hfutils.utils.path

This module provides utilities for working with Hugging Face filesystem paths. It includes functions for normalizing paths, constructing Hugging Face filesystem paths, and parsing those paths into structured components. Additionally, it manages irregular repositories through caching and provides a data class for representing Hugging Face filesystem paths.

hf_normpath

hfutils.utils.path.hf_normpath(path) → str[source]

Normalize a given path.

This function takes a path as input and normalizes it by removing any redundant separators and converting it to a relative path. This can be useful for ensuring consistent path formatting across different operating systems.

Parameters:

path (Any) – The path to normalize.

Returns:

The normalized path.

Return type:

str

hf_fs_path

hfutils.utils.path.hf_fs_path(repo_id: str, filename: str, repo_type: Literal['dataset', 'model', 'space'] = 'dataset', revision: str | None = None) → str[source]

Get the huggingface filesystem path.

This function constructs a Hugging Face filesystem path from the given repository ID, filename, repository type, and optional revision. It ensures that the path is formatted correctly according to the Hugging Face conventions.

Parameters:

  • repo_id (str) – The repository ID.

  • filename (str) – The filename.

  • repo_type (RepoTypeTyping) – The type of repository. (default: ‘dataset’)

  • revision (Optional[str]) – The revision of the repository. (default: None)

Returns:

The huggingface filesystem path.

Return type:

str

parse_hf_fs_path

hfutils.utils.path.parse_hf_fs_path(path: str) → HfFileSystemPath[source]

Parse the huggingface filesystem path.

This function takes a Hugging Face filesystem path as input and parses it into its components: repository ID, filename, repository type, and revision. It validates the path format and raises an error if the path is invalid.

Parameters:

path (str) – The path to parse.

Returns:

The parsed huggingface filesystem path.

Return type:

HfFileSystemPath

Raises:

ValueError – If this path is invalid.

HfFileSystemPath

class hfutils.utils.path.HfFileSystemPath(repo_id: str, filename: str, repo_type: Literal['dataset', 'model', 'space'], revision: str | None)[source]

Huggingface FileSystem Path.

This data class represents a Hugging Face filesystem path, encapsulating the repository ID, filename, repository type, and optional revision. It is used to provide a structured representation of filesystem paths used in Hugging Face.

Parameters:

  • repo_id (str) – The repository ID.

  • filename (str) – The filename.

  • repo_type (RepoTypeTyping) – The type of repository.

  • revision (Optional[str]) – The revision of the repository.