hfutils.operate.base

get_hf_client

hfutils.operate.base.get_hf_client(hf_token: str | None = None) → HfApi[source]

Get the Hugging Face API client.

Parameters:

hf_token (str, optional) – Huggingface token for API client, use HF_TOKEN variable if not assigned.

Returns:

The Hugging Face API client.

Return type:

HfApi

get_hf_fs

hfutils.operate.base.get_hf_fs(hf_token: str | None = None) → HfFileSystem[source]

Get the Hugging Face file system.

Parameters:

hf_token (str, optional) – Huggingface token for API client, use HF_TOKEN variable if not assigned.

Returns:

The Hugging Face file system.

Return type:

HfFileSystem

list_files_in_repository

hfutils.operate.base.list_files_in_repository(repo_id: str, repo_type: ~typing.Literal['dataset', 'model', 'space'] = 'dataset', subdir: str = '', revision: str = 'main', ignore_patterns: ~typing.List[str] = <object object>, hf_token: str | None = None) → List[str][source]

List files in a Hugging Face repository based on the given parameters.

Parameters:

  • repo_id (str) – The identifier of the repository.

  • repo_type (RepoTypeTyping) – The type of the repository (‘dataset’, ‘model’, ‘space’).

  • subdir (str) – The subdirectory to list files from.

  • revision (str) – The revision of the repository (e.g., branch, tag, commit hash).

  • ignore_patterns (List[str]) – List of file patterns to ignore.

  • hf_token (str, optional) – Huggingface token for API client, use HF_TOKEN variable if not assigned.

Returns:

A list of file paths.

Return type:

List[str]