hfutils.utils.model

This module provides functionality for identifying model files based on their extensions and naming patterns.

It includes a comprehensive list of model file extensions, patterns for sharded model files, and specific patterns for Hugging Face model files. The main function, is_model_file(), determines whether a given filename corresponds to a model file based on these predefined patterns and extensions.

This module can be useful in various scenarios, such as:

• Automated model file detection in directories

• Validation of uploaded files in machine learning platforms

• Preprocessing steps in model loading pipelines

Usage:

from model_file_identifier import is_model_file

filename = "model.pt"
if is_model_file(filename):
    print(f"{filename} is a model file")
else:
    print(f"{filename} is not a model file")

is_model_file

hfutils.utils.model.is_model_file(filename: str | PathLike) → bool

Determine if a given filename corresponds to a model file.

This function checks if the provided filename matches any of the known model file extensions or patterns, including sharded model files and Hugging Face specific patterns.

Parameters:

filename (Union[str, os.PathLike]) – The name of the file to check. Can be a full path or just the filename.

Returns:

True if the filename corresponds to a model file, False otherwise.

Return type:

bool

Raises:

TypeError – If the filename is not a string or os.PathLike object.

Usage:

>>> is_model_file("model.pt")
True
>>> is_model_file("data.csv")
False
>>> is_model_file("model-00001-of-00005")
True
>>> is_model_file("pytorch_model.bin")
True

Note

This function is case-insensitive and works with both file names and full paths.