Hugging Face Hub models aren’t just static files; they’re dynamic entities you can push to and pull from, effectively acting as a versioned, collaborative, and distributed cache for your machine learning artifacts.
Let’s see this in action. Imagine you’ve trained a small BERT model for sentiment analysis. First, you need to set up your local environment and authenticate with Hugging Face.
pip install transformers datasets huggingface_hub
huggingface-cli login
Now, let’s create a dummy model and push it.
from transformers import AutoModelForSequenceClassification, AutoTokenizer
from huggingface_hub import HfApi, create_repo
# --- Configuration ---
model_id = "your-username/my-sentiment-model" # Replace with your HF username and a unique model name
local_model_path = "./my_sentiment_model"
# --- Create a dummy model and tokenizer ---
# In a real scenario, you'd load your trained model here
tokenizer = AutoTokenizer.from_pretrained("bert-base-uncased")
model = AutoModelForSequenceClassification.from_pretrained("bert-base-uncased", num_labels=2)
# Save the model and tokenizer locally
tokenizer.save_pretrained(local_model_path)
model.save_pretrained(local_model_path)
print(f"Dummy model and tokenizer saved locally to: {local_model_path}")
# --- Push to Hugging Face Hub ---
api = HfApi()
# Create a new repository on the Hub if it doesn't exist
try:
api.create_repo(repo_id=model_id, exist_ok=True)
print(f"Repository '{model_id}' created or already exists.")
except Exception as e:
print(f"Error creating repository: {e}")
exit()
# Upload the model files
print(f"Uploading model files from {local_model_path} to {model_id}...")
api.upload_folder(
folder_path=local_model_path,
repo_id=model_id,
repo_type="model",
)
print(f"Model successfully pushed to Hugging Face Hub: https://huggingface.co/{model_id}")
This script first defines a model_id (which is essentially a path on the Hub, like username/repo_name). It then creates a simple BERT model and tokenizer, saving them to a local directory. The core action is api.upload_folder, which takes your local directory and uploads its contents to the specified repo_id on the Hub. repo_type="model" clarifies that this is a model repository, distinct from dataset or space repositories. exist_ok=True in create_repo means if you run this again, it won’t error out if the repository already exists.
Once pushed, anyone with access (public or private, depending on your repo settings) can download this model. The transformers library handles this seamlessly with from_pretrained.
from transformers import AutoModelForSequenceClassification, AutoTokenizer
# --- Download from Hugging Face Hub ---
hub_model_id = "your-username/my-sentiment-model" # Use the same model_id from the push script
print(f"Downloading model from Hugging Face Hub: {hub_model_id}")
try:
model = AutoModelForSequenceClassification.from_pretrained(hub_model_id)
tokenizer = AutoTokenizer.from_pretrained(hub_model_id)
print("Model and tokenizer downloaded successfully.")
# You can now use the loaded model and tokenizer
print("Example inference:")
inputs = tokenizer("This is a great movie!", return_tensors="pt")
outputs = model(**inputs)
print(f"Model output logits: {outputs.logits}")
except Exception as e:
print(f"Error downloading model: {e}")
This from_pretrained call, when given a Hub model ID, automatically checks the Hugging Face Hub, downloads the necessary files (model weights, configuration, tokenizer files), and loads them into the specified classes. It’s essentially a smart, versioned, and distributed cache. The transformers library knows to look for config.json, pytorch_model.bin (or tf_model.h5), and tokenizer_config.json, vocab.txt, etc., on the Hub.
The "push" and "load" models concept revolves around managing these artifact repositories. A "push" operation uploads local files to a remote repository on the Hub. A "load" (or "pull") operation downloads files from a Hub repository to your local machine. The Hub acts as the central point of truth and collaboration. Versioning is handled implicitly by commits to the Git repository backing each Hub model. When you push, you’re making commits; when you load, you’re by default pulling the latest commit. You can also specify a particular commit hash or tag to load specific versions.
Behind the scenes, each Hugging Face Hub repository is a Git repository. When you upload_folder, the huggingface_hub library is essentially performing Git operations (add, commit, push) against the remote repository. This means you can also interact with these repositories using standard Git commands if you clone them locally, offering a powerful alternative for managing complex workflows or integrating with existing CI/CD pipelines.
The most surprising thing about the push/load model paradigm is that it completely abstracts away the complexities of distributed file storage and version control for ML artifacts. You don’t think about S3 buckets, Git LFS, or manual versioning schemes; you just use push_to_hub or from_pretrained.
You’ll soon encounter the need to manage different versions of your models or to collaborate on a single model, leading into concepts like model branching and merging.