Hugging Face’s Trainer class can log directly to Weights & Biases, but it’s not just a simple wandb.init() call; the Trainer needs to be explicitly told to use W&B as its logging backend.
Let’s see it in action. Imagine you’re training a BERT model for text classification.
from transformers import Trainer, TrainingArguments
from datasets import load_dataset
import wandb
# Authenticate and initialize W&B
# wandb login --relogin
# wandb.init(project="huggingface-wandb-example") # This is not strictly necessary when using Trainer's W&B integration
# Load a small dataset
dataset = load_dataset("imdb", split="train[:1000]")
dataset = dataset.train_test_split(test_size=0.1)
# Tokenize the dataset
from transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("bert-base-uncased")
def tokenize_function(examples):
return tokenizer(examples["text"], padding="max_length", truncation=True)
tokenized_datasets = dataset.map(tokenize_function, batched=True)
# Load a pre-trained model
from transformers import AutoModelForSequenceClassification
model = AutoModelForSequenceClassification.from_pretrained("bert-base-uncased", num_labels=2)
# Define training arguments
# Crucially, set `report_to="wandb"`
training_args = TrainingArguments(
output_dir="./results",
num_train_epochs=1,
per_device_train_batch_size=8,
per_device_eval_batch_size=8,
warmup_steps=100,
weight_decay=0.01,
logging_dir="./logs",
logging_steps=10,
evaluation_strategy="steps",
eval_steps=100,
report_to="wandb", # <-- This is the key!
run_name="bert-imdb-training-run" # Optional: custom name for the W&B run
)
# Initialize the Trainer
trainer = Trainer(
model=model,
args=training_args,
train_dataset=tokenized_datasets["train"],
eval_dataset=tokenized_datasets["test"],
tokenizer=tokenizer,
# You can also add compute_metrics here for W&B to log metrics
# compute_metrics=compute_metrics_function
)
# Start training
trainer.train()
# When the run finishes, you'll see a link to your W&B dashboard
# printed in the console output, e.g.:
# "wandb: Synced bert-imdb-training-run: https://wandb.ai/your_username/huggingface-wandb-example/runs/..."
The core problem this solves is bridging the gap between the Hugging Face Trainer’s internal training loop and the external logging capabilities of Weights & Biases. Without report_to="wandb", the Trainer defaults to logging to the console and saving logs locally to training_args.logging_dir. By setting this argument, you instruct the Trainer to instantiate a W&B logger and push metrics like loss, learning rate, epoch, step, and any custom metrics you provide (e.g., accuracy, F1 score) to your W&B project.
Internally, when report_to is set to "wandb", the Trainer looks for the wandb library. If found, it initializes a W&B run using your environment variables or wandb.login() credentials. It then hooks into the Trainer’s internal logging mechanisms. Every time the Trainer logs progress (e.g., at logging_steps), it captures the relevant metrics and sends them to the active W&B run. This includes training loss, evaluation loss, learning rate, step count, and epoch count. If you provide a compute_metrics function to the Trainer, those computed metrics will also be logged.
The exact levers you control are primarily within TrainingArguments. report_to is the most critical, dictating the logging backend. run_name allows you to give your W&B run a descriptive title. logging_dir still applies for local fallback logs, but W&B becomes the primary destination. The Trainer itself can be configured with a compute_metrics callable, which is a function that takes EvalPrediction objects and returns a dictionary of metrics, which W&B will then automatically log alongside standard training metrics.
Most people don’t realize that the Trainer automatically logs the learning_rate as a separate metric that can be plotted over time in W&B. It’s not just loss; the Trainer is designed to expose key hyperparameters and their evolution during training, and W&B provides a powerful visualization layer for this.
The next step is to explore how to log model checkpoints and predictions automatically to W&B.