How to evaluate the Transformer Trainer

If you initialized a 🤗 Trainer object, it will do the training boiler plate for you. Using the TrainingArguments, you can additionally customize your training process.

One important argument is the evaluation_strategy which is set to “no” by default, thus no evaluation is done while training. You can set it up either per steps (using eval_steps) or at the end of each epoch. Make sure to set up an evaluation dataset beforehand.

# example snippet

# define hyperparameters
training_args = TrainingArguments(

# create TensorBoard Writer and assigns it to a Callback object
writer = SummaryWriter("path/to/tensorboardlogs")
tbcallback = TensorBoardCallback(writer)

# defines model
trainer = Trainer(


We also set a TensorboardCallback, so that metrics are directly send to Tensorboard after each evaluation.

Output metrics of default evaluation in tensorboard

Further Readings:

Fine-tuning a model with the Trainer API, access under:

Leave a Comment

Your email address will not be published. Required fields are marked *

hungsblog | Nguyen Hung Manh | Dresden
Scroll to Top