Experiment Tracker#

About#

The Aim Experiment Tracker app helps to track and manage ML experiments.

For most ML frameworks, we have extended the experiment tracker app and created an app per integration.

Each app provides the views for the experiments specifically tracked by that framework.

Together, these apps provide acomplete ML experiment tracking and management experience for wide variety of type of data - from metrics to text to audio.

The Experiment Tracker app also provides the base experiment tracking and management abstractions - the users can extend them to their preferences.

All relevant ML framework integrations are provided out of the box.

Integrates seamlessly with your favorite tools

Installation#

Installed out of the box with as part of default Aim installation. Users can unregister and replace with any other app.

Usage#

The Aim default experiment tracker app usage is straightforward. Users just need to integrate the Run and the

Integrate with code#

The Experiment Tracker can be added to the code like this:

from aimstack.experiment_tracker import Run, Experiment
from aimstack.base import Metric

run = Run()

run['hparams'] = {
  'lr': 0.001,
  'batch_size': 32,
  'epochs': 10,
}

metric = Metir(run, name='loss', context={'subset': 'train'})

for i in range(1000):
  metric.track(i, epoch=1)

Start the UI#

Once the code is run, the UI is very straightforward to start. With the default Aim installation, just run:

aim ui

The UI will be available at http://0.0.0.0:43800

Experiment Tracking UI#

The Experiment Tracking UI has three main sections:

  • Overview page: a high-level overview of what’s been tracked and their quantities

  • Experiment page: a grouped view of training runs

  • Single Run page: a detailed view of a single training run

  • Runs page: a searchable list of all runs

These views together with the Aim explorers provide a complete experiment tracking and management solution.

Using Explorers with Experiment Tracking#

Aim Explorers are specialized in comparing large quantities of logs of the same type.

There are wide variety of explorers available out of the box - from metrics to text to audio.

The Experiment Tracker app provides the base experiment management while the Explorers are the backbone of metric comparison. And overall comparison of lots of other type of logs.

Aim Experiment Tracker

Using Reports with Experiment Tracking#

Aim also has Reports available that are highly integrated with all the rest of the pages. Any logged data can be rendered in the Reports.

Reports are a knowledge-base on top of the logs. They are a great way to share knowledge and insights with the team.

Integrating to existing code#

Aim Experiment Tracker is integrated with all the major ML frameworks via the Integration Apps.

If you are already using one of the supported frameworks, you can start using Aim right away.

Example: HuggingFace integration#

Here is how the HuggingFace integration works:

from aimstack.huggingface import AimCallback

trainer = Trainer(
  model=model,
  args=training_args,
  train_dataset=train_dataset,
  eval_dataset=eval_dataset,
  callbacks=[AimCallback()],
)

trainer.train()