Metrics Overview

mlipx provides several tools and integrations for comparing and visualizing metrics across experiments and nodes. This section outlines how to use these features to evaluate model performance and gain insights into various tasks.

Comparing Metrics Using mlipx compare

With the mlipx compare command, you can directly compare results from the same Node or experiment using the ZnDraw visualization tool. For example:

mlipx compare mace-mpa-0_StructureOptimization orb-v2_0_StructureOptimization

This allows you to study the performance of different models for a single task in great detail. Every Node in mlipx defines its own comparison method for this.

Integrations with DVC and MLFlow

To enable a broader overview of metrics and enhance experiment tracking, mlipx integrates with both DVC and mlflow. These tools allow for efficient tracking, visualization, and comparison of metrics across multiple experiments.

MLFlow Integration

mlipx supports logging metrics to mlflow. To use this feature, ensure mlflow is installed:

pip install mlflow

Note

More information on how to setup MLFlow and run the server can be found in the MLFlow documentation.

Set the tracking URI to connect to your MLFlow server:

export MLFLOW_TRACKING_URI=http://localhost:5000

Use the zntrack mlflow-sync command to upload metrics to MLFlow. For this command, you need to specify the Nodes you want to sync.

Note

You can get an overview of all available Nodes using the zntrack list command. The use of glob patterns makes it easy to sync the same node for different models. To structure the experiments in MLFlow, you can specify a parent experiment.

A typical structure for syncing multiple Nodes would look like this:

zntrack mlflow-sync "*StructureOptimization" --experiment "mlipx" --parent "StructureOptimization"
zntrack mlflow-sync "*EnergyVolumeCurve" --experiment "mlipx" --parent "EnergyVolumeCurve"
zntrack mlflow-sync "*MolecularDynamics" --experiment "mlipx" --parent "MolecularDynamics"

With the MLFlow UI, you can visualize and compare metrics across experiments:

Additionally, mlipx logs plots to MLFlow, enabling comparisons of relaxation energies across models or direct visualizations of energy-volume curves:

Data Version Control (DVC)

Each Node in mlipx includes predefined metrics that can be accessed via the DVC command-line interface. Use the following commands to view metrics and plots:

dvc metrics show
dvc plots show

For more details on working with DVC, refer to the DVC documentation.

DVC also integrates seamlessly with Visual Studio Code through the DVC extension, providing a user-friendly interface to browse and compare metrics and plots: