Distributed evaluation

For the evaluation of different MLIP models, it is often necessary to run the evaluation in different environments due to package incompatibility. Another reason can be the computational cost of the evaluation.

Writing the evaluation in a workflow-like manner allows for the separation of tasks onto different hardware or software environments. For this purpose, the paraffin package was developed.

You can use paraffin submit to queue the evaluation of the selected stages. With paraffin worker --concurrency 5 you can start 5 workers to evaluate the stages.

Further, you can select which stage should be picked up by which worker by defining a paraffin.yaml file which supports wildcards.

queue:
    "B_X*": BQueue
    "A_X_AddNodeNumbers": AQueue

The above configuration will queue all stages starting with B_X to the BQueue and the stage A_X_AddNodeNumbers to the AQueue. You can then use paraffin worker --queue BQueue to only pick up the stages from the BQueue and vice versa.

The paraffin package is available on PyPI and can be installed via:

pip install paraffin