MD22 MD17 train and test data split

[jhmlam]

Hi @gabor1 @davkovacs @ilyes319 ,

Thank you for sharing, developing and maintaining the dataset and mace.
I saw on https://mace-docs.readthedocs.io/en/latest/examples/training_examples.html that there is a nanotube_test.xyz file. However, when I go to http://www.sgdml.org/#datasets, it seems that there is no official train test split for MD 17 and 22, though there are splitted xyz for the ccsd datasets there.

Is it possible to provide a simple script to do the split producing xyz similar to

https://github.com/ilyes319/mace-tutorials/blame/1172df9fb93cbaad6b3abda87526ec3484ef33fe/mace-users/MACE_users.ipynb#L469

such that it roughly matches Table I in the report Kovács DP, Batatia I, Arany ES, Csányi G. Evaluation of the MACE force field architecture: From medicinal chemistry to materials science. J Chem Phys. 2023 Jul 28;159(4):044118. doi: 10.1063/5.0155322. PMID: 37522405.

Thank you!

Best,
Jordy

[jhmlam]

Sorry for doubling. Indeed, which files "nanotube_test.xyz", "nanotube_large.xyz" are referring to on http://www.sgdml.org/#datasets. These files were not inside the Double-walled nanotube folder. So I cannot actually run the example exactly as written.

python <mace_repo_dir>/mace/cli/run_train.py \
    --name="nanotube_large_r55" \
    --train_file="nanotube_large.xyz" \
    --valid_fraction=0.05 \
    --test_file="nanotube_test.xyz" \
    --E0s="average" \
    --model="MACE" \
    --num_interactions=2 \
    --num_channels=256 \
    --max_L=2 \
    --correlation=3 \
    --r_max=5.0 \
    --forces_weight=1000 \
    --energy_weight=10 \
    --batch_size=2 \
    --valid_batch_size=2 \
    --max_num_epochs=650 \
    --start_swa=450 \
    --scheduler_patience=5 \
    --patience=15 \
    --eval_interval=3 \
    --ema \
    --swa \
    --swa_forces_weight=10 \
    --error_table='PerAtomMAE' \
    --default_dtype="float64"\
    --device=cuda \
    --seed=123 \
    --restart_latest \
    --save_cpu

[ilyes319]

Hello,

You can just run the following script that will select randomly 5% of the set for validation and 95% for training as detailed in the appendix of the paper. Make sure to convert the units from kcal/mol to eV before training for both energies and forces.

python <mace_repo_dir>/mace/cli/run_train.py \
    --name="nanotube_large_r55" \
    --train_file="md22_double-walled_nanotube.xyz" \
    --valid_fraction=0.05 \
    --test_file="md22_double-walled_nanotube.xyz" \
    --E0s="average" \
    --model="MACE" \
    --num_interactions=2 \
    --num_channels=256 \
    --max_L=2 \
    --correlation=3 \
    --r_max=5.0 \
    --forces_weight=1000 \
    --energy_weight=10 \
    --batch_size=2 \
    --valid_batch_size=2 \
    --max_num_epochs=650 \
    --start_swa=450 \
    --scheduler_patience=5 \
    --patience=15 \
    --eval_interval=3 \
    --ema \
    --swa \
    --swa_forces_weight=10 \
    --error_table='PerAtomMAE' \
    --default_dtype="float64"\
    --device=cuda \
    --seed=123 \
    --restart_latest \
    --save_cpu

[jhmlam]

Thank you @ilyes319 (also for the important remark on eV). Now it works. The flags

        --energy_key="Energy" \
        --forces_key="forces" \ 

were also used, otherwise it gives zero loss.