Is this necessary to run ab initio calculation to test MLFF

[duccoi]

Dear all,
I was advised to check the accuracy of MLFF by comparing the ab initio results and MLFF results independently.
1. After finishing the on-the-fly calculation, we extract the structure, say every 10 ps.
2. Run ab initio single-point calculation using structures obtained from the previous step to estimate the energy and force for each structure.
3. We compare the force and the energy between structures of ab initio and MLFF calculations.
This process is time-consuming because normally we would like to extract nearly 1000 structures for this comparison process. May I ask if this is necessary to do this process? Can we use data from the ML_REG file instead?

[ferenc_karsai]

The ML_REG file (and the ERR line in the ML_LOGFILE) contains only the fitting errors of the training data.
What you describe is to test the capability of the force field on independent data.
In the best case the error from both is the same. This is usually the case if you have test structures chosen from similar phases as the training data and at similar physical conditions.
However, the independent data can also be chosen from different phases and/or physical conditions. In that case it is not clear whether the force field will work well and test calculations as you described them are mandatory.

In my opinion if one wants to publish data then a verification on an independent test set is necessary.