* Note: This post contains 'preliminary valence parameter fitting results’, which was carried out with currently available QM data from 2nd generation training data sets.
Description
This post contains benchmark of preliminary valence parameter fitting (v1.2.0-preliminary).
Results of benchmarks for both the initial force field, pre-released force fields (v1.0.0, v1.1.0) and the re-fitted force field are provided here for performance comparison.
Benchmark Data
The benchmarks are performed with the first generation test sets(primary set, full set). Second generation test set generation is a WIP.
Benchmark Results
Two types of benchmarks were done: (1) QM vs MM optimized geometries and (2) the relative energies between conformers at QM optimized geometries.
The final objective function value(X2) from FB single point calculation gives a brief overview of the agreement between QM and MM. The lower X2 is, the better the force field reproduces QM structures and energetics.
X2 for primary set | X2 for full set | |
|---|---|---|
Initial force field | 1435 | 29,469 |
v1.0.0 | 948 | 20,672 |
v1.1.0 | 936 | 20,097 |
v1.2.0-preliminary | 766 | 16,939 |
To provide more intuitive insights on the benchmark results, we aggregated the resulting data and made the following plots.
1. Optget
To investigate the improved performance in reproducing QM optimized geometries, the weighted root-mean-square error (WRMSE) of each molecule, which is weighted root-mean-square deviation of internal coordinates of MM optimized geometry from QM optimized geometry was calculated and compared. ( Metrics for bond, angle, improper torsion are set to be 0.05 Angstrom, 8 degree and 20 degree respectively and torsion contributions were intentionally excluded.)
Note: y values in the plots (Δ WRMSE) are the difference in the WRMSE between different v1.2.0-pre and v1.1.0; Negative y value indicates better reproduction in v1.2.0-pre compared to v1.1.0.
2. Abinitio Targets
To investigate the improved performance of the new parameter set in reproducing QM relative energies between conformers, QM vs MM relative energies between conformers at QM optimized geometries were calculated.