* Note: This post contains 'preliminary valence parameter fitting results’, which was carried out with currently available QM data from 2nd generation training data sets.
Description
This post contains benchmark of preliminary valence parameter fitting (v1.2.0-preliminary).
Results of benchmarks for both the initial force field, pre-released force fields (v1.0.0, v1.1.0) and the re-fitted force field are provided here for performance comparison.
Fitting Data and Results
Fitting targets: 581 1-D torsion profiles; 2,974 optimized geometries; 278 vibrational frequencies
Input force field : same initial force field used in v1.1.0 fitting
The objective function decreased from
1.02809e+04to3.21676e+03in 28 steps.
Benchmark Data
The benchmarks are performed with the first generation test sets: neighboring (primary) set and diverse (full) set. Second generation test set generation is a WIP.
Benchmark Results
Two types of benchmarks were done: (1) QM vs MM optimized geometries and (2) the relative energies between conformers at QM optimized geometries.
The final objective function value(X2) from FB single point calculation gives a brief overview of the agreement between QM and MM. The lower X2 is, the better the force field reproduces QM structures and energetics.
X2 for neighboring (primary) set | X2 for diverse (full) set | |
|---|---|---|
Initial force field | 1435 | 29,469 |
v1.0.0 | 948 | 20,672 |
v1.1.0 | 936 | 20,097 |
v1.2.0-preliminary | 766 | 16,939 |
To provide more intuitive insights on the benchmark results, we aggregated the resulting data and made the following plots.
1. Optget
To investigate the improved performance in reproducing QM optimized geometries, the weighted root-mean-square error (WRMSE) of each molecule, which is weighted root-mean-square deviation of internal coordinates of MM optimized geometry from QM optimized geometry was calculated and compared. ( Metrics for bond, angle, improper torsion are set to be 0.05 Angstrom, 8 degree and 20 degree respectively and torsion contributions were intentionally excluded.)
y values in the plots (Δ WRMSE) are the difference in the WRMSE between different v1.2.0-pre and v1.1.0; Negative y value indicates better reproduction in v1.2.0-pre compared to v1.1.0. The average change in WRMSE is -1.248, indicating that overall the v1.2.0-pre better performs in reproducing QM optimized geometry than v1.1.0.
2. Abinitio Targets
To investigate the improved performance of the new parameter set in reproducing QM relative energies between conformers, QM vs MM relative energies between conformers “at QM optimized geometries” were calculated.
The distribution from v1.2.0-pre is more centered to zero and mean absolute deviation(MAD) is smaller than v1.1.0 and v1.0.0, indicating that overall the v1.2.0-pre better performs in reproducing QM energetics than the old versions.