...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
Fitting Data and Results
Fitting targets: 2nd generation training sets (link for the details of the training set generation scheme: http://doi.org/10.5281/zenodo.3777278)
...
*Note that there are three uncovered torsion parameters(t114, t125, t146) in the torsion training dataset, which are due to the failed torsiondrive calculation carried out inside QCArchive.
Input force field : version Parsley 1.1.0 parsley (http://doi.org/10.5281/zenodo.3695094)
The objective function decreased from
8.710e+03to6.843e+03in 31 steps.
(1) v1.2.0-RC1 vs. v1.2.0-RC2: Final objective function
Initial X2 | Final X2 | Number of iterations | |
|---|---|---|---|
v1.2.0-RC1(https://openforcefield.atlassian.net/wiki/x/AQC4F v1.2.0-RC1 fitting summary ) | 3.619E+04 | 6.877E+03 | 57 steps |
| objective value (X2) | ||
initial guess | 29,469 | ||
v1.1.0 | 20,097 | ||
v1.2.0-preliminary (link: http://doi.org/10.5281/zenodo.3781313 ) | 16,939 | ||
v1.2.0-RC2 | 8.710E+03 | 6.843E+03 |
Benchmark Results
Benchmark data
For the calculation, full benchmark set was used (25168 optimized geometries, plus relative energies of 2005 molecules). Detailed of the molecule selection can be found here: release-1-benchmarking/QM_molecule_selection
(1) Comparison of objective values from single point calculations on benchmark full set
Two types of benchmarks were done to compare the performances: (1) QM vs MM optimized geometries and (2) the relative energies between conformers at “QM optimized geometries”. The final objective function value(X2) from FB single point calculation gives a brief overview of the agreement between QM and MM. The lower X2 is, the better the force field reproduces QM structures and energetics.
31 steps |
(2) v1.2.0-RC1
...
16,713
...
v1.2.0-RC2
...
16,910
...
vs. v1.2.0-RC2:
...
Direct parameter comparison
...
| View file | ||
|---|---|---|
|
No notable differences in equilibrium bond lengths and equilibrium angles while showing notable differences in k values of have been found from the direct parameter comparison, while some angle/ torsion parameters. Angle terms with significant different final k values are noticeably different between the two optimized parameter sets.
The bar charts shows the angle k value differences between RC1(blue bars) and RC2.
...
Three angle terms with noticeably different k values between RC1 and RC2:
a6([#1:1]-[*;r3:2]~;!@[*:3], k value in SMIRNOFF99Frosst: 100 kcal/mol/radian2)a3([*;r3:1]1~;@[*;r3:2]~;@[*;r3:3]1, k value in SMIRNOFF99Frosst: 200 kcal/mol/radian2)a15([#8X1:1]~[#6X3:2]~[#8:3], k value in SMIRNOFF99Frosst: 126 kcal/mol/radian2)
: Based solely on my intuition without no strong evidence, RC2 angle k values for the angle terms(doesn’t change much from 1.1.0 during the optimzation) seem physical; 400kcal/mol/radian2 for angle seems too large compare to other angle k values. Final gradients for the angle k values are also higher in RC1 (a6: 5.580e+00 , a15 : 4.750e+00) compared to the gradients in RC2 (a6: 1.599e+00 , a15 : 1.391e+00)
...
: But scatter plot seems slight Scatter plots seem slightly better in RC1. Also one thing I noticed is that due to the larger equilibrium angle over QM angle values in v1.1.0, RC2 which used v1.1.0 as its initial guess also ended up having large equilibrium value (~ 135 large initial guess of a15 equilibrium angle has been used in RC2 fitting, which led to larger final equilibrium angle(~135 degree) compared to the resulting final equilibrium value angle in RC1(~ 129 degree). ( The optimization issue about resulting in final equilibrium values substantially off from QM values a15 equilibrium angle in v1.1.0 is around 137 degree, which is larger than angles observed in QM optimized geometries.) → For more discussion on this, please see the 5/28 ff-release call meeting note.
( The issue with converging to a final equilibrium value substantially off from values in QM optimized geometries is one of the known problems issues that we are currently working on. )
...
Cf. But it seems like having final equilibrium angle different with
...
QM values is not always the case we want to avoid. Here’s one example
...
. As you can see in the scatter plot below, the fitting converged to a larger a38 equilibrium angle (~ 147 degree
...
) which is larger than QM values(~ 110 degree
...
). And the large equilibrium angle has been found to be
...
beneficial in locating hydroxyl hydrogen in phosphono group far away enough from its neighboring oxygens, preventing unphysical
...
intramolecular H-bonding between the hydrogen and the neighbor oxygens.
...
Torsion Three torsion terms with significant significantly different final k values between RC1 and RC2:
t146,t147: having 6 periodicities cosine functionst15([*:1]-[#6X4;r3:2]-@[#6X4;r3:3]-[*:4]),t16([#6X4;r3:1]-[#6X4;r3:2]-[#6X4;r3:3]-[*:4]): in-ring rotation
...
rotations
| View file | ||
|---|---|---|
|
Benchmark Results
Benchmark data
For the calculation, full benchmark set was used (25168 optimized geometries, plus relative energies of 2005 molecules). Details of the molecule selection for the test set can be found here: release-1-benchmarking/QM_molecule_selection
(1) Comparison of objective values from single point calculations on benchmark full set
Two types of benchmarks were done using ForceBalance to compare the performances: (1) QM vs MM optimized geometries and (2) the relative energies between conformers at “QM optimized geometries”. The final objective function value(X2) from FB single point calculation gives a brief overview of the agreement between QM and MM. The lower X2 is, the better the force field reproduces QM structures and energetics. Objective value from RC1 is lower than the objective value from RC2.
| objective value (X2) |
|---|---|
initial guess | 29,469 |
v1.1.0 | 20,097 |
v1.2.0-preliminary (link: http://doi.org/10.5281/zenodo.3781313 ) | 16,939 |
v1.2.0-RC1 | 16,713 |
v1.2.0-RC2 | 16,910 |
(2) v1.2.0-RC1 vs. v1.2.0-RC2: Comparison of RMSD for each parameter
RC2 decrease RMSD of internal coordinates assigned to t135, t146, t34 and t97 while increasing RMSDs for t54 and t55.
| View file | ||
|---|---|---|
|
(3) v1.2.0-RC1 vs. v1.2.0-RC2: Optimized geometries
Specific improvement in optimized geometries with certain functional groups(phosphono group, sulfamate acetate) found in RC1 is also shown in RC2.
TODO: revert WRMSE → objective value
TODO: implementation of RMSE calculation from FB output
...
QM optimized geometry of CC(O)([P@@](=O)(O)[O-])[P@](=O)(O)[O-]. ( orange: MM optimized geometry with v1.1.0 force field, green: v1.2.0-RC1 force field, magenta: v1.2.0-RC2 force field)
(4) v1.2.0-RC1 vs. v1.2.0-RC2: Relative energies between conformers at “QM optimized geometries”
Comparison Performance comparison of performances of RC1 and RC2 in reproducing QM relative energies between conformers was carried out. Two different ways to calculate MM relative energies were used. Two different ways to calculate MM relative energies were used. For the left figureIn the first approach, MM relative energies were calculated by taking a difference between subtracting MM energy at each point and the QM minimum from MM energy at the QM minimum. And for the right figure, each point (distribution figure on the left). Second approaches calculated MM relative energies were obtained by taking a difference between by subtracting MM energy at each point and QM minimum from MM energy at QM minimumeach point(distribution figure on the right). Both candidates have smaller MAD and shorter tails than v1.1.0, indicating slight better performances over v1.1.0, while RC2 shows a slightly better performance over RC1.
...
(5) v1.2.0-RC1 vs. v1.2.
...
0-RC2: Molecules having [#7X2]-!@[#7X3]
RC2 is slightly worse than RC1 in reproducing QM optimized geometries (RMSD and TFD) while showing slightly better performance in reproducing QM energetics (ddE)
...
*Figures will be trimmed soon!