Results
- Simon Boothroyd
For reference the labels used throughout this page and plots correspond to:
openff-1.0.0- the unmodified parsley force field.h_mix_rho_x- a force field optimised against and .h_mix_rho_x_rho_pure- a force field optimised against , , and .h_mix_rho_x_rho_pure_h_vap- a force field optimised against , , and .h_mix_v_excess- a force field optimised against , .rho_pure_h_vap- a force field optimised against and .
All uncertainties reported and shown here are 95% confidence intervals calculated using 2000 bootstrapping iterations with replacement.
Alcohol-Ester Refit
Presented in this section are the results of refitting the openff-1.0.0 force field against a training set containing only alcohol and ester components.
Optimisation Results
For each of the different training sets employed, the objective function was observed to rapidly decrease after the first iteration, after which there was only marginal improvements with the objective function being seen to fluctuate around a minimum due to noise in evaluating the objective function and its gradient (Figure 1).
Figure 1. The value of the objective function at each iteration for each refit performed.
In general most of the refit parameters changed only slightly from their initial values (Figure 2), the exception being…
Figure 2. The change in the each parameter for each of the different training sets relative to their starting value (taken from the openff-1.0.0 force field).
In general the target properties improved after refitting, the main exception being the enthalpy of vaporisation. When the the enthalpy of vaporisation was included as a target property, it was observed that while the property improved for esters, its performance was degraded for alcohols (Figure 3).
Figure 3. The average RMSE of the enthalpy of vaporisation at each iteration relative to the RMSE at the first iteration estimated using the initial force field parameters. A positive value indicates the property got worse, while a negative value indicates the property got better.
A possible cause for this may be seen on inspection of the first derivative of the enthalpy of vaporisation's contribution to the object function (Figure 4).
Figure 4. The first derivative of the enthalpy of vaporisation's contribution to the object function at iteration 0. Solid lines represent the true direction and magnitude of each gradient, while the dashed lines represent the same gradients but which have been normalised to enhance clarity when the magnitude of a gradient is small.
In general each parameter seems to be constrained in the same direction, with the exception of the [#1:1]-[#6X4] hydrogen parameter and the [#6X4:1] carbon parameter. In both cases, the contribution from ester data points is significantly larger and in an opposite direction than the contribution from the alcohol data points.
Benchmark Results
Presented in this section are the results of benchmarking each of the refit force fields against the set test set curated for the alcohol-ester only portion of the study.
In general, it appears there was an improvement in the performance of each refit force field across the board (Figure 5 and 6). As was expected, fitting against the enthalpy of vaporisation did improve the performance of such in the benchmark set, while fitting against the mixture properties did again improve their performance in the benchmark set.
Interesting to note the fitting against the pure enthalpy of vaporisation and density did offer some improvement of the mixture properties, while fitting against the mixture properties did offer some improvement for the pure benchmarked properties.
Figure 5. The average RMSE and R2 for each type of property, and for each optimised force field.
Figure 6. The overall results of benchmarking the optimised force fields against the test set.
Figure 7. The average RMSE and R2 for each type of property partitioned by chemical environment. Here X > Y corresponds roughly to a 75:25 mix of environment X and Y, X ~ Y a roughly 50:50 mix and X < Y a roughly 25:75 mix.
Expanded Refit
Presented in this section are the results of refitting the openff-1.0.0 force field against a training set containing alcohol, ester, ether, ketone and alkane-only functionality. This is an expansion of the alcohol-ester only study presented above.
Optimisation Results
As with the initial alcohol-ester study, the objective function was observed to rapidly decrease after the first iteration, after which there was only marginal improvements (Figure 8).
Figure 8. The value of the objective function at each iteration for each refit performed.
Figure 9. The average RMSE of each type of property at each iteration relative to the RMSE of each type of property estimated using the initial force field parameters.
In general most of the refit parameters changed only slightly from their initial values, varying less than 5% in most cases (Figure 10)
Figure 10. The change in the each parameter for each of the different training sets relative to their starting value (taken from the openff-1.0.0 force field).
The gradient of each properties contribution to the objective function during the first and second iterations were extracted and are displayed in Figure 11. In general, each property contributes largely appears to constrain the different parameters in the same direction, with the exception of XXX.
a)
b)
Figure 11. The contribution of each type of property to the gradient of the objective function at iteration 0 when refitting against a) enthalpy of mixing and binary mass density and b) pure density and enthalpy of vaporisation. Solid lines represent the true direction and magnitude of each gradient, while the dashed lines represent the same gradients but which have been normalised to enhance clarity when the magnitude of a gradient is small.
Benchmark Results
Presented in this section are the results of benchmarking each of the refit force fields against the set test set curated for the expanded portion of the study.
Figure 12. The average RMSE and R2 for each type of property, and for each optimised force field.
Figure 13. The overall results of benchmarking the optimised force fields against the test set.
Figure 14. The average RMSE and R2 for each type of property partitioned by chemical environment. Here X > Y corresponds roughly to a 75:25 mix of environment X and Y, X ~ Y a roughly 50:50 mix and X < Y a roughly 25:75 mix.