Page Comparison

...

Bayesian methods give access to posterior distributions from which the correlations (including more complex, non-linear relationships) between BCC parameters can be easily discerned. Further the computation of Bayes factors allow us to quantitatively measure where extra model complexity (in this case extra ‘atom’ types or BCC ‘types’). Combined, Bayes methods should allow us to gain data driven insight into where we perhaps have too many atom types, and where there are types missing.

...

In particular, N sets of different atom types could be proposed (e.g. one set may have the highly delocalised and delocalised N merged into a single atom type), their Bayes factor computed, and inference made off of that as to which the ESP data supports the most.

...

The ESP data will be computed on a FCC grid (spacing TBD) and using a aug-cc-pV(D+d)Z basis and the pw6b95 method as was highlighted by the RESP2 publication as yielding a strong balance of performance and accuracy.

Preliminary (non-Bayesian) studies

One of the first questions in using a Bayesian inference approach (or any optimization/parameter search) is whether the target we are using in our posterior is representative of the properties we want to get right.

Specifically, in this case, our posterior is computed by comparing BCC charges to electric field values. The advantage of this method is twofold:

1. This is the canonical method of training BCC parameters (and electrostatics more generally).

2. It is relatively computationally cheap, because the electric fields do not be recomputed every time the BCC parameters are changed.

However, this does raise a question of whether this agreement with this RESP2-like electric field target is representative of our overall target (i.e. physical properties such as solvation free energies, mixture enthalpies, densities, etc).

More granularly, the question is whether improvements in the forcebalance optimization target correspond to improvements in physical property performance.

Does the ForceBalance target correspond to physical property accuracy?

In order to test how the FB target differentiates different parameter sets, I ran a set of 50 optimizations against the same target, but with perturbed initial BCC values (drawn from Norm(mu, 2*mu), where mu is the off-1.2.0 value). The optimizations have very different starting values, but all reach relatively similar final values.

...

Looking at the values of these BCC sets, we see a wide variety despite the similar objective functions:

...

In order to get a sense of the magnitude of the differences between these BCC sets, I identified the two most different sets and benchmarked them against some solvation free energy data for ethers and alcohols (two moieties known to have some problems with the current FF). Even though these sets have different BCC values, their final objective functions were extremely similar (2.157 vs 2.177):

...

In this benchmark, we observe a trade-off in quality of reproducing ethers and alcohols:

For the set that improved alcohols (perturb_32), we find that there is a systematic reduction of error in solvation free energies (perturb_32 overpredicts Gsolv less):

For the set that improved ethers (perturb_5), this is less clear, but still seems to happen (perturb_5 underpredicts Gsolv less):

This isn’t clear cut evidence either way for how predictive the objective function is (additional benchmarks are ongoing), since these sets do better at different moieties. However, it does identify a tradeoff in BCC values: alcohols and ethers seem to pull parameters in opposite directions!