...
With these results in hand, I next repeated the process but replacing every Sage parameter with the corresponding average parameter from Espaloma. As shown below, the results are more different from the esp-tors-10 results, as expected. And positively, esp-full appears to perform a bit better by all three metrics. This is without any re-fitting, so Espaloma’s average parameters for our SMIRKS patterns perform slightly better than our re-fit Sage 2.1.0 values.
...
The results above were actually still relying on the code that identified the large deviations from Sage parameters, so not all parameters were replaced. Additionally, that code only looked at force constants, and only the first force constant for torsions. In contrast, the graph below shows the esp-full-full (really full) results, where all of the values have been replaced for all force constants and equilibrium bonds and angles. Clearly these look much worse for the Espaloma values within the SMIRNOFF format. This makes me a bit suspicious of the torsions especially because those values look the most different from Sage at first glance, at least compared to the bonds and angles, which are more recognizable.
...