2024-01-24 Meeting notes

 Date

Jan 24, 2024

 Participants

  • @Matt Thompson

  • @Lily Wang

  • @Alexandra McIsaac

  • @Brent Westbrook

  • @Jeffrey Wagner

 Discussion topics

Item

Notes

Item

Notes

General updates

  • Bayly: “Are we comparing bonds and angles between QM and MM in general?”

    • There’s code for internal RMSD benchmarks:

    • LW – Would love to have this in the benchmarking suite. Given that code already exists it may be straightforward.

    • MT – Example code looks promising. Could use some explanation about what internal coordinates mean in general.

    • LM – Typically, internal coordinates are bonds, angles, dihedrals. So you can define the geometry of a molecule using EITHER cartesian coords OR internal coords, where the coords are bond lengths, bond angles, and torsion angles.

    • MT – Is this like converting between cartesian/polar/cylindrical coords?

    • LM – Not exactly, it’s slightly different in this context, so the transform will be a bit more involved.

    • MT – … Ok, looking at part of this it makes some sense. I’ll try implementing this later… Could become a new method on the class like get_internal_coordinate_rmsd. This is helpful.

    •  

  • Okay to continue using OpenMM for MM minimization?

    • LM – Not sure if this was about a different set of benchmarks, and I was unclear about the resolution/outcome. Not sure if this was about this benchmarking code or the industry benchmarking project.

    • MT – Also not sure about the context, I just picked up that there are different ways to do MM minimizations.

    • LW – Let’s stick with OpenMM for now but that may change depending on that LM finds. Question is broadly what to do when QM and MM energy surfaces differ, and OMM minimizaiton and geometric optimizaiton bring the same starting structure to different minima. So if LM finds that one method reliably goes to more useful minima we may change that. But generally the OpenMM minima currently seem “more relevant” than geomeTRIC minima.

    • MT – So these fits are looking at minimziations, and GeomeTRIC and OMM arrive at different minima…. Does geomeTRIC genreally get a lower energy?

    • LM – GeomeTRIC would gove a lower energy, but OMM would remain closer to the original conformer . And it’s possible that forcing geometric to use cartesian may give more similar results to OpenMM. So by the standards of “sticking close to the input conformer”, openMM does a better job.

    •  

    •  

    •  

    •  

Trello update

  • https://trello.com/b/dzvFZnv4/infrastructure?filter=label:benchmarking

  • MT – Importance of evaluator interface?

    • LW – TBH, not super important. Mostly because SFEs are blocked, and we don’t want to be juggling envs to be able to use old deps for SFEs.

    • MT – How valuable would it be to run these calcs routinely?

    • LW – It’d be useful to run these, for example for a likely FF release in April, but it’d raise questions about why different versions are being used.

    • JW – Would OpenFE SFE workflow be called through evaluator? Or would it be called separately?

    • LW – Good question. I have two conflicting positions - 1) A lot of people want that functionality, and it used to be there, but 2) I like things to be modular, and adding to evaluator would be quite complicated.

    • JW – If we want to fit to SFEs, don’t we need to have them in evaluator as a ForceBalance target?

    • LW – It’d be very expensive to fit to SFEs, not sure if we should wait on OpenFE and just implement out own. I could implement my own ion SFE, and I don’t think OpenFE wants to support ion SFEs because of charge change issues.

    • MT – I understand evaluator to calculate phys props and provide gradients. dFreeEnergy/dParameter sounds crazy to me, but I guess that’s what JSetiadi just did. I’m not sure whether OpenFE will provide this.

    • JW –

      • Do we currently have what we need for fitting?

      • Do we currently have what we need for benchmarking?

      • Do we need to do anything to line up eventual integration?

    • MT – Can you confirm that Evaluator is only used in fitting for vdW refits? And that those happen very infrequently/only for major versions?

      • LW – I’ve been doing a lot of vdW refits as part of vsites project and NN charges. But vdW refits will genrally just come with major releases - One with Rosemary protein charges, one with Rosemary graph charges, and one with Thyme vsites.

      •  

    • Summary:

      • Not so useful to have an Evaluator interface if SFEs aren’t supported

      • Not clear if OpenFE will support SFEs of ions

      • Not clear if Evaluator can/should have an interface to OpenFE-based free energy calcuations

      • Free energies only used for benchmarking not fitting

  • MT – Are things generally going alright in fitting world? I know I need to get vsites into FB. But anything elsE?

    • LW – Going well so far. I’m using old version of evaluator. But generally updated fitting stack seems solid.

    • LM – Working well for me.

    • MT – Good to know, currently in our automated fitting we don’t have many numerical tests, so I appreciate the update on status.

    •  

  • LW – Nonurgent, but pairwise energy targets could be handy. Currently with ddEs we compare to minimum conf. But doing all pairs could be interesting as an additional target.

    • MT – so evaluating diff between MM and QM as a all-to-all matrix instead of a 1D list..

    • LW – Yes, PB and BS were looking into this but it timed out.

    • MT – What do you do to get a result from the matrix?

    • LW – Currently for ddEs we have a huge list of conformer energy differences that we average (sometimes with weihgts), and so with the pairwise approach I’d just propose flattening the matrix and averaging again. Possible that there’s more research to do on weighting, and some overlap with fitting method.

    • MT – This seems possible once we want to go for it.

  • Also, with ddEs, we compare everything to min conf, but if RMSDs are too far away then it may not be useful. So maybe adding an geometry threshold would be helpful.

    • MT – So, if RMSD was way off, we’d remove from dataset? Affecting like 1ish% of mols?

    • LW – Yes, small %.

    • MT – I can probably get started on this when you want, will have a few details to discuss but I understand the concept.

    • LW – Could probably start now, but with a big default threshold (like 10A).

 Action items

 Decisions