2022-07-05 Protein-ligand benchmarks meeting notes

Need for ProteinComponent to support multiple structures?

John Chodera

• JC – In general, we’ll have multiple xray structures available for a single transformation. It’ll be important that we have some way to represent this. For example, we have >800 structures for COVID moonshot. In some cases, we have multiple reference structures for the same compounds. It would be great if we could gain information from all of these to make more accurate predictions. So, can a single node hold multiple protein structures? Will one edge strictly go between two protein conformations?

  • RG – I’d suggest that different protein conformations be different nodes on the graph, connected by special edges that have 0 energy difference.

  • JC – That sounds reasonable.

  • IA – RG and I talked about this earlier, and we also thought about evaluating the free energy difference between two protein conformations. For example, you’d see this in HSP90 with the loop around the binding site.

  • JC – Also, different ligand protonation states can lead to different preferences in pocket conformation.

  • LN – This can also help evaluate uncertainty.

fah-alchemy - current board status

David Dotson

• fah-alchemy : Phase 1 - MVP

• 8 weeks out from 9/1 deadline for biopolymer benchmarking

  • DD – Recall that there are two milestones - OpenFF for Sep 1, and ASAP for Oct 1.

• first protocol is fah_alchemy.protocols.FAHNonEquilibriumCyclingProtocol, developing in parallel to perses.protocols.NonEquilibriumCyclingProtocol from components in perses

  • DD – Talked with RG about which parts of F@H alchemy can be reused for OPenFE, determined that there’s not much in common.

• FAHScheduler and FAHExecutor development to follow

• DD – Worked on S3 external storage infra

• IP – Will these components be using gufe objects? .... Perses will be the home for a variety of protocols, which could be executable on their own.

  • JC (chat) – David will just draft the skeleton that uses the gufe API and objects and then the perses devs can fill in the body.

  • IP – Sounds good.

ProtocolSettings taxonomy working group update

Mike Henry

• MH – Met with MThompson, learned that SMIRNOFF spec contains a lot of info about how to get procedure right. Though in talking with RG, we determined that we don’t want people to be forced to use the SMIRNOFF spec, if they have another FF that they’re prefer to use that doesn’t have all that info.

  • JC – Is the idea to have a data harness that sits around the force field object that can fill in items?

  • MH – Yes.

  • JC – I like that idea. I’m happy to work with that working group. And I can help implement this in openmmforcefields

  • JW – Note that there are many default values in SMIRNOFF spec, so it may provide reasonable defaults for your infra as well.

  • MH – That’s true. But I want to make sure that we’re a bit flexible for folks with really specific protocols.

• MH – I’ll meet with folks about this and present more in the next meeting.

  • DD – Do you want to open a PR to gufe with your initial thoughts?

  • MH – Sure, I’ll do that when it’s ready to go.

• RG – I’m working on a replica exchange protocol. I’m wondering if this should get pushed into Perses. I do have some hesitation regarding versioning (like, if all the protocols are in perses, how can you independently version just one of them?)

  • JC – This touches on questions like “are protocols tied to engines?”

  • RG – This protocol may be OpenMM-specific, at least initially

  • JC – Maybe a good rule would be that OMM-specific things could go in Perses, and engine-agnostic things go in GUFE. It seems like anything we choose now will lead to complexity/code duplication, so we should just accept that that will happen.

protein-ligand-benchmark - 0.3.0 update

David Dotson

• blocked by protein-ligand-benchmark#52

  • DD – IA, we mostly have resolution on the PDB front, but could you weigh in?

  • IA – I highlighted some things that should change in the formatting of our PDBs - Things like HETATMs and other cases. I’m happy with the current state of the discussion and will bring the PDB files in line as soon as I can. I don’t have access to schrodinger so the hydrogen names may change again. There are more things that I’d like to discuss but we can make further changes in the next release.

  • DD – With the schrodinger/non-schrodinger touchpoints here, it seems like this preparation pathway is getting complex. Is there a way to smooth this out?

  • IA – I don’t think it will be too complex. There are other tools that assign amber-compliant atom names.

  • DD – One approach we could take for future targets is to enumerate specific characteristics and check for them using an automated tool. So what you’re doing manually this time could be turned into automatic validation.

  • IA – I’d talked with JC about this - A lot of what I’m doing is based on the issues I’m running into when trying to load these structures into AMBER and BioSimSpace.

  • JW – I saw some discussion on this issue about how to handle multiple polymer/small molecule components. The OpenFF toolkit will eventually be able to load multiple proteins from one PDB.

    • IP – Components in separate files is OK

    • IA – I’m in favor of multiple components in one file. In 0.5 I’d love to discuss the API further and figure out how to avoid using pathlib.

    • DD – Happy to discuss this further in the future, right now the API expects separate files.

  • IA – Regarding separating components, I’m fine to keep them in separate files.

  • JW – exactly one protein in a PDB file is current support; so having cofactors and other things in separate files would be best. In the future we can support more complex things

• attempt to retain original ligand networks / atom maps?

  • DD – IA and RG were going to generate new ligand networks with the updated inputs.

  • IA – We’re currently passing everything through the OpenFE toolkits. Using lomap to make a minimum spanning graph. We’re hand-validating this, since we know that lomap has limitations. Ben was working on a wrapper for perses to… mappings in a yaml file.

  • DD – PLBenchmark up until now had edges for a given target already defined in the 00_data input folder. But what now we’re going to have optionally host pre-defined networks in 03_networks.

    • JW – Sounds good

  • DD – I’m working on casting our old edge data into the new format. Unfortunately, some original ligands couldn’t be re-docked, so we can’t make the same graph. So we can make connected graphs for all but two targets (network is totally disconnected because the original network was connected through a node that no longer exists)

  • IA – How were networks originally defined?

    • JW + IP – We’d need to ask DHahn

  • (Some discussion about the impact of not exactly replicating DHahn networks/studies)

  • JW – I’d approve NOT requiring reproduction of those results. The structures are redocked, the infrastructure/protocols are different, etc. There’s so many other things that could throw off our results that I wouldn’t say this is a clear value add.

  • DD – Let’s make a formal decision about this at the next meeting

    • IA – we should still check with DHahn about how these networks were made, to ensure there’s not some special experimental rationale for the way that DHahn made the networks.

• IP – Have we heard about BioSimSpace’s use of these tools?

  • DD – Other folks are working on a manuscript. I’m not sure that melissa/toni are watching this PR and what they think of the discussion. They’ll need to chime in if they agree/disagree with something.

Gromacs support - initial protocols for OpenFF benchmarking

David Dotson

• OpenFE plans for Gromacs protocols?

  • RG – We don’t have any concrete plans for this currently, though we’re still formulating our year 2 plans. It’s not clear how much effort we want to put into supporting different engines - If the methods are designed to be the same, why implement them twice? Realistically if we DID implement a gromacs backend it would be to support a protocol that can’t be handled anywhere else.

  • DD – Agree with that rationale. It would be interesting to see whether different engines give the same answers given the same protocol, but that’s not that valuable. But what would be more informative to be able to compare entirely different protocols to see what works best.

  • IA – Also note that top-level protocol settings will differ a lot between engines. So those could also lead to different results. A large-scale comparison would be a LOT of work.

  • DD – The ASAP effort won’t be using gromacs, it’ll instead use OpenMM. I think that OpenFF may prefer to use GROMACS, but I’m not sure. Right now the project board is all OpenMM… Nonequilibrium switching vs. replica exchange? The latter is seen as more accurate, but it isn’t compatible with F@H.

  • JW – I’ll ask the OFF team about whether we’ll need GROMACS support and pmx, and will have an answer for next week

  • IA – OpenFE is going to be using replica exchange. But it’s not clear it’s better than nonequilibirum switching. In theory they should give the same answer.

• concerns with use of pmx, other alternatives?

  • (no comments)

• IP – In general, we’ll want a way in the future to know whether differences are due to changes in engine or method. This will help us validate GUFE.

• RG – I’m planning on there being a breakpoint before simulation, where you can dump out the files/configurations. So that would be a good place to inspect differences in methods.