2022-11-22 Protein-ligand benchmarks meeting notes

JW + JC : Sketching out planned submission operations, personnel-time requirements, submission prioritization:

• What will actual submission operations will look like?

• Will we provide submission support/debugging (and to whom)?

• How will priority be assigned to submissions.

• JC – Requirements for OpenFF are that we intend to benchmark various FFs against the protein-ligand benchmark set. Then there will be a bit of engineering here. But you’re just looking to do Rosemary vs. Sage vs. GAFF. So eventually we can make this automation, but we can do the first few manually/using raw scripts.

• JW – THat sounds good, but how will we manage priority between the orgs here?

• JC – We can handle that on the fly. We get an even share of F@H between us, and ASAP will be bursty while OpenFF will be longer-running.

• DD – ASAP’s deliverables are more time-sensitive

• JC – I’d like ASAP’s submissions to take priority, but there will be a lot of gaps between submissions

• DD – Can scope this based on (org, project, (something) ) tuple. Can assign weights that way.

• JC – There’s priority as well?

• DD – I’ve looked into replacing taskqueues (linked lists) with task hubs, which would let us use more flexible weighting schemes.

  • https://github.com/openforcefield/fah-alchemy/issues/23

• JC – How about we do equal allocation until we hit a throughput conflict?

  • JW – That sounds good to me.

• JW – Also submission bugfixing/troubleshooting - The QC submission review staff has become defacto submission troubleshooters, which sometimes eats up days or weeks. Who will be doing that in this case, and what are the limits to time expenditure?

  • JC – We have Hugo coming online, and JScheen can help too

  • JW – We’ll have DD working on this from our end, and later MOsato

  • DD – I can do this for the first months.

• (General) – It may be good to have a shared resource pool that we all contribute to so there’s a person which primary responsibility for maintaining this infra

• RG – I don’t think this would be super beneficial for us, since we’ll be changing things that may go beyond the scope of what F@H supports. The way we’re planning on doing things won’t be amenable to porting to F@H. The GROMACS stuff is quite complex and I wont' be able to propagate our workflow changes to that.

• IA (chat) – we'll definitely be interested in being tied into the PLB simulations, but I agree with Richard, we would need time to think it through

• DD – The F@H compute service is just one system that can tie into this system, we may also be able to connect this with local compute, PRP, etc. So this may become useful in the future.

• JC – Beyond the engine, you’ll be able to use this to experiment with different network planning sterategies

• JW – I’m fine to commit to giving 10 hrs/week of OpenFF personnel-time for the first half of 2023 - First DD, then MOsato.

• JC – Yes, probably more than that from my side.

DD : fah-alchemy - current board status : fah-alchemy : Phase 1 - MVP

• focused on 0.1.0 milestone, with outstanding issues required for deployment :

• still driving for 11/29 deadline for MVP

  • Test suite coverage at 81%.

  • Now have authentication on both user and compute API, abstracted as separate FastAPI APIRouters

  • Both user and compute clients feature same authentication machinery, different authentication entities

  • Have ProtocolDAG execution in SynchronousComputeService; hitting issues with serialization of ProtocolUnitResult outputs; would like help from David W.H. Swenson

• DS : fah-alchemy CLI -

• current state and next steps

• DS – Goal is to be a simple wrapper with very liottle code. It seems to work for launching the APIs and services, what I ahven’t done yet is testing. I tried to do is single-process testing which is taking too long, but it looks like I’ll need to parallelize tests, so the error messagesm amy come out in the wrong order. May require some env variables or additional commandline parameters. Any thoughts?

  • DD – Commandlne parameters are a bit more secure, env variables are less secrure and since they may leak secrets mroe easily in debug logs

  • DD – Also good to have the run only have one security key. Something with JWT tokens.

• JC – Will there be documentation for launch API somethwere?

  • DS – The CLI docs are easy to add to sphinx - this is native in click.

  • DD – Docs are a priority in release 0.2, they aren’t a hard requirment in release 0.1

  • JC – Understood, it’d be good to have examples for what they’ll look like available.

DD : help wanted - deployment issues :

https://github.com/openforcefield/fah-alchemy/issues?q=is%3Aopen+is%3Aissue+label%3Adeployment

• JC – MH said he’d be happy to help with this, feel free to reach out!

• DD – I’ve assigned him to this.

• JW – Maybe LN as well?

• LN – What could I help with?

• DD – Most easily separable items would include the full deployment picture

• LN – That could work well, talk after Thanksgivign? I’m interested in robustness of the system and work deduplication. I can help most with technical stuff, can’t get too deep into the science though.

• DD – Great - I’ll expand out some items with details and see if I notice separable parts.

IP : Nonequilibrium Cycling Protocol (perses#1066) update:

• IP – I finished writing all the settings as GUFE settings bojects, I’m using the baseprotocolsettings, didn’t see a way to use the other settings objects. So this is similar to RG’s approach for the repex protocol.

• IP – I got the simulation unit working with that, now I have to make the protocol run, and I’m facing some issues that I could use a quick glance at.

• (IP shares screen)

• (DD helps)

• ….

• (soething about unit serialization?)

• JC – Is there a common test input that we can use? It sounds like the root cause might be quantities from mixed unit packages?

• DS – Currently most of the inputs are coming from default valuess, so the issue here is kinda that we’re changing the defaults

• DD – …

• JC – How do settings know whether they’re compatible with a protocol?

• RG – You get default settings form a classmethod on a protocol the settigns are defined in the same module as the protocol . But I wouldn’t expect defaults from one protocol to work in another. It’s not organized very well currentyt since everything is defined together. So it would be hard to make a new protocol outside that file at the moment.

• DD – IP, you need to make a subclass of models.py::Settings

• (some question as to exactly what type something like termperature should be, which unit/models package, which settings object this should be in (whether it would confkict with temperate in ThermoSetytings)

• DS – Maybe there needs to be a specific pip-installed version of units package as well?

• (Tehre may be some unreleased changes in openff-models, OpenFF is happy to make a new release+conda package, Matt’s waiting to hear from Mike)

DD : requirements for using OpenFE REPEX protocol for initial testing alongside Perses NonEquilibriumCycling?

• RG – This is complicated… (see Fathom recording)

IA : protein-ligand-benchmark : blockers and priorities

• Adding missing thrombin entries and then good to go

• Waiting on #81

• Waiting on one of the networks to return (PR is up)

• IA – IP , are the network files in #83 suitable for you?

  • IP – I’ll look this week

• IA – JC, there was some thing with AMBER files.

  • JC – I think maybe Toni May mentioned this

  • RG – I think it was tleap in particular

  • IA – I can do that. Though I recall that we discussed whether we want parsing tests for PLB 0.3 and we decided that we only wated OPenMM compatibility

  • JC – Is the issue that we’re not PDB compliant, or that tleap doesn’t read PDB compliant files?

  • IA – I’ll follow up on this. It may have to do with the recognition of protonated residues.

  • (General) – Is this a problem with tleap/pdb4ambewr, or with or inputs? We should make sure our files are PDB compliant, but beyond that we can just inform the AMBER devs.

  • IA – I’ll follow up on this.

• #82

  • IA – What to do with gz files?

  • JC – Move to Git release artifact and expunge from git history

  • IA – Will do.

Transcripts