JW – Details on Janssen benchmarking? Scope? DH – I’ll be doing the legwork on the Janssen side. I’m not sure what is required in the other companies. Would be good to have more tutorial-style resources to show others how to get to similar plots of Victoria Lim. JW – Would prefer acting in a support role on this and help unblock issues. Would like to avoid talking to 10 companies at once. DH – Agree. I’m also looking at doing an analysis on PDB ligands. JW – I’ll assume that Janssen will take point on this and ask us for help unblocking as needed. Also we have a workflow for protonating PDB ligands –
Roundtable updates DH – Working on expanding benchmarking. Working on wrapping up Lim paper. Conformer benchmarking is on the to-do list. SB – Redone a lot of nonbonded code. 99% codecov. Babysitting nonbonded optimization. About a week in, probably about 75% done. Finishing a charge refitting study but encountering weird bugs/errors wrt grid spacing. Working on recharge CLI.
MT – Devops black hole. Lost track of scope and had some trouble with using the simplest solution/deciding when to explore new ideas or implement what I understand now. Conformer generation CLI PR ready for review. Still some ambiguous behavior – Looking forward to feedback. Preparing/speccing out interoperability work under MS’s new grant, will be in contact with Vanderbilt group who is also on the grant. Thinking about which parts of intermol I can reuse and things like equality operator. Also speccing out to_parmed in the short term. JW – would be interesting to programmatically compare different functional forms. Could be a very extensible design in the long run. MT – True. Alternatively, if everything can convert to the SAME format, then equality can be tested by converting to that format and testing exact equality of output.
DD - Mostly meetings last week, not a ton of implementation work. Current highest priority is submitting protein set. Found a potential bug in protein dihedral constraints (could be an off-by-one indexing error). Re: Benchmarking, TG and I are looking at building more functionality for benchmarking. I have a meeting with XL and GT this week about running ANI QCEngine jobs. How much overlap is there between DD and TG’s benchmarking infrastructure and industry benchmarking efforts? JW – I think that the industry benchmarking needs to take flight in the next few weeks, and the benchmarking infrastructure should be very abstract/configurable, but could take 3+ months to roll out. DD – Kinda disagree – We could just initially support+formalize the Lim benchmark routines. People won’t expect this infrastructure to work on the first run. SB – Agree with DD – Pursuing two routes will probably result in us not making progress on the long-term arm. MT – How bad would it be to release a 0.1 that breaks all the time? JW – Industry people won’t be tolerant of us making breaking changes every week. Industry-person time and attention is valuable and they’ll get annoyed if we’re having them regenerate their datasets with some regularity DD – How willing are industry people to test unstable code? Which groups are willing to use unstable APIs? JW – 2 or 3 (but struggles to name them) – likely Janssen because they have DH. DH – Industry people will struggle to find time for this sort of thing. Best tool would be “dump in SDF files → get results”. They likely don’t expect it to be THAT easy, but would prefer that it be like quite so simple. SB – Maybe MVP of benchmarking workflow should codify Lim routine. Then it can build from there. We could point industry benchmakers at the 0.0.1 release for the purposes of this study, and then let them upgrade. JW – What’s the difference between having very different 0.0.1 and 0.0.2 packages (where 001 is for this benchmarking study) and simple giing them different names/treating them as entirely different packages? (General) – To what extent do we want to control/constrain their conda environment? Should we fully pin their environment for the purposes of this benchmarking study? DD – Since we’re going to make a benchmarking infrastructure anyway, and the Lim paper is the most current benchmark, we will be reproducing that as an MVP regardless. JW – Concerned that building two script pathways to do the same job will make us commit a lot to finding differences between them, or risk throwing out datasets if different parts of them were generated usnig divergent pathways. DD + DH – People won’t expect exact equivalence/comparability between separately-generated parts of the dataset. MT – DH, what kinds of dataset/workflow differences AREN’T concerning to you? Package version RDK vs OE conf gen Using gaussian instead of psi4 Losing track of stereochemistry in 10% of molecules Misinterpreting constraints on constrained optimization Resulting datasets not able to made public
DH – We could ship initial workflows and ask other companies to run them on 10%ish of the Lim datasets. Then we could get an estimate of error resulting from running on different systems. This would also generate bug reports that we’ll need to fix, and it’ll be easy to share them because it’s a public set.
Stances: Janssen OpenFF One of the following: OFF will do nothing out of its way “Separate infrastructure” – OFF will update Lim scripts into a conda package “Parallel infrastructure” – OFF will make first iteration of benchmarks package mirror behavior of Lim study
We’ll start polishing benchmarkFF – JW and JH will improve deployability+packaging, DD and TG will begin working on improving extensibility/slotting it into larger automated benchmarking/mol submission information loop. DH will work on the industry side to do test runs and give feedback. DH will join DD/JH/GT/XL call later this week.
Losing atom ordering in PDB
JW (prepared notes, some out of date): Pavan onboarding -- Going well. He started with a Toolkit feature to fix PDB writing. Guided making some test cases, he write code to make them pass (controlling OE write flags), despite not having used PDB before. Also walked him through the meaning of SMILES and SMARTS, emphasizing information in graph molecules.
Infrastructure roadmap review went well, I think. Will continue this week.
Not sure how to approach committing to benchmarking infrastructure push. Seems like it will be "very high priority" as soon as we commit to it, because we want to make a good impression on pharma.
OpenMM C-F migration is moving forward again
Met with Marti Municoy (OFF-pele) Friday morning to apologize for being wrong about dihedral phases, talk about how he can get final values for interpolated torsions, thinking about ways for them to validate their torsion implementation with variable phase
Optimistic about System progress/interoperability. Worked with MT to make MVP for alpha release
Worked with JH to define MVP for bespoke workflow
FF port is coming along. Working on resonance bonds now, then will try to pull in and parameterize protein structures from a variety of sources.
|