2021-11-29 Core Developers meeting notes

General updates

• LW – This may be my last core-devs meeting. My fellowship ends tomorrow and I’m not on a work visa after that. Will head back to Australia after Dec and finish my PhD.

• JW – Roadmap revision is done, except espaloma question and some biopolymer benchmarking implementation details. CC will build prototypes of the latter and bring in infra team if more engineering is needed.

• MT – Big API breaking change coming to everything in early 2022 (target: Feb). This will be the result of switching the Toolkit to openff-units, and the Topology refactor. We’ll try to have all other ecosystem packages updated on the day that we do that release to control the breakage.

Roundtable updates

• CC

  • Got people to commit to getting outstanding LiveCOMS sections by the end of the week.

  • The last section left undone is the biopolymer benchmarking one, so I’ll have more clarity on those details soon

  • Submitted dipeptide torsiondrives. Updates TSCC workers environment, so those should be good to go shortly.

• MT

  • Short week last week.

  • Tried and made some progress on the reference energies package. Quickly ran into cases/details where the toolkit doesn’t handle things correctly, though it’s some question as to whether the toolkit should. Also found some issues with the SMIRNOFF spec. So I opened up some OFF-EPs, but this took a lot of work since it’s a lot of writing. The first ones were fairly straightforward and I expect easy approval. But there are some hard ones coming.

  • Got a support request from Roitberg group member. The issue was fairly unclear/somewhat low quality. Also the email went to my personal email which I don’t like. Generally I’ve seen a lot more issue/question traffic from external users, which is good, but we should make sure to be disciplined about it so we still get our own work done.

    • JW – I usually copy+paste email queries into the issue tracker

    • JW – JMitchell should be able to handle a large volume of user support if it starts getting on to public issue trackers.

  • Worked on toolkit refactor to make OpenMM optional. Big lift here is to remove openMM units and replace with openff units. Except for some possible edge cases with GBSA and VSites, this should be in good shape. Most of the test suite looks good.

  • For the topology and units refactor, we could use feedback and user testing – Please get in touch if you’re able to help here.

  • M1 builds are almost supported. One remaining big sticking point netcdf-fortran

  • Wrote+published interchange blog post https://openforcefield.org/community/news/science-updates/interchange-update-2021-11-10/

    • JW – With install instructions and working code examples!!

  • LW – Re: API breaking change – Should we stage other API breaking changes with the Feb release as well?

    • JW – We don’t really have a plan/policy, now that I think about it. For now, the best thing to do would be to open API-breaking PRs to the topology-refactor branch instead of master so that we can consolidate all the API breaks in the Feb release.

• DD

  • Last week sick Wednesday, Thursday; took Friday off.
    Worked two days last week.

    1. QCArchive

       • reviewed QM submissions from Pavan in support of OpenMM

       • we hit submission issues on some of these related to network routing on public QCArchive; worked with Ben on workaround

       • will review dipeptide submission today; apologies for the delay

       • also having production issues on Lilac; addressing today

       • Ben is continuing to work on refactor of QCFractal; aiming for next release and deployment in Jan/Feb. Working with him as a feedback loop.

    2. Protein-Ligand Benchmarks

       • met with John on Monday of the week previous

       • last week did design iterations on protein-ligand execution on Folding@Home; have v3 design, returning to this week and drawing out narrative doc for AWS client services and others' consumption

       • JW – It’d be good to make sure that we can have a MVP that doesn’t have protein prep/system setup happening on a remote host

         • DD – This is the initial plan.

    3. Partner Benchmarking

       • all partner results for Sage in, thanks to Lorenzo

       • OPLS results for public dataset in thanks to Lorenzo as well!

       • have feedback from partners via retrospective survey; reviewing with team this week to document our lessons-learned for next season

• LW

  • Short week last week

  • Refactored PsiRESP to use QCFractal (a while ago) and made docs (last fortnight), now asking for feedback if anyone uses it (last week):

  • Past two weeks: debugging with Simon (unsuccessfully so far) Yank writing to PDB error:

    •   File "/data/homezvol0/lilyw7/miniconda3/envs/psiresp-3.8/lib/python3.8/site-packages/mdtraj/formats/pdb/pdbfile.py", line 353, in write
          line = "ATOM  %5d %-4s %3s %1s%4d    %s%s%s  1.00 %5s      %-4s%2s  " % ( # Right-justify atom symbol
      ValueError: cannot convert float NaN to integer

    • Cannot reproduce on local machine (following what we think is the logic flow – loops etc make this a little unobvious)

    • I’ve run into this sort of error before – It’s probably not actually MDTraj’s fault, more likely YANK. Previous solution seemed to be downgrading to openmm 7.5.1 but this may not be solving it for us. Now clear why, and error is deeply buried in Evaluator workfow so it’s hard to pin down.

    • JW – I wonder what we can extract from this investigation - Some sort of principle when we engineer workflows to always have “access hatches” to intermediate data?

    • DD – Does this only happen on distributed DASK tasks?

      • LW – Looking into it. It’s so far happened on remote hosts, but I’m trying to reproduce on my laptop. Somewhat tricky since it took a lot of simulation to get there.

      • DD – My general solution in this kind of situation is to run things serially so I can use python -m pdb to get right to the error and look around.

      • LW – I’m not sure if setting it to one process will run it in serial. I’d need to look at this.

      • JW – I’d love to know if it’s possible to get a debugger itno there. We have lots of distribution architectures in our stack and each one has different tricks forinspecting the underlying runs.

  • Successfully used Evaluator + ForceBalance to automatically train a force field for properties that are not SolvationFreeEnergys on Slurm UCI infrastructure (hpc3)

    • Catch for hpc3: need to explicitly request gpus

    • Tiny dataset of properties for C/H molecules (alkanes, alkenes, and aromatics) did not show a lot of improvement / difference between am1bcc/resp2 charges. Trying again on a larger dataset of just alkane densities, as there are systematic difference between am1bcc/resp2 charges here and lots of density data

• JW –

  • Worked on ring stereo/pseudosymmetric representation problem - Looks like we should be able to make it work

  • Some molecule sanitization/canonicalization work for benchmarking - OPLS “sanitizes” inputs to have different graphs in some cases, like R[S+2]([O-])([O-])R → RS(=O)(=O)R, which breaks our analysis workflows. Used MolStandardize.rdMolStandardize.Normalize(rdmol)

  • Biopolymer topology refactor – Implementing quick molecule-identity checks, incorporating into parameterization loops.

  • Worked with AMBER dev (Kellon) to standardize on CCD. Will meet again this coming Fri.