Page Comparison

Support for torsion benchmarks

MT

• Mega PR:

  Github link macro
  link https://github.com/openforcefield/yammbs/pull/76

  • Early on we decided not to shoehorn TD models into moleculestore. Currently working at the level where we can process things in QCArchive. One question I’ve gotten is which question people care about. I’ve gotten suggestions from JHorton, LW, BW. Did development using 20 torsiondrives.

• Targeting version 0.2.0

• Any other metrics we want to add?

YDS updates

MT

• Torsions running as a prototype, possibly not ready for production yet

  • We want to get this running on AWS as well, seems to be working well as a prototype. BW and I should work on this.

  • BW – I’ve run some more benchmarks on opts which have gone fine. I pulled up the last few workflow runs from before thanksgiving, the last one crashed with a math domain error (happened during statistics stage, after minimizations).

  • MT – Let’s handle this between you and I. Seems like “coming up with useful torsion metrics” and “calculating metrics in YDS” are in a chicken and egg situation. Want to figure out how to break out of this cycle, and get something in released version. Two steps:

    • Get large scale torsion metrics running on YDS

    • Tag YAMMBS 0.2.0

    • Point YDS at YAMMBS 0.2.0 release

  • MT – PR is way too big to review (would take several days), though portions have been reviewed. Could see the previous runs as being evidence that things are running ok.

  • LW – The plan above sounds good to me.

  • BW – I think everything’s in place on the YDS side, except bumping version of YAMMBS that’s being pulled in YDS in the yaml file .

Trello updates

MT

https://trello.com/b/dzvFZnv4/infrastructure?filter=label:benchmarking

MT – What are other things that people might want out of benchmarking infrastructure in the future?

• LW – Not many fleshed-out ideas. My workflow is vaguely painful though. Sometimes we run run FE calcs and visualize the outputs with bootstrapping, and I currently have dozens of versions of the script that I’ve modified for different runs.

• …

• LW – What is envisioned as the interface for SFEs? Evaluator?

  • MT – … (we planned to eventually handle phys prop calcs, eg using evaluator) …

• JC – This reminds me of a package someone was making at nist - iprpy - they used it to test FFs for metals. It sounds like this does something similar to what YAMMBS does, but possibly more generalized. It has general calculation classes that can dispatch to ex. LAMMPS, and also have a web-based visualization frontend… I used this as an engine to run a hyperparameter scan, and it also has a job orchestrator - handles database side using homemade pacakge called yabbadabba. Suggested devs could generalize further but they were bandwidth-constrained.

  • https://www.ctcms.nist.gov/potentials/iprPy/setup.html

• MT – … How is compute scaling handled here?

  • JC – Runs on HPC, internally uses python multitasking to distribute jobs, some homegrown results storage and task distribution… My workflow in general benefitted from have my analysis pre-generate all plots so I could browse through them. Had issues with rerunning just a subset of calcs, would often require rerunning all calcs to get a second run of just a subset.