2023-08-28 Meeting notes

Cancelling upcoming meetings

Cancel the next two weeks, next meeting is 9/18, then we will decide on 9/15 and 10/2

• (General) – Yes

Trello review

MT –

• DEXP/plugin stuff - Nothing super clear as evidence of technical issues. Might just be an area where FF is really deficient.

• Phys prop benchmarks - Blocked by HFE workflow availability. YANK is outdated.

  • LW – OpenFE solution has been a month away for several months. I’m watching the relevant issue/PR but don’t recall seeing movement. I’ll send the link when I find it.

Lily’s results

https://openforcefieldgroup.slack.com/archives/C03T3LLVC1J/p1693256913567219

• MT – Agree that results look substantially different. Not sure how to proceed. Doubt that the issue is a 1-2% of molecules missing.

  • LW – Wondering whether BW’s comparisons are seeing the same. I think there may be a difference caused by using the Reference vs. CUDA platforms. I’m curious whether this is a case of everything being a little different, or many things being identical but a few things being off. I’d like to be able to check the individual molecules.

    • BW – Haven’t done the full pipeline of PB+SB’s scripts to compate, but I’ve run the internal benchmarking workflow three times.

  • MT – I could make a better way to look up molecule records/QCArchive names/intermediate results. I’d planned on the record ids to be unique global QCArchive IDs, but if you’re seeing them ranging from 1 to N contiguously then there’s probably an issue with how the library is getting IDs. What would be most useful - CMILES+geometry?

  • LW – CMILES+geometry would be great, or just the unique QCA ID.

  • MT – I’ll need to make some sort of API point to collect this from QCA.

• LW – It’s possible that I’ve got mixed versions, since I was updating the stack a few times since I’d made this env. Would take 24 hours to rerun.

  • MT – Would be great to re-run this with latest version of everything to verify the issue remains (and at least get QCA IDs)

  • MT – Worth figuring out what our target is for equivalence between workflows

  • LW – Let’s get more data to figure this out.

  • BW – How significant are these differences?

  • LW – In the plots I posted on slack, I’d conclude that one of those FFs was significantly better than the other (even though the difference was just the pipeline)

  • JW – I’d sleep better if we had some known causes identified for the differences. Like, if it’s just an RDKit RMSD calculation change between versions, that’s fine. But if we weren’t sure why there were big differences that wouldn’t be great.

• LW – How do we feel about usability?

  • BW – I like the new pipeline a lot more than the old

  • LM – The older README was better/more descriptive.

  • LW – Yeah, once we get JM back we can make a push on documentation.

  • LM – I’ve been keeping notes so I can help on the next round of documentation.

  • MT – Was the python structure being tricky?

  • LM – Wasn’t sure how to specify molecules, references, FFs.

  • BW – Oh, that may have actually been some stuff I built on top of MT’s workflow.

• JW – QCSubmit is chugging along, mostly technical changes, might be a handful of API changes. e.g. In the old one the IDs were the equivalent of strings, and now they’re integers. I’m probably >50% fixing the tests, but saving the hardest for last, so may take a while.