Partner benchmarking 2022
|
| DD – what would you like to see for benchmarking in 2022? DH – initially thought it would be good to repeat the benchmark every year, but not sure we would get more useful conclusions by just repeating at the moment might need to do something more specific, like looking at one compound pattern, e.g. charged molecules, biaryls, torsiondrives, etc. something to probe more non-bonded interactions in short, should be something different than Season 1
DH – know from another partner that it is important we show that OpenFF has made progress, use of the toolkit internally probably a better investment in time from partners this year to get OpenFF tooling applied by partners, integrated into actual workflows if a second benchmarking season leads to a workflow, that would be a positive outcome that gives us both DD – this is key; a season 2 is an opportunity to get internal use widespread among partners, and forces development team to not build bespoke tools to support a protocol, but instead engineer the components needed for general use that are then used in a protocol
JW – I wonder which format for these tools makes sense? we have Python library components, but then also need CLI-based tooling could the steps talk to each other with serialized OFFmols, but have users input molecules or output molecules as SDF
JW – benchmarking v2 is probably 3rd or 4th in big deliverables this year we need to get rosemary out first, though but do want to draw a distinction between benchmarking and CLI tools if we make CLI tools for benchmarking v2, it becomes easier, but perhaps overfit to the problem
DH – perhaps not the prerogative of OpenFF to build tools? Developing library components priority, partners can use to build tools DD – but if they don’t know how to build tools, this presents a problem for all parties DD – I had a good time working with XLucas, and I think this was productive. I wonder if we want to have a tooling effort where we work WITH the partners to design a CLI. By actively engaging, we can get direct feedback on which functionality is most useful. I’d be interested in that kind of work. DH – That’s a good idea. It could be framed as a benchmark, but focused on one partner. DD – This would be great
JW – I’d be interested in having the infrastructure team do some limited pharma user support/development, but I’m not sure when we’ll have time. How is timeline looking for your F@H work? DD – I could spearhead this in late 2022, run it similar to the benchmarking project. Stated goal would be to do “benchmarking season 2”, but really the design would emphasize the reusability of components and aim for people to adopt them for internal use.
|