2023-04-04 QC Meeting notes

Updates from MolSSI

• BP – Will put in order for new server in next few weeks, expect quick delivery. The price went down from what we’d expected.

• BP – Do you still have spare compute? We’d like to do some testing. I could send you manager configs.

  • DD + JW – Yes

  • DD – Yeah, happy to spin these up. We have a few clusters - A kubernetes cluster, an lsf, something at UCI and Max Planck. We usually submit managers as jobs and run pool

  • BP – Cool, you could do that, or I could help you set up Parsl. In the new version you can run parsl locally. Not sure if we’ll still have processpool support, maybe we’ll have threadpool.

  • BP – In the new version I should be able to have a single manager take from several queues.

  • DD – Using parsl may be helpful for alchemiscale… I’ve been taking a lot of the best parts of QCF for tihs. Would parsl manager need to be a single long-running process?

  • BP: Yeah, it does. Parsl is a bit more like a manager, DGA took inspiration from it to build QCF. You need to have something running on a head node. And, it doesn’t clean up that well, there would be some leftover processes that have to be killed.

  • BP – May reach out to parsl folks and try to fix up some of the rough bits. Project still seems active.

  • DD – Sounds good, I’m excited to learn about this

  • BP – Bit feature of the new managers is that you won’t need everything in the same environment - Using subprocess will allow us to use different conda envs, and things like docker and singularity. Would also allow using different versions of deps like psi4

  • DD – That’ll be handy - This was a big desire in the spice dataset work. Required lots of manual intervention at the time. Would this come with a system for task constraints, like “for these certian tasks, use these pinned deps”

  • BP – Looking forward to doing something like that but it’s not there now. Results already return which versions of deps were used to compute them.

  • DD – Happy to update managers any time, especially if we can keep you from needing to support multiple manager pathways

  • BP - Parsl is as difficult as dask, or pool, so figuring out one is good enough.

  • DD - JW, any updated on omsf servers?

    • JW - None so far. DLM may know better

  • JW - We have two NIH grants in line, one the renewal and the other is equipment supplement, no updates on both yet.
    

• PB – I’ll be taking a project scientist position starting April 17. I’ll still be working with DM but not in the same capacity. I’m happy to help with dataset review but I shouldn’t lead this meeting any more, since my new role will need priority.

  • DD will take over leading this meeting

  • (Meeting ownership transferred to DD)

Infrastructure advances

• PB – Nothing to mention here

Datasets

• PB – No datasets running, none coming together AFAIK.

User questions