2021-08-06 QCFractal User group meeting notes

• DD – BP is teaching this week, won’t be joining this week.

• DD – Industry data set had two issues, one regarding scale of error cycling (discussed below).

• DD – Questions on existing datasets/reprioritization?

  • PB – Is there room for more compute? One of my submissions is stalled even after upping priority. (#217)

  • DD – The PRP usage monitor shows under-utilization of many QM workers. Could be implicit solvent/PCM.

  • DD – Looking at the jobs, it is indeed PCM. #211

  • SB – We can drop those down in priority

  • JW – Could we make single-core workers that work on a tagged datasets?

    • DD – Yes, we could make special single-core workers on a compute tag

  • SB – Could we try to finish the basis_set=/uf portion of #211?

  • DD – I can push this through error cycling a few times. Unfortunately I can’t automate just resubmitting /uf jobs, but not the PCM ones. So I’ll take these off automatic error cycling, and I’ll manually error-cycle the /uf submissions.

• DD: will kill off PRP managers, start fresh in order to flush existing PCM jobs clogging resources

• DD – New submissions that we need reviewed?

  • DD – I have #218, the MM-after-QM dataset. This is a separate dataset entirely from the big industry dataset (the old one is MM-simultaneous-with-QM dataset).

• JW – We had talked about a grid optimization submission (Nitrogen-grid-optimization) that was somehow different from others. Will this different way of submitting possibly affect our issues of scale?

  • DD – No

• DD – We’ve had an issue with deploying psi4 where they deleted the rc package that we were installing from. So I’ve talked to them on the psi4 slack and they apologized and were looking to make a different release for reverse-compatibility

  • (General) – They just released a psi4 rc3 last night, which we should be able to deploy where we need

• SB – We have biopolymer grid optimization likely in the near future. So QCSubmit will need to update for this, as well as qcengine.

  • CC – And the plan for this is to use a gridoptimization as an ALTERNATIVE to a full 2D torsiondrive. Thinking of 2d scans of like (phi, psi), then (phi, chi1), (psi, chi1). Also looking to try doing this as a two-stage submission, where we submit first with a cheap QM method, and then once that’s done, use a more expensive level of theory.

  • DD – So idea would be like a gridoptimization/torsiondrive. Do those top out at 2D?

  • CC – Yes, but I’m just planning on 2D.

  • DD – So, we need QCSubmit support for 2d gripopt,

  • CC – We’d do a 2d gridopt or torsiondrive as the initial set, then a constrained optimization set as the second step (likely with all the torsions constrained)

  • DD – I think we have better support for torsiondrive at the moment, so that would be the best starting point, and we could study gridopts later

  • CC – Agree, I’m planning to do torsiondrives for other proof-of-concept studies.

• DD - let’s meet for a working session on manager deployment around Thursday next week; aiming to get new psi4 1.4 release in prod by then

D – Update on MM dataset submission

• CC – Is the best way forward to keep scaling up this submission, or break it into separate submissions?

• DD – I’ve been able to find a way to, instead of downloading every field for large datasets, to just grab a few fields like job status. Longer term solution will be to get rid of the need to do external error cycling entirely. So there, you could set an error-cycling policy on a per-collection basis. This would set a number of max retries, and the frequency/interval. Then the server would amnage t, and we wouldn’t need to push the whole dataset through a pipe onto an external computer.