2024-01-09 alchemiscale : dev group meeting notes

Participants

This meeting was cancelled due to D. Dotson illness; agenda with written updates below

Goals

• DD : development strategy through 2024

  • enabling the “living networks” usage pattern, in which networks are “grown” iteratively over time to take advantage of greater statistical power of ever larger networks for free energy estimates

  • automated Strategy execution for automatic, optimal compute allocation

  • anything else?

    • MH : admin interface points?

    • JC : fair-share scheduling to allow opening up use of resources to other users

      • don’t want projects using a shared resource to dominate

• alchemiscale development : first sprint of year spanning 1/10 - 1/22

  • architecture overview : PL Benchmarks on FAH - Architecture v6.drawio

  • coordination board : alchemiscale : Phase 3 - Folding@Home, new features, optimizations, targeted refactors

• DD : impending release 0.2.2 features:

  • the ability to set and get Task priority, set and get AlchemicalNetwork weight relative to others, and set and get actioned Task weights. These give users several levers for control for getting results of greater interest more quickly. Big thanks to Ian Kenney for working on these pieces in detail.

  • vast improvement to AlchemicalNetwork submission and AlchemicalNetwork, Transformation, and ChemicalSystem retrieval through smarter serialization via use of keyed dicts, thanks to work by Ian Kenney and myself to solve alchemiscale#216: `AlchemiscaleClient.create_network` scales poorly with increasing `AlchemicalNetwork` size · Issue #216 · OpenFreeEnergy/alchemiscale

    • users should see very fast submission times compared to previously, even on relatively slow internet connections

    • "large" networks (>1000 chemical systems) should be ingestible without issue

• DD : alchemiscale-fah status

  • still working on components of compute service in this PR, in particular:

    • graceful restarts of service that make use of its own leveldb-based index to avoid expensive filesystem searches

    • graceful replay of Tasks inflight if service goes down

    • selection of Tasks with compatible Protocols only

    • automatic certificate renewal

    • automatic project selection based on estimated computational effort (atom count, nonbonded settings)

    • need to build out test suite with mock WS presenting API endpoints we actually use

  • will pair up with Ian Kenney on this to accelerate development where possible

Discussion topics

Action items

Decisions