2020-10-15 Meeting notes

10

WBO interpolated parameters update

Jessica &
Pavan

20

Standard benchmarks

Jeff and Josh

• Which metrics?

  • benchmarkff dataset

  • Hyesu’s pre-release benchmarks

• Which data and software?

  • Prepare a clone of QCA data for local running/comparison (benchmarkff)

  • Make sure everyone can run Hyesu’s pre-release benchmarks

• Short-term solution vs. long-term solution?

  • Short term: Exactly the dataset/software used in existing studies

  • Long term: Carefully curated benchmark set, potentially hidden from team. Rules about which overlapping data can be used during training.

    • Could have tool to compare degree of overlap