Problem
The calibration diagnostics dashboard can currently consume the public microplex-us artifacts under artifacts/, but those files only expose summary-level parity/regression information:
pe_us_data_rebuild_parity.jsonlive_pe_us_data_rebuild_checkpoint_modelpass_regression_summary_20260410.jsonlive_pe_us_data_rebuild_checkpoint_national_irs_other_drilldown_20260410.json
The parity artifact reports headline counts and losses, e.g. n_targets_kept, n_national_targets, n_state_targets, win rates, and aggregate loss metrics, but it does not include the target-level rows. Because the microplex H5 and full target diagnostics are not public, downstream tools cannot produce a truthful full diff between microplex and policyengine-us-data by target or aggregate.
Requested artifact
Please publish a run-level artifact that contains the full per-target comparison against the same calibration target oracle used by policyengine-us-data. Either of these would work:
- A full target diagnostics JSON/CSV, committed or otherwise publicly downloadable.
- The microplex output H5, publicly downloadable, so downstream services can compute the same diagnostics.
Suggested target diagnostics schema
A row-oriented JSON or CSV would be easiest for dashboards/API consumers. Suggested fields:
run_id / artifact_idbaseline_dataset, e.g. enhanced_cps_2024.h5candidate_dataset, e.g. microplextarget_idvariableentitygeography / geo_level / state where applicableperiodtarget_valueus_data_aggregatemicroplex_aggregateus_data_absolute_errormicroplex_absolute_errorus_data_relative_errormicroplex_relative_errordelta_absolute_errordelta_relative_errorloss_contribution or equivalent weighted termfamily / target groupin_losssupported_by_microplex
Why this matters
The calibration diagnostics dashboard now has a microplex-vs-us-data page and API endpoint, but it can only show rollups from the current public artifacts. Analysts want to answer questions like:
- Which exact calibration targets does microplex beat or miss relative to us-data?
- Which aggregates drive the loss delta?
- Are failures concentrated in IRS SOI, state AGI distributions, ACA, SNAP, CTC, etc.?
- For a state/federal reform analysis, which upstream calibration nodes are trustworthy or suspect?
Without target-level rows or an H5, the dashboard has to label the view as aggregate-only and cannot provide a full per-target/per-aggregate diff.
Downstream consumer
This is needed by PolicyEngine/calibration-diagnostics PR #9 and follow-up API work. Once this artifact exists, the dashboard can add an endpoint like /microplex/targets or /microplex/diff to expose the full comparison table.
Problem
The calibration diagnostics dashboard can currently consume the public microplex-us artifacts under
artifacts/, but those files only expose summary-level parity/regression information:pe_us_data_rebuild_parity.jsonlive_pe_us_data_rebuild_checkpoint_modelpass_regression_summary_20260410.jsonlive_pe_us_data_rebuild_checkpoint_national_irs_other_drilldown_20260410.jsonThe parity artifact reports headline counts and losses, e.g.
n_targets_kept,n_national_targets,n_state_targets, win rates, and aggregate loss metrics, but it does not include the target-level rows. Because the microplex H5 and full target diagnostics are not public, downstream tools cannot produce a truthful full diff between microplex and policyengine-us-data by target or aggregate.Requested artifact
Please publish a run-level artifact that contains the full per-target comparison against the same calibration target oracle used by policyengine-us-data. Either of these would work:
Suggested target diagnostics schema
A row-oriented JSON or CSV would be easiest for dashboards/API consumers. Suggested fields:
run_id/artifact_idbaseline_dataset, e.g.enhanced_cps_2024.h5candidate_dataset, e.g.microplextarget_idvariableentitygeography/geo_level/statewhere applicableperiodtarget_valueus_data_aggregatemicroplex_aggregateus_data_absolute_errormicroplex_absolute_errorus_data_relative_errormicroplex_relative_errordelta_absolute_errordelta_relative_errorloss_contributionor equivalent weighted termfamily/ target groupin_losssupported_by_microplexWhy this matters
The calibration diagnostics dashboard now has a microplex-vs-us-data page and API endpoint, but it can only show rollups from the current public artifacts. Analysts want to answer questions like:
Without target-level rows or an H5, the dashboard has to label the view as aggregate-only and cannot provide a full per-target/per-aggregate diff.
Downstream consumer
This is needed by
PolicyEngine/calibration-diagnosticsPR #9 and follow-up API work. Once this artifact exists, the dashboard can add an endpoint like/microplex/targetsor/microplex/diffto expose the full comparison table.