Extract unified platform structure by Siel · Pull Request #276 · LAPKB/PMcore

Siel · 2026-04-28T21:46:18Z

Summary

extract the shared unified-platform refactor from feature/unified-platform-m1
keep the baseline NPAG, NPOD, and POSTPROB surface without the new parametric or extra non-parametric layers

…vious version the parametric algorithms were not actually supported on the same lelevel as the non-parametric ones. I redesigned the crate with solving this issue in mind. The structs are much clearer now, the dispach is smarter. On the negative side some elements might be overcomplicated as I am not 100% sure on how much flexibility some aspects will need, the current API might seem over-flexible. Will be improved over time

github-actions · 2026-04-28T21:48:44Z

Bencher Report

Branch	unified-platform-structure
Testbed	mhovd-pgx

🚨 1 Alert

Benchmark	Measure Units	View	Benchmark Result (Result Δ%)	Upper Boundary (Limit %)
bimodal_ke_npag	Latency seconds (s)	📈 plot 🚷 threshold 🚨 alert (🔔)	5.32 s (+9.94%) Baseline: 4.84 s	5.07 s (104.92%)

Click to view all benchmark results

Benchmark	Latency	Benchmark Result milliseconds (ms) (Result Δ%)	Upper Boundary milliseconds (ms) (Limit %)
bimodal_ke_npag	📈 view plot 🚷 view threshold 🚨 view alert (🔔)	5,323.70 ms (+9.94%) Baseline: 4,842.41 ms	5,073.95 ms (104.92%)
bimodal_ke_npod	📈 view plot 🚷 view threshold	967.93 ms (-28.54%) Baseline: 1,354.57 ms	1,608.45 ms (60.18%)
bimodal_ke_postprob	📈 view plot 🚷 view threshold	63.05 ms (-75.53%) Baseline: 257.60 ms	734.18 ms (8.59%)

🐰 View full continuous benchmarking report in Bencher

Copilot

Pull request overview

Extracts a unified “platform” API around modeling/compilation/estimation/results/output, migrating the existing nonparametric workflows (NPAG/NPOD/POSTPROB) off the old routines/* + structs/* surfaces.

Changes:

Introduces new public API modules (api, model, compile, estimation, results, output) and re-exports them via prelude.
Ports nonparametric execution into estimation::nonparametric workspaces/engines with shared result/output bundles.
Updates tests, examples, and benches to use ModelDefinition + EstimationProblem instead of the legacy settings/dispatch flow.

Reviewed changes

Copilot reviewed 110 out of 114 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
tests/structs_tests.rs	Updates imports to new nonparametric module paths.
tests/settings_tests.rs	Removes legacy `Settings` builder tests.
tests/results_summary_tests.rs	Adds summary/diagnostics/predictions surface test for new `FitResult`.
tests/output_writer_tests.rs	Adds coverage for shared output writer artifacts.
tests/onecomp.rs	Migrates algorithm tests to `EstimationProblem` API.
tests/nonparametric_engine_tests.rs	Adds engine-level workspace/fit-result smoke test.
tests/ipm_tests.rs	Updates IPM tests to new module paths.
tests/compile_layer_tests.rs	Adds compile-layer indexing/covariate extraction tests.
tests/api_smoke_tests.rs	Adds smoke tests for model builder, fit, compile/runtime preservation, log init.
tests/acceptance_baseline_tests.rs	Adds acceptance baseline test locking bimodal_ke NPAG outputs.
src/structs/mod.rs	Removes legacy structs module.
src/routines/mod.rs	Removes legacy routines module root.
src/routines/initialization/sobol.rs	Removes legacy Sobol initializer (migrated).
src/routines/initialization/latin.rs	Removes legacy Latin initializer (migrated).
src/routines/expansion/mod.rs	Removes legacy expansion module root.
src/routines/expansion/adaptative_grid.rs	Removes legacy adaptive grid expansion (migrated).
src/routines/estimation/qr.rs	Removes legacy QR routine (migrated).
src/routines/estimation/mod.rs	Removes legacy estimation module root.
src/routines/condensation/mod.rs	Removes legacy condensation routine (migrated/embedded).
src/results/summary.rs	Adds shared summary DTOs (`FitSummary`, `PopulationSummary`, etc.).
src/results/predictions.rs	Adds shared predictions bundle derivation for nonparametric results.
src/results/mod.rs	Introduces results module and re-exports bundles.
src/results/fit_result.rs	Adds unified `FitResult` wrapper around nonparametric workspace.
src/results/diagnostics.rs	Adds diagnostics bundle and nonparametric diagnostics derivation.
src/results/artifacts.rs	Adds artifact index (expected/missing/shared vs method-specific).
src/output/writer.rs	Adds result writer orchestration + shared predictions CSV writer.
src/output/shared.rs	Adds shared settings/summary/diagnostics writers and CSV helper.
src/output/nonparametric.rs	Adds nonparametric output plan (iterations/theta/posterior/covariates/predictions).
src/output/mod.rs	Introduces output module exports.
src/output/logging.rs	Refactors logging setup to use new `OutputPlan`/`LoggingOptions`.
src/output/file.rs	Adds `OutputFile` helper for safe folder/file creation.
src/model/variability.rs	Adds variability/random effects domain types.
src/model/parameter_space.rs	Adds parameter space/spec/domain/transform definitions + finite-range validation.
src/model/observation_spec.rs	Adds observation channel/spec modeling types.
src/model/mod.rs	Introduces `ModelDefinition` + builder with required-parameter validation.
src/model/metadata.rs	Adds optional model metadata container.
src/model/covariates.rs	Adds covariate spec/effects modeling types.
src/lib.rs	Reworks crate public surface + prelude exports to new unified API.
src/estimation/nonparametric/workspace.rs	Adds nonparametric workspace container + writers for theta/posterior/covariates.
src/estimation/nonparametric/weights.rs	Minor cleanup of weights API docs/comments.
src/estimation/nonparametric/summaries.rs	Adds fit/population/individual summaries derived from workspace state.
src/estimation/nonparametric/statistics.rs	Adds median/weighted median + mean/median utilities for summaries.
src/estimation/nonparametric/qr.rs	Adds QR decomposition helper for psi.
src/estimation/nonparametric/psi.rs	Adds ndarray conversion + minor cleanup.
src/estimation/nonparametric/prior/sobol.rs	Adds Sobol prior generator using new `ParameterSpace`.
src/estimation/nonparametric/prior/latin.rs	Adds Latin prior generator using new `ParameterSpace`.
src/estimation/nonparametric/predictions.rs	Updates predictions implementation to new module structure.
src/estimation/nonparametric/posterior.rs	Updates posterior calculation to new module structure.
src/estimation/nonparametric/mod.rs	Introduces nonparametric estimation module exports.
src/estimation/nonparametric/expansion.rs	Adds adaptive grid expansion implementation + test.
src/estimation/nonparametric/engine.rs	Adds engine adapter from compiled problems into algorithm runners.
src/estimation/nonparametric/cycles.rs	Updates cycle logging writer to new output file naming and inputs.
src/estimation/mod.rs	Introduces estimation module root.
src/compile/validation.rs	Adds problem validation for compile-time invariants.
src/compile/observation_index.rs	Adds compiled observation index representation.
src/compile/mod.rs	Adds compilation pipeline (design context, observation index, caches, log init).
src/compile/design_context.rs	Adds design context DTOs (subjects/occasions/structured covariates).
src/compile/compiled_problem.rs	Adds `CompiledProblem` container used by estimation engines.
src/compile/caches.rs	Adds cache configuration DTO for execution.
src/bestdose/predictions.rs	Updates BestDose to new nonparametric types + changes prediction interval handling.
src/bestdose/posterior.rs	Updates BestDose posterior refinement to use new config + native NPAG config.
src/bestdose/optimization.rs	Updates BestDose optimization to new weights type location.
src/bestdose/cost.rs	Updates BestDose cost/prediction interval usage and public export.
src/api/saem_config.rs	Adds SAEM configuration DTO in new API module.
src/api/progress.rs	Adds progress event DTOs for fit-with-progress API.
src/api/model_definition.rs	Re-exports model definition types via API module.
src/api/mod.rs	Introduces API module exports.
src/api/fit.rs	Adds `fit`/`fit_with_progress` entry points + basic progress test.
src/algorithms/nonparametric/postprob.rs	Refactors POSTPROB to produce `NonparametricWorkspace` and accept native config input.
src/algorithms/nonparametric/npod.rs	Refactors NPOD to unified inputs/outputs and native config.
src/algorithms/nonparametric/npag.rs	Refactors NPAG to unified inputs/outputs and native config.
src/algorithms/nonparametric/mod.rs	Adds nonparametric algorithms module docs/exports.
examples/vanco_sde/main.rs	Migrates example to new `ModelDefinition`/`EstimationProblem` API.
examples/two_eq_lag/main.rs	Removes legacy example using old settings/dispatch flow.
examples/theophylline/main.rs	Migrates example to new API.
examples/test_cobyla.rs	Adds COBYLA smoke example.
examples/new_iov/main.rs	Migrates example to new API.
examples/neely/main.rs	Removes legacy example using old settings/dispatch flow.
examples/meta/main.rs	Migrates example to new API and updates method selection.
examples/iov/main.rs	Migrates example to new API.
examples/drusano/main.rs	Migrates example to new API.
examples/drusano/config.toml	Removes legacy config file tied to old settings flow.
examples/bimodal_ke_saem/run_saemix.R	Adds R SAEMix comparison script for bimodal_ke.
examples/bimodal_ke/main.rs	Migrates example to new API and updates equation usage.
examples/bimodal_ke/config.toml	Removes legacy config file tied to old settings flow.
examples/bestdose_cov.rs	Adds BestDose covariate example using new APIs.
examples/bestdose_bounds.rs	Updates BestDose bounds example to new config/prior reader.
examples/bestdose_auc.rs	Updates BestDose AUC example to new config/prior reader.
examples/bestdose.rs	Updates BestDose example to new config/prior reader.
benches/bimodal_ke.rs	Migrates benchmark harness to `EstimationProblem` API.
Cargo.toml	Adds dependencies/features and adjusts test profile.
.gitignore	Updates ignored output artifacts and example output folders.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-04-28T21:50:39Z

+fn prediction_interval_hours(interval: f64) -> f64 {
+    if interval <= 0.0 {
+        return 1.0 / 60.0;
+    }
+
+    if interval < 1.0 {
+        interval
+    } else {
+        interval / 60.0
+    }


prediction_interval_hours treats interval == 1.0 as minutes (divides by 60), which makes a 1.0-hour interval turn into 1 minute. Since BestDoseConfig::prediction_interval is a f64 and defaults to hours (0.12), this boundary is likely wrong/ambiguous. Consider clarifying the unit (hours vs minutes) and/or adjusting the heuristic (e.g., treat <= 1.0 as hours, or use an explicit enum/unit).

Copilot · 2026-04-28T21:50:39Z

+        writer.write_field("id")?;
+        writer.write_field("point")?;
+        self.theta.param_names().iter().for_each(|name| {
+            writer.write_field(name).unwrap();
+        });
+        writer.write_field("prob")?;
+        writer.write_record(None::<&[u8]>)?;
+
+        let subjects = self.data.subjects();
+        self.posterior
+            .matrix()
+            .row_iter()
+            .enumerate()
+            .for_each(|(i, row)| {
+                let subject = subjects.get(i).unwrap();
+                let id = subject.id();
+
+                row.iter().enumerate().for_each(|(spp, prob)| {
+                    writer.write_field(id.clone()).unwrap();
+                    writer.write_field(spp.to_string()).unwrap();
+
+                    self.theta.matrix().row(spp).iter().for_each(|val| {
+                        writer.write_field(val.to_string()).unwrap();
+                    });
+
+                    writer.write_field(prob.to_string()).unwrap();
+                    writer.write_record(None::<&[u8]>).unwrap();
+                });


write_posterior uses multiple unwrap() calls while writing CSV fields/records. Any I/O/serialization failure (disk full, permission, broken pipe) will panic instead of returning an error. Please propagate these errors (e.g., replace for_each + unwrap() with try_for_each and ?).

Copilot · 2026-04-28T21:50:40Z

+pub fn median(data: &[f64]) -> f64 {
+    let mut data: Vec<f64> = data.to_vec();
+    data.sort_by(|a, b| a.partial_cmp(b).unwrap());
+
+    let size = data.len();
+    match size {
+        even if even % 2 == 0 => {
+            let fst = data.get(even / 2 - 1).unwrap();
+            let snd = data.get(even / 2).unwrap();
+            (fst + snd) / 2.0
+        }
+        odd => *data.get(odd / 2_usize).unwrap(),
+    }
+}
+
+pub fn weighted_median(data: &[f64], weights: &[f64]) -> f64 {
+    assert_eq!(
+        data.len(),
+        weights.len(),
+        "The length of data and weights must be the same"
+    );
+    assert!(
+        weights.iter().all(|&x| x >= 0.0),
+        "Weights must be non-negative, weights: {:?}",
+        weights
+    );


median/weighted_median are pub and currently panic on invalid inputs (partial_cmp(...).unwrap() and assert!/assert_eq!). Since these are part of the public surface (re-exported), prefer returning Result with a clear error (and handle NaNs explicitly) so callers can't crash the process with bad data/weights.

Copilot · 2026-04-28T21:50:40Z

+    if psi_norm.iter().any(|&x| x.is_nan()) {
+        dbg!(&psi);
+        bail!("NaN values found in psi_norm");
+    };


The error path logs dbg!(&psi); before returning bail!("NaN values found in psi_norm"). dbg! prints potentially large matrices to stderr and is not controllable via logging configuration. Prefer removing it or emitting a structured tracing::debug!/warn! message (e.g., include dims and a small sample).

Siel added 29 commits December 13, 2025 10:44

NPSAH v0.1

0380cd0

WIP: Testing new algorithms

38cf249

basic struct

2637a12

wip: SAEM's outputs

dd1b093

wip: cleaning outputs

4d3040d

wip: cleaning up

411833d

wip: cleaning up

c529859

playing with non-parametric algorithms

270c364

use git dependency

394c6b5

use git dependency

cf4476d

wip: saving progress so far

cc305bd

gh dependency

c11d38f

Merge branch 'main' into parametric

bd44d55

merged main

fda9b20

basic saem example seems to be working

388a950

remove bloat

dd4be93

cleaning a bit more

708d8c7

remove bloat

8c90072

update .gitignore

f9064aa

fmt

bd7a88b

chore: merged main into unified-platform

60d74c8

chore: fmt

c046f1a

wip:saving current state

3d0cfa8

use release pharmsol

5922da4

chore: sync local changes for unified-platform-m1

0a4d961

Merge origin/main into feature/unified-platform-m1-main-prep

15c26b4

Extract unified platform structure branch

bb4b6d4

Align structure split with prep tree

bf5f52c

Copilot AI review requested due to automatic review settings April 28, 2026 21:46

Copilot started reviewing on behalf of Siel April 28, 2026 21:46 View session

Run cargo fmt

bb25723

Copilot AI reviewed Apr 28, 2026

View reviewed changes

Siel added 9 commits April 29, 2026 07:56

chore: remove output data

7d89148

chore: remove output data

35d08de

feat: using latest version of pharmsol + clippy

6105d5b

chore: fmt

3742f22

feat: latest version of the API

bf9e45c

feat: latest version of the API

cbeb860

chore: update to pharmsol 0.27.0

e8576cd

chore: update benchmark

9b06db9

feature: support for pharmsol 0.27.1

668dd30

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Extract unified platform structure#276

Extract unified platform structure#276
Siel wants to merge 39 commits into
mainfrom
unified-platform-structure

Siel commented Apr 28, 2026

Uh oh!

github-actions Bot commented Apr 28, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Apr 28, 2026

Uh oh!

Copilot AI Apr 28, 2026

Uh oh!

Copilot AI Apr 28, 2026

Uh oh!

Copilot AI Apr 28, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

Siel commented Apr 28, 2026

Summary

Uh oh!

github-actions Bot commented Apr 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Bencher Report

🚨 1 Alert

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Apr 28, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 28, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 28, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 28, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

github-actions Bot commented Apr 28, 2026 •

edited

Loading