Integrate Design Documents into docs site#114
Open
slarson wants to merge 81 commits into
Open
Conversation
- Add 'Design Documents' section to mkdocs.yml nav (between History and Releases) - Create docs/design_documents/index.md with: - Mission alignment, phase roadmap summary - All 25 DDs organized by phase (Phase 0-4) - Links to GitHub for full specs, roadmap, integration map - How to contribute section - Update docs/index.md: - Add Design Documents to TOC (starred as NEW) - Prominently mention as starting point for technical blueprint Next: Update modeling.md to reference DDs, then open PR. Co-Authored-By: Claude Sonnet 4.5 (1M context) <noreply@anthropic.com>
Major content evolution per DOCS_SITE_EVOLUTION_PROPOSAL.md: - Rewrite modeling.md around Design Documents (5 scales, CyberElegans as history) - Create validation.md explaining DD010 3-tier framework - Create archived_projects.md contextualizing CyberElegans, Geppetto, movement_validation - Update faq.md with 959-cell goal, DD018 egg-laying, Phase 4 completion - Update projects.md with DD-to-project mapping table - Expand c302.md, sibernetic.md, geppetto.md with DD references and roadmaps - Add DD contribution workflow (10 steps) and contributor levels to github.md - Add "Path to 959 Cells" phase table to index.md - Restructure mkdocs.yml nav with "How It Works" and "History" sections - Fix contributor level names to match DD011 (Observer to Senior Contributor) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Copy all DD markdown files (DD001-DD023 + DD014.1/DD014.2 + support docs) from openworm-admin into docs/design_documents/ - Rewrite 93 external GitHub links across 11 files to local relative paths - Add all DDs to mkdocs.yml nav grouped by implementation phase - DDs now render natively on docs.openworm.org with full-text search - Only external DD links remaining: 2 GitHub tree-browsing links in design_documents/index.md (intentional — point to source repo) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Drop the phase label from the infrastructure work — "Phase 0, Phase A, Phase 1..." reads oddly with a letter in the middle of numbers. Now reads as "Infrastructure Bootstrap" without a phase number. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add "Continue Reading" sections to 18 pages guiding readers to next content - Add DD info boxes and cross-links to all project pages - Hyperlink all bare DD references in FAQ, archived projects, and DD metadata - Fix DD metadata linebreaks (trailing spaces for markdown <br>) - Hyperlink Related DD references in all 24 DD files - Fix broken archived_projects links in FAQ - Add "See today's version" links in archived projects pointing to current pages - Add "Design Document Era" section to full history page - Add DD013 Docker note to releases page Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Converted 393 bare DD### references to clickable [DD###](path) links across 13 files. Only DD016 (archived, no file) remains unlinked. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The sensorimotor loop diagram was in a code block, making DD links unclickable. Now renders with clickable DD links and arrow formatting. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Converted ~1,800 bare DD### references to clickable [DD###](file.md) links across all 27 DD files. Remaining bare refs are only in code blocks (YAML config examples, PlantUML diagrams) and page titles. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Enables toc permalink in MkDocs Material theme so every heading gets a clickable anchor link for easy sharing. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Inserted blank lines before unordered/ordered list items that immediately followed non-list content. Without the blank line, markdown renders these as inline paragraph text instead of proper <ul><li> elements. Fixed 504 instances across 31 files. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Each DD now shows which implementation phase it belongs to and what layer it covers, with a clickable link back to the relevant section in the Phase Roadmap. DDs with Quick Action Reference tables get two new rows; DDs without get a blockquote after the metadata. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Reordered so readers see the summary first, then the action table. Structure is now: metadata → TL;DR → Quick Action Reference → body. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
… page 229 bare citations (e.g., "Cook et al. 2019") linked to their DOI URLs across 27 files. New references.md page organizes all publications by topic with links to validation tiers. Added to nav and landing page. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…inks DD014 had its own "Phase 1/2/3" that didn't map 1:1 to the Roadmap (DD014 Phase 3 = Roadmap Phase 4). Renamed to "Viewer Stage 1/2/3" with explicit mapping table showing which Roadmap phase each stage belongs to. Fixed 87 mangled DD014.1/DD014.2 links created by the auto-linker wrapping "DD014" inside "DD014.1_..." filenames. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…inst Replace organizational framing (blog posts, paper opportunities, press releases, "shout about it") with concrete user experience: commands to run, behaviors to observe in the viewer, and specific experimental datasets each milestone validates against. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Linked Sarma et al. 2016, Goodman et al. 2002, Yemini et al. 2013, Raizen & Avery 1994, Thomas 1990 throughout DD010 body text and expanded the References section from 3 to 6 entries with DOI links. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
DD010 Tier 1 previously had no specific datasets — just vague "Goodman lab patch-clamp." Now lists 7 neuron classes with named papers: touch neurons (Goodman 2002, O'Hagan 2005, Suzuki 2003), AWC (Chalasani 2007), ASH (Hilliard 2004), AVA (Lindsay 2011), RIM (Liu 2018), pharyngeal (Raizen 1994). Documents the coverage gap (~7 classes with direct data out of 128) and why Tier 1 is non-blocking. Added new "Single-Cell Electrophysiology" section to master references page with all Tier 1 validation sources. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
For the ~121 neuron classes without direct electrophysiology, replaces a single vague sentence with a rigorous validation system: gene-to- electrical-property table (8 major channel genes), 5-step procedure, quantitative acceptance criteria (Spearman rho > 0.5, absence/presence checks), and a concrete testing command. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- DD008 (Data Integration Pipeline) → Phase A (infrastructure, data layer needed before Phase 1 CeNGEN ingestion) - DD014.1 (Visual Rendering Specification) → Phase 1 (companion to DD014 viewer, defines color palette and reference mockups) - DD015 (AI-Native Contributor Model) → Phase A (governance, alongside DD011/DD012) - DD022 (Environmental Modeling) → Phase 2 (substrates and gradients for closed-loop sensory behaviors) - DD023 (Proprioceptive Feedback) → Phase 2 (stretch receptors for stable motor coordination) Updates scope tables, key deliverables, success criteria, datasets, dependency summary, timeline summary (18 → 23 DDs), and blocking dependencies. Fixes DD008 Quick Action Reference from Phase 3 to Phase A. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…plan - Add WormBrowser (2012) history to DD014 Context section with features, technology, and John White's Feb 2026 suggestions (developmental stages, click-to-link, WormAtlas collaboration) - Rename DD014 Stage 3 to "WormSim 2.0" connecting to 2014 Kickstarter promise - Replace all viewer.openworm.org references with wormsim.openworm.org (10 files) - Add deployment timeline table: browser.openworm.org stays live through Phase 3, redirects to wormsim.openworm.org in Phase 4 after feature parity achieved - Add feature parity checklist (WormBrowser features + WormSim 2.0 additions) - Add Phase 1 quick win: click neuron → WormAtlas/WormBase links on existing WormBrowser at browser.openworm.org - Update DD_PHASE_ROADMAP Phase 4 milestone to "WormSim 2.0" Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Render 6 diagrams via PlantUML web service:
- integration_map.svg: Complete 23-DD dependency graph (was raw PlantUML)
- causal_loop.svg: Sensorimotor loop for modeling.md (replaces broken
Google Drawings link + mangled Unicode arrows)
- chain1-4 SVGs: Core loop, cell differentiation, closed-loop touch,
and visualization pipeline (were ASCII art in INTEGRATION_MAP.md)
- Add DD022 (Environmental Modeling) and DD023 (Proprioception) to
Integration Map PlantUML with dependency edges:
- DD022 -> DD019 (stimulus delivery), DD022 -> DD003 (substrate)
- DD003 -> DD023 (body curvature), DD023 -> DD001 (stretch current)
- DD022/DD023 -> DD014 (visualization exports)
- Update DD count from 21 to 23 in Integration Map header
- Preserve PlantUML source in collapsible details block for re-rendering
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The old diagram had 30 components and 78 edges — unreadable spaghetti. New approach: group DDs into 10 functional clusters (External Data, Data Access, Core Chain, Neural Extensions, Sensory/Motor, Organs, Whole Organism, Infrastructure, Validation, Visualization) with only major inter-cluster data flows shown. Color-coded arrows: green for core chain, red for closed loop, blue for validation, purple for viz. Detail preserved in the 4 chain diagrams (chain1-4 SVGs) which show per-edge data flow for each pathway. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Re-rendered all 6 PlantUML SVG diagrams with [[url]] links so clicking any DD component navigates to its design document. Switched from markdown image syntax to <object> tags for SVG interactivity support. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
… validation The ConnectomeToolbox already contains neuropeptidergic data as 'extrasynaptic' connections (Bentley 2016, Ripoll-Sánchez 2023) — DD006 was incorrectly proposing to add it. Fixed all references to consume via cect API (DD020) instead. Added Tier 1 functional connectivity validation using Randi 2023 unc-31 mutant data as a natural experiment isolating neuropeptide contribution (unc-31 = no dense-core vesicle release = no neuropeptide signaling). Added references: Pereira 2015, Beets 2022, Wang 2024, Randi 2023, Gleeson et al. ConnectomeToolbox manuscript. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Split Tier 2 into 2a (whole-network FC) and 2b (neuropeptide modulation via wt-vs-unc-31 comparison from Randi 2023). Add ConnectomeToolbox as data source alongside wormneuroatlas. Cross-reference DD006 validation. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Copy mockup PNGs from openworm-admin and embed in each mockup specification section. All 14 views now have visual reference images alongside their property tables. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…view Major changes: - Port MetaWorm (Zhao 2024) analysis into DD001, DD003, DD005, DD006, DD010, DD017, DD020, DD021, DD022 (29 citations across 7 files) - Add "Existing Code Resources" sections to 11 DDs identifying 18 reusable repos from the OpenWorm GitHub org - Create DD024 (Validation Data Acquisition Pipeline) - Create contributing_guide.md (extracted from dd_readme.md) - Consolidate overview: richer index.md with philosophical foundations, MetaWorm related work, cross-references by topic - Remove standalone analysis pages (CODE_REUSE, COMPREHENSIVE_ANALYSIS, dd_readme) — content incorporated into individual DDs - Add DD008 reconciliation note for OWMeta/cect phasing - Add DD001 Boyle & Cohen parameter borrowing clarification - Fix mkdocs.yml: add admonition/details/superfences extensions - Fix 3-space → 4-space list indentation across 12 files Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Comprehensive audit and cleanup of the entire openworm_docs site (35 files): - Fix broken links: dead repositories/ link, 3 Dropbox→DOI replacements, outdated openworm-admin reference, phantom nav entry - Add ~50 DOI hyperlinks to unhyperlinked citations across 10 DD files and references.md (Goodman, Gleeson, Cannon, Kato, Berman, Machamer, Pearl, Schafer, Brown, Pierce-Shimomura, Iino, Zhao, and many more) - Verify Chalfie 1985 DOI across all DDs (8 missing links added) - Upgrade ~115 http:// links to https:// across 18 files - Consolidate duplicated content: FAQ shortened with cross-links to canonical pages (background, DD index, validation) - Add cross-links between non-DD pages and governing Design Documents - Add historical banners to running-nc.md and Live-Video-Protocol.md - Fix image paths in browser.md and docker.md - Clarify DD008 neuropeptide data status (in cect, not yet in OWMeta) - Clean build: zero warnings, zero errors Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…oss-references Add ~30 new references extracted from all 25 Design Documents, create 4 new sections (Chemotaxis, Neural Circuit Variability, Computational Methods, Philosophical Foundations), and hyperlink every DD cross-reference in the Description column so readers can click through to the citing documents. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
v2 is deprecated and now hard-fails at job setup. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
pycairo build fails on Ubuntu 24.04 runners without this package. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The listComments response is already destructured, so comments is the array directly — no need for comments.data. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…al terms - Update DD count from 25 to 27 across 6 files (index, fullhistory, ai_agents, DD015, modeling) to reflect DD024-DD026 and DD014.1/DD014.2 - Add "Getting Started" section to DD001 with repo clone commands, Docker path, and native Python setup so newcomers can actually follow build instructions - Add 30 hyperlinks for domain-specific terms (Hodgkin-Huxley, NeuroML, LEMS, SPH, PCISPH, WCON, OME-Zarr, etc.) on first occurrence across DD001-DD003, DD020, DD_PHASE_ROADMAP, and INTEGRATION_MAP Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add "Getting Started" sections with clone URLs, Docker-vs-native paths, and dependency installation to all Phase 0 DDs so newcomers can follow the build instructions without prior knowledge: - DD002: Same c302 setup as DD001, with cross-reference for existing setup - DD003: Native C++ path with platform-specific OpenCL guidance, Docker recommended to avoid SDK setup complexity - DD020: Distinguish "use cect" (pip install) from "develop cect" (clone + editable install + pytest) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Update the DD012 RFC Process template to require a "Getting Started (Environment Setup)" sub-section in every DD's "How to Build & Test": - Template: expand flat bullet list into structured sub-sections (Prerequisites, Getting Started, Step-by-step, Green light criteria) - Getting Started must include: repo clone URLs, Docker path (recommended), native path with exact install commands, cross-references for shared repos, platform-specific notes where applicable - Quality criterion #10: "Complete Setup Path" — newcomer goes from blank machine to working environment without guessing - Anti-pattern #10: "Missing setup context" — build commands assume repos are already cloned without saying how Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Every non-exempt Design Document now has a "Getting Started (Environment Setup)" sub-section per the DD012 template standard. Includes clone URLs, Docker vs native paths, cross-references for shared repos, and platform-specific notes. DD013 and DD014 received full How to Build & Test sections (were previously missing). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…raft issues DD001 contained ~80 lines of Phase 1-2+ future work (multicompartmental neurons, ion channel library, synaptic optimization, spatial synapses) that made it read like a roadmap instead of a spec. This refactor: - Creates DD027 (Multicompartmental Neuron Models) from DD001's Level D and spatial synapse sections, with full DD012-compliant structure - Redistributes DD001 draft issues to owning DDs: DD005_draft_issues (6 ion channel issues), DD017_draft_issues (3 synaptic optimization issues), DD027_draft_issues (3 multicompartmental issues) - Trims DD001_draft_issues from 21 to 9 retained issues - Replaces extracted DD001 sections with concise forward references - Updates DD count site-wide (27→28), Phase Roadmap, Integration Map, mkdocs.yml nav, and cross-reference indices Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
DD001 should describe what exists now and note known approximations, not how the nervous system model will evolve. Replaced three evolution subsections (synaptic optimization, ion channel expansion, spatial synapses) with a concise "Known Approximations" section. Trimmed forward-looking reference annotations and removed future-only refs. Moved the digested technical details to Phase Roadmap where they belong (Phase 1 ion channel expansion, Phase 2 DD027 staging, Phase 3 synapse weight optimization). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Ring 1 (Open Courtyard) should be outermost and Ring 3 (Inner Sanctum) innermost — newcomers enter from outside, founder is protected at core. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
New DD specifying a real-time project health dashboard: validation scores, contributor activity, CI pipeline status, and phase progress. Static site (Chart.js + GitHub Pages) fed by a Python metrics collector. Updates site-wide DD count (28→29) and all cross-references. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
… map - Add DD028 key deliverable + success criterion to Phase A in roadmap - Add missing key deliverables for DD012, DD011, DD024 (items 11-13) - Add missing success criteria for DD012, DD011, DD024, DD028 - Add 5 missing Phase A DDs to Integration Map Phase Legend (DD008, DD011, DD012, DD015, DD024); reorganize legend by phase - Add DD028 component + edges to PlantUML diagram (Infrastructure package) - Add DD024 incoming edge from ext_beh (raw experimental data) - Add DD027 component + edges to PlantUML diagram (Neural Extensions package) - Restructure PlantUML layout: extract ML & Foundation package (DD025, DD017) from Infrastructure, position upstream of Neural Extensions and Core to convert 6 backward arrows into forward flow - Re-render integration_map.svg with all diagram changes - Remove all bio.rodeo references (9 occurrences across 5 files), keep underlying model names (AlphaFold 3, BioEmu-1, Boltz-2, ESM Cambrian) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…ampaign Closes the engagement design gap identified in the ExO Canvas (1/10 → 3/10). Adds 7 targeted improvements: Choose Your Level self-assessment, trigger-action engagement loop with State of the Worm newsletter, expanded public contributor dashboard, DD024 data digitization as L1 tasks, DD014 viewer as engagement hook, 3-wave reactivation campaign for 940 dormant applicants, and DD014/DD024/DD028 integration contract references. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…sking) Phase A was a monolithic 9-DD block mixing critical-path infrastructure with governance docs. This split clarifies what truly blocks progress (A1: DD008, DD013, DD021, DD024, DD028) vs what enables scaling in parallel (A2: DD011, DD012, DD015, DD025). Updated all 27 files: roadmap, overview, nav, 9 DD headers, integration map, 7 draft issue files (~150 refs), contextual body text, and supporting pages. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Phase 0 has 83 open stabilization issues across DD001-DD003/DD020. Replace misleading "Done"/"Complete"/"Already done" with "Functional" to convey that the simulation runs but stabilization work remains. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
5 tasks
Resolve conflicts from imgur image migration (PR #116) — keep local image paths from master alongside design document content additions. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Shifts the modernization narrative from Taichi-as-intermediate-layer to
hand-written native Metal (Apple Silicon) and CUDA (NVIDIA) substrates.
Recognizes the active `ow-native-gpu-0.1.0` consolidation (PR #230) and
PR #229 (CUDA scaffolding) as the strategic direction.
Headline changes:
- Compute Backends table: native Metal + native CUDA rows; Taichi
Metal/CUDA demoted to a single "Superseded" row
- New §"Why Native Ports, Not Taichi" subsection explaining the
historical Taichi coordinate-space divergence as motivation for the
hand-written-per-platform strategy
- §"The Result Quality Gap" rewritten around current native-Metal
per-demo parity status (5+ demos working)
- §"The Taichi Coordinate-Space Bug" subsection removed — folded into
the new "Why Native Ports" section as a historical note
- §"Backend Graduation Criteria" status table reflects native ports
- §"Stabilization Sequence" reordered around landing PR #230 + per-
demo parity gates
- New §"Community Contributions to Modernization" pointer to the
`contributor-acknowledgment` GitHub label (no contributor names in
the DD itself)
- `body.backend` valid values: taichi-{metal,cuda} → {metal,cuda}-native
- Implementation Status + Next Actions updated to current reality
- Key files list expanded with src/metal_diff/ + src/cuda/ paths
Differentiability is now framed as a byproduct of the native-port
implementation approach (cross-reference to DD017), not the motivation
for the modernization. This reflects the "we jumped the queue with
differentiable simulations" reframing — native hardware compatibility
is Phase 0, differentiability framework is DD017's Phase 3 scope.
All anchors referenced by the in-progress per-issue migration plan
remain stable. Build verified: mkdocs build succeeds; new anchors
(#why-native-ports-not-taichi, #community-contributions-to-modernization)
render cleanly.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
These three "DDs" were explicitly self-labeled as governance documents
(Status: Proposed (Governance) in their frontmatter) rather than
technical Design Documents. Moving them out of the DD series cleans up
DD semantics ("DDs spec technical artifacts") and puts contributor
onboarding in a discoverable top-level Contributing section.
New /docs/contributing/ section (parallel to existing Community/,
Projects/, Resources/):
- index.md — landing page explaining the section purpose
- contributor-progression.md — from DD011 (L0-L5 levels, badges,
BadgeList integration, reactivation campaign)
- decision-process.md — from DD012 (RFC process, DD template,
workflow, Mind-of-a-Worm enforcement)
- ai-contributors.md — from DD015 (AI agent registration, DD-to-issue
decomposition, AI-human coexistence, GitHub bot architecture)
Editorial pass on each migrated file is intentionally light:
- Drop "DDxxx:" prefix from title
- Drop the Phase A2 callout (no longer phase-gated)
- Convert frontmatter: Status: Proposed → Active; Author → Maintained by
- Update internal cross-refs to relative /contributing/ links
- Prefix all DD-references with ../design_documents/ (sibling dir)
- Content body otherwise unchanged
Stub redirects at old DD locations:
- design_documents/DD011_Contributor_Progression_Model.md
- design_documents/DD012_Design_Document_RFC_Process.md
- design_documents/DD015_AI_Contributor_Model.md
Each is a short page pointing at the new /contributing/ location with
a brief "Why this moved" explanation. Numbers retired from the series.
Cross-reference updates across the rest of the DD corpus:
- mkdocs.yml — new Contributing top-level section; Phase A2 nav under
Design Documents now contains only DD025 + a link to /contributing/
- DD_PHASE_ROADMAP.md — Phase A2 scope reduced to DD025; governance
pages noted as living continuously in /contributing/
- INTEGRATION_MAP.md — Governance package + table updated
- design_documents/index.md — quick links + governance section refer
to /contributing/ by descriptive labels
- All other DDs (DD010, DD013, DD014, DD020, DD021, DD028, draft
issues files, contributing_guide) — bulk-rewrote canonical link
targets and DDxxx labels
Build verified: mkdocs build succeeds with no new broken links
beyond pre-existing image-path and DD-rename warnings unrelated to
this migration.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Combines DD003 native-gpu reframe (dd003/native-gpu-reframe topic branch) with the /docs/contributing/ section migration of DD011/12/15. The two topic branches touched disjoint files so this is a clean merge. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Closes the gaps left by retiring DD011/12/15 (which moved to /docs/contributing/) and the historical DD016 gap. Net effect: the published DD series is now contiguous DD001-DD024, with DD012.1 and DD012.2 sub-numbers preserved as the only non-integer entries. Mapping (old → new): DD013 → DD011 (Simulation Stack Architecture) DD014 → DD012 (Dynamic Visualization Architecture) DD014.1 → DD012.1 (Visual Rendering Specification) DD014.2 → DD012.2 (Anatomical Mesh Deformation Pipeline) DD017 → DD013 (Hybrid Mechanistic-ML Framework) DD018 → DD014 (Egg-Laying System Architecture) DD019 → DD015 (Closed-Loop Touch Response) DD020 → DD016 (Connectome Data Access and Dataset Policy) DD021 → DD017 (Movement Analysis Toolbox and WCON Policy) DD022 → DD018 (Environmental Modeling and Stimulus Delivery) DD023 → DD019 (Proprioceptive Feedback and Motor Coordination) DD024 → DD020 (Validation Data Acquisition Pipeline) DD025 → DD021 (Protein Foundation Model Pipeline) DD026 → DD022 (Reservoir Computing Validation) DD027 → DD023 (Multicompartmental Neuron Models) DD028 → DD024 (Project Metrics Dashboard) Changes: - Renamed 22 DD files (and their _draft_issues.md siblings) via git mv in cascade order (smallest old number first, vacating slots before refilling) - Deleted DD011/12/15 stub redirect files — no longer needed after renumber overwrites the slots; the underlying content is canonical at /contributing/ - Two-pass sed across all docs/*.md + mkdocs.yml: DDxxx → DD_TMP_yyy → DDyyy (uppercase + lowercase label forms) — handles every link target, link text, prose mention, anchor reference, and PlantUML component ID - Fixed remaining references in docs/*.md, docs/Community/, docs/Projects/, docs/Resources/ that pointed at the deleted DD011/12/15 stub filenames — now point at /contributing/ Sequence note: this lands AFTER the DD003 native-gpu reframe and the /contributing/ migration both merged to integrate-design-documents, so the renumber is a single coherent pass over a stable state. No rebase needed. Build verified: mkdocs build succeeds; no new broken links. Pre- existing warnings (image paths, hypothetical DD names that never matched any real file) are unrelated. design-docs-drafts branch retains the pre-renumber filenames as a historical reference, in case anyone needs to look up an old DD by its original number. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…D001→DD002, DD002→DD003)
Repositions Sibernetic (the foundational body physics engine that
everything else couples to) as DD001, with Neural Circuit and Muscle
Model becoming DD002 and DD003 respectively. This reflects the
actual dependency direction: the body simulation is what neural and
muscle signals act upon — the body is the substrate, not a peer.
Mapping (old → new):
DD001 Neural Circuit Architecture → DD002 Neural Circuit Architecture
DD002 Muscle Model Architecture → DD003 Muscle Model Architecture
DD003 Body Physics Architecture → DD001 Body Physics Architecture
(and the corresponding _draft_issues.md files)
Implementation:
- Six git mv ops via DD0TMP{1,2,3}_ intermediates (cyclic swap;
can't go direct because target slots are occupied)
- Two-pass sed across all docs/*.md + mkdocs.yml: DDxxx → DD_TMP_y
→ DD00y (uppercase + lowercase label forms) — handles all link
targets, link text, prose mentions, anchors, PlantUML component
IDs, and the file H1 titles
- Verified: new DD001 H1 is "Body Physics", new DD002 H1 is
"Neural Circuit", new DD003 H1 is "Muscle Model"; Related fields
cross-reference correctly (e.g., new DD001 lists DD002+DD003+DD004)
Build verified: mkdocs build succeeds with zero new broken links
beyond pre-existing image-path and hypothetical-DD-name warnings.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…13 narrows accordingly
Audit of the sibernetic ow-native-gpu-0.1.0 branch revealed the
differentiable substrate is substantially more mature than DD001
acknowledged. Concrete state on the branch:
- 19 forward kernels with 19 paired analytic backward kernels
(hand-derived, all FD-validated at ε=1e-3, rel-err 1-5%)
- Multi-step xpbd_full_bwd walks K constraint iterations in reverse,
produces gradients on (x_init, v_init, ρ_rest, spring_K, viscosity,
α_density, floor_y, restitution, membrane params)
- 8 SGD harness scripts; 4 of 5 demos (demo1, demo2, one_sprig,
worm_alone) already tuned to OpenCL reference via gradient descent
- src/cuda/README.md mandates that the CUDA port mirror this
architecture file-for-file (paired backward per forward is a
structural requirement)
DD001 changes:
- TL;DR adds explicit "architecturally differentiable" framing
- Quick Action Reference gains Differentiability row
- Compute Backends table gains Differentiable column (Metal: ✅
end-to-end; CUDA: 🟡 structural mandate; OpenCL: ❌ forward only)
- New "Differentiability" section (~120 lines) covering:
* Why a substrate property, not a framework layer (3 structural
advantages: no drift, GPU-native gradients, free parameter grads)
* Per-kernel table of forward + paired backward + FD validation
* Multi-step xpbd_full_fwd/_bwd API
* Gradient-tuned demos table
* Implications for downstream subsystems (DD002, DD003, DD010, DD011)
* What's not yet differentiable (OpenCL, CUDA in progress, PyTorch)
* How to use the differentiable interface (CLI example, sgd_true.py
reference)
- Quality Criteria gains item 7: paired backward per forward is the
architectural contract; new kernels without backward must not land
- Integration Contract Outputs gain "Parameter gradients" row
- Implementation Status + Next Actions reflect the differentiable
reality + add joint neural↔body fitting as a Phase 3 prototype
DD013 changes:
- New "Scope reduced (2026-05)" admonition at top explaining the
architectural shift — original 4 components reduced to 2
- TL;DR rewritten: remaining scope is SPH surrogate + learned sensory
transduction (the differentiable substrate component moved to DD001,
the foundation-model component went to DD021, the neural-ODE
differentiability is deferred to Phase 3 design discussion)
- Goal & Success Criteria + Deliverables trimmed
- Quick Action Reference + config keys updated
- Related field reordered to lead with DD001 (Body Physics) since
DD013's SPH surrogate trains against DD001's differentiable substrate
Light-touch downstream annotations:
- DD002 (Neural Circuit): "Joint Neural ↔ Body Parameter Fitting"
forward-reference subsection
- DD003 (Muscle Model): note that muscle-through-body parameter
fitting becomes possible once muscle ODE is also differentiable
- DD010 (Validation Framework): "Gradient-Based Validation" forward
reference — Tier 3 metrics as loss functions; proposed `validate-grad`
mode
- DD011 (Simulation Stack): Inputs/Outputs gain gradient-buffer row;
proposed `body.differentiable: true` config flag (no-op for OpenCL)
Build verified: mkdocs build succeeds with zero new broken links
beyond pre-existing image-path and hypothetical-DD-name warnings.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Articulates the 8-phase predict→reference→inspect→refine→implement→ SGD-tune→render→compare workflow that emerged from the native-Metal port (ow-native-gpu-0.1.0 branch) and bakes it into the DD as a binding contract for every physics PR. This is the methodology companion to the architectural Differentiability section: the backward kernels prove the math is right; this workflow proves the physics is right. Three additions: 1. Quick Action Reference gains a "Validation methodology" row pointing at the new section. 2. Quality Criteria #8 added: every physics change MUST follow the 8-phase workflow. PRs without (a) benchmark config, (b) OpenCL reference trajectory, (c) Metal/CUDA trajectory dump, (d) side-by- side MP4, (e) visual + quantitative parity evidence are not ready to land. Hand-swept parameters are forbidden — SGD with saved convergence history required. 3. New top-level "Validation Methodology" section (~200 lines) placed after Quality Criteria and before Boundaries: - "Why This Is Horizontal" framing - 8 phases documented one-by-one with the operational commands (build, dump_metal_trajectory.py, sgd_*.py, render_*.py, ffmpeg hstack, frame extraction) - Worked example: one_sprig_test walked through all 8 phases showing how the 2× period surprise drove SGD-tuned K=555 - 10-item Mind-of-a-Worm PR Review Checklist that human and AI reviewers MUST verify before approving - 10 common gotchas distilled from the native-Metal port history (0.25 factor, rho_rest=0 NaN, CLI argv cascades, hand-sweep anti-pattern, physics-gap vs parameter-gap diagnosis, etc.) - Validation tooling reference (src/metal_diff/dump_metal_trajectory .py, render_*.py, sgd_*.py, test_*.py FD validators) - Explicit requirement that the CUDA substrate must include the same workflow tooling when it lands Sources: - Native-Metal port commits on ow-native-gpu-0.1.0 - Existing sibernetic-demo-port skill at ~/.claude/skills/sibernetic-demo-port/SKILL.md - The 19 FD validators + 8 SGD harness scripts in src/metal_diff/ Build verified: mkdocs build succeeds with zero new broken links. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
After DD001 added the 8-phase predict→reference→inspect→refine→ implement→tune→render→compare methodology (commit dac96b7), extract analogous methodology sections into the four DDs whose subsystems share the same validation-shape concern. Each DD's methodology is adapted to its domain's lived references and tooling, but all follow the same horizontal pattern and produce the same MoaW-checklist / common-gotchas artifacts. DD010 (Validation Framework) — the meta-template: - New "Validation Workflow Pattern" section after Three-Tier table - Articulates the 8-phase pattern as the cross-tier meta-template - Shows how Tier 1, Tier 2, Tier 3 each instantiate the pattern (one phase-table column per tier) - Per-tier artifact requirements table - 7 anti-patterns MoaW flags (skip prediction, hand-sweep, post-hoc thresholds, etc.) - Points at DD001 as the canonical worked example - Quality Criteria #5: workflow-pattern-followed becomes binding - Quick Action Reference adds "Workflow pattern" row DD002 (Neural Circuit) — per-cell/per-pair/per-network methodology: - New "Validation Methodology" section (~100 lines) - Renames existing "Validation Procedure" → "Validation Procedure (Quick Reference)" to keep its bash one-liners separate from the deeper methodology - 4 levels: per-cell electrophysiology, per-synapse, per-network functional correlations, round-trip with body via Sibernetic - Each level has its own 8-phase table with the relevant reference datasets (Goodman 2002, Randi 2023, Yemini 2013) and acceptance thresholds - Expression-consistency shortcut for the ~121 neuron classes without patch-clamp data - 10-item MoaW PR review checklist (neural-circuit-specific) - 5 common gotchas (don't confuse prediction with simulation output, don't break network when tuning cells, don't add edges to Cook 2019 topology, etc.) - Forward-reference to differentiable substrate (DD001) for joint neural↔body fitting once neural ODE side gains differentiability DD003 (Muscle Model) — twitch/curves/round-trip methodology: - New "Validation Methodology" section (~95 lines) - Renames existing "Testing Procedure" → "Testing Procedure (Quick Reference)" - 3 levels: twitch dynamics (calcium→force pulse shape against Boyle & Cohen 2008), force-velocity/force-length curves (Hill 1938, Gordon-Huxley-Julian 1966), round-trip body bend - Round-trip level explicitly uses DD001's tooling (dump_metal_ trajectory.py, side-by-side render) - 10-item MoaW PR review checklist (muscle-model-specific) - 5 common gotchas (don't copy neuron HH params to muscle, don't skip Level 3 for "internal" changes, etc.) DD011 (Simulation Stack) — integration-layer methodology: - New "Stack Integration Validation Methodology" section (~90 lines) - Distinguishes integration-layer concern from scientific-accuracy concern: "does the plumbing keep working when the science changes?" - Two-Gate Model: quick-test (<5 min, no-crash) vs validate (<2 hr, DD010 tier gates); explicit "what each gate proves / does NOT prove" table - 8-phase methodology adapted to Docker / compose / orchestrator concerns - Subsystem-Onboarding Workflow (7 numbered steps for adding a new subsystem to the stack) - 10-item MoaW PR review checklist (integration-layer-specific) - 8 common gotchas (Dockerfile layer ordering, transitive deps, quick-test vs validate confusion, versions.lock drift, etc.) - Cross-DD Validation Coordination table — defines which DD owns which validation layer, with the Integration Maintainer as cross-DD arbiter when validators conflict Build verified: mkdocs build succeeds with zero new broken links. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…contributing/ live The publish set for this initial docs.openworm.org release is: - DD001 (Sibernetic Body Physics) — the sole published DD - DD_PHASE_ROADMAP, INTEGRATION_MAP, index.md, contributing_guide.md — meta pages - /docs/contributing/ — the four governance pages Everything else (DD002, DD003, DD004..DD024 and their _draft_issues companions) is held back for one-by-one review per the original plan. Previously the publish-set files referenced held-back DDs as LIVE links, which leaked the held-back content into the published surface (any reader following a link would land on draft content). This commit unlinks every cross-DD reference whose target is NOT DD001 across all publish-set artifacts, preserving the textual mention so the prose still reads coherently — e.g.: Before: "See [DD004](DD004_Mechanical_Cell_Identity.md) for ..." After: "See DD004 for ..." Approach: a small Python script (`/tmp/unlink_held_back.py`, not committed) walks every link of the form `[DDxxx](...DDyyy_*.md)` across the publish set and unlinks any with yyy != 001. Handles both same-directory paths (used in design_documents/) and `../design_documents/` relative paths (used in contributing/). Final state across 8 files (488 link removals, 44 DD001 links preserved): | File | DD001 kept | Other unlinked | |---------------------------------------------------|-----------:|---------------:| | DD001_Body_Physics_Architecture.md | 0 | 37 | | DD_PHASE_ROADMAP.md | 11 | 121 | | INTEGRATION_MAP.md | 20 | 90 | | design_documents/index.md | 2 | 44 | | contributing_guide.md | 0 | 25 | | contributing/contributor-progression.md | 6 | 34 | | contributing/ai-contributors.md | 3 | 77 | | contributing/decision-process.md | 2 | 18 | | contributing/index.md | 0 | 0 | Build verified: mkdocs build succeeds with zero new broken links (text mentions don't have anchors to break). Intra-page anchors (#section) and /contributing/ internal links remain live. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…EADME pointer
Four focused changes responding to user feedback after preview inspection:
1. **XPBD discussion parallel to PCISPH.** Technical Approach now has
two side-by-side subsections: "Predictive-Corrective Pressure Solver
(PCISPH — OpenCL Reference)" and a new "Extended Position-Based
Dynamics (XPBD — Native Metal / CUDA Substrates)". The XPBD section
covers:
- The 3-phase loop (predict → constrain × K → update velocities)
- The constraint kinds the substrate supports (density, distance,
floor/box, membrane) and which kernel implements each
- How force kernels (viscous pair, SPH pressure-gradient buoyancy)
interleave with constraint projection for phenomena not cleanly
expressed as constraints
- Why XPBD was chosen over PCISPH for the native substrate:
analytic per-constraint Jacobians → analytic backward kernels,
bounded per-step state → multi-step xpbd_full_bwd, unconditional
stability → kinematic SGD across orders of magnitude
- Cross-approach parity: the native substrate is a XPBD/SPH hybrid
that matches OpenCL's PCISPH kinematics at the observable level
despite mathematically different inner schemes
- Why this is a PBD-revisitation that resolves the accuracy gap
that originally made PBD a rejected alternative (§Alternatives 4)
2. **Palyanov 2018 mentions linked to DOI.** Four bare-text "Palyanov
et al. 2018" references — including the §Validated Kinematic
Outputs header — now link to https://doi.org/10.1098/rstb.2017.0376
inline, matching the existing footnote and the format already used
for other citations in the document.
3. **Mind-of-a-Worm mentions linked to authoritative description.**
The two prose mentions (TL;DR table row, Validation Methodology
intro paragraph) now link to /contributing/ai-contributors.md
which is the authoritative governance page for the AI agent.
Section headers ("Mind-of-a-Worm PR Review Checklist") left as
bare text to avoid header-link visual noise.
4. **Build & Test replaced with pointer to Sibernetic README.** The
recap of Docker / native-build steps + Step-by-step bash recipe
was deleted (it duplicated content that lives in the Sibernetic
repo's README and was going stale). Replaced with:
- A pointer to the Sibernetic README as the canonical source
- A four-row Acceptance Criteria table (quick-test / validate /
parity / differentiability) — this is the DD's actual concern:
what defines a *correct* build
- The Validation Tooling Status note (preserved from old "Scripts
that may not exist yet" subsection) since it tracks an actual
gap in the validation tooling
Net diff: +54/-73 (-19 lines overall — the build recap shrank more
than XPBD expanded). Build verified: mkdocs build succeeds with zero
new broken links. Not pushed.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Nothing in the publish set was ever publicly released before, so references to "previously planned", "scope reduced", "originally had four components", or "if you arrived from the old URL" are all meta-version commentary about drafts no reader saw. Stripping them keeps the docs forward-only. What's preserved: explicit statements about changes from the version that lives on master (per user direction) and references to genuine project history that affected real users (Taichi backend existed and shipped in openworm.yml; OpenCL era of "build kernel, eyeball demo, ship" — real prior practice). Edits: DD013 (Hybrid Mechanistic-ML Framework): - Drop "Scope reduced (2026-05)" admonition entirely (5 lines) - TL;DR rewritten to lead with current 2-component scope; drop the 3-bullet "Scope reductions" section listing what moved/extracted - Success Criteria: drop italicized "(success criterion has moved to DD001)" trailing note - Deliverables: drop three italicized trailing notes about what's delivered elsewhere / extracted / future work — they were pure meta-version commentary - Quick Action Reference: drop parenthetical noting what moved DD_PHASE_ROADMAP: - Drop "Note on governance scope" blockquote with "previously listed here as DD011, DD012, DD015". Replace with one-line current-state statement - DD013 inventory row: strip "(Component 3 extracted to DD021)" parenthetical; replace with current-state description - Also corrected DD013's dependency list to include DD001 (now load-bearing for SPH surrogate training) DD001 §Differentiability: - "Why Differentiability Is a Substrate Property" subsection: drop opening paragraph that began "Earlier OpenWorm planning (DD013) treated 'differentiable simulation' as a future framework component..." Rewrite to describe the substrate-property choice on its own terms, with a forward-pointing reference to DD013 for the ML-augmentation framework that consumes the substrate. /contributing/index.md: - Drop trailing "If you arrived here from a DD011, DD012, or DD015 link..." paragraph. Nobody arrived from those URLs because the old URLs never existed publicly. Build verified: mkdocs build succeeds with zero new broken links. The remaining 3 "previously" mentions across the corpus all refer to real-world prior states (capability comparisons, contributor demographics, foundation-model retraining cycle), not draft history of this work. Not pushed. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
docs.openworm.org/preview/<branch>/with a link posted as a PR commentKey files
docs/design_documents/— all 27 DDs plus index, roadmap, integration map, contributing guidedocs/design_documents/*_draft_issues.md— pre-written GitHub issues for 6 DDsmkdocs.yml— expanded nav with full DD hierarchy.github/workflows/mkdocs.yml— PR preview deploy supportdocs/references.md,docs/validation.md,docs/archived_projects.md— new pagesTest plan
docs.openworm.org/preview/integrate-design-documents/!!!/???) render properly (requires pymdownx extensions)🤖 Generated with Claude Code