diff --git a/CLAUDE.md b/CLAUDE.md
index 8b1ef55..f5ec408 100644
--- a/CLAUDE.md
+++ b/CLAUDE.md
@@ -267,3 +267,9 @@ export default async function PostPage({ params }) {
- Guidance on concise `CLAUDE.md` files — see community write-ups like the Apidog "Claude.md" overview for keeping this file focused and high-signal
Keep this document up to date as the source of truth for how this blog is structured and extended.
+
+
+
+# Thoughts:
+
+All thoughs (even in sub-projects) should be in thoughts/shared/ (root), not in the sub projects.
\ No newline at end of file
diff --git a/app/blog/[slug]/page.tsx b/app/blog/[slug]/page.tsx
index 618e8aa..f7e27c8 100644
--- a/app/blog/[slug]/page.tsx
+++ b/app/blog/[slug]/page.tsx
@@ -67,6 +67,12 @@ export default async function BlogPostPage({ params }: { params: Promise<{ slug:
);
return
Diffusion Loop
+
+ Tool: think. Identify draft gaps and propose diverse research questions tied to
+ those gaps.
+
+ Expected: 3–5 targeted questions, each mapped to a draft gap with scope/priority notes. +
+ +
+ Tool: ConductResearch. Delegate distinct questions to sub-agents with explicit
+ instructions and expected returns.
+
+ Expected: cited findings (URLs + quotes) per sub-agent, deduped URLs, short summaries. +
+ +
+ Tool: refine_draft_report. Fold new findings into the draft; keep structure
+ concise to conserve context.
+
+ Expected: draft updated with citations/quotes; bullets or short paragraphs retained for clarity and + context efficiency. +
+ ++ Heuristic: diverse new searches should stop yielding new facts. If not, loop again. +
+
+ Expected: a clear decision to continue or call ResearchComplete, with rationale noted.
+
Diffusion Overview
++ The report covers [pillars] across labs, + highlighting [methods] with citations to + [sources]. +
+ ), + }, + { + label: 'Refined text', + render: ( ++ OpenAI: RLHF + eval gates. Anthropic: Constitutional AI + red-team. DeepMind: interpretability + + strict evals. Cited incidents and mitigations mapped to primary URLs. +
+ ), + }, +]; + +interface DraftDenoisingProps { + className?: string; +} + +export function DraftDenoising({ className }: DraftDenoisingProps) { + const ref = useRefDraft Denoising
+
+
+
+ The report converges toward a comprehensive, insight-rich, and readable deliverable with + clean citations that pass the FACT evaluation. +
+Parallel Sub-Agents
++ {stage === 'assign' && 'Assigning distinct questions...'} + {stage === 'research' && 'Monitoring parallel research...'} + {stage === 'collect' && 'Collecting findings...'} + {stage === 'refine' && 'Refining draft...'} + {stage === 'decide' && 'Assessing completeness...'} +
+
+
+ Topic: {agent.topic} +
++ Focus: {agent.focus} +
+ + {/* Status messages container - always render, control opacity */} ++ {stage === 'refine' ? 'Incorporating findings with citations...' : 'Draft updated with citations'} +
+How it works:
+
+
+ Note: At this time, Trivy tops this benchmark with a secret model. +
+Self-Balancing
+Outputs
++ Goal: close evidence gaps with primary sources before any polish. +
+Outputs
++ Goal: readable, insightful synthesis once facts are locked. +
++ Diffusion Deep Research represents a fundamental architectural shift in how AI systems + approach complex research tasks. Rather than generating research reports in a single pass, + this approach models the entire research process as a diffusion process—starting + with a “noisy” initial draft and iteratively “denoising” it through + cycles of information retrieval, reasoning, and revision. This post provides a comprehensive + technical overview of the architecture, its theoretical foundations, and a complete Go + implementation you can use today. +
+ +
+ Traditional AI research agents follow a linear paradigm: Query → Search → Synthesize → Report.
+ This suffers from fundamental limitations:
+
+ Diffusion models, originally developed for image generation, provide an elegant solution. Instead + of generating content in one pass, they start with a noisy initial state (random + noise for images, rough draft for research) and iteratively refine through multiple + denoising steps, using guidance signals to steer the refinement. +
+ ++ “The iterative nature of diffusion models naturally mirrors how humans actually conduct + research—cycles of searching, reasoning, and revision.” ++ +
+ — Google Research, Deep Researcher with Test-Time Diffusion, 2025 +
+ In classical diffusion models (DDPM, Stable Diffusion), the process consists of two phases: +
+
+ Forward Diffusion: Gradually add noise to data: x₀ → x₁ → x₂ → ... → xₜ (pure noise)
+
+ Reverse Diffusion: Learn to denoise step by step: xₜ → xₜ₋₁ → ... → x₁ → x₀ (clean data)
+
+ For research report generation, we reinterpret this process: +
+ +| Classical Diffusion | +Research Diffusion | +
|---|---|
| Random noise (xₜ) | +Initial draft from model knowledge | +
| Denoising step | +Research iteration + draft refinement | +
| Guidance signal | +Retrieved information from web search | +
| Clean output (x₀) | +Comprehensive, accurate research report | +
+ The key insight is that the initial draft generated purely from the LLM's + training data represents the “noisy” starting state. Each iteration of identifying gaps, + searching for information, and incorporating findings acts as a denoising step that + brings the report closer to ground truth. +
+ +Let:
+D₀ = Initial draft (from LLM training data only)Dₜ = Draft at iteration tR(Dₜ) = Research function that identifies gaps and retrieves informationU(Dₜ, R(Dₜ)) = Update function that incorporates research into draftThe diffusion process becomes:
+The process terminates when:
+R(Dₜ) returns no new information (information gap closed)+ The implementation consists of five primary phases, orchestrated through a state machine: +
+ ++ Transform the user query into a detailed research brief with sources, constraints, and scope. + This ensures all downstream research is grounded in explicit requirements. +
+ ++ Generate a draft from the LLM's internal knowledge only—no external + information retrieval yet. This is the “noisy” initial state that provides structure + to guide subsequent research. It may contain outdated or incomplete information, and that's + intentional. +
+ ++ The core innovation. Each iteration follows four steps: +
+conduct_research: Retrieve external info for “denoising”refine_draft: Remove “noise” (imprecision, incompleteness) from draft+ Apply quality optimization with Insightfulness + Helpfulness rules. Deduplicate findings by URL, + add granular breakdowns, detailed mapping tables, nuanced discussion, and proper citations. +
+ ++ The core innovation is the Self-Balancing Test-Time Diffusion algorithm, + encoded directly in the supervisor's system prompt. Here is the exact algorithm from + the Go implementation: +
+ ++ Below are the actual prompts from the Go implementation. Understanding these is crucial for + implementing your own diffusion research system. +
+ ++ This prompt orchestrates the entire diffusion loop. Note the explicit algorithm, hard limits, + and scaling rules: +
+ ++ Each sub-agent operates with isolated context and strict iteration limits: +
+ ++ Before returning to the supervisor, each sub-agent's findings are compressed while + preserving ALL information verbatim: +
+ ++ The final synthesis applies both Insightfulness and Helpfulness rules: +
+ ++ The algorithm explicitly separates information gap closing from{' '} + generation gap closing: +
+ ++ “There is a trade-off between the two gaps. We cannot optimize the generation gap + too early when the system is still optimizing the information gap because the generation + gap tends to bring more verbose and stylistic content that can distract from finding + missing information.” ++ +
+ — Paichun Lin, ThinkDepth.ai +
Stage 1 characteristics:
+Stage 2 characteristics:
++ Long-horizon research tasks face several context challenges. The diffusion approach + addresses each systematically: +
+ +| Problem | +Description | +Diffusion Solution | +
|---|---|---|
| Context Poisoning | +Hallucinations enter context | +Draft serves as verified state | +
| Context Distraction | +Too much context overwhelms focus | +Parallel sub-agents with isolated contexts | +
| Context Confusion | +Superfluous context influences output | +Structured finding format with compression | +
| Context Clash | +Parts of context disagree | +Supervisor resolves conflicts during refinement | +
+ The draft serves as a persistent, verified context that: +
+refine_draft call is validated+ Sub-researchers operate with isolated contexts—they cannot see each other's + work. This prevents topic A's findings from biasing topic B's research, keeps context + from growing unboundedly during parallel work, and avoids confusion from interleaved search results. +
+ +
+
+ RACE evaluates report generation quality through four dimensions: +
++ FACT evaluates information retrieval and grounding capabilities: +
+