Design a multi-agent research system

30-second answer

• An orchestrator decomposes the user's research question into 5-10 sub-questions, then dispatches them to parallel subagents. End-to-end latency equals the slowest subagent (30-40s), not the sum (200s+).
• Each subagent has access to a tool registry (web search, RAG retrieval, calculator). Tools are stateless and MCP-compatible.
• Every claim in the final report must link to a source. Sourceless claims are rejected by the grounding checker, not guessed at. Source-grounded generation reduces hallucinated citations from 70% to under 5%.
• A human approval checkpoint lets the user review the research plan (sub-questions + sources found) before the orchestrator synthesizes the final report.
• Conflicting findings from different subagents are surfaced explicitly with evidence weights rather than silently resolved. This mirrors how academic meta-analyses handle contradictory evidence.
• Hierarchical budget allocation keeps cost predictable: $2.00 per job with real-time monitoring and adaptive redistribution across agents.

Requirements and assumptions

Functional requirements

• The user submits a research question and receives a 2-3 page written report with inline citations.
• The system searches the web, internal knowledge bases, and structured data sources for relevant information.
• Every factual claim in the report links to a cited source. Claims without sources are flagged or dropped.
• The user can review and modify the research plan (sub-questions, sources) before the system generates the final report.
• Conflicting information from different sources is surfaced to the user rather than arbitrarily resolved.

Non-functional requirements

• End-to-end latency: 30-120 seconds for a comprehensive report (acceptable, not interactive).
• Each subagent has a hard 30-second timeout. Timed-out agents return partial findings rather than blocking the pipeline.
• The system runs up to 10 subagents in parallel.
• All retrieved source material is stored for 90 days for auditability.
• Cost per research job: under $2 at GPT-4o pricing (approximately 500K tokens total for 10 subagents).
• Citation accuracy: 95%+ of cited URLs must be alive and content-matched at report delivery time.

The hardest engineering problem is not the LLM calls. It is preventing hallucinated citations. Ungrounded systems can fabricate citations or point to the wrong content. Source-grounded generation with post-hoc verification is a strong default.

Assumptions

• The planner can search only approved source systems, and every retrieved passage receives a stable source_id and capture timestamp.
• Subagents are bounded workers: they have typed tasks, tool limits, deadlines, and a fixed budget. They cannot recursively create unlimited work.
• The user can inspect or edit the plan before synthesis. Conflicting evidence is preserved and shown rather than hidden by a single “best” answer.

5-minute approach

Design the system as a bounded map–reduce workflow. A planner turns the question into typed sub-questions, independent agents gather evidence in parallel, and a synthesizer writes only from source-linked findings before a grounding check verifies the report.

• Allocate tasks by coverage and cost, deduplicate overlapping searches, and cap each agent's tool calls.
• Store findings as claims paired with source excerpts, URLs, timestamps, and confidence—not as unstructured agent prose.
• Use a deadline so slow agents contribute partial results; reserve enough budget for synthesis and verification.
• Surface conflicts and unsupported claims in the report or send them back for review rather than silently inventing resolution.

The entities and endpoint below define the job, sub-task, evidence, and report lifecycle.

Core entities

ResearchJob

• job_id, user_id, original_question, status (decomposing | researching | awaiting_approval | synthesizing | complete), report_text, created_at, completed_at

SubQuestion

• subq_id, job_id, question_text, assigned_agent_id, question_type (factual | comparative | statistical | opinion), status, findings_json, sources[], confidence_score, elapsed_ms, token_budget

Source

• source_id, url, title, retrieved_text, retrieved_at, relevance_score, authority_score, subq_id

ReportClaim

• claim_id, report_id, claim_text, source_ids[], grounding_score, verified, verbatim_excerpt

API design

POST /api/research -- submit a research question

Request:  { "question": "What are the best practices for database sharding in 2025?", "depth": "comprehensive" }
Response: { "job_id": "job_abc", "status": "decomposing", "estimated_seconds": 60 }

GET /api/research/{job_id}/plan -- review research plan before synthesis

Response: {
  "sub_questions": [
    { "id": "subq_1", "text": "What are the main sharding strategies?", "type": "factual" },
    { "id": "subq_2", "text": "What do Postgres and MongoDB support natively?", "type": "comparative" }
  ],
  "sources_found": 34,
  "estimated_cost": "$1.20",
  "awaiting_approval": true
}

POST /api/research/{job_id}/approve -- approve or modify the plan

Request:  { "approved": true, "remove_subquestion_ids": ["subq_3"], "add_questions": ["What are the failure modes?"] }
Response: { "job_id": "job_abc", "status": "synthesizing" }

GET /api/research/{job_id}/report -- retrieve the completed report

Response: {
  "report_text": "...",
  "citations": [{ "id": "src_1", "url": "...", "verified": true }],
  "conflicts": [{ "claim": "...", "sources": ["src_2", "src_5"], "resolution": "surfaced" }],
  "grounding_score": 0.94,
  "total_cost": "$1.45"
}

GET /api/research/{job_id}/stream -- SSE stream for real-time progress

data: { "phase": "researching", "agent": "subagent_3", "progress": "Found 8 relevant sources" }
data: { "phase": "synthesizing", "progress": "Writing section 2 of 5" }

45-minute interview approach

Keep this as a design question about orchestration, evidence flow, and bounded parallelism—not a tour of autonomous-agent features.

• 0–5 min — Clarify scope: ask source types, citation format, user approval points, freshness requirements, report length, and whether sources may be private or paid.
• 5–10 min — Requirements and estimates: state report latency, subagent count, per-agent timeout, token/cost budget, source-retention period, and citation-accuracy target.
• 10–18 min — Interfaces and data model: define research jobs, plans, sub-tasks, tool calls, sources, evidence claims, conflicts, synthesis runs, and verification results.
• 18–28 min — High-level design: draw planner, user checkpoint, parallel workers, source cache, evidence store, synthesizer, grounding verifier, and report delivery.
• 28–38 min — Deep dive: explain decomposition, source quality, citation propagation, deduplication, disagreement handling, and synthesis context limits.
• 38–42 min — Scale and operations: cover fan-out limits, timeouts, budget enforcement, retries, provider outages, source freshness, and audit storage.
• 42–45 min — Trade-offs and close: compare more agents with overlap and cost, breadth with verification time, and full reports with partial results; state the safe behavior for missing evidence.

High-level design and data flow

The orchestrator receives the user question and calls a capable LLM (GPT-4o or Claude Sonnet) to decompose it into 5-10 sub-questions that collectively cover the topic. Decomposition is not trivial: a good orchestrator balances breadth (covering different angles) with specificity (sub-questions narrow enough for a single agent to answer well). Poor decomposition produces vague sub-questions that return generic web search results.

The orchestrator dispatches all sub-questions to the subagent pool simultaneously. Each subagent is an independent LLM process with access to the tool registry. Subagents run in parallel, with a 30-second timeout enforced by the orchestrator. A timed-out agent returns whatever partial findings it has accumulated. The orchestrator waits for all agents (or their timeouts) before proceeding.

Before synthesis, the system presents the research plan and sources found to the user as a human-in-the-loop checkpoint. This is the approval moment: the user can prune irrelevant sub-questions or add missing angles. After approval, the orchestrator deduplicates sources, identifies conflicting findings, and passes everything to a synthesis LLM that writes the final report with inline source citations.

The data flows through three distinct phases. Phase 1 (decomposition) takes 2-5 seconds: the orchestrator LLM generates sub-questions, classifies them by type, and estimates token budgets. Phase 2 (research) takes 20-30 seconds: all subagents run in parallel, each making 2-3 tool calls (search, fetch, summarize). Phase 3 (synthesis) takes 10-20 seconds: deduplication, conflict detection, human approval, report writing, and grounding verification.

The human checkpoint between phases 2 and 3 is the critical design decision. Without it, the system produces reports the user did not ask for. With it, the user sees the sub-questions and source count, can remove irrelevant angles, and add missing ones. This increases user trust and report relevance at the cost of 30-60 seconds of human review time. For automated research workflows (no human in the loop), this checkpoint becomes an optional quality gate controlled by a confidence threshold.