AI Research Agent

The Problem

When technical teams investigate emerging architectural patterns or specialized hardware benchmarks, standard LLM web-search integrations fail in three predictable ways:

1. Context Window Flooding: Naive retrieval feeds hundreds of low-signal snippets, diluting reasoning capability.
2. Unsupported Inferences: LLMs synthesize plausible-sounding claims without citing verifiable sentences from underlying whitepapers.
3. Premature Termination: Complex inquiries require iterative follow-up queries that standard single-turn RAG pipelines simply cannot handle.

The Constraints

• Zero Fabrication Tolerance: Every factual claim in the final generated technical brief must be grounded by an exact page/paragraph URI.
• Latency Budget: Full synthesis must complete in under 90 seconds while querying multiple academic repositories (arXiv, IEEE, GitHub repos).
• Execution Cost: Must avoid runaway subagent loops by capping exploration trees with strict token budget governors.

The Architecture

The system adopts an asynchronous ReAct + Hierarchical Planner topology built on top of LangGraph:

               ┌──────────────────────────┐
               │    User Research Goal    │
               └────────────┬─────────────┘
                            │
                            ▼
               ┌──────────────────────────┐
               │    Planner Agent (Decomp)│
               └──────┬────────────┬──────┘
                      │            │
         ┌────────────▼──┐      ┌──▼────────────┐
         │ ArXiv Search  │      │ GitHub Ingest │
         └──────┬────────┘      └──┬────────────┘
                │                  │
                └─────────┬────────┘
                          ▼
               ┌──────────────────────────┐
               │  Evidence Ranker (Rerank)│
               └──────────┬───────────────┘
                          │
                          ▼
               ┌──────────────────────────┐
               │  Verification & Synthesis│
               └──────────────────────────┘

How It Works: The 5-Step Pipeline

1. Query Decomposition: The goal is parsed into an execution DAG with atomic sub-hypotheses.
2. Targeted Ingestion: Specialized tools fetch authoritative markdown/PDF artifacts, stripping advertising and boilerplate.
3. Reranking & Clustering: Chunks are embedded with cosine rerankers to remove redundant documents.
4. Adversarial Fact-Check: A separate critique prompt challenges each synthesis assertion against the retrieved citations.
5. Structured Markdown Generation: The brief is assembled with complete citation links, confidence scores, and architectural diagrams.

What Failed & Engineering Decisions

The Failure: Context Dilution in Early Iterations

In early prototypes, retrieving the top 20 semantic chunks caused the model to lose the core thesis (the “lost-in-the-middle” effect). Fact verification dropped to 71% because contradictory claims across non-peer-reviewed blogs poisoned the synthesis context.

The Decision & Fix

We instituted a strict two-stage filtering policy:

• Authority Weighting: Academic preprints and official technical documentation receive 3x score weighting over informal blog posts.
• Context Compaction: Before handing documents to the synthesis model, a summarizer node extracts only direct quantitative evidence. This reduced token consumption by 48% while boosting citation accuracy to 94.2%.

Results & Impact

• Deployed across an internal engineering team to produce technical evaluation reports in minutes rather than hours.
• 68% reduction in hallucinated claims compared to standard naive RAG implementations.
• Full provenance tracking where every claim links directly to its source paragraph.