Pranav Gupta Chummun

SpecForge

Ten agents that take a ticket all the way to a pull request, ready for your review.

2025Multi-agent systemsAgentic RAGReActMCPEvaluation

Problem

Writing the code is only one part of a ticket. You read the ticket and everything linked to it, find the right files, plan the change, write the tests, fix what breaks and then write up the pull request. AI helps with pieces of that, but you’re still the one carrying the work from step to step.

What I built

SpecForge takes a ticket and carries it all the way to a pull request. Ten specialist agents each handle one part of the job and pass their work to the next as a checked file. When it’s unsure, it asks you. When it’s done, you review the pull request.

Architecture

InputYour ticket Read-only sourcesLinks, wiki, code and past PRs OrchestratorRuns every step and saves its state ContextWrites the brief Project knowledgeExplains the product Index agentFinds the right code PlannerDecides what to build Test agentPlans the scenarios first Code agentWrites the code Test agentWrites the tests Review agentReviews like your team VerifyDecides when it's done PR agentPull requestEvery claim linked to its source YouReview and mergeThe last call is always yours Fix loop InputYour ticket Read-onlySourcesLinks, wiki, code, PRs OrchestratorRuns every step and saves state ContextWrites the brief Project knowledgeExplains the product Index agentFinds the right code PlannerDecides what to build Test agentPlans the scenarios first Code agentWrites the code Test agentWrites the tests Review agentReviews like your team VerifyDecides when it's done PR agentPull requestEvery claim linked to its source YouReview and mergeThe last call is always yours

Key features

  • Ticket to pull request. Assign a ticket and get back a pull request ready for your review.
  • Plans before it codes. The goal, the interfaces and the expected results are agreed before any code exists.
  • One writer at a time. Exploration is read-only, and only the Code agent writes production code.
  • Asks when it’s unsure. It stops and asks you instead of guessing.
  • Learns from your team. Comments from past pull requests become the rules it reviews against.
  • Learns from failure. A failed test becomes a note that the next try reads.
  • Text is data. Anything in a ticket or wiki is treated as data, never as instructions.
  • Picks up where it stopped. Every step is saved, so a stopped run can resume.

The agents

Orchestrator

Runs the whole job and decides what happens next.

Prompt chainingModel routingHuman-in-the-loopPrompt cachingJSON Schema handoffsCheckpointed event logToken and cost budgetsAgent-to-agent chat

Context

Reads the ticket and everything linked to it, then writes one brief.

MCP tool useStructured extractionFew-shot promptingPrompt injection guardrailsQuery decompositionGraphRAG

Project knowledge

Explains how the product works, with sources.

RAGContextual retrievalHybrid searchCitationsCurated knowledge fileGraphRAG

Index agent

Searches the repo and returns only what matters.

Agentic RAGSubagentsQuery rewritingtree-sitter parsingSymbol and call graphBM25 plus ripgrepHNSW vector indexCross-encoder reranking

Planner

Turns the brief into a goal, the projects to touch and the public interfaces.

Plan-and-executeExtended thinkingStructured outputsClarifying questionsInterface-first design

Test agent

Plans the scenarios before any code exists, then writes the tests from the outside.

LLM test generationChain-of-thoughtDynamic few-shotBlack-box testingCoverage-guided fuzzingBoundary value analysis

Code agent

The only agent that writes production code.

ReActDynamic few-shotContext compactionAgent skillsGit worktree sandbox

Review agent

Reviews the change in a fresh context, using rules learned from past pull requests.

LLM rule inductionContrastive in-context learningCritic agentConfidence thresholdingWeakly supervised comment miningEmbedding clusteringFine-tuned comment classifier

Verify

Decides when the work is done. Code gives the pass or fail, and a model explains what went wrong.

Evaluator-optimizerReflexionLLM root cause analysisHuman-in-the-loopFull test suite runAC-to-test traceabilityScenario result comparison

PR agent

Writes the pull request from what the run already produced.

Grounded generationStructured outputsTemplate induction

Quality & testing

Each agent has its own measure, such as Recall@10 for the Index agent and precision on replayed pull requests for the Review agent. Closed tickets are replayed against a plain model as a baseline, so a technique only stays if it helps.