We had a problem. Three solutions, eighty projects, and almost no technical documentation. The kind of codebase where tribal knowledge lives in the heads of developers who may or may not still be on the team.
Manual documentation was the obvious answer. It was also never going to happen. Not thoroughly, anyway. Not with the depth that future developers would actually need.
So I built two custom agents in GitHub Copilot—a documentation planner that outlines what needs to be documented —and a documentation writer that creates the actual files based on the plan. The whole thing took about five days. Here’s what worked, what didn’t, and why splitting one agent into two made all the difference.
The approach that worked
I ended up with two separate agents: a documentation planner and a documentation writer.
The planner’s job is analysis. It walks through the entire solution structure, examines projects, and produces a phased documentation plan. The output is a readable markdown document with structured checklists — what needs documenting, in what order, grouped into logical phases with priorities.
The writer’s job is execution. It takes one topic from the plan, generates a single documentation file following a strict template, and checks that item off the list. Then it moves to the next one.
That separation was the key insight. But I didn’t start there.
The approach that didn’t
My first version was a single agent that did both planning and writing. Seemed efficient — why switch between agents when one could handle everything?
Two problems killed it.
First, context overload. By the time the agent had analyzed the codebase structure, built a mental model of what needed documenting, and started actually writing, it was juggling too much. The output lost focus. Documentation for one project would bleed into another. Priorities got muddled.
Second, scope creep within a single session. The agent would start documenting a feature, notice something adjacent, and wander off to document that instead. No clear boundaries meant no clear progress.
Splitting the agents forced the separation of concerns. The planner only plans. The writer only writes. Neither tries to do the other’s job.
Getting the guardrails right
Even with two agents, the writer gave me trouble early on.
In the first few runs, it would analyze code to understand a feature, then suggest improvements to the code itself. I’d ask it to document how a service worked, and it would rewrite the service. Not helpful when the goal is documentation, not refactoring.
The fix was to add explicit constraints to the agent instructions: you are documenting, not improving. Do not modify source files. Your only output is markdown.
The other issue was structural inconsistency. Without a defined template, each documentation file had a different shape. Some started with architecture diagrams. Some had feature lists up front. Some buried the “how to use this” section three pages down.
I locked down the structure:
- What problem does this solve? Who uses it? What’s the key capability?
- Feature list with brief descriptions
- Technology stack and NuGet dependencies
- How to actually use it
- Architecture overview as a Mermaid diagram
Every documentation file follows that template. No variation. The consistency makes the final documentation set navigable — you always know where to find what you’re looking for.
The markdown trick for working across sessions
Five days of documentation work don’t happen in one sitting. I needed a way to stop at the end of the day and pick up exactly where I left off the next morning.
The solution: the planner’s output lives in a markdown file that the writer both reads and updates.
When the writer finishes documenting a topic, they check off that item in the plan. The plan becomes a living document, a shared state between sessions and agents. The next day, I spin up the writer, it reads the plan, sees what’s done and what’s pending, and continues from the right spot.
No context lost. No duplicate work. No “wait, did I already document that project?”
This pattern — using markdown files as a persistent state that agents can read and write — has applications way beyond documentation. Any multi-session workflow benefits from providing agents with a place to track progress that persists across session boundaries.
With all this in place, was it actually worth the effort?
Three solutions, eighty projects, five days, one person.
The documentation is more thorough than anything I would have written manually. Each file has architecture diagrams. Each file lists dependencies. Each file follows the same structure, so developers can find what they need.
But let me be clear: the output wasn’t publish-ready the moment the agents finished.
Every generated document needed a read-through. Sometimes the agent misunderstood a feature’s purpose. Sometimes it got the relationship between components slightly wrong. Sometimes the Mermaid diagram was technically accurate but missed the conceptual point. The agents did 90–95% of the work, but the last 5–10% of validation and correction is still on you.
Here’s the thing, though: reviewing and tweaking existing documentation is a fundamentally different task than staring at a blank page. The agents gave me something to react to. “This paragraph is wrong” is easier than “What should this paragraph say?” The cognitive load shifts from generation to validation, and that’s a much lighter lift.
Would I have gotten there without the agents? Eventually, maybe. But realistically? I would have written high-level overviews for the critical projects and called it done. The long tail of smaller projects would have stayed undocumented.
The agents didn’t just save time—they raised the quality floor. Every project got the same treatment, not just the ones I personally understood best or found most interesting. Even with the review pass, five days for eighty projects is a pace I couldn’t have matched manually.
What I’d tell someone starting out
Start with two agents, not one. The temptation to build a single “super agent” is strong, but specialized agents with clear boundaries work better. Think microservices, not monolith.
Invest in guardrails early. The first version of your agent will do things you didn’t expect. That’s fine — but catch it before you’ve let it loose on your whole codebase. Define what it should not do as clearly as what it should.
Use markdown as your state layer. If your workflow spans multiple sessions or multiple agents, give them a shared document to coordinate through. It’s low-tech, and it works.
Always validate the output. This might be the most important lesson. Agents are confident writers—they’ll produce polished, structured documentation that reads authoritatively. But they get things wrong. They misinterpret relationships between components. They make assumptions that seem reasonable but aren’t quite right. Budget time for a human review pass on everything an agent produces. The 90–95% they give you is valuable, but shipping the remaining 5–10% of errors will cost you credibility with the developers who actually use the docs. Agents are drafters, not publishers.
And finally, your agents will reflect your clarity of thought. If your documentation lacks a consistent structure, your agents will too. Up-front planning—defining each file’s content, documentation order, and scope—directly drives agent effectiveness.
Agents amplify your clarity and process, not replace it. Success relies on the groundwork you lay.
