Writing Design Documents

In 2015, Google published its legendary "Bigtable" paper, a 14-page design document that launched a thousand databases. Cassandra, HBase, and countless internal systems trace their lineage to this single document. What made it so powerful wasn't the technology itself—it was the clarity with which the technology was explained: motivation, architecture, implementation details, and performance evaluation all woven together into a coherent narrative.

You may never write a paper that spawns an industry, but the skills exemplified in great design documents apply at every scale. A well-written design doc for your team's new feature prevents misunderstandings, reduces rework, and becomes a reference point for years. A poorly written doc creates confusion, leads to implementation debates, and is abandoned within weeks.

Design documents are where system design thinking becomes tangible. They synthesize requirements, architecture, diagrams, trade-offs, and implementation plans into an artifact that enables others to build, review, and maintain systems. This page teaches you how to write design docs that work.

Design documents serve multiple purposes for multiple audiences across different time horizons. Understanding these purposes helps you calibrate depth, detail, and structure.

Purpose 1: Alignment Before Implementation

The primary purpose of a design doc is to align stakeholders before significant engineering effort begins. It's dramatically cheaper to change a bullet point in a document than to refactor working code. Design docs surface disagreements when they're inexpensive to resolve.

A well-received design doc means:

• Engineering leadership agrees the approach is sound
• Product understands what they're getting and when
• Dependent teams know what interfaces to expect
• Security and compliance have flagged concerns early

Purpose 2: Reference During Implementation

During development, the design doc is the authoritative source for "what are we building?" When implementation questions arise ("Should we handle this edge case?"), the doc provides guidance. When scope creep threatens ("Can we also add X?"), the doc defines boundaries.

Purpose 3: Onboarding and Knowledge Transfer

Months or years later, new team members need to understand why the system exists and why it works the way it does. The design doc provides that context—far more effectively than reading through thousands of lines of code.

When to Write Design Docs:

Not every change needs a design document. The overhead of writing, reviewing, and maintaining is not zero. Guidelines:

Definitely write a design doc when:

• Development is estimated at more than 2 weeks
• Multiple engineers will collaborate on the implementation
• The change affects other teams' systems
• The change involves significant architectural decisions
• There are security, compliance, or regulatory implications
• The approach is novel or controversial

Probably skip the design doc when:

• It's a small, well-understood change
• You're the only person working on it
• There's an established pattern and this is another instance
• The change is easily reversible

When in doubt, write a lightweight design doc (1-2 pages). The writing process often reveals complexity you hadn't considered.

Design documents follow a general structure, though exact sections vary by organization. The key is progressing from context and motivation ("why") through architecture ("what") to implementation ("how").

Section 1: Summary/TL;DR (Required)

A one-paragraph summary that enables an executive to understand the project without reading further. Includes: problem being solved, proposed approach, key risk, timeline, and required resources.

Section 2: Motivation/Background (Required)

Why does this project exist? What user problem or business goal drives it? What happens if we do nothing? This section establishes urgency and importance.

Section 3: Goals and Non-Goals (Required)

Explicitly state what's in scope and what's not. Non-goals are as important as goals—they prevent scope creep and set expectations.

Section 4: High-Level Design (Required)

The architecture of the solution. This includes:

• System context diagram (how does this fit into the larger ecosystem?)
• Major components and their responsibilities
• Data flow at a high level
• Key interfaces with other systems

This section should be understandable without deep technical expertise.

Section 5: Detailed Design (Context-Dependent)

For engineering audiences, dive into implementation details:

• API definitions (endpoints, request/response formats)
• Data models (database schemas, event formats)
• Sequence diagrams for key flows
• Algorithms for complex logic
• Error handling and edge cases
• Performance considerations (caching, batching, indexing)

Section 6: Alternatives Considered (Required)

What other approaches did you evaluate? Why were they rejected? This demonstrates thorough analysis and prevents reviewers from suggesting options you've already dismissed.

Section 7: Trade-offs (Required)

Explicitly document what you're giving up with the chosen approach. This creates transparency and provides context for future decisions.

Section 8: Risks and Mitigations (Recommended)

What could go wrong? Technical risks, dependency risks, timeline risks. For each significant risk, describe mitigation strategies.

Section 9: Milestones and Timeline (Recommended)

Break the project into phases with deliverables and estimated dates. This enables progress tracking and creates opportunities for course correction.

Section 10: Open Questions (As Needed)

Questions you haven't resolved yet. These invite input during review and signal that you're aware of gaps rather than pretending certainty.

Technical writing is a skill distinct from technical thinking. Many engineers understand systems deeply but struggle to convey that understanding in prose. These techniques bridge the gap.

Technique 1: Lead with the Point

Every section, every paragraph, every sentence should lead with its main point. Don't build up to a conclusion—state the conclusion first, then support it.

❌ "We evaluated Redis, Memcached, and a custom solution. Memcached is simpler but lacks persistence. Redis has more features but is complex. The custom solution offers control but requires development time. The custom solution fits our needs best."

✅ "We're building a custom caching solution because off-the-shelf options don't meet our durability requirements. Redis lacks the specific eviction policies we need, and Memcached lacks persistence entirely."

Technique 2: Use Concrete Examples

Abstract descriptions are hard to evaluate. Concrete examples make ideas tangible.

❌ "The system will handle high throughput with low latency."

✅ "The system handles 10,000 requests per second at p99 latency under 50ms, based on load testing against a 3-node cluster."

Technique 3: Define Terms First Use

Assume readers don't share your context. When you introduce a term that might be unfamiliar or that you're using in a specific way, define it.

"We use 'partition key' to mean the primary identifier used for sharding data across nodes. In our system, user_id serves as the partition key."

Technique 4: Use Headings Generously

Headings create scan-ability. Readers should be able to identify sections relevant to them without reading everything. Use descriptive headings, not generic ones:

❌ "Architecture" ✅ "How Orders Flow from Cart to Fulfillment"

Technique 5: Create Visual Rhythm

Alternate between prose, lists, diagrams, and tables. Walls of text are intimidating. Visual variety maintains engagement and suits different learning styles.

Prose for narrative and rationale. Lists for enumeration. Tables for comparison. Diagrams for structure. Code blocks for precise API definitions.

Technique 6: Review Like a Newcomer

Before sharing, read your doc imagining you know nothing about the project. Where would you be confused? What questions remain unanswered? What terms are unexplained? Address these gaps before review.

Design documents aren't one-time artifacts—they evolve through distinct phases, and understanding this lifecycle improves how you write and maintain them.

Phase 1: Draft

The initial version, often rough and incomplete. Key characteristics:

• Contains open questions
• May have placeholder sections
• Shared with a small group for early feedback
• Rapidly iterated based on input

Don't aim for perfection in the draft. Aim for "good enough to get useful feedback." A draft that takes weeks to perfect misses early input that could have redirected the approach.

Phase 2: Review

The doc is complete enough for formal review. Key activities:

• Circulated to all stakeholders
• Scheduled review meeting (or asynchronous review period)
• Comments and concerns collected
• Revisions made based on feedback
• Approval obtained from required parties

Review is not a formality—it's where real alignment happens. Expect changes. A review that produces no feedback often means people didn't read the doc carefully.

Phase 3: Approved

The design is approved, and implementation begins. The doc becomes the reference for "what are we building?" Key practices:

• Mark the doc as "Approved" with date
• Update the doc when the design changes during implementation
• Note deviations from the original plan and why they happened
• Don't let the doc become stale

Phase 4: Living Document

Post-implementation, the doc shifts to maintenance mode:

• Serves as onboarding reference
• Updated when significant changes are made to the system
• Linked from README files and wiki pages
• May spawn supplemental docs for specific areas

Phase 5: Archived

When the system is deprecated or replaced:

• Mark as "Archived" with date and reason
• Optionally link to successor documentation
• Preserve for historical reference but clearly mark as outdated

Keeping Docs Alive:

The biggest failure in design documentation is stale docs that no longer match reality. Mitigation strategies:

• Include "Last Updated" dates prominently
• Schedule periodic reviews (quarterly for active systems)
• Make doc updates part of the PR process for significant changes
• Have a clear owner responsible for the document

The design review meeting (or asynchronous review process) is where alignment happens. Done well, it improves the design and builds buy-in. Done poorly, it's a waste of everyone's time.

Pre-Review Preparation (Author):

1. Share the doc early: Reviewers need time to read—2-3 days minimum for substantial docs.
2. Identify specific feedback requests: "I'm especially looking for input on the data model" focuses reviewers.
3. List open questions: Make it clear where you're uncertain.
4. Confirm attendees: Ensure the right people are invited (everyone whose approval is needed or who will be affected).

Pre-Review Preparation (Reviewers):

1. Actually read the document: Show up prepared. "I'll read it in the meeting" is disrespectful of everyone's time.
2. Add written comments in advance: Most feedback should happen asynchronously via doc comments.
3. Come with questions, not just critiques: Assume you might be missing context.

During the Review:

1. Don't re-read the doc: Assume everyone has read it. Quickly summarize (1-2 minutes) for context, then dive into discussion.
2. Focus on unresolved issues: Don't rehash comments already addressed. Focus on decisions that need live discussion.
3. Time-box discussions: If a topic isn't resolvable in 5 minutes, take it offline with a smaller group.
4. Capture action items: Who is doing what by when? Document decisions made.
5. End with clear next steps: Is this approved, approved with changes, or needs another review?

Handling Difficult Reviewers:

The Bikeshedder: Focuses on trivial details while ignoring major issues. → "Let's take naming discussions offline. Can we focus on the data model concerns first?"

The Absent Approver: Claims stake but never reads docs. → Set clear deadlines: "If I don't hear back by Friday, I'll assume approval."

The Scope Expander: Wants to add features during review. → "That's a great idea for v2. Let's capture it in the Future Work section."

The Perfect is the Enemy: Nothing is ever good enough. → "Given our timeline, is this a blocking concern or something we can iterate on?"

The Late Critic: Raises fundamental objections after approval. → Ensure they're in initial reviews. If concerns are valid, consider changes; if not, explain why the ship has sailed.

Asynchronous Reviews:

Not all design reviews need meetings. For smaller designs or distributed teams:

1. Set a review deadline (e.g., "Please comment by Friday EOD")
2. @-mention required reviewers to ensure visibility
3. Address comments in writing
4. Mark the doc as approved when sufficient consensus is reached
5. Hold a sync meeting only if unresolvable disagreements emerge

Learning from common failures accelerates your doc-writing skills. These mistakes appear repeatedly across organizations.

Mistake 1: Implementation Plan Masquerading as Design

The doc describes what files to create and what functions to write, but not why the architecture looks this way. This conflates planning with design.

Design docs should be comprehensible to someone who doesn't know your codebase. If understanding requires reading the code, the doc has failed.

Mistake 2: Diagram Without Context

A beautiful architecture diagram with no explanatory text. The diagram shows boxes and arrows, but what are the boxes? Why do the arrows exist? What decides routing?

Every diagram needs surrounding prose explaining what it shows, what's important, and what assumptions it encodes.

Mistake 3: No Alternatives Section

"We're using Kafka." Why? What else did you consider? Why not RabbitMQ? Why not direct HTTP?

Without alternatives, reviewers can't evaluate whether you explored the space. They'll either accept your choice blindly or suggest alternatives you already dismissed, wasting everyone's time.

Mistake 4: Wrong Abstraction Level

A doc that mixes strategic decisions ("should we adopt microservices?") with tactical details ("the User class has a name field") is confusing. Readers can't tell what's important.

Pick one level: strategic design docs focus on architecture and trade-offs; tactical docs focus on implementation details. Link between them if both are needed.

Mistake 5: Premature Detail

Specifying every edge case and error code before the high-level design is approved. Detailed work on an approach that might be rejected is wasted.

Start with enough detail to evaluate the approach. Add detailed design after high-level approval.

Mistake 6: One Author, No Review

The design exists only in one engineer's head, committed to paper for formality but never truly reviewed.

Design docs are communication tools. If they aren't read and challenged, they serve no purpose. Require actual review before approval.

Mistake 7: Perfect Document, No Action

Beautiful docs that never get implemented. Design is not an end in itself—it's a means to building systems.

Connect design docs to actionable work items. The doc should conclude with clear next steps and who owns them.

Design documents crystallize system design thinking into artifacts that enable teams to align, build, and maintain complex systems. They're where diagrams, trade-offs, and architectural decisions come together into coherent narratives that guide implementation.

Let's consolidate the key takeaways:

Module Complete:

This concludes Module 6: Documentation and Communication. You've learned how to:

1. Visualize architecture with different diagram types for different purposes
2. Create sequence diagrams that show dynamic system behavior over time
3. Communicate trade-offs to different audiences with appropriate framing
4. Write design documents that align teams and guide implementation

These skills are the bridge between technical design expertise and organizational effectiveness. Brilliant designs that can't be communicated don't get built. Clear documentation turns ideas into systems.

With this module complete, you've finished Chapter 2: System Design Thinking. You now have the mental models, analytical frameworks, and communication tools to approach system design problems systematically.