Skip to main content

System Design With AI

5 min read
Tech LeadCto

Software Arch

AI suggests patterns. It doesn't know your team's skills, your tech debt, or what you're trying to get off of. You do.

Cloud Arch

AI recommends services. It doesn't know your vendor strategy, compliance, or cost budget. You own the constraints.

Cto

AI can draft an architecture doc. The 'why' and 'what we're optimizing for' — that's executive judgment.

System Design With AI

TL;DR

  • AI can propose architectures, suggest components, and draft design docs. It knows patterns.
  • AI doesn't know your constraints: team skills, existing systems, budget, compliance, or "what we're actually optimizing for."
  • Use AI as a sparring partner. Never as the final authority.

System design is one of the highest-leverage human skills. AI can assist. It can also lead you into generic, wrong-for-you designs if you're not careful.

Where AI Helps

Pattern Suggestion

Prompt: "Design a system for real-time recommendations."

What you get: Kafka or similar for events. A serving layer. Maybe a feature store. Caching. Standard patterns from the literature.

Why it helps: You might not have considered all the options. AI surfaces patterns you can evaluate.

Trade-off Exploration

Prompt: "What are the trade-offs between synchronous vs. async for this API?"

What you get: Latency vs. complexity. Consistency vs. availability. CAP theorem references. Reasonable framing.

Why it helps: AI can structure the debate. You still decide.

Documentation Drafting

Prompt: "Create an ADR (Architecture Decision Record) for choosing PostgreSQL over MongoDB."

What you get: Template. Pros and cons. Maybe a decision and context.

What you add: Your actual context. "We chose Postgres because the team knows it and we're migrating off Mongo next quarter." AI doesn't know that.

Where AI Is Dangerous

Ignoring Your Constraints

  • "Use Kubernetes." — Do you have the team to run it? The budget? The need? AI suggests what's popular, not what fits.
  • "Microservices." — Maybe. Or maybe you're a 5-person team and a monolith is fine. AI doesn't know your org size.
  • "Event-driven." — Adds complexity. Do you need it? AI likes event-driven. It doesn't ask "for this use case?"

Generic Over Specific

  • AI gives you the textbook design. Your system has legacy. Tech debt. Political constraints ("we can't change that system, it's owned by another division"). AI designs in a vacuum. You design in reality.

Misunderstanding Requirements

  • "Design for 10K QPS." — AI might give you a design. But do you need 10K today or in 3 years? What's the growth curve? AI doesn't ask. It assumes.
  • "Design for 99.9% availability." — Do you mean 99.9% or "we've never had an outage and the board will notice"? Those are different. AI treats them the same.

Vendor and Lock-in

  • AI might suggest AWS, GCP, or vendor-specific services. Do you have a multi-cloud strategy? A vendor you're trying to reduce? AI doesn't know. It gives generic cloud patterns.

How to Use AI for System Design

  1. Prompt with context. "We're a 20-person team, migrating off legacy Oracle, need to support 5K concurrent users. Design options?" The more context, the better the suggestions — and the more you'll still need to filter.
  2. Treat output as options, not answers. AI suggests. You evaluate against your constraints.
  3. Always add the "why." AI-generated ADRs are sterile. Add "We chose X because of Y constraint." That's the part that matters in 2 years.
  4. Don't delegate "what are we optimizing for?" That's strategy. AI can't do it.

Quick Check

AI suggests 'Use Kubernetes' for your new service. What might it be missing?

You read papers, sketch designs, debate trade-offs. Days of research. Then you write the ADR from scratch.

Click "With AI" to see the difference →

Do This Next

  1. Ask AI to design a system you've actually built or are building. Compare its proposal to your reality. What did it get wrong? What would you have to explain?
  2. Document one constraint AI wouldn't know about your architecture. Team size, budget, compliance, politics. That's the context that makes your design right.