Security in System Design

In 2017, Equifax disclosed one of the most devastating data breaches in history. 147 million individuals—nearly half the U.S. population—had their most sensitive personal information exposed: Social Security numbers, birth dates, addresses, and driver's license numbers. The root cause? A known vulnerability in Apache Struts that had a patch available for two months before the breach occurred. The cost? Over $1.4 billion in direct expenses, a stock price collapse of 35%, and the resignation of multiple executives including the CEO.

This wasn't a sophisticated nation-state attack. It wasn't zero-day exploitation requiring genius-level hackers. It was a failure to treat security as a requirement—as fundamental to the system as the features it delivered.

Every system design decision you make either reinforces or undermines security. This page establishes why security cannot be bolted on after the fact and must instead be woven into the fabric of every architectural decision from day one.

One of the most dangerous misconceptions in software development is treating security as a feature that can be added to a backlog, prioritized against other features, and implemented when convenient. This framing is fundamentally flawed.

Features are additive; security is multiplicative.

Consider the difference:

• Adding a new search filter is a feature. It enhances the product but doesn't break existing functionality if delayed.
• Adding authentication is not a feature. Without it, every other feature is exposed. It's not enhancement—it's foundational.

Security operates as a systemic property—like structural integrity in a building. You don't add 'structural integrity' to a skyscraper in sprint 12. You design for it from the foundation up, and every subsequent decision either maintains or compromises it.

The architecture determines the attack surface.

When you make architectural decisions—choosing communication protocols, designing data flows, selecting authentication mechanisms, deciding on service boundaries—you are simultaneously designing your attack surface. These decisions are difficult or impossible to change later:

• If you store passwords in plaintext 'temporarily' to launch faster, remediation requires migrating every user
• If you communicate between services over unencrypted channels, encrypting later requires updating every service
• If you embed secrets in environment variables without rotation capability, adding rotation requires redesigning deployment

The cost of addressing security late is not linear—it's exponential. Studies consistently show that fixing a security flaw in production costs 30-100x more than addressing it during design.

The threat landscape facing modern systems has evolved dramatically. Understanding this landscape isn't about inducing paranoia—it's about informed design decisions. You cannot defend against threats you don't understand.

The democratization of attack tools:

Once, sophisticated attacks required significant expertise. Today, attack toolkits are readily available, automated scanning finds vulnerabilities in minutes, and 'hacking as a service' lowers the barrier for malicious actors. The question isn't if your system will be attacked—it's when and how often.

Attack surface expansion in modern architectures:

Modern distributed systems dramatically expand the attack surface compared to traditional monolithic applications. Consider what you're managing:

• Microservices: Each service is a potential entry point; inter-service communication must be secured
• APIs: Every API endpoint is externally accessible and must validate inputs, authenticate callers, authorize actions
• Cloud Infrastructure: Misconfigured IAM policies, exposed storage buckets, overly permissive security groups
• Containers and Orchestration: Container escapes, insecure images, secrets management in Kubernetes
• Third-Party Integrations: Every external service you depend on extends your attack surface

The complexity of modern systems means that security must be systematically addressed at every layer, not concentrated in a single point of enforcement.

Security often struggles to get priority in product discussions because benefits are invisible until breaches occur. Engineers and architects must learn to articulate security in terms that resonate with business stakeholders.

Translating security into business language:

Instead of discussing abstract vulnerabilities, frame security decisions in terms of business risks, regulatory requirements, and competitive advantages:

Compliance as a business enabler:

In many industries, security compliance is not optional—it's a prerequisite for doing business:

• Financial Services: PCI DSS compliance is mandatory for processing payments. Without it, you cannot accept credit cards.
• Healthcare: HIPAA compliance is required for any system touching patient health information. Violations carry penalties up to $1.5M per incident.
• Enterprise Sales: SOC 2 Type II reports are table stakes. Enterprise customers will not onboard without demonstrating security controls.
• Government Contracts: FedRAMP authorization is required for cloud services sold to federal agencies.

Security is not a cost center—it's a market access requirement. Build it in from the start, or rebuild your system when you try to scale into these markets.

Shift-left is the practice of addressing security concerns earlier in the development lifecycle—'shifting' activities traditionally performed late (testing, review, patching) to the 'left' side of the timeline (design, development, integration).

The traditional model (shift-right):

Design → Development → Testing → [Security Review] → Production → [Security Patching]

In this model, security reviews happen after the system is built. Findings require expensive rework, and production patches are reactive rather than preventive.

The shift-left model:

[Threat Modeling] → [Secure Design] → [Secure Coding] → [Automated Security Testing] → Production

Security is embedded at every stage. Architectural decisions incorporate security constraints. Code is written defensively. Automated tools catch issues in CI/CD before they reach production.

Implementing shift-left security:

Shift-left is not just a philosophy—it requires concrete practices:

1. Threat Modeling in Design: Before writing code, identify assets, threats, and mitigations. We'll cover this in detail in a later page.

2. Security Requirements: Include security acceptance criteria in user stories: 'User passwords must be hashed with bcrypt, cost factor 12.'

3. Secure Coding Training: Developers must understand common vulnerabilities (OWASP Top 10) and how to prevent them.

4. Automated Security Tools in CI/CD:

   • SAST (Static Analysis): Scans source code for vulnerabilities before merge
   • DAST (Dynamic Analysis): Tests running applications for exploitable issues
   • Dependency Scanning: Identifies known vulnerabilities in third-party libraries
   • Container Scanning: Checks base images for vulnerabilities

5. Pre-Commit Hooks: Prevent secrets, credentials, or insecure patterns from entering version control.

6. Security Champions: Embed security-minded engineers within development teams to provide guidance without bottlenecking.

Just as functional requirements describe what a system must do, security requirements describe how a system must protect itself and its users. These requirements must be explicit, measurable, and testable—not vague aspirations.

Categories of security requirements:

Writing effective security requirements:

Poor security requirement: 'The system should be secure.'

This is unmeasurable and untestable. What does 'secure' mean? Secure against what threats? To what degree?

Effective security requirements are specific:

In system design interviews, security is often an afterthought—candidates focus on scalability, data models, and API design while neglecting the security implications of their choices. This is a missed opportunity.

What interviewers observe:

Senior engineering candidates are expected to proactively identify security concerns without being prompted. When you design a system, you should naturally consider:

• 'How is this endpoint authenticated?'
• 'What happens if someone sends malicious input here?'
• 'Who has access to this data, and should they?'
• 'How do we protect this data in transit and at rest?'
• 'What are the threat vectors for this architecture?'

Candidates who raise these concerns demonstrate senior-level thinking—the ability to see not just what a system does, but what could go wrong.

Integrating security naturally:

Don't treat security as a separate section at the end of your design. Weave it into each component as you discuss it:

• Designing an API? Mention authentication in the same breath as the endpoint structure.
• Designing a database schema? Note which fields contain sensitive data and how they're protected.
• Designing service communication? Specify whether you're using mTLS between services.
• Designing storage? Mention encryption at rest and key management.

This demonstrates that security is part of your engineering intuition, not an afterthought.

Understanding what goes wrong helps you avoid repeating these mistakes. Here are patterns that consistently lead to security failures:

We've established the foundational mindset for security in system design. Let's consolidate the key takeaways:

What's next:

Now that we understand why security must be a requirement, we'll explore the practical strategy for implementing it: Defense in Depth. This principle ensures that no single point of failure can compromise your system, layering controls so that attackers must bypass multiple barriers to succeed.