Dominance

In 1970, when E.F. Codd published his paper on the relational model, the database market was dominated by established players using hierarchical and network databases. IBM's IMS (Information Management System) was the industry leader, deployed in thousands of enterprises worldwide. The CODASYL network model had broad industry backing and an official standard.

Codd's relational model was theoretical—an academic proposal from a mathematician, not a product. Industry veterans dismissed it as impractical. "You can't build a real database on set theory," they said. "The performance will never be acceptable."

They were wrong.

By the 1990s, the relational model had achieved near-total dominance. Oracle, DB2, Sybase, Informix, SQL Server, PostgreSQL—virtually every major database adopted relational principles. The hierarchical and network models, once industry standards, became legacy curiosities.

How did a mathematical abstraction triumph over entrenched, proven technology? The answer reveals fundamental truths about technology adoption, the value of abstraction, and why good ideas—eventually—win.

To appreciate why the relational model won, we must understand what it was competing against.

The Hierarchical Model (IMS Era)

IBM's IMS, launched in 1968, was the dominant database of the 1970s. It organized data in tree structures:

• A root record type
• Child record types branching below
• Relationships fixed at design time
• Navigation through explicit traversal

Strengths:

• Excellent performance for known access patterns
• Efficient storage for hierarchical data
• Well-suited to batch processing workloads
• Mature, stable, deeply integrated in enterprises

Weaknesses:

• Rigid structure—changing relationships required redesign
• Complex programming—developers needed to navigate pointer chains
• Poor ad-hoc queries—new queries might require schema changes
• Redundancy—data repeated across hierarchies

The Network Model (CODASYL)

The CODASYL (Conference on Data Systems Languages) model generalized hierarchies into graphs:

• Records could have multiple parent types
• Many-to-many relationships via owner-member sets
• More flexible than hierarchical
• Still navigation-based

Strengths:

• Could represent complex relationships
• Industry-standard specification
• Supported by multiple vendors
• Better than hierarchical for non-tree structures

Weaknesses:

• Even more complex pointer navigation
• "Spaghetti" data structures
• Steep learning curve
• Schema changes still disruptive

Codd's 1970 paper proposed something radically different: a data model based on mathematical relations, not physical pointers.

The Revolutionary Ideas

1. Data Independence Applications would work with logical tables, ignorant of physical storage. Change how data is stored without changing applications.

2. Declarative Queries Specify WHAT you want, not HOW to get it. The system figures out the access path.

3. Mathematical Foundation Operations defined formally, enabling automatic optimization and correctness proofs.

4. Simplicity Tables are intuitive. Anyone can understand rows and columns. No pointer navigation to learn.

5. Ad-hoc Query Capability Any query expressible in relational algebra/calculus could be run without programming—even queries not anticipated at design time.

The Contrast Was Stark

The hierarchical query required understanding the physical structure, navigating explicitly, and handling iteration. The relational query simply declared what was wanted.

This difference wasn't merely aesthetic. It meant:

• New queries took minutes to write, not days
• Businesspeople could formulate questions without programmers
• Database changes didn't require application rewrites
• Optimization could happen transparently

But there was a problem.

The 1970s saw intense debate about relational viability. Charles Bachman (network model inventor) and Ted Codd engaged in famous debates. The core question: Could relational systems ever match navigational performance?

The Performance Problem

Early relational prototypes (System R at IBM, INGRES at Berkeley) were indeed slower than IMS for comparable workloads. The abstraction penalty seemed real.

The Solution: Query Optimization

The breakthrough came from an unexpected direction. Because relational queries are declarative, the system has freedom in HOW to execute them. Query optimizers could:

1. Analyze the query mathematically
2. Consider multiple execution strategies (different join orders, index choices)
3. Estimate costs based on statistics
4. Choose the optimal plan automatically

A hand-tuned navigational program might be 10% faster than a naive relational execution. But a query optimizer could often find execution plans that no human programmer would consider.

The Pivotal Insight

As optimizer technology matured through the 1980s, something remarkable happened: relational systems became competitive with navigational systems for most workloads, and often faster for ad-hoc queries.

The key insight: abstraction enables optimization.

When programmers specify access paths (navigation), they lock in decisions made with limited information at coding time. When the system chooses access paths (declarative), it can use current statistics, current indexes, and current data distributions.

As databases grew and queries diversified, this advantage compounded. Navigational programs optimized for 1985 data patterns became suboptimal in 1990. Relational queries automatically adapted.

While performance debates raged among technologists, business decision-makers noticed something more important: relational systems made developers dramatically more productive.

Quantifying the Difference

Studies in the 1980s found:

• Ad-hoc queries: 10-100x faster to write in SQL vs. navigational code
• Application development: 2-5x faster with relational vs. hierarchical
• Schema changes: Days or hours vs. weeks or months
• Training time: SQL learnable in days; CODASYL took months

These productivity gains translated directly to business impact.

The Rise of 4GL and End-User Computing

Relational databases enabled a new category: 4th Generation Languages (4GLs) and end-user query tools.

Products like:

• Oracle's SQL*Plus
• Ashton-Tate's dBASE
• Microsoft Access
• Various report writers

These allowed non-programmers to extract data, generate reports, and perform analysis. This was impossible with navigational databases—you couldn't expect businesspeople to learn pointer navigation.

The democratization of data access was revolutionary. Information that required formal IT requests and programmer involvement became directly accessible. This alone justified relational adoption for many organizations.

Beyond technical merits, several industry factors accelerated relational adoption.

1. SQL Standardization (1986-1992)

The ANSI SQL standard (SQL-86, SQL-89, SQL-92) created a common language across vendors. This reduced vendor lock-in fears, enabling:

• Portable applications
• Competitive pricing
• Training investments that transferred between systems
• Third-party tool ecosystems

No such standard existed for hierarchical or network databases (IMS was IBM-specific; CODASYL implementations varied).

2. The Client-Server Revolution

As computing moved from mainframes to client-server architectures in the 1980s-90s:

• Relational databases fit naturally (SQL as the interface)
• Hierarchical systems were mainframe-centric
• Startups built new products on relational foundations
• The installed base advantage of IMS became a liability

3. Hardware Trends

Relational systems benefited from:

• Cheaper memory enabling caching and buffering
• Faster CPUs making optimization overhead trivial
• Larger disks allowing B-tree indexes
• RAID enabling concurrent I/O relational systems could exploit

If relational was so superior, did hierarchical and network databases disappear? Not entirely—but their role fundamentally changed.

IMS: Still Running, Rarely Growing

IBM's IMS still exists and runs critical workloads at major banks, airlines, and insurance companies. Why?

• Unmatched transaction performance for specific, repetitive workloads
• Billions invested in existing applications
• Risk aversion—systems with 40+ years of debugging are extremely stable
• If it ain't broke... mentality for mission-critical systems

But virtually no one builds NEW applications on IMS. It's maintained, not expanded.

CODASYL: Mostly Gone

The network model faded faster:

• No single vendor to champion it
• Complexity made migration more attractive
• Systems were replaced rather than maintained
• Few new practitioners learned it

The Market Reality Today

By revenue and deployment:

• Relational databases: ~75% of the market (Oracle, MySQL, PostgreSQL, SQL Server, DB2)
• NoSQL/Document databases: ~15% (MongoDB, DynamoDB, Cassandra)
• Legacy hierarchical/network: ~5% (IMS, remaining CODASYL)
• Other (graph, time-series, etc.): ~5%

Relational's dominance is so complete that even "NoSQL" systems increasingly adopt relational features (SQL support, transactions, joins). The model Codd proposed has become the default paradigm.

The relational model's triumph offers broader lessons about technology adoption and the value of good abstractions.

1. Abstraction Wins Over Time

Low-level control feels powerful but creates coupling. Declarative approaches seem slower initially but enable optimization and adaptation. As systems grow, abstraction's benefits compound.

2. Developer Productivity Matters More Than Raw Performance

Most applications aren't performance-limited. Development time, maintenance cost, and flexibility determine success. Technologies that optimize for developer experience win markets.

3. Standards Create Ecosystems

SQL's standardization enabled competition, reduced risk, and spawned tool ecosystems. Proprietary advantages are temporary; ecosystem advantages compound.

4. Mathematical Foundations Pay Off

Codd's grounding in set theory wasn't academic decoration—it enabled query optimization, formal constraint checking, and provable correctness. Good theory enables good practice.

5. Network Effects Amplify Adoption

Once relational gained momentum: more developers learned SQL, more tools supported it, more books were written, more problems were solved. This created a self-reinforcing cycle.