Hierarchical Model - Learning Module

Loading content...

0/241

IMS Example — The Industry-Defining Hierarchical DBMS

When Theory Became Industrial Reality

No discussion of hierarchical databases is complete without IBM's Information Management System (IMS)—the system that transformed hierarchical data modeling from computer science theory into industrial practice. Launched in 1968, IMS became the world's first widely successful database management system and established patterns that influenced database design for the next five decades.

IMS wasn't created in an academic laboratory or as a general-purpose research project. It was born from one of the most ambitious engineering endeavors in human history: the Apollo moon landing program. The need to track millions of parts, thousands of suppliers, and complex manufacturing processes with absolute reliability drove IBM and Rockwell (North American Aviation) to create a data management system unlike any before.

Today, more than 50 years later, IMS continues to process an astonishing volume of the world's transactions. Major banks, airlines, insurance companies, and governments run mission-critical workloads on IMS. Understanding IMS isn't merely historical appreciation—it's insight into a system architecture that processes trillions of dollars daily.

What You Will Master

By the end of this page, you will understand IMS's historical context, architectural principles, data organization mechanisms, programming model, and lasting contributions to database technology. You'll be equipped to appreciate why IMS solved problems that contemporary systems couldn't, and why its design principles remain relevant for understanding modern hierarchical data systems.

Historical Context: From Apollo to Enterprise Computing

The Apollo Challenge

In 1966, the United States was racing to land humans on the moon before the end of the decade. The Saturn V rocket—the launch vehicle for the Apollo missions—consisted of approximately 3 million parts, sourced from over 20,000 suppliers, manufactured across hundreds of facilities. Every bolt, wire, weld, and circuit had to be tracked, traced, and verified.

The existing data management approaches were completely inadequate:

File-based systems couldn't handle the relationship complexity. A single part might be used in dozens of assemblies, each assembly part of larger systems, each system interconnected with others.

Manual processes couldn't scale. Paper-based tracking was error-prone and couldn't provide the real-time status updates needed for millions of components in simultaneous production.

Existing software wasn't designed for hierarchical bill-of-materials (BOM) structures where assemblies contain sub-assemblies contain parts in deep nesting.

IBM, partnering with North American Aviation (the Saturn V's primary contractor), developed ICS (Information Control System) to address these challenges. ICS evolved into IMS, released commercially in 1968.

IMS Historical Timeline
Year	Version/Event	Significance
1966	ICS Development Begins	Apollo program drives initial development at IBM/Rockwell
1968	IMS V1 Released	First commercial hierarchical DBMS available
1969	Apollo 11 Landing	IMS-tracked systems help put humans on the moon
1970s	DL/I Standardization	Programmatic interface becomes industry standard
1974	IMS/VS Introduced	Virtual storage support dramatically increases capacity
1980s	IMS DB/DC	Combined database and transaction management
1990s	IMS Open Access	Java, JDBC, and web service interfaces added
2000s	IMS Enterprise Suite	Integration with modern middleware and tools
2010s	IMS 14+	JSON support, enhanced performance, cloud readiness
Present	IMS 15.x	Active development, processing trillions in transactions

Why Bills of Materials Matter

The bill-of-materials (BOM) problem—where complex products are assembled from hierarchical component structures—was the perfect use case for hierarchical databases. A Saturn V had assemblies containing sub-assemblies containing sub-sub-assemblies, naturally forming deep trees. This structure remains common in manufacturing, and IMS remains a leading BOM management platform.

IMS Architecture: A System Built for Reliability

IMS architecture reflects its mission-critical origins. Every design decision prioritizes reliability, performance, and recoverability—the qualities essential for systems where failures could halt rocket production or crash banking operations.

Converting Mermaid diagram...

Core IMS Components

•IMS Control Region — The master control component that manages IMS execution, schedules application programs, and coordinates resources. Runs as a single address space controlling the entire IMS subsystem.
•DL/I (Data Language/I) — The programmatic interface for database operations. Applications execute DL/I calls to retrieve, insert, update, and delete segments. DL/I abstracts physical storage details from applications.
•Buffer Manager — Caches database segments in memory to minimize I/O. Uses sophisticated algorithms to predict access patterns based on hierarchical navigation.
•Log Manager — Maintains before-images and after-images of all database changes for recovery. Enables point-in-time recovery and transaction rollback.
•Lock Manager — Coordinates concurrent access through hierarchical locking protocols. Can lock at segment, hierarchical path, or database level.

Dual Operational Modes

IMS operates in two primary modes:

IMS DB (Database Manager): Provides database management services without transaction management. Applications connect via DL/I calls, typically in batch processing mode.

IMS TM (Transaction Manager): Provides complete online transaction processing (OLTP) with message queuing, scheduling, and database access. Supports high-volume, concurrent transaction workloads.

IMS DB/DC (Combined): Most production environments run both components together, providing integrated database and transaction management with shared recovery and synchronization.

The ACID Foundation

IMS was providing ACID (Atomicity, Consistency, Isolation, Durability) transaction guarantees years before the term was formalized. Its logging, locking, and recovery mechanisms set the standard that relational databases would later adopt. When you use transactions in PostgreSQL or Oracle today, you're benefiting from concepts pioneered in IMS.

IMS Data Organization: Segments, Hierarchies, and Databases

IMS organizes data through a hierarchy of concepts: segments (the atomic data unit), segment types (the schema definition), database records (complete hierachical trees), and databases (collections of records).

IMS Data Concepts

•Segment — The basic unit of data access and storage in IMS. A segment contains one or more fields and corresponds to what other databases might call a 'record' or 'row.' Segments are accessed, stored, and locked as atomic units.
•Segment Type — The schema definition for a segment, specifying its fields, length, and position in the hierarchy. Analogous to a table definition in relational terms.
•Database Record — A complete hierarchical tree consisting of a root segment and all its descendant segments. One database record = one complete hierarchy instance.
•Database — A collection of database records of the same structure (same hierarchy schema). The complete physical and logical organization of related data.
•Physical Database (Phys DB) — The actual storage representation of data on disk, defined by a DBD (Database Description).
•Logical Database — A view over one or more physical databases, potentially combining segments from multiple sources through logical relationships.

Segment Types and the DBD

The Database Description (DBD) defines the hierarchical structure of an IMS database. It specifies:

Each segment type (name, size, fields)
Parent-child relationships between segment types
Physical storage organization
Access method (HDAM, HIDAM, HISAM, etc.)
Indexing and search fields

Sample Corporate DBD

IMS DBD

        DBD   NAME=CORPDB,ACCESS=HIDAM
**********************************************
* ROOT SEGMENT: COMPANY
**********************************************
        SEGM  NAME=COMPANY,BYTES=150,PARENT=0,
              POINTER=TWIN
        FIELD NAME=COMPCODE,BYTES=10,START=1,TYPE=C
        FIELD NAME=COMPNAME,BYTES=50,START=11,TYPE=C
        FIELD NAME=FOUNDED,BYTES=4,START=61,TYPE=P
        LCHILD NAME=(CORPINDX,CORPDB),POINTER=INDX
**********************************************
* LEVEL 1: DIVISION (child of COMPANY)
**********************************************
        SEGM  NAME=DIVISION,BYTES=100,PARENT=COMPANY,
              POINTER=(TWIN,TWINBWD)
        FIELD NAME=DIVCODE,BYTES=8,START=1,TYPE=C
        FIELD NAME=DIVNAME,BYTES=40,START=9,TYPE=C
        FIELD NAME=REGION,BYTES=20,START=49,TYPE=C
**********************************************
* LEVEL 2: DEPARTMENT (child of DIVISION)
**********************************************
        SEGM  NAME=DEPARTMT,BYTES=120,PARENT=DIVISION
        FIELD NAME=DEPTCODE,BYTES=10,START=1,TYPE=C
        FIELD NAME=DEPTNAME,BYTES=50,START=11,TYPE=C
        FIELD NAME=BUDGET,BYTES=8,START=61,TYPE=P
        FIELD NAME=MGR_ID,BYTES=10,START=69,TYPE=C
**********************************************
* LEVEL 3: EMPLOYEE (child of DEPARTMENT)
**********************************************
        SEGM  NAME=EMPLOYEE,BYTES=200,PARENT=DEPARTMT
        FIELD NAME=EMPNO,BYTES=10,START=1,TYPE=C,SEQ
        FIELD NAME=EMPNAME,BYTES=50,START=11,TYPE=C
        FIELD NAME=HIREDATE,BYTES=4,START=61,TYPE=P
        FIELD NAME=SALARY,BYTES=8,START=65,TYPE=P
        FIELD NAME=JOBCODE,BYTES=6,START=73,TYPE=C
**********************************************
* LEVEL 4: SKILL (child of EMPLOYEE)
**********************************************
        SEGM  NAME=SKILL,BYTES=50,PARENT=EMPLOYEE
        FIELD NAME=SKILLCD,BYTES=10,START=1,TYPE=C
        FIELD NAME=SKILLNM,BYTES=30,START=11,TYPE=C
        FIELD NAME=PROFLEVL,BYTES=1,START=41,TYPE=C
*  
        DBDGEN
        FINISH
        END

Understanding the DBD

The DBD defines a 5-level hierarchy: COMPANY (root) → DIVISION → DEPARTMT → EMPLOYEE → SKILL. Each SEGM statement defines a segment type with its parent explicitly named. The HIDAM access method provides both hierarchical access and index-based direct access to root segments. POINTER options define navigation structures between segments.

IMS Access Methods: Organizing Physical Storage

IMS provides multiple access methods—strategies for physically organizing data on disk. The choice of access method profoundly affects performance for different workload patterns.

IMS Primary Access Methods
Access Method	Full Name	Organization	Best For
HSAM	Hierarchical Sequential	Sequential file	Batch processing, data loads, backups
HISAM	Hierarchical Indexed Sequential	Indexed + overflow	Mixed sequential/random access
HDAM	Hierarchical Direct	Hash-based root access	High-volume random access
HIDAM	Hierarchical Indexed Direct	Index + direct	Random access with ordered traversal
PHDAM	Partitioned HDAM	Partitioned hash	Very large databases, parallel processing
PHIDAM	Partitioned HIDAM	Partitioned index	Large databases needing ordered access

HDAM (Hierarchical Direct Access Method) uses hashing to provide O(1) direct access to root segments. It's the most common access method for high-transaction-volume databases.

Structure:

Root Address Area: Hash buckets containing root segments or pointers
Overflow Area: Segments that don't fit in their hash bucket
Hierarchical pointers connect parents to children

Randomizing Module:

User-provides or uses IBM-supplied hashing algorithm
Key value → physical address of root segment
Good hash distribution critical for performance

Characteristics:

O(1) root segment access (with good hashing)
No key sequence maintained (hash-based order)
Excellent for random access workloads
Efficient insert/delete operations

Use Cases:

OLTP applications with random access patterns
Customer lookup by account number
Inventory queries by part number
Any high-volume random access workload

Trade-off: HDAM sacrifices sequential ordering for random access speed. Sequential scans require visiting all hash buckets.

DL/I: The Programming Interface

DL/I (Data Language/I) is the application programming interface for IMS databases. Unlike SQL's declarative approach, DL/I is navigational—programs explicitly navigate the hierarchical structure using positioning commands.

DL/I calls follow a consistent pattern: CALL CBLTDLI USING function-code, PCB, I/O-area, SSA...

Essential DL/I Call Functions
Function Code	Operation	Description
GU	Get Unique	Direct access to a specific segment using full key path
GN	Get Next	Get next segment in hierarchical sequence (preorder)
GNP	Get Next within Parent	Get next segment under current parent only
GHU/GHN/GHNP	Get Hold variants	Same as above but locks segment for update
ISRT	Insert	Add a new segment occurrence
DLET	Delete	Remove a segment (and all descendants)
REPL	Replace	Update the current segment's content
CHKP	Checkpoint	Establish a recovery point in the application

Segment Search Arguments (SSAs)

SSAs specify which segments to access. They can be unqualified (specify segment type only) or qualified (include selection criteria).

Unqualified SSA: EMPLOYEE — matches any EMPLOYEE segment

Qualified SSA: EMPLOYEE(EMPNO = '12345') — matches EMPLOYEE with specific key

Multiple SSAs establish the hierarchical path from root to target:

DL/I Call Examples (COBOL)

COBOL

      * Get Unique: Direct access to specific employee
       MOVE 'ACME    ' TO COMP-KEY.
       MOVE 'NORTH   ' TO DIV-KEY.
       MOVE 'ENG     ' TO DEPT-KEY.
       MOVE '12345   ' TO EMP-KEY.
       
       CALL 'CBLTDLI' USING GU-FUNC
                            PCB-MASK
                            EMPLOYEE-IO-AREA
                            COMPANY-SSA
                            DIVISION-SSA
                            DEPARTMT-SSA
                            EMPLOYEE-SSA.
       
       IF PCB-STATUS-CODE = SPACES
           DISPLAY 'Found: ' EMPLOYEE-NAME
       ELSE IF PCB-STATUS-CODE = 'GE'
           DISPLAY 'Employee not found'
       END-IF.
       
      * Get Next: Sequential traversal 
       CALL 'CBLTDLI' USING GN-FUNC
                            PCB-MASK
                            SEGMENT-IO-AREA.
       
      * Get Next within Parent: All employees in current dept
       CALL 'CBLTDLI' USING GNP-FUNC
                            PCB-MASK
                            EMPLOYEE-IO-AREA
                            EMPLOYEE-SSA-UNQUAL.
       
      * Insert: Add new skill for current employee
       MOVE 'PYTHON    ' TO SKILL-CODE.
       MOVE 'Python Programming' TO SKILL-NAME.
       MOVE '3' TO PROFICIENCY-LEVEL.
       
       CALL 'CBLTDLI' USING ISRT-FUNC
                            PCB-MASK
                            SKILL-IO-AREA
                            SKILL-SSA.
       
      * Update: Modify current segment (after GHU/GHN)
       ADD 5000 TO SALARY.
       
       CALL 'CBLTDLI' USING REPL-FUNC
                            PCB-MASK
                            EMPLOYEE-IO-AREA.
       
      * Delete: Remove current segment and all children
       CALL 'CBLTDLI' USING DLET-FUNC
                            PCB-MASK
                            EMPLOYEE-IO-AREA.

The Navigational Paradigm

DL/I's navigational approach differs fundamentally from SQL's declarative model. The programmer controls exactly how the database is traversed—which segments to visit, in what order, with what positioning. This gives fine-grained control but requires understanding the physical structure. The application 'knows' the schema, unlike SQL where the optimizer handles navigation.

Program Communication Blocks: The Application-Database Interface

The Program Communication Block (PCB) is the data structure that maintains the state of an application's view of the database. Each PCB represents one logical view (PSB) of a database and tracks the application's current position in the hierarchy.

PCB Components

•Database Name (DBD Name) — Identifies which database this PCB accesses. Links to the DBD definition.
•Segment Level Number — Indicates the hierarchical level of the last segment accessed (1 = root, 2 = first child level, etc.).
•Status Code — Two-character code indicating the result of the last DL/I call. Spaces = success, 'GE' = not found, 'GB' = end of database, etc.
•Processing Options — Defines what operations the application can perform (Get, Insert, Replace, Delete, All).
•Key Feedback Area — Contains the concatenated keys of all segments from root to current position, enabling the application to know exactly where it is in the hierarchy.
•Current Position — Internal pointers maintaining the application's position for Get Next operations.

Program Specification Block (PSB)

The PSB is the complete description of an application's database access requirements. It contains one or more PCBs, each defining access to a database or view.

PSB defines:

Which databases the program can access
Which segment types within each database are accessible (sensitivity)
What operations are permitted on each segment type
The sequence in which databases are presented to the program

Sample PSB Definition

PSB

        PCB   TYPE=DB,DBDNAME=CORPDB,PROCOPT=A,KEYLEN=78
        SENSEG NAME=COMPANY,PARENT=0
        SENSEG NAME=DIVISION,PARENT=COMPANY
        SENSEG NAME=DEPARTMT,PARENT=DIVISION
        SENSEG NAME=EMPLOYEE,PARENT=DEPARTMT,PROCOPT=G
        SENSEG NAME=SKILL,PARENT=EMPLOYEE,PROCOPT=GI
*
* PROCOPT=A: All operations (Get, Insert, Replace, Delete)
* PROCOPT=G: Get only (read-only access)  
* PROCOPT=GI: Get and Insert only
*
        PSBGEN LANG=COBOL,PSBNAME=EMPMAINT
        FINISH
        END

Sensitivity: IMS Security Model

SENSEG statements control what an application can 'see.' An application with sensitivity only to COMPANY and DIVISION cannot access EMPLOYEE data—DL/I calls referencing EMPLOYEE will fail. This built-in security model was innovative for its time and provides application-level data access control without changing the underlying database.

IMS Transaction Processing: Industrial-Strength OLTP

IMS TM (Transaction Manager) provides OLTP capabilities that process millions of transactions daily for the world's largest enterprises. Its design principles influence transaction processing systems to this day.

IMS Transaction Processing Capabilities
Capability	Description	Typical Scale
Transaction Rate	Sustained transaction throughput	50,000+ TPS per IMS system
Response Time	End-to-end transaction latency	Sub-second for most operations
Concurrent Users	Simultaneous active transactions	Thousands per system
Data Integrity	ACID compliance	Full commit/rollback support
Availability	System uptime	99.999% (5 nines) achievable
Recovery	Point-in-time restore capability	Transaction-level granularity

Message Processing

IMS TM uses message queues to decouple transaction input from processing:

Input Message Queue: Transactions enter from terminals/networks
Scheduling: IMS scheduler assigns messages to program instances
Processing: Application program processes message, accesses database
Output Message Queue: Response queued for delivery
Delivery: Response returned to originator

This queue-based architecture provides:

Load leveling: Process transactions at system's pace
Priority handling: High-priority transactions processed first
Recovery: Unprocessed messages survive system restarts
Scalability: Multiple program instances process in parallel

Commit Processing

IMS uses sync points to ensure transaction integrity:

Application requests commit (CHKP or end of message)
IMS writes log records for all changes
Log force to disk ensures durability
Locks released after successful write
Response sent only after commit completes

If failure occurs before commit, IMS automatically backs out all changes. This two-phase approach ensures no partial transactions persist.

Born for Mission-Critical

IMS's transaction processing architecture was designed for environments where failure isn't an option—banking systems where incorrect balances have legal consequences, airline reservations where double-bookings cause real problems, manufacturing systems where parts tracking affects safety. This mission-critical DNA persists in IMS's continued use for the world's most demanding workloads.

IMS's Lasting Impact on Database Technology

IMS's influence extends far beyond its continued production use. Concepts pioneered by IMS became foundational to all subsequent database systems.

IMS Innovations Adopted Industry-Wide

•Write-ahead logging for crash recovery
•Hierarchical locking protocols
•Application/database separation via API
•Transaction checkpoint/restart
•Buffer pool management
•Logical data views (schemas)
•Message queuing for decoupled processing

Modern Systems Echoing IMS

•MongoDB (hierarchical document model)
•XML databases (tree-structured data)
•JSON storage in SQL databases
•LDAP directories (hierarchical naming)
•File systems (folder hierarchies)
•Object databases (object graphs)
•React/Vue (component trees)

Why IMS Persists

Despite being over 50 years old, IMS continues to thrive:

Performance: For workloads it's designed for—high-volume, hierarchically-structured, OLTP—IMS remains extraordinarily fast. Decades of optimization have made it highly efficient on modern hardware.

Reliability: Organizations trust IMS for mission-critical workloads because it has proven itself over decades of production use. The risk of migrating away often outweighs the benefits.

Investment Protection: Massive codebases in COBOL and PL/I access IMS. Rewriting these applications would cost billions industrywide.

continued Enhancement: IBM continues active IMS development, adding JSON support, REST APIs, Java integration, and cloud deployment options. IMS evolves with technology rather than being abandoned.

Specialized Excellence: For hierarchical Bill-of-Materials, parts tracking, and similar workloads, IMS remains arguably the best tool. Generic relational solutions often can't match its specialized performance.

From Moon Landings to Modern Enterprise

IMS's journey from tracking Saturn V rocket components to processing trillions of dollars in modern financial transactions demonstrates the enduring power of well-designed hierarchical data structures. Understanding IMS isn't just historical appreciation—it's insight into architecting systems that must work reliably at scale, decade after decade.

Looking Ahead

With IMS as our concrete example of hierarchical database implementation, we're prepared to objectively evaluate the hierarchical model. The next page examines the advantages and limitations of hierarchical databases—what they do brilliantly, where they struggle, and why understanding both is essential for informed database technology decisions.