Cloud Object Storage: Architecture and Best Practices

Azure Blob Storage, part of Azure Storage accounts, represents Microsoft's approach to object storage. Launched as part of the broader Azure platform in 2010, Blob Storage reflects Microsoft's enterprise heritage while embracing cloud-native patterns. It powers not only external customer workloads but also internal Microsoft services including Microsoft 365, Xbox, and GitHub.

Blob Storage's architecture differs meaningfully from S3 and GCS. Rather than standalone buckets, Azure uses a hierarchical model: Storage Accounts contain Containers, which contain Blobs. This additional layer provides unique flexibility for organization, security, and billing that proves particularly valuable in enterprise environments with complex organizational structures.

Azure's storage hierarchy differs fundamentally from S3 and GCS. Understanding this model is essential for designing Azure storage solutions.

Storage Accounts: The Foundation

A storage account is the top-level resource that provides:

• Unified namespace: A single DNS name (mystorageaccount.blob.core.windows.net)
• Shared configuration: Redundancy options, access tier defaults, encryption settings
• Billing boundary: All storage within an account is billed together
• Access control scope: Shared access signatures and keys apply at this level
• Multiple storage services: A single account can host Blobs, Files, Queues, and Tables

This is conceptually different from S3/GCS where each bucket is largely independent. In Azure, the storage account creates a management and security boundary around related storage resources.

Storage Account Types

Azure offers several storage account types:

1. Standard general-purpose v2: The default, supporting Blobs, Files, Queues, Tables. Uses HDD-backed storage. Most cost-effective for typical workloads.

2. Premium block blobs: SSD-backed storage optimized for block blob workloads requiring low latency and high transaction rates.

3. Premium file shares: SSD-backed Azure Files for high-performance file shares.

4. Premium page blobs: SSD-backed storage for page blobs, primarily used for Azure VM disks.

Containers: The Second Layer

Within a storage account, containers organize blobs:

• Similar to S3 buckets but scoped within a storage account
• Container names must be unique within the storage account (not globally)
• Containers have their own access policies and metadata
• No nesting—containers cannot contain other containers

Azure distinguishes between three blob types, each optimized for different access patterns. This is more explicit than S3/GCS, which treat all objects uniformly.

Block Blobs

The most common blob type, optimized for streaming and cloud storage:

• Maximum size: 190.7 TB
• Composed of blocks, each up to 4GB (formerly 100MB before Azure's enhancement)
• Blocks can be uploaded in parallel, then committed together
• Ideal for: documents, images, videos, backups, logs—most object storage use cases

Block blobs support staged uploads similar to S3 multipart:

1. Put Block (upload individual blocks with IDs)
2. Put Block List (commit blocks in desired order)

This enables:

• Parallel uploads for large files
• Resume capability for failed uploads
• Block-level modification (uncommit and replace individual blocks)

Append Blobs

Optimized for append-only scenarios:

• Maximum size: 195 GB (composed of up to 50,000 append blocks)
• Only supports appending data; no modification of existing data
• Optimized for high-frequency append operations
• Ideal for: logging, streaming data written once and read sequentially

Append blobs eliminate the need to manage staging files for log aggregation. Multiple writers can append concurrently (with appropriate concurrency controls).

Page Blobs

Optimized for random read/write operations:

• Maximum size: 8 TB
• Data organized in 512-byte pages
• Supports random read/write by page offset
• Ideal for: virtual machine disks, database files, any workload requiring random access
• Azure VM boot disks and data disks are page blobs

Page blobs are conceptually different from object storage—they're more like block device storage exposed through the blob API.

Azure provides more granular redundancy options than S3 or GCS, allowing precise tradeoffs between durability, availability, and cost.

Locally Redundant Storage (LRS)

The simplest option:

• 3 synchronous copies within a single data center
• 11 nines of durability (99.999999999%)
• Protects against: disk failures, rack failures
• Does not protect against: data center disasters
• Lowest cost
• Use when: data can be reconstructed or is not critical

Zone-Redundant Storage (ZRS)

Data spread across availability zones:

• 3 synchronous copies across 3 availability zones in the same region
• Writes acknowledged only after all 3 zones confirm
• 12 nines of durability (99.9999999999%)
• Protects against: single zone failures, data center disasters within region
• Higher availability than LRS
• Use when: high availability within a region is priority

Geo-Redundant Storage (GRS)

Cross-region replication:

• 3 synchronous copies in primary region (LRS)
• 3 asynchronous copies in paired secondary region
• Total of 6 copies
• 16 nines of durability (99.99999999999999%)
• Protects against: regional disasters, natural calamities
• Secondary is read-only during normal operation
• Failover (manual or automatic) promotes secondary to primary

Geo-Zone-Redundant Storage (GZRS)

The most resilient option:

• 3 synchronous copies across availability zones in primary region (ZRS)
• 3 asynchronous copies in paired secondary region (LRS)
• Combines zone-level and regional protection
• Highest availability and durability
• Use for: mission-critical data requiring maximum resilience

Read-Access Variants (RA-GRS, RA-GZRS)

Standard GRS/GZRS only allow reading from the secondary during a failover. The Read-Access variants (RA-GRS, RA-GZRS) allow reading from the secondary region at any time:

• Secondary endpoint: mystorageaccount-secondary.blob.core.windows.net
• Data may be slightly stale (asynchronous replication)
• Useful for: read-heavy workloads that can distribute reads globally, DR testing

Comparison with S3/GCS

• S3 Standard: Similar to ZRS (3 AZ synchronous replication)
• S3 One Zone-IA: Similar to LRS (single zone)
• S3 Cross-Region Replication: User-configured, not a storage class; Azure's GRS is built-in
• GCS Multi-Region: Similar concept to GRS but across multiple active regions

Azure Blob Storage offers access tiers that control storage cost and access pricing. These can be set at the blob level (unlike S3 where it's object-level storage class) or as a default for the storage account.

Hot Tier

Optimized for data accessed frequently:

• Highest storage cost per GB
• Lowest access cost per operation
• No retrieval delays
• No minimum storage duration
• Use for: active data, applications, frequently accessed content

Cool Tier

For infrequently accessed data:

• ~50% lower storage cost than Hot
• Higher access cost per operation
• No retrieval delays
• 30-day minimum storage duration (early deletion incurs charges)
• Use for: short-term backup, disaster recovery source data

Cold Tier (Introduced 2023)

For rarely accessed data:

• Lower storage cost than Cool (~45% less)
• Higher access and transaction costs
• 90-day minimum storage duration
• Use for: compliance data, long-term backups accessed occasionally

Archive Tier

For long-term archival:

• Lowest storage cost (~90% less than Hot)
• Highest retrieval cost
• Offline storage: blobs must be rehydrated before access
• Rehydration times: High priority (< 1 hour), Standard (up to 15 hours)
• 180-day minimum storage duration
• Use for: regulatory archives, historical data rarely if ever accessed

Key Difference from GCS

Unlike GCS where Archive has identical read performance to Standard, Azure Archive is offline storage. You cannot read an archived blob directly—it must be rehydrated to Hot or Cool first. This is similar to S3 Glacier.

Lifecycle Management

Azure supports blob lifecycle management policies to automatically tier data:

{
  "rules": [
    {
      "name": "archiveOldData",
      "enabled": true,
      "type": "Lifecycle",
      "definition": {
        "filters": {
          "blobTypes": ["blockBlob"],
          "prefixMatch": ["logs/"]
        },
        "actions": {
          "baseBlob": {
            "tierToCool": {"daysAfterModificationGreaterThan": 30},
            "tierToCold": {"daysAfterModificationGreaterThan": 90},
            "tierToArchive": {"daysAfterModificationGreaterThan": 180},
            "delete": {"daysAfterModificationGreaterThan": 365}
          }
        }
      }
    }
  ]
}

Lifecycle policies support transitions based on modification date, access date (with access tracking enabled), version age, and more.

Azure Data Lake Storage Gen2 (ADLS Gen2) is a unique capability that transforms Blob Storage for big data analytics. It's implemented as an optional feature called Hierarchical Namespace (HNS) that can be enabled on storage accounts.

What Is Hierarchical Namespace?

Standard blob storage (like S3 and GCS) has a flat namespace—directories are an illusion created by key prefixes. Operations like 'rename folder' actually require copying and deleting every object in that prefix.

With HNS enabled, Azure Blob Storage gains a true hierarchical file system:

• Directories are real objects, not prefix conventions
• Rename operations are atomic and O(1) regardless of directory size
• ACLs can be set on directories and inherited by children
• POSIX-like permissions (rwx) for users and groups

Why This Matters for Analytics

ADLS Gen2 API and Access

ADLS Gen2 supports multiple access patterns:

1. Blob API: Standard Blob Storage operations work unchanged
2. Data Lake Storage API: File system operations (create, delete, rename, ACL management)
3. ABFS driver: Hadoop-compatible file system driver (abfs://container@account.dfs.core.windows.net/path)
4. NFS 3.0: Mount as a network file system (preview feature)

This dual-API approach lets you use blob storage patterns for applications while using file system patterns for analytics—on the same data.

When to Enable HNS

Azure Blob Storage provides multiple layers of access control, reflecting enterprise security requirements.

Storage Account Keys

Every storage account has two access keys that provide full administrative access:

• Primary and secondary keys for rotation without downtime
• Keys provide complete access to all data and operations
• Regenerating a key invalidates existing tokens using that key
• Generally avoid using account keys in applications; use more scoped credentials

Shared Access Signatures (SAS)

SAS tokens provide limited, time-bound access:

• Service SAS: Access to specific blob service resources
• Account SAS: Access across all storage services in an account
• User delegation SAS: Uses Azure AD credentials, no storage key exposure

SAS tokens can specify:

• Allowed operations (read, write, delete, list)
• Allowed IPs
• Valid time window
• Required protocols (HTTPS only)

Azure AD Authentication (RBAC)

Azure AD integration provides identity-based access:

• Assign built-in roles: Storage Blob Data Reader, Storage Blob Data Contributor, Storage Blob Data Owner
• Use managed identities for Azure resources (no credential management)
• Fine-grained permissions through custom roles
• Integration with Azure AD Conditional Access

Azure AD is the recommended access pattern for applications running in Azure.

ACLs for ADLS Gen2

With Hierarchical Namespace enabled, POSIX-style ACLs provide granular control:

• Owner, owning group, and other permissions (rwx)
• Named user and group entries
• Default ACLs inherited by new files in a directory
• Access ACLs controlling current permissions

Container-Level Access

Containers can be configured for public access (not recommended) or private:

• Private: All access requires authentication (default, recommended)
• Blob: Anonymous read access to blobs only
• Container: Anonymous read access to container listing and blobs

Public access can be completely disabled at the storage account level, preventing accidental exposure.

Let's synthesize the architectural differences across all three major cloud object storage services.

Data Model Comparison

Redundancy Comparison

When to Choose Each Provider

Choose AWS S3 when:

• Your infrastructure is primarily AWS
• You need the deepest integration with AWS services (Lambda, EMR, Athena, Redshift)
• Maximum ecosystem compatibility (most tools assume S3 API)
• Complex storage class optimization with many tiers
• Deep Glacier archival for lowest cost long-term storage

Choose Google Cloud Storage when:

• Your infrastructure is primarily GCP
• You need instant archive access (no retrieval delays)
• BigQuery is central to your analytics strategy
• You prefer simpler storage class model
• Composite objects are valuable for your workload

Choose Azure Blob Storage when:

• Your infrastructure is primarily Azure
• Enterprise features like AAD integration are essential
• You need hierarchical namespace for big data (ADLS Gen2)
• Append blobs fit your logging/streaming patterns
• You need built-in geo-redundancy (GRS/GZRS) with clear failover semantics

Let's consolidate the key insights about Azure Blob Storage:

Architectural Patterns for System Design

When designing systems with Azure Blob Storage:

• Use the three-tier hierarchy to organize by environment, project, or access pattern
• Choose blob type deliberately — append blobs for logs, page blobs for random access, block for everything else
• Enable HNS for analytics workloads but understand it's a permanent decision
• Design redundancy based on RPO/RTO — GRS for disaster recovery, ZRS for high availability
• Use lifecycle policies to automate tiering; enable access tracking for intelligent tiering

What's Next:

The next page explores storage classes and tiering in depth across all providers, examining the cost optimization strategies that can dramatically reduce storage spend at scale.