Special File Systems - Learning Module

Loading content...

0/227

devtmpfs - Dynamic Device File System

Where Do Device Files Come From?

When you plug in a USB drive, a new /dev/sdb file appears. When you connect a Bluetooth mouse, /dev/input/mouse0 materializes. When a Docker container starts, its /dev directory contains exactly the devices it needs. But how do these device files get created? Who decides their names, permissions, and major/minor numbers?

In the early Unix era, device files were created manually by administrators running mknod. A static /dev directory contained thousands of device files—most representing hardware that didn't exist on that particular system. This was wasteful, confusing, and made hotplug impossible.

devtmpfs solved this by creating a tmpfs instance where the kernel itself creates and removes device nodes dynamically as devices are detected. Combined with udev (which sets permissions and creates symlinks), devtmpfs provides a dynamic, minimal /dev that reflects only the actual hardware present.

What You Will Learn

By the end of this page, you will understand how devtmpfs works, its relationship with udev, the device number system (major/minor), and how modern Linux systems boot with a minimal /dev that grows dynamically. You'll see how this enables hotplug, containers, and embedded systems.

Historical Context: From Static to Dynamic /dev

The evolution of /dev management reflects Linux's journey from static configuration to dynamic, hotplug-aware systems. Understanding this history illuminates why devtmpfs exists and how it fits into the broader device management ecosystem.

Evolution of Linux Device Management
Era	Approach	Mechanism	Limitations
1970s-1990s	Static /dev	`mknod` commands, persistent on disk	Thousands of unused nodes, manual management
Early 2000s	devfs	Kernel creates nodes; first attempt at dynamic	Naming policy in kernel (bad design), removed in 2.6
2004-2009	udev on tmpfs	udev creates nodes in response to kernel events	Race conditions at early boot; slow for many devices
2009+	devtmpfs + udev	Kernel creates nodes; udev adjusts permissions	Best of both: kernel speed + userspace flexibility

The devfs mistake:

Linux 2.4 introduced devfs, which had the kernel create device nodes directly. However, this embedded naming policy into the kernel—a violation of Unix philosophy. Device names like disc0 instead of sda caused compatibility issues, and the code was unmaintainable.

The udev-only era:

dev 2.6 removed devfs and introduced udev—a userspace daemon that receives kernel device events and creates nodes accordingly. This was conceptually clean but had practical problems:

Early boot required a minimal static /dev (initramfs)
Race conditions between device detection and udev processing
Slow enumeration when many devices were present

The devtmpfs solution:

devtmpfs (introduced 2009, Linux 2.6.32) combines kernel efficiency with userspace flexibility:

Kernel creates nodes — Fast, no races, available immediately
Uses simple naming — Kernel uses default names (sda, tty0)
udev adjusts — Userspace sets permissions, creates symlinks, runs scripts

Separation of Concerns

devtmpfs demonstrates good system design: the kernel handles what it must (creating nodes when devices appear) while userspace handles policy (naming, permissions, symlinks). Neither duplicates the other's work.

Device Nodes Explained

Before diving into devtmpfs mechanics, we must understand what device nodes actually are—the fundamental interface between userspace and kernel device drivers.

A device node is a special file that provides a file-like interface to a device driver. When you read from or write to a device node, the kernel routes those operations to the appropriate driver. Device nodes are characterized by:

Device Node Characteristics

•Type — Character device (c) or block device (b)
•Major number — Identifies the driver handling this device
•Minor number — Identifies the specific device handled by that driver
•Permissions — Standard Unix permissions (owner, group, mode)
•Path — Location in filesystem (conventionally in /dev)

device_node_examples
Bash
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
# Viewing device nodes with their major:minor numbers
$ ls -la /dev/sda /dev/tty0 /dev/null /dev/random
brw-rw---- 1 root disk    8,   0 Jan 16 10:00 /dev/sda
crw--w---- 1 root tty     4,   0 Jan 16 10:00 /dev/tty0  
crw-rw-rw- 1 root root    1,   3 Jan 16 10:00 /dev/null
crw-rw-rw- 1 root root    1,   8 Jan 16 10:00 /dev/random
 
# Breaking down the output:
# 'b' = block device, 'c' = character device
# 8, 0 = major 8, minor 0 (SCSI disk driver, first disk)
# 4, 0 = major 4, minor 0 (TTY driver, console 0)
# 1, 3 = major 1, minor 3 (memory devices, null device)
# 1, 8 = major 1, minor 8 (memory devices, random)
 
# The kernel maintains a registry of major numbers
$ cat /proc/devices
Character devices:
  1 mem
  4 /dev/vc/0
  4 tty
  5 /dev/tty
  ...
  
Block devices:
  8 sd
  9 md
 11 sr
 ...
 
# Creating a device node manually (rarely needed today)
$ sudo mknod /dev/test_device c 100 0
$ ls -la /dev/test_device
crw-r--r-- 1 root root 100, 0 Jan 16 10:00 /dev/test_device
 
# What happens when you access a device?
$ echo "hello" > /dev/null   # Write routed to null driver
$ cat /dev/random | head -c 16 | xxd  # Read routed to random driver

Block vs Character devices:

Block devices — Data accessed in fixed-size blocks (sectors); support seeking, buffering, caching. Examples: disks (sd*), optical drives (sr*)
Character devices — Data accessed as a stream of bytes; no inherent structure. Examples: terminals (tty*), serial ports, mice, keyboards

Major/Minor number system:

Major numbers are assigned to drivers (often statically defined), while minor numbers distinguish instances. For example:

Major 8 = SCSI disk driver
Minor 0 = /dev/sda (first SCSI disk)
Minor 1 = /dev/sda1 (first partition of sda)
Minor 16 = /dev/sdb (second SCSI disk, minors 0-15 for sda partitions)

How devtmpfs Works

devtmpfs is a specialized tmpfs instance managed by the kernel. When kernel code calls device_add() to register a device, devtmpfs automatically creates the corresponding device node. When device_del() is called, the node is removed.

The devtmpfs lifecycle:

Converting Mermaid diagram...

devtmpfs_implementation
C (Kernel)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
/*
 * Simplified view of devtmpfs device node creation
 * Actual code in drivers/base/devtmpfs.c
 */
 
/* Called when device_add() is invoked */
int devtmpfs_create_node(struct device *dev)
{
    const char *nodename;
    umode_t mode = 0;
    kuid_t uid = GLOBAL_ROOT_UID;
    kgid_t gid = GLOBAL_ROOT_GID;
    int err;
 
    /* Get the device name (e.g., "sda", "tty0") */
    nodename = device_get_devnode(dev, &mode, &uid, &gid, &tmp);
    if (!nodename)
        return -ENOMEM;
 
    /* Determine the node type and create it */
    if (S_ISBLK(mode))
        err = vfs_mknod(/* path, mode, dev_t */);
    else
        err = vfs_create(/* path, mode */);
 
    /* Set ownership */
    if (!err)
        err = vfs_chown(/* path, uid, gid */);
 
    return err;
}
 
/* Called when device_del() is invoked */
int devtmpfs_delete_node(struct device *dev)
{
    const char *nodename;
    
    nodename = device_get_devnode(dev, NULL, NULL, NULL, NULL);
    if (!nodename)
        return -ENOMEM;
 
    /* Simply unlink the file */
    return vfs_unlink(/* path */);
}

Key implementation details:

Kernel thread — devtmpfs runs a kernel thread (kdevtmpfs) that handles node creation/deletion
Synchronous at boot — During early boot, operations are synchronous to ensure devices are ready
Asynchronous later — Once userspace starts, operations queue for the kernel thread
Default permissions — Kernel sets basic permissions (often 0600 root:root); udev adjusts
Simple naming — Kernel uses default names from drivers; udev creates additional symlinks

Kernel Configuration

devtmpfs is enabled via CONFIG_DEVTMPFS=y. The option CONFIG_DEVTMPFS_MOUNT=y makes the kernel mount devtmpfs at /dev automatically during boot, before init starts. This is essential for systems without initramfs.

devtmpfs and udev Collaboration

devtmpfs and udev form a collaborative partnership. The kernel creates device nodes quickly; udev enhances them with proper permissions, ownership, and symlinks. Neither could work as well alone.

devtmpfs Responsibilities

•Create device nodes immediately when devices register
•Delete nodes when devices unregister
•Use kernel-provided default names
•Set initial permissions (basic, restrictive)
•Provide instantaneous availability at boot

udev Responsibilities

•Apply user-defined permission rules
•Set correct ownership (user/group)
•Create persistent symlinks (by-id, by-path)
•Run helper scripts (notifications, automount)
•Handle complex naming policies

udev_and_devtmpfs
Bash
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
# ═══════════════════════════════════════════════════════════════
# See what devtmpfs creates vs what udev adds
# ═══════════════════════════════════════════════════════════════
 
# Kernel-created nodes (devtmpfs)
$ ls -la /dev/sda
brw-rw---- 1 root disk 8, 0 Jan 16 10:00 /dev/sda
# Note: group 'disk' was set by udev, not kernel
 
# udev-created symlinks
$ ls -la /dev/disk/by-id/
lrwxrwxrwx 1 root root 9 Jan 16 10:00 ata-Samsung_SSD_860_12345 -> ../../sda
lrwxrwxrwx 1 root root 9 Jan 16 10:00 wwn-0x50000000001 -> ../../sda
 
$ ls -la /dev/disk/by-uuid/
lrwxrwxrwx 1 root root 10 Jan 16 10:00 a1b2c3d4-... -> ../../sda1
 
# ═══════════════════════════════════════════════════════════════
# Example udev rule for a device
# ═══════════════════════════════════════════════════════════════
 
# /etc/udev/rules.d/99-usb-serial.rules
# Give specific USB serial device a consistent name and user access
SUBSYSTEM=="tty", ATTRS{idVendor}=="1234", ATTRS{idProduct}=="5678", \
    SYMLINK+="arduino", MODE="0666", GROUP="dialout"
 
# Result when this USB device is plugged in:
# 1. Kernel creates /dev/ttyUSB0 (devtmpfs)
# 2. udev receives uevent
# 3. udev matches rule, applies:
#    - Creates symlink /dev/arduino -> ttyUSB0
#    - Sets mode 0666 (world writable)
#    - Sets group to dialout
 
$ ls -la /dev/ttyUSB0 /dev/arduino
crw-rw-rw- 1 root dialout 188, 0 Jan 16 10:00 /dev/ttyUSB0
lrwxrwxrwx 1 root root    7 Jan 16 10:00 /dev/arduino -> ttyUSB0
 
# ═══════════════════════════════════════════════════════════════
# Timing: devtmpfs is immediate, udev takes time
# ═══════════════════════════════════════════════════════════════
 
# Watch device creation in real-time
$ udevadm monitor --property
 
# Plug in USB device...
KERNEL[123.456] add      /devices/pci.../usb1/1-1 (usb)
# [immediate] /dev/bus/usb/001/002 exists (devtmpfs)
 
UDEV[123.567] add       /devices/pci.../usb1/1-1 (usb)  
# [111ms later] symlinks created, permissions set
 
# The ~100ms gap is when udev rules are processed

Race Condition Prevention

devtmpfs eliminated the classic race condition where a script tried to access /dev/sda before udev created it. Now the kernel guarantees the node exists immediately. udev might not have set permissions yet, but the node is there—applications can wait with inotify if needed.

Boot Process and Early Device Access

One of devtmpfs's primary motivations was solving the early boot chicken-and-egg problem: you need devices to load init, but udev isn't running yet to create device nodes.

Converting Mermaid diagram...

Boot scenarios:

Boot without initramfs (CONFIG_DEVTMPFS_MOUNT=y):

•Kernel initializes, mounts devtmpfs at /dev automatically
•Device drivers probe hardware, nodes appear in /dev
•Kernel mounts root fs (specified by root= parameter)
•Kernel executes /sbin/init
•Init starts udev, which enhances existing nodes

This works for simple configurations where disk drivers are built into the kernel. No initramfs complexity needed.

boot_device_inspection
Bash
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
# Check if devtmpfs is mounted
$ mount | grep devtmpfs
devtmpfs on /dev type devtmpfs (rw,nosuid,size=16345678k,nr_inodes=4086419,mode=755)
 
# Check kernel configuration
$ zcat /proc/config.gz | grep DEVTMPFS
CONFIG_DEVTMPFS=y
CONFIG_DEVTMPFS_MOUNT=y
 
# See when devices were created (approximate, via dmesg)
$ dmesg | grep -E "(sd|nvme)" | head -5
[    2.123456] sd 0:0:0:0: [sda] 1953525168 512-byte logical blocks
[    2.123789] sd 0:0:0:0: [sda] Attached SCSI disk
 
# At this point, /dev/sda existed (before init started)
 
# Check boot timing for udev
$ systemd-analyze blame | grep udev
    432ms systemd-udev-trigger.service
    156ms systemd-udevd.service

The /dev Directory Structure

A modern Linux /dev directory contains both kernel-created nodes and udev-created symlinks/structure. Understanding this layout is essential for system administration and troubleshooting.

dev_directory_exploration
Bash
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
# ═══════════════════════════════════════════════════════════════
# Core device nodes (kernel-provided names)
# ═══════════════════════════════════════════════════════════════
 
$ ls -la /dev/sd*       # SCSI/SATA disks
brw-rw---- 1 root disk 8, 0 Jan 16 10:00 /dev/sda
brw-rw---- 1 root disk 8, 1 Jan 16 10:00 /dev/sda1
brw-rw---- 1 root disk 8, 2 Jan 16 10:00 /dev/sda2
 
$ ls -la /dev/nvme*     # NVMe SSDs
brw-rw---- 1 root disk 259, 0 Jan 16 10:00 /dev/nvme0n1
brw-rw---- 1 root disk 259, 1 Jan 16 10:00 /dev/nvme0n1p1
 
$ ls -la /dev/tty*      # Terminals
crw--w---- 1 root tty 4, 0 Jan 16 10:00 /dev/tty0
crw--w---- 1 root tty 4, 1 Jan 16 10:00 /dev/tty1
 
# ═══════════════════════════════════════════════════════════════
# Pseudo-device nodes (always present, memory-based)
# ═══════════════════════════════════════════════════════════════
 
$ ls -la /dev/null /dev/zero /dev/random /dev/urandom
crw-rw-rw- 1 root root 1, 3 Jan 16 10:00 /dev/null
crw-rw-rw- 1 root root 1, 5 Jan 16 10:00 /dev/zero
crw-rw-rw- 1 root root 1, 8 Jan 16 10:00 /dev/random
crw-rw-rw- 1 root root 1, 9 Jan 16 10:00 /dev/urandom
 
# ═══════════════════════════════════════════════════════════════
# udev-created symlink directories (persistent names)
# ═══════════════════════════════════════════════════════════════
 
$ ls /dev/disk/
by-id  by-label  by-partuuid  by-path  by-uuid
 
$ ls -la /dev/disk/by-id/
lrwxrwxrwx 1 root root  9 ... ata-Samsung_SSD_860_EVO_S1234567 -> ../../sda
lrwxrwxrwx 1 root root 10 ... ata-Samsung_SSD_860_EVO_S1234567-part1 -> ../../sda1
lrwxrwxrwx 1 root root  9 ... nvme-Samsung_SSD_970_EVO_XXXX -> ../../nvme0n1
lrwxrwxrwx 1 root root  9 ... wwn-0x5000000000000001 -> ../../sda
 
$ ls -la /dev/disk/by-uuid/
lrwxrwxrwx 1 root root 10 ... a1b2c3d4-e5f6-7890-abcd-ef1234567890 -> ../../sda1
 
# These symlinks remain consistent even if device order changes!
 
# ═══════════════════════════════════════════════════════════════
# Device subdirectories
# ═══════════════════════════════════════════════════════════════
 
$ ls /dev/input/            # Input devices (mice, keyboards)
by-id  by-path  event0  event1  event2  mice  mouse0
 
$ ls /dev/dri/              # Graphics (DRM)
by-path  card0  renderD128
 
$ ls /dev/snd/              # Sound (ALSA)
by-id  by-path  controlC0  hwC0D0  pcmC0D0c  pcmC0D0p  timer
 
$ ls /dev/bus/usb/          # USB device nodes (for libusb)
001  002  003  004

Common /dev Node Categories
Path Pattern	Type	Description	Created By
`/dev/sd[a-z]*`	Block	SCSI/SATA/USB disks	Kernel (devtmpfs)
`/dev/nvme*`	Block	NVMe SSDs	Kernel (devtmpfs)
`/dev/tty*`	Char	Terminals and consoles	Kernel (devtmpfs)
`/dev/pts/*`	Char	Pseudo-terminals (SSH, terminals)	devpts filesystem
`/dev/null`, `/dev/zero`	Char	Null sink, zero source	Kernel (always)
`/dev/random`, `/dev/urandom`	Char	Random number generators	Kernel (always)
`/dev/disk/by-*`	Symlinks	Persistent disk identifiers	udev rules
`/dev/input/event*`	Char	Input event devices	Kernel (devtmpfs)
`/dev/dri/*`	Char	GPU rendering nodes	Kernel (devtmpfs)
`/dev/shm`	Directory	POSIX shared memory mount point	System config

Containers and Minimal /dev

Containers don't use devtmpfs directly—instead, they receive a minimal, curated /dev directory. This isolation is crucial for security and resource management.

container_dev
Bash
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
# ═══════════════════════════════════════════════════════════════
# Docker container's default /dev
# ═══════════════════════════════════════════════════════════════
 
$ docker run --rm alpine ls -la /dev
total 4
drwxr-xr-x    5 root root 360 Jan 16 10:00 .
drwxr-xr-x    1 root root   6 Jan 16 10:00 ..
crw--w----    1 root tty  136, 0 Jan 16 10:00 console
lrwxrwxrwx    1 root root 11 Jan 16 10:00 core -> /proc/kcore
lrwxrwxrwx    1 root root 13 Jan 16 10:00 fd -> /proc/self/fd
crw-rw-rw-    1 root root   1, 7 Jan 16 10:00 full
drwxrwxrwt    2 root root  40 Jan 16 10:00 mqueue
crw-rw-rw-    1 root root   1, 3 Jan 16 10:00 null
lrwxrwxrwx    1 root root  8 Jan 16 10:00 ptmx -> pts/ptmx
drwxr-xr-x    2 root root   0 Jan 16 10:00 pts
crw-rw-rw-    1 root root   1, 8 Jan 16 10:00 random
drwxrwxrwt    2 root root  40 Jan 16 10:00 shm
lrwxrwxrwx    1 root root 15 Jan 16 10:00 stderr -> /proc/self/fd/2
lrwxrwxrwx    1 root root 15 Jan 16 10:00 stdin -> /proc/self/fd/0
lrwxrwxrwx    1 root root 15 Jan 16 10:00 stdout -> /proc/self/fd/1
crw-rw-rw-    1 root root   5, 0 Jan 16 10:00 tty
crw-rw-rw-    1 root root   1, 9 Jan 16 10:00 urandom
crw-rw-rw-    1 root root   1, 5 Jan 16 10:00 zero
 
# Note: No sda, tty0, etc. - only essential pseudo-devices
 
# ═══════════════════════════════════════════════════════════════
# Granting access to host devices
# ═══════════════════════════════════════════════════════════════
 
# Add a single device
$ docker run --device=/dev/video0 myapp
 
# Add GPU access (NVIDIA)
$ docker run --gpus all nvidia/cuda nvidia-smi
 
# Add all devices (dangerous!)
$ docker run --privileged ubuntu ls /dev
# Shows entire host /dev - rarely appropriate
 
# ═══════════════════════════════════════════════════════════════
# How container runtimes create /dev
# ═══════════════════════════════════════════════════════════════
 
# 1. Create tmpfs at container's /dev
mount -t tmpfs -o size=65536k,mode=755 tmpfs /container_root/dev
 
# 2. Create essential device nodes
mknod /container_root/dev/null c 1 3
mknod /container_root/dev/zero c 1 5
mknod /container_root/dev/full c 1 7
mknod /container_root/dev/random c 1 8
mknod /container_root/dev/urandom c 1 9
mknod /container_root/dev/tty c 5 0
...
 
# 3. Create symbolic links
ln -s /proc/self/fd /container_root/dev/fd
ln -s /proc/self/fd/0 /container_root/dev/stdin
...
 
# 4. Optionally bind-mount specific host devices
mount --bind /dev/nvidia0 /container_root/dev/nvidia0

Security Implications

The minimal container /dev is a security boundary. Access to /dev/sda would allow reading the host's disk. Access to /dev/kmem would allow reading kernel memory. The curated /dev ensures containers can only interact with safe, isolated pseudo-devices unless explicitly granted access.

Embedded and Minimal Systems

devtmpfs is particularly valuable for embedded systems and minimal Linux installations where running full udev is impractical or unnecessary.

Embedded System Approaches

•devtmpfs-only — Kernel creates all devices with default names and permissions. Simple, deterministic, no userspace dependency. Common in busybox-based systems.
•devtmpfs + mdev — Busybox's lightweight udev alternative. Reads simple rules, sets permissions. Much smaller than udev (~30KB vs >1MB).
•devtmpfs + eudev — Fork of udev without systemd dependency. Full udev features for non-systemd distributions.
•Static /dev — For truly minimal, fixed-function devices. Nodes created at build time, no runtime creation.

embedded_dev_config
Bash / Configuration
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
# ═══════════════════════════════════════════════════════════════
# Minimal initramfs with devtmpfs only (no udev)
# ═══════════════════════════════════════════════════════════════
 
# Kernel command line
root=/dev/mmcblk0p2 rootfstype=ext4 rootwait
 
# init script (minimal)
#!/bin/sh
# devtmpfs is auto-mounted by kernel (CONFIG_DEVTMPFS_MOUNT=y)
# or mount manually:
mount -t devtmpfs devtmpfs /dev
 
# Mount other filesystems
mount -t proc proc /proc
mount -t sysfs sysfs /sys
 
# Root device is now available at /dev/mmcblk0p2
# Mount and switch to real root
mount /dev/mmcblk0p2 /mnt/root
exec switch_root /mnt/root /sbin/init
 
# ═══════════════════════════════════════════════════════════════
# Busybox mdev configuration
# ═══════════════════════════════════════════════════════════════
 
# /etc/mdev.conf - simple rules
# Format: <device regex> <uid>:<gid> <permissions> [=|>|!] [<path>] [@|$|*] <command>
 
# Set permissions on specific devices
null            root:root 0666
zero            root:root 0666
random          root:root 0666
urandom         root:root 0666
console         root:tty  0600
tty             root:tty  0666
tty[0-9]*       root:tty  0620
 
# Block devices
sd[a-z].*       root:disk 0660
 
# USB serial adapters
ttyUSB[0-9]*    root:dialout 0660
 
# Create symlink for first USB serial
ttyUSB0         root:dialout 0660 =ttyUSB0 @ln -sf ttyUSB0 /dev/gps
 
# ═══════════════════════════════════════════════════════════════
# Enable mdev as hotplug handler
# ═══════════════════════════════════════════════════════════════
 
# Tell kernel to call mdev for hotplug events
echo /sbin/mdev > /proc/sys/kernel/hotplug
 
# Or use netlink-based mdevd for async operation
mdevd &

When devtmpfs Alone Is Enough

For embedded devices with fixed hardware configurations, devtmpfs alone often suffices. The kernel creates all needed nodes. If you only need consistent permissions (e.g., /dev/ttyS0 as 0666), you can either chmod in init scripts or use minimal mdev rules.

Summary: devtmpfs

devtmpfs revolutionized Linux device management by moving device node creation into the kernel while preserving userspace policy control.

Key Takeaways

•Device nodes explained — Special files with type (char/block) and major:minor numbers that route I/O to drivers
•Kernel creates nodes — devtmpfs creates device files immediately when devices register, eliminating race conditions
•udev enhances — Userspace sets permissions, creates symlinks, runs scripts—policy stays out of the kernel
•Boot simplification — Early boot has device nodes before init starts; no static /dev or complex initramfs needed
•Persistent naming — udev creates stable symlinks (/dev/disk/by-uuid/) while kernel uses simple names (sda)
•Container isolation — Containers get minimal, curated /dev preventing access to host hardware
•Embedded friendly — Can function without udev for minimal systems; works with lightweight alternatives like mdev

What's next:

We've explored kernel-space virtual file systems that expose system information (/proc, /sys) and provide dynamic storage (tmpfs, devtmpfs). Our final page examines FUSE (Filesystem in Userspace)—the framework that lets ordinary programs become file system drivers, enabling everything from network file systems to archive mounting to custom virtual file systems.

Page Complete

You now understand how devtmpfs works with udev to provide dynamic device management, the device node system (major/minor numbers), boot process integration, and its role in containers and embedded systems. This knowledge is fundamental to Linux system administration and device driver development.

devtmpfs - Dynamic Device File System

Where Do Device Files Come From?

What You Will Learn

Historical Context: From Static to Dynamic /dev

Evolution of Linux Device Management
Era	Approach	Mechanism	Limitations
1970s-1990s	Static /dev	`mknod` commands, persistent on disk	Thousands of unused nodes, manual management
Early 2000s	devfs	Kernel creates nodes; first attempt at dynamic	Naming policy in kernel (bad design), removed in 2.6
2004-2009	udev on tmpfs	udev creates nodes in response to kernel events	Race conditions at early boot; slow for many devices
2009+	devtmpfs + udev	Kernel creates nodes; udev adjusts permissions	Best of both: kernel speed + userspace flexibility

The devfs mistake:

The udev-only era:

dev 2.6 removed devfs and introduced udev—a userspace daemon that receives kernel device events and creates nodes accordingly. This was conceptually clean but had practical problems:

Early boot required a minimal static /dev (initramfs)
Race conditions between device detection and udev processing
Slow enumeration when many devices were present

The devtmpfs solution:

devtmpfs (introduced 2009, Linux 2.6.32) combines kernel efficiency with userspace flexibility:

Kernel creates nodes — Fast, no races, available immediately
Uses simple naming — Kernel uses default names (sda, tty0)
udev adjusts — Userspace sets permissions, creates symlinks, runs scripts

Separation of Concerns

Device Nodes Explained

Before diving into devtmpfs mechanics, we must understand what device nodes actually are—the fundamental interface between userspace and kernel device drivers.

Device Node Characteristics

•Type — Character device (c) or block device (b)
•Major number — Identifies the driver handling this device
•Minor number — Identifies the specific device handled by that driver
•Permissions — Standard Unix permissions (owner, group, mode)
•Path — Location in filesystem (conventionally in /dev)

device_node_examples
Bash
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
# Viewing device nodes with their major:minor numbers
$ ls -la /dev/sda /dev/tty0 /dev/null /dev/random
brw-rw---- 1 root disk    8,   0 Jan 16 10:00 /dev/sda
crw--w---- 1 root tty     4,   0 Jan 16 10:00 /dev/tty0  
crw-rw-rw- 1 root root    1,   3 Jan 16 10:00 /dev/null
crw-rw-rw- 1 root root    1,   8 Jan 16 10:00 /dev/random
 
# Breaking down the output:
# 'b' = block device, 'c' = character device
# 8, 0 = major 8, minor 0 (SCSI disk driver, first disk)
# 4, 0 = major 4, minor 0 (TTY driver, console 0)
# 1, 3 = major 1, minor 3 (memory devices, null device)
# 1, 8 = major 1, minor 8 (memory devices, random)
 
# The kernel maintains a registry of major numbers
$ cat /proc/devices
Character devices:
  1 mem
  4 /dev/vc/0
  4 tty
  5 /dev/tty
  ...
  
Block devices:
  8 sd
  9 md
 11 sr
 ...
 
# Creating a device node manually (rarely needed today)
$ sudo mknod /dev/test_device c 100 0
$ ls -la /dev/test_device
crw-r--r-- 1 root root 100, 0 Jan 16 10:00 /dev/test_device
 
# What happens when you access a device?
$ echo "hello" > /dev/null   # Write routed to null driver
$ cat /dev/random | head -c 16 | xxd  # Read routed to random driver

Block vs Character devices:

Block devices — Data accessed in fixed-size blocks (sectors); support seeking, buffering, caching. Examples: disks (sd*), optical drives (sr*)
Character devices — Data accessed as a stream of bytes; no inherent structure. Examples: terminals (tty*), serial ports, mice, keyboards

Major/Minor number system:

Major numbers are assigned to drivers (often statically defined), while minor numbers distinguish instances. For example:

Major 8 = SCSI disk driver
Minor 0 = /dev/sda (first SCSI disk)
Minor 1 = /dev/sda1 (first partition of sda)
Minor 16 = /dev/sdb (second SCSI disk, minors 0-15 for sda partitions)

How devtmpfs Works

The devtmpfs lifecycle:

Converting Mermaid diagram...

devtmpfs_implementation
C (Kernel)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
/*
 * Simplified view of devtmpfs device node creation
 * Actual code in drivers/base/devtmpfs.c
 */
 
/* Called when device_add() is invoked */
int devtmpfs_create_node(struct device *dev)
{
    const char *nodename;
    umode_t mode = 0;
    kuid_t uid = GLOBAL_ROOT_UID;
    kgid_t gid = GLOBAL_ROOT_GID;
    int err;
 
    /* Get the device name (e.g., "sda", "tty0") */
    nodename = device_get_devnode(dev, &mode, &uid, &gid, &tmp);
    if (!nodename)
        return -ENOMEM;
 
    /* Determine the node type and create it */
    if (S_ISBLK(mode))
        err = vfs_mknod(/* path, mode, dev_t */);
    else
        err = vfs_create(/* path, mode */);
 
    /* Set ownership */
    if (!err)
        err = vfs_chown(/* path, uid, gid */);
 
    return err;
}
 
/* Called when device_del() is invoked */
int devtmpfs_delete_node(struct device *dev)
{
    const char *nodename;
    
    nodename = device_get_devnode(dev, NULL, NULL, NULL, NULL);
    if (!nodename)
        return -ENOMEM;
 
    /* Simply unlink the file */
    return vfs_unlink(/* path */);
}

Key implementation details:

Kernel thread — devtmpfs runs a kernel thread (kdevtmpfs) that handles node creation/deletion
Synchronous at boot — During early boot, operations are synchronous to ensure devices are ready
Asynchronous later — Once userspace starts, operations queue for the kernel thread
Default permissions — Kernel sets basic permissions (often 0600 root:root); udev adjusts
Simple naming — Kernel uses default names from drivers; udev creates additional symlinks

Kernel Configuration

devtmpfs and udev Collaboration

devtmpfs and udev form a collaborative partnership. The kernel creates device nodes quickly; udev enhances them with proper permissions, ownership, and symlinks. Neither could work as well alone.

devtmpfs Responsibilities

•Create device nodes immediately when devices register
•Delete nodes when devices unregister
•Use kernel-provided default names
•Set initial permissions (basic, restrictive)
•Provide instantaneous availability at boot

udev Responsibilities

•Apply user-defined permission rules
•Set correct ownership (user/group)
•Create persistent symlinks (by-id, by-path)
•Run helper scripts (notifications, automount)
•Handle complex naming policies

udev_and_devtmpfs
Bash
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
# ═══════════════════════════════════════════════════════════════
# See what devtmpfs creates vs what udev adds
# ═══════════════════════════════════════════════════════════════
 
# Kernel-created nodes (devtmpfs)
$ ls -la /dev/sda
brw-rw---- 1 root disk 8, 0 Jan 16 10:00 /dev/sda
# Note: group 'disk' was set by udev, not kernel
 
# udev-created symlinks
$ ls -la /dev/disk/by-id/
lrwxrwxrwx 1 root root 9 Jan 16 10:00 ata-Samsung_SSD_860_12345 -> ../../sda
lrwxrwxrwx 1 root root 9 Jan 16 10:00 wwn-0x50000000001 -> ../../sda
 
$ ls -la /dev/disk/by-uuid/
lrwxrwxrwx 1 root root 10 Jan 16 10:00 a1b2c3d4-... -> ../../sda1
 
# ═══════════════════════════════════════════════════════════════
# Example udev rule for a device
# ═══════════════════════════════════════════════════════════════
 
# /etc/udev/rules.d/99-usb-serial.rules
# Give specific USB serial device a consistent name and user access
SUBSYSTEM=="tty", ATTRS{idVendor}=="1234", ATTRS{idProduct}=="5678", \
    SYMLINK+="arduino", MODE="0666", GROUP="dialout"
 
# Result when this USB device is plugged in:
# 1. Kernel creates /dev/ttyUSB0 (devtmpfs)
# 2. udev receives uevent
# 3. udev matches rule, applies:
#    - Creates symlink /dev/arduino -> ttyUSB0
#    - Sets mode 0666 (world writable)
#    - Sets group to dialout
 
$ ls -la /dev/ttyUSB0 /dev/arduino
crw-rw-rw- 1 root dialout 188, 0 Jan 16 10:00 /dev/ttyUSB0
lrwxrwxrwx 1 root root    7 Jan 16 10:00 /dev/arduino -> ttyUSB0
 
# ═══════════════════════════════════════════════════════════════
# Timing: devtmpfs is immediate, udev takes time
# ═══════════════════════════════════════════════════════════════
 
# Watch device creation in real-time
$ udevadm monitor --property
 
# Plug in USB device...
KERNEL[123.456] add      /devices/pci.../usb1/1-1 (usb)
# [immediate] /dev/bus/usb/001/002 exists (devtmpfs)
 
UDEV[123.567] add       /devices/pci.../usb1/1-1 (usb)  
# [111ms later] symlinks created, permissions set
 
# The ~100ms gap is when udev rules are processed

Race Condition Prevention

Boot Process and Early Device Access

One of devtmpfs's primary motivations was solving the early boot chicken-and-egg problem: you need devices to load init, but udev isn't running yet to create device nodes.

Converting Mermaid diagram...

Boot scenarios:

Boot without initramfs (CONFIG_DEVTMPFS_MOUNT=y):

•Kernel initializes, mounts devtmpfs at /dev automatically
•Device drivers probe hardware, nodes appear in /dev
•Kernel mounts root fs (specified by root= parameter)
•Kernel executes /sbin/init
•Init starts udev, which enhances existing nodes

This works for simple configurations where disk drivers are built into the kernel. No initramfs complexity needed.

boot_device_inspection
Bash
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
# Check if devtmpfs is mounted
$ mount | grep devtmpfs
devtmpfs on /dev type devtmpfs (rw,nosuid,size=16345678k,nr_inodes=4086419,mode=755)
 
# Check kernel configuration
$ zcat /proc/config.gz | grep DEVTMPFS
CONFIG_DEVTMPFS=y
CONFIG_DEVTMPFS_MOUNT=y
 
# See when devices were created (approximate, via dmesg)
$ dmesg | grep -E "(sd|nvme)" | head -5
[    2.123456] sd 0:0:0:0: [sda] 1953525168 512-byte logical blocks
[    2.123789] sd 0:0:0:0: [sda] Attached SCSI disk
 
# At this point, /dev/sda existed (before init started)
 
# Check boot timing for udev
$ systemd-analyze blame | grep udev
    432ms systemd-udev-trigger.service
    156ms systemd-udevd.service

The /dev Directory Structure

A modern Linux /dev directory contains both kernel-created nodes and udev-created symlinks/structure. Understanding this layout is essential for system administration and troubleshooting.

dev_directory_exploration
Bash
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
# ═══════════════════════════════════════════════════════════════
# Core device nodes (kernel-provided names)
# ═══════════════════════════════════════════════════════════════
 
$ ls -la /dev/sd*       # SCSI/SATA disks
brw-rw---- 1 root disk 8, 0 Jan 16 10:00 /dev/sda
brw-rw---- 1 root disk 8, 1 Jan 16 10:00 /dev/sda1
brw-rw---- 1 root disk 8, 2 Jan 16 10:00 /dev/sda2
 
$ ls -la /dev/nvme*     # NVMe SSDs
brw-rw---- 1 root disk 259, 0 Jan 16 10:00 /dev/nvme0n1
brw-rw---- 1 root disk 259, 1 Jan 16 10:00 /dev/nvme0n1p1
 
$ ls -la /dev/tty*      # Terminals
crw--w---- 1 root tty 4, 0 Jan 16 10:00 /dev/tty0
crw--w---- 1 root tty 4, 1 Jan 16 10:00 /dev/tty1
 
# ═══════════════════════════════════════════════════════════════
# Pseudo-device nodes (always present, memory-based)
# ═══════════════════════════════════════════════════════════════
 
$ ls -la /dev/null /dev/zero /dev/random /dev/urandom
crw-rw-rw- 1 root root 1, 3 Jan 16 10:00 /dev/null
crw-rw-rw- 1 root root 1, 5 Jan 16 10:00 /dev/zero
crw-rw-rw- 1 root root 1, 8 Jan 16 10:00 /dev/random
crw-rw-rw- 1 root root 1, 9 Jan 16 10:00 /dev/urandom
 
# ═══════════════════════════════════════════════════════════════
# udev-created symlink directories (persistent names)
# ═══════════════════════════════════════════════════════════════
 
$ ls /dev/disk/
by-id  by-label  by-partuuid  by-path  by-uuid
 
$ ls -la /dev/disk/by-id/
lrwxrwxrwx 1 root root  9 ... ata-Samsung_SSD_860_EVO_S1234567 -> ../../sda
lrwxrwxrwx 1 root root 10 ... ata-Samsung_SSD_860_EVO_S1234567-part1 -> ../../sda1
lrwxrwxrwx 1 root root  9 ... nvme-Samsung_SSD_970_EVO_XXXX -> ../../nvme0n1
lrwxrwxrwx 1 root root  9 ... wwn-0x5000000000000001 -> ../../sda
 
$ ls -la /dev/disk/by-uuid/
lrwxrwxrwx 1 root root 10 ... a1b2c3d4-e5f6-7890-abcd-ef1234567890 -> ../../sda1
 
# These symlinks remain consistent even if device order changes!
 
# ═══════════════════════════════════════════════════════════════
# Device subdirectories
# ═══════════════════════════════════════════════════════════════
 
$ ls /dev/input/            # Input devices (mice, keyboards)
by-id  by-path  event0  event1  event2  mice  mouse0
 
$ ls /dev/dri/              # Graphics (DRM)
by-path  card0  renderD128
 
$ ls /dev/snd/              # Sound (ALSA)
by-id  by-path  controlC0  hwC0D0  pcmC0D0c  pcmC0D0p  timer
 
$ ls /dev/bus/usb/          # USB device nodes (for libusb)
001  002  003  004

Common /dev Node Categories
Path Pattern	Type	Description	Created By
`/dev/sd[a-z]*`	Block	SCSI/SATA/USB disks	Kernel (devtmpfs)
`/dev/nvme*`	Block	NVMe SSDs	Kernel (devtmpfs)
`/dev/tty*`	Char	Terminals and consoles	Kernel (devtmpfs)
`/dev/pts/*`	Char	Pseudo-terminals (SSH, terminals)	devpts filesystem
`/dev/null`, `/dev/zero`	Char	Null sink, zero source	Kernel (always)
`/dev/random`, `/dev/urandom`	Char	Random number generators	Kernel (always)
`/dev/disk/by-*`	Symlinks	Persistent disk identifiers	udev rules
`/dev/input/event*`	Char	Input event devices	Kernel (devtmpfs)
`/dev/dri/*`	Char	GPU rendering nodes	Kernel (devtmpfs)
`/dev/shm`	Directory	POSIX shared memory mount point	System config

Containers and Minimal /dev

Containers don't use devtmpfs directly—instead, they receive a minimal, curated /dev directory. This isolation is crucial for security and resource management.

container_dev
Bash
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
# ═══════════════════════════════════════════════════════════════
# Docker container's default /dev
# ═══════════════════════════════════════════════════════════════
 
$ docker run --rm alpine ls -la /dev
total 4
drwxr-xr-x    5 root root 360 Jan 16 10:00 .
drwxr-xr-x    1 root root   6 Jan 16 10:00 ..
crw--w----    1 root tty  136, 0 Jan 16 10:00 console
lrwxrwxrwx    1 root root 11 Jan 16 10:00 core -> /proc/kcore
lrwxrwxrwx    1 root root 13 Jan 16 10:00 fd -> /proc/self/fd
crw-rw-rw-    1 root root   1, 7 Jan 16 10:00 full
drwxrwxrwt    2 root root  40 Jan 16 10:00 mqueue
crw-rw-rw-    1 root root   1, 3 Jan 16 10:00 null
lrwxrwxrwx    1 root root  8 Jan 16 10:00 ptmx -> pts/ptmx
drwxr-xr-x    2 root root   0 Jan 16 10:00 pts
crw-rw-rw-    1 root root   1, 8 Jan 16 10:00 random
drwxrwxrwt    2 root root  40 Jan 16 10:00 shm
lrwxrwxrwx    1 root root 15 Jan 16 10:00 stderr -> /proc/self/fd/2
lrwxrwxrwx    1 root root 15 Jan 16 10:00 stdin -> /proc/self/fd/0
lrwxrwxrwx    1 root root 15 Jan 16 10:00 stdout -> /proc/self/fd/1
crw-rw-rw-    1 root root   5, 0 Jan 16 10:00 tty
crw-rw-rw-    1 root root   1, 9 Jan 16 10:00 urandom
crw-rw-rw-    1 root root   1, 5 Jan 16 10:00 zero
 
# Note: No sda, tty0, etc. - only essential pseudo-devices
 
# ═══════════════════════════════════════════════════════════════
# Granting access to host devices
# ═══════════════════════════════════════════════════════════════
 
# Add a single device
$ docker run --device=/dev/video0 myapp
 
# Add GPU access (NVIDIA)
$ docker run --gpus all nvidia/cuda nvidia-smi
 
# Add all devices (dangerous!)
$ docker run --privileged ubuntu ls /dev
# Shows entire host /dev - rarely appropriate
 
# ═══════════════════════════════════════════════════════════════
# How container runtimes create /dev
# ═══════════════════════════════════════════════════════════════
 
# 1. Create tmpfs at container's /dev
mount -t tmpfs -o size=65536k,mode=755 tmpfs /container_root/dev
 
# 2. Create essential device nodes
mknod /container_root/dev/null c 1 3
mknod /container_root/dev/zero c 1 5
mknod /container_root/dev/full c 1 7
mknod /container_root/dev/random c 1 8
mknod /container_root/dev/urandom c 1 9
mknod /container_root/dev/tty c 5 0
...
 
# 3. Create symbolic links
ln -s /proc/self/fd /container_root/dev/fd
ln -s /proc/self/fd/0 /container_root/dev/stdin
...
 
# 4. Optionally bind-mount specific host devices
mount --bind /dev/nvidia0 /container_root/dev/nvidia0

Security Implications

Embedded and Minimal Systems

devtmpfs is particularly valuable for embedded systems and minimal Linux installations where running full udev is impractical or unnecessary.

Embedded System Approaches

•devtmpfs-only — Kernel creates all devices with default names and permissions. Simple, deterministic, no userspace dependency. Common in busybox-based systems.
•devtmpfs + mdev — Busybox's lightweight udev alternative. Reads simple rules, sets permissions. Much smaller than udev (~30KB vs >1MB).
•devtmpfs + eudev — Fork of udev without systemd dependency. Full udev features for non-systemd distributions.
•Static /dev — For truly minimal, fixed-function devices. Nodes created at build time, no runtime creation.

embedded_dev_config
Bash / Configuration
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
# ═══════════════════════════════════════════════════════════════
# Minimal initramfs with devtmpfs only (no udev)
# ═══════════════════════════════════════════════════════════════
 
# Kernel command line
root=/dev/mmcblk0p2 rootfstype=ext4 rootwait
 
# init script (minimal)
#!/bin/sh
# devtmpfs is auto-mounted by kernel (CONFIG_DEVTMPFS_MOUNT=y)
# or mount manually:
mount -t devtmpfs devtmpfs /dev
 
# Mount other filesystems
mount -t proc proc /proc
mount -t sysfs sysfs /sys
 
# Root device is now available at /dev/mmcblk0p2
# Mount and switch to real root
mount /dev/mmcblk0p2 /mnt/root
exec switch_root /mnt/root /sbin/init
 
# ═══════════════════════════════════════════════════════════════
# Busybox mdev configuration
# ═══════════════════════════════════════════════════════════════
 
# /etc/mdev.conf - simple rules
# Format: <device regex> <uid>:<gid> <permissions> [=|>|!] [<path>] [@|$|*] <command>
 
# Set permissions on specific devices
null            root:root 0666
zero            root:root 0666
random          root:root 0666
urandom         root:root 0666
console         root:tty  0600
tty             root:tty  0666
tty[0-9]*       root:tty  0620
 
# Block devices
sd[a-z].*       root:disk 0660
 
# USB serial adapters
ttyUSB[0-9]*    root:dialout 0660
 
# Create symlink for first USB serial
ttyUSB0         root:dialout 0660 =ttyUSB0 @ln -sf ttyUSB0 /dev/gps
 
# ═══════════════════════════════════════════════════════════════
# Enable mdev as hotplug handler
# ═══════════════════════════════════════════════════════════════
 
# Tell kernel to call mdev for hotplug events
echo /sbin/mdev > /proc/sys/kernel/hotplug
 
# Or use netlink-based mdevd for async operation
mdevd &

When devtmpfs Alone Is Enough

Summary: devtmpfs

devtmpfs revolutionized Linux device management by moving device node creation into the kernel while preserving userspace policy control.

Key Takeaways

•Device nodes explained — Special files with type (char/block) and major:minor numbers that route I/O to drivers
•Kernel creates nodes — devtmpfs creates device files immediately when devices register, eliminating race conditions
•udev enhances — Userspace sets permissions, creates symlinks, runs scripts—policy stays out of the kernel
•Boot simplification — Early boot has device nodes before init starts; no static /dev or complex initramfs needed
•Persistent naming — udev creates stable symlinks (/dev/disk/by-uuid/) while kernel uses simple names (sda)
•Container isolation — Containers get minimal, curated /dev preventing access to host hardware
•Embedded friendly — Can function without udev for minimal systems; works with lightweight alternatives like mdev

What's next:

Page Complete