Task Queue — Thread Pools | OneNoughtOne

0/107

Operating SystemsThread Pools

Thread Pools

LevelIntermediate

Duration75 mins

TopicThread Pools

3 / 5

Task Queue

The Buffer Between Work and Workers

In any producer-consumer system, the buffer between producers and consumers is the critical control point. It determines how much work can be pending, how load spikes are absorbed, what happens when consumption can't keep up with production, and how work is prioritized. In thread pools, this buffer is the task queue (also called the work queue). It's far more than a simple list of tasks waiting to be executed. The queue's type, capacity, and ordering policy fundamentally shape the pool's behavior under load—determining whether the system degrades gracefully or catastrophically when stressed. Many thread pool performance problems trace to queue misconfiguration. An unbounded queue can exhaust memory during load spikes. A bounded queue with the wrong rejection policy can drop critical work. A FIFO queue can cause priority inversions. Understanding task queues deeply prevents these failures.

What You Will Learn

By the end of this page, you will understand task queue fundamentals, the various queue types available, capacity and bounding strategies, ordering policies, rejection handling, and the performance implications of queue design. You'll be able to select and configure the right queue for any workload.

Queue Fundamentals

A task queue in a thread pool is a thread-safe data structure that holds tasks waiting for execution. It provides the essential decoupling between task submission (by arbitrary application threads) and task execution (by pool worker threads). Core Requirements: A task queue must satisfy several requirements to work correctly in a thread pool:

Task Queue Requirements

•Thread Safety — Multiple producers may submit tasks concurrently, and multiple workers may dequeue tasks concurrently. All operations must be safe without external synchronization.
•Blocking Support — Workers must be able to wait efficiently when the queue is empty, rather than busy-waiting and consuming CPU.
•Bounded or Unbounded — The queue must implement a clear capacity policy, either unbounded (grow indefinitely) or bounded (reject/block when full).
•Ordering Semantics — The queue must define and maintain task ordering: FIFO, LIFO, priority-based, or other policies.
•Introspection — The queue should expose its current size, remaining capacity, and other metrics for monitoring.

The BlockingQueue Interface (Java): Java's BlockingQueue interface captures these requirements and is the standard abstraction for thread pool queues:

blocking_queue_interface
Java
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
public interface BlockingQueue<E> extends Queue<E> {
    
    // === Producer Operations ===
    
    // Add, throw exception if full
    boolean add(E e);
    
    // Add, return false if full
    boolean offer(E e);
    
    // Add, block until space available
    void put(E e) throws InterruptedException;
    
    // Add, block with timeout
    boolean offer(E e, long timeout, TimeUnit unit) 
        throws InterruptedException;
    
    // === Consumer Operations ===
    
    // Remove and return, block if empty
    E take() throws InterruptedException;
    
    // Remove and return, block with timeout
    E poll(long timeout, TimeUnit unit) 
        throws InterruptedException;
    
    // === Inspection ===
    
    int remainingCapacity();  // Space before full
    int size();               // Current element count
    boolean contains(Object o);
    
    // === Batch Operations ===
    
    // Remove all and add to collection
    int drainTo(Collection<? super E> c);
    int drainTo(Collection<? super E> c, int maxElements);
}

Queue Operations in Thread Pool Context: Producers (task submitters) use offer() (non-blocking) or put() (blocking) to add tasks. Consumers (workers) use take() (blocking) or poll(timeout) to retrieve tasks. Pool manager uses size(), remainingCapacity(), and drainTo() for monitoring and shutdown. The choice between blocking and non-blocking operations affects how the pool behaves when the queue is full (for producers) or empty (for consumers).

The Blocking Wait

When a worker calls take() on an empty queue, it enters an efficient wait state—typically using OS synchronization primitives (futex on Linux, WaitForSingleObject on Windows). The worker consumes no CPU while waiting and is awakened only when work arrives. This is why idle pools have negligible performance impact.

Queue Types

Different queue implementations offer different tradeoffs between performance, ordering, and behavior. Understanding these options is essential for selecting the right queue for your workload.

LinkedBlockingQueue: A FIFO queue backed by a linked list. Can be bounded or unbounded. Characteristics: Ordering : FIFO (First-In-First-Out) Capacity : Optional bound; unbounded if not specified Concurrency : Separate locks for head and tail (two-lock queue) Memory : Node allocation per element (GC overhead) Performance: Better concurrency than ArrayBlockingQueue (separate locks) Additional GC pressure from node objects Excellent for high-throughput producer-consumer When to Use: Default choice for general-purpose pools High concurrency with many producers/consumers When you want unbounded buffering (with caution)

linked_blocking_queue
Java
1
2
3
4
5
6
7
8
9
10
11
// Unbounded LinkedBlockingQueue (careful - can exhaust memory!)
BlockingQueue<Runnable> unbounded = new LinkedBlockingQueue<>();
 
// Bounded LinkedBlockingQueue (recommended)
BlockingQueue<Runnable> bounded = new LinkedBlockingQueue<>(1000);
 
// In ThreadPoolExecutor
ExecutorService pool = new ThreadPoolExecutor(
    4, 4, 0, TimeUnit.SECONDS,
    new LinkedBlockingQueue<>(1000)
);

Queue Capacity and Bounding

The decision between bounded and unbounded queues is one of the most consequential choices in thread pool configuration. Each has significant implications for system behavior under load. Unbounded Queues:

Unbounded Queue Advantages

•Never rejects tasks due to queue capacity
•Absorbs arbitrary traffic bursts
•Simple configuration (no capacity decision)
•Producers never block on queue

Unbounded Queue Dangers

•Can exhaust memory during load spikes
•Masks capacity problems (queue grows silently)
•Latency grows unboundedly as queue builds
•No backpressure signal to producers

Bounded Queues: Bounded queues provide natural backpressure—when the queue is full, producers are forced to slow down (block) or handle rejection. This prevents memory exhaustion and provides early warning of capacity issues. Choosing Queue Capacity: Queue capacity determines how much burst traffic the pool can absorb. Consider the following analysis:

queue_capacity_analysis

Analysis

Given:
  - Pool size: N workers
  - Average task duration: D seconds
  - Maximum acceptable wait time: W seconds
  - Task arrival rate during burst: R tasks/second
 
Analysis:
 
  Pool throughput = N / D tasks/second
 
  Queue drains at rate: N / D
  Queue fills at rate: R (during burst)
 
  If R > N/D, queue grows during burst at rate: R - N/D
 
  To survive a burst of duration B seconds:
    Queue capacity >= (R - N/D) × B
 
  To limit wait time to W seconds:
    Queue capacity <= W × (N / D)
 
Example:
  N = 8 workers
  D = 0.1 seconds (100ms per task)
  Max throughput = 8 / 0.1 = 80 tasks/second
  
  Burst: 200 tasks/second for 10 seconds
  Queue growth rate: 200 - 80 = 120 tasks/second
  Queue needed: 120 × 10 = 1200 tasks
  
  With queue capacity 1200:
  - Queue fills during burst
  - Drains after burst ends
  - No rejections
  
  But: longest wait = 1200 / 80 = 15 seconds
  If max wait is 5 seconds, limit queue to 400 tasks

Memory Considerations: Queue capacity affects memory usage: Each task object consumes memory (typically 64-256 bytes for simple Runnables) Tasks may reference larger objects (request data, result buffers) Queue nodes add overhead (LinkedBlockingQueue: ~40 bytes/node) Rule of Thumb: Start with a bounded queue sized to handle expected burst duration while keeping wait times acceptable. Monitor queue depth in production and adjust based on observed behavior.

The Unbounded Queue Trap

Executors.newFixedThreadPool() uses an unbounded LinkedBlockingQueue. This is convenient but dangerous in production. Under sustained overload, the queue grows until OutOfMemoryError crashes the JVM. Always consider bounded queues with explicit rejection handling for production systems.

Rejection Policies

When a bounded queue is full and the pool cannot accept more tasks, the pool must decide what to do. This decision is encapsulated in the rejection policy (or saturation policy). When Rejection Occurs: A task is rejected when: The work queue is full (cannot accept more tasks) The pool has reached maximum size (cannot create more workers) The pool has been shut down (not accepting new tasks) Standard Rejection Policies (Java):

Built-in Rejection Policies
Policy	Behavior	Use Case
AbortPolicy	Throws RejectedExecutionException	Default. Task submitter must handle failure.
CallerRunsPolicy	Executes task in the calling thread	Backpressure—slows producer when pool is saturated.
DiscardPolicy	Silently discards the task	Best-effort processing, okay to lose work.
DiscardOldestPolicy	Discards oldest queued task, retries submit	Newest work more important than oldest.

rejection_policies
Java
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
// AbortPolicy (default) - throws exception
ExecutorService abortPool = new ThreadPoolExecutor(
    4, 8, 60, TimeUnit.SECONDS,
    new ArrayBlockingQueue<>(100),
    new ThreadPoolExecutor.AbortPolicy()  // Explicit default
);
 
try {
    abortPool.execute(task);
} catch (RejectedExecutionException e) {
    // Must handle rejection
    logger.error("Task rejected: queue full", e);
}
 
// CallerRunsPolicy - backpressure pattern
ExecutorService backpressurePool = new ThreadPoolExecutor(
    4, 8, 60, TimeUnit.SECONDS,
    new ArrayBlockingQueue<>(100),
    new ThreadPoolExecutor.CallerRunsPolicy()
);
// If queue full, task runs in submitting thread
// This slows down the producer, providing natural flow control
 
// DiscardPolicy - silent drop
ExecutorService bestEffortPool = new ThreadPoolExecutor(
    4, 8, 60, TimeUnit.SECONDS,
    new ArrayBlockingQueue<>(100),
    new ThreadPoolExecutor.DiscardPolicy()
);
// Task is silently dropped - no exception
 
// DiscardOldestPolicy - drop oldest
ExecutorService latestWinsPool = new ThreadPoolExecutor(
    4, 8, 60, TimeUnit.SECONDS,
    new ArrayBlockingQueue<>(100),
    new ThreadPoolExecutor.DiscardOldestPolicy()
);
// Drops oldest queued task, then retries adding new task

Custom Rejection Policies: For complex requirements, implement a custom RejectedExecutionHandler :

custom_rejection
Java
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
// Custom: Log and persist to backup queue
class PersistentRejectionHandler implements RejectedExecutionHandler {
    private final BlockingQueue<Runnable> backupQueue;
    private final Counter rejectionCounter;
    
    @Override
    public void rejectedExecution(Runnable r, ThreadPoolExecutor executor) {
        rejectionCounter.increment();
        
        if (executor.isShutdown()) {
            logger.warn("Task rejected: pool shutdown");
            return;
        }
        
        logger.warn("Task rejected: pool saturated. Queue size: {}", 
            executor.getQueue().size());
        
        // Persist to backup queue for later processing
        if (!backupQueue.offer(r)) {
            logger.error("Backup queue also full! Task lost: {}", r);
            throw new RejectedExecutionException("All queues exhausted");
        }
    }
}
 
// Custom: Block until space available (with timeout)
class BlockingRejectionHandler implements RejectedExecutionHandler {
    private final long timeoutMs;
    
    @Override
    public void rejectedExecution(Runnable r, ThreadPoolExecutor executor) {
        try {
            boolean accepted = executor.getQueue()
                .offer(r, timeoutMs, TimeUnit.MILLISECONDS);
            
            if (!accepted) {
                throw new RejectedExecutionException(
                    "Timed out waiting for queue space");
            }
        } catch (InterruptedException e) {
            Thread.currentThread().interrupt();
            throw new RejectedExecutionException("Interrupted", e);
        }
    }
}

CallerRunsPolicy as Backpressure

CallerRunsPolicy is powerful for flow control. When the pool is saturated, the submitting thread (often handling an incoming request) executes the task directly. This slows down request processing, naturally reducing the arrival rate. Combined with bounded queues, this creates an effective backpressure mechanism.

Queue Ordering and Fairness

The order in which tasks are dequeued affects latency distribution, fairness between producers, and priority handling. Understanding ordering semantics is essential for predictable behavior. FIFO (First-In-First-Out): The most common ordering. Tasks are executed in submission order. Properties: Latency : Bounded by queue depth and service time Fairness : Tasks from all producers treated equally based on arrival time Starvation : No starvation—every task eventually executes Priority : No priority differentiation LIFO (Last-In-First-Out): Recent tasks execute first. Useful for cache-hot processing: Recently touched data more likely in cache Better cache locality, potentially faster execution Old tasks may starve under sustained load Common in work-stealing for forked tasks Priority: Tasks execute in priority order: Critical tasks preempt less important work Low-priority tasks may starve if high-priority keeps arriving Requires Comparable tasks or explicit Comparator O(log n) insertion overhead

queue_ordering
Java
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
// FIFO - Standard ordering
BlockingQueue<Runnable> fifo = new LinkedBlockingQueue<>();
// Tasks executed in submission order
 
// LIFO - Using a Deque as stack
LinkedBlockingDeque<Runnable> lifoDirect = new LinkedBlockingDeque<>();
// Add to front, take from front:
lifoDirect.putFirst(task);
task = lifoDirect.takeFirst();
 
// Priority - Custom comparator
BlockingQueue<Runnable> priority = new PriorityBlockingQueue<>(
    100,
    Comparator.comparing((Runnable r) -> {
        if (r instanceof PrioritizedTask) {
            return ((PrioritizedTask) r).getPriority();
        }
        return 0;  // Default priority
    }).reversed()  // Higher value = higher priority
);
 
// Deadline-based priority
BlockingQueue<Task> deadline = new PriorityBlockingQueue<>(
    100,
    Comparator.comparing(Task::getDeadline)  // Earliest deadline first
);

Fairness in Blocking: When multiple threads contend (producers waiting for queue space, or workers waiting for tasks), the queue may or may not guarantee FIFO ordering of the waiting threads. Non-fair (default): Threads acquire access in arbitrary order Higher throughput due to less synchronization overhead Some threads may wait longer than others ("barging") Fair: Threads acquire access in arrival order Lower throughput due to additional ordering overhead Guaranteed bounded wait time relative to other waiters Available via ArrayBlockingQueue(capacity, fair=true)

Fairness vs Throughput Tradeoff

Fair queues reduce throughput by 10-50% due to strict ordering. Use fair queues only when bounded wait time is critical (e.g., avoiding producer starvation). For most pools, non-fair queues are preferred.

Priority Inversion and Starvation: With priority queues, low-priority tasks may never execute if high-priority tasks keep arriving. This is priority starvation. Mitigation strategies: Aging — Increase task priority over time. After waiting long enough, even low-priority tasks become high-priority. Proportional Scheduling — Reserve some fraction of capacity for each priority level. Separate Pools — Use separate pools for different priorities, with guaranteed capacity for each.

priority_aging
Java
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
// Aging to prevent starvation
class AgingTask implements Runnable, Comparable<AgingTask> {
    private final int basePriority;
    private final long creationTime;
    private final Runnable task;
    private final long agingIntervalMs = 1000;  // Boost every second
    private final int agingBoost = 1;
    
    public AgingTask(int priority, Runnable task) {
        this.basePriority = priority;
        this.creationTime = System.currentTimeMillis();
        this.task = task;
    }
    
    public int getEffectivePriority() {
        long age = System.currentTimeMillis() - creationTime;
        int aging = (int) (age / agingIntervalMs) * agingBoost;
        return basePriority + aging;  // Priority increases over time
    }
    
    @Override
    public int compareTo(AgingTask other) {
        // Higher effective priority = earlier execution
        return Integer.compare(
            other.getEffectivePriority(),
            this.getEffectivePriority()
        );
    }
    
    @Override
    public void run() {
        task.run();
    }
}

Queue Performance Characteristics

Queue choice significantly impacts pool performance. Understanding the performance characteristics of each queue type helps you select appropriately. Operation Complexity:

Queue Operation Complexity
Queue Type	Offer/Put	Poll/Take	Size	Notes
LinkedBlockingQueue	O(1)	O(1)	O(1)	Two-lock; excellent concurrent throughput
ArrayBlockingQueue	O(1)	O(1)	O(1)	Single lock; lower concurrency
SynchronousQueue	O(1)*	O(1)*	O(1) = 0	*Blocks until matched
PriorityBlockingQueue	O(log n)	O(log n)	O(1)	Heap maintenance overhead
DelayQueue	O(log n)	O(log n)	O(1)	Heap ordered by delay
ConcurrentLinkedQueue	O(1)	O(1)	O(n)!	Non-blocking; size() is expensive

Throughput Under Contention: When many threads compete for queue access, lock design becomes critical: LinkedBlockingQueue: Separate locks for head (take) and tail (put) Producers don't block consumers and vice versa Best throughput under high contention ArrayBlockingQueue: Single lock for both operations Producers and consumers compete Worse throughput under contention, but more predictable Lock-Free Queues (ConcurrentLinkedQueue): No locks, uses CAS operations Highest theoretical throughput Not directly usable with ThreadPoolExecutor (not BlockingQueue) Useful for building custom non-blocking pools

queue_benchmark

Benchmark Results

Queue Throughput Benchmark (8 producers, 8 consumers)
Operations per second (higher is better):
 
Queue Type                   | 1KB Tasks | 64B Tasks
-----------------------------|-----------|----------
LinkedBlockingQueue          | 2,100,000 | 4,500,000
ArrayBlockingQueue (unfair)  | 1,400,000 | 3,200,000
ArrayBlockingQueue (fair)    | 800,000   | 1,800,000
SynchronousQueue (unfair)    | 1,900,000 | 4,200,000
SynchronousQueue (fair)      | 1,100,000 | 2,500,000
 
Notes:
- LinkedBlockingQueue excels due to two-lock design
- Fair queues are ~40-50% slower due to ordering overhead
- SynchronousQueue is fast but has no buffering
- Task size affects GC pressure more than queue operations
 
Memory Overhead per queued task:
- LinkedBlockingQueue: ~48 bytes (node object)
- ArrayBlockingQueue: ~0 bytes (pre-allocated array)
- PriorityBlockingQueue: ~16 bytes (heap array slot)

Memory Profile: ArrayBlockingQueue pre-allocates the entire array at construction. Memory is O(capacity) regardless of current size. LinkedBlockingQueue allocates nodes on demand. Memory is O(size), but with per-node overhead. More GC work. PriorityBlockingQueue grows dynamically. Memory is O(size), but with potential reallocation overhead. Choosing Based on Workload:

Queue Selection Guide
Workload	Recommended Queue	Rationale
General purpose	LinkedBlockingQueue (bounded)	Good throughput, flexible sizing
Predictable memory	ArrayBlockingQueue	Pre-allocated, no GC surprises
Latency-critical	SynchronousQueue	No queueing delay
Priority handling	PriorityBlockingQueue	Heap-ordered execution
Scheduled tasks	DelayQueue	Time-based ordering
Need fairness	ArrayBlockingQueue(fair)	Bounded wait ordering

Profile Before Optimizing

Queue overhead is rarely the bottleneck in real applications—task execution usually dominates. Profile your actual workload before optimizing queue choice. The "best" queue depends on your specific access patterns, contention levels, and memory constraints.

Queue Monitoring and Debugging

Effective monitoring of task queues is essential for understanding pool behavior, detecting problems early, and tuning configuration. Here are key metrics and debugging techniques. Essential Queue Metrics:

Metrics to Monitor

•Queue Size — Current number of pending tasks. High values indicate backlog.
•Queue Capacity — Maximum tasks the queue can hold (for bounded queues).
•Queue Utilization — size / capacity. Approaching 100% means rejections imminent.
•Enqueue Rate — Tasks added per second. Indicates load.
•Dequeue Rate — Tasks removed per second. Should match enqueue rate in steady state.
•Queue Wait Time — Time from enqueue to dequeue. Indicates latency.
•Rejection Count — Tasks that couldn't be queued. Non-zero is alarming.

queue_monitoring
Java
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
class MonitoredQueue<E> implements BlockingQueue<E> {
    private final BlockingQueue<E> delegate;
    private final AtomicLong enqueueCount = new AtomicLong();
    private final AtomicLong dequeueCount = new AtomicLong();
    private final AtomicLong rejectionCount = new AtomicLong();
    
    // Metrics registration (e.g., Micrometer)
    public void registerMetrics(MeterRegistry registry) {
        Gauge.builder("pool.queue.size", delegate::size)
            .register(registry);
        Gauge.builder("pool.queue.capacity", delegate::remainingCapacity)
            .register(registry);
        registry.counter("pool.queue.enqueued").count(enqueueCount::get);
        registry.counter("pool.queue.dequeued").count(dequeueCount::get);
        registry.counter("pool.queue.rejected").count(rejectionCount::get);
    }
    
    @Override
    public boolean offer(E e) {
        boolean success = delegate.offer(e);
        if (success) {
            enqueueCount.incrementAndGet();
        } else {
            rejectionCount.incrementAndGet();
        }
        return success;
    }
    
    @Override
    public E poll(long timeout, TimeUnit unit) throws InterruptedException {
        E result = delegate.poll(timeout, unit);
        if (result != null) {
            dequeueCount.incrementAndGet();
        }
        return result;
    }
    
    // ... delegate other methods
}
 
// Using ThreadPoolExecutor's built-in monitoring
ThreadPoolExecutor pool = ...;
 
// Snapshot metrics
int queueSize = pool.getQueue().size();
int poolSize = pool.getPoolSize();
int activeCount = pool.getActiveCount();
long completedTasks = pool.getCompletedTaskCount();
long totalTasks = pool.getTaskCount();
long pendingTasks = totalTasks - completedTasks;

Diagnosing Queue Problems: Problem: Queue Growing Unboundedly Symptoms: Queue size increases steadily, memory usage grows, eventually OOM. Diagnosis: Arrival rate exceeds processing rate. Either: Tasks are taking too long (check task metrics) Pool is undersized (check CPU utilization) Blocking operations in tasks (check thread dumps) Resolution: Increase pool size, optimize tasks, use bounded queue with rejection. Problem: High Rejection Rate Symptoms: Tasks being rejected, rejection counter increasing. Diagnosis: Bounded queue is full and max threads are busy. Resolution: Increase queue capacity, increase max pool size, or implement backpressure. Problem: High Queue Wait Time Symptoms: Tasks wait long before execution, latency spikes. Diagnosis: Queue is deep relative to processing rate. Resolution: Reduce queue capacity (reject faster), increase workers, optimize tasks.

queue_debugging
Java
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
// Debugging: inspect queue contents
public void debugQueueContents(ThreadPoolExecutor pool) {
    BlockingQueue<Runnable> queue = pool.getQueue();
    
    System.out.println("Queue size: " + queue.size());
    System.out.println("Remaining capacity: " + queue.remainingCapacity());
    
    // Peek at queued tasks (don't remove them!)
    int sampleSize = Math.min(10, queue.size());
    int i = 0;
    for (Runnable r : queue) {
        if (i++ >= sampleSize) break;
        System.out.println("  Task: " + describeTask(r));
    }
}
 
private String describeTask(Runnable r) {
    if (r instanceof FutureTask) {
        // FutureTask wraps callable - toString may help
        return "FutureTask: " + r.toString();
    }
    // Custom task types
    if (r instanceof DescriptiveTask) {
        return ((DescriptiveTask) r).getDescription();
    }
    return r.getClass().getSimpleName() + "@" + 
           Integer.toHexString(r.hashCode());
}
 
// Thread dump to see what workers are doing
public void debugWorkers() {
    Map<Thread, StackTraceElement[]> stacks = Thread.getAllStackTraces();
    for (Map.Entry<Thread, StackTraceElement[]> entry : stacks.entrySet()) {
        Thread t = entry.getKey();
        if (t.getName().startsWith("pool-")) {  // Pool worker threads
            System.out.println(t.getName() + " - " + t.getState());
            for (StackTraceElement frame : entry.getValue()) {
                System.out.println("  " + frame);
            }
        }
    }
}

Be Careful with Queue Iteration

Iterating over a BlockingQueue for debugging is safe but not atomic. The queue may change during iteration. For diagnostics only—don't make decisions based on iterated content. Also, calling size() on ConcurrentLinkedQueue is O(n) and can be slow.

Operating SystemsThread Pools

Thread Pools

LevelIntermediate

Duration75 mins

TopicThread Pools

3 / 5

Task Queue

The Buffer Between Work and Workers

In any producer-consumer system, the buffer between producers and consumers is the critical control point. It determines how much work can be pending, how load spikes are absorbed, what happens when consumption can't keep up with production, and how work is prioritized. In thread pools, this buffer is the task queue (also called the work queue). It's far more than a simple list of tasks waiting to be executed. The queue's type, capacity, and ordering policy fundamentally shape the pool's behavior under load—determining whether the system degrades gracefully or catastrophically when stressed. Many thread pool performance problems trace to queue misconfiguration. An unbounded queue can exhaust memory during load spikes. A bounded queue with the wrong rejection policy can drop critical work. A FIFO queue can cause priority inversions. Understanding task queues deeply prevents these failures.

What You Will Learn

By the end of this page, you will understand task queue fundamentals, the various queue types available, capacity and bounding strategies, ordering policies, rejection handling, and the performance implications of queue design. You'll be able to select and configure the right queue for any workload.

Queue Fundamentals

A task queue in a thread pool is a thread-safe data structure that holds tasks waiting for execution. It provides the essential decoupling between task submission (by arbitrary application threads) and task execution (by pool worker threads). Core Requirements: A task queue must satisfy several requirements to work correctly in a thread pool:

Task Queue Requirements

•Thread Safety — Multiple producers may submit tasks concurrently, and multiple workers may dequeue tasks concurrently. All operations must be safe without external synchronization.
•Blocking Support — Workers must be able to wait efficiently when the queue is empty, rather than busy-waiting and consuming CPU.
•Bounded or Unbounded — The queue must implement a clear capacity policy, either unbounded (grow indefinitely) or bounded (reject/block when full).
•Ordering Semantics — The queue must define and maintain task ordering: FIFO, LIFO, priority-based, or other policies.
•Introspection — The queue should expose its current size, remaining capacity, and other metrics for monitoring.

The BlockingQueue Interface (Java): Java's BlockingQueue interface captures these requirements and is the standard abstraction for thread pool queues:

blocking_queue_interface
Java
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
public interface BlockingQueue<E> extends Queue<E> {
    
    // === Producer Operations ===
    
    // Add, throw exception if full
    boolean add(E e);
    
    // Add, return false if full
    boolean offer(E e);
    
    // Add, block until space available
    void put(E e) throws InterruptedException;
    
    // Add, block with timeout
    boolean offer(E e, long timeout, TimeUnit unit) 
        throws InterruptedException;
    
    // === Consumer Operations ===
    
    // Remove and return, block if empty
    E take() throws InterruptedException;
    
    // Remove and return, block with timeout
    E poll(long timeout, TimeUnit unit) 
        throws InterruptedException;
    
    // === Inspection ===
    
    int remainingCapacity();  // Space before full
    int size();               // Current element count
    boolean contains(Object o);
    
    // === Batch Operations ===
    
    // Remove all and add to collection
    int drainTo(Collection<? super E> c);
    int drainTo(Collection<? super E> c, int maxElements);
}

Queue Operations in Thread Pool Context: Producers (task submitters) use offer() (non-blocking) or put() (blocking) to add tasks. Consumers (workers) use take() (blocking) or poll(timeout) to retrieve tasks. Pool manager uses size(), remainingCapacity(), and drainTo() for monitoring and shutdown. The choice between blocking and non-blocking operations affects how the pool behaves when the queue is full (for producers) or empty (for consumers).

The Blocking Wait

When a worker calls take() on an empty queue, it enters an efficient wait state—typically using OS synchronization primitives (futex on Linux, WaitForSingleObject on Windows). The worker consumes no CPU while waiting and is awakened only when work arrives. This is why idle pools have negligible performance impact.

Queue Types

Different queue implementations offer different tradeoffs between performance, ordering, and behavior. Understanding these options is essential for selecting the right queue for your workload.

LinkedBlockingQueue: A FIFO queue backed by a linked list. Can be bounded or unbounded. Characteristics: Ordering : FIFO (First-In-First-Out) Capacity : Optional bound; unbounded if not specified Concurrency : Separate locks for head and tail (two-lock queue) Memory : Node allocation per element (GC overhead) Performance: Better concurrency than ArrayBlockingQueue (separate locks) Additional GC pressure from node objects Excellent for high-throughput producer-consumer When to Use: Default choice for general-purpose pools High concurrency with many producers/consumers When you want unbounded buffering (with caution)

linked_blocking_queue
Java
1
2
3
4
5
6
7
8
9
10
11
// Unbounded LinkedBlockingQueue (careful - can exhaust memory!)
BlockingQueue<Runnable> unbounded = new LinkedBlockingQueue<>();
 
// Bounded LinkedBlockingQueue (recommended)
BlockingQueue<Runnable> bounded = new LinkedBlockingQueue<>(1000);
 
// In ThreadPoolExecutor
ExecutorService pool = new ThreadPoolExecutor(
    4, 4, 0, TimeUnit.SECONDS,
    new LinkedBlockingQueue<>(1000)
);

Queue Capacity and Bounding

The decision between bounded and unbounded queues is one of the most consequential choices in thread pool configuration. Each has significant implications for system behavior under load. Unbounded Queues:

Unbounded Queue Advantages

•Never rejects tasks due to queue capacity
•Absorbs arbitrary traffic bursts
•Simple configuration (no capacity decision)
•Producers never block on queue

Unbounded Queue Dangers

•Can exhaust memory during load spikes
•Masks capacity problems (queue grows silently)
•Latency grows unboundedly as queue builds
•No backpressure signal to producers

Bounded Queues: Bounded queues provide natural backpressure—when the queue is full, producers are forced to slow down (block) or handle rejection. This prevents memory exhaustion and provides early warning of capacity issues. Choosing Queue Capacity: Queue capacity determines how much burst traffic the pool can absorb. Consider the following analysis:

queue_capacity_analysis

Analysis

Given:
  - Pool size: N workers
  - Average task duration: D seconds
  - Maximum acceptable wait time: W seconds
  - Task arrival rate during burst: R tasks/second
 
Analysis:
 
  Pool throughput = N / D tasks/second
 
  Queue drains at rate: N / D
  Queue fills at rate: R (during burst)
 
  If R > N/D, queue grows during burst at rate: R - N/D
 
  To survive a burst of duration B seconds:
    Queue capacity >= (R - N/D) × B
 
  To limit wait time to W seconds:
    Queue capacity <= W × (N / D)
 
Example:
  N = 8 workers
  D = 0.1 seconds (100ms per task)
  Max throughput = 8 / 0.1 = 80 tasks/second
  
  Burst: 200 tasks/second for 10 seconds
  Queue growth rate: 200 - 80 = 120 tasks/second
  Queue needed: 120 × 10 = 1200 tasks
  
  With queue capacity 1200:
  - Queue fills during burst
  - Drains after burst ends
  - No rejections
  
  But: longest wait = 1200 / 80 = 15 seconds
  If max wait is 5 seconds, limit queue to 400 tasks

Memory Considerations: Queue capacity affects memory usage: Each task object consumes memory (typically 64-256 bytes for simple Runnables) Tasks may reference larger objects (request data, result buffers) Queue nodes add overhead (LinkedBlockingQueue: ~40 bytes/node) Rule of Thumb: Start with a bounded queue sized to handle expected burst duration while keeping wait times acceptable. Monitor queue depth in production and adjust based on observed behavior.

The Unbounded Queue Trap

Executors.newFixedThreadPool() uses an unbounded LinkedBlockingQueue. This is convenient but dangerous in production. Under sustained overload, the queue grows until OutOfMemoryError crashes the JVM. Always consider bounded queues with explicit rejection handling for production systems.

Rejection Policies

When a bounded queue is full and the pool cannot accept more tasks, the pool must decide what to do. This decision is encapsulated in the rejection policy (or saturation policy). When Rejection Occurs: A task is rejected when: The work queue is full (cannot accept more tasks) The pool has reached maximum size (cannot create more workers) The pool has been shut down (not accepting new tasks) Standard Rejection Policies (Java):

Built-in Rejection Policies
Policy	Behavior	Use Case
AbortPolicy	Throws RejectedExecutionException	Default. Task submitter must handle failure.
CallerRunsPolicy	Executes task in the calling thread	Backpressure—slows producer when pool is saturated.
DiscardPolicy	Silently discards the task	Best-effort processing, okay to lose work.
DiscardOldestPolicy	Discards oldest queued task, retries submit	Newest work more important than oldest.

rejection_policies
Java
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
// AbortPolicy (default) - throws exception
ExecutorService abortPool = new ThreadPoolExecutor(
    4, 8, 60, TimeUnit.SECONDS,
    new ArrayBlockingQueue<>(100),
    new ThreadPoolExecutor.AbortPolicy()  // Explicit default
);
 
try {
    abortPool.execute(task);
} catch (RejectedExecutionException e) {
    // Must handle rejection
    logger.error("Task rejected: queue full", e);
}
 
// CallerRunsPolicy - backpressure pattern
ExecutorService backpressurePool = new ThreadPoolExecutor(
    4, 8, 60, TimeUnit.SECONDS,
    new ArrayBlockingQueue<>(100),
    new ThreadPoolExecutor.CallerRunsPolicy()
);
// If queue full, task runs in submitting thread
// This slows down the producer, providing natural flow control
 
// DiscardPolicy - silent drop
ExecutorService bestEffortPool = new ThreadPoolExecutor(
    4, 8, 60, TimeUnit.SECONDS,
    new ArrayBlockingQueue<>(100),
    new ThreadPoolExecutor.DiscardPolicy()
);
// Task is silently dropped - no exception
 
// DiscardOldestPolicy - drop oldest
ExecutorService latestWinsPool = new ThreadPoolExecutor(
    4, 8, 60, TimeUnit.SECONDS,
    new ArrayBlockingQueue<>(100),
    new ThreadPoolExecutor.DiscardOldestPolicy()
);
// Drops oldest queued task, then retries adding new task

Custom Rejection Policies: For complex requirements, implement a custom RejectedExecutionHandler :

custom_rejection
Java
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
// Custom: Log and persist to backup queue
class PersistentRejectionHandler implements RejectedExecutionHandler {
    private final BlockingQueue<Runnable> backupQueue;
    private final Counter rejectionCounter;
    
    @Override
    public void rejectedExecution(Runnable r, ThreadPoolExecutor executor) {
        rejectionCounter.increment();
        
        if (executor.isShutdown()) {
            logger.warn("Task rejected: pool shutdown");
            return;
        }
        
        logger.warn("Task rejected: pool saturated. Queue size: {}", 
            executor.getQueue().size());
        
        // Persist to backup queue for later processing
        if (!backupQueue.offer(r)) {
            logger.error("Backup queue also full! Task lost: {}", r);
            throw new RejectedExecutionException("All queues exhausted");
        }
    }
}
 
// Custom: Block until space available (with timeout)
class BlockingRejectionHandler implements RejectedExecutionHandler {
    private final long timeoutMs;
    
    @Override
    public void rejectedExecution(Runnable r, ThreadPoolExecutor executor) {
        try {
            boolean accepted = executor.getQueue()
                .offer(r, timeoutMs, TimeUnit.MILLISECONDS);
            
            if (!accepted) {
                throw new RejectedExecutionException(
                    "Timed out waiting for queue space");
            }
        } catch (InterruptedException e) {
            Thread.currentThread().interrupt();
            throw new RejectedExecutionException("Interrupted", e);
        }
    }
}

CallerRunsPolicy as Backpressure

CallerRunsPolicy is powerful for flow control. When the pool is saturated, the submitting thread (often handling an incoming request) executes the task directly. This slows down request processing, naturally reducing the arrival rate. Combined with bounded queues, this creates an effective backpressure mechanism.

Queue Ordering and Fairness

The order in which tasks are dequeued affects latency distribution, fairness between producers, and priority handling. Understanding ordering semantics is essential for predictable behavior. FIFO (First-In-First-Out): The most common ordering. Tasks are executed in submission order. Properties: Latency : Bounded by queue depth and service time Fairness : Tasks from all producers treated equally based on arrival time Starvation : No starvation—every task eventually executes Priority : No priority differentiation LIFO (Last-In-First-Out): Recent tasks execute first. Useful for cache-hot processing: Recently touched data more likely in cache Better cache locality, potentially faster execution Old tasks may starve under sustained load Common in work-stealing for forked tasks Priority: Tasks execute in priority order: Critical tasks preempt less important work Low-priority tasks may starve if high-priority keeps arriving Requires Comparable tasks or explicit Comparator O(log n) insertion overhead

queue_ordering
Java
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
// FIFO - Standard ordering
BlockingQueue<Runnable> fifo = new LinkedBlockingQueue<>();
// Tasks executed in submission order
 
// LIFO - Using a Deque as stack
LinkedBlockingDeque<Runnable> lifoDirect = new LinkedBlockingDeque<>();
// Add to front, take from front:
lifoDirect.putFirst(task);
task = lifoDirect.takeFirst();
 
// Priority - Custom comparator
BlockingQueue<Runnable> priority = new PriorityBlockingQueue<>(
    100,
    Comparator.comparing((Runnable r) -> {
        if (r instanceof PrioritizedTask) {
            return ((PrioritizedTask) r).getPriority();
        }
        return 0;  // Default priority
    }).reversed()  // Higher value = higher priority
);
 
// Deadline-based priority
BlockingQueue<Task> deadline = new PriorityBlockingQueue<>(
    100,
    Comparator.comparing(Task::getDeadline)  // Earliest deadline first
);

Fairness in Blocking: When multiple threads contend (producers waiting for queue space, or workers waiting for tasks), the queue may or may not guarantee FIFO ordering of the waiting threads. Non-fair (default): Threads acquire access in arbitrary order Higher throughput due to less synchronization overhead Some threads may wait longer than others ("barging") Fair: Threads acquire access in arrival order Lower throughput due to additional ordering overhead Guaranteed bounded wait time relative to other waiters Available via ArrayBlockingQueue(capacity, fair=true)

Fairness vs Throughput Tradeoff

Fair queues reduce throughput by 10-50% due to strict ordering. Use fair queues only when bounded wait time is critical (e.g., avoiding producer starvation). For most pools, non-fair queues are preferred.

Priority Inversion and Starvation: With priority queues, low-priority tasks may never execute if high-priority tasks keep arriving. This is priority starvation. Mitigation strategies: Aging — Increase task priority over time. After waiting long enough, even low-priority tasks become high-priority. Proportional Scheduling — Reserve some fraction of capacity for each priority level. Separate Pools — Use separate pools for different priorities, with guaranteed capacity for each.

priority_aging
Java
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
// Aging to prevent starvation
class AgingTask implements Runnable, Comparable<AgingTask> {
    private final int basePriority;
    private final long creationTime;
    private final Runnable task;
    private final long agingIntervalMs = 1000;  // Boost every second
    private final int agingBoost = 1;
    
    public AgingTask(int priority, Runnable task) {
        this.basePriority = priority;
        this.creationTime = System.currentTimeMillis();
        this.task = task;
    }
    
    public int getEffectivePriority() {
        long age = System.currentTimeMillis() - creationTime;
        int aging = (int) (age / agingIntervalMs) * agingBoost;
        return basePriority + aging;  // Priority increases over time
    }
    
    @Override
    public int compareTo(AgingTask other) {
        // Higher effective priority = earlier execution
        return Integer.compare(
            other.getEffectivePriority(),
            this.getEffectivePriority()
        );
    }
    
    @Override
    public void run() {
        task.run();
    }
}

Queue Performance Characteristics

Queue choice significantly impacts pool performance. Understanding the performance characteristics of each queue type helps you select appropriately. Operation Complexity:

Queue Operation Complexity
Queue Type	Offer/Put	Poll/Take	Size	Notes
LinkedBlockingQueue	O(1)	O(1)	O(1)	Two-lock; excellent concurrent throughput
ArrayBlockingQueue	O(1)	O(1)	O(1)	Single lock; lower concurrency
SynchronousQueue	O(1)*	O(1)*	O(1) = 0	*Blocks until matched
PriorityBlockingQueue	O(log n)	O(log n)	O(1)	Heap maintenance overhead
DelayQueue	O(log n)	O(log n)	O(1)	Heap ordered by delay
ConcurrentLinkedQueue	O(1)	O(1)	O(n)!	Non-blocking; size() is expensive

Throughput Under Contention: When many threads compete for queue access, lock design becomes critical: LinkedBlockingQueue: Separate locks for head (take) and tail (put) Producers don't block consumers and vice versa Best throughput under high contention ArrayBlockingQueue: Single lock for both operations Producers and consumers compete Worse throughput under contention, but more predictable Lock-Free Queues (ConcurrentLinkedQueue): No locks, uses CAS operations Highest theoretical throughput Not directly usable with ThreadPoolExecutor (not BlockingQueue) Useful for building custom non-blocking pools

queue_benchmark

Benchmark Results

Queue Throughput Benchmark (8 producers, 8 consumers)
Operations per second (higher is better):
 
Queue Type                   | 1KB Tasks | 64B Tasks
-----------------------------|-----------|----------
LinkedBlockingQueue          | 2,100,000 | 4,500,000
ArrayBlockingQueue (unfair)  | 1,400,000 | 3,200,000
ArrayBlockingQueue (fair)    | 800,000   | 1,800,000
SynchronousQueue (unfair)    | 1,900,000 | 4,200,000
SynchronousQueue (fair)      | 1,100,000 | 2,500,000
 
Notes:
- LinkedBlockingQueue excels due to two-lock design
- Fair queues are ~40-50% slower due to ordering overhead
- SynchronousQueue is fast but has no buffering
- Task size affects GC pressure more than queue operations
 
Memory Overhead per queued task:
- LinkedBlockingQueue: ~48 bytes (node object)
- ArrayBlockingQueue: ~0 bytes (pre-allocated array)
- PriorityBlockingQueue: ~16 bytes (heap array slot)

Memory Profile: ArrayBlockingQueue pre-allocates the entire array at construction. Memory is O(capacity) regardless of current size. LinkedBlockingQueue allocates nodes on demand. Memory is O(size), but with per-node overhead. More GC work. PriorityBlockingQueue grows dynamically. Memory is O(size), but with potential reallocation overhead. Choosing Based on Workload:

Queue Selection Guide
Workload	Recommended Queue	Rationale
General purpose	LinkedBlockingQueue (bounded)	Good throughput, flexible sizing
Predictable memory	ArrayBlockingQueue	Pre-allocated, no GC surprises
Latency-critical	SynchronousQueue	No queueing delay
Priority handling	PriorityBlockingQueue	Heap-ordered execution
Scheduled tasks	DelayQueue	Time-based ordering
Need fairness	ArrayBlockingQueue(fair)	Bounded wait ordering

Profile Before Optimizing

Queue overhead is rarely the bottleneck in real applications—task execution usually dominates. Profile your actual workload before optimizing queue choice. The "best" queue depends on your specific access patterns, contention levels, and memory constraints.

Queue Monitoring and Debugging

Effective monitoring of task queues is essential for understanding pool behavior, detecting problems early, and tuning configuration. Here are key metrics and debugging techniques. Essential Queue Metrics:

Metrics to Monitor

•Queue Size — Current number of pending tasks. High values indicate backlog.
•Queue Capacity — Maximum tasks the queue can hold (for bounded queues).
•Queue Utilization — size / capacity. Approaching 100% means rejections imminent.
•Enqueue Rate — Tasks added per second. Indicates load.
•Dequeue Rate — Tasks removed per second. Should match enqueue rate in steady state.
•Queue Wait Time — Time from enqueue to dequeue. Indicates latency.
•Rejection Count — Tasks that couldn't be queued. Non-zero is alarming.

queue_monitoring
Java
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
class MonitoredQueue<E> implements BlockingQueue<E> {
    private final BlockingQueue<E> delegate;
    private final AtomicLong enqueueCount = new AtomicLong();
    private final AtomicLong dequeueCount = new AtomicLong();
    private final AtomicLong rejectionCount = new AtomicLong();
    
    // Metrics registration (e.g., Micrometer)
    public void registerMetrics(MeterRegistry registry) {
        Gauge.builder("pool.queue.size", delegate::size)
            .register(registry);
        Gauge.builder("pool.queue.capacity", delegate::remainingCapacity)
            .register(registry);
        registry.counter("pool.queue.enqueued").count(enqueueCount::get);
        registry.counter("pool.queue.dequeued").count(dequeueCount::get);
        registry.counter("pool.queue.rejected").count(rejectionCount::get);
    }
    
    @Override
    public boolean offer(E e) {
        boolean success = delegate.offer(e);
        if (success) {
            enqueueCount.incrementAndGet();
        } else {
            rejectionCount.incrementAndGet();
        }
        return success;
    }
    
    @Override
    public E poll(long timeout, TimeUnit unit) throws InterruptedException {
        E result = delegate.poll(timeout, unit);
        if (result != null) {
            dequeueCount.incrementAndGet();
        }
        return result;
    }
    
    // ... delegate other methods
}
 
// Using ThreadPoolExecutor's built-in monitoring
ThreadPoolExecutor pool = ...;
 
// Snapshot metrics
int queueSize = pool.getQueue().size();
int poolSize = pool.getPoolSize();
int activeCount = pool.getActiveCount();
long completedTasks = pool.getCompletedTaskCount();
long totalTasks = pool.getTaskCount();
long pendingTasks = totalTasks - completedTasks;

Diagnosing Queue Problems: Problem: Queue Growing Unboundedly Symptoms: Queue size increases steadily, memory usage grows, eventually OOM. Diagnosis: Arrival rate exceeds processing rate. Either: Tasks are taking too long (check task metrics) Pool is undersized (check CPU utilization) Blocking operations in tasks (check thread dumps) Resolution: Increase pool size, optimize tasks, use bounded queue with rejection. Problem: High Rejection Rate Symptoms: Tasks being rejected, rejection counter increasing. Diagnosis: Bounded queue is full and max threads are busy. Resolution: Increase queue capacity, increase max pool size, or implement backpressure. Problem: High Queue Wait Time Symptoms: Tasks wait long before execution, latency spikes. Diagnosis: Queue is deep relative to processing rate. Resolution: Reduce queue capacity (reject faster), increase workers, optimize tasks.

queue_debugging
Java
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
// Debugging: inspect queue contents
public void debugQueueContents(ThreadPoolExecutor pool) {
    BlockingQueue<Runnable> queue = pool.getQueue();
    
    System.out.println("Queue size: " + queue.size());
    System.out.println("Remaining capacity: " + queue.remainingCapacity());
    
    // Peek at queued tasks (don't remove them!)
    int sampleSize = Math.min(10, queue.size());
    int i = 0;
    for (Runnable r : queue) {
        if (i++ >= sampleSize) break;
        System.out.println("  Task: " + describeTask(r));
    }
}
 
private String describeTask(Runnable r) {
    if (r instanceof FutureTask) {
        // FutureTask wraps callable - toString may help
        return "FutureTask: " + r.toString();
    }
    // Custom task types
    if (r instanceof DescriptiveTask) {
        return ((DescriptiveTask) r).getDescription();
    }
    return r.getClass().getSimpleName() + "@" + 
           Integer.toHexString(r.hashCode());
}
 
// Thread dump to see what workers are doing
public void debugWorkers() {
    Map<Thread, StackTraceElement[]> stacks = Thread.getAllStackTraces();
    for (Map.Entry<Thread, StackTraceElement[]> entry : stacks.entrySet()) {
        Thread t = entry.getKey();
        if (t.getName().startsWith("pool-")) {  // Pool worker threads
            System.out.println(t.getName() + " - " + t.getState());
            for (StackTraceElement frame : entry.getValue()) {
                System.out.println("  " + frame);
            }
        }
    }
}

Be Careful with Queue Iteration

Iterating over a BlockingQueue for debugging is safe but not atomic. The queue may change during iteration. For diagnostics only—don't make decisions based on iterated content. Also, calling size() on ConcurrentLinkedQueue is O(n) and can be slow.

Task Queue

The Buffer Between Work and Workers

Queue Fundamentals

Queue Types

Queue Capacity and Bounding

Rejection Policies

Queue Ordering and Fairness

Queue Performance Characteristics

Queue Monitoring and Debugging

There’s more to understand.

Task Queue

The Buffer Between Work and Workers

Queue Fundamentals

Queue Types

Queue Capacity and Bounding

Rejection Policies

Queue Ordering and Fairness

Queue Performance Characteristics

Queue Monitoring and Debugging

There’s more to understand.