# Concurrency and Async Optimization for Java

This guide covers Java's threading models, virtual threads, structured concurrency, lock hierarchies, concurrent data structures, and common concurrency performance pitfalls. For thread-level profiling (JFR JavaMonitorEnter, async-profiler lock mode), see the async agent prompt.

## Thread Model

### Platform threads

Traditional Java threads. Each maps 1:1 to an OS thread. Creation involves a kernel call, ~1 MB of stack allocation, and OS scheduling overhead.

| Characteristic | Value |
|---------------|-------|
| Stack size | 512 KB - 1 MB default (`-Xss`) |
| Creation cost | ~1 ms (OS kernel call + stack allocation) |
| Context switch cost | ~1-10 us (kernel mode switch) |
| Max practical count | ~1,000-10,000 per JVM (limited by OS memory for stacks) |
| Scheduling | OS kernel scheduler (preemptive) |

### Virtual threads (JDK 21+)

Lightweight threads managed by the JVM runtime, not the OS kernel. Many virtual threads are multiplexed onto a small pool of platform threads (carrier threads).

| Characteristic | Value |
|---------------|-------|
| Stack size | Starts at ~1 KB, grows/shrinks as needed (stored on heap) |
| Creation cost | ~1 us (JVM-managed, no kernel call) |
| Context switch cost | ~100-200 ns (userspace, no kernel transition) |
| Max practical count | Millions per JVM |
| Scheduling | JVM's ForkJoinPool scheduler (cooperative at I/O points) |

```java
// Creating virtual threads (JDK 21+)
Thread vt = Thread.ofVirtual().start(() -> {
    // runs on a virtual thread
    var result = httpClient.send(request, BodyHandlers.ofString());
    process(result);
});

// Executor that creates a new virtual thread per task
try (var executor = Executors.newVirtualThreadPerTaskExecutor()) {
    List<Future<String>> futures = tasks.stream()
        .map(task -> executor.submit(() -> processTask(task)))
        .toList();
    // collect results
}
```

## Virtual Threads: When and How

### When to use virtual threads

- **I/O-bound workloads**: HTTP clients, database queries, file I/O, message queue consumers. Virtual threads yield automatically at blocking I/O points, allowing other virtual threads to run on the same carrier.
- **High-concurrency servers**: Handling thousands of concurrent connections where each connection spends most of its time waiting for I/O.
- **Replacing thread pools for I/O**: Instead of a bounded thread pool with 200 threads, use virtual threads -- one per task, no pool sizing concerns.

### When NOT to use virtual threads

- **CPU-bound work**: Virtual threads don't make CPU work faster. Use platform thread pools (`ForkJoinPool`, `ThreadPoolExecutor`) for CPU-intensive tasks.
- **Short-lived tasks with no I/O**: The scheduling overhead of virtual threads (even though small) isn't justified if the task completes in microseconds without blocking.

### The pinning problem

A virtual thread is "pinned" to its carrier thread when it enters a `synchronized` block or calls a `native` method that blocks. While pinned, the carrier thread cannot run other virtual threads -- the concurrency benefit is lost.

```java
// BAD: synchronized pins the virtual thread to its carrier
public synchronized String getData() {
    return httpClient.send(request, BodyHandlers.ofString()).body();
    // Virtual thread is pinned for the entire HTTP call
    // Carrier thread is blocked -- cannot serve other virtual threads
}

// GOOD: ReentrantLock does NOT pin
private final ReentrantLock lock = new ReentrantLock();

public String getData() {
    lock.lock();
    try {
        return httpClient.send(request, BodyHandlers.ofString()).body();
        // Virtual thread yields at the HTTP call -- carrier is free
    } finally {
        lock.unlock();
    }
}
```

### Detecting pinning

```bash
# JVM flag to log pinning events
-Djdk.tracePinnedThreads=full    # full stack trace
-Djdk.tracePinnedThreads=short   # one-line summary

# JFR event
jdk.VirtualThreadPinned  # fires when a virtual thread is pinned
```

### Migration guide: platform threads to virtual threads

1. Replace `Executors.newFixedThreadPool(N)` with `Executors.newVirtualThreadPerTaskExecutor()`
2. Replace `synchronized` blocks that contain I/O with `ReentrantLock`
3. Remove thread pool sizing logic (virtual threads don't need pools)
4. **Do NOT pool virtual threads** -- they are cheap to create and should be one-per-task
5. Verify no third-party libraries use `synchronized` around I/O (common in JDBC drivers, legacy HTTP clients)
6. Test under load -- pinning issues only manifest under concurrency

## Structured Concurrency (JDK 21 Preview)

Structured concurrency ensures that child tasks are bounded by the lifetime of their parent scope. When the scope exits, all child tasks are either completed or cancelled.

### StructuredTaskScope.ShutdownOnFailure

Cancels remaining tasks if any task fails:

```java
try (var scope = new StructuredTaskScope.ShutdownOnFailure()) {
    Subtask<User> user = scope.fork(() -> fetchUser(id));
    Subtask<List<Order>> orders = scope.fork(() -> fetchOrders(id));
    Subtask<Recommendations> recs = scope.fork(() -> fetchRecs(id));

    scope.join();           // wait for all tasks
    scope.throwIfFailed();  // propagate first exception

    return buildDashboard(user.get(), orders.get(), recs.get());
}
// All subtasks guaranteed completed or cancelled when scope exits
```

### StructuredTaskScope.ShutdownOnSuccess

Cancels remaining tasks as soon as one succeeds (race pattern):

```java
try (var scope = new StructuredTaskScope.ShutdownOnSuccess<String>()) {
    scope.fork(() -> fetchFromPrimary(key));
    scope.fork(() -> fetchFromReplica(key));
    scope.fork(() -> fetchFromCache(key));

    scope.join();
    return scope.result();  // returns the first successful result
}
```

## CompletableFuture Patterns

### Composition

```java
// Sequential: thenApply (transform result), thenCompose (chain async operations)
CompletableFuture<User> user = fetchUser(id);
CompletableFuture<Dashboard> dashboard = user
    .thenCompose(u -> fetchOrders(u.getId()))       // async chain
    .thenApply(orders -> buildDashboard(orders));    // sync transform

// Concurrent: allOf (wait for all), anyOf (wait for first)
CompletableFuture<Void> all = CompletableFuture.allOf(future1, future2, future3);
all.thenRun(() -> {
    // all three complete -- safe to call .join()
    Result r1 = future1.join();
    Result r2 = future2.join();
    Result r3 = future3.join();
});
```

### Common mistakes

```java
// MISTAKE 1: blocking in thenApply (runs on common ForkJoinPool by default)
future.thenApply(data -> {
    return slowBlockingCall(data);  // blocks a ForkJoinPool thread
});
// FIX: use thenApplyAsync with a dedicated executor
future.thenApplyAsync(data -> slowBlockingCall(data), ioExecutor);

// MISTAKE 2: join() in a loop -- sequential, defeats the purpose
List<String> results = futures.stream()
    .map(CompletableFuture::join)  // blocks on each sequentially
    .toList();
// FIX: wait for all first, then collect
CompletableFuture.allOf(futures.toArray(CompletableFuture[]::new)).join();
List<String> results = futures.stream()
    .map(CompletableFuture::join)  // now instant -- all already complete
    .toList();

// MISTAKE 3: no timeout
future.join();  // blocks forever if the task never completes
// FIX: use orTimeout (JDK 9+)
future.orTimeout(5, TimeUnit.SECONDS)
    .exceptionally(ex -> fallbackValue);
```

## Thread Pools

### ForkJoinPool (work-stealing)

Default for `parallelStream()`, `CompletableFuture.supplyAsync()`, and virtual thread carriers.

| Characteristic | Value |
|---------------|-------|
| Algorithm | Work-stealing: idle threads steal tasks from busy threads' queues |
| Best for | Recursive divide-and-conquer, parallel streams, short CPU-bound tasks |
| Thread count | Default: `Runtime.getRuntime().availableProcessors()` |
| Queue type | Per-thread deques (double-ended queues) |

```java
// Custom ForkJoinPool for CPU-bound work
ForkJoinPool pool = new ForkJoinPool(
    Runtime.getRuntime().availableProcessors(),  // parallelism
    ForkJoinPool.defaultForkJoinWorkerThreadFactory,
    null,    // uncaught exception handler
    true     // asyncMode: FIFO scheduling (better for event-style tasks)
);
```

### ThreadPoolExecutor (bounded)

For explicit control over pool sizing, queue behavior, and rejection policy.

```java
ThreadPoolExecutor pool = new ThreadPoolExecutor(
    corePoolSize,      // threads to keep alive even when idle
    maxPoolSize,        // maximum threads
    60L, TimeUnit.SECONDS,  // idle thread keepalive
    new LinkedBlockingQueue<>(1000),  // bounded work queue
    new ThreadPoolExecutor.CallerRunsPolicy()  // rejection: caller thread runs the task
);
```

### Executors factory methods and their pitfalls

| Factory method | Pool type | Pitfall |
|---------------|-----------|---------|
| `newFixedThreadPool(N)` | N threads, unbounded queue | **Queue grows without bound** if tasks arrive faster than processed. Can cause OOM. |
| `newCachedThreadPool()` | 0 to Integer.MAX_VALUE threads | **Unbounded thread creation** under load. Each new task can create a new thread. |
| `newSingleThreadExecutor()` | 1 thread, unbounded queue | Same unbounded queue problem as fixed pool. |
| `newScheduledThreadPool(N)` | N threads + delayed queue | Queue is unbounded. Schedule at fixed rate can accumulate tasks if execution exceeds period. |
| `newVirtualThreadPerTaskExecutor()` | Virtual threads (JDK 21+) | No pooling needed but ensure tasks don't use `synchronized` with I/O. |

**Recommendation**: For production code, use `ThreadPoolExecutor` directly with explicit bounded queue and rejection policy. The factory methods are convenient but hide dangerous defaults.

## Lock Hierarchy

From coarsest (simplest, most overhead) to finest (most complex, least overhead):

### synchronized (intrinsic lock)

```java
public synchronized void update() { ... }
// or
synchronized (lockObject) { ... }
```

- Simplest API, built into the language
- Reentrant (same thread can re-acquire)
- Non-interruptible (cannot cancel a thread waiting for the lock)
- No try-lock or timed-lock capability
- **Pins virtual threads** -- avoid in virtual thread contexts

### ReentrantLock

```java
private final ReentrantLock lock = new ReentrantLock();

public void update() {
    lock.lock();
    try {
        // critical section
    } finally {
        lock.unlock();
    }
}
```

- Interruptible: `lock.lockInterruptibly()`
- Timed: `lock.tryLock(5, TimeUnit.SECONDS)`
- Fair mode: `new ReentrantLock(true)` -- FIFO ordering, higher overhead
- **Does NOT pin virtual threads** -- preferred replacement for `synchronized`

### StampedLock (JDK 8+)

```java
private final StampedLock sl = new StampedLock();

// Optimistic read (no lock acquired -- just a version check)
public double distanceFromOrigin() {
    long stamp = sl.tryOptimisticRead();
    double x = this.x, y = this.y;
    if (!sl.validate(stamp)) {  // check if a write happened
        stamp = sl.readLock();  // fall back to read lock
        try {
            x = this.x;
            y = this.y;
        } finally {
            sl.unlockRead(stamp);
        }
    }
    return Math.sqrt(x * x + y * y);
}

// Write lock
public void move(double deltaX, double deltaY) {
    long stamp = sl.writeLock();
    try {
        x += deltaX;
        y += deltaY;
    } finally {
        sl.unlockWrite(stamp);
    }
}
```

- Supports optimistic reads (no lock acquired -- readers check a version stamp)
- Read-write separation: multiple concurrent readers, exclusive writer
- **Not reentrant** -- do not call writeLock() while holding a read lock
- Highest performance for read-heavy workloads with infrequent writes

### Lock-free (Atomic classes, VarHandle)

```java
private final AtomicInteger counter = new AtomicInteger(0);

public int increment() {
    return counter.incrementAndGet();  // CAS-based, no lock
}

// AtomicReference for compare-and-swap on objects
private final AtomicReference<State> state = new AtomicReference<>(INITIAL);

public boolean transition(State expected, State next) {
    return state.compareAndSet(expected, next);
}
```

- No locks, no blocking, no deadlocks
- Uses CPU CAS (compare-and-swap) instructions
- Best for simple counters, flags, and state machines
- Complex lock-free algorithms are extremely hard to get right -- prefer higher-level constructs

### VarHandle (JDK 9+)

Low-level memory access with explicit ordering semantics. Replacement for `Unsafe` field access.

```java
private static final VarHandle COUNT;
static {
    try {
        COUNT = MethodHandles.lookup().findVarHandle(MyClass.class, "count", int.class);
    } catch (ReflectiveOperationException e) {
        throw new ExceptionInInitializerError(e);
    }
}
private volatile int count;

public void increment() {
    COUNT.getAndAdd(this, 1);  // atomic increment via VarHandle
}
```

## Concurrent Data Structures

### ConcurrentHashMap

The workhorse concurrent map. Reads are lock-free. Writes lock only the affected bucket (segment-level locking was replaced with per-bin locking in JDK 8).

```java
// compute-family methods: atomic read-modify-write
ConcurrentHashMap<String, AtomicLong> counters = new ConcurrentHashMap<>();

// computeIfAbsent: atomic initialization
counters.computeIfAbsent(key, k -> new AtomicLong()).incrementAndGet();

// merge: atomic combine
map.merge(key, 1L, Long::sum);  // increment or initialize to 1

// compute: atomic full read-modify-write
map.compute(key, (k, v) -> v == null ? 1L : v + 1);
```

**Warning**: The lambda passed to `compute*` methods runs under the bin lock. Keep it fast -- no I/O, no heavy computation inside the lambda.

### CopyOnWriteArrayList

```java
// Every write (add, set, remove) copies the entire internal array
CopyOnWriteArrayList<Handler> handlers = new CopyOnWriteArrayList<>();

// Reads are lock-free and iterate over a snapshot
for (Handler h : handlers) {  // snapshot iterator -- never throws ConcurrentModificationException
    h.handle(event);
}

// Writes copy the array -- O(n) per write
handlers.add(newHandler);  // copies entire array
```

**Use only when**: Reads vastly outnumber writes (event listener lists, configuration registries). Each write is O(n) in list size.

### ConcurrentLinkedQueue

Lock-free FIFO queue based on CAS operations. Unbounded.

```java
ConcurrentLinkedQueue<Task> queue = new ConcurrentLinkedQueue<>();
queue.offer(task);       // non-blocking add
Task t = queue.poll();   // non-blocking remove (null if empty)
```

### BlockingQueue variants

| Implementation | Bounded | Ordering | Use case |
|---------------|---------|----------|----------|
| `ArrayBlockingQueue` | Yes (fixed capacity) | FIFO | Producer-consumer with backpressure |
| `LinkedBlockingQueue` | Optional (default unbounded) | FIFO | High-throughput producer-consumer |
| `PriorityBlockingQueue` | No (unbounded) | Priority order | Task scheduling by priority |
| `SynchronousQueue` | Zero capacity | Direct handoff | Thread rendezvous (no buffering) |
| `LinkedTransferQueue` | No (unbounded) | FIFO | High-performance producer-consumer (JDK 7+) |

```java
// Bounded producer-consumer with backpressure
BlockingQueue<Task> queue = new ArrayBlockingQueue<>(1000);

// Producer blocks if queue is full
queue.put(task);           // blocks until space available
queue.offer(task, 5, TimeUnit.SECONDS);  // blocks with timeout

// Consumer blocks if queue is empty
Task t = queue.take();     // blocks until task available
Task t = queue.poll(5, TimeUnit.SECONDS);  // blocks with timeout
```

## Lock Contention Diagnosis

### JFR events

```bash
# Record lock contention events
jcmd <pid> JFR.start name=locks settings=profile duration=60s filename=/tmp/locks.jfr
```

Key events:
- `jdk.JavaMonitorEnter`: Thread waited to acquire a `synchronized` lock. High counts or long durations indicate contention.
- `jdk.JavaMonitorWait`: Thread called `Object.wait()`. Long durations may indicate inefficient signaling.
- `jdk.ThreadPark`: Thread parked (waiting on LockSupport.park, ReentrantLock, etc.).

### Thread dumps

```bash
# Take a thread dump
jcmd <pid> Thread.print
# or
kill -3 <pid>  # prints to stderr
# or
jstack <pid>
```

Look for multiple threads in `BLOCKED` state waiting on the same monitor. The thread holding the monitor is in `RUNNABLE` state inside the synchronized block.

### async-profiler lock mode

```bash
# Profile lock contention
./profiler.sh -d 30 -e lock -f /tmp/lock-profile.html <pid>
```

Shows a flamegraph of where threads spend time waiting for locks. The hottest frames indicate the most contended locks.

## Reactive Patterns: Reactor / RxJava

### When to use reactive

- High-concurrency, non-blocking I/O with complex composition (fan-out, retry, timeout, circuit-breaking)
- Streaming data processing with backpressure
- Already using a reactive framework (Spring WebFlux, Vert.x)

### When to prefer virtual threads instead

- Simple request-response patterns (fetch data, process, return)
- Code that reads sequentially (virtual threads allow sequential style without blocking)
- Team unfamiliar with reactive programming (steep learning curve)
- Debugging and stack traces matter (reactive stack traces are notoriously unhelpful)

### Performance comparison

| Dimension | Virtual threads | Reactive (Reactor/RxJava) |
|-----------|----------------|--------------------------|
| Throughput (I/O-bound) | Similar | Similar |
| Latency (simple request) | Lower (no operator overhead) | Higher (operator chain overhead) |
| Memory per concurrent task | ~1 KB (stack) | ~0.5-2 KB (subscription chain) |
| Debugging | Full stack traces | Truncated, assembly-required |
| Backpressure | Manual (bounded queues) | Built-in (Reactive Streams) |
| Composition | Sequential code | Operator chains (map, flatMap, zip) |

**Decision**: For new JDK 21+ projects, prefer virtual threads with structured concurrency for most use cases. Use reactive libraries when you need built-in backpressure, complex event stream composition, or are extending an existing reactive codebase.

## Pitfalls

- **Double-checked locking without volatile**: The classic broken pattern. The field MUST be `volatile` (or use `AtomicReference`/`VarHandle`) for double-checked locking to work correctly under the Java Memory Model.
- **Thread.sleep() in virtual threads**: Works correctly (yields the carrier thread) but `Thread.sleep(0)` does NOT yield -- use `Thread.yield()` for explicit yielding.
- **Parallel streams for I/O**: Parallel streams use the common ForkJoinPool (default: CPU count threads). Blocking I/O in parallel streams can exhaust this shared pool and affect ALL parallel stream users in the JVM.
- **ConcurrentHashMap.size() is O(n)**: Traverses all bins. Use `mappingCount()` for a `long` estimate, or maintain a separate counter.
- **Unbounded Executors**: `newCachedThreadPool()` and `newFixedThreadPool(N)` with default unbounded queue can cause OOM under load. Always use bounded queues and explicit rejection policies in production.
- **Virtual thread pooling**: Do NOT pool virtual threads. They are designed to be created per-task. Pooling adds overhead and defeats the purpose.
- **synchronized in virtual thread hot path**: Every `synchronized` block that contains a blocking call (I/O, sleep, lock acquisition) pins the virtual thread to its carrier. This silently degrades throughput under load. Replace with `ReentrantLock`.
