Process vs Thread vs Goroutine

Section: Core Mental Models | Difficulty: Basic | Frequency: Asked Often

A process owns memory and resources; threads share the process's address space and are scheduled by the OS; goroutines are user-space tasks multiplexed onto a small pool of OS threads by the Go runtime.

Key Points

• Process: isolated address space, expensive context switch (~tens of μs), IPC required to share data
• OS thread: ~1 MB stack, context switch ~1–10 μs, kernel-scheduled
• Goroutine: ~2 KB initial stack (grows), context switch ~hundreds of ns, runtime-scheduled
• Java virtual threads (21+): JVM-managed, M:N model, millions per JVM
• Python threads share the GIL, only one executes Python bytecode at a time
• Prefer processes for CPU-bound parallelism in Python; threads for I/O

Interview Follow-up Questions

Gotchas

• Goroutines started in a loop without context cancellation are the #1 source of leaks in production Go services
• Virtual threads pin to their carrier inside synchronized blocks, prefer ReentrantLock for I/O-heavy code
• Python threads do NOT give CPU parallelism, measure and switch to multiprocessing if compute-bound
• Mixing asyncio and threading is a footgun, use asyncio.to_thread() for blocking calls
• runtime.NumGoroutine() growing unbounded over time = leak; alert on it

Concurrency vs Parallelism

Section: Core Mental Models | Difficulty: Basic | Frequency: Asked Often

Concurrency is structuring a program to deal with many things at once; parallelism is actually doing many things at the same time. One can exist without the other.

Key Points

• Concurrency = composition of independently executing tasks (a structure)
• Parallelism = simultaneous execution of multiple tasks (an outcome)
• Single-core CPU + threads = concurrency without parallelism (interleaved)
• GPU compute or multi-core SIMD = parallelism without classical concurrency
• Web server handling 10K connections = concurrency; matrix multiplication on 8 cores = parallelism
• Async/event loops give concurrency without threads, single-threaded but interleaved

Interview Follow-up Questions

Gotchas

• More threads != more parallelism, once thread count exceeds core count, the cost is context-switching without throughput gain
• asyncio is single-threaded, a CPU-bound task in an async function blocks the entire event loop
• Thread pools sized for I/O are wrong for CPU-bound work and vice versa

Thread & Goroutine Lifecycle

Section: Core Mental Models | Difficulty: Basic | Frequency: Asked Often

A Java thread moves through NEW → RUNNABLE → BLOCKED/WAITING/TIMED_WAITING → TERMINATED. A goroutine has no public state, it runs until its function returns; the runtime parks/unparks it on channel and lock operations. Python threads expose start/join/is_alive but no fine-grained state.

Key Points

• Java thread states: NEW, RUNNABLE, BLOCKED (waiting for monitor), WAITING, TIMED_WAITING, TERMINATED
• RUNNABLE in Java covers both 'running' and 'ready to run', the JVM doesn't expose CPU-on/off
• BLOCKED = waiting for a synchronized monitor; WAITING = parked via wait/join/park
• Goroutines have no public state, they're either running, runnable, or parked
• Cancellation in Go is via context.Context, propagated explicitly through call chains
• Cancellation in Java is via Thread.interrupt(), checked cooperatively
• Cancellation in Python is via threading.Event polling or asyncio.Task.cancel()

Interview Follow-up Questions

Gotchas

• Thread.sleep() and Object.wait() CLEAR the interrupt flag, must re-set with Thread.currentThread().interrupt()
• Forgetting wg.Add() before 'go' creates a race with wg.Wait(), Add must be called from the parent goroutine
• context.WithCancel without calling cancel() leaks the context's goroutine
• Daemon threads (Java setDaemon, Python daemon=True) are killed at shutdown, avoid for cleanup work
• Calling notify() vs notifyAll(), notify wakes ONE waiter; spurious wakeups require waiting in a loop
• Swallowing asyncio.CancelledError prevents shutdown, re-raise after cleanup

Critical Sections & Shared State

Section: Core Mental Models | Difficulty: Basic | Frequency: Asked Often

A critical section is the part of code that touches shared mutable state and must run without interleaving from other threads. Identify it first, then pick a primitive (mutex, atomic, channel) to protect it. Most concurrency bugs come from missing or oversized critical sections.

Key Points

• Shared mutable state is the source of every race. Remove sharing or remove mutation, and the problem disappears.
• A critical section is any code path that reads-then-writes (or writes-then-reads) shared state and depends on the data not changing in between.
• Two failures to avoid: forgetting to protect a critical section, and protecting too much (locking around I/O).
• The right size of a critical section is the smallest set of statements that preserve the invariant in question.

Interview Follow-up Questions

Race Conditions, Why Concurrent Bugs Happen

Section: Core Mental Models | Difficulty: Basic | Frequency: Asked Often

A race condition is when the program's correctness depends on the unpredictable interleaving of two or more threads' operations on shared state. Fix it by removing the sharing, or by making the operations atomic with a lock or atomic primitive.

Key Points

• A race condition is a CORRECTNESS bug, not a 'maybe slow' bug, the wrong answer can be observed
• Three ingredients: shared mutable state, multiple threads, unsynchronized access
• Remove ANY one of the three and the race goes away
• Read-modify-write operations like counter++ are NEVER atomic, they're three instructions
• Compilers and CPUs reorder reads/writes, what looks sequential in source isn't
• Race conditions hide on x86 (strong memory model) and explode on ARM

Interview Follow-up Questions

Gotchas

• Tests almost always pass, races need real load and timing to manifest
• x86 hides races that ARM exposes, test on both
• Even atomic.Add doesn't help with check-then-act: if (atomic.Load() == 0) atomic.Store(1) is racy
• The Go race detector slows code 5-10x, run it in CI, not production
• Java's volatile is for visibility, not atomicity of compound ops

Thread Safety Levels

Section: Core Mental Models | Difficulty: Basic | Frequency: Asked Often

Code is not thread-safe or unsafe in one bucket. There are levels: immutable, thread-confined, conditionally thread-safe, fully thread-safe, and lock-free. Knowing the level a class promises is the difference between safe code and a postmortem.

Key Points

• Immutable: no writes after construction. Always safe to share.
• Thread-confined: only ever touched by one thread. Safe by isolation, not by locking.
• Conditionally thread-safe: individual methods are safe, but compound operations need external synchronisation.
• Fully thread-safe: every method and every reasonable sequence of calls is safe under concurrent use.
• Lock-free: thread-safe AND no thread can be permanently blocked by another. Stronger than fully thread-safe.

Interview Follow-up Questions

Mutex, Semaphore, Condition Variable

Section: Core Mental Models | Difficulty: Intermediate | Frequency: Asked Often

A mutex enforces 'one at a time'. A semaphore counts permits, N threads at a time. A condition variable lets threads wait for a state change. They're the three primitives every higher-level concurrency tool is built on.

Key Points

• Mutex = binary lock, mutual exclusion, ONE owner
• Semaphore = counted permits, allows N concurrent holders
• Condition variable = wait until a predicate becomes true, used WITH a mutex
• Mutex 'protects' invariants; semaphore 'limits' concurrency; condition variable 'signals' state changes
• Always wait on a condition variable in a while-loop, never if (spurious wakeups + stolen wakeups)
• Counting semaphore (N>1) for resource pools; binary semaphore (N=1) is 'almost a mutex' but lacks ownership

Interview Follow-up Questions

Gotchas

• Forgetting to release a lock on the error path → permanent deadlock. Always use defer/finally/with.
• Using if instead of while around await/wait → spurious + stolen wakeups corrupt invariants
• Locking and waiting on different lock instances, the code looks locked but isn't
• Calling signal() without holding the lock, the wait/check race is back
• Counting semaphore release without prior acquire → permit leak (raises in BoundedSemaphore)
• Mutex copy in Go (passing a struct by value), go vet catches it

Synchronisation vs Coordination

Section: Core Mental Models | Difficulty: Intermediate | Frequency: Sometimes

Synchronisation protects shared state. Coordination orchestrates the order or timing of work. Mutexes synchronise. Channels, latches, barriers, futures coordinate. Mixing them up leads to overcomplicated code that locks where it should signal, and signals where it should lock.

Key Points

• Synchronisation: 'no two threads in this section at once'. Mutex, RWLock, atomic, synchronized.
• Coordination: 'do this after that'. Channel, latch, barrier, future, semaphore, condition variable.
• Misapplication: using a mutex to wait for an event (busy-loop with mutex). Or using a channel to protect a variable (overkill).
• Channels are coordination AND synchronisation in one move because the send happens-before the receive.
• When in doubt, ask what's actually needed. Mutual exclusion of access? Or sequencing of work?

Interview Follow-up Questions

Optimistic vs Pessimistic Concurrency Control

Section: Core Mental Models | Difficulty: Intermediate | Frequency: Asked Often

Pessimistic: lock the data while working on it; nobody else can touch it. Cheap when contention is high, costs a lock acquire on every operation. Optimistic: read freely, do the work, commit only if nothing else changed (CAS, version number, MVCC). Nearly free when contention is low, requires retry under contention. Pick pessimistic when conflicts are common, optimistic when they're rare.

Key Points

• Pessimistic = 'assume conflict, prevent it' (locks). Optimistic = 'assume no conflict, detect it' (CAS, version checks).
• Pessimistic cost is constant per operation regardless of contention; optimistic cost is near-zero under low contention but grows (retries) under high contention.
• Optimistic concurrency uses one of three mechanisms: CAS on a memory word, a version column / ETag in a row, or MVCC snapshots in a database.
• The break-even point is the conflict rate. Roughly: under 10% conflict, optimistic wins. Above 50% conflict, pessimistic wins. In between, measure.
• Optimistic doesn't avoid contention, it relocates it: instead of waiting for the lock, the caller waits for a successful retry. The work done before the failed commit is wasted.
• Hybrid systems pick per-operation: PostgreSQL's MVCC is optimistic for reads (snapshots) and pessimistic for writes (row locks). HTTP If-Match uses optimistic versioning at the protocol layer.

Interview Follow-up Questions

Gotchas

• Optimistic without retry: a single CAS failure becomes a user-visible error. Always handle the false return, either by retrying or surfacing a conflict.
• Pessimistic + user think time: holding a row lock across a UI session is a deadlock factory. Move locks to commit time (optimistic) or shrink the lock window.
• Mixing optimistic and pessimistic on the same data without coordination: an optimistic writer can win the race against a pessimistic reader's expected ordering.
• Versioning a row but never bumping the version on every write path: a code path that forgets to increment version silently breaks optimistic checks.
• Counting CAS failure as a 'lock' in profiles: it isn't; millions of failed CAS in an atomic counter means contention to fix, not lock-elimination work to do.

Deadlock, Livelock, Starvation

Section: Core Mental Models | Difficulty: Intermediate | Frequency: Asked Often

Deadlock, threads block each other forever. Livelock, threads are running but making no progress. Starvation, one thread never gets the resource it needs because others keep grabbing it first. All three look similar from the outside; the fix is different for each.

Key Points

• Deadlock = circular wait, Thread A holds X waits for Y, Thread B holds Y waits for X
• Coffman conditions: mutual exclusion + hold-and-wait + no preemption + circular wait
• Break ANY ONE of the four → no deadlock
• Lock ordering (always acquire locks in a fixed global order) is the most common deadlock prevention
• Livelock = threads keep retrying and stepping on each other, no blocking, no progress
• Starvation = unfair scheduling, a thread is permanently outrun by others

Interview Follow-up Questions

Gotchas

• Reentrant locks (Java ReentrantLock, Python RLock) prevent self-deadlock but NOT mutual deadlock between threads
• Holding a lock while calling user-supplied code (a callback, a remote service) is a deadlock waiting to happen, that code might try to take a held lock
• Go's runtime detects only 'all goroutines asleep' deadlocks, partial deadlocks (some goroutines stuck, others running) are silent
• Java's jstack and -XX:+PrintConcurrentLocks show deadlocked threads, use them in production hangs
• Database transactions can deadlock too, ORDER BY-then-FOR UPDATE on different tables is the SQL version

Spurious and Stolen Wakeups

Section: Core Mental Models | Difficulty: Intermediate | Frequency: Asked Often

A waiting thread can wake up without anyone calling notify (spurious), or be beaten to the lock by another thread that consumed the condition (stolen). The defence is to always check the condition in a while loop after wait returns, never if.

Key Points

• Spurious wakeup: the OS or runtime wakes a waiting thread for no reason. Allowed by every POSIX-shaped wait API.
• Stolen wakeup: someone signalled, the thread woke up, but another thread grabbed the resource before the first could re-acquire the lock.
• Both are why every pthread / Java / Python tutorial says 'wait inside while, never if.'
• wait/await releases the lock atomically, parks the thread, re-acquires the lock on wake. The re-check happens after re-acquire.
• Channels and semaphores avoid the issue because they have an integer count, not a free-form predicate.

Interview Follow-up Questions

Priority Inversion

Section: Core Mental Models | Difficulty: Advanced | Frequency: Sometimes

A high-priority thread waits on a lock held by a low-priority thread. A medium-priority thread runs, preempting the low-priority holder, which means the high-priority thread also waits indirectly. The fix is priority inheritance: bump the holder's priority to match the highest waiter.

Key Points

• Three threads, one lock. High waits on Low for the lock. Medium runs, preempting Low. High is blocked indefinitely.
• Famous case: Mars Pathfinder, 1997. The rover kept rebooting due to a priority inversion in the VxWorks scheduler.
• Priority inheritance: while Low holds the lock, raise its priority to High's so Medium cannot preempt.
• Common in real-time systems, audio drivers, embedded code, kernel synchronisation. Less common in Java/Go/Python.
• JVM and most modern OS schedulers do not implement priority inheritance by default; problem solved by avoiding strict thread priorities in the first place.

Interview Follow-up Questions

Memory Visibility & CPU Reordering

Section: Core Mental Models | Difficulty: Advanced | Frequency: Asked Often

Modern CPUs and compilers reorder reads and writes for performance. Without explicit synchronization, one thread's writes may NEVER become visible to another thread, or may appear in a different order. This is the deepest reason concurrent code breaks, and skipping it means writing code that "works on a laptop, fails in production."

Key Points

• Skipping this lesson leads to broken concurrent code
• CPUs have per-core caches, a write on Core 1 may sit in Core 1's cache forever, invisible to Core 2
• Compilers reorder instructions for optimization; CPUs reorder them for pipelining
• The 'obvious sequential order' of source code is not what runs on the hardware
• x86 has a strong memory model and hides most reordering bugs; ARM and POWER expose them
• ALL modern languages have a memory model, Java JMM, Go, C++11, even Python (informally)
• Synchronization primitives (volatile, atomic, mutex, channel) insert MEMORY BARRIERS that flush caches and prevent reordering

Interview Follow-up Questions

Gotchas

• Tests pass on x86 dev laptops, fail on ARM CI runners, TEST ON BOTH
• Compiler optimizations can hoist a volatile-less read out of a loop entirely → 'why is my thread spinning forever?'
• long/double on 32-bit JVMs aren't atomic without volatile, torn reads possible
• Java: writing to a field after publishing a reference to its containing object can be invisible to readers who don't synchronize
• Go: copying a sync.Mutex breaks it (go vet catches); copying a struct with atomic fields breaks them
• Python's GIL gives bytecode-level atomicity but NOT visibility for the publication pattern

Linearizability and Sequential Consistency

Section: Core Mental Models | Difficulty: Advanced | Frequency: Sometimes

Linearizability says every operation appears to take effect at one instant between its call and its return, and that instant respects real time across threads. Sequential consistency is weaker: each thread's operations stay in program order and there is some single global order, but it doesn't have to match wall-clock time. Linearizability composes across objects; sequential consistency does not.

Key Points

• Linearizability: each operation has a single 'effect point' that lies between its invocation and its response, and the order of effect points is consistent with real time.
• Sequential consistency: there exists some interleaving of all operations that respects each thread's program order, but the interleaving does not have to match real time.
• Linearizability is composable: if every object is linearizable, the system is linearizable. Sequential consistency is not.
• Java volatile reads and writes on a single variable are linearizable; combining multiple volatiles or doing read-modify-write is not. Use java.util.concurrent.atomic for linearizable RMW.
• Redis is not linearizable by default. etcd, ZooKeeper, and DynamoDB conditional writes are.
• Linearizability has a real cost: in an async system with failures, you can't have linearizable reads with sub-RTT latency (Attiya and Welch, 1994).

Interview Follow-up Questions

Gotchas

• Composing two sequentially consistent objects can give a non-SC system. Composing two linearizable objects always gives a linearizable system.
• A linearizable register is what most papers mean by 'atomic register.' The terms are interchangeable in the literature.
• Don't conflate linearizability with serializability. Linearizability is about single-object real-time order. Serializability is about multi-operation transactions.
• A snapshot read across two unrelated linearizable objects is not itself linearizable. A transaction or a globally-ordered log is required for that.

Atomic Operations & CAS

Section: Core Mental Models | Difficulty: Advanced | Frequency: Asked Often

Atomic operations are indivisible CPU instructions that read-modify-write a single memory location safely, without locks. Compare-and-swap (CAS) is the foundation, "if this value is still X, replace it with Y", and it's how every lock-free data structure is built.

Key Points

• An atomic operation is a single CPU instruction that can't be interrupted partway through
• CAS = compare-and-swap: 'if memory[addr] == expected, write new and return success; else fail and return current value'
• Lock-free programming uses CAS in retry loops, never blocks, but may spin under contention
• Atomics give you single-variable safety; for multi-variable invariants you still need a lock
• ABA problem: a value goes A → B → A while you're not looking; CAS thinks nothing changed
• LongAdder/striped counters trade memory for throughput under high contention

Interview Follow-up Questions

Gotchas

• ABA problem on linked-list CAS, value comes back to original after intermediate change
• Reading an atomic field non-atomically (`int x = atomic.Load(); other code; use x`), value can change between read and use
• AtomicInteger.incrementAndGet() is atomic; AtomicInteger.get() + 1 followed by set() is NOT
• Hot CAS loop on a single AtomicLong from many threads = CPU bottleneck, switch to LongAdder
• Java: AtomicInteger.lazySet vs set, lazySet doesn't establish happens-before, can lose visibility
• Go: atomic operations require *value pointer (& prefix); using value-by-value is silently wrong

False Sharing

Section: Core Mental Models | Difficulty: Advanced | Frequency: Sometimes

Two unrelated variables can sit on the same CPU cache line. When two threads write to them concurrently, the cache line ping-pongs between cores even though the variables are not actually shared. The code is correct but slow. Fix by padding to one variable per cache line.

Key Points

• CPU cache lines are 64 bytes on x86 and ARM. The smallest unit of cache coherence is the line, not the variable.
• If two threads write variables in the same line, every write invalidates the other core's cache copy.
• Symptom: linear-looking code that does not scale with cores. Adding threads makes it slower.
• Fix: pad each hot variable to a full cache line, or use frameworks (LongAdder, padded atomics) that already do it.
• Most common in counters, queues, lock state, per-thread statistics.

Interview Follow-up Questions

Backpressure: Strategies and Signals

Section: Core Mental Models | Difficulty: Intermediate | Frequency: Asked Often

When the producer is faster than the consumer, something has to give. Backpressure is how the system tells the producer to slow down (or what happens when it can't). Four strategies: block the producer, drop new work, shed load with errors, or propagate the slowdown back through the call chain. Picking the wrong one is how you turn a slow consumer into a memory blowup or a thundering retry storm.

Key Points

• Without backpressure, a slow consumer forces unbounded memory growth or work loss
• Block the producer (bounded queue blocks on full): natural backpressure, default for most pipelines
• Drop incoming (drop newest or drop oldest): for telemetry where stale data is worse than missing data
• Shed load (return 503 / fail fast): for request paths where the client can retry or degrade
• Propagate (the slow downstream signals the upstream caller): for end-to-end flow control across services

Interview Follow-up Questions

Gotchas

• Unbounded queue = no backpressure = memory blowup under sustained overload
• Drop without metrics = silent data loss (count and alert on drops)
• Shed without Retry-After = client retries immediately, makes things worse
• Block in a request handler = thread starvation; combine with timeouts
• Propagate without timeouts = nothing actually propagates; ctx.Done is the channel

The Colored Function Problem

Section: Core Mental Models | Difficulty: Intermediate | Frequency: Sometimes

In some languages (Python, JavaScript, Rust), async functions and sync functions are different colours. Sync code can't call async without an event loop; async code can't call sync blocking calls without freezing the loop. The colour propagates upward through every caller. Go and Java 21+ virtual threads avoid the problem by making the runtime handle async transparently.

Key Points

• Python / JavaScript / Rust: async is a function colour. Async can call sync; sync cannot directly call async (needs a loop).
• Once async is added at any layer, every caller above must also be async, or pay an awkward bridge cost.
• Go avoids it: every function looks blocking; the runtime parks goroutines on I/O. No async/await syntax.
• Java 21+ virtual threads do the same: write blocking-style code; the JVM provides async-like efficiency.
• Cross-colour bridging exists (asyncio.to_thread, asyncio.run, asgiref.sync) but it's friction, not a clean solution.

Interview Follow-up Questions

Gotchas

• Calling an async function in sync code without a loop returns a coroutine/Promise, not a value
• Calling sync blocking I/O in async code freezes the entire event loop
• Mixing thread-based code (threading.Lock) with asyncio is the worst of both worlds
• Adopting async at one layer forces async on every caller above (the 'colouring' propagates)
• Even Go and Java virtual threads can starve when blocking C extensions don't release the runtime

When NOT to Use Concurrency

Section: Core Mental Models | Difficulty: Intermediate | Frequency: Sometimes

The best concurrent code is usually no concurrent code. Every added thread or task costs memory, debugging difficulty, and a chance for race conditions. Reach for concurrency only after measuring a real bottleneck where the workload's nature (I/O wait, CPU parallelism, latency hiding) actually benefits.

Key Points

• Concurrency is a cost, pay it only when the benefit is measured, not assumed
• Premature concurrency is worse than premature optimization, it can't be taken back without rewriting
• If the bottleneck is the database, adding threads in the app server doesn't help
• Under 1000 simultaneous users, async/await is rarely needed
• Single-threaded event-loop services (Redis, Node.js) prove single-threaded scales further than people think
• The ONE-CORE rule: if it can be solved on one core, do it on one core

Interview Follow-up Questions

Gotchas

• Threading.Thread() in a script that runs <1 minute → complexity cost paid for nothing
• asyncio.gather on 10 items → not enough concurrency to matter; just loop
• Adding sharding before measuring single-shard limits → premature, costs cohesion
• Concurrent code reads as 'sophisticated' in code reviews, bias toward simplicity
• 'It might scale' is not a reason to add concurrency, measure first

Producer-Consumer Pattern

Section: Core Patterns | Difficulty: Intermediate | Frequency: Asked Often

One or more producers add work items to a bounded buffer; one or more consumers remove and process them. The buffer decouples their rates; the synchronization makes the buffer safe and prevents busy-waiting.

Key Points

• Bounded buffer: producers block when full, consumers block when empty
• Two condition variables (notFull, notEmpty), never one, or you risk lost wakeups
• Always wait in a while-loop, never if, to handle spurious and stolen wakeups
• In Go, a buffered channel IS the entire pattern, no extra synchronization needed
• In Python, queue.Queue is the canonical impl, don't roll your own
• Graceful shutdown: producer closes channel / sends sentinel; consumer detects and exits

Interview Follow-up Questions

Gotchas

• Closing a Go channel twice → panic. Sending on a closed channel → panic.
• wg.Add() must run BEFORE 'go ...' or it races with wg.Wait()
• Spurious wakeups: never use 'if' instead of 'while' around await()
• BlockingQueue.put vs offer vs add: put blocks, offer returns false, add throws, pick the right one
• Forgetting to handle InterruptedException = thread silently lives on
• Python: forgetting to put the sentinel = consumer hangs forever

Worker Pool Pattern

Section: Core Patterns | Difficulty: Intermediate | Frequency: Asked Often

A fixed pool of N worker threads/goroutines pulls jobs off a shared queue and processes them in parallel. It bounds resource usage (you can't run out of memory by spawning a thread per request) and keeps overhead constant under load.

Key Points

• Fixed N workers > thread-per-task: bounded memory, predictable overhead
• Workers loop: pull from queue, process, repeat, until queue closes
• Result delivery: separate result channel, Future objects, or callback
• Sizing: ~core count for CPU work, much higher for I/O, use Goetz's formula
• Always have a shutdown story: close queue → workers drain → join all
• Error propagation matters: one worker failing should signal the rest

Interview Follow-up Questions

Gotchas

• Java's newFixedThreadPool uses an unbounded LinkedBlockingQueue by default, bursts → OOM. Use ThreadPoolExecutor explicitly with ArrayBlockingQueue.
• Forgetting to call shutdown() on an ExecutorService → JVM doesn't exit
• Forgetting to close the work channel in Go → workers block forever waiting for more
• Calling .get() on a Future inside a worker → can deadlock if the pool is full and the worker is waiting for another task to complete
• Task throws an exception, future swallows it, always log future.exception() or wrap in try/catch
• ProcessPoolExecutor with closures or lambdas → pickle errors; use top-level functions

Pipeline Pattern

Section: Core Patterns | Difficulty: Intermediate | Frequency: Asked Often

A pipeline chains stages where each stage transforms data and passes it to the next. Each stage runs concurrently; bounded queues between stages give backpressure for free. Dominant pattern for stream processing, ETL, and any 'producer → transform → transform → consumer' shape.

Key Points

• Each stage is a concurrent worker (or pool) reading from one queue, writing to the next.
• Bounded queues between stages give automatic backpressure: a slow stage throttles upstream.
• The slowest stage is the bottleneck. Throughput equals throughput of the slowest stage.
• Shutdown propagates downstream: close the input, the stage drains and closes the output.
• Errors are tricky: cancel the whole pipeline, or skip-and-log, or dead-letter the bad item. Pick a policy.

Interview Follow-up Questions

Gotchas

• Forgetting to close the output channel leaves downstream goroutines blocked forever
• Unbounded queues between stages defeat backpressure; bound everything
• Errors in the middle of the pipeline can leave stages running with no input/output
• Slow consumer at the end backs up everything; monitor queue depths

Fan-Out / Fan-In

Section: Core Patterns | Difficulty: Intermediate | Frequency: Asked Often

Fan-out: distribute work to multiple workers concurrently. Fan-in: merge their results back into a single stream. The two together parallelise the middle of a pipeline. Standard shape: one input, N workers each doing the same transformation, one output combining results.

Key Points

• Fan-out: one input channel/queue feeds N workers. Each worker reads independently.
• Fan-in: N output channels merge into one. With channels, this is N goroutines copying to a shared output channel.
• Order is not preserved by default; preserving it requires a sequence number and a re-sort downstream.
• Worker count = degree of parallelism. Cap it (errgroup.SetLimit, semaphore, ThreadPoolExecutor maxWorkers).
• Combine with retry, circuit breaker, timeout for production-grade fan-out.

Interview Follow-up Questions

Gotchas

• Forgetting to close the output channel leaves the consumer blocked forever
• Unbounded worker count exhausts FDs and overwhelms downstream
• Order is lost unless explicitly preserved with sequence numbers
• One slow worker doesn't slow others if input is shared, but slows the overall completion
• Errors in one worker can leak resources if not propagated through cancellation

Futures and Promises

Section: Core Patterns | Difficulty: Intermediate | Frequency: Asked Often

A future is a placeholder for a value that will be available later. A promise is the producer side that fulfils the future. The pair allows starting work asynchronously and consuming the result with backpressure (the consumer waits if not ready). Foundational primitive in async programming.

Key Points

• Future = consumer-side handle (read result, wait, attach callbacks). Promise = producer-side handle (set value or error).
• Many languages have only one type that does both jobs: CompletableFuture (Java), Future (Python), Promise (JS), Future (Rust).
• Composition: chain futures (then/map/flatMap), wait for all (allOf/gather), wait for any (anyOf).
• Cancellation is the hard part. Many implementations support cancelling the future, but stopping the actual work depends on the producer cooperating.
• Don't block in a future-returning function. The whole point is async; calling .get() inside an async function defeats the purpose.

Interview Follow-up Questions

Gotchas

• Calling .get() / await inside async code defeats the purpose; chain instead
• Cancellation marks the future cancelled but may not stop the underlying work
• Java CompletableFuture's default executor (ForkJoinPool.commonPool) is shared; specify an executor for I/O-bound chains
• Composing futures with shared mutable state still needs synchronisation
• exceptionally / catch resets the error chain; subsequent stages run on the recovered value, not the error

Reactor Pattern and the Event Loop

Section: Core Patterns | Difficulty: Intermediate | Frequency: Asked Often

One thread, one loop. The loop asks the kernel which file descriptors are ready, fires the registered handlers for each, and goes back to the kernel. No threads block. This single design powers Node.js, Python asyncio, Netty, Tokio, libuv, nginx, and the Go netpoller. It is the answer to 'how do I serve 100K connections without 100K threads?'

Key Points

• Three pieces: a demultiplexer (epoll, kqueue, IOCP, io_uring), a queue of handlers keyed by FD, and a loop that drives them.
• Loop body: block in epoll_wait, get a list of ready FDs, run each FD's handler to completion, then loop again.
• Handlers must be non-blocking. Any blocking call (sync I/O, sleep, CPU-heavy work) freezes the entire loop and stalls every other connection.
• CPU-heavy work goes to a worker pool (Node.js worker_threads, Python's run_in_executor); the result comes back as a callback or future on the loop.
• Single-threaded by design. Removes data races inside the handler. Costs multi-core scaling unless multiple loops run (one per core, share-nothing).
• The 'colored functions' problem comes from this pattern: handlers and the things they call must agree on the same async ABI (callback, promise, async/await).

Interview Follow-up Questions

Gotchas

• A blocking call inside a handler freezes the loop. This includes sleep(), input(), requests.get(), psycopg2 queries, and most non-async libraries.
• Forgetting to await an async function: it returns a coroutine object that does nothing. Linters catch this; reviews don't always.
• The loop runs callbacks in the order they're scheduled, not the order their I/O completed. Never assume FIFO across handlers if work is queued via call_soon.
• Exceptions in handlers can crash the loop in some libraries; in asyncio they get logged and lost unless the task is awaited. Always check for unawaited tasks in production.

Async vs Blocking I/O

Section: Core Patterns | Difficulty: Intermediate | Frequency: Asked Often

Blocking I/O: one thread per concurrent operation, thread sleeps during I/O wait. Async I/O: one thread handles many operations via an event loop, syscalls are non-blocking with callbacks/coroutines. Async wins for many concurrent connections; blocking wins for few but heavy ones, or when the language doesn't have great async support.

Key Points

• Blocking model: thread per request. Each thread has memory cost (pthreads default ~8 MB virtual / a few hundred KB committed; JVM default 512KB-1MB), thread-create cost, context switch cost.
• Async model: event loop reads ready I/O events, dispatches to coroutines. One thread can multiplex thousands of connections.
• Async needs the entire stack to be async: blocking call inside an async function freezes the loop.
• Choose based on workload: many connections + small per-connection work = async. Few connections + heavy work = threads.
• Virtual threads (Java 21+, Go goroutines) blur the line: write blocking-style code, runtime provides async-like efficiency.

Interview Follow-up Questions

Gotchas

• Calling blocking code from async freezes the event loop
• Mixing thread-based libraries (threading.Lock) with asyncio races horribly
• Per-thread memory cost adds up: 10K threads = ~10 GB just for stack address space (mostly virtual)
• Virtual threads inherit thread-locals, but stack traces look different from platform threads
• Async stack traces are harder to read (the event loop is in the middle)

Retry with Exponential Backoff and Jitter

Section: Core Patterns | Difficulty: Intermediate | Frequency: Asked Often

Retry transient failures with delays that double each attempt (exponential backoff). Add jitter (randomness) so a thousand clients don't synchronise their retries and DDoS the recovering server. Cap the total attempts and the total elapsed time. Only retry idempotent operations.

Key Points

• Exponential backoff: delay = base * 2^attempt. Avoids hammering the failing service.
• Jitter is required, not optional. Without it, retries synchronise into a thundering herd.
• Bound retries: max attempts AND max total elapsed time. Both, not either.
• Only retry transient errors (5xx, timeouts, connection refused). Never retry 4xx (client errors).
• Only retry idempotent operations. POST is dangerous; GET, PUT, DELETE are usually safe.

Interview Follow-up Questions

Gotchas

• No jitter = thundering herd when downstream recovers
• Retrying non-idempotent operations (POST without idempotency key) double-charges users
• No total-time bound = retries last longer than the original request budget
• Retrying 4xx errors wastes everyone's time
• Retry inside a retry (caller retry plus the HTTP client's) creates exponential blowup

Circuit Breaker Pattern

Section: Core Patterns | Difficulty: Intermediate | Frequency: Asked Often

Stop calling a downstream that is failing. After N failures in a window, the breaker opens and immediately fails subsequent calls without hitting the downstream. After a cooldown, it allows a probe; if successful, close again. Prevents one slow downstream from cascading into the calling service.

Key Points

• Three states: closed (normal), open (fail fast), half-open (probe single call).
• Trip on failure rate or absolute count over a sliding window. 'X failures per minute' is more useful than 'X consecutive failures'.
• Open state has a cooldown (typically 30s-5min). After cooldown, breaker is half-open: allows ONE probe call.
• Without a circuit breaker, a slow downstream ties up threads/goroutines waiting on timeouts; the caller slows down trying to call something that won't respond.
• Per-downstream breakers, not global. One slow service shouldn't block calls to a healthy one.

Interview Follow-up Questions

Gotchas

• Single global breaker for all downstreams: one bad service blocks everything
• No probe in half-open state: breaker stays open forever
• Counting client errors (4xx) as breaker failures: trips on user mistakes, not service health
• Threshold too low: trips on benign blips. Too high: trips too late, after damage is done
• Forgetting fallback when breaker opens: callers see CircuitOpenError instead of degraded response

Bulkhead Pattern

Section: Core Patterns | Difficulty: Intermediate | Frequency: Sometimes

Isolate resources so a failure in one part of the system doesn't drain everything. Separate thread pool, connection pool, or rate limit per downstream so one slow downstream can't tie up resources needed by healthy ones. Named after ship hulls divided into compartments to prevent total flooding.

Key Points

• Each downstream gets its own resource budget (threads, connections, semaphore permits).
• When one downstream is slow, only its budget exhausts. Calls to other downstreams keep flowing.
• Without bulkheads, a single thread pool serving all downstreams gets fully tied up by the slow one. Latency cascades.
• Pair with circuit breaker (fail-fast) and timeout (don't wait forever) for full resilience.
• Cost: more threads/connections to manage. The win is bounded blast radius.

Interview Follow-up Questions

Gotchas

• One shared pool with no per-downstream cap = slow downstream blocks all calls
• Bulkhead too small = false rejections under normal traffic
• Bulkhead too big = downstream still gets overloaded; bulkhead provided no protection
• Forgetting to release the semaphore on exception leaks permits permanently
• Bulkhead without timeout = each in-flight call can still hang forever; combine them

Idempotency

Section: Core Patterns | Difficulty: Intermediate | Frequency: Asked Often

An idempotent operation produces the same result whether called once or many times. Critical for retries: if a charge call is idempotent, a network blip that triggers a retry doesn't double-charge the customer. Achieved via idempotency keys (client-supplied unique IDs) or natural idempotence (PUT, DELETE, set operations).

Key Points

• Idempotent operations: GET, PUT (replace), DELETE, set-x-to-y. Calling twice == calling once.
• NON-idempotent: POST that creates, INSERT, charge. Calling twice produces two records or two charges.
• Idempotency key: client generates a unique ID per logical operation, server deduplicates on it.
• Server stores 'this key was seen, here is the response' for some retention window.
• Without idempotency, retries can double-charge, double-create, double-send. With it, retries are safe.

Interview Follow-up Questions

Gotchas

• Generating a new key per retry defeats deduplication; one logical operation = one key
• Storing the key without storing the response means retries fail to replay the original outcome
• Not handling concurrent requests with the same key (need a lock) leads to double-execution
• Treating different requests with the same key as duplicates leads to wrong responses
• POST/INSERT without idempotency keys is a bug waiting to happen under network failures

Strict Alternating Output

Section: Core Patterns | Difficulty: Medium | Frequency: Asked Often

Two threads must alternate strictly, A then B then A then B, n times. The canonical solution uses a Semaphore pair, with thread A holding the 'A's turn' permit and signaling B's after each step. Same pattern with wait/notify, asyncio.Event, or Go channels.

Key Points

• Two semaphores, started with permits 1 and 0 respectively → enforces order
• Each thread acquires its semaphore, prints, then releases the OTHER's semaphore
• Equivalent patterns: wait/notify pair, two condition variables, two channels
• Generalizes to N-thread round-robin (next lesson)

Interview Follow-up Questions

Three-Way Round-Robin Output

Section: Core Patterns | Difficulty: Medium | Frequency: Asked Often

Three threads must produce 010203040..., Zero, then alternating Odd/Even. Three semaphores form a token-passing relay: Zero hands to Odd-or-Even based on parity; the parity thread hands back to Zero.

Key Points

• Three semaphores: zeroSem (starts 1), oddSem (0), evenSem (0)
• Zero always runs after every digit; parity thread alternates 'who's next' based on counter
• Generalizes the strict-alternation pattern to conditional next-turn
• wait/notify version: shared int + condition variables, harder to get right

Interview Follow-up Questions

Coordinated Conditional Output

Section: Core Patterns | Difficulty: Medium | Frequency: Asked Often

Four threads, Number, Fizz, Buzz, FizzBuzz, coordinate to print FizzBuzz for 1..n. The right thread is whichever matches the current number's divisibility. Pattern: shared counter + per-thread Semaphore, dispatcher wakes only the right thread.

Key Points

• Four threads, one shared counter, four semaphores
• A 'dispatcher' increments the counter and releases exactly the matching thread's semaphore
• Or: each thread waits on a condition that matches its rule (i % 3, i % 5, i % 15)
• Same token-passing pattern as FooBar but routed by predicate

Interview Follow-up Questions

Ordered Three-Thread Execution (Print in Order)

Section: Core Patterns | Difficulty: Easy | Frequency: Asked Often

Three threads start in arbitrary order; ensure first() runs before second() runs before third(). The minimal solution: two binary semaphores starting at 0, where each method releases the next's semaphore on completion. Same pattern with CountDownLatch.

Key Points

• Two semaphores both start at 0 → first method must signal before second can proceed
• first() prints, then releases s1; second() acquires s1, prints, releases s2; third() acquires s2
• CountDownLatch (Java) is a clean alternative, semantically more obvious
• Generalizes to N-stage pipelines via N-1 semaphores or N-1 latches

Interview Follow-up Questions

N-of-M Barrier Synchronization

Section: Core Patterns | Difficulty: Hard | Frequency: Asked Often

Hydrogen and Oxygen threads must pair into water, exactly 2 H + 1 O before any can release. The pattern: two semaphores controlling H/O entry + a CyclicBarrier of 3 ensuring all three threads of a molecule are present before any prints.

Key Points

• Pattern: rate-limiting semaphore + barrier
• Hydrogen semaphore = 2 permits; Oxygen semaphore = 1 permit; both refill after barrier
• CyclicBarrier(3) ensures the 2 H + 1 O appear together before any prints
• After barrier, semaphores reset, next molecule's H and O begin

Interview Follow-up Questions

Dining Philosophers

Section: Core Patterns | Difficulty: Hard | Frequency: Asked Often

Five philosophers, five forks, alternating thinking and eating. Each needs both adjacent forks to eat, so naively-acquired forks deadlock. Classic solutions: enforce odd/even acquisition order, limit to 4 eaters at once, or use a waiter (arbitrator).

Key Points

• Naive solution (each grabs left then right) deadlocks, circular wait
• Solution 1: odd philosophers grab left first, even grab right first → asymmetry breaks cycle
• Solution 2: a Semaphore(N-1) limits concurrent eaters → no full cycle possible
• Solution 3: arbitrator (waiter) serializes fork pickups
• Modern Go solution: channel-based fork ownership eliminates locks entirely

Interview Follow-up Questions

Readers-Writers Problem

Section: Core Patterns | Difficulty: Medium | Frequency: Asked Often

Many readers can access shared data concurrently, but writers need exclusive access. The classic problem: which side gets priority? Reader-pref starves writers; writer-pref starves readers; fair locks alternate. Java/Go/Python all ship reader-writer locks.

Key Points

• Readers can run concurrently with each other; writers need exclusive access
• Reader-preference: writer can starve under sustained reader load
• Writer-preference: reader can starve when writers always queue
• Fair / FIFO: alternates by arrival order, slightly slower, no starvation
• Most stdlib RW locks are writer-preferring (Java, Go, Python's read-write semaphore)

Interview Follow-up Questions

Sleeping Barber

Section: Core Patterns | Difficulty: Medium | Frequency: Sometimes

A barber sleeps when no customers; a customer wakes the barber if asleep, otherwise sits in a finite waiting room (or leaves if full). Classic producer-consumer with a bounded buffer + signal-on-empty pattern. Two semaphores + a counted slot map onto it cleanly.

Key Points

• Three roles: barber, customer, finite waiting room
• Two semaphores: customers (producer signals barber) + barber (signals next haircut starts)
• Mutex protects the seat counter; bounded chairs reject if full
• Producer-consumer with a bounded buffer in different clothes

Interview Follow-up Questions

Cigarette Smokers Problem

Section: Core Patterns | Difficulty: Hard | Frequency: Rare

Three smokers each have one of three ingredients (tobacco, paper, matches). An agent randomly drops two of the three on the table; the smoker with the missing ingredient must roll and smoke. Tests whether the candidate can avoid 'helper threads' (Patil's restriction) and use only counting semaphores cleanly.

Key Points

• Three smokers, each with infinite supply of one ingredient
• Agent puts two ingredients on the table; smoker with the third must take and smoke
• Patil's restriction: NO conditional statements in the agent code (just put + signal)
• Trick: use 'pusher' threads as helpers, they observe what's available and signal the matching smoker
• Modern view: this 'difficulty' is contrived and the helpers pattern is fine

Interview Follow-up Questions

Santa Claus Problem

Section: Core Patterns | Difficulty: Hard | Frequency: Rare

Santa sleeps until either 9 reindeer return (deliver toys) or 3 elves arrive (consult). Reindeer take priority over elves. Tests selective wakeup + group barrier + priority, three classic primitives composed into one problem.

Key Points

• Santa sleeps until: 9 reindeer ready OR 3 elves waiting
• Reindeer have priority, Santa serves them first if both are queued
• Each group needs barrier-style synchronization (all 9 / all 3 must be present)
• Composition: priority + group-of-N barrier + producer-consumer signaling

Interview Follow-up Questions

Categorical Mutual Exclusion

Section: Core Patterns | Difficulty: Medium | Frequency: Sometimes

A bathroom can hold N people of the same category but never mix categories. People of category A enter while bathroom is empty or only A's are inside; same for B's. Generalizes Readers-Writers, readers and writers are categories, with N=1 for writers.

Key Points

• Two states: 'A only', 'B only', or empty
• Same-category entry is unrestricted (up to capacity if any)
• Cross-category entry must wait until current category empties
• Generalization of Readers-Writers, same problem, more symmetric
• Risk: starvation, if one category keeps arriving, the other never enters

Interview Follow-up Questions

Goroutines, Spawning, Scheduling, Lifecycle

Section: Go Concurrency | Difficulty: Basic | Frequency: Asked Often

Goroutines are user-space tasks multiplexed onto a small pool of OS threads by the Go runtime's M:P:G scheduler. Spawn with `go func() {...}()`, ~2KB initial stack, cheap to create, millions are practical. No public state machine; lifetime is 'until the function returns.'

Key Points

• Goroutines are NOT OS threads, multiplexed onto N OS threads where N = GOMAXPROCS
• Stack starts at ~2KB, grows on demand up to ~1GB
• M:P:G model: M = OS thread, P = logical processor, G = goroutine
• Work stealing: idle P's steal goroutines from busy P's run queues
• No stop/kill, goroutines exit only when their function returns

Interview Follow-up Questions

Gotchas

• Closure capture in loops (pre-Go 1.22): always pass loop vars as args
• wg.Add() must be BEFORE 'go ...' or it races with wg.Wait()
• Spawning unbounded goroutines = potential OOM under load
• tight pure-CPU loops without yields could starve the scheduler before Go 1.14

GMP Scheduler: How Goroutines Actually Run

Section: Go Concurrency | Difficulty: Advanced | Frequency: Asked Often

G = goroutine, M = OS thread, P = logical processor. The runtime keeps GOMAXPROCS P's; each P holds a local run queue of runnable G's. M's must hold a P to run G's. Blocking syscalls hand the P off to another M so other goroutines keep running. Idle P's steal work from busy ones. The network poller and sysmon thread keep things moving. The whole machine runs in user space and is what makes 100K goroutines cheap.

Key Points

• G = goroutine (small struct + stack), M = OS thread (machine), P = logical processor (GOMAXPROCS of these). M needs a P to run G's.
• Each P has a local run queue of 256 runnable G's plus a 'runnext' LIFO slot for newly created goroutines. Overflow goes to the global queue.
• Work stealing: when a P's local queue is empty, it steals half of another P's queue. Keeps load balanced without explicit coordination.
• Blocking syscalls: the M holding a P releases the P (handoff), another M (or a fresh one) picks the P up and continues running other G's. The blocked G stays attached to its M.
• Channel and lock waits don't block an M. The G is parked in user space (a wait list inside the channel/mutex), the M moves on to other work.
• Network poller (epoll/kqueue/IOCP) is one shared component. When network I/O completes, it puts the waiting G back on a P's run queue.
• sysmon is a dedicated OS thread (no P) that preempts long-running G's, retakes P's stuck in syscalls too long, and triggers GC.
• Async preemption via signals (since Go 1.14): even a tight loop with no function calls can be preempted. Before 1.14, such a loop could starve the scheduler.

Interview Follow-up Questions

Gotchas

• Runaway G creation under load: spawning a goroutine per inbound message without bounds will OOM the process when the queue can't drain fast enough. Always have a worker pool or semaphore.
• cgo calls block an M for the entire C call. Millions of cgo calls per second will exhaust the M pool and require raising the soft cap (default 10000).
• GOMAXPROCS too low (default 1 in Go pre-1.5, or in containerized environments without automaxprocs) silently serializes everything.
• GOMAXPROCS too high on a small cgroup cuts performance because the OS scheduler thrashes between threads that compete for limited CPU.
• Long syscalls (slow disks) can spike the M count; suddenly seeing hundreds of M's is a sign of blocking syscalls under load.

Channels, Buffered, Unbuffered, Patterns

Section: Go Concurrency | Difficulty: Intermediate | Frequency: Asked Often

Channels are typed conduits between goroutines. Unbuffered = synchronous handoff (sender blocks until receiver ready). Buffered = bounded queue. Closing signals 'no more values' to receivers. The cornerstone of Go's 'share by communicating' philosophy.

Key Points

• Unbuffered: synchronous, send blocks until receive, vice versa
• Buffered: send blocks only when buffer full; receive blocks only when empty
• close(ch) = 'no more values'; receivers see ok=false; sending on closed = panic
• Nil channels block forever, useful in select to disable a case
• Channels are not free, for tight CPU loops, mutex is faster

Interview Follow-up Questions

Gotchas

• Closing a channel twice → panic
• Sending on a closed channel → panic (so receiver should never close)
• Range loop on a never-closed channel → blocks forever (goroutine leak)
• Make sure only ONE goroutine closes the channel; coordinate with WaitGroup if multiple senders
• Buffered channels with capacity = number of sends → potential deadlock if a receive is missed

select Deep Dive

Section: Go Concurrency | Difficulty: Intermediate | Frequency: Asked Often

select waits on multiple channel operations and proceeds with whichever is ready first. Add a default for non-blocking operations; combine with time.After for timeouts; with ctx.Done() for cancellation. The cornerstone of channel composition.

Key Points

• select picks ONE ready case; if multiple ready, randomly chosen
• default makes select non-blocking, if no case ready, default runs
• time.After(d) returns a channel that fires once after d → easy timeouts
• ctx.Done() returns a channel, select on it for cancellation
• select on nil channel = case never fires → useful for dynamic channel disabling

Interview Follow-up Questions

Gotchas

• select with no cases blocks forever (used to park the main goroutine intentionally)
• select with only default is just an immediate non-blocking check
• time.After in tight loops leaks timers, use NewTimer + Stop
• default in select on multiple channels can cause starvation if always-ready cases dominate
• Nil channel cases never fire, both feature and footgun

context.Context, Cancellation & Deadlines

Section: Go Concurrency | Difficulty: Intermediate | Frequency: Asked Often

context.Context propagates cancellation, deadlines, and request-scoped values through call chains. Every long-running operation should accept a Context and select on ctx.Done(). The universal cancellation primitive in modern Go.

Key Points

• Context flows through the call graph, pass as first parameter to every function that may block
• WithCancel: parent function cancels manually
• WithTimeout / WithDeadline: auto-cancellation after duration
• ctx.Done() returns a channel that closes on cancellation
• Always defer cancel() to release resources, even when not strictly needed

Interview Follow-up Questions

Gotchas

• Forgetting `defer cancel()` → goroutine leak (the cleanup goroutine sticks around)
• Storing context as a struct field; convention says pass it explicitly
• Putting too much in context.Value, easy to abuse for global state
• Mixing cancellation: ctx is cancelled but inner code didn't check ctx.Done() → operation continues
• context.Background() in a function that should take a ctx, silently disables cancellation

Goroutine Leak Prevention

Section: Go Concurrency | Difficulty: Intermediate | Frequency: Asked Often

Every spawned goroutine must have a known exit story, context cancellation, channel close, natural completion, or timeout. Goroutines that block forever are the #1 production scaling bug in Go services. Detect via runtime.NumGoroutine and pprof/goroutine.

Key Points

• Symptom: NumGoroutine grows unbounded under load
• Diagnosis: pprof/goroutine shows what each leaked goroutine is blocked on
• Pattern: every send/receive needs a select-with-ctx.Done() escape
• Bounded buffered channels absorb bursts; default-on-full handles overflow
• Test for leaks: run handler 10K times in test, check NumGoroutine before/after

Interview Follow-up Questions

Gotchas

• context.WithCancel without `defer cancel()` leaks the timer goroutine
• for {} loops without select-on-ctx.Done leak forever
• Channel send with no receiver blocks forever, wrap in select + default OR + ctx.Done
• time.After in tight loops creates one timer per call; use NewTimer + Stop
• Goroutines waiting on http.Request body to fully drain, close the body explicitly

Go Memory Model & Sync Guarantees

Section: Go Concurrency | Difficulty: Intermediate | Frequency: Asked Often

Go's memory model defines when one goroutine is guaranteed to observe writes made by another. The rule is short, synchronize using channels, sync primitives, or sync/atomic, or you have a data race.

Key Points

• Go's mantra: 'Don't communicate by sharing memory; share memory by communicating'
• A send on a channel happens-before the corresponding receive completes
• A receive from an unbuffered channel happens-before the send completes
• Mutex.Unlock() happens-before the next Mutex.Lock() returns
• sync/atomic loads/stores establish happens-before (since Go 1.19)
• go run -race detects data races at runtime; run it in CI

Interview Follow-up Questions

Gotchas

• Copying a struct that contains a sync.Mutex breaks it; go vet will warn
• Closing a channel twice panics; sending to a closed channel panics
• select with no default + no ready case blocks forever, common cause of deadlock
• Goroutine leaked on channel send: writer blocks forever if no one reads and no buffer
• atomic operations require the value's address, atomic.AddInt64(&x, 1), not (x, 1)

sync.Mutex and sync.RWMutex

Section: Go Concurrency | Difficulty: Intermediate | Frequency: Asked Often

sync.Mutex is Go's basic mutex (Lock/Unlock). sync.RWMutex allows many readers OR one writer. Always use defer Unlock right after Lock. Mutexes are not reentrant in Go: same goroutine locking twice deadlocks. Pass mutexes by pointer; copying a mutex copies its state (including held-ness).

Key Points

• Mutexes are NOT reentrant in Go. Same goroutine locking twice deadlocks.
• Always Lock then defer Unlock. Skipping defer is the source of most lock leaks.
• Mutex zero value is unlocked and ready to use. No initialisation needed.
• Copying a Mutex copies its state. Embed by value carefully; pass structs containing mutexes by pointer.
• RWMutex is slower than Mutex for write-heavy or short-critical-section workloads. Profile.

Interview Follow-up Questions

Gotchas

• Mutex is not reentrant; same goroutine locking twice deadlocks
• Copying a struct that contains a Mutex copies the lock state; pass by pointer
• RWMutex is slower than Mutex for short critical sections; profile before assuming
• Forgetting defer Unlock leaks the lock on panic or early return
• Holding a mutex across a channel send/receive can deadlock with the channel's other end

sync Package: Once, WaitGroup, Cond, Pool, Map

Section: Go Concurrency | Difficulty: Intermediate | Frequency: Asked Often

Beyond mutexes, sync provides Once (one-time init), WaitGroup (wait for N goroutines), Cond (wait/signal on a predicate), Pool (reusable object cache), and Map (lock-free read-mostly map). Each solves a specific shape; reaching for the wrong one leads to ugly code.

Key Points

• sync.Once.Do guarantees the function runs exactly once across all goroutines.
• WaitGroup.Add(n) MUST happen before launching goroutines; Done in defer; Wait blocks until counter is 0.
• sync.Cond is rarely the right tool in Go; channels usually express the same intent more clearly.
• sync.Pool reduces GC pressure for short-lived objects but is NOT a connection pool: items can be reaped any time.
• sync.Map wins for read-mostly disjoint-key workloads. For write-heavy or read-write-mixed, plain map + RWMutex is faster.

Interview Follow-up Questions

Gotchas

• WaitGroup.Add inside a goroutine races with Wait; Add before launching
• sync.Pool items can be dropped at any GC; never use for finite resources
• sync.Cond.Wait must be inside a for-loop on the predicate (spurious wakeups)
• sync.Map cannot be ranged AND mutated safely; use Range carefully
• Calling Signal/Broadcast without holding the Cond's Locker can cause missed wakeups

sync/atomic: Lock-Free Single-Variable Updates

Section: Go Concurrency | Difficulty: Intermediate | Frequency: Asked Often

sync/atomic provides lock-free atomic operations on integers and pointers. Add, Load, Store, Swap, CompareAndSwap. Go 1.19+ added typed atomic.Int64, atomic.Pointer[T], atomic.Bool which are cleaner than the function-style API. Use for counters, hot flags, lock-free data structures.

Key Points

• Atomic ops are lock-free: single hardware instruction (LOCK XADD on x86, LDXR/STXR on ARM).
• Go 1.19+ typed API (atomic.Int64) is preferred over the function-style API.
• CompareAndSwap is the building block for lock-free algorithms; the retry loop pattern is identical to Java.
• atomic.Value supports any type but is replace-only (no compound update). Use atomic.Pointer[T] when possible.
• Plain reads/writes to int64 are NOT atomic on 32-bit platforms; use atomic.Int64 if portability matters.

Interview Follow-up Questions

Gotchas

• Function-style API requires 8-byte alignment for int64; typed API handles this
• atomic.Value requires same concrete type on every Store after the first
• Plain int64 reads/writes are NOT atomic on 32-bit platforms
• Atomic operations are NOT enough for compound state (multiple fields); use a mutex or restructure to one immutable struct
• CAS retry loops can spin under high contention; benchmark before assuming lock-free is faster

sync.Map vs map+RWMutex

Section: Go Concurrency | Difficulty: Intermediate | Frequency: Sometimes

sync.Map is the standard library's lock-free concurrent map. It wins for two specific shapes: write-once-read-many entries, and disjoint-key access from many goroutines. For mixed read/write on the same keys, plain map + sync.RWMutex is faster. The Go authors put a warning in the docs to this effect.

Key Points

• sync.Map is untyped (any/any). No generics in standard library; type assertions everywhere.
• Optimised for write-once-read-many or disjoint-key access. Other patterns: use map+RWMutex.
• Range is weakly consistent: it iterates a snapshot, but mutations during Range may or may not be reflected.
• LoadOrStore is the atomic 'get or insert' that map+RWMutex needs separate Lock/Unlock for.
• If you need iteration order, range, or generics, use plain map with RWMutex. sync.Map is a sharp tool for specific shapes.

Interview Follow-up Questions

Gotchas

• sync.Map is untyped; type assertions on every Load are easy to get wrong
• Range is weakly consistent; not safe for code that needs an exact snapshot
• Storing nil values is allowed but easily confused with 'not present'; check the bool
• For most workloads, map+RWMutex outperforms sync.Map; default to it unless you have measured otherwise
• Range can block forever if f never returns, including on writes that block on the map

errgroup and golang.org/x/sync/semaphore

Section: Go Concurrency | Difficulty: Intermediate | Frequency: Asked Often

errgroup.Group is the structured fan-out for Go: launch goroutines via Go(func), Wait for all, first error cancels siblings via Context. semaphore.Weighted bounds concurrency with weighted permits. Together they cover most real fan-out cases cleaner than raw WaitGroup + manual cancel plumbing.

Key Points

• errgroup.WithContext returns a Group AND a derived Context that is cancelled when any goroutine returns an error.
• First error wins; subsequent errors from siblings are dropped (only logged).
• g.SetLimit(n) (added in x/sync errgroup v0.0.0-2022) caps in-flight goroutines without a separate semaphore.
• semaphore.Weighted permits >1 weight per acquisition; useful when tasks have different cost (memory, CPU).
• Both are in golang.org/x/sync (not stdlib) but are de facto standard for Go fan-out.

Interview Follow-up Questions

Gotchas

• Forgetting to capture the loop variable (item := item) before passing to g.Go creates a race in pre-Go-1.22 code
• errgroup only returns the FIRST error; subsequent errors are silently dropped
• A goroutine that ignores ctx runs to completion even after error cancellation
• semaphore.Weighted.Acquire blocks until ctx done or weight available; check ctx.Err() to distinguish
• errgroup zero value works but loses cancellation; always use WithContext for fan-out

Go Runtime Tuning: GOMAXPROCS, GC, Scheduler

Section: Go Concurrency | Difficulty: Advanced | Frequency: Sometimes

GOMAXPROCS sets how many OS threads execute Go code concurrently (default = cpu count). GOGC controls when GC runs (default 100 = collect when heap doubles). GOMEMLIMIT (Go 1.19+) caps RSS to keep within container limits. The Go scheduler is M:N; you almost never need to think about it, but knowing the levers helps when you do.

Key Points

• GOMAXPROCS defaults to cpu count. In containers without CFS-quota awareness, that may be the host's cores, not your limit. Use uber-go/automaxprocs.
• GOGC=100 means GC when heap doubles since last GC. Higher values trade memory for fewer collections.
• GOMEMLIMIT is a soft cap on total memory. Without it, Go can OOM in containers even with GOGC tuned low.
• The scheduler is M:N: many goroutines on a few OS threads. Goroutines park and resume cheaply (microseconds).
• GODEBUG=schedtrace=1000 prints scheduler stats every 1s. Useful for diagnosing scheduling stalls.

Interview Follow-up Questions

Gotchas

• GOMAXPROCS default is host CPU count, NOT container CPU limit; use automaxprocs
• Without GOMEMLIMIT, Go can OOM in tight containers even with GOGC tuned
• Running pprof with too many concurrent profiles slows the program; profile sparingly in prod
• GODEBUG=schedtrace adds overhead; not for prod, only for diagnostics
• runtime.NumGoroutine() shows total; doesn't tell you which are blocked vs runnable

Threads, Runnable, Virtual Threads

Section: Java Concurrency | Difficulty: Basic | Frequency: Asked Often

Java's concurrency unit is the thread. Platform threads wrap OS threads (1MB stack, ~10K max). Virtual threads (Java 21+) are JVM-managed M:N tasks (~few KB), millions per JVM. Use Runnable/Callable to define work; Thread/Executors to schedule.

Key Points

• Platform thread = OS thread; Virtual thread = JVM-managed M:N (Java 21+)
• Use Runnable for fire-and-forget; Callable for results; Thread.start to launch
• Virtual threads work with all blocking JDK APIs, no library rewrite needed
• Thread.join() waits for completion; thread.getState() gives runtime state
• Avoid creating threads directly in production code; use ExecutorService

Interview Follow-up Questions

Gotchas

• Thread.stop() is deprecated and unsafe, never use
• Thread.sleep() and Object.wait() CLEAR the interrupt flag, re-set with Thread.currentThread().interrupt()
• Don't reuse Thread instances, start() throws on a started/terminated thread
• Virtual threads pin to their carrier inside synchronized blocks, use ReentrantLock for I/O-heavy code on virtual threads
• Daemon threads are killed at JVM exit, don't use for cleanup work

synchronized vs ReentrantLock

Section: Java Concurrency | Difficulty: Intermediate | Frequency: Asked Often

synchronized is the language-level mutex, concise, JVM-optimized, automatic release on exception. ReentrantLock is the API-level mutex, adds tryLock with timeout, fairness, multiple Conditions, lock interruptibility. Pick based on which features the call site needs.

Key Points

• synchronized: cheap, JVM-optimized, auto-release. Default choice.
• ReentrantLock: tryLock(timeout), fairness, multiple Conditions, interruptible
• Both reentrant, same thread can re-acquire
• synchronized.notify() / notifyAll() vs ReentrantLock.newCondition().signal()
• Java 21+ virtual threads + synchronized = pinning (use ReentrantLock instead for I/O-heavy code)

Interview Follow-up Questions

Gotchas

• ReentrantLock without try/finally → exception leaks the lock
• synchronized on String literals or autoboxed Integers → shared monitors, deadlocks across unrelated code
• Fairness in locks adds overhead, use only when measurement justifies it
• Reading a long/double without synchronization on 32-bit JVMs is NOT atomic
• Holding a lock during an external call (network, callback) → deadlock if callee tries to lock something held by caller

AbstractQueuedSynchronizer (AQS)

Section: Java Concurrency | Difficulty: Advanced | Frequency: Asked Often

AQS is the framework underneath every lock in java.util.concurrent. It provides a single int state variable, a CLH-style wait queue, and the park/unpark plumbing. Subclasses define what the state means and when acquire/release succeeds. ReentrantLock, Semaphore, CountDownLatch, ReadWriteLock, FutureTask, ThreadPoolExecutor's worker locks, and StampedLock are all AQS subclasses or close cousins.

Key Points

• One volatile int state field. Subclasses interpret it: 0/1 for ReentrantLock, hold count for reentrancy, available permits for Semaphore, count for CountDownLatch.
• FIFO wait queue (a CLH variant): each waiter is a Node with a Thread and a status. Spin briefly, then park.
• Subclasses override tryAcquire / tryRelease (exclusive) or tryAcquireShared / tryReleaseShared (shared). Everything else (queueing, parking, signalling) is provided.
• Supports interruptible, timed, and fair acquisition out of the box.
• ConditionObject (the Condition impl) lives on top of AQS too; await/signal use a separate condition queue that migrates nodes back to the main wait queue on signal.
• Java 21 added VirtualThread support: AQS parks via LockSupport, which knows how to unmount a virtual thread from its carrier.

Interview Follow-up Questions

Gotchas

• Forgetting to make state writes happen via setState/compareAndSetState. They are volatile-safe; direct writes to other fields are not.
• In tryRelease, only return true when the lock is fully released (state == 0 for a reentrant mutex). Returning true early wakes a waiter that will then fail tryAcquire and re-park, wasting a context switch.
• Mixing exclusive and shared modes in the same synchronizer is possible but tricky (see ReentrantReadWriteLock). Don't try without first reading the source.
• Signal-on-Condition without holding the lock throws IllegalMonitorStateException. ConditionObject checks the AQS owner.

volatile in Java

Section: Java Concurrency | Difficulty: Intermediate | Frequency: Asked Often

volatile gives a single field two guarantees: every read sees the latest write from any thread (visibility), and reads and writes cannot be reordered across the access (ordering). It does not make compound operations like value++ atomic. Use it for status flags, write-once references, and the published reference in double-checked locking.

Key Points

• Visibility: a write to a volatile field is immediately visible to any thread that reads it.
• Ordering: writes that happened before the volatile write are visible to a reader that observes it. Reads after the volatile read see those writes too.
• NOT atomic for compound operations. value++ is read-modify-write, not one operation.
• Long and double on 32-bit JVMs are atomic only when declared volatile.
• For atomicity, use AtomicInteger / AtomicLong / AtomicReference. When both are needed, those classes already provide volatile semantics.

Interview Follow-up Questions

Gotchas

• volatile does not lock; multiple writers can still race on the same field
• volatile array reference is volatile, but reading array[i] is NOT (the element access is plain). Use AtomicReferenceArray for per-element ordering
• Long and double need volatile to be atomic on 32-bit JVMs
• Compound updates (value++, value = compute(value)) are not atomic under volatile alone
• synchronized provides everything volatile does, plus mutual exclusion. Do not stack them on the same field

Java Memory Model & Happens-Before

Section: Java Concurrency | Difficulty: Intermediate | Frequency: Asked Often

The JMM defines what reads can observe what writes across threads. The happens-before relation is the rule set, if action A happens-before action B, then A's effects are visible to B.

Key Points

• Without happens-before, the JVM/CPU can reorder reads/writes for performance
• volatile guarantees: visibility (latest value) + ordering (no reordering across the access)
• synchronized guarantees: mutual exclusion + happens-before on lock release → next acquire
• final fields: safely visible after constructor completes, no synchronization needed
• Thread.start() happens-before any action in the started thread
• Action in a thread happens-before another thread's successful join() return

Interview Follow-up Questions

Gotchas

• Reading a volatile array reference is volatile, but reading array[i] is NOT, use AtomicReferenceArray
• synchronized on a String literal or autoboxed Boolean can deadlock unrelated code (shared instances)
• Constructors that publish 'this' before completion break the final-field guarantee
• long and double are NOT atomic on 32-bit JVMs without volatile
• Python CPython's atomic-bytecode behavior is not portable to PyPy or free-threaded CPython

Java Atomics: AtomicInteger, AtomicReference, LongAdder

Section: Java Concurrency | Difficulty: Intermediate | Frequency: Asked Often

java.util.concurrent.atomic gives lock-free single-variable updates: AtomicInteger / AtomicLong / AtomicReference for counters and references, LongAdder for high-contention counters, AtomicReferenceArray for per-element ordering. All built on hardware compare-and-swap.

Key Points

• All atomic classes provide volatile-style visibility plus atomic compound operations like incrementAndGet, compareAndSet, accumulateAndGet.
• Use AtomicLong for counters with low contention. Use LongAdder when many threads write the same counter and reads are rare.
• compareAndSet is the building block. Most other methods are CAS retry loops.
• AtomicReference.compareAndSet protects against the wrong WRITE, but not against the wrong READ. Reading a reference and dereferencing it does not interlock with concurrent writes.
• AtomicStampedReference adds a version counter to defeat the ABA problem.

Interview Follow-up Questions

Gotchas

• AtomicLong.get() then .set() is two operations, not one. Use accumulateAndGet or compareAndSet.
• AtomicReferenceArray protects per-element ordering; a volatile array reference does NOT.
• Under heavy contention, CAS retry loops can spin a lot. Move to LongAdder or sharded counters.
• Compound operations across multiple atomics need a lock; atomics are single-variable only.
• lazySet on Atomic* skips the release barrier. Faster, but no happens-before. Use only when the implications are well understood.

ThreadLocal & Inheritable State

Section: Java Concurrency | Difficulty: Intermediate | Frequency: Sometimes

ThreadLocal stores per-thread values, every thread sees its own copy. Used for: request context, SimpleDateFormat (not thread-safe), DB connections per thread. Pitfalls: leaks in thread pools, doesn't inherit by default, virtual threads explode memory.

Key Points

• Each thread has its own value, isolated, no synchronization needed
• Common uses: request-id for logging, per-thread DB connections, SimpleDateFormat
• InheritableThreadLocal copies parent's value to child threads
• Pitfall: thread pool reuses threads, ThreadLocal values persist across requests → leaks
• ScopedValue is the modern replacement (preview in 21, finalised in Java 25): bounded scope, no leaks

Interview Follow-up Questions

Gotchas

• ThreadLocal in a pool without remove() in finally → leak
• InheritableThreadLocal copies AT CREATION, async dispatch doesn't propagate (need explicit handoff)
• Virtual threads + many ThreadLocals → memory explosion (use ScopedValue)
• ThreadLocal.set(null) is NOT the same as remove(), null entry stays in the map
• MDC in async chains often loses the value, middleware must propagate or wrap

ExecutorService & Thread Pools

Section: Java Concurrency | Difficulty: Intermediate | Frequency: Asked Often

ExecutorService manages a pool of threads + a work queue. Use ThreadPoolExecutor with bounded queue + named threads + sensible rejection policy in production. newFixedThreadPool/newCachedThreadPool have unbounded queues, production hazards.

Key Points

• ThreadPoolExecutor (the actual class), full control
• Executors.newXxx, convenience but with footguns (unbounded queues)
• Always: bounded queue, named threads, explicit rejection policy
• shutdown() vs shutdownNow(), graceful vs forceful
• Java 21+ virtual thread executor, request-per-thread reborn

Interview Follow-up Questions

Gotchas

• Executors.newFixedThreadPool: unbounded queue → potential OOM
• Executors.newCachedThreadPool: unbounded thread count → potential OS thread exhaustion
• Forgetting to call shutdown() → JVM doesn't exit
• Future.get() without timeout → can wait forever if task hangs
• Worker thread throwing uncaught exception → killed silently; pool spawns replacement; bug invisible without UncaughtExceptionHandler

CompletableFuture & Async Composition

Section: Java Concurrency | Difficulty: Intermediate | Frequency: Asked Often

CompletableFuture composes async operations into pipelines: thenApply (transform), thenCompose (chain), allOf/anyOf (combine), exceptionally (recover), orTimeout (deadline). The Java equivalent of JS Promise chains, with explicit executor binding.

Key Points

• supplyAsync: launch async computation; returns CompletableFuture<T>
• thenApply: transform result (sync). thenCompose: chain another async operation
• allOf: wait for all. anyOf: wait for first.
• Always specify the Executor, default ForkJoinPool.commonPool() is shared
• exceptionally / handle for error recovery; orTimeout for deadlines

Interview Follow-up Questions

Gotchas

• Default executor is ForkJoinPool.commonPool(), shared, easily exhausted
• thenApply where thenCompose was needed → CF<CF<T>> (nested future)
• cancel(true) doesn't interrupt the running thread
• exceptionally hides the exception unless logged; whenComplete is better for visibility
• join() can hang if no timeout, always pair with orTimeout for production

ConcurrentHashMap

Section: Java Concurrency | Difficulty: Intermediate | Frequency: Asked Often

Java's general-purpose concurrent map. Lock-free reads, CAS-based and per-bin locking on writes. Use computeIfAbsent and merge for atomic compound operations. Far better than synchronized HashMap or Collections.synchronizedMap for everything except trivial workloads.

Key Points

• Reads are completely lock-free. Writes lock per-bin (the linked list at one hash bucket).
• Java 8+ replaced segment locks with per-bin synchronisation + CAS. Resize is incremental and lock-free.
• containsKey + put is racy. Use putIfAbsent or computeIfAbsent.
• size() is approximate under concurrent mutation. mappingCount() returns long for huge maps.
• Iterators are weakly consistent: they reflect the state at some point during iteration, not a snapshot.

Interview Follow-up Questions

Gotchas

• computeIfAbsent function should be fast and side-effect-free. While it runs, the bin is locked.
• Recursive update inside computeIfAbsent (calling map.put on the same map) can deadlock or throw.
• size() is O(N) and approximate. Cache it when called often.
• Default initial capacity is 16, default concurrency level was historically 16. In Java 8+ concurrencyLevel is just a hint.
• Null keys and null values are forbidden. Throws NullPointerException.

BlockingQueue Family

Section: Java Concurrency | Difficulty: Intermediate | Frequency: Asked Often

BlockingQueue is the Java interface for thread-safe producer-consumer queues. Pick by overflow behaviour and ordering: ArrayBlockingQueue (bounded), LinkedBlockingQueue (default unbounded, dangerous), PriorityBlockingQueue (ordered), SynchronousQueue (no buffer, hand-off), DelayQueue (scheduled), TransferQueue (sender knows when received).

Key Points

• put blocks if full, take blocks if empty. offer/poll have non-blocking and timed variants.
• ArrayBlockingQueue is the default safe choice for production: bounded, simple, predictable.
• LinkedBlockingQueue defaults to UNBOUNDED. Always pass a capacity for production.
• SynchronousQueue has zero buffer: producer waits for consumer to take directly. Used by Executors.newCachedThreadPool.
• All implementations are fully thread-safe: put/take/poll/offer can be called from any thread.

Interview Follow-up Questions

Gotchas

• LinkedBlockingQueue with no arg = unbounded = OOM risk under burst
• Mixing add/remove (throw) with put/take (block) leads to inconsistent error handling
• PriorityBlockingQueue is unbounded; use a separate semaphore for bounded priority work
• Iterator on a BlockingQueue is weakly consistent, not a snapshot
• Forgetting to handle InterruptedException in put/take leaves the thread alive but the cancellation lost

Sync Utilities: Latch, Barrier, Semaphore, Phaser, Exchanger

Section: Java Concurrency | Difficulty: Intermediate | Frequency: Asked Often

Five coordination primitives in java.util.concurrent. CountDownLatch: one-shot start signal. CyclicBarrier: rendezvous N threads, reusable. Semaphore: limit concurrent access to N permits. Phaser: dynamic-party barrier. Exchanger: hand off a value between two threads.

Key Points

• CountDownLatch is one-shot. Once it hits zero, it stays zero. Cannot reset.
• CyclicBarrier is reusable. After N threads arrive, it triggers an optional action and resets.
• Semaphore.acquire() blocks for a permit. Use it for connection pools, rate limiters, bounded concurrent execution.
• Phaser is the flexible cousin of CyclicBarrier: parties can register and deregister mid-flight.
• Exchanger pairs exactly two threads. Each calls exchange(value) and gets the other's value.

Interview Follow-up Questions

Gotchas

• CountDownLatch cannot be reset. For a reusable variant, use CyclicBarrier or Phaser.
• Forgetting to release a Semaphore permit (no try/finally) leaks permits permanently
• CyclicBarrier with a barrier action: that action throws? All waiting threads get BrokenBarrierException.
• Unfair Semaphore can starve a thread under sustained contention.
• Phaser deregistration is one-shot; deregistering twice from one thread will desync the count.

ReadWriteLock & StampedLock

Section: Java Concurrency | Difficulty: Advanced | Frequency: Sometimes

ReentrantReadWriteLock allows many concurrent readers OR one writer. StampedLock adds an optimistic-read mode that does not block writers. Use ReadWriteLock when reads dominate writes by 10x or more. Use StampedLock when reads vastly dominate and a retry is tolerable. Both are sharper than they look.

Key Points

• ReentrantReadWriteLock: many readers OR one writer at a time. Readers do not block readers.
• Read locks are reentrant. Write locks are reentrant. Holding write lock and acquiring read = OK. Holding read lock and acquiring write = DEADLOCK on the same thread.
• StampedLock has optimistic read: no lock acquisition, validate after the read. Falls back to read lock if validation fails.
• StampedLock is NOT reentrant. Calling readLock twice on the same thread deadlocks.
• Both are heavier than synchronized for low-contention or write-dominant code. Measure.

Interview Follow-up Questions

Gotchas

• Read lock cannot be upgraded to write lock on ReentrantReadWriteLock. Drop and re-acquire.
• StampedLock readers must be side-effect-free and cheap. Optimistic read may execute and then re-execute.
• StampedLock is NOT reentrant. Holding read or write and re-acquiring the same hangs.
• Unfair ReadWriteLock can starve writers under sustained read load.
• For very short critical sections, plain synchronized often beats both.

ForkJoinPool & Parallel Streams

Section: Java Concurrency | Difficulty: Intermediate | Frequency: Sometimes

ForkJoinPool is Java's work-stealing thread pool optimised for divide-and-conquer recursion. RecursiveTask / RecursiveAction split work, fork() schedules, join() waits. Parallel streams run on the common pool. Sharp tool: useful for CPU-bound recursive work, terrible for blocking I/O.

Key Points

• Work stealing: each worker has its own deque; when idle, it steals from another worker's deque tail.
• Designed for divide-and-conquer with no blocking. Tasks split until small, compute, combine.
• Common pool is shared across the JVM. Parallel streams use it. Blocking inside it starves everyone.
• Default parallelism = available cores - 1. Override with -Djava.util.concurrent.ForkJoinPool.common.parallelism.
• For CPU-bound recursive work, fork-join can outperform a fixed thread pool. For I/O, use a dedicated executor.

Interview Follow-up Questions

Gotchas

• Blocking inside the common pool starves every other parallel stream caller
• Recursive tasks that share mutable state defeat the model. Compose pure functions.
• Parallel stream parallelism = commonPool size, NOT number of stream sources
• fork() then immediately join() on the same task is just compute(); fork-join shines when one side runs inline and the other forks
• Setting common.parallelism too high (>cores) wastes context switches; too low (=1) defeats the point

Structured Concurrency (Java 21+)

Section: Java Concurrency | Difficulty: Advanced | Frequency: Sometimes

StructuredTaskScope (Java 21 preview, finalised in 25) ties subtasks to a parent scope. The scope owns lifetimes: when it exits, all subtasks are cancelled or joined. Replaces the unstructured spawn-and-forget pattern of plain ExecutorService for fan-out work. Pairs naturally with virtual threads.

Key Points

• Subtasks are scoped to a try-with-resources block. Scope close = all subtasks cancelled or joined.
• Fork-join is structured: parent waits for children, children inherit deadlines and cancellation.
• anySuccessful joiner: first success cancels siblings. Used for hedged calls, fastest-replica reads.
• allSuccessfulOrThrow: all must succeed or all are cancelled. Used for fan-out where every result matters.
• Pairs naturally with virtual threads: cheap to spawn 1000 subtasks, scope cleans up reliably.

Interview Follow-up Questions

Gotchas

• scope.join() must be called before reading any subtask result. Otherwise IllegalStateException.
• Subtask.get() before join() also throws. Order matters.
• Forgetting try-with-resources defeats the structuring. Always use the resource block.
• Joiners are typed by result type. Mixing forks with different return types complicates the joiner.
• Subtasks inherit the parent's interrupt status. Cancel propagates downward through nested scopes.

JMH: Benchmarking Concurrent Java

Section: Java Concurrency | Difficulty: Advanced | Frequency: Sometimes

JMH (Java Microbenchmark Harness) is the only Java microbenchmarking tool worth trusting. Hand-rolled timing loops give meaningless numbers because the JIT optimises away the work, dead-code-eliminates results, and warm-up effects dominate. JMH manages all of that and produces statistically sound measurements.

Key Points

• JIT warmup matters: cold code runs interpreted, warm code is fully JIT-compiled. Measure only warm.
• Dead code elimination silently deletes unconsumed results. Use Blackhole or return values.
• Loop unrolling and constant folding can vaporise simple benchmarks. Inputs must come from @State.
• @Threads(N) measures contention. Per-thread state vs shared state matters.
• Reported numbers are op/sec or ns/op with confidence intervals. Trust the CIs.

Interview Follow-up Questions

Gotchas

• Hand-rolled timing loops are almost always wrong. Use JMH.
• Returning the result is the easy way to defeat DCE; use Blackhole when a return is not possible
• @State scope decides shared vs per-thread; getting it wrong measures the wrong thing
• Single-fork results are noisy; multiple forks expose JVM-to-JVM variance
• Runtime in dev IDE is misleading; build the JAR and run from CLI

The Python GIL, What It Is and Why It Matters

Section: Python Concurrency | Difficulty: Intermediate | Frequency: Asked Often

The Global Interpreter Lock is a mutex that serialises execution of Python bytecode in CPython. It makes single-threaded code fast and simple, but means threads cannot run Python code in parallel, even on a 64-core machine. Reach for multiprocessing for CPU-bound work, threading or asyncio for I/O.

Key Points

• The GIL is a CPython implementation detail, not part of the language spec
• Threads release the GIL around blocking I/O calls, so threading IS useful for I/O work
• Threads do NOT help CPU-bound code in CPython, they serialise via the GIL
• GIL switches between threads roughly every 5ms (sys.setswitchinterval)
• PyPy, Jython, and IronPython have no GIL; CPython 3.13+ has experimental free-threaded mode (PEP 703)
• Even with the GIL, common bytecode operations like += are NOT atomic, bytecode boundaries can interleave

Interview Follow-up Questions

Gotchas

• Adding threading to CPU-bound code makes it slower, measure before adding
• multiprocessing on Windows/macOS uses 'spawn' by default, globals from main aren't copied
• Pickle errors when passing un-picklable objects (lambdas, locks) to ProcessPoolExecutor
• asyncio + a CPU-heavy synchronous call → entire event loop stalls until it finishes
• Mixing threads and asyncio is a footgun, prefer asyncio.to_thread() for blocking calls
• The GIL is released around blocking I/O, but NOT around CPU loops in pure Python

threading vs multiprocessing, Picking the Right Tool

Section: Python Concurrency | Difficulty: Intermediate | Frequency: Asked Often

threading shares memory and is great for I/O-bound work because the GIL releases around blocking calls. multiprocessing spawns separate processes that bypass the GIL, the only way to get real CPU parallelism in stock CPython.

Key Points

• threading: shared memory, light, GIL-bound, pick for I/O concurrency
• multiprocessing: separate memory, ~50ms startup each, no GIL, pick for CPU parallelism
• concurrent.futures gives a unified API over both, Executor, submit(), map(), Future
• Process startup method differs by OS: 'fork' on Linux (fast, copy-on-write), 'spawn' on macOS/Windows (slower, isolated)
• multiprocessing requires picklable arguments, lambdas, locks, sockets won't cross the boundary
• Process pools are best for medium-large CPU tasks; small tasks lose to IPC overhead

Interview Follow-up Questions

Gotchas

• Forgetting `if __name__ == '__main__':` guard on Windows, re-imports module recursively, infinite spawn
• Module-level state set by the parent isn't visible to spawn-mode children, must pass explicitly
• Pickle errors on lambdas, use def or functools.partial
• ProcessPoolExecutor swallows worker crashes, wrap in try/except and check fut.exception()
• fork on macOS is officially discouraged (3.8+) due to subtle bugs with system libraries
• Closing a Pool inside the with block is automatic; outside, pool.close() + pool.join() are required or workers leak

asyncio Fundamentals

Section: Python Concurrency | Difficulty: Intermediate | Frequency: Asked Often

asyncio runs many tasks on a single thread by interleaving them at await points. It scales to 100K+ concurrent I/O operations because each task is just a few KB instead of a thread's megabytes, but a single CPU-heavy synchronous call freezes the entire event loop.

Key Points

• asyncio is single-threaded, concurrency without parallelism
• await is a yield point: control returns to the event loop, other tasks run
• Tasks are cheap (~few KB); spawning 100K is realistic
• ANY blocking call (requests.get, time.sleep, DB driver) stalls the entire event loop
• asyncio.to_thread() bridges blocking code into the async world
• The library being called must be async-aware, sync libraries don't 'become' async

Interview Follow-up Questions

Gotchas

• Forgetting `await` calls a coroutine but doesn't run it; emits a 'coroutine was never awaited' warning
• Calling sync DB drivers inside async, silent event loop stall, tail latency spikes
• Swallowing CancelledError, breaks asyncio.wait_for and graceful shutdown
• Creating tasks without awaiting/storing them, task may be garbage collected mid-run
• Mixing asyncio.run() inside another running event loop, RuntimeError; use await or get_event_loop()
• Using time.sleep in async code; asyncio.sleep is the right tool (the sync one blocks the loop)

Python threading: Lock, RLock, Condition, Event, Barrier

Section: Python Concurrency | Difficulty: Intermediate | Frequency: Asked Often

threading module ships the usual suspects: Lock (mutex), RLock (reentrant), Condition (wait/notify), Event (one-shot signal), Semaphore (counted permits), Barrier (rendezvous N threads). The GIL makes Python thread interleaving look atomic for single bytecodes but not for multi-step operations, so these primitives are still needed.

Key Points

• Lock acquired twice on the same thread = deadlock. RLock allows recursive entry.
• Even with the GIL, operations like dict[k] += 1 are NOT atomic (load, add, store across multiple bytecodes).
• Condition.wait() releases the lock and re-acquires on wakeup. Always pair with a predicate loop to handle spurious wakeups.
• Event is the right tool for one-time signals (shutdown flag, ready signal). Don't roll a custom Lock+flag.
• queue.Queue is built on these and provides a thread-safe producer-consumer for free.

Interview Follow-up Questions

Gotchas

• Holding a Lock across a yield from / await blocks the whole asyncio loop
• Forgetting `with` and using acquire/release manually leaks locks on exception
• Condition.wait() must be called inside a loop on the predicate; single 'if' is wrong
• Barrier with action: the action runs ONCE on whichever thread arrives last; exceptions break the barrier
• BoundedSemaphore raises if release() exceeds initial count; Semaphore silently allows it

queue Module: Queue, LifoQueue, PriorityQueue

Section: Python Concurrency | Difficulty: Basic | Frequency: Asked Often

queue.Queue is the standard thread-safe FIFO. LifoQueue is a stack. PriorityQueue is a heap. SimpleQueue is FIFO without size bound, optimised for short-lived. All handle the locking internally; pair with a sentinel (None) to signal shutdown to consumers.

Key Points

• queue.Queue is thread-safe by design. No external lock needed.
• maxsize > 0 makes the queue bounded; put() blocks when full. Backpressure for free.
• Standard shutdown: producer puts a sentinel (None); consumer breaks when it sees one.
• task_done() + join() let the producer wait for all submitted work to be processed.
• queue.Queue is for THREADS. asyncio has its own asyncio.Queue. multiprocessing has multiprocessing.Queue.

Interview Follow-up Questions

Gotchas

• queue.Queue is for threads only; asyncio coroutines must use asyncio.Queue
• PriorityQueue tuples need a tiebreaker; otherwise equal priorities try to compare payloads
• task_done() called more times than put() raises ValueError
• maxsize=0 is the default and means UNBOUNDED. Always set explicitly for production.
• join() blocks the calling thread; if no consumers are running, it blocks forever

concurrent.futures: ThreadPoolExecutor & ProcessPoolExecutor

Section: Python Concurrency | Difficulty: Intermediate | Frequency: Asked Often

concurrent.futures is the high-level API for running callables on a thread or process pool. ThreadPoolExecutor for I/O-bound work (the GIL is released during blocking I/O). ProcessPoolExecutor for CPU-bound work (one Python interpreter per process, true parallelism). Submit returns a Future; map streams results in input order.

Key Points

• ThreadPoolExecutor: I/O-bound (HTTP, DB, file). The GIL releases during blocking syscalls so multiple threads make progress.
• ProcessPoolExecutor: CPU-bound (parsing, image processing, math). True parallelism, but pickling cost on every call.
• executor.map preserves input order. as_completed yields in completion order. Pick by what the downstream consumer needs.
• Always use a context manager (with). Forgetting to shutdown leaks worker threads or processes.
• Default max_workers: min(32, cpu_count+4) for ThreadPool, cpu_count for ProcessPool. Tune to the workload.

Interview Follow-up Questions

Gotchas

• Forgetting `with` context manager leaks workers (especially processes, which keep file handles open)
• ProcessPoolExecutor on Windows uses spawn, not fork; everything in __main__ guard
• executor.map raises only on iteration; exceptions in early items are deferred
• Default max_workers is often wrong for the workload; size based on bottleneck (cores for CPU, latency * QPS for I/O)
• Cancelled futures still complete pending work in the queue if other threads pick them up

asyncio Patterns: gather, TaskGroup, Semaphore

Section: Python Concurrency | Difficulty: Intermediate | Frequency: Asked Often

asyncio.gather runs awaitables concurrently and returns results in input order. TaskGroup (Python 3.11+) is the structured-concurrency replacement: better cancellation, better error propagation. asyncio.Semaphore caps in-flight concurrency. asyncio.timeout adds a deadline. These four cover most async fan-out patterns.

Key Points

• gather runs N awaitables concurrently; result is a list in input order.
• TaskGroup replaces gather for new code: any task failure cancels siblings, the group raises ExceptionGroup.
• asyncio.Semaphore is the concurrency limit. Without it, gather over 10,000 URLs spawns 10,000 connections.
• asyncio.timeout(s) is a context manager. The whole block must finish in s seconds or it cancels.
• Bare `asyncio.create_task(coro)` without keeping a reference may be garbage-collected mid-flight.

Interview Follow-up Questions

Gotchas

• create_task without keeping a reference may be GC'd before completion
• gather with return_exceptions=False raises immediately; siblings keep running
• Forgetting `await` on an async function returns a coroutine, not a result
• Calling sync blocking code (time.sleep, requests.get) from async blocks the event loop
• asyncio.Lock is for asyncio coroutines; threading.Lock will block the event loop

ContextVar: Async-Safe Request Context

Section: Python Concurrency | Difficulty: Intermediate | Frequency: Sometimes

ContextVar (Python 3.7+) is the async-safe replacement for threading.local. Each task gets its own isolated copy of the variable; values set in a parent task propagate to children but mutations don't leak back. The right tool for request IDs, trace IDs, user context, and any per-request state in async code.

Key Points

• Each asyncio Task gets a copy of the parent's context. Sets in the task don't affect siblings or parent.
• Replaces threading.local for async: thread-locals don't isolate across asyncio tasks on the same thread.
• Used for request-scoped data: trace ID, request ID, authenticated user, tenant ID.
• var.set() returns a Token; pass it to var.reset(token) to restore the previous value (rare in async code).
• Frameworks (FastAPI, Starlette, structlog) use ContextVar internally for request-scoped state.

Interview Follow-up Questions

Gotchas

• threading.local in async code: tasks on the same thread share state, race city
• ThreadPoolExecutor.submit does NOT copy context; trace IDs disappear in pool work
• var.set() inside a finally block of a task does NOT affect the parent (per-task isolation)
• Forgetting default + var.get() raises LookupError; catch or pre-set in middleware
• Context is captured at create_task time, not at await time; mutations between create and await don't propagate

Multiprocessing Deep Dive

Section: Python Concurrency | Difficulty: Advanced | Frequency: Sometimes

multiprocessing spawns separate Python processes to escape the GIL. Pool, Process, Queue, Pipe, Manager, shared memory. The catch: every argument and return value is pickled across the boundary, start methods (fork/spawn/forkserver) behave very differently, and shared state requires explicit machinery.

Key Points

• Each process has its own Python interpreter and its own GIL. True CPU parallelism.
• Arguments and return values must be picklable. Lambdas, closures, file handles, locks all fail.
• Start method matters: fork (Unix default, fast, copy-on-write) vs spawn (Windows, slower, fresh interpreter) vs forkserver.
• Shared state needs Manager (proxied, slow) or shared_memory (fast, raw bytes).
• multiprocessing.Pool has the same map/imap/apply API as concurrent.futures' ProcessPoolExecutor; the latter is usually preferred.

Interview Follow-up Questions

Gotchas

• Forgetting `if __name__ == '__main__':` on Windows/macOS spawn causes infinite recursion
• Lambdas, local functions, and closures cannot be pickled; use module-level callables
• Manager proxies are SLOW; every read or write is an IPC round trip
• fork after threading is undefined behaviour in many libraries; prefer spawn or forkserver
• Forgetting shm.unlink() leaks shared memory until reboot on some platforms

Bridging Sync and Async

Section: Python Concurrency | Difficulty: Advanced | Frequency: Sometimes

asyncio.to_thread runs sync code on a thread without blocking the event loop. asyncio.run_coroutine_threadsafe runs async code from a thread. asgiref.sync provides sync_to_async / async_to_sync helpers. Mixing the two worlds is messy; isolate the boundary and pick one direction per layer.

Key Points

• Calling sync blocking code from async (without to_thread) freezes the event loop.
• Calling async code from sync (without proper bridging) raises RuntimeError or silently never runs.
• asyncio.to_thread offloads the sync call to the default thread pool. Returns awaitable.
• From a non-event-loop thread, asyncio.run_coroutine_threadsafe schedules a coroutine on the loop and returns a concurrent.futures.Future.
• When in doubt, isolate the boundary: a layer is fully sync OR fully async, never mixed.

Interview Follow-up Questions

Gotchas

• Calling time.sleep / requests.get / sync DB drivers from async blocks the event loop
• asyncio.run inside a running loop raises RuntimeError; only call from sync code
• asyncio.run_coroutine_threadsafe needs a loop already running on another thread
• to_thread workers come from a fixed pool; thousands of concurrent to_thread calls queue up
• asgiref helpers create per-call loops if needed; expensive in tight loops

PEP 703: No-GIL Python

Section: Python Concurrency | Difficulty: Advanced | Frequency: Sometimes

PEP 703 makes the GIL optional. Free-threaded CPython (3.13+ build option, supported but not default in 3.14) lets multiple Python threads run bytecode in parallel. Threads finally help CPU-bound work. Cost: per-object reference counting needs atomics, single-threaded performance regresses ~10-15%, C extensions need rebuilding.

Key Points

• Free-threaded Python (3.13+) builds without the GIL. Threads achieve true parallelism.
• Only available in a separate build (python3.13t). Default CPython still has the GIL.
• C extensions must opt in via Py_mod_gil. Old extensions trigger a re-enabling of the GIL.
• Single-threaded perf regresses (~10-15% in 3.13) due to atomic reference counting and biased locking on objects.
• Multi-threaded scaling on CPU-bound code finally works without multiprocessing.

Interview Follow-up Questions

Gotchas

• Single-threaded perf regresses ~10-15% on free-threaded build (atomic refcounts, biased locking overhead)
• Loading any GIL-required C extension re-enables the GIL process-wide
• Race conditions hidden by GIL serialisation become visible; correctness audits matter
• Free-threaded build is python3.13t (separate binary); pip install wheels are limited
• Some libraries gate behaviour on sys.version; check for free-threaded build separately

Connection Pool

Section: Real-World Scenarios | Difficulty: Intermediate | Frequency: Asked Often

A pool of pre-established connections (database, HTTP, gRPC) reused across requests. Building a connection per request is too slow (TLS handshake, TCP setup, auth). Pool provides bounded resource usage, faster requests, and back-pressure when the limit is reached.

Key Points

• Connection establishment is expensive: TCP handshake, TLS handshake, auth. Pool amortises this.
• Pool size = max concurrent in-flight requests. Sized by downstream limit, not by app traffic.
• Acquire blocks (or fails) when pool is empty; this is the back-pressure mechanism.
• Health check: validate connection on borrow; broken connections must be discarded, not handed out.
• Idle timeout: connections that have been idle too long are closed (server may have closed them already).

Interview Follow-up Questions

Gotchas

• Pool size larger than database can handle: every request fails when DB max_connections hit
• Forgetting to release connection (no try-with-resources): pool leak, eventual exhaustion
• No idle timeout: server-closed connections stay in the pool, fail on next borrow
• Long-running query holds a connection: pool exhausted by 1 slow query
• Same pool for read and write: write spike starves reads, or vice versa

Background Job Queue

Section: Real-World Scenarios | Difficulty: Intermediate | Frequency: Asked Often

Persist jobs to durable storage (Postgres, Redis, SQS), workers pull and process them. Need: idempotent handlers (jobs can be delivered twice), visibility timeout (a worker that crashes mid-job must release the job for retry), exponential retry with dead-letter queue, scheduled/delayed jobs, priority queuing.

Key Points

• Durable storage so jobs survive restart. Memory-only queues lose work on crash.
• Visibility timeout: when a worker pulls a job, hide it for N seconds. If the worker doesn't ack, it reappears.
• Idempotency on the handler side: jobs can be delivered more than once (visibility timeout expiry, worker crashes after work but before ack).
• Dead-letter queue (DLQ) for jobs that fail repeatedly. Don't retry forever; surface to humans.
• Sidekiq/Resque (Redis), RQ (Redis), Celery (RabbitMQ or Redis broker), BullMQ (Redis), AWS SQS, and Postgres SKIP LOCKED are popular implementations. Only Sidekiq/RQ/BullMQ are Redis-native; Celery's primary broker is RabbitMQ (Redis is supported but not the default).

Interview Follow-up Questions

Gotchas

• No visibility timeout: worker crash mid-job means the job is lost OR re-runs immediately and floods retries
• Non-idempotent handler with at-least-once delivery: side effects double on retry
• Unbounded retries: a permanently failing job retries forever, drowning out healthy work
• No DLQ inspection: dead jobs accumulate, no one notices the silent failure
• Treating queue as a database: long-running jobs hold locks, block compaction, slow queries

In-Memory Cache with TTL Invalidation

Section: Real-World Scenarios | Difficulty: Medium | Frequency: Asked Often

Thread-safe cache where each entry expires after T seconds. Core decisions: lazy expiry vs background sweep; ConcurrentHashMap vs RWLock; cache-stampede prevention via single-flight. Tests understanding of expiration semantics + concurrency together.

Key Points

• Two expiry strategies: lazy (check on read) vs active sweep (background timer)
• ConcurrentHashMap (Java) / sync.Map (Go) / dict + RLock (Python) for storage
• Cache stampede: when key expires, many concurrent reads all try to recompute → use single-flight
• Hot-key contention: shard the cache by key hash for high throughput

Interview Follow-up Questions

Token Bucket & Sliding Window Rate Limiters

Section: Real-World Scenarios | Difficulty: Medium | Frequency: Asked Often

A rate limiter caps request rate per key. Token bucket: refill N tokens/sec, requests consume one. Sliding window: count requests in the last second. Both need atomic counter updates and either lazy refill or scheduled refill, choice depends on traffic patterns and burst tolerance.

Key Points

• Token bucket: refill rate, max bucket capacity (allows bursts)
• Sliding window: log of timestamps, count those in [now-1s, now]
• Per-key state requires concurrent map; per-bucket update requires atomic
• Distributed rate limiting: Redis SETNX + INCR + EXPIRE, or atomic Lua script
• Real production: Stripe's algorithm uses sliding window + leaky bucket per dimension

Interview Follow-up Questions

Retry Library: Production Implementation

Section: Real-World Scenarios | Difficulty: Intermediate | Frequency: Asked Often

Build a retry library: classify errors (transient vs permanent), exponential backoff with jitter, max-attempts AND max-duration, per-call deadline budget, optional circuit breaker integration. Two ways to get it wrong: retry permanent errors, or retry without jitter.

Key Points

• Classify errors: only retry transient (5xx, timeout, connection refused). Never retry 4xx.
• Exponential backoff with FULL jitter: delay = random(0, base * 2^attempt).
• Bound both attempts AND total elapsed time. Either alone is not enough.
• Pass remaining deadline to each attempt; don't run a 5s call when 1s of budget remains.
• Compose with circuit breaker: retries hit the breaker first, fail fast if open.

Interview Follow-up Questions

Gotchas

• Retrying without a per-attempt timeout means slow downstream + slow retries = unbounded latency
• Retrying a non-idempotent POST without an idempotency key duplicates the side effect
• Stacked retries (application retry + the HTTP client's retry) multiply attempts exponentially
• Using elapsed time but not respecting context cancellation: retry continues after caller gave up
• Treating all exceptions as retryable: validation errors and OOM get retried too

Circuit Breaker: Production Implementation

Section: Real-World Scenarios | Difficulty: Intermediate | Frequency: Asked Often

Build a circuit breaker that wraps downstream calls. State machine: closed → (failures) → open → (cooldown) → half-open → (probe) → closed/open. Per-downstream instance. Trip on failure rate over a sliding window with a minimum-call floor. Half-open allows N probes; success rate decides if we close or reopen.

Key Points

• Three states: closed (normal), open (fail-fast), half-open (probing). Atomic state transitions.
• Trip policy: failure rate over a sliding window (last N calls or last T seconds), with a minimum-call floor.
• Cooldown in open state: 30s-5min typical. After cooldown, transition to half-open.
• Half-open allows N probes (typically 3-10). Track their outcomes; success rate decides next state.
• Per-downstream breaker. Never one global breaker. Different downstreams have different reliability.

Interview Follow-up Questions

Gotchas

• One global breaker for all downstreams: one bad downstream stops everything
• Counting client errors (4xx) as breaker failures: trips on user mistakes, not service health
• Half-open with no probe cap: surge of probe traffic on a recovering downstream
• No fallback when breaker opens: caller sees CircuitOpenException with no graceful path
• Breaker without timeout: slow calls can pile up before the breaker even sees them as failures

Idempotency Key: Stripe-Style Implementation

Section: Real-World Scenarios | Difficulty: Intermediate | Frequency: Asked Often

Server-side dedup of retries on non-idempotent operations. Client sends Idempotency-Key header (unique per logical operation). Server: lock on key → check cache → if cached, replay response → else process and cache → release. TTL on cache (24h typical). Detect key collisions via request fingerprint.

Key Points

• Client generates ONE key per logical operation, reuses on retry. NOT one per HTTP attempt.
• Server-side: lock on key (so concurrent retries don't both run), check cache, process, store response, release.
• Cache stores: response body + status code + relevant headers. Replay must be byte-identical.
• TTL: typically 24h. Long enough for client retries, short enough that storage doesn't grow forever.
• Detect key collisions: store a fingerprint (hash of request body) and compare. If different, return 422 instead of cached response.

Interview Follow-up Questions

Gotchas

• Generating new key per HTTP attempt: server sees each retry as a fresh request
• Caching only response body, not status code: replay returns wrong status
• No collision detection: same key for different operations replays wrong response
• No TTL on cached response: storage grows without bound
• Lock without TTL: crashed processor blocks subsequent retries forever

API Aggregator (Backend-for-Frontend)

Section: Real-World Scenarios | Difficulty: Intermediate | Frequency: Asked Often

Service that calls N downstream services concurrently and merges results into a single response. Common in mobile/web BFFs. Patterns: fan out with errgroup/TaskGroup/gather, per-call timeouts, partial-result tolerance (return what we have if some fail), per-downstream circuit breakers.

Key Points

• Fan out concurrently: total latency = max of downstreams, not sum.
• Per-call timeout: one slow downstream shouldn't drag the whole response.
• Partial-result tolerance: decide which downstreams are critical, which are optional.
• Per-downstream circuit breaker: when one is unhealthy, fail it fast and degrade gracefully.
• Cache aggressively: page-level cache and per-downstream cache reduce fan-out cost.

Interview Follow-up Questions

Gotchas

• Sequential awaits instead of concurrent fan-out: latency becomes sum, not max
• No per-call timeout: one slow downstream blocks the whole page
• Treating all downstreams as critical: any single failure breaks the page
• No circuit breaker per downstream: calls keep timing out for an unhealthy service
• No caching: every request fans out to all backends; database hammered

Polite Concurrent Web Crawler

Section: Real-World Scenarios | Difficulty: Hard | Frequency: Asked Often

Crawl 1M URLs concurrently while: (1) bounding total in-flight requests, (2) deduplicating visited URLs, (3) respecting per-domain politeness (max N concurrent per host). Bounded worker pool + concurrent set + per-domain semaphore.

Key Points

• Worker pool with N workers fetches from a shared work queue
• Concurrent set (visited) prevents fetching the same URL twice
• Per-domain semaphore caps requests-per-host (politeness)
• BFS-style traversal: each fetched page enqueues newly-discovered links
• Termination: when queue empty AND no in-flight workers

Interview Follow-up Questions

Lock Convoys

Section: Performance & Tradeoffs | Difficulty: Advanced | Frequency: Sometimes

A lock convoy happens when threads contending for a lock end up serialized in the kernel wait queue. The first thread runs and releases. The next thread is woken from a kernel sleep, which costs a context switch and a cache miss. The lock is fair, but throughput drops because every acquire pays a kernel round trip even when the critical section is microseconds. Common in fair mutexes under steady contention.

Key Points

• All N threads end up in the kernel wait queue. Each acquire pays a wakeup, a context switch, and cold-cache restart.
• Critical section is short, but wall-clock time per acquire is dominated by the wakeup cost (microseconds), not the section itself (nanoseconds).
• Triggered by fair locks (FIFO wakeup), high contention, and short critical sections. The trio is the warning sign.
• Symptom: throughput plateaus or drops as you add threads, but no thread is CPU bound; perf shows time inside futex/wait, not inside the critical section.
• Fixes: reduce contention (sharding, RCU, lock-free), prefer adaptive/unfair locks (default in glibc, Java ReentrantLock without fair=true), or restructure so the section is so short that nobody ever blocks.

Interview Follow-up Questions

Gotchas

• Adding a 'just to be safe' fair lock to fix a starvation complaint can tank throughput by 100x.
• Convoys are time-of-day sensitive: light load looks fine, peak load suddenly cliffs. Load-test under realistic concurrency.
• GC pauses and STW phases extend critical sections enough to push a healthy unfair mutex into convoy mode for a few seconds.
• Profilers that sample the running thread won't show the convoy: time is inside the kernel scheduler, not application code. Use mutex/contention profilers.

Scheduled Callback Executor (HashedWheelTimer)

Section: Real-World Scenarios | Difficulty: Hard | Frequency: Sometimes

Schedule callbacks to run at future times. Naive heap-based scheduler scales to thousands; for millions, use a HashedWheelTimer, bucketize callbacks by absolute deadline, sweep one bucket per tick. Used by Netty, Linux kernel timer wheel, Kafka.

Key Points

• Heap-based: O(log N) insert, O(log N) extract, fine for thousands
• Hashed wheel: O(1) insert, O(N/buckets) per tick, scales to millions
• Trade precision for throughput: wheel has tick granularity (~10ms)
• Single tick thread + worker pool dispatches expired callbacks

Interview Follow-up Questions

In-Process Pub-Sub Bus

Section: Real-World Scenarios | Difficulty: Hard | Frequency: Asked Often

An in-memory pub-sub: publishers send to topics, subscribers receive matching messages. Decisions: per-subscriber bounded queue (slow consumer doesn't block fast); ordered or fan-out; sync or async delivery. Trades latency for backpressure.

Key Points

• Topics → subscriber list; publish iterates and delivers (or queues per subscriber)
• Each subscriber has bounded queue → slow consumer doesn't backpressure publisher
• Drop-on-full vs block-on-full, slow-consumer policy is THE design decision
• Concurrent map for topic→subscribers (CHM, sync.Map, dict + RLock)
• Used by Spring Events, Akka Streams, in-process EventBus libraries

Interview Follow-up Questions

DAG Task Runner with Backpressure

Section: Real-World Scenarios | Difficulty: Hard | Frequency: Sometimes

Run tasks with dependencies, each task waits for its prerequisites. Topological-order scheduling + worker pool + bounded concurrency. Pattern: per-task CountDownLatch / Future / WaitGroup, with dependency graph driving submission.

Key Points

• Topological order: tasks run as soon as their dependencies complete
• Per-task synchronization: Future, CompletableFuture, CountDownLatch, errgroup
• Worker pool bounds concurrency; max-in-flight is the backpressure knob
• Failure propagation: if a task fails, downstream tasks are cancelled (or retried)
• Used by Apache Airflow, Argo Workflows, Make, build systems

Interview Follow-up Questions

Inventory Reservation with Concurrent Holds

Section: Real-World Scenarios | Difficulty: Hard | Frequency: Asked Often

Booking system: 100 concurrent users grab the last seat. Only one wins. The losers see 'sold out' before completing checkout. Pattern: optimistic locking with version-stamped CAS, OR pessimistic per-item lock, OR atomic decrement on a Redis counter.

Key Points

• Optimistic: version + CAS, read inventory, attempt update, retry if version changed
• Pessimistic: lock per item, serializes high-contention items but cleaner semantics
• Soft holds: reserve for N minutes, expire if checkout doesn't complete
• Distributed: Redis DECR, or Postgres SELECT FOR UPDATE, or two-phase commit

Interview Follow-up Questions

Distributed Lock: Redis or ZooKeeper

Section: Real-World Scenarios | Difficulty: Advanced | Frequency: Asked Often

Mutual exclusion across processes/machines. Three implementations: SETNX with TTL on Redis (cheap, mostly correct), Redlock (Redis cluster, controversial safety), ZooKeeper ephemeral nodes (strongly correct, more ops). All require a TTL/lease so a crashed holder doesn't block forever. Always design for: 'what if the holder thinks it has the lock but actually doesn't?'

Key Points

• Redis SET key value NX EX ttl is the simplest distributed lock. Atomic, TTL-based, fast.
• Lock holder must store a unique token (UUID) and check it on release; otherwise release-by-someone-else is possible.
• TTL is required: if the holder crashes, the lock auto-expires. Pick TTL > expected critical-section duration.
• Fencing tokens (monotonic ID) protect against zombies: 'I had the lock, then GC paused me, lock expired, someone else took it, then I came back and acted on stale belief'.
• ZooKeeper / etcd / Consul give stronger guarantees (linearisable, sequential ephemeral nodes) at higher operational cost.

Interview Follow-up Questions

Gotchas

• No TTL: crashed holder blocks the lock forever
• No token check on release: someone else's release deletes the lock
• Long-running work without lease extension: TTL expires mid-work, two holders concurrent
• Treating SETNX as linearisable: it is not under all failure modes
• No fencing token in critical paths: zombies can corrupt data after lock expiry

Heartbeat & Leader Election

Section: Real-World Scenarios | Difficulty: Advanced | Frequency: Asked Often

Heartbeat: each instance periodically writes 'I'm alive' to shared storage. Leader election: instances compete to become the single coordinator; loser instances watch the leader's heartbeat. When the leader's heartbeat stops, a new election runs. Backed by ZooKeeper, etcd, Consul, or a simpler Redis lease.

Key Points

• Heartbeat: leader writes a key with TTL every N seconds; if it stops writing, the key expires.
• Election: candidates try to atomically claim the leader key. The winner is the leader. Losers watch.
• Lease > heartbeat interval × 3: tolerates transient network blips without flapping.
• Always design for split-brain: two instances that both think they are leader. Fencing tokens prevent corruption.
• ZooKeeper, etcd, Consul, Kubernetes leases give correct primitives. Redis with TTL is simpler but has known edge cases.

Interview Follow-up Questions

Gotchas

• TTL too short: harmless network blip causes leadership flap
• Renewing on a separate connection: connection drops independently, lease lost while leader is fine
• Doing work after losing leadership: the new leader is also writing; conflict
• No fencing token: stale leader corrupts data after lease expiry
• Single-region election with no fencing: cross-region partition can have two regional leaders

Saga Pattern: Distributed Transactions Without 2PC

Section: Real-World Scenarios | Difficulty: Advanced | Frequency: Asked Often

A saga executes a multi-step business transaction across services as a sequence of local transactions, each with a compensating transaction that undoes it. If any step fails, run the compensations for the steps already done. Replaces two-phase commit (2PC) for the microservices era. Two flavours: orchestrated (a coordinator drives) or choreographed (each service reacts to events).

Key Points

• Local transactions per step. Each has a compensating transaction (cancel order, refund payment, restore inventory).
• On failure at step N, run compensations for steps N-1, N-2, ..., 1 in reverse order.
• Compensations must be idempotent: they may be replayed. Use idempotency keys.
• Orchestrated: one coordinator decides next step, calls each service, handles failures. Easier to reason about.
• Choreographed: each service publishes events, others react. Decouples services but harder to debug.

Interview Follow-up Questions

Gotchas

• Forgetting idempotency keys: replays double-charge, double-refund
• Not persisting state between steps: coordinator crash loses progress
• Choreographed saga across many services: workflow is invisible, debugging is awful
• Treating compensations as 'just like forward, in reverse': they're not, semantics differ (refund != negative charge)
• No alert on compensation failure: silent inconsistency

Outbox Pattern: Reliable Event Publishing

Section: Real-World Scenarios | Difficulty: Advanced | Frequency: Asked Often

Write the event to an 'outbox' table in the SAME transaction as the business write. A separate process polls the table and publishes events to Kafka/queue/webhook. Solves the dual-write problem: a database AND a message broker can't be written to atomically, so events would otherwise be lost or duplicated when one of the writes fails.

Key Points

• Dual write problem: 'INSERT order' + 'publish OrderCreated event' is two writes, no atomic guarantee. Either can fail.
• Outbox: write the event row in the SAME DB transaction as the business write. Either both happen or neither.
• A relay process polls the outbox, publishes each event to the broker, marks it published (or deletes it).
• At-least-once: the relay can publish, crash, restart, publish again. Subscribers must be idempotent.
• CDC alternative: tools like Debezium read the database WAL and publish change events without an outbox table.

Interview Follow-up Questions

Gotchas

• Forgetting to put the outbox INSERT in the same transaction defeats the pattern
• Relay marks events 'sent' before actually publishing: events lost on crash
• Single relay instance: bottleneck and SPOF; run multiple with SKIP LOCKED
• Letting the outbox table grow unbounded: relay queries slow down, table bloats
• Consumers not idempotent: at-least-once delivery causes duplicate side effects

Cancellation Propagation Across Services

Section: Real-World Scenarios | Difficulty: Advanced | Frequency: Asked Often

When a request is cancelled (client disconnect, deadline exceeded, parent abort), the cancellation must propagate through every downstream call so wasted work stops everywhere. gRPC ships deadlines in headers; HTTP relies on connection close detection; Go does it with context; Java with CompletionStage / virtual thread interrupts. Done wrong, zombie work ties up resources after the user has gone.

Key Points

• Cancellation is end-to-end: a single layer that ignores it leaves zombie work running
• Client disconnect: the server must detect socket close and stop processing (HTTP request.context() in Go, request.is_disconnected() in Starlette)
• gRPC propagates a deadline header (grpc-timeout); intermediate services subtract elapsed time and pass the remainder downstream
• HTTP has no standard deadline header; pass via X-Request-Timeout or rely on connection lifetime
• Forgotten cancellation = thread/goroutine pool exhaustion as zombies pile up

Interview Follow-up Questions

Gotchas

• Forgetting to pass ctx to a downstream call: that layer ignores the deadline, zombies pile up
• Long CPU loops without ctx checks: cancellation can't interrupt CPU work, only I/O
• Deadline too tight at the edge: real requests time out under normal load
• Deadline too loose at the edge: zombies pile up before deadline hits
• Treating cancellation as exceptional: it's a normal part of every request lifecycle

CPU-bound vs I/O-bound

Section: Performance & Tradeoffs | Difficulty: Basic | Frequency: Asked Often

A CPU-bound task spends most of its time doing computation; an I/O-bound task spends most of its time waiting. Different workloads need really different concurrency strategies, and picking the wrong one makes the code slower, not faster.

Key Points

• CPU-bound: bottleneck is compute. Speedup requires more cores OR faster code.
• I/O-bound: bottleneck is waiting (network, disk, DB). Speedup requires more concurrency to overlap waits.
• Threads help I/O-bound work even with the GIL, Python releases the GIL around blocking calls
• Threads do NOT help CPU-bound work in Python (GIL serializes execution), use multiprocessing
• Async/await scales I/O-bound to 100K+ tasks but does nothing for CPU-bound
• Right pool size: CPU-bound ≈ core count; I/O-bound ≈ much higher (often 100s)

Interview Follow-up Questions

Gotchas

• A function can be CPU-bound in one workload and I/O-bound in another, measure on actual data
• Hashing/JSON-parsing/compression in 'I/O' code paths is often the hidden CPU bottleneck
• GPU-accelerated compute is technically I/O from the CPU's perspective, different concurrency model again
• Network I/O can become CPU-bound at very high throughput (parsing dominates)
• asyncio + a CPU-heavy task = stalled event loop and timeouts on every concurrent task

Throughput vs Latency Tuning

Section: Performance & Tradeoffs | Difficulty: Intermediate | Frequency: Asked Often

Throughput = ops/sec the system can sustain. Latency = time per op. They trade off via batching, queuing, and concurrency. Higher concurrency → higher throughput but worse tail latency. Little's Law: average concurrency = throughput × average latency.

Key Points

• Throughput: system-level capacity (req/sec)
• Latency: per-op time, especially p50/p99/p99.9 percentiles
• Little's Law: L = λW (concurrent in-flight = throughput × avg latency)
• Batching trades latency for throughput (waits for batch, then processes many)
• Queue length grows when arrival rate exceeds service rate, exponential queueing

Interview Follow-up Questions

Gotchas

• Average latency hides tail, always look at p99/p99.9
• Adding threads beyond optimal N HURTS throughput (USL)
• Unbounded queues turn brief overload into permanent latency degradation
• Cache hit rate drops when concurrency increases (cache-line contention, eviction)

Context Switch Cost

Section: Performance & Tradeoffs | Difficulty: Intermediate | Frequency: Sometimes

OS thread context switch: ~5μs (kernel-mediated, save/restore registers, TLB flush). Goroutine / virtual thread context switch: ~hundreds of ns (user-space, no kernel transition). Async task switch: ~tens of ns (function call). Choosing the right unit affects throughput at scale.

Key Points

• OS thread switch: ~1-5μs typical, includes kernel mode switch + TLB pollution
• Goroutine / virtual thread switch: ~200-500ns, user-space only
• asyncio task switch: ~50-100ns, just a coroutine resumption
• At 1M req/sec, a 1μs context switch costs 1 second of CPU per second, 100% of one core
• Mass goroutine spawning is fine; mass thread spawning hits scheduler limits

Interview Follow-up Questions

Gotchas

• Mass-spawning OS threads (e.g., one per request) hits OS scheduler limits at ~10K threads
• vmstat cs only counts OS-level switches; goroutine and async switches invisible
• TLB pollution makes the FIRST few accesses after a switch much slower than steady-state
• Hyper-threading helps hide some thread-switch latency by running another logical core
• Synchronized blocks in Java pin virtual threads to carriers, losing the cheap-switch benefit

Memory Overhead Comparison

Section: Performance & Tradeoffs | Difficulty: Intermediate | Frequency: Sometimes

OS thread: ~1MB stack reserved. Goroutine: ~2KB initial, grows on demand. Virtual thread (Java 21): ~few KB. asyncio task: ~1KB. The difference dictates whether you can run 10K vs 1M concurrent units on the same hardware.

Key Points

• OS thread: 1MB default stack reserved (most unused) → 10K threads = 10GB RSS
• Goroutine: 2KB initial, grows up to 1GB → 10K goroutines = 20MB
• Virtual thread (Java 21): ~few KB stack → 1M virtual threads ~ few GB
• asyncio task: ~1KB per task (Python) → 1M tasks ~ 1GB
• Per-thread/task overhead matters when you serve millions of concurrent connections

Interview Follow-up Questions

Gotchas

• 1M threads on Linux = ~1MB×1M = 1TB RSS, won't fit on most servers
• Async stacks live across awaits, long-lived tasks accumulate stack memory
• ThreadLocal × virtual threads = memory explosion (use ScopedValue in Java 21+)
• Goroutine stack growth costs CPU during the copy, usually invisible but profiles can show it

Amdahl's Law

Section: Performance & Tradeoffs | Difficulty: Intermediate | Frequency: Sometimes

Maximum speedup from parallelisation = 1 / (s + p/N), where s is serial fraction, p is parallel fraction, N is processor count. Even small serial fractions cap maximum speedup. 5% serial = 20x maximum, no matter how many cores. The implication: optimising the serial part is often the highest-leverage work.

Key Points

• Speedup(N) = 1 / (s + (1-s)/N). As N → ∞, speedup → 1/s.
• 5% serial = 20x maximum speedup. 1% serial = 100x. 10% = 10x.
• Real systems have multiple serial parts: I/O, lock contention, sequential bottlenecks.
• Highest-leverage optimisation: reduce the serial fraction, not just add cores.
• Universal Scalability Law (USL) extends Amdahl with a contention term, often more realistic.

Interview Follow-up Questions

Gotchas

• Adding cores past the Amdahl ceiling is wasted money
• 'Embarrassingly parallel' workloads still have a serial setup/teardown that limits speedup at scale
• Lock contention is a hidden serial fraction; profile to find it
• Per-core wins (better algorithm, less contention) sometimes beat throwing more cores
• Real systems are often USL-shaped (degradation past peak), not Amdahl-shaped (asymptote)

Universal Scalability Law (USL)

Section: Performance & Tradeoffs | Difficulty: Advanced | Frequency: Sometimes

USL extends Amdahl's Law with a coherency term. Speedup = N / (1 + α(N-1) + βN(N-1)). The β term captures the case where adding cores actually slows you down (cache coherency, lock contention). Real systems peak at some N and degrade beyond it. USL fits this; Amdahl doesn't.

Key Points

• α (contention): the part that serialises, like Amdahl's serial fraction.
• β (coherency): the cost of coordinating between cores; quadratic in N.
• Real systems often peak at finite N, then degrade. USL predicts the peak; Amdahl doesn't.
• Fitting USL to your benchmark data tells you where to invest: reduce contention OR coordination.
• Practical use: capacity planning. Know the peak before you scale past it and waste money.

Interview Follow-up Questions

Gotchas

• Fitting USL to too few data points (3-4) gives unreliable α, β estimates
• Treating the asymptote as the goal: real systems peak and decline; the peak is the goal
• Ignoring the curve shape: high α and high β need different fixes
• Adding cores past the USL peak makes you slower AND more expensive
• USL is per-workload; same hardware can have very different curves for different workloads

Tail Latency: Why p99 Matters More Than Average

Section: Performance & Tradeoffs | Difficulty: Advanced | Frequency: Asked Often

p99 (the 99th percentile latency) is the user experience; the average is a lie. Tail latency amplifies in fan-out: a request that fans out to 10 services has a p99 dominated by the slowest of 10 backends, not by their average. Mitigations: hedged requests, tied requests, micro-partitioning, smaller fan-out, better tail metrics.

Key Points

• Average hides outliers. The p99 IS the user experience for 1% of requests, which is millions when you serve millions.
• Tail amplification: a request fanning out to N backends has p99 ≈ max of N independent draws. With N=10, p99 ≈ p99.9 of one backend.
• Mitigations: hedged requests (send to two replicas, take first), tied requests (cancel the loser), micro-batching, fewer fan-outs.
• Coordinated omission: load testing tools that wait for slow responses miss tail latency. Use HDR Histogram, k6, or wrk2.
• Read 'The Tail at Scale' by Dean & Barroso (2013); foundational paper, still relevant.

Interview Follow-up Questions

Gotchas

• Reporting average latency in dashboards: hides the tail entirely
• Coordinated omission in load tests: reported p99 understates real p99 by 10-100x
• Fan-out without thinking about amplification: p99 of the user request is much worse than p99 of any backend
• Hedging non-idempotent operations: double-execution under network failures
• Prometheus summary metrics with default config: percentiles drift over the reporting window

Benchmarking Methodology

Section: Performance & Tradeoffs | Difficulty: Intermediate | Frequency: Sometimes

Microbenchmarks lie unless they're done carefully. Warm up the JIT/runtime. Defeat dead-code elimination. Use realistic input. Measure with confidence intervals (multiple runs). Benchmark the right thing (the operation, not the noise around it). End-to-end load tests for system-level decisions, microbenchmarks for code-level decisions.

Key Points

• Warmup is critical: JIT, branch predictor, caches all need to stabilise.
• Dead-code elimination silently deletes your benchmark; consume the result.
• Multiple runs in fresh JVMs (forks) catch JVM-to-JVM variance.
• Report confidence intervals; if they overlap, you can't claim one is faster.
• Microbenchmark vs load test: microbenchmark for 'is op A faster than op B'; load test for 'does the system handle 1000 RPS'.

Interview Follow-up Questions

Gotchas

• Hand-rolled timing loops give numbers that are off by 10x or more
• Single run = no idea of variance; could be off by a factor of 2
• Discarding the benchmark result lets DCE delete your loop
• Reporting averages without intervals hides 'no significant difference'
• Benchmarking in dev/IDE = unreliable; always run benchmarks from CLI

Measuring Lock Contention

Section: Performance & Tradeoffs | Difficulty: Advanced | Frequency: Sometimes

Lock contention is when threads spend significant time waiting for locks instead of running. Measure with: profiler block-time view, JFR/async-profiler lock events, perf-events for futex contention. The fix is usually one of: shrink the critical section, shard the lock, replace with lock-free, or remove shared state entirely.

Key Points

• Symptom: high CPU time spent in lock acquisition / blocked / parked states.
• Java: JFR events 'jdk.JavaMonitorEnter', async-profiler 'lock' mode, jstack thread dumps.
• Linux: perf trace -e 'sched:*' or perf record with off-cpu profiling.
• Fix priority: shrink critical section > shard lock > lock-free structure > redesign to avoid sharing.
• False sharing looks like contention; measure cache-miss rate to distinguish.

Interview Follow-up Questions

Gotchas

• CPU at 100%: contention is NOT the bottleneck; the system is CPU-bound, optimise the code
• CPU low but throughput limited: lock or I/O is the bottleneck; profile to find which
• Confusing high CPU with high contention: profile both on-CPU AND off-CPU
• Profiler overhead: high sampling rates can themselves cause perceived contention
• Optimising a non-bottleneck: makes the system more complex without helping

NUMA & Cache Effects

Section: Performance & Tradeoffs | Difficulty: Advanced | Frequency: Sometimes

Multi-socket servers have NUMA: each CPU socket has its own memory; accessing the other socket's memory is slower (1.5-3x). Pinning threads to NUMA nodes and allocating memory locally can dramatically improve performance for memory-bound workloads. False sharing is the related cache-line problem within a socket.

Key Points

• NUMA: each socket has its own memory bank. Cross-socket access is slower than local.
• Linux: numactl --hardware shows NUMA topology. numactl --cpunodebind=0 --membind=0 pins to node 0.
• JVM, Go runtime, Postgres can be NUMA-aware; check docs and tune.
• False sharing: two cores writing different fields on the same cache line cause cache-line bouncing.
• Fix false sharing with padding (@Contended in Java, _Alignas(64) in C) so independent fields land on different cache lines.

Interview Follow-up Questions

Gotchas

• Cross-socket memory access is 1.5-3x slower; bind threads and memory to the same node
• False sharing on hot atomics can dominate even simple workloads
• JVM and Go runtimes are mostly NUMA-blind by default; manual binding helps
• Large heaps that span sockets thrash; either pin or use multiple smaller processes
• Padding wastes memory; only apply where you've measured false sharing

Parallel Merge Sort & ForkJoin

Section: Performance & Tradeoffs | Difficulty: Medium | Frequency: Sometimes

Divide-and-conquer naturally parallelizes, recursive halves run on different cores. Java's ForkJoinPool is purpose-built for this; Go uses goroutines + WaitGroups; Python uses concurrent.futures. Threshold check stops parallelism for small subarrays, fork overhead exceeds the parallelism win.

Key Points

• Divide-and-conquer = natural parallelism (each half is independent)
• Threshold: stop forking when subarray < N (~~ 1000 elements)
• Java ForkJoinPool: work-stealing scheduler optimized for D&C
• Sequential merge step is unavoidable bottleneck, Amdahl's law
• Speedup is logarithmic vs linear: 8 cores ≠ 8x speedup

Interview Follow-up Questions

Gotchas

• ForkJoinPool.commonPool() is shared, heavy use starves parallel streams
• ProcessPoolExecutor pickles arguments, large arrays have IPC overhead
• Threshold too low → fork overhead dominates
• Threshold too high → underutilization on big arrays
• Parallel merge sort has O(n) extra memory for the merge buffer

Parallel MapReduce: Word Frequency Counter

Section: Performance & Tradeoffs | Difficulty: Medium | Frequency: Sometimes

Map: each worker processes a chunk of input → partial counts. Reduce: merge partial counts into final result. Map step is embarrassingly parallel; reduce step is the synchronization point. Java's parallelStream, Python's multiprocessing.Pool.map, Go's errgroup, all express this.

Key Points

• Map: chunk input, each worker computes partial result independently
• Reduce: merge partial results, must be associative (and ideally commutative)
• Embarrassingly parallel: no inter-worker communication during map
• Reduce strategies: tree-reduce (parallel), sequential merge, ConcurrentHashMap
• Foundation of distributed computing, Hadoop, Spark, BigQuery all extend this

Interview Follow-up Questions

Gotchas

• Mutable accumulators (like StringBuilder) need synchronization or per-worker copies
• parallelStream uses commonPool, heavy use starves other parallelStream callers
• Reduce step can be the bottleneck if maps are huge, consider tree-reduce
• Float addition is not strictly associative, parallel reduce gives slightly different results than sequential

Parallel Graph Traversal (BFS/DFS)

Section: Performance & Tradeoffs | Difficulty: Hard | Frequency: Sometimes

Parallel BFS: process each level in parallel; serial frontier-to-frontier transitions. Parallel DFS: harder due to depth-recursion; usually use task-pool of subgraphs. Concurrent visited-set required either way; ConcurrentHashMap or Bloom filter.

Key Points

• BFS parallelizes naturally per-level, all nodes at level k processed concurrently
• DFS harder, recursive nature serializes; use bounded task pool of subgraphs
• Visited set must be concurrent, CHM, sync.Map, atomic bitset
• Frontier transitions are barriers, synchronize all workers between levels
• Used in PageRank, social-graph queries, dependency resolution at scale

Interview Follow-up Questions

Gotchas

• If workers don't synchronize between levels, processing crosses levels and BFS becomes wrong
• Concurrent map's `add` semantics matter, use `newKeySet().add()` (returns true if newly added) for dedup
• Level barrier requires ALL workers from level k to finish before level k+1 starts, wg.Wait()
• DFS isn't level-based, naive parallelization breaks ordering
• Memory-bound when frontier is huge, bound concurrency to avoid OOM

Lock-Free Programming, CAS Loops

Section: Lock-Free Programming | Difficulty: Advanced | Frequency: Sometimes

Lock-free algorithms use atomic compare-and-swap (CAS) instead of mutexes. The pattern: read current value, compute new value, CAS-install, retry on failure. At least one thread always makes progress (no deadlocks, no waiting on a slow holder), but individual threads can starve under contention. Pays off when contention is high and locks would serialise.

Key Points

• Lock-free = at least one thread is always making progress (system-wide). Individual threads may retry many times.
• The CAS retry loop is the building block: load → compute → CAS → retry on failure.
• No deadlocks possible (no thread blocks on another). No priority inversion. No waiting on a slow holder.
• Costs: ABA problem in pointer-based structures, memory reclamation in non-GC languages, cache-line contention under hot CAS.
• Use when locks are measurably the bottleneck. For most code, mutexes are simpler AND faster.

Interview Follow-up Questions

Gotchas

• Treating 'lock-free' as 'always faster', measure first; mutexes often win for simple cases
• ABA in pointer structures, without GC, requires extra machinery
• Hot CAS contention causes cache-line ping-pong; lock-free counters can be slower than sharded
• Multi-field invariants need locks or pointer-swap; CAS only protects one word
• Lock-free does NOT mean wait-free; individual threads can starve

Memory Ordering: Acquire, Release, Relaxed, SeqCst

Section: Lock-Free Programming | Difficulty: Advanced | Frequency: Sometimes

C++/Rust let the programmer specify how strict an atomic operation's memory ordering must be. SeqCst (sequentially consistent) is strongest and slowest. Acquire-release is the right default for publishing data. Relaxed gives no ordering, only atomicity (counters). Java and Go atomics are SeqCst by default; there's no choice to make.

Key Points

• SeqCst: every thread sees the same global order of all SeqCst operations. Strongest, slowest.
• Acquire (on load) + Release (on store): publish-subscribe pattern. Writes before release are visible after acquire.
• Relaxed: atomic, but no ordering. Use for counters where order doesn't matter.
• Java atomics and Go's sync/atomic give SeqCst always. C++ and Rust allow per-operation choice.
• Wrong relaxation = wrong code. Acquire-release is the default if unsure; relax only after measuring AND reasoning carefully.

Interview Follow-up Questions

Gotchas

• Mixing relaxed with code that depends on ordering = subtle bugs
• Acquire on store / release on load = compiler error; can't apply to wrong direction
• Java atomics are SeqCst; they can't be relaxed
• Memory ordering is ABOUT the compiler AND the CPU; both can reorder
• Reasoning by 'it works on my machine' fails on weakly-ordered ARM

The ABA Problem & Tagged Pointers

Section: Lock-Free Programming | Difficulty: Advanced | Frequency: Asked Often

ABA: thread A reads pointer P (= A); thread B pops A and pushes a new node at the same address; thread A's CAS succeeds even though the structure changed. Solutions: tagged pointers (pack version counter), hazard pointers (announce in-use), epoch-based reclamation, GC.

Key Points

• Pure CAS is unsafe when the same address can be reused (typical in C/C++ with manual memory)
• Java/Go avoid ABA via GC, freed memory not reused while references exist
• Tagged pointers: pack a version counter into unused pointer bits (low 3 bits typically)
• Hazard pointers: each thread declares pointers it's actively using; readers don't free those
• Epoch-based reclamation: defer free until all threads have moved past the freeing epoch

Interview Follow-up Questions

Gotchas

• Reusing nodes from a pool reintroduces ABA even in GC languages
• AtomicStampedReference adds overhead, use only when ABA is a real risk
• Tagged pointers fail on architectures with 64-bit virtual addresses (some ARM)
• Hazard pointers add 1 cache line per thread per concurrent operation, memory cost

Treiber Stack: The Canonical Lock-Free Stack

Section: Lock-Free Programming | Difficulty: Advanced | Frequency: Sometimes

Lock-free stack using CAS on the head pointer. Push: build new node pointing at current head, CAS to install. Pop: read head, CAS-replace with head.next. The textbook lock-free data structure; suffers from the ABA problem and from contention on the head.

Key Points

• Push: allocate node, set node.next = head, CAS(&head, oldHead, node).
• Pop: load head, then CAS(&head, oldHead, oldHead.next). Retry on CAS failure.
• Vulnerable to ABA: between read and CAS, head can be popped, freed, and re-pushed at the same address.
• Memory reclamation is the hard part: when is it safe to free a popped node? Other threads might still be reading it.
• Single-cache-line contention on head: not great for many concurrent threads. Lock-free MPSC queues outperform for high contention.

Interview Follow-up Questions

Gotchas

• ABA: in C/C++ without GC or hazard pointers, popped nodes can be reused, breaking CAS
• Memory reclamation: when is it safe to free a popped node?
• Cache-line contention on head: scales poorly with many writers
• Push allocates a node; allocator contention can dominate the lock-free win
• Lock-free does NOT mean wait-free; threads can starve indefinitely under bad scheduling

Hazard Pointers

Section: Lock-Free Programming | Difficulty: Advanced | Frequency: Sometimes

Memory reclamation scheme for lock-free structures in non-GC languages. Each thread publishes the pointers it's about to dereference (hazard pointers). Reclamation defers freeing a node until no thread's hazard pointer references it. Solves the 'when can I free this?' problem without a GC.

Key Points

• Each thread maintains a small fixed array of hazard pointers (typically 1-8).
• Before dereferencing a pointer obtained via shared atomic, publish it as a hazard pointer.
• On reclamation: collect all hazard pointers across threads, only free nodes not in the set.
• Bounds on memory reclamation latency: a node can be freed at most O(N * H) operations after being unlinked, where N=threads, H=hazards per thread.
• Alternative to epoch-based reclamation; lower latency for individual frees, more bookkeeping per access.

Interview Follow-up Questions

Gotchas

• Publishing hazard pointer AFTER dereferencing = use-after-free
• Forgetting to clear hazard pointer = node never freed (memory leak)
• Too few hazard slots per thread = need to retire-and-restart on contention
• Hazard scan is O(threads × slots); doesn't scale to thousands of threads
• Mixing hazard pointers with epoch-based reclamation in the same structure: confusing

RCU: Read-Copy-Update

Section: Lock-Free Programming | Difficulty: Advanced | Frequency: Sometimes

Read-Copy-Update: writers create a new copy and atomically swap the pointer; readers see either the old or the new, never a partial state. Reads are nearly free (often just a pointer load). Writers are slower but rare. Used heavily in the Linux kernel for read-mostly data structures.

Key Points

• Reads are extremely cheap: dereference a pointer. No locks, no atomics on the read path.
• Writes: copy the data, modify, atomically swap the pointer to the new copy.
• Old copy must be reclaimed only after all readers that might be using it have moved on.
• Grace period: readers register a 'reader region'; reclamation waits for all reader regions to end.
• Best for read-mostly data: routing tables, configuration, dispatch tables. Not for hot writes.

Interview Follow-up Questions

Gotchas

• Mutating in place defeats RCU; copy + swap is required
• Writers must wait for the grace period before freeing the old copy (in non-GC code)
• RCU is read-mostly; for write-heavy data, the copy cost dominates
• Multi-step updates (modify two related structures consistently) need extra coordination
• Long reader regions delay reclamation; in the kernel, this can pin memory

Wait-Free vs Lock-Free vs Obstruction-Free

Section: Lock-Free Programming | Difficulty: Advanced | Frequency: Sometimes

Three progress guarantees. Wait-free: every thread completes in bounded steps regardless of others. Lock-free: at least one thread makes progress at any time (system-wide progress). Obstruction-free: a thread makes progress if it runs alone for long enough. Strict ordering: wait-free > lock-free > obstruction-free > blocking.

Key Points

• Wait-free: hardest. Bounded per-thread steps. No starvation possible.
• Lock-free: at least one thread makes progress always. Individual threads may starve.
• Obstruction-free: a thread makes progress if no other thread interferes. Helped by exponential backoff.
• Blocking (lock-based): a thread holding a lock can pause everyone else indefinitely.
• Most production 'lock-free' code is actually lock-free, not wait-free. Wait-free is a research goal; lock-free is the practical bar.

Interview Follow-up Questions

Gotchas

• 'Lock-free' marketing often means 'doesn't block on a mutex' rather than the formal definition
• Lock-free does NOT prevent individual thread starvation
• Wait-free algorithms have bad constant factors; rarely worth it in practice
• Obstruction-free without backoff = livelock under contention
• Blocking algorithms inside a lock-free data structure (e.g., malloc) defeat the guarantee

Spinlocks: When Busy-Waiting Wins

Section: Lock-Free Programming | Difficulty: Advanced | Frequency: Sometimes

A spinlock is a mutex where the waiter spins in a tight CAS loop instead of going to sleep. Faster than mutex when the critical section is shorter than a context switch (~1µs). Awful when the critical section is long. Used inside OS kernels, sometimes in HFT systems; almost never appropriate in application code.

Key Points

• Spin in a CAS loop instead of parking the thread. No context switch cost.
• Wins when critical section < ~1µs (context switch cost) and there are spare cores.
• Loses badly when critical section is long: spinning threads burn 100% CPU doing nothing.
• Adaptive mutexes (Linux futex, Java synchronized): spin briefly, then park. Usually the right default.
• Spinlocks should NOT be used in user code without guaranteeing very short critical sections AND a spare core.

Interview Follow-up Questions

Gotchas

• Spinlock with a long critical section = wasted CPU and bad latency
• Spinning waiter preventing the lock holder from being scheduled = livelock-like behaviour
• Plain TAS spinlock causes cache-line ping-pong; use TTAS
• Forgetting PAUSE on x86 wastes power and hurts co-running hyperthreads
• Userspace spinlocks where the OS can preempt the holder are almost always wrong

Lock-Free MPSC Queue

Section: Lock-Free Programming | Difficulty: Hard | Frequency: Sometimes

A multi-producer, single-consumer queue using CAS on linked-list pointers. Producers append at tail; consumer pops from head. No locks. The Michael-Scott (1996) design is the textbook impl. Used in Go's runtime, Linux kernel, low-latency trading.

Key Points

• Linked list with atomic head and tail pointers
• enqueue: CAS to append; on failure, retry with new tail
• dequeue: CAS to advance head; on failure, retry
• ABA-prone, needs hazard pointers, GC (Java), or tagged pointers
• Note: the Michael-Scott algorithm shown below is actually MPMC; a true MPSC variant simplifies the consumer side (no CAS on head)

Interview Follow-up Questions

Gotchas

• ABA in C/C++: freed node reused at same address, CAS thinks unchanged
• Memory ordering: need acquire/release semantics or full fences
• Tail can lag, dequeue must help advance tail before returning empty
• Cache line contention: head and tail on same cache line → false sharing; pad to 64 bytes

LMAX Disruptor

Section: Lock-Free Programming | Difficulty: Expert | Frequency: Sometimes

A pre-allocated ring buffer plus per-consumer sequence cursors. Producers claim slots with CAS; consumers wait until the producer cursor advances past their position. Slots are reused (no per-message allocation), data lives in cache lines under explicit control (no false sharing), and the wait strategy is pluggable (busy spin, yield, block). Built by LMAX for trading, hits 25M+ messages per second per pipeline.

Key Points

• Ring buffer of fixed size (must be a power of two so index = sequence & (size - 1) is one AND instruction).
• Slots are pre-allocated objects that get reused; producers fill in fields, consumers read them. Zero per-message allocation.
• Producer cursor: an AtomicLong (cache-line padded) holding the highest published sequence. Consumers spin until their target sequence <= producer cursor.
• Consumer cursor: each consumer has its own cursor. Producers wait if the slowest consumer is too far behind (overwrites would lose data).
• Wait strategies are pluggable: BusySpinWaitStrategy (lowest latency, burns CPU), YieldingWaitStrategy (spin then yield), BlockingWaitStrategy (park on a condition).
• False sharing is the silent killer; every cursor is padded to occupy its own cache line.
• Single-producer mode skips the producer-side CAS and is much faster than multi-producer mode.

Interview Follow-up Questions

Gotchas

• Forgetting padding. The single most common mistake: a 'simple ring buffer' loses 80% of its throughput to false sharing between cursors.
• Using Object[] of allocated objects, not pre-allocated slot fields. Defeats the no-allocation property.
• Multi-producer mode without availability tracking: consumers will read uninitialized slots.
• Wait strategies that burn CPU in production. BusySpinWaitStrategy is for dedicated cores; on shared boxes it starves other work.
• Treating the ring as a dynamically resizable queue. The size is fixed at construction; pick it based on max in-flight, not average.

Sharded HashMap

Section: Lock-Free Programming | Difficulty: Hard | Frequency: Asked Often

Naive synchronized HashMap serializes every operation. Sharded version splits into N independent maps, each with its own lock. Hash key → shard index → operate on that shard's map. Java's ConcurrentHashMap does this with 16+ shards. Reads parallel, writes serialized only within shard.

Key Points

• N independent shards, each with own lock + map
• Hash key → shard index = key.hashCode() % N (better: use bit masking with power-of-2 N)
• Concurrent reads/writes on different shards proceed in parallel
• Resize: per-shard, or global (rare); ConcurrentHashMap uses tree-based bins instead
• Java 8+ ConcurrentHashMap uses CAS + per-bin synchronized, even finer than striping

Interview Follow-up Questions

Gotchas

• Bad hash function → uneven shard distribution → some shards always hot
• Iterating across shards is non-atomic, snapshot is inconsistent
• size() across shards requires summing each, non-atomic without freeze
• Resize is per-shard, not global, capacity can drift unevenly
• Don't use string %= shardCount on user-provided keys, collision attacks

Concurrent Bloom Filter (CAS-Based)

Section: Lock-Free Programming | Difficulty: Hard | Frequency: Sometimes

Bloom filter: probabilistic set membership with no false negatives, configurable false positives. Concurrent version: bits stored in atomic longs; insert sets bits via OR atomically; query reads atomically. Naturally lock-free because OR is monotonic.

Key Points

• Bloom filter: array of bits + N hash functions; insert sets N bits, query checks all N
• False positive: all N bits happen to be set by other entries (configurable rate)
• False negative: never (if all N bits are 0, definitely not in set)
• Concurrent: atomic OR, naturally lock-free, monotonic (bits only go 0→1)
• Cannot delete (would require resetting bits, but other entries may use them)

Interview Follow-up Questions

Gotchas

• False positive rate degrades as filter fills, pre-size for expected load
• Hash-function quality matters, bad hashes → uneven bit distribution → worse FP rate
• Cannot delete, once a bit is set it can't be safely cleared
• Multiple Bloom filters with different hash seeds aren't interoperable
• Use a counting Bloom filter or Cuckoo filter when delete is required

Concurrent Priority Queue (Skiplist-Based)

Section: Lock-Free Programming | Difficulty: Hard | Frequency: Sometimes

Lock-free priority queue is hard, heap-based requires global rebalance. Skiplist-based works: probabilistic levels, ordered by priority, CAS-based insert/delete. Java's ConcurrentSkipListMap is the production-grade impl. Use it; don't roll a custom one.

Key Points

• Heap-based PQ: hard to make lock-free (global rebalancing on every op)
• Skiplist: probabilistic balanced structure, naturally lock-free
• Insert: choose random level, CAS pointers in at each level
• Java's ConcurrentSkipListMap is the canonical implementation
• PriorityBlockingQueue (Java) is heap-based with a single lock, fine for moderate scale

Interview Follow-up Questions

Gotchas

• PriorityBlockingQueue.iterator() is NOT in priority order, only poll() is
• ConcurrentSkipListMap.firstEntry() is a snapshot, value may have been polled by another thread by the time it's used
• Skiplist memory overhead: ~4x of equivalent array (multi-level pointers per node)
• size() on ConcurrentSkipListMap is O(n); don't call in hot paths

Concurrent Trie (RCU-Style Read Path)

Section: Lock-Free Programming | Difficulty: Hard | Frequency: Rare

Concurrent trie: prefix-indexed tree where reads traverse a stable snapshot (no locks) and writes copy-then-swap nodes (RCU style). Lock-free reads at the cost of some write overhead. Used in Linux routing tables, JDK's CHAMP, persistent data structures.

Key Points

• Trie = tree indexed by string prefix; common in routing, autocomplete, dictionaries
• RCU (read-copy-update): readers see consistent snapshots; writers copy + atomic swap
• Lock-free reads: traverse atomic-pointer fields; never block
• Writers serialize on root pointer CAS, but contention only on the changed branch
• Memory: temporary copies on update (free old when no readers reference)

Interview Follow-up Questions

Detecting Race Conditions in Production

Section: Debugging Concurrency | Difficulty: Intermediate | Frequency: Sometimes

Race conditions almost never reproduce locally. Catch them with built-in race detectors (go run -race, ThreadSanitizer, jcstress) integrated into CI, plus production monitoring for symptoms (counter mismatches, sporadic test failures, "impossible" log states).

Key Points

• Race conditions hide in unit tests, they need real load and timing
• go run -race is built into Go, use it in CI; 5-10x runtime cost
• Java has no built-in race detector; use jcstress for stress tests, JFR for production tracing
• Python has the GIL but still has races, no native detector; rely on stress tests and review
• Symptoms in production: sporadic test failures, counter mismatches, 'impossible' log states
• Always test on multiple architectures, x86 hides races that ARM exposes

Interview Follow-up Questions

Gotchas

• go run -race only catches races that ACTUALLY OCCUR during the run, coverage matters
• Race detector doesn't catch logical races (check-then-act with atomics)
• Python's GIL hides races at bytecode boundaries; releases between bytecodes expose them
• Tests passing 1000 times in a row doesn't prove no race, the unlucky 1001st time may fail in production
• x86 strong memory model hides races that fire on ARM, test on both architectures
• Sporadic test failures dismissed as 'flaky' are often races, investigate, don't retry

Race Detection in CI

Section: Debugging Concurrency | Difficulty: Intermediate | Frequency: Sometimes

Run race detectors as part of CI. Go: -race. C++/Rust: ThreadSanitizer (TSan). Java: jcstress for memory model assumptions, plus quality concurrent tests. The detectors find concurrent unprotected access regardless of platform; they catch the bugs hidden by x86's strong ordering before they hit ARM in production.

Key Points

• Go: go test -race covers most data races. Run in CI on every PR.
• C++/Rust: ThreadSanitizer (TSan). Compile with -fsanitize=thread, run tests.
• Java: jcstress for memory-model bugs (lock-free code). Standard JUnit for higher-level concurrency.
• Race detectors slow execution 2-10x and use more memory; run in CI, not production.
• Stress-test concurrent code with high goroutine/thread counts; race detector + load surfaces real bugs.

Interview Follow-up Questions

Gotchas

• Not running -race / TSan in CI = bugs hide until ARM production
• Tests that don't actually exercise concurrency: race detector reports nothing
• Long test suites with -race timeout because of 10x slowdown; tune timeouts
• Race-free does not mean correct; algorithmic concurrency bugs (deadlock, livelock) still possible
• Skipping race detection on 'performance' tests: those are exactly where races hide

Profiling Concurrent Code

Section: Debugging Concurrency | Difficulty: Intermediate | Frequency: Sometimes

On-CPU profilers (perf, async-profiler, py-spy) show where threads spend CPU. Off-CPU profilers show where threads wait (locks, I/O). Concurrent code needs both: 'CPU is fine but throughput won't go higher' means off-CPU work, not on-CPU. Lock profilers and goroutine analyses are the specific tools.

Key Points

• On-CPU profile: where threads spend CPU. Use for CPU-bound bottlenecks.
• Off-CPU profile: where threads wait (block, sleep, park). Use for concurrency bottlenecks.
• Lock profilers: per-lock contention time. Async-profiler 'lock' mode, Go's mutex profiler.
• Thread/goroutine dumps: snapshot of every thread's stack. Use for hangs and deadlocks.
• Tracing (OpenTelemetry, runtime/trace): per-request timeline; useful for tail latency.

Interview Follow-up Questions

Gotchas

• Profiling only on-CPU when the bottleneck is contention: misses the problem
• Single thread dump is a snapshot; take 3-4 separated by seconds for hang diagnosis
• Profiling overhead can itself cause symptoms; use sampling at production scale
• Reading flame graphs without normalising by sample count: large boxes might be common, not slow
• Conflating tracing with profiling: tracing reports on specific requests, profiling on aggregate behaviour

Diagnosing Deadlocks and Hangs

Section: Debugging Concurrency | Difficulty: Intermediate | Frequency: Sometimes

When a service hangs, dump every thread/goroutine's stack trace. The pattern of who's waiting on what reveals whether it's a deadlock (cyclic wait), a livelock (busy spinning), or starvation (one thread never scheduled). Tools, jstack, py-spy, go pprof, kill -3.

Key Points

• Step 1 of any hang: dump every thread/goroutine's stack, without it, the diagnosis is guesswork
• Java: `jstack <pid>` or `kill -3 <pid>`; output shows 'Found X deadlocked threads'
• Go: send SIGQUIT or use /debug/pprof/goroutine?debug=2, full stack of every goroutine
• Python: `py-spy dump --pid <pid>` for sync code; asyncio.all_tasks() for async
• Deadlock pattern: thread A blocked on lock held by B, B blocked on lock held by A
• Hang ≠ deadlock, could also be infinite loop, blocked syscall, network timeout, or starvation

Interview Follow-up Questions

Gotchas

• kill -3 in production sends SIGQUIT, the Go runtime PRINTS THE DUMP AND EXITS the process. Use SIGUSR1/2 or a pprof endpoint instead.
• Java's `jstack` requires the same JVM version as the target process, and same UID; usually requires sudo or being the service user
• py-spy needs ptrace permissions; on hardened Linux, set CAP_SYS_PTRACE or run as root
• asyncio hangs don't show up as 'thread blocked', only one thread, looks idle. Use asyncio.all_tasks() instead.
• Goroutine dump alone doesn't show channel state, the dump shows 'chan receive' but not 'who has the channel'. Combine with code review.
• Hung threads waiting on a network socket look like deadlock, verify with strace/tcpdump before assuming

Bug Hunt: Find the Race in This Counter

Section: Debugging Concurrency | Difficulty: Basic | Frequency: Asked Often

The classic compound-update race. The counter looks atomic but isn't, read, increment, write are three separate steps that can interleave. Fix with an atomic primitive or a lock around the read-modify-write.

Key Points

• Compound operations (++ , += , !x) are NEVER atomic without explicit synchronization
• Two threads can both read 5, both compute 6, both write 6, losing one increment
• Tests pass on dev laptops; the race fires under production load
• Three correct fixes, atomic primitive (best), lock, or per-thread accumulator with merge

Interview Follow-up Questions

Gotchas

• Tests pass on x86, fail on ARM, same code, different memory model
• Java: AtomicInteger.get() then .set() is NOT atomic (two ops); use accumulateAndGet/updateAndGet
• Python: even integer assignment may need synchronization on shared globals
• Go: counter++ on int doesn't flag without -race; always run with -race in CI
• Locking around the wrong thing, protecting the read but not the write

Bug Hunt: Why Does This Deadlock Under Load?

Section: Debugging Concurrency | Difficulty: Intermediate | Frequency: Asked Often

Two threads each acquire two locks, in opposite order. Most of the time it works because the timing doesn't line up, under load, eventually it does, and both threads block forever waiting for each other. Fix with global lock ordering, or tryLock with timeout.

Key Points

• Under low load, the unlucky interleaving is rare, the bug looks like a flaky hang
• Under high load, it becomes nearly inevitable
• Four Coffman conditions must all hold; break any ONE → no deadlock
• The cheapest fix: enforce a global total ordering on lock acquisition

Interview Follow-up Questions

Gotchas

• Reentrant locks (Java ReentrantLock, Python RLock) prevent SELF-deadlock but NOT mutual deadlock between threads
• Go's runtime only catches 'all goroutines asleep', partial deadlocks go undetected
• Holding a lock while calling user-supplied callback code is a deadlock waiting to happen
• Sorting locks by hash is fine for objects, but pointers across processes (shared memory) need stable IDs

Bug Hunt: Why Does My RWMutex Deadlock When I Re-enter?

Section: Debugging Concurrency | Difficulty: Intermediate | Frequency: Sometimes

RWMutex is not reentrant. A goroutine holding the read lock that calls a method which also tries to acquire the read lock can deadlock if a writer is waiting in between. Java's ReentrantReadWriteLock is reentrant by design; sync.RWMutex is intentionally not. The fix is to restructure so locks aren't reentered.

Key Points

• sync.RWMutex (Go): NOT reentrant, RLock-then-Lock or RLock-then-RLock can deadlock
• ReentrantReadWriteLock (Java): IS reentrant, same thread can re-acquire its own lock
• Common trap: reader calls a method that also reads; a writer arrives between them; deadlock
• Fix: restructure to acquire the lock once at the outer scope, or pass an already-locked context

Interview Follow-up Questions

Gotchas

• Go: sync.RWMutex is writer-preferring, once Lock() is called, new RLocks block
• Java: StampedLock and Semaphore are NOT reentrant; ReentrantLock and ReentrantReadWriteLock are
• Python: threading.Lock is NOT reentrant; threading.RLock IS
• Calling external code (callbacks, listeners) while holding a lock can introduce reentrancy
• ReentrantLock is fine for nested calls; using one to call into completely unknown code is still risky (might block on something else)

Bug Hunt: Lost Wakeup with Condition Variable

Section: Debugging Concurrency | Difficulty: Advanced | Frequency: Sometimes

Producer signals condition before consumer waits → consumer waits forever. Classic bug from forgetting to check the predicate inside the lock before waiting. Fix: always check predicate inside a while loop while holding the lock; the predicate guards both the wait AND the wakeup.

Key Points

• Lost wakeup: signal happens before wait, signal is dropped (condition variables don't queue signals).
• Fix: hold the lock, check predicate in a while loop, wait if predicate is false.
• Signal must be inside the same lock to make the predicate visible to the waker.
• Spurious wakeups also require while loop (not if): wait can return without anyone signalling.
• In Go, channels have built-in 'queued' semantics that avoid this; sync.Cond can lose signals.

Interview Follow-up Questions

Gotchas

• Using if instead of while around wait: spurious wakeups break correctness
• Signalling outside the lock: predicate update may not be visible to waiter in time
• Using notify when multiple unrelated waiters wait on the same condition
• Forgetting to update the predicate before signal: wakes waiter that finds predicate false
• Believing the bug is rare: lost-wakeup happens reliably under load testing

Bug Hunt: The Broken Double-Checked Locking

Section: Debugging Concurrency | Difficulty: Advanced | Frequency: Asked Often

Pre-Java-5, double-checked locking was famously broken because the JMM allowed the constructor's writes to be reordered with the publication of the reference. A reader could see a non-null reference whose object's fields were still uninitialized. The fix is volatile (Java) or a static-holder idiom, and similar gotchas apply in Go and Python.

Key Points

• DCL trick: 'check, lock, check again, construct', saves a lock on the fast path
• Bug: without volatile, the constructor's writes can be reordered with the assignment to the field
• A reader sees a non-null reference; reads object fields; sees garbage / nulls
• Static-holder idiom (Java) is simpler and uses class-init guarantees instead
• Go's sync.Once is the right answer; in Python, module-level construction works

Interview Follow-up Questions

Gotchas

• Java: forgetting volatile on the singleton field is the classic broken-DCL bug
• Java: the static-holder idiom doesn't run init until get() is called, true laziness
• Python: DCL without holding the lock for both check and set has a race on free-threaded CPython
• Go: sync.Once.Do swallows panics, wrap critical init with a panic-aware version if needed
• Reading instance.someField on the fast path skips ALL synchronization, must guarantee instance was safely published
• Enum singletons in Java prevent reflection-based instantiation; regular ones don't

Bug Hunt: Goroutine Count Growing 10/sec

Section: Debugging Concurrency | Difficulty: Intermediate | Frequency: Sometimes

A goroutine that blocks on a channel send/receive without a cancellation path lives forever. The fix is always a select on ctx.Done() or a buffered channel large enough to absorb all sends. The same bug appears in Java (futures never cancelled) and Python (asyncio tasks orphaned).

Key Points

• A goroutine that blocks forever holds memory + captured variables + channels
• Symptom: runtime.NumGoroutine() growing unbounded under load
• Diagnosis: /debug/pprof/goroutine?debug=2 shows what each goroutine is waiting on
• Fix pattern: every goroutine that sends/receives must have a cancellation path

Interview Follow-up Questions

Gotchas

• fire-and-forget goroutines without ctx.Done() are the #1 source of Go production leaks
• Java's CompletableFuture without orTimeout silently inherits 'wait forever' semantics
• Python: asyncio.create_task without keeping a reference can be GC'd mid-execution
• context.WithCancel without calling cancel() leaks the underlying timer goroutine
• Logging goroutines that block on a slow downstream → biggest leak risk in HTTP handlers
• Bounded queues + drop-on-full is often better than infinite buffering

Bug Hunt: Why Is This LongAdder Slower Than AtomicLong?

Section: Debugging Concurrency | Difficulty: Advanced | Frequency: Sometimes

LongAdder shards counters across threads to win under contention, but its sum() walks every shard, so reads are O(N). When read rate is high relative to write rate, AtomicLong (single CAS) is faster. The bug is using the right tool for the wrong workload.

Key Points

• AtomicLong: cheap reads (one load), CAS retries on contention (slow when many writers contend)
• LongAdder: cheap writes under contention (per-thread cells), expensive reads (sum walks all cells)
• Read-heavy + few writers → AtomicLong wins
• Write-heavy + many writers → LongAdder wins
• Always benchmark with the actual workload, intuition lies

Interview Follow-up Questions

Gotchas

• Many counters are read on every request (rate limiters, header injection) → AtomicLong wins
• Metrics counters scraped via Prometheus/StatsD → LongAdder may win
• Sharded counters are eventually consistent, sum() can return slightly old values
• False sharing across cache lines kills sharded perf if cells aren't padded, JDK pads automatically; rolling-your-own often forgets
• Profile-driven optimization beats intuition every time

Bug Hunt: False Sharing in a Striped Counter

Section: Debugging Concurrency | Difficulty: Advanced | Frequency: Sometimes

Striped counter built from an array of AtomicLong shows worse throughput at high thread count than a single AtomicLong. Bug: the array elements share cache lines; threads writing different stripes cause cache-line ping-pong. Fix: pad each element to its own cache line (LongAdder, @Contended, or manual padding).

Key Points

• Symptom: striped counter scales worse than expected; CPU at 100%, throughput plateaus or degrades.
• Root cause: array elements occupy adjacent memory; multiple elements share each cache line.
• Two cores writing to different elements on the same line: the line bounces between L1 caches.
• Fix: pad each element to 64 bytes (cache line size). Java's LongAdder does this internally.
• Diagnose: profile shows hot atomic-add op, cache miss rate is high, IPC (instructions per cycle) is low.

Interview Follow-up Questions

Gotchas

• Striped counters without padding scale worse than non-striped
• Padding only some atomics: misses the bouncing on the unpadded ones
• @Contended without -XX:-RestrictContended is silently ignored on application classes
• False sharing in field layout (two atomic fields adjacent in a struct): same problem as arrays
• Padding waste: 64 bytes per atomic adds up; use sparingly

Bug Hunt: Memory Ordering on ARM

Section: Debugging Concurrency | Difficulty: Advanced | Frequency: Sometimes

Code passes tests on x86, fails on ARM. The bug: missing memory barrier (volatile in Java, atomic in C++, sync in Go) on a flag that the writer sets and the reader checks. x86's strong ordering hides the bug; ARM's weaker ordering exposes it. Fix: use proper synchronization primitives, never plain reads of shared variables.

Key Points

• x86 has strong ordering: most reorderings forbidden by hardware. Bugs hidden.
• ARM/POWER have weak ordering: writes can be reordered, loads can speculate ahead.
• The bug usually looks like: writer thread sets flag, reader thread sees the flag set but old data.
• Fix: use volatile / atomic / proper synchronisation. Never plain reads of shared mutable state.
• Detect via stress testing on ARM hardware (Apple Silicon, AWS Graviton, Raspberry Pi).

Interview Follow-up Questions

Gotchas

• Testing only on x86 = bugs hide until ARM deployment
• 'Works in dev, fails in prod' when prod is on ARM
• Race detector catches issues regardless of platform; run it in CI
• C++ default memory_order is SeqCst (safe but slow); explicit order_relaxed needs justification
• Java plain int reads/writes are NOT atomic for long; use volatile or AtomicLong

Bug Hunt: Works on x86, Fails on ARM, Why?

Section: Debugging Concurrency | Difficulty: Advanced | Frequency: Sometimes

x86 has a strong memory model that hides reordering bugs at runtime. ARM and POWER have weaker memory models that expose them. Code missing proper synchronization can 'work' on Intel and break on Apple Silicon, AWS Graviton, or mobile. The fix is to add a memory barrier, volatile, atomic, or a lock.

Key Points

• x86 uses TSO (Total Store Order), most reads and writes can't be reordered with each other
• ARM, POWER, RISC-V allow more reordering; the same code 'works' less reliably
• Compilers reorder too, even on x86, missing synchronization can fire on a JIT optimization
• Memory barriers (volatile, atomic store/load, lock acquire/release) prevent reordering

Interview Follow-up Questions

Gotchas

• Tests pass on x86 dev laptops, fail on ARM CI runners, TEST ON BOTH
• JIT can hoist a non-volatile read out of a loop entirely → 'why is my thread spinning forever?'
• Java: long/double on 32-bit JVMs aren't atomic without volatile, torn reads possible
• Python: free-threaded CPython (3.13+) loosens visibility guarantees that GIL-based code accidentally relied on
• Go: copying a struct that contains atomic fields silently breaks them
• x86 hides write-write reordering; ARM does not

Code Review: This Lock-Free Counter

Section: Debugging Concurrency | Difficulty: Advanced | Frequency: Sometimes

A candidate's 'lock-free counter' looks slick, atomic operations, no synchronized blocks. But the get-then-set pattern isn't atomic, the read isn't ordered with the write, and overflow handling is missing. Walk through what an interviewer wants reviewers to find.

Key Points

• An atomic.get() followed by atomic.set() is NOT atomic, same race as ++ on volatile
• Compound operations need compareAndSet (CAS) or accumulateAndGet, not get+set
• Reading 'unconditionally' before checking is a read-then-act race
• Overflow on counters that monotonically increase is a real production bug

Interview Follow-up Questions

Gotchas

• atomic.get() then atomic.set() is two operations, not atomic
• atomic.set(atomic.get() + 1) is the textbook racy increment
• Reading outside a lock then acting inside it, if the read informed the action, it's a race
• Naming a class 'LockFree' doesn't make it correct
• Java's accumulateAndGet/updateAndGet build CAS retry loops automatically, prefer them when applicable

Code Review: Go Context Misuse

Section: Debugging Concurrency | Difficulty: Intermediate | Frequency: Asked Often

Common Go context bugs: not passing context downstream, not checking ctx.Done() in loops, storing context in struct fields, ignoring cancel cleanup, calling cancel from inside the cancelled goroutine. Each one creates a goroutine leak or unresponsive cancellation.

Key Points

• Always pass ctx as the FIRST parameter; don't store in struct fields.
• Always defer cancel() after creating one with WithTimeout/WithCancel; otherwise leak.
• Pass ctx down through every call that does I/O or might block.
• In loops, check ctx.Done() either via select or ctx.Err().
• Returning early from cancelled context is a goroutine's responsibility; runtime won't kill it.

Interview Follow-up Questions

Gotchas

• Forgetting defer cancel() leaks the cancellation goroutine
• Ignoring ctx in HTTP calls or DB queries: cancellation has no effect
• Storing ctx in struct: subsequent calls inherit stale ctx
• Using context.Background() in handlers: no inherited cancellation; should be the request's ctx
• Putting business data in ctx values: anti-pattern; use explicit parameters

Code Review: This asyncio Service for Blocking Calls

Section: Debugging Concurrency | Difficulty: Intermediate | Frequency: Sometimes

An async FastAPI handler calling a sync DB driver, sync HTTP client, and time.sleep stalls the entire event loop. Tail latency goes from 50ms to 500ms across all concurrent requests. Walk through every blocking call and either replace with async-aware libraries or wrap with asyncio.to_thread.

Key Points

• async def + a synchronous call inside = silent disaster, the event loop stalls
• All blocking I/O must be either async-native or wrapped in asyncio.to_thread
• asyncio.create_task without a reference can be garbage-collected mid-run
• asyncio.Lock and threading.Lock are NOT interchangeable

Interview Follow-up Questions

Gotchas

• requests.get inside async def → silent stall, no warning
• time.sleep inside async def → freezes everything
• threading.Lock in asyncio code → blocks the event loop instead of yielding
• asyncio.create_task without keeping reference → task can be GC'd mid-run
• Mixing sync DB drivers with FastAPI → tail latency surge under load
• asyncio.gather with N=1M tasks → memory blow-up; bound with Semaphore

Production Postmortem: Concurrency Outages

Section: Debugging Concurrency | Difficulty: Advanced | Frequency: Sometimes

Real concurrency outages share patterns: thread pool exhaustion from a slow downstream, missing timeouts that turn local slowness into global outage, retry storms after a brief failure, deadlock under load that's invisible at low traffic. The fixes are usually defensive (timeouts, bulkheads, circuit breakers) rather than algorithmic.

Key Points

• Most outages have multiple contributing causes; rarely one bug.
• Common pattern: 'X got slow → no timeout → Y got tied up → cascade'.
• Timeline analysis: when did metrics change? Tie to deploys, traffic, downstream events.
• Action items: not just 'fix the bug' but 'prevent the class of bug': timeouts, bulkheads, alerts.
• The blameless postmortem: focus on systemic gaps, not individual mistakes.

Interview Follow-up Questions

Gotchas

• Postmortem that blames an individual instead of fixing the system
• Action items without owners or due dates: never get done
• Fixing only the immediate bug, not the class of bug
• Not exercising the postmortem's findings: 'we'll add timeouts later'
• Skipping postmortems for 'small' incidents that turn out to be patterns