Context Switch Cost

Why context switch cost matters

Every concurrency primitive has a cost per switch. At small scale (hundreds of switches/sec) it's invisible. At scale (10K-1M switches/sec) it becomes a measurable percentage of CPU time. Choosing a primitive whose switch cost is 10x cheaper can mean the difference between maxing out a server at 100K rps vs 1M rps.

Order-of-magnitude numbers

The numbers vary by ~3x with hardware/OS, but the orders-of-magnitude relationships hold.

When this dominates

How to measure

• Linux system-wide: vmstat 1 shows cs (context switches per second).
• Per-process: pidstat -w 1 shows voluntary and involuntary cs.
• Go: runtime/pprof goroutine and mutex profiles show coordination overhead.
• Java: JFR JavaThreadStatistics event tracks thread states.
• Async (Python/JS): harder, track task-resume rate via custom instrumentation.

Reproduce the numbers locally

A pipe ping-pong is the standard microbenchmark for OS thread switch cost. Two threads (or two processes) each write one byte and read one byte, forever; each round trip is two context switches. Divide.

// pipe_pong.c -- one process, two threads ping-ponging via a pipe
// Build: cc -O2 -pthread pipe_pong.c -o pipe_pong
// Run:   taskset -c 0,1 ./pipe_pong         # pin to two cores to avoid noise
#include <pthread.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <time.h>
#include <unistd.h>

#define ROUNDS 1000000

int p1[2], p2[2];
char b;

void *pong(void *_) {
    for (long i = 0; i < ROUNDS; i++) {
        read(p1[0], &b, 1);
        write(p2[1], &b, 1);
    }
    return NULL;
}

int main() {
    pipe(p1); pipe(p2);
    pthread_t t;
    pthread_create(&t, NULL, pong, NULL);

    struct timespec t0, t1;
    clock_gettime(CLOCK_MONOTONIC, &t0);

    for (long i = 0; i < ROUNDS; i++) {
        write(p1[1], &b, 1);
        read(p2[0], &b, 1);
    }
    clock_gettime(CLOCK_MONOTONIC, &t1);

    double sec = (t1.tv_sec - t0.tv_sec) + (t1.tv_nsec - t0.tv_nsec) / 1e9;
    printf("%.0f ns per round trip (%.0f ns per switch)\n",
           sec * 1e9 / ROUNDS, sec * 1e9 / ROUNDS / 2);
    return 0;
}

// Typical output on a modern Linux box:
//   ~3000-5000 ns per round trip = ~1500-2500 ns per OS thread switch

For goroutines, the equivalent benchmark is two goroutines exchanging on a channel:

// chan_pong_test.go -- run with: go test -bench=Pong -benchtime=1000000x
package main

import "testing"

func BenchmarkChanPong(b *testing.B) {
    a := make(chan struct{}, 1)
    z := make(chan struct{}, 1)
    go func() {
        for i := 0; i < b.N; i++ {
            <-a
            z <- struct{}{}
        }
    }()
    b.ResetTimer()
    for i := 0; i < b.N; i++ {
        a <- struct{}{}
        <-z
    }
}

// Typical: 200-400 ns per round trip = 100-200 ns per goroutine switch.
// Compare with the pipe_pong number above to see the order-of-magnitude gap.

Why the gap is so big

An OS thread switch is the kernel taking the CPU away from one thread and giving it to another. The work happens in two distinct buckets, and the second one is usually larger than the first.

Bucket 1: the kernel work. Five steps, each with a real cost.

That alone is roughly 500 ns to 1 µs of pure kernel overhead.

Bucket 2: the cold caches and cold TLB. This is the part most people miss, and it is usually bigger than the kernel work.

After the switch, thread B is running on a core whose caches are full of thread A's data. B's stack, B's hot variables, and B's recently-touched globals are not in L1. Every memory access B makes for the next hundred instructions or so misses L1 and has to refill from L2, L3, or main memory. The TLB (the CPU's cache of virtual-to-physical address mappings) also has stale entries. Even with PCID / ASID tagging, which lets the kernel avoid a full TLB flush, many entries get invalidated.

The "first hundred accesses are slow" effect is often the largest cost of a context switch, larger than the kernel work itself. Two threads that fit comfortably in cache individually do not fit together; switching evicts each one's data on the way out.

A goroutine switch or a virtual-thread switch avoids most of this. Both stay in user space, so steps 1 and 5 (the user-kernel transitions) do not happen. The "save and restore registers" work is a small struct copy in user space rather than a kernel call. The runtime tries to keep the same goroutine on the same OS thread when possible, which keeps the cache warm. The total is roughly 100 to 200 ns, an order of magnitude faster than the OS thread switch and two orders of magnitude faster than crossing the network. That gap is why services that fan out lots of small concurrent units (web servers, RPC clients, ingestion pipelines) benefit so much from goroutines or virtual threads over OS-thread-per-request.