errgroup and golang.org/x/sync/semaphore
errgroup.Group is the structured fan-out for Go: launch goroutines via Go(func), Wait for all, first error cancels siblings via Context. semaphore.Weighted bounds concurrency with weighted permits. Together they cover most real fan-out cases cleaner than raw WaitGroup + manual cancel plumbing.
What it is
errgroup.Group is to Go what StructuredTaskScope is to Java 21+: structured fan-out with cancellation on first error. It lives in golang.org/x/sync/errgroup, not the standard library, but it has been stable for years and is the de facto standard for any "do N things concurrently and wait" pattern.
semaphore.Weighted (in golang.org/x/sync/semaphore) provides a counted semaphore where each acquisition can take more than one permit. Used when tasks have different resource costs.
Together, these two cover most real-world Go fan-out work cleanly.
errgroup: the canonical fan-out
The pattern is always the same:
- Get a Group and a derived Context:
g, ctx := errgroup.WithContext(parent). - Launch goroutines via
g.Go(func() error { ... }). - Wait via
g.Wait(). Returns the first error from any goroutine. - Inside each goroutine, respect
ctx: pass it to downstream calls, check it on long loops.
What this provides:
- Wait for all: g.Wait blocks until every Go has returned.
- Cancel on first error: when any goroutine returns non-nil, the derived ctx is cancelled. Other goroutines that respect ctx exit early.
- Bounded concurrency: SetLimit caps in-flight goroutines.
Compared to manual sync.WaitGroup + cancellation channel + error tracking, errgroup is half the code and harder to get wrong.
semaphore.Weighted: cost-aware concurrency
A counted semaphore. Acquire(ctx, w) blocks until w permits are available; Release(w) returns them.
The weighted variant matters when tasks have different costs. A small image takes 1 MB to process; a big one takes 50 MB. With a plain count semaphore, the cap has to assume worst case. Weighted lets small tasks pile up while big tasks throttle naturally.
For "just cap at N concurrent" without weights, errgroup's SetLimit is simpler. Reach for semaphore.Weighted when the cost varies.
When SetLimit is enough
For most "cap fan-out at N" patterns, SetLimit on the errgroup does the job:
g, ctx := errgroup.WithContext(ctx)
g.SetLimit(50)
for _, item := range items {
item := item
g.Go(func() error { return process(ctx, item) })
}
return g.Wait()
This loops over items, blocks at 50 in-flight, propagates the first error, cancels siblings.
When finer control is needed (different stages with different limits, weighted resource budgets, semaphores shared across components), reach for semaphore.Weighted.
Common Go-specific traps
The most common Go concurrency bug is goroutine leaks: a goroutine that outlives its caller because nobody waits for it or cancels it. errgroup eliminates the most common version of this bug by structuring fan-out: every goroutine launched via Go is waited for in g.Wait.
The next one is ignoring the context. errgroup provides a derived ctx that cancels on first error, but only goroutines that pass that ctx to downstream calls (HTTP requests, DB queries, channel operations in select) actually exit early. A goroutine doing a busy compute that never checks ctx will run to completion.
The last one is not capturing the loop variable. Pre-Go-1.22, for _, item := range items { go func() { use(item) } } shares one item variable across all goroutines, all of which see the last value. Always item := item before the Go. (Go 1.22+ scopes loop variables per iteration, fixing this.)
Primitives by language
- errgroup.Group / errgroup.WithContext
- g.Go(func() error), launches goroutine
- g.Wait(), waits for all, returns first error
- semaphore.NewWeighted(n) / Acquire(ctx, w) / Release(w)
Implementation
Three independent calls. errgroup runs them concurrently. If any returns an error, the context cancels, and the other goroutines see ctx.Done() and exit early. g.Wait returns the first error.
1 package main
2
3 import (
4 "context"
5 "golang.org/x/sync/errgroup"
6 )
7
8 func loadOrder(ctx context.Context, uid string) (*Order, error) {
9 g, ctx := errgroup.WithContext(ctx)
10
11 var user *User
12 var cart *Cart
13 var recs []*Item
14
15 g.Go(func() error {
16 u, err := fetchUser(ctx, uid)
17 user = u
18 return err
19 })
20 g.Go(func() error {
21 c, err := fetchCart(ctx, uid)
22 cart = c
23 return err
24 })
25 g.Go(func() error {
26 r, err := fetchRecs(ctx, uid)
27 recs = r
28 return err
29 })
30
31 if err := g.Wait(); err != nil {
32 return nil, err // first error wins
33 }
34 return &Order{User: user, Cart: cart, Recs: recs}, nil
35 }SetLimit caps the number of in-flight Go calls. Subsequent g.Go blocks until a slot is free. Replaces the 'errgroup + semaphore' pattern for the common case of 'cap concurrency at N'.
1 package main
2
3 import (
4 "context"
5 "golang.org/x/sync/errgroup"
6 )
7
8 func processBatch(ctx context.Context, urls []string) error {
9 g, ctx := errgroup.WithContext(ctx)
10 g.SetLimit(20) // at most 20 concurrent
11
12 for _, u := range urls {
13 u := u
14 g.Go(func() error {
15 return fetch(ctx, u)
16 })
17 }
18 return g.Wait()
19 }Each task acquires a weight proportional to its memory cost. The semaphore caps total in-flight weight, not just count. Useful when tasks vary widely in size: small ones run many in parallel, big ones run alone or in pairs.
1 package main
2
3 import (
4 "context"
5 "golang.org/x/sync/semaphore"
6 )
7
8 // Total budget: 100 MB worth of in-flight work
9 var memBudget = semaphore.NewWeighted(100)
10
11 func processFile(ctx context.Context, sizeMB int64, path string) error {
12 if err := memBudget.Acquire(ctx, sizeMB); err != nil {
13 return err // ctx cancelled
14 }
15 defer memBudget.Release(sizeMB)
16
17 return process(path)
18 }
19
20 // Many concurrent processFile calls; total in-flight memory bounded at 100 MBCombine errgroup with range-over-func iteration: process each yielded item in a goroutine, cap concurrency, propagate first error. Pattern that turns up everywhere modern Go is written.
1 package main
2
3 import (
4 "context"
5 "iter"
6 "golang.org/x/sync/errgroup"
7 )
8
9 func processStream(ctx context.Context, items iter.Seq[Item]) error {
10 g, ctx := errgroup.WithContext(ctx)
11 g.SetLimit(10)
12
13 for item := range items {
14 item := item
15 g.Go(func() error {
16 select {
17 case <-ctx.Done():
18 return ctx.Err()
19 default:
20 }
21 return handle(ctx, item)
22 })
23 }
24 return g.Wait()
25 }Key points
- •errgroup.WithContext returns a Group AND a derived Context that is cancelled when any goroutine returns an error.
- •First error wins; subsequent errors from siblings are dropped (only logged).
- •g.SetLimit(n) (added in x/sync errgroup v0.0.0-2022) caps in-flight goroutines without a separate semaphore.
- •semaphore.Weighted permits >1 weight per acquisition; useful when tasks have different cost (memory, CPU).
- •Both are in golang.org/x/sync (not stdlib) but are de facto standard for Go fan-out.
Follow-up questions
▸errgroup vs sync.WaitGroup: when to use which?
▸What happens to slow goroutines after the first error?
▸How does SetLimit differ from a Semaphore?
▸Why are these in x/sync rather than stdlib?
Gotchas
- !Forgetting to capture the loop variable (item := item) before passing to g.Go creates a race in pre-Go-1.22 code
- !errgroup only returns the FIRST error; subsequent errors are silently dropped
- !A goroutine that ignores ctx runs to completion even after error cancellation
- !semaphore.Weighted.Acquire blocks until ctx done or weight available; check ctx.Err() to distinguish
- !errgroup zero value works but loses cancellation; always use WithContext for fan-out