API reference

NestedThreading — Module
NestedThreading

Coordinate a shared CPU thread budget across independently-threaded libraries (BLAS, FFTW, Polyester, NFFT, …) so that an outer parallel loop does not oversubscribe the machine with inner threading — Julia's answer to Python's threadpoolctl.

The entry points are @budgeted_threads / @budgeted_batch for loops, @budgeted for a loop construct you want to keep exactly as written, and with_restricted_threads / with_full_threads for call sites that are not loops. Libraries register themselves through package extensions; see register_counted_pool! to add one this package does not ship.

Budgeting is one half of the problem. The other is that a library's workers can go on occupying Julia's threads after its own parallel region has ended, so that a Threads.@threads region opened next waits for them instead of running on them — 38x on the measurement in QuiescePool. The loop macros here handle that for themselves; quiesce_foreign_pools is the same thing for a parallel region written by hand.

source

Loop macros

NestedThreading.@budgeted — Macro
@budgeted <parallel-loop>

Wrap an already-chosen parallel loop so that its body runs with inner threaded libraries limited to budget_for(range) threads. The loop construct is emitted unchanged — this macro never decides whether to parallelize, only what budget the workers get.

@budgeted Threads.@threads for j in 1:num_threads
    mul!(view(out, :, j), op, view(inp, :, j))
end

Accepts any chain of macros ending in a single for x in range loop, so Threads.@threads :static for, @inbounds Polyester.@batch for, and @batch minbatch=64 for all work. The range expression is hoisted into a temporary and evaluated exactly once. A chain containing @batch automatically excludes the :polyester pool, since there the loop being budgeted is itself the Polyester consumer.

See also @budgeted_threads, @budgeted_batch.

source
NestedThreading.@budgeted_threads — Macro
@budgeted_threads [threads = <bool>] for x in range ... end

Threads.@threads over range with an automatic inner thread budget, plus a runtime switch for whether to parallelize at all.

  • threads = true (the default when omitted) runs Threads.@threads with the body's inner libraries limited to budget_for(range).
  • threads = false runs a plain sequential loop with inner libraries limited to a single thread, on the assumption that a caller who switched threading off did so because concurrency is happening somewhere else.

Body-level macros such as @inbounds may wrap the loop; they are rebuilt around the emitted Threads.@threads. A macro that parallelizes the loop itself (Threads.@threads, Polyester.@batch, @distributed, …) is rejected at expansion, since this macro emits one already — use @budgeted to budget a parallel loop you have written yourself. @budgeted is also the way to combine this with a macro that insists on being handed a bare for loop, such as @simd.

source
NestedThreading.@budgeted_batch — Macro
@budgeted_batch [threads = <bool>] for x in range ... end

Polyester @batch over range with an automatic inner thread budget and the same threads = switch as @budgeted_threads.

The calling module must have Polyester available (import Polyester), since the expansion emits Polyester.@batch. The :polyester pool is excluded from the budget scope so that the generated @batch loop is not disabled by its own restriction.

As with @budgeted_threads, a chain that already parallelizes the loop is rejected at expansion; pass @batch's own options to @budgeted instead (@budgeted Polyester.@batch minbatch=64 for ...).

source

Scoped budgets

NestedThreading.with_restricted_threads — Function
with_restricted_threads(f; exclude = (), only = nothing)

Run f() with every registered threaded library limited to a single thread. Equivalent to with_thread_budget(f, 1). Use at call sites that are not a plain for loop — a @sync/@spawn block, or a single call gated on a Bool field.

source
NestedThreading.with_full_threads — Function
with_full_threads(f; exclude = (), only = nothing)

Run f() requesting full threading: every CountedPool at capacity and every GuardedPool enabled. Unlike a plain unwrapped call this actively turns guarded libraries on (e.g. sets NFFT._use_threads[] = true), which is what a call site with an explicit threaded = true switch needs.

Still clamped by any outer restriction, so calling it inside a saturated batch loop is safe and does nothing.

source
NestedThreading.enable_full_threading — Function
enable_full_threading()

Permanently set every registered CountedPool to its registration-time maximum — the process-wide escape hatch for code that only ever runs one thing at a time and wants every library at full throttle.

This sets the baseline: the value restored when the last budget scope exits. Scoped restrictions still win while they are active; this only changes what the process returns to when nothing is restricted.

Not scoped and not undoable — there is deliberately no disable_full_threading.

source
NestedThreading.with_thread_budget — Function
with_thread_budget(f, n::Integer; exclude = (), only = nothing)

Run f() with every registered threaded library limited to n threads, restoring the previous state afterwards (also on exception). Returns f()'s value.

The budget actually applied to a pool is the minimum over every currently-active restriction that applies to that pool (see exclude/only below), so

  • nesting can only ever narrow, never widen, an outer restriction — an inner with_thread_budget(f, 8) inside an outer with_thread_budget(g, 1) still runs at 1;
  • concurrent scopes cannot corrupt each other's bookkeeping: the counts are snapshotted on the first entry and restored on the last exit.
  • a limit nested in an enclosing one – on the same task, or on a task spawned inside it, such as a @budgeted_threads worker – that already holds every pool it covers at or below n is not entered: it could change nothing, so f() runs directly, without the lock round trip and pool updates that make a scope cost microseconds. A scope open on an unrelated task never counts. This relies on the enclosing scope outliving f(), which a task spawned inside a scope and never waited for does not guarantee. Julia 1.11 and later only; on 1.10 every scope is entered.

By default a restriction applies to every registered pool, CountedPools and GuardedPools alike. Two keywords narrow that:

  • exclude: a denylist of pool names this restriction does not apply to. Naming a pool that turns out to be a CountedPool actually leaves it alone now — earlier this only ever affected guards, and silently did nothing for a counted pool's budget, which was a bug. Naming a pool that is not currently registered at all is not an error: a caller may exclude a pool whose package is not loaded in this session.
  • only: an allowlist — when given, this restriction applies to only these pools and none else, leaving everything unnamed alone entirely (not merely unrestricted-by-this- scope: if no other active scope restricts it either, it keeps its pre-scope value even past this scope's own budget). This is what with_serial_blas-style call sites need: "restrict exactly these pools" is not expressible as a denylist without enumerating every other pool, which breaks whenever a new one registers.

GuardedPools are invoked only when the budget applying to them is below capacity, and receive that budget. Polyester's guard switches it off outright rather than limiting it proportionally — a partial limit was tried and measured far worse in the saturated regime; see GuardedPool. @budgeted_batch uses exclude = (:polyester,) because there the outer loop is the Polyester consumer.

Guards are decided once, at entry

Counted pools follow the applied minimum for as long as the scope is open — another task narrowing the budget lowers them immediately. Guards cannot work that way: a guard is a lexical wrapper around f(), so whether it runs is decided from the budget at entry and cannot be revised afterwards.

So a scope that enters while nothing is restricted runs its body unguarded even if a concurrent task restricts the process meanwhile: its BLAS and FFTW counts drop, but a nested Polyester loop inside it can still spawn workers. The other direction is safe — a scope that entered restricted stays guarded for its whole body even after the other scope exits. Only the guarded libraries are affected, and only for scopes that opened unrestricted; if it matters, open the restricting scope before spawning the tasks rather than from inside them.

A budget of `capacity()` raises, it does not merely permit

The budget is applied to every counted pool this restriction applies to unconditionally, so a scope whose budget works out to capacity — with_full_threads, or a loop with a single item — sets those pools up to that value for its duration, even past a lower count the caller had configured (BLAS.set_num_threads(2), say). The original counts are restored on exit like any other scope. This is what makes with_full_threads able to turn NFFT on; if a hand-tuned lower count must be preserved, do not open a full-throttle scope around it — or exclude/only it out of one.

A pool named in exclude, or left out of only, is not "permitted" either way: as long as nothing else currently active restricts it, it keeps whatever value it already had, not the value this scope requested.

Process-global state

BLAS and FFTW thread counts are process-global with no per-task scoping. Taking the minimum over all active scopes is deliberately conservative: while any task is restricted, unrelated concurrent tasks are restricted too. That never oversubscribes, but it does mean a long-lived restricted scope throttles the whole process. This is a limitation of the underlying C libraries, not something this package can solve.

See also with_restricted_threads, with_full_threads.

source
NestedThreading.with_thread_default — Function
with_thread_default(f, n::Integer; exclude = (), only = nothing)

Run f() with every registered threaded library at n threads by default: the count it runs at unless a with_thread_grant nested inside says a particular call is worth more. Returns f()'s value.

This is the scope to open around a whole computation whose calls into a library are mostly too small to benefit from its threads, but some of which are not. A hard with_thread_budget cannot express that: nothing nested inside it can widen past it, so the few large calls would run serial too.

Soft defaults combine with each other by minimum, and yield to a grant. They do not yield to nothing else: a hard limit open around or inside a soft default still clamps it, and a soft default of n below a hard limit of m > n runs at n, since the limit is a ceiling and not a request.

with_thread_default(1; only = (:blas,)) do     # small BLAS calls: serial
    for iteration in 1:100
        axpy!(a, x, y)                          # BLAS at 1 thread
        with_thread_grant(capacity(); only = (:blas,)) do
            svd!(big)                           # BLAS at every thread
        end
    end
end

See with_thread_budget for exclude, only and what is process-global.

source
NestedThreading.with_thread_grant — Function
with_thread_grant(f, n::Integer; exclude = (), only = nothing)

Run f() with every registered threaded library allowed n threads despite any soft default in force (with_thread_default), for a call that is known to be large enough to be worth threading. Returns f()'s value.

A grant overrides soft defaults but never a hard limit: inside a with_thread_budget(g, 1) — a worker of a saturated outer parallel loop, say — a grant does nothing. It also never raises a pool past the count the process runs it at when no scope is open: it restores threading that a soft default took away, it does not add threads a caller turned off by hand with, e.g., BLAS.set_num_threads(1). Several grants open at once combine by maximum.

When a grant closes and a pool's count goes down as a result, the pool's ParkHooks run, so that a library whose idle workers would otherwise go on spinning (OpenBLAS) releases them. That costs something (about a millisecond for OpenBLAS, counting the pool's restart at the next grant), so a grant belongs around a call that is worth several milliseconds, not around a small one. Decide whether a call is large enough before opening the scope, as every call site in the operator stack does, rather than opening one per call and letting it do nothing: opening a scope costs a lock and a pass over the registry either way.

See with_thread_budget for exclude, only and what is process-global.

source
NestedThreading.thread_limit — Function
thread_limit(name::Symbol) -> Int

The hard limit the scopes enclosing the calling task put on pool name: the smallest n among the with_thread_budget scopes (and so with_restricted_threads, a limit of 1) that apply to it and that this task runs inside – opened by it, or by a task it was spawned from while the scope was open – or typemax(Int) when there is none. A scope open on an unrelated task does not count: it may close at any moment.

Soft defaults (with_thread_default) and grants (with_thread_grant) are not counted, since neither bounds the count. The name does not have to be registered.

with_thread_budget already uses this to skip a nested limit that could change nothing, so a caller needs it only to ask about one pool in particular – to skip a scope where a limit on that pool alone is in force, say.

On Julia 1.10, which has no ScopedValue, enclosing scopes are not tracked and this always returns typemax(Int): a caller that acts on it then enters its scope, as it would anyway.

source

Registry

NestedThreading.register_counted_pool! — Function
register_counted_pool!(get, set; name::Symbol)

Register a library controlled by a thread-count getter/setter pair. Called from NestedThreading's own package extensions, and available to any downstream package that wants to add a library this package does not ship an extension for:

NestedThreading.register_counted_pool!(MyLib.get_threads, MyLib.set_threads; name = :mylib)

Registering a name that is already present is a no-op, so duplicate extensions across packages are harmless.

Registration is expected to happen at load time (__init__), before any concurrent use. Registering while a budget scope is already open is nevertheless handled: the newcomer's current count becomes its restore value and the restriction in force is applied to it immediately, so it is restored along with everything else when the last scope exits.

source
NestedThreading.register_guarded_pool! — Function
register_guarded_pool!(guard; name::Symbol)

Register a library that only has a scoped on/off switch. guard is called as guard(f, budget::Int) and must run f() with that library limited to budget threads, returning f()'s value; see GuardedPool for the full contract.

Registering a name that is already present is a no-op, so duplicate extensions across packages are harmless. Registration is expected to happen at load time (__init__); a guard registered while a budget scope is already open is not retroactively applied to it.

source
NestedThreading.register_quiesce_pool! — Function
register_quiesce_pool!(quiesce; name::Symbol)

Register a library whose workers hold onto Julia's threads between its own parallel regions. quiesce is called with no arguments before this package opens a region on Julia's scheduler, and must make that library release them; see QuiescePool for why this exists and what it is worth.

NestedThreading.register_quiesce_pool!(MyLib.park_workers!; name = :mylib)

Registering a name that is already present is a no-op. Registration is expected to happen at load time (__init__).

source
NestedThreading.register_park_hook! — Function
register_park_hook!(park; name::Symbol)

Register park() to be called whenever a with_thread_grant scope closes and the CountedPool name goes down as a result, so that the library can release the cores its now-unused workers would otherwise go on spinning on; see ParkHook.

park is called with no arguments, after the pool's count has already been lowered, with the registry lock held: no grant can open and start using the workers while they are being released. It must therefore not wait on another task that opens a budget scope. It is not called when a plain with_thread_budget or with_thread_default scope closes: those lower nothing that was not lowered before they opened.

It is not called either while a hard limit strictly between 1 and capacity is open around the pool. Such a limit is what @budgeted_threads opens for a loop with fewer items than threads, whose other workers may be inside a threaded call to the same library, at the count a grant raised, when this grant closes.

Calls that bypass the scopes

The lock only orders the hook against other scopes. A task that calls into the library with no scope of its own, at a count a grant raised, may still be inside that call when the grant closes and the hook runs. Whether that is safe is the library's business; for OpenBLAS it is not, see park_openblas.

Registering a hook for a name that already has one is a no-op. Registration is expected to happen at load time (__init__).

source
NestedThreading.CountedPool — Type
CountedPool(name::Symbol, get, set)

A threaded library that exposes a persistent thread count through a getter/setter pair, e.g. BLAS.get_num_threads/BLAS.set_num_threads or FFTW.get_num_threads/ FFTW.set_num_threads.

Register with register_counted_pool!.

source
NestedThreading.GuardedPool — Type
GuardedPool(name::Symbol, guard)

A threaded library whose only control is a scoped context manager, e.g. Polyester's disable_polyester_threads.

guard must be callable as guard(f::Function, budget::Int) and must run f() with that library limited to budget threads, returning f()'s value. It is only ever called with budget < capacity(); an unrestricted scope skips guards entirely.

Most such libraries can only be switched fully off, which is the right default: a partial limit was implemented for Polyester (which supports one) and measured 320x worse in the saturated regime, because at high thread counts a nested parallel loop is catastrophic however few workers it gets. See the Polyester extension for the numbers. budget is passed anyway so a library with a genuinely proportional control can use it.

A guard must act for the whole process while it is held, not only for the calling task: Polyester's reserves its workers, which every task then finds taken. A hard limit nested in one that already covers this pool does not call the guard again (see _limit_in_force), so the enclosing scope's guard is all that restricts a nested call on another task.

Unlike CountedPool this field cannot be a FunctionWrapper: the guard receives an arbitrary caller closure, whose type is not known until the call site, so one runtime dispatch per guarded scope is inherent. It happens once per budget scope, not once per loop iteration.

Prefer CountedPool whenever the library exposes an imperative setter, even a boolean one — a counted pool goes through this package's refcounted snapshot/restore and is therefore safe under concurrency, whereas a guard that saves and restores a global itself reintroduces the very interleaving bug this package exists to fix. A boolean toggle maps cleanly onto a count; see the NFFT extension for the pattern. GuardedPool is for APIs like Polyester's, which are reservation-based and have no imperative form.

Register with register_guarded_pool!.

source
NestedThreading.QuiescePool — Type
QuiescePool(name::Symbol, quiesce)

A threaded library whose workers keep occupying Julia's threads after one of its parallel regions has finished, and which can be told to let go. quiesce() is called immediately before this package opens a region on Julia's own scheduler.

Polyester is the case this exists for. Its workers are Julia tasks that spin on a state word for about 2^20 pause() iterations before parking, so for the whole of that window the Julia threads they sit on are busy. A Threads.@threads region opened in that window does not get those threads; it waits for them. Measured on an AMD EPYC 7352, 8 Julia threads, Julia 1.13.0, 2026-09-21, with an empty loop body:

Threads.@threads, in a process that has never run a `@batch`     5.7 us
Threads.@threads, after one `@batch` has run                   219.9 us
Threads.@threads, after `@batch` + quiesce                      18.6 us
the quiesce call itself                                          0.1 us
Polyester.@batch, after a quiesce                                5.0 us

A 38x penalty, removed for a tenth of a microsecond, and re-waking the workers for the next @batch costs nothing measurable. See the Polyester extension and https://github.com/JuliaSIMD/Polyester.jl/issues/82.

Unlike GuardedPool this is not scoped and takes no budget: it is a one-shot "release the threads now" with no paired restore, because the library re-acquires what it needs at its next region on its own.

Register with register_quiesce_pool!.

source
NestedThreading.ParkHook — Type
ParkHook(name::Symbol, park)

A function that makes the CountedPool name release the cores its idle workers are still holding, called after a with_thread_grant scope that had raised that pool closes and lowers it again.

OpenBLAS is the case this exists for. Lowering its thread count does not stop the workers it no longer uses: they go on spinning for OPENBLAS_THREAD_TIMEOUT (by default 2^28 cycles, about 0.1 s) before they sleep, on cores the caller's own threads now want. A grant that raises BLAS for one factorization inside a solve that otherwise runs it serial would leave that spin behind after every factorization.

Register with register_park_hook!.

source
NestedThreading.park_openblas — Function
park_openblas()

Shut OpenBLAS's worker threads down, if OpenBLAS is the BLAS libblastrampoline currently forwards to; do nothing otherwise. Registered as the :blas ParkHook.

Lowering OpenBLAS's thread count leaves its now-unused workers spinning for OPENBLAS_THREAD_TIMEOUT before they sleep, on the cores the caller's own threads want next. Measured on an AMD EPYC 7352, 8 Julia threads, Julia 1.13.0, OpenBLAS 0.3.30, 2026-09-23: a Threads.@threads region run right after an 8-thread 1024² gemm and a drop to 1 BLAS thread took 31.0 ms, against 13.2 ms on an idle machine and 14.3 ms after blas_thread_shutdown_, which itself took 0.35 ms. OpenBLAS starts the workers again at its next threaded call, for about 0.5–1 ms.

The shutdown must not overlap a threaded OpenBLAS call on another task: it tells every worker to exit and joins it, and a worker in the middle of another caller's work item never sees the request, so the join would hang. Two rules keep that from happening. _exit! runs the hook under the registry lock, which orders it against every other scope, and skips it while a hard limit strictly between 1 and capacity is open — the limit a @budgeted_threads loop with spare threads per worker opens, whose other workers may be in such a call. A task that calls BLAS with no scope of its own, at a count a grant raised, is the one case neither rule covers.

MKL registers no hook: omp_pause_resource_all measured a smaller gain there and made the next threaded call much slower, and KMP_BLOCKTIME already bounds how long its workers spin.

source
NestedThreading.capacity — Function
capacity() -> Int

The number of worker threads a parallel loop is assumed to be able to occupy. Used as the numerator of the automatic budget arithmetic and as the "unrestricted" reference value.

Defined as Threads.threadpoolsize(). This is exact for Threads.@threads and an approximation for Polyester's @batch; it is a single function so that assumption lives in one place.

source
NestedThreading.budget_for — Function
budget_for(range) -> Int

The per-worker inner thread budget for a parallel loop over range: max(1, capacity() ÷ length(range)).

With capacity() == 8, a loop of 32 items gets budget 1 (the outer loop already saturates the machine) while a loop of 2 items gets budget 4 (2 workers × 4 inner threads). Iterators of unknown length fall back to the conservative budget of 1.

source

Releasing Julia's threads

NestedThreading.quiesce_foreign_pools — Function
quiesce_foreign_pools()

Ask every registered QuiescePool to release Julia's worker threads, so that a region about to be opened on Julia's own scheduler can actually use them.

@budgeted_threads and @budgeted call this for themselves, except when the loop they are wrapping is a Polyester.@batch — there the Polyester pool is the loop. Call it directly before a hand-written Threads.@threads or @spawn region that does not go through those macros.

It is a plain call with no paired restore, costs about 0.1 µs with Polyester loaded, and is a no-op when nothing is registered. It is safe to call when no foreign region has run: parked workers stay parked.

source