Benchmarks

The comparison harness in benchmark/comparison/ reconstructs the same cases with Ristretto, BART, SigPy, MRIReco.jl, MIRT.jl and MRpro, from the same prepared data, on the same node (see its README). Every table below is a snapshot: the Hardware and Versions sections say when it was measured and which release of each toolbox it measured.

Method. Each toolbox solves the same problem: the same encoding model, the same regularizer, with λ calibrated per toolbox so that all of them reach the same accuracy, since the toolboxes scale the data and the regularizer differently. The fair number is then time to accuracy: how long each toolbox needs to bring the NRMSE against the ground truth below a common target (the accuracy race). Fixed-iteration timings (matched effort) are listed too, but an iteration costs and achieves different things in different solvers. Times are the fastest of three runs after a warm-up, each planning its FFTs from scratch, from data on the host to an image on the host.

FFT planning. FFTW chooses how to compute a transform when it is planned. FFTW_ESTIMATE guesses from the sizes and plans in well under a millisecond; FFTW_MEASURE times candidate algorithms on the machine, which costs about 0.1–0.2 s per 2D transform and 1–1.5 s per 3D one and gives a plan that runs faster. What FFTW learns by measuring ("wisdom") can be saved and reused by a later plan of the same transform. The toolboxes differ here:

• BART plans with FFTW_ESTIMATE by default; with BART_USE_FFTW_WISDOM=1 it plans with FFTW_MEASURE and saves the wisdom to files, which later calls read.
• MRIReco.jl always plans with FFTW_MEASURE and keeps the wisdom only in memory, so every new Julia session measures again.
• Ristretto chooses per reconstruction: FFTW_MEASURE when the transforms the solve will run repay the measuring, else FFTW_ESTIMATE (fft_planning = :auto, the default), and saves the wisdom to disk for later sessions.
• SigPy and MRpro do not use FFTW: NumPy's and PyTorch's FFTs plan without a measuring step. On the GPU every toolbox uses cuFFT.

The benchmarks charge every run its planning: no wisdom survives from one run to the next, neither in memory nor on disk. MRIReco therefore pays its measuring in every run, about 0.3 s per solve on the 128² Shepp–Logan case, most of its time there. BART is timed both ways: the BART (MEASURE) column measures in every run, the BART column estimates. Measuring costs BART about 0.3 s per 2D solve, several times its estimating time there, and about breaks even on the 3D and cine cases, where the faster plans run long enough to repay it.

Start-up and warm-up

The tables below favour the toolboxes that are slow to start. They time warm solves, while a user also pays to start the toolbox: a BART call is a new process that starts in milliseconds, a Python toolbox has to be imported, and a Julia toolbox also compiles each solver the first time it is called. Measured on one case (Shepp–Logan 2D, 8 coils, Cartesian; ℓ₁-wavelet / CG-SENSE), 8 threads, in fresh processes (benchmark/startup/):

The Julia numbers assume precompiled packages; precompiling Ristretto itself once takes about two minutes more. For BART every call is a fresh process, so its first and warm solves are the same and include the process start and file I/O that the tables below leave out. The first solve in a Julia session is therefore two to three orders of magnitude slower than the warm one, which matters for a script that reconstructs one image and hardly at all for a session that reconstructs many.

Time to accuracy

Target NRMSE against the ground truth in the method column; time per toolbox, the fastest in bold, the NRMSE reached in brackets. 8 threads, OpenBLAS. BART's TGV is not quite the same penalty: since BART 1.0 it weights the two terms by (1, √3) and halves the symmetrised gradient, so its λ is calibrated separately and only the target is shared.

Matched effort

ℓ₁-wavelet regularized SENSE, 20 iterations of each toolbox's default solver for the problem.

Thread scaling

Ristretto's time for the matched-effort ℓ₁-wavelet reconstruction, from 1 to 16 threads. Single-coil 2D problems are too small to gain from threads and run serially by design.

GPU

The matched-effort rows on an NVIDIA GPU (CUDA), copies to and from the device included.

Hardware

• openblas, 8 thread(s): AMD EPYC 7763 64-Core Processor; Julia 1.13.1; BLAS libopenblas64_.so; measured 2026-10-10 to 2026-10-12.
• cuda, 1 thread(s): AMD EPYC 7763 64-Core Processor, GPU NVIDIA A100-SXM4-40GB; Julia 1.13.0, 1.13.1; BLAS libopenblas64_.so; measured 2026-10-02 to 2026-10-12.

Versions

The toolbox versions behind the CPU tables (8 threads, OpenBLAS):

and behind the GPU table: