Our current daily driver, using EXL3 experts and vLLM with changes to prompt reuse, prefill scheduling, and Engram reads.
Three DGX Sparks · one endpointRank 0 serves the API; all three run inference.
Release
v2.0.1 · Experimental
Hardware
3 × NVIDIA DGX Spark (GB10, 128 GB each)
Serving
EXL3 experts · vLLM · TP3
Measured on our three Sparks.
Tempo measurements, September 2026. Earlier test records call this build L5-P. Different workloads answer different questions.
Short speed screen
Code, prose, and parallel streams.
Aggregate end-to-end output. Measured on our three DGX Sparks.
Code · C1 · 1 offered stream
TempoOur measurement64.0 tok/s
MiaOur measurement59.2 tok/s
1.08× Mia’s measured code throughput in this cell.
How throughput scales tok/s · higher is better
16010550
C1C2C3C4
TempoMia
Y-axis: 50–160 tok/s · does not start at zero
Same short screen, different recipes. Completion tokens across all streams ÷ shared HTTP wall time, including initial wait. One wave per category and concurrency, 150–256-token caps. These fixtures measure speed, not answer quality.
Comparisons stop at 4 offered streams, within Mia’s active-request cap. Tempo allows 8 active requests. These are recipe-level comparisons, not an isolated kernel or quantization test.
Tony’s published aggregate reference is available in “8-category mean”. It is separate from the code and prose measurements shown here.
Accepted backend tokens ÷ 180 seconds, including unfinished output, health probes, and small launch/drain overhead. Same initial tasks; later tool histories diverge. Repeat uses fresh Pi sessions with caches intact.
One cohort, no output-quality score or productivity claim. Three agents stay within every recipe’s active-request cap. Tony’s local run includes fabric, storage, loader, and allocator adaptations; it is separate from his published short-screen figures.
Single stream, after first output. Ranges across 3 prose and 4 code tasks; one answer per task. (Completion tokens − 1) ÷ (HTTP duration − TTFT), including transport, finalization, and speculative chunks. Not GPU-only decode.
146.12 tok/s in the separate six-stream short-answer test. Eight categories, 150–256-token caps, one wave per category; transport fixtures, not answer-quality grades. Mean of eight category wave rates, including initial wait.
Three-minute agent tasks
The repeated Work round.
81.98 tok/s
Aggregate output over the full task window
060120 tok/s
One three-minute task round; later tool trajectories diverge; no output-quality score.
Same initial payload repeated in fresh Pi sessions, with caches intact. Server-wide accepted tokens ÷ 180 seconds, including unfinished output, health probes, and small launch/drain overhead. Active cap: 8.
110.49 tok/s
Aggregate output over the full task window
060120 tok/s
One three-minute task round; later tool trajectories diverge; no output-quality score.
Same initial payload repeated in fresh Pi sessions, with caches intact. Server-wide accepted tokens ÷ 180 seconds, including unfinished output, health probes, and small launch/drain overhead. Active cap: 8.
Time to first token
An uncached prompt and its exact repeat.
03060 seconds
Uncached prompt42.623 s
Exact repeat0.708 s
Three-trial medians, one request at a time. Client request start to first nonempty streamed output.
Exact repeat reuses the same prefix with server caches intact. Uncached means a new prompt, with kernels already warm.
03060 seconds
Uncached prompt49.659 s
Exact repeat0.471 s
Three-trial medians, one request at a time. Client request start to first nonempty streamed output.
Exact repeat reuses the same prefix with server caches intact. Uncached means a new prompt, with kernels already warm.
Tempo builds on Tony and Kai’s DeepSeek Spark serving recipe ↗ and bot-lab-21’s model files, made smaller with WestWaters’ Pollard method (EXL3). Mia’s three-Spark recipe ↗ provides a comparison. Its ideas for combining model calculations and sharing memory helped inform Tempo; Tempo does not reuse its code.
Retain reusable prefixes at valid state boundaries, including changed suffixes and tool continuations.
Prefill scheduling
Keep large chunks for a request running alone; share a smaller budget when another stream is generating.
Engram reads
Read packed weights and scales together. The preparation preserves the values and quantization.
Run Tempo
You need three DGX Sparks, fast direct network connections between them (RoCE), and enough disk space. The install guide first checks that your machines have what they need.
Tempo downloads all its model files from bot-lab-21’s DeepSeek release, including a smaller helper model that helps generate answers faster. The official DeepSeek release named in our docs tells you where the model comes from; it is not another download. JSPARK3 Tempo on Hugging Face hosts the recipe, not model files.
Experimental serving recipe; historical measurements come from one three-Spark fleet.
A short request arriving two seconds into a 64K prefill waited 40.458 seconds.
300,000 tokens is configured context, not certified usable context or C8 capacity.
Uncached prompt measurements use warmed kernels; they are not cold machine starts.
Work counts unfinished output and does not score quality. Mia active cap 4 differs from Tempo cap 8.
Native vision and optional Pi desktop use are separate from the throughput cohort.
Release, evidence, and credits +
v2.0.1 · experimental. JSPARK3 names our current three-Spark daily driver, chosen through internal benchmarks and use. It does not claim to win every benchmark.
DeepSeek created the model. Tony and Kai’s serving work, bot-lab-21’s EXL3 experts, and vLLM underpin this recipe. Code, weights, and drafts retain their upstream terms. This is serving engineering, with no new fine-tune or model merge.
