Skip to results
JSPARK3

JSPARK3 v2

Tempo

DeepSeek-V4.1 Flash
on three DGX Sparks.

Our current daily driver, using EXL3 experts and vLLM with changes to prompt reuse, prefill scheduling, and Engram reads.

Three Sparks. One model.An illustrative three-node RoCE fabric. Rank zero exposes the API; all three machines participate in tensor-parallel execution.00 / API HEAD01 / WORKER02 / WORKERONE ENDPOINTTHREE-WAY EXECUTION / RoCE FABRIC
Three DGX Sparks · one endpointRank 0 serves the API; all three run inference.
Release
v2.0.0 · Experimental
Hardware
3 × NVIDIA DGX Spark (GB10, 128 GB each)
Serving
EXL3 experts · vLLM · TP3

Measured on our three Sparks.

L5-P measurements, September 2026. Different workloads answer different questions.

Time to first token

An uncached prompt and its exact repeat.

Prompt length
Uncached prompt42.623 s
Exact repeat0.708 s

Three-trial medians, one request at a time. Client request start to first nonempty streamed output.

Exact repeat reuses the same prefix with server caches intact. Uncached means a new prompt, with kernels already warm.

Uncached prompt49.659 s
Exact repeat0.471 s

Three-trial medians, one request at a time. Client request start to first nonempty streamed output.

Exact repeat reuses the same prefix with server caches intact. Uncached means a new prompt, with kernels already warm.

Three-minute agent tasks

The repeated Work round.

Concurrent agents

81.98 tok/s

Aggregate output over the full task window

One three-minute task round; later tool trajectories diverge; no output-quality score.

Same initial payload repeated in fresh Pi sessions, with caches intact. Server-wide accepted tokens ÷ 180 seconds, including unfinished output, health probes, and small launch/drain overhead. Active cap: 8.

110.49 tok/s

Aggregate output over the full task window

One three-minute task round; later tool trajectories diverge; no output-quality score.

Same initial payload repeated in fresh Pi sessions, with caches intact. Server-wide accepted tokens ÷ 180 seconds, including unfinished output, health probes, and small launch/drain overhead. Active cap: 8.

Full benchmark report and methods ↗Release measurement data ↗

Prose
33.21–34.28 tok/s
Code
67.31–75.64 tok/s

Single stream, after first output. Ranges across 3 prose and 4 code tasks; one answer per task. (Completion tokens − 1) ÷ (HTTP duration − TTFT), including transport, finalization, and speculative chunks. Not GPU-only decode.

146.12 tok/s in the separate six-stream short-answer test. Eight categories, 150–256-token caps, one wave per category; transport fixtures, not answer-quality grades. Mean of eight category wave rates, including initial wait.

What changed

Prompt reuse
Retain reusable prefixes at valid state boundaries, including changed suffixes and tool continuations.
Prefill scheduling
Keep large chunks for a request running alone; share a smaller budget when another stream is generating.
Engram reads
Read packed weights and scales together. The preparation preserves the values and quantization.

Run Tempo

Three DGX Sparks, a working RoCE fabric, and local storage for the pinned model, experts, draft, and prepared Engram files. The install guide starts with a fit check and configuration worksheet.

Install v2.0.0 ↗
Known limitations
  • Experimental serving recipe; historical measurements come from one three-Spark fleet.
  • Functional code passed 2/4 tasks; this is not a broad quality ranking.
  • A short request arriving two seconds into a 64K prefill waited 40.458 seconds.
  • 300,000 tokens is configured context, not certified usable context or C8 capacity.
  • Uncached prompt measurements use warmed kernels; they are not cold machine starts.
  • Work counts unfinished output and does not score quality. Mia active cap 4 differs from Tempo cap 8.
  • Native vision and optional Pi desktop use are separate from the throughput cohort.
Release, evidence, and credits

v2.0.0 · experimental. JSPARK3 names our current three-Spark daily driver, chosen through internal benchmarks and use. It does not claim to win every benchmark.

DeepSeek created the model. Tony and Kai’s serving work, bot-lab-21’s EXL3 experts, and vLLM underpin this recipe. Code, weights, and drafts retain their upstream terms. This is serving engineering, with no new fine-tune or model merge.

Versioned release · GitHub source · Hugging Face recipe · Release evidence

Generated release summary · Previous driver: GLM-5.3 Flash / Cadence