Three DGX Sparks, one model server.
I had three DGX Sparks and every recipe out there was written for two or four. JSpark3 is the setup that makes three work: GLM-5.3 Flash running across all of them as one endpoint, about 1.4x faster than the two-box recipe. Every piece is pinned so you can rebuild it exactly, and the numbers are published with the misses left in.
Scope
Architecture
Each Spark holds one tensor-parallel shard and one third of the routed experts. The three fabric legs form a triangle so every rank reaches each peer directly.
JSpark3 v1: one GLM-5.3 Flash endpoint across three NVIDIA DGX Sparks
TP 3 · EP 3 · EXL3/TR3 4-bpw target · DFlash2 k=7 draft · FP8 KV cache · 1,000,000-token configured context
Client
OpenAI-compatible HTTP
POST /v1/chat/completions
model: glm-5.3-flash
DGX Spark · rank 0 · API head
GB10 (SM 12.1), aarch64 · vLLM mp worker
TP 1/3: attention, KDA, shared expert, LM head · EP: 96 of 288 routed experts
DFlash2 draft, TP 3 · FP8 KV · prefix cache
64 GiB cgroup · CUDA graphs 8/16/24/32/48
W8A16 overlay: 169 modules / 225 tensors
Control and identity
Management LAN: SSH, Gloo/TP sockets, API
Per-rank preflight: GID, MTU, routes, image digest, checkpoint bytes, manifest, memory
Start order 2 → 1 → 0 · hash-bound manifest
Refuses cgroup, NCCL, overlay, loader drift
Verify: shards, graphs, focused witness
DGX Spark · rank 1 · headless
GB10 (SM 12.1), aarch64 · vLLM mp worker
TP shard 2/3 · EP: 96 of 288 routed experts
DFlash2 draft, TP 3 · FP8 KV · prefix cache
64 GiB cgroup · CUDA graphs 8/16/24/32/48
same overlay, same hash-gated transforms
DGX Spark · rank 2 · headless
GB10 (SM 12.1), aarch64 · vLLM mp worker
TP shard 3/3 · EP: 96 of 288 routed experts
DFlash2 draft, TP 3 · FP8 KV · prefix cache
64 GiB cgroup · CUDA graphs 8/16/24/32/48
same overlay, same hash-gated transforms
Reading the diagram
Orange lines: three RoCE-v2 fabric legs, each on its own network at MTU 9000; every Spark owns two. Dashed: management.
Model bytes are never inside the image or the recipe; each rank bind-mounts the pinned checkpoint read-only.
Every serving byte is validated before start. W8A16 converts the BF16 trunk to INT8 Marlin at load; experts stay EXL3, the 34 KDA f/g modules are excluded. The DFlash2 draft is built over all three ranks (draft TP 1 in the profile is ignored).
Tensor parallel 3 and expert parallel 3. Every rank holds a shard of attention, KDA, the shared expert and the LM head, plus 96 of the 288 routed experts. The DFlash2 draft is built over all three ranks; the profile's draft TP 1 setting is ignored by this loader, so the diagram shows what is actually loaded.
Three RoCE-v2 legs, each on its own network at MTU 9000. Every Spark owns two fabric interfaces. Preflight checks the GID, MTU and routes on each rank before anything starts.
The BF16 trunk is converted to INT8 Marlin at load: 169 modules, 225 tensors, group ladder 128/64/32. The 34 KDA f/g modules are excluded. It frees 1,595,392,320 bytes per rank.
Start order 2, then 1, then 0, bound to a hash-checked release manifest. The entrypoint refuses on cgroup, NCCL, overlay or loader drift. Verify confirms shards, graphs and a focused witness.
Evidence
The authors' pinned benchmark scripts, run unchanged against the JSpark3 release build except for the endpoint address and model name. Author values are the numbers they published.
Decode, tok/s (mean of three runs)
Hello (17-token stop)
1.23xratio
runs 46.106 / 46.405 / 45.820
runs 37.9 / 36.9 / 37.3
Structured count 1 to 200
1.21xratio
runs 83.988 / 84.580 / 84.161
runs 69.0 / 68.5 / 71.2
is_prime code
1.18xratio
runs 65.840 / 68.246 / 65.544
runs 52.3 / 58.7 / 58.2
Draft acceptance 0.8324 against FlyCockpit's 0.815. FlyCockpit's runs were first-serve at GPU memory utilization 0.87; these were warm-server runs at 0.83, the release envelope.
Median of five 400-token runs, tok/s
Structured count 1 to 200
1.35xratio
Prose hash-map
1.36xratio
Accepted-per-draft ratios match theirs almost exactly: 0.9533 against 0.959 on structured, 0.3357 against 0.341 on prose. The draft behaves the same; the engine is faster.
| Estimator | Mia TP2, 2 Sparks | JSpark3, structured count3 Sparks | JSpark3, clamp code3 Sparks |
|---|---|---|---|
| Concurrency C1 | |||
| per-stream decode tok/s | 62.9 | 86.56 | 84.47 |
| time to first token | 719 ms | 461 ms | 391 ms |
| Concurrency C2 | |||
| per-stream decode tok/s | 51.7 | 79.26 | 70.56 |
| aggregate decode tok/s | 103.3 | 76.95 | 139.77 |
| Concurrency C4 | |||
| per-stream decode tok/s | 37.1 | 50.08 | 62.80 |
| aggregate decode tok/s | 146.5 | 200.29 | 251.13 |
Mia publishes one value for its high-accept prompt family without saying which prompt, so both of ours are shown.
At two streams the structured aggregate falls below Mia's published figure: the release build serializes low-concurrency work below its eight-sequence graph floor, the same artifact the internal ablation shows at three streams.
Measured on 2026-09-02 against the release build held immutable, three Sparks, warm server. FlyCockpit T0 script at commit 9093765c; Mia tests/bench_decode.py at commit c190db1a; sparkDash release 1.8.5 at commit e93fc87d.
Single-stream decode, tok/s
Structured count
1.41xratio
Code
1.49xratio
Prose
1.45xratio
Same frozen 24-request screen on the same fleet, thinking off, temperature 0, 400 max tokens. The Mia recipe is the current two-Spark release at commit c190db1a, run here with one compatibility repair.
Aggregate decode, tok/s
One agent prompt, independent runs; each agent chose its own path.
Two Mia TP2 lineages and one FlyCockpit-derived build were run on this fleet with disclosed adaptations, and each was given the same agent prompt as JSpark3. Same prompt, independent trajectories: product evidence, not an engine-rate comparison.
44.583 tok/s aggregate decode over 132 requests and 32,618 generated tokens; mean time to first token 3.704 s.
Commit 0e2e78f, run locally with site, storage, API, and safety adaptations. 24.913 tok/s aggregate decode over 43 requests and 105,198 generated tokens.
Commit c190db1a, runnable at full context only with the GLM53_INDEXER_WORKSPACE=rightsize repair: an adapted reproduction, not an exact one. 24.728 tok/s aggregate decode over 24 requests and 76,540 generated tokens.
Commit 9093765c with minimal correctness and safety adaptations, not the literal upstream launcher. 29.042 tok/s aggregate decode over 58 requests and 130,971 generated tokens.
The public comparison is against recipes that were publicly available before JSpark3. Their numbers are author-reported; ours come from our own fleet. Prompts, instruments, and envelopes differ, and the node count is on every row, so read the table as context, not a ranking. No percentage is computed between rows.
Every row states hardware count, quantization and speculation, context, workload, and source. Sources are pinned commits, listed with full URLs on the repository's benchmarks page.
Internal ablation
The control is JSpark3 itself with one switch off. Same three Sparks, same pinned checkpoint and DFlash2 draft, same container image, same TP3/EP3 topology and serving envelope, same request sets and estimator. The only change is that the selective W8A16/Marlin trunk overlay is disabled, so the trunk serves in BF16 as it came from upstream. That build is unreleased internal development evidence, not a product and not a market comparison; it exists to isolate what the overlay alone changed.
Every bar below is JSpark3 measured against that control. Two single-stream rows are shown: the campaign medians against the control's earlier battery, and a strict same-day pair (candidate battery r3 against control battery r6). Green went up, red went down, one scale.
Single-stream decode · campaign medians vs the control's earlier battery
Code
+3.75%
Structured count
+5.74%
Prose
+2.62%
Single-stream decode · same-day pair, candidate r3 vs control r6
Code
+7.27%
Structured count
+6.63%
Prose
+8.35%
C6 per-stream median
+2.91%
Matched concurrency waves · aggregate service throughput
C12
+0.16%
C24
+1.21%
C48
+3.47%
Matched long prefill · 113,908 tokens
Prefill proxy
−3.38%
What the overlay improved. Single-stream decode medians: code +3.75%, structured count +5.74%, prose +2.62% against the control's earlier battery; +7.27%, +6.63% and +8.35% in the strict same-day pair. Token pacing: median inter-token interval 98.645 to 91.912 ms (−6.83%), p99 −10.27%, worst interval 364.416 to 148.344 ms (−59.29%). Aggregate throughput at 48 streams +3.47%. 1,595,392,320 bytes of weight memory freed per rank.
What it cost, and what was missed. Long prefill −3.38% with time to first token +3.50% on 113,908 tokens. Fairness did not improve. Time to first token at 48 streams reached a p90 of 96.722 s. Two internal promotion gates were missed: a code median of 66.257 against a 67.0 floor, and a demonstration pacing run of 14 against a limit below 5.
Evidence grade: engineering evidence. One fleet, operated by the project, with no third-party reproduction yet. No correctness, stability, or safety failure was observed in any run. Estimators, sample sizes, receipts, and the three evidence classes are on the repository's benchmarks page.
Reproducibility
The construction is exact or the recipe does not start. The measurements are fully specified but the numbers are not guaranteed; your fleet will differ, the bytes will not.
Any checkpoint revision, draft revision, image digest, or transform source that is not the pinned one.
A cgroup other than 64 GiB with swap off; any NCCL_PROTO, NCCL_ALGO, or NCCL_IB_ADDR_RANGE override; overlay or KDA environment drift.
A preflight row that differs from the expected row, a preflight checksum mismatch, a missing or unbound image receipt, or an existing container with the release name.
Overlay, loader-hook, and transform before/after hash drift; a missing or duplicated seam. Every rank validates every serving byte before start.
cp .env.example .env
./scripts/clean-room-setup.sh --env-file .env --output preflight.json
preflight_sha=$(sha256sum preflight.json | cut -d' ' -f1)
./scripts/start.sh --env-file .env \
--preflight preflight.json --preflight-sha256 "$preflight_sha" \
--confirm START-JSPARK3
./scripts/health.sh --env-file .env --manifest jspark3-release-manifest.json
./scripts/verify.sh --env-file .env --manifest jspark3-release-manifest.json \
--output verify.json --log-output verify-rank0.logEvery command has --dry-run. Confirmation tokens are typed, never defaulted. The full nine-step installation, from bare Docker hosts to a verified endpoint, is in the repository's install page.
Provenance
Nothing here is a floating tag. The recipe checks each of these before it serves a token.
Mia-AiLab/GLM-5.3-Flash-EXL3-TR3-4bpw at revision 25a44fdbf16862a46b7cc9921142c6c81350af2f, itself byte-identical to brandonmusic/GLM-5.3-Flash-tr3-4bpw at 5ab363a8dcf6405955fd5f99671e01a1c9fb124b. The JSpark3 Hugging Face repository re-hosts this revision shard for shard with the same hashes; the preflight accepts either source because the bytes are identicalincoai/GLM-5.3-Flash-DFlash2 at revision dc77ff1c99eeb2df044ee3d4f0094eb033fee410, k=7ghcr.io/miaai-lab/glm-5.3-flash-2x-dgx-sparks at digest sha256:9bb1557a4234fce63d59599e44d10747eabd742beb337eebf9e7070be8a0fd58, launched by digest, not redistributed487ecf187 as shipped inside the pinned image; five hash-gated runtime transforms are applied at startGLM-5.3-Flash-EXL3-3x-DGX-Sparks at 9093765c757bd1976372196e44af84a67cf86bad; vcruz305 GLM-5.3-Flash-EXL3-K2-DGX-Spark-recipe at 622cb878d66f703c597bd6baaa2423caa1786f99glm-5.3-flashLicensing
The recipe is ours to license. The model bytes it loads are not. Read this before deploying for anything commercial.
Recipe code
The scripts, overlays, transforms, tooling, and documentation, with third-party notices. Use, modify, redistribute.
Target checkpoint
Source-available and attribution-required; not OSI open source. Brandon M. Music created the EXL3/TR3 checkpoint, Mia-AiLab re-hosts it, and JSpark3 re-hosts that revision in turn under the same license, with the required attribution reproduced verbatim. Downstream copies stay under this license.
DFlash2 draft
Research and evaluation only. Commercial use of the draft requires a license from Inco. Without the draft the recipe still serves, without speculative decoding.
The assembled endpoint is therefore neither unrestricted open source nor commercial-ready. The repository's licensing page lists every term and its practical effect, and the ownership statements the project does not make.
Credits
JSpark3 did not train, fine-tune, or quantize anything. The contribution is the three-Spark architecture, the runtime adaptation, the operating envelope, the experimental campaign, and the reproducible serving recipe.
Brandon M. Music for the ShapleyMcg EXL3/TR3 4-bpw checkpoint that every rank loads. MiaAI-Lab for the two-Spark EXL3 recipe, the pinned container image, and the checkpoint re-host that JSpark3 mirrors in turn. FlyCockpit for the three-Spark EXL3 lineage the TP3 overlay transform is reconstructed from. vcruz305 for the K-pool tail correction. tonyd2wild for scheduler and concurrency benchmarking context. sfxnz for DGX Spark serving context. Inco AI for DFlash2. Z.AI for GLM-5.3 Flash. The vLLM project for the engine.
Every upstream author is credited by name, repository, and pinned commit or revision in the repository's third-party notices.
The repository ships CITATION.cff and CITATION.bib. Cite the release by name, version, and repository.
JSpark3 v1, version 1.0.0 (2026)
https://github.com/jakejharris/jspark3