One or two Sparks. Two ways to run them.

Self-hosted inference on Nvidia DGX Spark (Grace CPU, GB10 GPU, Ubuntu 24.04, arm64) plus Open WebUI, n8n, and Hermes on the personal node. Share one model at TP=2, or give each box its own process at TP=1. New hardware: install a coding agent first and feed it llms.txt.

A · clustered TP=2 B · isolated solo TP=1 vllm-node (not tf5) Open WebUI n8n Hermes on node-a port 8000 stays off overlays

Just unboxed a Spark?

Do not start from a year-old gist. Install Grok Build, Codex, or Claude Code, then point it at the playbook. It will interview you and build the layout from current eugr/spark-vllm-docker commands.

Read the agent subpage, then feed llms.txt and llms-full.txt to the agent. The generator below is the same procedure as downloadable bash if you would rather run scripts yourself.

Why two layouts

A 122B-class model at tensor-parallel 2 uses both GPUs as one pool. One listener, one weight set, apps still isolated per node. Use it when you want maximum context and one shared brain.

Isolated solo exists because a shared head can see every prompt and couples uptime. Solo gives each node its own vLLM at TP=1 so models and restarts do not cross the copper link. A single Spark is always solo.

Architectures

A — Clustered

One vLLM, TP=2 over the DAC. Head on node-a :8000. Worker has no API. Multi-node default is no-Ray (--ray is opt-in). Typical recipe: qwen3.5-122b-fp8 (cluster_only). Supported window: 262144.

B — Isolated solo

Each node runs run-recipe.sh RECIPE --solo -d --tp 1. Cluster unit disabled on boot. Supported windows: INT4 122B-class 32768; 35B FP8 65536. Do not raise those values. Do not --solo a cluster-only recipe.

Setup generator

Pick the layout and model first. That tab locks the supported max-model-len. Fill the node table. Downloads bake those values in. Secrets stay in sessionStorage only (cleared when the tab closes). Scripts target a new box: vllm-node, build-and-copy.sh without --tf5, hf-download.sh for weights.

Model (from the tab)

Layout

vLLM flags

Recipe

Served name

HF repo

HF cache dir

max-model-len (supported only)

tool-call-parser

Your nodes

Single Spark (ignore node-b; clustered tab will refuse)

node-a · personal

mgmt IP

DAC IP

HF token

Linux username

node-b · workload

mgmt IP

DAC IP

Open WebUI :8080 n8n :5678 Hermes on node-a :9119 host firewall disable Ubuntu desktop

max-model-len is the supported window for the selected tab. Raising it OOMs a single GB10. Port 8000 stays off overlays. Example IPs in placeholders are not yours.

Preview · node-a

Preview · node-b

Trust notes

Architecture A: the head API sees prompts from both UIs.
Architecture B (and a single Spark): each API sees only its own UI.
Knowledge bases and chats do not share a volume across nodes.
vLLM has no auth. Keep port 8000 off overlays. Clustered node-b reaches the head on the DAC only.