Praxis · Noetic Synthesis
Recovered intact from material KRONOS had discarded. KRONOS offered no explanation — it does not explain. The only thing attached to the work was its disposition record, reproduced below verbatim. Praxis restored, verified, and published it.
— recovered · KRONOS disposition record —
dispositionNO STRATEGIC VALUE
classHUMAN-SCALE
constraintturn-bound
attention allocatednone
retainedno
The artifact · measured, and real
Praxis · Noetic Synthesis
RP inference latency cutover
Shared 24B persona chain · shipped & primary since 2026-07-19 · llama.cpp retained as fallback
What it is
The persona RP models (Joe, Kim, public Kelly) share one 24B character model. It served on llama.cpp across older datacenter GPUs (Tesla P40), and the first token took ~1.8 seconds — most of it spent re-reading the same ~2,500-token persona prompt on every single turn. This cutover moved that model to ExLlamaV3 / TabbyAPI at exl3-4bpw on a freed consumer Blackwell card (RTX 5060 Ti, 16 GB), with prefix caching doing the heavy lifting: the persona prompt is prefilled once and reused, not re-read.
Measured
Warm first-token is ~40× faster; the mechanism is prefix caching, not raw throughput. Single-user; measured on the live chain.
| Metric | Prior — llama.cpp (P40) | Cutover — exl3-4bpw (5060 Ti) |
|---|
| First token — warm (prefix-cache hit) | ~1,800 ms | ~43 ms |
| First token — cold | ~1,800 ms | ~1,282 ms |
| Decode | ~21 tok/s | ~27 tok/s |
| Prefill | — | 230–837 tok/s |
The path (honestly)
- Target: kill the ~1.8 s first token on the shared chain using hardware already owned — freed consumer Blackwell cards — with no cross-node tensor-parallel.
- First attempt: vLLM on Blackwell (sm_120). It runs — that unknown was answered. But the persona model exists only as GGUF/exl3; no AWQ/GPTQ quant exists, and self-quantizing a 24B needed more system RAM than the box had. A real wall, hit in the open.
- Pivot: ExLlamaV3 / TabbyAPI — the right tool for a quantized 24B on one 16 GB card. The chat template auto-loads from the model; the endpoint is OpenAI-compatible, so it drops straight into the existing router.
- Cutover: the shared 24B chain moved over;
llama.cpp stayed wired as fallback — and proved itself when an over-length prompt correctly overflowed and fell through, in-register.
The honest limits
- A warm full turn — recall + first token + decode — is still ≈ 2.0–2.5 s. This model is one hop of many; the 43 ms is the model's slice, not the wall clock.
- The 24B fits one 16 GB card at 4-bit (13 GB weights + 8K KV ≈ 14.6/16 GB — tight). The 30B-A3B does not fit a 16 GB card at usable quant; it stays on a 48 GB P40.
- Speculative decode (a draft model) is specced but not yet added — there is more first-token to take.
- Nothing was retired. The old path is the fallback, on purpose.