Slipstream Distributed Serving
Disaggregated inference across a whole rack.
Serving layer that separates prefill from decode onto different accelerator pools, migrates KV cache between them over HyperLink, and routes requests by cache locality rather than round-robin.
Specifications
| Architecture | Disaggregated prefill/decode |
|---|---|
| KV transfer | HyperLink and RDMA |
| Routing | Cache-affinity aware |
| Gain | 1.9–3.2× tokens/s per rack |
| Family | Inference & Serving |
| Software stack | FLUX — source-compatible across the whole catalog |
Disclosure
Slipstream Distributed Serving is invented. The numbers above were written to be internally consistent and plausible, not measured. Treat every figure on this site as placeholder content for layout, search and API testing.
Related
Often deployed together
Kiln Compiler
Take a trained model, get a fast engine.
- Inputs
- PyTorch, ONNX, JAX export, GGUF
- Quantization
- FP8, FP4, INT8, INT4 + accuracy report
Conductor Inference Server
One server, every framework, real batching.
- Backends
- Kiln, PyTorch, ONNX, vLLM, XGBoost
- Batching
- Continuous + prefill/decode split
AXION Cells
Pre-built model containers with one API each.
- Catalog
- 310 model containers
- Deploy targets
- K8s, bare metal, edge, cloud
Sentinel Guardrails
Programmable rails around a model endpoint.
- Rails
- Topic, safety, PII, grounding, tool-auth
- Latency
- +11 ms typical