ruvllm-pi-cluster-v1.0.0-iter28

aigc3d/LAMruvllm-pi-cluster-v1.0.0-iter28May 5, 2026by ruvnet

AI Summary

Shipped the first multi-Pi LLM cluster, achieving 20.5 tokens/second aggregate throughput on 4x Pi 5 hardware.

Key Highlights

  • First multi-Pi LLM cluster
  • 20.5 tok/s aggregate on 4x Pi 5
  • New `ruvllm-pi-worker` binary
  • Aarch64 cross-build chain

New Features

  • Multi-Pi cluster support
  • New worker binary
  • Cross-compilation support

Full Release Notes

ADR-179 / 28-iteration `/loop` ships the first **multi-Pi LLM cluster** in this repo.

## Headline

**SOTA: 20.5 tok/s aggregate** on TinyLlama-1.1B Q4_K_M across 4× Pi 5 + AI HAT+. 7.9× over iter-13 single-Pi fp16 baseline. Total cluster power ~28 W. Hardware ~$400.

## What ships

- New `ruvllm-pi-worker` bin in `ruvector-hailo-cluster` (sibling to `ruvector-hailo-worker` from ADR-167)
- aarch64 cross-build chain — `rustls-tls` swap on `hf-hub`, Cortex-A76 `+lse +rcpc +fp16 +crc` rustflags
- New `hub-download` cargo feature on `ruvllm` — gates HF Hub auto-download behind opt-in (default-on for workstation, off for cross-builds)
- `deploy/ruvllm-pi-worker.{service,env.example}` + `install-ruvllm-pi-worker.sh`
- `deploy/ruvllm-cluster-smoke.sh` — parallel cluster bench harness
- `crates/ruvector-hailo-cluster/RUVLLM_CLUSTER_PLAN.md` — full 28-iter log
- 4 new ADRs: 179 (this work), 180/181/182 (next-phase scoping)

## Cluster topology

| Node | Tailnet IP | :50051 (embed) | :50053 (LLM) |
|---|---|---|---|
| cognitum-v0 | 100.77.59.83 | ✓ | ✓ |
| cognitum-cluster-1 | 100.80.54.16 | ✓ | ✓ |
| cognitum-cluster-2 | 100.77.220.24 | ✓ | ✓ |
| cognitum-cluster-3 | 100.73.75.53 | ✓ | ✓ |

## Convergence trajectory

| Iter | Quant | nPi | tok/s/Pi | Aggregate | vs base |
|---|---|---:|---:|---:|---:|
| 13 | fp16 | 1 | 2.9 | 2.6 | 1.0× |
| 23 | fp16 | 3 | 2.9 | 8.7 | 3.3× |
| 24 | Q4_K_M | 1 | 9.0 | 9.0 | 3.5× |
| **26** | **Q4_K_M** | **4** | **5.1** | **20.5** | **7.9×** |
| 27 | Q3_K_S/Q2_K | 1 | 7.9 | strike 1 | |
| 28 | multi-inflight | 1 | 7.0 | strike 2 → converged | |

## Crates published

- `ruvllm 2.2.0` → crates.io
- `ruvllm-cli 2.2.0` → crates.io (new on registry)

## Out of scope (next ADRs)

- **ADR-180** ServingEngine continuous batching wiring (~2× projected, ~50-80 tok/s aggregate target)
- **ADR-181** in-tree pi_quant + BitNet b1.58 (~2-3× projected, ~80-100 tok/s target)
- **ADR-182** Hailo-10H hardware migration (~5-10× projected, ~150-300 tok/s target)