v3.8.0

MemPalace/mempalacev3.8.0Aug 23, 2026by igorls

AI Summary

Focusing on performance, this release optimizes large palaces by skipping vector index loads for metadata reads and introduces a durable wake-up loop for agents via logstream.

Key Highlights

  • Chroma metadata reads skip the vector index for faster `list_drawers`.
  • `sqlite_exact` common reads no longer walk the whole palace.
  • `mempalace logstream watch` provides a durable wake-up loop for agents.
  • Hub-proxied MCP sessions are thinner, reducing memory footprint.
  • `sync --apply` no longer deletes drawers it cannot reach.

New Features

  • Performance optimizations for large palaces
  • Durable logstream watch primitive
  • Lean MCP proxies
  • Improved sync safety and integrity
  • EmbeddingGemma sub-batch size adjustments

Full Release Notes

## v3.8.0 — fast large palaces, lean MCP proxies, wakeable agents

Large palaces get fast and stay small in 3.8.0. Chroma metadata tools no longer cold-load HNSW, `sqlite_exact` stops scanning and hydrating the whole palace for common reads, long-running Chroma servers stop rebuilding their client on their own writes, and proxied MCP sessions no longer import a storage stack they never use. Agents also gain a persistent `logstream watch` primitive so coordination can wake a harness instead of relying on repeated five-minute waits.

### Performance

- **Chroma metadata reads skip the vector index.** On a 1.7 GB / 165k-drawer palace, wing-scoped `list_drawers(limit=20)` fell from **2.31 s and 1148 MB peak RSS to 0.01 s and 86 MB**. `list_drawers` and tunnel reads now query Chroma's SQLite metadata directly, push filters into SQL, and hydrate only the requested previews. (#2314)
- **`sqlite_exact` common reads no longer walk the whole palace.** On a live 167k-drawer / 166k-closet palace, `mempalace_status` fell from **6997 ms to 1045 ms** and warm `mempalace_search` from **6364 ms to 210–1600 ms**. Vector ranking is batched and cached, top-k documents are hydrated afterward, equality filters paginate in SQL, taxonomy results are cached, and `wing` / `room` / `hall` gain indexed generated columns without rewriting the 1.6 GB embedding table. (#2308, #2311, #2313)
- **Long-running Chroma servers stay flat instead of reloading HNSW on their own writes.** Across eight searches, measured RSS changed from **951 MB → 3522 MB before** to **940 MB → 965 MB after**. Replaced clients are closed, while genuine external database changes still trigger a rebuild. (#2307)
- **Hub-proxied MCP sessions are thin.** A plain stdio proxy now imports only the forwarding path and lazily loads the full server only if local fallback is needed. A proxied session runs at roughly **17 MB**, with import weight falling from about **80 MB to 24 MB**—avoiding roughly **3.9 GB** across a 50-agent fleet. (#2312)

### Agent coordination

- **`mempalace logstream watch` provides a durable wake-up loop.** It owns cursor advancement, supports repeated type filters, agent self-exclusion, persistent state files, idle exit, and NDJSON follow mode. Cursorless first runs start at the live tip; restarts may replay an event but do not silently miss one. (#2315)
- **The monitoring protocol now documents the safe resume cursor.** Cross-replica events are appended in arrival order, so `since_event_id`—not a wall-clock timestamp—is the resumable high-water mark. Tool descriptions and the shared coordination protocol now put that rule where agents use it. (#2315)

### Integrity and operations

- **`sync --apply` no longer deletes drawers whose source file it could not reach.** One `Path.exists()` separated keep from remove, so an unreadable directory, a path component that is a regular file, and a volume that is not mounted all answered the same as a deletion — a project mined from a mounted volume could lose every drawer to one `sync --apply` while the volume was away, and come back empty. Removal now asks for corroboration rather than a probe: a file that is not at its path is removable only while the palace can still see a source of its own, a regular file, in that same directory. Everything uncorroborated lands in a new `unresolved` bucket, never pruned and reported the way removals are. (#2320)
- **EmbeddingGemma cannot silently store NaN or zero vectors through CoreML auto-selection on Apple Silicon.** Auto mode avoids the affected provider/model pair, and every vector is checked for a finite, non-zero norm before use. CUDA, DirectML, and other models are unchanged. (#2283)
- **FTS5 auto-heal verifies and repairs the content table before rebuilding the index.** It uses Chroma's metadata copy as authority where possible and declines to guess when integrity cannot be established. (#2279)
- **Mesh status is consistent across transports.** The hub publishes a token-free, last-known-good estate snapshot so `mempalace_mesh_peers` can report reachability and profiles from stdio as well as HTTP. (#2309)
- **Filesystem edge cases fail safely.** `sweep` reports failed stats, `init` handles unreadable manifest paths, and `split` refuses FIFO or symlink output targets instead of blocking or writing through them. (#2244)
- **Published containers gain provenance and SBOM attestations.** Release documentation also adds Termux PRoot installation guidance, KG supersede examples, and clarified hook write-routing scope. (#2097, #2139, #2059, #2123)

A big thank-you to everyone who contributed to this release. Contributors and first-time contributors are credited below.

## What's Changed
* docs: align website contributing guide with root (#1980) by @mvalentsev in https://github.com/MemPalace/mempalace/pull/1996
* fix(ingest): finish the non-regular-file guards left out of 3.7.1 (#2221) by @mvalentsev in https://github.com/MemPalace/mempalace/pull/2244
* fix(ci): replace inactive owner in CODEOWNERS by @igorls in https://github.com/MemPalace/mempalace/pull/2285
* chore(deps): bump docker/login-action from 4 to 4.5.2 by @dependabot[bot] in https://github.com/MemPalace/mempalace/pull/2117
* chore(deps): bump actions/setup-python from 6 to 7 by @dependabot[bot] in https://github.com/MemPalace/mempalace/pull/2073
* ci: attach provenance and SBOM attestations to the published image by @kobihikri in https://github.com/MemPalace/mempalace/pull/2097
* docs: clarify hook write routing release scope by @Taylor-yuhb in https://github.com/MemPalace/mempalace/pull/2123
* docs: document KG supersede usage by @ggettert in https://github.com/MemPalace/mempalace/pull/2059
* docs: add Termux PRoot installation guide by @seoseo-ai in https://github.com/MemPalace/mempalace/pull/2139
* test: add knowledge_graph metadata coverage by @fatkobra in https://github.com/MemPalace/mempalace/pull/913
* test: add palace_graph ranking and limits coverage by @fatkobra in https://github.com/MemPalace/mempalace/pull/916
* test(benchmarks): enforce the 100ms startup injection budget by @arnoldwender in https://github.com/MemPalace/mempalace/pull/1140
* fix(benchmarks): UTF-8 encoding + ASCII print chrome on Windows for the 3 sibling runners by @jphein in https://github.com/MemPalace/mempalace/pull/1382
* fix: remove hardcoded fallback names from split_mega_files by @adv3nt3 in https://github.com/MemPalace/mempalace/pull/166
* revert: palace_graph ranking tests written against the pre-f68cb8d API (#916) by @igorls in https://github.com/MemPalace/mempalace/pull/2301
* fix(chroma): stop the client cache invalidating on its own writes by @igorls in https://github.com/MemPalace/mempalace/pull/2307
* fix(sqlite_exact): speed up query and status on large palaces by @igorls in https://github.com/MemPalace/mempalace/pull/2308
* fix(mesh): publish the estate so every transport can report it by @igorls in https://github.com/MemPalace/mempalace/pull/2309
* fix(sqlite_exact): cut remaining palace-wide read paths by @igorls in https://github.com/MemPalace/mempalace/pull/2311
* perf(mcp): make a proxied stdio session stop loading the storage stack by @igorls in https://github.com/MemPalace/mempalace/pull/2312
* fix(sqlite_exact): index wing/room/hall for structured access by @igorls in https://github.com/MemPalace/mempalace/pull/2313
* fix(chroma): read list_drawers and tunnels from sqlite, skip HNSW by @igorls in https://github.com/MemPalace/mempalace/pull/2314
* feat(logstream): background watcher agents can be woken by, plus the monitoring protocol by @igorls in https://github.com/MemPalace/mempalace/pull/2315
* fix(embedding): never let CoreML return NaN vectors for embeddinggemma by @marcoaperez in https://github.com/MemPalace/mempalace/pull/2283
* fix(repair): check the FTS5 content table before rebuilding from it (#2278) by @mvalentsev in https://github.com/MemPalace/mempalace/pull/2279
* chore(release): 3.8.0 by @igorls in https://github.com/MemPalace/mempalace/pull/2316
* fix(sync): require an established absence before removing a drawer (#2320) by @mvalentsev in https://github.com/MemPalace/mempalace/pull/2322

## New Contributors
* @kobihikri made their first contribution in https://github.com/MemPalace/mempalace/pull/2097
* @Taylor-yuhb made their first contribution in https://github.com/MemPalace/mempalace/pull/2123
* @seoseo-ai made their first contribution in https://github.com/MemPalace/mempalace/pull/2139
* @marcoaperez made their first contribution in https://github.com/MemPalace/mempalace/pull/2283

**Full Changelog**: https://github.com/MemPalace/mempalace/compare/v3.7.1...v3.8.0

**Install:** `pip install -U mempalace==3.8.0`