v0.8.2-beta-hotfix7

clockworklabs/SpacetimeDBv0.8.2-beta-hotfix7Feb 15, 2024by bfops

AI Summary

A performance-focused release involving memory architecture refactoring and parallel evaluation of subscription queries.

Key Highlights

  • Memory architecture refactor reduces overhead when reading values.
  • Incremental subscription queries are now evaluated in parallel.
  • Improved join evaluation performance.
  • Lookups on indexed columns are faster.

New Features

  • Parallel subscription query evaluation
  • Memory architecture improvements

Full Release Notes

Our release 0.8.2 is available now! It includes tons of performance improvements and some great bugfixes!

As a reminder you can upgrade from the previous release with the `spacetime upgrade` command. If you are on macOS and you've installed through brew we recommend using `brew install clockworklabs/tap/spacetime` to upgrade.

## Highlights
- **Memory architecture refactor**: we're working on rearchitecting the way we store and retrieve data from the database. This is our first batch of improvements which reduces the amount of overhead when reading values from the database. This affects both subscription query performance and module performance.
- Tons of performance improvements
  - Incremental subscription queries are now evaluated in parallel
  - Join evaluation performance improvements
  - Lookups on an indexed column are faster
- Fixed `spacetime generate --delete-files` works properly without using `--force`
- Bug fixes
  - Fixed correctness issues with subscription queries
  - Fixed some issues related to unwrapping on None
 
## What's Changed
* core: Sort constraint definitions before comparing by @kim in https://github.com/clockworklabs/SpacetimeDB/pull/727
* Allows to convert into/from micros_since_epoch:u64 for TimeStamp by @mamcx in https://github.com/clockworklabs/SpacetimeDB/pull/731
* Fix callgrind action again by @kazimuth in https://github.com/clockworklabs/SpacetimeDB/pull/718
* Fix local callgrind runs by @kazimuth in https://github.com/clockworklabs/SpacetimeDB/pull/723
* Expand callgrind benches by @kazimuth in https://github.com/clockworklabs/SpacetimeDB/pull/728
* Version upgrade to 0.8.1 by @jdetter in https://github.com/clockworklabs/SpacetimeDB/pull/729
* Fix _delete_by_rel by @coolreader18 in https://github.com/clockworklabs/SpacetimeDB/pull/733
* Add `ColList`: a compact replacement of `NonEmpty<ColId>`, 8 vs 32 bytes by @Centril in https://github.com/clockworklabs/SpacetimeDB/pull/730
* feat: subscription to use read type tx by @Shubham8287 in https://github.com/clockworklabs/SpacetimeDB/pull/685
* Fix --delete-files command without using --force by @jdetter in https://github.com/clockworklabs/SpacetimeDB/pull/739
* Use `ColList` more and improve it by @Centril in https://github.com/clockworklabs/SpacetimeDB/pull/738
* Fix for reducer queue metric by @jdetter in https://github.com/clockworklabs/SpacetimeDB/pull/737
* Fix row count for the 'sql' command by @mamcx in https://github.com/clockworklabs/SpacetimeDB/pull/745
* Cleanup SpacetimeDB name usages by @jdetter in https://github.com/clockworklabs/SpacetimeDB/pull/744
* SATS: add support needed for BFLATN => BSATN directly by @Centril in https://github.com/clockworklabs/SpacetimeDB/pull/750
* Check benchmarks on CI + disable callgrind benchmarks on non-Linux by @kazimuth in https://github.com/clockworklabs/SpacetimeDB/pull/752
* Fix some bugs in `concat_byte_chunks`, `serialize_bsatn` by @Centril in https://github.com/clockworklabs/SpacetimeDB/pull/755
* made subscription broadcast multithreaded by @Shubham8287 in https://github.com/clockworklabs/SpacetimeDB/pull/741
* Split the old `locking_tx_datastore` by @Centril in https://github.com/clockworklabs/SpacetimeDB/pull/757
* Fixed broken quickstart links in readme by @jdetter in https://github.com/clockworklabs/SpacetimeDB/pull/761
* Fix more broken links by @jdetter in https://github.com/clockworklabs/SpacetimeDB/pull/762
* Add the `spacetimedb_table` crate by @Centril in https://github.com/clockworklabs/SpacetimeDB/pull/763
* perf(759): Track the number of active subscription queries by @joshua-spacetime in https://github.com/clockworklabs/SpacetimeDB/pull/760
* spacetimedb_table: nix spacetimedb-core dep + build_unique_error fixes by @Centril in https://github.com/clockworklabs/SpacetimeDB/pull/767
* feat(748): Add compile time feature flag for table metrics by @joshua-spacetime in https://github.com/clockworklabs/SpacetimeDB/pull/749
* Fixed issue with perfcnt not working on certain platforms and architectures by @cloutiertyler in https://github.com/clockworklabs/SpacetimeDB/pull/768
* Split old datastore more by @Centril in https://github.com/clockworklabs/SpacetimeDB/pull/770
* pointers_yielded -> num_pointers_yielded by @Centril in https://github.com/clockworklabs/SpacetimeDB/pull/773
* core: Lock `unwritten_commit` across I/O by @kim in https://github.com/clockworklabs/SpacetimeDB/pull/775
* Use rayon for broadcast_commit_event by @coolreader18 in https://github.com/clockworklabs/SpacetimeDB/pull/779
* Integrate mem arch into `locking_tx_datastore` by @Centril in https://github.com/clockworklabs/SpacetimeDB/pull/769
* reducer to wait until subsciption broadcast finishes by @Shubham8287 in https://github.com/clockworklabs/SpacetimeDB/pull/776
* Handle returned status codes instead of Rust values by @bfops in https://github.com/clockworklabs/SpacetimeDB/pull/783
* fix: Put query metrics behind feature flag by @joshua-spacetime in https://github.com/clockworklabs/SpacetimeDB/pull/784
* feat(query): Make current plan type hashable by @joshua-spacetime in https://github.com/clockworklabs/SpacetimeDB/pull/782
* Fix assorted typos by @bfops in https://github.com/clockworklabs/SpacetimeDB/pull/785
* `trait ReadColumn`, to read a single column from a `RowRef` by @gefjon in https://github.com/clockworklabs/SpacetimeDB/pull/789
* feat(query): Add a utility for checking table refs by @joshua-spacetime in https://github.com/clockworklabs/SpacetimeDB/pull/790
* BTreeIndex: Specialize on primitive key types for great performance by @gefjon in https://github.com/clockworklabs/SpacetimeDB/pull/793
* Unify benchmark runs (to some extent) by @kazimuth in https://github.com/clockworklabs/SpacetimeDB/pull/742
* Fix private not building with latest master by @bfops in https://github.com/clockworklabs/SpacetimeDB/pull/798
* BTreeIndex: use `ReadColumn` instead of `ProductValue` fields by @gefjon in https://github.com/clockworklabs/SpacetimeDB/pull/796
* Remove separate "callgrind please" script by @kazimuth in https://github.com/clockworklabs/SpacetimeDB/pull/800
* 1_000_000 insert + update benchmarks by @kazimuth in https://github.com/clockworklabs/SpacetimeDB/pull/764
* Add rust-version to Cargo.toml by @coolreader18 in https://github.com/clockworklabs/SpacetimeDB/pull/802
* Simplify broadcast_commit_event by @coolreader18 in https://github.com/clockworklabs/SpacetimeDB/pull/791
* Fix build errors in spacetimedb_table benches by @coolreader18 in https://github.com/clockworklabs/SpacetimeDB/pull/787
* perf(query): Moves Code::Block instead of cloning during eval by @joshua-spacetime in https://github.com/clockworklabs/SpacetimeDB/pull/805

**Full Changelog**: https://github.com/clockworklabs/SpacetimeDB/compare/v0.8.1-beta...v0.8.2-beta-hotfix7