v0.8.0-beta

clockworklabs/SpacetimeDBv0.8.0-betaDec 9, 2023by cloutiertyler

AI Summary

A major release switching to the Wasmtime engine, adding multi-column indexes, and improving metrics and storage stability.

Key Highlights

  • Switched execution engine from Wasmer to Wasmtime.
  • Added multi-column index support.
  • New performance metrics tracking for modules on Testnet.
  • Stability and correctness improvements to persistent storage.
  • Significant performance improvements.

Breaking Changes

  • ABI has changed; modules must be rebuilt.
  • Switched execution engine from Wasmer to Wasmtime.

New Features

  • Multi-column indexes
  • Wasmtime engine integration
  • Performance metrics tracking

Full Release Notes

We're here with the next big release for SpacetimeDB, **v0.8.0**! 🚀🚀🚀

We've merged in more than 70 new PRs since v0.7.3 and we're super excited to share all the new stuff!

As a reminder you can upgrade from v0.7.3 with the `spacetime upgrade` command. Please note that our ABI has changed with this release and you will therefore need to rebuild your modules.

## Highlights

The most impactful changes in this release are:
- New performance metrics tracking for your modules on the Testnet!
- Stability and correctness improvements to the persistent storage.
- Multi-column index support
- Switched to Wasmtime execution engine
- Significant performance improvements

Currently much of our effort is focused around improving performance and stability with the goal of launching the Mainnet in April of next year!

## What's Changed
* core: Track the number of bytes read when iterating over the WAL by @kim in https://github.com/clockworklabs/SpacetimeDB/pull/496
* Created a route to check what emails are already connected to an identity by @NateTheDev1 in https://github.com/clockworklabs/SpacetimeDB/pull/477
* Capture logs in integration tests by @RReverser in https://github.com/clockworklabs/SpacetimeDB/pull/534
* Update the pull request template by @kulakowski in https://github.com/clockworklabs/SpacetimeDB/pull/541
* Downgrading some logs to trace instead of info by @jdetter in https://github.com/clockworklabs/SpacetimeDB/pull/542
* core: Add truncation to message log by @kim in https://github.com/clockworklabs/SpacetimeDB/pull/526
* Misc refactoring preparing for multi col indices by @Centril in https://github.com/clockworklabs/SpacetimeDB/pull/497
* Moving object defs to SATS by @mamcx in https://github.com/clockworklabs/SpacetimeDB/pull/460
* Add integration test for the query macro by @mamcx in https://github.com/clockworklabs/SpacetimeDB/pull/538
* Desparse Tar Files When Building by @jdetter in https://github.com/clockworklabs/SpacetimeDB/pull/544
* delete_by_col_eq: optimize with smallvec for 1 row case by @Centril in https://github.com/clockworklabs/SpacetimeDB/pull/547
* Disable latest docker tag by @drogus in https://github.com/clockworklabs/SpacetimeDB/pull/546
* Fixes client API breaks introduced by #460 by @cloutiertyler in https://github.com/clockworklabs/SpacetimeDB/pull/558
* Nix `impl TryFrom<&str> for IndexType` by @Centril in https://github.com/clockworklabs/SpacetimeDB/pull/559
* core: Replay database prefix by @kim in https://github.com/clockworklabs/SpacetimeDB/pull/527
* Refactoring IDs for system objects by @mamcx in https://github.com/clockworklabs/SpacetimeDB/pull/439
* Move 'constraints' markers to primitive and fix incompatibility with deploy by @mamcx in https://github.com/clockworklabs/SpacetimeDB/pull/561
* Fix out of order subscription updates by @coolreader18 in https://github.com/clockworklabs/SpacetimeDB/pull/556
* perf(553): Optimize incremental join evaluation by @joshua-spacetime in https://github.com/clockworklabs/SpacetimeDB/pull/557
* Add the `slim_slice` data-structures by @Centril in https://github.com/clockworklabs/SpacetimeDB/pull/528
* `system_tables`: fix `st_indexes_schema` inconsistency by @Centril in https://github.com/clockworklabs/SpacetimeDB/pull/569
* Simplify contributor check by @coolreader18 in https://github.com/clockworklabs/SpacetimeDB/pull/565
* core: Refine panic message when decoding fails during replay by @kim in https://github.com/clockworklabs/SpacetimeDB/pull/568
* Switch from wasmer to wasmtime by @coolreader18 in https://github.com/clockworklabs/SpacetimeDB/pull/457
* core: Make `Segment` pub and serde by @kim in https://github.com/clockworklabs/SpacetimeDB/pull/574
* core: Move replay + error correction logic to commit log by @kim in https://github.com/clockworklabs/SpacetimeDB/pull/554
* refactor(571): Track module queue length with histogram by @joshua-spacetime in https://github.com/clockworklabs/SpacetimeDB/pull/576
* test: Create indexes for restarting tests by @kim in https://github.com/clockworklabs/SpacetimeDB/pull/573
* core: Only write actual large objects to odb by @kim in https://github.com/clockworklabs/SpacetimeDB/pull/575
* perf(572): Track the number of reducer panics by @joshua-spacetime in https://github.com/clockworklabs/SpacetimeDB/pull/578
* Update to my main NuGet author handle by @RReverser in https://github.com/clockworklabs/SpacetimeDB/pull/585
* Error for no abi version by @coolreader18 in https://github.com/clockworklabs/SpacetimeDB/pull/580
* perf: Track maximum values for transaction cpu time and reducer queue length by @joshua-spacetime in https://github.com/clockworklabs/SpacetimeDB/pull/583
* Update to rust 1.74 by @coolreader18 in https://github.com/clockworklabs/SpacetimeDB/pull/589
* Add spacetimedb::sql::query_debug_info::QueryDebugInfo by @kazimuth in https://github.com/clockworklabs/SpacetimeDB/pull/560
* Rename HostType::Wasmtime to just Wasm by @coolreader18 in https://github.com/clockworklabs/SpacetimeDB/pull/592
* chore: Add functions for resetting max value metrics by @joshua-spacetime in https://github.com/clockworklabs/SpacetimeDB/pull/600
* perf(591): Distinguish query metrics as sql, subscription or incremental update by @joshua-spacetime in https://github.com/clockworklabs/SpacetimeDB/pull/605
* Hotfix for log replay bug, fix type for constraints column by @mamcx in https://github.com/clockworklabs/SpacetimeDB/pull/604
* perf(590): Label query metrics with query text by @joshua-spacetime in https://github.com/clockworklabs/SpacetimeDB/pull/606
* core: Load large objects eagerly during log replay by @kim in https://github.com/clockworklabs/SpacetimeDB/pull/594
* perf: Record execution and compilation time metrics per query by @joshua-spacetime in https://github.com/clockworklabs/SpacetimeDB/pull/609
* perf(610): More granular histogram buckets for timing metrics by @joshua-spacetime in https://github.com/clockworklabs/SpacetimeDB/pull/613
* Format binary data as hex in sql/satn by @mamcx in https://github.com/clockworklabs/SpacetimeDB/pull/521
* fix: Remove newlines from QueryDebugInfo by @joshua-spacetime in https://github.com/clockworklabs/SpacetimeDB/pull/616
* fix(CommitLog): Do not truncate the log on out of order errors by @joshua-spacetime in https://github.com/clockworklabs/SpacetimeDB/pull/588
* Update wasmtime (fixes fuel tracking) by @coolreader18 in https://github.com/clockworklabs/SpacetimeDB/pull/598
* Move to .NET 8 WASI support by @RReverser in https://github.com/clockworklabs/SpacetimeDB/pull/587
* core: Make `advertise_addr` of `Node` optional by @kim in https://github.com/clockworklabs/SpacetimeDB/pull/618
* refactor: Remove global disk and memory tracking by @joshua-spacetime in https://github.com/clockworklabs/SpacetimeDB/pull/620
* Allow to see the context of a Unsupported query for a subscription by @mamcx in https://github.com/clockworklabs/SpacetimeDB/pull/621
* refactor(metrics): Remove dependencies of metric builder macros by @joshua-spacetime in https://github.com/clockworklabs/SpacetimeDB/pull/624
* Refactoring bootstrapping and adding full support for constraints & multi-column indexes by @mamcx in https://github.com/clockworklabs/SpacetimeDB/pull/596
* Version upgrade to 0.8.0 by @jdetter in https://github.com/clockworklabs/SpacetimeDB/pull/622
* Support arbitrary RHS in C# Query by @RReverser in https://github.com/clockworklabs/SpacetimeDB/pull/586
* fix: Temporarily disable flaky smoketests by @joshua-spacetime in https://github.com/clockworklabs/SpacetimeDB/pull/629
* C#: don't show "Extracting..." message if already unpacked by @RReverser in https://github.com/clockworklabs/SpacetimeDB/pull/628
* Spacetime Upgrade Force Flag by @jdetter in https://github.com/clockworklabs/SpacetimeDB/pull/631
* refactor(metrics): Move metric builder macros into their own crate by @joshua-spacetime in https://github.com/clockworklabs/SpacetimeDB/pull/625
* Try to fsync message log on drop by @kim in https://github.com/clockworklabs/SpacetimeDB/pull/627
* Added RWLock on committed state by @Shubham8287 in https://github.com/clockworklabs/SpacetimeDB/pull/599
* Various fixes for C# column attrs  by @RReverser in https://github.com/clockworklabs/SpacetimeDB/pull/635
* ci: Tame the doge by @kim in https://github.com/clockworklabs/SpacetimeDB/pull/632
* chore: Update tracy to 0.10.4 by @joshua-spacetime in https://github.com/clockworklabs/SpacetimeDB/pull/644
* core: Log db open progress at INFO by @kim in https://github.com/clockworklabs/SpacetimeDB/pull/646
* refactor(metrics): Move table size metric into lib by @joshua-spacetime in https://github.com/clockworklabs/SpacetimeDB/pull/636
* chore(vm): Pass database address to the compiler by @joshua-spacetime in https://github.com/clockworklabs/SpacetimeDB/pull/637
* Cache the 'row_type' on TableSchema so is not re-build in each access by @mamcx in https://github.com/clockworklabs/SpacetimeDB/pull/638
* Fixes for performance drop related to missing index selection by @mamcx in https://github.com/clockworklabs/SpacetimeDB/pull/647
* Add timestamp to module logs by @coolreader18 in https://github.com/clockworklabs/SpacetimeDB/pull/641
* [CI] Merge docker images from multiple platforms by @drogus in https://github.com/clockworklabs/SpacetimeDB/pull/640
* Improve TypeScript generated code (fixes and improvements) by @drogus in https://github.com/clockworklabs/SpacetimeDB/pull/603

## New Contributors
* @NateTheDev1 made their first contribution in https://github.com/clockworklabs/SpacetimeDB/pull/477
* @Shubham8287 made their first contribution in https://github.com/clockworklabs/SpacetimeDB/pull/599

**Full Changelog**: https://github.com/clockworklabs/SpacetimeDB/compare/v0.7.3-beta-hotfix1...v0.8.0-beta