4.38

TrafficGuard/typedai4.38Jul 6, 2026by chrislusf

AI Summary

This release focuses on Erasure Coding (EC) scrubbing parity between Rust and Go volume servers, alongside improvements to storage topology, S3 API, and Kubernetes/Helm integrations.

Key Highlights

Comprehensive EC scrubbing fixes including deleted needle handling and bitrot checksum verification.
Rust volume server parity with Go for scrubbing, read paths, and master lookup parsing.
Topology and balancing improvements including empty disk reporting and physical disk usage gating.
S3 API enhancements such as bucket quotas, directory path probes, and paginated bucket listings.
Admin UI and Shell command updates including ec.shard.unmount and Traefik IngressRouteTCP support.

New Features

EC scrubbing modes (FULL and LOCAL) with distributed needle walks.
Bucket quota enforcement in Admin UI based on logical size.
Traefik IngressRouteTCP for gRPC with TLS passthrough.
ec.shard.unmount command with --delete option.
Lazy, bounded, entry-free per-bucket caches.

Full Release Notes

## What's Changed

Erasure Coding & Scrub

The largest theme — Rust volume-server scrub parity plus EC correctness.

EC scrub
- #10130 Fix scrubbing of deleted needles on EC volumes
- #10144 EC LOCAL needle walk (split from FULL)
- #10147 suppress deleted-needle size mismatch in EC LOCAL scrub
- #10149 EC FULL scrub — distributed local+remote needle walk
- #10152 correct EC FULL scrub for deleted needles, shard-location cache, parity coverage
- #10154 EC bitrot CHECKSUM scrub on the Rust volume server
- #10172 reject oversized bitrot payload before narrowing to u32
- #10176 expose force_deleted_needles_check in ScrubEcVolume RPC and shell

Volume scrub (Rust parity)
- #10141 correct backwards FULL/LOCAL mode comments in Rust
- #10142 align Rust INDEX scrub to Go's idx.CheckIndexFile
- #10143 walk the on-disk .idx in Volume::scrub
- #10148 don't flag offset-0 logical tombstones

EC storage / read paths
- #10140 honor wide EC ratios in Rust read_ec_shard_config
- #10145 search sibling disk locations when rebuilding missing EC shards + .ecx
- #10187 read chunk-manifest chunks stored on EC volumes (Rust)
- #10217 detect truncated .ecx instead of treating it as clean EOF

Rust volume server (non-scrub)

- #10128 parse master lookup when publicUrl is omitted
- #10139 test coverage for type=replicate fan-out writes
- #10189 compare live compaction_revision instead of stale last_compact_revision
- #10233 pin rustls to aws-lc-rs so TLS gRPC startup doesn't panic

Topology, balancing & placement

- #10161 keep physical disk 0 distinct in SplitByPhysicalDisk
- #10166 report empty disks (per-disk type + capacity in heartbeat)
- #10162 volume.balance: gate on real physical disk usage
- #10167 skip physically near-full disks when placing EC shards
- #10169 share replica-placement logic between shell and worker
- #10171 worker: project the moved volume when gating on disk fullness
- #10174 extract the bytes-aware density metric to a shared package

Master

- #10202 let the growth initiator wait instead of shedding itself
- #10226 repair the lookup index after a vacuum that raced a disconnect

Volume server (Go)

- #10185 drop stale volume-location cache on under-replication
- #10229 bound intra-cluster HTTP so an unresponsive peer can't hang reads/writes

S3 API

- #10156 invalidate stale reader-cache locations on chunk read failure
- #10186 return IncompleteBody instead of 500 for truncated PUT bodies
- #10192 embedded IAM inline policy honors prefix-scoped resources
- #10207 surface transient store errors during multipart resume/copy
- #10213 lazy, bounded, entry-free per-bucket caches
- #10214 paginate ListBuckets and serve from a bucket owner index
- #10224 enforce bucket quota on logical size + read-only state in Admin UI
- #10225 answer directory-path probes like AWS (Flink savepoint restore)
- #10236 FULL_OBJECT checksumming for CRC

Filer

- #10158 use DROP TABLE IF EXISTS in SQL stores
- #10190 avoid ReaderCache WaitGroup reuse race between reads and destroy
- #10196 applyStorageDefaultsToEntry before CreateEntry
- #10203 stream offloaded metadata-log entries to fix concurrent-write OOM
- #10232 default filemeta CREATE TABLE for postgres2/mysql2 when unset
- #10234 default cassandra2 timeout and ydb prefix for env-var configs

Admin UI

- #10150 remove non-functional EC repair button
- #10155 restore cluster volume page CSV export
- #10170 respect filerGroup for cluster discovery
- #10175 fix 'send on closed channel' panic in worker gRPC server
- #10227 browse Iceberg table data

Shell commands

- #10126 avoid duplicate volume.list parent headers
- #10129 collection pattern to delete empty volumes
- #10159 honor explicit fs.mergeVolumes from/to direction
- #10204 add ec.shard.unmount command
- #10219 pass collection in ec.shard.unmount --delete request

Remote storage & sync

- #10151 remote/gcs: tolerate already-deleted object in DeleteFile
- #10184 remote.meta.sync: sync directories and remove remote-deleted files
- #10228 add -delete option to remote.copy.local
- #10177 filer.sync.verify: reclassify chunk-slice-order ETag diffs as CHUNK_REORDER

Kubernetes / Helm

- #10198 add certificates.dnsNames for custom SANs in cert-manager certs
- #10205 filer HTTP + gRPC ingress for the Helm chart
- #10223 Traefik IngressRouteTCP for gRPC with TLS passthrough

KMS

- #10173 cache decrypted data keys so cache_enabled/cache_ttl take effect

Core / networking / util

- #10164 centralize genUploadUrl in UploadOption
- #10212 don't let the activity timeout clobber externally-set conn deadlines

Tests & CI

- #10208 fuse-tests: don't declare mount ready before it is mounted
- #10215 fuse-tests: pass glog flags before the mount subcommand
- #10209 keep mini-allocated ports below the ephemeral floor

## New Contributors
* @doma17 made their first contribution in https://github.com/seaweedfs/seaweedfs/pull/10155
* @ycjlin made their first contribution in https://github.com/seaweedfs/seaweedfs/pull/10228
* @cole-h made their first contribution in https://github.com/seaweedfs/seaweedfs/pull/10236

**Full Changelog**: https://github.com/seaweedfs/seaweedfs/compare/4.37...4.38