v1.0.0-beta3
juicedata/juicefsv1.0.0-beta3May 5, 2022by SandyXSD
AI Summary
The third beta release for v1.0.0, introducing etcd support, Redis Cluster compatibility, and significant changes to session management and trash handling.
Key Highlights
- Supports etcd as a new metadata engine
- Supports Redis Cluster and compatible services (Amazon MemoryDB)
- New session management format (old clients cannot detect sessions)
- Compacted slices are kept for `trash-days` if trash is enabled
- Improved `juicefs sync` command
Breaking Changes
- Old clients are unable to detect sessions with version 1.0.0-beta3 or higher via `juicefs status` or `juicefs destroy` command
New Features
- etcd metadata engine support
- Redis Cluster support
- Non-UTF8 file name handling for SQL engines
- TLS support for TiKV and object storage
- Oracle Object Storage support
- Umask setting for S3 Gateway
- Graphite metrics for Java SDK
- Hashed prefix for objects
- Customized storage type option
Full Release Notes
JuiceFS v1.0.0-beta3 is the third beta release for v1.0. It has 247 commits from 22 contributors, thanks to @SandyXSD @zhoucheng361 @davies @zhijian-pro @yuhr123 @sanwan @AIXjing @rayw000 @xiaogaozi @Suave @showjason @tangyoupeng @201341 @solracsf @guo-sj @chnliyong @DeanThompson @zwwhdls @wph95 @lidaohang @sjp00556 @DEvmIb ! # Highlights - Supports [etcd](https://etcd.io/) as a new metadata engine. It can be a handy choice when you only need a small volume but cares more about the data availability and persistence. - Supports Redis Cluster and other compatible services (Amazon MemoryDB for Redis) as metadata engines. - When using SQL metadata engines, file names not encoded by UTF-8 can now be properly handled after manual modification to the table schema, see [details](https://juicefs.com/docs/community/release_notes/#upgrade-to-juicefs-v100-beta3). - A new session management format is introduced. Old clients are unable to detect sessions with version 1.0.0-beta3 or higher via `juicefs status` or `juicefs destroy` command, see [details](https://juicefs.com/docs/community/release_notes/#upgrade-to-juicefs-v100-beta3). - If trash is enabled, compacted slices are kept as well in case they are needed to recover file data. These slices will be cleaned up automatically after `trash-days`, and can be deleted manually via `juicefs gc` command. - A lot of improvements have been made to `juicefs sync` command. - A lot of protection checks against unintentional misuse have been added. # New - Supports etcd as metadata engine (#1638) - Supports Redis in cluster mode while using one slot for each file system (#1696) - Supports handling file names not encoded by UTF-8 for SQL metadata engines (#1762) - Supports TLS when using TiKV as metadata engine or object storage (#1653, #1778) - Supports Oracle Object Storage as data storage (#1516) - Supports setting umask for S3 Gateway (#1537) - Java SDK now supports pushing metrics to Graphite (#1586) - Added a new option `--heartbeat` for the `mount` command to adjust heartbeat interval (#1591, #1865) - Added many improvements for `sync` command to make it more handy (#1554, #1619, #1651, #1836, #1897, #1901) - Added a new option `--hash-prefix` for the `format` command to add a hashed prefix for objects (#1657) - Added a new client option `--storage` to allow customized storage type (#1912) # Changed - compacted slices will be kept for `trash-days` if trash is enabled (#1790) - cmd: support using integer for duration flags (#1796) - cmd: use homedir as default working directory for non-root users (#1869) - cmd/format: create a uuid object in the target bucket (#1548) - cmd/dump&load: dump load behavior supports non-ascii characters (#1691) - cmd/dump: omit empty value in dumped JSON (#1676) - cmd/dump: remove secret key (#1569) - meta: encrypt the secret-key and encrypt-key in setting (#1562) - meta: create subdir automatically (#1712) - meta: specify the format field preventing update (#1776) - meta: escape meta password from env (#1879) - meta/redis: check redis version (#1584) - meta/redis: use smaller retry backoff in sentinel mode (#1620) - meta/redis: retry transaction for connection error or EXECABORT (#1637) - meta/sql: retry transaction after too many connections (#1876) - meta/sql: add primary key for all tables (#1913, #1919) - meta&chunk: Set max retries of meta & chunk according to the config `io-retries` (#1713, #1800) - chunk: limit number of upload goroutines (#1625) - chunk/store: limit max retry for async upload as well (#1673) - object/obs: Verify Etag from OBS (#1715) - object/redis: implement listAll api for redis (#1777) - fuse: automatically add `ro` option if mount with `--read-only` (#1661) - vfs/backup: reduce the limit for skipping backup (#1659) - sync: reduce memory allocation when write into files (#1644) - SDK: use uint32 for uid,gid (#1648) - SDK: handle removeXAttr return code (#1775) - Dependency: switch to Go 1.17 (#1594) - Dependency: fix jwt replace (#1534) - Dependency: upgrade golang-cross version to v1.17.8 (#1539) - Dependency: upgrade tikv to v2.0.0 (#1643) - Dependency: reduce dep from minio (#1645) - Dependency: upgrade gjson to 1.9.3 (#1647) - Dependency: upgrade sdk for object storage (#1665) - Dependency: upgrade qiniu sdk (#1697) # Bugfix - cmd/format: fix setting quota (#1658) - cmd/mount: fix parsing of cache dir (#1758) - cmd/warmup: fix handling of relative paths (#1735) - cmd/sync: fix sync command not working when destination is webdav (#1636) - cmd/gateway: fix s3 gateway DeleteObjects panic (#1527) - meta: forbid empty name for dentry (#1687) - meta: lock counters when loading entries (#1703) - meta: fix snap not released if error occurs when dumping meta (#1669) - meta: don't update parent attribute if it's a trash directory (#1580) - meta/redis: fix loading large directory into Redis (#1858) - meta/redis: update used space/inodes in memory whenever needed (#1573) - meta/sql: use upsert to update xattr for PG (#1825) - meta/sql: split insert batch (#1831) - meta/sql: fix wrong result from scanning SQL database (#1854) - chunk/cache: Fix read disk_cache not existed remove cache key (#1677) - object: fallback to List only if ListAll is not supported (#1623) - object/b2: check returned bucket from B2 (#1745) - object/encrypt: fix parse rsa from pem (#1724) - object/encrypt: Add JFS_RSA_PASSPHRASE environment variable prompt information (#1706) - object/sharding: fix ListAll returning invalid objects (#1616) - object/ceph: fix listAll hangs if there are many objects (#1891) - vfs: write control file asynchronously (#1747) - vfs: fix getlk in access log (#1788) - sync: Fix copied and copiedBytes (#1801) - utils: fix the problem that the progress bar loses the log (#1756) - SDK: rename libjfs atomic (#1939)