v1.0.0-beta1

juicedata/juicefsv1.0.0-beta1Jan 13, 2022by davies

AI Summary

The first beta release for v1.0.0, introducing critical features like metadata backups, trash bin, and configuration management commands.

Key Highlights

  • Automatic hourly metadata backups to object storage
  • Trash bin feature for removed or overwritten files
  • Added `config` command to update volume configuration
  • Added `destroy` command to clean up data and metadata
  • Data integrity verification with `--check-all` flag

New Features

  • Metadata backup to object storage
  • Trash bin management
  • Configuration management command
  • Data destruction command
  • S3 Gateway etag preservation
  • Anonymous access to S3 in sync command

Full Release Notes

JuiceFS v1.0.0-beta1 is the first beta release for v1.0, arrived three months after v0.17. It has 300+ commits from 22 contributors, thanks to @SandyXSD @davies @xiaogaozi @yuhr123 @zhijian-pro @sanwan @zwwhdls @tangyoupeng @Suave @chiyutianyi @201341 @suzaku @reusee @tisonkun @chenjie4255 @dragonly @nature1995 @fredchen2022 @Shoothzj @nsampre @supermario1990 @sjp00556 !

**JuiceFS v1.0.0-beta1 is released under the Apache License 2.0.**

# New

* Backups the whole metadata as a compressed JSON file into object storage in every hour by default, so we can get most of the data back in case of losing the entire meta database. (#975)
* Added trash bin: removed or overwritten file will be moved to trash bin, will be deleted after configured days. (#1066)
* Added `config` command to update configuration of an existing volume (#1137)
* Added `destroy` command to clean up all data & metadata of a volume (#1164)
* Added a option to limit the concurrent deletes (#917)
* Added a option to register prometheus metrics api to consul (#910)
* Added a option to S3 gateway to keep the etag information of uploaded objects (#1154)
* Added `--check-all` and `--check-new` to verify data integrity (#1208)
* `sync` command supports anonymous access to S3 (#1228)

# Changed

* meta: don't return EINVAL when encountering unknown flags (#862) 
* cmd/bench: remove default benchmark PATH (#864)
* hadoop: sort list res to compatible with hdfs (#889)
* expose metrics in S3 gateway (#897)
* cleanup broken mountpoint (#912)
* cmd/info: add '--recursive' flag to get summary of a tree (#935)
* cache: force sync upload if local disk is too full for writeback (#943)
* Speed up metadata dump&load for redis (#954)
* fsck: list broken files (#958)
* write to json file by stream (#970) 
* speedup listing on file (#622)
* meta: unify transaction backoff for redis/tkv (#999)
* speed up metadata dumping and loading for Tikv V2 (#998)
* check AK and SK for gateway (#1001)
* add a option to provide customized bucket endpoint (#1008) 
* change frsize to 4096 (#1016)
* Speed up dump for sql engine (#1006)
* format support provide only bucket name (#1022) 
* meta/tkv: fix clean stale session (#1041)
* add namespace and label to existing metrics (#1056)
* release memory after dumping meta (#1093)
* meta: unify counters for all metadata engines (#1119)
* Keep SGID for directory (#1133)
* Speedup hadoop concat by delete src concurrently (#1163)
* change default cache-size to 100GB (#1169)
* metrics: expose only JuiceFS metrics to prometheus (#1185)
* cleanup reference of unused slice in gc command (#1249) 
* Adjust log level for xorm and TiKV when they are in use (#1229)
* hadoop: Users in supergroup are superusers (#1202) 
* Check permission with multiple groups (#1205)
* Ret the region of the object storage compatible with the s3 protocol from the env (#1171)
* Added prefix to metrics in gateway (#1189)
* improve s3 url parsing (#1234)

# Bugfix

* go-fuse: return ENOSYS if xattr is disabled (#863)
* Fix WebDAV backend (#899)
* meta: always update cache after got newest attributes (#902)
* sql.go: fix delete file range error (#904)
* meta: update parent when rename (#905)
* sqlmeta: fix delete chunk sql error (#931) 
* Fix the bug that file credential authentication fails when using GCS (#933)
* hadoop: compatible with kitedata in sqoop parquet use case (#941)
* Fix redis-sentinel addresses parsing (#953) 
* utils: fix cond and its test (#983)
* workaround for List with marker (#984)
* sync: ignore broken symlink (#1015)
* meta/sql: fix delete condition (#1018)
* metadata: should not skip error if open a file not exist (#1035)
* Fix minio with sync (#1055)
* meta: remove xattr only when nlink <= 0 (#1078)
* meta/sql: fix parentDst nlink in rename (#1082)
* Fix upload bandwidth limit (#1086)
* Fix lua script handling big inode number (#1095) 
* meta: fix lookup of corrupt entries (#1098)
* Fix potential metadata corrupt in Redis caused by gc (#1110)
* Fix fmtKey to sliceKey (#1143)
* Fix mkdir/rename/delete/rmr with trailing slash (#1149)
* Fix gateway GetObjectInfo http code 500 to 404 (#1158)
* meta: fix nlink of parentDst in rename (#1170)
* Sync: distinguish DeleteSrc from DeleteDSt (#1197)
* Fix subdir in S3 gateway (#1201)