v1.3.0

This is the fourth major release of JuiceFS, and is an LTS version as well that will be maintained for 24 months.
Since v1.2.0 it has 696 commits from 60 contributors, thanks to
@zhijian-pro @jiefenghuang @polyrabbit @zhoucheng361 @davies @chenjie4255 @YunhuiChen @tangyoupeng @CodingPoeta @CaitinChen @Xuyuchao-juice @SandyXSD @anysql @xiaogaozi @yuhr123 @zxh326 @kyungwan-nam @Suave @timfeirg @zwwhdls @wenjianhn @winglq @cxljs @dsd2077 @ddrid @daimashusheng @SonglinLife @williamstein @mt5225 @lidaohang @JoyLiuc @Looka149 @YinhaoHu @justforlxz @yetone @wpleonardo @taadis @shenxingwuying @prOOrc @myron369 @menwenjun @hawkli-1994 @bitsark @chnliyong @austinhmh @wzhallright @201341 @servusdei2018 @mrdrivingduck @maxjustus @lianneli @davidzhangbj @sjp00556 @jerrycaimin @aucusaga @solracsf @fuyufjh @veinkr @DASWU @achetronic !

# Highlights

- Python SDK (#5437,#5516,#5573,#5779,#5852,#5847,#5853,#5931,#5998...)
- Improved tools and compatibility on Windows (#5376,#5630,#5629,#5643,#5661,#5714,#5724...)
- Speedup backup using protobuf (#5344,#5348,#5352,#5356,#5733,#5379,#5383,#5404...)
- Support Apache Ranger (#5285,#5430,#6016)

# New

- fuse: add `--async_dio` option to enable asynchronous handling of direct I/O (#4943,#4986)
- fuse: add `--max-fuse-io` option to config max write size for request (#5087,#6017)
- fuse: add timeout for FUSE requests (#5670,#5681,#5690)
- fuse: enable kernel symlink cache (#5766)
- fuse: support /dev/fd/N mount point for `--fusemount` (#5076,#5082,#5976)
- cmd/mount: add `max-readahead` option to set the max readahead buffer (#5225,#5229,#5704)
- cmd/mount: add `--sort-dir` option to sort entries by name within a directory (#5031)
- cmd/mount: add `readdir-cache` option to enable kernel caching readdir entries (#5462)
- cmd/mount: add `negative-entry-cache` option to allow kernel caching negative dir entry lookups (#5788,#6038)
- cmd/mount: add `cache-large-write` option to cache blocks after uploading (#5261)
- cmd/mount: add `cache-items` option to limit number of cache items (#5489,#5495)
- cmd/mount: add `check-storage` option to perform object storage test at startup (#5281)
- cmd/mount: add `enable-cap` and `enable-selinux` options to enable corresponding xattr requests (#5406,#5962)
- cmd/mount: add `all-squash` option to map all users to a specified one (#5395)
- cmd/mount: add `umask` option to overwrite the one from app (#5756,#5942)
- cmd/mount: add `disable-transparent-hugepage` option to disable transparent huge page (#6147)
- cmd/mount: supervisor process pass signal to service process (#5238,#5245,#5246,#5278,#5624)
- vfs: support `O_TMPFILE` flag (#5309,#5433)
- meta: support readdir streaming (#5162,#5207,#5223,#5242,#5482,#5504,#5895)
- meta: add `fast-statfs` option to speed up statfs (#6168,#6171)
- meta/tikv: support `open-tso-follower-proxy` and `max-tso-batch-wait-interval` options (#6166)
- meta/sql: support multiple volumes sharing a single db (#5791,#5804)
- cache: add env `JFS_DROP_OSCACHE` to enable dropping page cache (#4981)
- cmd/format: support TLS configuration via bucket URL parameters (#5241)
- cmd/sync: add `max-failure` option to abort on error threshold (#5142)
- cmd/sync: add `start-time` and `end-time` options to filter file (#6072,#6121)
- cmd/sync: add `check-change` option to verify file size (#6093)
- cmd/quota: add `--create` option to create directory if not exists (#5196)
- cmd/rmr: add `skip-trash` option to delete files directly (#5384,#5818)
- cmd: add `threads` option for rmr tool, webdav, gateway and libjfs to delete files (#5522,#5535)
- cmd/gateway: add `object-meta` option to enable object metadata api (#5536)
- cmd/sync: add `files-from` option to specify the list of files or directories to sync (#5639,#5864,#5894)
- cmd/objbench: add `shards` to support object storage shards (#5755)
- cmd/dump&load: support zstd compression (#4921)
- cmd/webdav: support PROPPATCH method (#5041)
- sdk/hadoop: support subdir option in hadoop sdk (#6097,#6141)

# Changed

- vfs: limit buffer usage for read requests (#5557)
- meta: improve statfs performance (#5161)
- meta/kv: improve mknod performance via batch get (#5056)
- meta: improve performance via inode prefetching. (#5130,#5481)
- meta: reduce `GetAttr` calls (#5540)
- meta: optimizations for reducing transaction conflicts (#5308,#5330,#5574)
- meta: adjust the interval to one hour for deleted files cleanup (#5343)
- meta: cancel background tasks when closing session (#5486,#5613,#5943,#5986,#6108,#6060)
- meta/kv: improve scan performance and robustness (#5298,#5314,#5080)
- meta/kv: simplify transactions in point get scenarios to reduce one RPC call for TiKV (#6136,#6150)
- meta/sql: improve performance by simplifying transaction and reducing locks (#5377,#5460)
- meta/sql: set default transaction isolation level to repeatable-read (#5432,#5456)
- meta/sql: support more connection options (#5512,#5806)
- meta/sql: improve compatibility by updating exact required fields (#6153,#6158)
- meta/redis: support tls server name configuration (#5182)
- cache: inherit atime and track logic size when re-scan (#5692)
- cache: adjust cache capacity and max items by freeratio (#5970)
- cache: change the default value of `verify-cache-checksum` option to `extend` (#6103)
- cmd: exit compact timely and log a message when long-running background tasks start (#6033)
- cmd: refuse more than expected positional arguments (#6154)
- cmd/mount: the `upload-hours` option allow using `-` as a separator (#5494)
- cmd/mount: set the process as an `IO_FLUSHER` to avoid potential hang (#6144,#6157)
- cmd/gateway: add one more hierarchy to temporary multipart dir to reduce transaction conflicts (#5549)
- cmd/gateway: improve the gateway performance in read-only mode (#6187)
- cmd/gateway: some performance optimizations (#5163,#5548,#5544)
- cmd/gateway: delete intermediate temporary files promptly to reduce inode usage (#6080)
- cmd/gateway: implement ListObjectVersions (#6173)
- cmd/sync: support more environment variables in cluster mode (#5364,#5422)
- cmd/sync: improve performance by calculating checksum in flight to reduce redundant reads from the source (#5524)
- cmd/warmup: accelerate big file warmup (#5713,#5886,#5897)
- cmd/warmup: improve performance by reducing file operations (#5408)
- cmd/warmup: show cache location info (#5559)
- cmd/info: add column for absolute file offset (#6037)
- cmd/fsck: support recalculating used space and inodes (#5331,#6036)
- cmd/restore: output restored dirs and stats (#5380)
- cmd/gc: use `MaxDeletes` as remove concurrency (#5532)
- cmd/gc: improve scanning performance for pending-deletion files (#5683)
- cmd/backup: use multipart upload to transfer large backup files (#5564)
- cmd/stats: add timestamp info and `count` option (#5528)
- cmd/debug: collect more detailed information about goroutines (#5656)
- object: upgrade aws sdk to v2 (#5968,#5987,#5992)
- object: remove object storage support for scs, jss, upyun (#5972, #5975, #5974)
- object: improve list performance via v2 API (#5250,#5393,#5434)
- object: display full address of s3 compatible object storage in log (#6151)
- object: disable s3 sdk retries since JuiceFS has retry mechanism (#5644,#6113)
- object/gs: improve gs performance by using client pool (#5020)
- object/nfs: improve `Readdir` performance by increasing batch size (#5689)
- object/minio: support `REDUCED_REDUNDANCY` storage class for MinIO (#5413)
- object/sftp: supports verification of known hosts (#5581)
- object/bos: reduce memory allocation in bos (#5657)
- object/bos: support crc32 check (#5883)
- object/oss: support parsing regionId from environment variable and from VPC-type endpoint (#6202)
- hadoop: support skipNBytes (#5028)
- metrics: update grafana dashboard (#4980,#5248)
- metrics: add method label for transaction conflicts (#5762)
- metrics: add `total_inodes` and `total_space` metrics (#5424)
- metrics: expand the FUSE latency histogram range (#6199)

# Bugfix

- vfs: remove object files if metadata commit fails with ENOENT (#5133)
- vfs: allow `ftruncate` on an opened and deleted file (#5664)
- meta: fix the issue that trash files may be dangling (#5414)
- meta: evict symlink cache periodically (#5131)
- meta: fix permission check of open and mknod (#4948)
- meta: fix volume stats update issues (#4979,#5221,#5272,#5280,#5303,#5305)
- meta: fix the issue that the username cannot contain the "@" character (#5761)
- meta: fix potential issues with context reuse (#5372,#5665)
- meta: exit `doCleanupSlices` timely (#6081)
- meta: check for incorrect filenames (#6073)
- meta: fix stats for trash directories (#6099)
- meta: fix all-zero read response due to invalid openFile.chunks cache (#6192)
- meta: fix leaking objects when quota is full (#6200)
- meta/kv: return EINVAL for empty xattr value when using TiKV (#6104)
- meta/sql: fix nanosecond of link and readlink (#5496)
- meta/sql: fix sqlite3 "database is locked" issue (#5586)
- meta/sql: fix flush quotas and stats deadlock (#5706,#5712)
- meta/sql: fix database retry issues (#5440,#5485)
- meta/sql: fix concurrency issue in json backup dump (#5930)
- meta/redis: fix potential loss of attributes issue (#5709)
- meta/redis: check `MGet` error to avoid panic (#5773)
- meta/redis: fix parsing options for redis cluster (#6129)
- chunk: remove unexpected read retries (#5199)
- chunk: deduplicate prefetch/partial-block/full-block requests to avoid read amplification (#4947，#5219)
- cache/stage: retry uploading stage blocks upon disk read failures (#5084)
- cache/stage: add stage write timeout with object storage fallback to prevent hangs (#5230,#5563,#5575)
- cache: disable writeback during compaction to prevent EIO from multi-client races (#5767)
- cache: maintain a margin based on the free-space ratio during cleanup to prevent client interference (#5216)
- cache: fix inoperative `--cache-expire` option in memory cache (#5313)
- cache: fix cache duplication issue caused by concurrency (#5623)
- cache: fix potential high latency caused by locks in `getDiskUsage` (#5857)
- cache: fix sleep duration in cache expiration loop (#6165)
- cmd/mount: fix process management issues (#5064,#5172,#5641,#5881)
- cmd/mount: fix potential hangs during stack trace collection by adding timeout (#5325)
- cmd/mount: support using ":" to separate multiple `cache-dir` in fstab (#5719)
- cmd/mount: fix the mount path to an absolute path in fstab (#5951)
- cmd/debug: fix potential hangs by adding timeout (#5050,#5627)
- cmd/sync: fix command-line interaction issues (#5138,#5744,#5752,#6115)
- cmd/sync: fix parallel list issues (#5645,#5646)
- cmd/sync: fix statistical issues (#5658,#5763)
- cmd/sync: fix manager ip issues (#5597,#5950,#5947)
- cmd/sync: ignore special files of JuiceFS (#5523)
- cmd/sync: fix leaked directories issue (#5705,#5777,#5787)
- cmd/sync: fix missing error log for worker process (#5362)
- cmd/sync: use a dedicated buffer pool to avoid contention with writes (#5568)
- cmd/sync: fix the issue that worker might get an outdated binary. (#5899)
- cmd/sync: fix check-all in cluster mode (#5904)
- cmd/sync: fix uneven task distribution among workers. (#5957)
- cmd/sync: return error if uid or gid cannot be found (#5995)
- cmd/sync: fix copied counter (#6114)
- cmd/sync: fix mtime comparison for check-change (#6134)
- cmd/sync: fix progress bar for file list (#6162)
- cmd/gateway: fix missing memory buffer usage metrics (#5560)
- cmd/gateway: fix potential etag inconsistency caused by concurrency (#5258,#5378)
- cmd/gateway: fix some compatibility issues (#6004,#6006)
- cmd/gateway: fix some panic issues (#5098, #5164,#5807)
- cmd/fsck: fix the missing trash files statistics in volume stat (#6105)
- cmd/clone: skip trash files (#5109)
- cmd/gc: fix misclassification of pending-delete objects as leaked (#5198)
- cmd/config: fix bucket update issue (#5653)
- cmd/objbench: fix time and size statistics (#5264,#5267,#6201)
- cmd/warmup: fix handling files with holes (#6034)
- cmd/warmup: prevent division by zero when checking for empty files or directories (#6074)
- object: fix error handling for directory creation and deletion (#6176)
- object: fix burst bandwidth (#6102)
- object: fix the issue where modifying mtime also changes atime (#6026)
- object: fix sha256 checksum in s3 is added again (#6100)
- object: add validation for ciphertext length to prevent panic caused by data corruption in object storage (#6107)
- object: fix error handling in HEAD (#6120)
- object: fix object mtime precision issue (#6127)
- object/sftp: fix connection reuse issue (#6123)
- object/ks3: check endpoint format to avoid panic (#5074)
- object/s3: fix long timeout issue when AWS IMDSv2 is disabled (#5231)
- object/tos: fix wrong region of TOS (#6191)
- object/jfs: fix file creation logic (#5792)
- hadoop: fix juicefs.umask does not work (#5337)
- hadoop: fix background task thread leak (#5363)
- hadoop: fix HDFS command not displaying JuiceFS quota (#5939)
- utils/cond: fix timer issue (#5631,#5634)
- log: fix quote parsing issue in accesslog (#5588)
- build: fix static linking for ARM64 binary (#5612)

# Deps

- deps: bump `github.com/winfsp/cgofuse` from v1.5.0 to v1.6.0 (#5840)
- deps: bump MySQL driver `github.com/go-sql-driver/mysql` from v1.7.1(2023-04-25) to v1.9.1(2025-03-20) (#5834)
- deps: bump PostgreSQL driver `github.com/jackc/pgx/v5` from v5.5.4 to v5.7.3 (#5835)
- deps: bump sqlite3 driver `github.com/mattn/go-sqlite3` from v1.14.16 to v1.14.24 (#5555)
- deps: bump BadgerDB `github.com/dgraph-io/badger/v4` from v4.2.0 to v4.5.1 (#5626)
- deps: bump Redis client `github.com/redis/go-redis/v9` from v9.0.2 to v9.7.3 (#5832)
- deps: bump TiKV SDK from v2.0.1 to v2.0.7 (#5850)
- deps: bump baidu bce sdk from v0.9.200 to v0.9.221 (#5820)
- deps: bump tencent cos sdk to v0.7.45 to v0.7.63 (#5821)
- deps: bump volcengine tos sdk from v2.7.0 to v2.7.8 (#5838)
- deps: change aliyun oss sdk to `github.com/aliyun/alibabacloud-oss-go-sdk-v2` v1.2.1 (#5830,#5878)
- deps: bump ks3 sdk from v1.2.2 to v1.6.0 (#5829)
- deps: bump ibm cos sdk from v1.10.0 to v1.12.1 (#5828)
- deps: bump qiniu sdk from v7.15.0 to v7.25.2 (#5831)
- deps: bump gs sdk from v1.28.1(2022-12-02) to v1.48.0(2024-12-05) (#5849)
- deps: bump azure sdk from v1.0.0 to 1.3.1 (#5819)
- deps: bump `github.com/ncw/swift/v2` from v2.0.1 to v2.0.3 (#5836)
- deps: bump `github.com/colinmarc/hdfs/v2` from v2.3.0 to v2.4.0 (#5827)
- deps: bump `github.com/studio-b12/gowebdav` from 3282f94193f2(2023-02-04) to v0.10.0 (#5839)
- deps: bump prometheus from v1.15.1 to v1.21.1 (#5843)
- deps: bump `github.com/goccy/go-json` from v0.10.2 to v0.10.5 (#5841)
- deps: bump `github.com/grafana/pyroscope-go` from v1.0.4 to v1.2.1 (#5842)
- deps: bump `golang.org/x/crypto` from 0.21.0 to 0.31.0 (#5374)
- deps: bump `golang.org/x/net` from v0.23.0 to v0.33.0 (#5412)
- deps: bump `github.com/DataDog/zstd` from v1.5.0 to v1.5.6 (#5741)
- deps: bump `github.com/remyoudompheng/bigfft` to latest version (#5747)
- deps: bump `go.uber.org/automaxprocs` from v1.5.2 to v1.6.0 (#5844)