Bcachefs File-System Benchmarks

Bcachefs file-system benchmarks by Michael Larabel for a future article.

HTML result view exported from: https://openbenchmarking.org/result/2311274-NE-SSDSTORAG18&rdt&grw.

Bcachefs File-System BenchmarksProcessorMotherboardChipsetMemoryDiskGraphicsAudioMonitorNetworkOSKernelDesktopDisplay ServerOpenGLCompilerFile-SystemScreen ResolutionBcachefsBtrfsEXT4XFSF2FSAMD Ryzen Threadripper 7980X 64-Cores @ 8.21GHz (64 Cores / 128 Threads)ASUS Pro WS TRX50-SAGE WIFI (0217 BIOS)AMD Device 14a4128GB2000GB Corsair MP700 PRO + 1000GB Western Digital WDS100T1X0E-00AFY0AMD Radeon RX 7900 XT 20GB (2025/1249MHz)Realtek ALC1220DELL U2723QEAquantia Device 04c0 + Intel I226-LM + MEDIATEK MT7922 802.11ax PCIUbuntu 23.106.7.0-060700rc2daily20231126-generic (x86_64)GNOME Shell 45.0X Server 1.21.1.7 + Wayland4.6 Mesa 23.2.1-1ubuntu3 (LLVM 15.0.7 DRM 3.56)GCC 13.2.0BcacheFS3840x2160btrfsext4xfsf2fsOpenBenchmarking.orgKernel Details- Transparent Huge Pages: madviseCompiler Details- --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-link-serialization=2 --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-defaulted --enable-offload-targets=nvptx-none=/build/gcc-13-XYspKM/gcc-13-13.2.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-13-XYspKM/gcc-13-13.2.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-build-config=bootstrap-lto-lean --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v Disk Details- Bcachefs: NONE / fix_errors=yes,nojournal_transaction_names,relatime,rw / Block Size: 512- Btrfs: NONE / discard=async,relatime,rw,space_cache=v2,ssd,subvol=/,subvolid=5 / Block Size: 4096- EXT4: NONE / relatime,rw / Block Size: 4096- XFS: NONE / attr2,inode64,logbsize=32k,logbufs=8,noquota,relatime,rw / Block Size: 4096- F2FS: NONE / acl,active_logs=6,alloc_mode=default,background_gc=on,barrier,checkpoint_merge,discard,discard_unit=block,errors=continue,extent_cache,flush_merge,fsync_mode=posix,inline_data,inline_dentry,inline_xattr,lazytime,memory=normal,mode=adaptive,no_heap,nogc_merge,relatime,rw,user_xattr / Block Size: 4096 Processor Details- Scaling Governor: amd-pstate-epp powersave (EPP: balance_performance) - CPU Microcode: 0xa108105Security Details- gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_rstack_overflow: Mitigation of Safe RET + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Enhanced / Automatic IBRS IBPB: conditional STIBP: always-on RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected

Bcachefs File-System Benchmarksdbench: 12fio: Rand Read - IO_uring - No - Yes - 4KB - 32 - Default Test Directoryfio: Rand Write - IO_uring - No - Yes - 4KB - 1 - Default Test Directoryfio: Seq Read - IO_uring - No - Yes - 2MB - 1 - Default Test Directoryfio: Seq Read - IO_uring - No - Yes - 2MB - 32 - Default Test Directoryfio: Seq Write - IO_uring - No - Yes - 2MB - 1 - Default Test Directorycockroach: KV, 50% Reads - 128sqlite: 2sqlite: 4sqlite: 8sqlite: 16pgbench: 1000 - 1000 - Read Writepgbench: 1000 - 1000 - Read Write - Average LatencyBcachefsBtrfsEXT4XFSF2FS747.406158110682010331.1610111.731779.4148184.151.59352.81257.21466.8736558157.7011582.944925667521111396.0210720.094929.7845262.369.93171.19973.37978.2243028733.0401611.99157367722181911448.2110798.651631.1970152.844.14966.35695.298150.2413270131.020808.244156572022324711483.7510883.3111010.6970505.534.05952.83587.274154.6505138119.9762039.79156876722172811387.5010432.6910995.6484252.641.21757.77284.342104.8752748836.535OpenBenchmarking.org

Dbench

Client Count: 12

OpenBenchmarking.orgMB/s, More Is BetterDbench 4.0Client Count: 12BcachefsBtrfsEXT4XFSF2FS400800120016002000SE +/- 16.80, N = 9SE +/- 9.48, N = 3SE +/- 18.01, N = 9SE +/- 0.63, N = 3SE +/- 1.03, N = 3747.411582.941611.99808.242039.791. (CC) gcc options: -lpopt -O2

Flexible IO Tester

Type: Random Read - Engine: IO_uring - Buffered: No - Direct: Yes - Block Size: 4KB - Job Count: 32 - Disk Target: Default Test Directory

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.35Type: Random Read - Engine: IO_uring - Buffered: No - Direct: Yes - Block Size: 4KB - Job Count: 32 - Disk Target: Default Test DirectoryBcachefsBtrfsEXT4XFSF2FS300K600K900K1200K1500KSE +/- 1208.88, N = 3SE +/- 3682.01, N = 3SE +/- 912.07, N = 3SE +/- 3432.72, N = 3SE +/- 3142.11, N = 3158110492566157367715657201568767MIN: 487738.18 / MAX: 499795.26MIN: 1571879.93 / MAX: 1574845.88MIN: 1559531.51 / MAX: 1571388.76MIN: 1563469.75 / MAX: 1574343.531. (CC) gcc options: -rdynamic -ltcmalloc -ll -lnuma -lz -lm -laio -lpthread -ldl -lcurl -lssl -lcrypto -std=gnu99 -ffast-math -include -O3 -fcommon -march=native

Flexible IO Tester

Type: Random Write - Engine: IO_uring - Buffered: No - Direct: Yes - Block Size: 4KB - Job Count: 1 - Disk Target: Default Test Directory

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.35Type: Random Write - Engine: IO_uring - Buffered: No - Direct: Yes - Block Size: 4KB - Job Count: 1 - Disk Target: Default Test DirectoryBcachefsBtrfsEXT4XFSF2FS50K100K150K200K250KSE +/- 66.80, N = 15SE +/- 220.39, N = 3SE +/- 934.79, N = 3SE +/- 1470.61, N = 3SE +/- 2479.24, N = 3682075211221819223247221728MIN: 74826.59 / MAX: 75589.96MIN: 220427.04 / MAX: 223595.96MIN: 221503.62 / MAX: 226169.77MIN: 217209.02 / MAX: 225754.91. (CC) gcc options: -rdynamic -ltcmalloc -ll -lnuma -lz -lm -laio -lpthread -ldl -lcurl -lssl -lcrypto -std=gnu99 -ffast-math -include -O3 -fcommon -march=native

Flexible IO Tester

Type: Sequential Read - Engine: IO_uring - Buffered: No - Direct: Yes - Block Size: 2MB - Job Count: 1 - Disk Target: Default Test Directory

OpenBenchmarking.orgMiB/s, More Is BetterFlexible IO Tester 3.35Type: Sequential Read - Engine: IO_uring - Buffered: No - Direct: Yes - Block Size: 2MB - Job Count: 1 - Disk Target: Default Test DirectoryBcachefsBtrfsEXT4XFSF2FS2K4K6K8K10KSE +/- 76.79, N = 3SE +/- 2.04, N = 3SE +/- 5.93, N = 3SE +/- 3.31, N = 3SE +/- 8.08, N = 310331.1611396.0211448.2111483.7511387.50MIN: 10187.06 / MAX: 10449.23MIN: 11393.15 / MAX: 11399.97MIN: 11439.15 / MAX: 11459.36MIN: 11477.88 / MAX: 11489.35MIN: 11373.02 / MAX: 11400.951. (CC) gcc options: -rdynamic -ltcmalloc -ll -lnuma -lz -lm -laio -lpthread -ldl -lcurl -lssl -lcrypto -std=gnu99 -ffast-math -include -O3 -fcommon -march=native

Flexible IO Tester

Type: Sequential Read - Engine: IO_uring - Buffered: No - Direct: Yes - Block Size: 2MB - Job Count: 32 - Disk Target: Default Test Directory

OpenBenchmarking.orgMiB/s, More Is BetterFlexible IO Tester 3.35Type: Sequential Read - Engine: IO_uring - Buffered: No - Direct: Yes - Block Size: 2MB - Job Count: 32 - Disk Target: Default Test DirectoryBcachefsBtrfsEXT4XFSF2FS2K4K6K8K10KSE +/- 53.06, N = 3SE +/- 44.48, N = 3SE +/- 58.12, N = 3SE +/- 64.98, N = 3SE +/- 91.69, N = 810111.7310720.0910798.6510883.3110432.69MIN: 10037.07 / MAX: 10214.38MIN: 10631.46 / MAX: 10770.97MIN: 10696.74 / MAX: 10898.04MIN: 10754.18 / MAX: 10960.58MIN: 10119.85 / MAX: 10855.141. (CC) gcc options: -rdynamic -ltcmalloc -ll -lnuma -lz -lm -laio -lpthread -ldl -lcurl -lssl -lcrypto -std=gnu99 -ffast-math -include -O3 -fcommon -march=native

Flexible IO Tester

Type: Sequential Write - Engine: IO_uring - Buffered: No - Direct: Yes - Block Size: 2MB - Job Count: 1 - Disk Target: Default Test Directory

OpenBenchmarking.orgMiB/s, More Is BetterFlexible IO Tester 3.35Type: Sequential Write - Engine: IO_uring - Buffered: No - Direct: Yes - Block Size: 2MB - Job Count: 1 - Disk Target: Default Test DirectoryBcachefsBtrfsEXT4XFSF2FS2K4K6K8K10KSE +/- 100.08, N = 15SE +/- 44.22, N = 3SE +/- 60.81, N = 3SE +/- 5.97, N = 3SE +/- 3.58, N = 31779.414929.781631.1911010.6910995.64MIN: 1385.17 / MAX: 3033.5MIN: 4841.36 / MAX: 4975.3MIN: 1043.13 / MAX: 925220.58MIN: 10999 / MAX: 11018.62MIN: 10989.72 / MAX: 11002.081. (CC) gcc options: -rdynamic -ltcmalloc -ll -lnuma -lz -lm -laio -lpthread -ldl -lcurl -lssl -lcrypto -std=gnu99 -ffast-math -include -O3 -fcommon -march=native

CockroachDB

Workload: KV, 50% Reads - Concurrency: 128

OpenBenchmarking.orgops/s, More Is BetterCockroachDB 22.2Workload: KV, 50% Reads - Concurrency: 128BcachefsBtrfsEXT4XFSF2FS20K40K60K80K100KSE +/- 2640.04, N = 12SE +/- 198.41, N = 3SE +/- 267.13, N = 3SE +/- 177.77, N = 3SE +/- 173.63, N = 348184.145262.370152.870505.584252.6

SQLite

Threads / Copies: 2

OpenBenchmarking.orgSeconds, Fewer Is BetterSQLite 3.41.2Threads / Copies: 2BcachefsBtrfsEXT4XFSF2FS1632486480SE +/- 0.53, N = 3SE +/- 0.40, N = 3SE +/- 1.23, N = 12SE +/- 0.27, N = 3SE +/- 0.47, N = 351.5969.9344.1534.0641.221. (CC) gcc options: -O2 -lreadline -ltermcap -lz -lm

SQLite

Threads / Copies: 4

OpenBenchmarking.orgSeconds, Fewer Is BetterSQLite 3.41.2Threads / Copies: 4BcachefsBtrfsEXT4XFSF2FS1632486480SE +/- 0.33, N = 3SE +/- 0.16, N = 3SE +/- 0.60, N = 3SE +/- 0.17, N = 3SE +/- 0.66, N = 352.8171.2066.3652.8457.771. (CC) gcc options: -O2 -lreadline -ltermcap -lz -lm

SQLite

Threads / Copies: 8

OpenBenchmarking.orgSeconds, Fewer Is BetterSQLite 3.41.2Threads / Copies: 8BcachefsBtrfsEXT4XFSF2FS20406080100SE +/- 0.51, N = 3SE +/- 0.30, N = 3SE +/- 0.06, N = 3SE +/- 0.25, N = 3SE +/- 0.67, N = 957.2173.3895.3087.2784.341. (CC) gcc options: -O2 -lreadline -ltermcap -lz -lm

SQLite

Threads / Copies: 16

OpenBenchmarking.orgSeconds, Fewer Is BetterSQLite 3.41.2Threads / Copies: 16BcachefsBtrfsEXT4XFSF2FS306090120150SE +/- 0.12, N = 3SE +/- 0.89, N = 4SE +/- 0.30, N = 3SE +/- 0.38, N = 3SE +/- 1.37, N = 366.8778.22150.24154.65104.881. (CC) gcc options: -O2 -lreadline -ltermcap -lz -lm

PostgreSQL

Scaling Factor: 1000 - Clients: 1000 - Mode: Read Write

OpenBenchmarking.orgTPS, More Is BetterPostgreSQL 16Scaling Factor: 1000 - Clients: 1000 - Mode: Read WriteBcachefsBtrfsEXT4XFSF2FS11K22K33K44K55KSE +/- 393.97, N = 12SE +/- 231.44, N = 12SE +/- 1417.85, N = 9SE +/- 3124.70, N = 9SE +/- 570.95, N = 126558302873270151381274881. (CC) gcc options: -fno-strict-aliasing -fwrapv -O2 -lpgcommon -lpgport -lpq -lm

PostgreSQL

Scaling Factor: 1000 - Clients: 1000 - Mode: Read Write - Average Latency

OpenBenchmarking.orgms, Fewer Is BetterPostgreSQL 16Scaling Factor: 1000 - Clients: 1000 - Mode: Read Write - Average LatencyBcachefsBtrfsEXT4XFSF2FS306090120150SE +/- 7.94, N = 12SE +/- 0.26, N = 12SE +/- 1.27, N = 9SE +/- 1.06, N = 9SE +/- 0.68, N = 12157.7033.0431.0219.9836.541. (CC) gcc options: -fno-strict-aliasing -fwrapv -O2 -lpgcommon -lpgport -lpq -lm

Phoronix Test Suite v10.8.5