2019 BCachefs Linux Benchmarks

Intel Optane 900p benchmarks with various file-systems including the latest Bcachefs file-system for a future article on Phoronix.

HTML result view exported from: https://openbenchmarking.org/result/1906200-HV-STORAGETE98.

2019 BCachefs Linux BenchmarksProcessorMotherboardChipsetMemoryDiskGraphicsAudioMonitorNetworkOSKernelDesktopDisplay ServerOpenGLCompilerFile-SystemScreen ResolutionEXT4XFSBtrfsF2FSZFSBcachefsAMD Ryzen Threadripper 2990WX 32-Core @ 3.00GHz (32 Cores / 64 Threads)ASUS ROG ZENITH EXTREME (1701 BIOS)AMD 17h4 x 8192 MB DDR4-3200MT/s F4-3200C14-8GFX280GB INTEL SSDPED1D280GA + Samsung SSD 970 EVO 500GBAMD Radeon RX 64 8GB (1590/800MHz)Realtek ALC1220ASUS VP28UIntel I211 + Qualcomm Atheros QCA6174 802.11ac + Wilocity Wil6200 802.11adUbuntu 19.045.2.0-999-generic (x86_64) 20190605GNOME Shell 3.32.0X Server4.5 Mesa 19.0.2 (LLVM 8.0.0)GCC 8.3.0ext43840x2160xfsbtrfsf2fsSamsung SSD 970 EVO 500GB + 2 x 280GB INTEL SSDPED1D280GAzfs2 x 280GB INTEL SSDPED1D280GA + Samsung SSD 970 EVO 500GB5.1.0-bcachefs (x86_64)BcacheFSOpenBenchmarking.orgCompiler Details- --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++ --enable-libmpx --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-targets=nvptx-none --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib --with-tune=generic --without-cuda-driver -v Disk Details- EXT4: NONE / relatime,rw- XFS: NONE / attr2,inode64,noquota,relatime,rw- Btrfs: NONE / relatime,rw,space_cache,ssd,subvol=/,subvolid=5- F2FS: NONE / acl,active_logs=6,alloc_mode=default,background_gc=on,discard,extent_cache,flush_merge,fsync_mode=posix,inline_data,inline_dentry,inline_xattr,lazytime,mode=adaptive,no_heap,relatime,rw,user_xattr- Bcachefs: NONE / relatime,rwProcessor Details- Scaling Governor: acpi-cpufreq ondemandPython Details- Python 2.7.16 + Python 3.7.3Security Details- EXT4: l1tf: Not affected + mds: Not affected + meltdown: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl and seccomp + spectre_v1: Mitigation of __user pointer sanitization + spectre_v2: Mitigation of Full AMD retpoline IBPB: conditional STIBP: disabled RSB filling- XFS: l1tf: Not affected + mds: Not affected + meltdown: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl and seccomp + spectre_v1: Mitigation of __user pointer sanitization + spectre_v2: Mitigation of Full AMD retpoline IBPB: conditional STIBP: disabled RSB filling- Btrfs: l1tf: Not affected + mds: Not affected + meltdown: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl and seccomp + spectre_v1: Mitigation of __user pointer sanitization + spectre_v2: Mitigation of Full AMD retpoline IBPB: conditional STIBP: disabled RSB filling- F2FS: l1tf: Not affected + mds: Not affected + meltdown: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl and seccomp + spectre_v1: Mitigation of __user pointer sanitization + spectre_v2: Mitigation of Full AMD retpoline IBPB: conditional STIBP: disabled RSB filling- ZFS: l1tf: Not affected + mds: Not affected + meltdown: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl and seccomp + spectre_v1: Mitigation of __user pointer sanitization + spectre_v2: Mitigation of Full AMD retpoline IBPB: conditional STIBP: disabled RSB filling- Bcachefs: l1tf: Not affected + meltdown: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl and seccomp + spectre_v1: Mitigation of __user pointer sanitization + spectre_v2: Mitigation of Full AMD retpoline IBPB: conditional STIBP: disabled RSB fillingDisk Scheduler Details- ZFS: NONE

2019 BCachefs Linux Benchmarksstartup-time: Only Seq Reads - xterm - Default Test Directorystartup-time: Seq Reads + Writes - xterm - Default Test Directorystartup-time: Only Seq Reads - GNOME Terminal - Default Test Directorystartup-time: Only Seq Reads - LibreOffice Writer - Default Test Directorystartup-time: Seq Reads + Writes - GNOME Terminal - Default Test Directorystartup-time: Seq Reads + Writes - LibreOffice Writer - Default Test Directorysqlite: Timed SQLite Insertionsfio: Rand Read - Linux AIO - No - Yes - 2MB - Default Test Directoryfio: Rand Read - Linux AIO - No - Yes - 4KB - Default Test Directoryfio: Rand Read - Linux AIO - No - Yes - 4KB - Default Test Directoryfs-mark: 1000 Files, 1MB Sizefs-mark: 5000 Files, 1MB Size, 4 Threadsfs-mark: 4000 Files, 32 Sub Dirs, 1MB Sizedbench: 12compilebench: Compilecompilebench: Read Compiled Treepgbench: Mostly RAM - Normal Load - Read Onlypgbench: Mostly RAM - Normal Load - Read Writepgbench: Buffer Test - Normal Load - Read WriteEXT4XFSBtrfsF2FSZFSBcachefs0.520.641.991.353.101.682.062613110628300069869844091972158025442828983449230.530.722.001.342.971.7126131050269000881151588659062683150425748031481432930.670.381.440.661.070.666.69261055714253374414496611870253414042153899686200880.360.401.030.721.040.682.0226131063272000790139082268182466169224025320938416570.150.200.320.260.340.335.64440423459733630.27865.30595.93600.701420.74678.869529.103491.0919394.730.610.782.241.542.731.802.85260615940767859.501546.50862.331803.872806.591373.0342316.6316129.6133033.99OpenBenchmarking.org

Application Start-up Time

Background I/O Mix: Only Sequential Reads - Application To Start: xterm - Disk Target: Default Test Directory

OpenBenchmarking.orgsec, Fewer Is BetterApplication Start-up Time 3.4.0Background I/O Mix: Only Sequential Reads - Application To Start: xterm - Disk Target: Default Test DirectoryEXT4XFSBtrfsF2FSZFSBcachefs0.15080.30160.45240.60320.754SE +/- 0.00, N = 3SE +/- 0.01, N = 3SE +/- 0.11, N = 9SE +/- 0.00, N = 12SE +/- 0.00, N = 3SE +/- 0.00, N = 30.520.530.670.360.150.61

Application Start-up Time

Background I/O Mix: Sequential Reads + Writes - Application To Start: xterm - Disk Target: Default Test Directory

OpenBenchmarking.orgsec, Fewer Is BetterApplication Start-up Time 3.4.0Background I/O Mix: Sequential Reads + Writes - Application To Start: xterm - Disk Target: Default Test DirectoryEXT4XFSBtrfsF2FSZFSBcachefs0.17550.3510.52650.7020.8775SE +/- 0.00, N = 3SE +/- 0.08, N = 9SE +/- 0.04, N = 12SE +/- 0.03, N = 12SE +/- 0.03, N = 12SE +/- 0.04, N = 90.640.720.380.400.200.78

Application Start-up Time

Background I/O Mix: Only Sequential Reads - Application To Start: GNOME Terminal - Disk Target: Default Test Directory

OpenBenchmarking.orgsec, Fewer Is BetterApplication Start-up Time 3.4.0Background I/O Mix: Only Sequential Reads - Application To Start: GNOME Terminal - Disk Target: Default Test DirectoryEXT4XFSBtrfsF2FSZFSBcachefs0.5041.0081.5122.0162.52SE +/- 0.00, N = 3SE +/- 0.02, N = 3SE +/- 0.17, N = 12SE +/- 0.01, N = 3SE +/- 0.00, N = 3SE +/- 0.04, N = 121.992.001.441.030.322.24

Application Start-up Time

Background I/O Mix: Only Sequential Reads - Application To Start: LibreOffice Writer - Disk Target: Default Test Directory

OpenBenchmarking.orgsec, Fewer Is BetterApplication Start-up Time 3.4.0Background I/O Mix: Only Sequential Reads - Application To Start: LibreOffice Writer - Disk Target: Default Test DirectoryEXT4XFSBtrfsF2FSZFSBcachefs0.34650.6931.03951.3861.7325SE +/- 0.01, N = 3SE +/- 0.01, N = 8SE +/- 0.02, N = 12SE +/- 0.01, N = 4SE +/- 0.01, N = 12SE +/- 0.01, N = 31.351.340.660.720.261.54

Application Start-up Time

Background I/O Mix: Sequential Reads + Writes - Application To Start: GNOME Terminal - Disk Target: Default Test Directory

OpenBenchmarking.orgsec, Fewer Is BetterApplication Start-up Time 3.4.0Background I/O Mix: Sequential Reads + Writes - Application To Start: GNOME Terminal - Disk Target: Default Test DirectoryEXT4XFSBtrfsF2FSZFSBcachefs0.69751.3952.09252.793.4875SE +/- 0.03, N = 3SE +/- 0.22, N = 9SE +/- 0.08, N = 12SE +/- 0.05, N = 12SE +/- 0.03, N = 9SE +/- 0.10, N = 123.102.971.071.040.342.73

Application Start-up Time

Background I/O Mix: Sequential Reads + Writes - Application To Start: LibreOffice Writer - Disk Target: Default Test Directory

OpenBenchmarking.orgsec, Fewer Is BetterApplication Start-up Time 3.4.0Background I/O Mix: Sequential Reads + Writes - Application To Start: LibreOffice Writer - Disk Target: Default Test DirectoryEXT4XFSBtrfsF2FSZFSBcachefs0.4050.811.2151.622.025SE +/- 0.03, N = 12SE +/- 0.05, N = 12SE +/- 0.03, N = 9SE +/- 0.03, N = 10SE +/- 0.03, N = 12SE +/- 0.02, N = 31.681.710.660.680.331.80

SQLite

Timed SQLite Insertions

OpenBenchmarking.orgSeconds, Fewer Is BetterSQLite 3.22Timed SQLite InsertionsEXT4BtrfsF2FSZFSBcachefs246810SE +/- 0.01, N = 3SE +/- 0.11, N = 3SE +/- 0.00, N = 2SE +/- 0.02, N = 3SE +/- 0.03, N = 32.066.692.025.642.851. (CC) gcc options: -O2 -lz -ldl -lpthread

Flexible IO Tester

Type: Random Read - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 2MB - Disk Target: Default Test Directory

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.1Type: Random Read - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 2MB - Disk Target: Default Test DirectoryEXT4XFSBtrfsF2FSZFSBcachefs9001800270036004500SE +/- 0.33, N = 3SE +/- 39.01, N = 3SE +/- 1.20, N = 3261326132610261344042606-lnuma -libverbs-lnuma -libverbs1. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -U_FORTIFY_SOURCE -ll -lrt -laio -lz -lm -lpthread -ldl

Flexible IO Tester

Type: Random Read - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.1Type: Random Read - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test DirectoryEXT4XFSBtrfsF2FSZFSBcachefs2004006008001000SE +/- 7.31, N = 3SE +/- 2.31, N = 3SE +/- 6.07, N = 15SE +/- 15.82, N = 3SE +/- 0.67, N = 3SE +/- 1.67, N = 3110610505571063234159-lnuma -libverbs-lnuma -libverbs1. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -U_FORTIFY_SOURCE -ll -lrt -laio -lz -lm -lpthread -ldl

Flexible IO Tester

Type: Random Read - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.1Type: Random Read - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test DirectoryEXT4XFSBtrfsF2FSZFSBcachefs60K120K180K240K300KSE +/- 2081.67, N = 3SE +/- 577.35, N = 3SE +/- 1545.71, N = 15SE +/- 4041.45, N = 3SE +/- 185.59, N = 3SE +/- 384.42, N = 32830002690001425332720005973340767-lnuma -libverbs-lnuma -libverbs1. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -U_FORTIFY_SOURCE -ll -lrt -laio -lz -lm -lpthread -ldl

FS-Mark

Test: 1000 Files, 1MB Size

OpenBenchmarking.orgFiles/s, More Is BetterFS-Mark 3.3Test: 1000 Files, 1MB SizeEXT4XFSBtrfsF2FSZFSBcachefs2004006008001000SE +/- 11.91, N = 3SE +/- 3.72, N = 3SE +/- 6.56, N = 3SE +/- 3.49, N = 3SE +/- 2.84, N = 3SE +/- 4.06, N = 3698.00881.00744.00790.00630.27859.501. (CC) gcc options: -static

FS-Mark

Test: 5000 Files, 1MB Size, 4 Threads

OpenBenchmarking.orgFiles/s, More Is BetterFS-Mark 3.3Test: 5000 Files, 1MB Size, 4 ThreadsXFSBtrfsF2FSZFSBcachefs30060090012001500SE +/- 12.55, N = 13SE +/- 4.48, N = 3SE +/- 4.24, N = 3SE +/- 5.71, N = 3SE +/- 13.72, N = 31515.001449.001390.00865.301546.501. (CC) gcc options: -static

FS-Mark

Test: 4000 Files, 32 Sub Dirs, 1MB Size

OpenBenchmarking.orgFiles/s, More Is BetterFS-Mark 3.3Test: 4000 Files, 32 Sub Dirs, 1MB SizeEXT4XFSBtrfsF2FSZFSBcachefs2004006008001000SE +/- 13.00, N = 12SE +/- 4.10, N = 3SE +/- 6.99, N = 15SE +/- 9.81, N = 15SE +/- 2.05, N = 3SE +/- 0.70, N = 3698.00886.00661.00822.00595.93862.331. (CC) gcc options: -static

Dbench

Client Count: 12

OpenBenchmarking.orgMB/s, More Is BetterDbench 4.0Client Count: 12EXT4XFSBtrfsF2FSZFSBcachefs15003000450060007500SE +/- 3.54, N = 3SE +/- 9.21, N = 3SE +/- 0.79, N = 3SE +/- 3.59, N = 3SE +/- 1.49, N = 3SE +/- 1.97, N = 34409.005906.001870.006818.00600.701803.871. (CC) gcc options: -lpopt -O2

Compile Bench

Test: Compile

OpenBenchmarking.orgMB/s, More Is BetterCompile Bench 0.6Test: CompileEXT4XFSBtrfsF2FSZFSBcachefs6001200180024003000SE +/- 14.01, N = 3SE +/- 12.65, N = 3SE +/- 20.47, N = 13SE +/- 7.03, N = 3SE +/- 21.86, N = 3SE +/- 23.27, N = 31972.002683.002534.002466.001420.742806.59

Compile Bench

Test: Read Compiled Tree

OpenBenchmarking.orgMB/s, More Is BetterCompile Bench 0.6Test: Read Compiled TreeEXT4XFSBtrfsF2FSZFSBcachefs400800120016002000SE +/- 22.84, N = 3SE +/- 25.60, N = 3SE +/- 14.69, N = 3SE +/- 4.92, N = 3SE +/- 38.85, N = 3SE +/- 1.13, N = 31580.001504.001404.001692.00678.861373.03

PostgreSQL pgbench

Scaling: Mostly RAM - Test: Normal Load - Mode: Read Only

OpenBenchmarking.orgTPS, More Is BetterPostgreSQL pgbench 10.3Scaling: Mostly RAM - Test: Normal Load - Mode: Read OnlyEXT4XFSBtrfsF2FSZFSBcachefs60K120K180K240K300KSE +/- 1834.32, N = 3SE +/- 490.41, N = 3SE +/- 2461.40, N = 9SE +/- 4049.06, N = 3SE +/- 74.87, N = 3SE +/- 287.62, N = 3254428.00257480.00215389.00240253.009529.1042316.631. (CC) gcc options: -fno-strict-aliasing -fwrapv -O2 -lpgcommon -lpgport -lpq -lpthread -lrt -lcrypt -ldl -lm

PostgreSQL pgbench

Scaling: Mostly RAM - Test: Normal Load - Mode: Read Write

OpenBenchmarking.orgTPS, More Is BetterPostgreSQL pgbench 10.3Scaling: Mostly RAM - Test: Normal Load - Mode: Read WriteEXT4XFSBtrfsF2FSZFSBcachefs7K14K21K28K35KSE +/- 199.16, N = 3SE +/- 277.25, N = 3SE +/- 143.98, N = 3SE +/- 161.12, N = 3SE +/- 8.49, N = 3SE +/- 136.39, N = 328983.0031481.009686.0020938.003491.0916129.611. (CC) gcc options: -fno-strict-aliasing -fwrapv -O2 -lpgcommon -lpgport -lpq -lpthread -lrt -lcrypt -ldl -lm

PostgreSQL pgbench

Scaling: Buffer Test - Test: Normal Load - Mode: Read Write

OpenBenchmarking.orgTPS, More Is BetterPostgreSQL pgbench 10.3Scaling: Buffer Test - Test: Normal Load - Mode: Read WriteEXT4XFSBtrfsF2FSZFSBcachefs10K20K30K40K50KSE +/- 282.98, N = 3SE +/- 197.57, N = 3SE +/- 7.74, N = 3SE +/- 267.77, N = 3SE +/- 41.57, N = 3SE +/- 25.84, N = 344923.0043293.0020088.0041657.0019394.7333033.991. (CC) gcc options: -fno-strict-aliasing -fwrapv -O2 -lpgcommon -lpgport -lpq -lpthread -lrt -lcrypt -ldl -lm

Phoronix Test Suite v10.8.4