Ubuntu 19.10 ZFS vs. EXT4

AMD Ryzen 9 3900X ZFS vs. EXT4 tests by Michael Larabel for a future article.

HTML result view exported from: https://openbenchmarking.org/result/1910109-AS-UBUNTUZFS73.

Ubuntu 19.10 ZFS vs. EXT4ProcessorMotherboardChipsetMemoryDiskGraphicsAudioMonitorNetworkOSKernelDesktopDisplay ServerDisplay DriverOpenGLCompilerFile-SystemScreen ResolutionZFSEXT4AMD Ryzen 9 3900X 12-Core @ 3.80GHz (12 Cores / 24 Threads)ASUS ROG CROSSHAIR VIII HERO (WI-FI) (1001 BIOS)AMD Starship/Matisse16384MB2000GB Force MP600AMD Radeon RX 56/64 8GB (1630/945MHz)AMD Vega 10 HDMI AudioASUS VP28URealtek Device 8125 + Intel I211 + Intel Device 2723Ubuntu 19.105.3.0-17-generic (x86_64)GNOME Shell 3.34.1X Server 1.20.5amdgpu 19.0.14.5 Mesa 19.2.0 (LLVM 9.0.0)GCC 9.2.1 20191008zfs3840x2160ext4OpenBenchmarking.orgCompiler Details- --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,gm2 --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-multiarch --enable-multilib --enable-nls --enable-offload-targets=nvptx-none,hsa --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v Processor Details- Scaling Governor: acpi-cpufreq ondemandDisk Scheduler Details- ZFS: NONEJava Details- OpenJDK Runtime Environment (build 11.0.5-ea+9-post-Ubuntu-1ubuntu1)Python Details- Python 2.7.16+ + Python 3.7.5rc1Security Details- l1tf: Not affected + mds: Not affected + meltdown: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl and seccomp + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Full AMD retpoline IBPB: conditional STIBP: always-on RSB filling Disk Details- EXT4: NONE / errors=remount-ro,relatime,rw

Ubuntu 19.10 ZFS vs. EXT4fio: Rand Read - Linux AIO - No - No - 4KB - Default Test Directoryfio: Rand Read - Linux AIO - No - No - 4KB - Default Test Directoryfio: Rand Write - Linux AIO - No - No - 4KB - Default Test Directoryfio: Rand Write - Linux AIO - No - No - 4KB - Default Test Directoryfio: Seq Read - Linux AIO - No - No - 2MB - Default Test Directoryfs-mark: 1000 Files, 1MB Sizecompilebench: Compilecompilebench: Initial Createcompilebench: Read Compiled Treebuild-gcc: Time To Compilesqlite-speedtest: Timed Time - Size 1,000cassandra: Writesrocksdb: Rand Fillrocksdb: Rand Readrocksdb: Seq Fillrocksdb: Read While WritingZFSEXT428372433102.402623340861513.081505.70209.281343.29705.54145.999567456439563237260600927172349414873806679352393332970578.202438.50716.663987.50671.1656.961096919594286339626111108422848469OpenBenchmarking.org

Flexible IO Tester

Type: Random Read - IO Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 4KB - Disk Target: Default Test Directory

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.16Type: Random Read - IO Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 4KB - Disk Target: Default Test DirectoryZFSEXT430060090012001500SE +/- 2.33, N = 3SE +/- 8.35, N = 328314871. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -U_FORTIFY_SOURCE -march=native -ll -lrt -laio -lm -lpthread -ldl

Flexible IO Tester

Type: Random Read - IO Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 4KB - Disk Target: Default Test Directory

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.16Type: Random Read - IO Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 4KB - Disk Target: Default Test DirectoryZFSEXT480K160K240K320K400KSE +/- 633.33, N = 3SE +/- 2333.33, N = 3724333806671. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -U_FORTIFY_SOURCE -march=native -ll -lrt -laio -lm -lpthread -ldl

Flexible IO Tester

Type: Random Write - IO Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 4KB - Disk Target: Default Test Directory

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.16Type: Random Write - IO Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 4KB - Disk Target: Default Test DirectoryZFSEXT42004006008001000SE +/- 1.60, N = 3SE +/- 52.81, N = 12102.40935.001. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -U_FORTIFY_SOURCE -march=native -ll -lrt -laio -lm -lpthread -ldl

Flexible IO Tester

Type: Random Write - IO Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 4KB - Disk Target: Default Test Directory

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.16Type: Random Write - IO Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 4KB - Disk Target: Default Test DirectoryZFSEXT450K100K150K200K250KSE +/- 417.67, N = 3SE +/- 13569.87, N = 12262332393331. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -U_FORTIFY_SOURCE -march=native -ll -lrt -laio -lm -lpthread -ldl

Flexible IO Tester

Type: Sequential Read - IO Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 2MB - Disk Target: Default Test Directory

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.16Type: Sequential Read - IO Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 2MB - Disk Target: Default Test DirectoryZFSEXT49001800270036004500SE +/- 2.19, N = 3SE +/- 307.39, N = 12408629701. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -U_FORTIFY_SOURCE -march=native -ll -lrt -laio -lm -lpthread -ldl

FS-Mark

Test: 1000 Files, 1MB Size

OpenBenchmarking.orgFiles/s, More Is BetterFS-Mark 3.3Test: 1000 Files, 1MB SizeZFSEXT430060090012001500SE +/- 14.58, N = 9SE +/- 2.69, N = 31513.08578.201. (CC) gcc options: -static

Compile Bench

Test: Compile

OpenBenchmarking.orgMB/s, More Is BetterCompile Bench 0.6Test: CompileZFSEXT45001000150020002500SE +/- 19.76, N = 3SE +/- 41.19, N = 31505.702438.50

Compile Bench

Test: Initial Create

OpenBenchmarking.orgMB/s, More Is BetterCompile Bench 0.6Test: Initial CreateZFSEXT4150300450600750SE +/- 2.22, N = 3SE +/- 3.45, N = 3209.28716.66

Compile Bench

Test: Read Compiled Tree

OpenBenchmarking.orgMB/s, More Is BetterCompile Bench 0.6Test: Read Compiled TreeZFSEXT49001800270036004500SE +/- 9.17, N = 3SE +/- 61.38, N = 31343.293987.50

Timed GCC Compilation

Time To Compile

OpenBenchmarking.orgSeconds, Fewer Is BetterTimed GCC Compilation 8.2Time To CompileZFSEXT4150300450600750SE +/- 1.39, N = 3SE +/- 0.60, N = 3705.54671.16

SQLite Speedtest

Timed Time - Size 1,000

OpenBenchmarking.orgSeconds, Fewer Is BetterSQLite Speedtest 3.30Timed Time - Size 1,000ZFSEXT4306090120150SE +/- 0.45, N = 3SE +/- 0.24, N = 3145.9956.961. (CC) gcc options: -O2 -ldl -lpthread

Apache Cassandra

Test: Writes

OpenBenchmarking.orgOp/s, More Is BetterApache Cassandra 3.11.4Test: WritesZFSEXT420K40K60K80K100KSE +/- 1313.71, N = 3SE +/- 1079.62, N = 895674109691

Facebook RocksDB

Test: Random Fill

OpenBenchmarking.orgOp/s, More Is BetterFacebook RocksDB 6.3.6Test: Random FillZFSEXT4200K400K600K800K1000KSE +/- 1525.01, N = 3SE +/- 6909.48, N = 35643959594281. (CXX) g++ options: -O3 -march=native -std=c++11 -fno-builtin-memcmp -fno-rtti -rdynamic -lpthread

Facebook RocksDB

Test: Random Read

OpenBenchmarking.orgOp/s, More Is BetterFacebook RocksDB 6.3.6Test: Random ReadZFSEXT414M28M42M56M70MSE +/- 948389.35, N = 3SE +/- 666074.26, N = 363237260633962611. (CXX) g++ options: -O3 -march=native -std=c++11 -fno-builtin-memcmp -fno-rtti -rdynamic -lpthread

Facebook RocksDB

Test: Sequential Fill

OpenBenchmarking.orgOp/s, More Is BetterFacebook RocksDB 6.3.6Test: Sequential FillZFSEXT4200K400K600K800K1000KSE +/- 4171.10, N = 3SE +/- 6209.26, N = 360092711108421. (CXX) g++ options: -O3 -march=native -std=c++11 -fno-builtin-memcmp -fno-rtti -rdynamic -lpthread

Facebook RocksDB

Test: Read While Writing

OpenBenchmarking.orgOp/s, More Is BetterFacebook RocksDB 6.3.6Test: Read While WritingZFSEXT4600K1200K1800K2400K3000KSE +/- 26538.96, N = 3SE +/- 34441.37, N = 5172349428484691. (CXX) g++ options: -O3 -march=native -std=c++11 -fno-builtin-memcmp -fno-rtti -rdynamic -lpthread

Phoronix Test Suite v10.8.4