F2FS XFS EXT4 Hard Drive + SSD Linux Benchmarks

Linux file-system benchmarks by Michael Larabel for a future article.

HTML result view exported from: https://openbenchmarking.org/result/1901281-SP-F2FSBENCH59.

ProcessorMotherboardChipsetMemoryDiskGraphicsAudioMonitorNetworkOSKernelDesktopDisplay ServerOpenGLCompilerFile-SystemScreen ResolutionHDDSSD EXT4 F2FS XFS EXT4 F2FS XFSAMD Ryzen Threadripper 2950X 16-Core @ 3.50GHz (16 Cores / 32 Threads)MSI MEG X399 CREATION (MS-7B92) v1.0 (1.10 BIOS)AMD Family 17h4 x 8192 MB DDR4-1600MT/s F4-3200C14-8GTZRSamsung SSD 970 EVO 250GB + 150GB Western Digital WD1500HLHX-0llvmpipe 31GBRealtek ALC1220ASUS PB2782 x Intel I211 + Intel-AC 9260Ubuntu 18.105.0.0-999-generic (x86_64) 20190125GNOME Shell 3.30.1X Server3.3 Mesa 18.2.2 (LLVM 7.0 128 bits)GCC 8.2.0ext42560x1440f2fsxfsSamsung SSD 970 EVO 250GB + 1000GB Samsung SSD 860ext4f2fsxfsOpenBenchmarking.orgCompiler Details- --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++ --enable-libmpx --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-targets=nvptx-none --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib --with-tune=generic --without-cuda-driver -v Disk Details- HDD: EXT4: MQ-DEADLINE / relatime,rw- HDD: F2FS: MQ-DEADLINE / acl,active_logs=6,alloc_mode=default,background_gc=on,discard,extent_cache,flush_merge,fsync_mode=posix,inline_data,inline_dentry,inline_xattr,lazytime,mode=adaptive,no_heap,relatime,rw,user_xattr - HDD: XFS: MQ-DEADLINE / attr2,inode64,noquota,relatime,rw- SSD: EXT4: MQ-DEADLINE / relatime,rw- SSD: F2FS: MQ-DEADLINE / acl,active_logs=6,alloc_mode=default,background_gc=on,discard,extent_cache,flush_merge,fsync_mode=posix,inline_data,inline_dentry,inline_xattr,lazytime,mode=adaptive,no_heap,relatime,rw,user_xattr - SSD: XFS: MQ-DEADLINE / attr2,inode64,noquota,relatime,rwProcessor Details- Scaling Governor: acpi-cpufreq ondemandPython Details- Python 2.7.15+ + Python 3.6.7Security Details- __user pointer sanitization + Full AMD retpoline IBPB: conditional STIBP: disabled RSB filling + SSB disabled via prctl and seccomp

startup-time: Only Seq Reads - xterm - Default Test Directorystartup-time: Seq Reads + Writes - xterm - Default Test Directorystartup-time: Seq Reads + Writes - GNOME Terminal - Default Test Directorystartup-time: Seq Reads + Writes - LibreOffice Writer - Default Test Directorysqlite: Timed SQLite Insertionsfio: Rand Read - Linux AIO - No - Yes - 4KB - Default Test Directoryfio: Rand Read - Linux AIO - No - Yes - 4MB - Default Test Directoryfio: Rand Write - Linux AIO - No - Yes - 4KB - Default Test Directoryfio: Rand Write - Linux AIO - No - Yes - 4MB - Default Test Directoryfio: Seq Read - Linux AIO - No - Yes - 4KB - Default Test Directoryfio: Seq Read - Linux AIO - No - Yes - 4KB - Default Test Directoryfio: Seq Read - Linux AIO - No - Yes - 4MB - Default Test Directoryfio: Seq Write - Linux AIO - No - Yes - 4KB - Default Test Directoryfio: Seq Write - Linux AIO - No - Yes - 4KB - Default Test Directoryfio: Seq Write - Linux AIO - No - Yes - 4MB - Default Test Directoryfs-mark: 1000 Files, 1MB Sizefs-mark: 5000 Files, 1MB Size, 4 Threadsfs-mark: 4000 Files, 32 Sub Dirs, 1MB Sizeblogbench: Readblogbench: Writecompilebench: Compilecompilebench: Initial Createpgbench: Buffer Test - Normal Load - Read Onlypgbench: Buffer Test - Normal Load - Read Writefio: Rand Read - Linux AIO - No - Yes - 4KB - Default Test Directoryfio: Rand Write - Linux AIO - No - Yes - 4KB - Default Test DirectoryHDDSSD EXT4 F2FS XFS EXT4 F2FS XFS16.548.3755.4771.582742.141152.41117115294001471032623314327.4067.3327.3125062994932131451230742262124.1010.3136.2427.572892.351112.43105114291671321022610013042.5357.5948.5323339442996915126303978181815.448.5637.0138.202272.281092.36106115293001261002560012532.6745.1734.502318138265318944153052566562.371.647.185.4234.75365552357523440112667552415106333523170.4091.2942.07242227976431295489308177169793533912672.171.494.573.1841.81365551356523403103175467423108000523223.37109.0848.0222752653125871104305022184993467911332.431.516.605.0739.83365551356522444114000535419107333522181.63297.17188.2723083805667182443030545118879336791133OpenBenchmarking.org

Application Start-up Time

Background I/O Mix: Only Sequential Reads - Application To Start: xterm - Disk Target: Default Test Directory

HDDSSDOpenBenchmarking.orgsec, Fewer Is BetterApplication Start-up Time 2.4.0Background I/O Mix: Only Sequential Reads - Application To Start: xterm - Disk Target: Default Test DirectoryEXT4F2FSXFS612182430SE +/- 0.41, N = 9SE +/- 0.45, N = 3SE +/- 0.20, N = 9SE +/- 0.06, N = 9SE +/- 0.05, N = 9SE +/- 0.08, N = 916.5424.1015.442.372.172.43

Application Start-up Time

Background I/O Mix: Sequential Reads + Writes - Application To Start: xterm - Disk Target: Default Test Directory

HDDSSDOpenBenchmarking.orgsec, Fewer Is BetterApplication Start-up Time 2.4.0Background I/O Mix: Sequential Reads + Writes - Application To Start: xterm - Disk Target: Default Test DirectoryEXT4F2FSXFS3691215SE +/- 0.15, N = 9SE +/- 0.15, N = 3SE +/- 0.09, N = 3SE +/- 0.00, N = 3SE +/- 0.02, N = 9SE +/- 0.01, N = 38.3710.318.561.641.491.51

Application Start-up Time

Background I/O Mix: Sequential Reads + Writes - Application To Start: GNOME Terminal - Disk Target: Default Test Directory

HDDSSDOpenBenchmarking.orgsec, Fewer Is BetterApplication Start-up Time 2.4.0Background I/O Mix: Sequential Reads + Writes - Application To Start: GNOME Terminal - Disk Target: Default Test DirectoryEXT4F2FSXFS1224364860SE +/- 0.90, N = 4SE +/- 0.55, N = 9SE +/- 0.34, N = 3SE +/- 0.04, N = 3SE +/- 0.03, N = 3SE +/- 0.02, N = 355.4736.2437.017.184.576.60

Application Start-up Time

Background I/O Mix: Sequential Reads + Writes - Application To Start: LibreOffice Writer - Disk Target: Default Test Directory

HDDSSDOpenBenchmarking.orgsec, Fewer Is BetterApplication Start-up Time 2.4.0Background I/O Mix: Sequential Reads + Writes - Application To Start: LibreOffice Writer - Disk Target: Default Test DirectoryEXT4F2FSXFS1632486480SE +/- 0.57, N = 3SE +/- 0.45, N = 9SE +/- 0.48, N = 3SE +/- 0.02, N = 3SE +/- 0.05, N = 3SE +/- 0.01, N = 371.5827.5738.205.423.185.07

SQLite

Timed SQLite Insertions

HDDSSDOpenBenchmarking.orgSeconds, Fewer Is BetterSQLite 3.22Timed SQLite InsertionsEXT4F2FSXFS60120180240300SE +/- 3.82, N = 3SE +/- 2.38, N = 3SE +/- 1.84, N = 3SE +/- 0.04, N = 3SE +/- 0.15, N = 3SE +/- 0.09, N = 3274.00289.00227.0034.7541.8139.831. (CC) gcc options: -O2 -ldl -lpthread

Flexible IO Tester

Type: Random Read - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory

HDDSSDOpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.1Type: Random Read - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test DirectoryEXT4F2FSXFS80160240320400SE +/- 0.01, N = 3SE +/- 0.02, N = 3SE +/- 0.02, N = 32.142.352.28365.00365.00365.001. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -U_FORTIFY_SOURCE -lgfapi -lglusterfs -lrt -laio -lm -lpthread -ldl

Flexible IO Tester

Type: Random Read - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4MB - Disk Target: Default Test Directory

HDDSSDOpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.1Type: Random Read - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4MB - Disk Target: Default Test DirectoryEXT4F2FSXFS120240360480600SE +/- 1.50, N = 6SE +/- 0.67, N = 3SE +/- 0.58, N = 31151111095525515511. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -U_FORTIFY_SOURCE -lgfapi -lglusterfs -lrt -laio -lm -lpthread -ldl

Flexible IO Tester

Type: Random Write - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory

HDDSSDOpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.1Type: Random Write - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test DirectoryEXT4F2FSXFS80160240320400SE +/- 0.01, N = 3SE +/- 0.04, N = 3SE +/- 0.05, N = 3SE +/- 0.88, N = 3SE +/- 0.88, N = 32.412.432.36357.00356.00356.001. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -U_FORTIFY_SOURCE -lgfapi -lglusterfs -lrt -laio -lm -lpthread -ldl

Flexible IO Tester

Type: Random Write - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4MB - Disk Target: Default Test Directory

HDDSSDOpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.1Type: Random Write - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4MB - Disk Target: Default Test DirectoryEXT4F2FSXFS110220330440550SE +/- 0.88, N = 3SE +/- 0.33, N = 31171051065235235221. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -U_FORTIFY_SOURCE -lgfapi -lglusterfs -lrt -laio -lm -lpthread -ldl

Flexible IO Tester

Type: Sequential Read - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory

HDDSSDOpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.1Type: Sequential Read - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test DirectoryEXT4F2FSXFS100200300400500SE +/- 0.58, N = 3SE +/- 0.58, N = 3SE +/- 0.67, N = 3SE +/- 3.51, N = 3SE +/- 5.97, N = 121151141154404034441. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -U_FORTIFY_SOURCE -lgfapi -lglusterfs -lrt -laio -lm -lpthread -ldl

Flexible IO Tester

Type: Sequential Read - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory

HDDSSDOpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.1Type: Sequential Read - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test DirectoryEXT4F2FSXFS20K40K60K80K100KSE +/- 115.47, N = 3SE +/- 133.33, N = 3SE +/- 200.00, N = 3SE +/- 666.67, N = 3SE +/- 1560.11, N = 122940029167293001126671031751140001. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -U_FORTIFY_SOURCE -lgfapi -lglusterfs -lrt -laio -lm -lpthread -ldl

Flexible IO Tester

Type: Sequential Read - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4MB - Disk Target: Default Test Directory

HDDSSDOpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.1Type: Sequential Read - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4MB - Disk Target: Default Test DirectoryEXT4F2FSXFS120240360480600SE +/- 1.15, N = 3SE +/- 2.40, N = 3SE +/- 0.33, N = 31471321265524675351. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -U_FORTIFY_SOURCE -lgfapi -lglusterfs -lrt -laio -lm -lpthread -ldl

Flexible IO Tester

Type: Sequential Write - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory

HDDSSDOpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.1Type: Sequential Write - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test DirectoryEXT4F2FSXFS90180270360450SE +/- 0.33, N = 3SE +/- 3.93, N = 3SE +/- 2.19, N = 31031021004154234191. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -U_FORTIFY_SOURCE -lgfapi -lglusterfs -lrt -laio -lm -lpthread -ldl

Flexible IO Tester

Type: Sequential Write - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory

HDDSSDOpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.1Type: Sequential Write - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test DirectoryEXT4F2FSXFS20K40K60K80K100KSE +/- 33.33, N = 3SE +/- 881.92, N = 3SE +/- 666.67, N = 32623326100256001063331080001073331. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -U_FORTIFY_SOURCE -lgfapi -lglusterfs -lrt -laio -lm -lpthread -ldl

Flexible IO Tester

Type: Sequential Write - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4MB - Disk Target: Default Test Directory

HDDSSDOpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.1Type: Sequential Write - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4MB - Disk Target: Default Test DirectoryEXT4F2FSXFS110220330440550SE +/- 0.58, N = 3SE +/- 0.33, N = 31431301255235235221. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -U_FORTIFY_SOURCE -lgfapi -lglusterfs -lrt -laio -lm -lpthread -ldl

FS-Mark

Test: 1000 Files, 1MB Size

HDDSSDOpenBenchmarking.orgFiles/s, More Is BetterFS-Mark 3.3Test: 1000 Files, 1MB SizeEXT4F2FSXFS50100150200250SE +/- 0.26, N = 3SE +/- 0.03, N = 3SE +/- 0.19, N = 3SE +/- 0.35, N = 3SE +/- 1.18, N = 3SE +/- 0.72, N = 327.4042.5332.67170.40223.37181.631. (CC) gcc options: -static

FS-Mark

Test: 5000 Files, 1MB Size, 4 Threads

HDDSSDOpenBenchmarking.orgFiles/s, More Is BetterFS-Mark 3.3Test: 5000 Files, 1MB Size, 4 ThreadsEXT4F2FSXFS60120180240300SE +/- 0.27, N = 3SE +/- 4.34, N = 9SE +/- 0.33, N = 3SE +/- 30.78, N = 9SE +/- 37.09, N = 9SE +/- 2.96, N = 367.3357.5945.1791.29109.08297.171. (CC) gcc options: -static

FS-Mark

Test: 4000 Files, 32 Sub Dirs, 1MB Size

HDDSSDOpenBenchmarking.orgFiles/s, More Is BetterFS-Mark 3.3Test: 4000 Files, 32 Sub Dirs, 1MB SizeEXT4F2FSXFS4080120160200SE +/- 0.45, N = 9SE +/- 0.09, N = 3SE +/- 0.50, N = 12SE +/- 0.03, N = 3SE +/- 0.67, N = 5SE +/- 0.37, N = 327.3148.5334.5042.0748.02188.271. (CC) gcc options: -static

BlogBench

Test: Read

HDDSSDOpenBenchmarking.orgFinal Score, More Is BetterBlogBench 1.1Test: ReadEXT4F2FSXFS500K1000K1500K2000K2500KSE +/- 7597.92, N = 3SE +/- 5617.21, N = 3SE +/- 13393.45, N = 3SE +/- 9217.94, N = 3SE +/- 15627.71, N = 3SE +/- 1370.49, N = 32506299233394423181382422279227526523083801. (CC) gcc options: -O2 -pthread

BlogBench

Test: Write

HDDSSDOpenBenchmarking.orgFinal Score, More Is BetterBlogBench 1.1Test: WriteEXT4F2FSXFS16003200480064008000SE +/- 15.31, N = 3SE +/- 50.33, N = 3SE +/- 41.73, N = 3SE +/- 122.35, N = 3SE +/- 186.00, N = 3SE +/- 1099.95, N = 34932299626537643312556671. (CC) gcc options: -O2 -pthread

Compile Bench

Test: Compile

HDDSSDOpenBenchmarking.orgMB/s, More Is BetterCompile Bench 0.6Test: CompileEXT4F2FSXFS400800120016002000SE +/- 13.47, N = 3SE +/- 14.49, N = 3SE +/- 24.89, N = 3SE +/- 20.39, N = 3SE +/- 11.13, N = 3SE +/- 21.25, N = 31314915189412958711824

Compile Bench

Test: Initial Create

HDDSSDOpenBenchmarking.orgMB/s, More Is BetterCompile Bench 0.6Test: Initial CreateEXT4F2FSXFS110220330440550SE +/- 7.05, N = 3SE +/- 5.64, N = 3SE +/- 6.08, N = 3SE +/- 1.89, N = 3SE +/- 1.52, N = 3SE +/- 4.67, N = 3512126415489104430

PostgreSQL pgbench

Scaling: Buffer Test - Test: Normal Load - Mode: Read Only

HDDSSDOpenBenchmarking.orgTPS, More Is BetterPostgreSQL pgbench 10.3Scaling: Buffer Test - Test: Normal Load - Mode: Read OnlyEXT4F2FSXFS70K140K210K280K350KSE +/- 1240.78, N = 3SE +/- 782.15, N = 3SE +/- 973.91, N = 3SE +/- 1185.25, N = 3SE +/- 449.92, N = 3SE +/- 1316.47, N = 33074223039783052563081773050223054511. (CC) gcc options: -fno-strict-aliasing -fwrapv -O2 -lpgcommon -lpgport -lpq -lpthread -lrt -lcrypt -ldl -lm

PostgreSQL pgbench

Scaling: Buffer Test - Test: Normal Load - Mode: Read Write

HDDSSDOpenBenchmarking.orgTPS, More Is BetterPostgreSQL pgbench 10.3Scaling: Buffer Test - Test: Normal Load - Mode: Read WriteEXT4F2FSXFS400800120016002000SE +/- 7.75, N = 3SE +/- 21.37, N = 3SE +/- 14.97, N = 12SE +/- 53.24, N = 9SE +/- 77.95, N = 12SE +/- 95.92, N = 1262118186561697184918871. (CC) gcc options: -fno-strict-aliasing -fwrapv -O2 -lpgcommon -lpgport -lpq -lpthread -lrt -lcrypt -ldl -lm

Flexible IO Tester

Type: Random Read - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.1Type: Random Read - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test DirectoryEXT4F2FSXFS20K40K60K80K100KSE +/- 33.33, N = 3SE +/- 33.33, N = 3SE +/- 33.33, N = 39353393467933671. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -U_FORTIFY_SOURCE -lgfapi -lglusterfs -lrt -laio -lm -lpthread -ldl

Flexible IO Tester

Type: Random Write - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.1Type: Random Write - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test DirectoryEXT4F2FSXFS20K40K60K80K100KSE +/- 240.37, N = 3SE +/- 233.33, N = 3SE +/- 33.33, N = 39126791133911331. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -U_FORTIFY_SOURCE -lgfapi -lglusterfs -lrt -laio -lm -lpthread -ldl

Phoronix Test Suite v10.8.4