F2FS XFS EXT4 Hard Drive + SSD Linux Benchmarks

Linux file-system benchmarks by Michael Larabel for a future article.

HTML result view exported from: https://openbenchmarking.org/result/1901281-SP-F2FSBENCH59&rdt&grr.

ProcessorMotherboardChipsetMemoryDiskGraphicsAudioMonitorNetworkOSKernelDesktopDisplay ServerOpenGLCompilerFile-SystemScreen ResolutionHDDSSD EXT4 F2FS XFS XFS EXT4 F2FSAMD Ryzen Threadripper 2950X 16-Core @ 3.50GHz (16 Cores / 32 Threads)MSI MEG X399 CREATION (MS-7B92) v1.0 (1.10 BIOS)AMD Family 17h4 x 8192 MB DDR4-1600MT/s F4-3200C14-8GTZRSamsung SSD 970 EVO 250GB + 150GB Western Digital WD1500HLHX-0llvmpipe 31GBRealtek ALC1220ASUS PB2782 x Intel I211 + Intel-AC 9260Ubuntu 18.105.0.0-999-generic (x86_64) 20190125GNOME Shell 3.30.1X Server3.3 Mesa 18.2.2 (LLVM 7.0 128 bits)GCC 8.2.0ext42560x1440f2fsxfsSamsung SSD 970 EVO 250GB + 1000GB Samsung SSD 860ext4f2fsOpenBenchmarking.orgCompiler Details- --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++ --enable-libmpx --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-targets=nvptx-none --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib --with-tune=generic --without-cuda-driver -v Disk Details- HDD: EXT4: MQ-DEADLINE / relatime,rw- HDD: F2FS: MQ-DEADLINE / acl,active_logs=6,alloc_mode=default,background_gc=on,discard,extent_cache,flush_merge,fsync_mode=posix,inline_data,inline_dentry,inline_xattr,lazytime,mode=adaptive,no_heap,relatime,rw,user_xattr - HDD: XFS: MQ-DEADLINE / attr2,inode64,noquota,relatime,rw- SSD: XFS: MQ-DEADLINE / attr2,inode64,noquota,relatime,rw- SSD: EXT4: MQ-DEADLINE / relatime,rw- SSD: F2FS: MQ-DEADLINE / acl,active_logs=6,alloc_mode=default,background_gc=on,discard,extent_cache,flush_merge,fsync_mode=posix,inline_data,inline_dentry,inline_xattr,lazytime,mode=adaptive,no_heap,relatime,rw,user_xattr Processor Details- Scaling Governor: acpi-cpufreq ondemandPython Details- Python 2.7.15+ + Python 3.6.7Security Details- __user pointer sanitization + Full AMD retpoline IBPB: conditional STIBP: disabled RSB filling + SSB disabled via prctl and seccomp

fs-mark: 5000 Files, 1MB Size, 4 Threadsstartup-time: Only Seq Reads - xterm - Default Test Directoryblogbench: Readpgbench: Buffer Test - Normal Load - Read Writestartup-time: Seq Reads + Writes - GNOME Terminal - Default Test Directorystartup-time: Seq Reads + Writes - LibreOffice Writer - Default Test Directorystartup-time: Seq Reads + Writes - xterm - Default Test Directoryfs-mark: 4000 Files, 32 Sub Dirs, 1MB Sizecompilebench: Compilesqlite: Timed SQLite Insertionspgbench: Buffer Test - Normal Load - Read Onlyfio: Seq Read - Linux AIO - No - Yes - 4KB - Default Test Directoryfio: Seq Read - Linux AIO - No - Yes - 4KB - Default Test Directoryfio: Rand Read - Linux AIO - No - Yes - 4MB - Default Test Directoryfio: Rand Read - Linux AIO - No - Yes - 4KB - Default Test Directoryfio: Rand Read - Linux AIO - No - Yes - 4KB - Default Test Directoryfio: Rand Write - Linux AIO - No - Yes - 4MB - Default Test Directoryfio: Seq Write - Linux AIO - No - Yes - 4MB - Default Test Directoryfio: Seq Read - Linux AIO - No - Yes - 4MB - Default Test Directoryfio: Rand Write - Linux AIO - No - Yes - 4KB - Default Test Directoryfio: Seq Write - Linux AIO - No - Yes - 4KB - Default Test Directoryfio: Seq Write - Linux AIO - No - Yes - 4KB - Default Test Directoryfio: Rand Write - Linux AIO - No - Yes - 4KB - Default Test Directoryfs-mark: 1000 Files, 1MB Sizecompilebench: Initial Createblogbench: WriteHDDSSD EXT4 F2FS XFS XFS EXT4 F2FS67.3316.54250629962155.4771.588.3727.311314274307422294001151152.141171431472.412623310327.40512493257.5924.102333944181836.2427.5710.3148.53915289303978291671141112.351051301322.432610010242.53126299645.1715.44231813865637.0138.208.5634.501894227305256293001151092.281061251262.362560010032.674152653297.172.43230838018876.605.071.51188.27182439.833054511140004445513659336752252253535610733341991133181.63430566791.292.37242227916977.185.421.6442.07129534.753081771126674405523659353352352355235710633341591267170.404897643109.082.17227526518494.573.181.4948.0287141.813050221031754035513659346752352346735610800042391133223.371043125OpenBenchmarking.org

FS-Mark

Test: 5000 Files, 1MB Size, 4 Threads

HDDSSDOpenBenchmarking.orgFiles/s, More Is BetterFS-Mark 3.3Test: 5000 Files, 1MB Size, 4 ThreadsEXT4F2FSXFS60120180240300SE +/- 0.27, N = 3SE +/- 4.34, N = 9SE +/- 0.33, N = 3SE +/- 2.96, N = 3SE +/- 30.78, N = 9SE +/- 37.09, N = 967.3357.5945.17297.1791.29109.081. (CC) gcc options: -static

Application Start-up Time

Background I/O Mix: Only Sequential Reads - Application To Start: xterm - Disk Target: Default Test Directory

HDDSSDOpenBenchmarking.orgsec, Fewer Is BetterApplication Start-up Time 2.4.0Background I/O Mix: Only Sequential Reads - Application To Start: xterm - Disk Target: Default Test DirectoryEXT4F2FSXFS612182430SE +/- 0.41, N = 9SE +/- 0.45, N = 3SE +/- 0.20, N = 9SE +/- 0.08, N = 9SE +/- 0.06, N = 9SE +/- 0.05, N = 916.5424.1015.442.432.372.17

BlogBench

Test: Read

HDDSSDOpenBenchmarking.orgFinal Score, More Is BetterBlogBench 1.1Test: ReadEXT4F2FSXFS500K1000K1500K2000K2500KSE +/- 7597.92, N = 3SE +/- 5617.21, N = 3SE +/- 13393.45, N = 3SE +/- 1370.49, N = 3SE +/- 9217.94, N = 3SE +/- 15627.71, N = 32506299233394423181382308380242227922752651. (CC) gcc options: -O2 -pthread

PostgreSQL pgbench

Scaling: Buffer Test - Test: Normal Load - Mode: Read Write

HDDSSDOpenBenchmarking.orgTPS, More Is BetterPostgreSQL pgbench 10.3Scaling: Buffer Test - Test: Normal Load - Mode: Read WriteEXT4F2FSXFS400800120016002000SE +/- 7.75, N = 3SE +/- 21.37, N = 3SE +/- 14.97, N = 12SE +/- 95.92, N = 12SE +/- 53.24, N = 9SE +/- 77.95, N = 1262118186561887169718491. (CC) gcc options: -fno-strict-aliasing -fwrapv -O2 -lpgcommon -lpgport -lpq -lpthread -lrt -lcrypt -ldl -lm

Application Start-up Time

Background I/O Mix: Sequential Reads + Writes - Application To Start: GNOME Terminal - Disk Target: Default Test Directory

HDDSSDOpenBenchmarking.orgsec, Fewer Is BetterApplication Start-up Time 2.4.0Background I/O Mix: Sequential Reads + Writes - Application To Start: GNOME Terminal - Disk Target: Default Test DirectoryEXT4F2FSXFS1224364860SE +/- 0.90, N = 4SE +/- 0.55, N = 9SE +/- 0.34, N = 3SE +/- 0.02, N = 3SE +/- 0.04, N = 3SE +/- 0.03, N = 355.4736.2437.016.607.184.57

Application Start-up Time

Background I/O Mix: Sequential Reads + Writes - Application To Start: LibreOffice Writer - Disk Target: Default Test Directory

HDDSSDOpenBenchmarking.orgsec, Fewer Is BetterApplication Start-up Time 2.4.0Background I/O Mix: Sequential Reads + Writes - Application To Start: LibreOffice Writer - Disk Target: Default Test DirectoryEXT4F2FSXFS1632486480SE +/- 0.57, N = 3SE +/- 0.45, N = 9SE +/- 0.48, N = 3SE +/- 0.01, N = 3SE +/- 0.02, N = 3SE +/- 0.05, N = 371.5827.5738.205.075.423.18

Application Start-up Time

Background I/O Mix: Sequential Reads + Writes - Application To Start: xterm - Disk Target: Default Test Directory

HDDSSDOpenBenchmarking.orgsec, Fewer Is BetterApplication Start-up Time 2.4.0Background I/O Mix: Sequential Reads + Writes - Application To Start: xterm - Disk Target: Default Test DirectoryEXT4F2FSXFS3691215SE +/- 0.15, N = 9SE +/- 0.15, N = 3SE +/- 0.09, N = 3SE +/- 0.01, N = 3SE +/- 0.00, N = 3SE +/- 0.02, N = 98.3710.318.561.511.641.49

FS-Mark

Test: 4000 Files, 32 Sub Dirs, 1MB Size

HDDSSDOpenBenchmarking.orgFiles/s, More Is BetterFS-Mark 3.3Test: 4000 Files, 32 Sub Dirs, 1MB SizeEXT4F2FSXFS4080120160200SE +/- 0.45, N = 9SE +/- 0.09, N = 3SE +/- 0.50, N = 12SE +/- 0.37, N = 3SE +/- 0.03, N = 3SE +/- 0.67, N = 527.3148.5334.50188.2742.0748.021. (CC) gcc options: -static

Compile Bench

Test: Compile

HDDSSDOpenBenchmarking.orgMB/s, More Is BetterCompile Bench 0.6Test: CompileEXT4F2FSXFS400800120016002000SE +/- 13.47, N = 3SE +/- 14.49, N = 3SE +/- 24.89, N = 3SE +/- 21.25, N = 3SE +/- 20.39, N = 3SE +/- 11.13, N = 31314915189418241295871

SQLite

Timed SQLite Insertions

HDDSSDOpenBenchmarking.orgSeconds, Fewer Is BetterSQLite 3.22Timed SQLite InsertionsEXT4F2FSXFS60120180240300SE +/- 3.82, N = 3SE +/- 2.38, N = 3SE +/- 1.84, N = 3SE +/- 0.09, N = 3SE +/- 0.04, N = 3SE +/- 0.15, N = 3274.00289.00227.0039.8334.7541.811. (CC) gcc options: -O2 -ldl -lpthread

PostgreSQL pgbench

Scaling: Buffer Test - Test: Normal Load - Mode: Read Only

HDDSSDOpenBenchmarking.orgTPS, More Is BetterPostgreSQL pgbench 10.3Scaling: Buffer Test - Test: Normal Load - Mode: Read OnlyEXT4F2FSXFS70K140K210K280K350KSE +/- 1240.78, N = 3SE +/- 782.15, N = 3SE +/- 973.91, N = 3SE +/- 1316.47, N = 3SE +/- 1185.25, N = 3SE +/- 449.92, N = 33074223039783052563054513081773050221. (CC) gcc options: -fno-strict-aliasing -fwrapv -O2 -lpgcommon -lpgport -lpq -lpthread -lrt -lcrypt -ldl -lm

Flexible IO Tester

Type: Sequential Read - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory

HDDSSDOpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.1Type: Sequential Read - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test DirectoryEXT4F2FSXFS20K40K60K80K100KSE +/- 115.47, N = 3SE +/- 133.33, N = 3SE +/- 200.00, N = 3SE +/- 666.67, N = 3SE +/- 1560.11, N = 122940029167293001140001126671031751. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -U_FORTIFY_SOURCE -lgfapi -lglusterfs -lrt -laio -lm -lpthread -ldl

Flexible IO Tester

Type: Sequential Read - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory

HDDSSDOpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.1Type: Sequential Read - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test DirectoryEXT4F2FSXFS100200300400500SE +/- 0.58, N = 3SE +/- 0.58, N = 3SE +/- 0.67, N = 3SE +/- 3.51, N = 3SE +/- 5.97, N = 121151141154444404031. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -U_FORTIFY_SOURCE -lgfapi -lglusterfs -lrt -laio -lm -lpthread -ldl

Flexible IO Tester

Type: Random Read - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4MB - Disk Target: Default Test Directory

HDDSSDOpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.1Type: Random Read - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4MB - Disk Target: Default Test DirectoryEXT4F2FSXFS120240360480600SE +/- 1.50, N = 6SE +/- 0.67, N = 3SE +/- 0.58, N = 31151111095515525511. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -U_FORTIFY_SOURCE -lgfapi -lglusterfs -lrt -laio -lm -lpthread -ldl

Flexible IO Tester

Type: Random Read - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory

HDDSSDOpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.1Type: Random Read - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test DirectoryEXT4F2FSXFS80160240320400SE +/- 0.01, N = 3SE +/- 0.02, N = 3SE +/- 0.02, N = 32.142.352.28365.00365.00365.001. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -U_FORTIFY_SOURCE -lgfapi -lglusterfs -lrt -laio -lm -lpthread -ldl

Flexible IO Tester

Type: Random Read - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.1Type: Random Read - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test DirectoryXFSEXT4F2FS20K40K60K80K100KSE +/- 33.33, N = 3SE +/- 33.33, N = 3SE +/- 33.33, N = 39336793533934671. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -U_FORTIFY_SOURCE -lgfapi -lglusterfs -lrt -laio -lm -lpthread -ldl

Flexible IO Tester

Type: Random Write - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4MB - Disk Target: Default Test Directory

HDDSSDOpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.1Type: Random Write - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4MB - Disk Target: Default Test DirectoryEXT4F2FSXFS110220330440550SE +/- 0.88, N = 3SE +/- 0.33, N = 31171051065225235231. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -U_FORTIFY_SOURCE -lgfapi -lglusterfs -lrt -laio -lm -lpthread -ldl

Flexible IO Tester

Type: Sequential Write - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4MB - Disk Target: Default Test Directory

HDDSSDOpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.1Type: Sequential Write - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4MB - Disk Target: Default Test DirectoryEXT4F2FSXFS110220330440550SE +/- 0.58, N = 3SE +/- 0.33, N = 31431301255225235231. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -U_FORTIFY_SOURCE -lgfapi -lglusterfs -lrt -laio -lm -lpthread -ldl

Flexible IO Tester

Type: Sequential Read - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4MB - Disk Target: Default Test Directory

HDDSSDOpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.1Type: Sequential Read - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4MB - Disk Target: Default Test DirectoryEXT4F2FSXFS120240360480600SE +/- 1.15, N = 3SE +/- 0.33, N = 3SE +/- 2.40, N = 31471321265355524671. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -U_FORTIFY_SOURCE -lgfapi -lglusterfs -lrt -laio -lm -lpthread -ldl

Flexible IO Tester

Type: Random Write - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory

HDDSSDOpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.1Type: Random Write - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test DirectoryEXT4F2FSXFS80160240320400SE +/- 0.01, N = 3SE +/- 0.04, N = 3SE +/- 0.05, N = 3SE +/- 0.88, N = 3SE +/- 0.88, N = 32.412.432.36356.00357.00356.001. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -U_FORTIFY_SOURCE -lgfapi -lglusterfs -lrt -laio -lm -lpthread -ldl

Flexible IO Tester

Type: Sequential Write - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory

HDDSSDOpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.1Type: Sequential Write - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test DirectoryEXT4F2FSXFS20K40K60K80K100KSE +/- 33.33, N = 3SE +/- 666.67, N = 3SE +/- 881.92, N = 32623326100256001073331063331080001. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -U_FORTIFY_SOURCE -lgfapi -lglusterfs -lrt -laio -lm -lpthread -ldl

Flexible IO Tester

Type: Sequential Write - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory

HDDSSDOpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.1Type: Sequential Write - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test DirectoryEXT4F2FSXFS90180270360450SE +/- 0.33, N = 3SE +/- 2.19, N = 3SE +/- 3.93, N = 31031021004194154231. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -U_FORTIFY_SOURCE -lgfapi -lglusterfs -lrt -laio -lm -lpthread -ldl

Flexible IO Tester

Type: Random Write - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.1Type: Random Write - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test DirectoryXFSEXT4F2FS20K40K60K80K100KSE +/- 33.33, N = 3SE +/- 240.37, N = 3SE +/- 233.33, N = 39113391267911331. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -U_FORTIFY_SOURCE -lgfapi -lglusterfs -lrt -laio -lm -lpthread -ldl

FS-Mark

Test: 1000 Files, 1MB Size

HDDSSDOpenBenchmarking.orgFiles/s, More Is BetterFS-Mark 3.3Test: 1000 Files, 1MB SizeEXT4F2FSXFS50100150200250SE +/- 0.26, N = 3SE +/- 0.03, N = 3SE +/- 0.19, N = 3SE +/- 0.72, N = 3SE +/- 0.35, N = 3SE +/- 1.18, N = 327.4042.5332.67181.63170.40223.371. (CC) gcc options: -static

Compile Bench

Test: Initial Create

HDDSSDOpenBenchmarking.orgMB/s, More Is BetterCompile Bench 0.6Test: Initial CreateEXT4F2FSXFS110220330440550SE +/- 7.05, N = 3SE +/- 5.64, N = 3SE +/- 6.08, N = 3SE +/- 4.67, N = 3SE +/- 1.89, N = 3SE +/- 1.52, N = 3512126415430489104

BlogBench

Test: Write

HDDSSDOpenBenchmarking.orgFinal Score, More Is BetterBlogBench 1.1Test: WriteEXT4F2FSXFS16003200480064008000SE +/- 15.31, N = 3SE +/- 50.33, N = 3SE +/- 41.73, N = 3SE +/- 1099.95, N = 3SE +/- 122.35, N = 3SE +/- 186.00, N = 34932299626535667764331251. (CC) gcc options: -O2 -pthread

Phoronix Test Suite v10.8.4