Linux 5.14 SSD File-System Benchmarks After Changes F2FS: Processor: AMD Ryzen 9 5900X 12-Core @ 3.70GHz (12 Cores / 24 Threads), Motherboard: ASUS ROG CROSSHAIR VIII HERO (3501 BIOS), Chipset: AMD Starship/Matisse, Memory: 16GB, Disk: 1000GB Western Digital WD_BLACK SN850 1TB + 1000GB Western Digital WDS100T1X0E-00AFY0, Graphics: AMD DIMGREY_CAVEFISH 8GB (2900/1000MHz), Audio: AMD Device ab28, Monitor: ASUS VP28U, Network: Realtek RTL8125 2.5GbE + Intel I211 OS: Ubuntu 20.04, Kernel: 5.14.0-051400rc7-generic (x86_64) 20210822, Desktop: GNOME Shell 3.36.9, Display Server: X Server 1.20.11, OpenGL: 4.6 Mesa 21.3.0-devel (git-2b4b310 2021-08-21 focal-oibaf-ppa) (LLVM 12.0.1), Vulkan: 1.2.185, Compiler: GCC 9.3.0, File-System: f2fs, Screen Resolution: 3840x2160 Btrfs: Processor: AMD Ryzen 9 5900X 12-Core @ 3.70GHz (12 Cores / 24 Threads), Motherboard: ASUS ROG CROSSHAIR VIII HERO (3501 BIOS), Chipset: AMD Starship/Matisse, Memory: 16GB, Disk: 1000GB Western Digital WD_BLACK SN850 1TB + 1000GB Western Digital WDS100T1X0E-00AFY0, Graphics: AMD DIMGREY_CAVEFISH 8GB (2900/1000MHz), Audio: AMD Device ab28, Monitor: ASUS VP28U, Network: Realtek RTL8125 2.5GbE + Intel I211 OS: Ubuntu 20.04, Kernel: 5.14.0-051400rc7-generic (x86_64) 20210822, Desktop: GNOME Shell 3.36.9, Display Server: X Server 1.20.11, OpenGL: 4.6 Mesa 21.3.0-devel (git-2b4b310 2021-08-21 focal-oibaf-ppa) (LLVM 12.0.1), Vulkan: 1.2.185, Compiler: GCC 9.3.0, File-System: btrfs, Screen Resolution: 3840x2160 EXT4: Processor: AMD Ryzen 9 5900X 12-Core @ 3.70GHz (12 Cores / 24 Threads), Motherboard: ASUS ROG CROSSHAIR VIII HERO (3501 BIOS), Chipset: AMD Starship/Matisse, Memory: 16GB, Disk: 1000GB Western Digital WD_BLACK SN850 1TB + 1000GB Western Digital WDS100T1X0E-00AFY0, Graphics: AMD DIMGREY_CAVEFISH 8GB (2900/1000MHz), Audio: AMD Device ab28, Monitor: ASUS VP28U, Network: Realtek RTL8125 2.5GbE + Intel I211 OS: Ubuntu 20.04, Kernel: 5.14.0-051400rc7-generic (x86_64) 20210822, Desktop: GNOME Shell 3.36.9, Display Server: X Server 1.20.11, OpenGL: 4.6 Mesa 21.3.0-devel (git-2b4b310 2021-08-21 focal-oibaf-ppa) (LLVM 12.0.1), Vulkan: 1.2.185, Compiler: GCC 9.3.0, File-System: ext4, Screen Resolution: 3840x2160 XFS: Processor: AMD Ryzen 9 5900X 12-Core @ 3.70GHz (12 Cores / 24 Threads), Motherboard: ASUS ROG CROSSHAIR VIII HERO (3501 BIOS), Chipset: AMD Starship/Matisse, Memory: 16GB, Disk: 1000GB Western Digital WD_BLACK SN850 1TB + 1000GB Western Digital WDS100T1X0E-00AFY0, Graphics: AMD DIMGREY_CAVEFISH 8GB (2900/1000MHz), Audio: AMD Device ab28, Monitor: ASUS VP28U, Network: Realtek RTL8125 2.5GbE + Intel I211 OS: Ubuntu 20.04, Kernel: 5.14.0-051400rc7-generic (x86_64) 20210822, Desktop: GNOME Shell 3.36.9, Display Server: X Server 1.20.11, OpenGL: 4.6 Mesa 21.3.0-devel (git-2b4b310 2021-08-21 focal-oibaf-ppa) (LLVM 12.0.1), Vulkan: 1.2.185, Compiler: GCC 9.3.0, File-System: xfs, Screen Resolution: 3840x2160 ext4 - 4 x 4001GB CT4000P3PSSD8: Processor: AMD Ryzen 7 1700X Eight-Core @ 3.40GHz (8 Cores / 16 Threads), Motherboard: ASUS PRIME X470-PRO (6202 BIOS), Chipset: AMD 17h, Memory: 128GB, Disk: 4 x 4001GB CT4000P3PSSD8 + 500GB Samsung SSD 860 + 4001GB Western Digital WD4000F9YZ-0 + 2 x 4001GB HGST HUS726T4TAL + 4001GB Western Digital WD4002FYYZ-0, Graphics: MSI NVIDIA GeForce GT 710 978MB, Audio: NVIDIA GK208 HDMI/DP, Network: Mellanox MT27520 + Intel I211 OS: Ubuntu 22.04, Kernel: 6.2.0-39-generic (x86_64), Display Server: X Server 1.19.6, Display Driver: NVIDIA, Vulkan: 1.0.65, Compiler: GCC 11.4.0, File-System: ext4, Screen Resolution: 1024x768 Tests with bad nvme sector size 512 and bad zfs block size: Processor: AMD Ryzen 7 1700X Eight-Core @ 3.40GHz (8 Cores / 16 Threads), Motherboard: ASUS PRIME X470-PRO (6202 BIOS), Chipset: AMD 17h, Memory: 128GB, Disk: 4 x 4001GB CT4000P3PSSD8 + 500GB Samsung SSD 860 + 4001GB Western Digital WD4000F9YZ-0 + 2 x 4001GB HGST HUS726T4TAL + 4001GB Western Digital WD4002FYYZ-0, Graphics: MSI NVIDIA GeForce GT 710 978MB, Audio: NVIDIA GK208 HDMI/DP, Network: Mellanox MT27520 + Intel I211 OS: Ubuntu 22.04, Kernel: 6.2.0-39-generic (x86_64), Display Server: X Server 1.19.6, Display Driver: NVIDIA, Vulkan: 1.0.65, Compiler: GCC 11.4.0, File-System: zfs, Screen Resolution: 1024x768 Tests with low-level reformatted nvme 4k sectors and new zfs pool ashift=12: Processor: AMD Ryzen 7 1700X Eight-Core @ 3.40GHz (8 Cores / 16 Threads), Motherboard: ASUS PRIME X470-PRO (6202 BIOS), Chipset: AMD 17h, Memory: 128GB, Disk: 4 x 4001GB CT4000P3PSSD8 + 500GB Samsung SSD 860 + 4001GB Western Digital WD4000F9YZ-0 + 2 x 4001GB HGST HUS726T4TAL + 4001GB Western Digital WD4002FYYZ-0, Graphics: MSI NVIDIA GeForce GT 710 979MB, Audio: NVIDIA GK208 HDMI/DP, Network: Mellanox MT27520 + Intel I211 OS: Ubuntu 22.04, Kernel: 6.2.0-39-generic (x86_64), Display Server: X Server 1.19.6, Display Driver: NVIDIA, Vulkan: 1.0.65, Compiler: GCC 11.4.0, File-System: zfs, Screen Resolution: 800x600 SQLite 3.30.1 Threads / Copies: 1 Seconds < Lower Is Better F2FS ........................................................................ 9.127 | Btrfs ....................................................................... 13.385 | EXT4 ........................................................................ 7.221 | XFS ......................................................................... 8.274 | Tests with bad nvme sector size 512 and bad zfs block size .................. 222.729 | Tests with low-level reformatted nvme 4k sectors and new zfs pool ashift=12 . 98.706 | SQLite 3.30.1 Threads / Copies: 8 Seconds < Lower Is Better F2FS ........................................................................ 18.17 | Btrfs ....................................................................... 24.35 | EXT4 ........................................................................ 28.78 | XFS ......................................................................... 38.91 | Tests with bad nvme sector size 512 and bad zfs block size .................. 472.04 | Tests with low-level reformatted nvme 4k sectors and new zfs pool ashift=12 . 138.89 | Flexible IO Tester 3.25 Type: Random Read - Engine: IO_uring - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory IOPS > Higher Is Better F2FS ........................................................................ 753667 | Btrfs ....................................................................... 554333 | EXT4 ........................................................................ 760000 | XFS ......................................................................... 755667 | Flexible IO Tester 3.25 Type: Random Write - Engine: IO_uring - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory IOPS > Higher Is Better F2FS ........................................................................ 285000 | Btrfs ....................................................................... 143333 | EXT4 ........................................................................ 712000 | XFS ......................................................................... 715667 | Flexible IO Tester 3.25 Type: Sequential Read - Engine: IO_uring - Buffered: No - Direct: Yes - Block Size: 2MB - Disk Target: Default Test Directory MB/s > Higher Is Better F2FS ........................................................................ 3591 | Btrfs ....................................................................... 3253 | EXT4 ........................................................................ 3591 | XFS ......................................................................... 3593 | Flexible IO Tester 3.25 Type: Sequential Write - Engine: IO_uring - Buffered: No - Direct: Yes - Block Size: 2MB - Disk Target: Default Test Directory IOPS > Higher Is Better F2FS ........................................................................ 1635 | Btrfs ....................................................................... 1633 | EXT4 ........................................................................ 1634 | XFS ......................................................................... 1635 | FS-Mark 3.3 Test: 1000 Files, 1MB Size Files/s > Higher Is Better F2FS ........................................................................ 1190.5 | Btrfs ....................................................................... 872.2 | EXT4 ........................................................................ 843.0 | XFS ......................................................................... 827.2 | Tests with bad nvme sector size 512 and bad zfs block size .................. 55.9 | Tests with low-level reformatted nvme 4k sectors and new zfs pool ashift=12 . 199.2 | FS-Mark 3.3 Test: 5000 Files, 1MB Size, 4 Threads Files/s > Higher Is Better F2FS ........................................................................ 2491.7 | Btrfs ....................................................................... 1841.0 | EXT4 ........................................................................ 1798.9 | XFS ......................................................................... 1774.5 | Tests with bad nvme sector size 512 and bad zfs block size .................. 50.1 | Tests with low-level reformatted nvme 4k sectors and new zfs pool ashift=12 . 384.2 | FS-Mark 3.3 Test: 4000 Files, 32 Sub Dirs, 1MB Size Files/s > Higher Is Better F2FS ........................................................................ 1193.5 | Btrfs ....................................................................... 859.2 | EXT4 ........................................................................ 838.5 | XFS ......................................................................... 829.4 | Tests with bad nvme sector size 512 and bad zfs block size .................. 62.5 | Tests with low-level reformatted nvme 4k sectors and new zfs pool ashift=12 . 201.5 | Dbench 4.0 Client Count: 6 MB/s > Higher Is Better F2FS ........................................................................ 3234.57 | Btrfs ....................................................................... 2005.65 | EXT4 ........................................................................ 2474.15 | XFS ......................................................................... 1751.81 | Tests with bad nvme sector size 512 and bad zfs block size .................. 87.04 | Tests with low-level reformatted nvme 4k sectors and new zfs pool ashift=12 . 310.37 | IOR 3.3.0 Block Size: 16MB - Disk Target: Default Test Directory MB/s > Higher Is Better F2FS ........................................................................ 2291.22 | Btrfs ....................................................................... 2291.05 | EXT4 ........................................................................ 2091.14 | XFS ......................................................................... 2445.03 | Tests with bad nvme sector size 512 and bad zfs block size .................. 1254.95 | Tests with low-level reformatted nvme 4k sectors and new zfs pool ashift=12 . 2847.57 | MariaDB 10.6.4 Clients: 128 Queries Per Second > Higher Is Better F2FS ........................................................................ 1081 | Btrfs ....................................................................... 856 | EXT4 ........................................................................ 1037 | XFS ......................................................................... 1025 | MariaDB 10.6.4 Clients: 256 Queries Per Second > Higher Is Better F2FS ........................................................................ 841 | Btrfs ....................................................................... 725 | EXT4 ........................................................................ 791 | XFS ......................................................................... 799 | MariaDB 10.6.4 Clients: 512 Queries Per Second > Higher Is Better F2FS ........................................................................ 779 | Btrfs ....................................................................... 706 | EXT4 ........................................................................ 748 | XFS ......................................................................... 751 | PostgreSQL pgbench 13.0 Scaling Factor: 100 - Clients: 50 - Mode: Read Write TPS > Higher Is Better F2FS ........................................................................ 41083 | Btrfs ....................................................................... 24356 | EXT4 ........................................................................ 41200 | XFS ......................................................................... 41646 | Tests with bad nvme sector size 512 and bad zfs block size .................. 539 | Tests with low-level reformatted nvme 4k sectors and new zfs pool ashift=12 . 5521 | PostgreSQL pgbench 13.0 Scaling Factor: 100 - Clients: 50 - Mode: Read Write - Average Latency ms < Lower Is Better F2FS ........................................................................ 1.217 | Btrfs ....................................................................... 2.053 | EXT4 ........................................................................ 1.214 | XFS ......................................................................... 1.201 | Tests with bad nvme sector size 512 and bad zfs block size .................. 105.739 | Tests with low-level reformatted nvme 4k sectors and new zfs pool ashift=12 . 9.057 | PostgreSQL pgbench 13.0 Scaling Factor: 1000 - Clients: 50 - Mode: Read Only TPS > Higher Is Better F2FS ........................................................................ 291011 | Btrfs ....................................................................... 311379 | EXT4 ........................................................................ 338599 | XFS ......................................................................... 340083 | Tests with bad nvme sector size 512 and bad zfs block size .................. 39405 | Tests with low-level reformatted nvme 4k sectors and new zfs pool ashift=12 . 37664 | PostgreSQL pgbench 13.0 Scaling Factor: 1000 - Clients: 50 - Mode: Read Only - Average Latency ms < Lower Is Better F2FS ........................................................................ 0.172 | Btrfs ....................................................................... 0.161 | EXT4 ........................................................................ 0.148 | XFS ......................................................................... 0.147 | Tests with bad nvme sector size 512 and bad zfs block size .................. 1.269 | Tests with low-level reformatted nvme 4k sectors and new zfs pool ashift=12 . 1.327 | PostgreSQL pgbench 13.0 Scaling Factor: 1000 - Clients: 50 - Mode: Read Write TPS > Higher Is Better F2FS ........................................................................ 34439 | Btrfs ....................................................................... 20361 | EXT4 ........................................................................ 29455 | XFS ......................................................................... 37059 | Tests with bad nvme sector size 512 and bad zfs block size .................. 468 | Tests with low-level reformatted nvme 4k sectors and new zfs pool ashift=12 . 5119 | PostgreSQL pgbench 13.0 Scaling Factor: 1000 - Clients: 50 - Mode: Read Write - Average Latency ms < Lower Is Better F2FS ........................................................................ 1.452 | Btrfs ....................................................................... 2.456 | EXT4 ........................................................................ 1.760 | XFS ......................................................................... 1.350 | Tests with bad nvme sector size 512 and bad zfs block size .................. 113.799 | Tests with low-level reformatted nvme 4k sectors and new zfs pool ashift=12 . 9.769 |