Linux 5.6 IO Scheduler Benchmarks Linux 5.6 I/O scheduler benchmarks on NVMe SSD. Benchmarks by Michael Larabel for a future article. None: Processor: AMD Ryzen Threadripper 3970X 32-Core @ 3.70GHz (32 Cores / 64 Threads), Motherboard: ASUS ROG ZENITH II EXTREME (0702 BIOS), Chipset: AMD Starship/Matisse, Memory: 64GB, Disk: 1000GB Force MP600, Graphics: Gigabyte AMD Radeon RX 5500/5500M / Pro 5500M 8GB (1890/875MHz), Audio: AMD Navi 10 HDMI Audio, Monitor: ASUS MG28U, Network: Aquantia AQC107 NBase-T/IEEE + Intel I211 + Intel Wi-Fi 6 AX200 OS: Ubuntu 20.04, Kernel: 5.6.0-050600rc7-generic (x86_64) 20200322, Desktop: GNOME Shell 3.35.91, Display Server: X Server 1.20.7, Display Driver: modesetting 1.20.7, OpenGL: 4.6 Mesa 20.0.0 (LLVM 9.0.1), Compiler: GCC 9.3.0, File-System: ext4, Screen Resolution: 1280x800 MQ-Deadline: Processor: AMD Ryzen Threadripper 3970X 32-Core @ 3.70GHz (32 Cores / 64 Threads), Motherboard: ASUS ROG ZENITH II EXTREME (0702 BIOS), Chipset: AMD Starship/Matisse, Memory: 64GB, Disk: 1000GB Force MP600, Graphics: Gigabyte AMD Radeon RX 5500/5500M / Pro 5500M 8GB (1890/875MHz), Audio: AMD Navi 10 HDMI Audio, Monitor: ASUS MG28U, Network: Aquantia AQC107 NBase-T/IEEE + Intel I211 + Intel Wi-Fi 6 AX200 OS: Ubuntu 20.04, Kernel: 5.6.0-050600rc7-generic (x86_64) 20200322, Desktop: GNOME Shell 3.35.91, Display Server: X Server 1.20.7, Display Driver: modesetting 1.20.7, OpenGL: 4.6 Mesa 20.0.0 (LLVM 9.0.1), Compiler: GCC 9.3.0, File-System: ext4, Screen Resolution: 1280x800 Kyber: Processor: AMD Ryzen Threadripper 3970X 32-Core @ 3.70GHz (32 Cores / 64 Threads), Motherboard: ASUS ROG ZENITH II EXTREME (0702 BIOS), Chipset: AMD Starship/Matisse, Memory: 64GB, Disk: 1000GB Force MP600, Graphics: Gigabyte AMD Radeon RX 5500/5500M / Pro 5500M 8GB (1890/875MHz), Audio: AMD Navi 10 HDMI Audio, Monitor: ASUS MG28U, Network: Aquantia AQC107 NBase-T/IEEE + Intel I211 + Intel Wi-Fi 6 AX200 OS: Ubuntu 20.04, Kernel: 5.6.0-050600rc7-generic (x86_64) 20200322, Desktop: GNOME Shell 3.35.91, Display Server: X Server 1.20.7, Display Driver: modesetting 1.20.7, OpenGL: 4.6 Mesa 20.0.0 (LLVM 9.0.1), Compiler: GCC 9.3.0, File-System: ext4, Screen Resolution: 1280x800 BFQ low-latency: Processor: AMD Ryzen Threadripper 3970X 32-Core @ 3.70GHz (32 Cores / 64 Threads), Motherboard: ASUS ROG ZENITH II EXTREME (0702 BIOS), Chipset: AMD Starship/Matisse, Memory: 64GB, Disk: 1000GB Force MP600, Graphics: Gigabyte AMD Radeon RX 5500/5500M / Pro 5500M 8GB (1890/875MHz), Audio: AMD Navi 10 HDMI Audio, Monitor: ASUS MG28U, Network: Aquantia AQC107 NBase-T/IEEE + Intel I211 + Intel Wi-Fi 6 AX200 OS: Ubuntu 20.04, Kernel: 5.6.0-050600rc7-generic (x86_64) 20200322, Desktop: GNOME Shell 3.35.91, Display Server: X Server 1.20.7, Display Driver: modesetting 1.20.7, OpenGL: 4.6 Mesa 20.0.0 (LLVM 9.0.1), Compiler: GCC 9.3.0, File-System: ext4, Screen Resolution: 3840x2160 BFQ: Processor: AMD Ryzen Threadripper 3970X 32-Core @ 3.70GHz (32 Cores / 64 Threads), Motherboard: ASUS ROG ZENITH II EXTREME (0702 BIOS), Chipset: AMD Starship/Matisse, Memory: 64GB, Disk: 1000GB Force MP600, Graphics: Gigabyte AMD Radeon RX 5500/5500M / Pro 5500M 8GB (1890/875MHz), Audio: AMD Navi 10 HDMI Audio, Monitor: ASUS MG28U, Network: Aquantia AQC107 NBase-T/IEEE + Intel I211 + Intel Wi-Fi 6 AX200 OS: Ubuntu 20.04, Kernel: 5.6.0-050600rc7-generic (x86_64) 20200322, Desktop: GNOME Shell 3.35.91, Display Server: X Server 1.20.7, Display Driver: modesetting 1.20.7, OpenGL: 4.6 Mesa 20.0.0 (LLVM 9.0.1), Compiler: GCC 9.3.0, File-System: ext4, Screen Resolution: 1280x800 SQLite 3.30.1 Threads / Copies: 1 Seconds < Lower Is Better None ............ 14.42 |==================================================== MQ-Deadline ..... 14.39 |==================================================== Kyber ........... 15.08 |====================================================== BFQ low-latency . 14.72 |===================================================== BFQ ............. 14.60 |==================================================== SQLite 3.30.1 Threads / Copies: 8 Seconds < Lower Is Better None ............ 44.70 |================ MQ-Deadline ..... 65.07 |======================= Kyber ........... 142.94 |================================================== BFQ low-latency . 151.36 |===================================================== BFQ ............. 107.81 |====================================== Flexible IO Tester 3.18 Type: Random Read - Engine: IO_uring - Buffered: Yes - Direct: No - Block Size: 4KB - Disk Target: Default Test Directory MB/s > Higher Is Better None ............ 1381 |======================================================= MQ-Deadline ..... 1119 |============================================= Kyber ........... 1114 |============================================ BFQ low-latency . 787 |=============================== BFQ ............. 800 |================================ Flexible IO Tester 3.18 Type: Random Read - Engine: IO_uring - Buffered: Yes - Direct: No - Block Size: 4KB - Disk Target: Default Test Directory IOPS > Higher Is Better None ............ 353667 |===================================================== MQ-Deadline ..... 286333 |=========================================== Kyber ........... 285167 |=========================================== BFQ low-latency . 201600 |============================== BFQ ............. 204500 |=============================== Flexible IO Tester 3.18 Type: Random Write - Engine: IO_uring - Buffered: Yes - Direct: No - Block Size: 4KB - Disk Target: Default Test Directory MB/s > Higher Is Better None ............ 979 |======================================================== MQ-Deadline ..... 975 |======================================================== Kyber ........... 883 |=================================================== BFQ low-latency . 940 |====================================================== BFQ ............. 960 |======================================================= Flexible IO Tester 3.18 Type: Random Write - Engine: IO_uring - Buffered: Yes - Direct: No - Block Size: 4KB - Disk Target: Default Test Directory IOPS > Higher Is Better None ............ 250667 |===================================================== MQ-Deadline ..... 249467 |===================================================== Kyber ........... 226142 |================================================ BFQ low-latency . 240750 |=================================================== BFQ ............. 245750 |==================================================== Flexible IO Tester 3.18 Type: Sequential Read - Engine: IO_uring - Buffered: Yes - Direct: No - Block Size: 2MB - Disk Target: Default Test Directory MB/s > Higher Is Better None ............ 3825 |==================================================== MQ-Deadline ..... 3967 |====================================================== Kyber ........... 4021 |======================================================= BFQ low-latency . 3792 |==================================================== BFQ ............. 4009 |======================================================= Flexible IO Tester 3.18 Type: Sequential Read - Engine: IO_uring - Buffered: Yes - Direct: No - Block Size: 2MB - Disk Target: Default Test Directory IOPS > Higher Is Better None ............ 1909 |==================================================== MQ-Deadline ..... 1980 |====================================================== Kyber ........... 2007 |======================================================= BFQ low-latency . 1893 |==================================================== BFQ ............. 2001 |======================================================= Flexible IO Tester 3.18 Type: Sequential Read - Engine: IO_uring - Buffered: Yes - Direct: No - Block Size: 4KB - Disk Target: Default Test Directory MB/s > Higher Is Better None ............ 1483 |========================================== MQ-Deadline ..... 1665 |================================================ Kyber ........... 1927 |======================================================= BFQ low-latency . 1373 |======================================= BFQ ............. 1921 |======================================================= Flexible IO Tester 3.18 Type: Sequential Read - Engine: IO_uring - Buffered: Yes - Direct: No - Block Size: 4KB - Disk Target: Default Test Directory IOPS > Higher Is Better None ............ 379333 |========================================= MQ-Deadline ..... 426000 |============================================== Kyber ........... 493333 |===================================================== BFQ low-latency . 351333 |====================================== BFQ ............. 492000 |===================================================== Flexible IO Tester 3.18 Type: Sequential Write - Engine: IO_uring - Buffered: Yes - Direct: No - Block Size: 2MB - Disk Target: Default Test Directory MB/s > Higher Is Better None ............ 1979 |======================================================= MQ-Deadline ..... 1701 |=============================================== Kyber ........... 1776 |================================================= BFQ low-latency . 1378 |====================================== BFQ ............. 1414 |======================================= Flexible IO Tester 3.18 Type: Sequential Write - Engine: IO_uring - Buffered: Yes - Direct: No - Block Size: 2MB - Disk Target: Default Test Directory IOPS > Higher Is Better None ............ 986 |======================================================== MQ-Deadline ..... 847 |================================================ Kyber ........... 884 |================================================== BFQ low-latency . 686 |======================================= BFQ ............. 703 |======================================== Flexible IO Tester 3.18 Type: Sequential Write - Engine: IO_uring - Buffered: Yes - Direct: No - Block Size: 4KB - Disk Target: Default Test Directory MB/s > Higher Is Better None ............ 1184 |======================================================= MQ-Deadline ..... 1166 |====================================================== Kyber ........... 1155 |====================================================== BFQ low-latency . 1015 |=============================================== BFQ ............. 1027 |================================================ Flexible IO Tester 3.18 Type: Sequential Write - Engine: IO_uring - Buffered: Yes - Direct: No - Block Size: 4KB - Disk Target: Default Test Directory IOPS > Higher Is Better None ............ 303000 |===================================================== MQ-Deadline ..... 298667 |==================================================== Kyber ........... 296000 |==================================================== BFQ low-latency . 260000 |============================================= BFQ ............. 263000 |============================================== FS-Mark 3.3 Test: 5000 Files, 1MB Size, 4 Threads Files/s > Higher Is Better None ............ 1238.2 |===================================================== MQ-Deadline ..... 773.5 |================================= Kyber ........... 593.5 |========================= BFQ low-latency . 647.2 |============================ BFQ ............. 364.5 |================ FS-Mark 3.3 Test: 4000 Files, 32 Sub Dirs, 1MB Size Files/s > Higher Is Better None ............ 592.3 |====================================================== MQ-Deadline ..... 591.4 |====================================================== Kyber ........... 545.8 |================================================== BFQ low-latency . 590.6 |====================================================== BFQ ............. 528.4 |================================================ IOR 3.2.1 Write Test MB/s > Higher Is Better None ............ 797.73 |===================================================== MQ-Deadline ..... 794.35 |===================================================== Kyber ........... 781.43 |==================================================== BFQ low-latency . 799.22 |===================================================== BFQ ............. 770.48 |=================================================== IOR 3.2.1 Read Test MB/s > Higher Is Better None ............ 995.19 |===================================================== MQ-Deadline ..... 977.23 |==================================================== Kyber ........... 977.87 |==================================================== BFQ low-latency . 978.41 |==================================================== BFQ ............. 986.32 |===================================================== Timed GCC Compilation 9.3.0 Time To Compile Seconds < Lower Is Better None ............ 641.99 |===================================================== MQ-Deadline ..... 641.41 |===================================================== Kyber ........... 642.18 |===================================================== BFQ low-latency . 642.19 |===================================================== BFQ ............. 642.04 |===================================================== Timed GDB GNU Debugger Compilation 9.1 Time To Compile Seconds < Lower Is Better None ............ 80.41 |====================================================== MQ-Deadline ..... 80.25 |====================================================== Kyber ........... 80.36 |====================================================== BFQ low-latency . 80.89 |====================================================== BFQ ............. 80.80 |====================================================== LevelDB 1.22 Benchmark: Hot Read Microseconds Per Op < Lower Is Better None ............ 40.96 |====================================================== MQ-Deadline ..... 41.02 |====================================================== Kyber ........... 40.89 |====================================================== BFQ low-latency . 41.05 |====================================================== BFQ ............. 41.06 |====================================================== LevelDB 1.22 Benchmark: Fill Sync MB/s > Higher Is Better None ............ 3.6 |======================================================== MQ-Deadline ..... 3.6 |======================================================== Kyber ........... 3.6 |======================================================== BFQ low-latency . 3.5 |====================================================== BFQ ............. 3.5 |====================================================== LevelDB 1.22 Benchmark: Fill Sync Microseconds Per Op < Lower Is Better None ............ 1954.38 |================================================== MQ-Deadline ..... 1966.91 |=================================================== Kyber ........... 1967.16 |=================================================== BFQ low-latency . 1998.44 |=================================================== BFQ ............. 2019.67 |==================================================== LevelDB 1.22 Benchmark: Overwrite MB/s > Higher Is Better None ............ 26.7 |======================================================= MQ-Deadline ..... 26.7 |======================================================= Kyber ........... 26.6 |======================================================= BFQ low-latency . 26.6 |======================================================= BFQ ............. 26.8 |======================================================= LevelDB 1.22 Benchmark: Overwrite Microseconds Per Op < Lower Is Better None ............ 264.66 |===================================================== MQ-Deadline ..... 265.35 |===================================================== Kyber ........... 265.72 |===================================================== BFQ low-latency . 265.55 |===================================================== BFQ ............. 264.25 |===================================================== LevelDB 1.22 Benchmark: Random Fill MB/s > Higher Is Better None ............ 26.7 |======================================================= MQ-Deadline ..... 26.6 |======================================================= Kyber ........... 26.6 |======================================================= BFQ low-latency . 26.7 |======================================================= BFQ ............. 26.7 |======================================================= LevelDB 1.22 Benchmark: Random Fill Microseconds Per Op < Lower Is Better None ............ 265.44 |===================================================== MQ-Deadline ..... 265.98 |===================================================== Kyber ........... 265.72 |===================================================== BFQ low-latency . 264.87 |===================================================== BFQ ............. 265.21 |===================================================== LevelDB 1.22 Benchmark: Random Read Microseconds Per Op < Lower Is Better None ............ 41.27 |===================================================== MQ-Deadline ..... 41.30 |===================================================== Kyber ........... 41.05 |===================================================== BFQ low-latency . 40.81 |===================================================== BFQ ............. 41.82 |====================================================== LevelDB 1.22 Benchmark: Seek Random Microseconds Per Op < Lower Is Better None ............ 66.58 |====================================================== MQ-Deadline ..... 66.89 |====================================================== Kyber ........... 66.61 |====================================================== BFQ low-latency . 66.23 |===================================================== BFQ ............. 66.34 |====================================================== LevelDB 1.22 Benchmark: Random Delete Microseconds Per Op < Lower Is Better None ............ 256.30 |===================================================== MQ-Deadline ..... 257.51 |===================================================== Kyber ........... 257.43 |===================================================== BFQ low-latency . 257.17 |===================================================== BFQ ............. 258.15 |===================================================== LevelDB 1.22 Benchmark: Sequential Fill MB/s > Higher Is Better None ............ 27.5 |======================================================= MQ-Deadline ..... 27.4 |======================================================= Kyber ........... 27.6 |======================================================= BFQ low-latency . 27.5 |======================================================= BFQ ............. 27.5 |======================================================= LevelDB 1.22 Benchmark: Sequential Fill Microseconds Per Op < Lower Is Better None ............ 257.33 |===================================================== MQ-Deadline ..... 257.85 |===================================================== Kyber ........... 256.08 |===================================================== BFQ low-latency . 256.94 |===================================================== BFQ ............. 257.01 |===================================================== KeyDB 5.3.1 Ops/sec > Higher Is Better None ............ 519045.76 |================================================= MQ-Deadline ..... 522808.47 |================================================== Kyber ........... 527545.50 |================================================== BFQ low-latency . 525133.31 |================================================== BFQ ............. 522119.62 |================================================= MariaDB 10.3.8 Clients: 64 Queries Per Second > Higher Is Better None ............ 1134 |======================================================= MQ-Deadline ..... 327 |================ Kyber ........... 217 |=========== BFQ low-latency . 318 |=============== BFQ ............. 207 |========== PostgreSQL pgbench 12.0 Scaling: Buffer Test - Test: Normal Load - Mode: Read Write TPS > Higher Is Better None ............ 34866.08 |=================================================== MQ-Deadline ..... 12530.75 |================== Kyber ........... 12539.32 |================== BFQ low-latency . 12441.08 |================== BFQ ............. 12284.35 |================== Apache Cassandra 3.11.4 Test: Writes Op/s > Higher Is Better None ............ 291627 |===================================================== MQ-Deadline ..... 271238 |================================================= Kyber ........... 279252 |=================================================== BFQ low-latency . 273739 |================================================== BFQ ............. 282883 |=================================================== Apache Cassandra 3.11.4 Test: Mixed 1:1 Op/s > Higher Is Better None ............ 3753 |======================================================= MQ-Deadline ..... 226 |=== Kyber ........... 165 |== BFQ low-latency . 141 |== BFQ ............. 137 |== Facebook RocksDB 6.3.6 Test: Random Fill Op/s > Higher Is Better None ............ 1205145 |==================================================== MQ-Deadline ..... 1185563 |=================================================== Kyber ........... 1193020 |=================================================== BFQ low-latency . 1188747 |=================================================== BFQ ............. 1178558 |=================================================== Facebook RocksDB 6.3.6 Test: Random Read Op/s > Higher Is Better None ............ 181322024 |================================================= MQ-Deadline ..... 179996909 |================================================= Kyber ........... 183324454 |================================================== BFQ low-latency . 180517371 |================================================= BFQ ............. 184232888 |================================================== Facebook RocksDB 6.3.6 Test: Sequential Fill Op/s > Higher Is Better None ............ 1235558 |==================================================== MQ-Deadline ..... 1237822 |==================================================== Kyber ........... 1216032 |=================================================== BFQ low-latency . 1220974 |=================================================== BFQ ............. 1225970 |==================================================== Facebook RocksDB 6.3.6 Test: Random Fill Sync Op/s > Higher Is Better None ............ 34082 |====================================================== MQ-Deadline ..... 31209 |================================================= Kyber ........... 33715 |===================================================== BFQ low-latency . 29761 |=============================================== BFQ ............. 23275 |===================================== Facebook RocksDB 6.3.6 Test: Read While Writing Op/s > Higher Is Better None ............ 6164535 |=================================================== MQ-Deadline ..... 6184288 |=================================================== Kyber ........... 6131530 |=================================================== BFQ low-latency . 6116190 |=================================================== BFQ ............. 6294210 |==================================================== Apache HBase 2.2.3 Test: Increment - Clients: 32 Rows Per Second > Higher Is Better None ............ 110680 |===================================================== MQ-Deadline ..... 110917 |===================================================== Kyber ........... 111460 |===================================================== BFQ low-latency . 110196 |==================================================== BFQ ............. 111730 |===================================================== Apache HBase 2.2.3 Test: Increment - Clients: 32 Microseconds - Average Latency < Lower Is Better None ............ 287 |======================================================== MQ-Deadline ..... 286 |======================================================== Kyber ........... 284 |======================================================= BFQ low-latency . 288 |======================================================== BFQ ............. 284 |======================================================= Apache HBase 2.2.3 Test: Random Read - Clients: 32 Rows Per Second > Higher Is Better None ............ 239965 |===================================================== MQ-Deadline ..... 239244 |===================================================== Kyber ........... 240821 |===================================================== BFQ low-latency . 241114 |===================================================== BFQ ............. 238769 |==================================================== Apache HBase 2.2.3 Test: Random Read - Clients: 32 Microseconds - Average Latency < Lower Is Better None ............ 131 |======================================================== MQ-Deadline ..... 131 |======================================================== Kyber ........... 130 |======================================================= BFQ low-latency . 130 |======================================================= BFQ ............. 132 |======================================================== Apache HBase 2.2.3 Test: Sequential Read - Clients: 32 Rows Per Second > Higher Is Better None ............ 185345 |===================================================== MQ-Deadline ..... 170160 |================================================= Kyber ........... 166203 |================================================ BFQ low-latency . 177460 |=================================================== BFQ ............. 173354 |================================================== Apache HBase 2.2.3 Test: Sequential Read - Clients: 32 Microseconds - Average Latency < Lower Is Better None ............ 172 |================================================== MQ-Deadline ..... 187 |======================================================= Kyber ........... 191 |======================================================== BFQ low-latency . 179 |==================================================== BFQ ............. 184 |====================================================== Apache HBase 2.2.3 Test: Async Random Read - Clients: 32 Rows Per Second > Higher Is Better None ............ 167371 |================================================== MQ-Deadline ..... 166000 |================================================== Kyber ........... 175746 |===================================================== BFQ low-latency . 162581 |================================================= BFQ ............. 163602 |================================================= Apache HBase 2.2.3 Test: Async Random Read - Clients: 32 Microseconds - Average Latency < Lower Is Better None ............ 188 |====================================================== MQ-Deadline ..... 190 |======================================================= Kyber ........... 179 |==================================================== BFQ low-latency . 194 |======================================================== BFQ ............. 193 |======================================================== Apache HBase 2.2.3 Test: Async Random Write - Clients: 32 Rows Per Second > Higher Is Better None ............ 61372 |==================================================== MQ-Deadline ..... 63706 |====================================================== Kyber ........... 60197 |=================================================== BFQ low-latency . 46461 |======================================= BFQ ............. 62761 |===================================================== Apache HBase 2.2.3 Test: Async Random Write - Clients: 32 Microseconds - Average Latency < Lower Is Better None ............ 522 |========================================== MQ-Deadline ..... 501 |======================================== Kyber ........... 535 |=========================================== BFQ low-latency . 695 |======================================================== BFQ ............. 508 |=========================================