Linux 4.17 IO Scheduler Testing Tests by Michael Larabel for a future article. HDD: BFQ: Processor: 2 x Intel Xeon Gold 6138 @ 3.70GHz (40 Cores / 80 Threads), Motherboard: TYAN S7106 (V1.00 BIOS), Chipset: Intel Sky Lake-E DMI3 Registers, Memory: 12 x 8192 MB DDR4-2666MT/s Micron 9ASF1G72PZ-2G6B1, Disk: 256GB Samsung SSD 850 + 2000GB Seagate ST2000DM006-2DM1 + 2 x 120GB TOSHIBA-TR150, Graphics: ASPEED ASPEED Family, Monitor: VE228, Network: Intel I210 Gigabit Connection OS: Ubuntu 18.04, Kernel: 4.17.0-999-generic (x86_64) 20180529, Desktop: GNOME Shell 3.28.1, Compiler: GCC 7.3.0, File-System: ext4, Screen Resolution: 1920x1080 HDD: BFQ low_latency: Processor: 2 x Intel Xeon Gold 6138 @ 3.70GHz (40 Cores / 80 Threads), Motherboard: TYAN S7106 (V1.00 BIOS), Chipset: Intel Sky Lake-E DMI3 Registers, Memory: 12 x 8192 MB DDR4-2666MT/s Micron 9ASF1G72PZ-2G6B1, Disk: 256GB Samsung SSD 850 + 2000GB Seagate ST2000DM006-2DM1 + 2 x 120GB TOSHIBA-TR150, Graphics: ASPEED ASPEED Family, Monitor: VE228, Network: Intel I210 Gigabit Connection OS: Ubuntu 18.04, Kernel: 4.17.0-999-generic (x86_64) 20180529, Desktop: GNOME Shell 3.28.1, Compiler: GCC 7.3.0, File-System: ext4, Screen Resolution: 1920x1080 HDD: CFQ: Processor: 2 x Intel Xeon Gold 6138 @ 3.70GHz (40 Cores / 80 Threads), Motherboard: TYAN S7106 (V1.00 BIOS), Chipset: Intel Sky Lake-E DMI3 Registers, Memory: 12 x 8192 MB DDR4-2666MT/s Micron 9ASF1G72PZ-2G6B1, Disk: 256GB Samsung SSD 850 + 2000GB Seagate ST2000DM006-2DM1 + 2 x 120GB TOSHIBA-TR150, Graphics: ASPEED ASPEED Family, Monitor: VE228, Network: Intel I210 Gigabit Connection OS: Ubuntu 18.04, Kernel: 4.17.0-999-generic (x86_64) 20180529, Desktop: GNOME Shell 3.28.1, Compiler: GCC 7.3.0, File-System: ext4, Screen Resolution: 1920x1080 HDD: DEADLINE: Processor: 2 x Intel Xeon Gold 6138 @ 3.70GHz (40 Cores / 80 Threads), Motherboard: TYAN S7106 (V1.00 BIOS), Chipset: Intel Sky Lake-E DMI3 Registers, Memory: 12 x 8192 MB DDR4-2666MT/s Micron 9ASF1G72PZ-2G6B1, Disk: 256GB Samsung SSD 850 + 2000GB Seagate ST2000DM006-2DM1 + 2 x 120GB TOSHIBA-TR150, Graphics: ASPEED ASPEED Family, Monitor: VE228, Network: Intel I210 Gigabit Connection OS: Ubuntu 18.04, Kernel: 4.17.0-999-generic (x86_64) 20180529, Desktop: GNOME Shell 3.28.1, Compiler: GCC 7.3.0, File-System: ext4, Screen Resolution: 1920x1080 HDD: Kyber: Processor: 2 x Intel Xeon Gold 6138 @ 3.70GHz (40 Cores / 80 Threads), Motherboard: TYAN S7106 (V1.00 BIOS), Chipset: Intel Sky Lake-E DMI3 Registers, Memory: 12 x 8192 MB DDR4-2666MT/s Micron 9ASF1G72PZ-2G6B1, Disk: 256GB Samsung SSD 850 + 2000GB Seagate ST2000DM006-2DM1 + 2 x 120GB TOSHIBA-TR150, Graphics: ASPEED ASPEED Family, Monitor: VE228, Network: Intel I210 Gigabit Connection OS: Ubuntu 18.04, Kernel: 4.17.0-999-generic (x86_64) 20180529, Desktop: GNOME Shell 3.28.1, Compiler: GCC 7.3.0, File-System: ext4, Screen Resolution: 1920x1080 HDD: MQ-DEADLINE: Processor: 2 x Intel Xeon Gold 6138 @ 3.70GHz (40 Cores / 80 Threads), Motherboard: TYAN S7106 (V1.00 BIOS), Chipset: Intel Sky Lake-E DMI3 Registers, Memory: 12 x 8192 MB DDR4-2666MT/s Micron 9ASF1G72PZ-2G6B1, Disk: 256GB Samsung SSD 850 + 2000GB Seagate ST2000DM006-2DM1 + 2 x 120GB TOSHIBA-TR150, Graphics: ASPEED ASPEED Family, Monitor: VE228, Network: Intel I210 Gigabit Connection OS: Ubuntu 18.04, Kernel: 4.17.0-999-generic (x86_64) 20180529, Desktop: GNOME Shell 3.28.1, Compiler: GCC 7.3.0, File-System: ext4, Screen Resolution: 1920x1080 HDD: Noop: Processor: 2 x Intel Xeon Gold 6138 @ 3.70GHz (40 Cores / 80 Threads), Motherboard: TYAN S7106 (V1.00 BIOS), Chipset: Intel Sky Lake-E DMI3 Registers, Memory: 12 x 8192 MB DDR4-2666MT/s Micron 9ASF1G72PZ-2G6B1, Disk: 256GB Samsung SSD 850 + 2000GB Seagate ST2000DM006-2DM1 + 2 x 120GB TOSHIBA-TR150, Graphics: ASPEED ASPEED Family, Monitor: VE228, Network: Intel I210 Gigabit Connection OS: Ubuntu 18.04, Kernel: 4.17.0-999-generic (x86_64) 20180529, Desktop: GNOME Shell 3.28.1, Compiler: GCC 7.3.0, File-System: ext4, Screen Resolution: 1920x1080 SSD: BFQ: Processor: 2 x Intel Xeon Gold 6138 @ 3.70GHz (40 Cores / 80 Threads), Motherboard: TYAN S7106 (V1.00 BIOS), Chipset: Intel Sky Lake-E DMI3 Registers, Memory: 12 x 8192 MB DDR4-2666MT/s Micron 9ASF1G72PZ-2G6B1, Disk: 256GB Samsung SSD 850 + 2000GB Seagate ST2000DM006-2DM1 + 2 x 120GB TOSHIBA-TR150, Graphics: ASPEED ASPEED Family, Monitor: VE228, Network: Intel I210 Gigabit Connection OS: Ubuntu 18.04, Kernel: 4.17.0-999-generic (x86_64) 20180529, Desktop: GNOME Shell 3.28.1, Compiler: GCC 7.3.0, File-System: ext4, Screen Resolution: 1920x1080 SSD: BFQ low_latency: Processor: 2 x Intel Xeon Gold 6138 @ 3.70GHz (40 Cores / 80 Threads), Motherboard: TYAN S7106 (V1.00 BIOS), Chipset: Intel Sky Lake-E DMI3 Registers, Memory: 12 x 8192 MB DDR4-2666MT/s Micron 9ASF1G72PZ-2G6B1, Disk: 256GB Samsung SSD 850 + 2000GB Seagate ST2000DM006-2DM1 + 2 x 120GB TOSHIBA-TR150, Graphics: ASPEED ASPEED Family, Monitor: VE228, Network: Intel I210 Gigabit Connection OS: Ubuntu 18.04, Kernel: 4.17.0-999-generic (x86_64) 20180529, Desktop: GNOME Shell 3.28.1, Compiler: GCC 7.3.0, File-System: ext4, Screen Resolution: 1920x1080 SSD: CFQ: Processor: 2 x Intel Xeon Gold 6138 @ 3.70GHz (40 Cores / 80 Threads), Motherboard: TYAN S7106 (V1.00 BIOS), Chipset: Intel Sky Lake-E DMI3 Registers, Memory: 12 x 8192 MB DDR4-2666MT/s Micron 9ASF1G72PZ-2G6B1, Disk: 256GB Samsung SSD 850 + 2000GB Seagate ST2000DM006-2DM1 + 2 x 120GB TOSHIBA-TR150, Graphics: ASPEED ASPEED Family, Monitor: VE228, Network: Intel I210 Gigabit Connection OS: Ubuntu 18.04, Kernel: 4.17.0-999-generic (x86_64) 20180529, Desktop: GNOME Shell 3.28.1, Compiler: GCC 7.3.0, File-System: ext4, Screen Resolution: 1920x1080 SSD: DEADLINE: Processor: 2 x Intel Xeon Gold 6138 @ 3.70GHz (40 Cores / 80 Threads), Motherboard: TYAN S7106 (V1.00 BIOS), Chipset: Intel Sky Lake-E DMI3 Registers, Memory: 12 x 8192 MB DDR4-2666MT/s Micron 9ASF1G72PZ-2G6B1, Disk: 256GB Samsung SSD 850 + 2000GB Seagate ST2000DM006-2DM1 + 2 x 120GB TOSHIBA-TR150, Graphics: ASPEED ASPEED Family, Monitor: VE228, Network: Intel I210 Gigabit Connection OS: Ubuntu 18.04, Kernel: 4.17.0-999-generic (x86_64) 20180529, Desktop: GNOME Shell 3.28.1, Compiler: GCC 7.3.0, File-System: ext4, Screen Resolution: 1920x1080 SSD: Kyber: Processor: 2 x Intel Xeon Gold 6138 @ 3.70GHz (40 Cores / 80 Threads), Motherboard: TYAN S7106 (V1.00 BIOS), Chipset: Intel Sky Lake-E DMI3 Registers, Memory: 12 x 8192 MB DDR4-2666MT/s Micron 9ASF1G72PZ-2G6B1, Disk: 256GB Samsung SSD 850 + 2000GB Seagate ST2000DM006-2DM1 + 2 x 120GB TOSHIBA-TR150, Graphics: ASPEED ASPEED Family, Monitor: VE228, Network: Intel I210 Gigabit Connection OS: Ubuntu 18.04, Kernel: 4.17.0-999-generic (x86_64) 20180529, Desktop: GNOME Shell 3.28.1, Compiler: GCC 7.3.0, File-System: ext4, Screen Resolution: 1920x1080 SSD: MQ-DEADLINE: Processor: 2 x Intel Xeon Gold 6138 @ 3.70GHz (40 Cores / 80 Threads), Motherboard: TYAN S7106 (V1.00 BIOS), Chipset: Intel Sky Lake-E DMI3 Registers, Memory: 12 x 8192 MB DDR4-2666MT/s Micron 9ASF1G72PZ-2G6B1, Disk: 256GB Samsung SSD 850 + 2000GB Seagate ST2000DM006-2DM1 + 2 x 120GB TOSHIBA-TR150, Graphics: ASPEED ASPEED Family, Monitor: VE228, Network: Intel I210 Gigabit Connection OS: Ubuntu 18.04, Kernel: 4.17.0-999-generic (x86_64) 20180529, Desktop: GNOME Shell 3.28.1, Compiler: GCC 7.3.0, File-System: ext4, Screen Resolution: 1920x1080 SSD: Noop: Processor: 2 x Intel Xeon Gold 6138 @ 3.70GHz (40 Cores / 80 Threads), Motherboard: TYAN S7106 (V1.00 BIOS), Chipset: Intel Sky Lake-E DMI3 Registers, Memory: 12 x 8192 MB DDR4-2666MT/s Micron 9ASF1G72PZ-2G6B1, Disk: 256GB Samsung SSD 850 + 2000GB Seagate ST2000DM006-2DM1 + 2 x 120GB TOSHIBA-TR150, Graphics: ASPEED ASPEED Family, Monitor: VE228, Network: Intel I210 Gigabit Connection OS: Ubuntu 18.04, Kernel: 4.17.0-999-generic (x86_64) 20180529, Desktop: GNOME Shell 3.28.1, Compiler: GCC 7.3.0, File-System: ext4, Screen Resolution: 1920x1080 Application Start-up Time 2.3.0 Background I/O Mix: Only Sequential Reads - Application To Start: GNOME Terminal sec < Lower Is Better HDD: BFQ ............. 6.75 |================================================ HDD: BFQ low_latency . 6.67 |=============================================== HDD: CFQ ............. 5.47 |======================================= HDD: DEADLINE ........ 5.48 |======================================= HDD: Kyber ........... 6.51 |============================================== HDD: MQ-DEADLINE ..... 7.04 |================================================== HDD: Noop ............ 5.48 |======================================= SSD: BFQ ............. 6.86 |================================================= SSD: BFQ low_latency . 6.95 |================================================= SSD: CFQ ............. 5.50 |======================================= SSD: DEADLINE ........ 5.34 |====================================== SSD: Kyber ........... 6.86 |================================================= SSD: MQ-DEADLINE ..... 7.07 |================================================== SSD: Noop ............ 5.23 |===================================== Application Start-up Time 2.3.0 Background I/O Mix: Sequential Reads + Writes - Application To Start: GNOME Terminal sec < Lower Is Better HDD: BFQ ............. 5.02 |==== HDD: BFQ low_latency . 5.24 |===== HDD: CFQ ............. 46.07 |======================================== HDD: DEADLINE ........ 39.27 |================================== HDD: Kyber ........... 5.22 |===== HDD: MQ-DEADLINE ..... 5.66 |===== HDD: Noop ............ 56.56 |================================================= SSD: BFQ ............. 5.23 |===== SSD: BFQ low_latency . 5.26 |===== SSD: CFQ ............. 40.88 |=================================== SSD: DEADLINE ........ 54.70 |=============================================== SSD: Kyber ........... 5.50 |===== SSD: MQ-DEADLINE ..... 5.48 |===== SSD: Noop ............ 53.68 |=============================================== SQLite 3.22 Timed SQLite Insertions Seconds < Lower Is Better HDD: BFQ ............. 705.15 |=========================================== HDD: BFQ low_latency . 790.19 |================================================ HDD: CFQ ............. 769.04 |=============================================== HDD: DEADLINE ........ 707.30 |=========================================== HDD: Kyber ........... 698.82 |========================================== HDD: MQ-DEADLINE ..... 716.51 |============================================ HDD: Noop ............ 708.38 |=========================================== SSD: BFQ ............. 45.13 |=== SSD: BFQ low_latency . 50.68 |=== SSD: CFQ ............. 41.59 |=== SSD: DEADLINE ........ 42.54 |=== SSD: Kyber ........... 44.51 |=== SSD: MQ-DEADLINE ..... 42.94 |=== SSD: Noop ............ 43.44 |=== Flexible IO Tester 3.1 Type: Random Read - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory MB/s > Higher Is Better HDD: BFQ ............. 1.57 | HDD: BFQ low_latency . 1.57 | HDD: CFQ ............. 1.57 | HDD: DEADLINE ........ 1.58 | HDD: Kyber ........... 1.56 | HDD: MQ-DEADLINE ..... 1.56 | HDD: Noop ............ 1.55 | SSD: BFQ ............. 233.00 |=============================================== SSD: BFQ low_latency . 213.00 |=========================================== SSD: CFQ ............. 224.00 |============================================== SSD: DEADLINE ........ 227.00 |============================================== SSD: Kyber ........... 224.00 |============================================== SSD: MQ-DEADLINE ..... 236.00 |================================================ SSD: Noop ............ 206.00 |========================================== Flexible IO Tester 3.1 Type: Random Write - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory MB/s > Higher Is Better HDD: BFQ ............. 1.06 | HDD: BFQ low_latency . 1.08 | HDD: CFQ ............. 1.05 | HDD: DEADLINE ........ 1.05 | HDD: Kyber ........... 1.04 | HDD: MQ-DEADLINE ..... 1.04 | HDD: Noop ............ 1.06 | SSD: BFQ ............. 97.53 |================= SSD: BFQ low_latency . 240.00 |========================================= SSD: CFQ ............. 281.00 |================================================ SSD: DEADLINE ........ 175.78 |============================== SSD: Kyber ........... 182.00 |=============================== SSD: MQ-DEADLINE ..... 171.00 |============================= SSD: Noop ............ 133.00 |======================= Flexible IO Tester 3.1 Type: Sequential Write - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory MB/s > Higher Is Better HDD: BFQ ............. 188.00 |====================== HDD: BFQ low_latency . 60.68 |======= HDD: CFQ ............. 196.00 |======================= HDD: DEADLINE ........ 196.00 |======================= HDD: Kyber ........... 168.00 |=================== HDD: MQ-DEADLINE ..... 185.00 |===================== HDD: Noop ............ 198.00 |======================= SSD: BFQ ............. 416.00 |================================================ SSD: BFQ low_latency . 417.00 |================================================ SSD: CFQ ............. 416.00 |================================================ SSD: DEADLINE ........ 411.00 |=============================================== SSD: Kyber ........... 412.00 |=============================================== SSD: MQ-DEADLINE ..... 414.00 |================================================ SSD: Noop ............ 400.00 |============================================== Flexible IO Tester 3.1 Type: Sequential Write - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory IOPS > Higher Is Better HDD: BFQ ............. 47867 |====================== HDD: BFQ low_latency . 15533 |======= HDD: CFQ ............. 50267 |======================= HDD: DEADLINE ........ 50000 |====================== HDD: Kyber ........... 42900 |=================== HDD: MQ-DEADLINE ..... 47267 |===================== HDD: Noop ............ 50533 |======================= SSD: BFQ ............. 106333 |================================================ SSD: BFQ low_latency . 106667 |================================================ SSD: CFQ ............. 106333 |================================================ SSD: DEADLINE ........ 105000 |=============================================== SSD: Kyber ........... 106000 |================================================ SSD: MQ-DEADLINE ..... 106120 |================================================ SSD: Noop ............ 102250 |============================================== FS-Mark 3.3 Test: 1000 Files, 1MB Size Files/s > Higher Is Better HDD: BFQ ............. 17.40 |=============== HDD: BFQ low_latency . 17.23 |=============== HDD: CFQ ............. 17.47 |=============== HDD: DEADLINE ........ 17.40 |=============== HDD: Kyber ........... 17.37 |=============== HDD: MQ-DEADLINE ..... 17.33 |=============== HDD: Noop ............ 17.37 |=============== SSD: BFQ ............. 55.73 |=============================================== SSD: BFQ low_latency . 47.62 |======================================== SSD: CFQ ............. 58.05 |================================================= SSD: DEADLINE ........ 55.45 |=============================================== SSD: Kyber ........... 55.82 |=============================================== SSD: MQ-DEADLINE ..... 56.20 |=============================================== SSD: Noop ............ 51.15 |=========================================== FS-Mark 3.3 Test: 5000 Files, 1MB Size, 4 Threads Files/s > Higher Is Better HDD: BFQ ............. 48.40 |=============================================== HDD: BFQ low_latency . 48.40 |=============================================== HDD: CFQ ............. 47.87 |=============================================== HDD: DEADLINE ........ 48.67 |=============================================== HDD: Kyber ........... 47.73 |============================================== HDD: MQ-DEADLINE ..... 48.53 |=============================================== HDD: Noop ............ 48.67 |=============================================== SSD: BFQ ............. 49.47 |================================================ SSD: BFQ low_latency . 47.00 |============================================== SSD: CFQ ............. 50.40 |================================================= SSD: DEADLINE ........ 47.67 |============================================== SSD: Kyber ........... 49.33 |================================================ SSD: MQ-DEADLINE ..... 45.80 |============================================= SSD: Noop ............ 48.53 |=============================================== FS-Mark 3.3 Test: 4000 Files, 32 Sub Dirs, 1MB Size Files/s > Higher Is Better HDD: BFQ ............. 17.37 |===================== HDD: BFQ low_latency . 17.30 |==================== HDD: CFQ ............. 17.33 |===================== HDD: DEADLINE ........ 17.33 |===================== HDD: Kyber ........... 17.37 |===================== HDD: MQ-DEADLINE ..... 17.37 |===================== HDD: Noop ............ 17.40 |===================== SSD: BFQ ............. 41.22 |================================================= SSD: BFQ low_latency . 40.74 |================================================ SSD: CFQ ............. 41.42 |================================================= SSD: DEADLINE ........ 41.05 |================================================= SSD: Kyber ........... 40.98 |================================================ SSD: MQ-DEADLINE ..... 35.80 |========================================== SSD: Noop ............ 40.98 |================================================ Dbench 4.0 Client Count: 12 MB/s > Higher Is Better HDD: BFQ ............. 30.43 |=== HDD: BFQ low_latency . 29.60 |== HDD: CFQ ............. 30.53 |=== HDD: DEADLINE ........ 30.56 |=== HDD: Kyber ........... 27.48 |== HDD: MQ-DEADLINE ..... 27.81 |== HDD: Noop ............ 30.49 |=== SSD: BFQ ............. 486.68 |========================================= SSD: BFQ low_latency . 486.71 |========================================= SSD: CFQ ............. 560.40 |=============================================== SSD: DEADLINE ........ 538.55 |============================================= SSD: Kyber ........... 517.23 |=========================================== SSD: MQ-DEADLINE ..... 547.19 |============================================== SSD: Noop ............ 570.93 |================================================ IOzone 3.465 Record Size: 1MB - File Size: 8GB - Disk Test: Write Performance MB/s > Higher Is Better HDD: BFQ ............. 179.09 |=============================================== HDD: BFQ low_latency . 181.58 |=============================================== HDD: CFQ ............. 182.55 |================================================ HDD: DEADLINE ........ 184.40 |================================================ HDD: Kyber ........... 174.82 |============================================== HDD: MQ-DEADLINE ..... 183.72 |================================================ HDD: Noop ............ 183.62 |================================================ SSD: BFQ ............. 102.59 |=========================== SSD: BFQ low_latency . 94.18 |========================= SSD: CFQ ............. 97.12 |========================= SSD: DEADLINE ........ 103.75 |=========================== SSD: Kyber ........... 96.00 |========================= SSD: MQ-DEADLINE ..... 98.69 |========================== SSD: Noop ............ 105.59 |=========================== PostgreSQL pgbench 10.3 Scaling: Buffer Test - Test: Normal Load - Mode: Read Write TPS > Higher Is Better HDD: BFQ ............. 301.02 |====== HDD: BFQ low_latency . 354.98 |======= HDD: CFQ ............. 320.96 |====== HDD: DEADLINE ........ 245.09 |===== HDD: Kyber ........... 511.81 |========== HDD: MQ-DEADLINE ..... 522.59 |=========== HDD: Noop ............ 253.04 |===== SSD: BFQ ............. 2301.40 |============================================== SSD: BFQ low_latency . 1961.43 |======================================== SSD: CFQ ............. 2264.96 |============================================== SSD: DEADLINE ........ 2289.32 |============================================== SSD: Kyber ........... 1970.86 |======================================== SSD: MQ-DEADLINE ..... 1757.42 |=================================== SSD: Noop ............ 2329.09 |=============================================== Flexible IO Tester 3.1 Type: Random Read - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory IOPS > Higher Is Better SSD: BFQ ............. 59700 |================================================ SSD: BFQ low_latency . 54600 |============================================ SSD: CFQ ............. 57300 |============================================== SSD: DEADLINE ........ 58000 |=============================================== SSD: Kyber ........... 57233 |============================================== SSD: MQ-DEADLINE ..... 60500 |================================================= SSD: Noop ............ 52700 |=========================================== Flexible IO Tester 3.1 Type: Random Write - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory IOPS > Higher Is Better SSD: BFQ ............. 24783 |================= SSD: BFQ low_latency . 61300 |========================================== SSD: CFQ ............. 72067 |================================================= SSD: DEADLINE ........ 45050 |=============================== SSD: Kyber ........... 46600 |================================ SSD: MQ-DEADLINE ..... 43650 |============================== SSD: Noop ............ 33817 |=======================