Btrfs Linux 4.10 RAID 4-Disk SSD Btrfs multi-disk tests on the Linux 4.10 kernel with various RAID options. Tests by Michael Larabel for a future article on Phoronix.com. Single Disk: Processor: Intel Core i7-6800K @ 3.80GHz (12 Cores), Motherboard: MSI X99A WORKSTATION (MS-7A54) v1.0, Chipset: Intel Xeon E7 v4/Xeon, Memory: 16384MB, Disk: 120GB Samsung SSD 850 + 4 x 120GB TOSHIBA-TR150, Graphics: NVIDIA GeForce GTX TITAN X 12288MB, Audio: Realtek ALC1150, Monitor: ASUS PB278, Network: Intel Connection OS: Ubuntu 16.04, Kernel: 4.10.0-999-generic (x86_64) 20170126, Desktop: Unity 7.4.0, Display Server: X Server 1.18.4, Display Driver: nouveau 1.0.12, OpenGL: 3.3 Mesa 11.2.0 Gallium 0.4, Vulkan: 1.0.8, Compiler: GCC 5.4.0 20160609 + CUDA 8.0, File-System: btrfs, Screen Resolution: 2560x1440 2-Disk RAID0: Processor: Intel Core i7-6800K @ 3.80GHz (12 Cores), Motherboard: MSI X99A WORKSTATION (MS-7A54) v1.0, Chipset: Intel Xeon E7 v4/Xeon, Memory: 16384MB, Disk: 120GB Samsung SSD 850 + 4 x 120GB TOSHIBA-TR150, Graphics: NVIDIA GeForce GTX TITAN X 12288MB, Audio: Realtek ALC1150, Monitor: ASUS PB278, Network: Intel Connection OS: Ubuntu 16.04, Kernel: 4.10.0-999-generic (x86_64) 20170126, Desktop: Unity 7.4.0, Display Server: X Server 1.18.4, Display Driver: nouveau 1.0.12, OpenGL: 3.3 Mesa 11.2.0 Gallium 0.4, Vulkan: 1.0.8, Compiler: GCC 5.4.0 20160609 + CUDA 8.0, File-System: btrfs, Screen Resolution: 2560x1440 2-Disk RAID1: Processor: Intel Core i7-6800K @ 3.80GHz (12 Cores), Motherboard: MSI X99A WORKSTATION (MS-7A54) v1.0, Chipset: Intel Xeon E7 v4/Xeon, Memory: 16384MB, Disk: 120GB Samsung SSD 850 + 4 x 120GB TOSHIBA-TR150, Graphics: NVIDIA GeForce GTX TITAN X 12288MB, Audio: Realtek ALC1150, Monitor: ASUS PB278, Network: Intel Connection OS: Ubuntu 16.04, Kernel: 4.10.0-999-generic (x86_64) 20170126, Desktop: Unity 7.4.0, Display Server: X Server 1.18.4, Display Driver: nouveau 1.0.12, OpenGL: 3.3 Mesa 11.2.0 Gallium 0.4, Vulkan: 1.0.8, Compiler: GCC 5.4.0 20160609 + CUDA 8.0, File-System: btrfs, Screen Resolution: 2560x1440 4-Disk RAID0: Processor: Intel Core i7-6800K @ 3.80GHz (12 Cores), Motherboard: MSI X99A WORKSTATION (MS-7A54) v1.0, Chipset: Intel Xeon E7 v4/Xeon, Memory: 16384MB, Disk: 120GB Samsung SSD 850 + 4 x 120GB TOSHIBA-TR150, Graphics: NVIDIA GeForce GTX TITAN X 12288MB, Audio: Realtek ALC1150, Monitor: ASUS PB278, Network: Intel Connection OS: Ubuntu 16.04, Kernel: 4.10.0-999-generic (x86_64) 20170126, Desktop: Unity 7.4.0, Display Server: X Server 1.18.4, Display Driver: nouveau 1.0.12, OpenGL: 3.3 Mesa 11.2.0 Gallium 0.4, Vulkan: 1.0.8, Compiler: GCC 5.4.0 20160609 + CUDA 8.0, File-System: btrfs, Screen Resolution: 2560x1440 4-Disk RAID1: Processor: Intel Core i7-6800K @ 3.80GHz (12 Cores), Motherboard: MSI X99A WORKSTATION (MS-7A54) v1.0, Chipset: Intel Xeon E7 v4/Xeon, Memory: 16384MB, Disk: 120GB Samsung SSD 850 + 4 x 120GB TOSHIBA-TR150, Graphics: NVIDIA GeForce GTX TITAN X 12288MB, Audio: Realtek ALC1150, Monitor: ASUS PB278, Network: Intel Connection OS: Ubuntu 16.04, Kernel: 4.10.0-999-generic (x86_64) 20170126, Desktop: Unity 7.4.0, Display Server: X Server 1.18.4, Display Driver: nouveau 1.0.12, OpenGL: 3.3 Mesa 11.2.0 Gallium 0.4, Vulkan: 1.0.8, Compiler: GCC 5.4.0 20160609 + CUDA 8.0, File-System: btrfs, Screen Resolution: 2560x1440 4-Disk RAID10: Processor: Intel Core i7-6800K @ 3.80GHz (12 Cores), Motherboard: MSI X99A WORKSTATION (MS-7A54) v1.0, Chipset: Intel Xeon E7 v4/Xeon, Memory: 16384MB, Disk: 120GB Samsung SSD 850 + 4 x 120GB TOSHIBA-TR150, Graphics: NVIDIA GeForce GTX TITAN X 12288MB, Audio: Realtek ALC1150, Monitor: ASUS PB278, Network: Intel Connection OS: Ubuntu 16.04, Kernel: 4.10.0-999-generic (x86_64) 20170126, Desktop: Unity 7.4.0, Display Server: X Server 1.18.4, Display Driver: nouveau 1.0.12, OpenGL: 3.3 Mesa 11.2.0 Gallium 0.4, Vulkan: 1.0.8, Compiler: GCC 5.4.0 20160609 + CUDA 8.0, File-System: btrfs, Screen Resolution: 2560x1440 4-Disk RAID5: Processor: Intel Core i7-6800K @ 3.80GHz (12 Cores), Motherboard: MSI X99A WORKSTATION (MS-7A54) v1.0, Chipset: Intel Xeon E7 v4/Xeon, Memory: 16384MB, Disk: 120GB Samsung SSD 850 + 4 x 120GB TOSHIBA-TR150, Graphics: NVIDIA GeForce GTX TITAN X 12288MB, Audio: Realtek ALC1150, Monitor: ASUS PB278, Network: Intel Connection OS: Ubuntu 16.04, Kernel: 4.10.0-999-generic (x86_64) 20170126, Desktop: Unity 7.4.0, Display Server: X Server 1.18.4, Display Driver: nouveau 1.0.12, OpenGL: 3.3 Mesa 11.2.0 Gallium 0.4, Vulkan: 1.0.8, Compiler: GCC 5.4.0 20160609 + CUDA 8.0, File-System: btrfs, Screen Resolution: 2560x1440 4-Disk RAID6: Processor: Intel Core i7-6800K @ 3.80GHz (12 Cores), Motherboard: MSI X99A WORKSTATION (MS-7A54) v1.0, Chipset: Intel Xeon E7 v4/Xeon, Memory: 16384MB, Disk: 120GB Samsung SSD 850 + 4 x 120GB TOSHIBA-TR150, Graphics: NVIDIA GeForce GTX TITAN X 12288MB, Audio: Realtek ALC1150, Monitor: ASUS PB278, Network: Intel Connection OS: Ubuntu 16.04, Kernel: 4.10.0-999-generic (x86_64) 20170126, Desktop: Unity 7.4.0, Display Server: X Server 1.18.4, Display Driver: nouveau 1.0.12, OpenGL: 3.3 Mesa 11.2.0 Gallium 0.4, Vulkan: 1.0.8, Compiler: GCC 5.4.0 20160609 + CUDA 8.0, File-System: btrfs, Screen Resolution: 2560x1440 seistro-RAID5-8disks-ext4-4.12.1: Processor: 2 x Intel Xeon E5-2630 0 @ 2.80GHz (24 Cores), Motherboard: Dell 0XH7F2, Chipset: Intel Xeon E5/Core, Memory: 387072MB, Disk: 6998GB PERC H710P + 1199GB PERC H710P, Graphics: Matrox s G200eR2, Monitor: Smart Cable, Network: Broadcom NetXtreme BCM5720 Gigabit PCIe OS: Debian 8.8, Kernel: 4.12.1-041201-generic (x86_64), Compiler: GCC 5.3.1 20160224, File-System: ext4, Screen Resolution: 1024x768 4.12.1-seistro-raid10-4disks: Processor: 2 x Intel Xeon E5-2630 0 @ 2.80GHz (24 Cores), Motherboard: Dell 0XH7F2, Chipset: Intel Xeon E5/Core, Memory: 387072MB, Disk: 6998GB PERC H710P + 1199GB PERC H710P, Graphics: Matrox s G200eR2, Monitor: Smart Cable, Network: Broadcom NetXtreme BCM5720 Gigabit PCIe OS: Debian 8.8, Kernel: 4.12.1-041201-generic (x86_64), Compiler: GCC 5.3.1 20160224, File-System: ext4, Screen Resolution: 1024x768 4.12.1-seistro-raid10-4disks-ext4: Processor: 2 x Intel Xeon E5-2630 0 @ 2.80GHz (24 Cores), Motherboard: Dell 0XH7F2, Chipset: Intel Xeon E5/Core, Memory: 387072MB, Disk: 6998GB PERC H710P + 1199GB PERC H710P, Graphics: Matrox s G200eR2, Monitor: Smart Cable, Network: Broadcom NetXtreme BCM5720 Gigabit PCIe OS: Debian 8.8, Kernel: 4.12.1-041201-generic (x86_64), Compiler: GCC 5.3.1 20160224, File-System: ext4, Screen Resolution: 1024x768 SQLite 3.8.10.2 Test Target: Default Test Directory Seconds < Lower Is Better Single Disk ....................... 85.56 |========================= 2-Disk RAID0 ...................... 87.56 |========================= 2-Disk RAID1 ...................... 105.90 |============================== 4-Disk RAID0 ...................... 85.22 |======================== 4-Disk RAID1 ...................... 91.33 |========================== 4-Disk RAID10 ..................... 97.09 |============================ 4-Disk RAID5 ...................... 107.49 |=============================== 4-Disk RAID6 ...................... 122.07 |=================================== seistro-RAID5-8disks-ext4-4.12.1 .. 12.99 |==== 4.12.1-seistro-raid10-4disks-ext4 . 3.98 |= Flexible IO Tester 2.1.13 Type: Random Read - IO Engine: Libaio - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory - Result: MB/s MB/s > Higher Is Better Single Disk ....................... 210.37 |========== 2-Disk RAID0 ...................... 439.91 |===================== 2-Disk RAID1 ...................... 213.46 |========== 4-Disk RAID0 ...................... 720.89 |=================================== 4-Disk RAID1 ...................... 413.46 |==================== 4-Disk RAID10 ..................... 424.44 |===================== 4-Disk RAID5 ...................... 585.34 |============================ 4-Disk RAID6 ...................... 422.97 |===================== seistro-RAID5-8disks-ext4-4.12.1 .. 2.32 | 4.12.1-seistro-raid10-4disks-ext4 . 245.69 |============ Flexible IO Tester 2.1.13 Type: Random Write - IO Engine: Libaio - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory - Result: MB/s MB/s > Higher Is Better Single Disk ....................... 91.71 |==================== 2-Disk RAID0 ...................... 126.24 |============================ 2-Disk RAID1 ...................... 49.58 |=========== 4-Disk RAID0 ...................... 160.19 |=================================== 4-Disk RAID1 ...................... 95.64 |===================== 4-Disk RAID10 ..................... 103.63 |======================= 4-Disk RAID5 ...................... 156.98 |================================== 4-Disk RAID6 ...................... 119.44 |========================== seistro-RAID5-8disks-ext4-4.12.1 .. 6.28 |= 4.12.1-seistro-raid10-4disks-ext4 . 149.18 |================================= Flexible IO Tester 2.1.13 Type: Sequential Read - IO Engine: Libaio - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory - Result: MB/s MB/s > Higher Is Better Single Disk ....................... 243.91 |============ 2-Disk RAID0 ...................... 525.49 |========================== 2-Disk RAID1 ...................... 331.32 |================ 4-Disk RAID0 ...................... 719.56 |=================================== 4-Disk RAID1 ...................... 267.38 |============= 4-Disk RAID10 ..................... 558.34 |=========================== 4-Disk RAID5 ...................... 678.77 |================================= 4-Disk RAID6 ...................... 591.20 |============================= seistro-RAID5-8disks-ext4-4.12.1 .. 2.40 | 4.12.1-seistro-raid10-4disks-ext4 . 254.28 |============ Flexible IO Tester 2.1.13 Type: Sequential Write - IO Engine: Libaio - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory - Result: MB/s MB/s > Higher Is Better Single Disk ....................... 90.36 |========= 2-Disk RAID0 ...................... 125.88 |============= 2-Disk RAID1 ...................... 49.50 |===== 4-Disk RAID0 ...................... 166.98 |================= 4-Disk RAID1 ...................... 105.29 |=========== 4-Disk RAID10 ..................... 116.27 |============ 4-Disk RAID5 ...................... 113.33 |============ 4-Disk RAID6 ...................... 101.58 |========== seistro-RAID5-8disks-ext4-4.12.1 .. 103.67 |=========== 4.12.1-seistro-raid10-4disks-ext4 . 339.34 |=================================== FS-Mark 3.3 Test: 1000 Files, 1MB Size Files/s > Higher Is Better Single Disk ....................... 61.03 |===== 2-Disk RAID0 ...................... 78.17 |======= 2-Disk RAID1 ...................... 59.35 |===== 4-Disk RAID0 ...................... 121.72 |=========== 4-Disk RAID1 ...................... 78.57 |======= 4-Disk RAID10 ..................... 63.17 |====== 4-Disk RAID5 ...................... 82.83 |======= 4-Disk RAID6 ...................... 83.70 |======= seistro-RAID5-8disks-ext4-4.12.1 .. 61.90 |===== 4.12.1-seistro-raid10-4disks-ext4 . 399.68 |=================================== FS-Mark 3.3 Test: 5000 Files, 1MB Size, 4 Threads Files/s > Higher Is Better Single Disk ....................... 50.73 |==== 2-Disk RAID0 ...................... 80.60 |======= 2-Disk RAID1 ...................... 42.95 |==== 4-Disk RAID0 ...................... 132.25 |=========== 4-Disk RAID1 ...................... 61.37 |===== 4-Disk RAID10 ..................... 65.67 |===== 4-Disk RAID5 ...................... 100.83 |======== 4-Disk RAID6 ...................... 66.60 |===== seistro-RAID5-8disks-ext4-4.12.1 .. 184.12 |=============== 4.12.1-seistro-raid10-4disks-ext4 . 424.50 |=================================== FS-Mark 3.3 Test: 4000 Files, 32 Sub Dirs, 1MB Size Files/s > Higher Is Better Single Disk ....................... 46.63 |===== 2-Disk RAID0 ...................... 74.55 |======= 2-Disk RAID1 ...................... 42.05 |==== 4-Disk RAID0 ...................... 135.77 |============= 4-Disk RAID1 ...................... 61.42 |====== 4-Disk RAID10 ..................... 65.63 |====== 4-Disk RAID5 ...................... 84.28 |======== 4-Disk RAID6 ...................... 57.65 |====== 4.12.1-seistro-raid10-4disks-ext4 . 354.45 |=================================== BlogBench 1.0 Test: Write Final Score > Higher Is Better Single Disk ....................... 4942 |========= 2-Disk RAID0 ...................... 6617 |============ 2-Disk RAID1 ...................... 4939 |========= 4-Disk RAID0 ...................... 7537 |============== 4-Disk RAID1 ...................... 5542 |========== 4-Disk RAID10 ..................... 6420 |============ 4-Disk RAID5 ...................... 7079 |============= 4-Disk RAID6 ...................... 5808 |=========== 4.12.1-seistro-raid10-4disks-ext4 . 19876 |==================================== Compile Bench 0.6 Test: Compile MB/s > Higher Is Better Single Disk ....................... 816.97 |==================== 2-Disk RAID0 ...................... 986.31 |======================== 2-Disk RAID1 ...................... 784.53 |=================== 4-Disk RAID0 ...................... 1413.04 |================================== 4-Disk RAID1 ...................... 915.54 |====================== 4-Disk RAID10 ..................... 903.57 |====================== 4-Disk RAID5 ...................... 1001.93 |======================== 4-Disk RAID6 ...................... 878.04 |===================== 4.12.1-seistro-raid10-4disks-ext4 . 1140.21 |=========================== Compile Bench 0.6 Test: Initial Create MB/s > Higher Is Better Single Disk ....................... 126.47 |=============== 2-Disk RAID0 ...................... 271.68 |================================ 2-Disk RAID1 ...................... 118.82 |============== 4-Disk RAID0 ...................... 296.61 |=================================== 4-Disk RAID1 ...................... 226.87 |=========================== 4-Disk RAID10 ..................... 246.89 |============================= 4-Disk RAID5 ...................... 263.94 |=============================== 4-Disk RAID6 ...................... 237.06 |============================ 4.12.1-seistro-raid10-4disks-ext4 . 177.50 |===================== PostgreSQL pgbench 9.4.3 Scaling: On-Disk - Test: Normal Load - Mode: Read Write TPS > Higher Is Better Single Disk ....................... 1196.72 |============== 2-Disk RAID0 ...................... 1626.47 |=================== 2-Disk RAID1 ...................... 1147.83 |============== 4-Disk RAID0 ...................... 2887.00 |================================== 4-Disk RAID1 ...................... 972.52 |=========== 4-Disk RAID10 ..................... 1427.30 |================= 4-Disk RAID5 ...................... 2012.79 |======================== 4-Disk RAID6 ...................... 1303.60 |=============== 4.12.1-seistro-raid10-4disks-ext4 . 112.32 |=