Btrfs RAID, EXT4 MD RAID 4 x SSD Linux Tests
Btrfs RAID and EXT4 MDADM RAID tests on Linux 4.15 for a future article on Phoronix.


Btrfs 1 Disk: 

	Processor: Intel Xeon Silver 4108 @ 3.00GHz (8 Cores / 16 Threads), Motherboard: TYAN S7100AG2NR, Chipset: Intel Device 2020, Memory: 6 x 4096 MB DDR4-2400MHz Micron 9ASF51272PZ-2G6B1, Disk: 4 x 120GB ADATA SU700 + 240GB Force MP500, Graphics: ASPEED ASPEED Family, Audio: Realtek ALC892, Monitor: VE228, Network: Intel I350 Gigabit Connection

	OS: Ubuntu 17.04, Kernel: 4.15.0-999-generic (x86_64) 20171212, Desktop: Unity 7.5.0, Display Driver: modesetting 1.19.3, Compiler: GCC 6.3.0 20170406, File-System: btrfs, Screen Resolution: 1920x1080

Btrfs 2 Disk RAID0: 

	Processor: Intel Xeon Silver 4108 @ 3.00GHz (8 Cores / 16 Threads), Motherboard: TYAN S7100AG2NR, Chipset: Intel Device 2020, Memory: 6 x 4096 MB DDR4-2400MHz Micron 9ASF51272PZ-2G6B1, Disk: 4 x 120GB ADATA SU700 + 240GB Force MP500, Graphics: ASPEED ASPEED Family, Audio: Realtek ALC892, Monitor: VE228, Network: Intel I350 Gigabit Connection

	OS: Ubuntu 17.04, Kernel: 4.15.0-999-generic (x86_64) 20171212, Desktop: Unity 7.5.0, Display Driver: modesetting 1.19.3, Compiler: GCC 6.3.0 20170406, File-System: btrfs, Screen Resolution: 1920x1080

Btrfs 2 Disk RAID1: 

	Processor: Intel Xeon Silver 4108 @ 3.00GHz (8 Cores / 16 Threads), Motherboard: TYAN S7100AG2NR, Chipset: Intel Device 2020, Memory: 6 x 4096 MB DDR4-2400MHz Micron 9ASF51272PZ-2G6B1, Disk: 4 x 120GB ADATA SU700 + 240GB Force MP500, Graphics: ASPEED ASPEED Family, Audio: Realtek ALC892, Monitor: VE228, Network: Intel I350 Gigabit Connection

	OS: Ubuntu 17.04, Kernel: 4.15.0-999-generic (x86_64) 20171212, Desktop: Unity 7.5.0, Display Driver: modesetting 1.19.3, Compiler: GCC 6.3.0 20170406, File-System: btrfs, Screen Resolution: 1920x1080

Btrfs 4 Disk RAID0: 

	Processor: Intel Xeon Silver 4108 @ 3.00GHz (8 Cores / 16 Threads), Motherboard: TYAN S7100AG2NR, Chipset: Intel Device 2020, Memory: 6 x 4096 MB DDR4-2400MHz Micron 9ASF51272PZ-2G6B1, Disk: 4 x 120GB ADATA SU700 + 240GB Force MP500, Graphics: ASPEED ASPEED Family, Audio: Realtek ALC892, Monitor: VE228, Network: Intel I350 Gigabit Connection

	OS: Ubuntu 17.04, Kernel: 4.15.0-999-generic (x86_64) 20171212, Display Driver: modesetting 1.19.3, Compiler: GCC 6.3.0 20170406, File-System: btrfs, Screen Resolution: 1920x1080

Btrfs 4 Disk RAID1: 

	Processor: Intel Xeon Silver 4108 @ 3.00GHz (8 Cores / 16 Threads), Motherboard: TYAN S7100AG2NR, Chipset: Intel Device 2020, Memory: 6 x 4096 MB DDR4-2400MHz Micron 9ASF51272PZ-2G6B1, Disk: 4 x 120GB ADATA SU700 + 240GB Force MP500, Graphics: ASPEED ASPEED Family, Audio: Realtek ALC892, Monitor: VE228, Network: Intel I350 Gigabit Connection

	OS: Ubuntu 17.04, Kernel: 4.15.0-999-generic (x86_64) 20171212, Desktop: Unity 7.5.0, Display Driver: modesetting 1.19.3, Compiler: GCC 6.3.0 20170406, File-System: btrfs, Screen Resolution: 1920x1080

Btrfs 4 Disk RAID10: 

	Processor: Intel Xeon Silver 4108 @ 3.00GHz (8 Cores / 16 Threads), Motherboard: TYAN S7100AG2NR, Chipset: Intel Device 2020, Memory: 6 x 4096 MB DDR4-2400MHz Micron 9ASF51272PZ-2G6B1, Disk: 4 x 120GB ADATA SU700 + 240GB Force MP500, Graphics: ASPEED ASPEED Family, Audio: Realtek ALC892, Monitor: VE228, Network: Intel I350 Gigabit Connection

	OS: Ubuntu 17.04, Kernel: 4.15.0-999-generic (x86_64) 20171212, Desktop: Unity 7.5.0, Display Driver: modesetting 1.19.3, Compiler: GCC 6.3.0 20170406, File-System: btrfs, Screen Resolution: 1920x1080

Btrfs 4 Disk RAID5: 

	Processor: Intel Xeon Silver 4108 @ 3.00GHz (8 Cores / 16 Threads), Motherboard: TYAN S7100AG2NR, Chipset: Intel Device 2020, Memory: 6 x 4096 MB DDR4-2400MHz Micron 9ASF51272PZ-2G6B1, Disk: 4 x 120GB ADATA SU700 + 240GB Force MP500, Graphics: ASPEED ASPEED Family, Audio: Realtek ALC892, Monitor: VE228, Network: Intel I350 Gigabit Connection

	OS: Ubuntu 17.04, Kernel: 4.15.0-999-generic (x86_64) 20171212, Desktop: Unity 7.5.0, Display Driver: modesetting 1.19.3, Compiler: GCC 6.3.0 20170406, File-System: btrfs, Screen Resolution: 1920x1080

Btrfs 4 Disk RAID6: 

	Processor: Intel Xeon Silver 4108 @ 3.00GHz (8 Cores / 16 Threads), Motherboard: TYAN S7100AG2NR, Chipset: Intel Device 2020, Memory: 6 x 4096 MB DDR4-2400MHz Micron 9ASF51272PZ-2G6B1, Disk: 4 x 120GB ADATA SU700 + 240GB Force MP500, Graphics: ASPEED ASPEED Family, Audio: Realtek ALC892, Monitor: VE228, Network: Intel I350 Gigabit Connection

	OS: Ubuntu 17.04, Kernel: 4.15.0-999-generic (x86_64) 20171212, Desktop: Unity 7.5.0, Display Driver: modesetting 1.19.3, Compiler: GCC 6.3.0 20170406, File-System: btrfs, Screen Resolution: 1920x1080

EXT4 1 Disk: 

	Processor: Intel Xeon Silver 4108 @ 3.00GHz (8 Cores / 16 Threads), Motherboard: TYAN S7100AG2NR, Chipset: Intel Device 2020, Memory: 6 x 4096 MB DDR4-2400MHz Micron 9ASF51272PZ-2G6B1, Disk: 4 x 120GB ADATA SU700 + 240GB Force MP500, Graphics: ASPEED ASPEED Family, Audio: Realtek ALC892, Monitor: VE228, Network: Intel I350 Gigabit Connection

	OS: Ubuntu 17.04, Kernel: 4.15.0-999-generic (x86_64) 20171212, Desktop: Unity 7.5.0, Display Driver: modesetting 1.19.3, Compiler: GCC 6.3.0 20170406, File-System: ext4, Screen Resolution: 1920x1080

EXT4 2 Disk RAID1: 

	Processor: Intel Xeon Silver 4108 @ 3.00GHz (8 Cores / 16 Threads), Motherboard: TYAN S7100AG2NR, Chipset: Intel Device 2020, Memory: 6 x 4096 MB DDR4-2400MHz Micron 9ASF51272PZ-2G6B1, Disk: 4 x 120GB ADATA SU700 + 240GB Force MP500, Graphics: ASPEED ASPEED Family, Audio: Realtek ALC892, Monitor: VE228, Network: Intel I350 Gigabit Connection

	OS: Ubuntu 17.04, Kernel: 4.15.0-999-generic (x86_64) 20171212, Desktop: Unity 7.5.0, Display Driver: modesetting 1.19.3, Compiler: GCC 6.3.0 20170406, File-System: ext4, Screen Resolution: 1920x1080

EXT4 2 Disk RAID0: 

	Processor: Intel Xeon Silver 4108 @ 3.00GHz (8 Cores / 16 Threads), Motherboard: TYAN S7100AG2NR, Chipset: Intel Device 2020, Memory: 6 x 4096 MB DDR4-2400MHz Micron 9ASF51272PZ-2G6B1, Disk: 4 x 120GB ADATA SU700 + 240GB Force MP500, Graphics: ASPEED ASPEED Family, Audio: Realtek ALC892, Monitor: VE228, Network: Intel I350 Gigabit Connection

	OS: Ubuntu 17.04, Kernel: 4.15.0-999-generic (x86_64) 20171212, Desktop: Unity 7.5.0, Display Driver: modesetting 1.19.3, Compiler: GCC 6.3.0 20170406, File-System: ext4, Screen Resolution: 1920x1080

EXT4 4 Disk RAID0: 

	Processor: Intel Xeon Silver 4108 @ 3.00GHz (8 Cores / 16 Threads), Motherboard: TYAN S7100AG2NR, Chipset: Intel Device 2020, Memory: 6 x 4096 MB DDR4-2400MHz Micron 9ASF51272PZ-2G6B1, Disk: 4 x 120GB ADATA SU700 + 240GB Force MP500, Graphics: ASPEED ASPEED Family, Audio: Realtek ALC892, Monitor: VE228, Network: Intel I350 Gigabit Connection

	OS: Ubuntu 17.04, Kernel: 4.15.0-999-generic (x86_64) 20171212, Desktop: Unity 7.5.0, Display Driver: modesetting 1.19.3, Compiler: GCC 6.3.0 20170406, File-System: ext4, Screen Resolution: 1920x1080

EXT4 4 Disk RAID1: 

	Processor: Intel Xeon Silver 4108 @ 3.00GHz (8 Cores / 16 Threads), Motherboard: TYAN S7100AG2NR, Chipset: Intel Device 2020, Memory: 6 x 4096 MB DDR4-2400MHz Micron 9ASF51272PZ-2G6B1, Disk: 4 x 120GB ADATA SU700 + 240GB Force MP500, Graphics: ASPEED ASPEED Family, Audio: Realtek ALC892, Monitor: VE228, Network: Intel I350 Gigabit Connection

	OS: Ubuntu 17.04, Kernel: 4.15.0-999-generic (x86_64) 20171212, Desktop: Unity 7.5.0, Display Driver: modesetting 1.19.3, Compiler: GCC 6.3.0 20170406, File-System: ext4, Screen Resolution: 1920x1080

EXT4 4 Disk RAID5: 

	Processor: Intel Xeon Silver 4108 @ 3.00GHz (8 Cores / 16 Threads), Motherboard: TYAN S7100AG2NR, Chipset: Intel Device 2020, Memory: 6 x 4096 MB DDR4-2400MHz Micron 9ASF51272PZ-2G6B1, Disk: 4 x 120GB ADATA SU700 + 240GB Force MP500, Graphics: ASPEED ASPEED Family, Audio: Realtek ALC892, Monitor: VE228, Network: Intel I350 Gigabit Connection

	OS: Ubuntu 17.04, Kernel: 4.15.0-999-generic (x86_64) 20171212, Desktop: Unity 7.5.0, Display Driver: modesetting 1.19.3, Compiler: GCC 6.3.0 20170406, File-System: ext4, Screen Resolution: 1920x1080

EXT4 4 Disk RAID6: 

	Processor: Intel Xeon Silver 4108 @ 3.00GHz (8 Cores / 16 Threads), Motherboard: TYAN S7100AG2NR, Chipset: Intel Device 2020, Memory: 6 x 4096 MB DDR4-2400MHz Micron 9ASF51272PZ-2G6B1, Disk: 4 x 120GB ADATA SU700 + 240GB Force MP500, Graphics: ASPEED ASPEED Family, Audio: Realtek ALC892, Monitor: VE228, Network: Intel I350 Gigabit Connection

	OS: Ubuntu 17.04, Kernel: 4.15.0-999-generic (x86_64) 20171212, Display Driver: modesetting 1.19.3, Compiler: GCC 6.3.0 20170406, File-System: ext4, Screen Resolution: 1920x1080

EXT4 4 Disk RAID10: 

	Processor: Intel Xeon Silver 4108 @ 3.00GHz (8 Cores / 16 Threads), Motherboard: TYAN S7100AG2NR, Chipset: Intel Device 2020, Memory: 6 x 4096 MB DDR4-2400MHz Micron 9ASF51272PZ-2G6B1, Disk: 4 x 120GB ADATA SU700 + 240GB Force MP500, Graphics: ASPEED ASPEED Family, Audio: Realtek ALC892, Monitor: VE228, Network: Intel I350 Gigabit Connection

	OS: Ubuntu 17.04, Kernel: 4.15.0-999-generic (x86_64) 20171212, Display Driver: modesetting 1.19.3, Compiler: GCC 6.3.0 20170406, File-System: ext4, Screen Resolution: 1920x1080

zfs_nvme_2driveraid0_intel: 

	Processor: 2 x Intel Xeon E5-2620 v4 @ 3.00GHz (16 Cores / 32 Threads), Motherboard: Supermicro X10DRU-i+ (2.0c BIOS), Chipset: Intel Xeon E7 v4/Xeon, Memory: 16 x 64 GB DDR4-2133MT/s Micron 72ASS8G72LZ-2G6B2, Disk: 2 x 128GB SAMSUNG MZ7LN128 + 2 x 2048GB Samsung SSD 850 + 2 x 1000GB INTEL SSDPE2KX010T7, Graphics: ASPEED ASPEED Family, Network: Intel 10-Gigabit X540-AT2

	OS: Fedora 27, Kernel: 4.15.3-300.fc27.x86_64 (x86_64), Compiler: GCC 7.3.1 20180130, File-System: zfs, Screen Resolution: 1024x768


SQLite 3.8.10.2
Test Target: Default Test Directory
Seconds < Lower Is Better
Btrfs 1 Disk ............... 60.90  |====
Btrfs 2 Disk RAID0 ......... 59.21  |====
Btrfs 2 Disk RAID1 ......... 64.93  |====
Btrfs 4 Disk RAID0 ......... 62.10  |====
Btrfs 4 Disk RAID1 ......... 66.73  |====
Btrfs 4 Disk RAID10 ........ 66.91  |====
Btrfs 4 Disk RAID5 ......... 140.27 |=========
Btrfs 4 Disk RAID6 ......... 139.08 |=========
EXT4 1 Disk ................ 27.74  |==
EXT4 2 Disk RAID1 .......... 394.48 |=========================
EXT4 2 Disk RAID0 .......... 37.72  |==
EXT4 4 Disk RAID0 .......... 41.22  |===
EXT4 4 Disk RAID1 .......... 454.42 |=============================
EXT4 4 Disk RAID5 .......... 418.50 |==========================
EXT4 4 Disk RAID6 .......... 669.24 |==========================================
EXT4 4 Disk RAID10 ......... 48.21  |===
zfs_nvme_2driveraid0_intel . 4.96   |


Flexible IO Tester 2.1.13
Type: Random Read - IO Engine: Libaio - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory
MB/s > Higher Is Better
Btrfs 1 Disk ........ 124.36 |=============
Btrfs 2 Disk RAID0 .. 220.66 |========================
Btrfs 2 Disk RAID1 .. 124.10 |=============
Btrfs 4 Disk RAID0 .. 292.70 |===============================
Btrfs 4 Disk RAID1 .. 241.61 |==========================
Btrfs 4 Disk RAID10 . 221.64 |========================
Btrfs 4 Disk RAID5 .. 384.54 |=========================================
Btrfs 4 Disk RAID6 .. 367.54 |=======================================
EXT4 1 Disk ......... 123.76 |=============
EXT4 2 Disk RAID1 ... 245.71 |==========================
EXT4 2 Disk RAID0 ... 242.12 |==========================
EXT4 4 Disk RAID0 ... 455.50 |=================================================
EXT4 4 Disk RAID1 ... 458.92 |=================================================
EXT4 4 Disk RAID5 ... 427.96 |==============================================
EXT4 4 Disk RAID6 ... 416.81 |=============================================
EXT4 4 Disk RAID10 .. 438.83 |===============================================


Flexible IO Tester 2.1.13
Type: Random Read - IO Engine: Libaio - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory
IOPS > Higher Is Better
Btrfs 1 Disk ........ 31086  |=============
Btrfs 2 Disk RAID0 .. 55162  |========================
Btrfs 2 Disk RAID1 .. 31021  |=============
Btrfs 4 Disk RAID0 .. 73172  |===============================
Btrfs 4 Disk RAID1 .. 60399  |==========================
Btrfs 4 Disk RAID10 . 55406  |========================
Btrfs 4 Disk RAID5 .. 96132  |=========================================
Btrfs 4 Disk RAID6 .. 91881  |=======================================
EXT4 1 Disk ......... 30936  |=============
EXT4 2 Disk RAID1 ... 61424  |==========================
EXT4 2 Disk RAID0 ... 60527  |==========================
EXT4 4 Disk RAID0 ... 113870 |=================================================
EXT4 4 Disk RAID1 ... 114726 |=================================================
EXT4 4 Disk RAID5 ... 106986 |==============================================
EXT4 4 Disk RAID6 ... 104198 |=============================================
EXT4 4 Disk RAID10 .. 109704 |===============================================


Flexible IO Tester 2.1.13
Type: Random Write - IO Engine: Libaio - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory
MB/s > Higher Is Better
Btrfs 1 Disk ........ 82.35  |==============
Btrfs 2 Disk RAID0 .. 64.76  |===========
Btrfs 2 Disk RAID1 .. 74.21  |=============
Btrfs 4 Disk RAID0 .. 66.17  |============
Btrfs 4 Disk RAID1 .. 40.61  |=======
Btrfs 4 Disk RAID10 . 34.41  |======
Btrfs 4 Disk RAID5 .. 68.31  |============
Btrfs 4 Disk RAID6 .. 68.21  |============
EXT4 1 Disk ......... 167.24 |=============================
EXT4 2 Disk RAID1 ... 33.57  |======
EXT4 2 Disk RAID0 ... 245.04 |===========================================
EXT4 4 Disk RAID0 ... 279.03 |=================================================
EXT4 4 Disk RAID1 ... 35.41  |======
EXT4 4 Disk RAID5 ... 25.93  |=====
EXT4 4 Disk RAID6 ... 17.09  |===
EXT4 4 Disk RAID10 .. 116.34 |====================


Flexible IO Tester 2.1.13
Type: Random Write - IO Engine: Libaio - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory
IOPS > Higher Is Better
Btrfs 1 Disk ........ 20583 |===============
Btrfs 2 Disk RAID0 .. 16186 |============
Btrfs 2 Disk RAID1 .. 18548 |=============
Btrfs 4 Disk RAID0 .. 16540 |============
Btrfs 4 Disk RAID1 .. 10149 |=======
Btrfs 4 Disk RAID10 . 8599  |======
Btrfs 4 Disk RAID5 .. 17074 |============
Btrfs 4 Disk RAID6 .. 17049 |============
EXT4 1 Disk ......... 41806 |==============================
EXT4 2 Disk RAID1 ... 8389  |======
EXT4 2 Disk RAID0 ... 61257 |============================================
EXT4 4 Disk RAID0 ... 69754 |==================================================
EXT4 4 Disk RAID1 ... 8850  |======
EXT4 4 Disk RAID5 ... 6480  |=====
EXT4 4 Disk RAID6 ... 4270  |===
EXT4 4 Disk RAID10 .. 29080 |=====================


Flexible IO Tester 2.1.13
Type: Sequential Read - IO Engine: Libaio - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory
MB/s > Higher Is Better
Btrfs 1 Disk ........ 109.25 |=============
Btrfs 2 Disk RAID0 .. 234.64 |=============================
Btrfs 2 Disk RAID1 .. 123.40 |===============
Btrfs 4 Disk RAID0 .. 302.93 |=====================================
Btrfs 4 Disk RAID1 .. 188.29 |=======================
Btrfs 4 Disk RAID10 . 197.04 |========================
Btrfs 4 Disk RAID5 .. 402.86 |=================================================
Btrfs 4 Disk RAID6 .. 323.48 |=======================================
EXT4 1 Disk ......... 76.04  |=========
EXT4 2 Disk RAID1 ... 76.70  |=========
EXT4 2 Disk RAID0 ... 144.92 |==================
EXT4 4 Disk RAID0 ... 143.94 |==================
EXT4 4 Disk RAID1 ... 82.30  |==========
EXT4 4 Disk RAID5 ... 151.50 |==================
EXT4 4 Disk RAID6 ... 196.79 |========================
EXT4 4 Disk RAID10 .. 217.34 |==========================


Flexible IO Tester 2.1.13
Type: Sequential Read - IO Engine: Libaio - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory
IOPS > Higher Is Better
Btrfs 1 Disk ........ 27309  |=============
Btrfs 2 Disk RAID0 .. 58657  |=============================
Btrfs 2 Disk RAID1 .. 30847  |===============
Btrfs 4 Disk RAID0 .. 75730  |=====================================
Btrfs 4 Disk RAID1 .. 47068  |=======================
Btrfs 4 Disk RAID10 . 49255  |========================
Btrfs 4 Disk RAID5 .. 100711 |=================================================
Btrfs 4 Disk RAID6 .. 80866  |=======================================
EXT4 1 Disk ......... 19008  |=========
EXT4 2 Disk RAID1 ... 19171  |=========
EXT4 2 Disk RAID0 ... 36226  |==================
EXT4 4 Disk RAID0 ... 35981  |==================
EXT4 4 Disk RAID1 ... 20571  |==========
EXT4 4 Disk RAID5 ... 37872  |==================
EXT4 4 Disk RAID6 ... 49194  |========================
EXT4 4 Disk RAID10 .. 54331  |==========================


Flexible IO Tester 2.1.13
Type: Sequential Write - IO Engine: Libaio - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory
MB/s > Higher Is Better
Btrfs 1 Disk ........ 81.37  |===================
Btrfs 2 Disk RAID0 .. 69.23  |================
Btrfs 2 Disk RAID1 .. 69.42  |================
Btrfs 4 Disk RAID0 .. 61.04  |==============
Btrfs 4 Disk RAID1 .. 47.71  |===========
Btrfs 4 Disk RAID10 . 43.08  |==========
Btrfs 4 Disk RAID5 .. 72.47  |=================
Btrfs 4 Disk RAID6 .. 73.17  |=================
EXT4 1 Disk ......... 213.91 |=================================================
EXT4 2 Disk RAID1 ... 36.73  |========
EXT4 2 Disk RAID0 ... 103.37 |========================
EXT4 4 Disk RAID0 ... 171.16 |=======================================
EXT4 4 Disk RAID1 ... 33.09  |========
EXT4 4 Disk RAID5 ... 33.22  |========
EXT4 4 Disk RAID6 ... 16.89  |====
EXT4 4 Disk RAID10 .. 71.29  |================


Flexible IO Tester 2.1.13
Type: Sequential Write - IO Engine: Libaio - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory
IOPS > Higher Is Better
Btrfs 1 Disk ........ 20340 |===================
Btrfs 2 Disk RAID0 .. 17304 |================
Btrfs 2 Disk RAID1 .. 17352 |================
Btrfs 4 Disk RAID0 .. 15256 |==============
Btrfs 4 Disk RAID1 .. 11923 |===========
Btrfs 4 Disk RAID10 . 10767 |==========
Btrfs 4 Disk RAID5 .. 18113 |=================
Btrfs 4 Disk RAID6 .. 18289 |=================
EXT4 1 Disk ......... 53473 |==================================================
EXT4 2 Disk RAID1 ... 9178  |=========
EXT4 2 Disk RAID0 ... 25838 |========================
EXT4 4 Disk RAID0 ... 42786 |========================================
EXT4 4 Disk RAID1 ... 8268  |========
EXT4 4 Disk RAID5 ... 8301  |========
EXT4 4 Disk RAID6 ... 4219  |====
EXT4 4 Disk RAID10 .. 17818 |=================


FS-Mark 3.3
Test: 1000 Files, 1MB Size
Files/s > Higher Is Better
Btrfs 1 Disk ............... 133.87 |==============
Btrfs 2 Disk RAID0 ......... 187.30 |====================
Btrfs 2 Disk RAID1 ......... 140.87 |===============
Btrfs 4 Disk RAID0 ......... 189.18 |====================
Btrfs 4 Disk RAID1 ......... 137.77 |===============
Btrfs 4 Disk RAID10 ........ 177.53 |===================
Btrfs 4 Disk RAID5 ......... 124.42 |=============
Btrfs 4 Disk RAID6 ......... 138.18 |===============
EXT4 1 Disk ................ 119.13 |=============
EXT4 2 Disk RAID1 .......... 28.73  |===
EXT4 2 Disk RAID0 .......... 71.93  |========
EXT4 4 Disk RAID0 .......... 101.43 |===========
EXT4 4 Disk RAID1 .......... 21.93  |==
EXT4 4 Disk RAID5 .......... 28.60  |===
EXT4 4 Disk RAID6 .......... 24.88  |===
EXT4 4 Disk RAID10 ......... 55.20  |======
zfs_nvme_2driveraid0_intel . 393.57 |==========================================


FS-Mark 3.3
Test: 5000 Files, 1MB Size, 4 Threads
Files/s > Higher Is Better
Btrfs 1 Disk ............... 148.15 |===========
Btrfs 2 Disk RAID0 ......... 322.97 |=========================
Btrfs 2 Disk RAID1 ......... 169.60 |=============
Btrfs 4 Disk RAID0 ......... 445.50 |==================================
Btrfs 4 Disk RAID1 ......... 190.00 |==============
Btrfs 4 Disk RAID10 ........ 287.30 |======================
Btrfs 4 Disk RAID5 ......... 280.90 |=====================
Btrfs 4 Disk RAID6 ......... 258.10 |====================
EXT4 1 Disk ................ 64.67  |=====
EXT4 2 Disk RAID1 .......... 35.47  |===
EXT4 2 Disk RAID0 .......... 79.87  |======
EXT4 4 Disk RAID0 .......... 175.33 |=============
EXT4 4 Disk RAID1 .......... 27.00  |==
EXT4 4 Disk RAID5 .......... 52.10  |====
EXT4 4 Disk RAID6 .......... 32.80  |===
EXT4 4 Disk RAID10 ......... 70.27  |=====
zfs_nvme_2driveraid0_intel . 550.63 |==========================================


FS-Mark 3.3
Test: 4000 Files, 32 Sub Dirs, 1MB Size
Files/s > Higher Is Better
Btrfs 1 Disk ............... 131.40 |==============
Btrfs 2 Disk RAID0 ......... 180.03 |===================
Btrfs 2 Disk RAID1 ......... 133.83 |==============
Btrfs 4 Disk RAID0 ......... 206.00 |======================
Btrfs 4 Disk RAID1 ......... 132.33 |==============
Btrfs 4 Disk RAID10 ........ 164.07 |==================
Btrfs 4 Disk RAID5 ......... 135.73 |===============
Btrfs 4 Disk RAID6 ......... 146.93 |================
EXT4 1 Disk ................ 41.35  |====
EXT4 2 Disk RAID1 .......... 29.75  |===
EXT4 2 Disk RAID0 .......... 53.13  |======
EXT4 4 Disk RAID0 .......... 101.37 |===========
EXT4 4 Disk RAID1 .......... 28.23  |===
EXT4 4 Disk RAID5 .......... 29.67  |===
EXT4 4 Disk RAID6 .......... 38.43  |====
EXT4 4 Disk RAID10 ......... 48.00  |=====
zfs_nvme_2driveraid0_intel . 388.03 |==========================================


BlogBench 1.0
Test: Read
Final Score > Higher Is Better
Btrfs 1 Disk ............... 2402392 |======================================
Btrfs 2 Disk RAID0 ......... 2372735 |======================================
Btrfs 2 Disk RAID1 ......... 2388227 |======================================
Btrfs 4 Disk RAID0 ......... 2400675 |======================================
Btrfs 4 Disk RAID1 ......... 2363039 |=====================================
Btrfs 4 Disk RAID10 ........ 2359105 |=====================================
Btrfs 4 Disk RAID5 ......... 2326341 |=====================================
Btrfs 4 Disk RAID6 ......... 2162115 |==================================
EXT4 1 Disk ................ 2477510 |=======================================
EXT4 2 Disk RAID1 .......... 2417606 |======================================
EXT4 2 Disk RAID0 .......... 2389094 |======================================
EXT4 4 Disk RAID0 .......... 1577870 |=========================
EXT4 4 Disk RAID1 .......... 2361663 |=====================================
EXT4 4 Disk RAID5 .......... 2534779 |========================================
EXT4 4 Disk RAID6 .......... 2585486 |=========================================
EXT4 4 Disk RAID10 ......... 2314883 |=====================================
zfs_nvme_2driveraid0_intel . 535474  |========


BlogBench 1.0
Test: Write
Final Score > Higher Is Better
Btrfs 1 Disk ............... 5079 |==========================
Btrfs 2 Disk RAID0 ......... 5746 |=============================
Btrfs 2 Disk RAID1 ......... 4773 |========================
Btrfs 4 Disk RAID0 ......... 5732 |=============================
Btrfs 4 Disk RAID1 ......... 5397 |===========================
Btrfs 4 Disk RAID10 ........ 5577 |============================
Btrfs 4 Disk RAID5 ......... 5136 |==========================
Btrfs 4 Disk RAID6 ......... 4655 |=======================
EXT4 1 Disk ................ 4212 |=====================
EXT4 2 Disk RAID1 .......... 4825 |========================
EXT4 2 Disk RAID0 .......... 5610 |============================
EXT4 4 Disk RAID0 .......... 8724 |============================================
EXT4 4 Disk RAID1 .......... 4692 |========================
EXT4 4 Disk RAID5 .......... 6243 |===============================
EXT4 4 Disk RAID6 .......... 5135 |==========================
EXT4 4 Disk RAID10 ......... 7769 |=======================================
zfs_nvme_2driveraid0_intel . 2995 |===============


Dbench 4.0
Client Count: 12
MB/s > Higher Is Better
Btrfs 1 Disk ............... 291.84  |=========
Btrfs 2 Disk RAID0 ......... 309.27  |==========
Btrfs 2 Disk RAID1 ......... 289.99  |=========
Btrfs 4 Disk RAID0 ......... 328.55  |===========
Btrfs 4 Disk RAID1 ......... 314.25  |==========
Btrfs 4 Disk RAID10 ........ 348.16  |===========
Btrfs 4 Disk RAID5 ......... 311.67  |==========
Btrfs 4 Disk RAID6 ......... 302.24  |==========
EXT4 1 Disk ................ 218.25  |=======
EXT4 2 Disk RAID1 .......... 160.56  |=====
EXT4 2 Disk RAID0 .......... 374.87  |============
EXT4 4 Disk RAID0 .......... 605.34  |===================
EXT4 4 Disk RAID1 .......... 91.30   |===
EXT4 4 Disk RAID5 .......... 129.00  |====
EXT4 4 Disk RAID6 .......... 106.08  |===
EXT4 4 Disk RAID10 ......... 302.57  |==========
zfs_nvme_2driveraid0_intel . 1278.83 |=========================================


IOzone 3.465
Record Size: 4Kb - File Size: 8GB - Disk Test: Write Performance
MB/s > Higher Is Better
Btrfs 1 Disk ............... 37.97  |==
Btrfs 2 Disk RAID0 ......... 76.16  |====
Btrfs 2 Disk RAID1 ......... 46.52  |==
Btrfs 4 Disk RAID0 ......... 900.57 |==========================================
Btrfs 4 Disk RAID1 ......... 182.49 |=========
Btrfs 4 Disk RAID10 ........ 142.13 |=======
Btrfs 4 Disk RAID5 ......... 114.48 |=====
Btrfs 4 Disk RAID6 ......... 75.62  |====
EXT4 1 Disk ................ 48.72  |==
EXT4 2 Disk RAID1 .......... 43.68  |==
EXT4 2 Disk RAID0 .......... 103.42 |=====
EXT4 4 Disk RAID0 .......... 168.87 |========
EXT4 4 Disk RAID1 .......... 42.09  |==
EXT4 4 Disk RAID5 .......... 80.43  |====
EXT4 4 Disk RAID6 .......... 59.66  |===
EXT4 4 Disk RAID10 ......... 94.16  |====
zfs_nvme_2driveraid0_intel . 448.55 |=====================


PostgreSQL pgbench 10.0
Scaling: Mostly RAM - Test: Normal Load - Mode: Read Write
TPS > Higher Is Better
Btrfs 1 Disk ........ 881.08  |========================
Btrfs 2 Disk RAID0 .. 1133.63 |===============================
Btrfs 2 Disk RAID1 .. 628.35  |=================
Btrfs 4 Disk RAID0 .. 1643.89 |=============================================
Btrfs 4 Disk RAID1 .. 1642.71 |=============================================
Btrfs 4 Disk RAID10 . 1041.85 |============================
Btrfs 4 Disk RAID5 .. 1370.47 |=====================================
Btrfs 4 Disk RAID6 .. 684.05  |===================
EXT4 1 Disk ......... 1086.05 |=============================
EXT4 2 Disk RAID1 ... 760.78  |=====================
EXT4 2 Disk RAID0 ... 1771.23 |================================================
EXT4 4 Disk RAID0 ... 1729.77 |===============================================
EXT4 4 Disk RAID1 ... 0.10    |
EXT4 4 Disk RAID5 ... 1033.44 |============================
EXT4 4 Disk RAID6 ... 1020.35 |============================
EXT4 4 Disk RAID10 .. 1310.87 |====================================