FreeBSD ZFS RAID Benchmarks AMD EPYC 7601 32-Core testing on FreeBSD via the Phoronix Test Suite. RAIDZ1 20 Disks: Processor: AMD EPYC 7601 32-Core @ 2.20GHz (128 Cores), Motherboard: Dell PowerEdge R7425, Chipset: AMD [AMD] Family 17h (Models 00h-0fh) Root Complex, Memory: 524032MB, Disk: 20 x ATA Samsung SSD 860 1B6Q + DP BP14G+EXP 2.17 + SSDSCKJB120G7R N201DL43 OS: FreeBSD, Kernel: 12.0-RC3 (x86_64), Compiler: Clang 6.0.1 (SVN 335540), File-System: zfs, Screen Resolution: 1024x768 Stripe 20 Disks: Processor: AMD EPYC 7601 32-Core @ 2.20GHz (128 Cores), Motherboard: Dell PowerEdge R7425, Chipset: AMD [AMD] Family 17h (Models 00h-0fh) Root Complex, Memory: 524032MB, Disk: 20 x ATA Samsung SSD 860 1B6Q + DP BP14G+EXP 2.17 + SSDSCKJB120G7R N201DL43 OS: FreeBSD, Kernel: 12.0-RC3 (x86_64), Compiler: Clang 6.0.1 (SVN 335540), File-System: zfs, Screen Resolution: 1024x768 RAIDZ3 20 Disks: Processor: AMD EPYC 7601 32-Core @ 2.20GHz (128 Cores), Motherboard: Dell PowerEdge R7425, Chipset: AMD [AMD] Family 17h (Models 00h-0fh) Root Complex, Memory: 524032MB, Disk: 20 x ATA Samsung SSD 860 1B6Q + DP BP14G+EXP 2.17 + SSDSCKJB120G7R N201DL43 OS: FreeBSD, Kernel: 12.0-RC3 (x86_64), Compiler: Clang 6.0.1 (SVN 335540), File-System: zfs, Screen Resolution: 1024x768 Single Disk: Processor: AMD EPYC 7601 32-Core @ 2.20GHz (128 Cores), Motherboard: Dell PowerEdge R7425, Chipset: AMD [AMD] Family 17h (Models 00h-0fh) Root Complex, Memory: 524032MB, Disk: ATA Samsung SSD 860 1B6Q + DP BP14G+EXP 2.17 + SSDSCKJB120G7R N201DL43 OS: FreeBSD, Kernel: 12.0-RC3 (x86_64), Compiler: Clang 6.0.1 (SVN 335540), File-System: zfs, Screen Resolution: 1024x768 SQLite 3.22 Timed SQLite Insertions Seconds < Lower Is Better RAIDZ1 20 Disks . 35.14 |================================================ Stripe 20 Disks . 26.18 |=================================== RAIDZ3 20 Disks . 39.91 |====================================================== Single Disk ..... 25.02 |================================== Flexible IO Tester 3.1 Type: Random Read - IO Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 2MB - Disk Target: Default Test Directory IOPS > Higher Is Better RAIDZ1 20 Disks . 19200 |====================================================== Stripe 20 Disks . 19100 |====================================================== RAIDZ3 20 Disks . 19133 |====================================================== Single Disk ..... 18467 |==================================================== Flexible IO Tester 3.1 Type: Random Read - IO Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory MB/s > Higher Is Better RAIDZ1 20 Disks . 238 |======================================================== Stripe 20 Disks . 238 |======================================================== RAIDZ3 20 Disks . 238 |======================================================== Single Disk ..... 240 |======================================================== Flexible IO Tester 3.1 Type: Random Read - IO Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory IOPS > Higher Is Better RAIDZ1 20 Disks . 60933 |====================================================== Stripe 20 Disks . 61000 |====================================================== RAIDZ3 20 Disks . 60900 |====================================================== Single Disk ..... 61267 |====================================================== Flexible IO Tester 3.1 Type: Random Write - IO Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 2MB - Disk Target: Default Test Directory MB/s > Higher Is Better RAIDZ1 20 Disks . 8125 |====================================================== Stripe 20 Disks . 6576 |============================================ RAIDZ3 20 Disks . 8290 |======================================================= Flexible IO Tester 3.1 Type: Random Write - IO Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory MB/s > Higher Is Better RAIDZ1 20 Disks . 92.25 |========================= Stripe 20 Disks . 90.87 |========================= RAIDZ3 20 Disks . 79.13 |====================== Single Disk ..... 192.00 |===================================================== Flexible IO Tester 3.1 Type: Random Write - IO Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory IOPS > Higher Is Better RAIDZ1 20 Disks . 23600 |========================== Stripe 20 Disks . 23267 |========================== RAIDZ3 20 Disks . 20267 |====================== Single Disk ..... 49167 |====================================================== Flexible IO Tester 3.1 Type: Sequential Read - IO Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 2MB - Disk Target: Default Test Directory IOPS > Higher Is Better RAIDZ1 20 Disks . 18900 |===================================================== Stripe 20 Disks . 19333 |====================================================== RAIDZ3 20 Disks . 19133 |===================================================== Single Disk ..... 18833 |===================================================== Flexible IO Tester 3.1 Type: Sequential Read - IO Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory MB/s > Higher Is Better RAIDZ1 20 Disks . 238 |====================================================== Stripe 20 Disks . 245 |======================================================== RAIDZ3 20 Disks . 241 |======================================================= Single Disk ..... 240 |======================================================= Flexible IO Tester 3.1 Type: Sequential Read - IO Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory IOPS > Higher Is Better RAIDZ1 20 Disks . 60867 |==================================================== Stripe 20 Disks . 62767 |====================================================== RAIDZ3 20 Disks . 61633 |===================================================== Single Disk ..... 61433 |===================================================== Flexible IO Tester 3.1 Type: Sequential Write - IO Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 2MB - Disk Target: Default Test Directory MB/s > Higher Is Better RAIDZ1 20 Disks . 8085 |====================================================== Stripe 20 Disks . 6995 |=============================================== RAIDZ3 20 Disks . 8207 |======================================================= Flexible IO Tester 3.1 Type: Sequential Write - IO Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory MB/s > Higher Is Better RAIDZ1 20 Disks . 180 |================================================== Stripe 20 Disks . 179 |================================================== RAIDZ3 20 Disks . 171 |================================================ Single Disk ..... 200 |======================================================== Flexible IO Tester 3.1 Type: Sequential Write - IO Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory IOPS > Higher Is Better RAIDZ1 20 Disks . 46133 |================================================= Stripe 20 Disks . 45800 |================================================ RAIDZ3 20 Disks . 43833 |============================================== Single Disk ..... 51267 |====================================================== BlogBench 1.1 Test: Read Final Score > Higher Is Better RAIDZ1 20 Disks . 163372 |===================================================== Stripe 20 Disks . 161812 |==================================================== RAIDZ3 20 Disks . 156970 |=================================================== Single Disk ..... 161269 |==================================================== BlogBench 1.1 Test: Write Final Score > Higher Is Better RAIDZ1 20 Disks . 3147 |====================================================== Stripe 20 Disks . 3156 |====================================================== RAIDZ3 20 Disks . 3018 |==================================================== Single Disk ..... 3208 |======================================================= IOzone 3.465 Record Size: 1MB - File Size: 8GB - Disk Test: Read Performance MB/s > Higher Is Better RAIDZ1 20 Disks . 6639 |======================================================= Stripe 20 Disks . 6691 |======================================================= RAIDZ3 20 Disks . 6694 |======================================================= Single Disk ..... 6646 |======================================================= IOzone 3.465 Record Size: 4Kb - File Size: 8GB - Disk Test: Read Performance MB/s > Higher Is Better RAIDZ1 20 Disks . 2096 |======================================================= Stripe 20 Disks . 2107 |======================================================= RAIDZ3 20 Disks . 2111 |======================================================= Single Disk ..... 2107 |======================================================= IOzone 3.465 Record Size: 1MB - File Size: 8GB - Disk Test: Write Performance MB/s > Higher Is Better RAIDZ1 20 Disks . 2587 |========================================= Stripe 20 Disks . 3043 |================================================ RAIDZ3 20 Disks . 2594 |========================================= Single Disk ..... 3499 |======================================================= IOzone 3.465 Record Size: 4Kb - File Size: 8GB - Disk Test: Write Performance MB/s > Higher Is Better RAIDZ1 20 Disks . 445 |=================================================== Stripe 20 Disks . 399 |============================================== RAIDZ3 20 Disks . 419 |================================================ Single Disk ..... 488 |======================================================== IOzone 3.465 Record Size: 64Kb - File Size: 8GB - Disk Test: Read Performance MB/s > Higher Is Better RAIDZ1 20 Disks . 6077 |======================================================= Stripe 20 Disks . 6107 |======================================================= RAIDZ3 20 Disks . 6117 |======================================================= Single Disk ..... 6055 |====================================================== IOzone 3.465 Record Size: 1MB - File Size: 512MB - Disk Test: Read Performance MB/s > Higher Is Better RAIDZ1 20 Disks . 6389 |===================================================== Stripe 20 Disks . 6380 |===================================================== RAIDZ3 20 Disks . 6440 |===================================================== Single Disk ..... 6630 |======================================================= IOzone 3.465 Record Size: 4Kb - File Size: 512MB - Disk Test: Read Performance MB/s > Higher Is Better RAIDZ1 20 Disks . 2085 |====================================================== Stripe 20 Disks . 2072 |====================================================== RAIDZ3 20 Disks . 2117 |======================================================= Single Disk ..... 2109 |======================================================= IOzone 3.465 Record Size: 64Kb - File Size: 8GB - Disk Test: Write Performance MB/s > Higher Is Better RAIDZ1 20 Disks . 1650 |========================================= Stripe 20 Disks . 1803 |============================================= RAIDZ3 20 Disks . 1658 |========================================= Single Disk ..... 2206 |======================================================= IOzone 3.465 Record Size: 1MB - File Size: 512MB - Disk Test: Write Performance MB/s > Higher Is Better RAIDZ1 20 Disks . 2048 |================================================ Stripe 20 Disks . 1911 |============================================ RAIDZ3 20 Disks . 1671 |======================================= Single Disk ..... 2370 |======================================================= IOzone 3.465 Record Size: 4Kb - File Size: 512MB - Disk Test: Write Performance MB/s > Higher Is Better RAIDZ1 20 Disks . 371 |=================================================== Stripe 20 Disks . 388 |===================================================== RAIDZ3 20 Disks . 361 |================================================= Single Disk ..... 409 |======================================================== IOzone 3.465 Record Size: 64Kb - File Size: 512MB - Disk Test: Read Performance MB/s > Higher Is Better RAIDZ1 20 Disks . 5921 |======================================================= Stripe 20 Disks . 5972 |======================================================= RAIDZ3 20 Disks . 5614 |==================================================== Single Disk ..... 5693 |==================================================== IOzone 3.465 Record Size: 64Kb - File Size: 512MB - Disk Test: Write Performance MB/s > Higher Is Better RAIDZ1 20 Disks . 1372 |============================================ Stripe 20 Disks . 1331 |========================================== RAIDZ3 20 Disks . 1342 |=========================================== Single Disk ..... 1728 |======================================================= Compile Bench 0.6 Test: Compile MB/s > Higher Is Better RAIDZ1 20 Disks . 925 |============================================= Stripe 20 Disks . 1045 |================================================== RAIDZ3 20 Disks . 890 |=========================================== Single Disk ..... 1141 |======================================================= Compile Bench 0.6 Test: Initial Create MB/s > Higher Is Better RAIDZ1 20 Disks . 136 |====================================================== Stripe 20 Disks . 139 |======================================================== RAIDZ3 20 Disks . 133 |===================================================== Single Disk ..... 140 |======================================================== PostMark 1.51 Disk Transaction Performance TPS > Higher Is Better RAIDZ1 20 Disks . 3232 |========================================= Stripe 20 Disks . 3318 |========================================== RAIDZ3 20 Disks . 3260 |========================================== Single Disk ..... 4312 |======================================================= PostgreSQL pgbench 10.3 Scaling: Buffer Test - Test: Normal Load - Mode: Read Only TPS > Higher Is Better RAIDZ1 20 Disks . 41943 |===================================================== Stripe 20 Disks . 42403 |====================================================== RAIDZ3 20 Disks . 41800 |===================================================== Single Disk ..... 42780 |====================================================== PostgreSQL pgbench 10.3 Scaling: Buffer Test - Test: Normal Load - Mode: Read Write TPS > Higher Is Better RAIDZ1 20 Disks . 1915 |===================================== Stripe 20 Disks . 2880 |======================================================= RAIDZ3 20 Disks . 1878 |==================================== Single Disk ..... 2141 |========================================= OSBench Test: Create Files us Per Event < Lower Is Better RAIDZ1 20 Disks . 76.73 |=================================================== Stripe 20 Disks . 80.55 |====================================================== RAIDZ3 20 Disks . 80.07 |====================================================== Single Disk ..... 77.71 |====================================================