FreeBSD ZFS RAID Benchmarks AMD EPYC 7601 32-Core testing on FreeBSD via the Phoronix Test Suite. RAIDZ1 20 Disks: Processor: AMD EPYC 7601 32-Core @ 2.20GHz (128 Cores), Motherboard: Dell PowerEdge R7425, Chipset: AMD [AMD] Family 17h (Models 00h-0fh) Root Complex, Memory: 524032MB, Disk: 20 x ATA Samsung SSD 860 1B6Q + DP BP14G+EXP 2.17 + SSDSCKJB120G7R N201DL43 OS: FreeBSD, Kernel: 12.0-RC3 (x86_64), Compiler: Clang 6.0.1 (SVN 335540), File-System: zfs, Screen Resolution: 1024x768 Stripe 20 Disks: Processor: AMD EPYC 7601 32-Core @ 2.20GHz (128 Cores), Motherboard: Dell PowerEdge R7425, Chipset: AMD [AMD] Family 17h (Models 00h-0fh) Root Complex, Memory: 524032MB, Disk: 20 x ATA Samsung SSD 860 1B6Q + DP BP14G+EXP 2.17 + SSDSCKJB120G7R N201DL43 OS: FreeBSD, Kernel: 12.0-RC3 (x86_64), Compiler: Clang 6.0.1 (SVN 335540), File-System: zfs, Screen Resolution: 1024x768 RAIDZ3 20 Disks: Processor: AMD EPYC 7601 32-Core @ 2.20GHz (128 Cores), Motherboard: Dell PowerEdge R7425, Chipset: AMD [AMD] Family 17h (Models 00h-0fh) Root Complex, Memory: 524032MB, Disk: 20 x ATA Samsung SSD 860 1B6Q + DP BP14G+EXP 2.17 + SSDSCKJB120G7R N201DL43 OS: FreeBSD, Kernel: 12.0-RC3 (x86_64), Compiler: Clang 6.0.1 (SVN 335540), File-System: zfs, Screen Resolution: 1024x768 SQLite 3.22 Timed SQLite Insertions Seconds < Lower Is Better RAIDZ1 20 Disks . 35.14 |================================================ Stripe 20 Disks . 26.18 |=================================== RAIDZ3 20 Disks . 39.91 |====================================================== Flexible IO Tester 3.1 Type: Random Read - IO Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 2MB - Disk Target: Default Test Directory IOPS > Higher Is Better RAIDZ1 20 Disks . 19200 |====================================================== Stripe 20 Disks . 19100 |====================================================== RAIDZ3 20 Disks . 19133 |====================================================== Flexible IO Tester 3.1 Type: Random Read - IO Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory MB/s > Higher Is Better RAIDZ1 20 Disks . 238 |======================================================== Stripe 20 Disks . 238 |======================================================== RAIDZ3 20 Disks . 238 |======================================================== Flexible IO Tester 3.1 Type: Random Read - IO Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory IOPS > Higher Is Better RAIDZ1 20 Disks . 60933 |====================================================== Stripe 20 Disks . 61000 |====================================================== RAIDZ3 20 Disks . 60900 |====================================================== Flexible IO Tester 3.1 Type: Random Write - IO Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 2MB - Disk Target: Default Test Directory MB/s > Higher Is Better RAIDZ1 20 Disks . 8125 |====================================================== Stripe 20 Disks . 6576 |============================================ RAIDZ3 20 Disks . 8290 |======================================================= Flexible IO Tester 3.1 Type: Random Write - IO Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory MB/s > Higher Is Better RAIDZ1 20 Disks . 92.25 |====================================================== Stripe 20 Disks . 90.87 |===================================================== RAIDZ3 20 Disks . 79.13 |============================================== Flexible IO Tester 3.1 Type: Random Write - IO Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory IOPS > Higher Is Better RAIDZ1 20 Disks . 23600 |====================================================== Stripe 20 Disks . 23267 |===================================================== RAIDZ3 20 Disks . 20267 |============================================== Flexible IO Tester 3.1 Type: Sequential Read - IO Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 2MB - Disk Target: Default Test Directory IOPS > Higher Is Better RAIDZ1 20 Disks . 18900 |===================================================== Stripe 20 Disks . 19333 |====================================================== RAIDZ3 20 Disks . 19133 |===================================================== Flexible IO Tester 3.1 Type: Sequential Read - IO Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory MB/s > Higher Is Better RAIDZ1 20 Disks . 238 |====================================================== Stripe 20 Disks . 245 |======================================================== RAIDZ3 20 Disks . 241 |======================================================= Flexible IO Tester 3.1 Type: Sequential Read - IO Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory IOPS > Higher Is Better RAIDZ1 20 Disks . 60867 |==================================================== Stripe 20 Disks . 62767 |====================================================== RAIDZ3 20 Disks . 61633 |===================================================== Flexible IO Tester 3.1 Type: Sequential Write - IO Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 2MB - Disk Target: Default Test Directory MB/s > Higher Is Better RAIDZ1 20 Disks . 8085 |====================================================== Stripe 20 Disks . 6995 |=============================================== RAIDZ3 20 Disks . 8207 |======================================================= Flexible IO Tester 3.1 Type: Sequential Write - IO Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory MB/s > Higher Is Better RAIDZ1 20 Disks . 180 |======================================================== Stripe 20 Disks . 179 |======================================================== RAIDZ3 20 Disks . 171 |===================================================== Flexible IO Tester 3.1 Type: Sequential Write - IO Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory IOPS > Higher Is Better RAIDZ1 20 Disks . 46133 |====================================================== Stripe 20 Disks . 45800 |====================================================== RAIDZ3 20 Disks . 43833 |=================================================== BlogBench 1.1 Test: Read Final Score > Higher Is Better RAIDZ1 20 Disks . 163372 |===================================================== Stripe 20 Disks . 161812 |==================================================== RAIDZ3 20 Disks . 156970 |=================================================== BlogBench 1.1 Test: Write Final Score > Higher Is Better RAIDZ1 20 Disks . 3147 |======================================================= Stripe 20 Disks . 3156 |======================================================= RAIDZ3 20 Disks . 3018 |===================================================== IOzone 3.465 Record Size: 1MB - File Size: 8GB - Disk Test: Read Performance MB/s > Higher Is Better RAIDZ1 20 Disks . 6639 |======================================================= Stripe 20 Disks . 6691 |======================================================= RAIDZ3 20 Disks . 6694 |======================================================= IOzone 3.465 Record Size: 4Kb - File Size: 8GB - Disk Test: Read Performance MB/s > Higher Is Better RAIDZ1 20 Disks . 2096 |======================================================= Stripe 20 Disks . 2107 |======================================================= RAIDZ3 20 Disks . 2111 |======================================================= IOzone 3.465 Record Size: 1MB - File Size: 8GB - Disk Test: Write Performance MB/s > Higher Is Better RAIDZ1 20 Disks . 2587 |=============================================== Stripe 20 Disks . 3043 |======================================================= RAIDZ3 20 Disks . 2594 |=============================================== IOzone 3.465 Record Size: 4Kb - File Size: 8GB - Disk Test: Write Performance MB/s > Higher Is Better RAIDZ1 20 Disks . 445 |======================================================== Stripe 20 Disks . 399 |================================================== RAIDZ3 20 Disks . 419 |===================================================== IOzone 3.465 Record Size: 64Kb - File Size: 8GB - Disk Test: Read Performance MB/s > Higher Is Better RAIDZ1 20 Disks . 6077 |======================================================= Stripe 20 Disks . 6107 |======================================================= RAIDZ3 20 Disks . 6117 |======================================================= IOzone 3.465 Record Size: 1MB - File Size: 512MB - Disk Test: Read Performance MB/s > Higher Is Better RAIDZ1 20 Disks . 6389 |======================================================= Stripe 20 Disks . 6380 |====================================================== RAIDZ3 20 Disks . 6440 |======================================================= IOzone 3.465 Record Size: 4Kb - File Size: 512MB - Disk Test: Read Performance MB/s > Higher Is Better RAIDZ1 20 Disks . 2085 |====================================================== Stripe 20 Disks . 2072 |====================================================== RAIDZ3 20 Disks . 2117 |======================================================= IOzone 3.465 Record Size: 64Kb - File Size: 8GB - Disk Test: Write Performance MB/s > Higher Is Better RAIDZ1 20 Disks . 1650 |================================================== Stripe 20 Disks . 1803 |======================================================= RAIDZ3 20 Disks . 1658 |=================================================== IOzone 3.465 Record Size: 1MB - File Size: 512MB - Disk Test: Write Performance MB/s > Higher Is Better RAIDZ1 20 Disks . 2048 |======================================================= Stripe 20 Disks . 1911 |=================================================== RAIDZ3 20 Disks . 1671 |============================================= IOzone 3.465 Record Size: 4Kb - File Size: 512MB - Disk Test: Write Performance MB/s > Higher Is Better RAIDZ1 20 Disks . 371 |====================================================== Stripe 20 Disks . 388 |======================================================== RAIDZ3 20 Disks . 361 |==================================================== IOzone 3.465 Record Size: 64Kb - File Size: 512MB - Disk Test: Read Performance MB/s > Higher Is Better RAIDZ1 20 Disks . 5921 |======================================================= Stripe 20 Disks . 5972 |======================================================= RAIDZ3 20 Disks . 5614 |==================================================== IOzone 3.465 Record Size: 64Kb - File Size: 512MB - Disk Test: Write Performance MB/s > Higher Is Better RAIDZ1 20 Disks . 1372 |======================================================= Stripe 20 Disks . 1331 |===================================================== RAIDZ3 20 Disks . 1342 |====================================================== Compile Bench 0.6 Test: Compile MB/s > Higher Is Better RAIDZ1 20 Disks . 925 |================================================= Stripe 20 Disks . 1045 |======================================================= RAIDZ3 20 Disks . 890 |=============================================== Compile Bench 0.6 Test: Initial Create MB/s > Higher Is Better RAIDZ1 20 Disks . 136 |======================================================= Stripe 20 Disks . 139 |======================================================== RAIDZ3 20 Disks . 133 |====================================================== PostMark 1.51 Disk Transaction Performance TPS > Higher Is Better RAIDZ1 20 Disks . 3232 |====================================================== Stripe 20 Disks . 3318 |======================================================= RAIDZ3 20 Disks . 3260 |====================================================== PostgreSQL pgbench 10.3 Scaling: Buffer Test - Test: Normal Load - Mode: Read Only TPS > Higher Is Better RAIDZ1 20 Disks . 41943 |===================================================== Stripe 20 Disks . 42403 |====================================================== RAIDZ3 20 Disks . 41800 |===================================================== PostgreSQL pgbench 10.3 Scaling: Buffer Test - Test: Normal Load - Mode: Read Write TPS > Higher Is Better RAIDZ1 20 Disks . 1915 |===================================== Stripe 20 Disks . 2880 |======================================================= RAIDZ3 20 Disks . 1878 |==================================== OSBench Test: Create Files us Per Event < Lower Is Better RAIDZ1 20 Disks . 76.73 |=================================================== Stripe 20 Disks . 80.55 |====================================================== RAIDZ3 20 Disks . 80.07 |======================================================