45Drives_XL60 Intel Xeon Silver 4216 testing on FreeBSD via the Phoronix Test Suite. 10 VDEV with Six Drives Per VDEV RaidZ2: Processor: Intel Xeon Silver 4216 (32 Cores), Motherboard: 45Drives XL60 0123456789, Memory: 256GB, Disk: 648814GB OS: FreeBSD, Kernel: 12.2-RELEASE-p10 (x86_64), Compiler: Clang 10.0.1, File-System: zfs 6 VDEV with Ten Drives Per VDEV RaidZ2: Processor: Intel Xeon Silver 4216 (32 Cores), Motherboard: 45Drives XL60 0123456789, Memory: 256GB, Disk: 648814GB OS: FreeBSD, Kernel: 12.2-RELEASE-p10 (x86_64), Compiler: Clang 10.0.1, File-System: zfs 6 VDEVs with Ten Drives Per RaidZ2: Processor: Intel Xeon Silver 4216 (32 Cores), Motherboard: 45Drives XL60 0123456789, Memory: 256GB, Disk: 742118GB OS: FreeBSD, Kernel: 12.2-RELEASE-p10 (x86_64), Compiler: Clang 10.0.1, File-System: zfs 10 VDEVs RaidZ2: Processor: Intel Xeon Silver 4216 (32 Cores), Motherboard: 45Drives XL60 0123456789, Memory: 256GB, Disk: 742118GB OS: FreeBSD, Kernel: 12.2-RELEASE-p10 (x86_64), Compiler: Clang 10.0.1, File-System: zfs Flexible IO Tester 3.25 Type: Random Read - Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory MB/s > Higher Is Better 10 VDEV with Six Drives Per VDEV RaidZ2 . 697 |================================ 10 VDEV with Six Drives Per VDEV RaidZ2 . 693 |================================ 6 VDEV with Ten Drives Per VDEV RaidZ2 .. 696 |================================ 6 VDEVs with Ten Drives Per RaidZ2 ...... 702 |================================ 10 VDEVs RaidZ2 ......................... 690 |=============================== Flexible IO Tester 3.25 Type: Random Read - Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory IOPS > Higher Is Better 10 VDEV with Six Drives Per VDEV RaidZ2 . 178333 |============================= 10 VDEV with Six Drives Per VDEV RaidZ2 . 177000 |============================= 6 VDEV with Ten Drives Per VDEV RaidZ2 .. 178000 |============================= 6 VDEVs with Ten Drives Per RaidZ2 ...... 179667 |============================= 10 VDEVs RaidZ2 ......................... 176333 |============================ Flexible IO Tester 3.25 Type: Random Write - Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory MB/s > Higher Is Better 10 VDEV with Six Drives Per VDEV RaidZ2 . 461 |================================ 6 VDEV with Ten Drives Per VDEV RaidZ2 .. 370 |========================== 6 VDEVs with Ten Drives Per RaidZ2 ...... 372 |========================== 10 VDEVs RaidZ2 ......................... 355 |========================= Flexible IO Tester 3.25 Type: Random Write - Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory IOPS > Higher Is Better 10 VDEV with Six Drives Per VDEV RaidZ2 . 118000 |============================= 6 VDEV with Ten Drives Per VDEV RaidZ2 .. 94700 |======================= 6 VDEVs with Ten Drives Per RaidZ2 ...... 95333 |======================= 10 VDEVs RaidZ2 ......................... 90533 |====================== Flexible IO Tester 3.25 Type: Random Read - Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 128KB - Disk Target: Default Test Directory IOPS > Higher Is Better 10 VDEV with Six Drives Per VDEV RaidZ2 . 140333 |========================= 6 VDEV with Ten Drives Per VDEV RaidZ2 .. 158667 |============================ 6 VDEVs with Ten Drives Per RaidZ2 ...... 161667 |============================= 10 VDEVs RaidZ2 ......................... 158333 |============================ Flexible IO Tester 3.25 Type: Random Write - Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 128KB - Disk Target: Default Test Directory IOPS > Higher Is Better 10 VDEV with Six Drives Per VDEV RaidZ2 . 95400 |============================== 6 VDEV with Ten Drives Per VDEV RaidZ2 .. 70867 |====================== 6 VDEVs with Ten Drives Per RaidZ2 ...... 76167 |======================== 10 VDEVs RaidZ2 ......................... 74233 |======================= Flexible IO Tester 3.25 Type: Sequential Read - Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory MB/s > Higher Is Better 10 VDEV with Six Drives Per VDEV RaidZ2 . 756 |=============================== 6 VDEV with Ten Drives Per VDEV RaidZ2 .. 754 |=============================== 6 VDEVs with Ten Drives Per RaidZ2 ...... 770 |================================ 10 VDEVs RaidZ2 ......................... 764 |================================ Flexible IO Tester 3.25 Type: Sequential Read - Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory IOPS > Higher Is Better 10 VDEV with Six Drives Per VDEV RaidZ2 . 193667 |============================= 6 VDEV with Ten Drives Per VDEV RaidZ2 .. 193000 |============================ 6 VDEVs with Ten Drives Per RaidZ2 ...... 197000 |============================= 10 VDEVs RaidZ2 ......................... 195333 |============================= Flexible IO Tester 3.25 Type: Sequential Write - Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory MB/s > Higher Is Better 10 VDEV with Six Drives Per VDEV RaidZ2 . 515 |================================ 6 VDEV with Ten Drives Per VDEV RaidZ2 .. 503 |=============================== 6 VDEVs with Ten Drives Per RaidZ2 ...... 494 |=============================== 10 VDEVs RaidZ2 ......................... 494 |=============================== Flexible IO Tester 3.25 Type: Sequential Write - Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory IOPS > Higher Is Better 10 VDEV with Six Drives Per VDEV RaidZ2 . 132000 |============================= 6 VDEV with Ten Drives Per VDEV RaidZ2 .. 128667 |============================ 6 VDEVs with Ten Drives Per RaidZ2 ...... 126667 |============================ 10 VDEVs RaidZ2 ......................... 126000 |============================ Flexible IO Tester 3.25 Type: Sequential Read - Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 128KB - Disk Target: Default Test Directory IOPS > Higher Is Better 10 VDEV with Six Drives Per VDEV RaidZ2 . 134667 |========================= 6 VDEV with Ten Drives Per VDEV RaidZ2 .. 153667 |============================= 6 VDEVs with Ten Drives Per RaidZ2 ...... 156333 |============================= 10 VDEVs RaidZ2 ......................... 156333 |============================= Flexible IO Tester 3.25 Type: Sequential Write - Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 128KB - Disk Target: Default Test Directory IOPS > Higher Is Better 10 VDEV with Six Drives Per VDEV RaidZ2 . 94133 |============================== 6 VDEV with Ten Drives Per VDEV RaidZ2 .. 70633 |======================= 6 VDEVs with Ten Drives Per RaidZ2 ...... 76767 |======================== 10 VDEVs RaidZ2 ......................... 75067 |======================== Flexible IO Tester 3.25 Type: Random Write - Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 128KB - Disk Target: Default Test Directory MB/s > Higher Is Better 6 VDEV with Ten Drives Per VDEV RaidZ2 . 8858 |============================== 6 VDEVs with Ten Drives Per RaidZ2 ..... 9524 |================================ 10 VDEVs RaidZ2 ........................ 9280 |=============================== Flexible IO Tester 3.25 Type: Sequential Write - Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 128KB - Disk Target: Default Test Directory MB/s > Higher Is Better 6 VDEV with Ten Drives Per VDEV RaidZ2 . 8832 |============================= 6 VDEVs with Ten Drives Per RaidZ2 ..... 9593 |================================ 10 VDEVs RaidZ2 ........................ 9380 |===============================