45drivesxl60 2 x AMD EPYC 7513 32-Core testing with a Dell PowerEdge R7525 [0590KW] (2.6.6 BIOS) and Matrox G200eW3 on Debian GNU/Linux 11 via the Phoronix Test Suite. ,,"10 Per VDEV RaidZ2","5 Per VDEV RaidZ2","Mirrors","DELL ENT NVME SSD1","DELL SAMSUNG SSD4","DELL ENTERPRISE NVME","SAMSUNG ZFS MIRROR","DELLENT ZFS MIRROR","PERC RAID1 SAMSUNG 256-A-D-C","PERC RAID1 DELL ENT. H755","4xNVME ZFS MIRROR","DELLENT ZFS MIRROR2","SAMSUNG ZFS MIRROR2" Processor,,Intel Xeon Silver 4216 (32 Cores),Intel Xeon Silver 4216 (32 Cores),Intel Xeon Silver 4216 (32 Cores),2 x AMD EPYC 7513 32-Core (64 Cores / 128 Threads),2 x AMD EPYC 7513 32-Core (64 Cores / 128 Threads),2 x AMD EPYC 7513 32-Core (64 Cores / 128 Threads),2 x AMD EPYC 7513 32-Core (64 Cores / 128 Threads),2 x AMD EPYC 7513 32-Core (64 Cores / 128 Threads),2 x AMD EPYC 7513 32-Core (64 Cores / 128 Threads),2 x AMD EPYC 7513 32-Core (64 Cores / 128 Threads),2 x AMD EPYC 7513 32-Core (64 Cores / 128 Threads),2 x AMD EPYC 7513 32-Core (64 Cores / 128 Threads),2 x AMD EPYC 7513 32-Core (64 Cores / 128 Threads) Motherboard,,45Drives XL60 0123456789,45Drives XL60 0123456789,45Drives XL60 0123456789,Dell PowerEdge R7525 [0590KW] (2.6.6 BIOS),Dell PowerEdge R7525 [0590KW] (2.6.6 BIOS),Dell PowerEdge R7525 [0590KW] (2.6.6 BIOS),Dell PowerEdge R7525 [0590KW] (2.6.6 BIOS),Dell PowerEdge R7525 [0590KW] (2.6.6 BIOS),Dell PowerEdge R7525 [0590KW] (2.6.6 BIOS),Dell PowerEdge R7525 [0590KW] (2.6.6 BIOS),Dell PowerEdge R7525 [0590KW] (2.6.6 BIOS),Dell PowerEdge R7525 [0590KW] (2.6.6 BIOS),Dell PowerEdge R7525 [0590KW] (2.6.6 BIOS) Memory,,256GB,256GB,256GB,8 x 64 GB DDR4-3200MT/s 36ASF8G72PZ-3G2B2,8 x 64 GB DDR4-3200MT/s 36ASF8G72PZ-3G2B2,8 x 64 GB DDR4-3200MT/s 36ASF8G72PZ-3G2B2,8 x 64 GB DDR4-3200MT/s 36ASF8G72PZ-3G2B2,8 x 64 GB DDR4-3200MT/s 36ASF8G72PZ-3G2B2,8 x 64 GB DDR4-3200MT/s 36ASF8G72PZ-3G2B2,8 x 64 GB DDR4-3200MT/s 36ASF8G72PZ-3G2B2,8 x 64 GB DDR4-3200MT/s 36ASF8G72PZ-3G2B2,8 x 64 GB DDR4-3200MT/s 36ASF8G72PZ-3G2B2,8 x 64 GB DDR4-3200MT/s 36ASF8G72PZ-3G2B2 Disk,,742117GB,742117GB,742117GB,240GB DELLBOSS VD + 2 x 1920GB Dell Ent NVMe CM + 2 x 3841GB SAMSUNG MZQL23T8,240GB DELLBOSS VD + 2 x 1920GB Dell Ent NVMe CM + 2 x 3841GB SAMSUNG MZQL23T8,240GB DELLBOSS VD + 2 x 1920GB Dell Ent NVMe CM + 2 x 3841GB SAMSUNG MZQL23T8,240GB DELLBOSS VD + 2 x 1920GB Dell Ent NVMe CM + 2 x 3841GB SAMSUNG MZQL23T8,240GB DELLBOSS VD + 2 x 1920GB Dell Ent NVMe CM + 2 x 3841GB SAMSUNG MZQL23T8,240GB DELLBOSS VD + 2 x 1920GB Dell Ent NVMe CM + 2 x 3841GB SAMSUNG MZQL23T8,240GB DELLBOSS VD + 2 x 1920GB Dell Ent NVMe CM + 2 x 3841GB SAMSUNG MZQL23T8,240GB DELLBOSS VD + 2 x 1920GB Dell Ent NVMe CM + 2 x 3841GB SAMSUNG MZQL23T8,240GB DELLBOSS VD + 2 x 1920GB Dell Ent NVMe CM + 2 x 3841GB SAMSUNG MZQL23T8,240GB DELLBOSS VD + 2 x 1920GB Dell Ent NVMe CM + 2 x 3841GB SAMSUNG MZQL23T8 Chipset,,,,,AMD Starship/Matisse,AMD Starship/Matisse,AMD Starship/Matisse,AMD Starship/Matisse,AMD Starship/Matisse,AMD Starship/Matisse,AMD Starship/Matisse,AMD Starship/Matisse,AMD Starship/Matisse,AMD Starship/Matisse Graphics,,,,,Matrox G200eW3,Matrox G200eW3,Matrox G200eW3,Matrox G200eW3,Matrox G200eW3,Matrox G200eW3,Matrox G200eW3,Matrox G200eW3,Matrox G200eW3,Matrox G200eW3 Monitor,,,,,DELL E172FP,DELL E172FP,DELL E172FP,DELL E172FP,DELL E172FP,DELL E172FP,DELL E172FP,DELL E172FP,DELL E172FP,DELL E172FP Network,,,,,4 x Intel X710 for 10GBASE-T + 2 x Broadcom NetXtreme BCM5720 2-port PCIe,4 x Intel X710 for 10GBASE-T + 2 x Broadcom NetXtreme BCM5720 2-port PCIe,4 x Intel X710 for 10GBASE-T + 2 x Broadcom NetXtreme BCM5720 2-port PCIe,4 x Intel X710 for 10GBASE-T + 2 x Broadcom NetXtreme BCM5720 2-port PCIe,4 x Intel X710 for 10GBASE-T + 2 x Broadcom NetXtreme BCM5720 2-port PCIe,4 x Intel X710 for 10GBASE-T + 2 x Broadcom NetXtreme BCM5720 2-port PCIe,4 x Intel X710 for 10GBASE-T + 2 x Broadcom NetXtreme BCM5720 2-port PCIe,4 x Intel X710 for 10GBASE-T + 2 x Broadcom NetXtreme BCM5720 2-port PCIe,4 x Intel X710 for 10GBASE-T + 2 x Broadcom NetXtreme BCM5720 2-port PCIe,4 x Intel X710 for 10GBASE-T + 2 x Broadcom NetXtreme BCM5720 2-port PCIe OS,,FreeBSD,FreeBSD,FreeBSD,Debian GNU/Linux 11,Debian GNU/Linux 11,Debian GNU/Linux 11,Debian GNU/Linux 11,Debian GNU/Linux 11,Debian GNU/Linux 11,Debian GNU/Linux 11,Debian GNU/Linux 11,Debian GNU/Linux 11,Debian GNU/Linux 11 Kernel,,12.2-RELEASE-p10 (x86_64),12.2-RELEASE-p10 (x86_64),12.2-RELEASE-p10 (x86_64),5.15.30-2-pve (x86_64),5.15.30-2-pve (x86_64),5.15.30-2-pve (x86_64),5.15.30-2-pve (x86_64),5.15.30-2-pve (x86_64),5.15.30-2-pve (x86_64),5.15.30-2-pve (x86_64),5.15.30-2-pve (x86_64),5.15.30-2-pve (x86_64),5.15.30-2-pve (x86_64) Compiler,,Clang 10.0.1,Clang 10.0.1,Clang 10.0.1,GCC 10.2.1 20210110,GCC 10.2.1 20210110,GCC 10.2.1 20210110,GCC 10.2.1 20210110,GCC 10.2.1 20210110,GCC 10.2.1 20210110,GCC 10.2.1 20210110,GCC 10.2.1 20210110,GCC 10.2.1 20210110,GCC 10.2.1 20210110 File-System,,zfs,zfs,zfs,ext4,ext4,ext4,zfs,zfs,ext4,ext4,zfs,zfs,zfs Vulkan,,,,,1.0.2,1.0.2,1.0.2,1.0.2,1.0.2,1.0.2,1.0.2,1.0.2,1.0.2,1.0.2 Screen Resolution,,,,,1280x1024,1280x1024,1280x1024,1280x1024,1280x1024,1280x1024,1280x1024,1280x1024,1280x1024,1280x1024 ,,"10 Per VDEV RaidZ2","5 Per VDEV RaidZ2","Mirrors","DELL ENT NVME SSD1","DELL SAMSUNG SSD4","DELL ENTERPRISE NVME","SAMSUNG ZFS MIRROR","DELLENT ZFS MIRROR","PERC RAID1 SAMSUNG 256-A-D-C","PERC RAID1 DELL ENT. H755","4xNVME ZFS MIRROR","DELLENT ZFS MIRROR2","SAMSUNG ZFS MIRROR2" "Flexible IO Tester - Type: Random Read - Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory (IOPS)",HIB,175333,177333,178333,6991,9825,12100,99133,98583,20533,46800,93392,107000,97383 "Flexible IO Tester - Type: Random Write - Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory (IOPS)",HIB,92300,110333,129333,18260,6242,61233,180000,185000,32633,42942,118800,182000,180333 "Flexible IO Tester - Type: Random Read - Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 128KB - Disk Target: Default Test Directory (IOPS)",HIB,156667,157000,156333,1561,6327,7633,58733,57367,9613,12767,57225,57167,57600 "Flexible IO Tester - Type: Random Write - Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 128KB - Disk Target: Default Test Directory (IOPS)",HIB,72700,93500,95733,1617,16400,18300,49633,51633,22400,18700,44233,51433,51867 "Flexible IO Tester - Type: Sequential Read - Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory (IOPS)",HIB,188000,188000,189333,7183,63067,22500,440133,434467,34367,52100,459867,423929,461000 "Flexible IO Tester - Type: Sequential Write - Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory (IOPS)",HIB,127000,135667,137667,17871,6385,61500,353917,333333,39767,48167,315333,319600,335000 "Flexible IO Tester - Type: Sequential Read - Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 128KB - Disk Target: Default Test Directory (IOPS)",HIB,146933,131333,132600,1553,17133,9457,54433,52847,21700,13167,53707,53575,52067 "Flexible IO Tester - Type: Sequential Write - Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 128KB - Disk Target: Default Test Directory (IOPS)",HIB,73033,91300,99900,1623,16633,18300,49140,48446,22400,19787,38433,49700,48936 "Flexible IO Tester - Type: Random Read - Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory (MB/s)",HIB,685,692,697,27.3,38.4,47.2,388,386,80.5,183,365,418,381 "Flexible IO Tester - Type: Random Write - Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory (MB/s)",HIB,360,432,505,71.3,24.4,240,703,721,128,168,464,711,704 "Flexible IO Tester - Type: Random Write - Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 128KB - Disk Target: Default Test Directory (MB/s)",HIB,9083,,,203,2054,2286,6210,6454,2798,2334,5530,6430,6482 "Flexible IO Tester - Type: Sequential Read - Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory (MB/s)",HIB,735,735,739,28.1,246,87.6,1720,1698,134,204,1797,1656,1801 "Flexible IO Tester - Type: Sequential Write - Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory (MB/s)",HIB,495,529,537,70.3,24.3,240,1382,1302,157,188,1232,1248,1309 "Flexible IO Tester - Type: Sequential Write - Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 128KB - Disk Target: Default Test Directory (MB/s)",HIB,9131,,,203,2080,2285,6145,6074,2801,2475,4805,6213,6117 "Flexible IO Tester - Type: Random Read - Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 128KB - Disk Target: Default Test Directory (MB/s)",HIB,,,,196,791,955,7340,7172,1202,1591,7155,7146,7198 "Flexible IO Tester - Type: Sequential Read - Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 128KB - Disk Target: Default Test Directory (MB/s)",HIB,,,,195,2138,1182,6804,6622,2754,1645,6724,6698,6511