45drivesxl60 2 x AMD EPYC 7513 32-Core testing with a Dell PowerEdge R7525 [0590KW] (2.6.6 BIOS) and Matrox G200eW3 on Debian GNU/Linux 11 via the Phoronix Test Suite.
HTML result view exported from: https://openbenchmarking.org/result/2209212-NE-2209211NE32&grr&sro .
45drivesxl60 Processor Motherboard Memory Disk Chipset Graphics Monitor Network OS Kernel Compiler File-System Vulkan Screen Resolution 10 Per VDEV RaidZ2 5 Per VDEV RaidZ2 Mirrors DELL ENT NVME SSD1 DELL SAMSUNG SSD4 DELL ENTERPRISE NVME SAMSUNG ZFS MIRROR DELLENT ZFS MIRROR PERC RAID1 SAMSUNG 256-A-D-C PERC RAID1 DELL ENT. H755 4xNVME ZFS MIRROR DELLENT ZFS MIRROR2 SAMSUNG ZFS MIRROR2 Intel Xeon Silver 4216 (32 Cores) 45Drives XL60 0123456789 256GB 742117GB FreeBSD 12.2-RELEASE-p10 (x86_64) Clang 10.0.1 zfs 2 x AMD EPYC 7513 32-Core (64 Cores / 128 Threads) Dell PowerEdge R7525 [0590KW] (2.6.6 BIOS) AMD Starship/Matisse 8 x 64 GB DDR4-3200MT/s 36ASF8G72PZ-3G2B2 240GB DELLBOSS VD + 2 x 1920GB Dell Ent NVMe CM + 2 x 3841GB SAMSUNG MZQL23T8 Matrox G200eW3 DELL E172FP 4 x Intel X710 for 10GBASE-T + 2 x Broadcom NetXtreme BCM5720 2-port PCIe Debian GNU/Linux 11 5.15.30-2-pve (x86_64) 1.0.2 GCC 10.2.1 20210110 ext4 1280x1024 zfs ext4 zfs OpenBenchmarking.org Kernel Details - DELL ENT NVME SSD1, DELL SAMSUNG SSD4, DELL ENTERPRISE NVME, SAMSUNG ZFS MIRROR, DELLENT ZFS MIRROR, PERC RAID1 SAMSUNG 256-A-D-C, PERC RAID1 DELL ENT. H755, 4xNVME ZFS MIRROR, DELLENT ZFS MIRROR2, SAMSUNG ZFS MIRROR2: Transparent Huge Pages: madvise Compiler Details - DELL ENT NVME SSD1, DELL SAMSUNG SSD4, DELL ENTERPRISE NVME, SAMSUNG ZFS MIRROR, DELLENT ZFS MIRROR, PERC RAID1 SAMSUNG 256-A-D-C, PERC RAID1 DELL ENT. H755, 4xNVME ZFS MIRROR, DELLENT ZFS MIRROR2, SAMSUNG ZFS MIRROR2: --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-link-mutex --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-targets=nvptx-none=/build/gcc-10-Km9U7s/gcc-10-10.2.1/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-10-Km9U7s/gcc-10-10.2.1/debian/tmp-gcn/usr,hsa --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-build-config=bootstrap-lto-lean --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v Disk Details - DELL ENT NVME SSD1: MQ-DEADLINE / errors=remount-ro,relatime,rw / Block Size: 4096 - DELL SAMSUNG SSD4: MQ-DEADLINE / errors=remount-ro,relatime,rw / Block Size: 4096 - DELL ENTERPRISE NVME: MQ-DEADLINE / errors=remount-ro,relatime,rw / Block Size: 4096 - SAMSUNG ZFS MIRROR: MQ-DEADLINE / errors=remount-ro,relatime,rw / Block Size: 131072 - DELLENT ZFS MIRROR: MQ-DEADLINE / errors=remount-ro,relatime,rw / Block Size: 131072 - PERC RAID1 SAMSUNG 256-A-D-C: MQ-DEADLINE / errors=remount-ro,relatime,rw / Block Size: 4096 - PERC RAID1 DELL ENT. H755: MQ-DEADLINE / errors=remount-ro,relatime,rw / Block Size: 4096 - 4xNVME ZFS MIRROR: MQ-DEADLINE / errors=remount-ro,relatime,rw / Block Size: 131072 - DELLENT ZFS MIRROR2: MQ-DEADLINE / errors=remount-ro,relatime,rw / Block Size: 131072 - SAMSUNG ZFS MIRROR2: MQ-DEADLINE / errors=remount-ro,relatime,rw / Block Size: 131072 Processor Details - DELL ENT NVME SSD1, DELL SAMSUNG SSD4, DELL ENTERPRISE NVME, SAMSUNG ZFS MIRROR, DELLENT ZFS MIRROR, PERC RAID1 SAMSUNG 256-A-D-C, PERC RAID1 DELL ENT. H755, 4xNVME ZFS MIRROR, DELLENT ZFS MIRROR2, SAMSUNG ZFS MIRROR2: CPU Microcode: 0xa00115d Security Details - DELL ENT NVME SSD1, DELL SAMSUNG SSD4, DELL ENTERPRISE NVME, SAMSUNG ZFS MIRROR, DELLENT ZFS MIRROR, PERC RAID1 SAMSUNG 256-A-D-C, PERC RAID1 DELL ENT. H755, 4xNVME ZFS MIRROR, DELLENT ZFS MIRROR2, SAMSUNG ZFS MIRROR2: itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl and seccomp + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional IBRS_FW STIBP: always-on RSB filling + srbds: Not affected + tsx_async_abort: Not affected
45drivesxl60 fio: Seq Write - POSIX AIO - No - Yes - 4KB - Default Test Directory fio: Seq Write - POSIX AIO - No - Yes - 4KB - Default Test Directory fio: Rand Write - POSIX AIO - No - Yes - 4KB - Default Test Directory fio: Rand Write - POSIX AIO - No - Yes - 4KB - Default Test Directory fio: Seq Read - POSIX AIO - No - Yes - 128KB - Default Test Directory fio: Seq Read - POSIX AIO - No - Yes - 4KB - Default Test Directory fio: Seq Read - POSIX AIO - No - Yes - 4KB - Default Test Directory fio: Seq Read - POSIX AIO - No - Yes - 128KB - Default Test Directory fio: Seq Write - POSIX AIO - No - Yes - 128KB - Default Test Directory fio: Rand Read - POSIX AIO - No - Yes - 4KB - Default Test Directory fio: Seq Write - POSIX AIO - No - Yes - 128KB - Default Test Directory fio: Rand Read - POSIX AIO - No - Yes - 4KB - Default Test Directory fio: Rand Read - POSIX AIO - No - Yes - 128KB - Default Test Directory fio: Rand Read - POSIX AIO - No - Yes - 128KB - Default Test Directory fio: Rand Write - POSIX AIO - No - Yes - 128KB - Default Test Directory fio: Rand Write - POSIX AIO - No - Yes - 128KB - Default Test Directory 10 Per VDEV RaidZ2 5 Per VDEV RaidZ2 Mirrors DELL ENT NVME SSD1 DELL SAMSUNG SSD4 DELL ENTERPRISE NVME SAMSUNG ZFS MIRROR DELLENT ZFS MIRROR PERC RAID1 SAMSUNG 256-A-D-C PERC RAID1 DELL ENT. H755 4xNVME ZFS MIRROR DELLENT ZFS MIRROR2 SAMSUNG ZFS MIRROR2 127000 495 92300 360 146933 188000 735 9131 685 73033 175333 156667 72700 9083 135667 529 110333 432 131333 188000 735 692 91300 177333 157000 93500 137667 537 129333 505 132600 189333 739 697 99900 178333 156333 95733 17871 70.3 18260 71.3 1553 7183 28.1 195 203 27.3 1623 6991 196 1561 1617 203 6385 24.3 6242 24.4 17133 63067 246 2138 2080 38.4 16633 9825 791 6327 16400 2054 61500 240 61233 240 9457 22500 87.6 1182 2285 47.2 18300 12100 955 7633 18300 2286 353917 1382 180000 703 54433 440133 1720 6804 6145 388 49140 99133 7340 58733 49633 6210 333333 1302 185000 721 52847 434467 1698 6622 6074 386 48446 98583 7172 57367 51633 6454 39767 157 32633 128 21700 34367 134 2754 2801 80.5 22400 20533 1202 9613 22400 2798 48167 188 42942 168 13167 52100 204 1645 2475 183 19787 46800 1591 12767 18700 2334 315333 1232 118800 464 53707 459867 1797 6724 4805 365 38433 93392 7155 57225 44233 5530 319600 1248 182000 711 53575 423929 1656 6698 6213 418 49700 107000 7146 57167 51433 6430 335000 1309 180333 704 52067 461000 1801 6511 6117 381 48936 97383 7198 57600 51867 6482 OpenBenchmarking.org
Flexible IO Tester Type: Sequential Write - Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory OpenBenchmarking.org IOPS, More Is Better Flexible IO Tester 3.25 Type: Sequential Write - Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory 10 Per VDEV RaidZ2 4xNVME ZFS MIRROR 5 Per VDEV RaidZ2 DELL ENT NVME SSD1 DELL ENTERPRISE NVME DELL SAMSUNG SSD4 DELLENT ZFS MIRROR DELLENT ZFS MIRROR2 Mirrors PERC RAID1 DELL ENT. H755 PERC RAID1 SAMSUNG 256-A-D-C SAMSUNG ZFS MIRROR SAMSUNG ZFS MIRROR2 80K 160K 240K 320K 400K SE +/- 7030.32, N = 15 SE +/- 333.33, N = 3 SE +/- 180.48, N = 14 SE +/- 57.74, N = 3 SE +/- 1.20, N = 3 SE +/- 3073.18, N = 15 SE +/- 4423.75, N = 15 SE +/- 333.33, N = 3 SE +/- 185.59, N = 3 SE +/- 384.42, N = 3 SE +/- 5411.07, N = 12 SE +/- 3386.25, N = 6 127000 315333 135667 17871 61500 6385 333333 319600 137667 48167 39767 353917 335000
Flexible IO Tester Type: Sequential Write - Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory OpenBenchmarking.org MB/s, More Is Better Flexible IO Tester 3.25 Type: Sequential Write - Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory 10 Per VDEV RaidZ2 4xNVME ZFS MIRROR 5 Per VDEV RaidZ2 DELL ENT NVME SSD1 DELL ENTERPRISE NVME DELL SAMSUNG SSD4 DELLENT ZFS MIRROR DELLENT ZFS MIRROR2 Mirrors PERC RAID1 DELL ENT. H755 PERC RAID1 SAMSUNG 256-A-D-C SAMSUNG ZFS MIRROR SAMSUNG ZFS MIRROR2 300 600 900 1200 1500 SE +/- 27.37, N = 15 SE +/- 0.73, N = 14 SE +/- 0.30, N = 3 SE +/- 11.95, N = 15 SE +/- 17.29, N = 15 SE +/- 1.20, N = 3 SE +/- 0.67, N = 3 SE +/- 21.11, N = 12 SE +/- 13.25, N = 6 495.0 1232.0 529.0 70.3 240.0 24.3 1302.0 1248.0 537.0 188.0 157.0 1382.0 1309.0
Flexible IO Tester Type: Random Write - Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory OpenBenchmarking.org IOPS, More Is Better Flexible IO Tester 3.25 Type: Random Write - Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory 10 Per VDEV RaidZ2 4xNVME ZFS MIRROR 5 Per VDEV RaidZ2 DELL ENT NVME SSD1 DELL ENTERPRISE NVME DELL SAMSUNG SSD4 DELLENT ZFS MIRROR DELLENT ZFS MIRROR2 Mirrors PERC RAID1 DELL ENT. H755 PERC RAID1 SAMSUNG 256-A-D-C SAMSUNG ZFS MIRROR SAMSUNG ZFS MIRROR2 40K 80K 120K 160K 200K SE +/- 416.33, N = 3 SE +/- 1313.66, N = 15 SE +/- 881.92, N = 3 SE +/- 222.92, N = 15 SE +/- 66.67, N = 3 SE +/- 1.86, N = 3 SE +/- 1527.53, N = 3 SE +/- 1944.47, N = 15 SE +/- 1452.97, N = 3 SE +/- 1055.04, N = 12 SE +/- 283.12, N = 15 SE +/- 2728.45, N = 3 92300 118800 110333 18260 61233 6242 185000 182000 129333 42942 32633 180000 180333
Flexible IO Tester Type: Random Write - Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory OpenBenchmarking.org MB/s, More Is Better Flexible IO Tester 3.25 Type: Random Write - Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory 10 Per VDEV RaidZ2 4xNVME ZFS MIRROR 5 Per VDEV RaidZ2 DELL ENT NVME SSD1 DELL ENTERPRISE NVME DELL SAMSUNG SSD4 DELLENT ZFS MIRROR DELLENT ZFS MIRROR2 Mirrors PERC RAID1 DELL ENT. H755 PERC RAID1 SAMSUNG 256-A-D-C SAMSUNG ZFS MIRROR SAMSUNG ZFS MIRROR2 160 320 480 640 800 SE +/- 1.76, N = 3 SE +/- 4.95, N = 15 SE +/- 2.33, N = 3 SE +/- 0.86, N = 15 SE +/- 0.33, N = 3 SE +/- 0.00, N = 3 SE +/- 6.23, N = 3 SE +/- 7.59, N = 15 SE +/- 5.61, N = 3 SE +/- 4.12, N = 12 SE +/- 1.07, N = 15 SE +/- 0.67, N = 3 SE +/- 10.09, N = 3 360.0 464.0 432.0 71.3 240.0 24.4 721.0 711.0 505.0 168.0 128.0 703.0 704.0
Flexible IO Tester Type: Sequential Read - Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 128KB - Disk Target: Default Test Directory OpenBenchmarking.org IOPS, More Is Better Flexible IO Tester 3.25 Type: Sequential Read - Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 128KB - Disk Target: Default Test Directory 10 Per VDEV RaidZ2 4xNVME ZFS MIRROR 5 Per VDEV RaidZ2 DELL ENT NVME SSD1 DELL ENTERPRISE NVME DELL SAMSUNG SSD4 DELLENT ZFS MIRROR DELLENT ZFS MIRROR2 Mirrors PERC RAID1 DELL ENT. H755 PERC RAID1 SAMSUNG 256-A-D-C SAMSUNG ZFS MIRROR SAMSUNG ZFS MIRROR2 30K 60K 90K 120K 150K SE +/- 2369.38, N = 15 SE +/- 548.72, N = 15 SE +/- 333.33, N = 3 SE +/- 4.10, N = 3 SE +/- 4.33, N = 3 SE +/- 33.33, N = 3 SE +/- 522.53, N = 15 SE +/- 673.76, N = 4 SE +/- 1400.00, N = 5 SE +/- 88.19, N = 3 SE +/- 351.19, N = 3 SE +/- 693.62, N = 3 SE +/- 837.89, N = 15 146933 53707 131333 1553 9457 17133 52847 53575 132600 13167 21700 54433 52067
Flexible IO Tester Type: Sequential Read - Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory OpenBenchmarking.org IOPS, More Is Better Flexible IO Tester 3.25 Type: Sequential Read - Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory 10 Per VDEV RaidZ2 4xNVME ZFS MIRROR 5 Per VDEV RaidZ2 DELL ENT NVME SSD1 DELL ENTERPRISE NVME DELL SAMSUNG SSD4 DELLENT ZFS MIRROR DELLENT ZFS MIRROR2 Mirrors PERC RAID1 DELL ENT. H755 PERC RAID1 SAMSUNG 256-A-D-C SAMSUNG ZFS MIRROR SAMSUNG ZFS MIRROR2 100K 200K 300K 400K 500K SE +/- 577.35, N = 3 SE +/- 8694.21, N = 15 SE +/- 577.35, N = 3 SE +/- 11.72, N = 3 SE +/- 33.33, N = 3 SE +/- 7940.17, N = 15 SE +/- 11135.26, N = 14 SE +/- 333.33, N = 3 SE +/- 100.00, N = 3 SE +/- 33.33, N = 3 SE +/- 10885.49, N = 15 SE +/- 1527.53, N = 3 188000 459867 188000 7183 22500 63067 434467 423929 189333 52100 34367 440133 461000
Flexible IO Tester Type: Sequential Read - Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory OpenBenchmarking.org MB/s, More Is Better Flexible IO Tester 3.25 Type: Sequential Read - Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory 10 Per VDEV RaidZ2 4xNVME ZFS MIRROR 5 Per VDEV RaidZ2 DELL ENT NVME SSD1 DELL ENTERPRISE NVME DELL SAMSUNG SSD4 DELLENT ZFS MIRROR DELLENT ZFS MIRROR2 Mirrors PERC RAID1 DELL ENT. H755 PERC RAID1 SAMSUNG 256-A-D-C SAMSUNG ZFS MIRROR SAMSUNG ZFS MIRROR2 400 800 1200 1600 2000 SE +/- 1.76, N = 3 SE +/- 33.83, N = 15 SE +/- 1.76, N = 3 SE +/- 0.06, N = 3 SE +/- 0.32, N = 3 SE +/- 0.33, N = 3 SE +/- 30.86, N = 15 SE +/- 43.36, N = 14 SE +/- 1.67, N = 3 SE +/- 0.33, N = 3 SE +/- 42.44, N = 15 SE +/- 6.77, N = 3 735.0 1797.0 735.0 28.1 87.6 246.0 1698.0 1656.0 739.0 204.0 134.0 1720.0 1801.0
Flexible IO Tester Type: Sequential Read - Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 128KB - Disk Target: Default Test Directory OpenBenchmarking.org MB/s, More Is Better Flexible IO Tester 3.25 Type: Sequential Read - Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 128KB - Disk Target: Default Test Directory 4xNVME ZFS MIRROR DELL ENT NVME SSD1 DELL ENTERPRISE NVME DELL SAMSUNG SSD4 DELLENT ZFS MIRROR DELLENT ZFS MIRROR2 PERC RAID1 DELL ENT. H755 PERC RAID1 SAMSUNG 256-A-D-C SAMSUNG ZFS MIRROR SAMSUNG ZFS MIRROR2 1500 3000 4500 6000 7500 SE +/- 70.16, N = 15 SE +/- 0.67, N = 3 SE +/- 4.18, N = 3 SE +/- 59.56, N = 15 SE +/- 83.02, N = 4 SE +/- 7.62, N = 3 SE +/- 6.81, N = 3 SE +/- 88.50, N = 3 SE +/- 104.53, N = 15 6724 195 1182 2138 6622 6698 1645 2754 6804 6511 1. (CC) gcc options: -rdynamic -ll -lnuma -lrt -lz -lpthread -lm -ldl -laio -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native
Flexible IO Tester Type: Sequential Write - Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 128KB - Disk Target: Default Test Directory OpenBenchmarking.org MB/s, More Is Better Flexible IO Tester 3.25 Type: Sequential Write - Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 128KB - Disk Target: Default Test Directory 10 Per VDEV RaidZ2 4xNVME ZFS MIRROR DELL ENT NVME SSD1 DELL ENTERPRISE NVME DELL SAMSUNG SSD4 DELLENT ZFS MIRROR DELLENT ZFS MIRROR2 PERC RAID1 DELL ENT. H755 PERC RAID1 SAMSUNG 256-A-D-C SAMSUNG ZFS MIRROR SAMSUNG ZFS MIRROR2 2K 4K 6K 8K 10K SE +/- 114.13, N = 3 SE +/- 17.49, N = 3 SE +/- 0.67, N = 3 SE +/- 0.67, N = 3 SE +/- 1.20, N = 3 SE +/- 85.11, N = 13 SE +/- 44.24, N = 3 SE +/- 17.87, N = 15 SE +/- 1.86, N = 3 SE +/- 46.95, N = 10 SE +/- 78.92, N = 14 9131 4805 203 2285 2080 6074 6213 2475 2801 6145 6117
Flexible IO Tester Type: Random Read - Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory OpenBenchmarking.org MB/s, More Is Better Flexible IO Tester 3.25 Type: Random Read - Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory 10 Per VDEV RaidZ2 4xNVME ZFS MIRROR 5 Per VDEV RaidZ2 DELL ENT NVME SSD1 DELL ENTERPRISE NVME DELL SAMSUNG SSD4 DELLENT ZFS MIRROR DELLENT ZFS MIRROR2 Mirrors PERC RAID1 DELL ENT. H755 PERC RAID1 SAMSUNG 256-A-D-C SAMSUNG ZFS MIRROR SAMSUNG ZFS MIRROR2 150 300 450 600 750 SE +/- 3.06, N = 3 SE +/- 21.18, N = 12 SE +/- 2.67, N = 3 SE +/- 0.06, N = 3 SE +/- 0.00, N = 3 SE +/- 0.00, N = 3 SE +/- 11.53, N = 12 SE +/- 2.08, N = 3 SE +/- 2.85, N = 3 SE +/- 7.35, N = 15 SE +/- 1.33, N = 3 SE +/- 13.16, N = 12 685.0 365.0 692.0 27.3 47.2 38.4 386.0 418.0 697.0 183.0 80.5 388.0 381.0
Flexible IO Tester Type: Sequential Write - Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 128KB - Disk Target: Default Test Directory OpenBenchmarking.org IOPS, More Is Better Flexible IO Tester 3.25 Type: Sequential Write - Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 128KB - Disk Target: Default Test Directory 10 Per VDEV RaidZ2 4xNVME ZFS MIRROR 5 Per VDEV RaidZ2 DELL ENT NVME SSD1 DELL ENTERPRISE NVME DELL SAMSUNG SSD4 DELLENT ZFS MIRROR DELLENT ZFS MIRROR2 Mirrors PERC RAID1 DELL ENT. H755 PERC RAID1 SAMSUNG 256-A-D-C SAMSUNG ZFS MIRROR SAMSUNG ZFS MIRROR2 20K 40K 60K 80K 100K SE +/- 924.36, N = 3 SE +/- 133.33, N = 3 SE +/- 709.46, N = 3 SE +/- 4.84, N = 3 SE +/- 33.33, N = 3 SE +/- 752.31, N = 13 SE +/- 351.19, N = 3 SE +/- 1081.67, N = 3 SE +/- 141.04, N = 15 SE +/- 376.59, N = 10 SE +/- 633.98, N = 14 73033 38433 91300 1623 18300 16633 48446 49700 99900 19787 22400 49140 48936
Flexible IO Tester Type: Random Read - Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory OpenBenchmarking.org IOPS, More Is Better Flexible IO Tester 3.25 Type: Random Read - Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory 10 Per VDEV RaidZ2 4xNVME ZFS MIRROR 5 Per VDEV RaidZ2 DELL ENT NVME SSD1 DELL ENTERPRISE NVME DELL SAMSUNG SSD4 DELLENT ZFS MIRROR DELLENT ZFS MIRROR2 Mirrors PERC RAID1 DELL ENT. H755 PERC RAID1 SAMSUNG 256-A-D-C SAMSUNG ZFS MIRROR SAMSUNG ZFS MIRROR2 40K 80K 120K 160K 200K SE +/- 666.67, N = 3 SE +/- 5435.76, N = 12 SE +/- 666.67, N = 3 SE +/- 14.33, N = 3 SE +/- 4.63, N = 3 SE +/- 2927.48, N = 12 SE +/- 577.35, N = 3 SE +/- 666.67, N = 3 SE +/- 1886.59, N = 15 SE +/- 366.67, N = 3 SE +/- 3366.00, N = 12 175333 93392 177333 6991 12100 9825 98583 107000 178333 46800 20533 99133 97383
Flexible IO Tester Type: Random Read - Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 128KB - Disk Target: Default Test Directory OpenBenchmarking.org MB/s, More Is Better Flexible IO Tester 3.25 Type: Random Read - Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 128KB - Disk Target: Default Test Directory 4xNVME ZFS MIRROR DELL ENT NVME SSD1 DELL ENTERPRISE NVME DELL SAMSUNG SSD4 DELLENT ZFS MIRROR DELLENT ZFS MIRROR2 PERC RAID1 DELL ENT. H755 PERC RAID1 SAMSUNG 256-A-D-C SAMSUNG ZFS MIRROR SAMSUNG ZFS MIRROR2 1600 3200 4800 6400 8000 SE +/- 83.90, N = 4 SE +/- 0.33, N = 3 SE +/- 0.33, N = 3 SE +/- 5.24, N = 3 SE +/- 69.68, N = 3 SE +/- 48.90, N = 3 SE +/- 3.18, N = 3 SE +/- 3.38, N = 3 SE +/- 74.71, N = 3 SE +/- 71.04, N = 3 7155 196 955 791 7172 7146 1591 1202 7340 7198 1. (CC) gcc options: -rdynamic -ll -lnuma -lrt -lz -lpthread -lm -ldl -laio -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native
Flexible IO Tester Type: Random Read - Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 128KB - Disk Target: Default Test Directory OpenBenchmarking.org IOPS, More Is Better Flexible IO Tester 3.25 Type: Random Read - Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 128KB - Disk Target: Default Test Directory 10 Per VDEV RaidZ2 4xNVME ZFS MIRROR 5 Per VDEV RaidZ2 DELL ENT NVME SSD1 DELL ENTERPRISE NVME DELL SAMSUNG SSD4 DELLENT ZFS MIRROR DELLENT ZFS MIRROR2 Mirrors PERC RAID1 DELL ENT. H755 PERC RAID1 SAMSUNG 256-A-D-C SAMSUNG ZFS MIRROR SAMSUNG ZFS MIRROR2 30K 60K 90K 120K 150K SE +/- 333.33, N = 3 SE +/- 668.80, N = 4 SE +/- 577.35, N = 3 SE +/- 2.89, N = 3 SE +/- 3.79, N = 3 SE +/- 43.75, N = 3 SE +/- 578.31, N = 3 SE +/- 384.42, N = 3 SE +/- 333.33, N = 3 SE +/- 33.33, N = 3 SE +/- 26.74, N = 3 SE +/- 600.93, N = 3 SE +/- 577.35, N = 3 156667 57225 157000 1561 7633 6327 57367 57167 156333 12767 9613 58733 57600
Flexible IO Tester Type: Random Write - Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 128KB - Disk Target: Default Test Directory OpenBenchmarking.org IOPS, More Is Better Flexible IO Tester 3.25 Type: Random Write - Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 128KB - Disk Target: Default Test Directory 10 Per VDEV RaidZ2 4xNVME ZFS MIRROR 5 Per VDEV RaidZ2 DELL ENT NVME SSD1 DELL ENTERPRISE NVME DELL SAMSUNG SSD4 DELLENT ZFS MIRROR DELLENT ZFS MIRROR2 Mirrors PERC RAID1 DELL ENT. H755 PERC RAID1 SAMSUNG 256-A-D-C SAMSUNG ZFS MIRROR SAMSUNG ZFS MIRROR2 20K 40K 60K 80K 100K SE +/- 862.17, N = 3 SE +/- 536.45, N = 3 SE +/- 757.19, N = 3 SE +/- 10.41, N = 3 SE +/- 480.74, N = 3 SE +/- 296.27, N = 3 SE +/- 517.47, N = 3 SE +/- 57.74, N = 3 SE +/- 120.19, N = 3 SE +/- 333.33, N = 3 72700 44233 93500 1617 18300 16400 51633 51433 95733 18700 22400 49633 51867
Flexible IO Tester Type: Random Write - Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 128KB - Disk Target: Default Test Directory OpenBenchmarking.org MB/s, More Is Better Flexible IO Tester 3.25 Type: Random Write - Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 128KB - Disk Target: Default Test Directory 10 Per VDEV RaidZ2 4xNVME ZFS MIRROR DELL ENT NVME SSD1 DELL ENTERPRISE NVME DELL SAMSUNG SSD4 DELLENT ZFS MIRROR DELLENT ZFS MIRROR2 PERC RAID1 DELL ENT. H755 PERC RAID1 SAMSUNG 256-A-D-C SAMSUNG ZFS MIRROR SAMSUNG ZFS MIRROR2 2K 4K 6K 8K 10K SE +/- 108.58, N = 3 SE +/- 65.54, N = 3 SE +/- 1.45, N = 3 SE +/- 1.53, N = 3 SE +/- 58.30, N = 3 SE +/- 39.43, N = 3 SE +/- 6.39, N = 3 SE +/- 1.33, N = 3 SE +/- 15.21, N = 3 SE +/- 40.18, N = 3 9083 5530 203 2286 2054 6454 6430 2334 2798 6210 6482
Phoronix Test Suite v10.8.5