45drivesxl60 2 x AMD EPYC 7513 32-Core testing with a Dell PowerEdge R7525 [0590KW] (2.6.6 BIOS) and Matrox G200eW3 on Debian GNU/Linux 11 via the Phoronix Test Suite.
HTML result view exported from: https://openbenchmarking.org/result/2209212-NE-2209211NE32&grs&sor .
45drivesxl60 Processor Motherboard Memory Disk Chipset Graphics Monitor Network OS Kernel Compiler File-System Vulkan Screen Resolution 10 Per VDEV RaidZ2 5 Per VDEV RaidZ2 Mirrors DELL ENT NVME SSD1 DELL SAMSUNG SSD4 DELL ENTERPRISE NVME SAMSUNG ZFS MIRROR DELLENT ZFS MIRROR PERC RAID1 SAMSUNG 256-A-D-C PERC RAID1 DELL ENT. H755 4xNVME ZFS MIRROR DELLENT ZFS MIRROR2 SAMSUNG ZFS MIRROR2 Intel Xeon Silver 4216 (32 Cores) 45Drives XL60 0123456789 256GB 742117GB FreeBSD 12.2-RELEASE-p10 (x86_64) Clang 10.0.1 zfs 2 x AMD EPYC 7513 32-Core (64 Cores / 128 Threads) Dell PowerEdge R7525 [0590KW] (2.6.6 BIOS) AMD Starship/Matisse 8 x 64 GB DDR4-3200MT/s 36ASF8G72PZ-3G2B2 240GB DELLBOSS VD + 2 x 1920GB Dell Ent NVMe CM + 2 x 3841GB SAMSUNG MZQL23T8 Matrox G200eW3 DELL E172FP 4 x Intel X710 for 10GBASE-T + 2 x Broadcom NetXtreme BCM5720 2-port PCIe Debian GNU/Linux 11 5.15.30-2-pve (x86_64) 1.0.2 GCC 10.2.1 20210110 ext4 1280x1024 zfs ext4 zfs OpenBenchmarking.org Kernel Details - DELL ENT NVME SSD1, DELL SAMSUNG SSD4, DELL ENTERPRISE NVME, SAMSUNG ZFS MIRROR, DELLENT ZFS MIRROR, PERC RAID1 SAMSUNG 256-A-D-C, PERC RAID1 DELL ENT. H755, 4xNVME ZFS MIRROR, DELLENT ZFS MIRROR2, SAMSUNG ZFS MIRROR2: Transparent Huge Pages: madvise Compiler Details - DELL ENT NVME SSD1, DELL SAMSUNG SSD4, DELL ENTERPRISE NVME, SAMSUNG ZFS MIRROR, DELLENT ZFS MIRROR, PERC RAID1 SAMSUNG 256-A-D-C, PERC RAID1 DELL ENT. H755, 4xNVME ZFS MIRROR, DELLENT ZFS MIRROR2, SAMSUNG ZFS MIRROR2: --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-link-mutex --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-targets=nvptx-none=/build/gcc-10-Km9U7s/gcc-10-10.2.1/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-10-Km9U7s/gcc-10-10.2.1/debian/tmp-gcn/usr,hsa --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-build-config=bootstrap-lto-lean --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v Disk Details - DELL ENT NVME SSD1: MQ-DEADLINE / errors=remount-ro,relatime,rw / Block Size: 4096 - DELL SAMSUNG SSD4: MQ-DEADLINE / errors=remount-ro,relatime,rw / Block Size: 4096 - DELL ENTERPRISE NVME: MQ-DEADLINE / errors=remount-ro,relatime,rw / Block Size: 4096 - SAMSUNG ZFS MIRROR: MQ-DEADLINE / errors=remount-ro,relatime,rw / Block Size: 131072 - DELLENT ZFS MIRROR: MQ-DEADLINE / errors=remount-ro,relatime,rw / Block Size: 131072 - PERC RAID1 SAMSUNG 256-A-D-C: MQ-DEADLINE / errors=remount-ro,relatime,rw / Block Size: 4096 - PERC RAID1 DELL ENT. H755: MQ-DEADLINE / errors=remount-ro,relatime,rw / Block Size: 4096 - 4xNVME ZFS MIRROR: MQ-DEADLINE / errors=remount-ro,relatime,rw / Block Size: 131072 - DELLENT ZFS MIRROR2: MQ-DEADLINE / errors=remount-ro,relatime,rw / Block Size: 131072 - SAMSUNG ZFS MIRROR2: MQ-DEADLINE / errors=remount-ro,relatime,rw / Block Size: 131072 Processor Details - DELL ENT NVME SSD1, DELL SAMSUNG SSD4, DELL ENTERPRISE NVME, SAMSUNG ZFS MIRROR, DELLENT ZFS MIRROR, PERC RAID1 SAMSUNG 256-A-D-C, PERC RAID1 DELL ENT. H755, 4xNVME ZFS MIRROR, DELLENT ZFS MIRROR2, SAMSUNG ZFS MIRROR2: CPU Microcode: 0xa00115d Security Details - DELL ENT NVME SSD1, DELL SAMSUNG SSD4, DELL ENTERPRISE NVME, SAMSUNG ZFS MIRROR, DELLENT ZFS MIRROR, PERC RAID1 SAMSUNG 256-A-D-C, PERC RAID1 DELL ENT. H755, 4xNVME ZFS MIRROR, DELLENT ZFS MIRROR2, SAMSUNG ZFS MIRROR2: itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl and seccomp + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional IBRS_FW STIBP: always-on RSB filling + srbds: Not affected + tsx_async_abort: Not affected
45drivesxl60 fio: Seq Read - POSIX AIO - No - Yes - 128KB - Default Test Directory fio: Seq Write - POSIX AIO - No - Yes - 128KB - Default Test Directory fio: Rand Write - POSIX AIO - No - Yes - 128KB - Default Test Directory fio: Seq Write - POSIX AIO - No - Yes - 4KB - Default Test Directory fio: Seq Write - POSIX AIO - No - Yes - 4KB - Default Test Directory fio: Seq Write - POSIX AIO - No - Yes - 128KB - Default Test Directory fio: Rand Write - POSIX AIO - No - Yes - 128KB - Default Test Directory fio: Rand Read - POSIX AIO - No - Yes - 128KB - Default Test Directory fio: Seq Read - POSIX AIO - No - Yes - 128KB - Default Test Directory fio: Rand Write - POSIX AIO - No - Yes - 4KB - Default Test Directory fio: Rand Write - POSIX AIO - No - Yes - 4KB - Default Test Directory fio: Rand Read - POSIX AIO - No - Yes - 128KB - Default Test Directory fio: Seq Read - POSIX AIO - No - Yes - 4KB - Default Test Directory fio: Seq Read - POSIX AIO - No - Yes - 4KB - Default Test Directory fio: Rand Read - POSIX AIO - No - Yes - 4KB - Default Test Directory fio: Rand Read - POSIX AIO - No - Yes - 4KB - Default Test Directory 10 Per VDEV RaidZ2 5 Per VDEV RaidZ2 Mirrors DELL ENT NVME SSD1 DELL SAMSUNG SSD4 DELL ENTERPRISE NVME SAMSUNG ZFS MIRROR DELLENT ZFS MIRROR PERC RAID1 SAMSUNG 256-A-D-C PERC RAID1 DELL ENT. H755 4xNVME ZFS MIRROR DELLENT ZFS MIRROR2 SAMSUNG ZFS MIRROR2 146933 73033 72700 495 127000 9131 9083 92300 360 156667 188000 735 175333 685 131333 91300 93500 529 135667 110333 432 157000 188000 735 177333 692 132600 99900 95733 537 137667 129333 505 156333 189333 739 178333 697 1553 1623 1617 70.3 17871 203 203 196 195 18260 71.3 1561 7183 28.1 6991 27.3 17133 16633 16400 24.3 6385 2080 2054 791 2138 6242 24.4 6327 63067 246 9825 38.4 9457 18300 18300 240 61500 2285 2286 955 1182 61233 240 7633 22500 87.6 12100 47.2 54433 49140 49633 1382 353917 6145 6210 7340 6804 180000 703 58733 440133 1720 99133 388 52847 48446 51633 1302 333333 6074 6454 7172 6622 185000 721 57367 434467 1698 98583 386 21700 22400 22400 157 39767 2801 2798 1202 2754 32633 128 9613 34367 134 20533 80.5 13167 19787 18700 188 48167 2475 2334 1591 1645 42942 168 12767 52100 204 46800 183 53707 38433 44233 1232 315333 4805 5530 7155 6724 118800 464 57225 459867 1797 93392 365 53575 49700 51433 1248 319600 6213 6430 7146 6698 182000 711 57167 423929 1656 107000 418 52067 48936 51867 1309 335000 6117 6482 7198 6511 180333 704 57600 461000 1801 97383 381 OpenBenchmarking.org
Flexible IO Tester Type: Sequential Read - Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 128KB - Disk Target: Default Test Directory OpenBenchmarking.org IOPS, More Is Better Flexible IO Tester 3.25 Type: Sequential Read - Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 128KB - Disk Target: Default Test Directory 10 Per VDEV RaidZ2 Mirrors 5 Per VDEV RaidZ2 SAMSUNG ZFS MIRROR 4xNVME ZFS MIRROR DELLENT ZFS MIRROR2 DELLENT ZFS MIRROR SAMSUNG ZFS MIRROR2 PERC RAID1 SAMSUNG 256-A-D-C DELL SAMSUNG SSD4 PERC RAID1 DELL ENT. H755 DELL ENTERPRISE NVME DELL ENT NVME SSD1 30K 60K 90K 120K 150K SE +/- 2369.38, N = 15 SE +/- 1400.00, N = 5 SE +/- 333.33, N = 3 SE +/- 693.62, N = 3 SE +/- 548.72, N = 15 SE +/- 673.76, N = 4 SE +/- 522.53, N = 15 SE +/- 837.89, N = 15 SE +/- 351.19, N = 3 SE +/- 33.33, N = 3 SE +/- 88.19, N = 3 SE +/- 4.33, N = 3 SE +/- 4.10, N = 3 146933 132600 131333 54433 53707 53575 52847 52067 21700 17133 13167 9457 1553
Flexible IO Tester Type: Sequential Write - Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 128KB - Disk Target: Default Test Directory OpenBenchmarking.org IOPS, More Is Better Flexible IO Tester 3.25 Type: Sequential Write - Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 128KB - Disk Target: Default Test Directory Mirrors 5 Per VDEV RaidZ2 10 Per VDEV RaidZ2 DELLENT ZFS MIRROR2 SAMSUNG ZFS MIRROR SAMSUNG ZFS MIRROR2 DELLENT ZFS MIRROR 4xNVME ZFS MIRROR PERC RAID1 SAMSUNG 256-A-D-C PERC RAID1 DELL ENT. H755 DELL ENTERPRISE NVME DELL SAMSUNG SSD4 DELL ENT NVME SSD1 20K 40K 60K 80K 100K SE +/- 1081.67, N = 3 SE +/- 709.46, N = 3 SE +/- 924.36, N = 3 SE +/- 351.19, N = 3 SE +/- 376.59, N = 10 SE +/- 633.98, N = 14 SE +/- 752.31, N = 13 SE +/- 133.33, N = 3 SE +/- 141.04, N = 15 SE +/- 33.33, N = 3 SE +/- 4.84, N = 3 99900 91300 73033 49700 49140 48936 48446 38433 22400 19787 18300 16633 1623
Flexible IO Tester Type: Random Write - Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 128KB - Disk Target: Default Test Directory OpenBenchmarking.org IOPS, More Is Better Flexible IO Tester 3.25 Type: Random Write - Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 128KB - Disk Target: Default Test Directory Mirrors 5 Per VDEV RaidZ2 10 Per VDEV RaidZ2 SAMSUNG ZFS MIRROR2 DELLENT ZFS MIRROR DELLENT ZFS MIRROR2 SAMSUNG ZFS MIRROR 4xNVME ZFS MIRROR PERC RAID1 SAMSUNG 256-A-D-C PERC RAID1 DELL ENT. H755 DELL ENTERPRISE NVME DELL SAMSUNG SSD4 DELL ENT NVME SSD1 20K 40K 60K 80K 100K SE +/- 517.47, N = 3 SE +/- 757.19, N = 3 SE +/- 862.17, N = 3 SE +/- 333.33, N = 3 SE +/- 480.74, N = 3 SE +/- 296.27, N = 3 SE +/- 120.19, N = 3 SE +/- 536.45, N = 3 SE +/- 57.74, N = 3 SE +/- 10.41, N = 3 95733 93500 72700 51867 51633 51433 49633 44233 22400 18700 18300 16400 1617
Flexible IO Tester Type: Sequential Write - Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory OpenBenchmarking.org MB/s, More Is Better Flexible IO Tester 3.25 Type: Sequential Write - Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory SAMSUNG ZFS MIRROR SAMSUNG ZFS MIRROR2 DELLENT ZFS MIRROR DELLENT ZFS MIRROR2 4xNVME ZFS MIRROR Mirrors 5 Per VDEV RaidZ2 10 Per VDEV RaidZ2 DELL ENTERPRISE NVME PERC RAID1 DELL ENT. H755 PERC RAID1 SAMSUNG 256-A-D-C DELL ENT NVME SSD1 DELL SAMSUNG SSD4 300 600 900 1200 1500 SE +/- 21.11, N = 12 SE +/- 13.25, N = 6 SE +/- 11.95, N = 15 SE +/- 17.29, N = 15 SE +/- 27.37, N = 15 SE +/- 1.20, N = 3 SE +/- 0.67, N = 3 SE +/- 0.73, N = 14 SE +/- 0.30, N = 3 1382.0 1309.0 1302.0 1248.0 1232.0 537.0 529.0 495.0 240.0 188.0 157.0 70.3 24.3
Flexible IO Tester Type: Sequential Write - Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory OpenBenchmarking.org IOPS, More Is Better Flexible IO Tester 3.25 Type: Sequential Write - Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory SAMSUNG ZFS MIRROR SAMSUNG ZFS MIRROR2 DELLENT ZFS MIRROR DELLENT ZFS MIRROR2 4xNVME ZFS MIRROR Mirrors 5 Per VDEV RaidZ2 10 Per VDEV RaidZ2 DELL ENTERPRISE NVME PERC RAID1 DELL ENT. H755 PERC RAID1 SAMSUNG 256-A-D-C DELL ENT NVME SSD1 DELL SAMSUNG SSD4 80K 160K 240K 320K 400K SE +/- 5411.07, N = 12 SE +/- 3386.25, N = 6 SE +/- 3073.18, N = 15 SE +/- 4423.75, N = 15 SE +/- 7030.32, N = 15 SE +/- 333.33, N = 3 SE +/- 333.33, N = 3 SE +/- 57.74, N = 3 SE +/- 185.59, N = 3 SE +/- 384.42, N = 3 SE +/- 180.48, N = 14 SE +/- 1.20, N = 3 353917 335000 333333 319600 315333 137667 135667 127000 61500 48167 39767 17871 6385
Flexible IO Tester Type: Sequential Write - Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 128KB - Disk Target: Default Test Directory OpenBenchmarking.org MB/s, More Is Better Flexible IO Tester 3.25 Type: Sequential Write - Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 128KB - Disk Target: Default Test Directory 10 Per VDEV RaidZ2 DELLENT ZFS MIRROR2 SAMSUNG ZFS MIRROR SAMSUNG ZFS MIRROR2 DELLENT ZFS MIRROR 4xNVME ZFS MIRROR PERC RAID1 SAMSUNG 256-A-D-C PERC RAID1 DELL ENT. H755 DELL ENTERPRISE NVME DELL SAMSUNG SSD4 DELL ENT NVME SSD1 2K 4K 6K 8K 10K SE +/- 114.13, N = 3 SE +/- 44.24, N = 3 SE +/- 46.95, N = 10 SE +/- 78.92, N = 14 SE +/- 85.11, N = 13 SE +/- 17.49, N = 3 SE +/- 1.86, N = 3 SE +/- 17.87, N = 15 SE +/- 0.67, N = 3 SE +/- 1.20, N = 3 SE +/- 0.67, N = 3 9131 6213 6145 6117 6074 4805 2801 2475 2285 2080 203
Flexible IO Tester Type: Random Write - Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 128KB - Disk Target: Default Test Directory OpenBenchmarking.org MB/s, More Is Better Flexible IO Tester 3.25 Type: Random Write - Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 128KB - Disk Target: Default Test Directory 10 Per VDEV RaidZ2 SAMSUNG ZFS MIRROR2 DELLENT ZFS MIRROR DELLENT ZFS MIRROR2 SAMSUNG ZFS MIRROR 4xNVME ZFS MIRROR PERC RAID1 SAMSUNG 256-A-D-C PERC RAID1 DELL ENT. H755 DELL ENTERPRISE NVME DELL SAMSUNG SSD4 DELL ENT NVME SSD1 2K 4K 6K 8K 10K SE +/- 108.58, N = 3 SE +/- 40.18, N = 3 SE +/- 58.30, N = 3 SE +/- 39.43, N = 3 SE +/- 15.21, N = 3 SE +/- 65.54, N = 3 SE +/- 1.33, N = 3 SE +/- 6.39, N = 3 SE +/- 1.53, N = 3 SE +/- 1.45, N = 3 9083 6482 6454 6430 6210 5530 2798 2334 2286 2054 203
Flexible IO Tester Type: Random Read - Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 128KB - Disk Target: Default Test Directory OpenBenchmarking.org MB/s, More Is Better Flexible IO Tester 3.25 Type: Random Read - Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 128KB - Disk Target: Default Test Directory SAMSUNG ZFS MIRROR SAMSUNG ZFS MIRROR2 DELLENT ZFS MIRROR 4xNVME ZFS MIRROR DELLENT ZFS MIRROR2 PERC RAID1 DELL ENT. H755 PERC RAID1 SAMSUNG 256-A-D-C DELL ENTERPRISE NVME DELL SAMSUNG SSD4 DELL ENT NVME SSD1 1600 3200 4800 6400 8000 SE +/- 74.71, N = 3 SE +/- 71.04, N = 3 SE +/- 69.68, N = 3 SE +/- 83.90, N = 4 SE +/- 48.90, N = 3 SE +/- 3.18, N = 3 SE +/- 3.38, N = 3 SE +/- 0.33, N = 3 SE +/- 5.24, N = 3 SE +/- 0.33, N = 3 7340 7198 7172 7155 7146 1591 1202 955 791 196 1. (CC) gcc options: -rdynamic -ll -lnuma -lrt -lz -lpthread -lm -ldl -laio -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native
Flexible IO Tester Type: Sequential Read - Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 128KB - Disk Target: Default Test Directory OpenBenchmarking.org MB/s, More Is Better Flexible IO Tester 3.25 Type: Sequential Read - Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 128KB - Disk Target: Default Test Directory SAMSUNG ZFS MIRROR 4xNVME ZFS MIRROR DELLENT ZFS MIRROR2 DELLENT ZFS MIRROR SAMSUNG ZFS MIRROR2 PERC RAID1 SAMSUNG 256-A-D-C DELL SAMSUNG SSD4 PERC RAID1 DELL ENT. H755 DELL ENTERPRISE NVME DELL ENT NVME SSD1 1500 3000 4500 6000 7500 SE +/- 88.50, N = 3 SE +/- 70.16, N = 15 SE +/- 83.02, N = 4 SE +/- 59.56, N = 15 SE +/- 104.53, N = 15 SE +/- 6.81, N = 3 SE +/- 4.18, N = 3 SE +/- 7.62, N = 3 SE +/- 0.67, N = 3 6804 6724 6698 6622 6511 2754 2138 1645 1182 195 1. (CC) gcc options: -rdynamic -ll -lnuma -lrt -lz -lpthread -lm -ldl -laio -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native
Flexible IO Tester Type: Random Write - Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory OpenBenchmarking.org IOPS, More Is Better Flexible IO Tester 3.25 Type: Random Write - Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory DELLENT ZFS MIRROR DELLENT ZFS MIRROR2 SAMSUNG ZFS MIRROR2 SAMSUNG ZFS MIRROR Mirrors 4xNVME ZFS MIRROR 5 Per VDEV RaidZ2 10 Per VDEV RaidZ2 DELL ENTERPRISE NVME PERC RAID1 DELL ENT. H755 PERC RAID1 SAMSUNG 256-A-D-C DELL ENT NVME SSD1 DELL SAMSUNG SSD4 40K 80K 120K 160K 200K SE +/- 1527.53, N = 3 SE +/- 1944.47, N = 15 SE +/- 2728.45, N = 3 SE +/- 1452.97, N = 3 SE +/- 1313.66, N = 15 SE +/- 881.92, N = 3 SE +/- 416.33, N = 3 SE +/- 66.67, N = 3 SE +/- 1055.04, N = 12 SE +/- 283.12, N = 15 SE +/- 222.92, N = 15 SE +/- 1.86, N = 3 185000 182000 180333 180000 129333 118800 110333 92300 61233 42942 32633 18260 6242
Flexible IO Tester Type: Random Write - Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory OpenBenchmarking.org MB/s, More Is Better Flexible IO Tester 3.25 Type: Random Write - Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory DELLENT ZFS MIRROR DELLENT ZFS MIRROR2 SAMSUNG ZFS MIRROR2 SAMSUNG ZFS MIRROR Mirrors 4xNVME ZFS MIRROR 5 Per VDEV RaidZ2 10 Per VDEV RaidZ2 DELL ENTERPRISE NVME PERC RAID1 DELL ENT. H755 PERC RAID1 SAMSUNG 256-A-D-C DELL ENT NVME SSD1 DELL SAMSUNG SSD4 160 320 480 640 800 SE +/- 6.23, N = 3 SE +/- 7.59, N = 15 SE +/- 10.09, N = 3 SE +/- 0.67, N = 3 SE +/- 5.61, N = 3 SE +/- 4.95, N = 15 SE +/- 2.33, N = 3 SE +/- 1.76, N = 3 SE +/- 0.33, N = 3 SE +/- 4.12, N = 12 SE +/- 1.07, N = 15 SE +/- 0.86, N = 15 SE +/- 0.00, N = 3 721.0 711.0 704.0 703.0 505.0 464.0 432.0 360.0 240.0 168.0 128.0 71.3 24.4
Flexible IO Tester Type: Random Read - Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 128KB - Disk Target: Default Test Directory OpenBenchmarking.org IOPS, More Is Better Flexible IO Tester 3.25 Type: Random Read - Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 128KB - Disk Target: Default Test Directory 5 Per VDEV RaidZ2 10 Per VDEV RaidZ2 Mirrors SAMSUNG ZFS MIRROR SAMSUNG ZFS MIRROR2 DELLENT ZFS MIRROR 4xNVME ZFS MIRROR DELLENT ZFS MIRROR2 PERC RAID1 DELL ENT. H755 PERC RAID1 SAMSUNG 256-A-D-C DELL ENTERPRISE NVME DELL SAMSUNG SSD4 DELL ENT NVME SSD1 30K 60K 90K 120K 150K SE +/- 577.35, N = 3 SE +/- 333.33, N = 3 SE +/- 333.33, N = 3 SE +/- 600.93, N = 3 SE +/- 577.35, N = 3 SE +/- 578.31, N = 3 SE +/- 668.80, N = 4 SE +/- 384.42, N = 3 SE +/- 33.33, N = 3 SE +/- 26.74, N = 3 SE +/- 3.79, N = 3 SE +/- 43.75, N = 3 SE +/- 2.89, N = 3 157000 156667 156333 58733 57600 57367 57225 57167 12767 9613 7633 6327 1561
Flexible IO Tester Type: Sequential Read - Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory OpenBenchmarking.org IOPS, More Is Better Flexible IO Tester 3.25 Type: Sequential Read - Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory SAMSUNG ZFS MIRROR2 4xNVME ZFS MIRROR SAMSUNG ZFS MIRROR DELLENT ZFS MIRROR DELLENT ZFS MIRROR2 Mirrors 5 Per VDEV RaidZ2 10 Per VDEV RaidZ2 DELL SAMSUNG SSD4 PERC RAID1 DELL ENT. H755 PERC RAID1 SAMSUNG 256-A-D-C DELL ENTERPRISE NVME DELL ENT NVME SSD1 100K 200K 300K 400K 500K SE +/- 1527.53, N = 3 SE +/- 8694.21, N = 15 SE +/- 10885.49, N = 15 SE +/- 7940.17, N = 15 SE +/- 11135.26, N = 14 SE +/- 333.33, N = 3 SE +/- 577.35, N = 3 SE +/- 577.35, N = 3 SE +/- 33.33, N = 3 SE +/- 100.00, N = 3 SE +/- 33.33, N = 3 SE +/- 11.72, N = 3 461000 459867 440133 434467 423929 189333 188000 188000 63067 52100 34367 22500 7183
Flexible IO Tester Type: Sequential Read - Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory OpenBenchmarking.org MB/s, More Is Better Flexible IO Tester 3.25 Type: Sequential Read - Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory SAMSUNG ZFS MIRROR2 4xNVME ZFS MIRROR SAMSUNG ZFS MIRROR DELLENT ZFS MIRROR DELLENT ZFS MIRROR2 Mirrors 5 Per VDEV RaidZ2 10 Per VDEV RaidZ2 DELL SAMSUNG SSD4 PERC RAID1 DELL ENT. H755 PERC RAID1 SAMSUNG 256-A-D-C DELL ENTERPRISE NVME DELL ENT NVME SSD1 400 800 1200 1600 2000 SE +/- 6.77, N = 3 SE +/- 33.83, N = 15 SE +/- 42.44, N = 15 SE +/- 30.86, N = 15 SE +/- 43.36, N = 14 SE +/- 1.67, N = 3 SE +/- 1.76, N = 3 SE +/- 1.76, N = 3 SE +/- 0.33, N = 3 SE +/- 0.33, N = 3 SE +/- 0.32, N = 3 SE +/- 0.06, N = 3 1801.0 1797.0 1720.0 1698.0 1656.0 739.0 735.0 735.0 246.0 204.0 134.0 87.6 28.1
Flexible IO Tester Type: Random Read - Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory OpenBenchmarking.org IOPS, More Is Better Flexible IO Tester 3.25 Type: Random Read - Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory Mirrors 5 Per VDEV RaidZ2 10 Per VDEV RaidZ2 DELLENT ZFS MIRROR2 SAMSUNG ZFS MIRROR DELLENT ZFS MIRROR SAMSUNG ZFS MIRROR2 4xNVME ZFS MIRROR PERC RAID1 DELL ENT. H755 PERC RAID1 SAMSUNG 256-A-D-C DELL ENTERPRISE NVME DELL SAMSUNG SSD4 DELL ENT NVME SSD1 40K 80K 120K 160K 200K SE +/- 666.67, N = 3 SE +/- 666.67, N = 3 SE +/- 666.67, N = 3 SE +/- 577.35, N = 3 SE +/- 366.67, N = 3 SE +/- 2927.48, N = 12 SE +/- 3366.00, N = 12 SE +/- 5435.76, N = 12 SE +/- 1886.59, N = 15 SE +/- 4.63, N = 3 SE +/- 14.33, N = 3 178667 177333 175333 107000 99133 98583 97383 93392 46800 20533 12100 9825 6991
Flexible IO Tester Type: Random Read - Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory OpenBenchmarking.org MB/s, More Is Better Flexible IO Tester 3.25 Type: Random Read - Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory Mirrors 5 Per VDEV RaidZ2 10 Per VDEV RaidZ2 DELLENT ZFS MIRROR2 SAMSUNG ZFS MIRROR DELLENT ZFS MIRROR SAMSUNG ZFS MIRROR2 4xNVME ZFS MIRROR PERC RAID1 DELL ENT. H755 PERC RAID1 SAMSUNG 256-A-D-C DELL ENTERPRISE NVME DELL SAMSUNG SSD4 DELL ENT NVME SSD1 150 300 450 600 750 SE +/- 2.85, N = 3 SE +/- 2.67, N = 3 SE +/- 3.06, N = 3 SE +/- 2.08, N = 3 SE +/- 1.33, N = 3 SE +/- 11.53, N = 12 SE +/- 13.16, N = 12 SE +/- 21.18, N = 12 SE +/- 7.35, N = 15 SE +/- 0.00, N = 3 SE +/- 0.00, N = 3 SE +/- 0.06, N = 3 697.0 692.0 685.0 418.0 388.0 386.0 381.0 365.0 183.0 80.5 47.2 38.4 27.3
Phoronix Test Suite v10.8.5