2020-06-24-2159

no tuning, dedup=on, compression=on, sync=on, ashift=9

Compare your own system(s) to this result file with the Phoronix Test Suite by running the command: phoronix-test-suite benchmark 2006259-NI-2006251NI96
Jump To Table - Results

View

Do Not Show Noisy Results
Do Not Show Results With Incomplete Data
Do Not Show Results With Little Change/Spread
List Notable Results

Statistics

Show Overall Harmonic Mean(s)
Show Overall Geometric Mean
Show Wins / Losses Counts (Pie Chart)
Normalize Results
Remove Outliers Before Calculating Averages

Graph Settings

Force Line Graphs Where Applicable
Convert To Scalar Where Applicable
Prefer Vertical Bar Graphs

Multi-Way Comparison

Condense Multi-Option Tests Into Single Result Graphs

Table

Show Detailed System Result Table

Run Management

Highlight
Result
Hide
Result
Result
Identifier
Performance Per
Dollar
Date
Run
  Test
  Duration
zfs_6x_ssd_in_2x_raidz1
June 24 2020
  2 Hours, 24 Minutes
zfs_6x_ssd_in_2x_raidz1_ashift=12
June 24 2020
 
6 x 481GB HSCP0480S5xnNMRI
June 24 2020
  2 Hours, 29 Minutes
zfs_ashift=9_compression=on_dedup=off
June 25 2020
  1 Hour, 57 Minutes
zfs_ashift=9_compression=on_dedup=on
June 25 2020
  1 Hour, 19 Minutes
zfs_ashift=9_compression=on_dedup=on_sync=yes
June 25 2020
  1 Hour, 50 Minutes
Invert Hiding All Results Option
  1 Hour, 40 Minutes

Only show results where is faster than
Only show results matching title/arguments (delimit multiple options with a comma):
Do not show results matching title/arguments (delimit multiple options with a comma):

zfs_6x_ssd_in_2x_raidz1

Compiler Notes: --build=x86_64-redhat-linux --disable-libmpx --disable-libunwind-exceptions --enable-__cxa_atexit --enable-bootstrap --enable-cet --enable-checking=release --enable-gnu-indirect-function --enable-gnu-unique-object --enable-initfini-array --enable-languages=c,c++,fortran,lto --enable-multilib --enable-offload-targets=nvptx-none --enable-plugin --enable-shared --enable-threads=posix --mandir=/usr/share/man --with-arch_32=x86-64 --with-gcc-major-version-only --with-isl --with-linker-hash-style=gnu --with-tune=generic --without-cuda-driver
Disk Notes: MQ-DEADLINE / attr2,inode64,noquota,relatime,rw,seclabel
Processor Notes: Scaling Governor: intel_pstate performance - CPU Microcode: 0x43
Security Notes: SELinux + itlb_multihit: KVM: Mitigation of Split huge pages + l1tf: Mitigation of PTE Inversion; VMX: conditional cache flushes SMT vulnerable + mds: Mitigation of Clear buffers; SMT vulnerable + meltdown: Mitigation of PTI + spec_store_bypass: Mitigation of SSB disabled via prctl and seccomp + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Full generic retpoline IBPB: conditional IBRS_FW STIBP: conditional RSB filling + srbds: Not affected + tsx_async_abort: Not affected

zfs_6x_ssd_in_2x_raidz1_ashift=12

6 x 481GB HSCP0480S5xnNMRI

zfs_ashift=9_compression=on_dedup=off

zfs_ashift=9_compression=on_dedup=on

zfs_ashift=9_compression=on_dedup=on_sync=yes

Processor: 2 x Intel Xeon E5-2640 v3 @ 3.40GHz (16 Cores / 32 Threads), Motherboard: Dell 0CNCJW (2.11.0 BIOS), Chipset: Intel Xeon E7 v3/Xeon, Memory: 16 x GB DDR4-2133MT/s HMA42GR7MFR4N-TF, Disk: 6 x 481GB HSCP0480S5xnNMRI + 99GB PERC H730 Mini + 16GB Cruzer Blade + 16GB Cruzer Fit, Graphics: Matrox G200eR2, Monitor: DELL E196FP, Network: 2 x Intel 10-Gigabit X540-AT2 + 2 x Intel I350 + Mellanox MT27500

OS: CentOS Linux 8, Kernel: 4.18.0-193.6.3.el8_2.x86_64 (x86_64), Compiler: GCC 8.3.1 20191121, File-System: xfs, Screen Resolution: 1280x1024

2020-06-24-2159OpenBenchmarking.orgPhoronix Test Suite2 x Intel Xeon E5-2640 v3 @ 3.40GHz (16 Cores / 32 Threads)Dell 0CNCJW (2.11.0 BIOS)Intel Xeon E7 v3/Xeon16 x GB DDR4-2133MT/s HMA42GR7MFR4N-TF6 x 481GB HSCP0480S5xnNMRI + 99GB PERC H730 Mini + 16GB Cruzer Blade + 16GB Cruzer FitMatrox G200eR2DELL E196FP2 x Intel 10-Gigabit X540-AT2 + 2 x Intel I350 + Mellanox MT27500CentOS Linux 84.18.0-193.6.3.el8_2.x86_64 (x86_64)GCC 8.3.1 20191121xfs1280x1024ProcessorMotherboardChipsetMemoryDiskGraphicsMonitorNetworkOSKernelCompilerFile-SystemScreen Resolution2020-06-24-2159 BenchmarksSystem Logs- --build=x86_64-redhat-linux --disable-libmpx --disable-libunwind-exceptions --enable-__cxa_atexit --enable-bootstrap --enable-cet --enable-checking=release --enable-gnu-indirect-function --enable-gnu-unique-object --enable-initfini-array --enable-languages=c,c++,fortran,lto --enable-multilib --enable-offload-targets=nvptx-none --enable-plugin --enable-shared --enable-threads=posix --mandir=/usr/share/man --with-arch_32=x86-64 --with-gcc-major-version-only --with-isl --with-linker-hash-style=gnu --with-tune=generic --without-cuda-driver - MQ-DEADLINE / attr2,inode64,noquota,relatime,rw,seclabel- Scaling Governor: intel_pstate performance - CPU Microcode: 0x43- SELinux + itlb_multihit: KVM: Mitigation of Split huge pages + l1tf: Mitigation of PTE Inversion; VMX: conditional cache flushes SMT vulnerable + mds: Mitigation of Clear buffers; SMT vulnerable + meltdown: Mitigation of PTI + spec_store_bypass: Mitigation of SSB disabled via prctl and seccomp + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Full generic retpoline IBPB: conditional IBRS_FW STIBP: conditional RSB filling + srbds: Not affected + tsx_async_abort: Not affected

2020-06-24-2159fio: Rand Read - Sync - No - Yes - 4KB - /mnt/ztest1fio: Rand Read - Sync - No - Yes - 4KB - /mnt/ztest1fio: Rand Read - Sync - No - Yes - 16KB - /mnt/ztest1fio: Rand Read - Sync - No - Yes - 16KB - /mnt/ztest1fio: Rand Read - Sync - No - Yes - 64KB - /mnt/ztest1fio: Rand Read - Sync - No - Yes - 64KB - /mnt/ztest1fio: Rand Write - Sync - No - Yes - 4KB - /mnt/ztest1fio: Rand Write - Sync - No - Yes - 4KB - /mnt/ztest1fio: Rand Write - Sync - No - Yes - 16KB - /mnt/ztest1fio: Rand Write - Sync - No - Yes - 16KB - /mnt/ztest1fio: Rand Write - Sync - No - Yes - 64KB - /mnt/ztest1fio: Rand Write - Sync - No - Yes - 64KB - /mnt/ztest1fio: Seq Read - Sync - No - Yes - 4KB - /mnt/ztest1fio: Seq Read - Sync - No - Yes - 4KB - /mnt/ztest1fio: Rand Read - Linux AIO - No - Yes - 4KB - /mnt/ztest1fio: Rand Read - Linux AIO - No - Yes - 4KB - /mnt/ztest1fio: Seq Read - Sync - No - Yes - 16KB - /mnt/ztest1fio: Seq Read - Sync - No - Yes - 16KB - /mnt/ztest1fio: Seq Read - Sync - No - Yes - 64KB - /mnt/ztest1fio: Seq Read - Sync - No - Yes - 64KB - /mnt/ztest1fio: Seq Write - Sync - No - Yes - 4KB - /mnt/ztest1fio: Seq Write - Sync - No - Yes - 4KB - /mnt/ztest1fio: Rand Read - Linux AIO - No - Yes - 16KB - /mnt/ztest1fio: Rand Read - Linux AIO - No - Yes - 16KB - /mnt/ztest1fio: Rand Read - Linux AIO - No - Yes - 64KB - /mnt/ztest1fio: Rand Read - Linux AIO - No - Yes - 64KB - /mnt/ztest1fio: Rand Write - Linux AIO - No - Yes - 4KB - /mnt/ztest1fio: Rand Write - Linux AIO - No - Yes - 4KB - /mnt/ztest1fio: Seq Write - Sync - No - Yes - 16KB - /mnt/ztest1fio: Seq Write - Sync - No - Yes - 16KB - /mnt/ztest1fio: Seq Write - Sync - No - Yes - 64KB - /mnt/ztest1fio: Seq Write - Sync - No - Yes - 64KB - /mnt/ztest1fio: Rand Write - Linux AIO - No - Yes - 16KB - /mnt/ztest1fio: Rand Write - Linux AIO - No - Yes - 16KB - /mnt/ztest1fio: Rand Write - Linux AIO - No - Yes - 64KB - /mnt/ztest1fio: Rand Write - Linux AIO - No - Yes - 64KB - /mnt/ztest1fio: Seq Read - Linux AIO - No - Yes - 4KB - /mnt/ztest1fio: Seq Read - Linux AIO - No - Yes - 4KB - /mnt/ztest1fio: Seq Read - Linux AIO - No - Yes - 16KB - /mnt/ztest1fio: Seq Read - Linux AIO - No - Yes - 16KB - /mnt/ztest1fio: Seq Read - Linux AIO - No - Yes - 64KB - /mnt/ztest1fio: Seq Read - Linux AIO - No - Yes - 64KB - /mnt/ztest1fio: Seq Write - Linux AIO - No - Yes - 4KB - /mnt/ztest1fio: Seq Write - Linux AIO - No - Yes - 4KB - /mnt/ztest1fio: Seq Write - Linux AIO - No - Yes - 16KB - /mnt/ztest1fio: Seq Write - Linux AIO - No - Yes - 16KB - /mnt/ztest1fio: Seq Write - Linux AIO - No - Yes - 64KB - /mnt/ztest1fio: Seq Write - Linux AIO - No - Yes - 64KB - /mnt/ztest1zfs_6x_ssd_in_2x_raidz1zfs_6x_ssd_in_2x_raidz1_ashift=126 x 481GB HSCP0480S5xnNMRIzfs_ashift=9_compression=on_dedup=offzfs_ashift=9_compression=on_dedup=onzfs_ashift=9_compression=on_dedup=on_sync=yes946242133253516226745537277321154027718456671879300671396357667872223333325420833348227716770017933322881466674092654502125413316251040002106334756624222516112570710862783332753176000438570167615157333142391067181128967928237467240715433342606816723158887776495201865298671416362667860220000324420766748977833370318033322831460004153662562185566715811011502097335677264636416132580011172863332820180667449571900628161000144192233207233167968247667251116066743947030032182367961615331669267001447371000868222333323320700048097663374319033323211486674117658671814630018621190002176348006203964015442470011312896672821180600445471260614157000144192187196131600966247333254716300045037170034488033103766333200932133143536766788622666732952110004911786007541926672358151000424267883215550331862119000220835300736471001608257331121287000285818266745347253360615533314719413321503436710022570002860182667499879960333849001005640331974317251484379667897229667353522606753628579275119200025071606674548727672135450018321170002209353337294663316072570011362910002956189333476576233608155333148094700215534467OpenBenchmarking.org

Flexible IO Tester

Fio is an advanced disk benchmark that depends upon the kernel's AIO access library. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.18Type: Random Read - Engine: Sync - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: /mnt/ztest1zfs_6x_ssd_in_2x_raidz16 x 481GB HSCP0480S5xnNMRIzfs_ashift=9_compression=on_dedup=offzfs_ashift=9_compression=on_dedup=onzfs_ashift=9_compression=on_dedup=on_sync=yes2004006008001000SE +/- 11.46, N = 15SE +/- 8.94, N = 15SE +/- 12.78, N = 3SE +/- 6.98, N = 3SE +/- 2.60, N = 394692896896610021. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native -lrt -laio -lpthread -lm -ldl
OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.18Type: Random Read - Engine: Sync - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: /mnt/ztest1zfs_6x_ssd_in_2x_raidz16 x 481GB HSCP0480S5xnNMRIzfs_ashift=9_compression=on_dedup=offzfs_ashift=9_compression=on_dedup=onzfs_ashift=9_compression=on_dedup=on_sync=yes2004006008001000Min: 897 / Avg: 945.6 / Max: 1023Min: 904 / Avg: 927.8 / Max: 1012Min: 952 / Avg: 967.67 / Max: 993Min: 957 / Avg: 966.33 / Max: 980Min: 998 / Avg: 1002.33 / Max: 10071. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native -lrt -laio -lpthread -lm -ldl

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.18Type: Random Read - Engine: Sync - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: /mnt/ztest1zfs_6x_ssd_in_2x_raidz16 x 481GB HSCP0480S5xnNMRIzfs_ashift=9_compression=on_dedup=offzfs_ashift=9_compression=on_dedup=onzfs_ashift=9_compression=on_dedup=on_sync=yes60K120K180K240K300KSE +/- 2946.94, N = 15SE +/- 2302.93, N = 15SE +/- 3179.80, N = 3SE +/- 1855.92, N = 3SE +/- 577.35, N = 32421332374672476672473332570001. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native -lrt -laio -lpthread -lm -ldl
OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.18Type: Random Read - Engine: Sync - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: /mnt/ztest1zfs_6x_ssd_in_2x_raidz16 x 481GB HSCP0480S5xnNMRIzfs_ashift=9_compression=on_dedup=offzfs_ashift=9_compression=on_dedup=onzfs_ashift=9_compression=on_dedup=on_sync=yes40K80K120K160K200KMin: 230000 / Avg: 242133.33 / Max: 262000Min: 231000 / Avg: 237466.67 / Max: 259000Min: 244000 / Avg: 247666.67 / Max: 254000Min: 245000 / Avg: 247333.33 / Max: 251000Min: 256000 / Avg: 257000 / Max: 2580001. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native -lrt -laio -lpthread -lm -ldl

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.18Type: Random Read - Engine: Sync - Buffered: No - Direct: Yes - Block Size: 16KB - Disk Target: /mnt/ztest1zfs_6x_ssd_in_2x_raidz16 x 481GB HSCP0480S5xnNMRIzfs_ashift=9_compression=on_dedup=offzfs_ashift=9_compression=on_dedup=onzfs_ashift=9_compression=on_dedup=on_sync=yes6001200180024003000SE +/- 53.47, N = 15SE +/- 9.68, N = 3SE +/- 8.33, N = 3SE +/- 8.82, N = 3SE +/- 10.00, N = 3253524072511254728601. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native -lrt -laio -lpthread -lm -ldl
OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.18Type: Random Read - Engine: Sync - Buffered: No - Direct: Yes - Block Size: 16KB - Disk Target: /mnt/ztest1zfs_6x_ssd_in_2x_raidz16 x 481GB HSCP0480S5xnNMRIzfs_ashift=9_compression=on_dedup=offzfs_ashift=9_compression=on_dedup=onzfs_ashift=9_compression=on_dedup=on_sync=yes5001000150020002500Min: 2400 / Avg: 2534.6 / Max: 2954Min: 2388 / Avg: 2407.33 / Max: 2418Min: 2499 / Avg: 2511 / Max: 2527Min: 2534 / Avg: 2547.33 / Max: 2564Min: 2850 / Avg: 2860 / Max: 28801. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native -lrt -laio -lpthread -lm -ldl

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.18Type: Random Read - Engine: Sync - Buffered: No - Direct: Yes - Block Size: 16KB - Disk Target: /mnt/ztest1zfs_6x_ssd_in_2x_raidz16 x 481GB HSCP0480S5xnNMRIzfs_ashift=9_compression=on_dedup=offzfs_ashift=9_compression=on_dedup=onzfs_ashift=9_compression=on_dedup=on_sync=yes40K80K120K160K200KSE +/- 3412.12, N = 15SE +/- 666.67, N = 3SE +/- 666.67, N = 3SE +/- 577.35, N = 3SE +/- 666.67, N = 31622671543331606671630001826671. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native -lrt -laio -lpthread -lm -ldl
OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.18Type: Random Read - Engine: Sync - Buffered: No - Direct: Yes - Block Size: 16KB - Disk Target: /mnt/ztest1zfs_6x_ssd_in_2x_raidz16 x 481GB HSCP0480S5xnNMRIzfs_ashift=9_compression=on_dedup=offzfs_ashift=9_compression=on_dedup=onzfs_ashift=9_compression=on_dedup=on_sync=yes30K60K90K120K150KMin: 154000 / Avg: 162266.67 / Max: 189000Min: 153000 / Avg: 154333.33 / Max: 155000Min: 160000 / Avg: 160666.67 / Max: 162000Min: 162000 / Avg: 163000 / Max: 164000Min: 182000 / Avg: 182666.67 / Max: 1840001. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native -lrt -laio -lpthread -lm -ldl

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.18Type: Random Read - Engine: Sync - Buffered: No - Direct: Yes - Block Size: 64KB - Disk Target: /mnt/ztest1zfs_6x_ssd_in_2x_raidz16 x 481GB HSCP0480S5xnNMRIzfs_ashift=9_compression=on_dedup=offzfs_ashift=9_compression=on_dedup=onzfs_ashift=9_compression=on_dedup=on_sync=yes11002200330044005500SE +/- 121.70, N = 15SE +/- 2.40, N = 3SE +/- 8.67, N = 3SE +/- 2.52, N = 3SE +/- 86.11, N = 15455342604394450349981. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native -lrt -laio -lpthread -lm -ldl
OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.18Type: Random Read - Engine: Sync - Buffered: No - Direct: Yes - Block Size: 64KB - Disk Target: /mnt/ztest1zfs_6x_ssd_in_2x_raidz16 x 481GB HSCP0480S5xnNMRIzfs_ashift=9_compression=on_dedup=offzfs_ashift=9_compression=on_dedup=onzfs_ashift=9_compression=on_dedup=on_sync=yes9001800270036004500Min: 4264 / Avg: 4553.33 / Max: 5506Min: 4255 / Avg: 4259.67 / Max: 4263Min: 4379 / Avg: 4394.33 / Max: 4409Min: 4500 / Avg: 4503 / Max: 4508Min: 4346 / Avg: 4998.13 / Max: 51901. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native -lrt -laio -lpthread -lm -ldl

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.18Type: Random Read - Engine: Sync - Buffered: No - Direct: Yes - Block Size: 64KB - Disk Target: /mnt/ztest1zfs_6x_ssd_in_2x_raidz16 x 481GB HSCP0480S5xnNMRIzfs_ashift=9_compression=on_dedup=offzfs_ashift=9_compression=on_dedup=onzfs_ashift=9_compression=on_dedup=on_sync=yes20K40K60K80K100KSE +/- 1917.72, N = 15SE +/- 33.33, N = 3SE +/- 115.47, N = 3SE +/- 351.19, N = 3SE +/- 1378.16, N = 1572773681677030071700799601. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native -lrt -laio -lpthread -lm -ldl
OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.18Type: Random Read - Engine: Sync - Buffered: No - Direct: Yes - Block Size: 64KB - Disk Target: /mnt/ztest1zfs_6x_ssd_in_2x_raidz16 x 481GB HSCP0480S5xnNMRIzfs_ashift=9_compression=on_dedup=offzfs_ashift=9_compression=on_dedup=onzfs_ashift=9_compression=on_dedup=on_sync=yes14K28K42K56K70KMin: 68200 / Avg: 72773.33 / Max: 88100Min: 68100 / Avg: 68166.67 / Max: 68200Min: 70100 / Avg: 70300 / Max: 70500Min: 71000 / Avg: 71700 / Max: 72100Min: 69500 / Avg: 79960 / Max: 830001. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native -lrt -laio -lpthread -lm -ldl

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.18Type: Random Write - Engine: Sync - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: /mnt/ztest1zfs_6x_ssd_in_2x_raidz16 x 481GB HSCP0480S5xnNMRIzfs_ashift=9_compression=on_dedup=offzfs_ashift=9_compression=on_dedup=onzfs_ashift=9_compression=on_dedup=on_sync=yes70140210280350SE +/- 3.16, N = 15SE +/- 3.26, N = 15SE +/- 2.33, N = 3SE +/- 3.21, N = 32112313213443331. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native -lrt -laio -lpthread -lm -ldl
OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.18Type: Random Write - Engine: Sync - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: /mnt/ztest1zfs_6x_ssd_in_2x_raidz16 x 481GB HSCP0480S5xnNMRIzfs_ashift=9_compression=on_dedup=offzfs_ashift=9_compression=on_dedup=onzfs_ashift=9_compression=on_dedup=on_sync=yes60120180240300Min: 189 / Avg: 211.13 / Max: 230Min: 211 / Avg: 230.53 / Max: 259Min: 340 / Avg: 343.67 / Max: 348Min: 328 / Avg: 333 / Max: 3391. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native -lrt -laio -lpthread -lm -ldl

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.18Type: Random Write - Engine: Sync - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: /mnt/ztest1zfs_6x_ssd_in_2x_raidz16 x 481GB HSCP0480S5xnNMRIzfs_ashift=9_compression=on_dedup=offzfs_ashift=9_compression=on_dedup=onzfs_ashift=9_compression=on_dedup=on_sync=yes20K40K60K80K100KSE +/- 803.41, N = 15SE +/- 840.57, N = 15SE +/- 1097.47, N = 3SE +/- 638.57, N = 3SE +/- 1096.97, N = 354027588878236788033849001. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native -lrt -laio -lpthread -lm -ldl
OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.18Type: Random Write - Engine: Sync - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: /mnt/ztest1zfs_6x_ssd_in_2x_raidz16 x 481GB HSCP0480S5xnNMRIzfs_ashift=9_compression=on_dedup=offzfs_ashift=9_compression=on_dedup=onzfs_ashift=9_compression=on_dedup=on_sync=yes15K30K45K60K75KMin: 48400 / Avg: 54026.67 / Max: 58900Min: 54100 / Avg: 58886.67 / Max: 66300Min: 80500 / Avg: 82366.67 / Max: 84300Min: 87000 / Avg: 88033.33 / Max: 89200Min: 83000 / Avg: 84900 / Max: 868001. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native -lrt -laio -lpthread -lm -ldl

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.18Type: Random Write - Engine: Sync - Buffered: No - Direct: Yes - Block Size: 16KB - Disk Target: /mnt/ztest1zfs_6x_ssd_in_2x_raidz16 x 481GB HSCP0480S5xnNMRIzfs_ashift=9_compression=on_dedup=offzfs_ashift=9_compression=on_dedup=onzfs_ashift=9_compression=on_dedup=on_sync=yes2004006008001000SE +/- 10.11, N = 3SE +/- 7.20, N = 15SE +/- 1.45, N = 3SE +/- 5.13, N = 3718776961103710051. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native -lrt -laio -lpthread -lm -ldl
OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.18Type: Random Write - Engine: Sync - Buffered: No - Direct: Yes - Block Size: 16KB - Disk Target: /mnt/ztest1zfs_6x_ssd_in_2x_raidz16 x 481GB HSCP0480S5xnNMRIzfs_ashift=9_compression=on_dedup=offzfs_ashift=9_compression=on_dedup=onzfs_ashift=9_compression=on_dedup=on_sync=yes2004006008001000Min: 701 / Avg: 718.33 / Max: 736Min: 746 / Avg: 775.73 / Max: 847Min: 959 / Avg: 961.33 / Max: 964Min: 1027 / Avg: 1037 / Max: 10441. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native -lrt -laio -lpthread -lm -ldl

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.18Type: Random Write - Engine: Sync - Buffered: No - Direct: Yes - Block Size: 16KB - Disk Target: /mnt/ztest1zfs_6x_ssd_in_2x_raidz16 x 481GB HSCP0480S5xnNMRIzfs_ashift=9_compression=on_dedup=offzfs_ashift=9_compression=on_dedup=onzfs_ashift=9_compression=on_dedup=on_sync=yes14K28K42K56K70KSE +/- 717.25, N = 3SE +/- 473.11, N = 15SE +/- 88.19, N = 3SE +/- 328.30, N = 3SE +/- 145.30, N = 345667495206153366333640331. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native -lrt -laio -lpthread -lm -ldl
OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.18Type: Random Write - Engine: Sync - Buffered: No - Direct: Yes - Block Size: 16KB - Disk Target: /mnt/ztest1zfs_6x_ssd_in_2x_raidz16 x 481GB HSCP0480S5xnNMRIzfs_ashift=9_compression=on_dedup=offzfs_ashift=9_compression=on_dedup=onzfs_ashift=9_compression=on_dedup=on_sync=yes11K22K33K44K55KMin: 44900 / Avg: 45666.67 / Max: 47100Min: 47000 / Avg: 49520 / Max: 54200Min: 61400 / Avg: 61533.33 / Max: 61700Min: 65700 / Avg: 66333.33 / Max: 66800Min: 63800 / Avg: 64033.33 / Max: 643001. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native -lrt -laio -lpthread -lm -ldl

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.18Type: Random Write - Engine: Sync - Buffered: No - Direct: Yes - Block Size: 64KB - Disk Target: /mnt/ztest1zfs_6x_ssd_in_2x_raidz16 x 481GB HSCP0480S5xnNMRIzfs_ashift=9_compression=on_dedup=offzfs_ashift=9_compression=on_dedup=onzfs_ashift=9_compression=on_dedup=on_sync=yes400800120016002000SE +/- 25.71, N = 3SE +/- 32.20, N = 3SE +/- 20.83, N = 4SE +/- 15.88, N = 3SE +/- 21.79, N = 3187918651669200919741. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native -lrt -laio -lpthread -lm -ldl
OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.18Type: Random Write - Engine: Sync - Buffered: No - Direct: Yes - Block Size: 64KB - Disk Target: /mnt/ztest1zfs_6x_ssd_in_2x_raidz16 x 481GB HSCP0480S5xnNMRIzfs_ashift=9_compression=on_dedup=offzfs_ashift=9_compression=on_dedup=onzfs_ashift=9_compression=on_dedup=on_sync=yes400800120016002000Min: 1834 / Avg: 1879.33 / Max: 1923Min: 1804 / Avg: 1865.33 / Max: 1913Min: 1613 / Avg: 1668.5 / Max: 1702Min: 1982 / Avg: 2009.33 / Max: 2037Min: 1939 / Avg: 1974 / Max: 20141. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native -lrt -laio -lpthread -lm -ldl

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.18Type: Random Write - Engine: Sync - Buffered: No - Direct: Yes - Block Size: 64KB - Disk Target: /mnt/ztest1zfs_6x_ssd_in_2x_raidz16 x 481GB HSCP0480S5xnNMRIzfs_ashift=9_compression=on_dedup=offzfs_ashift=9_compression=on_dedup=onzfs_ashift=9_compression=on_dedup=on_sync=yes7K14K21K28K35KSE +/- 433.33, N = 3SE +/- 504.42, N = 3SE +/- 331.66, N = 4SE +/- 260.34, N = 3SE +/- 292.62, N = 430067298672670032133317251. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native -lrt -laio -lpthread -lm -ldl
OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.18Type: Random Write - Engine: Sync - Buffered: No - Direct: Yes - Block Size: 64KB - Disk Target: /mnt/ztest1zfs_6x_ssd_in_2x_raidz16 x 481GB HSCP0480S5xnNMRIzfs_ashift=9_compression=on_dedup=offzfs_ashift=9_compression=on_dedup=onzfs_ashift=9_compression=on_dedup=on_sync=yes6K12K18K24K30KMin: 29300 / Avg: 30066.67 / Max: 30800Min: 28900 / Avg: 29866.67 / Max: 30600Min: 25800 / Avg: 26700 / Max: 27200Min: 31700 / Avg: 32133.33 / Max: 32600Min: 31000 / Avg: 31725 / Max: 322001. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native -lrt -laio -lpthread -lm -ldl

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.18Type: Sequential Read - Engine: Sync - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: /mnt/ztest1zfs_6x_ssd_in_2x_raidz16 x 481GB HSCP0480S5xnNMRIzfs_ashift=9_compression=on_dedup=offzfs_ashift=9_compression=on_dedup=onzfs_ashift=9_compression=on_dedup=on_sync=yes30060090012001500SE +/- 4.18, N = 3SE +/- 2.40, N = 3SE +/- 21.61, N = 3SE +/- 7.69, N = 3SE +/- 4.73, N = 3139614161447143514841. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native -lrt -laio -lpthread -lm -ldl
OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.18Type: Sequential Read - Engine: Sync - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: /mnt/ztest1zfs_6x_ssd_in_2x_raidz16 x 481GB HSCP0480S5xnNMRIzfs_ashift=9_compression=on_dedup=offzfs_ashift=9_compression=on_dedup=onzfs_ashift=9_compression=on_dedup=on_sync=yes30060090012001500Min: 1388 / Avg: 1396.33 / Max: 1401Min: 1413 / Avg: 1416.33 / Max: 1421Min: 1420 / Avg: 1447.33 / Max: 1490Min: 1424 / Avg: 1435.33 / Max: 1450Min: 1477 / Avg: 1484 / Max: 14931. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native -lrt -laio -lpthread -lm -ldl

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.18Type: Sequential Read - Engine: Sync - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: /mnt/ztest1zfs_6x_ssd_in_2x_raidz16 x 481GB HSCP0480S5xnNMRIzfs_ashift=9_compression=on_dedup=offzfs_ashift=9_compression=on_dedup=onzfs_ashift=9_compression=on_dedup=on_sync=yes80K160K240K320K400KSE +/- 1333.33, N = 3SE +/- 666.67, N = 3SE +/- 5567.76, N = 3SE +/- 1763.83, N = 3SE +/- 1201.85, N = 33576673626673710003676673796671. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native -lrt -laio -lpthread -lm -ldl
OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.18Type: Sequential Read - Engine: Sync - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: /mnt/ztest1zfs_6x_ssd_in_2x_raidz16 x 481GB HSCP0480S5xnNMRIzfs_ashift=9_compression=on_dedup=offzfs_ashift=9_compression=on_dedup=onzfs_ashift=9_compression=on_dedup=on_sync=yes70K140K210K280K350KMin: 355000 / Avg: 357666.67 / Max: 359000Min: 362000 / Avg: 362666.67 / Max: 364000Min: 364000 / Avg: 371000 / Max: 382000Min: 365000 / Avg: 367666.67 / Max: 371000Min: 378000 / Avg: 379666.67 / Max: 3820001. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native -lrt -laio -lpthread -lm -ldl

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.18Type: Random Read - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: /mnt/ztest1zfs_6x_ssd_in_2x_raidz16 x 481GB HSCP0480S5xnNMRIzfs_ashift=9_compression=on_dedup=offzfs_ashift=9_compression=on_dedup=onzfs_ashift=9_compression=on_dedup=on_sync=yes2004006008001000SE +/- 13.38, N = 3SE +/- 2.65, N = 3SE +/- 5.84, N = 3SE +/- 4.06, N = 3SE +/- 2.08, N = 38728608688868971. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native -lrt -laio -lpthread -lm -ldl
OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.18Type: Random Read - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: /mnt/ztest1zfs_6x_ssd_in_2x_raidz16 x 481GB HSCP0480S5xnNMRIzfs_ashift=9_compression=on_dedup=offzfs_ashift=9_compression=on_dedup=onzfs_ashift=9_compression=on_dedup=on_sync=yes160320480640800Min: 857 / Avg: 872.33 / Max: 899Min: 856 / Avg: 860 / Max: 865Min: 856 / Avg: 867.67 / Max: 874Min: 879 / Avg: 885.67 / Max: 893Min: 893 / Avg: 897 / Max: 9001. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native -lrt -laio -lpthread -lm -ldl

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.18Type: Random Read - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: /mnt/ztest1zfs_6x_ssd_in_2x_raidz16 x 481GB HSCP0480S5xnNMRIzfs_ashift=9_compression=on_dedup=offzfs_ashift=9_compression=on_dedup=onzfs_ashift=9_compression=on_dedup=on_sync=yes50K100K150K200K250KSE +/- 3382.96, N = 3SE +/- 577.35, N = 3SE +/- 1666.67, N = 3SE +/- 1201.85, N = 3SE +/- 333.33, N = 32233332200002223332266672296671. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native -lrt -laio -lpthread -lm -ldl
OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.18Type: Random Read - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: /mnt/ztest1zfs_6x_ssd_in_2x_raidz16 x 481GB HSCP0480S5xnNMRIzfs_ashift=9_compression=on_dedup=offzfs_ashift=9_compression=on_dedup=onzfs_ashift=9_compression=on_dedup=on_sync=yes40K80K120K160K200KMin: 219000 / Avg: 223333.33 / Max: 230000Min: 219000 / Avg: 220000 / Max: 221000Min: 219000 / Avg: 222333.33 / Max: 224000Min: 225000 / Avg: 226666.67 / Max: 229000Min: 229000 / Avg: 229666.67 / Max: 2300001. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native -lrt -laio -lpthread -lm -ldl

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.18Type: Sequential Read - Engine: Sync - Buffered: No - Direct: Yes - Block Size: 16KB - Disk Target: /mnt/ztest1zfs_6x_ssd_in_2x_raidz16 x 481GB HSCP0480S5xnNMRIzfs_ashift=9_compression=on_dedup=offzfs_ashift=9_compression=on_dedup=onzfs_ashift=9_compression=on_dedup=on_sync=yes8001600240032004000SE +/- 3.76, N = 3SE +/- 25.32, N = 3SE +/- 3.28, N = 3SE +/- 7.22, N = 3SE +/- 35.37, N = 15325432443233329535351. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native -lrt -laio -lpthread -lm -ldl
OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.18Type: Sequential Read - Engine: Sync - Buffered: No - Direct: Yes - Block Size: 16KB - Disk Target: /mnt/ztest1zfs_6x_ssd_in_2x_raidz16 x 481GB HSCP0480S5xnNMRIzfs_ashift=9_compression=on_dedup=offzfs_ashift=9_compression=on_dedup=onzfs_ashift=9_compression=on_dedup=on_sync=yes6001200180024003000Min: 3247 / Avg: 3253.67 / Max: 3260Min: 3194 / Avg: 3244 / Max: 3276Min: 3228 / Avg: 3232.67 / Max: 3239Min: 3282 / Avg: 3294.67 / Max: 3307Min: 3257 / Avg: 3535.4 / Max: 36301. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native -lrt -laio -lpthread -lm -ldl

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.18Type: Sequential Read - Engine: Sync - Buffered: No - Direct: Yes - Block Size: 16KB - Disk Target: /mnt/ztest1zfs_6x_ssd_in_2x_raidz16 x 481GB HSCP0480S5xnNMRIzfs_ashift=9_compression=on_dedup=offzfs_ashift=9_compression=on_dedup=onzfs_ashift=9_compression=on_dedup=on_sync=yes50K100K150K200K250KSE +/- 333.33, N = 3SE +/- 1855.92, N = 3SE +/- 577.35, N = 3SE +/- 2266.67, N = 152083332076672070002110002260671. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native -lrt -laio -lpthread -lm -ldl
OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.18Type: Sequential Read - Engine: Sync - Buffered: No - Direct: Yes - Block Size: 16KB - Disk Target: /mnt/ztest1zfs_6x_ssd_in_2x_raidz16 x 481GB HSCP0480S5xnNMRIzfs_ashift=9_compression=on_dedup=offzfs_ashift=9_compression=on_dedup=onzfs_ashift=9_compression=on_dedup=on_sync=yes40K80K120K160K200KMin: 208000 / Avg: 208333.33 / Max: 209000Min: 204000 / Avg: 207666.67 / Max: 210000Min: 210000 / Avg: 211000 / Max: 212000Min: 208000 / Avg: 226066.67 / Max: 2320001. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native -lrt -laio -lpthread -lm -ldl

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.18Type: Sequential Read - Engine: Sync - Buffered: No - Direct: Yes - Block Size: 64KB - Disk Target: /mnt/ztest1zfs_6x_ssd_in_2x_raidz16 x 481GB HSCP0480S5xnNMRIzfs_ashift=9_compression=on_dedup=offzfs_ashift=9_compression=on_dedup=onzfs_ashift=9_compression=on_dedup=on_sync=yes11002200330044005500SE +/- 4.04, N = 3SE +/- 10.17, N = 3SE +/- 4.73, N = 3SE +/- 6.23, N = 3SE +/- 43.97, N = 12482248974809491153621. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native -lrt -laio -lpthread -lm -ldl
OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.18Type: Sequential Read - Engine: Sync - Buffered: No - Direct: Yes - Block Size: 64KB - Disk Target: /mnt/ztest1zfs_6x_ssd_in_2x_raidz16 x 481GB HSCP0480S5xnNMRIzfs_ashift=9_compression=on_dedup=offzfs_ashift=9_compression=on_dedup=onzfs_ashift=9_compression=on_dedup=on_sync=yes9001800270036004500Min: 4817 / Avg: 4822 / Max: 4830Min: 4877 / Avg: 4896.67 / Max: 4911Min: 4800 / Avg: 4809 / Max: 4816Min: 4899 / Avg: 4911.33 / Max: 4919Min: 4881 / Avg: 5362.33 / Max: 54271. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native -lrt -laio -lpthread -lm -ldl

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.18Type: Sequential Read - Engine: Sync - Buffered: No - Direct: Yes - Block Size: 64KB - Disk Target: /mnt/ztest1zfs_6x_ssd_in_2x_raidz16 x 481GB HSCP0480S5xnNMRIzfs_ashift=9_compression=on_dedup=offzfs_ashift=9_compression=on_dedup=onzfs_ashift=9_compression=on_dedup=on_sync=yes20K40K60K80K100KSE +/- 66.67, N = 3SE +/- 176.38, N = 3SE +/- 328.30, N = 3SE +/- 100.00, N = 3SE +/- 702.86, N = 1277167783337663378600857921. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native -lrt -laio -lpthread -lm -ldl
OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.18Type: Sequential Read - Engine: Sync - Buffered: No - Direct: Yes - Block Size: 64KB - Disk Target: /mnt/ztest1zfs_6x_ssd_in_2x_raidz16 x 481GB HSCP0480S5xnNMRIzfs_ashift=9_compression=on_dedup=offzfs_ashift=9_compression=on_dedup=onzfs_ashift=9_compression=on_dedup=on_sync=yes15K30K45K60K75KMin: 77100 / Avg: 77166.67 / Max: 77300Min: 78000 / Avg: 78333.33 / Max: 78600Min: 76000 / Avg: 76633.33 / Max: 77100Min: 78400 / Avg: 78600 / Max: 78700Min: 78100 / Avg: 85791.67 / Max: 868001. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native -lrt -laio -lpthread -lm -ldl

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.18Type: Sequential Write - Engine: Sync - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: /mnt/ztest1zfs_6x_ssd_in_2x_raidz16 x 481GB HSCP0480S5xnNMRIzfs_ashift=9_compression=on_dedup=offzfs_ashift=9_compression=on_dedup=onzfs_ashift=9_compression=on_dedup=on_sync=yes160320480640800SE +/- 4.26, N = 3SE +/- 5.81, N = 3SE +/- 7.06, N = 3SE +/- 2.73, N = 3SE +/- 4.00, N = 37007037437547511. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native -lrt -laio -lpthread -lm -ldl
OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.18Type: Sequential Write - Engine: Sync - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: /mnt/ztest1zfs_6x_ssd_in_2x_raidz16 x 481GB HSCP0480S5xnNMRIzfs_ashift=9_compression=on_dedup=offzfs_ashift=9_compression=on_dedup=onzfs_ashift=9_compression=on_dedup=on_sync=yes130260390520650Min: 694 / Avg: 699.67 / Max: 708Min: 694 / Avg: 703.33 / Max: 714Min: 730 / Avg: 743.33 / Max: 754Min: 750 / Avg: 753.67 / Max: 759Min: 743 / Avg: 751 / Max: 7551. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native -lrt -laio -lpthread -lm -ldl

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.18Type: Sequential Write - Engine: Sync - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: /mnt/ztest1zfs_6x_ssd_in_2x_raidz16 x 481GB HSCP0480S5xnNMRIzfs_ashift=9_compression=on_dedup=offzfs_ashift=9_compression=on_dedup=onzfs_ashift=9_compression=on_dedup=on_sync=yes40K80K120K160K200KSE +/- 881.92, N = 3SE +/- 1452.97, N = 3SE +/- 1763.83, N = 3SE +/- 666.67, N = 3SE +/- 1000.00, N = 31793331803331903331926671920001. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native -lrt -laio -lpthread -lm -ldl
OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.18Type: Sequential Write - Engine: Sync - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: /mnt/ztest1zfs_6x_ssd_in_2x_raidz16 x 481GB HSCP0480S5xnNMRIzfs_ashift=9_compression=on_dedup=offzfs_ashift=9_compression=on_dedup=onzfs_ashift=9_compression=on_dedup=on_sync=yes30K60K90K120K150KMin: 178000 / Avg: 179333.33 / Max: 181000Min: 178000 / Avg: 180333.33 / Max: 183000Min: 187000 / Avg: 190333.33 / Max: 193000Min: 192000 / Avg: 192666.67 / Max: 194000Min: 190000 / Avg: 192000 / Max: 1930001. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native -lrt -laio -lpthread -lm -ldl

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.18Type: Random Read - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 16KB - Disk Target: /mnt/ztest1zfs_6x_ssd_in_2x_raidz16 x 481GB HSCP0480S5xnNMRIzfs_ashift=9_compression=on_dedup=offzfs_ashift=9_compression=on_dedup=onzfs_ashift=9_compression=on_dedup=on_sync=yes5001000150020002500SE +/- 5.17, N = 3SE +/- 1.86, N = 3SE +/- 16.37, N = 3SE +/- 9.61, N = 3SE +/- 3.84, N = 3228822832321235825071. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native -lrt -laio -lpthread -lm -ldl
OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.18Type: Random Read - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 16KB - Disk Target: /mnt/ztest1zfs_6x_ssd_in_2x_raidz16 x 481GB HSCP0480S5xnNMRIzfs_ashift=9_compression=on_dedup=offzfs_ashift=9_compression=on_dedup=onzfs_ashift=9_compression=on_dedup=on_sync=yes400800120016002000Min: 2278 / Avg: 2288.33 / Max: 2294Min: 2279 / Avg: 2282.67 / Max: 2285Min: 2289 / Avg: 2321.33 / Max: 2342Min: 2339 / Avg: 2358 / Max: 2370Min: 2500 / Avg: 2507.33 / Max: 25131. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native -lrt -laio -lpthread -lm -ldl

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.18Type: Random Read - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 16KB - Disk Target: /mnt/ztest1zfs_6x_ssd_in_2x_raidz16 x 481GB HSCP0480S5xnNMRIzfs_ashift=9_compression=on_dedup=offzfs_ashift=9_compression=on_dedup=onzfs_ashift=9_compression=on_dedup=on_sync=yes30K60K90K120K150KSE +/- 333.33, N = 3SE +/- 881.92, N = 3SE +/- 577.35, N = 3SE +/- 333.33, N = 31466671460001486671510001606671. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native -lrt -laio -lpthread -lm -ldl
OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.18Type: Random Read - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 16KB - Disk Target: /mnt/ztest1zfs_6x_ssd_in_2x_raidz16 x 481GB HSCP0480S5xnNMRIzfs_ashift=9_compression=on_dedup=offzfs_ashift=9_compression=on_dedup=onzfs_ashift=9_compression=on_dedup=on_sync=yes30K60K90K120K150KMin: 146000 / Avg: 146666.67 / Max: 147000Min: 147000 / Avg: 148666.67 / Max: 150000Min: 150000 / Avg: 151000 / Max: 152000Min: 160000 / Avg: 160666.67 / Max: 1610001. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native -lrt -laio -lpthread -lm -ldl

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.18Type: Random Read - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 64KB - Disk Target: /mnt/ztest1zfs_6x_ssd_in_2x_raidz16 x 481GB HSCP0480S5xnNMRIzfs_ashift=9_compression=on_dedup=offzfs_ashift=9_compression=on_dedup=onzfs_ashift=9_compression=on_dedup=on_sync=yes10002000300040005000SE +/- 3.53, N = 3SE +/- 37.97, N = 15SE +/- 28.04, N = 3SE +/- 49.30, N = 6SE +/- 12.66, N = 3409241534117424245481. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native -lrt -laio -lpthread -lm -ldl
OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.18Type: Random Read - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 64KB - Disk Target: /mnt/ztest1zfs_6x_ssd_in_2x_raidz16 x 481GB HSCP0480S5xnNMRIzfs_ashift=9_compression=on_dedup=offzfs_ashift=9_compression=on_dedup=onzfs_ashift=9_compression=on_dedup=on_sync=yes8001600240032004000Min: 4085 / Avg: 4091.67 / Max: 4097Min: 4033 / Avg: 4152.67 / Max: 4508Min: 4080 / Avg: 4117 / Max: 4172Min: 4136 / Avg: 4242 / Max: 4479Min: 4523 / Avg: 4548 / Max: 45641. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native -lrt -laio -lpthread -lm -ldl

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.18Type: Random Read - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 64KB - Disk Target: /mnt/ztest1zfs_6x_ssd_in_2x_raidz16 x 481GB HSCP0480S5xnNMRIzfs_ashift=9_compression=on_dedup=offzfs_ashift=9_compression=on_dedup=onzfs_ashift=9_compression=on_dedup=on_sync=yes16K32K48K64K80KSE +/- 28.87, N = 4SE +/- 583.81, N = 16SE +/- 425.57, N = 3SE +/- 793.06, N = 6SE +/- 185.59, N = 365450662566586767883727671. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native -lrt -laio -lpthread -lm -ldl
OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.18Type: Random Read - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 64KB - Disk Target: /mnt/ztest1zfs_6x_ssd_in_2x_raidz16 x 481GB HSCP0480S5xnNMRIzfs_ashift=9_compression=on_dedup=offzfs_ashift=9_compression=on_dedup=onzfs_ashift=9_compression=on_dedup=on_sync=yes13K26K39K52K65KMin: 65400 / Avg: 65450 / Max: 65500Min: 64500 / Avg: 66256.25 / Max: 72100Min: 65300 / Avg: 65866.67 / Max: 66700Min: 66200 / Avg: 67883.33 / Max: 71700Min: 72400 / Avg: 72766.67 / Max: 730001. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native -lrt -laio -lpthread -lm -ldl

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.18Type: Random Write - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: /mnt/ztest1zfs_6x_ssd_in_2x_raidz16 x 481GB HSCP0480S5xnNMRIzfs_ashift=9_compression=on_dedup=offzfs_ashift=9_compression=on_dedup=onzfs_ashift=9_compression=on_dedup=on_sync=yes50100150200250SE +/- 2.40, N = 3SE +/- 5.09, N = 12SE +/- 2.48, N = 4SE +/- 0.58, N = 32122181812152131. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native -lrt -laio -lpthread -lm -ldl
OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.18Type: Random Write - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: /mnt/ztest1zfs_6x_ssd_in_2x_raidz16 x 481GB HSCP0480S5xnNMRIzfs_ashift=9_compression=on_dedup=offzfs_ashift=9_compression=on_dedup=onzfs_ashift=9_compression=on_dedup=on_sync=yes4080120160200Min: 207 / Avg: 211.67 / Max: 215Min: 184 / Avg: 217.67 / Max: 240Min: 175 / Avg: 181 / Max: 186Min: 214 / Avg: 215 / Max: 2161. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native -lrt -laio -lpthread -lm -ldl

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.18Type: Random Write - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: /mnt/ztest1zfs_6x_ssd_in_2x_raidz16 x 481GB HSCP0480S5xnNMRIzfs_ashift=9_compression=on_dedup=offzfs_ashift=9_compression=on_dedup=onzfs_ashift=9_compression=on_dedup=on_sync=yes12K24K36K48K60KSE +/- 648.93, N = 3SE +/- 1279.40, N = 12SE +/- 678.23, N = 4SE +/- 202.76, N = 354133556674630055033545001. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native -lrt -laio -lpthread -lm -ldl
OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.18Type: Random Write - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: /mnt/ztest1zfs_6x_ssd_in_2x_raidz16 x 481GB HSCP0480S5xnNMRIzfs_ashift=9_compression=on_dedup=offzfs_ashift=9_compression=on_dedup=onzfs_ashift=9_compression=on_dedup=on_sync=yes10K20K30K40K50KMin: 52900 / Avg: 54133.33 / Max: 55100Min: 47200 / Avg: 55666.67 / Max: 61400Min: 44700 / Avg: 46300 / Max: 47700Min: 54700 / Avg: 55033.33 / Max: 554001. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native -lrt -laio -lpthread -lm -ldl

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.18Type: Sequential Write - Engine: Sync - Buffered: No - Direct: Yes - Block Size: 16KB - Disk Target: /mnt/ztest1zfs_6x_ssd_in_2x_raidz16 x 481GB HSCP0480S5xnNMRIzfs_ashift=9_compression=on_dedup=offzfs_ashift=9_compression=on_dedup=onzfs_ashift=9_compression=on_dedup=on_sync=yes400800120016002000SE +/- 7.64, N = 3SE +/- 17.99, N = 6SE +/- 7.23, N = 3SE +/- 16.68, N = 3SE +/- 18.48, N = 3162515811862186218321. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native -lrt -laio -lpthread -lm -ldl
OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.18Type: Sequential Write - Engine: Sync - Buffered: No - Direct: Yes - Block Size: 16KB - Disk Target: /mnt/ztest1zfs_6x_ssd_in_2x_raidz16 x 481GB HSCP0480S5xnNMRIzfs_ashift=9_compression=on_dedup=offzfs_ashift=9_compression=on_dedup=onzfs_ashift=9_compression=on_dedup=on_sync=yes30060090012001500Min: 1615 / Avg: 1625 / Max: 1640Min: 1499 / Avg: 1580.67 / Max: 1629Min: 1849 / Avg: 1862 / Max: 1874Min: 1829 / Avg: 1862.33 / Max: 1880Min: 1800 / Avg: 1832 / Max: 18641. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native -lrt -laio -lpthread -lm -ldl

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.18Type: Sequential Write - Engine: Sync - Buffered: No - Direct: Yes - Block Size: 16KB - Disk Target: /mnt/ztest1zfs_6x_ssd_in_2x_raidz16 x 481GB HSCP0480S5xnNMRIzfs_ashift=9_compression=on_dedup=offzfs_ashift=9_compression=on_dedup=onzfs_ashift=9_compression=on_dedup=on_sync=yes30K60K90K120K150KSE +/- 577.35, N = 3SE +/- 1152.89, N = 6SE +/- 577.35, N = 3SE +/- 1000.00, N = 3SE +/- 1154.70, N = 31040001011501190001190001170001. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native -lrt -laio -lpthread -lm -ldl
OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.18Type: Sequential Write - Engine: Sync - Buffered: No - Direct: Yes - Block Size: 16KB - Disk Target: /mnt/ztest1zfs_6x_ssd_in_2x_raidz16 x 481GB HSCP0480S5xnNMRIzfs_ashift=9_compression=on_dedup=offzfs_ashift=9_compression=on_dedup=onzfs_ashift=9_compression=on_dedup=on_sync=yes20K40K60K80K100KMin: 103000 / Avg: 104000 / Max: 105000Min: 95900 / Avg: 101150 / Max: 104000Min: 118000 / Avg: 119000 / Max: 120000Min: 117000 / Avg: 119000 / Max: 120000Min: 115000 / Avg: 117000 / Max: 1190001. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native -lrt -laio -lpthread -lm -ldl

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.18Type: Sequential Write - Engine: Sync - Buffered: No - Direct: Yes - Block Size: 64KB - Disk Target: /mnt/ztest1zfs_6x_ssd_in_2x_raidz16 x 481GB HSCP0480S5xnNMRIzfs_ashift=9_compression=on_dedup=offzfs_ashift=9_compression=on_dedup=onzfs_ashift=9_compression=on_dedup=on_sync=yes5001000150020002500SE +/- 31.54, N = 4SE +/- 10.20, N = 3SE +/- 30.28, N = 3SE +/- 2.65, N = 3SE +/- 12.86, N = 3210620972176220822091. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native -lrt -laio -lpthread -lm -ldl
OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.18Type: Sequential Write - Engine: Sync - Buffered: No - Direct: Yes - Block Size: 64KB - Disk Target: /mnt/ztest1zfs_6x_ssd_in_2x_raidz16 x 481GB HSCP0480S5xnNMRIzfs_ashift=9_compression=on_dedup=offzfs_ashift=9_compression=on_dedup=onzfs_ashift=9_compression=on_dedup=on_sync=yes400800120016002000Min: 2035 / Avg: 2105.75 / Max: 2186Min: 2085 / Avg: 2096.67 / Max: 2117Min: 2116 / Avg: 2176.33 / Max: 2211Min: 2203 / Avg: 2208 / Max: 2212Min: 2189 / Avg: 2209 / Max: 22331. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native -lrt -laio -lpthread -lm -ldl

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.18Type: Sequential Write - Engine: Sync - Buffered: No - Direct: Yes - Block Size: 64KB - Disk Target: /mnt/ztest1zfs_6x_ssd_in_2x_raidz16 x 481GB HSCP0480S5xnNMRIzfs_ashift=9_compression=on_dedup=offzfs_ashift=9_compression=on_dedup=onzfs_ashift=9_compression=on_dedup=on_sync=yes8K16K24K32K40KSE +/- 319.83, N = 4SE +/- 166.67, N = 3SE +/- 503.32, N = 3SE +/- 57.74, N = 3SE +/- 202.76, N = 333475335673480035300353331. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native -lrt -laio -lpthread -lm -ldl
OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.18Type: Sequential Write - Engine: Sync - Buffered: No - Direct: Yes - Block Size: 64KB - Disk Target: /mnt/ztest1zfs_6x_ssd_in_2x_raidz16 x 481GB HSCP0480S5xnNMRIzfs_ashift=9_compression=on_dedup=offzfs_ashift=9_compression=on_dedup=onzfs_ashift=9_compression=on_dedup=on_sync=yes6K12K18K24K30KMin: 32600 / Avg: 33475 / Max: 34000Min: 33400 / Avg: 33566.67 / Max: 33900Min: 33800 / Avg: 34800 / Max: 35400Min: 35200 / Avg: 35300 / Max: 35400Min: 35000 / Avg: 35333.33 / Max: 357001. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native -lrt -laio -lpthread -lm -ldl

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.18Type: Random Write - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 16KB - Disk Target: /mnt/ztest1zfs_6x_ssd_in_2x_raidz16 x 481GB HSCP0480S5xnNMRIzfs_ashift=9_compression=on_dedup=offzfs_ashift=9_compression=on_dedup=onzfs_ashift=9_compression=on_dedup=on_sync=yes160320480640800SE +/- 6.87, N = 8SE +/- 12.87, N = 14SE +/- 5.18, N = 15SE +/- 8.09, N = 3SE +/- 1.00, N = 36627266207367291. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native -lrt -laio -lpthread -lm -ldl
OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.18Type: Random Write - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 16KB - Disk Target: /mnt/ztest1zfs_6x_ssd_in_2x_raidz16 x 481GB HSCP0480S5xnNMRIzfs_ashift=9_compression=on_dedup=offzfs_ashift=9_compression=on_dedup=onzfs_ashift=9_compression=on_dedup=on_sync=yes130260390520650Min: 637 / Avg: 661.75 / Max: 699Min: 655 / Avg: 725.79 / Max: 834Min: 594 / Avg: 619.6 / Max: 660Min: 720 / Avg: 735.67 / Max: 747Min: 727 / Avg: 729 / Max: 7301. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native -lrt -laio -lpthread -lm -ldl

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.18Type: Random Write - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 16KB - Disk Target: /mnt/ztest1zfs_6x_ssd_in_2x_raidz16 x 481GB HSCP0480S5xnNMRIzfs_ashift=9_compression=on_dedup=offzfs_ashift=9_compression=on_dedup=onzfs_ashift=9_compression=on_dedup=on_sync=yes10K20K30K40K50KSE +/- 506.30, N = 8SE +/- 825.66, N = 14SE +/- 329.62, N = 15SE +/- 513.16, N = 3SE +/- 66.67, N = 342225463643964047100466331. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native -lrt -laio -lpthread -lm -ldl
OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.18Type: Random Write - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 16KB - Disk Target: /mnt/ztest1zfs_6x_ssd_in_2x_raidz16 x 481GB HSCP0480S5xnNMRIzfs_ashift=9_compression=on_dedup=offzfs_ashift=9_compression=on_dedup=onzfs_ashift=9_compression=on_dedup=on_sync=yes8K16K24K32K40KMin: 40000 / Avg: 42225 / Max: 44700Min: 41900 / Avg: 46364.29 / Max: 53300Min: 38000 / Avg: 39640 / Max: 42200Min: 46100 / Avg: 47100 / Max: 47800Min: 46500 / Avg: 46633.33 / Max: 467001. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native -lrt -laio -lpthread -lm -ldl

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.18Type: Random Write - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 64KB - Disk Target: /mnt/ztest1zfs_6x_ssd_in_2x_raidz16 x 481GB HSCP0480S5xnNMRIzfs_ashift=9_compression=on_dedup=offzfs_ashift=9_compression=on_dedup=onzfs_ashift=9_compression=on_dedup=on_sync=yes30060090012001500SE +/- 20.14, N = 15SE +/- 4.10, N = 3SE +/- 26.54, N = 3SE +/- 13.45, N = 3161116131544160816071. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native -lrt -laio -lpthread -lm -ldl
OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.18Type: Random Write - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 64KB - Disk Target: /mnt/ztest1zfs_6x_ssd_in_2x_raidz16 x 481GB HSCP0480S5xnNMRIzfs_ashift=9_compression=on_dedup=offzfs_ashift=9_compression=on_dedup=onzfs_ashift=9_compression=on_dedup=on_sync=yes30060090012001500Min: 1490 / Avg: 1610.8 / Max: 1756Min: 1607 / Avg: 1613.33 / Max: 1621Min: 1579 / Avg: 1608 / Max: 1661Min: 1588 / Avg: 1607 / Max: 16331. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native -lrt -laio -lpthread -lm -ldl

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.18Type: Random Write - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 64KB - Disk Target: /mnt/ztest1zfs_6x_ssd_in_2x_raidz16 x 481GB HSCP0480S5xnNMRIzfs_ashift=9_compression=on_dedup=offzfs_ashift=9_compression=on_dedup=onzfs_ashift=9_compression=on_dedup=on_sync=yes6K12K18K24K30KSE +/- 313.88, N = 15SE +/- 57.74, N = 3SE +/- 433.33, N = 3SE +/- 208.17, N = 325707258002470025733257001. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native -lrt -laio -lpthread -lm -ldl
OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.18Type: Random Write - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 64KB - Disk Target: /mnt/ztest1zfs_6x_ssd_in_2x_raidz16 x 481GB HSCP0480S5xnNMRIzfs_ashift=9_compression=on_dedup=offzfs_ashift=9_compression=on_dedup=onzfs_ashift=9_compression=on_dedup=on_sync=yes4K8K12K16K20KMin: 23800 / Avg: 25706.67 / Max: 28100Min: 25700 / Avg: 25800 / Max: 25900Min: 25300 / Avg: 25733.33 / Max: 26600Min: 25400 / Avg: 25700 / Max: 261001. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native -lrt -laio -lpthread -lm -ldl

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.18Type: Sequential Read - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: /mnt/ztest1zfs_6x_ssd_in_2x_raidz16 x 481GB HSCP0480S5xnNMRIzfs_ashift=9_compression=on_dedup=offzfs_ashift=9_compression=on_dedup=onzfs_ashift=9_compression=on_dedup=on_sync=yes2004006008001000SE +/- 14.90, N = 3SE +/- 2.73, N = 3SE +/- 6.35, N = 3SE +/- 3.84, N = 3SE +/- 1.86, N = 3108611171131112111361. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native -lrt -laio -lpthread -lm -ldl
OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.18Type: Sequential Read - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: /mnt/ztest1zfs_6x_ssd_in_2x_raidz16 x 481GB HSCP0480S5xnNMRIzfs_ashift=9_compression=on_dedup=offzfs_ashift=9_compression=on_dedup=onzfs_ashift=9_compression=on_dedup=on_sync=yes2004006008001000Min: 1069 / Avg: 1086.33 / Max: 1116Min: 1112 / Avg: 1117.33 / Max: 1121Min: 1120 / Avg: 1131 / Max: 1142Min: 1115 / Avg: 1120.67 / Max: 1128Min: 1134 / Avg: 1136.33 / Max: 11401. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native -lrt -laio -lpthread -lm -ldl

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.18Type: Sequential Read - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: /mnt/ztest1zfs_6x_ssd_in_2x_raidz16 x 481GB HSCP0480S5xnNMRIzfs_ashift=9_compression=on_dedup=offzfs_ashift=9_compression=on_dedup=onzfs_ashift=9_compression=on_dedup=on_sync=yes60K120K180K240K300KSE +/- 3844.19, N = 3SE +/- 666.67, N = 3SE +/- 1452.97, N = 32783332863332896672870002910001. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native -lrt -laio -lpthread -lm -ldl
OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.18Type: Sequential Read - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: /mnt/ztest1zfs_6x_ssd_in_2x_raidz16 x 481GB HSCP0480S5xnNMRIzfs_ashift=9_compression=on_dedup=offzfs_ashift=9_compression=on_dedup=onzfs_ashift=9_compression=on_dedup=on_sync=yes50K100K150K200K250KMin: 274000 / Avg: 278333.33 / Max: 286000Min: 285000 / Avg: 286333.33 / Max: 287000Min: 287000 / Avg: 289666.67 / Max: 2920001. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native -lrt -laio -lpthread -lm -ldl

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.18Type: Sequential Read - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 16KB - Disk Target: /mnt/ztest1zfs_6x_ssd_in_2x_raidz16 x 481GB HSCP0480S5xnNMRIzfs_ashift=9_compression=on_dedup=offzfs_ashift=9_compression=on_dedup=onzfs_ashift=9_compression=on_dedup=on_sync=yes6001200180024003000SE +/- 5.90, N = 3SE +/- 2.40, N = 3SE +/- 35.95, N = 5SE +/- 26.21, N = 3SE +/- 4.81, N = 3275328202821285829561. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native -lrt -laio -lpthread -lm -ldl
OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.18Type: Sequential Read - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 16KB - Disk Target: /mnt/ztest1zfs_6x_ssd_in_2x_raidz16 x 481GB HSCP0480S5xnNMRIzfs_ashift=9_compression=on_dedup=offzfs_ashift=9_compression=on_dedup=onzfs_ashift=9_compression=on_dedup=on_sync=yes5001000150020002500Min: 2741 / Avg: 2752.67 / Max: 2760Min: 2815 / Avg: 2819.67 / Max: 2823Min: 2778 / Avg: 2820.8 / Max: 2964Min: 2829 / Avg: 2857.67 / Max: 2910Min: 2947 / Avg: 2956.33 / Max: 29631. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native -lrt -laio -lpthread -lm -ldl

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.18Type: Sequential Read - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 16KB - Disk Target: /mnt/ztest1zfs_6x_ssd_in_2x_raidz16 x 481GB HSCP0480S5xnNMRIzfs_ashift=9_compression=on_dedup=offzfs_ashift=9_compression=on_dedup=onzfs_ashift=9_compression=on_dedup=on_sync=yes40K80K120K160K200KSE +/- 577.35, N = 3SE +/- 333.33, N = 3SE +/- 2357.97, N = 5SE +/- 1666.67, N = 3SE +/- 333.33, N = 31760001806671806001826671893331. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native -lrt -laio -lpthread -lm -ldl
OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.18Type: Sequential Read - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 16KB - Disk Target: /mnt/ztest1zfs_6x_ssd_in_2x_raidz16 x 481GB HSCP0480S5xnNMRIzfs_ashift=9_compression=on_dedup=offzfs_ashift=9_compression=on_dedup=onzfs_ashift=9_compression=on_dedup=on_sync=yes30K60K90K120K150KMin: 175000 / Avg: 176000 / Max: 177000Min: 180000 / Avg: 180666.67 / Max: 181000Min: 178000 / Avg: 180600 / Max: 190000Min: 181000 / Avg: 182666.67 / Max: 186000Min: 189000 / Avg: 189333.33 / Max: 1900001. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native -lrt -laio -lpthread -lm -ldl

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.18Type: Sequential Read - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 64KB - Disk Target: /mnt/ztest1zfs_6x_ssd_in_2x_raidz16 x 481GB HSCP0480S5xnNMRIzfs_ashift=9_compression=on_dedup=offzfs_ashift=9_compression=on_dedup=onzfs_ashift=9_compression=on_dedup=on_sync=yes10002000300040005000SE +/- 4.37, N = 3SE +/- 9.17, N = 3SE +/- 37.20, N = 15SE +/- 6.12, N = 3SE +/- 14.93, N = 3438544954454453447651. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native -lrt -laio -lpthread -lm -ldl
OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.18Type: Sequential Read - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 64KB - Disk Target: /mnt/ztest1zfs_6x_ssd_in_2x_raidz16 x 481GB HSCP0480S5xnNMRIzfs_ashift=9_compression=on_dedup=offzfs_ashift=9_compression=on_dedup=onzfs_ashift=9_compression=on_dedup=on_sync=yes8001600240032004000Min: 4380 / Avg: 4385.33 / Max: 4394Min: 4483 / Avg: 4495 / Max: 4513Min: 4305 / Avg: 4453.87 / Max: 4804Min: 4524 / Avg: 4533.67 / Max: 4545Min: 4737 / Avg: 4765 / Max: 47881. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native -lrt -laio -lpthread -lm -ldl

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.18Type: Sequential Read - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 64KB - Disk Target: /mnt/ztest1zfs_6x_ssd_in_2x_raidz16 x 481GB HSCP0480S5xnNMRIzfs_ashift=9_compression=on_dedup=offzfs_ashift=9_compression=on_dedup=onzfs_ashift=9_compression=on_dedup=on_sync=yes16K32K48K64K80KSE +/- 66.67, N = 3SE +/- 152.75, N = 3SE +/- 594.56, N = 15SE +/- 88.19, N = 3SE +/- 233.33, N = 370167719007126072533762331. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native -lrt -laio -lpthread -lm -ldl
OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.18Type: Sequential Read - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 64KB - Disk Target: /mnt/ztest1zfs_6x_ssd_in_2x_raidz16 x 481GB HSCP0480S5xnNMRIzfs_ashift=9_compression=on_dedup=offzfs_ashift=9_compression=on_dedup=onzfs_ashift=9_compression=on_dedup=on_sync=yes13K26K39K52K65KMin: 70100 / Avg: 70166.67 / Max: 70300Min: 71700 / Avg: 71900 / Max: 72200Min: 68900 / Avg: 71260 / Max: 76900Min: 72400 / Avg: 72533.33 / Max: 72700Min: 75800 / Avg: 76233.33 / Max: 766001. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native -lrt -laio -lpthread -lm -ldl

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.18Type: Sequential Write - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: /mnt/ztest1zfs_6x_ssd_in_2x_raidz16 x 481GB HSCP0480S5xnNMRIzfs_ashift=9_compression=on_dedup=offzfs_ashift=9_compression=on_dedup=onzfs_ashift=9_compression=on_dedup=on_sync=yes140280420560700SE +/- 6.89, N = 3SE +/- 5.90, N = 3SE +/- 1.00, N = 3SE +/- 2.89, N = 3SE +/- 2.33, N = 36156286146066081. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native -lrt -laio -lpthread -lm -ldl
OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.18Type: Sequential Write - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: /mnt/ztest1zfs_6x_ssd_in_2x_raidz16 x 481GB HSCP0480S5xnNMRIzfs_ashift=9_compression=on_dedup=offzfs_ashift=9_compression=on_dedup=onzfs_ashift=9_compression=on_dedup=on_sync=yes110220330440550Min: 602 / Avg: 615.33 / Max: 625Min: 621 / Avg: 628.33 / Max: 640Min: 612 / Avg: 614 / Max: 615Min: 601 / Avg: 606 / Max: 611Min: 603 / Avg: 607.67 / Max: 6101. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native -lrt -laio -lpthread -lm -ldl

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.18Type: Sequential Write - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: /mnt/ztest1zfs_6x_ssd_in_2x_raidz16 x 481GB HSCP0480S5xnNMRIzfs_ashift=9_compression=on_dedup=offzfs_ashift=9_compression=on_dedup=onzfs_ashift=9_compression=on_dedup=on_sync=yes30K60K90K120K150KSE +/- 1763.83, N = 3SE +/- 1527.53, N = 3SE +/- 881.92, N = 3SE +/- 666.67, N = 31573331610001570001553331553331. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native -lrt -laio -lpthread -lm -ldl
OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.18Type: Sequential Write - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: /mnt/ztest1zfs_6x_ssd_in_2x_raidz16 x 481GB HSCP0480S5xnNMRIzfs_ashift=9_compression=on_dedup=offzfs_ashift=9_compression=on_dedup=onzfs_ashift=9_compression=on_dedup=on_sync=yes30K60K90K120K150KMin: 154000 / Avg: 157333.33 / Max: 160000Min: 159000 / Avg: 161000 / Max: 164000Min: 154000 / Avg: 155333.33 / Max: 157000Min: 154000 / Avg: 155333.33 / Max: 1560001. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native -lrt -laio -lpthread -lm -ldl

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.18Type: Sequential Write - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 16KB - Disk Target: /mnt/ztest1zfs_6x_ssd_in_2x_raidz16 x 481GB HSCP0480S5xnNMRIzfs_ashift=9_compression=on_dedup=offzfs_ashift=9_compression=on_dedup=onzfs_ashift=9_compression=on_dedup=on_sync=yes30060090012001500SE +/- 16.33, N = 3SE +/- 19.47, N = 3SE +/- 12.05, N = 15SE +/- 8.54, N = 3SE +/- 1.53, N = 3142314411441147114801. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native -lrt -laio -lpthread -lm -ldl
OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.18Type: Sequential Write - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 16KB - Disk Target: /mnt/ztest1zfs_6x_ssd_in_2x_raidz16 x 481GB HSCP0480S5xnNMRIzfs_ashift=9_compression=on_dedup=offzfs_ashift=9_compression=on_dedup=onzfs_ashift=9_compression=on_dedup=on_sync=yes30060090012001500Min: 1393 / Avg: 1423.33 / Max: 1449Min: 1404 / Avg: 1441 / Max: 1470Min: 1399 / Avg: 1440.53 / Max: 1510Min: 1454 / Avg: 1471 / Max: 1481Min: 1478 / Avg: 1480 / Max: 14831. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native -lrt -laio -lpthread -lm -ldl

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.18Type: Sequential Write - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 16KB - Disk Target: /mnt/ztest1zfs_6x_ssd_in_2x_raidz16 x 481GB HSCP0480S5xnNMRIzfs_ashift=9_compression=on_dedup=offzfs_ashift=9_compression=on_dedup=onzfs_ashift=9_compression=on_dedup=on_sync=yes20K40K60K80K100KSE +/- 1052.51, N = 3SE +/- 1234.68, N = 3SE +/- 772.68, N = 15SE +/- 569.60, N = 3SE +/- 100.00, N = 391067922339218794133947001. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native -lrt -laio -lpthread -lm -ldl
OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.18Type: Sequential Write - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 16KB - Disk Target: /mnt/ztest1zfs_6x_ssd_in_2x_raidz16 x 481GB HSCP0480S5xnNMRIzfs_ashift=9_compression=on_dedup=offzfs_ashift=9_compression=on_dedup=onzfs_ashift=9_compression=on_dedup=on_sync=yes16K32K48K64K80KMin: 89100 / Avg: 91066.67 / Max: 92700Min: 89900 / Avg: 92233.33 / Max: 94100Min: 89500 / Avg: 92186.67 / Max: 96600Min: 93000 / Avg: 94133.33 / Max: 94800Min: 94600 / Avg: 94700 / Max: 949001. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native -lrt -laio -lpthread -lm -ldl

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.18Type: Sequential Write - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 64KB - Disk Target: /mnt/ztest1zfs_6x_ssd_in_2x_raidz16 x 481GB HSCP0480S5xnNMRIzfs_ashift=9_compression=on_dedup=offzfs_ashift=9_compression=on_dedup=onzfs_ashift=9_compression=on_dedup=on_sync=yes5001000150020002500SE +/- 21.62, N = 3SE +/- 33.95, N = 3SE +/- 30.83, N = 3SE +/- 5.51, N = 3SE +/- 4.41, N = 3181120721961215021551. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native -lrt -laio -lpthread -lm -ldl
OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.18Type: Sequential Write - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 64KB - Disk Target: /mnt/ztest1zfs_6x_ssd_in_2x_raidz16 x 481GB HSCP0480S5xnNMRIzfs_ashift=9_compression=on_dedup=offzfs_ashift=9_compression=on_dedup=onzfs_ashift=9_compression=on_dedup=on_sync=yes400800120016002000Min: 1771 / Avg: 1811.33 / Max: 1845Min: 2029 / Avg: 2072 / Max: 2139Min: 1916 / Avg: 1961 / Max: 2020Min: 2140 / Avg: 2150 / Max: 2159Min: 2148 / Avg: 2154.67 / Max: 21631. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native -lrt -laio -lpthread -lm -ldl

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.18Type: Sequential Write - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 64KB - Disk Target: /mnt/ztest1zfs_6x_ssd_in_2x_raidz16 x 481GB HSCP0480S5xnNMRIzfs_ashift=9_compression=on_dedup=offzfs_ashift=9_compression=on_dedup=onzfs_ashift=9_compression=on_dedup=on_sync=yes7K14K21K28K35KSE +/- 352.77, N = 3SE +/- 523.87, N = 3SE +/- 422.30, N = 4SE +/- 88.19, N = 3SE +/- 66.67, N = 328967331673160034367344671. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native -lrt -laio -lpthread -lm -ldl
OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.18Type: Sequential Write - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 64KB - Disk Target: /mnt/ztest1zfs_6x_ssd_in_2x_raidz16 x 481GB HSCP0480S5xnNMRIzfs_ashift=9_compression=on_dedup=offzfs_ashift=9_compression=on_dedup=onzfs_ashift=9_compression=on_dedup=on_sync=yes6K12K18K24K30KMin: 28300 / Avg: 28966.67 / Max: 29500Min: 32500 / Avg: 33166.67 / Max: 34200Min: 30600 / Avg: 31600 / Max: 32300Min: 34200 / Avg: 34366.67 / Max: 34500Min: 34400 / Avg: 34466.67 / Max: 346001. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native -lrt -laio -lpthread -lm -ldl

48 Results Shown

Flexible IO Tester:
  Rand Read - Sync - No - Yes - 4KB - /mnt/ztest1:
    MB/s
    IOPS
  Rand Read - Sync - No - Yes - 16KB - /mnt/ztest1:
    MB/s
    IOPS
  Rand Read - Sync - No - Yes - 64KB - /mnt/ztest1:
    MB/s
    IOPS
  Rand Write - Sync - No - Yes - 4KB - /mnt/ztest1:
    MB/s
    IOPS
  Rand Write - Sync - No - Yes - 16KB - /mnt/ztest1:
    MB/s
    IOPS
  Rand Write - Sync - No - Yes - 64KB - /mnt/ztest1:
    MB/s
    IOPS
  Seq Read - Sync - No - Yes - 4KB - /mnt/ztest1:
    MB/s
    IOPS
  Rand Read - Linux AIO - No - Yes - 4KB - /mnt/ztest1:
    MB/s
    IOPS
  Seq Read - Sync - No - Yes - 16KB - /mnt/ztest1:
    MB/s
    IOPS
  Seq Read - Sync - No - Yes - 64KB - /mnt/ztest1:
    MB/s
    IOPS
  Seq Write - Sync - No - Yes - 4KB - /mnt/ztest1:
    MB/s
    IOPS
  Rand Read - Linux AIO - No - Yes - 16KB - /mnt/ztest1:
    MB/s
    IOPS
  Rand Read - Linux AIO - No - Yes - 64KB - /mnt/ztest1:
    MB/s
    IOPS
  Rand Write - Linux AIO - No - Yes - 4KB - /mnt/ztest1:
    MB/s
    IOPS
  Seq Write - Sync - No - Yes - 16KB - /mnt/ztest1:
    MB/s
    IOPS
  Seq Write - Sync - No - Yes - 64KB - /mnt/ztest1:
    MB/s
    IOPS
  Rand Write - Linux AIO - No - Yes - 16KB - /mnt/ztest1:
    MB/s
    IOPS
  Rand Write - Linux AIO - No - Yes - 64KB - /mnt/ztest1:
    MB/s
    IOPS
  Seq Read - Linux AIO - No - Yes - 4KB - /mnt/ztest1:
    MB/s
    IOPS
  Seq Read - Linux AIO - No - Yes - 16KB - /mnt/ztest1:
    MB/s
    IOPS
  Seq Read - Linux AIO - No - Yes - 64KB - /mnt/ztest1:
    MB/s
    IOPS
  Seq Write - Linux AIO - No - Yes - 4KB - /mnt/ztest1:
    MB/s
    IOPS
  Seq Write - Linux AIO - No - Yes - 16KB - /mnt/ztest1:
    MB/s
    IOPS
  Seq Write - Linux AIO - No - Yes - 64KB - /mnt/ztest1:
    MB/s
    IOPS

zfs_6x_ssd_in_2x_raidz1

Compiler Notes: --build=x86_64-redhat-linux --disable-libmpx --disable-libunwind-exceptions --enable-__cxa_atexit --enable-bootstrap --enable-cet --enable-checking=release --enable-gnu-indirect-function --enable-gnu-unique-object --enable-initfini-array --enable-languages=c,c++,fortran,lto --enable-multilib --enable-offload-targets=nvptx-none --enable-plugin --enable-shared --enable-threads=posix --mandir=/usr/share/man --with-arch_32=x86-64 --with-gcc-major-version-only --with-isl --with-linker-hash-style=gnu --with-tune=generic --without-cuda-driver
Disk Notes: MQ-DEADLINE / attr2,inode64,noquota,relatime,rw,seclabel
Processor Notes: Scaling Governor: intel_pstate performance - CPU Microcode: 0x43
Security Notes: SELinux + itlb_multihit: KVM: Mitigation of Split huge pages + l1tf: Mitigation of PTE Inversion; VMX: conditional cache flushes SMT vulnerable + mds: Mitigation of Clear buffers; SMT vulnerable + meltdown: Mitigation of PTI + spec_store_bypass: Mitigation of SSB disabled via prctl and seccomp + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Full generic retpoline IBPB: conditional IBRS_FW STIBP: conditional RSB filling + srbds: Not affected + tsx_async_abort: Not affected

Testing initiated at 24 June 2020 21:59 by user .

zfs_6x_ssd_in_2x_raidz1_ashift=12

Compiler Notes: --build=x86_64-redhat-linux --disable-libmpx --disable-libunwind-exceptions --enable-__cxa_atexit --enable-bootstrap --enable-cet --enable-checking=release --enable-gnu-indirect-function --enable-gnu-unique-object --enable-initfini-array --enable-languages=c,c++,fortran,lto --enable-multilib --enable-offload-targets=nvptx-none --enable-plugin --enable-shared --enable-threads=posix --mandir=/usr/share/man --with-arch_32=x86-64 --with-gcc-major-version-only --with-isl --with-linker-hash-style=gnu --with-tune=generic --without-cuda-driver
Disk Notes: MQ-DEADLINE / attr2,inode64,noquota,relatime,rw,seclabel
Processor Notes: Scaling Governor: intel_pstate performance - CPU Microcode: 0x43
Security Notes: SELinux + itlb_multihit: KVM: Mitigation of Split huge pages + l1tf: Mitigation of PTE Inversion; VMX: conditional cache flushes SMT vulnerable + mds: Mitigation of Clear buffers; SMT vulnerable + meltdown: Mitigation of PTI + spec_store_bypass: Mitigation of SSB disabled via prctl and seccomp + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Full generic retpoline IBPB: conditional IBRS_FW STIBP: conditional RSB filling + srbds: Not affected + tsx_async_abort: Not affected

Testing initiated at 24 June 2020 23:34 by user .

6 x 481GB HSCP0480S5xnNMRI

Compiler Notes: --build=x86_64-redhat-linux --disable-libmpx --disable-libunwind-exceptions --enable-__cxa_atexit --enable-bootstrap --enable-cet --enable-checking=release --enable-gnu-indirect-function --enable-gnu-unique-object --enable-initfini-array --enable-languages=c,c++,fortran,lto --enable-multilib --enable-offload-targets=nvptx-none --enable-plugin --enable-shared --enable-threads=posix --mandir=/usr/share/man --with-arch_32=x86-64 --with-gcc-major-version-only --with-isl --with-linker-hash-style=gnu --with-tune=generic --without-cuda-driver
Disk Notes: MQ-DEADLINE / attr2,inode64,noquota,relatime,rw,seclabel
Processor Notes: Scaling Governor: intel_pstate performance - CPU Microcode: 0x43
Security Notes: SELinux + itlb_multihit: KVM: Mitigation of Split huge pages + l1tf: Mitigation of PTE Inversion; VMX: conditional cache flushes SMT vulnerable + mds: Mitigation of Clear buffers; SMT vulnerable + meltdown: Mitigation of PTI + spec_store_bypass: Mitigation of SSB disabled via prctl and seccomp + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Full generic retpoline IBPB: conditional IBRS_FW STIBP: conditional RSB filling + srbds: Not affected + tsx_async_abort: Not affected

Testing initiated at 24 June 2020 23:35 by user .

zfs_ashift=9_compression=on_dedup=off

Compiler Notes: --build=x86_64-redhat-linux --disable-libmpx --disable-libunwind-exceptions --enable-__cxa_atexit --enable-bootstrap --enable-cet --enable-checking=release --enable-gnu-indirect-function --enable-gnu-unique-object --enable-initfini-array --enable-languages=c,c++,fortran,lto --enable-multilib --enable-offload-targets=nvptx-none --enable-plugin --enable-shared --enable-threads=posix --mandir=/usr/share/man --with-arch_32=x86-64 --with-gcc-major-version-only --with-isl --with-linker-hash-style=gnu --with-tune=generic --without-cuda-driver
Disk Notes: MQ-DEADLINE / attr2,inode64,noquota,relatime,rw,seclabel
Processor Notes: Scaling Governor: intel_pstate performance - CPU Microcode: 0x43
Security Notes: SELinux + itlb_multihit: KVM: Mitigation of Split huge pages + l1tf: Mitigation of PTE Inversion; VMX: conditional cache flushes SMT vulnerable + mds: Mitigation of Clear buffers; SMT vulnerable + meltdown: Mitigation of PTI + spec_store_bypass: Mitigation of SSB disabled via prctl and seccomp + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Full generic retpoline IBPB: conditional IBRS_FW STIBP: conditional RSB filling + srbds: Not affected + tsx_async_abort: Not affected

Testing initiated at 25 June 2020 11:25 by user .

zfs_ashift=9_compression=on_dedup=on

Compiler Notes: --build=x86_64-redhat-linux --disable-libmpx --disable-libunwind-exceptions --enable-__cxa_atexit --enable-bootstrap --enable-cet --enable-checking=release --enable-gnu-indirect-function --enable-gnu-unique-object --enable-initfini-array --enable-languages=c,c++,fortran,lto --enable-multilib --enable-offload-targets=nvptx-none --enable-plugin --enable-shared --enable-threads=posix --mandir=/usr/share/man --with-arch_32=x86-64 --with-gcc-major-version-only --with-isl --with-linker-hash-style=gnu --with-tune=generic --without-cuda-driver
Disk Notes: MQ-DEADLINE / attr2,inode64,noquota,relatime,rw,seclabel
Processor Notes: Scaling Governor: intel_pstate performance - CPU Microcode: 0x43
Security Notes: SELinux + itlb_multihit: KVM: Mitigation of Split huge pages + l1tf: Mitigation of PTE Inversion; VMX: conditional cache flushes SMT vulnerable + mds: Mitigation of Clear buffers; SMT vulnerable + meltdown: Mitigation of PTI + spec_store_bypass: Mitigation of SSB disabled via prctl and seccomp + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Full generic retpoline IBPB: conditional IBRS_FW STIBP: conditional RSB filling + srbds: Not affected + tsx_async_abort: Not affected

Testing initiated at 25 June 2020 12:43 by user .

zfs_ashift=9_compression=on_dedup=on_sync=yes

Processor: 2 x Intel Xeon E5-2640 v3 @ 3.40GHz (16 Cores / 32 Threads), Motherboard: Dell 0CNCJW (2.11.0 BIOS), Chipset: Intel Xeon E7 v3/Xeon, Memory: 16 x GB DDR4-2133MT/s HMA42GR7MFR4N-TF, Disk: 6 x 481GB HSCP0480S5xnNMRI + 99GB PERC H730 Mini + 16GB Cruzer Blade + 16GB Cruzer Fit, Graphics: Matrox G200eR2, Monitor: DELL E196FP, Network: 2 x Intel 10-Gigabit X540-AT2 + 2 x Intel I350 + Mellanox MT27500

OS: CentOS Linux 8, Kernel: 4.18.0-193.6.3.el8_2.x86_64 (x86_64), Compiler: GCC 8.3.1 20191121, File-System: xfs, Screen Resolution: 1280x1024

Compiler Notes: --build=x86_64-redhat-linux --disable-libmpx --disable-libunwind-exceptions --enable-__cxa_atexit --enable-bootstrap --enable-cet --enable-checking=release --enable-gnu-indirect-function --enable-gnu-unique-object --enable-initfini-array --enable-languages=c,c++,fortran,lto --enable-multilib --enable-offload-targets=nvptx-none --enable-plugin --enable-shared --enable-threads=posix --mandir=/usr/share/man --with-arch_32=x86-64 --with-gcc-major-version-only --with-isl --with-linker-hash-style=gnu --with-tune=generic --without-cuda-driver
Disk Notes: MQ-DEADLINE / attr2,inode64,noquota,relatime,rw,seclabel
Processor Notes: Scaling Governor: intel_pstate performance - CPU Microcode: 0x43
Security Notes: SELinux + itlb_multihit: KVM: Mitigation of Split huge pages + l1tf: Mitigation of PTE Inversion; VMX: conditional cache flushes SMT vulnerable + mds: Mitigation of Clear buffers; SMT vulnerable + meltdown: Mitigation of PTI + spec_store_bypass: Mitigation of SSB disabled via prctl and seccomp + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Full generic retpoline IBPB: conditional IBRS_FW STIBP: conditional RSB filling + srbds: Not affected + tsx_async_abort: Not affected

Testing initiated at 25 June 2020 14:32 by user .