truenas-fio Supermicro with Intel Xeon E3-1230 v6 32Gb RAM 8HDD in 2 mirrored VDEVS with 4 WD HDD truenas-fio-nolog-1: Processor: Intel Xeon E3-1230 v6 @ 3.50GHz (8 Cores), Motherboard: Supermicro Super Server 0123456789, Memory: 32GB, Disk: Generic NVMe Device OS: FreeBSD, Kernel: 13.1-RELEASE-p7 (x86_64), Compiler: Clang 14.0.5, File-System: zfs Flexible IO Tester 3.35 Type: Sequential Read - IO Engine: POSIX AIO - Buffered: Yes - Direct: No - Block Size: 256KB - Disk Target: Default Test Directory MB/s > Higher Is Better truenas-fio-nolog-1 . 7216 |=================================================== Flexible IO Tester 3.35 Type: Sequential Write - IO Engine: POSIX AIO - Buffered: Yes - Direct: No - Block Size: 2MB - Disk Target: Default Test Directory MB/s > Higher Is Better truenas-fio-nolog-1 . 7834 |=================================================== Flexible IO Tester 3.35 Type: Sequential Write - IO Engine: POSIX AIO - Buffered: Yes - Direct: No - Block Size: 8KB - Disk Target: Default Test Directory MB/s > Higher Is Better truenas-fio-nolog-1 . 1236 |=================================================== Flexible IO Tester Type: Sequential Read - IO Engine: POSIX AIO - Buffered: Yes - Direct: No - Block Size: 8KB - Disk Target: Default Test Directory IOPS > Higher Is Better truenas-fio-nolog-1 . 332667 |================================================= Flexible IO Tester Type: Sequential Read - IO Engine: POSIX AIO - Buffered: Yes - Direct: No - Block Size: 32KB - Disk Target: Default Test Directory IOPS > Higher Is Better truenas-fio-nolog-1 . 237667 |================================================= Flexible IO Tester 3.35 Type: Sequential Write - IO Engine: POSIX AIO - Buffered: Yes - Direct: No - Block Size: 256KB - Disk Target: Default Test Directory MB/s > Higher Is Better truenas-fio-nolog-1 . 7985 |=================================================== Flexible IO Tester 3.35 Type: Sequential Read - IO Engine: POSIX AIO - Buffered: Yes - Direct: No - Block Size: 4KB - Disk Target: Default Test Directory MB/s > Higher Is Better truenas-fio-nolog-1 . 1416 |=================================================== Flexible IO Tester 3.35 Type: Random Read - IO Engine: POSIX AIO - Buffered: Yes - Direct: No - Block Size: 16KB - Disk Target: Default Test Directory MB/s > Higher Is Better truenas-fio-nolog-1 . 2140 |=================================================== Flexible IO Tester Type: Random Read - IO Engine: POSIX AIO - Buffered: Yes - Direct: No - Block Size: 16KB - Disk Target: Default Test Directory IOPS > Higher Is Better truenas-fio-nolog-1 . 136667 |================================================= Flexible IO Tester 3.35 Type: Random Read - IO Engine: POSIX AIO - Buffered: Yes - Direct: No - Block Size: 8KB - Disk Target: Default Test Directory MB/s > Higher Is Better truenas-fio-nolog-1 . 1113 |=================================================== Flexible IO Tester Type: Random Read - IO Engine: POSIX AIO - Buffered: Yes - Direct: No - Block Size: 8KB - Disk Target: Default Test Directory IOPS > Higher Is Better truenas-fio-nolog-1 . 142667 |================================================= Flexible IO Tester 3.35 Type: Random Read - IO Engine: POSIX AIO - Buffered: Yes - Direct: No - Block Size: 4MB - Disk Target: Default Test Directory MB/s > Higher Is Better truenas-fio-nolog-1 . 6403 |=================================================== Flexible IO Tester Type: Random Read - IO Engine: POSIX AIO - Buffered: Yes - Direct: No - Block Size: 4KB - Disk Target: Default Test Directory IOPS > Higher Is Better truenas-fio-nolog-1 . 142000 |================================================= Flexible IO Tester 3.35 Type: Random Read - IO Engine: POSIX AIO - Buffered: Yes - Direct: No - Block Size: 4KB - Disk Target: Default Test Directory MB/s > Higher Is Better truenas-fio-nolog-1 . 555 |==================================================== Flexible IO Tester 3.35 Type: Sequential Write - IO Engine: POSIX AIO - Buffered: Yes - Direct: No - Block Size: 16KB - Disk Target: Default Test Directory MB/s > Higher Is Better truenas-fio-nolog-1 . 1258 |=================================================== Flexible IO Tester 3.35 Type: Random Read - IO Engine: POSIX AIO - Buffered: Yes - Direct: No - Block Size: 32KB - Disk Target: Default Test Directory MB/s > Higher Is Better truenas-fio-nolog-1 . 3690 |=================================================== Flexible IO Tester 3.35 Type: Sequential Write - IO Engine: POSIX AIO - Buffered: Yes - Direct: No - Block Size: 64KB - Disk Target: Default Test Directory MB/s > Higher Is Better truenas-fio-nolog-1 . 8037 |=================================================== Flexible IO Tester Type: Random Read - IO Engine: POSIX AIO - Buffered: Yes - Direct: No - Block Size: 32KB - Disk Target: Default Test Directory IOPS > Higher Is Better truenas-fio-nolog-1 . 118000 |================================================= Flexible IO Tester 3.35 Type: Random Read - IO Engine: POSIX AIO - Buffered: Yes - Direct: No - Block Size: 2MB - Disk Target: Default Test Directory MB/s > Higher Is Better truenas-fio-nolog-1 . 6427 |=================================================== Flexible IO Tester 3.35 Type: Random Read - IO Engine: POSIX AIO - Buffered: Yes - Direct: No - Block Size: 64KB - Disk Target: Default Test Directory MB/s > Higher Is Better truenas-fio-nolog-1 . 5347 |=================================================== Flexible IO Tester 3.35 Type: Sequential Read - IO Engine: POSIX AIO - Buffered: Yes - Direct: No - Block Size: 4MB - Disk Target: Default Test Directory MB/s > Higher Is Better truenas-fio-nolog-1 . 6103 |=================================================== Flexible IO Tester Type: Random Read - IO Engine: POSIX AIO - Buffered: Yes - Direct: No - Block Size: 64KB - Disk Target: Default Test Directory IOPS > Higher Is Better truenas-fio-nolog-1 . 85567 |================================================== Flexible IO Tester Type: Sequential Read - IO Engine: POSIX AIO - Buffered: Yes - Direct: No - Block Size: 16KB - Disk Target: Default Test Directory IOPS > Higher Is Better truenas-fio-nolog-1 . 314000 |================================================= Flexible IO Tester 3.35 Type: Random Write - IO Engine: POSIX AIO - Buffered: Yes - Direct: No - Block Size: 1MB - Disk Target: Default Test Directory MB/s > Higher Is Better truenas-fio-nolog-1 . 7797 |=================================================== Flexible IO Tester Type: Sequential Read - IO Engine: POSIX AIO - Buffered: Yes - Direct: No - Block Size: 64KB - Disk Target: Default Test Directory IOPS > Higher Is Better truenas-fio-nolog-1 . 117000 |================================================= Flexible IO Tester 3.35 Type: Random Write - IO Engine: POSIX AIO - Buffered: Yes - Direct: No - Block Size: 2MB - Disk Target: Default Test Directory MB/s > Higher Is Better truenas-fio-nolog-1 . 7740 |=================================================== Flexible IO Tester Type: Sequential Write - IO Engine: POSIX AIO - Buffered: Yes - Direct: No - Block Size: 4KB - Disk Target: Default Test Directory IOPS > Higher Is Better truenas-fio-nolog-1 . 271000 |================================================= Flexible IO Tester 3.35 Type: Random Write - IO Engine: POSIX AIO - Buffered: Yes - Direct: No - Block Size: 4KB - Disk Target: Default Test Directory MB/s > Higher Is Better truenas-fio-nolog-1 . 1100 |=================================================== Flexible IO Tester 3.35 Type: Sequential Read - IO Engine: POSIX AIO - Buffered: Yes - Direct: No - Block Size: 128KB - Disk Target: Default Test Directory MB/s > Higher Is Better truenas-fio-nolog-1 . 7083 |=================================================== Flexible IO Tester Type: Random Write - IO Engine: POSIX AIO - Buffered: Yes - Direct: No - Block Size: 4KB - Disk Target: Default Test Directory IOPS > Higher Is Better truenas-fio-nolog-1 . 281667 |================================================= Flexible IO Tester 3.35 Type: Sequential Read - IO Engine: POSIX AIO - Buffered: Yes - Direct: No - Block Size: 512KB - Disk Target: Default Test Directory MB/s > Higher Is Better truenas-fio-nolog-1 . 7056 |=================================================== Flexible IO Tester 3.35 Type: Random Write - IO Engine: POSIX AIO - Buffered: Yes - Direct: No - Block Size: 4MB - Disk Target: Default Test Directory MB/s > Higher Is Better truenas-fio-nolog-1 . 7795 |=================================================== Flexible IO Tester 3.35 Type: Sequential Write - IO Engine: POSIX AIO - Buffered: Yes - Direct: No - Block Size: 32KB - Disk Target: Default Test Directory MB/s > Higher Is Better truenas-fio-nolog-1 . 7099 |=================================================== Flexible IO Tester 3.35 Type: Random Write - IO Engine: POSIX AIO - Buffered: Yes - Direct: No - Block Size: 8KB - Disk Target: Default Test Directory MB/s > Higher Is Better truenas-fio-nolog-1 . 1213 |=================================================== Flexible IO Tester 3.35 Type: Sequential Write - IO Engine: POSIX AIO - Buffered: Yes - Direct: No - Block Size: 128KB - Disk Target: Default Test Directory MB/s > Higher Is Better truenas-fio-nolog-1 . 7936 |=================================================== Flexible IO Tester Type: Random Write - IO Engine: POSIX AIO - Buffered: Yes - Direct: No - Block Size: 8KB - Disk Target: Default Test Directory IOPS > Higher Is Better truenas-fio-nolog-1 . 155000 |================================================= Flexible IO Tester 3.35 Type: Sequential Write - IO Engine: POSIX AIO - Buffered: Yes - Direct: No - Block Size: 512KB - Disk Target: Default Test Directory MB/s > Higher Is Better truenas-fio-nolog-1 . 7922 |=================================================== Flexible IO Tester 3.35 Type: Random Read - IO Engine: POSIX AIO - Buffered: Yes - Direct: No - Block Size: 128KB - Disk Target: Default Test Directory MB/s > Higher Is Better truenas-fio-nolog-1 . 6989 |=================================================== Flexible IO Tester 3.35 Type: Sequential Read - IO Engine: POSIX AIO - Buffered: Yes - Direct: No - Block Size: 2MB - Disk Target: Default Test Directory MB/s > Higher Is Better truenas-fio-nolog-1 . 6138 |=================================================== Flexible IO Tester Type: Random Read - IO Engine: POSIX AIO - Buffered: Yes - Direct: No - Block Size: 128KB - Disk Target: Default Test Directory IOPS > Higher Is Better truenas-fio-nolog-1 . 55900 |================================================== Flexible IO Tester Type: Sequential Read - IO Engine: POSIX AIO - Buffered: Yes - Direct: No - Block Size: 4KB - Disk Target: Default Test Directory IOPS > Higher Is Better truenas-fio-nolog-1 . 362667 |================================================= Flexible IO Tester 3.35 Type: Random Read - IO Engine: POSIX AIO - Buffered: Yes - Direct: No - Block Size: 256KB - Disk Target: Default Test Directory MB/s > Higher Is Better truenas-fio-nolog-1 . 6975 |=================================================== Flexible IO Tester 3.35 Type: Sequential Read - IO Engine: POSIX AIO - Buffered: Yes - Direct: No - Block Size: 8KB - Disk Target: Default Test Directory MB/s > Higher Is Better truenas-fio-nolog-1 . 2600 |=================================================== Flexible IO Tester Type: Random Read - IO Engine: POSIX AIO - Buffered: Yes - Direct: No - Block Size: 256KB - Disk Target: Default Test Directory IOPS > Higher Is Better truenas-fio-nolog-1 . 27900 |================================================== Flexible IO Tester 3.35 Type: Sequential Read - IO Engine: POSIX AIO - Buffered: Yes - Direct: No - Block Size: 16KB - Disk Target: Default Test Directory MB/s > Higher Is Better truenas-fio-nolog-1 . 4906 |=================================================== Flexible IO Tester 3.35 Type: Random Read - IO Engine: POSIX AIO - Buffered: Yes - Direct: No - Block Size: 512KB - Disk Target: Default Test Directory MB/s > Higher Is Better truenas-fio-nolog-1 . 6731 |=================================================== Flexible IO Tester 3.35 Type: Sequential Read - IO Engine: POSIX AIO - Buffered: Yes - Direct: No - Block Size: 32KB - Disk Target: Default Test Directory MB/s > Higher Is Better truenas-fio-nolog-1 . 7426 |=================================================== Flexible IO Tester Type: Random Read - IO Engine: POSIX AIO - Buffered: Yes - Direct: No - Block Size: 512KB - Disk Target: Default Test Directory IOPS > Higher Is Better truenas-fio-nolog-1 . 13500 |================================================== Flexible IO Tester 3.35 Type: Sequential Read - IO Engine: POSIX AIO - Buffered: Yes - Direct: No - Block Size: 64KB - Disk Target: Default Test Directory MB/s > Higher Is Better truenas-fio-nolog-1 . 7328 |=================================================== Flexible IO Tester 3.35 Type: Random Write - IO Engine: POSIX AIO - Buffered: Yes - Direct: No - Block Size: 16KB - Disk Target: Default Test Directory MB/s > Higher Is Better truenas-fio-nolog-1 . 1260 |=================================================== Flexible IO Tester 3.35 Type: Sequential Write - IO Engine: POSIX AIO - Buffered: Yes - Direct: No - Block Size: 1MB - Disk Target: Default Test Directory MB/s > Higher Is Better truenas-fio-nolog-1 . 7843 |=================================================== Flexible IO Tester Type: Random Write - IO Engine: POSIX AIO - Buffered: Yes - Direct: No - Block Size: 16KB - Disk Target: Default Test Directory IOPS > Higher Is Better truenas-fio-nolog-1 . 80667 |================================================== Flexible IO Tester 3.35 Type: Sequential Write - IO Engine: POSIX AIO - Buffered: Yes - Direct: No - Block Size: 4KB - Disk Target: Default Test Directory MB/s > Higher Is Better truenas-fio-nolog-1 . 1059 |=================================================== Flexible IO Tester 3.35 Type: Random Write - IO Engine: POSIX AIO - Buffered: Yes - Direct: No - Block Size: 32KB - Disk Target: Default Test Directory MB/s > Higher Is Better truenas-fio-nolog-1 . 6824 |=================================================== Flexible IO Tester 3.35 Type: Sequential Write - IO Engine: POSIX AIO - Buffered: Yes - Direct: No - Block Size: 4MB - Disk Target: Default Test Directory MB/s > Higher Is Better truenas-fio-nolog-1 . 7830 |=================================================== Flexible IO Tester Type: Random Write - IO Engine: POSIX AIO - Buffered: Yes - Direct: No - Block Size: 32KB - Disk Target: Default Test Directory IOPS > Higher Is Better truenas-fio-nolog-1 . 218333 |================================================= Flexible IO Tester Type: Sequential Write - IO Engine: POSIX AIO - Buffered: Yes - Direct: No - Block Size: 8KB - Disk Target: Default Test Directory IOPS > Higher Is Better truenas-fio-nolog-1 . 158333 |================================================= Flexible IO Tester 3.35 Type: Random Write - IO Engine: POSIX AIO - Buffered: Yes - Direct: No - Block Size: 64KB - Disk Target: Default Test Directory MB/s > Higher Is Better truenas-fio-nolog-1 . 8095 |=================================================== Flexible IO Tester Type: Sequential Read - IO Engine: POSIX AIO - Buffered: Yes - Direct: No - Block Size: 128KB - Disk Target: Default Test Directory IOPS > Higher Is Better truenas-fio-nolog-1 . 56633 |================================================== Flexible IO Tester Type: Random Write - IO Engine: POSIX AIO - Buffered: Yes - Direct: No - Block Size: 64KB - Disk Target: Default Test Directory IOPS > Higher Is Better truenas-fio-nolog-1 . 129667 |================================================= Flexible IO Tester Type: Sequential Read - IO Engine: POSIX AIO - Buffered: Yes - Direct: No - Block Size: 256KB - Disk Target: Default Test Directory IOPS > Higher Is Better truenas-fio-nolog-1 . 28867 |================================================== Flexible IO Tester 3.35 Type: Random Write - IO Engine: POSIX AIO - Buffered: Yes - Direct: No - Block Size: 128KB - Disk Target: Default Test Directory MB/s > Higher Is Better truenas-fio-nolog-1 . 7984 |=================================================== Flexible IO Tester Type: Sequential Read - IO Engine: POSIX AIO - Buffered: Yes - Direct: No - Block Size: 512KB - Disk Target: Default Test Directory IOPS > Higher Is Better truenas-fio-nolog-1 . 14100 |================================================== Flexible IO Tester Type: Random Write - IO Engine: POSIX AIO - Buffered: Yes - Direct: No - Block Size: 128KB - Disk Target: Default Test Directory IOPS > Higher Is Better truenas-fio-nolog-1 . 63867 |================================================== Flexible IO Tester Type: Sequential Write - IO Engine: POSIX AIO - Buffered: Yes - Direct: No - Block Size: 16KB - Disk Target: Default Test Directory IOPS > Higher Is Better truenas-fio-nolog-1 . 80533 |================================================== Flexible IO Tester 3.35 Type: Random Write - IO Engine: POSIX AIO - Buffered: Yes - Direct: No - Block Size: 256KB - Disk Target: Default Test Directory MB/s > Higher Is Better truenas-fio-nolog-1 . 8024 |=================================================== Flexible IO Tester Type: Sequential Write - IO Engine: POSIX AIO - Buffered: Yes - Direct: No - Block Size: 32KB - Disk Target: Default Test Directory IOPS > Higher Is Better truenas-fio-nolog-1 . 227000 |================================================= Flexible IO Tester Type: Random Write - IO Engine: POSIX AIO - Buffered: Yes - Direct: No - Block Size: 256KB - Disk Target: Default Test Directory IOPS > Higher Is Better truenas-fio-nolog-1 . 32100 |================================================== Flexible IO Tester Type: Sequential Write - IO Engine: POSIX AIO - Buffered: Yes - Direct: No - Block Size: 64KB - Disk Target: Default Test Directory IOPS > Higher Is Better truenas-fio-nolog-1 . 128667 |================================================= Flexible IO Tester 3.35 Type: Random Write - IO Engine: POSIX AIO - Buffered: Yes - Direct: No - Block Size: 512KB - Disk Target: Default Test Directory MB/s > Higher Is Better truenas-fio-nolog-1 . 7963 |=================================================== Flexible IO Tester Type: Sequential Write - IO Engine: POSIX AIO - Buffered: Yes - Direct: No - Block Size: 128KB - Disk Target: Default Test Directory IOPS > Higher Is Better truenas-fio-nolog-1 . 63467 |================================================== Flexible IO Tester Type: Random Write - IO Engine: POSIX AIO - Buffered: Yes - Direct: No - Block Size: 512KB - Disk Target: Default Test Directory IOPS > Higher Is Better truenas-fio-nolog-1 . 15933 |================================================== Flexible IO Tester Type: Sequential Write - IO Engine: POSIX AIO - Buffered: Yes - Direct: No - Block Size: 256KB - Disk Target: Default Test Directory IOPS > Higher Is Better truenas-fio-nolog-1 . 31933 |================================================== Flexible IO Tester 3.35 Type: Sequential Read - IO Engine: POSIX AIO - Buffered: Yes - Direct: No - Block Size: 1MB - Disk Target: Default Test Directory MB/s > Higher Is Better truenas-fio-nolog-1 . 6162 |=================================================== Flexible IO Tester Type: Sequential Write - IO Engine: POSIX AIO - Buffered: Yes - Direct: No - Block Size: 512KB - Disk Target: Default Test Directory IOPS > Higher Is Better truenas-fio-nolog-1 . 15867 |================================================== Flexible IO Tester 3.35 Type: Random Read - IO Engine: POSIX AIO - Buffered: Yes - Direct: No - Block Size: 1MB - Disk Target: Default Test Directory MB/s > Higher Is Better truenas-fio-nolog-1 . 6444 |===================================================