Debian-XCP-ng-TrueNAS

Xen HVM domU 4.13 testing on Debian 11 via the Phoronix Test Suite.

Compare your own system(s) to this result file with the Phoronix Test Suite by running the command: phoronix-test-suite benchmark 2108249-IB-DEBIANXCP11
Jump To Table - Results

View

Do Not Show Noisy Results
Do Not Show Results With Incomplete Data
Do Not Show Results With Little Change/Spread
List Notable Results
Show Result Confidence Charts
Allow Limiting Results To Certain Suite(s)

Statistics

Show Overall Harmonic Mean(s)
Show Overall Geometric Mean
Show Wins / Losses Counts (Pie Chart)
Normalize Results
Remove Outliers Before Calculating Averages

Graph Settings

Force Line Graphs Where Applicable
Convert To Scalar Where Applicable
Prefer Vertical Bar Graphs

Multi-Way Comparison

Condense Multi-Option Tests Into Single Result Graphs

Table

Show Detailed System Result Table

Run Management

Highlight
Result
Toggle/Hide
Result
Result
Identifier
View Logs
Performance Per
Dollar
Date
Run
  Test
  Duration
NFS on TrueNAS
August 24 2021
  1 Hour, 31 Minutes
iSCSI on TrueNAS
August 24 2021
  1 Hour, 25 Minutes
Invert Behavior (Only Show Selected Data)
  1 Hour, 28 Minutes
Only show results matching title/arguments (delimit multiple options with a comma):
Do not show results matching title/arguments (delimit multiple options with a comma):

NFS on TrueNAS

Kernel Notes: Transparent Huge Pages: always
Compiler Notes: --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-link-mutex --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-targets=nvptx-none=/build/gcc-10-Km9U7s/gcc-10-10.2.1/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-10-Km9U7s/gcc-10-10.2.1/debian/tmp-gcn/usr,hsa --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-build-config=bootstrap-lto-lean --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v
Disk Notes: MQ-DEADLINE / errors=remount-ro,relatime,rw / Block Size: 4096
Processor Notes: CPU Microcode: 0x28
Security Notes: itlb_multihit: KVM: Mitigation of VMX unsupported + l1tf: Mitigation of PTE Inversion + mds: Mitigation of Clear buffers; SMT Host state unknown + meltdown: Mitigation of PTI + spec_store_bypass: Mitigation of SSB disabled via prctl and seccomp + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Full generic retpoline IBPB: conditional IBRS_FW STIBP: disabled RSB filling + srbds: Unknown: Dependent on hypervisor status + tsx_async_abort: Not affected

iSCSI on TrueNAS

Processor: 8 x Intel Core i7-4790 (15 Cores), Motherboard: Xen HVM domU v4.13, Chipset: Intel 440FX 82441FX PMC, Memory: 1 x 4088 MB RAM, Disk: 57GB, Graphics: bochs-drmdrmfb

OS: Debian 11, Kernel: 5.10.0-8-amd64 (x86_64), Compiler: GCC 10.2.1 20210110, File-System: ext4, Screen Resolution: 1024x768, System Layer: Xen HVM domU 4.13

Debian-XCP-ng-TrueNASOpenBenchmarking.orgPhoronix Test Suite8 x Intel Core i7-4790 (15 Cores)Xen HVM domU v4.13Intel 440FX 82441FX PMC1 x 4088 MB RAM57GBbochs-drmdrmfbDebian 115.10.0-8-amd64 (x86_64)GCC 10.2.1 20210110ext41024x768Xen HVM domU 4.13ProcessorMotherboardChipsetMemoryDiskGraphicsOSKernelCompilerFile-SystemScreen ResolutionSystem LayerDebian-XCP-ng-TrueNAS BenchmarksSystem Logs- Transparent Huge Pages: always- --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-link-mutex --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-targets=nvptx-none=/build/gcc-10-Km9U7s/gcc-10-10.2.1/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-10-Km9U7s/gcc-10-10.2.1/debian/tmp-gcn/usr,hsa --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-build-config=bootstrap-lto-lean --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v - MQ-DEADLINE / errors=remount-ro,relatime,rw / Block Size: 4096 - CPU Microcode: 0x28- itlb_multihit: KVM: Mitigation of VMX unsupported + l1tf: Mitigation of PTE Inversion + mds: Mitigation of Clear buffers; SMT Host state unknown + meltdown: Mitigation of PTI + spec_store_bypass: Mitigation of SSB disabled via prctl and seccomp + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Full generic retpoline IBPB: conditional IBRS_FW STIBP: disabled RSB filling + srbds: Unknown: Dependent on hypervisor status + tsx_async_abort: Not affected

NFS on TrueNAS vs. iSCSI on TrueNAS ComparisonPhoronix Test SuiteBaseline+22.2%+22.2%+44.4%+44.4%+66.6%+66.6%+88.8%+88.8%88.9%88.6%46.3%46.2%44.8%44.1%40.3%40.3%34%34%27.4%24.4%22%18.9%17.1%17.1%13.3%13.2%11.4%11.4%9.8%9.7%7.9%7.7%7.7%7.6%7.4%2.7%Rand Write - Linux AIO - No - No - 4KBRand Write - Linux AIO - No - No - 4KBRand Write - Linux AIO - No - No - 16KBRand Write - Linux AIO - No - No - 16KBRand Read - Linux AIO - No - No - 4KBRand Read - Linux AIO - No - No - 4KBRand Read - Linux AIO - No - No - 16KBRand Read - Linux AIO - No - No - 16KBSeq Read - Linux AIO - No - No - 4KBSeq Read - Linux AIO - No - No - 4KB64Seq Write - Linux AIO - No - No - 64KB25.3%Seq Write - Linux AIO - No - No - 64KB25.1%12832Seq Write - Linux AIO - No - No - 1MB21.9%Seq Write - Linux AIO - No - No - 1MB21.8%Seq Write - Linux AIO - No - No - 256KB20.6%Seq Write - Linux AIO - No - No - 256KB20.4%8Seq Read - Linux AIO - No - No - 16KBSeq Read - Linux AIO - No - No - 16KBSeq Write - Linux AIO - No - No - 16KB14.9%Seq Write - Linux AIO - No - No - 16KB14.8%Seq Write - Linux AIO - No - No - 4KBSeq Write - Linux AIO - No - No - 4KBRand Write - Linux AIO - No - No - 1MB12.3%Rand Write - Linux AIO - No - No - 1MB12.2%Seq Read - Linux AIO - No - No - 1MBSeq Read - Linux AIO - No - No - 1MBRand Write - Linux AIO - No - No - 64KBRand Write - Linux AIO - No - No - 64KB1Seq Read - Linux AIO - No - No - 64KBSeq Read - Linux AIO - No - No - 64KBSeq Read - Linux AIO - No - No - 256KBRand Read - Linux AIO - No - No - 256KB7.5%Rand Read - Linux AIO - No - No - 256KB7.5%Seq Read - Linux AIO - No - No - 256KBRand Write - Linux AIO - No - No - 256KB3.6%Rand Write - Linux AIO - No - No - 256KB3.5%Rand Read - Linux AIO - No - No - 1MB3.3%Rand Read - Linux AIO - No - No - 1MB3.2%Rand Read - Linux AIO - No - No - 64KBFlexible IO TesterFlexible IO TesterFlexible IO TesterFlexible IO TesterFlexible IO TesterFlexible IO TesterFlexible IO TesterFlexible IO TesterFlexible IO TesterFlexible IO TesterSQLiteFlexible IO TesterFlexible IO TesterSQLiteSQLiteFlexible IO TesterFlexible IO TesterFlexible IO TesterFlexible IO TesterSQLiteFlexible IO TesterFlexible IO TesterFlexible IO TesterFlexible IO TesterFlexible IO TesterFlexible IO TesterFlexible IO TesterFlexible IO TesterFlexible IO TesterFlexible IO TesterFlexible IO TesterFlexible IO TesterSQLiteFlexible IO TesterFlexible IO TesterFlexible IO TesterFlexible IO TesterFlexible IO TesterFlexible IO TesterFlexible IO TesterFlexible IO TesterFlexible IO TesterFlexible IO TesterFlexible IO TesterNFS on TrueNASiSCSI on TrueNAS

Debian-XCP-ng-TrueNASfio: Rand Write - Linux AIO - No - No - 4KB - Default Test Directoryfio: Rand Write - Linux AIO - No - No - 4KB - Default Test Directoryfio: Rand Write - Linux AIO - No - No - 16KB - Default Test Directoryfio: Rand Write - Linux AIO - No - No - 16KB - Default Test Directoryfio: Rand Read - Linux AIO - No - No - 4KB - Default Test Directoryfio: Rand Read - Linux AIO - No - No - 4KB - Default Test Directoryfio: Rand Read - Linux AIO - No - No - 16KB - Default Test Directoryfio: Rand Read - Linux AIO - No - No - 16KB - Default Test Directoryfio: Seq Read - Linux AIO - No - No - 4KB - Default Test Directoryfio: Seq Read - Linux AIO - No - No - 4KB - Default Test Directorysqlite: 64fio: Seq Write - Linux AIO - No - No - 64KB - Default Test Directoryfio: Seq Write - Linux AIO - No - No - 64KB - Default Test Directorysqlite: 128sqlite: 32fio: Seq Write - Linux AIO - No - No - 1MB - Default Test Directoryfio: Seq Write - Linux AIO - No - No - 1MB - Default Test Directoryfio: Seq Write - Linux AIO - No - No - 256KB - Default Test Directoryfio: Seq Write - Linux AIO - No - No - 256KB - Default Test Directorysqlite: 8fio: Seq Read - Linux AIO - No - No - 16KB - Default Test Directoryfio: Seq Read - Linux AIO - No - No - 16KB - Default Test Directoryfio: Seq Write - Linux AIO - No - No - 16KB - Default Test Directoryfio: Seq Write - Linux AIO - No - No - 16KB - Default Test Directoryfio: Seq Write - Linux AIO - No - No - 4KB - Default Test Directoryfio: Seq Write - Linux AIO - No - No - 4KB - Default Test Directoryfio: Rand Write - Linux AIO - No - No - 1MB - Default Test Directoryfio: Rand Write - Linux AIO - No - No - 1MB - Default Test Directoryfio: Seq Read - Linux AIO - No - No - 1MB - Default Test Directoryfio: Seq Read - Linux AIO - No - No - 1MB - Default Test Directoryfio: Rand Write - Linux AIO - No - No - 64KB - Default Test Directoryfio: Rand Write - Linux AIO - No - No - 64KB - Default Test Directorysqlite: 1fio: Seq Read - Linux AIO - No - No - 64KB - Default Test Directoryfio: Seq Read - Linux AIO - No - No - 64KB - Default Test Directoryfio: Seq Read - Linux AIO - No - No - 256KB - Default Test Directoryfio: Rand Read - Linux AIO - No - No - 256KB - Default Test Directoryfio: Rand Read - Linux AIO - No - No - 256KB - Default Test Directoryfio: Seq Read - Linux AIO - No - No - 256KB - Default Test Directoryfio: Rand Write - Linux AIO - No - No - 256KB - Default Test Directoryfio: Rand Write - Linux AIO - No - No - 256KB - Default Test Directoryfio: Rand Read - Linux AIO - No - No - 1MB - Default Test Directoryfio: Rand Read - Linux AIO - No - No - 1MB - Default Test Directoryfio: Rand Read - Linux AIO - No - No - 64KB - Default Test Directoryfio: Rand Read - Linux AIO - No - No - 64KB - Default Test DirectoryNFS on TrueNASiSCSI on TrueNAS68.31750022914633174445336053870067767265130.1458683543239.13185.076563559221755540.785528008255463490024963400583587911915744846610.37594615133381842661067956582232310621065933149001293300033521400252641678495430090833355102.1526931434192.27269.734462459183846134.3056183396647530400282718005195231015101981775119.61910191630041073969993102756222451028103295815000OpenBenchmarking.org

Flexible IO Tester

FIO, the Flexible I/O Tester, is an advanced Linux disk benchmark supporting multiple I/O engines and a wealth of options. FIO was written by Jens Axboe for testing of the Linux I/O subsystem and schedulers. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.25Type: Random Write - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 4KB - Disk Target: Default Test DirectoryNFS on TrueNASiSCSI on TrueNAS306090120150SE +/- 0.10, N = 368.3129.01. (CC) gcc options: -rdynamic -ll -lrt -lz -lpthread -lm -ldl -laio -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.25Type: Random Write - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 4KB - Disk Target: Default Test DirectoryNFS on TrueNASiSCSI on TrueNAS7K14K21K28K35KSE +/- 57.74, N = 317500330001. (CC) gcc options: -rdynamic -ll -lrt -lz -lpthread -lm -ldl -laio -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.25Type: Random Write - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 16KB - Disk Target: Default Test DirectoryNFS on TrueNASiSCSI on TrueNAS70140210280350SE +/- 1.20, N = 3SE +/- 0.88, N = 32293351. (CC) gcc options: -rdynamic -ll -lrt -lz -lpthread -lm -ldl -laio -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.25Type: Random Write - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 16KB - Disk Target: Default Test DirectoryNFS on TrueNASiSCSI on TrueNAS5K10K15K20K25KSE +/- 33.33, N = 3SE +/- 57.74, N = 314633214001. (CC) gcc options: -rdynamic -ll -lrt -lz -lpthread -lm -ldl -laio -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.25Type: Random Read - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 4KB - Disk Target: Default Test DirectoryNFS on TrueNASiSCSI on TrueNAS601201802403001742521. (CC) gcc options: -rdynamic -ll -lrt -lz -lpthread -lm -ldl -laio -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.25Type: Random Read - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 4KB - Disk Target: Default Test DirectoryNFS on TrueNASiSCSI on TrueNAS14K28K42K56K70KSE +/- 66.67, N = 3SE +/- 617.34, N = 344533641671. (CC) gcc options: -rdynamic -ll -lrt -lz -lpthread -lm -ldl -laio -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.25Type: Random Read - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 16KB - Disk Target: Default Test DirectoryNFS on TrueNASiSCSI on TrueNAS2004006008001000SE +/- 4.04, N = 3SE +/- 3.00, N = 36058491. (CC) gcc options: -rdynamic -ll -lrt -lz -lpthread -lm -ldl -laio -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.25Type: Random Read - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 16KB - Disk Target: Default Test DirectoryNFS on TrueNASiSCSI on TrueNAS12K24K36K48K60KSE +/- 251.66, N = 3SE +/- 200.00, N = 338700543001. (CC) gcc options: -rdynamic -ll -lrt -lz -lpthread -lm -ldl -laio -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.25Type: Sequential Read - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 4KB - Disk Target: Default Test DirectoryNFS on TrueNASiSCSI on TrueNAS20K40K60K80K100KSE +/- 317.98, N = 3SE +/- 533.33, N = 367767908331. (CC) gcc options: -rdynamic -ll -lrt -lz -lpthread -lm -ldl -laio -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.25Type: Sequential Read - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 4KB - Disk Target: Default Test DirectoryNFS on TrueNASiSCSI on TrueNAS80160240320400SE +/- 1.45, N = 3SE +/- 2.00, N = 32653551. (CC) gcc options: -rdynamic -ll -lrt -lz -lpthread -lm -ldl -laio -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native

SQLite

This is a simple benchmark of SQLite. At present this test profile just measures the time to perform a pre-defined number of insertions on an indexed database. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgSeconds, Fewer Is BetterSQLite 3.30.1Threads / Copies: 64NFS on TrueNASiSCSI on TrueNAS306090120150SE +/- 0.22, N = 3SE +/- 0.23, N = 3130.15102.151. (CC) gcc options: -O2 -lz -lm -ldl -lpthread

Flexible IO Tester

FIO, the Flexible I/O Tester, is an advanced Linux disk benchmark supporting multiple I/O engines and a wealth of options. FIO was written by Jens Axboe for testing of the Linux I/O subsystem and schedulers. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.25Type: Sequential Write - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 64KB - Disk Target: Default Test DirectoryNFS on TrueNASiSCSI on TrueNAS2K4K6K8K10KSE +/- 19.46, N = 3SE +/- 3.38, N = 3868369311. (CC) gcc options: -rdynamic -ll -lrt -lz -lpthread -lm -ldl -laio -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.25Type: Sequential Write - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 64KB - Disk Target: Default Test DirectoryNFS on TrueNASiSCSI on TrueNAS120240360480600SE +/- 1.20, N = 35434341. (CC) gcc options: -rdynamic -ll -lrt -lz -lpthread -lm -ldl -laio -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native

SQLite

This is a simple benchmark of SQLite. At present this test profile just measures the time to perform a pre-defined number of insertions on an indexed database. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgSeconds, Fewer Is BetterSQLite 3.30.1Threads / Copies: 128NFS on TrueNASiSCSI on TrueNAS50100150200250SE +/- 0.26, N = 3SE +/- 0.33, N = 3239.13192.271. (CC) gcc options: -O2 -lz -lm -ldl -lpthread

OpenBenchmarking.orgSeconds, Fewer Is BetterSQLite 3.30.1Threads / Copies: 32NFS on TrueNASiSCSI on TrueNAS20406080100SE +/- 0.20, N = 3SE +/- 0.28, N = 385.0869.731. (CC) gcc options: -O2 -lz -lm -ldl -lpthread

Flexible IO Tester

FIO, the Flexible I/O Tester, is an advanced Linux disk benchmark supporting multiple I/O engines and a wealth of options. FIO was written by Jens Axboe for testing of the Linux I/O subsystem and schedulers. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.25Type: Sequential Write - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 1MB - Disk Target: Default Test DirectoryNFS on TrueNASiSCSI on TrueNAS120240360480600SE +/- 2.00, N = 3SE +/- 1.53, N = 35634621. (CC) gcc options: -rdynamic -ll -lrt -lz -lpthread -lm -ldl -laio -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.25Type: Sequential Write - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 1MB - Disk Target: Default Test DirectoryNFS on TrueNASiSCSI on TrueNAS120240360480600SE +/- 2.19, N = 3SE +/- 1.33, N = 35594591. (CC) gcc options: -rdynamic -ll -lrt -lz -lpthread -lm -ldl -laio -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.25Type: Sequential Write - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 256KB - Disk Target: Default Test DirectoryNFS on TrueNASiSCSI on TrueNAS5001000150020002500SE +/- 30.03, N = 4SE +/- 12.77, N = 3221718381. (CC) gcc options: -rdynamic -ll -lrt -lz -lpthread -lm -ldl -laio -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.25Type: Sequential Write - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 256KB - Disk Target: Default Test DirectoryNFS on TrueNASiSCSI on TrueNAS120240360480600SE +/- 7.44, N = 4SE +/- 3.18, N = 35554611. (CC) gcc options: -rdynamic -ll -lrt -lz -lpthread -lm -ldl -laio -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native

SQLite

This is a simple benchmark of SQLite. At present this test profile just measures the time to perform a pre-defined number of insertions on an indexed database. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgSeconds, Fewer Is BetterSQLite 3.30.1Threads / Copies: 8NFS on TrueNASiSCSI on TrueNAS918273645SE +/- 0.16, N = 3SE +/- 0.40, N = 340.7934.311. (CC) gcc options: -O2 -lz -lm -ldl -lpthread

Flexible IO Tester

FIO, the Flexible I/O Tester, is an advanced Linux disk benchmark supporting multiple I/O engines and a wealth of options. FIO was written by Jens Axboe for testing of the Linux I/O subsystem and schedulers. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.25Type: Sequential Read - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 16KB - Disk Target: Default Test DirectoryNFS on TrueNASiSCSI on TrueNAS13K26K39K52K65KSE +/- 57.74, N = 3SE +/- 669.16, N = 352800618331. (CC) gcc options: -rdynamic -ll -lrt -lz -lpthread -lm -ldl -laio -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.25Type: Sequential Read - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 16KB - Disk Target: Default Test DirectoryNFS on TrueNASiSCSI on TrueNAS2004006008001000SE +/- 0.88, N = 3SE +/- 10.73, N = 38259661. (CC) gcc options: -rdynamic -ll -lrt -lz -lpthread -lm -ldl -laio -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.25Type: Sequential Write - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 16KB - Disk Target: Default Test DirectoryNFS on TrueNASiSCSI on TrueNAS120240360480600SE +/- 3.18, N = 3SE +/- 2.19, N = 35464751. (CC) gcc options: -rdynamic -ll -lrt -lz -lpthread -lm -ldl -laio -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.25Type: Sequential Write - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 16KB - Disk Target: Default Test DirectoryNFS on TrueNASiSCSI on TrueNAS7K14K21K28K35KSE +/- 200.00, N = 3SE +/- 152.75, N = 334900304001. (CC) gcc options: -rdynamic -ll -lrt -lz -lpthread -lm -ldl -laio -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.25Type: Sequential Write - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 4KB - Disk Target: Default Test DirectoryNFS on TrueNASiSCSI on TrueNAS60120180240300SE +/- 0.58, N = 3SE +/- 0.67, N = 32492821. (CC) gcc options: -rdynamic -ll -lrt -lz -lpthread -lm -ldl -laio -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.25Type: Sequential Write - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 4KB - Disk Target: Default Test DirectoryNFS on TrueNASiSCSI on TrueNAS15K30K45K60K75KSE +/- 230.94, N = 3SE +/- 416.33, N = 363400718001. (CC) gcc options: -rdynamic -ll -lrt -lz -lpthread -lm -ldl -laio -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.25Type: Random Write - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 1MB - Disk Target: Default Test DirectoryNFS on TrueNASiSCSI on TrueNAS130260390520650SE +/- 2.52, N = 3SE +/- 2.52, N = 35835191. (CC) gcc options: -rdynamic -ll -lrt -lz -lpthread -lm -ldl -laio -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.25Type: Random Write - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 1MB - Disk Target: Default Test DirectoryNFS on TrueNASiSCSI on TrueNAS130260390520650SE +/- 2.73, N = 3SE +/- 2.73, N = 35875231. (CC) gcc options: -rdynamic -ll -lrt -lz -lpthread -lm -ldl -laio -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.25Type: Sequential Read - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 1MB - Disk Target: Default Test DirectoryNFS on TrueNASiSCSI on TrueNAS2004006008001000SE +/- 6.77, N = 3SE +/- 10.27, N = 391110151. (CC) gcc options: -rdynamic -ll -lrt -lz -lpthread -lm -ldl -laio -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.25Type: Sequential Read - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 1MB - Disk Target: Default Test DirectoryNFS on TrueNASiSCSI on TrueNAS2004006008001000SE +/- 6.77, N = 3SE +/- 10.27, N = 391510191. (CC) gcc options: -rdynamic -ll -lrt -lz -lpthread -lm -ldl -laio -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.25Type: Random Write - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 64KB - Disk Target: Default Test DirectoryNFS on TrueNASiSCSI on TrueNAS2K4K6K8K10KSE +/- 56.51, N = 3SE +/- 25.54, N = 3744881771. (CC) gcc options: -rdynamic -ll -lrt -lz -lpthread -lm -ldl -laio -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.25Type: Random Write - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 64KB - Disk Target: Default Test DirectoryNFS on TrueNASiSCSI on TrueNAS110220330440550SE +/- 3.33, N = 3SE +/- 1.53, N = 34665111. (CC) gcc options: -rdynamic -ll -lrt -lz -lpthread -lm -ldl -laio -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native

SQLite

This is a simple benchmark of SQLite. At present this test profile just measures the time to perform a pre-defined number of insertions on an indexed database. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgSeconds, Fewer Is BetterSQLite 3.30.1Threads / Copies: 1NFS on TrueNASiSCSI on TrueNAS3691215SE +/- 0.140, N = 4SE +/- 0.127, N = 510.3759.6191. (CC) gcc options: -O2 -lz -lm -ldl -lpthread

Flexible IO Tester

FIO, the Flexible I/O Tester, is an advanced Linux disk benchmark supporting multiple I/O engines and a wealth of options. FIO was written by Jens Axboe for testing of the Linux I/O subsystem and schedulers. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.25Type: Sequential Read - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 64KB - Disk Target: Default Test DirectoryNFS on TrueNASiSCSI on TrueNAS2004006008001000SE +/- 2.52, N = 3SE +/- 4.06, N = 394610191. (CC) gcc options: -rdynamic -ll -lrt -lz -lpthread -lm -ldl -laio -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.25Type: Sequential Read - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 64KB - Disk Target: Default Test DirectoryNFS on TrueNASiSCSI on TrueNAS3K6K9K12K15KSE +/- 66.67, N = 3SE +/- 57.74, N = 315133163001. (CC) gcc options: -rdynamic -ll -lrt -lz -lpthread -lm -ldl -laio -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.25Type: Sequential Read - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 256KB - Disk Target: Default Test DirectoryNFS on TrueNASiSCSI on TrueNAS9001800270036004500SE +/- 39.67, N = 3SE +/- 45.96, N = 3381841071. (CC) gcc options: -rdynamic -ll -lrt -lz -lpthread -lm -ldl -laio -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.25Type: Random Read - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 256KB - Disk Target: Default Test DirectoryNFS on TrueNASiSCSI on TrueNAS9001800270036004500SE +/- 6.24, N = 3SE +/- 13.00, N = 3426639691. (CC) gcc options: -rdynamic -ll -lrt -lz -lpthread -lm -ldl -laio -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.25Type: Random Read - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 256KB - Disk Target: Default Test DirectoryNFS on TrueNASiSCSI on TrueNAS2004006008001000SE +/- 1.45, N = 3SE +/- 3.18, N = 310679931. (CC) gcc options: -rdynamic -ll -lrt -lz -lpthread -lm -ldl -laio -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.25Type: Sequential Read - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 256KB - Disk Target: Default Test DirectoryNFS on TrueNASiSCSI on TrueNAS2004006008001000SE +/- 9.84, N = 3SE +/- 11.39, N = 395610271. (CC) gcc options: -rdynamic -ll -lrt -lz -lpthread -lm -ldl -laio -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.25Type: Random Write - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 256KB - Disk Target: Default Test DirectoryNFS on TrueNASiSCSI on TrueNAS130260390520650SE +/- 4.18, N = 3SE +/- 4.10, N = 35825621. (CC) gcc options: -rdynamic -ll -lrt -lz -lpthread -lm -ldl -laio -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.25Type: Random Write - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 256KB - Disk Target: Default Test DirectoryNFS on TrueNASiSCSI on TrueNAS5001000150020002500SE +/- 16.01, N = 3SE +/- 16.29, N = 3232322451. (CC) gcc options: -rdynamic -ll -lrt -lz -lpthread -lm -ldl -laio -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.25Type: Random Read - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 1MB - Disk Target: Default Test DirectoryNFS on TrueNASiSCSI on TrueNAS2004006008001000SE +/- 0.88, N = 3106210281. (CC) gcc options: -rdynamic -ll -lrt -lz -lpthread -lm -ldl -laio -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.25Type: Random Read - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 1MB - Disk Target: Default Test DirectoryNFS on TrueNASiSCSI on TrueNAS2004006008001000SE +/- 1.20, N = 3SE +/- 10.97, N = 3106510321. (CC) gcc options: -rdynamic -ll -lrt -lz -lpthread -lm -ldl -laio -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.25Type: Random Read - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 64KB - Disk Target: Default Test DirectoryNFS on TrueNASiSCSI on TrueNAS2004006008001000SE +/- 2.91, N = 3SE +/- 11.24, N = 39339581. (CC) gcc options: -rdynamic -ll -lrt -lz -lpthread -lm -ldl -laio -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.25Type: Random Read - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 64KB - Disk Target: Default Test DirectoryNFS on TrueNASiSCSI on TrueNAS3K6K9K12K15KSE +/- 503.32, N = 314900150001. (CC) gcc options: -rdynamic -ll -lrt -lz -lpthread -lm -ldl -laio -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native

45 Results Shown

Flexible IO Tester:
  Rand Write - Linux AIO - No - No - 4KB - Default Test Directory:
    MB/s
    IOPS
  Rand Write - Linux AIO - No - No - 16KB - Default Test Directory:
    MB/s
    IOPS
  Rand Read - Linux AIO - No - No - 4KB - Default Test Directory:
    MB/s
    IOPS
  Rand Read - Linux AIO - No - No - 16KB - Default Test Directory:
    MB/s
    IOPS
  Seq Read - Linux AIO - No - No - 4KB - Default Test Directory:
    IOPS
    MB/s
SQLite
Flexible IO Tester:
  Seq Write - Linux AIO - No - No - 64KB - Default Test Directory:
    IOPS
    MB/s
SQLite:
  128
  32
Flexible IO Tester:
  Seq Write - Linux AIO - No - No - 1MB - Default Test Directory:
    MB/s
    IOPS
  Seq Write - Linux AIO - No - No - 256KB - Default Test Directory:
    IOPS
    MB/s
SQLite
Flexible IO Tester:
  Seq Read - Linux AIO - No - No - 16KB - Default Test Directory:
    IOPS
    MB/s
  Seq Write - Linux AIO - No - No - 16KB - Default Test Directory:
    MB/s
    IOPS
  Seq Write - Linux AIO - No - No - 4KB - Default Test Directory:
    MB/s
    IOPS
  Rand Write - Linux AIO - No - No - 1MB - Default Test Directory:
    IOPS
    MB/s
  Seq Read - Linux AIO - No - No - 1MB - Default Test Directory:
    IOPS
    MB/s
  Rand Write - Linux AIO - No - No - 64KB - Default Test Directory:
    IOPS
    MB/s
SQLite
Flexible IO Tester:
  Seq Read - Linux AIO - No - No - 64KB - Default Test Directory:
    MB/s
    IOPS
  Seq Read - Linux AIO - No - No - 256KB - Default Test Directory:
    IOPS
  Rand Read - Linux AIO - No - No - 256KB - Default Test Directory:
    IOPS
    MB/s
  Seq Read - Linux AIO - No - No - 256KB - Default Test Directory:
    MB/s
  Rand Write - Linux AIO - No - No - 256KB - Default Test Directory:
    MB/s
    IOPS
  Rand Read - Linux AIO - No - No - 1MB - Default Test Directory:
    IOPS
    MB/s
  Rand Read - Linux AIO - No - No - 64KB - Default Test Directory:
    MB/s
    IOPS

NFS on TrueNAS

Kernel Notes: Transparent Huge Pages: always
Compiler Notes: --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-link-mutex --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-targets=nvptx-none=/build/gcc-10-Km9U7s/gcc-10-10.2.1/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-10-Km9U7s/gcc-10-10.2.1/debian/tmp-gcn/usr,hsa --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-build-config=bootstrap-lto-lean --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v
Disk Notes: MQ-DEADLINE / errors=remount-ro,relatime,rw / Block Size: 4096
Processor Notes: CPU Microcode: 0x28
Security Notes: itlb_multihit: KVM: Mitigation of VMX unsupported + l1tf: Mitigation of PTE Inversion + mds: Mitigation of Clear buffers; SMT Host state unknown + meltdown: Mitigation of PTI + spec_store_bypass: Mitigation of SSB disabled via prctl and seccomp + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Full generic retpoline IBPB: conditional IBRS_FW STIBP: disabled RSB filling + srbds: Unknown: Dependent on hypervisor status + tsx_async_abort: Not affected

Testing initiated at 24 August 2021 06:02 by user root.

iSCSI on TrueNAS

Processor: 8 x Intel Core i7-4790 (15 Cores), Motherboard: Xen HVM domU v4.13, Chipset: Intel 440FX 82441FX PMC, Memory: 1 x 4088 MB RAM, Disk: 57GB, Graphics: bochs-drmdrmfb

OS: Debian 11, Kernel: 5.10.0-8-amd64 (x86_64), Compiler: GCC 10.2.1 20210110, File-System: ext4, Screen Resolution: 1024x768, System Layer: Xen HVM domU 4.13

Kernel Notes: Transparent Huge Pages: always
Compiler Notes: --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-link-mutex --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-targets=nvptx-none=/build/gcc-10-Km9U7s/gcc-10-10.2.1/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-10-Km9U7s/gcc-10-10.2.1/debian/tmp-gcn/usr,hsa --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-build-config=bootstrap-lto-lean --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v
Disk Notes: MQ-DEADLINE / errors=remount-ro,relatime,rw / Block Size: 4096
Processor Notes: CPU Microcode: 0x28
Security Notes: itlb_multihit: KVM: Mitigation of VMX unsupported + l1tf: Mitigation of PTE Inversion + mds: Mitigation of Clear buffers; SMT Host state unknown + meltdown: Mitigation of PTI + spec_store_bypass: Mitigation of SSB disabled via prctl and seccomp + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Full generic retpoline IBPB: conditional IBRS_FW STIBP: disabled RSB filling + srbds: Unknown: Dependent on hypervisor status + tsx_async_abort: Not affected

Testing initiated at 24 August 2021 06:58 by user root.