io-benchmarks

2 x Intel Xeon Platinum 8168 testing with a Dell 06G98X and mgadrmfb on CentOS Linux 7 via the Phoronix Test Suite.

HTML result view exported from: https://openbenchmarking.org/result/1801228-TY-IOBENCHMA97&grs&sor&rro.

io-benchmarksProcessorMotherboardChipsetMemoryDiskGraphicsNetworkMonitorOSKernelCompilerFile-SystemScreen Resolutionlinux-dell-ssd-lvm-xfslinux-dell-ssd-zfs-lz4-nvme-zillinux-dell-ssd-zfs-nocompress-nvme-zillinux-dell-ssd-zfs-lz4linux-dell-ssd-lvm-xfs-nvme-xloglinux-dell-ssd-lvm-xfs-nvdimm-xloglinux-dell-nvme-lvm-xfs-nvdimm-xloglinux-dell-nvdimm-lvm-xfslinux-dell-ssd-zfs-lz4-nvdimm-zil2 x Intel Xeon Platinum 8168 @ 2.70GHz (96 Cores)Dell 06G98XIntel Device 202023.59375 x 32768 MB DDR4-2666MHz2 x 240GB SSDSCKJB240G7R + 2 x 119GB PERC H740P Adp + 18 x 960GB PERC H740P Adp + 4 x 3201GB Dell Flash PM1725a 3.2TB SFFmgadrmfbIntel 10G X550TCentOS Linux 73.10.0-693.el7.x86_64 (x86_64)GCC 4.8.5 20150623xfs1024x7683.10.0-693.11.1.el7.x86_64 (x86_64)zfsxfs32 GB + 32 GB + 32 GB + 32 GB + 32 GB + 32 GB + 32 GB + 32 GB + 32 GB + 16384 MB + 16384 MB + 16384 MB + 32 GB + 32 GB + 32 GB + 32 GB + 32 GB + 32 GB + 32 GB + 32 GB + 32 GB + 16384 MB + 16384 MB + 16384 MB DDR4-2666MHzAcer V206HQL1600x900zfsOpenBenchmarking.orgCompiler Details- --build=x86_64-redhat-linux --disable-libgcj --disable-libunwind-exceptions --enable-__cxa_atexit --enable-bootstrap --enable-checking=release --enable-gnu-indirect-function --enable-gnu-unique-object --enable-initfini-array --enable-languages=c,c++,objc,obj-c++,java,fortran,ada,go,lto --enable-plugin --enable-shared --enable-threads=posix --mandir=/usr/share/man --with-arch_32=x86-64 --with-linker-hash-style=gnu --with-tune=genericDisk Details- none / data=ordered,discard,noatime,rw / raid1 sdb1[1] sda1[0]System Details- Python 2.7.5.

io-benchmarksfs-mark: 1000 Files, 1MB Sizedbench: 1 Clientsdbench: 48 Clientsdbench: 12 Clientstiobench: 64MB Rand Read - 32 Threadsaio-stress: Rand Writecompilebench: Read Compiled Treecompilebench: Initial Createsqlite: Timed SQLite Insertionsdbench: 128 Clientspostmark: Disk Transaction Performancetiobench: 64MB Rand Write - 32 Threadscompilebench: Compilecompress-gzip: 2GB File Compressionapache: Static Web Page Servingunpack-linux: linux-2.6.32.tar.bz2linux-dell-ssd-lvm-xfslinux-dell-ssd-zfs-lz4-nvme-zillinux-dell-ssd-zfs-nocompress-nvme-zillinux-dell-ssd-zfs-lz4linux-dell-ssd-lvm-xfs-nvme-xloglinux-dell-ssd-lvm-xfs-nvdimm-xloglinux-dell-nvme-lvm-xfs-nvdimm-xloglinux-dell-nvdimm-lvm-xfslinux-dell-ssd-zfs-lz4-nvdimm-zil652.90754.7012164.105944.49169458.152166.61653.95300.6611575.8340762003.231495.2514.1433542.639.58659.57499.312623.541988.2611794.78817.24747.42178.312597.003874785.56890.9211.7939290.5810.28638.77491.372833.782008.6011761.56820.70704.01177.912885.983845083.09877.6111.7938935.489.85440.73470.152594.971818.8911764.13822.21746.83176.323.292769.853443333.12867.2911.7939453.8810.26650.88773.779228.495915.99225302.262411.91764.68353.823.0011296.1048071802.721660.2712.0138442.449.23766.521140.2016777.839595.54235164.212380.18789.42355.7013157.4348382564.721689.2711.8638319.459.05118.47168.27460.91325.32241012.582320.17794.51338.2650.29909.80471719861.391648.4511.7938740.199.081002.301282.6610880.409177.12236829.942332.891402.07328.693749.51465822032.501635.0511.8037659.758.75654.73506.662667.962022.8111726.14823.40739.41178.562666.773705060.50873.9111.8039200.909.84OpenBenchmarking.org

FS-Mark

1000 Files, 1MB Size

OpenBenchmarking.orgFiles/s, More Is BetterFS-Mark 3.31000 Files, 1MB Sizelinux-dell-nvme-lvm-xfs-nvdimm-xloglinux-dell-ssd-zfs-lz4linux-dell-ssd-zfs-nocompress-nvme-zillinux-dell-ssd-lvm-xfs-nvme-xloglinux-dell-ssd-lvm-xfslinux-dell-ssd-zfs-lz4-nvdimm-zillinux-dell-ssd-zfs-lz4-nvme-zillinux-dell-ssd-lvm-xfs-nvdimm-xloglinux-dell-nvdimm-lvm-xfs2004006008001000SE +/- 0.94, N = 3SE +/- 4.25, N = 3SE +/- 3.74, N = 3SE +/- 32.53, N = 6SE +/- 12.10, N = 3SE +/- 11.62, N = 3SE +/- 4.07, N = 3SE +/- 10.55, N = 6SE +/- 13.22, N = 3118.47440.73638.77650.88652.90654.73659.57766.521002.301. (CC) gcc options: -static

Dbench

1 Clients

OpenBenchmarking.orgMB/s, More Is BetterDbench 4.01 Clientslinux-dell-nvme-lvm-xfs-nvdimm-xloglinux-dell-ssd-zfs-lz4linux-dell-ssd-zfs-nocompress-nvme-zillinux-dell-ssd-zfs-lz4-nvme-zillinux-dell-ssd-zfs-lz4-nvdimm-zillinux-dell-ssd-lvm-xfslinux-dell-ssd-lvm-xfs-nvme-xloglinux-dell-ssd-lvm-xfs-nvdimm-xloglinux-dell-nvdimm-lvm-xfs30060090012001500SE +/- 0.04, N = 3SE +/- 0.68, N = 3SE +/- 4.30, N = 3SE +/- 1.98, N = 3SE +/- 0.58, N = 3SE +/- 0.14, N = 3SE +/- 1.84, N = 3SE +/- 2.25, N = 3SE +/- 3.92, N = 3168.27470.15491.37499.31506.66754.70773.771140.201282.661. (CC) gcc options: -lpopt -O2

Dbench

48 Clients

OpenBenchmarking.orgMB/s, More Is BetterDbench 4.048 Clientslinux-dell-nvme-lvm-xfs-nvdimm-xloglinux-dell-ssd-zfs-lz4linux-dell-ssd-zfs-lz4-nvme-zillinux-dell-ssd-zfs-lz4-nvdimm-zillinux-dell-ssd-zfs-nocompress-nvme-zillinux-dell-ssd-lvm-xfs-nvme-xloglinux-dell-nvdimm-lvm-xfslinux-dell-ssd-lvm-xfslinux-dell-ssd-lvm-xfs-nvdimm-xlog4K8K12K16K20KSE +/- 9.53, N = 6SE +/- 2.97, N = 3SE +/- 2.87, N = 3SE +/- 3.44, N = 3SE +/- 3.38, N = 3SE +/- 22.23, N = 3SE +/- 5.86, N = 3SE +/- 14.78, N = 3SE +/- 65.46, N = 3460.912594.972623.542667.962833.789228.4910880.4012164.1016777.831. (CC) gcc options: -lpopt -O2

Dbench

12 Clients

OpenBenchmarking.orgMB/s, More Is BetterDbench 4.012 Clientslinux-dell-nvme-lvm-xfs-nvdimm-xloglinux-dell-ssd-zfs-lz4linux-dell-ssd-zfs-lz4-nvme-zillinux-dell-ssd-zfs-nocompress-nvme-zillinux-dell-ssd-zfs-lz4-nvdimm-zillinux-dell-ssd-lvm-xfs-nvme-xloglinux-dell-ssd-lvm-xfslinux-dell-nvdimm-lvm-xfslinux-dell-ssd-lvm-xfs-nvdimm-xlog2K4K6K8K10KSE +/- 7.21, N = 6SE +/- 6.76, N = 3SE +/- 6.47, N = 3SE +/- 2.41, N = 3SE +/- 0.84, N = 3SE +/- 33.03, N = 3SE +/- 5.85, N = 3SE +/- 4.49, N = 3SE +/- 41.86, N = 3325.321818.891988.262008.602022.815915.995944.499177.129595.541. (CC) gcc options: -lpopt -O2

Threaded I/O Tester

64MB Random Read - 32 Threads

OpenBenchmarking.orgMB/s, More Is BetterThreaded I/O Tester 2017050364MB Random Read - 32 Threadslinux-dell-ssd-zfs-lz4-nvdimm-zillinux-dell-ssd-zfs-nocompress-nvme-zillinux-dell-ssd-zfs-lz4linux-dell-ssd-zfs-lz4-nvme-zillinux-dell-ssd-lvm-xfslinux-dell-ssd-lvm-xfs-nvme-xloglinux-dell-ssd-lvm-xfs-nvdimm-xloglinux-dell-nvdimm-lvm-xfslinux-dell-nvme-lvm-xfs-nvdimm-xlog50K100K150K200K250KSE +/- 13.11, N = 3SE +/- 26.70, N = 3SE +/- 7.18, N = 3SE +/- 14.01, N = 3SE +/- 2942.45, N = 3SE +/- 946.54, N = 3SE +/- 779.38, N = 3SE +/- 2174.12, N = 3SE +/- 795.21, N = 311726.1411761.5611764.1311794.78169458.15225302.26235164.21236829.94241012.581. (CC) gcc options: -O2

AIO-Stress

Random Write

OpenBenchmarking.orgMB/s, More Is BetterAIO-Stress 0.21Random Writelinux-dell-ssd-zfs-lz4-nvme-zillinux-dell-ssd-zfs-nocompress-nvme-zillinux-dell-ssd-zfs-lz4linux-dell-ssd-zfs-lz4-nvdimm-zillinux-dell-ssd-lvm-xfslinux-dell-nvme-lvm-xfs-nvdimm-xloglinux-dell-nvdimm-lvm-xfslinux-dell-ssd-lvm-xfs-nvdimm-xloglinux-dell-ssd-lvm-xfs-nvme-xlog5001000150020002500SE +/- 15.72, N = 3SE +/- 12.60, N = 3SE +/- 18.67, N = 6SE +/- 4.80, N = 3SE +/- 6.77, N = 3SE +/- 33.55, N = 3SE +/- 1.22, N = 3SE +/- 0.61, N = 3SE +/- 2.17, N = 3817.24820.70822.21823.402166.612320.172332.892380.182411.911. (CC) gcc options: -pthread -laio

Compile Bench

Test: Read Compiled Tree

OpenBenchmarking.orgMB/s, More Is BetterCompile Bench 0.6Test: Read Compiled Treelinux-dell-ssd-lvm-xfslinux-dell-ssd-zfs-nocompress-nvme-zillinux-dell-ssd-zfs-lz4-nvdimm-zillinux-dell-ssd-zfs-lz4linux-dell-ssd-zfs-lz4-nvme-zillinux-dell-ssd-lvm-xfs-nvme-xloglinux-dell-ssd-lvm-xfs-nvdimm-xloglinux-dell-nvme-lvm-xfs-nvdimm-xloglinux-dell-nvdimm-lvm-xfs30060090012001500SE +/- 4.18, N = 3SE +/- 12.71, N = 3SE +/- 25.59, N = 3SE +/- 23.49, N = 3SE +/- 16.95, N = 3SE +/- 0.26, N = 3SE +/- 5.28, N = 3SE +/- 6.43, N = 3SE +/- 5.28, N = 3653.95704.01739.41746.83747.42764.68789.42794.511402.07

Compile Bench

Test: Initial Create

OpenBenchmarking.orgMB/s, More Is BetterCompile Bench 0.6Test: Initial Createlinux-dell-ssd-zfs-lz4linux-dell-ssd-zfs-nocompress-nvme-zillinux-dell-ssd-zfs-lz4-nvme-zillinux-dell-ssd-zfs-lz4-nvdimm-zillinux-dell-ssd-lvm-xfslinux-dell-nvdimm-lvm-xfslinux-dell-nvme-lvm-xfs-nvdimm-xloglinux-dell-ssd-lvm-xfs-nvme-xloglinux-dell-ssd-lvm-xfs-nvdimm-xlog80160240320400SE +/- 0.42, N = 3SE +/- 0.29, N = 3SE +/- 0.46, N = 3SE +/- 1.18, N = 3SE +/- 0.60, N = 3SE +/- 4.11, N = 3SE +/- 1.07, N = 3SE +/- 2.45, N = 3SE +/- 1.51, N = 3176.32177.91178.31178.56300.66328.69338.26353.82355.70

SQLite

Timed SQLite Insertions

OpenBenchmarking.orgSeconds, Fewer Is BetterSQLite 3.8.10.2Timed SQLite Insertionslinux-dell-nvme-lvm-xfs-nvdimm-xloglinux-dell-ssd-zfs-lz4linux-dell-ssd-lvm-xfs-nvme-xlog1122334455SE +/- 0.11, N = 3SE +/- 0.01, N = 350.293.293.001. (CC) gcc options: -O2 -ldl -lpthread

Dbench

128 Clients

OpenBenchmarking.orgMB/s, More Is BetterDbench 4.0128 Clientslinux-dell-nvme-lvm-xfs-nvdimm-xloglinux-dell-ssd-zfs-lz4-nvme-zillinux-dell-ssd-zfs-lz4-nvdimm-zillinux-dell-ssd-zfs-lz4linux-dell-ssd-zfs-nocompress-nvme-zillinux-dell-nvdimm-lvm-xfslinux-dell-ssd-lvm-xfs-nvme-xloglinux-dell-ssd-lvm-xfslinux-dell-ssd-lvm-xfs-nvdimm-xlog3K6K9K12K15KSE +/- 10.78, N = 3SE +/- 5.57, N = 3SE +/- 1.18, N = 3SE +/- 1.14, N = 3SE +/- 2.36, N = 3SE +/- 2.92, N = 3SE +/- 9.78, N = 3SE +/- 16.67, N = 3SE +/- 10.59, N = 3909.802597.002666.772769.852885.983749.5111296.1011575.8313157.431. (CC) gcc options: -lpopt -O2

PostMark

Disk Transaction Performance

OpenBenchmarking.orgTPS, More Is BetterPostMark 1.51Disk Transaction Performancelinux-dell-ssd-zfs-lz4linux-dell-ssd-zfs-lz4-nvdimm-zillinux-dell-ssd-zfs-nocompress-nvme-zillinux-dell-ssd-zfs-lz4-nvme-zillinux-dell-ssd-lvm-xfslinux-dell-nvdimm-lvm-xfslinux-dell-nvme-lvm-xfs-nvdimm-xloglinux-dell-ssd-lvm-xfs-nvme-xloglinux-dell-ssd-lvm-xfs-nvdimm-xlog10002000300040005000SE +/- 7.12, N = 6SE +/- 9.77, N = 6SE +/- 8.03, N = 6SE +/- 6.14, N = 6SE +/- 22.00, N = 3SE +/- 29.00, N = 3SE +/- 51.39, N = 3SE +/- 31.33, N = 3344370384387407646584717480748381. (CC) gcc options: -O3

Threaded I/O Tester

64MB Random Write - 32 Threads

OpenBenchmarking.orgMB/s, More Is BetterThreaded I/O Tester 2017050364MB Random Write - 32 Threadslinux-dell-ssd-lvm-xfs-nvme-xloglinux-dell-ssd-lvm-xfslinux-dell-ssd-lvm-xfs-nvdimm-xloglinux-dell-ssd-zfs-lz4linux-dell-ssd-zfs-lz4-nvme-zillinux-dell-ssd-zfs-lz4-nvdimm-zillinux-dell-ssd-zfs-nocompress-nvme-zillinux-dell-nvme-lvm-xfs-nvdimm-xloglinux-dell-nvdimm-lvm-xfs5K10K15K20K25KSE +/- 74.86, N = 6SE +/- 16.82, N = 3SE +/- 35.96, N = 5SE +/- 55.57, N = 6SE +/- 42.49, N = 3SE +/- 86.10, N = 4SE +/- 102.34, N = 3SE +/- 300.03, N = 5SE +/- 122.32, N = 31802.722003.232564.723333.124785.565060.505083.0919861.3922032.501. (CC) gcc options: -O2

Compile Bench

Test: Compile

OpenBenchmarking.orgMB/s, More Is BetterCompile Bench 0.6Test: Compilelinux-dell-ssd-zfs-lz4linux-dell-ssd-zfs-lz4-nvdimm-zillinux-dell-ssd-zfs-nocompress-nvme-zillinux-dell-ssd-zfs-lz4-nvme-zillinux-dell-ssd-lvm-xfslinux-dell-nvdimm-lvm-xfslinux-dell-nvme-lvm-xfs-nvdimm-xloglinux-dell-ssd-lvm-xfs-nvme-xloglinux-dell-ssd-lvm-xfs-nvdimm-xlog400800120016002000SE +/- 3.19, N = 3SE +/- 1.98, N = 3SE +/- 2.72, N = 3SE +/- 4.43, N = 3SE +/- 26.11, N = 6SE +/- 28.14, N = 3SE +/- 10.69, N = 3SE +/- 24.68, N = 3SE +/- 2.52, N = 3867.29873.91877.61890.921495.251635.051648.451660.271689.27

Gzip Compression

2GB File Compression

OpenBenchmarking.orgSeconds, Fewer Is BetterGzip Compression2GB File Compressionlinux-dell-ssd-lvm-xfslinux-dell-ssd-lvm-xfs-nvme-xloglinux-dell-ssd-lvm-xfs-nvdimm-xloglinux-dell-ssd-zfs-lz4-nvdimm-zillinux-dell-nvdimm-lvm-xfslinux-dell-nvme-lvm-xfs-nvdimm-xloglinux-dell-ssd-zfs-lz4linux-dell-ssd-zfs-nocompress-nvme-zillinux-dell-ssd-zfs-lz4-nvme-zil48121620SE +/- 0.01, N = 3SE +/- 0.10, N = 3SE +/- 0.03, N = 3SE +/- 0.00, N = 3SE +/- 0.00, N = 3SE +/- 0.00, N = 3SE +/- 0.00, N = 3SE +/- 0.00, N = 3SE +/- 0.00, N = 314.1412.0111.8611.8011.8011.7911.7911.7911.79

Apache Benchmark

Static Web Page Serving

OpenBenchmarking.orgRequests Per Second, More Is BetterApache Benchmark 2.4.7Static Web Page Servinglinux-dell-ssd-lvm-xfslinux-dell-nvdimm-lvm-xfslinux-dell-ssd-lvm-xfs-nvdimm-xloglinux-dell-ssd-lvm-xfs-nvme-xloglinux-dell-nvme-lvm-xfs-nvdimm-xloglinux-dell-ssd-zfs-nocompress-nvme-zillinux-dell-ssd-zfs-lz4-nvdimm-zillinux-dell-ssd-zfs-lz4-nvme-zillinux-dell-ssd-zfs-lz48K16K24K32K40KSE +/- 320.98, N = 3SE +/- 747.07, N = 3SE +/- 213.45, N = 3SE +/- 218.33, N = 3SE +/- 774.16, N = 3SE +/- 573.27, N = 5SE +/- 770.83, N = 3SE +/- 785.56, N = 3SE +/- 87.69, N = 333542.6337659.7538319.4538442.4438740.1938935.4839200.9039290.5839453.881. (CC) gcc options: -shared -fPIC -O2 -pthread

Unpacking The Linux Kernel

linux-2.6.32.tar.bz2

OpenBenchmarking.orgSeconds, Fewer Is BetterUnpacking The Linux Kernellinux-2.6.32.tar.bz2linux-dell-ssd-zfs-lz4-nvme-zillinux-dell-ssd-zfs-lz4linux-dell-ssd-zfs-nocompress-nvme-zillinux-dell-ssd-zfs-lz4-nvdimm-zillinux-dell-ssd-lvm-xfslinux-dell-ssd-lvm-xfs-nvme-xloglinux-dell-nvme-lvm-xfs-nvdimm-xloglinux-dell-ssd-lvm-xfs-nvdimm-xloglinux-dell-nvdimm-lvm-xfs3691215SE +/- 0.11, N = 4SE +/- 0.03, N = 4SE +/- 0.13, N = 4SE +/- 0.12, N = 4SE +/- 0.13, N = 8SE +/- 0.28, N = 8SE +/- 0.16, N = 8SE +/- 0.13, N = 8SE +/- 0.01, N = 410.2810.269.859.849.589.239.089.058.75


Phoronix Test Suite v10.8.5