FreeBSD 12.0 ZFS vs. ZoL Benchmarks

FreeBSD 12.0 ZFS benchmarks for a future article.

Compare your own system(s) to this result file with the Phoronix Test Suite by running the command: phoronix-test-suite benchmark 1904258-SP-1904234HV78
Jump To Table - Results

View

Do Not Show Noisy Results
Do Not Show Results With Incomplete Data
Do Not Show Results With Little Change/Spread
List Notable Results

Limit displaying results to tests within:

Disk Test Suite 3 Tests
Server 2 Tests

Statistics

Show Overall Harmonic Mean(s)
Show Overall Geometric Mean
Show Geometric Means Per-Suite/Category
Show Wins / Losses Counts (Pie Chart)
Normalize Results
Remove Outliers Before Calculating Averages

Graph Settings

Force Line Graphs Where Applicable
Convert To Scalar Where Applicable
Disable Color Branding
Prefer Vertical Bar Graphs

Additional Graphs

Show Perf Per Core/Thread Calculation Graphs Where Applicable
Show Perf Per Clock Calculation Graphs Where Applicable

Multi-Way Comparison

Condense Multi-Option Tests Into Single Result Graphs

Table

Show Detailed System Result Table

Run Management

Highlight
Result
Hide
Result
Result
Identifier
Performance Per
Dollar
Date
Run
  Test
  Duration
FreeBSD iX ZoL
April 22 2019
  2 Hours, 3 Minutes
FreeBSD ZFS
April 22 2019
  2 Hours, 13 Minutes
FreeBSD UFS
April 22 2019
  2 Hours, 15 Minutes
Ubuntu 18.04 EXT4
April 23 2019
  3 Hours, 22 Minutes
Ubuntu 18.04 ZoL
April 23 2019
  2 Hours, 13 Minutes
Debian Stretch ZoL
April 25 2019
  2 Hours, 33 Minutes
Invert Hiding All Results Option
  2 Hours, 27 Minutes

Only show results where is faster than
Only show results matching title/arguments (delimit multiple options with a comma):
Do not show results matching title/arguments (delimit multiple options with a comma):


FreeBSD 12.0 ZFS vs. ZoL BenchmarksProcessorMotherboardChipsetMemoryDiskGraphicsAudioMonitorNetworkOSKernelCompilerFile-SystemScreen ResolutionDesktopDisplay ServerDisplay DriverOpenGLFreeBSD iX ZoLFreeBSD ZFSFreeBSD UFSUbuntu 18.04 EXT4Ubuntu 18.04 ZoLDebian Stretch ZoLIntel Xeon E3-1275 v6 @ 3.80GHz (8 Cores)ASUS P10S-M WSIntel Xeon E3-1200 v6/7th Gen Core Registers16384MBSamsung SSD 970 EVO Plus 500GBIntel HD P630Intel Sunrise Point HDAFreeBSD12.0-STABLE (x86_64)Clang 8.0.0 (SVN 356365)zfs800x60012.0-RELEASE (x86_64)Clang 6.0.1 (SVN 335540)ufsIntel Xeon E3-1275 v6 @ 4.20GHz (4 Cores / 8 Threads)ASUS P10S-M WS (4401 BIOS)Intel Xeon E3-1200 v6/7thIntel HD P630 3GB (1150MHz)Realtek ALC1150Acer B286HK + DELL P2415Q2 x Intel I210Ubuntu 18.044.18.0-17-generic (x86_64)GNOME Shell 3.28.3X Server 1.20.1modesetting 1.20.14.5 Mesa 18.2.8GCC 7.3.0ext41920x1080zfsAMD Ryzen Threadripper 2950X 16-Core @ 3.50GHz (16 Cores / 32 Threads)ASRock X399 Taichi (P3.30 BIOS)AMD Device 14504 x 16384 MB DDR4-1200MHz3 x 4001GB TOSHIBA MD04ACA4 + 2 x 4001GB Western Digital WD40EZRZ-00G + 4001GB HGST HDS724040AL + 4001GB Seagate ST4000VN008-2DR1 + 4001GB Western Digital WD40EZRX-00S + 512GB SPCC Solid State + 60GB Corsair Force LS + 4001GB TOSHIBA HDWE140 + 3 x 128GB ADATA SP900NS38amdgpudrmfb (1206/1750MHz)AMD Device aae02 x Intel I211 + Mellanox MT26448 + Intel Device 24fbDebian 9.84.19.0-0.bpo.4-amd64 (x86_64)GCC 6.3.0 20170516OpenBenchmarking.orgPython Details- FreeBSD iX ZoL: Python 3.6.8Security Details- FreeBSD iX ZoL: KPTI- FreeBSD ZFS: KPTI- FreeBSD UFS: KPTI- Ubuntu 18.04 EXT4: KPTI + __user pointer sanitization + Full generic retpoline IBPB IBRS_FW STIBP + SSB disabled via prctl and seccomp + PTE Inversion; VMX: conditional cache flushes SMT vulnerable- Ubuntu 18.04 ZoL: KPTI + __user pointer sanitization + Full generic retpoline IBPB IBRS_FW STIBP + SSB disabled via prctl and seccomp + PTE Inversion; VMX: conditional cache flushes SMT vulnerable- Debian Stretch ZoL: __user pointer sanitization + Full AMD retpoline IBPB: conditional STIBP: disabled RSB filling + SSB disabled via prctl and seccompCompiler Details- Ubuntu 18.04 EXT4: --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++ --enable-libmpx --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-targets=nvptx-none --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib --with-tune=generic --without-cuda-driver -v- Ubuntu 18.04 ZoL: --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++ --enable-libmpx --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-targets=nvptx-none --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib --with-tune=generic --without-cuda-driver -v- Debian Stretch ZoL: --build=x86_64-linux-gnu --disable-browser-plugin --disable-vtable-verify --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-gtk-cairo --enable-java-awt=gtk --enable-java-home --enable-languages=c,ada,c++,java,go,d,fortran,objc,obj-c++ --enable-libmpx --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-arch-directory=amd64 --with-default-libstdcxx-abi=new --with-multilib-list=m32,m64,mx32 --with-target-system-zlib --with-tune=generic -vDisk Details- Ubuntu 18.04 EXT4: NONE / errors=remount-ro,relatime,rwProcessor Details- Ubuntu 18.04 EXT4: Scaling Governor: intel_pstate powersave- Ubuntu 18.04 ZoL: Scaling Governor: intel_pstate powersave- Debian Stretch ZoL: Scaling Governor: acpi-cpufreq conservativeDisk Scheduler Details- Ubuntu 18.04 ZoL: NONE

FreeBSD iX ZoLFreeBSD ZFSFreeBSD UFSUbuntu 18.04 EXT4Ubuntu 18.04 ZoLDebian Stretch ZoLLogarithmic Result OverviewPhoronix Test SuitePostMarkBlogBenchIOzoneIOzoneDbenchBlogBenchDbenchD.T.PRead4Kb - 512MB - Write Performance4Kb - 512MB - Read Performance1Write6

FreeBSD 12.0 ZFS vs. ZoL Benchmarkssqlite: Timed SQLite Insertionsblogbench: Readblogbench: Writeiozone: 4Kb - 512MB - Read Performanceiozone: 4Kb - 512MB - Write Performancedbench: 1dbench: 6postmark: Disk Transaction PerformanceFreeBSD iX ZoLFreeBSD ZFSFreeBSD UFSUbuntu 18.04 EXT4Ubuntu 18.04 ZoLDebian Stretch ZoL22.38137813541782489911132423707627.21131384634172514855252490694712556592665406016728452843379639.35585358358735644131360.99226669727.04332905189322237893100.66312388613.825689994226192064.12207.582831083OpenBenchmarking.org

SQLite

This is a simple benchmark of SQLite. At present this test profile just measures the time to perform a pre-defined number of insertions on an indexed database. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgSeconds, Fewer Is BetterSQLite 3.22Timed SQLite InsertionsFreeBSD iX ZoLFreeBSD ZFSUbuntu 18.04 EXT4Ubuntu 18.04 ZoLDebian Stretch ZoL918273645SE +/- 0.05, N = 3SE +/- 0.04, N = 3SE +/- 0.03, N = 3SE +/- 0.07, N = 3SE +/- 0.22, N = 322.3827.2139.3527.0413.82
OpenBenchmarking.orgSeconds, Fewer Is BetterSQLite 3.22Timed SQLite InsertionsFreeBSD iX ZoLFreeBSD ZFSUbuntu 18.04 EXT4Ubuntu 18.04 ZoLDebian Stretch ZoL816243240Min: 22.3 / Avg: 22.38 / Max: 22.48Min: 27.17 / Avg: 27.21 / Max: 27.3Min: 39.29 / Avg: 39.35 / Max: 39.39Min: 26.91 / Avg: 27.04 / Max: 27.13Min: 13.46 / Avg: 13.82 / Max: 14.23

BlogBench

BlogBench is designed to replicate the load of a real-world busy file server by stressing the file-system with multiple threads of random reads, writes, and rewrites. The behavior is mimicked of that of a blog by creating blogs with content and pictures, modifying blog posts, adding comments to these blogs, and then reading the content of the blogs. All of these blogs generated are created locally with fake content and pictures. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgFinal Score, More Is BetterBlogBench 1.1Test: ReadFreeBSD iX ZoLFreeBSD ZFSFreeBSD UFSUbuntu 18.04 EXT4Ubuntu 18.04 ZoLDebian Stretch ZoL300K600K900K1200K1500KSE +/- 24290.33, N = 9SE +/- 12811.27, N = 9SE +/- 59738.49, N = 9SE +/- 14788.62, N = 9SE +/- 6586.79, N = 9SE +/- 66989.71, N = 9137813513138461255659585358332905568999clangclangclanggccgccgcc
OpenBenchmarking.orgFinal Score, More Is BetterBlogBench 1.1Test: ReadFreeBSD iX ZoLFreeBSD ZFSFreeBSD UFSUbuntu 18.04 EXT4Ubuntu 18.04 ZoLDebian Stretch ZoL200K400K600K800K1000KMin: 1275731 / Avg: 1378135.11 / Max: 1510057Min: 1245463 / Avg: 1313846 / Max: 1350556Min: 1010614 / Avg: 1255658.78 / Max: 1592100Min: 537035 / Avg: 585357.56 / Max: 684003Min: 318462 / Avg: 332904.89 / Max: 383790Min: 227991 / Avg: 568999 / Max: 884857

OpenBenchmarking.orgFinal Score, More Is BetterBlogBench 1.1Test: WriteFreeBSD iX ZoLFreeBSD ZFSFreeBSD UFSUbuntu 18.04 EXT4Ubuntu 18.04 ZoLDebian Stretch ZoL8K16K24K32K40KSE +/- 358.54, N = 3SE +/- 494.73, N = 3SE +/- 581.92, N = 3SE +/- 845.64, N = 3SE +/- 2614.11, N = 3SE +/- 800.94, N = 341783417266535873189324226clangclanggccgccgcc
OpenBenchmarking.orgFinal Score, More Is BetterBlogBench 1.1Test: WriteFreeBSD iX ZoLFreeBSD ZFSFreeBSD UFSUbuntu 18.04 EXT4Ubuntu 18.04 ZoLDebian Stretch ZoL6K12K18K24K30KMin: 3650 / Avg: 4177.67 / Max: 4862Min: 2436 / Avg: 3416.67 / Max: 4021Min: 1841 / Avg: 2665.33 / Max: 3789Min: 34395 / Avg: 35873.33 / Max: 37324Min: 13715 / Avg: 18932 / Max: 21837Min: 3024 / Avg: 4226 / Max: 5744

IOzone

The IOzone benchmark tests the hard disk drive / file-system performance. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgMB/s, More Is BetterIOzone 3.465Record Size: 4Kb - File Size: 512MB - Disk Test: Read PerformanceFreeBSD iX ZoLFreeBSD ZFSFreeBSD UFSUbuntu 18.04 EXT4Ubuntu 18.04 ZoLDebian Stretch ZoL12002400360048006000SE +/- 22.05, N = 3SE +/- 22.22, N = 11SE +/- 3.91, N = 3SE +/- 92.53, N = 12SE +/- 18.29, N = 3SE +/- 299.08, N = 9248925144060564422371920
OpenBenchmarking.orgMB/s, More Is BetterIOzone 3.465Record Size: 4Kb - File Size: 512MB - Disk Test: Read PerformanceFreeBSD iX ZoLFreeBSD ZFSFreeBSD UFSUbuntu 18.04 EXT4Ubuntu 18.04 ZoLDebian Stretch ZoL10002000300040005000Min: 2452.11 / Avg: 2488.67 / Max: 2528.3Min: 2463.36 / Avg: 2514.01 / Max: 2681.95Min: 4052.69 / Avg: 4060.22 / Max: 4065.78Min: 4630.17 / Avg: 5644.03 / Max: 5779.83Min: 2211.15 / Avg: 2236.9 / Max: 2272.26Min: 49.34 / Avg: 1919.53 / Max: 2673.01

OpenBenchmarking.orgMB/s, More Is BetterIOzone 3.465Record Size: 4Kb - File Size: 512MB - Disk Test: Write PerformanceFreeBSD iX ZoLFreeBSD ZFSFreeBSD UFSUbuntu 18.04 EXT4Ubuntu 18.04 ZoLDebian Stretch ZoL400800120016002000SE +/- 1.30, N = 3SE +/- 13.24, N = 3SE +/- 8.99, N = 3SE +/- 18.66, N = 3SE +/- 4.04, N = 3SE +/- 13.46, N = 9911.00855.001672.001313.00893.0064.12
OpenBenchmarking.orgMB/s, More Is BetterIOzone 3.465Record Size: 4Kb - File Size: 512MB - Disk Test: Write PerformanceFreeBSD iX ZoLFreeBSD ZFSFreeBSD UFSUbuntu 18.04 EXT4Ubuntu 18.04 ZoLDebian Stretch ZoL30060090012001500Min: 908.88 / Avg: 910.55 / Max: 913.11Min: 830.7 / Avg: 855.14 / Max: 876.19Min: 1661.4 / Avg: 1672.46 / Max: 1690.25Min: 1275.8 / Avg: 1312.62 / Max: 1336.32Min: 884.89 / Avg: 892.97 / Max: 897.33Min: 36.69 / Avg: 64.12 / Max: 165.75

Dbench

Dbench is a benchmark designed by the Samba project as a free alternative to netbench, but dbench contains only file-system calls for testing the disk performance. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgMB/s, More Is BetterDbench 4.0Client Count: 1FreeBSD iX ZoLFreeBSD ZFSFreeBSD UFSUbuntu 18.04 EXT4Ubuntu 18.04 ZoLDebian Stretch ZoL2004006008001000SE +/- 0.09, N = 3SE +/- 0.15, N = 3SE +/- 0.70, N = 3SE +/- 0.83, N = 9SE +/- 0.07, N = 3SE +/- 1.58, N = 3132.00252.00845.0060.99100.66207.58
OpenBenchmarking.orgMB/s, More Is BetterDbench 4.0Client Count: 1FreeBSD iX ZoLFreeBSD ZFSFreeBSD UFSUbuntu 18.04 EXT4Ubuntu 18.04 ZoLDebian Stretch ZoL150300450600750Min: 131.68 / Avg: 131.84 / Max: 132.01Min: 252 / Avg: 252.23 / Max: 252.51Min: 844.08 / Avg: 845.39 / Max: 846.46Min: 57.99 / Avg: 60.99 / Max: 65.59Min: 100.56 / Avg: 100.66 / Max: 100.79Min: 204.72 / Avg: 207.58 / Max: 210.16

OpenBenchmarking.orgMB/s, More Is BetterDbench 4.0Client Count: 6FreeBSD iX ZoLFreeBSD ZFSFreeBSD UFSUbuntu 18.04 EXT4Ubuntu 18.04 ZoLDebian Stretch ZoL6001200180024003000SE +/- 2.00, N = 3SE +/- 0.52, N = 3SE +/- 13.00, N = 3SE +/- 3.63, N = 3SE +/- 0.34, N = 3SE +/- 2.54, N = 34234902843226312283
OpenBenchmarking.orgMB/s, More Is BetterDbench 4.0Client Count: 6FreeBSD iX ZoLFreeBSD ZFSFreeBSD UFSUbuntu 18.04 EXT4Ubuntu 18.04 ZoLDebian Stretch ZoL5001000150020002500Min: 421.39 / Avg: 423.49 / Max: 427.48Min: 488.95 / Avg: 489.83 / Max: 490.75Min: 2818 / Avg: 2842.57 / Max: 2862.22Min: 219.36 / Avg: 226.08 / Max: 231.82Min: 311.75 / Avg: 312.12 / Max: 312.8Min: 278.09 / Avg: 282.69 / Max: 286.84

PostMark

This is a test of NetApp's PostMark benchmark designed to simulate small-file testing similar to the tasks endured by web and mail servers. This test profile will set PostMark to perform 25,000 transactions with 500 files simultaneously with the file sizes ranging between 5 and 512 kilobytes. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgTPS, More Is BetterPostMark 1.51Disk Transaction PerformanceFreeBSD iX ZoLFreeBSD ZFSFreeBSD UFSUbuntu 18.04 EXT4Ubuntu 18.04 ZoLDebian Stretch ZoL15003000450060007500SE +/- 66.00, N = 3SE +/- 111.44, N = 3SE +/- 57.70, N = 15SE +/- 59.33, N = 3SE +/- 20.00, N = 3SE +/- 4.10, N = 3707669473796669738861083clangclangclanggccgccgcc
OpenBenchmarking.orgTPS, More Is BetterPostMark 1.51Disk Transaction PerformanceFreeBSD iX ZoLFreeBSD ZFSFreeBSD UFSUbuntu 18.04 EXT4Ubuntu 18.04 ZoLDebian Stretch ZoL12002400360048006000Min: 6944 / Avg: 7076 / Max: 7142Min: 6756 / Avg: 6947.33 / Max: 7142Min: 3424 / Avg: 3796 / Max: 4166Min: 6578 / Avg: 6696.67 / Max: 6756Min: 3846 / Avg: 3886 / Max: 3906Min: 1077 / Avg: 1083.33 / Max: 1091

8 Results Shown

SQLite
BlogBench:
  Read
  Write
IOzone:
  4Kb - 512MB - Read Performance
  4Kb - 512MB - Write Performance
Dbench:
  1
  6
PostMark