FreeBSD 12.0 ZFS vs. ZoL Benchmarks

FreeBSD 12.0 ZFS benchmarks for a future article.

Compare your own system(s) to this result file with the Phoronix Test Suite by running the command: phoronix-test-suite benchmark 1904234-HV-FREEBSDZF72
Jump To Table - Results

View

Do Not Show Noisy Results
Do Not Show Results With Incomplete Data
Do Not Show Results With Little Change/Spread
List Notable Results
Show Result Confidence Charts
Allow Limiting Results To Certain Suite(s)

Statistics

Show Overall Harmonic Mean(s)
Show Overall Geometric Mean
Show Wins / Losses Counts (Pie Chart)
Normalize Results
Remove Outliers Before Calculating Averages

Graph Settings

Force Line Graphs Where Applicable
Convert To Scalar Where Applicable
Disable Color Branding
Prefer Vertical Bar Graphs

Additional Graphs

Show Perf Per Core/Thread Calculation Graphs Where Applicable
Show Perf Per Clock Calculation Graphs Where Applicable

Multi-Way Comparison

Condense Multi-Option Tests Into Single Result Graphs

Table

Show Detailed System Result Table

Run Management

Highlight
Result
Toggle/Hide
Result
Result
Identifier
Performance Per
Dollar
Date
Run
  Test
  Duration
FreeBSD iX ZoL
April 22 2019
  2 Hours, 3 Minutes
FreeBSD ZFS
April 22 2019
  2 Hours, 13 Minutes
FreeBSD UFS
April 22 2019
  2 Hours, 15 Minutes
Ubuntu 18.04 EXT4
April 23 2019
  3 Hours, 22 Minutes
Ubuntu 18.04 ZoL
April 23 2019
  2 Hours, 13 Minutes
Invert Behavior (Only Show Selected Data)
  2 Hours, 26 Minutes

Only show results where is faster than
Only show results matching title/arguments (delimit multiple options with a comma):
Do not show results matching title/arguments (delimit multiple options with a comma):

FreeBSD iX ZoL

Processor: Intel Xeon E3-1275 v6 @ 3.80GHz (8 Cores), Motherboard: ASUS P10S-M WS, Chipset: Intel Xeon E3-1200 v6/7th Gen Core Registers, Memory: 16384MB, Disk: Samsung SSD 970 EVO Plus 500GB, Graphics: Intel HD P630, Audio: Intel Sunrise Point HDA

OS: FreeBSD, Kernel: 12.0-STABLE (x86_64), Compiler: Clang 8.0.0 (SVN 356365), File-System: zfs, Screen Resolution: 800x600

Python Notes: Python 3.6.8
Security Notes: KPTI

FreeBSD ZFS

OS: FreeBSD, Kernel: 12.0-RELEASE (x86_64), Compiler: Clang 6.0.1 (SVN 335540), File-System: zfs, Screen Resolution: 800x600

Security Notes: KPTI

FreeBSD UFS

OS: FreeBSD, Kernel: 12.0-RELEASE (x86_64), Compiler: Clang 6.0.1 (SVN 335540), File-System: ufs, Screen Resolution: 800x600

Ubuntu 18.04 EXT4

Processor: Intel Xeon E3-1275 v6 @ 4.20GHz (4 Cores / 8 Threads), Motherboard: ASUS P10S-M WS (4401 BIOS), Chipset: Intel Xeon E3-1200 v6/7th, Memory: 16384MB, Disk: Samsung SSD 970 EVO Plus 500GB, Graphics: Intel HD P630 3GB (1150MHz), Audio: Realtek ALC1150, Monitor: Acer B286HK + DELL P2415Q, Network: 2 x Intel I210

OS: Ubuntu 18.04, Kernel: 4.18.0-17-generic (x86_64), Desktop: GNOME Shell 3.28.3, Display Server: X Server 1.20.1, Display Driver: modesetting 1.20.1, OpenGL: 4.5 Mesa 18.2.8, Compiler: GCC 7.3.0, File-System: ext4, Screen Resolution: 1920x1080

Compiler Notes: --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++ --enable-libmpx --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-targets=nvptx-none --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib --with-tune=generic --without-cuda-driver -v
Disk Notes: NONE / errors=remount-ro,relatime,rw
Processor Notes: Scaling Governor: intel_pstate powersave
Security Notes: KPTI + __user pointer sanitization + Full generic retpoline IBPB IBRS_FW STIBP + SSB disabled via prctl and seccomp + PTE Inversion; VMX: conditional cache flushes SMT vulnerable

Ubuntu 18.04 ZoL

OS: Ubuntu 18.04, Kernel: 4.18.0-17-generic (x86_64), Desktop: GNOME Shell 3.28.3, Display Server: X Server 1.20.1, Display Driver: modesetting 1.20.1, OpenGL: 4.5 Mesa 18.2.8, Compiler: GCC 7.3.0, File-System: zfs, Screen Resolution: 1920x1080

Compiler Notes: --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++ --enable-libmpx --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-targets=nvptx-none --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib --with-tune=generic --without-cuda-driver -v
Processor Notes: Scaling Governor: intel_pstate powersave
Disk Scheduler Notes: NONE
Security Notes: KPTI + __user pointer sanitization + Full generic retpoline IBPB IBRS_FW STIBP + SSB disabled via prctl and seccomp + PTE Inversion; VMX: conditional cache flushes SMT vulnerable

FreeBSD 12.0 ZFS vs. ZoL BenchmarksProcessorMotherboardChipsetMemoryDiskGraphicsAudioMonitorNetworkOSKernelCompilerFile-SystemScreen ResolutionDesktopDisplay ServerDisplay DriverOpenGLFreeBSD iX ZoLFreeBSD ZFSFreeBSD UFSUbuntu 18.04 EXT4Ubuntu 18.04 ZoLIntel Xeon E3-1275 v6 @ 3.80GHz (8 Cores)ASUS P10S-M WSIntel Xeon E3-1200 v6/7th Gen Core Registers16384MBSamsung SSD 970 EVO Plus 500GBIntel HD P630Intel Sunrise Point HDAFreeBSD12.0-STABLE (x86_64)Clang 8.0.0 (SVN 356365)zfs800x60012.0-RELEASE (x86_64)Clang 6.0.1 (SVN 335540)ufsIntel Xeon E3-1275 v6 @ 4.20GHz (4 Cores / 8 Threads)ASUS P10S-M WS (4401 BIOS)Intel Xeon E3-1200 v6/7thIntel HD P630 3GB (1150MHz)Realtek ALC1150Acer B286HK + DELL P2415Q2 x Intel I210Ubuntu 18.044.18.0-17-generic (x86_64)GNOME Shell 3.28.3X Server 1.20.1modesetting 1.20.14.5 Mesa 18.2.8GCC 7.3.0ext41920x1080zfsOpenBenchmarking.orgPython Details- FreeBSD iX ZoL: Python 3.6.8Security Details- FreeBSD iX ZoL: KPTI- FreeBSD ZFS: KPTI- FreeBSD UFS: KPTI- Ubuntu 18.04 EXT4: KPTI + __user pointer sanitization + Full generic retpoline IBPB IBRS_FW STIBP + SSB disabled via prctl and seccomp + PTE Inversion; VMX: conditional cache flushes SMT vulnerable - Ubuntu 18.04 ZoL: KPTI + __user pointer sanitization + Full generic retpoline IBPB IBRS_FW STIBP + SSB disabled via prctl and seccomp + PTE Inversion; VMX: conditional cache flushes SMT vulnerable Compiler Details- Ubuntu 18.04 EXT4, Ubuntu 18.04 ZoL: --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++ --enable-libmpx --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-targets=nvptx-none --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib --with-tune=generic --without-cuda-driver -v Disk Details- Ubuntu 18.04 EXT4: NONE / errors=remount-ro,relatime,rwProcessor Details- Ubuntu 18.04 EXT4, Ubuntu 18.04 ZoL: Scaling Governor: intel_pstate powersaveDisk Scheduler Details- Ubuntu 18.04 ZoL: NONE

FreeBSD iX ZoLFreeBSD ZFSFreeBSD UFSUbuntu 18.04 EXT4Ubuntu 18.04 ZoLResult OverviewPhoronix Test Suite100%421%743%1064%1386%BlogBenchIOzoneDbenchBlogBenchDbenchIOzonePostMarkRead4Kb - 512MB - Read Performance1Write64Kb - 512MB - Write PerformanceD.T.P

FreeBSD 12.0 ZFS vs. ZoL Benchmarksdbench: 1dbench: 6iozone: 4Kb - 512MB - Read Performanceiozone: 4Kb - 512MB - Write Performancepostmark: Disk Transaction Performanceblogbench: Readblogbench: Writesqlite: Timed SQLite InsertionsFreeBSD iX ZoLFreeBSD ZFSFreeBSD UFSUbuntu 18.04 EXT4Ubuntu 18.04 ZoL132423248991170761378135417822.38252490251485569471313846341727.2184528434060167237961255659266560.992265644131366975853583587339.35100.66312223789338863329051893227.04OpenBenchmarking.org

Dbench

Dbench is a benchmark designed by the Samba project as a free alternative to netbench, but dbench contains only file-system calls for testing the disk performance. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgMB/s, More Is BetterDbench 4.0Client Count: 1FreeBSD UFSFreeBSD ZFSFreeBSD iX ZoLUbuntu 18.04 ZoLUbuntu 18.04 EXT42004006008001000SE +/- 0.70, N = 3SE +/- 0.15, N = 3SE +/- 0.09, N = 3SE +/- 0.07, N = 3SE +/- 0.83, N = 9845.00252.00132.00100.6660.99

OpenBenchmarking.orgMB/s, More Is BetterDbench 4.0Client Count: 6FreeBSD UFSFreeBSD ZFSFreeBSD iX ZoLUbuntu 18.04 ZoLUbuntu 18.04 EXT46001200180024003000SE +/- 13.00, N = 3SE +/- 0.52, N = 3SE +/- 2.00, N = 3SE +/- 0.34, N = 3SE +/- 3.63, N = 32843490423312226

IOzone

The IOzone benchmark tests the hard disk drive / file-system performance. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgMB/s, More Is BetterIOzone 3.465Record Size: 4Kb - File Size: 512MB - Disk Test: Read PerformanceUbuntu 18.04 EXT4FreeBSD UFSFreeBSD ZFSFreeBSD iX ZoLUbuntu 18.04 ZoL12002400360048006000SE +/- 92.53, N = 12SE +/- 3.91, N = 3SE +/- 22.22, N = 11SE +/- 22.05, N = 3SE +/- 18.29, N = 356444060251424892237

OpenBenchmarking.orgMB/s, More Is BetterIOzone 3.465Record Size: 4Kb - File Size: 512MB - Disk Test: Write PerformanceFreeBSD UFSUbuntu 18.04 EXT4FreeBSD iX ZoLUbuntu 18.04 ZoLFreeBSD ZFS400800120016002000SE +/- 8.99, N = 3SE +/- 18.66, N = 3SE +/- 1.30, N = 3SE +/- 4.04, N = 3SE +/- 13.24, N = 316721313911893855

PostMark

This is a test of NetApp's PostMark benchmark designed to simulate small-file testing similar to the tasks endured by web and mail servers. This test profile will set PostMark to perform 25,000 transactions with 500 files simultaneously with the file sizes ranging between 5 and 512 kilobytes. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgTPS, More Is BetterPostMark 1.51Disk Transaction PerformanceFreeBSD iX ZoLFreeBSD ZFSUbuntu 18.04 EXT4Ubuntu 18.04 ZoLFreeBSD UFS15003000450060007500SE +/- 66.00, N = 3SE +/- 111.44, N = 3SE +/- 59.33, N = 3SE +/- 20.00, N = 3SE +/- 57.70, N = 1570766947669738863796clangclanggccgccclang

BlogBench

BlogBench is designed to replicate the load of a real-world busy file server by stressing the file-system with multiple threads of random reads, writes, and rewrites. The behavior is mimicked of that of a blog by creating blogs with content and pictures, modifying blog posts, adding comments to these blogs, and then reading the content of the blogs. All of these blogs generated are created locally with fake content and pictures. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgFinal Score, More Is BetterBlogBench 1.1Test: ReadFreeBSD iX ZoLFreeBSD ZFSFreeBSD UFSUbuntu 18.04 EXT4Ubuntu 18.04 ZoL300K600K900K1200K1500KSE +/- 24290.33, N = 9SE +/- 12811.27, N = 9SE +/- 59738.49, N = 9SE +/- 14788.62, N = 9SE +/- 6586.79, N = 9137813513138461255659585358332905clangclangclanggccgcc

OpenBenchmarking.orgFinal Score, More Is BetterBlogBench 1.1Test: WriteUbuntu 18.04 EXT4Ubuntu 18.04 ZoLFreeBSD iX ZoLFreeBSD ZFSFreeBSD UFS8K16K24K32K40KSE +/- 845.64, N = 3SE +/- 2614.11, N = 3SE +/- 358.54, N = 3SE +/- 494.73, N = 3SE +/- 581.92, N = 33587318932417834172665gccgccclangclang

SQLite

This is a simple benchmark of SQLite. At present this test profile just measures the time to perform a pre-defined number of insertions on an indexed database. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgSeconds, Fewer Is BetterSQLite 3.22Timed SQLite InsertionsFreeBSD iX ZoLUbuntu 18.04 ZoLFreeBSD ZFSUbuntu 18.04 EXT4918273645SE +/- 0.05, N = 3SE +/- 0.07, N = 3SE +/- 0.04, N = 3SE +/- 0.03, N = 322.3827.0427.2139.35

8 Results Shown

Dbench:
  1
  6
IOzone:
  4Kb - 512MB - Read Performance
  4Kb - 512MB - Write Performance
PostMark
BlogBench:
  Read
  Write
SQLite

FreeBSD iX ZoL

Processor: Intel Xeon E3-1275 v6 @ 3.80GHz (8 Cores), Motherboard: ASUS P10S-M WS, Chipset: Intel Xeon E3-1200 v6/7th Gen Core Registers, Memory: 16384MB, Disk: Samsung SSD 970 EVO Plus 500GB, Graphics: Intel HD P630, Audio: Intel Sunrise Point HDA

OS: FreeBSD, Kernel: 12.0-STABLE (x86_64), Compiler: Clang 8.0.0 (SVN 356365), File-System: zfs, Screen Resolution: 800x600

Python Notes: Python 3.6.8
Security Notes: KPTI

Testing initiated at 22 April 2019 15:37 by user .

FreeBSD ZFS

Processor: Intel Xeon E3-1275 v6 @ 3.80GHz (8 Cores), Motherboard: ASUS P10S-M WS, Chipset: Intel Xeon E3-1200 v6/7th Gen Core Registers, Memory: 16384MB, Disk: Samsung SSD 970 EVO Plus 500GB, Graphics: Intel HD P630, Audio: Intel Sunrise Point HDA

OS: FreeBSD, Kernel: 12.0-RELEASE (x86_64), Compiler: Clang 6.0.1 (SVN 335540), File-System: zfs, Screen Resolution: 800x600

Security Notes: KPTI

Testing initiated at 22 April 2019 19:48 by user .

FreeBSD UFS

Processor: Intel Xeon E3-1275 v6 @ 3.80GHz (8 Cores), Motherboard: ASUS P10S-M WS, Chipset: Intel Xeon E3-1200 v6/7th Gen Core Registers, Memory: 16384MB, Disk: Samsung SSD 970 EVO Plus 500GB, Graphics: Intel HD P630, Audio: Intel Sunrise Point HDA

OS: FreeBSD, Kernel: 12.0-RELEASE (x86_64), Compiler: Clang 6.0.1 (SVN 335540), File-System: ufs, Screen Resolution: 800x600

Security Notes: KPTI

Testing initiated at 22 April 2019 22:26 by user .

Ubuntu 18.04 EXT4

Processor: Intel Xeon E3-1275 v6 @ 4.20GHz (4 Cores / 8 Threads), Motherboard: ASUS P10S-M WS (4401 BIOS), Chipset: Intel Xeon E3-1200 v6/7th, Memory: 16384MB, Disk: Samsung SSD 970 EVO Plus 500GB, Graphics: Intel HD P630 3GB (1150MHz), Audio: Realtek ALC1150, Monitor: Acer B286HK + DELL P2415Q, Network: 2 x Intel I210

OS: Ubuntu 18.04, Kernel: 4.18.0-17-generic (x86_64), Desktop: GNOME Shell 3.28.3, Display Server: X Server 1.20.1, Display Driver: modesetting 1.20.1, OpenGL: 4.5 Mesa 18.2.8, Compiler: GCC 7.3.0, File-System: ext4, Screen Resolution: 1920x1080

Compiler Notes: --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++ --enable-libmpx --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-targets=nvptx-none --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib --with-tune=generic --without-cuda-driver -v
Disk Notes: NONE / errors=remount-ro,relatime,rw
Processor Notes: Scaling Governor: intel_pstate powersave
Security Notes: KPTI + __user pointer sanitization + Full generic retpoline IBPB IBRS_FW STIBP + SSB disabled via prctl and seccomp + PTE Inversion; VMX: conditional cache flushes SMT vulnerable

Testing initiated at 23 April 2019 11:17 by user phoronix.

Ubuntu 18.04 ZoL

Processor: Intel Xeon E3-1275 v6 @ 4.20GHz (4 Cores / 8 Threads), Motherboard: ASUS P10S-M WS (4401 BIOS), Chipset: Intel Xeon E3-1200 v6/7th, Memory: 16384MB, Disk: Samsung SSD 970 EVO Plus 500GB, Graphics: Intel HD P630 3GB (1150MHz), Audio: Realtek ALC1150, Monitor: Acer B286HK + DELL P2415Q, Network: 2 x Intel I210

OS: Ubuntu 18.04, Kernel: 4.18.0-17-generic (x86_64), Desktop: GNOME Shell 3.28.3, Display Server: X Server 1.20.1, Display Driver: modesetting 1.20.1, OpenGL: 4.5 Mesa 18.2.8, Compiler: GCC 7.3.0, File-System: zfs, Screen Resolution: 1920x1080

Compiler Notes: --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++ --enable-libmpx --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-targets=nvptx-none --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib --with-tune=generic --without-cuda-driver -v
Processor Notes: Scaling Governor: intel_pstate powersave
Disk Scheduler Notes: NONE
Security Notes: KPTI + __user pointer sanitization + Full generic retpoline IBPB IBRS_FW STIBP + SSB disabled via prctl and seccomp + PTE Inversion; VMX: conditional cache flushes SMT vulnerable

Testing initiated at 23 April 2019 14:58 by user phoronix.