FreeBSD ZFS SSD RAID vs. Ubuntu Linux RAID Benchmarks

FreeBSD ZFS and Linux RAID storage performance benchmarks for a future article.

Compare your own system(s) to this result file with the Phoronix Test Suite by running the command: phoronix-test-suite benchmark 1812090-SK-ZFSBTRFS470
Jump To Table - Results

View

Do Not Show Noisy Results
Do Not Show Results With Incomplete Data
Do Not Show Results With Little Change/Spread
List Notable Results

Limit displaying results to tests within:

CPU Massive 3 Tests
Database Test Suite 2 Tests
Disk Test Suite 5 Tests
Common Kernel Benchmarks 2 Tests
Server 3 Tests

Statistics

Show Overall Harmonic Mean(s)
Show Overall Geometric Mean
Show Geometric Means Per-Suite/Category
Show Wins / Losses Counts (Pie Chart)
Normalize Results
Remove Outliers Before Calculating Averages

Graph Settings

Force Line Graphs Where Applicable
Convert To Scalar Where Applicable
Prefer Vertical Bar Graphs

Additional Graphs

Show Perf Per Core/Thread Calculation Graphs Where Applicable

Multi-Way Comparison

Condense Multi-Option Tests Into Single Result Graphs

Table

Show Detailed System Result Table

Run Management

Highlight
Result
Hide
Result
Result
Identifier
Performance Per
Dollar
Date
Run
  Test
  Duration
ZFS Single
December 08 2018
  2 Hours, 13 Minutes
ZFS Stripe
December 07 2018
  2 Hours, 4 Minutes
ZFS RAIDZ1
December 07 2018
  2 Hours, 22 Minutes
ZFS RAIDZ3
December 08 2018
  1 Hour, 35 Minutes
ZFS RAID10
December 08 2018
  2 Hours, 2 Minutes
Btrfs Single
December 08 2018
  2 Hours, 51 Minutes
Btrfs RAID10
December 08 2018
  1 Hour, 47 Minutes
Btrfs RAID0
December 08 2018
  1 Hour, 34 Minutes
EXT4 Single
December 08 2018
  1 Hour, 51 Minutes
EXT4 RAID10
December 09 2018
  2 Hours, 31 Minutes
EXT4 RAID0
December 09 2018
  1 Hour, 24 Minutes
Invert Hiding All Results Option
  2 Hours, 1 Minute

Only show results where is faster than
Only show results matching title/arguments (delimit multiple options with a comma):
Do not show results matching title/arguments (delimit multiple options with a comma):


FreeBSD ZFS SSD RAID vs. Ubuntu Linux RAID BenchmarksProcessorMotherboardChipsetMemoryDisk ChipsetGraphicsMonitorNetworkOSKernelCompilerFile-SystemScreen ResolutionDesktopDisplay ServerZFS SingleZFS StripeZFS RAIDZ1ZFS RAIDZ3ZFS RAID10Btrfs SingleBtrfs RAID10Btrfs RAID0EXT4 SingleEXT4 RAID10EXT4 RAID0AMD EPYC 7601 32-Core @ 2.20GHz (128 Cores)Dell PowerEdge R7425AMD [AMD] Family 17h (Models 00h-0fh) Root Complex524032MBATA Samsung SSD 860 1B6Q + DP BP14G+EXP 2.17 + SSDSCKJB120G7R N201DL43FreeBSD12.0-RC3 (x86_64)Clang 6.0.1 (SVN 335540)zfs1024x76820 x ATA Samsung SSD 860 1B6Q + DP BP14G+EXP 2.17 + SSDSCKJB120G7R N201DL43AMD [AMD] Family 17h (Models 00h-0fh) Root Complex20 x ATA Samsung SSD 860 1B6Q + DP BP14G+EXP 2.17 + SSDSCKJB120G7R N201DL432 x AMD EPYC 7601 32-Core (64 Cores / 128 Threads)Dell 02MJ3T (1.2.5 BIOS)AMD Family 17h16 x 32 GB DDR4-2400MT/s 36ASF4G72PZ-2G6D220 x 500GB Samsung SSD 860 + 120GB SSDSCKJB120G7RMatrox G200eW3VE228Broadcom BCM57416 NetXtreme-E Dual-Media 10G RDMAUbuntu 18.104.20.0-999-generic (x86_64) 20181207GNOME Shell 3.30.1X ServerGCC 8.2.0btrfs1600x1200ext4OpenBenchmarking.orgPython Details- ZFS Single, ZFS Stripe, ZFS RAIDZ1, ZFS RAIDZ3: Python 2.7.15 + Python 3.6.6

ZFS SingleZFS StripeZFS RAIDZ1ZFS RAIDZ3ZFS RAID10Btrfs SingleBtrfs RAID10Btrfs RAID0EXT4 SingleEXT4 RAID10EXT4 RAID0Result OverviewPhoronix Test Suite100%300%500%699%899%PostgreSQL pgbenchBlogBenchIOzoneSQLiteFlexible IO TesterCompile BenchPostMark

FreeBSD ZFS SSD RAID vs. Ubuntu Linux RAID Benchmarkssqlite: Timed SQLite Insertionsfio: Rand Write - POSIX AIO - No - Yes - 4KB - Default Test Directoryfio: Seq Read - POSIX AIO - No - Yes - 2MB - Default Test Directoryfio: Seq Read - POSIX AIO - No - Yes - 4KB - Default Test Directoryfio: Seq Write - POSIX AIO - No - Yes - 2MB - Default Test Directoryfio: Seq Write - POSIX AIO - No - Yes - 4KB - Default Test Directoryblogbench: Readblogbench: Writeiozone: 1MB - 8GB - Write Performanceiozone: 4Kb - 8GB - Write Performancecompilebench: Compilecompilebench: Initial Createpostmark: Disk Transaction Performancepgbench: Buffer Test - Normal Load - Read Onlypgbench: Buffer Test - Normal Load - Read Writefio: Seq Read - POSIX AIO - No - Yes - 2MB - Default Test DirectoryZFS SingleZFS StripeZFS RAIDZ1ZFS RAIDZ3ZFS RAID10Btrfs SingleBtrfs RAID10Btrfs RAID0EXT4 SingleEXT4 RAID10EXT4 RAID025.0219218833240200161269320834994881141140431242780214126.1890.87193332456995179161812315630433991045139331842403288035.1492.2518900238808518016337231472587445925136323241943191539.9179.1319133241820717115697030182594419890133326041800187826.5492.17191332416989186160529313030264541043140333342388282890.1970.2336.1736573.278788419292299279223523250345310463536301127.7958.6428.33213360.53841095763413551055222230251735405125414424122.1468.6431.96335770.8087685382361454113022652935068535945572541535.1689.07102.1838694.777558002622026525315683965173533638318038263.2378.0780.23132081.0581458726163913707158239252085322743052113159.1890.4089.30154492.88768694283891507146515733975208536273118331701OpenBenchmarking.org

SQLite

This is a simple benchmark of SQLite. At present this test profile just measures the time to perform a pre-defined number of insertions on an indexed database. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgSeconds, Fewer Is BetterSQLite 3.22Timed SQLite InsertionsZFS SingleZFS StripeZFS RAIDZ1ZFS RAIDZ3ZFS RAID10Btrfs SingleBtrfs RAID10Btrfs RAID0EXT4 SingleEXT4 RAID10EXT4 RAID0306090120150SE +/- 0.03, N = 3SE +/- 0.17, N = 3SE +/- 0.55, N = 3SE +/- 0.27, N = 3SE +/- 0.15, N = 3SE +/- 1.66, N = 12SE +/- 2.52, N = 9SE +/- 0.10, N = 3SE +/- 0.09, N = 3SE +/- 1.16, N = 3SE +/- 0.72, N = 325.0226.1835.1439.9126.5490.19127.79122.1435.1663.2359.18
OpenBenchmarking.orgSeconds, Fewer Is BetterSQLite 3.22Timed SQLite InsertionsZFS SingleZFS StripeZFS RAIDZ1ZFS RAIDZ3ZFS RAID10Btrfs SingleBtrfs RAID10Btrfs RAID0EXT4 SingleEXT4 RAID10EXT4 RAID020406080100Min: 24.97 / Avg: 25.02 / Max: 25.07Min: 25.93 / Avg: 26.18 / Max: 26.51Min: 34.42 / Avg: 35.14 / Max: 36.23Min: 39.46 / Avg: 39.91 / Max: 40.39Min: 26.3 / Avg: 26.54 / Max: 26.83Min: 88.07 / Avg: 90.19 / Max: 108.39Min: 123.7 / Avg: 127.79 / Max: 147.87Min: 121.96 / Avg: 122.14 / Max: 122.31Min: 34.99 / Avg: 35.16 / Max: 35.3Min: 60.93 / Avg: 63.23 / Max: 64.6Min: 58.44 / Avg: 59.18 / Max: 60.62

Flexible IO Tester

Fio is an advanced disk benchmark that depends upon the kernel's AIO access library. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.1Type: Random Write - IO Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test DirectoryZFS SingleZFS StripeZFS RAIDZ1ZFS RAIDZ3ZFS RAID10Btrfs SingleBtrfs RAID10Btrfs RAID0EXT4 SingleEXT4 RAID10EXT4 RAID04080120160200SE +/- 0.54, N = 3SE +/- 1.51, N = 4SE +/- 0.50, N = 3SE +/- 0.92, N = 3SE +/- 0.37, N = 3SE +/- 0.78, N = 12SE +/- 1.01, N = 5SE +/- 0.43, N = 3SE +/- 2.06, N = 11SE +/- 1.37, N = 3192.0090.8792.2579.1392.1770.2358.6468.6489.0778.0790.40
OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.1Type: Random Write - IO Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test DirectoryZFS SingleZFS StripeZFS RAIDZ1ZFS RAIDZ3ZFS RAID10Btrfs SingleBtrfs RAID10Btrfs RAID0EXT4 SingleEXT4 RAID10EXT4 RAID04080120160200Min: 90.1 / Avg: 90.87 / Max: 91.9Min: 89.6 / Avg: 92.25 / Max: 96.5Min: 78.2 / Avg: 79.13 / Max: 79.9Min: 90.6 / Avg: 92.17 / Max: 93.8Min: 69.5 / Avg: 70.23 / Max: 70.7Min: 56.4 / Avg: 58.64 / Max: 64.5Min: 64.7 / Avg: 68.64 / Max: 70.1Min: 88.2 / Avg: 89.07 / Max: 89.5Min: 70.6 / Avg: 78.07 / Max: 94.5Min: 87.7 / Avg: 90.4 / Max: 92.1

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.1Type: Random Write - IO Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test DirectoryZFS SingleZFS StripeZFS RAIDZ1ZFS RAIDZ3ZFS RAID10Btrfs SingleBtrfs RAID10Btrfs RAID0EXT4 SingleEXT4 RAID10EXT4 RAID011K22K33K44K55KSE +/- 33.33, N = 3SE +/- 120.19, N = 3SE +/- 393.70, N = 4SE +/- 145.30, N = 3SE +/- 176.38, N = 3SE +/- 100.00, N = 3SE +/- 198.42, N = 12SE +/- 273.13, N = 5SE +/- 100.00, N = 3SE +/- 525.84, N = 11SE +/- 371.18, N = 34916723267236002026723267180001501717640228001998223133
OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.1Type: Random Write - IO Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test DirectoryZFS SingleZFS StripeZFS RAIDZ1ZFS RAIDZ3ZFS RAID10Btrfs SingleBtrfs RAID10Btrfs RAID0EXT4 SingleEXT4 RAID10EXT4 RAID09K18K27K36K45KMin: 49100 / Avg: 49166.67 / Max: 49200Min: 23100 / Avg: 23266.67 / Max: 23500Min: 22900 / Avg: 23600 / Max: 24700Min: 20000 / Avg: 20266.67 / Max: 20500Min: 23000 / Avg: 23266.67 / Max: 23600Min: 17800 / Avg: 18000 / Max: 18100Min: 14400 / Avg: 15016.67 / Max: 16500Min: 16600 / Avg: 17640 / Max: 18200Min: 22600 / Avg: 22800 / Max: 22900Min: 18100 / Avg: 19981.82 / Max: 24200Min: 22400 / Avg: 23133.33 / Max: 23600

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.1Type: Sequential Read - IO Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 2MB - Disk Target: Default Test DirectoryZFS SingleZFS StripeZFS RAIDZ1ZFS RAIDZ3ZFS RAID104K8K12K16K20KSE +/- 272.85, N = 3SE +/- 33.33, N = 3SE +/- 251.66, N = 3SE +/- 176.38, N = 3SE +/- 120.19, N = 318833193331890019133191331. (CC) clang options: -rdynamic -std=gnu99 -ffast-math -include -O3 -U_FORTIFY_SOURCE -ll -lrdmacm -libverbs -lz -lm -lpthread -lrt
OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.1Type: Sequential Read - IO Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 2MB - Disk Target: Default Test DirectoryZFS SingleZFS StripeZFS RAIDZ1ZFS RAIDZ3ZFS RAID103K6K9K12K15KMin: 18300 / Avg: 18833.33 / Max: 19200Min: 19300 / Avg: 19333.33 / Max: 19400Min: 18400 / Avg: 18900 / Max: 19200Min: 18800 / Avg: 19133.33 / Max: 19400Min: 18900 / Avg: 19133.33 / Max: 193001. (CC) clang options: -rdynamic -std=gnu99 -ffast-math -include -O3 -U_FORTIFY_SOURCE -ll -lrdmacm -libverbs -lz -lm -lpthread -lrt

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.1Type: Sequential Read - IO Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test DirectoryZFS SingleZFS StripeZFS RAIDZ1ZFS RAIDZ3ZFS RAID10Btrfs SingleBtrfs RAID10Btrfs RAID0EXT4 SingleEXT4 RAID10EXT4 RAID050100150200250SE +/- 0.88, N = 3SE +/- 2.00, N = 3SE +/- 1.67, N = 3SE +/- 2.08, N = 3SE +/- 0.64, N = 3SE +/- 0.30, N = 3SE +/- 0.33, N = 12SE +/- 2.48, N = 12SE +/- 1.15, N = 12SE +/- 1.28, N = 12240.00245.00238.00241.00241.0036.1728.3331.96102.1880.2389.30
OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.1Type: Sequential Read - IO Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test DirectoryZFS SingleZFS StripeZFS RAIDZ1ZFS RAIDZ3ZFS RAID10Btrfs SingleBtrfs RAID10Btrfs RAID0EXT4 SingleEXT4 RAID10EXT4 RAID04080120160200Min: 238 / Avg: 239.67 / Max: 241Min: 243 / Avg: 245 / Max: 249Min: 236 / Avg: 237.67 / Max: 241Min: 237 / Avg: 241 / Max: 244Min: 34.9 / Avg: 36.17 / Max: 36.9Min: 27.9 / Avg: 28.33 / Max: 28.9Min: 30.4 / Avg: 31.96 / Max: 34.2Min: 92.8 / Avg: 102.18 / Max: 116Min: 75.8 / Avg: 80.23 / Max: 90.8Min: 83.6 / Avg: 89.3 / Max: 101

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.1Type: Sequential Read - IO Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test DirectoryZFS SingleZFS StripeZFS RAIDZ1ZFS RAIDZ3ZFS RAID10EXT4 SingleEXT4 RAID10EXT4 RAID013K26K39K52K65KSE +/- 233.33, N = 3SE +/- 566.67, N = 3SE +/- 470.22, N = 3SE +/- 145.30, N = 3SE +/- 589.73, N = 3SE +/- 648.11, N = 12SE +/- 294.94, N = 12SE +/- 322.98, N = 126143362767608676163361433260672057522850
OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.1Type: Sequential Read - IO Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test DirectoryZFS SingleZFS StripeZFS RAIDZ1ZFS RAIDZ3ZFS RAID10EXT4 SingleEXT4 RAID10EXT4 RAID011K22K33K44K55KMin: 61000 / Avg: 61433.33 / Max: 61800Min: 62200 / Avg: 62766.67 / Max: 63900Min: 60300 / Avg: 60866.67 / Max: 61800Min: 61400 / Avg: 61633.33 / Max: 61900Min: 60700 / Avg: 61433.33 / Max: 62600Min: 23800 / Avg: 26066.67 / Max: 29800Min: 19400 / Avg: 20575 / Max: 23200Min: 21400 / Avg: 22850 / Max: 25800

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.1Type: Sequential Write - IO Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 2MB - Disk Target: Default Test DirectoryZFS StripeZFS RAIDZ1ZFS RAIDZ3ZFS RAID10Btrfs SingleBtrfs RAID10Btrfs RAID0EXT4 SingleEXT4 RAID10EXT4 RAID02K4K6K8K10KSE +/- 113.76, N = 12SE +/- 72.09, N = 3SE +/- 87.61, N = 12SE +/- 80.57, N = 12SE +/- 7.37, N = 3SE +/- 12.77, N = 3SE +/- 33.94, N = 11SE +/- 1.76, N = 3SE +/- 50.88, N = 10SE +/- 5.46, N = 369958085820769893652133335738613201544
OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.1Type: Sequential Write - IO Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 2MB - Disk Target: Default Test DirectoryZFS StripeZFS RAIDZ1ZFS RAIDZ3ZFS RAID10Btrfs SingleBtrfs RAID10Btrfs RAID0EXT4 SingleEXT4 RAID10EXT4 RAID014002800420056007000Min: 6480 / Avg: 6995.17 / Max: 7621Min: 7948 / Avg: 8085.33 / Max: 8192Min: 7873 / Avg: 8206.67 / Max: 8860Min: 6662 / Avg: 6989.08 / Max: 7446Min: 354 / Avg: 365 / Max: 379Min: 2110 / Avg: 2133.33 / Max: 2154Min: 3105 / Avg: 3356.91 / Max: 3484Min: 383 / Avg: 385.67 / Max: 389Min: 877 / Avg: 1319.7 / Max: 1462Min: 1533 / Avg: 1543.67 / Max: 1551

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.1Type: Sequential Write - IO Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test DirectoryZFS SingleZFS StripeZFS RAIDZ1ZFS RAIDZ3ZFS RAID10Btrfs SingleBtrfs RAID10Btrfs RAID0EXT4 SingleEXT4 RAID10EXT4 RAID04080120160200SE +/- 0.88, N = 3SE +/- 2.00, N = 3SE +/- 0.58, N = 3SE +/- 0.89, N = 3SE +/- 0.58, N = 3SE +/- 1.11, N = 3SE +/- 0.47, N = 3SE +/- 2.14, N = 12SE +/- 2.12, N = 9200.00179.00180.00171.00186.0073.2760.5370.8094.7781.0592.88
OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.1Type: Sequential Write - IO Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test DirectoryZFS SingleZFS StripeZFS RAIDZ1ZFS RAIDZ3ZFS RAID10Btrfs SingleBtrfs RAID10Btrfs RAID0EXT4 SingleEXT4 RAID10EXT4 RAID04080120160200Min: 199 / Avg: 200.33 / Max: 202Min: 167 / Avg: 171 / Max: 173Min: 185 / Avg: 186 / Max: 187Min: 71.5 / Avg: 73.27 / Max: 74.3Min: 59.4 / Avg: 60.53 / Max: 61.3Min: 68.6 / Avg: 70.8 / Max: 72.2Min: 93.9 / Avg: 94.77 / Max: 95.5Min: 70.2 / Avg: 81.05 / Max: 93.1Min: 85.6 / Avg: 92.88 / Max: 107

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.1Type: Sequential Write - IO Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test DirectoryZFS SingleZFS StripeZFS RAIDZ1ZFS RAIDZ3ZFS RAID10Btrfs SingleBtrfs RAID10Btrfs RAID0EXT4 SingleEXT4 RAID10EXT4 RAID011K22K33K44K55KSE +/- 166.67, N = 3SE +/- 145.30, N = 3SE +/- 470.22, N = 3SE +/- 120.19, N = 3SE +/- 272.85, N = 3SE +/- 152.75, N = 3SE +/- 272.85, N = 3SE +/- 120.19, N = 3SE +/- 590.00, N = 12SE +/- 552.21, N = 95126745800461334383347567188331550018133242332065823778
OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.1Type: Sequential Write - IO Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test DirectoryZFS SingleZFS StripeZFS RAIDZ1ZFS RAIDZ3ZFS RAID10Btrfs SingleBtrfs RAID10Btrfs RAID0EXT4 SingleEXT4 RAID10EXT4 RAID09K18K27K36K45KMin: 51100 / Avg: 51266.67 / Max: 51600Min: 45900 / Avg: 46133.33 / Max: 46400Min: 42900 / Avg: 43833.33 / Max: 44400Min: 47400 / Avg: 47566.67 / Max: 47800Min: 18300 / Avg: 18833.33 / Max: 19200Min: 15200 / Avg: 15500 / Max: 15700Min: 17600 / Avg: 18133.33 / Max: 18500Min: 24000 / Avg: 24233.33 / Max: 24400Min: 17000 / Avg: 20658.33 / Max: 23800Min: 21900 / Avg: 23777.78 / Max: 27500

BlogBench

BlogBench is designed to replicate the load of a real-world busy file server by stressing the file-system with multiple threads of random reads, writes, and rewrites. The behavior is mimicked of that of a blog by creating blogs with content and pictures, modifying blog posts, adding comments to these blogs, and then reading the content of the blogs. All of these blogs generated are created locally with fake content and pictures. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgFinal Score, More Is BetterBlogBench 1.1Test: ReadZFS SingleZFS StripeZFS RAIDZ1ZFS RAIDZ3ZFS RAID10Btrfs SingleBtrfs RAID10Btrfs RAID0EXT4 SingleEXT4 RAID10EXT4 RAID0200K400K600K800K1000KSE +/- 2686.95, N = 3SE +/- 1752.00, N = 3SE +/- 2001.90, N = 8SE +/- 1041.60, N = 3SE +/- 1266.85, N = 3SE +/- 7126.36, N = 3SE +/- 14408.83, N = 4SE +/- 11480.70, N = 3SE +/- 5393.35, N = 3SE +/- 6970.23, N = 3SE +/- 7484.94, N = 3161269161812163372156970160529878841841095876853755800814587768694clangclangclangclangclanggccgccgccgccgccgcc
OpenBenchmarking.orgFinal Score, More Is BetterBlogBench 1.1Test: ReadZFS SingleZFS StripeZFS RAIDZ1ZFS RAIDZ3ZFS RAID10Btrfs SingleBtrfs RAID10Btrfs RAID0EXT4 SingleEXT4 RAID10EXT4 RAID0150K300K450K600K750KMin: 157183 / Avg: 161269.33 / Max: 166335Min: 158768 / Avg: 161812.33 / Max: 164837Min: 154322 / Avg: 163371.5 / Max: 171595Min: 155621 / Avg: 156969.67 / Max: 159019Min: 158973 / Avg: 160529.33 / Max: 163039Min: 864897 / Avg: 878840.67 / Max: 888369Min: 798378 / Avg: 841094.75 / Max: 859298Min: 853938 / Avg: 876853 / Max: 889574Min: 749658 / Avg: 755799.67 / Max: 766550Min: 800891 / Avg: 814587 / Max: 823686Min: 757247 / Avg: 768694 / Max: 782772

OpenBenchmarking.orgFinal Score, More Is BetterBlogBench 1.1Test: WriteZFS SingleZFS StripeZFS RAIDZ1ZFS RAIDZ3ZFS RAID10Btrfs SingleBtrfs RAID10Btrfs RAID0EXT4 SingleEXT4 RAID10EXT4 RAID06K12K18K24K30KSE +/- 52.94, N = 3SE +/- 42.15, N = 3SE +/- 90.82, N = 3SE +/- 33.71, N = 3SE +/- 20.88, N = 3SE +/- 100.04, N = 3SE +/- 204.99, N = 3SE +/- 198.91, N = 3SE +/- 387.87, N = 3SE +/- 1151.85, N = 3SE +/- 522.62, N = 332083156314730183130929276348236262202616328389clangclangclangclangclanggccgccgccgccgccgcc
OpenBenchmarking.orgFinal Score, More Is BetterBlogBench 1.1Test: WriteZFS SingleZFS StripeZFS RAIDZ1ZFS RAIDZ3ZFS RAID10Btrfs SingleBtrfs RAID10Btrfs RAID0EXT4 SingleEXT4 RAID10EXT4 RAID05K10K15K20K25KMin: 3120 / Avg: 3208 / Max: 3303Min: 3083 / Avg: 3156.33 / Max: 3229Min: 3047 / Avg: 3146.67 / Max: 3328Min: 2978 / Avg: 3018 / Max: 3085Min: 3107 / Avg: 3130.33 / Max: 3172Min: 9139 / Avg: 9291.67 / Max: 9480Min: 7229 / Avg: 7633.67 / Max: 7893Min: 7852 / Avg: 8236 / Max: 8518Min: 25777 / Avg: 26220 / Max: 26993Min: 24005 / Avg: 26163.33 / Max: 27940Min: 27347 / Avg: 28389.33 / Max: 28978

IOzone

The IOzone benchmark tests the hard disk drive / file-system performance. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgMB/s, More Is BetterIOzone 3.465Record Size: 1MB - File Size: 8GB - Disk Test: Write PerformanceZFS SingleZFS StripeZFS RAIDZ1ZFS RAIDZ3ZFS RAID10Btrfs SingleBtrfs RAID10Btrfs RAID0EXT4 SingleEXT4 RAID10EXT4 RAID08001600240032004000SE +/- 67.16, N = 3SE +/- 21.58, N = 3SE +/- 35.29, N = 3SE +/- 11.48, N = 3SE +/- 30.39, N = 3SE +/- 5.21, N = 3SE +/- 2.21, N = 3SE +/- 3.61, N = 3SE +/- 8.87, N = 12SE +/- 47.64, N = 12SE +/- 3.14, N = 334993043258725943026299135514542659131507
OpenBenchmarking.orgMB/s, More Is BetterIOzone 3.465Record Size: 1MB - File Size: 8GB - Disk Test: Write PerformanceZFS SingleZFS StripeZFS RAIDZ1ZFS RAIDZ3ZFS RAID10Btrfs SingleBtrfs RAID10Btrfs RAID0EXT4 SingleEXT4 RAID10EXT4 RAID06001200180024003000Min: 3398.32 / Avg: 3499.2 / Max: 3626.44Min: 3017.55 / Avg: 3043.01 / Max: 3085.92Min: 2517.91 / Avg: 2586.53 / Max: 2635.13Min: 2570.86 / Avg: 2593.54 / Max: 2608.03Min: 2987.27 / Avg: 3025.82 / Max: 3085.79Min: 290.63 / Avg: 298.54 / Max: 308.36Min: 1351.24 / Avg: 1354.91 / Max: 1358.86Min: 1447.72 / Avg: 1454.45 / Max: 1460.09Min: 247.69 / Avg: 265.22 / Max: 345.14Min: 702.51 / Avg: 912.84 / Max: 1127.21Min: 1501.35 / Avg: 1507.38 / Max: 1511.88

OpenBenchmarking.orgMB/s, More Is BetterIOzone 3.465Record Size: 4Kb - File Size: 8GB - Disk Test: Write PerformanceZFS SingleZFS StripeZFS RAIDZ1ZFS RAIDZ3ZFS RAID10Btrfs SingleBtrfs RAID10Btrfs RAID0EXT4 SingleEXT4 RAID10EXT4 RAID030060090012001500SE +/- 7.91, N = 3SE +/- 4.31, N = 12SE +/- 5.97, N = 3SE +/- 5.15, N = 12SE +/- 7.09, N = 12SE +/- 8.63, N = 12SE +/- 13.12, N = 7SE +/- 0.69, N = 3SE +/- 0.05, N = 3SE +/- 39.02, N = 12SE +/- 2.54, N = 3488399445419454279105511302537071465
OpenBenchmarking.orgMB/s, More Is BetterIOzone 3.465Record Size: 4Kb - File Size: 8GB - Disk Test: Write PerformanceZFS SingleZFS StripeZFS RAIDZ1ZFS RAIDZ3ZFS RAID10Btrfs SingleBtrfs RAID10Btrfs RAID0EXT4 SingleEXT4 RAID10EXT4 RAID030060090012001500Min: 475.13 / Avg: 488.3 / Max: 502.48Min: 374.6 / Avg: 398.54 / Max: 416.23Min: 433.55 / Avg: 445.48 / Max: 451.6Min: 398.29 / Avg: 419.44 / Max: 445.34Min: 416.73 / Avg: 454.31 / Max: 491.44Min: 259.99 / Avg: 279.37 / Max: 336.81Min: 987.53 / Avg: 1055.11 / Max: 1078.64Min: 1129.52 / Avg: 1130.22 / Max: 1131.6Min: 253.37 / Avg: 253.43 / Max: 253.54Min: 529.87 / Avg: 706.54 / Max: 1062.97Min: 1459.94 / Avg: 1464.79 / Max: 1468.55

Compile Bench

Compilebench tries to age a filesystem by simulating some of the disk IO common in creating, compiling, patching, stating and reading kernel trees. It indirectly measures how well filesystems can maintain directory locality as the disk fills up and directories age. This current test is setup to use the makej mode with 10 initial directories Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgMB/s, More Is BetterCompile Bench 0.6Test: CompileZFS SingleZFS StripeZFS RAIDZ1ZFS RAIDZ3ZFS RAID10Btrfs SingleBtrfs RAID10Btrfs RAID0EXT4 SingleEXT4 RAID10EXT4 RAID05001000150020002500SE +/- 6.65, N = 3SE +/- 13.52, N = 3SE +/- 15.82, N = 4SE +/- 5.79, N = 3SE +/- 4.57, N = 3SE +/- 26.00, N = 12SE +/- 23.47, N = 3SE +/- 25.78, N = 12SE +/- 10.55, N = 3SE +/- 11.73, N = 3SE +/- 5.33, N = 3114110459258901043223522222265156815821573
OpenBenchmarking.orgMB/s, More Is BetterCompile Bench 0.6Test: CompileZFS SingleZFS StripeZFS RAIDZ1ZFS RAIDZ3ZFS RAID10Btrfs SingleBtrfs RAID10Btrfs RAID0EXT4 SingleEXT4 RAID10EXT4 RAID0400800120016002000Min: 1131.81 / Avg: 1141.33 / Max: 1154.14Min: 1027.81 / Avg: 1045.3 / Max: 1071.91Min: 898.69 / Avg: 924.53 / Max: 965.65Min: 878.64 / Avg: 889.55 / Max: 898.35Min: 1037.17 / Avg: 1043.18 / Max: 1052.15Min: 2089.9 / Avg: 2234.69 / Max: 2397.93Min: 2176.38 / Avg: 2221.56 / Max: 2255.15Min: 2155.3 / Avg: 2265.45 / Max: 2382.28Min: 1550.49 / Avg: 1567.5 / Max: 1586.81Min: 1561.29 / Avg: 1582.13 / Max: 1601.89Min: 1564.88 / Avg: 1573.45 / Max: 1583.21

OpenBenchmarking.orgMB/s, More Is BetterCompile Bench 0.6Test: Initial CreateZFS SingleZFS StripeZFS RAIDZ1ZFS RAIDZ3ZFS RAID10Btrfs SingleBtrfs RAID10Btrfs RAID0EXT4 SingleEXT4 RAID10EXT4 RAID090180270360450SE +/- 0.48, N = 3SE +/- 0.07, N = 3SE +/- 0.55, N = 3SE +/- 0.56, N = 3SE +/- 0.76, N = 3SE +/- 4.88, N = 3SE +/- 7.97, N = 3SE +/- 1.07, N = 3SE +/- 2.88, N = 3SE +/- 4.11, N = 3SE +/- 2.27, N = 3140139136133140232302293396392397
OpenBenchmarking.orgMB/s, More Is BetterCompile Bench 0.6Test: Initial CreateZFS SingleZFS StripeZFS RAIDZ1ZFS RAIDZ3ZFS RAID10Btrfs SingleBtrfs RAID10Btrfs RAID0EXT4 SingleEXT4 RAID10EXT4 RAID070140210280350Min: 139.41 / Avg: 140.29 / Max: 141.06Min: 139.14 / Avg: 139.27 / Max: 139.34Min: 134.65 / Avg: 135.73 / Max: 136.4Min: 131.44 / Avg: 132.56 / Max: 133.15Min: 138.13 / Avg: 139.57 / Max: 140.71Min: 224.36 / Avg: 231.87 / Max: 241.01Min: 286.75 / Avg: 301.66 / Max: 314.02Min: 291 / Avg: 293.13 / Max: 294.4Min: 392.94 / Avg: 396.48 / Max: 402.18Min: 385.6 / Avg: 391.59 / Max: 399.46Min: 394.31 / Avg: 396.92 / Max: 401.43

PostMark

This is a test of NetApp's PostMark benchmark designed to simulate small-file testing similar to the tasks endured by web and mail servers. This test profile will set PostMark to perform 25,000 transactions with 500 files simultaneously with the file sizes ranging between 5 and 512 kilobytes. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgTPS, More Is BetterPostMark 1.51Disk Transaction PerformanceZFS SingleZFS StripeZFS RAIDZ1ZFS RAIDZ3ZFS RAID10Btrfs SingleBtrfs RAID10Btrfs RAID0EXT4 SingleEXT4 RAID10EXT4 RAID011002200330044005500SE +/- 99.00, N = 9SE +/- 14.67, N = 3SE +/- 13.67, N = 3SE +/- 14.33, N = 3SE +/- 34.00, N = 3SE +/- 35.33, N = 3SE +/- 34.00, N = 3SE +/- 35.33, N = 343123318323232603333503451735068517352085208clangclangclangclangclanggccgccgccgccgccgcc
OpenBenchmarking.orgTPS, More Is BetterPostMark 1.51Disk Transaction PerformanceZFS SingleZFS StripeZFS RAIDZ1ZFS RAIDZ3ZFS RAID10Btrfs SingleBtrfs RAID10Btrfs RAID0EXT4 SingleEXT4 RAID10EXT4 RAID09001800270036004500Min: 3787 / Avg: 4312.11 / Max: 4716Min: 3289 / Avg: 3318.33 / Max: 3333Min: 3205 / Avg: 3232.33 / Max: 3246Min: 3246 / Avg: 3260.33 / Max: 3289Min: 5000 / Avg: 5034 / Max: 5102Min: 5102 / Avg: 5172.67 / Max: 5208Min: 5000 / Avg: 5068 / Max: 5102Min: 5102 / Avg: 5172.67 / Max: 5208

PostgreSQL pgbench

This is a simple benchmark of PostgreSQL using pgbench. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgTPS, More Is BetterPostgreSQL pgbench 10.3Scaling: Buffer Test - Test: Normal Load - Mode: Read OnlyZFS SingleZFS StripeZFS RAIDZ1ZFS RAIDZ3ZFS RAID10Btrfs SingleBtrfs RAID10Btrfs RAID0EXT4 SingleEXT4 RAID10EXT4 RAID0120K240K360K480K600KSE +/- 280.54, N = 3SE +/- 259.57, N = 3SE +/- 113.65, N = 3SE +/- 64.59, N = 3SE +/- 311.71, N = 3SE +/- 1958.73, N = 3SE +/- 4320.38, N = 3SE +/- 9821.82, N = 3SE +/- 10599.09, N = 3SE +/- 8283.17, N = 3SE +/- 3156.55, N = 34278042403419434180042388531046540512535945533638532274536273
OpenBenchmarking.orgTPS, More Is BetterPostgreSQL pgbench 10.3Scaling: Buffer Test - Test: Normal Load - Mode: Read OnlyZFS SingleZFS StripeZFS RAIDZ1ZFS RAIDZ3ZFS RAID10Btrfs SingleBtrfs RAID10Btrfs RAID0EXT4 SingleEXT4 RAID10EXT4 RAID090K180K270K360K450KMin: 42297.2 / Avg: 42780.31 / Max: 43268.98Min: 41944.05 / Avg: 42402.56 / Max: 42842.65Min: 41806.43 / Avg: 41942.71 / Max: 42168.39Min: 41698.75 / Avg: 41799.89 / Max: 41920.05Min: 41782.68 / Avg: 42387.73 / Max: 42820.37Min: 528542.31 / Avg: 531045.58 / Max: 534906.84Min: 533213.59 / Avg: 540512.1 / Max: 548167.09Min: 522387.59 / Avg: 535944.58 / Max: 555034.07Min: 520517.57 / Avg: 533638.38 / Max: 554617.7Min: 520777.95 / Avg: 532274.4 / Max: 548352.51Min: 529974.25 / Avg: 536273.44 / Max: 539785.64

OpenBenchmarking.orgTPS, More Is BetterPostgreSQL pgbench 10.3Scaling: Buffer Test - Test: Normal Load - Mode: Read WriteZFS SingleZFS StripeZFS RAIDZ1ZFS RAIDZ3ZFS RAID10Btrfs SingleBtrfs RAID10Btrfs RAID0EXT4 SingleEXT4 RAID10EXT4 RAID03K6K9K12K15KSE +/- 32.07, N = 9SE +/- 68.03, N = 8SE +/- 24.99, N = 7SE +/- 32.38, N = 3SE +/- 33.38, N = 8SE +/- 224.33, N = 9SE +/- 48.84, N = 3SE +/- 75.55, N = 3SE +/- 47.94, N = 3SE +/- 157.77, N = 9SE +/- 88.59, N = 3214128801915187828283536541457253180305211833
OpenBenchmarking.orgTPS, More Is BetterPostgreSQL pgbench 10.3Scaling: Buffer Test - Test: Normal Load - Mode: Read WriteZFS SingleZFS StripeZFS RAIDZ1ZFS RAIDZ3ZFS RAID10Btrfs SingleBtrfs RAID10Btrfs RAID0EXT4 SingleEXT4 RAID10EXT4 RAID02K4K6K8K10KMin: 1958.16 / Avg: 2140.63 / Max: 2277.48Min: 2534.41 / Avg: 2879.91 / Max: 3073.92Min: 1849.61 / Avg: 1914.86 / Max: 2011.46Min: 1820.9 / Avg: 1878.28 / Max: 1932.97Min: 2697.7 / Avg: 2827.76 / Max: 2970.88Min: 2696.72 / Avg: 3535.86 / Max: 4831.26Min: 5319.87 / Avg: 5413.98 / Max: 5483.66Min: 5637.71 / Avg: 5724.56 / Max: 5875.07Min: 3093.4 / Avg: 3179.54 / Max: 3259.06Min: 2589.76 / Avg: 3052.23 / Max: 4168.95Min: 11655.42 / Avg: 11832.6 / Max: 11922.4

Flexible IO Tester

Fio is an advanced disk benchmark that depends upon the kernel's AIO access library. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.1Type: Sequential Read - IO Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 2MB - Disk Target: Default Test DirectoryBtrfs SingleBtrfs RAID10Btrfs RAID0EXT4 SingleEXT4 RAID10EXT4 RAID0400800120016002000SE +/- 2.65, N = 3SE +/- 29.83, N = 12SE +/- 24.87, N = 12SE +/- 10.91, N = 12301424415382113117011. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -U_FORTIFY_SOURCE -lrt -laio -lm -lpthread -ldl
OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.1Type: Sequential Read - IO Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 2MB - Disk Target: Default Test DirectoryBtrfs SingleBtrfs RAID10Btrfs RAID0EXT4 SingleEXT4 RAID10EXT4 RAID030060090012001500Min: 296 / Avg: 301 / Max: 305Min: 330 / Avg: 423.67 / Max: 586Min: 319 / Avg: 414.67 / Max: 545Min: 1091 / Avg: 1131 / Max: 12191. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -U_FORTIFY_SOURCE -lrt -laio -lm -lpthread -ldl