Btrfs RAID 0/1/5/6/10 Disk Testing

Btrfs RAID benchmarks with four Intel SSDs using RAID 0, 1, 5, 6, and 10 levels on Linux 3.18 kernel. Benchmarks by Michael Larabel for a future article on Phoronix.com.

Compare your own system(s) to this result file with the Phoronix Test Suite by running the command: phoronix-test-suite benchmark 1410291-LI-BTRFSRAID01
Jump To Table - Results

View

Do Not Show Noisy Results
Do Not Show Results With Incomplete Data
Do Not Show Results With Little Change/Spread
List Notable Results
Show Result Confidence Charts
Allow Limiting Results To Certain Suite(s)

Statistics

Show Overall Harmonic Mean(s)
Show Overall Geometric Mean
Show Wins / Losses Counts (Pie Chart)
Normalize Results
Remove Outliers Before Calculating Averages

Graph Settings

Force Line Graphs Where Applicable
Convert To Scalar Where Applicable
Prefer Vertical Bar Graphs

Multi-Way Comparison

Condense Multi-Option Tests Into Single Result Graphs

Table

Show Detailed System Result Table

Run Management

Highlight
Result
Toggle/Hide
Result
Result
Identifier
View Logs
Performance Per
Dollar
Date
Run
  Test
  Duration
Single Disk
October 28 2014
 
4 Disks, Non-RAID
October 28 2014
 
RAID 0
October 28 2014
 
RAID 1
October 28 2014
 
RAID 1+0
October 28 2014
 
RAID 5
October 28 2014
 
RAID 6
October 29 2014
 
Invert Behavior (Only Show Selected Data)
 

Only show results where is faster than
Only show results matching title/arguments (delimit multiple options with a comma):
Do not show results matching title/arguments (delimit multiple options with a comma):

Single Disk

Compiler Notes: --build=x86_64-linux-gnu --disable-browser-plugin --disable-vtable-verify --disable-werror --enable-checking=release --enable-clocale=gnu --enable-gnu-unique-object --enable-gtk-cairo --enable-java-awt=gtk --enable-java-home --enable-languages=c,c++,java,go,d,fortran,objc,obj-c++ --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-arch-directory=amd64 --with-multilib-list=m32,m64,mx32 --with-tune=generic -v
Disk Notes: DEADLINE / relatime,rw,space_cache,ssd
Processor Notes: Scaling Governor: intel_pstate powersave
System Notes: Disk Scheduler: DEADLINE. Python 2.7.8.

4 Disks, Non-RAID

RAID 0

RAID 1

RAID 1+0

RAID 5

RAID 6

Processor: Intel Core i7-5960X @ 4.00GHz (16 Cores), Motherboard: Gigabyte X99-UD4-CF, Chipset: Intel Xeon E5 v3/Core, Memory: 16384MB, Disk: 4 x 120GB INTEL SSDSC2BW12 + 128GB Crucial_CT128MX1, Graphics: XFX AMD Radeon HD 7950/8950 OEM / R9 280 3072MB, Audio: Intel C610/X99, Monitor: SyncMaster, Network: Intel Connection

OS: Ubuntu 14.10, Kernel: 3.18.0-031800rc1-generic (x86_64), Desktop: Unity 7.3.1, Display Server: X Server 1.16.0, Display Driver: radeon 7.4.0, OpenGL: 3.3 Mesa 10.3.0 Gallium 0.4, Compiler: GCC 4.9.1, File-System: btrfs, Screen Resolution: 2560x1600

Btrfs RAID 0/1/5/6/10 Disk TestingOpenBenchmarking.orgPhoronix Test SuiteIntel Core i7-5960X @ 4.00GHz (16 Cores)Gigabyte X99-UD4-CFIntel Xeon E5 v3/Core16384MB4 x 120GB INTEL SSDSC2BW12 + 128GB Crucial_CT128MX1XFX AMD Radeon HD 7950/8950 OEM / R9 280 3072MBIntel C610/X99SyncMasterIntel ConnectionUbuntu 14.103.18.0-031800rc1-generic (x86_64)Unity 7.3.1X Server 1.16.0radeon 7.4.03.3 Mesa 10.3.0 Gallium 0.4GCC 4.9.1btrfs2560x1600ProcessorMotherboardChipsetMemoryDiskGraphicsAudioMonitorNetworkOSKernelDesktopDisplay ServerDisplay DriverOpenGLCompilerFile-SystemScreen ResolutionBtrfs RAID 0/1/5/6/10 Disk Testing BenchmarksSystem Logs- --build=x86_64-linux-gnu --disable-browser-plugin --disable-vtable-verify --disable-werror --enable-checking=release --enable-clocale=gnu --enable-gnu-unique-object --enable-gtk-cairo --enable-java-awt=gtk --enable-java-home --enable-languages=c,c++,java,go,d,fortran,objc,obj-c++ --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-arch-directory=amd64 --with-multilib-list=m32,m64,mx32 --with-tune=generic -v - DEADLINE / relatime,rw,space_cache,ssd- Scaling Governor: intel_pstate powersave- Disk Scheduler: DEADLINE. Python 2.7.8.

Single Disk4 Disks, Non-RAIDRAID 0RAID 1RAID 1+0RAID 5RAID 6Result OverviewPhoronix Test Suite100%175%250%325%400%Flexible IO TesterFlexible IO TesterFlexible IO TesterFlexible IO TesterIOzoneCompile BenchFS-MarkFS-MarkFS-MarkSeq Read - Libaio - No - 16KB - MB/sRand Write - Libaio - No - 16KB - MB/sRand Read - Libaio - No - 16KB - MB/sSeq Write - Libaio - No - 16KB - MB/s4Kb - 8GB - Write PerformanceInitial Create5.F.1.S.4.T1.F.1.S4.F.3.S.D.1.S

Btrfs RAID 0/1/5/6/10 Disk Testingfio: Rand Read - Libaio - No - 16KB - Default Test Directory - MB/sfio: Rand Write - Libaio - No - 16KB - Default Test Directory - MB/sfio: Seq Read - Libaio - No - 16KB - Default Test Directory - MB/sfio: Seq Write - Libaio - No - 16KB - Default Test Directory - MB/sfs-mark: 1000 Files, 1MB Sizefs-mark: 5000 Files, 1MB Size, 4 Threadsfs-mark: 4000 Files, 32 Sub Dirs, 1MB Sizeiozone: 4Kb - 8GB - Write Performancecompilebench: Initial CreateSingle Disk4 Disks, Non-RAIDRAID 0RAID 1RAID 1+0RAID 5RAID 6236.89170.35239.98169.4040.5379.9340.80481.77162.15787.97436.31796.84344.8841.2084.4041.031133.91176.58758.26497.23824.97319.4141.4383.7340.731142.43206.74233.00128.00323.80130.3235.1069.0335.63584.85187.72469.10267.48450.78272.1938.3377.0738.50599.32177.54742.06248.93869.68242.5538.8779.8739.17876.77189.59653.54214.56960.89208.9638.2075.4737.73598.43195.02OpenBenchmarking.org

Flexible IO Tester

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 2.1.11Type: Random Read - IO Engine: Libaio - Buffered: No - Block Size: 16KB - Disk Target: Default Test Directory - Result: MB/sSingle Disk4 Disks, Non-RAIDRAID 0RAID 1RAID 1+0RAID 5RAID 62004006008001000SE +/- 2.80, N = 3SE +/- 40.14, N = 6SE +/- 23.17, N = 6SE +/- 0.84, N = 3SE +/- 10.47, N = 6SE +/- 52.17, N = 6SE +/- 67.86, N = 6236.89787.97758.26233.00469.10742.06653.541. (CC) gcc options: -rdynamic -std=gnu99 -O3 -ffast-math -include -lrt -laio -lz -lm -lpthread -ldl

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 2.1.11Type: Random Write - IO Engine: Libaio - Buffered: No - Block Size: 16KB - Disk Target: Default Test Directory - Result: MB/sSingle Disk4 Disks, Non-RAIDRAID 0RAID 1RAID 1+0RAID 5RAID 6110220330440550SE +/- 0.33, N = 3SE +/- 16.68, N = 6SE +/- 13.92, N = 6SE +/- 0.39, N = 3SE +/- 2.33, N = 3SE +/- 3.37, N = 6SE +/- 2.95, N = 6170.35436.31497.23128.00267.48248.93214.561. (CC) gcc options: -rdynamic -std=gnu99 -O3 -ffast-math -include -lrt -laio -lz -lm -lpthread -ldl

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 2.1.11Type: Sequential Read - IO Engine: Libaio - Buffered: No - Block Size: 16KB - Disk Target: Default Test Directory - Result: MB/sSingle Disk4 Disks, Non-RAIDRAID 0RAID 1RAID 1+0RAID 5RAID 62004006008001000SE +/- 3.46, N = 3SE +/- 16.37, N = 6SE +/- 22.85, N = 6SE +/- 0.96, N = 3SE +/- 1.47, N = 3SE +/- 40.14, N = 6SE +/- 8.58, N = 3239.98796.84824.97323.80450.78869.68960.891. (CC) gcc options: -rdynamic -std=gnu99 -O3 -ffast-math -include -lrt -laio -lz -lm -lpthread -ldl

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 2.1.11Type: Sequential Write - IO Engine: Libaio - Buffered: No - Block Size: 16KB - Disk Target: Default Test Directory - Result: MB/sSingle Disk4 Disks, Non-RAIDRAID 0RAID 1RAID 1+0RAID 5RAID 670140210280350SE +/- 0.04, N = 3SE +/- 5.97, N = 6SE +/- 3.82, N = 3SE +/- 2.46, N = 6SE +/- 0.90, N = 3SE +/- 0.41, N = 3SE +/- 1.21, N = 3169.40344.88319.41130.32272.19242.55208.961. (CC) gcc options: -rdynamic -std=gnu99 -O3 -ffast-math -include -lrt -laio -lz -lm -lpthread -ldl

FS-Mark

OpenBenchmarking.orgFiles/s, More Is BetterFS-Mark 3.3Test: 1000 Files, 1MB SizeSingle Disk4 Disks, Non-RAIDRAID 0RAID 1RAID 1+0RAID 5RAID 6918273645SE +/- 0.07, N = 3SE +/- 0.21, N = 3SE +/- 0.28, N = 3SE +/- 0.06, N = 3SE +/- 0.32, N = 3SE +/- 0.17, N = 3SE +/- 0.06, N = 340.5341.2041.4335.1038.3338.8738.201. (CC) gcc options: -static

OpenBenchmarking.orgFiles/s, More Is BetterFS-Mark 3.3Test: 5000 Files, 1MB Size, 4 ThreadsSingle Disk4 Disks, Non-RAIDRAID 0RAID 1RAID 1+0RAID 5RAID 620406080100SE +/- 0.07, N = 3SE +/- 0.23, N = 3SE +/- 0.35, N = 3SE +/- 0.12, N = 3SE +/- 0.13, N = 3SE +/- 0.13, N = 3SE +/- 0.27, N = 379.9384.4083.7369.0377.0779.8775.471. (CC) gcc options: -static

OpenBenchmarking.orgFiles/s, More Is BetterFS-Mark 3.3Test: 4000 Files, 32 Sub Dirs, 1MB SizeSingle Disk4 Disks, Non-RAIDRAID 0RAID 1RAID 1+0RAID 5RAID 6918273645SE +/- 0.06, N = 3SE +/- 0.12, N = 3SE +/- 0.03, N = 3SE +/- 0.03, N = 3SE +/- 0.25, N = 3SE +/- 0.28, N = 3SE +/- 0.07, N = 340.8041.0340.7335.6338.5039.1737.731. (CC) gcc options: -static

IOzone

The IOzone benchmark tests the hard disk drive / file-system performance. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgMB/s, More Is BetterIOzone 3.405Record Size: 4Kb - File Size: 8GB - Disk Test: Write PerformanceSingle Disk4 Disks, Non-RAIDRAID 0RAID 1RAID 1+0RAID 5RAID 62004006008001000SE +/- 0.08, N = 3SE +/- 0.84, N = 3SE +/- 1.16, N = 3SE +/- 3.64, N = 3SE +/- 0.36, N = 3SE +/- 0.45, N = 3SE +/- 0.24, N = 3481.771133.911142.43584.85599.32876.77598.431. (CC) gcc options: -O3

Compile Bench

Compilebench tries to age a filesystem by simulating some of the disk IO common in creating, compiling, patching, stating and reading kernel trees. It indirectly measures how well filesystems can maintain directory locality as the disk fills up and directories age. This current test is setup to use the makej mode with 10 initial directories Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgMB/s, More Is BetterCompile Bench 0.6Test: Initial CreateSingle Disk4 Disks, Non-RAIDRAID 0RAID 1RAID 1+0RAID 5RAID 650100150200250SE +/- 3.45, N = 3SE +/- 0.57, N = 3SE +/- 3.10, N = 3SE +/- 9.02, N = 3SE +/- 0.81, N = 3SE +/- 9.32, N = 3SE +/- 2.89, N = 3162.15176.58206.74187.72177.54189.59195.02

Single Disk

Compiler Notes: --build=x86_64-linux-gnu --disable-browser-plugin --disable-vtable-verify --disable-werror --enable-checking=release --enable-clocale=gnu --enable-gnu-unique-object --enable-gtk-cairo --enable-java-awt=gtk --enable-java-home --enable-languages=c,c++,java,go,d,fortran,objc,obj-c++ --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-arch-directory=amd64 --with-multilib-list=m32,m64,mx32 --with-tune=generic -v
Disk Notes: DEADLINE / relatime,rw,space_cache,ssd
Processor Notes: Scaling Governor: intel_pstate powersave
System Notes: Disk Scheduler: DEADLINE. Python 2.7.8.

Testing initiated at 28 October 2014 13:02 by user phoronix.

4 Disks, Non-RAID

Compiler Notes: --build=x86_64-linux-gnu --disable-browser-plugin --disable-vtable-verify --disable-werror --enable-checking=release --enable-clocale=gnu --enable-gnu-unique-object --enable-gtk-cairo --enable-java-awt=gtk --enable-java-home --enable-languages=c,c++,java,go,d,fortran,objc,obj-c++ --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-arch-directory=amd64 --with-multilib-list=m32,m64,mx32 --with-tune=generic -v
Disk Notes: DEADLINE / relatime,rw,space_cache,ssd
Processor Notes: Scaling Governor: intel_pstate powersave
System Notes: Disk Scheduler: DEADLINE. Python 2.7.8.

Testing initiated at 28 October 2014 15:29 by user phoronix.

RAID 0

Compiler Notes: --build=x86_64-linux-gnu --disable-browser-plugin --disable-vtable-verify --disable-werror --enable-checking=release --enable-clocale=gnu --enable-gnu-unique-object --enable-gtk-cairo --enable-java-awt=gtk --enable-java-home --enable-languages=c,c++,java,go,d,fortran,objc,obj-c++ --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-arch-directory=amd64 --with-multilib-list=m32,m64,mx32 --with-tune=generic -v
Disk Notes: DEADLINE / relatime,rw,space_cache,ssd
Processor Notes: Scaling Governor: intel_pstate powersave
System Notes: Disk Scheduler: DEADLINE. Python 2.7.8.

Testing initiated at 28 October 2014 17:19 by user phoronix.

RAID 1

Compiler Notes: --build=x86_64-linux-gnu --disable-browser-plugin --disable-vtable-verify --disable-werror --enable-checking=release --enable-clocale=gnu --enable-gnu-unique-object --enable-gtk-cairo --enable-java-awt=gtk --enable-java-home --enable-languages=c,c++,java,go,d,fortran,objc,obj-c++ --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-arch-directory=amd64 --with-multilib-list=m32,m64,mx32 --with-tune=generic -v
Disk Notes: DEADLINE / relatime,rw,space_cache,ssd
Processor Notes: Scaling Governor: intel_pstate powersave
System Notes: Disk Scheduler: DEADLINE. Python 2.7.8.

Testing initiated at 28 October 2014 19:14 by user phoronix.

RAID 1+0

Compiler Notes: --build=x86_64-linux-gnu --disable-browser-plugin --disable-vtable-verify --disable-werror --enable-checking=release --enable-clocale=gnu --enable-gnu-unique-object --enable-gtk-cairo --enable-java-awt=gtk --enable-java-home --enable-languages=c,c++,java,go,d,fortran,objc,obj-c++ --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-arch-directory=amd64 --with-multilib-list=m32,m64,mx32 --with-tune=generic -v
Disk Notes: DEADLINE / relatime,rw,space_cache,ssd
Processor Notes: Scaling Governor: intel_pstate powersave
System Notes: Disk Scheduler: DEADLINE. Python 2.7.8.

Testing initiated at 28 October 2014 20:45 by user phoronix.

RAID 5

Compiler Notes: --build=x86_64-linux-gnu --disable-browser-plugin --disable-vtable-verify --disable-werror --enable-checking=release --enable-clocale=gnu --enable-gnu-unique-object --enable-gtk-cairo --enable-java-awt=gtk --enable-java-home --enable-languages=c,c++,java,go,d,fortran,objc,obj-c++ --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-arch-directory=amd64 --with-multilib-list=m32,m64,mx32 --with-tune=generic -v
Disk Notes: DEADLINE / relatime,rw,space_cache,ssd
Processor Notes: Scaling Governor: intel_pstate powersave
System Notes: Disk Scheduler: DEADLINE. Python 2.7.8.

Testing initiated at 28 October 2014 22:12 by user phoronix.

RAID 6

Processor: Intel Core i7-5960X @ 4.00GHz (16 Cores), Motherboard: Gigabyte X99-UD4-CF, Chipset: Intel Xeon E5 v3/Core, Memory: 16384MB, Disk: 4 x 120GB INTEL SSDSC2BW12 + 128GB Crucial_CT128MX1, Graphics: XFX AMD Radeon HD 7950/8950 OEM / R9 280 3072MB, Audio: Intel C610/X99, Monitor: SyncMaster, Network: Intel Connection

OS: Ubuntu 14.10, Kernel: 3.18.0-031800rc1-generic (x86_64), Desktop: Unity 7.3.1, Display Server: X Server 1.16.0, Display Driver: radeon 7.4.0, OpenGL: 3.3 Mesa 10.3.0 Gallium 0.4, Compiler: GCC 4.9.1, File-System: btrfs, Screen Resolution: 2560x1600

Compiler Notes: --build=x86_64-linux-gnu --disable-browser-plugin --disable-vtable-verify --disable-werror --enable-checking=release --enable-clocale=gnu --enable-gnu-unique-object --enable-gtk-cairo --enable-java-awt=gtk --enable-java-home --enable-languages=c,c++,java,go,d,fortran,objc,obj-c++ --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-arch-directory=amd64 --with-multilib-list=m32,m64,mx32 --with-tune=generic -v
Disk Notes: DEADLINE / relatime,rw,space_cache,ssd
Processor Notes: Scaling Governor: intel_pstate powersave
System Notes: Disk Scheduler: DEADLINE. Python 2.7.8.

Testing initiated at 29 October 2014 07:46 by user phoronix.