zfs ssd mirror, ashift=13, recordsize 64k, qcow2 image Compiler Notes: --build=x86_64-linux-gnu --disable-browser-plugin --disable-vtable-verify --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-gtk-cairo --enable-java-awt=gtk --enable-java-home --enable-languages=c,ada,c++,java,go,d,fortran,objc,obj-c++ --enable-libmpx --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-arch-directory=amd64 --with-default-libstdcxx-abi=new --with-multilib-list=m32,m64,mx32 --with-target-system-zlib --with-tune=generic -vDisk Notes: CFQ / data=ordered,errors=remount-ro,relatime,rwSystem Notes: Python 2.7.13.
zfs ssd mirror, ashift=13, recordsize 64k, raw image zfs ssd mirror, ashift=13, recordsize 64k, zvol zfs ssd mirror, ashift=13, recordsize 8k, qcow2 image zfs ssd mirror, ashift=13, recordsize 8k, raw image zfs ssd mirror, ashift=13, recordsize 8k, zvol Processor: 8 x Intel Celeron_4x0 (Conroe/Merom Class Core 2) @ 1.86GHz (8 Cores), Motherboard: QEMU Standard PC (i440FX + PIIX 1996), Chipset: Intel 440FX- 82441FX PMC, Memory: 1 x 1024 MB RAM QEMU, Disk: 54GB QEMU HDD, Graphics: Cirrus Logic GD 5446, Network: Realtek RTL-8100/8101L/8139
OS: Debian 9.1, Kernel: 4.9.0-4-amd64 (x86_64), Compiler: GCC 6.3.0 20170516, File-System: ext4, Screen Resolution: 1024x768, System Layer: qemu
pve-ubuntu-local-1 Processor: Common KVM @ 3.09GHz (2 Cores) , Motherboard: QEMU Standard PC (i440FX + PIIX 1996), Chipset: Intel 440FX- 82441FX PMC, Memory: 2048MB , Disk: 34GB QEMU HDD , Graphics: Device 1234:1111 , Network: Red Hat Virtio device
OS: Ubuntu 16.04, Kernel: 4.4.0-116-generic (x86_64), Compiler: GCC 5.4.0 20160609, File-System: ext4, Screen Resolution: 800x600, System Layer: qemu
Compiler Notes: --build=x86_64-linux-gnu --disable-browser-plugin --disable-vtable-verify --disable-werror --enable-checking=release --enable-clocale=gnu --enable-gnu-unique-object --enable-gtk-cairo --enable-java-awt=gtk --enable-java-home --enable-languages=c,ada,c++,java,go,d,fortran,objc,obj-c++ --enable-libmpx --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-arch-directory=amd64 --with-default-libstdcxx-abi=new --with-multilib-list=m32,m64,mx32 --with-tune=generic -vDisk Notes: DEADLINE / data=ordered,errors=remount-ro,relatime,rwSystem Notes: Disk Scheduler: DEADLINE. Python 2.7.12.
ZFS SSD Benchmark: RAW IMAGE vs QCOW2 vs ZVOL for KVM Processor Motherboard Chipset Memory Disk Graphics Network OS Kernel Compiler File-System Screen Resolution System Layer zfs ssd mirror, ashift=13, recordsize 64k, qcow2 image zfs ssd mirror, ashift=13, recordsize 64k, raw image zfs ssd mirror, ashift=13, recordsize 64k, zvol zfs ssd mirror, ashift=13, recordsize 8k, qcow2 image zfs ssd mirror, ashift=13, recordsize 8k, raw image zfs ssd mirror, ashift=13, recordsize 8k, zvol pve-ubuntu-local-1 8 x Intel Celeron_4x0 (Conroe/Merom Class Core 2) @ 1.86GHz (8 Cores) QEMU Standard PC (i440FX + PIIX 1996) Intel 440FX- 82441FX PMC 1 x 1024 MB RAM QEMU 54GB QEMU HDD Cirrus Logic GD 5446 Realtek RTL-8100/8101L/8139 Debian 9.1 4.9.0-4-amd64 (x86_64) GCC 6.3.0 20170516 ext4 1024x768 qemu Common KVM @ 3.09GHz (2 Cores) 2048MB 34GB QEMU HDD Device 1234:1111 Red Hat Virtio device Ubuntu 16.04 4.4.0-116-generic (x86_64) GCC 5.4.0 20160609 800x600 OpenBenchmarking.org Compiler Details - zfs ssd mirror, ashift=13, recordsize 64k, qcow2 image: --build=x86_64-linux-gnu --disable-browser-plugin --disable-vtable-verify --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-gtk-cairo --enable-java-awt=gtk --enable-java-home --enable-languages=c,ada,c++,java,go,d,fortran,objc,obj-c++ --enable-libmpx --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-arch-directory=amd64 --with-default-libstdcxx-abi=new --with-multilib-list=m32,m64,mx32 --with-target-system-zlib --with-tune=generic -v - zfs ssd mirror, ashift=13, recordsize 64k, raw image: --build=x86_64-linux-gnu --disable-browser-plugin --disable-vtable-verify --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-gtk-cairo --enable-java-awt=gtk --enable-java-home --enable-languages=c,ada,c++,java,go,d,fortran,objc,obj-c++ --enable-libmpx --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-arch-directory=amd64 --with-default-libstdcxx-abi=new --with-multilib-list=m32,m64,mx32 --with-target-system-zlib --with-tune=generic -v - zfs ssd mirror, ashift=13, recordsize 64k, zvol: --build=x86_64-linux-gnu --disable-browser-plugin --disable-vtable-verify --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-gtk-cairo --enable-java-awt=gtk --enable-java-home --enable-languages=c,ada,c++,java,go,d,fortran,objc,obj-c++ --enable-libmpx --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-arch-directory=amd64 --with-default-libstdcxx-abi=new --with-multilib-list=m32,m64,mx32 --with-target-system-zlib --with-tune=generic -v - zfs ssd mirror, ashift=13, recordsize 8k, qcow2 image: --build=x86_64-linux-gnu --disable-browser-plugin --disable-vtable-verify --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-gtk-cairo --enable-java-awt=gtk --enable-java-home --enable-languages=c,ada,c++,java,go,d,fortran,objc,obj-c++ --enable-libmpx --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-arch-directory=amd64 --with-default-libstdcxx-abi=new --with-multilib-list=m32,m64,mx32 --with-target-system-zlib --with-tune=generic -v - zfs ssd mirror, ashift=13, recordsize 8k, raw image: --build=x86_64-linux-gnu --disable-browser-plugin --disable-vtable-verify --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-gtk-cairo --enable-java-awt=gtk --enable-java-home --enable-languages=c,ada,c++,java,go,d,fortran,objc,obj-c++ --enable-libmpx --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-arch-directory=amd64 --with-default-libstdcxx-abi=new --with-multilib-list=m32,m64,mx32 --with-target-system-zlib --with-tune=generic -v - zfs ssd mirror, ashift=13, recordsize 8k, zvol: --build=x86_64-linux-gnu --disable-browser-plugin --disable-vtable-verify --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-gtk-cairo --enable-java-awt=gtk --enable-java-home --enable-languages=c,ada,c++,java,go,d,fortran,objc,obj-c++ --enable-libmpx --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-arch-directory=amd64 --with-default-libstdcxx-abi=new --with-multilib-list=m32,m64,mx32 --with-target-system-zlib --with-tune=generic -v - pve-ubuntu-local-1: --build=x86_64-linux-gnu --disable-browser-plugin --disable-vtable-verify --disable-werror --enable-checking=release --enable-clocale=gnu --enable-gnu-unique-object --enable-gtk-cairo --enable-java-awt=gtk --enable-java-home --enable-languages=c,ada,c++,java,go,d,fortran,objc,obj-c++ --enable-libmpx --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-arch-directory=amd64 --with-default-libstdcxx-abi=new --with-multilib-list=m32,m64,mx32 --with-tune=generic -v Disk Details - zfs ssd mirror, ashift=13, recordsize 64k, qcow2 image: CFQ / data=ordered,errors=remount-ro,relatime,rw - zfs ssd mirror, ashift=13, recordsize 64k, raw image: CFQ / data=ordered,errors=remount-ro,relatime,rw - zfs ssd mirror, ashift=13, recordsize 64k, zvol: CFQ / data=ordered,errors=remount-ro,relatime,rw - zfs ssd mirror, ashift=13, recordsize 8k, qcow2 image: CFQ / data=ordered,errors=remount-ro,relatime,rw - zfs ssd mirror, ashift=13, recordsize 8k, raw image: CFQ / data=ordered,errors=remount-ro,relatime,rw - zfs ssd mirror, ashift=13, recordsize 8k, zvol: CFQ / data=ordered,errors=remount-ro,relatime,rw - pve-ubuntu-local-1: DEADLINE / data=ordered,errors=remount-ro,relatime,rw System Details - zfs ssd mirror, ashift=13, recordsize 64k, qcow2 image: Python 2.7.13. - zfs ssd mirror, ashift=13, recordsize 64k, raw image: Python 2.7.13. - zfs ssd mirror, ashift=13, recordsize 64k, zvol: Python 2.7.13. - zfs ssd mirror, ashift=13, recordsize 8k, qcow2 image: Python 2.7.13. - zfs ssd mirror, ashift=13, recordsize 8k, raw image: Python 2.7.13. - zfs ssd mirror, ashift=13, recordsize 8k, zvol: Python 2.7.13. - pve-ubuntu-local-1: Disk Scheduler: DEADLINE. Python 2.7.12.
zfs ssd mirror, ashift=13, recordsize 64k, qcow2 image zfs ssd mirror, ashift=13, recordsize 64k, raw image zfs ssd mirror, ashift=13, recordsize 64k, zvol zfs ssd mirror, ashift=13, recordsize 8k, qcow2 image zfs ssd mirror, ashift=13, recordsize 8k, raw image zfs ssd mirror, ashift=13, recordsize 8k, zvol pve-ubuntu-local-1 Result Overview Phoronix Test Suite 100% 220% 339% 459% 579% Dbench Compile Bench IOzone AIO-Stress
ZFS SSD Benchmark: RAW IMAGE vs QCOW2 vs ZVOL for KVM dbench: 6 dbench: 12 iozone: 64Kb - 8GB - Write Performance iozone: 4Kb - 8GB - Write Performance dbench: 256 compilebench: Read Compiled Tree iozone: 1MB - 8GB - Write Performance iozone: 64Kb - 8GB - Read Performance iozone: 1MB - 8GB - Read Performance iozone: 4Kb - 8GB - Read Performance aio-stress: Rand Write dbench: 1 compilebench: Initial Create compilebench: Compile dbench: 128 dbench: 48 zfs ssd mirror, ashift=13, recordsize 64k, qcow2 image zfs ssd mirror, ashift=13, recordsize 64k, raw image zfs ssd mirror, ashift=13, recordsize 64k, zvol zfs ssd mirror, ashift=13, recordsize 8k, qcow2 image zfs ssd mirror, ashift=13, recordsize 8k, raw image zfs ssd mirror, ashift=13, recordsize 8k, zvol pve-ubuntu-local-1 102.98 162.59 156.04 161.59 118.70 64.69 156.30 225.08 215.34 223.31 125.74 42.33 82.40 142.88 203.76 294.03 105.25 166.12 160.28 162.04 120.18 64.83 157.79 226.37 216.63 224.74 131.81 42.60 86.85 147.18 207.32 292.22 88.54 133.55 60.03 63.10 62.00 60.72 66.79 88.90 86.07 94.99 111.33 42.72 50.11 81.59 102.07 177.89 89.99 137.50 58.87 61.43 55.57 56.72 65.22 174.56 165.90 173.68 93.70 42.20 42.54 61.65 89.75 176.41 91.96 137.66 55.72 57.00 55.77 55.72 60.00 172.38 167.07 174.63 98.01 42.18 48.40 72.13 97.79 181.38 86.79 128.76 67.62 74.06 58.47 46.58 72.07 84.86 80.50 84.95 100.87 40.87 49.32 91.69 88.41 153.63 826.51 779.05 207.41 207.63 201.67 166.58 206.19 262.94 245.63 256.87 49.15 430.74 182.82 262.57 308.19 792.05 OpenBenchmarking.org
Dbench Dbench is a benchmark designed by the Samba project as a free alternative to netbench, but dbench contains only file-system calls for testing the disk performance. Learn more via the OpenBenchmarking.org test page.
Result
OpenBenchmarking.org MB/s, More Is Better Dbench 4.0 Client Count: 6 pve-ubuntu-local-1 zfs ssd mirror, ashift=13, recordsize 64k, qcow2 image zfs ssd mirror, ashift=13, recordsize 64k, raw image zfs ssd mirror, ashift=13, recordsize 64k, zvol zfs ssd mirror, ashift=13, recordsize 8k, qcow2 image zfs ssd mirror, ashift=13, recordsize 8k, raw image zfs ssd mirror, ashift=13, recordsize 8k, zvol 200 400 600 800 1000 SE +/- 15.67, N = 3 SE +/- 0.91, N = 3 SE +/- 1.07, N = 3 SE +/- 1.27, N = 3 SE +/- 0.90, N = 3 SE +/- 1.36, N = 3 SE +/- 0.16, N = 3 826.51 102.98 105.25 88.54 89.99 91.96 86.79 1. (CC) gcc options: -lpopt -O2
Result Confidence
OpenBenchmarking.org MB/s, More Is Better Dbench 4.0 Client Count: 6 pve-ubuntu-local-1 zfs ssd mirror, ashift=13, recordsize 64k, qcow2 image zfs ssd mirror, ashift=13, recordsize 64k, raw image zfs ssd mirror, ashift=13, recordsize 64k, zvol zfs ssd mirror, ashift=13, recordsize 8k, qcow2 image zfs ssd mirror, ashift=13, recordsize 8k, raw image zfs ssd mirror, ashift=13, recordsize 8k, zvol 150 300 450 600 750 Min: 795.53 / Avg: 826.51 / Max: 846.08 Min: 101.58 / Avg: 102.98 / Max: 104.68 Min: 104.08 / Avg: 105.25 / Max: 107.38 Min: 87.03 / Avg: 88.54 / Max: 91.07 Min: 88.53 / Avg: 89.99 / Max: 91.62 Min: 89.9 / Avg: 91.96 / Max: 94.52 Min: 86.5 / Avg: 86.79 / Max: 87.06 1. (CC) gcc options: -lpopt -O2
Result
OpenBenchmarking.org MB/s, More Is Better Dbench 4.0 Client Count: 12 pve-ubuntu-local-1 zfs ssd mirror, ashift=13, recordsize 64k, qcow2 image zfs ssd mirror, ashift=13, recordsize 64k, raw image zfs ssd mirror, ashift=13, recordsize 64k, zvol zfs ssd mirror, ashift=13, recordsize 8k, qcow2 image zfs ssd mirror, ashift=13, recordsize 8k, raw image zfs ssd mirror, ashift=13, recordsize 8k, zvol 200 400 600 800 1000 SE +/- 9.36, N = 3 SE +/- 0.75, N = 3 SE +/- 0.58, N = 3 SE +/- 0.41, N = 3 SE +/- 1.47, N = 3 SE +/- 1.52, N = 3 SE +/- 0.84, N = 3 779.05 162.59 166.12 133.55 137.50 137.66 128.76 1. (CC) gcc options: -lpopt -O2
Result Confidence
OpenBenchmarking.org MB/s, More Is Better Dbench 4.0 Client Count: 12 pve-ubuntu-local-1 zfs ssd mirror, ashift=13, recordsize 64k, qcow2 image zfs ssd mirror, ashift=13, recordsize 64k, raw image zfs ssd mirror, ashift=13, recordsize 64k, zvol zfs ssd mirror, ashift=13, recordsize 8k, qcow2 image zfs ssd mirror, ashift=13, recordsize 8k, raw image zfs ssd mirror, ashift=13, recordsize 8k, zvol 140 280 420 560 700 Min: 764.77 / Avg: 779.05 / Max: 796.68 Min: 161.12 / Avg: 162.59 / Max: 163.61 Min: 165.44 / Avg: 166.12 / Max: 167.27 Min: 132.77 / Avg: 133.55 / Max: 134.14 Min: 134.66 / Avg: 137.5 / Max: 139.59 Min: 135.61 / Avg: 137.66 / Max: 140.62 Min: 127.18 / Avg: 128.76 / Max: 130.06 1. (CC) gcc options: -lpopt -O2
Flexible IO Tester
Result
OpenBenchmarking.org MB/s, More Is Better Flexible IO Tester 3.1 Type: Sequential Write - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 64KB - Disk Target: Default Test Directory zfs ssd mirror, ashift=13, recordsize 64k, qcow2 image zfs ssd mirror, ashift=13, recordsize 64k, raw image zfs ssd mirror, ashift=13, recordsize 64k, zvol zfs ssd mirror, ashift=13, recordsize 8k, qcow2 image zfs ssd mirror, ashift=13, recordsize 8k, raw image zfs ssd mirror, ashift=13, recordsize 8k, zvol 30 60 90 120 150 SE +/- 0.67, N = 3 SE +/- 0.59, N = 3 SE +/- 0.03, N = 3 SE +/- 0.41, N = 3 SE +/- 3.31, N = 6 139.00 138.00 71.37 70.77 74.27 36.43 1. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -U_FORTIFY_SOURCE -lrt -laio -lm -lpthread -ldl
Result Confidence
OpenBenchmarking.org MB/s, More Is Better Flexible IO Tester 3.1 Type: Sequential Write - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 64KB - Disk Target: Default Test Directory zfs ssd mirror, ashift=13, recordsize 64k, qcow2 image zfs ssd mirror, ashift=13, recordsize 64k, raw image zfs ssd mirror, ashift=13, recordsize 64k, zvol zfs ssd mirror, ashift=13, recordsize 8k, qcow2 image zfs ssd mirror, ashift=13, recordsize 8k, raw image zfs ssd mirror, ashift=13, recordsize 8k, zvol 30 60 90 120 150 Min: 138 / Avg: 139.33 / Max: 140 Min: 70.2 / Avg: 71.37 / Max: 72.1 Min: 70.7 / Avg: 70.77 / Max: 70.8 Min: 73.5 / Avg: 74.27 / Max: 74.9 Min: 30 / Avg: 36.43 / Max: 52.3 1. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -U_FORTIFY_SOURCE -lrt -laio -lm -lpthread -ldl
IOzone
Result
OpenBenchmarking.org MB/s, More Is Better IOzone 3.465 Record Size: 64Kb - File Size: 8GB - Disk Test: Write Performance pve-ubuntu-local-1 zfs ssd mirror, ashift=13, recordsize 64k, qcow2 image zfs ssd mirror, ashift=13, recordsize 64k, raw image zfs ssd mirror, ashift=13, recordsize 64k, zvol zfs ssd mirror, ashift=13, recordsize 8k, qcow2 image zfs ssd mirror, ashift=13, recordsize 8k, raw image zfs ssd mirror, ashift=13, recordsize 8k, zvol 50 100 150 200 250 SE +/- 3.16, N = 3 SE +/- 2.73, N = 6 SE +/- 2.94, N = 6 SE +/- 1.77, N = 6 SE +/- 0.92, N = 6 SE +/- 0.37, N = 3 SE +/- 3.14, N = 6 207.41 156.04 160.28 60.03 58.87 55.72 67.62 1. (CC) gcc options: -O3
Result Confidence
OpenBenchmarking.org MB/s, More Is Better IOzone 3.465 Record Size: 64Kb - File Size: 8GB - Disk Test: Write Performance pve-ubuntu-local-1 zfs ssd mirror, ashift=13, recordsize 64k, qcow2 image zfs ssd mirror, ashift=13, recordsize 64k, raw image zfs ssd mirror, ashift=13, recordsize 64k, zvol zfs ssd mirror, ashift=13, recordsize 8k, qcow2 image zfs ssd mirror, ashift=13, recordsize 8k, raw image zfs ssd mirror, ashift=13, recordsize 8k, zvol 40 80 120 160 200 Min: 201.52 / Avg: 207.41 / Max: 212.34 Min: 147.87 / Avg: 156.04 / Max: 164.35 Min: 153.6 / Avg: 160.28 / Max: 173.13 Min: 51.66 / Avg: 60.03 / Max: 63.61 Min: 57.22 / Avg: 58.87 / Max: 63.21 Min: 54.99 / Avg: 55.72 / Max: 56.21 Min: 60.89 / Avg: 67.62 / Max: 81.49 1. (CC) gcc options: -O3
Result
OpenBenchmarking.org MB/s, More Is Better IOzone 3.465 Record Size: 4Kb - File Size: 8GB - Disk Test: Write Performance pve-ubuntu-local-1 zfs ssd mirror, ashift=13, recordsize 64k, qcow2 image zfs ssd mirror, ashift=13, recordsize 64k, raw image zfs ssd mirror, ashift=13, recordsize 64k, zvol zfs ssd mirror, ashift=13, recordsize 8k, qcow2 image zfs ssd mirror, ashift=13, recordsize 8k, raw image zfs ssd mirror, ashift=13, recordsize 8k, zvol 50 100 150 200 250 SE +/- 2.78, N = 3 SE +/- 1.79, N = 3 SE +/- 1.03, N = 3 SE +/- 1.08, N = 3 SE +/- 1.02, N = 6 SE +/- 1.60, N = 6 SE +/- 1.13, N = 3 207.63 161.59 162.04 63.10 61.43 57.00 74.06 1. (CC) gcc options: -O3
Result Confidence
OpenBenchmarking.org MB/s, More Is Better IOzone 3.465 Record Size: 4Kb - File Size: 8GB - Disk Test: Write Performance pve-ubuntu-local-1 zfs ssd mirror, ashift=13, recordsize 64k, qcow2 image zfs ssd mirror, ashift=13, recordsize 64k, raw image zfs ssd mirror, ashift=13, recordsize 64k, zvol zfs ssd mirror, ashift=13, recordsize 8k, qcow2 image zfs ssd mirror, ashift=13, recordsize 8k, raw image zfs ssd mirror, ashift=13, recordsize 8k, zvol 40 80 120 160 200 Min: 202.54 / Avg: 207.63 / Max: 212.11 Min: 158.77 / Avg: 161.59 / Max: 164.92 Min: 160.4 / Avg: 162.04 / Max: 163.93 Min: 61.35 / Avg: 63.1 / Max: 65.07 Min: 58.11 / Avg: 61.43 / Max: 64.56 Min: 52.64 / Avg: 57 / Max: 62.51 Min: 72.03 / Avg: 74.06 / Max: 75.93 1. (CC) gcc options: -O3
Dbench Dbench is a benchmark designed by the Samba project as a free alternative to netbench, but dbench contains only file-system calls for testing the disk performance. Learn more via the OpenBenchmarking.org test page.
Result
OpenBenchmarking.org MB/s, More Is Better Dbench 4.0 Client Count: 256 pve-ubuntu-local-1 zfs ssd mirror, ashift=13, recordsize 64k, qcow2 image zfs ssd mirror, ashift=13, recordsize 64k, raw image zfs ssd mirror, ashift=13, recordsize 64k, zvol zfs ssd mirror, ashift=13, recordsize 8k, qcow2 image zfs ssd mirror, ashift=13, recordsize 8k, raw image zfs ssd mirror, ashift=13, recordsize 8k, zvol 40 80 120 160 200 SE +/- 1.90, N = 3 SE +/- 0.23, N = 3 SE +/- 0.40, N = 3 SE +/- 2.00, N = 6 SE +/- 0.85, N = 3 SE +/- 2.11, N = 6 SE +/- 1.39, N = 6 201.67 118.70 120.18 62.00 55.57 55.77 58.47 1. (CC) gcc options: -lpopt -O2
Result Confidence
OpenBenchmarking.org MB/s, More Is Better Dbench 4.0 Client Count: 256 pve-ubuntu-local-1 zfs ssd mirror, ashift=13, recordsize 64k, qcow2 image zfs ssd mirror, ashift=13, recordsize 64k, raw image zfs ssd mirror, ashift=13, recordsize 64k, zvol zfs ssd mirror, ashift=13, recordsize 8k, qcow2 image zfs ssd mirror, ashift=13, recordsize 8k, raw image zfs ssd mirror, ashift=13, recordsize 8k, zvol 40 80 120 160 200 Min: 199.14 / Avg: 201.67 / Max: 205.39 Min: 118.32 / Avg: 118.7 / Max: 119.12 Min: 119.74 / Avg: 120.18 / Max: 120.97 Min: 56.76 / Avg: 62 / Max: 69.12 Min: 53.96 / Avg: 55.57 / Max: 56.82 Min: 49.06 / Avg: 55.77 / Max: 61.92 Min: 56.01 / Avg: 58.47 / Max: 64.77 1. (CC) gcc options: -lpopt -O2
Compile Bench Compilebench tries to age a filesystem by simulating some of the disk IO common in creating, compiling, patching, stating and reading kernel trees. It indirectly measures how well filesystems can maintain directory locality as the disk fills up and directories age. This current test is setup to use the makej mode with 10 initial directories Learn more via the OpenBenchmarking.org test page.
Result
OpenBenchmarking.org MB/s, More Is Better Compile Bench 0.6 Test: Read Compiled Tree pve-ubuntu-local-1 zfs ssd mirror, ashift=13, recordsize 64k, qcow2 image zfs ssd mirror, ashift=13, recordsize 64k, raw image zfs ssd mirror, ashift=13, recordsize 64k, zvol zfs ssd mirror, ashift=13, recordsize 8k, qcow2 image zfs ssd mirror, ashift=13, recordsize 8k, raw image zfs ssd mirror, ashift=13, recordsize 8k, zvol 40 80 120 160 200 SE +/- 2.00, N = 3 SE +/- 0.14, N = 3 SE +/- 0.33, N = 3 SE +/- 2.51, N = 3 SE +/- 0.52, N = 3 SE +/- 0.15, N = 3 SE +/- 0.65, N = 3 166.58 64.69 64.83 60.72 56.72 55.72 46.58
Result Confidence
OpenBenchmarking.org MB/s, More Is Better Compile Bench 0.6 Test: Read Compiled Tree pve-ubuntu-local-1 zfs ssd mirror, ashift=13, recordsize 64k, qcow2 image zfs ssd mirror, ashift=13, recordsize 64k, raw image zfs ssd mirror, ashift=13, recordsize 64k, zvol zfs ssd mirror, ashift=13, recordsize 8k, qcow2 image zfs ssd mirror, ashift=13, recordsize 8k, raw image zfs ssd mirror, ashift=13, recordsize 8k, zvol 30 60 90 120 150 Min: 163.71 / Avg: 166.58 / Max: 170.44 Min: 64.52 / Avg: 64.69 / Max: 64.96 Min: 64.19 / Avg: 64.83 / Max: 65.29 Min: 58.05 / Avg: 60.72 / Max: 65.73 Min: 55.68 / Avg: 56.72 / Max: 57.26 Min: 55.49 / Avg: 55.72 / Max: 55.99 Min: 45.51 / Avg: 46.58 / Max: 47.74
Flexible IO Tester
Result
OpenBenchmarking.org MB/s, More Is Better Flexible IO Tester 3.1 Type: Random Write - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 1MB - Disk Target: Default Test Directory zfs ssd mirror, ashift=13, recordsize 64k, qcow2 image zfs ssd mirror, ashift=13, recordsize 64k, raw image zfs ssd mirror, ashift=13, recordsize 64k, zvol zfs ssd mirror, ashift=13, recordsize 8k, qcow2 image zfs ssd mirror, ashift=13, recordsize 8k, raw image zfs ssd mirror, ashift=13, recordsize 8k, zvol 30 60 90 120 150 SE +/- 0.67, N = 3 SE +/- 0.58, N = 3 SE +/- 2.09, N = 6 SE +/- 0.09, N = 3 SE +/- 0.31, N = 3 SE +/- 1.08, N = 6 143.00 146.00 54.75 70.23 71.20 42.05 1. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -U_FORTIFY_SOURCE -lrt -laio -lm -lpthread -ldl
Result Confidence
OpenBenchmarking.org MB/s, More Is Better Flexible IO Tester 3.1 Type: Random Write - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 1MB - Disk Target: Default Test Directory zfs ssd mirror, ashift=13, recordsize 64k, qcow2 image zfs ssd mirror, ashift=13, recordsize 64k, raw image zfs ssd mirror, ashift=13, recordsize 64k, zvol zfs ssd mirror, ashift=13, recordsize 8k, qcow2 image zfs ssd mirror, ashift=13, recordsize 8k, raw image zfs ssd mirror, ashift=13, recordsize 8k, zvol 30 60 90 120 150 Min: 142 / Avg: 143.33 / Max: 144 Min: 145 / Avg: 146 / Max: 147 Min: 50.1 / Avg: 54.75 / Max: 63.8 Min: 70.1 / Avg: 70.23 / Max: 70.4 Min: 70.8 / Avg: 71.2 / Max: 71.8 Min: 38.9 / Avg: 42.05 / Max: 45.3 1. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -U_FORTIFY_SOURCE -lrt -laio -lm -lpthread -ldl
IOzone
Result
OpenBenchmarking.org MB/s, More Is Better IOzone 3.465 Record Size: 1MB - File Size: 8GB - Disk Test: Write Performance pve-ubuntu-local-1 zfs ssd mirror, ashift=13, recordsize 64k, qcow2 image zfs ssd mirror, ashift=13, recordsize 64k, raw image zfs ssd mirror, ashift=13, recordsize 64k, zvol zfs ssd mirror, ashift=13, recordsize 8k, qcow2 image zfs ssd mirror, ashift=13, recordsize 8k, raw image zfs ssd mirror, ashift=13, recordsize 8k, zvol 50 100 150 200 250 SE +/- 1.30, N = 3 SE +/- 0.82, N = 3 SE +/- 0.85, N = 3 SE +/- 0.50, N = 3 SE +/- 2.01, N = 6 SE +/- 1.42, N = 6 SE +/- 2.02, N = 6 206.19 156.30 157.79 66.79 65.22 60.00 72.07 1. (CC) gcc options: -O3
Result Confidence
OpenBenchmarking.org MB/s, More Is Better IOzone 3.465 Record Size: 1MB - File Size: 8GB - Disk Test: Write Performance pve-ubuntu-local-1 zfs ssd mirror, ashift=13, recordsize 64k, qcow2 image zfs ssd mirror, ashift=13, recordsize 64k, raw image zfs ssd mirror, ashift=13, recordsize 64k, zvol zfs ssd mirror, ashift=13, recordsize 8k, qcow2 image zfs ssd mirror, ashift=13, recordsize 8k, raw image zfs ssd mirror, ashift=13, recordsize 8k, zvol 40 80 120 160 200 Min: 204.82 / Avg: 206.19 / Max: 208.79 Min: 154.67 / Avg: 156.3 / Max: 157.23 Min: 156.74 / Avg: 157.79 / Max: 159.47 Min: 66.24 / Avg: 66.79 / Max: 67.8 Min: 58.73 / Avg: 65.22 / Max: 71.05 Min: 57.1 / Avg: 60 / Max: 65.15 Min: 67.3 / Avg: 72.07 / Max: 81.1 1. (CC) gcc options: -O3
Flexible IO Tester
Result
OpenBenchmarking.org MB/s, More Is Better Flexible IO Tester 3.1 Type: Sequential Write - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 1MB - Disk Target: Default Test Directory zfs ssd mirror, ashift=13, recordsize 64k, qcow2 image zfs ssd mirror, ashift=13, recordsize 64k, raw image zfs ssd mirror, ashift=13, recordsize 64k, zvol zfs ssd mirror, ashift=13, recordsize 8k, qcow2 image zfs ssd mirror, ashift=13, recordsize 8k, raw image zfs ssd mirror, ashift=13, recordsize 8k, zvol 30 60 90 120 150 SE +/- 2.33, N = 3 SE +/- 1.94, N = 6 SE +/- 1.08, N = 3 SE +/- 0.93, N = 3 SE +/- 6.42, N = 6 147.00 152.00 66.28 72.03 74.37 45.33 1. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -U_FORTIFY_SOURCE -lrt -laio -lm -lpthread -ldl
Result Confidence
OpenBenchmarking.org MB/s, More Is Better Flexible IO Tester 3.1 Type: Sequential Write - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 1MB - Disk Target: Default Test Directory zfs ssd mirror, ashift=13, recordsize 64k, qcow2 image zfs ssd mirror, ashift=13, recordsize 64k, raw image zfs ssd mirror, ashift=13, recordsize 64k, zvol zfs ssd mirror, ashift=13, recordsize 8k, qcow2 image zfs ssd mirror, ashift=13, recordsize 8k, raw image zfs ssd mirror, ashift=13, recordsize 8k, zvol 30 60 90 120 150 Min: 143 / Avg: 146.67 / Max: 151 Min: 61.2 / Avg: 66.28 / Max: 71.4 Min: 70 / Avg: 72.03 / Max: 73.7 Min: 72.7 / Avg: 74.37 / Max: 75.9 Min: 33.6 / Avg: 45.33 / Max: 75.1 1. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -U_FORTIFY_SOURCE -lrt -laio -lm -lpthread -ldl
IOzone
Result
OpenBenchmarking.org MB/s, More Is Better IOzone 3.465 Record Size: 64Kb - File Size: 8GB - Disk Test: Read Performance pve-ubuntu-local-1 zfs ssd mirror, ashift=13, recordsize 64k, qcow2 image zfs ssd mirror, ashift=13, recordsize 64k, raw image zfs ssd mirror, ashift=13, recordsize 64k, zvol zfs ssd mirror, ashift=13, recordsize 8k, qcow2 image zfs ssd mirror, ashift=13, recordsize 8k, raw image zfs ssd mirror, ashift=13, recordsize 8k, zvol 60 120 180 240 300 SE +/- 10.90, N = 6 SE +/- 0.72, N = 3 SE +/- 0.36, N = 3 SE +/- 1.37, N = 4 SE +/- 0.49, N = 3 SE +/- 0.49, N = 3 SE +/- 2.08, N = 6 262.94 225.08 226.37 88.90 174.56 172.38 84.86 1. (CC) gcc options: -O3
Result Confidence
OpenBenchmarking.org MB/s, More Is Better IOzone 3.465 Record Size: 64Kb - File Size: 8GB - Disk Test: Read Performance pve-ubuntu-local-1 zfs ssd mirror, ashift=13, recordsize 64k, qcow2 image zfs ssd mirror, ashift=13, recordsize 64k, raw image zfs ssd mirror, ashift=13, recordsize 64k, zvol zfs ssd mirror, ashift=13, recordsize 8k, qcow2 image zfs ssd mirror, ashift=13, recordsize 8k, raw image zfs ssd mirror, ashift=13, recordsize 8k, zvol 50 100 150 200 250 Min: 225.93 / Avg: 262.94 / Max: 284.84 Min: 223.68 / Avg: 225.08 / Max: 226.04 Min: 225.64 / Avg: 226.37 / Max: 226.77 Min: 85.77 / Avg: 88.9 / Max: 92.46 Min: 173.86 / Avg: 174.56 / Max: 175.51 Min: 171.49 / Avg: 172.38 / Max: 173.17 Min: 80.99 / Avg: 84.86 / Max: 92.32 1. (CC) gcc options: -O3
Result
OpenBenchmarking.org MB/s, More Is Better IOzone 3.465 Record Size: 1MB - File Size: 8GB - Disk Test: Read Performance pve-ubuntu-local-1 zfs ssd mirror, ashift=13, recordsize 64k, qcow2 image zfs ssd mirror, ashift=13, recordsize 64k, raw image zfs ssd mirror, ashift=13, recordsize 64k, zvol zfs ssd mirror, ashift=13, recordsize 8k, qcow2 image zfs ssd mirror, ashift=13, recordsize 8k, raw image zfs ssd mirror, ashift=13, recordsize 8k, zvol 50 100 150 200 250 SE +/- 10.35, N = 6 SE +/- 0.79, N = 3 SE +/- 0.57, N = 3 SE +/- 0.58, N = 3 SE +/- 1.22, N = 3 SE +/- 1.25, N = 3 SE +/- 1.26, N = 3 245.63 215.34 216.63 86.07 165.90 167.07 80.50 1. (CC) gcc options: -O3
Result Confidence
OpenBenchmarking.org MB/s, More Is Better IOzone 3.465 Record Size: 1MB - File Size: 8GB - Disk Test: Read Performance pve-ubuntu-local-1 zfs ssd mirror, ashift=13, recordsize 64k, qcow2 image zfs ssd mirror, ashift=13, recordsize 64k, raw image zfs ssd mirror, ashift=13, recordsize 64k, zvol zfs ssd mirror, ashift=13, recordsize 8k, qcow2 image zfs ssd mirror, ashift=13, recordsize 8k, raw image zfs ssd mirror, ashift=13, recordsize 8k, zvol 40 80 120 160 200 Min: 208.44 / Avg: 245.63 / Max: 279.85 Min: 214.03 / Avg: 215.34 / Max: 216.76 Min: 215.5 / Avg: 216.63 / Max: 217.24 Min: 84.98 / Avg: 86.07 / Max: 86.95 Min: 164.47 / Avg: 165.9 / Max: 168.32 Min: 165.56 / Avg: 167.07 / Max: 169.54 Min: 78.74 / Avg: 80.5 / Max: 82.93 1. (CC) gcc options: -O3
Flexible IO Tester
Result
OpenBenchmarking.org MB/s, More Is Better Flexible IO Tester 3.1 Type: Random Write - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 64KB - Disk Target: Default Test Directory zfs ssd mirror, ashift=13, recordsize 64k, qcow2 image zfs ssd mirror, ashift=13, recordsize 64k, raw image zfs ssd mirror, ashift=13, recordsize 64k, zvol zfs ssd mirror, ashift=13, recordsize 8k, qcow2 image zfs ssd mirror, ashift=13, recordsize 8k, raw image zfs ssd mirror, ashift=13, recordsize 8k, zvol 30 60 90 120 150 SE +/- 0.67, N = 3 SE +/- 0.88, N = 3 SE +/- 0.81, N = 3 SE +/- 0.25, N = 3 SE +/- 0.03, N = 3 SE +/- 3.30, N = 6 113.00 118.00 66.40 59.40 61.33 38.75 1. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -U_FORTIFY_SOURCE -lrt -laio -lm -lpthread -ldl
Result Confidence
OpenBenchmarking.org MB/s, More Is Better Flexible IO Tester 3.1 Type: Random Write - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 64KB - Disk Target: Default Test Directory zfs ssd mirror, ashift=13, recordsize 64k, qcow2 image zfs ssd mirror, ashift=13, recordsize 64k, raw image zfs ssd mirror, ashift=13, recordsize 64k, zvol zfs ssd mirror, ashift=13, recordsize 8k, qcow2 image zfs ssd mirror, ashift=13, recordsize 8k, raw image zfs ssd mirror, ashift=13, recordsize 8k, zvol 20 40 60 80 100 Min: 112 / Avg: 113.33 / Max: 114 Min: 116 / Avg: 117.67 / Max: 119 Min: 65 / Avg: 66.4 / Max: 67.8 Min: 58.9 / Avg: 59.4 / Max: 59.7 Min: 61.3 / Avg: 61.33 / Max: 61.4 Min: 30.9 / Avg: 38.75 / Max: 50.4 1. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -U_FORTIFY_SOURCE -lrt -laio -lm -lpthread -ldl
IOzone
Result
OpenBenchmarking.org MB/s, More Is Better IOzone 3.465 Record Size: 4Kb - File Size: 8GB - Disk Test: Read Performance pve-ubuntu-local-1 zfs ssd mirror, ashift=13, recordsize 64k, qcow2 image zfs ssd mirror, ashift=13, recordsize 64k, raw image zfs ssd mirror, ashift=13, recordsize 64k, zvol zfs ssd mirror, ashift=13, recordsize 8k, qcow2 image zfs ssd mirror, ashift=13, recordsize 8k, raw image zfs ssd mirror, ashift=13, recordsize 8k, zvol 60 120 180 240 300 SE +/- 13.31, N = 6 SE +/- 0.88, N = 3 SE +/- 2.04, N = 3 SE +/- 1.27, N = 3 SE +/- 0.12, N = 3 SE +/- 0.79, N = 3 SE +/- 2.19, N = 6 256.87 223.31 224.74 94.99 173.68 174.63 84.95 1. (CC) gcc options: -O3
Result Confidence
OpenBenchmarking.org MB/s, More Is Better IOzone 3.465 Record Size: 4Kb - File Size: 8GB - Disk Test: Read Performance pve-ubuntu-local-1 zfs ssd mirror, ashift=13, recordsize 64k, qcow2 image zfs ssd mirror, ashift=13, recordsize 64k, raw image zfs ssd mirror, ashift=13, recordsize 64k, zvol zfs ssd mirror, ashift=13, recordsize 8k, qcow2 image zfs ssd mirror, ashift=13, recordsize 8k, raw image zfs ssd mirror, ashift=13, recordsize 8k, zvol 50 100 150 200 250 Min: 219.31 / Avg: 256.87 / Max: 289.54 Min: 221.57 / Avg: 223.31 / Max: 224.4 Min: 220.66 / Avg: 224.74 / Max: 226.99 Min: 93.66 / Avg: 94.99 / Max: 97.53 Min: 173.55 / Avg: 173.68 / Max: 173.92 Min: 173.19 / Avg: 174.63 / Max: 175.92 Min: 81.2 / Avg: 84.95 / Max: 93.88 1. (CC) gcc options: -O3
AIO-Stress AIO-Stress is an a-synchronous I/O benchmark created by SuSE. Current this profile uses a 2048MB test file and a 64KB record size. Learn more via the OpenBenchmarking.org test page.
Result
OpenBenchmarking.org MB/s, More Is Better AIO-Stress 0.21 Test: Random Write pve-ubuntu-local-1 zfs ssd mirror, ashift=13, recordsize 64k, qcow2 image zfs ssd mirror, ashift=13, recordsize 64k, raw image zfs ssd mirror, ashift=13, recordsize 64k, zvol zfs ssd mirror, ashift=13, recordsize 8k, qcow2 image zfs ssd mirror, ashift=13, recordsize 8k, raw image zfs ssd mirror, ashift=13, recordsize 8k, zvol 30 60 90 120 150 SE +/- 0.83, N = 4 SE +/- 0.78, N = 3 SE +/- 0.94, N = 3 SE +/- 0.51, N = 3 SE +/- 1.19, N = 3 SE +/- 0.50, N = 3 SE +/- 3.01, N = 6 49.15 125.74 131.81 111.33 93.70 98.01 100.87 1. (CC) gcc options: -pthread -laio
Result Confidence
OpenBenchmarking.org MB/s, More Is Better AIO-Stress 0.21 Test: Random Write pve-ubuntu-local-1 zfs ssd mirror, ashift=13, recordsize 64k, qcow2 image zfs ssd mirror, ashift=13, recordsize 64k, raw image zfs ssd mirror, ashift=13, recordsize 64k, zvol zfs ssd mirror, ashift=13, recordsize 8k, qcow2 image zfs ssd mirror, ashift=13, recordsize 8k, raw image zfs ssd mirror, ashift=13, recordsize 8k, zvol 20 40 60 80 100 Min: 47.97 / Avg: 49.15 / Max: 51.54 Min: 124.26 / Avg: 125.74 / Max: 126.89 Min: 130.01 / Avg: 131.81 / Max: 133.2 Min: 110.53 / Avg: 111.33 / Max: 112.29 Min: 91.33 / Avg: 93.7 / Max: 95.11 Min: 97.11 / Avg: 98.01 / Max: 98.82 Min: 89.63 / Avg: 100.87 / Max: 109.01 1. (CC) gcc options: -pthread -laio
Flexible IO Tester
Result
OpenBenchmarking.org MB/s, More Is Better Flexible IO Tester 3.1 Type: Random Write - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 8KB - Disk Target: Default Test Directory zfs ssd mirror, ashift=13, recordsize 64k, qcow2 image zfs ssd mirror, ashift=13, recordsize 64k, raw image zfs ssd mirror, ashift=13, recordsize 64k, zvol zfs ssd mirror, ashift=13, recordsize 8k, qcow2 image zfs ssd mirror, ashift=13, recordsize 8k, raw image zfs ssd mirror, ashift=13, recordsize 8k, zvol 7 14 21 28 35 SE +/- 0.24, N = 3 SE +/- 0.26, N = 3 SE +/- 0.21, N = 4 SE +/- 0.19, N = 3 SE +/- 0.20, N = 3 SE +/- 0.10, N = 3 15.97 17.20 12.05 25.73 29.20 20.70 1. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -U_FORTIFY_SOURCE -lrt -laio -lm -lpthread -ldl
Result Confidence
OpenBenchmarking.org MB/s, More Is Better Flexible IO Tester 3.1 Type: Random Write - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 8KB - Disk Target: Default Test Directory zfs ssd mirror, ashift=13, recordsize 64k, qcow2 image zfs ssd mirror, ashift=13, recordsize 64k, raw image zfs ssd mirror, ashift=13, recordsize 64k, zvol zfs ssd mirror, ashift=13, recordsize 8k, qcow2 image zfs ssd mirror, ashift=13, recordsize 8k, raw image zfs ssd mirror, ashift=13, recordsize 8k, zvol 6 12 18 24 30 Min: 15.5 / Avg: 15.97 / Max: 16.3 Min: 16.8 / Avg: 17.2 / Max: 17.7 Min: 11.6 / Avg: 12.05 / Max: 12.5 Min: 25.5 / Avg: 25.73 / Max: 26.1 Min: 28.8 / Avg: 29.2 / Max: 29.4 Min: 20.5 / Avg: 20.7 / Max: 20.8 1. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -U_FORTIFY_SOURCE -lrt -laio -lm -lpthread -ldl
Result
OpenBenchmarking.org MB/s, More Is Better Flexible IO Tester 3.1 Type: Sequential Read - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 1MB - Disk Target: Default Test Directory zfs ssd mirror, ashift=13, recordsize 64k, qcow2 image zfs ssd mirror, ashift=13, recordsize 64k, raw image zfs ssd mirror, ashift=13, recordsize 64k, zvol zfs ssd mirror, ashift=13, recordsize 8k, qcow2 image zfs ssd mirror, ashift=13, recordsize 8k, raw image zfs ssd mirror, ashift=13, recordsize 8k, zvol 100 200 300 400 500 SE +/- 1.20, N = 3 SE +/- 0.67, N = 3 SE +/- 5.61, N = 3 SE +/- 0.67, N = 3 SE +/- 10.48, N = 6 291 289 458 226 229 472 1. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -U_FORTIFY_SOURCE -lrt -laio -lm -lpthread -ldl
Result Confidence
OpenBenchmarking.org MB/s, More Is Better Flexible IO Tester 3.1 Type: Sequential Read - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 1MB - Disk Target: Default Test Directory zfs ssd mirror, ashift=13, recordsize 64k, qcow2 image zfs ssd mirror, ashift=13, recordsize 64k, raw image zfs ssd mirror, ashift=13, recordsize 64k, zvol zfs ssd mirror, ashift=13, recordsize 8k, qcow2 image zfs ssd mirror, ashift=13, recordsize 8k, raw image zfs ssd mirror, ashift=13, recordsize 8k, zvol 80 160 240 320 400 Min: 289 / Avg: 291.33 / Max: 293 Min: 288 / Avg: 289.33 / Max: 290 Min: 447 / Avg: 457.67 / Max: 466 Min: 225 / Avg: 226.33 / Max: 227 Min: 450 / Avg: 471.83 / Max: 505 1. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -U_FORTIFY_SOURCE -lrt -laio -lm -lpthread -ldl
Result
OpenBenchmarking.org MB/s, More Is Better Flexible IO Tester 3.1 Type: Random Read - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 1MB - Disk Target: Default Test Directory zfs ssd mirror, ashift=13, recordsize 64k, qcow2 image zfs ssd mirror, ashift=13, recordsize 64k, raw image zfs ssd mirror, ashift=13, recordsize 64k, zvol zfs ssd mirror, ashift=13, recordsize 8k, qcow2 image zfs ssd mirror, ashift=13, recordsize 8k, raw image zfs ssd mirror, ashift=13, recordsize 8k, zvol 100 200 300 400 500 SE +/- 1.20, N = 3 SE +/- 2.08, N = 3 SE +/- 10.34, N = 6 SE +/- 1.00, N = 3 SE +/- 1.00, N = 3 SE +/- 7.46, N = 6 292 294 454 228 230 473 1. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -U_FORTIFY_SOURCE -lrt -laio -lm -lpthread -ldl
Result Confidence
OpenBenchmarking.org MB/s, More Is Better Flexible IO Tester 3.1 Type: Random Read - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 1MB - Disk Target: Default Test Directory zfs ssd mirror, ashift=13, recordsize 64k, qcow2 image zfs ssd mirror, ashift=13, recordsize 64k, raw image zfs ssd mirror, ashift=13, recordsize 64k, zvol zfs ssd mirror, ashift=13, recordsize 8k, qcow2 image zfs ssd mirror, ashift=13, recordsize 8k, raw image zfs ssd mirror, ashift=13, recordsize 8k, zvol 80 160 240 320 400 Min: 290 / Avg: 291.67 / Max: 294 Min: 291 / Avg: 294 / Max: 298 Min: 412 / Avg: 454.33 / Max: 484 Min: 226 / Avg: 228 / Max: 229 Min: 229 / Avg: 230 / Max: 232 Min: 443 / Avg: 472.67 / Max: 487 1. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -U_FORTIFY_SOURCE -lrt -laio -lm -lpthread -ldl
Dbench Dbench is a benchmark designed by the Samba project as a free alternative to netbench, but dbench contains only file-system calls for testing the disk performance. Learn more via the OpenBenchmarking.org test page.
Result
OpenBenchmarking.org MB/s, More Is Better Dbench 4.0 Client Count: 1 pve-ubuntu-local-1 zfs ssd mirror, ashift=13, recordsize 64k, qcow2 image zfs ssd mirror, ashift=13, recordsize 64k, raw image zfs ssd mirror, ashift=13, recordsize 64k, zvol zfs ssd mirror, ashift=13, recordsize 8k, qcow2 image zfs ssd mirror, ashift=13, recordsize 8k, raw image zfs ssd mirror, ashift=13, recordsize 8k, zvol 90 180 270 360 450 SE +/- 3.32, N = 3 SE +/- 0.08, N = 3 SE +/- 0.09, N = 3 SE +/- 0.01, N = 3 SE +/- 0.02, N = 3 SE +/- 0.05, N = 3 SE +/- 0.15, N = 3 430.74 42.33 42.60 42.72 42.20 42.18 40.87 1. (CC) gcc options: -lpopt -O2
Result Confidence
OpenBenchmarking.org MB/s, More Is Better Dbench 4.0 Client Count: 1 pve-ubuntu-local-1 zfs ssd mirror, ashift=13, recordsize 64k, qcow2 image zfs ssd mirror, ashift=13, recordsize 64k, raw image zfs ssd mirror, ashift=13, recordsize 64k, zvol zfs ssd mirror, ashift=13, recordsize 8k, qcow2 image zfs ssd mirror, ashift=13, recordsize 8k, raw image zfs ssd mirror, ashift=13, recordsize 8k, zvol 80 160 240 320 400 Min: 424.17 / Avg: 430.74 / Max: 434.88 Min: 42.19 / Avg: 42.33 / Max: 42.45 Min: 42.43 / Avg: 42.6 / Max: 42.72 Min: 42.69 / Avg: 42.72 / Max: 42.74 Min: 42.17 / Avg: 42.2 / Max: 42.25 Min: 42.11 / Avg: 42.18 / Max: 42.27 Min: 40.71 / Avg: 40.87 / Max: 41.17 1. (CC) gcc options: -lpopt -O2
Flexible IO Tester
Result
OpenBenchmarking.org MB/s, More Is Better Flexible IO Tester 3.1 Type: Sequential Read - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 64KB - Disk Target: Default Test Directory zfs ssd mirror, ashift=13, recordsize 64k, qcow2 image zfs ssd mirror, ashift=13, recordsize 64k, raw image zfs ssd mirror, ashift=13, recordsize 64k, zvol zfs ssd mirror, ashift=13, recordsize 8k, qcow2 image zfs ssd mirror, ashift=13, recordsize 8k, raw image zfs ssd mirror, ashift=13, recordsize 8k, zvol 80 160 240 320 400 SE +/- 1.33, N = 3 SE +/- 1.76, N = 3 SE +/- 7.51, N = 6 SE +/- 0.58, N = 3 SE +/- 1.33, N = 3 SE +/- 4.51, N = 3 306 303 347 244 252 353 1. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -U_FORTIFY_SOURCE -lrt -laio -lm -lpthread -ldl
Result Confidence
OpenBenchmarking.org MB/s, More Is Better Flexible IO Tester 3.1 Type: Sequential Read - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 64KB - Disk Target: Default Test Directory zfs ssd mirror, ashift=13, recordsize 64k, qcow2 image zfs ssd mirror, ashift=13, recordsize 64k, raw image zfs ssd mirror, ashift=13, recordsize 64k, zvol zfs ssd mirror, ashift=13, recordsize 8k, qcow2 image zfs ssd mirror, ashift=13, recordsize 8k, raw image zfs ssd mirror, ashift=13, recordsize 8k, zvol 60 120 180 240 300 Min: 305 / Avg: 306.33 / Max: 309 Min: 300 / Avg: 302.67 / Max: 306 Min: 318 / Avg: 347.33 / Max: 369 Min: 243 / Avg: 244 / Max: 245 Min: 249 / Avg: 251.67 / Max: 253 Min: 348 / Avg: 353 / Max: 362 1. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -U_FORTIFY_SOURCE -lrt -laio -lm -lpthread -ldl
Result
OpenBenchmarking.org MB/s, More Is Better Flexible IO Tester 3.1 Type: Random Read - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 8KB - Disk Target: Default Test Directory zfs ssd mirror, ashift=13, recordsize 64k, qcow2 image zfs ssd mirror, ashift=13, recordsize 64k, raw image zfs ssd mirror, ashift=13, recordsize 64k, zvol zfs ssd mirror, ashift=13, recordsize 8k, qcow2 image zfs ssd mirror, ashift=13, recordsize 8k, raw image zfs ssd mirror, ashift=13, recordsize 8k, zvol 9 18 27 36 45 SE +/- 0.10, N = 3 SE +/- 0.03, N = 3 SE +/- 0.52, N = 3 SE +/- 0.65, N = 4 SE +/- 0.40, N = 3 SE +/- 0.38, N = 3 38.60 38.93 40.73 40.65 36.80 39.20 1. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -U_FORTIFY_SOURCE -lrt -laio -lm -lpthread -ldl
Result Confidence
OpenBenchmarking.org MB/s, More Is Better Flexible IO Tester 3.1 Type: Random Read - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 8KB - Disk Target: Default Test Directory zfs ssd mirror, ashift=13, recordsize 64k, qcow2 image zfs ssd mirror, ashift=13, recordsize 64k, raw image zfs ssd mirror, ashift=13, recordsize 64k, zvol zfs ssd mirror, ashift=13, recordsize 8k, qcow2 image zfs ssd mirror, ashift=13, recordsize 8k, raw image zfs ssd mirror, ashift=13, recordsize 8k, zvol 8 16 24 32 40 Min: 38.5 / Avg: 38.6 / Max: 38.8 Min: 38.9 / Avg: 38.93 / Max: 39 Min: 39.7 / Avg: 40.73 / Max: 41.4 Min: 39 / Avg: 40.65 / Max: 42.1 Min: 36 / Avg: 36.8 / Max: 37.2 Min: 38.5 / Avg: 39.2 / Max: 39.8 1. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -U_FORTIFY_SOURCE -lrt -laio -lm -lpthread -ldl
Result
OpenBenchmarking.org MB/s, More Is Better Flexible IO Tester 3.1 Type: Random Read - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 64KB - Disk Target: Default Test Directory zfs ssd mirror, ashift=13, recordsize 64k, qcow2 image zfs ssd mirror, ashift=13, recordsize 64k, raw image zfs ssd mirror, ashift=13, recordsize 64k, zvol zfs ssd mirror, ashift=13, recordsize 8k, qcow2 image zfs ssd mirror, ashift=13, recordsize 8k, raw image zfs ssd mirror, ashift=13, recordsize 8k, zvol 50 100 150 200 250 SE +/- 1.45, N = 3 SE +/- 0.58, N = 3 SE +/- 2.52, N = 3 SE +/- 0.88, N = 3 237 239 230 232 229 230 1. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -U_FORTIFY_SOURCE -lrt -laio -lm -lpthread -ldl
Result Confidence
OpenBenchmarking.org MB/s, More Is Better Flexible IO Tester 3.1 Type: Random Read - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 64KB - Disk Target: Default Test Directory zfs ssd mirror, ashift=13, recordsize 64k, qcow2 image zfs ssd mirror, ashift=13, recordsize 64k, raw image zfs ssd mirror, ashift=13, recordsize 64k, zvol zfs ssd mirror, ashift=13, recordsize 8k, qcow2 image zfs ssd mirror, ashift=13, recordsize 8k, raw image zfs ssd mirror, ashift=13, recordsize 8k, zvol 40 80 120 160 200 Min: 237 / Avg: 239.33 / Max: 242 Min: 229 / Avg: 230 / Max: 231 Min: 229 / Avg: 232 / Max: 237 Min: 229 / Avg: 230.33 / Max: 232 1. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -U_FORTIFY_SOURCE -lrt -laio -lm -lpthread -ldl
Compile Bench Compilebench tries to age a filesystem by simulating some of the disk IO common in creating, compiling, patching, stating and reading kernel trees. It indirectly measures how well filesystems can maintain directory locality as the disk fills up and directories age. This current test is setup to use the makej mode with 10 initial directories Learn more via the OpenBenchmarking.org test page.
Result
OpenBenchmarking.org MB/s, More Is Better Compile Bench 0.6 Test: Initial Create pve-ubuntu-local-1 zfs ssd mirror, ashift=13, recordsize 64k, qcow2 image zfs ssd mirror, ashift=13, recordsize 64k, raw image zfs ssd mirror, ashift=13, recordsize 64k, zvol zfs ssd mirror, ashift=13, recordsize 8k, qcow2 image zfs ssd mirror, ashift=13, recordsize 8k, raw image zfs ssd mirror, ashift=13, recordsize 8k, zvol 40 80 120 160 200 SE +/- 10.19, N = 3 SE +/- 6.42, N = 3 SE +/- 3.08, N = 3 SE +/- 5.96, N = 3 SE +/- 8.32, N = 3 SE +/- 1.91, N = 3 SE +/- 3.57, N = 3 182.82 82.40 86.85 50.11 42.54 48.40 49.32
Result Confidence
OpenBenchmarking.org MB/s, More Is Better Compile Bench 0.6 Test: Initial Create pve-ubuntu-local-1 zfs ssd mirror, ashift=13, recordsize 64k, qcow2 image zfs ssd mirror, ashift=13, recordsize 64k, raw image zfs ssd mirror, ashift=13, recordsize 64k, zvol zfs ssd mirror, ashift=13, recordsize 8k, qcow2 image zfs ssd mirror, ashift=13, recordsize 8k, raw image zfs ssd mirror, ashift=13, recordsize 8k, zvol 30 60 90 120 150 Min: 163.36 / Avg: 182.82 / Max: 197.79 Min: 69.68 / Avg: 82.4 / Max: 90.26 Min: 80.72 / Avg: 86.85 / Max: 90.52 Min: 39.26 / Avg: 50.11 / Max: 59.82 Min: 26.3 / Avg: 42.54 / Max: 53.76 Min: 44.83 / Avg: 48.4 / Max: 51.36 Min: 45.55 / Avg: 49.32 / Max: 56.46
Result
OpenBenchmarking.org MB/s, More Is Better Compile Bench 0.6 Test: Compile pve-ubuntu-local-1 zfs ssd mirror, ashift=13, recordsize 64k, qcow2 image zfs ssd mirror, ashift=13, recordsize 64k, raw image zfs ssd mirror, ashift=13, recordsize 64k, zvol zfs ssd mirror, ashift=13, recordsize 8k, qcow2 image zfs ssd mirror, ashift=13, recordsize 8k, raw image zfs ssd mirror, ashift=13, recordsize 8k, zvol 60 120 180 240 300 SE +/- 4.39, N = 6 SE +/- 5.01, N = 6 SE +/- 5.21, N = 6 SE +/- 1.40, N = 3 SE +/- 4.97, N = 6 SE +/- 2.12, N = 6 SE +/- 2.73, N = 6 262.57 142.88 147.18 81.59 61.65 72.13 91.69
Result Confidence
OpenBenchmarking.org MB/s, More Is Better Compile Bench 0.6 Test: Compile pve-ubuntu-local-1 zfs ssd mirror, ashift=13, recordsize 64k, qcow2 image zfs ssd mirror, ashift=13, recordsize 64k, raw image zfs ssd mirror, ashift=13, recordsize 64k, zvol zfs ssd mirror, ashift=13, recordsize 8k, qcow2 image zfs ssd mirror, ashift=13, recordsize 8k, raw image zfs ssd mirror, ashift=13, recordsize 8k, zvol 50 100 150 200 250 Min: 249.08 / Avg: 262.57 / Max: 277.57 Min: 118.1 / Avg: 142.88 / Max: 150.7 Min: 121.26 / Avg: 147.18 / Max: 154.92 Min: 79.68 / Avg: 81.59 / Max: 84.31 Min: 39.42 / Avg: 61.65 / Max: 74.62 Min: 66.78 / Avg: 72.13 / Max: 78.69 Min: 84.43 / Avg: 91.69 / Max: 99.93
Dbench Dbench is a benchmark designed by the Samba project as a free alternative to netbench, but dbench contains only file-system calls for testing the disk performance. Learn more via the OpenBenchmarking.org test page.
Result
OpenBenchmarking.org MB/s, More Is Better Dbench 4.0 Client Count: 128 pve-ubuntu-local-1 zfs ssd mirror, ashift=13, recordsize 64k, qcow2 image zfs ssd mirror, ashift=13, recordsize 64k, raw image zfs ssd mirror, ashift=13, recordsize 64k, zvol zfs ssd mirror, ashift=13, recordsize 8k, qcow2 image zfs ssd mirror, ashift=13, recordsize 8k, raw image zfs ssd mirror, ashift=13, recordsize 8k, zvol 70 140 210 280 350 SE +/- 6.16, N = 3 SE +/- 0.99, N = 3 SE +/- 2.00, N = 3 SE +/- 4.96, N = 6 SE +/- 5.76, N = 6 SE +/- 5.85, N = 6 SE +/- 4.52, N = 6 308.19 203.76 207.32 102.07 89.75 97.79 88.41 1. (CC) gcc options: -lpopt -O2
Result Confidence
OpenBenchmarking.org MB/s, More Is Better Dbench 4.0 Client Count: 128 pve-ubuntu-local-1 zfs ssd mirror, ashift=13, recordsize 64k, qcow2 image zfs ssd mirror, ashift=13, recordsize 64k, raw image zfs ssd mirror, ashift=13, recordsize 64k, zvol zfs ssd mirror, ashift=13, recordsize 8k, qcow2 image zfs ssd mirror, ashift=13, recordsize 8k, raw image zfs ssd mirror, ashift=13, recordsize 8k, zvol 60 120 180 240 300 Min: 296.13 / Avg: 308.19 / Max: 316.39 Min: 201.78 / Avg: 203.76 / Max: 204.88 Min: 203.47 / Avg: 207.32 / Max: 210.18 Min: 84.97 / Avg: 102.07 / Max: 118.02 Min: 71.9 / Avg: 89.75 / Max: 110.12 Min: 79.78 / Avg: 97.79 / Max: 118.14 Min: 75.11 / Avg: 88.41 / Max: 104.93 1. (CC) gcc options: -lpopt -O2
Result
OpenBenchmarking.org MB/s, More Is Better Dbench 4.0 Client Count: 48 pve-ubuntu-local-1 zfs ssd mirror, ashift=13, recordsize 64k, qcow2 image zfs ssd mirror, ashift=13, recordsize 64k, raw image zfs ssd mirror, ashift=13, recordsize 64k, zvol zfs ssd mirror, ashift=13, recordsize 8k, qcow2 image zfs ssd mirror, ashift=13, recordsize 8k, raw image zfs ssd mirror, ashift=13, recordsize 8k, zvol 200 400 600 800 1000 SE +/- 2.73, N = 3 SE +/- 4.38, N = 3 SE +/- 4.21, N = 3 SE +/- 6.43, N = 6 SE +/- 7.08, N = 6 SE +/- 7.08, N = 6 SE +/- 6.84, N = 6 792.05 294.03 292.22 177.89 176.41 181.38 153.63 1. (CC) gcc options: -lpopt -O2
Result Confidence
OpenBenchmarking.org MB/s, More Is Better Dbench 4.0 Client Count: 48 pve-ubuntu-local-1 zfs ssd mirror, ashift=13, recordsize 64k, qcow2 image zfs ssd mirror, ashift=13, recordsize 64k, raw image zfs ssd mirror, ashift=13, recordsize 64k, zvol zfs ssd mirror, ashift=13, recordsize 8k, qcow2 image zfs ssd mirror, ashift=13, recordsize 8k, raw image zfs ssd mirror, ashift=13, recordsize 8k, zvol 140 280 420 560 700 Min: 786.89 / Avg: 792.05 / Max: 796.19 Min: 288.73 / Avg: 294.03 / Max: 302.71 Min: 284.17 / Avg: 292.22 / Max: 298.38 Min: 159.34 / Avg: 177.89 / Max: 197.47 Min: 153.47 / Avg: 176.41 / Max: 196.04 Min: 157.66 / Avg: 181.38 / Max: 203.24 Min: 135.63 / Avg: 153.63 / Max: 179.58 1. (CC) gcc options: -lpopt -O2
Flexible IO Tester
Result
OpenBenchmarking.org MB/s, More Is Better Flexible IO Tester 3.1 Type: Sequential Write - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 8KB - Disk Target: Default Test Directory zfs ssd mirror, ashift=13, recordsize 64k, qcow2 image zfs ssd mirror, ashift=13, recordsize 64k, raw image zfs ssd mirror, ashift=13, recordsize 64k, zvol zfs ssd mirror, ashift=13, recordsize 8k, qcow2 image zfs ssd mirror, ashift=13, recordsize 8k, raw image zfs ssd mirror, ashift=13, recordsize 8k, zvol 11 22 33 44 55 SE +/- 1.72, N = 6 SE +/- 1.80, N = 6 SE +/- 0.64, N = 6 SE +/- 1.80, N = 6 SE +/- 1.78, N = 6 SE +/- 0.57, N = 6 46.30 47.10 29.77 32.70 41.83 24.15 1. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -U_FORTIFY_SOURCE -lrt -laio -lm -lpthread -ldl
Result Confidence
OpenBenchmarking.org MB/s, More Is Better Flexible IO Tester 3.1 Type: Sequential Write - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 8KB - Disk Target: Default Test Directory zfs ssd mirror, ashift=13, recordsize 64k, qcow2 image zfs ssd mirror, ashift=13, recordsize 64k, raw image zfs ssd mirror, ashift=13, recordsize 64k, zvol zfs ssd mirror, ashift=13, recordsize 8k, qcow2 image zfs ssd mirror, ashift=13, recordsize 8k, raw image zfs ssd mirror, ashift=13, recordsize 8k, zvol 10 20 30 40 50 Min: 40.7 / Avg: 46.3 / Max: 51.1 Min: 40 / Avg: 47.1 / Max: 50.6 Min: 27.4 / Avg: 29.77 / Max: 31.8 Min: 25 / Avg: 32.7 / Max: 36.4 Min: 35 / Avg: 41.83 / Max: 46.6 Min: 21.9 / Avg: 24.15 / Max: 25.3 1. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -U_FORTIFY_SOURCE -lrt -laio -lm -lpthread -ldl
Result
OpenBenchmarking.org MB/s, More Is Better Flexible IO Tester 3.1 Type: Sequential Read - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 8KB - Disk Target: Default Test Directory zfs ssd mirror, ashift=13, recordsize 64k, qcow2 image zfs ssd mirror, ashift=13, recordsize 64k, raw image zfs ssd mirror, ashift=13, recordsize 64k, zvol zfs ssd mirror, ashift=13, recordsize 8k, qcow2 image zfs ssd mirror, ashift=13, recordsize 8k, raw image zfs ssd mirror, ashift=13, recordsize 8k, zvol 13 26 39 52 65 SE +/- 1.33, N = 6 SE +/- 0.23, N = 3 SE +/- 0.32, N = 3 SE +/- 1.52, N = 6 SE +/- 2.11, N = 6 SE +/- 1.10, N = 6 54.17 55.50 44.73 45.85 58.32 45.02 1. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -U_FORTIFY_SOURCE -lrt -laio -lm -lpthread -ldl
Result Confidence
OpenBenchmarking.org MB/s, More Is Better Flexible IO Tester 3.1 Type: Sequential Read - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 8KB - Disk Target: Default Test Directory zfs ssd mirror, ashift=13, recordsize 64k, qcow2 image zfs ssd mirror, ashift=13, recordsize 64k, raw image zfs ssd mirror, ashift=13, recordsize 64k, zvol zfs ssd mirror, ashift=13, recordsize 8k, qcow2 image zfs ssd mirror, ashift=13, recordsize 8k, raw image zfs ssd mirror, ashift=13, recordsize 8k, zvol 12 24 36 48 60 Min: 50.3 / Avg: 54.17 / Max: 59.2 Min: 55.1 / Avg: 55.5 / Max: 55.9 Min: 44.2 / Avg: 44.73 / Max: 45.3 Min: 42 / Avg: 45.85 / Max: 50.6 Min: 52.5 / Avg: 58.32 / Max: 65.6 Min: 41.3 / Avg: 45.02 / Max: 46.9 1. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -U_FORTIFY_SOURCE -lrt -laio -lm -lpthread -ldl
zfs ssd mirror, ashift=13, recordsize 64k, qcow2 image Compiler Notes: --build=x86_64-linux-gnu --disable-browser-plugin --disable-vtable-verify --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-gtk-cairo --enable-java-awt=gtk --enable-java-home --enable-languages=c,ada,c++,java,go,d,fortran,objc,obj-c++ --enable-libmpx --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-arch-directory=amd64 --with-default-libstdcxx-abi=new --with-multilib-list=m32,m64,mx32 --with-target-system-zlib --with-tune=generic -vDisk Notes: CFQ / data=ordered,errors=remount-ro,relatime,rwSystem Notes: Python 2.7.13.
Testing initiated at 17 March 2018 09:29 by user root.
zfs ssd mirror, ashift=13, recordsize 64k, raw image Compiler Notes: --build=x86_64-linux-gnu --disable-browser-plugin --disable-vtable-verify --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-gtk-cairo --enable-java-awt=gtk --enable-java-home --enable-languages=c,ada,c++,java,go,d,fortran,objc,obj-c++ --enable-libmpx --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-arch-directory=amd64 --with-default-libstdcxx-abi=new --with-multilib-list=m32,m64,mx32 --with-target-system-zlib --with-tune=generic -vDisk Notes: CFQ / data=ordered,errors=remount-ro,relatime,rwSystem Notes: Python 2.7.13.
Testing initiated at 16 March 2018 10:57 by user root.
zfs ssd mirror, ashift=13, recordsize 64k, zvol Compiler Notes: --build=x86_64-linux-gnu --disable-browser-plugin --disable-vtable-verify --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-gtk-cairo --enable-java-awt=gtk --enable-java-home --enable-languages=c,ada,c++,java,go,d,fortran,objc,obj-c++ --enable-libmpx --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-arch-directory=amd64 --with-default-libstdcxx-abi=new --with-multilib-list=m32,m64,mx32 --with-target-system-zlib --with-tune=generic -vDisk Notes: CFQ / data=ordered,errors=remount-ro,relatime,rwSystem Notes: Python 2.7.13.
Testing initiated at 18 March 2018 15:45 by user root.
zfs ssd mirror, ashift=13, recordsize 8k, qcow2 image Compiler Notes: --build=x86_64-linux-gnu --disable-browser-plugin --disable-vtable-verify --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-gtk-cairo --enable-java-awt=gtk --enable-java-home --enable-languages=c,ada,c++,java,go,d,fortran,objc,obj-c++ --enable-libmpx --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-arch-directory=amd64 --with-default-libstdcxx-abi=new --with-multilib-list=m32,m64,mx32 --with-target-system-zlib --with-tune=generic -vDisk Notes: CFQ / data=ordered,errors=remount-ro,relatime,rwSystem Notes: Python 2.7.13.
Testing initiated at 16 March 2018 17:39 by user root.
zfs ssd mirror, ashift=13, recordsize 8k, raw image Compiler Notes: --build=x86_64-linux-gnu --disable-browser-plugin --disable-vtable-verify --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-gtk-cairo --enable-java-awt=gtk --enable-java-home --enable-languages=c,ada,c++,java,go,d,fortran,objc,obj-c++ --enable-libmpx --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-arch-directory=amd64 --with-default-libstdcxx-abi=new --with-multilib-list=m32,m64,mx32 --with-target-system-zlib --with-tune=generic -vDisk Notes: CFQ / data=ordered,errors=remount-ro,relatime,rwSystem Notes: Python 2.7.13.
Testing initiated at 15 March 2018 18:44 by user root.
zfs ssd mirror, ashift=13, recordsize 8k, zvol Processor: 8 x Intel Celeron_4x0 (Conroe/Merom Class Core 2) @ 1.86GHz (8 Cores), Motherboard: QEMU Standard PC (i440FX + PIIX 1996), Chipset: Intel 440FX- 82441FX PMC, Memory: 1 x 1024 MB RAM QEMU, Disk: 54GB QEMU HDD, Graphics: Cirrus Logic GD 5446, Network: Realtek RTL-8100/8101L/8139
OS: Debian 9.1, Kernel: 4.9.0-4-amd64 (x86_64), Compiler: GCC 6.3.0 20170516, File-System: ext4, Screen Resolution: 1024x768, System Layer: qemu
Compiler Notes: --build=x86_64-linux-gnu --disable-browser-plugin --disable-vtable-verify --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-gtk-cairo --enable-java-awt=gtk --enable-java-home --enable-languages=c,ada,c++,java,go,d,fortran,objc,obj-c++ --enable-libmpx --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-arch-directory=amd64 --with-default-libstdcxx-abi=new --with-multilib-list=m32,m64,mx32 --with-target-system-zlib --with-tune=generic -vDisk Notes: CFQ / data=ordered,errors=remount-ro,relatime,rwSystem Notes: Python 2.7.13.
Testing initiated at 17 March 2018 16:46 by user root.
pve-ubuntu-local-1 Processor: Common KVM @ 3.09GHz (2 Cores), Motherboard: QEMU Standard PC (i440FX + PIIX 1996), Chipset: Intel 440FX- 82441FX PMC, Memory: 2048MB, Disk: 34GB QEMU HDD, Graphics: Device 1234:1111, Network: Red Hat Virtio device
OS: Ubuntu 16.04, Kernel: 4.4.0-116-generic (x86_64), Compiler: GCC 5.4.0 20160609, File-System: ext4, Screen Resolution: 800x600, System Layer: qemu
Compiler Notes: --build=x86_64-linux-gnu --disable-browser-plugin --disable-vtable-verify --disable-werror --enable-checking=release --enable-clocale=gnu --enable-gnu-unique-object --enable-gtk-cairo --enable-java-awt=gtk --enable-java-home --enable-languages=c,ada,c++,java,go,d,fortran,objc,obj-c++ --enable-libmpx --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-arch-directory=amd64 --with-default-libstdcxx-abi=new --with-multilib-list=m32,m64,mx32 --with-tune=generic -vDisk Notes: DEADLINE / data=ordered,errors=remount-ro,relatime,rwSystem Notes: Disk Scheduler: DEADLINE. Python 2.7.12.
Testing initiated at 24 March 2018 19:55 by user jj.