ZFS SSD Benchmark: RAW IMAGE vs QCOW2 vs ZVOL for KVM This benchmark show’s the performance of a zfs pool providing storage to a kvm virtual machine with three different formats: raw image files on plain dataset qcow2 image file on plain dataset zvol For each of those several filesystem benchmarks from the phoronix-benchmark-suite are run with two different recordsizes: 8k 64k Host Details: cpu: Intel® Xeon® CPU E5320 @ 1.86GHz ram: 6GB DDR2 sata: Intel Corporation 631xESB/632xESB SATA AHCI Controller (rev 09 os: Debian GNU/Linux 9.4 (stretch), fully upgraded as of 2018-03-15 zfs: 0.7.6-1~bpo9+1 (from stretch-backports) ssd: 2x samsung 840 pro 256GB KVM machine: cpu: host ram: 1024MB disk: bus=scsi,cache=writeback,io=threads os: Debian GNU/Linux 9.1 (stretch) fs: ext4 The zfs pool used consists of a single mirrored vdev with samsung 840 pro ssd’s. Each benchmark is run like this: drop all caches trim whole ssd’s create zfs pool (ashift=13,compression=lz4,atime=on,relatime=on benchmark,devices=off,exec=off,setuid=off,xattr=off) create disk: qemu-img create -f raw debian9.raw 50G qemu-img create -f qcow2 -o cluster_size=8k,preallocation=metadata,compat=1.1,lazy_refcounts=on debian9.qcow2 50G zfs create -o volblocksize=8k -V 50G benchmark/kvm/debian9 create kvm machine take timestamp let debian9 install automatically save install time install phoronix-test-suite and needed dependencies in the vm run the pts trim the filesystem and save the time it took copy the results to the kvm host destroy vm and zpool other test results: 8k_raw - install time: 619s, trim: 39.401s 64k raw - install time: 589s, trim: 0m9.593s 8k qcow2 - install time: 615s, trim: 1m45.421s 64k qcow2 - install time: 590s, trim: 0m7.598s 8k zvol - install time: 621s, trim: 0m11.353s 64k zvol - install time: 605s zfs ssd mirror, ashift=13, recordsize 64k, qcow2 image: Processor: 8 x Intel Celeron_4x0 (Conroe/Merom Class Core 2) @ 1.86GHz (8 Cores), Motherboard: QEMU Standard PC (i440FX + PIIX 1996), Chipset: Intel 440FX- 82441FX PMC, Memory: 1 x 1024 MB RAM QEMU, Disk: 54GB QEMU HDD, Graphics: Cirrus Logic GD 5446, Network: Realtek RTL-8100/8101L/8139 OS: Debian 9.1, Kernel: 4.9.0-4-amd64 (x86_64), Compiler: GCC 6.3.0 20170516, File-System: ext4, Screen Resolution: 1024x768, System Layer: qemu zfs ssd mirror, ashift=13, recordsize 64k, raw image: Processor: 8 x Intel Celeron_4x0 (Conroe/Merom Class Core 2) @ 1.86GHz (8 Cores), Motherboard: QEMU Standard PC (i440FX + PIIX 1996), Chipset: Intel 440FX- 82441FX PMC, Memory: 1 x 1024 MB RAM QEMU, Disk: 54GB QEMU HDD, Graphics: Cirrus Logic GD 5446, Network: Realtek RTL-8100/8101L/8139 OS: Debian 9.1, Kernel: 4.9.0-4-amd64 (x86_64), Compiler: GCC 6.3.0 20170516, File-System: ext4, Screen Resolution: 1024x768, System Layer: qemu zfs ssd mirror, ashift=13, recordsize 64k, zvol: Processor: 8 x Intel Celeron_4x0 (Conroe/Merom Class Core 2) @ 1.86GHz (8 Cores), Motherboard: QEMU Standard PC (i440FX + PIIX 1996), Chipset: Intel 440FX- 82441FX PMC, Memory: 1 x 1024 MB RAM QEMU, Disk: 54GB QEMU HDD, Graphics: Cirrus Logic GD 5446, Network: Realtek RTL-8100/8101L/8139 OS: Debian 9.1, Kernel: 4.9.0-4-amd64 (x86_64), Compiler: GCC 6.3.0 20170516, File-System: ext4, Screen Resolution: 1024x768, System Layer: qemu zfs ssd mirror, ashift=13, recordsize 8k, qcow2 image: Processor: 8 x Intel Celeron_4x0 (Conroe/Merom Class Core 2) @ 1.86GHz (8 Cores), Motherboard: QEMU Standard PC (i440FX + PIIX 1996), Chipset: Intel 440FX- 82441FX PMC, Memory: 1 x 1024 MB RAM QEMU, Disk: 54GB QEMU HDD, Graphics: Cirrus Logic GD 5446, Network: Realtek RTL-8100/8101L/8139 OS: Debian 9.1, Kernel: 4.9.0-4-amd64 (x86_64), Compiler: GCC 6.3.0 20170516, File-System: ext4, Screen Resolution: 1024x768, System Layer: qemu zfs ssd mirror, ashift=13, recordsize 8k, raw image: Processor: 8 x Intel Celeron_4x0 (Conroe/Merom Class Core 2) @ 1.86GHz (8 Cores), Motherboard: QEMU Standard PC (i440FX + PIIX 1996), Chipset: Intel 440FX- 82441FX PMC, Memory: 1 x 1024 MB RAM QEMU, Disk: 54GB QEMU HDD, Graphics: Cirrus Logic GD 5446, Network: Realtek RTL-8100/8101L/8139 OS: Debian 9.1, Kernel: 4.9.0-4-amd64 (x86_64), Compiler: GCC 6.3.0 20170516, File-System: ext4, Screen Resolution: 1024x768, System Layer: qemu zfs ssd mirror, ashift=13, recordsize 8k, zvol: Processor: 8 x Intel Celeron_4x0 (Conroe/Merom Class Core 2) @ 1.86GHz (8 Cores), Motherboard: QEMU Standard PC (i440FX + PIIX 1996), Chipset: Intel 440FX- 82441FX PMC, Memory: 1 x 1024 MB RAM QEMU, Disk: 54GB QEMU HDD, Graphics: Cirrus Logic GD 5446, Network: Realtek RTL-8100/8101L/8139 OS: Debian 9.1, Kernel: 4.9.0-4-amd64 (x86_64), Compiler: GCC 6.3.0 20170516, File-System: ext4, Screen Resolution: 1024x768, System Layer: qemu AIO-Stress 0.21 Test: Random Write MB/s > Higher Is Better zfs ssd mirror, ashift=13, recordsize 64k, qcow2 image . 125.74 |============= zfs ssd mirror, ashift=13, recordsize 64k, raw image ... 131.81 |============== zfs ssd mirror, ashift=13, recordsize 64k, zvol ........ 111.33 |============ zfs ssd mirror, ashift=13, recordsize 8k, qcow2 image .. 93.70 |========== zfs ssd mirror, ashift=13, recordsize 8k, raw image .... 98.01 |========== zfs ssd mirror, ashift=13, recordsize 8k, zvol ......... 100.87 |=========== Flexible IO Tester 3.1 Type: Random Read - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 1MB - Disk Target: Default Test Directory MB/s > Higher Is Better zfs ssd mirror, ashift=13, recordsize 64k, qcow2 image . 292 |========== zfs ssd mirror, ashift=13, recordsize 64k, raw image ... 294 |=========== zfs ssd mirror, ashift=13, recordsize 64k, zvol ........ 454 |================ zfs ssd mirror, ashift=13, recordsize 8k, qcow2 image .. 228 |======== zfs ssd mirror, ashift=13, recordsize 8k, raw image .... 230 |======== zfs ssd mirror, ashift=13, recordsize 8k, zvol ......... 473 |================= Flexible IO Tester 3.1 Type: Random Read - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 8KB - Disk Target: Default Test Directory MB/s > Higher Is Better zfs ssd mirror, ashift=13, recordsize 64k, qcow2 image . 38.60 |============== zfs ssd mirror, ashift=13, recordsize 64k, raw image ... 38.93 |============== zfs ssd mirror, ashift=13, recordsize 64k, zvol ........ 40.73 |=============== zfs ssd mirror, ashift=13, recordsize 8k, qcow2 image .. 40.65 |=============== zfs ssd mirror, ashift=13, recordsize 8k, raw image .... 36.80 |============== zfs ssd mirror, ashift=13, recordsize 8k, zvol ......... 39.20 |============== Flexible IO Tester 3.1 Type: Random Read - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 64KB - Disk Target: Default Test Directory MB/s > Higher Is Better zfs ssd mirror, ashift=13, recordsize 64k, qcow2 image . 237 |================= zfs ssd mirror, ashift=13, recordsize 64k, raw image ... 239 |================= zfs ssd mirror, ashift=13, recordsize 64k, zvol ........ 230 |================ zfs ssd mirror, ashift=13, recordsize 8k, qcow2 image .. 232 |================= zfs ssd mirror, ashift=13, recordsize 8k, raw image .... 229 |================ zfs ssd mirror, ashift=13, recordsize 8k, zvol ......... 230 |================ Flexible IO Tester 3.1 Type: Random Write - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 1MB - Disk Target: Default Test Directory MB/s > Higher Is Better zfs ssd mirror, ashift=13, recordsize 64k, qcow2 image . 143.00 |============== zfs ssd mirror, ashift=13, recordsize 64k, raw image ... 146.00 |============== zfs ssd mirror, ashift=13, recordsize 64k, zvol ........ 54.75 |===== zfs ssd mirror, ashift=13, recordsize 8k, qcow2 image .. 70.23 |======= zfs ssd mirror, ashift=13, recordsize 8k, raw image .... 71.20 |======= zfs ssd mirror, ashift=13, recordsize 8k, zvol ......... 42.05 |==== Flexible IO Tester 3.1 Type: Random Write - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 8KB - Disk Target: Default Test Directory MB/s > Higher Is Better zfs ssd mirror, ashift=13, recordsize 64k, qcow2 image . 15.97 |======== zfs ssd mirror, ashift=13, recordsize 64k, raw image ... 17.20 |========= zfs ssd mirror, ashift=13, recordsize 64k, zvol ........ 12.05 |====== zfs ssd mirror, ashift=13, recordsize 8k, qcow2 image .. 25.73 |============= zfs ssd mirror, ashift=13, recordsize 8k, raw image .... 29.20 |=============== zfs ssd mirror, ashift=13, recordsize 8k, zvol ......... 20.70 |=========== Flexible IO Tester 3.1 Type: Random Write - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 64KB - Disk Target: Default Test Directory MB/s > Higher Is Better zfs ssd mirror, ashift=13, recordsize 64k, qcow2 image . 113.00 |============= zfs ssd mirror, ashift=13, recordsize 64k, raw image ... 118.00 |============== zfs ssd mirror, ashift=13, recordsize 64k, zvol ........ 66.40 |======== zfs ssd mirror, ashift=13, recordsize 8k, qcow2 image .. 59.40 |======= zfs ssd mirror, ashift=13, recordsize 8k, raw image .... 61.33 |======= zfs ssd mirror, ashift=13, recordsize 8k, zvol ......... 38.75 |===== Flexible IO Tester 3.1 Type: Sequential Read - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 1MB - Disk Target: Default Test Directory MB/s > Higher Is Better zfs ssd mirror, ashift=13, recordsize 64k, qcow2 image . 291 |========== zfs ssd mirror, ashift=13, recordsize 64k, raw image ... 289 |========== zfs ssd mirror, ashift=13, recordsize 64k, zvol ........ 458 |================ zfs ssd mirror, ashift=13, recordsize 8k, qcow2 image .. 226 |======== zfs ssd mirror, ashift=13, recordsize 8k, raw image .... 229 |======== zfs ssd mirror, ashift=13, recordsize 8k, zvol ......... 472 |================= Flexible IO Tester 3.1 Type: Sequential Read - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 8KB - Disk Target: Default Test Directory MB/s > Higher Is Better zfs ssd mirror, ashift=13, recordsize 64k, qcow2 image . 54.17 |============== zfs ssd mirror, ashift=13, recordsize 64k, raw image ... 55.50 |============== zfs ssd mirror, ashift=13, recordsize 64k, zvol ........ 44.73 |============ zfs ssd mirror, ashift=13, recordsize 8k, qcow2 image .. 45.85 |============ zfs ssd mirror, ashift=13, recordsize 8k, raw image .... 58.32 |=============== zfs ssd mirror, ashift=13, recordsize 8k, zvol ......... 45.02 |============ Flexible IO Tester 3.1 Type: Sequential Read - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 64KB - Disk Target: Default Test Directory MB/s > Higher Is Better zfs ssd mirror, ashift=13, recordsize 64k, qcow2 image . 306 |=============== zfs ssd mirror, ashift=13, recordsize 64k, raw image ... 303 |=============== zfs ssd mirror, ashift=13, recordsize 64k, zvol ........ 347 |================= zfs ssd mirror, ashift=13, recordsize 8k, qcow2 image .. 244 |============ zfs ssd mirror, ashift=13, recordsize 8k, raw image .... 252 |============ zfs ssd mirror, ashift=13, recordsize 8k, zvol ......... 353 |================= Flexible IO Tester 3.1 Type: Sequential Write - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 1MB - Disk Target: Default Test Directory MB/s > Higher Is Better zfs ssd mirror, ashift=13, recordsize 64k, qcow2 image . 147.00 |============== zfs ssd mirror, ashift=13, recordsize 64k, raw image ... 152.00 |============== zfs ssd mirror, ashift=13, recordsize 64k, zvol ........ 66.28 |====== zfs ssd mirror, ashift=13, recordsize 8k, qcow2 image .. 72.03 |======= zfs ssd mirror, ashift=13, recordsize 8k, raw image .... 74.37 |======= zfs ssd mirror, ashift=13, recordsize 8k, zvol ......... 45.33 |==== Flexible IO Tester 3.1 Type: Sequential Write - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 8KB - Disk Target: Default Test Directory MB/s > Higher Is Better zfs ssd mirror, ashift=13, recordsize 64k, qcow2 image . 46.30 |=============== zfs ssd mirror, ashift=13, recordsize 64k, raw image ... 47.10 |=============== zfs ssd mirror, ashift=13, recordsize 64k, zvol ........ 29.77 |========= zfs ssd mirror, ashift=13, recordsize 8k, qcow2 image .. 32.70 |========== zfs ssd mirror, ashift=13, recordsize 8k, raw image .... 41.83 |============= zfs ssd mirror, ashift=13, recordsize 8k, zvol ......... 24.15 |======== Flexible IO Tester 3.1 Type: Sequential Write - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 64KB - Disk Target: Default Test Directory MB/s > Higher Is Better zfs ssd mirror, ashift=13, recordsize 64k, qcow2 image . 139.00 |============== zfs ssd mirror, ashift=13, recordsize 64k, raw image ... 138.00 |============== zfs ssd mirror, ashift=13, recordsize 64k, zvol ........ 71.37 |======= zfs ssd mirror, ashift=13, recordsize 8k, qcow2 image .. 70.77 |======= zfs ssd mirror, ashift=13, recordsize 8k, raw image .... 74.27 |======= zfs ssd mirror, ashift=13, recordsize 8k, zvol ......... 36.43 |==== Dbench 4.0 Client Count: 1 MB/s > Higher Is Better zfs ssd mirror, ashift=13, recordsize 64k, qcow2 image . 42.33 |=============== zfs ssd mirror, ashift=13, recordsize 64k, raw image ... 42.60 |=============== zfs ssd mirror, ashift=13, recordsize 64k, zvol ........ 42.72 |=============== zfs ssd mirror, ashift=13, recordsize 8k, qcow2 image .. 42.20 |=============== zfs ssd mirror, ashift=13, recordsize 8k, raw image .... 42.18 |=============== zfs ssd mirror, ashift=13, recordsize 8k, zvol ......... 40.87 |============== Dbench 4.0 Client Count: 6 MB/s > Higher Is Better zfs ssd mirror, ashift=13, recordsize 64k, qcow2 image . 102.98 |============== zfs ssd mirror, ashift=13, recordsize 64k, raw image ... 105.25 |============== zfs ssd mirror, ashift=13, recordsize 64k, zvol ........ 88.54 |============ zfs ssd mirror, ashift=13, recordsize 8k, qcow2 image .. 89.99 |============ zfs ssd mirror, ashift=13, recordsize 8k, raw image .... 91.96 |============ zfs ssd mirror, ashift=13, recordsize 8k, zvol ......... 86.79 |============ Dbench 4.0 Client Count: 12 MB/s > Higher Is Better zfs ssd mirror, ashift=13, recordsize 64k, qcow2 image . 162.59 |============== zfs ssd mirror, ashift=13, recordsize 64k, raw image ... 166.12 |============== zfs ssd mirror, ashift=13, recordsize 64k, zvol ........ 133.55 |=========== zfs ssd mirror, ashift=13, recordsize 8k, qcow2 image .. 137.50 |============ zfs ssd mirror, ashift=13, recordsize 8k, raw image .... 137.66 |============ zfs ssd mirror, ashift=13, recordsize 8k, zvol ......... 128.76 |=========== Dbench 4.0 Client Count: 48 MB/s > Higher Is Better zfs ssd mirror, ashift=13, recordsize 64k, qcow2 image . 294.03 |============== zfs ssd mirror, ashift=13, recordsize 64k, raw image ... 292.22 |============== zfs ssd mirror, ashift=13, recordsize 64k, zvol ........ 177.89 |======== zfs ssd mirror, ashift=13, recordsize 8k, qcow2 image .. 176.41 |======== zfs ssd mirror, ashift=13, recordsize 8k, raw image .... 181.38 |========= zfs ssd mirror, ashift=13, recordsize 8k, zvol ......... 153.63 |======= Dbench 4.0 Client Count: 128 MB/s > Higher Is Better zfs ssd mirror, ashift=13, recordsize 64k, qcow2 image . 203.76 |============== zfs ssd mirror, ashift=13, recordsize 64k, raw image ... 207.32 |============== zfs ssd mirror, ashift=13, recordsize 64k, zvol ........ 102.07 |======= zfs ssd mirror, ashift=13, recordsize 8k, qcow2 image .. 89.75 |====== zfs ssd mirror, ashift=13, recordsize 8k, raw image .... 97.79 |======= zfs ssd mirror, ashift=13, recordsize 8k, zvol ......... 88.41 |====== Dbench 4.0 Client Count: 256 MB/s > Higher Is Better zfs ssd mirror, ashift=13, recordsize 64k, qcow2 image . 118.70 |============== zfs ssd mirror, ashift=13, recordsize 64k, raw image ... 120.18 |============== zfs ssd mirror, ashift=13, recordsize 64k, zvol ........ 62.00 |======= zfs ssd mirror, ashift=13, recordsize 8k, qcow2 image .. 55.57 |====== zfs ssd mirror, ashift=13, recordsize 8k, raw image .... 55.77 |====== zfs ssd mirror, ashift=13, recordsize 8k, zvol ......... 58.47 |======= IOzone 3.465 Record Size: 1MB - File Size: 8GB - Disk Test: Read Performance MB/s > Higher Is Better zfs ssd mirror, ashift=13, recordsize 64k, qcow2 image . 215.34 |============== zfs ssd mirror, ashift=13, recordsize 64k, raw image ... 216.63 |============== zfs ssd mirror, ashift=13, recordsize 64k, zvol ........ 86.07 |====== zfs ssd mirror, ashift=13, recordsize 8k, qcow2 image .. 165.90 |=========== zfs ssd mirror, ashift=13, recordsize 8k, raw image .... 167.07 |=========== zfs ssd mirror, ashift=13, recordsize 8k, zvol ......... 80.50 |===== IOzone 3.465 Record Size: 4Kb - File Size: 8GB - Disk Test: Read Performance MB/s > Higher Is Better zfs ssd mirror, ashift=13, recordsize 64k, qcow2 image . 223.31 |============== zfs ssd mirror, ashift=13, recordsize 64k, raw image ... 224.74 |============== zfs ssd mirror, ashift=13, recordsize 64k, zvol ........ 94.99 |====== zfs ssd mirror, ashift=13, recordsize 8k, qcow2 image .. 173.68 |=========== zfs ssd mirror, ashift=13, recordsize 8k, raw image .... 174.63 |=========== zfs ssd mirror, ashift=13, recordsize 8k, zvol ......... 84.95 |===== IOzone 3.465 Record Size: 1MB - File Size: 8GB - Disk Test: Write Performance MB/s > Higher Is Better zfs ssd mirror, ashift=13, recordsize 64k, qcow2 image . 156.30 |============== zfs ssd mirror, ashift=13, recordsize 64k, raw image ... 157.79 |============== zfs ssd mirror, ashift=13, recordsize 64k, zvol ........ 66.79 |====== zfs ssd mirror, ashift=13, recordsize 8k, qcow2 image .. 65.22 |====== zfs ssd mirror, ashift=13, recordsize 8k, raw image .... 60.00 |===== zfs ssd mirror, ashift=13, recordsize 8k, zvol ......... 72.07 |====== IOzone 3.465 Record Size: 4Kb - File Size: 8GB - Disk Test: Write Performance MB/s > Higher Is Better zfs ssd mirror, ashift=13, recordsize 64k, qcow2 image . 161.59 |============== zfs ssd mirror, ashift=13, recordsize 64k, raw image ... 162.04 |============== zfs ssd mirror, ashift=13, recordsize 64k, zvol ........ 63.10 |===== zfs ssd mirror, ashift=13, recordsize 8k, qcow2 image .. 61.43 |===== zfs ssd mirror, ashift=13, recordsize 8k, raw image .... 57.00 |===== zfs ssd mirror, ashift=13, recordsize 8k, zvol ......... 74.06 |====== IOzone 3.465 Record Size: 64Kb - File Size: 8GB - Disk Test: Read Performance MB/s > Higher Is Better zfs ssd mirror, ashift=13, recordsize 64k, qcow2 image . 225.08 |============== zfs ssd mirror, ashift=13, recordsize 64k, raw image ... 226.37 |============== zfs ssd mirror, ashift=13, recordsize 64k, zvol ........ 88.90 |===== zfs ssd mirror, ashift=13, recordsize 8k, qcow2 image .. 174.56 |=========== zfs ssd mirror, ashift=13, recordsize 8k, raw image .... 172.38 |=========== zfs ssd mirror, ashift=13, recordsize 8k, zvol ......... 84.86 |===== IOzone 3.465 Record Size: 64Kb - File Size: 8GB - Disk Test: Write Performance MB/s > Higher Is Better zfs ssd mirror, ashift=13, recordsize 64k, qcow2 image . 156.04 |============== zfs ssd mirror, ashift=13, recordsize 64k, raw image ... 160.28 |============== zfs ssd mirror, ashift=13, recordsize 64k, zvol ........ 60.03 |===== zfs ssd mirror, ashift=13, recordsize 8k, qcow2 image .. 58.87 |===== zfs ssd mirror, ashift=13, recordsize 8k, raw image .... 55.72 |===== zfs ssd mirror, ashift=13, recordsize 8k, zvol ......... 67.62 |====== Compile Bench 0.6 Test: Compile MB/s > Higher Is Better zfs ssd mirror, ashift=13, recordsize 64k, qcow2 image . 142.88 |============== zfs ssd mirror, ashift=13, recordsize 64k, raw image ... 147.18 |============== zfs ssd mirror, ashift=13, recordsize 64k, zvol ........ 81.59 |======== zfs ssd mirror, ashift=13, recordsize 8k, qcow2 image .. 61.65 |====== zfs ssd mirror, ashift=13, recordsize 8k, raw image .... 72.13 |======= zfs ssd mirror, ashift=13, recordsize 8k, zvol ......... 91.69 |========= Compile Bench 0.6 Test: Initial Create MB/s > Higher Is Better zfs ssd mirror, ashift=13, recordsize 64k, qcow2 image . 82.40 |============== zfs ssd mirror, ashift=13, recordsize 64k, raw image ... 86.85 |=============== zfs ssd mirror, ashift=13, recordsize 64k, zvol ........ 50.11 |========= zfs ssd mirror, ashift=13, recordsize 8k, qcow2 image .. 42.54 |======= zfs ssd mirror, ashift=13, recordsize 8k, raw image .... 48.40 |======== zfs ssd mirror, ashift=13, recordsize 8k, zvol ......... 49.32 |========= Compile Bench 0.6 Test: Read Compiled Tree MB/s > Higher Is Better zfs ssd mirror, ashift=13, recordsize 64k, qcow2 image . 64.69 |=============== zfs ssd mirror, ashift=13, recordsize 64k, raw image ... 64.83 |=============== zfs ssd mirror, ashift=13, recordsize 64k, zvol ........ 60.72 |============== zfs ssd mirror, ashift=13, recordsize 8k, qcow2 image .. 56.72 |============= zfs ssd mirror, ashift=13, recordsize 8k, raw image .... 55.72 |============= zfs ssd mirror, ashift=13, recordsize 8k, zvol ......... 46.58 |===========