truenas-20210502fio KVM testing on Ubuntu 20.04 via the Phoronix Test Suite. iSCSI Sync L2ARC No ZIL: Processor: Common KVM (2 Cores), Motherboard: QEMU Standard PC (i440FX + PIIX 1996) (rel-1.14.0-0-g155821a1990b-prebuilt.qemu.org BIOS), Chipset: Intel 440FX 82441FX PMC, Memory: 2048MB, Disk: 34GB QEMU HDD + 3427GB iSCSI Disk, Graphics: bochs-drmdrmfb, Monitor: QEMU Monitor, Network: 2 x Red Hat Virtio device OS: Ubuntu 20.04, Kernel: 5.4.0-72-generic (x86_64), Compiler: GCC 9.3.0, File-System: ext4, Screen Resolution: 1024x768, System Layer: KVM iSCSI Sync L2ARC No ZIL Again: Processor: Common KVM (2 Cores), Motherboard: QEMU Standard PC (i440FX + PIIX 1996) (rel-1.14.0-0-g155821a1990b-prebuilt.qemu.org BIOS), Chipset: Intel 440FX 82441FX PMC, Memory: 2048MB, Disk: 34GB QEMU HDD + 3427GB iSCSI Disk, Graphics: bochs-drmdrmfb, Monitor: QEMU Monitor, Network: 2 x Red Hat Virtio device OS: Ubuntu 20.04, Kernel: 5.4.0-72-generic (x86_64), Compiler: GCC 9.3.0, File-System: ext4, Screen Resolution: 1024x768, System Layer: KVM iSCSI Sync L2ARC No ZIL Retry: Processor: Common KVM (2 Cores), Motherboard: QEMU Standard PC (i440FX + PIIX 1996) (rel-1.14.0-0-g155821a1990b-prebuilt.qemu.org BIOS), Chipset: Intel 440FX 82441FX PMC, Memory: 2048MB, Disk: 34GB QEMU HDD + 3427GB iSCSI Disk, Graphics: bochs-drmdrmfb, Monitor: QEMU Monitor, Network: 2 x Red Hat Virtio device OS: Ubuntu 20.04, Kernel: 5.4.0-72-generic (x86_64), Compiler: GCC 9.3.0, File-System: ext4, Screen Resolution: 1024x768, System Layer: KVM iSCSI Sync No L2ARC No ZIL: Processor: Common KVM (2 Cores), Motherboard: QEMU Standard PC (i440FX + PIIX 1996) (rel-1.14.0-0-g155821a1990b-prebuilt.qemu.org BIOS), Chipset: Intel 440FX 82441FX PMC, Memory: 2048MB, Disk: 34GB QEMU HDD + 3427GB iSCSI Disk, Graphics: bochs-drmdrmfb, Monitor: QEMU Monitor, Network: 2 x Red Hat Virtio device OS: Ubuntu 20.04, Kernel: 5.4.0-72-generic (x86_64), Compiler: GCC 9.3.0, File-System: ext4, Screen Resolution: 1024x768, System Layer: KVM iSCSI Sync ZIL: Processor: Common KVM (2 Cores), Motherboard: QEMU Standard PC (i440FX + PIIX 1996) (rel-1.14.0-0-g155821a1990b-prebuilt.qemu.org BIOS), Chipset: Intel 440FX 82441FX PMC, Memory: 2048MB, Disk: 34GB QEMU HDD + 3427GB iSCSI Disk, Graphics: bochs-drmdrmfb, Monitor: QEMU Monitor, Network: 2 x Red Hat Virtio device OS: Ubuntu 20.04, Kernel: 5.4.0-72-generic (x86_64), Compiler: GCC 9.3.0, File-System: ext4, Screen Resolution: 1024x768, System Layer: KVM iSCSI Sync ZIL Take 2: Processor: Common KVM (2 Cores), Motherboard: QEMU Standard PC (i440FX + PIIX 1996) (rel-1.14.0-0-g155821a1990b-prebuilt.qemu.org BIOS), Chipset: Intel 440FX 82441FX PMC, Memory: 2048MB, Disk: 34GB QEMU HDD + 3427GB iSCSI Disk, Graphics: bochs-drmdrmfb, Monitor: QEMU Monitor, Network: 2 x Red Hat Virtio device OS: Ubuntu 20.04, Kernel: 5.4.0-72-generic (x86_64), Compiler: GCC 9.3.0, File-System: ext4, Screen Resolution: 1024x768, System Layer: KVM Flexible IO Tester 3.25 Type: Random Read - Engine: Sync - Buffered: Yes - Direct: No - Block Size: 1MB - Disk Target: Default Test Directory MB/s > Higher Is Better iSCSI Sync L2ARC No ZIL . 191 |================================================ Flexible IO Tester 3.25 Type: Random Read - Engine: Sync - Buffered: Yes - Direct: No - Block Size: 1MB - Disk Target: Default Test Directory IOPS > Higher Is Better iSCSI Sync L2ARC No ZIL . 191 |================================================ Flexible IO Tester 3.25 Type: Random Read - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 1MB - Disk Target: Default Test Directory MB/s > Higher Is Better iSCSI Sync L2ARC No ZIL Again . 3.096 |= iSCSI Sync L2ARC No ZIL Retry . 4.616 |== iSCSI Sync No L2ARC No ZIL .... 9.494 |=== iSCSI Sync ZIL ................ 115.200 |====================================== Flexible IO Tester 3.25 Type: Random Read - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 1MB - Disk Target: Default Test Directory IOPS > Higher Is Better iSCSI Sync L2ARC No ZIL Again . 1 | iSCSI Sync L2ARC No ZIL Retry . 3 |= iSCSI Sync No L2ARC No ZIL .... 7 |=== iSCSI Sync ZIL ................ 112 |========================================== Flexible IO Tester 3.25 Type: Random Read - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory MB/s > Higher Is Better iSCSI Sync L2ARC No ZIL Again . 101.0 |======================================== iSCSI Sync No L2ARC No ZIL .... 101.0 |======================================== iSCSI Sync ZIL ................ 98.1 |======================================= iSCSI Sync ZIL Take 2 ......... 86.5 |================================== Flexible IO Tester 3.25 Type: Random Read - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory IOPS > Higher Is Better iSCSI Sync L2ARC No ZIL Again . 25867 |======================================== iSCSI Sync No L2ARC No ZIL .... 25800 |======================================== iSCSI Sync ZIL ................ 25133 |======================================= iSCSI Sync ZIL Take 2 ......... 22233 |================================== Flexible IO Tester 3.25 Type: Random Read - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 16KB - Disk Target: Default Test Directory MB/s > Higher Is Better iSCSI Sync L2ARC No ZIL Again . 212 |======================================== iSCSI Sync No L2ARC No ZIL .... 218 |========================================== iSCSI Sync ZIL ................ 220 |========================================== Flexible IO Tester 3.25 Type: Random Read - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 16KB - Disk Target: Default Test Directory IOPS > Higher Is Better iSCSI Sync L2ARC No ZIL Again . 13533 |======================================= iSCSI Sync No L2ARC No ZIL .... 13933 |======================================== iSCSI Sync ZIL ................ 13700 |======================================= Flexible IO Tester 3.25 Type: Random Read - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 64KB - Disk Target: Default Test Directory MB/s > Higher Is Better iSCSI Sync L2ARC No ZIL Again . 229 |========================================== iSCSI Sync ZIL ................ 162 |============================== Flexible IO Tester 3.25 Type: Random Read - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 64KB - Disk Target: Default Test Directory IOPS > Higher Is Better iSCSI Sync L2ARC No ZIL Again . 3657 |========================================= iSCSI Sync ZIL ................ 2588 |============================= Flexible IO Tester 3.25 Type: Random Write - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 1MB - Disk Target: Default Test Directory MB/s > Higher Is Better iSCSI Sync L2ARC No ZIL Again . 839 |========================================== iSCSI Sync No L2ARC No ZIL .... 104 |===== iSCSI Sync ZIL ................ 279 |============== Flexible IO Tester 3.25 Type: Random Write - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 1MB - Disk Target: Default Test Directory IOPS > Higher Is Better iSCSI Sync L2ARC No ZIL Again . 836 |========================================== iSCSI Sync No L2ARC No ZIL .... 100 |===== iSCSI Sync ZIL ................ 275 |============== Flexible IO Tester 3.25 Type: Random Write - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory MB/s > Higher Is Better iSCSI Sync L2ARC No ZIL Again . 74.400 |======================================= iSCSI Sync No L2ARC No ZIL .... 7.625 |==== iSCSI Sync ZIL ................ 75.000 |======================================= iSCSI Sync ZIL Take 2 ......... 74.600 |======================================= Flexible IO Tester 3.25 Type: Random Write - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory IOPS > Higher Is Better iSCSI Sync L2ARC No ZIL Again . 19000 |======================================== iSCSI Sync No L2ARC No ZIL .... 1902 |==== iSCSI Sync ZIL ................ 19200 |======================================== iSCSI Sync ZIL Take 2 ......... 19167 |======================================== Flexible IO Tester 3.25 Type: Random Write - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 16KB - Disk Target: Default Test Directory MB/s > Higher Is Better iSCSI Sync L2ARC No ZIL Again . 223.0 |======================================== iSCSI Sync No L2ARC No ZIL .... 18.4 |=== iSCSI Sync ZIL ................ 200.0 |==================================== Flexible IO Tester 3.25 Type: Random Write - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 16KB - Disk Target: Default Test Directory IOPS > Higher Is Better iSCSI Sync L2ARC No ZIL Again . 14333 |======================================== iSCSI Sync No L2ARC No ZIL .... 1174 |=== iSCSI Sync ZIL ................ 12767 |==================================== Flexible IO Tester 3.25 Type: Random Write - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 64KB - Disk Target: Default Test Directory MB/s > Higher Is Better iSCSI Sync L2ARC No ZIL Again . 459 |========================================== iSCSI Sync ZIL ................ 263 |======================== Flexible IO Tester 3.25 Type: Random Write - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 64KB - Disk Target: Default Test Directory IOPS > Higher Is Better iSCSI Sync L2ARC No ZIL Again . 7343 |========================================= iSCSI Sync ZIL ................ 4205 |======================= Flexible IO Tester 3.25 Type: Sequential Read - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 1MB - Disk Target: Default Test Directory MB/s > Higher Is Better iSCSI Sync L2ARC No ZIL Again . 12.494 |== iSCSI Sync No L2ARC No ZIL .... 277.104 |====================================== iSCSI Sync ZIL ................ 9.522 |= Flexible IO Tester 3.25 Type: Sequential Read - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 1MB - Disk Target: Default Test Directory IOPS > Higher Is Better iSCSI Sync L2ARC No ZIL Again . 9 |= iSCSI Sync No L2ARC No ZIL .... 274 |========================================== iSCSI Sync ZIL ................ 7 |= Flexible IO Tester 3.25 Type: Sequential Read - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory MB/s > Higher Is Better iSCSI Sync L2ARC No ZIL Again . 132 |====================================== iSCSI Sync No L2ARC No ZIL .... 140 |========================================= iSCSI Sync ZIL ................ 145 |========================================== iSCSI Sync ZIL Take 2 ......... 143 |========================================= Flexible IO Tester 3.25 Type: Sequential Read - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory IOPS > Higher Is Better iSCSI Sync L2ARC No ZIL Again . 33667 |==================================== iSCSI Sync No L2ARC No ZIL .... 35933 |======================================= iSCSI Sync ZIL ................ 37067 |======================================== iSCSI Sync ZIL Take 2 ......... 36533 |======================================= Flexible IO Tester 3.25 Type: Sequential Read - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 16KB - Disk Target: Default Test Directory MB/s > Higher Is Better iSCSI Sync L2ARC No ZIL Again . 189 |================================= iSCSI Sync No L2ARC No ZIL .... 227 |======================================= iSCSI Sync ZIL ................ 243 |========================================== Flexible IO Tester 3.25 Type: Sequential Read - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 16KB - Disk Target: Default Test Directory IOPS > Higher Is Better iSCSI Sync L2ARC No ZIL Again . 12086 |=============================== iSCSI Sync No L2ARC No ZIL .... 14500 |===================================== iSCSI Sync ZIL ................ 15533 |======================================== Flexible IO Tester 3.25 Type: Sequential Read - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 64KB - Disk Target: Default Test Directory MB/s > Higher Is Better iSCSI Sync L2ARC No ZIL Again . 23.900 |======================================= iSCSI Sync ZIL ................ 5.220 |========= Flexible IO Tester 3.25 Type: Sequential Read - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 64KB - Disk Target: Default Test Directory IOPS > Higher Is Better iSCSI Sync L2ARC No ZIL Again . 379 |========================================== iSCSI Sync ZIL ................ 78 |========= Flexible IO Tester 3.25 Type: Sequential Write - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 1MB - Disk Target: Default Test Directory MB/s > Higher Is Better iSCSI Sync L2ARC No ZIL Again . 560.0 |======================================== iSCSI Sync No L2ARC No ZIL .... 99.0 |======= iSCSI Sync ZIL ................ 274.0 |==================== Flexible IO Tester 3.25 Type: Sequential Write - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 1MB - Disk Target: Default Test Directory IOPS > Higher Is Better iSCSI Sync L2ARC No ZIL Again . 556 |========================================== iSCSI Sync No L2ARC No ZIL .... 95 |======= iSCSI Sync ZIL ................ 271 |==================== Flexible IO Tester 3.25 Type: Sequential Write - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory MB/s > Higher Is Better iSCSI Sync L2ARC No ZIL Again . 115.00 |======================================= iSCSI Sync No L2ARC No ZIL .... 9.81 |=== iSCSI Sync ZIL ................ 80.40 |=========================== iSCSI Sync ZIL Take 2 ......... 82.20 |============================ Flexible IO Tester 3.25 Type: Sequential Write - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory IOPS > Higher Is Better iSCSI Sync L2ARC No ZIL Again . 29500 |======================================== iSCSI Sync No L2ARC No ZIL .... 2509 |=== iSCSI Sync ZIL ................ 20233 |=========================== iSCSI Sync ZIL Take 2 ......... 21033 |============================= Flexible IO Tester 3.25 Type: Sequential Write - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 16KB - Disk Target: Default Test Directory MB/s > Higher Is Better iSCSI Sync L2ARC No ZIL Again . 349.0 |======================================== iSCSI Sync No L2ARC No ZIL .... 22.9 |=== iSCSI Sync ZIL ................ 199.0 |======================= Flexible IO Tester 3.25 Type: Sequential Write - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 16KB - Disk Target: Default Test Directory IOPS > Higher Is Better iSCSI Sync L2ARC No ZIL Again . 22367 |======================================== iSCSI Sync No L2ARC No ZIL .... 1460 |=== iSCSI Sync ZIL ................ 12700 |======================= Flexible IO Tester 3.25 Type: Sequential Write - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 64KB - Disk Target: Default Test Directory MB/s > Higher Is Better iSCSI Sync L2ARC No ZIL Again . 446 |========================================== iSCSI Sync ZIL ................ 248 |======================= Flexible IO Tester 3.25 Type: Sequential Write - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 64KB - Disk Target: Default Test Directory IOPS > Higher Is Better iSCSI Sync L2ARC No ZIL Again . 7134 |========================================= iSCSI Sync ZIL ................ 3962 |======================= Flexible IO Tester 3.25 Type: Random Read - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 32KB - Disk Target: Default Test Directory MB/s > Higher Is Better iSCSI Sync No L2ARC No ZIL . 226 |============================================= Flexible IO Tester 3.25 Type: Random Read - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 32KB - Disk Target: Default Test Directory IOPS > Higher Is Better iSCSI Sync No L2ARC No ZIL . 7234 |============================================ Flexible IO Tester 3.25 Type: Random Write - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 32KB - Disk Target: Default Test Directory MB/s > Higher Is Better iSCSI Sync No L2ARC No ZIL . 19.3 |============================================ Flexible IO Tester 3.25 Type: Random Write - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 32KB - Disk Target: Default Test Directory IOPS > Higher Is Better iSCSI Sync No L2ARC No ZIL . 614 |============================================= Flexible IO Tester 3.25 Type: Sequential Read - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 32KB - Disk Target: Default Test Directory MB/s > Higher Is Better iSCSI Sync No L2ARC No ZIL . 49.2 |============================================ Flexible IO Tester 3.25 Type: Sequential Read - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 32KB - Disk Target: Default Test Directory IOPS > Higher Is Better iSCSI Sync No L2ARC No ZIL . 1570 |============================================ Flexible IO Tester 3.25 Type: Sequential Write - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 32KB - Disk Target: Default Test Directory MB/s > Higher Is Better iSCSI Sync No L2ARC No ZIL . 26.9 |============================================ Flexible IO Tester 3.25 Type: Sequential Write - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 32KB - Disk Target: Default Test Directory IOPS > Higher Is Better iSCSI Sync No L2ARC No ZIL . 858 |============================================= Flexible IO Tester 3.25 Type: Random Read - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 2MB - Disk Target: Default Test Directory MB/s > Higher Is Better iSCSI Sync ZIL Take 2 . 9.526 |================================================ Flexible IO Tester 3.25 Type: Random Read - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 2MB - Disk Target: Default Test Directory IOPS > Higher Is Better iSCSI Sync ZIL Take 2 . 3 |==================================================== Flexible IO Tester 3.25 Type: Random Write - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 2MB - Disk Target: Default Test Directory MB/s > Higher Is Better iSCSI Sync ZIL Take 2 . 280 |================================================== Flexible IO Tester 3.25 Type: Random Write - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 2MB - Disk Target: Default Test Directory IOPS > Higher Is Better iSCSI Sync ZIL Take 2 . 137 |================================================== Flexible IO Tester 3.25 Type: Sequential Read - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 2MB - Disk Target: Default Test Directory MB/s > Higher Is Better iSCSI Sync ZIL Take 2 . 12.26 |================================================ Flexible IO Tester 3.25 Type: Sequential Read - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 2MB - Disk Target: Default Test Directory IOPS > Higher Is Better iSCSI Sync ZIL Take 2 . 4 |==================================================== Flexible IO Tester 3.25 Type: Sequential Write - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 2MB - Disk Target: Default Test Directory MB/s > Higher Is Better iSCSI Sync ZIL Take 2 . 279 |================================================== Flexible IO Tester 3.25 Type: Sequential Write - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 2MB - Disk Target: Default Test Directory IOPS > Higher Is Better iSCSI Sync ZIL Take 2 . 136 |==================================================