Linux 4.16 File-System Tests HDD and SSD file-system tests on Linux 4.16 for a future article on Phoronix. TR150 SSD: EXT4: Processor: 2 x Intel Xeon Gold 6138 @ 3.70GHz (40 Cores / 80 Threads), Motherboard: TYAN S7106 (V1.00 BIOS), Chipset: Intel Sky Lake-E DMI3 Registers, Memory: 12 x 8192 MB DDR4-2666MT/s Micron 9ASF1G72PZ-2G6B1, Disk: 256GB Samsung SSD 850 + 2000GB Seagate ST2000DM006-2DM1 + 2 x 120GB TOSHIBA-TR150, Graphics: llvmpipe 95360MB, Monitor: VE228, Network: Intel I210 Gigabit Connection OS: Ubuntu 18.04, Kernel: 4.16.0-999-generic (x86_64) 20180323, Desktop: GNOME Shell 3.28.0, Display Server: X Server 1.19.6, OpenGL: 3.3 Mesa 18.0.0-rc5 (LLVM 6.0 256 bits), Compiler: GCC 7.3.0, File-System: ext4, Screen Resolution: 1920x1080 TR150 SSD: F2FS: Processor: 2 x Intel Xeon Gold 6138 @ 3.70GHz (40 Cores / 80 Threads), Motherboard: TYAN S7106 (V1.00 BIOS), Chipset: Intel Sky Lake-E DMI3 Registers, Memory: 12 x 8192 MB DDR4-2666MT/s Micron 9ASF1G72PZ-2G6B1, Disk: 256GB Samsung SSD 850 + 2000GB Seagate ST2000DM006-2DM1 + 2 x 120GB TOSHIBA-TR150, Graphics: llvmpipe 95360MB, Monitor: VE228, Network: Intel I210 Gigabit Connection OS: Ubuntu 18.04, Kernel: 4.16.0-999-generic (x86_64) 20180323, Desktop: GNOME Shell 3.28.0, Display Server: X Server 1.19.6, OpenGL: 3.3 Mesa 18.0.0-rc5 (LLVM 6.0 256 bits), Compiler: GCC 7.3.0, File-System: f2fs, Screen Resolution: 1920x1080 TR150 SSD: Btrfs: Processor: 2 x Intel Xeon Gold 6138 @ 3.70GHz (40 Cores / 80 Threads), Motherboard: TYAN S7106 (V1.00 BIOS), Chipset: Intel Sky Lake-E DMI3 Registers, Memory: 12 x 8192 MB DDR4-2666MT/s Micron 9ASF1G72PZ-2G6B1, Disk: 256GB Samsung SSD 850 + 2000GB Seagate ST2000DM006-2DM1 + 2 x 120GB TOSHIBA-TR150, Graphics: llvmpipe 95360MB, Monitor: VE228, Network: Intel I210 Gigabit Connection OS: Ubuntu 18.04, Kernel: 4.16.0-999-generic (x86_64) 20180323, Desktop: GNOME Shell 3.28.0, Display Server: X Server 1.19.6, OpenGL: 3.3 Mesa 18.0.0-rc5 (LLVM 6.0 256 bits), Compiler: GCC 7.3.0, File-System: btrfs, Screen Resolution: 1920x1080 TR150 SSD: XFS: Processor: 2 x Intel Xeon Gold 6138 @ 3.70GHz (40 Cores / 80 Threads), Motherboard: TYAN S7106 (V1.00 BIOS), Chipset: Intel Sky Lake-E DMI3 Registers, Memory: 12 x 8192 MB DDR4-2666MT/s Micron 9ASF1G72PZ-2G6B1, Disk: 256GB Samsung SSD 850 + 2000GB Seagate ST2000DM006-2DM1 + 2 x 120GB TOSHIBA-TR150, Graphics: llvmpipe 95360MB, Monitor: VE228, Network: Intel I210 Gigabit Connection OS: Ubuntu 18.04, Kernel: 4.16.0-999-generic (x86_64) 20180323, Desktop: GNOME Shell 3.28.0, Display Server: X Server 1.19.6, OpenGL: 3.3 Mesa 18.0.0-rc5 (LLVM 6.0 256 bits), Compiler: GCC 7.3.0, File-System: xfs, Screen Resolution: 1920x1080 Seagate HDD: XFS: Processor: 2 x Intel Xeon Gold 6138 @ 3.70GHz (40 Cores / 80 Threads), Motherboard: TYAN S7106 (V1.00 BIOS), Chipset: Intel Sky Lake-E DMI3 Registers, Memory: 12 x 8192 MB DDR4-2666MT/s Micron 9ASF1G72PZ-2G6B1, Disk: 256GB Samsung SSD 850 + 2000GB Seagate ST2000DM006-2DM1 + 2 x 120GB TOSHIBA-TR150, Graphics: llvmpipe 95360MB, Monitor: VE228, Network: Intel I210 Gigabit Connection OS: Ubuntu 18.04, Kernel: 4.16.0-999-generic (x86_64) 20180323, Desktop: GNOME Shell 3.28.0, Display Server: X Server 1.19.6, OpenGL: 3.3 Mesa 18.0.0-rc5 (LLVM 6.0 256 bits), Compiler: GCC 7.3.0, File-System: xfs, Screen Resolution: 1920x1080 Seagate HDD: Btrfs: Processor: 2 x Intel Xeon Gold 6138 @ 3.70GHz (40 Cores / 80 Threads), Motherboard: TYAN S7106 (V1.00 BIOS), Chipset: Intel Sky Lake-E DMI3 Registers, Memory: 12 x 8192 MB DDR4-2666MT/s Micron 9ASF1G72PZ-2G6B1, Disk: 256GB Samsung SSD 850 + 2000GB Seagate ST2000DM006-2DM1 + 2 x 120GB TOSHIBA-TR150, Graphics: llvmpipe 95360MB, Monitor: VE228, Network: Intel I210 Gigabit Connection OS: Ubuntu 18.04, Kernel: 4.16.0-999-generic (x86_64) 20180323, Desktop: GNOME Shell 3.28.0, Display Server: X Server 1.19.6, OpenGL: 3.3 Mesa 18.0.0-rc5 (LLVM 6.0 256 bits), Compiler: GCC 7.3.0, File-System: btrfs, Screen Resolution: 1920x1080 Seagate HDD: EXT4: Processor: 2 x Intel Xeon Gold 6138 @ 3.70GHz (40 Cores / 80 Threads), Motherboard: TYAN S7106 (V1.00 BIOS), Chipset: Intel Sky Lake-E DMI3 Registers, Memory: 12 x 8192 MB DDR4-2666MT/s Micron 9ASF1G72PZ-2G6B1, Disk: 256GB Samsung SSD 850 + 2000GB Seagate ST2000DM006-2DM1 + 2 x 120GB TOSHIBA-TR150, Graphics: llvmpipe 95360MB, Monitor: VE228, Network: Intel I210 Gigabit Connection OS: Ubuntu 18.04, Kernel: 4.16.0-999-generic (x86_64) 20180323, Desktop: GNOME Shell 3.28.0, Display Server: X Server 1.19.6, OpenGL: 3.3 Mesa 18.0.0-rc5 (LLVM 6.0 256 bits), Compiler: GCC 7.3.0, File-System: ext4, Screen Resolution: 1920x1080 Virtio ZFS HDD Raid 0: Processor: Common KVM @ 3.91GHz (2 Cores), Motherboard: QEMU Standard PC (i440FX + PIIX 1996) (rel-1.10.2-0-g5f4c7b1-prebuilt.qemu-project.org BIOS), Memory: 2048MB, Disk: 34GB QEMU HDD + 30GB 2115, Graphics: bochsdrmfb OS: Debian 9.4, Kernel: 4.9.0-6-amd64 (x86_64), Compiler: GCC 6.3.0 20170516, File-System: ext4, Screen Resolution: 1024x768, System Layer: qemu Virtio ZFS HDD Raid 0 2: Processor: Common KVM @ 3.91GHz (4 Cores), Motherboard: QEMU Standard PC (i440FX + PIIX 1996) (rel-1.10.2-0-g5f4c7b1-prebuilt.qemu-project.org BIOS), Memory: 2048MB, Disk: 34GB QEMU HDD, Graphics: bochsdrmfb OS: Debian 9.4, Kernel: 4.9.0-6-amd64 (x86_64), Compiler: GCC 6.3.0 20170516, File-System: ext4, Screen Resolution: 1024x768, System Layer: qemu Virtio ZFS HDD Raid 10: Processor: Common KVM @ 3.91GHz (4 Cores), Motherboard: QEMU Standard PC (i440FX + PIIX 1996) (rel-1.10.2-0-g5f4c7b1-prebuilt.qemu-project.org BIOS), Memory: 2048MB, Disk: 34GB QEMU HDD, Graphics: bochsdrmfb OS: Debian 9.4, Kernel: 4.9.0-6-amd64 (x86_64), Compiler: GCC 6.3.0 20170516, File-System: ext4, Screen Resolution: 1024x768, System Layer: qemu Virtio ZFS HDD Raid 10 WBU: Processor: Common KVM @ 3.91GHz (4 Cores), Motherboard: QEMU Standard PC (i440FX + PIIX 1996) (rel-1.10.2-0-g5f4c7b1-prebuilt.qemu-project.org BIOS), Memory: 2048MB, Disk: 34GB QEMU HDD, Graphics: bochsdrmfb OS: Debian 9.4, Kernel: 4.9.0-6-amd64 (x86_64), Compiler: GCC 6.3.0 20170516, File-System: ext4, Screen Resolution: 1024x768, System Layer: qemu XenServer 7.4 Adaptec 6805 Raid 1 PV: Processor: AMD Turion II Neo N54L @ 2.20GHz (2 Cores), Memory: 4096MB, Disk: 15GB OS: Debian 9.4, Kernel: 4.9.0-6-amd64 (x86_64), Compiler: GCC 6.3.0 20170516, File-System: ext4, System Layer: vm-other Xen 4.7.4-4.1 Hypervisor Proxmox ZFS Raid 1 WT: Processor: Common KVM @ 2.20GHz (2 Cores), Motherboard: QEMU Standard PC (i440FX + PIIX 1996) (rel-1.10.2-0-g5f4c7b1-prebuilt.qemu-project.org BIOS), Memory: 4096MB, Disk: 15GB, Graphics: bochsdrmfb OS: Debian 9.4, Kernel: 4.9.0-6-amd64 (x86_64), Compiler: GCC 6.3.0 20170516, File-System: ext4, Screen Resolution: 1024x768, System Layer: qemu Proxmox ZFS Raid 1 WB: Processor: Common KVM @ 2.20GHz (2 Cores), Motherboard: QEMU Standard PC (i440FX + PIIX 1996) (rel-1.10.2-0-g5f4c7b1-prebuilt.qemu-project.org BIOS), Memory: 4096MB, Disk: 15GB, Graphics: bochsdrmfb OS: Debian 9.4, Kernel: 4.9.0-6-amd64 (x86_64), Compiler: GCC 6.3.0 20170516, File-System: ext4, Screen Resolution: 1024x768, System Layer: qemu Proxmox ZFS Raid 1 WB metadata: Processor: Common KVM @ 2.20GHz (2 Cores), Motherboard: QEMU Standard PC (i440FX + PIIX 1996) (rel-1.10.2-0-g5f4c7b1-prebuilt.qemu-project.org BIOS), Memory: 4096MB, Disk: 15GB, Graphics: bochsdrmfb OS: Debian 9.4, Kernel: 4.9.0-6-amd64 (x86_64), Compiler: GCC 6.3.0 20170516, File-System: ext4, Screen Resolution: 1024x768, System Layer: qemu Proxmox ZFS Raid 1 WB metadata throughput: Processor: Common KVM @ 2.20GHz (2 Cores), Motherboard: QEMU Standard PC (i440FX + PIIX 1996) (rel-1.10.2-0-g5f4c7b1-prebuilt.qemu-project.org BIOS), Memory: 4096MB, Disk: 15GB, Graphics: bochsdrmfb OS: Debian 9.4, Kernel: 4.9.0-6-amd64 (x86_64), Compiler: GCC 6.3.0 20170516, File-System: ext4, Screen Resolution: 1024x768, System Layer: qemu Proxmox ZFS Raid 1 WB 2: Processor: Common KVM @ 2.20GHz (2 Cores), Motherboard: QEMU Standard PC (i440FX + PIIX 1996) (rel-1.10.2-0-g5f4c7b1-prebuilt.qemu-project.org BIOS), Memory: 4096MB, Disk: 15GB, Graphics: bochsdrmfb OS: Debian 9.4, Kernel: 4.9.0-6-amd64 (x86_64), Compiler: GCC 6.3.0 20170516, File-System: ext4, Screen Resolution: 1024x768, System Layer: qemu Proxmox ZFS Raid 1 WB ZFS 0.7.6: Processor: Common KVM @ 2.20GHz (1 Core), Motherboard: QEMU Standard PC (i440FX + PIIX 1996) (rel-1.10.2-0-g5f4c7b1-prebuilt.qemu-project.org BIOS), Memory: 4096MB, Disk: 15GB, Graphics: bochsdrmfb OS: Debian 9.4, Kernel: 4.9.0-6-amd64 (x86_64), Compiler: GCC 6.3.0 20170516, File-System: ext4, Screen Resolution: 1024x768, System Layer: qemu Proxmox ZFS Raid 1 WB ZFS 0.7.6 iothread-native: Processor: Common KVM @ 2.20GHz (1 Core), Motherboard: QEMU Standard PC (i440FX + PIIX 1996) (rel-1.10.2-0-g5f4c7b1-prebuilt.qemu-project.org BIOS), Memory: 4096MB, Disk: 15GB, Graphics: bochsdrmfb OS: Debian 9.4, Kernel: 4.9.0-6-amd64 (x86_64), Compiler: GCC 6.3.0 20170516, File-System: ext4, Screen Resolution: 1024x768, System Layer: qemu Proxmox ZFS Raid 1 WB ZFS 0.7.6 iothread-native moswap: Processor: Common KVM @ 2.20GHz (1 Core), Motherboard: QEMU Standard PC (i440FX + PIIX 1996) (rel-1.10.2-0-g5f4c7b1-prebuilt.qemu-project.org BIOS), Memory: 4096MB, Disk: 15GB, Graphics: bochsdrmfb OS: Debian 9.4, Kernel: 4.9.0-6-amd64 (x86_64), Compiler: GCC 6.3.0 20170516, File-System: ext4, Screen Resolution: 1024x768, System Layer: qemu Proxmox ZFS Raid 1 N ZFS 0.7.6 iothread-native noswap nodiskwc: Processor: Common KVM @ 2.20GHz (1 Core), Motherboard: QEMU Standard PC (i440FX + PIIX 1996) (rel-1.10.2-0-g5f4c7b1-prebuilt.qemu-project.org BIOS), Memory: 4096MB, Disk: 15GB, Graphics: bochsdrmfb OS: Debian 9.4, Kernel: 4.9.0-6-amd64 (x86_64), Compiler: GCC 6.3.0 20170516, File-System: ext4, Screen Resolution: 1024x768, System Layer: qemu Proxmox ZFS Raid 1 N ZFS 0.7.6 iothread-native noswap nodiskwc 2: Processor: Common KVM @ 2.20GHz (1 Core), Motherboard: QEMU Standard PC (i440FX + PIIX 1996) (rel-1.10.2-0-g5f4c7b1-prebuilt.qemu-project.org BIOS), Memory: 4096MB, Disk: 15GB, Graphics: bochsdrmfb OS: Debian 9.4, Kernel: 4.9.0-6-amd64 (x86_64), Compiler: GCC 6.3.0 20170516, File-System: ext4, Screen Resolution: 1024x768, System Layer: qemu Proxmox ZFS Raid 1 N metadata ZFS 0.7.6 iothread-native noswap nodiskwc: Processor: Common KVM @ 2.20GHz (1 Core), Motherboard: QEMU Standard PC (i440FX + PIIX 1996) (rel-1.10.2-0-g5f4c7b1-prebuilt.qemu-project.org BIOS), Memory: 4096MB, Disk: 15GB, Graphics: bochsdrmfb OS: Debian 9.4, Kernel: 4.9.0-6-amd64 (x86_64), Compiler: GCC 6.3.0 20170516, File-System: ext4, Screen Resolution: 1024x768, System Layer: qemu Proxmox ZFS Raid 1 LXC 3: Processor: AMD Turion II Neo N54L @ 2.20GHz (2 Cores), Motherboard: HP ProLiant MicroServer (O41 BIOS), Memory: 4096MB, Disk: 2 x 500GB SAMSUNG HD502HI + 62GB Ultra Fit + 1000GB SAMSUNG HD103SI, Graphics: astdrmfb OS: Debian GNU/Linux 9, Kernel: 4.13.16-1-pve (x86_64), Compiler: GCC 6.3.0 20170516, File-System: zfs, Screen Resolution: 1024x768, System Layer: lxc AIO-Stress 0.21 Test: Random Write MB/s > Higher Is Better TR150 SSD: EXT4 ......................................................... 2301.32 | TR150 SSD: F2FS ......................................................... 3017.61 | TR150 SSD: Btrfs ........................................................ 2936.17 | TR150 SSD: XFS .......................................................... 2971.42 | Seagate HDD: XFS ........................................................ 2979.14 | Seagate HDD: Btrfs ...................................................... 2953.92 | Seagate HDD: EXT4 ....................................................... 2410.29 | Virtio ZFS HDD Raid 0 2 ................................................. 1341.61 | Virtio ZFS HDD Raid 10 .................................................. 1376.59 | Virtio ZFS HDD Raid 10 WBU .............................................. 748.84 | XenServer 7.4 Adaptec 6805 Raid 1 PV .................................... 47.81 | Proxmox ZFS Raid 1 WT ................................................... 87.08 | Proxmox ZFS Raid 1 WB ................................................... 333.49 | Proxmox ZFS Raid 1 WB metadata .......................................... 275.85 | Proxmox ZFS Raid 1 WB metadata throughput ............................... 267.77 | Proxmox ZFS Raid 1 WB 2 ................................................. 256.50 | Proxmox ZFS Raid 1 WB ZFS 0.7.6 iothread-native ......................... 216.53 | Proxmox ZFS Raid 1 WB ZFS 0.7.6 iothread-native moswap .................. 300.01 | Proxmox ZFS Raid 1 N ZFS 0.7.6 iothread-native noswap nodiskwc .......... 196.56 | Proxmox ZFS Raid 1 N ZFS 0.7.6 iothread-native noswap nodiskwc 2 ........ 205.72 | Proxmox ZFS Raid 1 N metadata ZFS 0.7.6 iothread-native noswap nodiskwc . 145.93 | Proxmox ZFS Raid 1 LXC 3 ................................................ 182.65 | SQLite 3.22 Timed SQLite Insertions Seconds < Lower Is Better TR150 SSD: EXT4 ......................................................... 41.27 | TR150 SSD: F2FS ......................................................... 34.89 | TR150 SSD: Btrfs ........................................................ 99.08 | TR150 SSD: XFS .......................................................... 36.76 | Seagate HDD: XFS ........................................................ 417.70 | Seagate HDD: Btrfs ...................................................... 1012.38 | Seagate HDD: EXT4 ....................................................... 576.02 | Virtio ZFS HDD Raid 0 2 ................................................. 246.03 | Virtio ZFS HDD Raid 10 .................................................. 329.31 | Virtio ZFS HDD Raid 10 WBU .............................................. 3.34 | XenServer 7.4 Adaptec 6805 Raid 1 PV .................................... 23.65 | Proxmox ZFS Raid 1 WT ................................................... 480.57 | Proxmox ZFS Raid 1 WB ................................................... 400.77 | Proxmox ZFS Raid 1 WB metadata .......................................... 489.60 | Proxmox ZFS Raid 1 WB metadata throughput ............................... 436.47 | Proxmox ZFS Raid 1 WB 2 ................................................. 396.77 | Proxmox ZFS Raid 1 WB ZFS 0.7.6 iothread-native ......................... 391.26 | Proxmox ZFS Raid 1 WB ZFS 0.7.6 iothread-native moswap .................. 388.95 | Proxmox ZFS Raid 1 N ZFS 0.7.6 iothread-native noswap nodiskwc 2 ........ 399.65 | Proxmox ZFS Raid 1 N metadata ZFS 0.7.6 iothread-native noswap nodiskwc . 449.30 | Proxmox ZFS Raid 1 LXC 3 ................................................ 270.05 | Flexible IO Tester 3.1 Type: Random Read - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory MB/s > Higher Is Better TR150 SSD: EXT4 ......................................................... 230.00 | TR150 SSD: F2FS ......................................................... 212.00 | TR150 SSD: Btrfs ........................................................ 212.00 | TR150 SSD: XFS .......................................................... 212.00 | Seagate HDD: XFS ........................................................ 1.53 | Seagate HDD: Btrfs ...................................................... 1.52 | Seagate HDD: EXT4 ....................................................... 1.45 | Virtio ZFS HDD Raid 0 2 ................................................. 258.00 | Virtio ZFS HDD Raid 10 .................................................. 2.98 | Virtio ZFS HDD Raid 10 WBU .............................................. 2.83 | XenServer 7.4 Adaptec 6805 Raid 1 PV .................................... 0.64 | Proxmox ZFS Raid 1 WT ................................................... 144.00 | Proxmox ZFS Raid 1 WB ................................................... 1.17 | Proxmox ZFS Raid 1 WB metadata .......................................... 1.09 | Proxmox ZFS Raid 1 WB metadata throughput ............................... 1.10 | Proxmox ZFS Raid 1 WB 2 ................................................. 1.54 | Proxmox ZFS Raid 1 WB ZFS 0.7.6 iothread-native ......................... 1.18 | Proxmox ZFS Raid 1 WB ZFS 0.7.6 iothread-native moswap .................. 1.50 | Proxmox ZFS Raid 1 N ZFS 0.7.6 iothread-native noswap nodiskwc 2 ........ 1.17 | Proxmox ZFS Raid 1 N metadata ZFS 0.7.6 iothread-native noswap nodiskwc . 1.07 | Flexible IO Tester 3.1 Type: Random Read - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory IOPS > Higher Is Better TR150 SSD: EXT4 ......... 58967 |========================================= TR150 SSD: F2FS ......... 54200 |====================================== TR150 SSD: Btrfs ........ 54367 |====================================== TR150 SSD: XFS .......... 54400 |====================================== Virtio ZFS HDD Raid 0 2 . 65633 |============================================== Proxmox ZFS Raid 1 WT ... 36733 |========================== Flexible IO Tester 3.1 Type: Random Write - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory MB/s > Higher Is Better TR150 SSD: EXT4 ......................................................... 275.00 | TR150 SSD: F2FS ......................................................... 282.00 | TR150 SSD: Btrfs ........................................................ 73.28 | TR150 SSD: XFS .......................................................... 273.00 | Seagate HDD: XFS ........................................................ 1.17 | Seagate HDD: Btrfs ...................................................... 23.02 | Seagate HDD: EXT4 ....................................................... 1.03 | Virtio ZFS HDD Raid 0 2 ................................................. 197.00 | Virtio ZFS HDD Raid 10 .................................................. 217.00 | Virtio ZFS HDD Raid 10 WBU .............................................. 207.00 | XenServer 7.4 Adaptec 6805 Raid 1 PV .................................... 1.01 | Proxmox ZFS Raid 1 WT ................................................... 0.42 | Proxmox ZFS Raid 1 WB ................................................... 17.46 | Proxmox ZFS Raid 1 WB metadata .......................................... 9.40 | Proxmox ZFS Raid 1 WB metadata throughput ............................... 8.48 | Proxmox ZFS Raid 1 WB 2 ................................................. 8.24 | Proxmox ZFS Raid 1 WB ZFS 0.7.6 iothread-native ......................... 9.00 | Proxmox ZFS Raid 1 WB ZFS 0.7.6 iothread-native moswap .................. 10.81 | Proxmox ZFS Raid 1 N ZFS 0.7.6 iothread-native noswap nodiskwc 2 ........ 1.15 | Proxmox ZFS Raid 1 N metadata ZFS 0.7.6 iothread-native noswap nodiskwc . 1.25 | Flexible IO Tester 3.1 Type: Random Write - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory IOPS > Higher Is Better TR150 SSD: EXT4 ............ 70250 |========================================== TR150 SSD: F2FS ............ 72167 |=========================================== TR150 SSD: Btrfs ........... 18750 |=========== TR150 SSD: XFS ............. 69967 |========================================== Seagate HDD: Btrfs ......... 27400 |================ Virtio ZFS HDD Raid 0 2 .... 50500 |============================== Virtio ZFS HDD Raid 10 ..... 55567 |================================= Virtio ZFS HDD Raid 10 WBU . 53000 |================================ Proxmox ZFS Raid 1 WB ...... 16100 |========== Flexible IO Tester 3.1 Type: Sequential Read - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory MB/s > Higher Is Better TR150 SSD: EXT4 ......................................................... 227.00 | TR150 SSD: F2FS ......................................................... 228.00 | TR150 SSD: Btrfs ........................................................ 327.00 | TR150 SSD: XFS .......................................................... 202.00 | Seagate HDD: XFS ........................................................ 148.00 | Seagate HDD: Btrfs ...................................................... 1.54 | Seagate HDD: EXT4 ....................................................... 155.00 | Virtio ZFS HDD Raid 0 2 ................................................. 255.00 | Virtio ZFS HDD Raid 10 .................................................. 250.00 | Virtio ZFS HDD Raid 10 WBU .............................................. 270.00 | XenServer 7.4 Adaptec 6805 Raid 1 PV .................................... 68.33 | Proxmox ZFS Raid 1 WT ................................................... 170.00 | Proxmox ZFS Raid 1 WB ................................................... 165.00 | Proxmox ZFS Raid 1 WB metadata .......................................... 158.00 | Proxmox ZFS Raid 1 WB metadata throughput ............................... 172.00 | Proxmox ZFS Raid 1 WB 2 ................................................. 160.00 | Proxmox ZFS Raid 1 WB ZFS 0.7.6 iothread-native ......................... 243.22 | Proxmox ZFS Raid 1 WB ZFS 0.7.6 iothread-native moswap .................. 290.53 | Proxmox ZFS Raid 1 N ZFS 0.7.6 iothread-native noswap nodiskwc 2 ........ 35.83 | Proxmox ZFS Raid 1 N metadata ZFS 0.7.6 iothread-native noswap nodiskwc . 37.40 | Flexible IO Tester 3.1 Type: Sequential Read - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory IOPS > Higher Is Better TR150 SSD: EXT4 .................................................. 58300 |=== TR150 SSD: F2FS .................................................. 58467 |=== TR150 SSD: Btrfs ................................................. 83633 |===== TR150 SSD: XFS ................................................... 51667 |=== Seagate HDD: XFS ................................................. 37800 |== Seagate HDD: EXT4 ................................................ 39767 |== Virtio ZFS HDD Raid 0 2 .......................................... 65167 |==== Virtio ZFS HDD Raid 10 ........................................... 63683 |==== Virtio ZFS HDD Raid 10 WBU ....................................... 68940 |==== XenServer 7.4 Adaptec 6805 Raid 1 PV ............................. 19100 |= Proxmox ZFS Raid 1 WT ............................................ 43533 |== Proxmox ZFS Raid 1 WB ............................................ 41933 |== Proxmox ZFS Raid 1 WB metadata ................................... 40200 |== Proxmox ZFS Raid 1 WB metadata throughput ........................ 44050 |== Proxmox ZFS Raid 1 WB 2 .......................................... 40833 |== Proxmox ZFS Raid 1 WB ZFS 0.7.6 iothread-native .................. 73780 |==== Proxmox ZFS Raid 1 WB ZFS 0.7.6 iothread-native moswap ........... 89220 |===== Proxmox ZFS Raid 1 N ZFS 0.7.6 iothread-native noswap nodiskwc 2 . 18900 |= Flexible IO Tester 3.1 Type: Sequential Write - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory MB/s > Higher Is Better TR150 SSD: EXT4 ......................................................... 413.00 | TR150 SSD: F2FS ......................................................... 416.00 | TR150 SSD: Btrfs ........................................................ 83.68 | TR150 SSD: XFS .......................................................... 395.00 | Seagate HDD: XFS ........................................................ 145.00 | Seagate HDD: Btrfs ...................................................... 4.04 | Seagate HDD: EXT4 ....................................................... 149.00 | Virtio ZFS HDD Raid 0 2 ................................................. 197.00 | Virtio ZFS HDD Raid 10 .................................................. 219.00 | Virtio ZFS HDD Raid 10 WBU .............................................. 228.00 | XenServer 7.4 Adaptec 6805 Raid 1 PV .................................... 95.57 | Proxmox ZFS Raid 1 WT ................................................... 0.54 | Proxmox ZFS Raid 1 WB ................................................... 21.60 | Proxmox ZFS Raid 1 WB metadata .......................................... 4.04 | Proxmox ZFS Raid 1 WB metadata throughput ............................... 2.65 | Proxmox ZFS Raid 1 WB 2 ................................................. 5.58 | Proxmox ZFS Raid 1 WB ZFS 0.7.6 iothread-native ......................... 23.50 | Proxmox ZFS Raid 1 WB ZFS 0.7.6 iothread-native moswap .................. 15.97 | Proxmox ZFS Raid 1 N ZFS 0.7.6 iothread-native noswap nodiskwc 2 ........ 17.40 | Proxmox ZFS Raid 1 N metadata ZFS 0.7.6 iothread-native noswap nodiskwc . 13.99 | Flexible IO Tester 3.1 Type: Sequential Write - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory IOPS > Higher Is Better TR150 SSD: EXT4 ........................................ 106000 |============== TR150 SSD: F2FS ........................................ 106333 |============== TR150 SSD: Btrfs ....................................... 21500 |=== TR150 SSD: XFS ......................................... 101167 |============= Seagate HDD: XFS ....................................... 37050 |===== Seagate HDD: EXT4 ...................................... 38233 |===== Virtio ZFS HDD Raid 0 2 ................................ 50150 |======= Virtio ZFS HDD Raid 10 ................................. 56000 |======= Virtio ZFS HDD Raid 10 WBU ............................. 58317 |======== XenServer 7.4 Adaptec 6805 Raid 1 PV ................... 24433 |=== Proxmox ZFS Raid 1 WB ZFS 0.7.6 iothread-native moswap . 11200 |= BlogBench 1.0 Test: Read Final Score > Higher Is Better TR150 SSD: EXT4 ......................................................... 2097180 | TR150 SSD: F2FS ......................................................... 2054932 | TR150 SSD: Btrfs ........................................................ 2269750 | TR150 SSD: XFS .......................................................... 2084376 | Seagate HDD: XFS ........................................................ 2197243 | Seagate HDD: Btrfs ...................................................... 2293433 | Seagate HDD: EXT4 ....................................................... 2415373 | Virtio ZFS HDD Raid 0 2 ................................................. 214407 | Virtio ZFS HDD Raid 10 .................................................. 195626 | Virtio ZFS HDD Raid 10 WBU .............................................. 228394 | XenServer 7.4 Adaptec 6805 Raid 1 PV .................................... 183352 | Proxmox ZFS Raid 1 WT ................................................... 257117 | Proxmox ZFS Raid 1 WB ................................................... 243293 | Proxmox ZFS Raid 1 WB metadata .......................................... 235041 | Proxmox ZFS Raid 1 WB metadata throughput ............................... 233116 | Proxmox ZFS Raid 1 WB 2 ................................................. 241096 | Proxmox ZFS Raid 1 WB ZFS 0.7.6 iothread-native ......................... 210892 | Proxmox ZFS Raid 1 WB ZFS 0.7.6 iothread-native moswap .................. 203102 | Proxmox ZFS Raid 1 N ZFS 0.7.6 iothread-native noswap nodiskwc 2 ........ 230306 | Proxmox ZFS Raid 1 N metadata ZFS 0.7.6 iothread-native noswap nodiskwc . 221821 | Proxmox ZFS Raid 1 LXC 3 ................................................ 159289 | BlogBench 1.0 Test: Write Final Score > Higher Is Better TR150 SSD: EXT4 ......................................................... 10325 | TR150 SSD: F2FS ......................................................... 9156 | TR150 SSD: Btrfs ........................................................ 3458 | TR150 SSD: XFS .......................................................... 4039 | Seagate HDD: XFS ........................................................ 2126 | Seagate HDD: Btrfs ...................................................... 2370 | Seagate HDD: EXT4 ....................................................... 6473 | Virtio ZFS HDD Raid 0 2 ................................................. 1930 | Virtio ZFS HDD Raid 10 .................................................. 1864 | Virtio ZFS HDD Raid 10 WBU .............................................. 1966 | XenServer 7.4 Adaptec 6805 Raid 1 PV .................................... 721 | Proxmox ZFS Raid 1 WT ................................................... 550 | Proxmox ZFS Raid 1 WB ................................................... 678 | Proxmox ZFS Raid 1 WB metadata .......................................... 739 | Proxmox ZFS Raid 1 WB metadata throughput ............................... 754 | Proxmox ZFS Raid 1 WB 2 ................................................. 734 | Proxmox ZFS Raid 1 WB ZFS 0.7.6 iothread-native ......................... 433 | Proxmox ZFS Raid 1 WB ZFS 0.7.6 iothread-native moswap .................. 800 | Proxmox ZFS Raid 1 N ZFS 0.7.6 iothread-native noswap nodiskwc 2 ........ 884 | Proxmox ZFS Raid 1 N metadata ZFS 0.7.6 iothread-native noswap nodiskwc . 871 | Proxmox ZFS Raid 1 LXC 3 ................................................ 786 | Dbench 4.0 Client Count: 6 MB/s > Higher Is Better TR150 SSD: EXT4 ......................................................... 369.21 | TR150 SSD: F2FS ......................................................... 276.31 | TR150 SSD: Btrfs ........................................................ 247.49 | TR150 SSD: XFS .......................................................... 442.69 | Seagate HDD: XFS ........................................................ 20.21 | Seagate HDD: Btrfs ...................................................... 45.74 | Seagate HDD: EXT4 ....................................................... 25.73 | Virtio ZFS HDD Raid 0 2 ................................................. 72.29 | Virtio ZFS HDD Raid 10 .................................................. 56.88 | Virtio ZFS HDD Raid 10 WBU .............................................. 1553.66 | XenServer 7.4 Adaptec 6805 Raid 1 PV .................................... 212.75 | Proxmox ZFS Raid 1 WT ................................................... 50.29 | Proxmox ZFS Raid 1 WB ................................................... 48.41 | Proxmox ZFS Raid 1 WB metadata .......................................... 41.18 | Proxmox ZFS Raid 1 WB metadata throughput ............................... 41.46 | Proxmox ZFS Raid 1 WB 2 ................................................. 49.10 | Proxmox ZFS Raid 1 WB ZFS 0.7.6 iothread-native ......................... 48.44 | Proxmox ZFS Raid 1 WB ZFS 0.7.6 iothread-native moswap .................. 49.69 | Proxmox ZFS Raid 1 N ZFS 0.7.6 iothread-native noswap nodiskwc 2 ........ 48.86 | Proxmox ZFS Raid 1 N metadata ZFS 0.7.6 iothread-native noswap nodiskwc . 41.25 | Proxmox ZFS Raid 1 LXC 3 ................................................ 91.57 | IOzone 3.465 Record Size: 4Kb - File Size: 8GB - Disk Test: Write Performance MB/s > Higher Is Better TR150 SSD: EXT4 ......................................................... 103.49 | TR150 SSD: F2FS ......................................................... 88.99 | TR150 SSD: Btrfs ........................................................ 93.20 | TR150 SSD: XFS .......................................................... 96.10 | Seagate HDD: XFS ........................................................ 153.08 | Seagate HDD: Btrfs ...................................................... 161.01 | Seagate HDD: EXT4 ....................................................... 155.45 | Virtio ZFS HDD Raid 0 2 ................................................. 101.53 | Virtio ZFS HDD Raid 10 .................................................. 109.03 | Virtio ZFS HDD Raid 10 WBU .............................................. 152.17 | XenServer 7.4 Adaptec 6805 Raid 1 PV .................................... 79.12 | Proxmox ZFS Raid 1 WT ................................................... 38.57 | Proxmox ZFS Raid 1 WB ................................................... 66.76 | Proxmox ZFS Raid 1 WB metadata .......................................... 66.20 | Proxmox ZFS Raid 1 WB metadata throughput ............................... 67.43 | Proxmox ZFS Raid 1 WB 2 ................................................. 68.57 | Proxmox ZFS Raid 1 WB ZFS 0.7.6 iothread-native ......................... 57.09 | Proxmox ZFS Raid 1 WB ZFS 0.7.6 iothread-native moswap .................. 83.20 | Proxmox ZFS Raid 1 N ZFS 0.7.6 iothread-native noswap nodiskwc 2 ........ 78.67 | Proxmox ZFS Raid 1 N metadata ZFS 0.7.6 iothread-native noswap nodiskwc . 75.92 | Proxmox ZFS Raid 1 LXC 3 ................................................ 238.21 | Compile Bench 0.6 Test: Compile MB/s > Higher Is Better TR150 SSD: EXT4 ......................................................... 1686.54 | TR150 SSD: F2FS ......................................................... 2271.93 | TR150 SSD: Btrfs ........................................................ 1371.80 | TR150 SSD: XFS .......................................................... 2101.38 | Seagate HDD: XFS ........................................................ 2092.61 | Seagate HDD: Btrfs ...................................................... 2177.85 | Seagate HDD: EXT4 ....................................................... 1678.76 | Virtio ZFS HDD Raid 0 2 ................................................. 538.22 | Virtio ZFS HDD Raid 10 .................................................. 495.46 | Virtio ZFS HDD Raid 10 WBU .............................................. 300.82 | XenServer 7.4 Adaptec 6805 Raid 1 PV .................................... 75.83 | Proxmox ZFS Raid 1 WT ................................................... 39.55 | Proxmox ZFS Raid 1 WB ................................................... 119.87 | Proxmox ZFS Raid 1 WB metadata .......................................... 129.16 | Proxmox ZFS Raid 1 WB metadata throughput ............................... 127.59 | Proxmox ZFS Raid 1 WB 2 ................................................. 121.69 | Proxmox ZFS Raid 1 WB ZFS 0.7.6 iothread-native ......................... 122.37 | Proxmox ZFS Raid 1 WB ZFS 0.7.6 iothread-native moswap .................. 136.28 | Proxmox ZFS Raid 1 N ZFS 0.7.6 iothread-native noswap nodiskwc 2 ........ 102.48 | Proxmox ZFS Raid 1 N metadata ZFS 0.7.6 iothread-native noswap nodiskwc . 100.66 | Proxmox ZFS Raid 1 LXC 3 ................................................ 234.53 | Compile Bench 0.6 Test: Initial Create MB/s > Higher Is Better TR150 SSD: EXT4 ......................................................... 505.73 | TR150 SSD: F2FS ......................................................... 550.41 | TR150 SSD: Btrfs ........................................................ 109.66 | TR150 SSD: XFS .......................................................... 395.06 | Seagate HDD: XFS ........................................................ 402.86 | Seagate HDD: Btrfs ...................................................... 262.66 | Seagate HDD: EXT4 ....................................................... 494.62 | Virtio ZFS HDD Raid 0 2 ................................................. 255.10 | Virtio ZFS HDD Raid 10 .................................................. 241.59 | Virtio ZFS HDD Raid 10 WBU .............................................. 212.93 | XenServer 7.4 Adaptec 6805 Raid 1 PV .................................... 68.28 | Proxmox ZFS Raid 1 WT ................................................... 51.18 | Proxmox ZFS Raid 1 WB ................................................... 83.03 | Proxmox ZFS Raid 1 WB metadata .......................................... 81.49 | Proxmox ZFS Raid 1 WB metadata throughput ............................... 72.65 | Proxmox ZFS Raid 1 WB 2 ................................................. 83.29 | Proxmox ZFS Raid 1 WB ZFS 0.7.6 iothread-native ......................... 73.81 | Proxmox ZFS Raid 1 WB ZFS 0.7.6 iothread-native moswap .................. 83.55 | Proxmox ZFS Raid 1 N ZFS 0.7.6 iothread-native noswap nodiskwc 2 ........ 82.72 | Proxmox ZFS Raid 1 N metadata ZFS 0.7.6 iothread-native noswap nodiskwc . 70.16 | Proxmox ZFS Raid 1 LXC 3 ................................................ 86.20 | Compile Bench 0.6 Test: Read Compiled Tree MB/s > Higher Is Better TR150 SSD: EXT4 ......................................................... 837.97 | TR150 SSD: F2FS ......................................................... 878.02 | TR150 SSD: Btrfs ........................................................ 881.31 | TR150 SSD: XFS .......................................................... 892.28 | Seagate HDD: XFS ........................................................ 833.32 | Seagate HDD: Btrfs ...................................................... 804.92 | Seagate HDD: EXT4 ....................................................... 801.75 | Virtio ZFS HDD Raid 0 2 ................................................. 807.12 | Virtio ZFS HDD Raid 10 .................................................. 887.85 | Virtio ZFS HDD Raid 10 WBU .............................................. 746.27 | XenServer 7.4 Adaptec 6805 Raid 1 PV .................................... 230.68 | Proxmox ZFS Raid 1 WT ................................................... 326.85 | Proxmox ZFS Raid 1 WB ................................................... 315.73 | Proxmox ZFS Raid 1 WB metadata .......................................... 313.92 | Proxmox ZFS Raid 1 WB metadata throughput ............................... 320.41 | Proxmox ZFS Raid 1 WB 2 ................................................. 313.49 | Proxmox ZFS Raid 1 WB ZFS 0.7.6 iothread-native ......................... 301.25 | Proxmox ZFS Raid 1 WB ZFS 0.7.6 iothread-native moswap .................. 305.55 | Proxmox ZFS Raid 1 N ZFS 0.7.6 iothread-native noswap nodiskwc 2 ........ 314.06 | Proxmox ZFS Raid 1 N metadata ZFS 0.7.6 iothread-native noswap nodiskwc . 318.96 | Proxmox ZFS Raid 1 LXC 3 ................................................ 73.35 | Unpacking The Linux Kernel linux-4.15.tar.xz Seconds < Lower Is Better TR150 SSD: EXT4 ......................................................... 6.40 | TR150 SSD: F2FS ......................................................... 6.67 | TR150 SSD: Btrfs ........................................................ 9.44 | TR150 SSD: XFS .......................................................... 6.82 | Seagate HDD: XFS ........................................................ 7.65 | Seagate HDD: Btrfs ...................................................... 7.25 | Seagate HDD: EXT4 ....................................................... 6.67 | Virtio ZFS HDD Raid 0 2 ................................................. 15.97 | Virtio ZFS HDD Raid 10 .................................................. 14.52 | Virtio ZFS HDD Raid 10 WBU .............................................. 10.65 | XenServer 7.4 Adaptec 6805 Raid 1 PV .................................... 15.93 | Proxmox ZFS Raid 1 WT ................................................... 29.63 | Proxmox ZFS Raid 1 WB ................................................... 30.03 | Proxmox ZFS Raid 1 WB metadata .......................................... 30.83 | Proxmox ZFS Raid 1 WB metadata throughput ............................... 25.42 | Proxmox ZFS Raid 1 WB 2 ................................................. 31.56 | Proxmox ZFS Raid 1 WB ZFS 0.7.6 iothread-native ......................... 31.63 | Proxmox ZFS Raid 1 WB ZFS 0.7.6 iothread-native moswap .................. 31.08 | Proxmox ZFS Raid 1 N ZFS 0.7.6 iothread-native noswap nodiskwc 2 ........ 30.53 | Proxmox ZFS Raid 1 N metadata ZFS 0.7.6 iothread-native noswap nodiskwc . 30.06 | Proxmox ZFS Raid 1 LXC 3 ................................................ 34.12 | Git Time To Complete Common Git Commands Seconds < Lower Is Better TR150 SSD: EXT4 ......................................................... 6.31 | TR150 SSD: F2FS ......................................................... 6.40 | TR150 SSD: Btrfs ........................................................ 6.36 | TR150 SSD: XFS .......................................................... 6.65 | Seagate HDD: XFS ........................................................ 6.76 | Seagate HDD: Btrfs ...................................................... 6.59 | Seagate HDD: EXT4 ....................................................... 6.57 | Virtio ZFS HDD Raid 0 2 ................................................. 6.67 | Virtio ZFS HDD Raid 10 .................................................. 7.15 | Virtio ZFS HDD Raid 10 WBU .............................................. 7.57 | XenServer 7.4 Adaptec 6805 Raid 1 PV .................................... 17.81 | Proxmox ZFS Raid 1 WT ................................................... 25.70 | Proxmox ZFS Raid 1 WB ................................................... 20.08 | Proxmox ZFS Raid 1 WB metadata .......................................... 20.56 | Proxmox ZFS Raid 1 WB metadata throughput ............................... 20.64 | Proxmox ZFS Raid 1 WB 2 ................................................. 20.39 | Proxmox ZFS Raid 1 WB ZFS 0.7.6 iothread-native ......................... 20.57 | Proxmox ZFS Raid 1 WB ZFS 0.7.6 iothread-native moswap .................. 21.14 | Proxmox ZFS Raid 1 N ZFS 0.7.6 iothread-native noswap nodiskwc 2 ........ 22.97 | Proxmox ZFS Raid 1 N metadata ZFS 0.7.6 iothread-native noswap nodiskwc . 20.67 | Proxmox ZFS Raid 1 LXC 3 ................................................ 79.74 |