ssd Bionic LXD container in Xenial host, ZFS on 2x(4x6TB HDD RAIDZ) SSD: Processor: Intel Core i5-5300U @ 2.90GHz (2 Cores / 4 Threads), Motherboard: Intel NUC5i5MYBE (MYBDWi5v.86A.0026.2015.0820.1501 BIOS), Chipset: Intel Broadwell-U-OPI, Memory: 16384MB, Disk: 512GB Samsung SSD 850, Graphics: Intel HD 5500 (900MHz), Audio: Intel Broadwell-U Audio, Monitor: HP w20, Network: Intel Connection OS: Ubuntu 18.04, Kernel: 4.15.0-13-generic (x86_64), Compiler: GCC 7.3.0, File-System: ext4, Screen Resolution: 1680x1050 Ceph 4SS ODSs, 1Gb eth: Processor: Intel Core i5-5300U @ 2.90GHz (2 Cores / 4 Threads), Motherboard: Intel NUC5i5MYBE (MYBDWi5v.86A.0026.2015.0820.1501 BIOS), Chipset: Intel Broadwell-U-OPI, Memory: 16384MB, Disk: 256GB Samsung SSD 850, Graphics: Intel HD 5500 (900MHz), Audio: Intel Broadwell-U Audio, Network: Intel Connection OS: Ubuntu 18.04, Kernel: 4.15.0-13-generic (x86_64), Compiler: GCC 7.3.0, File-System: ext4, System Layer: lxc NVMe: Processor: Intel Xeon E3-1280 v6 @ 4.20GHz (4 Cores / 8 Threads), Motherboard: Supermicro X11SSL-F v1.01 (2.0a BIOS), Memory: 64512MB, Disk: 2 x 800GB INTEL SSDSC2BB80 + 2 x 400GB INTEL SSDPEDME400G4, Graphics: astdrmfb OS: Ubuntu 18.04, Kernel: 4.4.0-93-generic (x86_64), Compiler: GCC 7.3.0, File-System: ext2, Screen Resolution: 1024x768, System Layer: lxc DRBD/RAID1/NVMe: Processor: Intel Xeon E3-1280 v6 @ 4.20GHz (4 Cores / 8 Threads), Motherboard: Supermicro X11SSL-F v1.01 (2.0a BIOS), Memory: 64512MB, Disk: 2 x 800GB INTEL SSDSC2BB80 + 2 x 400GB INTEL SSDPEDME400G4, Graphics: astdrmfb OS: Ubuntu 16.04, Kernel: 4.4.0-93-generic (x86_64), Compiler: GCC 5.4.0 20160609, File-System: ext4, Screen Resolution: 1024x768, System Layer: lxc LXD/ZFS/HD: Processor: Intel Xeon E3-1240 v3 @ 3.80GHz (4 Cores / 8 Threads), Motherboard: Supermicro X10SLL-F v1.02 (2.2 BIOS), Chipset: Intel Xeon E3-1200 v3 DRAM, Memory: 32768MB, Disk: 8 x 6001GB Western Digital WD60EFRX-68M + 2 x 120GB INTEL SSDSC2BB12, Graphics: ASPEED ASPEED Family, Network: Intel Connection I217-LM OS: Ubuntu 18.04, Kernel: 4.4.0-116-generic (x86_64), Compiler: GCC 7.3.0, File-System: zfs, Screen Resolution: 1024x768, System Layer: lxc AIO-Stress 0.21 Random Write MB/s > Higher Is Better SSD .................... 2292.94 |============================= Ceph 4SS ODSs, 1Gb eth . 2219.44 |============================ NVMe ................... 1918.11 |======================== DRBD/RAID1/NVMe ........ 3573.95 |============================================= LXD/ZFS/HD ............. 1382.49 |================= Apache Benchmark 2.4.29 Static Web Page Serving Requests Per Second > Higher Is Better SSD .................... 11945.86 |================= Ceph 4SS ODSs, 1Gb eth . 14605.66 |===================== NVMe ................... 30230.49 |============================================ DRBD/RAID1/NVMe ........ 30202.25 |============================================ LXD/ZFS/HD ............. 17967.17 |========================== Compile Bench 0.6 Test: Compile MB/s > Higher Is Better SSD .................... 862.26 |================ Ceph 4SS ODSs, 1Gb eth . 436.69 |======== NVMe ................... 1732.33 |================================ DRBD/RAID1/NVMe ........ 2471.56 |============================================= LXD/ZFS/HD ............. 1167.99 |===================== Compile Bench 0.6 Test: Initial Create MB/s > Higher Is Better SSD .................... 350.97 |=============================== Ceph 4SS ODSs, 1Gb eth . 187.37 |================= NVMe ................... 206.23 |================== DRBD/RAID1/NVMe ........ 520.65 |============================================== LXD/ZFS/HD ............. 192.40 |================= Compile Bench 0.6 Test: Read Compiled Tree MB/s > Higher Is Better SSD .................... 2042.16 |================================== Ceph 4SS ODSs, 1Gb eth . 1060.11 |================== NVMe ................... 2304.86 |======================================= DRBD/RAID1/NVMe ........ 2685.94 |============================================= LXD/ZFS/HD ............. 708.39 |============ Dbench 4.0 12 Clients MB/s > Higher Is Better SSD .................... 210.28 |== Ceph 4SS ODSs, 1Gb eth . 86.08 |= NVMe ................... 4050.65 |============================================= DRBD/RAID1/NVMe ........ 3142.73 |=================================== LXD/ZFS/HD ............. 139.65 |== Dbench 4.0 48 Clients MB/s > Higher Is Better SSD .................... 570.62 |===== Ceph 4SS ODSs, 1Gb eth . 189.28 |== NVMe ................... 4928.96 |============================================= DRBD/RAID1/NVMe ........ 3239.23 |============================== LXD/ZFS/HD ............. 174.95 |== Dbench 4.0 128 Clients MB/s > Higher Is Better SSD .................... 1010.77 |========== Ceph 4SS ODSs, 1Gb eth . 286.69 |=== NVMe ................... 4649.74 |============================================= DRBD/RAID1/NVMe ........ 2739.20 |=========================== LXD/ZFS/HD ............. 240.45 |== Dbench 4.0 1 Clients MB/s > Higher Is Better SSD .................... 42.76 |== Ceph 4SS ODSs, 1Gb eth . 17.89 |= NVMe ................... 808.69 |======================================== DRBD/RAID1/NVMe ........ 923.36 |============================================== LXD/ZFS/HD ............. 33.38 |== FS-Mark 3.3 1000 Files, 1MB Size Files/s > Higher Is Better SSD .................... 57.90 |====== Ceph 4SS ODSs, 1Gb eth . 17.37 |== NVMe ................... 421.70 |============================================= DRBD/RAID1/NVMe ........ 433.03 |============================================== LXD/ZFS/HD ............. 52.67 |====== Gzip Compression Linux Source Tree Archiving To .tar.gz Seconds < Lower Is Better SSD .................... 50.16 |============================================= Ceph 4SS ODSs, 1Gb eth . 52.77 |=============================================== NVMe ................... 36.06 |================================ DRBD/RAID1/NVMe ........ 34.08 |============================== LXD/ZFS/HD ............. 38.56 |================================== PostMark 1.51 Disk Transaction Performance TPS > Higher Is Better SSD .................... 4807 |================================ Ceph 4SS ODSs, 1Gb eth . 4261 |============================ NVMe ................... 5708 |====================================== DRBD/RAID1/NVMe ........ 7282 |================================================ LXD/ZFS/HD ............. 3157 |===================== SQLite 3.22 Timed SQLite Insertions Seconds < Lower Is Better SSD .................... 135.45 |=================== Ceph 4SS ODSs, 1Gb eth . 323.25 |============================================== NVMe ................... 2.73 | DRBD/RAID1/NVMe ........ 2.54 | LXD/ZFS/HD ............. 269.14 |====================================== Threaded I/O Tester 20170503 64MB Random Read - 32 Threads MB/s > Higher Is Better SSD .................... 73033.05 |============ Ceph 4SS ODSs, 1Gb eth . 47240.68 |======== NVMe ................... 237556.39 |====================================== DRBD/RAID1/NVMe ........ 270736.23 |=========================================== LXD/ZFS/HD ............. 14446.66 |== Threaded I/O Tester 20170503 64MB Random Write - 32 Threads MB/s > Higher Is Better SSD .................... 93.97 |= Ceph 4SS ODSs, 1Gb eth . 62.34 | NVMe ................... 5618.22 |========================================= DRBD/RAID1/NVMe ........ 6206.57 |============================================= LXD/ZFS/HD ............. 42.94 | Unpacking The Linux Kernel linux-4.15.tar.xz Seconds < Lower Is Better SSD .................... 8.26 |==================================== Ceph 4SS ODSs, 1Gb eth . 9.19 |======================================== NVMe ................... 10.85 |=============================================== DRBD/RAID1/NVMe ........ 5.64 |======================== LXD/ZFS/HD ............. 7.73 |=================================