ssd Bionic LXD container in Xenial host, ZFS on 2x(4x6TB HDD RAIDZ) Ceph/SSD, 1Gb/s: Processor: Intel Core i5-5300U @ 2.90GHz (2 Cores / 4 Threads), Motherboard: Intel NUC5i5MYBE (MYBDWi5v.86A.0026.2015.0820.1501 BIOS), Chipset: Intel Broadwell-U-OPI, Memory: 16384MB, Disk: 256GB Samsung SSD 850, Graphics: Intel HD 5500 (900MHz), Audio: Intel Broadwell-U Audio, Network: Intel Connection OS: Ubuntu 18.04, Kernel: 4.15.0-13-generic (x86_64), Compiler: GCC 7.3.0, File-System: ext4, System Layer: lxc DRBD/NVMe: Processor: Intel Xeon E3-1280 v6 @ 4.20GHz (4 Cores / 8 Threads), Motherboard: Supermicro X11SSL-F v1.01 (2.0a BIOS), Memory: 64512MB, Disk: 2 x 800GB INTEL SSDSC2BB80 + 2 x 400GB INTEL SSDPEDME400G4, Graphics: astdrmfb OS: Ubuntu 16.04, Kernel: 4.4.0-93-generic (x86_64), Compiler: GCC 5.4.0 20160609, File-System: ext4, Screen Resolution: 1024x768, System Layer: lxc Ubuntu Bionic, LXD/ZFS/HDD: Processor: Intel Xeon E3-1240 v3 @ 3.80GHz (4 Cores / 8 Threads), Motherboard: Supermicro X10SLL-F v1.02 (2.2 BIOS), Chipset: Intel Xeon E3-1200 v3 DRAM, Memory: 32768MB, Disk: 8 x 6001GB Western Digital WD60EFRX-68M + 2 x 120GB INTEL SSDSC2BB12, Graphics: ASPEED ASPEED Family, Network: Intel Connection I217-LM OS: Ubuntu 18.04, Kernel: 4.4.0-116-generic (x86_64), Compiler: GCC 7.3.0, File-System: zfs, Screen Resolution: 1024x768, System Layer: lxc Ubuntu Bionic, ext4/LVM/SSD Samsung 850: Processor: Intel Core i5-5300U @ 2.90GHz (2 Cores / 4 Threads), Motherboard: Intel NUC5i5MYBE (MYBDWi5v.86A.0026.2015.0820.1501 BIOS), Chipset: Intel Broadwell-U-OPI, Memory: 16384MB, Disk: 512GB Samsung SSD 850, Graphics: Intel HD 5500 (900MHz), Audio: Intel Broadwell-U Audio, Monitor: HP w20, Network: Intel Connection OS: Ubuntu 18.04, Kernel: 4.15.0-13-generic (x86_64), Compiler: GCC 7.3.0, File-System: ext4, Screen Resolution: 1680x1050 Ubuntu Bionic, ext4/LXC/NVMe: Processor: Intel Xeon E3-1280 v6 @ 4.20GHz (4 Cores / 8 Threads), Motherboard: Supermicro X11SSL-F v1.01 (2.0a BIOS), Memory: 64512MB, Disk: 2 x 800GB INTEL SSDSC2BB80 + 2 x 400GB INTEL SSDPEDME400G4, Graphics: astdrmfb OS: Ubuntu 18.04, Kernel: 4.4.0-93-generic (x86_64), Compiler: GCC 7.3.0, File-System: ext2, Screen Resolution: 1024x768, System Layer: lxc drbd-nvme: Processor: Intel Xeon E3-1280 v6 @ 4.20GHz (4 Cores / 8 Threads), Motherboard: Supermicro X11SSL-F v1.01 (2.0a BIOS), Memory: 64512MB, Disk: 2 x 800GB INTEL SSDSC2BB80 + 2 x 400GB INTEL SSDPEDME400G4, Graphics: astdrmfb OS: Ubuntu 16.04, Kernel: 4.4.0-93-generic (x86_64), Compiler: GCC 5.4.0 20160609, File-System: ext4, Screen Resolution: 1024x768, System Layer: lxc AIO-Stress 0.21 Random Write MB/s > Higher Is Better Ceph/SSD, 1Gb/s ......................... 2219.44 |================= DRBD/NVMe ............................... 3573.95 |============================ Ubuntu Bionic, LXD/ZFS/HDD .............. 1382.49 |=========== Ubuntu Bionic, ext4/LVM/SSD Samsung 850 . 2292.94 |================== Ubuntu Bionic, ext4/LXC/NVMe ............ 1918.11 |=============== drbd-nvme ............................... 3582.94 |============================ SQLite 3.22 Timed SQLite Insertions Seconds < Lower Is Better Ceph/SSD, 1Gb/s ......................... 323.25 |============================= DRBD/NVMe ............................... 2.54 | Ubuntu Bionic, LXD/ZFS/HDD .............. 269.14 |======================== Ubuntu Bionic, ext4/LVM/SSD Samsung 850 . 135.45 |============ Ubuntu Bionic, ext4/LXC/NVMe ............ 2.73 | FS-Mark 3.3 1000 Files, 1MB Size Files/s > Higher Is Better Ceph/SSD, 1Gb/s ......................... 17.37 |= DRBD/NVMe ............................... 433.03 |============================= Ubuntu Bionic, LXD/ZFS/HDD .............. 52.67 |==== Ubuntu Bionic, ext4/LVM/SSD Samsung 850 . 57.90 |==== Ubuntu Bionic, ext4/LXC/NVMe ............ 421.70 |============================ Dbench 4.0 12 Clients MB/s > Higher Is Better Ceph/SSD, 1Gb/s ......................... 86.08 |= DRBD/NVMe ............................... 3142.73 |====================== Ubuntu Bionic, LXD/ZFS/HDD .............. 139.65 |= Ubuntu Bionic, ext4/LVM/SSD Samsung 850 . 210.28 |= Ubuntu Bionic, ext4/LXC/NVMe ............ 4050.65 |============================ Dbench 4.0 48 Clients MB/s > Higher Is Better Ceph/SSD, 1Gb/s ......................... 189.28 |= DRBD/NVMe ............................... 3239.23 |================== Ubuntu Bionic, LXD/ZFS/HDD .............. 174.95 |= Ubuntu Bionic, ext4/LVM/SSD Samsung 850 . 570.62 |=== Ubuntu Bionic, ext4/LXC/NVMe ............ 4928.96 |============================ Dbench 4.0 128 Clients MB/s > Higher Is Better Ceph/SSD, 1Gb/s ......................... 286.69 |== DRBD/NVMe ............................... 2739.20 |================ Ubuntu Bionic, LXD/ZFS/HDD .............. 240.45 |= Ubuntu Bionic, ext4/LVM/SSD Samsung 850 . 1010.77 |====== Ubuntu Bionic, ext4/LXC/NVMe ............ 4649.74 |============================ Dbench 4.0 1 Clients MB/s > Higher Is Better Ceph/SSD, 1Gb/s ......................... 17.89 |= DRBD/NVMe ............................... 923.36 |============================= Ubuntu Bionic, LXD/ZFS/HDD .............. 33.38 |= Ubuntu Bionic, ext4/LVM/SSD Samsung 850 . 42.76 |= Ubuntu Bionic, ext4/LXC/NVMe ............ 808.69 |========================= Threaded I/O Tester 20170503 64MB Random Read - 32 Threads MB/s > Higher Is Better Ceph/SSD, 1Gb/s ......................... 47240.68 |===== DRBD/NVMe ............................... 270736.23 |========================== Ubuntu Bionic, LXD/ZFS/HDD .............. 14446.66 |= Ubuntu Bionic, ext4/LVM/SSD Samsung 850 . 73033.05 |======= Ubuntu Bionic, ext4/LXC/NVMe ............ 237556.39 |======================= Threaded I/O Tester 20170503 64MB Random Write - 32 Threads MB/s > Higher Is Better Ceph/SSD, 1Gb/s ......................... 62.34 | DRBD/NVMe ............................... 6206.57 |============================ Ubuntu Bionic, LXD/ZFS/HDD .............. 42.94 | Ubuntu Bionic, ext4/LVM/SSD Samsung 850 . 93.97 | Ubuntu Bionic, ext4/LXC/NVMe ............ 5618.22 |========================= Compile Bench 0.6 Test: Compile MB/s > Higher Is Better Ceph/SSD, 1Gb/s ......................... 436.69 |===== DRBD/NVMe ............................... 2471.56 |============================ Ubuntu Bionic, LXD/ZFS/HDD .............. 1167.99 |============= Ubuntu Bionic, ext4/LVM/SSD Samsung 850 . 862.26 |========== Ubuntu Bionic, ext4/LXC/NVMe ............ 1732.33 |==================== Compile Bench 0.6 Test: Initial Create MB/s > Higher Is Better Ceph/SSD, 1Gb/s ......................... 187.37 |========== DRBD/NVMe ............................... 520.65 |============================= Ubuntu Bionic, LXD/ZFS/HDD .............. 192.40 |=========== Ubuntu Bionic, ext4/LVM/SSD Samsung 850 . 350.97 |==================== Ubuntu Bionic, ext4/LXC/NVMe ............ 206.23 |=========== Compile Bench 0.6 Test: Read Compiled Tree MB/s > Higher Is Better Ceph/SSD, 1Gb/s ......................... 1060.11 |=========== DRBD/NVMe ............................... 2685.94 |============================ Ubuntu Bionic, LXD/ZFS/HDD .............. 708.39 |======= Ubuntu Bionic, ext4/LVM/SSD Samsung 850 . 2042.16 |===================== Ubuntu Bionic, ext4/LXC/NVMe ............ 2304.86 |======================== Unpacking The Linux Kernel linux-4.15.tar.xz Seconds < Lower Is Better Ceph/SSD, 1Gb/s ......................... 9.19 |========================= DRBD/NVMe ............................... 5.64 |================ Ubuntu Bionic, LXD/ZFS/HDD .............. 7.73 |===================== Ubuntu Bionic, ext4/LVM/SSD Samsung 850 . 8.26 |======================= Ubuntu Bionic, ext4/LXC/NVMe ............ 10.85 |============================== PostMark 1.51 Disk Transaction Performance TPS > Higher Is Better Ceph/SSD, 1Gb/s ......................... 4261 |================== DRBD/NVMe ............................... 7282 |=============================== Ubuntu Bionic, LXD/ZFS/HDD .............. 3157 |============= Ubuntu Bionic, ext4/LVM/SSD Samsung 850 . 4807 |==================== Ubuntu Bionic, ext4/LXC/NVMe ............ 5708 |======================== Gzip Compression Linux Source Tree Archiving To .tar.gz Seconds < Lower Is Better Ceph/SSD, 1Gb/s ......................... 52.77 |============================== DRBD/NVMe ............................... 34.08 |=================== Ubuntu Bionic, LXD/ZFS/HDD .............. 38.56 |====================== Ubuntu Bionic, ext4/LVM/SSD Samsung 850 . 50.16 |============================= Ubuntu Bionic, ext4/LXC/NVMe ............ 36.06 |===================== Apache Benchmark 2.4.29 Static Web Page Serving Requests Per Second > Higher Is Better Ceph/SSD, 1Gb/s ......................... 14605.66 |============= DRBD/NVMe ............................... 30202.25 |=========================== Ubuntu Bionic, LXD/ZFS/HDD .............. 17967.17 |================ Ubuntu Bionic, ext4/LVM/SSD Samsung 850 . 11945.86 |=========== Ubuntu Bionic, ext4/LXC/NVMe ............ 30230.49 |===========================