ssd Bionic LXD container in Xenial host, ZFS on 2x(4x6TB HDD RAIDZ) Ceph 4 SSD, 1Gb eth: Processor: Intel Core i5-5300U @ 2.90GHz (2 Cores / 4 Threads), Motherboard: Intel NUC5i5MYBE (MYBDWi5v.86A.0026.2015.0820.1501 BIOS), Chipset: Intel Broadwell-U-OPI, Memory: 16384MB, Disk: 256GB Samsung SSD 850, Graphics: Intel HD 5500 (900MHz), Audio: Intel Broadwell-U Audio, Network: Intel Connection OS: Ubuntu 18.04, Kernel: 4.15.0-13-generic (x86_64), Compiler: GCC 7.3.0, File-System: ext4, System Layer: lxc DRBD/RAID1/NVMe, 1Gb eth: Processor: Intel Xeon E3-1280 v6 @ 4.20GHz (4 Cores / 8 Threads), Motherboard: Supermicro X11SSL-F v1.01 (2.0a BIOS), Memory: 64512MB, Disk: 2 x 800GB INTEL SSDSC2BB80 + 2 x 400GB INTEL SSDPEDME400G4, Graphics: astdrmfb OS: Ubuntu 16.04, Kernel: 4.4.0-93-generic (x86_64), Compiler: GCC 5.4.0 20160609, File-System: ext4, Screen Resolution: 1024x768, System Layer: lxc LXD/ZFS/HD: Processor: Intel Xeon E3-1240 v3 @ 3.80GHz (4 Cores / 8 Threads), Motherboard: Supermicro X10SLL-F v1.02 (2.2 BIOS), Chipset: Intel Xeon E3-1200 v3 DRAM, Memory: 32768MB, Disk: 8 x 6001GB Western Digital WD60EFRX-68M + 2 x 120GB INTEL SSDSC2BB12, Graphics: ASPEED ASPEED Family, Network: Intel Connection I217-LM OS: Ubuntu 18.04, Kernel: 4.4.0-116-generic (x86_64), Compiler: GCC 7.3.0, File-System: zfs, Screen Resolution: 1024x768, System Layer: lxc NVMe: Processor: Intel Xeon E3-1280 v6 @ 4.20GHz (4 Cores / 8 Threads), Motherboard: Supermicro X11SSL-F v1.01 (2.0a BIOS), Memory: 64512MB, Disk: 2 x 800GB INTEL SSDSC2BB80 + 2 x 400GB INTEL SSDPEDME400G4, Graphics: astdrmfb OS: Ubuntu 18.04, Kernel: 4.4.0-93-generic (x86_64), Compiler: GCC 7.3.0, File-System: ext2, Screen Resolution: 1024x768, System Layer: lxc SSD: Processor: Intel Core i5-5300U @ 2.90GHz (2 Cores / 4 Threads), Motherboard: Intel NUC5i5MYBE (MYBDWi5v.86A.0026.2015.0820.1501 BIOS), Chipset: Intel Broadwell-U-OPI, Memory: 16384MB, Disk: 512GB Samsung SSD 850, Graphics: Intel HD 5500 (900MHz), Audio: Intel Broadwell-U Audio, Monitor: HP w20, Network: Intel Connection OS: Ubuntu 18.04, Kernel: 4.15.0-13-generic (x86_64), Compiler: GCC 7.3.0, File-System: ext4, Screen Resolution: 1680x1050 AIO-Stress 0.21 Random Write MB/s > Higher Is Better Ceph 4 SSD, 1Gb eth ...... 2219.44 |=========================== DRBD/RAID1/NVMe, 1Gb eth . 3573.95 |=========================================== LXD/ZFS/HD ............... 1382.49 |================= NVMe ..................... 1918.11 |======================= SSD ...................... 2292.94 |============================ SQLite 3.22 Timed SQLite Insertions Seconds < Lower Is Better Ceph 4 SSD, 1Gb eth ...... 323.25 |============================================ DRBD/RAID1/NVMe, 1Gb eth . 2.54 | LXD/ZFS/HD ............... 269.14 |===================================== NVMe ..................... 2.73 | SSD ...................... 135.45 |================== FS-Mark 3.3 1000 Files, 1MB Size Files/s > Higher Is Better Ceph 4 SSD, 1Gb eth ...... 17.37 |== DRBD/RAID1/NVMe, 1Gb eth . 433.03 |============================================ LXD/ZFS/HD ............... 52.67 |===== NVMe ..................... 421.70 |=========================================== SSD ...................... 57.90 |====== Dbench 4.0 12 Clients MB/s > Higher Is Better Ceph 4 SSD, 1Gb eth ...... 86.08 |= DRBD/RAID1/NVMe, 1Gb eth . 3142.73 |================================= LXD/ZFS/HD ............... 139.65 |= NVMe ..................... 4050.65 |=========================================== SSD ...................... 210.28 |== Dbench 4.0 48 Clients MB/s > Higher Is Better Ceph 4 SSD, 1Gb eth ...... 189.28 |== DRBD/RAID1/NVMe, 1Gb eth . 3239.23 |============================ LXD/ZFS/HD ............... 174.95 |== NVMe ..................... 4928.96 |=========================================== SSD ...................... 570.62 |===== Dbench 4.0 128 Clients MB/s > Higher Is Better Ceph 4 SSD, 1Gb eth ...... 286.69 |=== DRBD/RAID1/NVMe, 1Gb eth . 2739.20 |========================= LXD/ZFS/HD ............... 240.45 |== NVMe ..................... 4649.74 |=========================================== SSD ...................... 1010.77 |========= Dbench 4.0 1 Clients MB/s > Higher Is Better Ceph 4 SSD, 1Gb eth ...... 17.89 |= DRBD/RAID1/NVMe, 1Gb eth . 923.36 |============================================ LXD/ZFS/HD ............... 33.38 |== NVMe ..................... 808.69 |======================================= SSD ...................... 42.76 |== Threaded I/O Tester 20170503 64MB Random Read - 32 Threads MB/s > Higher Is Better Ceph 4 SSD, 1Gb eth ...... 47240.68 |======= DRBD/RAID1/NVMe, 1Gb eth . 270736.23 |========================================= LXD/ZFS/HD ............... 14446.66 |== NVMe ..................... 237556.39 |==================================== SSD ...................... 73033.05 |=========== Threaded I/O Tester 20170503 64MB Random Write - 32 Threads MB/s > Higher Is Better Ceph 4 SSD, 1Gb eth ...... 62.34 | DRBD/RAID1/NVMe, 1Gb eth . 6206.57 |=========================================== LXD/ZFS/HD ............... 42.94 | NVMe ..................... 5618.22 |======================================= SSD ...................... 93.97 |= Compile Bench 0.6 Test: Compile MB/s > Higher Is Better Ceph 4 SSD, 1Gb eth ...... 436.69 |======== DRBD/RAID1/NVMe, 1Gb eth . 2471.56 |=========================================== LXD/ZFS/HD ............... 1167.99 |==================== NVMe ..................... 1732.33 |============================== SSD ...................... 862.26 |=============== Compile Bench 0.6 Test: Initial Create MB/s > Higher Is Better Ceph 4 SSD, 1Gb eth ...... 187.37 |================ DRBD/RAID1/NVMe, 1Gb eth . 520.65 |============================================ LXD/ZFS/HD ............... 192.40 |================ NVMe ..................... 206.23 |================= SSD ...................... 350.97 |============================== Compile Bench 0.6 Test: Read Compiled Tree MB/s > Higher Is Better Ceph 4 SSD, 1Gb eth ...... 1060.11 |================= DRBD/RAID1/NVMe, 1Gb eth . 2685.94 |=========================================== LXD/ZFS/HD ............... 708.39 |=========== NVMe ..................... 2304.86 |===================================== SSD ...................... 2042.16 |================================= Unpacking The Linux Kernel linux-4.15.tar.xz Seconds < Lower Is Better Ceph 4 SSD, 1Gb eth ...... 9.19 |====================================== DRBD/RAID1/NVMe, 1Gb eth . 5.64 |======================= LXD/ZFS/HD ............... 7.73 |================================ NVMe ..................... 10.85 |============================================= SSD ...................... 8.26 |================================== PostMark 1.51 Disk Transaction Performance TPS > Higher Is Better Ceph 4 SSD, 1Gb eth ...... 4261 |=========================== DRBD/RAID1/NVMe, 1Gb eth . 7282 |============================================== LXD/ZFS/HD ............... 3157 |==================== NVMe ..................... 5708 |==================================== SSD ...................... 4807 |============================== Gzip Compression Linux Source Tree Archiving To .tar.gz Seconds < Lower Is Better Ceph 4 SSD, 1Gb eth ...... 52.77 |============================================= DRBD/RAID1/NVMe, 1Gb eth . 34.08 |============================= LXD/ZFS/HD ............... 38.56 |================================= NVMe ..................... 36.06 |=============================== SSD ...................... 50.16 |=========================================== Apache Benchmark 2.4.29 Static Web Page Serving Requests Per Second > Higher Is Better Ceph 4 SSD, 1Gb eth ...... 14605.66 |==================== DRBD/RAID1/NVMe, 1Gb eth . 30202.25 |========================================== LXD/ZFS/HD ............... 17967.17 |========================= NVMe ..................... 30230.49 |========================================== SSD ...................... 11945.86 |=================