Linux 4.19: Bare Metal vs. KVM Guest

Tests for a future article.

HTML result view exported from: https://openbenchmarking.org/result/1808279-RA-ROUND243586&sro&grs.

ProcessorMotherboardChipsetMemoryDiskGraphicsMonitorNetworkAudioOSKernelDesktopDisplay ServerDisplay DriverCompilerFile-SystemScreen ResolutionSystem LayerStockNew Microcode Bare Metal VM Guest Bare Metal2 x Intel Xeon Gold 6138 @ 3.70GHz (40 Cores / 80 Threads)TYAN S7106 (V1.01 BIOS)Intel Sky Lake-E DMI3 Registers96256MB256GB Samsung SSD 850 + 2000GB Seagate ST2000DM006-2DM1 + 2 x 120GB TOSHIBA-TR150ASPEED ASPEED FamilyVE228Intel I210 Gigabit ConnectionUbuntu 18.044.18.0-999-generic (x86_64) 20180824GNOME Shell 3.28.2X Server 1.19.6modesetting 1.19.6GCC 7.3.0ext41920x108064 x Intel Xeon (Skylake IBRS) @ 2.00GHz (64 Cores)QEMU Standard PC (i440FX + PIIX 1996) (1.10.2-1ubuntu1 BIOS)Intel 440FX- 82441FX PMC47104MB162GBRed Hat QXL paravirtual graphic cardQEMU GenericRed Hat Virtio device1024x768KVM2 x Intel Xeon Gold 6138 @ 3.70GHz (40 Cores / 80 Threads)TYAN S7106 (V1.01 BIOS)Intel Sky Lake-E DMI3 Registers96256MB256GB Samsung SSD 850 + 2000GB Seagate ST2000DM006-2DM1 + 2 x 120GB TOSHIBA-TR150ASPEED ASPEED FamilyVE228Intel I210 Gigabit Connection1920x1080OpenBenchmarking.orgCompiler Details- --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++ --enable-libmpx --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-targets=nvptx-none --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-as=/usr/bin/x86_64-linux-gnu-as --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-ld=/usr/bin/x86_64-linux-gnu-ld --with-multilib-list=m32,m64,mx32 --with-target-system-zlib --with-tune=generic --without-cuda-driver -v Disk Details- Stock: Bare Metal: CFQ / errors=remount-ro,relatime,rw- Stock: VM Guest: NONE / errors=remount-ro,relatime,rw- New Microcode: Bare Metal: CFQ / errors=remount-ro,relatime,rwProcessor Details- Stock: Bare Metal, New Microcode: Bare Metal: Scaling Governor: intel_pstate powersavePython Details- Python 2.7.15rc1 + Python 3.6.5Security Details- Stock: Bare Metal: KPTI + __user pointer sanitization + Full generic retpoline IBPB IBRS_FW + PTE Inversion; VMX: conditional cache flushes SMT vulnerable Protection- Stock: VM Guest: KPTI + __user pointer sanitization + Full generic retpoline IBPB IBRS_FW + PTE Inversion Protection- New Microcode: Bare Metal: KPTI + __user pointer sanitization + Full generic retpoline IBPB IBRS_FW + SSB disabled via prctl and seccomp + PTE Inversion; VMX: conditional cache flushes SMT vulnerable Protection

stress-ng: Semaphoresprimesieve: 1e12 Prime Number Generationpgbench: Buffer Test - Normal Load - Read Writemysqlslap: 32pgbench: Buffer Test - Normal Load - Read Onlybuild-llvm: Time To Compilecompilebench: Compilefio: Rand Read - Linux AIO - No - Yes - 2MB - Default Test Directorysysbench: CPUblender: Barbershop - CPU-Onlyttsiod-renderer: Phong Rendering With Soft-Shadow Mappingsysbench: Memorycompilebench: Initial Createfio: Seq Read - Linux AIO - No - Yes - 2MB - Default Test Directorycompilebench: Read Compiled Treex265: H.265 Video Encodingnginx: Static Web Page Servingstress-ng: System V Message Passingstress-ng: Context Switchingstress-ng: Socket Activityredis: SETredis: GEThackbench: 32 - Processbuild-linux-kernel: Time To Compilefs-mark: 4000 Files, 32 Sub Dirs, 1MB Sizefs-mark: 5000 Files, 1MB Size, 4 Threadsfs-mark: 1000 Files, 1MB Sizefio: Seq Write - Linux AIO - No - Yes - 2MB - Default Test Directoryfio: Rand Write - Linux AIO - No - Yes - 2MB - Default Test DirectoryStockNew Microcode Bare Metal VM Guest Bare Metal1831962.438.622248.82246585224.79169.531712.7953158755.65374.05742.6811764465.11501.795312469.9310.3124591.867053890.1422234776.323454.631656081.462142971.1135.7333.7059.10220.18145.40513511600043.1324.621075.59131384280.70208.781467.6551676.09423.31673.7410796572.79460.822351.2810.3627078.5811422458.9112704770.9314127.951455207.361999876.2971.5437.2132.3639.8746.8744911711890399.798.641694.68221588527.68171.021698.5045859370.55373.98731.6011803679.99487.434962478.3610.3223745.747163233.3122976365.083313.121688379.922059062.2536.7333.4039.7040.8937.18127.9671.58OpenBenchmarking.org

Stress-NG

Test: Semaphores

New MicrocodeStockOpenBenchmarking.orgBogo Ops/s, More Is BetterStress-NG 0.07.26Test: SemaphoresBare MetalVM Guest400K800K1200K1600K2000KSE +/- 26014.39, N = 6SE +/- 29092.76, N = 3SE +/- 925.56, N = 31890399.791831962.43600043.131. (CC) gcc options: -O2 -std=gnu99 -lm -lz -lcrypt -lrt -lpthread -laio -lc

Primesieve

1e12 Prime Number Generation

New MicrocodeStockOpenBenchmarking.orgSeconds, Fewer Is BetterPrimesieve 7.11e12 Prime Number GenerationBare MetalVM Guest612182430SE +/- 0.02, N = 3SE +/- 0.03, N = 3SE +/- 0.15, N = 38.648.6224.621. (CXX) g++ options: -O3 -lpthread

PostgreSQL pgbench

Scaling: Buffer Test - Test: Normal Load - Mode: Read Write

New MicrocodeStockOpenBenchmarking.orgTPS, More Is BetterPostgreSQL pgbench 10.3Scaling: Buffer Test - Test: Normal Load - Mode: Read WriteBare MetalVM Guest5001000150020002500SE +/- 23.41, N = 9SE +/- 25.12, N = 9SE +/- 5.91, N = 31694.682248.821075.591. (CC) gcc options: -fno-strict-aliasing -fwrapv -O2 -lpgcommon -lpgport -lpq -lpthread -lrt -lcrypt -ldl -lm

MariaDB

Clients: 32

New MicrocodeStockOpenBenchmarking.orgQueries Per Second, More Is BetterMariaDB 10.3.8Clients: 32Bare MetalVM Guest50100150200250SE +/- 2.85, N = 3SE +/- 1.96, N = 3SE +/- 0.75, N = 32212461311. (CXX) g++ options: -pie -fPIC -fstack-protector -fno-rtti -O2 -lpthread -llzma -laio -lz -lm -lpcre -lcrypt -lssl -lcrypto -ldl

PostgreSQL pgbench

Scaling: Buffer Test - Test: Normal Load - Mode: Read Only

New MicrocodeStockOpenBenchmarking.orgTPS, More Is BetterPostgreSQL pgbench 10.3Scaling: Buffer Test - Test: Normal Load - Mode: Read OnlyBare MetalVM Guest130K260K390K520K650KSE +/- 823.94, N = 3SE +/- 2323.77, N = 3SE +/- 2865.20, N = 3588527.68585224.79384280.701. (CC) gcc options: -fno-strict-aliasing -fwrapv -O2 -lpgcommon -lpgport -lpq -lpthread -lrt -lcrypt -ldl -lm

Timed LLVM Compilation

Time To Compile

New MicrocodeStockOpenBenchmarking.orgSeconds, Fewer Is BetterTimed LLVM Compilation 6.0.1Time To CompileBare MetalVM Guest50100150200250171.02169.53208.78

Compile Bench

Test: Compile

New MicrocodeStockOpenBenchmarking.orgMB/s, More Is BetterCompile Bench 0.6Test: CompileBare MetalVM Guest400800120016002000SE +/- 2.32, N = 3SE +/- 8.67, N = 3SE +/- 11.24, N = 31698.501712.791467.65

Flexible IO Tester

Type: Random Read - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 2MB - Disk Target: Default Test Directory

New MicrocodeStockOpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.1Type: Random Read - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 2MB - Disk Target: Default Test DirectoryBare Metal110220330440550SE +/- 0.33, N = 34585311. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -U_FORTIFY_SOURCE -ll -lrt -laio -lz -lm -lpthread -ldl

Sysbench

Test: CPU

New MicrocodeStockOpenBenchmarking.orgEvents Per Second, More Is BetterSysbench 2018-07-28Test: CPUBare MetalVM Guest13K26K39K52K65KSE +/- 296.24, N = 3SE +/- 583.55, N = 3SE +/- 5.72, N = 359370.5558755.6551676.091. (CC) gcc options: -pthread -O3 -funroll-loops -ggdb3 -march=core2 -rdynamic -ldl -laio -lm

Blender

Blend File: Barbershop - Compute: CPU-Only

New MicrocodeStockOpenBenchmarking.orgSeconds, Fewer Is BetterBlender 2.79aBlend File: Barbershop - Compute: CPU-OnlyBare MetalVM Guest90180270360450373.98374.05423.31

TTSIOD 3D Renderer

Phong Rendering With Soft-Shadow Mapping

New MicrocodeStockOpenBenchmarking.orgFPS, More Is BetterTTSIOD 3D Renderer 2.3bPhong Rendering With Soft-Shadow MappingBare MetalVM Guest160320480640800SE +/- 6.67, N = 3SE +/- 10.35, N = 3SE +/- 6.63, N = 3731.60742.68673.741. (CXX) g++ options: -O3 -fomit-frame-pointer -ffast-math -mtune=native -flto -msse -mrecip -mfpmath=sse -msse2 -mssse3 -lSDL -fopenmp -fwhole-program -lstdc++

Sysbench

Test: Memory

New MicrocodeStockOpenBenchmarking.orgEvents Per Second, More Is BetterSysbench 2018-07-28Test: MemoryBare MetalVM Guest3M6M9M12M15MSE +/- 188626.26, N = 3SE +/- 132541.95, N = 12SE +/- 139878.10, N = 311803679.9911764465.1110796572.791. (CC) gcc options: -pthread -O3 -funroll-loops -ggdb3 -march=core2 -rdynamic -ldl -laio -lm

Compile Bench

Test: Initial Create

New MicrocodeStockOpenBenchmarking.orgMB/s, More Is BetterCompile Bench 0.6Test: Initial CreateBare MetalVM Guest110220330440550SE +/- 1.19, N = 3SE +/- 3.51, N = 3SE +/- 15.44, N = 3487.43501.79460.82

Flexible IO Tester

Type: Sequential Read - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 2MB - Disk Target: Default Test Directory

New MicrocodeStockOpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.1Type: Sequential Read - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 2MB - Disk Target: Default Test DirectoryBare Metal1102203304405504965311. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -U_FORTIFY_SOURCE -ll -lrt -laio -lz -lm -lpthread -ldl

Compile Bench

Test: Read Compiled Tree

New MicrocodeStockOpenBenchmarking.orgMB/s, More Is BetterCompile Bench 0.6Test: Read Compiled TreeBare MetalVM Guest5001000150020002500SE +/- 27.68, N = 3SE +/- 12.91, N = 3SE +/- 12.12, N = 32478.362469.932351.28

Tinymembench

Standard Memset

New MicrocodeStockOpenBenchmarking.orgMB/s, More Is BetterTinymembench 2018-05-28Standard MemsetBare MetalVM Guest2K4K6K8K10K9151.709298.208980.201. (CC) gcc options: -O2 -lm

x265

H.265 Video Encoding

New MicrocodeStockOpenBenchmarking.orgFrames Per Second, More Is Betterx265 2.8H.265 Video EncodingBare MetalVM Guest3691215SE +/- 0.01, N = 3SE +/- 0.06, N = 3SE +/- 0.05, N = 310.3210.3110.361. (CXX) g++ options: -O3 -rdynamic -lpthread -lrt -ldl

NGINX Benchmark

Static Web Page Serving

New MicrocodeStockOpenBenchmarking.orgRequests Per Second, More Is BetterNGINX Benchmark 1.9.9Static Web Page ServingBare MetalVM Guest6K12K18K24K30KSE +/- 114.41, N = 3SE +/- 140.64, N = 3SE +/- 1495.37, N = 323745.7424591.8627078.581. (CC) gcc options: -lpthread -lcrypt -lcrypto -lz -O3 -march=native

Stress-NG

Test: System V Message Passing

New MicrocodeStockOpenBenchmarking.orgBogo Ops/s, More Is BetterStress-NG 0.07.26Test: System V Message PassingBare MetalVM Guest2M4M6M8M10MSE +/- 457349.76, N = 9SE +/- 342053.14, N = 12SE +/- 87029.51, N = 37163233.317053890.1411422458.911. (CC) gcc options: -O2 -std=gnu99 -lm -lz -lcrypt -lrt -lpthread -laio -lc

Stress-NG

Test: Context Switching

New MicrocodeStockOpenBenchmarking.orgBogo Ops/s, More Is BetterStress-NG 0.07.26Test: Context SwitchingBare MetalVM Guest5M10M15M20M25MSE +/- 550387.69, N = 12SE +/- 858271.57, N = 12SE +/- 57576.83, N = 322976365.0822234776.3212704770.931. (CC) gcc options: -O2 -std=gnu99 -lm -lz -lcrypt -lrt -lpthread -laio -lc

Stress-NG

Test: Socket Activity

New MicrocodeStockOpenBenchmarking.orgBogo Ops/s, More Is BetterStress-NG 0.07.26Test: Socket ActivityBare MetalVM Guest3K6K9K12K15KSE +/- 943.03, N = 12SE +/- 853.24, N = 12SE +/- 151.62, N = 33313.123454.6314127.951. (CC) gcc options: -O2 -std=gnu99 -lm -lz -lcrypt -lrt -lpthread -laio -lc

Redis

Test: SET

New MicrocodeStockOpenBenchmarking.orgRequests Per Second, More Is BetterRedis 4.0.8Test: SETBare MetalVM Guest400K800K1200K1600K2000KSE +/- 10888.60, N = 3SE +/- 19253.77, N = 3SE +/- 51445.85, N = 121688379.921656081.461455207.361. (CC) gcc options: -ggdb -rdynamic -lm -ldl -pthread

Redis

Test: GET

New MicrocodeStockOpenBenchmarking.orgRequests Per Second, More Is BetterRedis 4.0.8Test: GETBare MetalVM Guest500K1000K1500K2000K2500KSE +/- 6166.42, N = 3SE +/- 42615.86, N = 12SE +/- 67289.89, N = 122059062.252142971.111999876.291. (CC) gcc options: -ggdb -rdynamic -lm -ldl -pthread

Hackbench

Count: 32 - Type: Process

New MicrocodeStockOpenBenchmarking.orgSeconds, Fewer Is BetterHackbenchCount: 32 - Type: ProcessBare MetalVM Guest1632486480SE +/- 0.69, N = 3SE +/- 0.99, N = 12SE +/- 0.25, N = 336.7335.7371.541. (CC) gcc options: -lpthread

Timed Linux Kernel Compilation

Time To Compile

New MicrocodeStockOpenBenchmarking.orgSeconds, Fewer Is BetterTimed Linux Kernel Compilation 4.18Time To CompileBare MetalVM Guest918273645SE +/- 0.50, N = 12SE +/- 1.08, N = 12SE +/- 0.41, N = 1233.4033.7037.21

FS-Mark

Test: 4000 Files, 32 Sub Dirs, 1MB Size

New MicrocodeStockOpenBenchmarking.orgFiles/s, More Is BetterFS-Mark 3.3Test: 4000 Files, 32 Sub Dirs, 1MB SizeBare MetalVM Guest1326395265SE +/- 1.10, N = 12SE +/- 1.70, N = 12SE +/- 0.57, N = 939.7059.1032.361. (CC) gcc options: -static

FS-Mark

Test: 5000 Files, 1MB Size, 4 Threads

New MicrocodeStockOpenBenchmarking.orgFiles/s, More Is BetterFS-Mark 3.3Test: 5000 Files, 1MB Size, 4 ThreadsBare MetalVM Guest50100150200250SE +/- 1.32, N = 9SE +/- 18.59, N = 9SE +/- 1.83, N = 640.89220.1839.871. (CC) gcc options: -static

FS-Mark

Test: 1000 Files, 1MB Size

New MicrocodeStockOpenBenchmarking.orgFiles/s, More Is BetterFS-Mark 3.3Test: 1000 Files, 1MB SizeBare MetalVM Guest306090120150SE +/- 0.75, N = 12SE +/- 1.17, N = 3SE +/- 0.57, N = 337.18145.4046.871. (CC) gcc options: -static

Flexible IO Tester

Type: Sequential Write - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 2MB - Disk Target: Default Test Directory

New MicrocodeStockOpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.1Type: Sequential Write - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 2MB - Disk Target: Default Test DirectoryBare MetalVM Guest110220330440550SE +/- 4.91, N = 11SE +/- 0.67, N = 3SE +/- 95.50, N = 9127.96513.00449.001. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -U_FORTIFY_SOURCE -ll -lrt -laio -lz -lm -lpthread -ldl

Flexible IO Tester

Type: Random Write - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 2MB - Disk Target: Default Test Directory

New MicrocodeStockOpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.1Type: Random Write - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 2MB - Disk Target: Default Test DirectoryBare MetalVM Guest30060090012001500SE +/- 3.57, N = 12SE +/- 1.53, N = 3SE +/- 102.19, N = 1071.58511.001171.001. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -U_FORTIFY_SOURCE -ll -lrt -laio -lz -lm -lpthread -ldl


Phoronix Test Suite v10.8.4