Linux 4.19: Bare Metal vs. KVM Guest

Tests for a future article.

HTML result view exported from: https://openbenchmarking.org/result/1808279-RA-ROUND243586.

ProcessorMotherboardChipsetMemoryDiskGraphicsMonitorNetworkAudioOSKernelDesktopDisplay ServerDisplay DriverCompilerFile-SystemScreen ResolutionSystem LayerStockNew Microcode Bare Metal VM Guest Bare Metal2 x Intel Xeon Gold 6138 @ 3.70GHz (40 Cores / 80 Threads)TYAN S7106 (V1.01 BIOS)Intel Sky Lake-E DMI3 Registers96256MB256GB Samsung SSD 850 + 2000GB Seagate ST2000DM006-2DM1 + 2 x 120GB TOSHIBA-TR150ASPEED ASPEED FamilyVE228Intel I210 Gigabit ConnectionUbuntu 18.044.18.0-999-generic (x86_64) 20180824GNOME Shell 3.28.2X Server 1.19.6modesetting 1.19.6GCC 7.3.0ext41920x108064 x Intel Xeon (Skylake IBRS) @ 2.00GHz (64 Cores)QEMU Standard PC (i440FX + PIIX 1996) (1.10.2-1ubuntu1 BIOS)Intel 440FX- 82441FX PMC47104MB162GBRed Hat QXL paravirtual graphic cardQEMU GenericRed Hat Virtio device1024x768KVM2 x Intel Xeon Gold 6138 @ 3.70GHz (40 Cores / 80 Threads)TYAN S7106 (V1.01 BIOS)Intel Sky Lake-E DMI3 Registers96256MB256GB Samsung SSD 850 + 2000GB Seagate ST2000DM006-2DM1 + 2 x 120GB TOSHIBA-TR150ASPEED ASPEED FamilyVE228Intel I210 Gigabit Connection1920x1080OpenBenchmarking.orgCompiler Details- --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++ --enable-libmpx --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-targets=nvptx-none --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-as=/usr/bin/x86_64-linux-gnu-as --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-ld=/usr/bin/x86_64-linux-gnu-ld --with-multilib-list=m32,m64,mx32 --with-target-system-zlib --with-tune=generic --without-cuda-driver -v Disk Details- Stock: Bare Metal: CFQ / errors=remount-ro,relatime,rw- Stock: VM Guest: NONE / errors=remount-ro,relatime,rw- New Microcode: Bare Metal: CFQ / errors=remount-ro,relatime,rwProcessor Details- Stock: Bare Metal, New Microcode: Bare Metal: Scaling Governor: intel_pstate powersavePython Details- Python 2.7.15rc1 + Python 3.6.5Security Details- Stock: Bare Metal: KPTI + __user pointer sanitization + Full generic retpoline IBPB IBRS_FW + PTE Inversion; VMX: conditional cache flushes SMT vulnerable Protection- Stock: VM Guest: KPTI + __user pointer sanitization + Full generic retpoline IBPB IBRS_FW + PTE Inversion Protection- New Microcode: Bare Metal: KPTI + __user pointer sanitization + Full generic retpoline IBPB IBRS_FW + SSB disabled via prctl and seccomp + PTE Inversion; VMX: conditional cache flushes SMT vulnerable Protection

fio: Rand Read - Linux AIO - No - Yes - 2MB - Default Test Directoryfio: Rand Write - Linux AIO - No - Yes - 2MB - Default Test Directoryfio: Seq Read - Linux AIO - No - Yes - 2MB - Default Test Directoryfio: Seq Write - Linux AIO - No - Yes - 2MB - Default Test Directoryfs-mark: 1000 Files, 1MB Sizefs-mark: 5000 Files, 1MB Size, 4 Threadsfs-mark: 4000 Files, 32 Sub Dirs, 1MB Sizecompilebench: Compilecompilebench: Initial Createcompilebench: Read Compiled Treettsiod-renderer: Phong Rendering With Soft-Shadow Mappingx265: H.265 Video Encodingbuild-linux-kernel: Time To Compilebuild-llvm: Time To Compileprimesieve: 1e12 Prime Number Generationhackbench: 32 - Processmysqlslap: 32pgbench: Buffer Test - Normal Load - Read Onlypgbench: Buffer Test - Normal Load - Read Writeredis: GETredis: SETstress-ng: Semaphoresstress-ng: Socket Activitystress-ng: Context Switchingstress-ng: System V Message Passingsysbench: Memorysysbench: CPUblender: Barbershop - CPU-Onlynginx: Static Web Page ServingStockNew Microcode Bare Metal VM Guest Bare Metal531511531513145.40220.1859.101712.79501.792469.93742.6810.3133.70169.538.6235.73246585224.792248.822142971.111656081.461831962.433454.6322234776.327053890.1411764465.1158755.65374.0524591.86117144946.8739.8732.361467.65460.822351.28673.7410.3637.21208.7824.6271.54131384280.701075.591999876.291455207.36600043.1314127.9512704770.9311422458.9110796572.7951676.09423.3127078.5845871.58496127.9637.1840.8939.701698.50487.432478.36731.6010.3233.40171.028.6436.73221588527.681694.682059062.251688379.921890399.793313.1222976365.087163233.3111803679.9959370.55373.9823745.74OpenBenchmarking.org

Flexible IO Tester

Type: Random Read - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 2MB - Disk Target: Default Test Directory

StockNew MicrocodeOpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.1Type: Random Read - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 2MB - Disk Target: Default Test DirectoryBare Metal110220330440550SE +/- 0.33, N = 35314581. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -U_FORTIFY_SOURCE -ll -lrt -laio -lz -lm -lpthread -ldl

Flexible IO Tester

Type: Random Write - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 2MB - Disk Target: Default Test Directory

StockNew MicrocodeOpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.1Type: Random Write - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 2MB - Disk Target: Default Test DirectoryBare MetalVM Guest30060090012001500SE +/- 1.53, N = 3SE +/- 102.19, N = 10SE +/- 3.57, N = 12511.001171.0071.581. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -U_FORTIFY_SOURCE -ll -lrt -laio -lz -lm -lpthread -ldl

Flexible IO Tester

Type: Sequential Read - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 2MB - Disk Target: Default Test Directory

StockNew MicrocodeOpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.1Type: Sequential Read - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 2MB - Disk Target: Default Test DirectoryBare Metal1102203304405505314961. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -U_FORTIFY_SOURCE -ll -lrt -laio -lz -lm -lpthread -ldl

Flexible IO Tester

Type: Sequential Write - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 2MB - Disk Target: Default Test Directory

StockNew MicrocodeOpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.1Type: Sequential Write - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 2MB - Disk Target: Default Test DirectoryBare MetalVM Guest110220330440550SE +/- 0.67, N = 3SE +/- 95.50, N = 9SE +/- 4.91, N = 11513.00449.00127.961. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -U_FORTIFY_SOURCE -ll -lrt -laio -lz -lm -lpthread -ldl

FS-Mark

Test: 1000 Files, 1MB Size

StockNew MicrocodeOpenBenchmarking.orgFiles/s, More Is BetterFS-Mark 3.3Test: 1000 Files, 1MB SizeBare MetalVM Guest306090120150SE +/- 1.17, N = 3SE +/- 0.57, N = 3SE +/- 0.75, N = 12145.4046.8737.181. (CC) gcc options: -static

FS-Mark

Test: 5000 Files, 1MB Size, 4 Threads

StockNew MicrocodeOpenBenchmarking.orgFiles/s, More Is BetterFS-Mark 3.3Test: 5000 Files, 1MB Size, 4 ThreadsBare MetalVM Guest50100150200250SE +/- 18.59, N = 9SE +/- 1.83, N = 6SE +/- 1.32, N = 9220.1839.8740.891. (CC) gcc options: -static

FS-Mark

Test: 4000 Files, 32 Sub Dirs, 1MB Size

StockNew MicrocodeOpenBenchmarking.orgFiles/s, More Is BetterFS-Mark 3.3Test: 4000 Files, 32 Sub Dirs, 1MB SizeBare MetalVM Guest1326395265SE +/- 1.70, N = 12SE +/- 0.57, N = 9SE +/- 1.10, N = 1259.1032.3639.701. (CC) gcc options: -static

Compile Bench

Test: Compile

StockNew MicrocodeOpenBenchmarking.orgMB/s, More Is BetterCompile Bench 0.6Test: CompileBare MetalVM Guest400800120016002000SE +/- 8.67, N = 3SE +/- 11.24, N = 3SE +/- 2.32, N = 31712.791467.651698.50

Compile Bench

Test: Initial Create

StockNew MicrocodeOpenBenchmarking.orgMB/s, More Is BetterCompile Bench 0.6Test: Initial CreateBare MetalVM Guest110220330440550SE +/- 3.51, N = 3SE +/- 15.44, N = 3SE +/- 1.19, N = 3501.79460.82487.43

Compile Bench

Test: Read Compiled Tree

StockNew MicrocodeOpenBenchmarking.orgMB/s, More Is BetterCompile Bench 0.6Test: Read Compiled TreeBare MetalVM Guest5001000150020002500SE +/- 12.91, N = 3SE +/- 12.12, N = 3SE +/- 27.68, N = 32469.932351.282478.36

Tinymembench

Standard Memset

StockNew MicrocodeOpenBenchmarking.orgMB/s, More Is BetterTinymembench 2018-05-28Standard MemsetBare MetalVM Guest2K4K6K8K10K9298.208980.209151.701. (CC) gcc options: -O2 -lm

TTSIOD 3D Renderer

Phong Rendering With Soft-Shadow Mapping

StockNew MicrocodeOpenBenchmarking.orgFPS, More Is BetterTTSIOD 3D Renderer 2.3bPhong Rendering With Soft-Shadow MappingBare MetalVM Guest160320480640800SE +/- 10.35, N = 3SE +/- 6.63, N = 3SE +/- 6.67, N = 3742.68673.74731.601. (CXX) g++ options: -O3 -fomit-frame-pointer -ffast-math -mtune=native -flto -msse -mrecip -mfpmath=sse -msse2 -mssse3 -lSDL -fopenmp -fwhole-program -lstdc++

x265

H.265 Video Encoding

StockNew MicrocodeOpenBenchmarking.orgFrames Per Second, More Is Betterx265 2.8H.265 Video EncodingBare MetalVM Guest3691215SE +/- 0.06, N = 3SE +/- 0.05, N = 3SE +/- 0.01, N = 310.3110.3610.321. (CXX) g++ options: -O3 -rdynamic -lpthread -lrt -ldl

Timed Linux Kernel Compilation

Time To Compile

StockNew MicrocodeOpenBenchmarking.orgSeconds, Fewer Is BetterTimed Linux Kernel Compilation 4.18Time To CompileBare MetalVM Guest918273645SE +/- 1.08, N = 12SE +/- 0.41, N = 12SE +/- 0.50, N = 1233.7037.2133.40

Timed LLVM Compilation

Time To Compile

StockNew MicrocodeOpenBenchmarking.orgSeconds, Fewer Is BetterTimed LLVM Compilation 6.0.1Time To CompileBare MetalVM Guest50100150200250169.53208.78171.02

Primesieve

1e12 Prime Number Generation

StockNew MicrocodeOpenBenchmarking.orgSeconds, Fewer Is BetterPrimesieve 7.11e12 Prime Number GenerationBare MetalVM Guest612182430SE +/- 0.03, N = 3SE +/- 0.15, N = 3SE +/- 0.02, N = 38.6224.628.641. (CXX) g++ options: -O3 -lpthread

Hackbench

Count: 32 - Type: Process

StockNew MicrocodeOpenBenchmarking.orgSeconds, Fewer Is BetterHackbenchCount: 32 - Type: ProcessBare MetalVM Guest1632486480SE +/- 0.99, N = 12SE +/- 0.25, N = 3SE +/- 0.69, N = 335.7371.5436.731. (CC) gcc options: -lpthread

MariaDB

Clients: 32

StockNew MicrocodeOpenBenchmarking.orgQueries Per Second, More Is BetterMariaDB 10.3.8Clients: 32Bare MetalVM Guest50100150200250SE +/- 1.96, N = 3SE +/- 0.75, N = 3SE +/- 2.85, N = 32461312211. (CXX) g++ options: -pie -fPIC -fstack-protector -fno-rtti -O2 -lpthread -llzma -laio -lz -lm -lpcre -lcrypt -lssl -lcrypto -ldl

PostgreSQL pgbench

Scaling: Buffer Test - Test: Normal Load - Mode: Read Only

StockNew MicrocodeOpenBenchmarking.orgTPS, More Is BetterPostgreSQL pgbench 10.3Scaling: Buffer Test - Test: Normal Load - Mode: Read OnlyBare MetalVM Guest130K260K390K520K650KSE +/- 2323.77, N = 3SE +/- 2865.20, N = 3SE +/- 823.94, N = 3585224.79384280.70588527.681. (CC) gcc options: -fno-strict-aliasing -fwrapv -O2 -lpgcommon -lpgport -lpq -lpthread -lrt -lcrypt -ldl -lm

PostgreSQL pgbench

Scaling: Buffer Test - Test: Normal Load - Mode: Read Write

StockNew MicrocodeOpenBenchmarking.orgTPS, More Is BetterPostgreSQL pgbench 10.3Scaling: Buffer Test - Test: Normal Load - Mode: Read WriteBare MetalVM Guest5001000150020002500SE +/- 25.12, N = 9SE +/- 5.91, N = 3SE +/- 23.41, N = 92248.821075.591694.681. (CC) gcc options: -fno-strict-aliasing -fwrapv -O2 -lpgcommon -lpgport -lpq -lpthread -lrt -lcrypt -ldl -lm

Redis

Test: GET

StockNew MicrocodeOpenBenchmarking.orgRequests Per Second, More Is BetterRedis 4.0.8Test: GETBare MetalVM Guest500K1000K1500K2000K2500KSE +/- 42615.86, N = 12SE +/- 67289.89, N = 12SE +/- 6166.42, N = 32142971.111999876.292059062.251. (CC) gcc options: -ggdb -rdynamic -lm -ldl -pthread

Redis

Test: SET

StockNew MicrocodeOpenBenchmarking.orgRequests Per Second, More Is BetterRedis 4.0.8Test: SETBare MetalVM Guest400K800K1200K1600K2000KSE +/- 19253.77, N = 3SE +/- 51445.85, N = 12SE +/- 10888.60, N = 31656081.461455207.361688379.921. (CC) gcc options: -ggdb -rdynamic -lm -ldl -pthread

Stress-NG

Test: Semaphores

StockNew MicrocodeOpenBenchmarking.orgBogo Ops/s, More Is BetterStress-NG 0.07.26Test: SemaphoresBare MetalVM Guest400K800K1200K1600K2000KSE +/- 29092.76, N = 3SE +/- 925.56, N = 3SE +/- 26014.39, N = 61831962.43600043.131890399.791. (CC) gcc options: -O2 -std=gnu99 -lm -lz -lcrypt -lrt -lpthread -laio -lc

Stress-NG

Test: Socket Activity

StockNew MicrocodeOpenBenchmarking.orgBogo Ops/s, More Is BetterStress-NG 0.07.26Test: Socket ActivityBare MetalVM Guest3K6K9K12K15KSE +/- 853.24, N = 12SE +/- 151.62, N = 3SE +/- 943.03, N = 123454.6314127.953313.121. (CC) gcc options: -O2 -std=gnu99 -lm -lz -lcrypt -lrt -lpthread -laio -lc

Stress-NG

Test: Context Switching

StockNew MicrocodeOpenBenchmarking.orgBogo Ops/s, More Is BetterStress-NG 0.07.26Test: Context SwitchingBare MetalVM Guest5M10M15M20M25MSE +/- 858271.57, N = 12SE +/- 57576.83, N = 3SE +/- 550387.69, N = 1222234776.3212704770.9322976365.081. (CC) gcc options: -O2 -std=gnu99 -lm -lz -lcrypt -lrt -lpthread -laio -lc

Stress-NG

Test: System V Message Passing

StockNew MicrocodeOpenBenchmarking.orgBogo Ops/s, More Is BetterStress-NG 0.07.26Test: System V Message PassingBare MetalVM Guest2M4M6M8M10MSE +/- 342053.14, N = 12SE +/- 87029.51, N = 3SE +/- 457349.76, N = 97053890.1411422458.917163233.311. (CC) gcc options: -O2 -std=gnu99 -lm -lz -lcrypt -lrt -lpthread -laio -lc

Sysbench

Test: Memory

StockNew MicrocodeOpenBenchmarking.orgEvents Per Second, More Is BetterSysbench 2018-07-28Test: MemoryBare MetalVM Guest3M6M9M12M15MSE +/- 132541.95, N = 12SE +/- 139878.10, N = 3SE +/- 188626.26, N = 311764465.1110796572.7911803679.991. (CC) gcc options: -pthread -O3 -funroll-loops -ggdb3 -march=core2 -rdynamic -ldl -laio -lm

Sysbench

Test: CPU

StockNew MicrocodeOpenBenchmarking.orgEvents Per Second, More Is BetterSysbench 2018-07-28Test: CPUBare MetalVM Guest13K26K39K52K65KSE +/- 583.55, N = 3SE +/- 5.72, N = 3SE +/- 296.24, N = 358755.6551676.0959370.551. (CC) gcc options: -pthread -O3 -funroll-loops -ggdb3 -march=core2 -rdynamic -ldl -laio -lm

Blender

Blend File: Barbershop - Compute: CPU-Only

StockNew MicrocodeOpenBenchmarking.orgSeconds, Fewer Is BetterBlender 2.79aBlend File: Barbershop - Compute: CPU-OnlyBare MetalVM Guest90180270360450374.05423.31373.98

NGINX Benchmark

Static Web Page Serving

StockNew MicrocodeOpenBenchmarking.orgRequests Per Second, More Is BetterNGINX Benchmark 1.9.9Static Web Page ServingBare MetalVM Guest6K12K18K24K30KSE +/- 140.64, N = 3SE +/- 1495.37, N = 3SE +/- 114.41, N = 324591.8627078.5823745.741. (CC) gcc options: -lpthread -lcrypt -lcrypto -lz -O3 -march=native

Phoronix Test Suite v10.8.4