FreeBSD ZFS Hypervisor

AMD Ryzen 5 3600 6-Core testing on FreeBSD via the Phoronix Test Suite.

HTML result view exported from: https://openbenchmarking.org/result/2110053-IB-FREEBSDZF95&sor.

FreeBSD ZFS HypervisorProcessorChipsetMemoryDiskGraphicsAudioMotherboardOSKernelCompilerFile-SystemScreen ResolutionPhysicalHyper-V Server 2022 gen2 ZFS #1Hyper-V Server 2019 gen2 UFSHyper-V Server 2019 gen2 ZFSHyper-V Server 2022 gen2 UFSHyper-V Server 2022 gen2 ZFS #2Proxmox 7 ZFSXCP-ng 8.2 ZFSAMD Ryzen 5 3600 6-Core @ 3.60GHz (12 Cores)AMD [AMD] Starship/Matisse Root Complex64GBSamsung SSD 980 1TBASPEEDAMD X570 HDAFreeBSD13.0-RELEASE (x86_64)GCC 10.3.0 + Clang 11.0.1zfs800x600AMD Ryzen 5 3600 6-Core (4 Cores)Microsoft Virtual Machine Hyper-V UEFI v4.1 (Hyper-V BIOS)4096MBMsft Virtual Disk 1.013.0-RELEASE-p4 (x86_64)1024x768Microsoft Virtual Machine Hyper-V UEFI v4.0 (Hyper-V BIOS)ufs13.0-RELEASE (x86_64)zfsMicrosoft Virtual Machine Hyper-V UEFI v4.1 (Hyper-V BIOS)ufs13.0-RELEASE-p4 (x86_64)zfsCommon KVM (4 Cores)QEMU Standard PC (i440FX + PIIX 1996) pc-i440fx-6.0Intel 440FX 82441FX PMC [Natoma]47GBIntel 82371AB/EB/MB PIIX4 ACPI13.0-RELEASE (x86_64)AMD Ryzen 5 3600 6-Core (4 Cores)Xen HVM domU 4.13Cirrus Logic GD 544613.0-RELEASE-p4 (x86_64)OpenBenchmarking.orgCompiler Details- --build=x86_64-portbld-freebsd13.0 --disable-nls --enable-gnu-indirect-function --enable-languages=c,c++,objc,fortran --enable-multilib --enable-plugin --localstatedir=/var --mandir=/usr/local/man --with-as=/usr/local/bin/as --with-build-config=bootstrap-debug --with-gmp=/usr/local --with-ld=/usr/local/bin/ld --without-zstd

FreeBSD ZFS Hypervisorblogbench: Readblogbench: Writet-test1: 1t-test1: 2iperf: 192.168.1.27 - 5201 - 360 Seconds - TCP - 20john-the-ripper: Blowfishjohn-the-ripper: MD5compress-7zip: Compress Speed Testopenssl: SHA256openssl: RSA4096openssl: RSA4096pgbench: 1 - 100 - Read Onlypgbench: 1 - 100 - Read Only - Average Latencypgbench: 1 - 100 - Read Writepgbench: 1 - 100 - Read Write - Average Latencysqlite-speedtest: Timed Time - Size 1,000nginx: 100apache: 100phpbench: PHP Benchmark Suitepgbench: 1 - 100 - Read Onlypgbench: 1 - 100 - Read Only - Average Latencypgbench: 1 - 100 - Read Writepgbench: 1 - 100 - Read Write - Average LatencyPhysicalHyper-V Server 2022 gen2 ZFS #1Hyper-V Server 2019 gen2 UFSHyper-V Server 2019 gen2 ZFSHyper-V Server 2022 gen2 UFSHyper-V Server 2022 gen2 ZFS #2Proxmox 7 ZFSXCP-ng 8.2 ZFS1927724131028.2502.69188588673105264339191849355101748.8113146.42046490.489160862.18088.008224.70104396.27457046878994149816.0195.705941183812061088281696724313318.520825.1658.185203.7726738.70238950441922.263436249.64575448767116.4475.621939182211977485241733037107323.621469.3438932.279167259.832125.224203.9230532.74239316939801119916.3875.722940181212068684581818648249326.320817.2423882.360767130.363567.076203.7526539.34259131636646305016.7745.966940181811998083591695300020318.820691.5125.612204.2231019.81263094418122.392177756.287113245612679.1613.2959416510175935203331074437310780.851093.4670171.492635169.748607.512206.4622418.653213491355701539816.1484.64294161953628311954836943991001095.472872.1839881.191206748.805401.757205.5149812.86509500OpenBenchmarking.org

BlogBench

Test: Read

OpenBenchmarking.orgFinal Score, More Is BetterBlogBench 1.1Test: ReadPhysicalXCP-ng 8.2 ZFSProxmox 7 ZFSHyper-V Server 2019 gen2 ZFSHyper-V Server 2022 gen2 ZFS #1Hyper-V Server 2019 gen2 UFSHyper-V Server 2022 gen2 UFS400K800K1200K1600K2000KSE +/- 20896.13, N = 5SE +/- 69015.49, N = 9SE +/- 119452.26, N = 9SE +/- 51240.74, N = 9SE +/- 68010.49, N = 6SE +/- 37468.76, N = 12SE +/- 62232.41, N = 61927724135570111324569398018789947544876366461. (CC) gcc options: -O2 -pthread

BlogBench

Test: Write

OpenBenchmarking.orgFinal Score, More Is BetterBlogBench 1.1Test: WritePhysicalXCP-ng 8.2 ZFSHyper-V Server 2022 gen2 UFSHyper-V Server 2022 gen2 ZFS #1Proxmox 7 ZFSHyper-V Server 2019 gen2 ZFSHyper-V Server 2019 gen2 UFS3K6K9K12K15KSE +/- 2643.32, N = 3SE +/- 211.50, N = 2SE +/- 1135.65, N = 3SE +/- 383.67, N = 3SE +/- 495.20, N = 313102539830501498126711996711. (CC) gcc options: -O2 -pthread

t-test1

Threads: 1

OpenBenchmarking.orgSeconds, Fewer Is Bettert-test1 2017-01-13Threads: 1PhysicalProxmox 7 ZFSHyper-V Server 2022 gen2 ZFS #1XCP-ng 8.2 ZFSHyper-V Server 2019 gen2 ZFSHyper-V Server 2019 gen2 UFSHyper-V Server 2022 gen2 UFS48121620SE +/- 0.030, N = 3SE +/- 0.037, N = 3SE +/- 0.029, N = 3SE +/- 0.053, N = 3SE +/- 0.069, N = 3SE +/- 0.033, N = 3SE +/- 0.121, N = 38.2509.16116.01916.14816.38716.44716.7741. (CC) gcc options: -pthread

t-test1

Threads: 2

OpenBenchmarking.orgSeconds, Fewer Is Bettert-test1 2017-01-13Threads: 2PhysicalProxmox 7 ZFSXCP-ng 8.2 ZFSHyper-V Server 2019 gen2 UFSHyper-V Server 2022 gen2 ZFS #1Hyper-V Server 2019 gen2 ZFSHyper-V Server 2022 gen2 UFS1.34242.68484.02725.36966.712SE +/- 0.025, N = 7SE +/- 0.006, N = 3SE +/- 0.067, N = 15SE +/- 0.058, N = 15SE +/- 0.063, N = 3SE +/- 0.071, N = 15SE +/- 0.020, N = 32.6913.2954.6425.6215.7055.7225.9661. (CC) gcc options: -pthread

iPerf

Server Address: 192.168.1.27 - Server Port: 5201 - Duration: 360 Seconds - Test: TCP - Parallel: 20

OpenBenchmarking.orgMbits/sec, More Is BetteriPerf 3.7Server Address: 192.168.1.27 - Server Port: 5201 - Duration: 360 Seconds - Test: TCP - Parallel: 20XCP-ng 8.2 ZFSProxmox 7 ZFSHyper-V Server 2022 gen2 ZFS #1Hyper-V Server 2022 gen2 UFSHyper-V Server 2019 gen2 ZFSHyper-V Server 2019 gen2 UFSPhysical2004006008001000SE +/- 0.33, N = 3SE +/- 36.26, N = 99419419419409409398851. (CC) gcc options: -O3 -march=native -lssl -lcrypto -lm

John The Ripper

Test: Blowfish

OpenBenchmarking.orgReal C/S, More Is BetterJohn The Ripper 1.9.0-jumbo-1Test: BlowfishPhysicalProxmox 7 ZFSXCP-ng 8.2 ZFSHyper-V Server 2022 gen2 ZFS #1Hyper-V Server 2019 gen2 UFSHyper-V Server 2022 gen2 UFSHyper-V Server 2019 gen2 ZFS2K4K6K8K10KSE +/- 82.40, N = 15SE +/- 7.22, N = 3SE +/- 45.34, N = 3SE +/- 3.33, N = 3SE +/- 4.36, N = 3SE +/- 13.86, N = 3SE +/- 2.19, N = 38867651061951838182218181812-lssl -lcrypto -lgmp -pthread -lm -lz -ldl -lcrypt -lbz2-lssl -lcrypto -lgmp -pthread -lm -lz -ldl -lcrypt -lbz2-lssl -lcrypto -lgmp -pthread -lm -lz -ldl -lcrypt -lbz2-lssl -lcrypto -lgmp -pthread -lm -lz -ldl -lcrypt -lbz2-lpcap-lssl -lcrypto -lgmp -pthread -lm -lz -ldl -lcrypt -lbz2-lpcap1. (CC) gcc options: -m64 -fopenmp

John The Ripper

Test: MD5

OpenBenchmarking.orgReal C/S, More Is BetterJohn The Ripper 1.9.0-jumbo-1Test: MD5XCP-ng 8.2 ZFSPhysicalProxmox 7 ZFSHyper-V Server 2019 gen2 ZFSHyper-V Server 2022 gen2 ZFS #1Hyper-V Server 2022 gen2 UFSHyper-V Server 2019 gen2 UFS80K160K240K320K400KSE +/- 348.38, N = 3SE +/- 3778.81, N = 3SE +/- 863.36, N = 3SE +/- 394.82, N = 3SE +/- 579.82, N = 3SE +/- 628.29, N = 3SE +/- 313.45, N = 3362831310526175935120686120610119980119774-lssl -lcrypto -lgmp -pthread -lm -lz -ldl -lcrypt -lbz2-lssl -lcrypto -lgmp -pthread -lm -lz -ldl -lcrypt -lbz2-lssl -lcrypto -lgmp -pthread -lm -lz -ldl -lcrypt -lbz2-lpcap-lssl -lcrypto -lgmp -pthread -lm -lz -ldl -lcrypt -lbz2-lssl -lcrypto -lgmp -pthread -lm -lz -ldl -lcrypt -lbz2-lpcap1. (CC) gcc options: -m64 -fopenmp

7-Zip Compression

Compress Speed Test

OpenBenchmarking.orgMIPS, More Is Better7-Zip Compression 16.02Compress Speed TestPhysicalProxmox 7 ZFSXCP-ng 8.2 ZFSHyper-V Server 2022 gen2 ZFS #1Hyper-V Server 2019 gen2 UFSHyper-V Server 2019 gen2 ZFSHyper-V Server 2022 gen2 UFS9K18K27K36K45KSE +/- 413.92, N = 6SE +/- 4.91, N = 3SE +/- 139.45, N = 3SE +/- 9.61, N = 3SE +/- 54.58, N = 3SE +/- 45.92, N = 3SE +/- 55.19, N = 343391203331954888288524845883591. (CXX) g++ options: -pipe -lpthread

OpenSSL

Algorithm: SHA256

OpenBenchmarking.orgbyte/s, More Is BetterOpenSSL 3.0Algorithm: SHA256PhysicalXCP-ng 8.2 ZFSHyper-V Server 2019 gen2 ZFSHyper-V Server 2019 gen2 UFSHyper-V Server 2022 gen2 ZFS #1Hyper-V Server 2022 gen2 UFSProxmox 7 ZFS2000M4000M6000M8000M10000MSE +/- 4517236.92, N = 3SE +/- 4064000.36, N = 3SE +/- 30814745.08, N = 12SE +/- 14460466.96, N = 3SE +/- 7574201.44, N = 3SE +/- 5593250.35, N = 3SE +/- 664193.75, N = 391849355103694399100181864824917330371071696724313169530002010744373101. (CC) gcc options: -pthread -O3 -lssl -lcrypto

OpenSSL

Algorithm: RSA4096

OpenBenchmarking.orgsign/s, More Is BetterOpenSSL 3.0Algorithm: RSA4096PhysicalXCP-ng 8.2 ZFSProxmox 7 ZFSHyper-V Server 2019 gen2 ZFSHyper-V Server 2019 gen2 UFSHyper-V Server 2022 gen2 UFSHyper-V Server 2022 gen2 ZFS #1400800120016002000SE +/- 0.03, N = 3SE +/- 11.14, N = 13SE +/- 0.74, N = 3SE +/- 2.80, N = 3SE +/- 1.89, N = 3SE +/- 1.59, N = 3SE +/- 1.22, N = 31748.81095.4780.8326.3323.6318.8318.51. (CC) gcc options: -pthread -O3 -lssl -lcrypto

OpenSSL

Algorithm: RSA4096

OpenBenchmarking.orgverify/s, More Is BetterOpenSSL 3.0Algorithm: RSA4096PhysicalXCP-ng 8.2 ZFSProxmox 7 ZFSHyper-V Server 2019 gen2 UFSHyper-V Server 2022 gen2 ZFS #1Hyper-V Server 2019 gen2 ZFSHyper-V Server 2022 gen2 UFS20K40K60K80K100KSE +/- 40.47, N = 3SE +/- 728.16, N = 13SE +/- 104.96, N = 3SE +/- 695.73, N = 3SE +/- 37.37, N = 3SE +/- 84.84, N = 3SE +/- 74.87, N = 3113146.472872.151093.421469.320825.120817.220691.51. (CC) gcc options: -pthread -O3 -lssl -lcrypto

PostgreSQL pgbench

Scaling Factor: 1 - Clients: 100 - Mode: Read Only

OpenBenchmarking.orgTPS, More Is BetterPostgreSQL pgbench 13.0Scaling Factor: 1 - Clients: 100 - Mode: Read OnlyPhysicalXCP-ng 8.2 ZFSProxmox 7 ZFSHyper-V Server 2019 gen2 UFSHyper-V Server 2019 gen2 ZFS40K80K120K160K200KSE +/- 1738.86, N = 3SE +/- 24.29, N = 3SE +/- 120.61, N = 3SE +/- 475.16, N = 3SE +/- 144.69, N = 320464983988670174389342388-lpq-lpq-lpq1. (CC) gcc options: -fno-strict-aliasing -fwrapv -O2 -lpgcommon -lpgport -lexecinfo -lm

PostgreSQL pgbench

Scaling Factor: 1 - Clients: 100 - Mode: Read Only - Average Latency

OpenBenchmarking.orgms, Fewer Is BetterPostgreSQL pgbench 13.0Scaling Factor: 1 - Clients: 100 - Mode: Read Only - Average LatencyPhysicalXCP-ng 8.2 ZFSProxmox 7 ZFSHyper-V Server 2019 gen2 UFSHyper-V Server 2019 gen2 ZFS0.5311.0621.5932.1242.655SE +/- 0.004, N = 3SE +/- 0.000, N = 3SE +/- 0.003, N = 3SE +/- 0.025, N = 3SE +/- 0.008, N = 30.4891.1911.4922.2792.360-lpq-lpq-lpq1. (CC) gcc options: -fno-strict-aliasing -fwrapv -O2 -lpgcommon -lpgport -lexecinfo -lm

PostgreSQL pgbench

Scaling Factor: 1 - Clients: 100 - Mode: Read Write

OpenBenchmarking.orgTPS, More Is BetterPostgreSQL pgbench 13.0Scaling Factor: 1 - Clients: 100 - Mode: Read WriteXCP-ng 8.2 ZFSHyper-V Server 2019 gen2 UFSPhysicalHyper-V Server 2019 gen2 ZFSProxmox 7 ZFS400800120016002000SE +/- 50.23, N = 15SE +/- 11.36, N = 3SE +/- 3.35, N = 3SE +/- 9.40, N = 3SE +/- 48.39, N = 15206716721608767635-lpq-lpq-lpq1. (CC) gcc options: -fno-strict-aliasing -fwrapv -O2 -lpgcommon -lpgport -lexecinfo -lm

PostgreSQL pgbench

Scaling Factor: 1 - Clients: 100 - Mode: Read Write - Average Latency

OpenBenchmarking.orgms, Fewer Is BetterPostgreSQL pgbench 13.0Scaling Factor: 1 - Clients: 100 - Mode: Read Write - Average LatencyXCP-ng 8.2 ZFSHyper-V Server 2019 gen2 UFSPhysicalHyper-V Server 2019 gen2 ZFSProxmox 7 ZFS4080120160200SE +/- 1.27, N = 15SE +/- 0.41, N = 3SE +/- 0.13, N = 3SE +/- 1.59, N = 3SE +/- 12.35, N = 1548.8159.8362.18130.36169.75-lpq-lpq-lpq1. (CC) gcc options: -fno-strict-aliasing -fwrapv -O2 -lpgcommon -lpgport -lexecinfo -lm

SQLite Speedtest

Timed Time - Size 1,000

OpenBenchmarking.orgSeconds, Fewer Is BetterSQLite Speedtest 3.30Timed Time - Size 1,000PhysicalHyper-V Server 2019 gen2 UFSHyper-V Server 2022 gen2 UFSXCP-ng 8.2 ZFSHyper-V Server 2019 gen2 ZFSProxmox 7 ZFSHyper-V Server 2022 gen2 ZFS #1140280420560700SE +/- 0.04, N = 3SE +/- 0.45, N = 3SE +/- 0.17, N = 3SE +/- 5.15, N = 3SE +/- 1.48, N = 3SE +/- 1.39, N = 3SE +/- 0.40, N = 388.01125.22125.61401.76567.08607.51658.191. (CC) gcc options: -O2 -lz -lpthread

nginx

Concurrent Requests: 100

OpenBenchmarking.orgRequests Per Second, More Is Betternginx 1.21.1Concurrent Requests: 100PhysicalProxmox 7 ZFSXCP-ng 8.2 ZFSHyper-V Server 2022 gen2 UFSHyper-V Server 2019 gen2 UFSHyper-V Server 2022 gen2 ZFS #1Hyper-V Server 2019 gen2 ZFS50100150200250SE +/- 6.73, N = 12SE +/- 1.11, N = 3SE +/- 0.85, N = 3SE +/- 0.63, N = 3SE +/- 1.15, N = 3SE +/- 0.99, N = 3SE +/- 0.96, N = 3224.70206.46205.51204.22203.92203.77203.751. (CC) gcc options: -lcrypt -lz -O3 -march=native

Apache HTTP Server

Concurrent Requests: 100

OpenBenchmarking.orgRequests Per Second, More Is BetterApache HTTP Server 2.4.48Concurrent Requests: 100PhysicalXCP-ng 8.2 ZFSHyper-V Server 2022 gen2 UFSHyper-V Server 2019 gen2 UFSHyper-V Server 2022 gen2 ZFS #1Hyper-V Server 2019 gen2 ZFSProxmox 7 ZFS20K40K60K80K100KSE +/- 357.41, N = 3SE +/- 615.93, N = 4SE +/- 345.94, N = 4SE +/- 367.83, N = 4SE +/- 284.16, N = 5SE +/- 363.45, N = 3SE +/- 46.86, N = 3104396.2749812.8631019.8130532.7426738.7026539.3422418.651. (CC) gcc options: -shared -fPIC -O2

PHPBench

PHP Benchmark Suite

OpenBenchmarking.orgScore, More Is BetterPHPBench 0.8.1PHP Benchmark SuiteXCP-ng 8.2 ZFSPhysicalProxmox 7 ZFSHyper-V Server 2022 gen2 UFSHyper-V Server 2019 gen2 ZFSHyper-V Server 2019 gen2 UFSHyper-V Server 2022 gen2 ZFS #1110K220K330K440K550KSE +/- 319.82, N = 3SE +/- 360.51, N = 3SE +/- 44.94, N = 3SE +/- 1108.50, N = 3SE +/- 814.65, N = 3SE +/- 668.88, N = 3SE +/- 953.42, N = 3509500457046321349263094259131239316238950

PostgreSQL pgbench

Scaling Factor: 1 - Clients: 100 - Mode: Read Only

OpenBenchmarking.orgTPS, More Is BetterPostgreSQL pgbench 14.0Scaling Factor: 1 - Clients: 100 - Mode: Read OnlyHyper-V Server 2022 gen2 ZFS #1Hyper-V Server 2022 gen2 UFS9K18K27K36K45KSE +/- 141.62, N = 3SE +/- 125.66, N = 344192418121. (CC) gcc options: -fno-strict-aliasing -fwrapv -O2 -lpgcommon -lpgport -lpq -lpthread -lexecinfo -lm

PostgreSQL pgbench

Scaling Factor: 1 - Clients: 100 - Mode: Read Only - Average Latency

OpenBenchmarking.orgms, Fewer Is BetterPostgreSQL pgbench 14.0Scaling Factor: 1 - Clients: 100 - Mode: Read Only - Average LatencyHyper-V Server 2022 gen2 ZFS #1Hyper-V Server 2022 gen2 UFS0.53821.07641.61462.15282.691SE +/- 0.007, N = 3SE +/- 0.007, N = 32.2632.3921. (CC) gcc options: -fno-strict-aliasing -fwrapv -O2 -lpgcommon -lpgport -lpq -lpthread -lexecinfo -lm

PostgreSQL pgbench

Scaling Factor: 1 - Clients: 100 - Mode: Read Write

OpenBenchmarking.orgTPS, More Is BetterPostgreSQL pgbench 14.0Scaling Factor: 1 - Clients: 100 - Mode: Read WriteHyper-V Server 2022 gen2 UFSHyper-V Server 2022 gen2 ZFS #1400800120016002000SE +/- 19.81, N = 3SE +/- 48.63, N = 917774361. (CC) gcc options: -fno-strict-aliasing -fwrapv -O2 -lpgcommon -lpgport -lpq -lpthread -lexecinfo -lm

PostgreSQL pgbench

Scaling Factor: 1 - Clients: 100 - Mode: Read Write - Average Latency

OpenBenchmarking.orgms, Fewer Is BetterPostgreSQL pgbench 14.0Scaling Factor: 1 - Clients: 100 - Mode: Read Write - Average LatencyHyper-V Server 2022 gen2 UFSHyper-V Server 2022 gen2 ZFS #150100150200250SE +/- 0.62, N = 3SE +/- 22.82, N = 956.29249.651. (CC) gcc options: -fno-strict-aliasing -fwrapv -O2 -lpgcommon -lpgport -lpq -lpthread -lexecinfo -lm

Phoronix Test Suite v10.8.4