FreeBSD ZFS Hypervisor

AMD Ryzen 5 3600 6-Core testing on FreeBSD via the Phoronix Test Suite.

HTML result view exported from: https://openbenchmarking.org/result/2110053-IB-FREEBSDZF95&gru&rdt.

FreeBSD ZFS HypervisorProcessorChipsetMemoryDiskGraphicsAudioMotherboardOSKernelCompilerFile-SystemScreen ResolutionPhysicalXCP-ng 8.2 ZFSHyper-V Server 2019 gen2 UFSHyper-V Server 2019 gen2 ZFSProxmox 7 ZFSHyper-V Server 2022 gen2 UFSHyper-V Server 2022 gen2 ZFS #1Hyper-V Server 2022 gen2 ZFS #2AMD Ryzen 5 3600 6-Core @ 3.60GHz (12 Cores)AMD [AMD] Starship/Matisse Root Complex64GBSamsung SSD 980 1TBASPEEDAMD X570 HDAFreeBSD13.0-RELEASE (x86_64)GCC 10.3.0 + Clang 11.0.1zfs800x600AMD Ryzen 5 3600 6-Core (4 Cores)Xen HVM domU 4.13Intel 440FX 82441FX PMC [Natoma]4096MB47GBCirrus Logic GD 544613.0-RELEASE-p4 (x86_64)Microsoft Virtual Machine Hyper-V UEFI v4.0 (Hyper-V BIOS)Msft Virtual Disk 1.0ufs1024x76813.0-RELEASE (x86_64)zfsCommon KVM (4 Cores)QEMU Standard PC (i440FX + PIIX 1996) pc-i440fx-6.0Intel 440FX 82441FX PMC [Natoma]47GBIntel 82371AB/EB/MB PIIX4 ACPIAMD Ryzen 5 3600 6-Core (4 Cores)Microsoft Virtual Machine Hyper-V UEFI v4.1 (Hyper-V BIOS)Msft Virtual Disk 1.0ufs1024x76813.0-RELEASE-p4 (x86_64)zfsOpenBenchmarking.orgCompiler Details- --build=x86_64-portbld-freebsd13.0 --disable-nls --enable-gnu-indirect-function --enable-languages=c,c++,objc,fortran --enable-multilib --enable-plugin --localstatedir=/var --mandir=/usr/local/man --with-as=/usr/local/bin/as --with-build-config=bootstrap-debug --with-gmp=/usr/local --with-ld=/usr/local/bin/ld --without-zstd

FreeBSD ZFS Hypervisoropenssl: SHA256blogbench: Readblogbench: Writeiperf: 192.168.1.27 - 5201 - 360 Seconds - TCP - 20compress-7zip: Compress Speed Testjohn-the-ripper: Blowfishjohn-the-ripper: MD5nginx: 100apache: 100phpbench: PHP Benchmark Suiteopenssl: RSA4096pgbench: 1 - 100 - Read Onlypgbench: 1 - 100 - Read Writepgbench: 1 - 100 - Read Onlypgbench: 1 - 100 - Read Writeopenssl: RSA4096pgbench: 1 - 100 - Read Only - Average Latencypgbench: 1 - 100 - Read Write - Average Latencypgbench: 1 - 100 - Read Only - Average Latencypgbench: 1 - 100 - Read Write - Average Latencyt-test1: 1t-test1: 2sqlite-speedtest: Timed Time - Size 1,000PhysicalXCP-ng 8.2 ZFSHyper-V Server 2019 gen2 UFSHyper-V Server 2019 gen2 ZFSProxmox 7 ZFSHyper-V Server 2022 gen2 UFSHyper-V Server 2022 gen2 ZFS #1Hyper-V Server 2022 gen2 ZFS #29184935510192772413102885433918867310526224.70104396.274570461748.82046491608113146.40.48962.1808.2502.69188.008369439910013557015398941195486195362831205.5149812.865095001095.483988206772872.11.19148.80516.1484.642401.757173303710775448767193985241822119774203.9230532.74239316323.643893167221469.32.27959.83216.4475.621125.2241818648249939801119994084581812120686203.7526539.34259131326.34238876720817.22.360130.36316.3875.722567.076107443731011324561267941203336510175935206.4622418.65321349780.86701763551093.41.492169.7489.1613.295607.5121695300020636646305094083591818119980204.2231019.81263094318.841812177720691.52.39256.28716.7745.966125.6121696724313878994149894188281838120610203.7726738.70238950318.54419243620825.12.263249.64516.0195.705658.185OpenBenchmarking.org

OpenSSL

Algorithm: SHA256

OpenBenchmarking.orgbyte/s, More Is BetterOpenSSL 3.0Algorithm: SHA256PhysicalXCP-ng 8.2 ZFSHyper-V Server 2019 gen2 UFSHyper-V Server 2019 gen2 ZFSProxmox 7 ZFSHyper-V Server 2022 gen2 UFSHyper-V Server 2022 gen2 ZFS #12000M4000M6000M8000M10000MSE +/- 4517236.92, N = 3SE +/- 4064000.36, N = 3SE +/- 14460466.96, N = 3SE +/- 30814745.08, N = 12SE +/- 664193.75, N = 3SE +/- 5593250.35, N = 3SE +/- 7574201.44, N = 391849355103694399100173303710718186482491074437310169530002016967243131. (CC) gcc options: -pthread -O3 -lssl -lcrypto

BlogBench

Test: Read

OpenBenchmarking.orgFinal Score, More Is BetterBlogBench 1.1Test: ReadPhysicalXCP-ng 8.2 ZFSHyper-V Server 2019 gen2 UFSHyper-V Server 2019 gen2 ZFSProxmox 7 ZFSHyper-V Server 2022 gen2 UFSHyper-V Server 2022 gen2 ZFS #1400K800K1200K1600K2000KSE +/- 20896.13, N = 5SE +/- 69015.49, N = 9SE +/- 37468.76, N = 12SE +/- 51240.74, N = 9SE +/- 119452.26, N = 9SE +/- 62232.41, N = 6SE +/- 68010.49, N = 61927724135570175448793980111324566366468789941. (CC) gcc options: -O2 -pthread

BlogBench

Test: Write

OpenBenchmarking.orgFinal Score, More Is BetterBlogBench 1.1Test: WritePhysicalXCP-ng 8.2 ZFSHyper-V Server 2019 gen2 UFSHyper-V Server 2019 gen2 ZFSProxmox 7 ZFSHyper-V Server 2022 gen2 UFSHyper-V Server 2022 gen2 ZFS #13K6K9K12K15KSE +/- 2643.32, N = 3SE +/- 495.20, N = 3SE +/- 383.67, N = 3SE +/- 211.50, N = 2SE +/- 1135.65, N = 313102539867111991267305014981. (CC) gcc options: -O2 -pthread

iPerf

Server Address: 192.168.1.27 - Server Port: 5201 - Duration: 360 Seconds - Test: TCP - Parallel: 20

OpenBenchmarking.orgMbits/sec, More Is BetteriPerf 3.7Server Address: 192.168.1.27 - Server Port: 5201 - Duration: 360 Seconds - Test: TCP - Parallel: 20PhysicalXCP-ng 8.2 ZFSHyper-V Server 2019 gen2 UFSHyper-V Server 2019 gen2 ZFSProxmox 7 ZFSHyper-V Server 2022 gen2 UFSHyper-V Server 2022 gen2 ZFS #12004006008001000SE +/- 36.26, N = 9SE +/- 0.33, N = 38859419399409419409411. (CC) gcc options: -O3 -march=native -lssl -lcrypto -lm

7-Zip Compression

Compress Speed Test

OpenBenchmarking.orgMIPS, More Is Better7-Zip Compression 16.02Compress Speed TestPhysicalXCP-ng 8.2 ZFSHyper-V Server 2019 gen2 UFSHyper-V Server 2019 gen2 ZFSProxmox 7 ZFSHyper-V Server 2022 gen2 UFSHyper-V Server 2022 gen2 ZFS #19K18K27K36K45KSE +/- 413.92, N = 6SE +/- 139.45, N = 3SE +/- 54.58, N = 3SE +/- 45.92, N = 3SE +/- 4.91, N = 3SE +/- 55.19, N = 3SE +/- 9.61, N = 343391195488524845820333835988281. (CXX) g++ options: -pipe -lpthread

John The Ripper

Test: Blowfish

OpenBenchmarking.orgReal C/S, More Is BetterJohn The Ripper 1.9.0-jumbo-1Test: BlowfishPhysicalXCP-ng 8.2 ZFSHyper-V Server 2019 gen2 UFSHyper-V Server 2019 gen2 ZFSProxmox 7 ZFSHyper-V Server 2022 gen2 UFSHyper-V Server 2022 gen2 ZFS #12K4K6K8K10KSE +/- 82.40, N = 15SE +/- 45.34, N = 3SE +/- 4.36, N = 3SE +/- 2.19, N = 3SE +/- 7.22, N = 3SE +/- 13.86, N = 3SE +/- 3.33, N = 38867619518221812651018181838-lssl -lcrypto -lgmp -pthread -lm -lz -ldl -lcrypt -lbz2-lssl -lcrypto -lgmp -pthread -lm -lz -ldl -lcrypt -lbz2-lpcap-lpcap-lssl -lcrypto -lgmp -pthread -lm -lz -ldl -lcrypt -lbz2-lssl -lcrypto -lgmp -pthread -lm -lz -ldl -lcrypt -lbz2-lssl -lcrypto -lgmp -pthread -lm -lz -ldl -lcrypt -lbz21. (CC) gcc options: -m64 -fopenmp

John The Ripper

Test: MD5

OpenBenchmarking.orgReal C/S, More Is BetterJohn The Ripper 1.9.0-jumbo-1Test: MD5PhysicalXCP-ng 8.2 ZFSHyper-V Server 2019 gen2 UFSHyper-V Server 2019 gen2 ZFSProxmox 7 ZFSHyper-V Server 2022 gen2 UFSHyper-V Server 2022 gen2 ZFS #180K160K240K320K400KSE +/- 3778.81, N = 3SE +/- 348.38, N = 3SE +/- 313.45, N = 3SE +/- 394.82, N = 3SE +/- 863.36, N = 3SE +/- 628.29, N = 3SE +/- 579.82, N = 3310526362831119774120686175935119980120610-lssl -lcrypto -lgmp -pthread -lm -lz -ldl -lcrypt -lbz2-lssl -lcrypto -lgmp -pthread -lm -lz -ldl -lcrypt -lbz2-lpcap-lpcap-lssl -lcrypto -lgmp -pthread -lm -lz -ldl -lcrypt -lbz2-lssl -lcrypto -lgmp -pthread -lm -lz -ldl -lcrypt -lbz2-lssl -lcrypto -lgmp -pthread -lm -lz -ldl -lcrypt -lbz21. (CC) gcc options: -m64 -fopenmp

nginx

Concurrent Requests: 100

OpenBenchmarking.orgRequests Per Second, More Is Betternginx 1.21.1Concurrent Requests: 100PhysicalXCP-ng 8.2 ZFSHyper-V Server 2019 gen2 UFSHyper-V Server 2019 gen2 ZFSProxmox 7 ZFSHyper-V Server 2022 gen2 UFSHyper-V Server 2022 gen2 ZFS #150100150200250SE +/- 6.73, N = 12SE +/- 0.85, N = 3SE +/- 1.15, N = 3SE +/- 0.96, N = 3SE +/- 1.11, N = 3SE +/- 0.63, N = 3SE +/- 0.99, N = 3224.70205.51203.92203.75206.46204.22203.771. (CC) gcc options: -lcrypt -lz -O3 -march=native

Apache HTTP Server

Concurrent Requests: 100

OpenBenchmarking.orgRequests Per Second, More Is BetterApache HTTP Server 2.4.48Concurrent Requests: 100PhysicalXCP-ng 8.2 ZFSHyper-V Server 2019 gen2 UFSHyper-V Server 2019 gen2 ZFSProxmox 7 ZFSHyper-V Server 2022 gen2 UFSHyper-V Server 2022 gen2 ZFS #120K40K60K80K100KSE +/- 357.41, N = 3SE +/- 615.93, N = 4SE +/- 367.83, N = 4SE +/- 363.45, N = 3SE +/- 46.86, N = 3SE +/- 345.94, N = 4SE +/- 284.16, N = 5104396.2749812.8630532.7426539.3422418.6531019.8126738.701. (CC) gcc options: -shared -fPIC -O2

PHPBench

PHP Benchmark Suite

OpenBenchmarking.orgScore, More Is BetterPHPBench 0.8.1PHP Benchmark SuitePhysicalXCP-ng 8.2 ZFSHyper-V Server 2019 gen2 UFSHyper-V Server 2019 gen2 ZFSProxmox 7 ZFSHyper-V Server 2022 gen2 UFSHyper-V Server 2022 gen2 ZFS #1110K220K330K440K550KSE +/- 360.51, N = 3SE +/- 319.82, N = 3SE +/- 668.88, N = 3SE +/- 814.65, N = 3SE +/- 44.94, N = 3SE +/- 1108.50, N = 3SE +/- 953.42, N = 3457046509500239316259131321349263094238950

OpenSSL

Algorithm: RSA4096

OpenBenchmarking.orgsign/s, More Is BetterOpenSSL 3.0Algorithm: RSA4096PhysicalXCP-ng 8.2 ZFSHyper-V Server 2019 gen2 UFSHyper-V Server 2019 gen2 ZFSProxmox 7 ZFSHyper-V Server 2022 gen2 UFSHyper-V Server 2022 gen2 ZFS #1400800120016002000SE +/- 0.03, N = 3SE +/- 11.14, N = 13SE +/- 1.89, N = 3SE +/- 2.80, N = 3SE +/- 0.74, N = 3SE +/- 1.59, N = 3SE +/- 1.22, N = 31748.81095.4323.6326.3780.8318.8318.51. (CC) gcc options: -pthread -O3 -lssl -lcrypto

PostgreSQL pgbench

Scaling Factor: 1 - Clients: 100 - Mode: Read Only

OpenBenchmarking.orgTPS, More Is BetterPostgreSQL pgbench 13.0Scaling Factor: 1 - Clients: 100 - Mode: Read OnlyPhysicalXCP-ng 8.2 ZFSHyper-V Server 2019 gen2 UFSHyper-V Server 2019 gen2 ZFSProxmox 7 ZFS40K80K120K160K200KSE +/- 1738.86, N = 3SE +/- 24.29, N = 3SE +/- 475.16, N = 3SE +/- 144.69, N = 3SE +/- 120.61, N = 320464983988438934238867017-lpq-lpq-lpq1. (CC) gcc options: -fno-strict-aliasing -fwrapv -O2 -lpgcommon -lpgport -lexecinfo -lm

PostgreSQL pgbench

Scaling Factor: 1 - Clients: 100 - Mode: Read Write

OpenBenchmarking.orgTPS, More Is BetterPostgreSQL pgbench 13.0Scaling Factor: 1 - Clients: 100 - Mode: Read WritePhysicalXCP-ng 8.2 ZFSHyper-V Server 2019 gen2 UFSHyper-V Server 2019 gen2 ZFSProxmox 7 ZFS400800120016002000SE +/- 3.35, N = 3SE +/- 50.23, N = 15SE +/- 11.36, N = 3SE +/- 9.40, N = 3SE +/- 48.39, N = 15160820671672767635-lpq-lpq-lpq1. (CC) gcc options: -fno-strict-aliasing -fwrapv -O2 -lpgcommon -lpgport -lexecinfo -lm

PostgreSQL pgbench

Scaling Factor: 1 - Clients: 100 - Mode: Read Only

OpenBenchmarking.orgTPS, More Is BetterPostgreSQL pgbench 14.0Scaling Factor: 1 - Clients: 100 - Mode: Read OnlyHyper-V Server 2022 gen2 UFSHyper-V Server 2022 gen2 ZFS #19K18K27K36K45KSE +/- 125.66, N = 3SE +/- 141.62, N = 341812441921. (CC) gcc options: -fno-strict-aliasing -fwrapv -O2 -lpgcommon -lpgport -lpq -lpthread -lexecinfo -lm

PostgreSQL pgbench

Scaling Factor: 1 - Clients: 100 - Mode: Read Write

OpenBenchmarking.orgTPS, More Is BetterPostgreSQL pgbench 14.0Scaling Factor: 1 - Clients: 100 - Mode: Read WriteHyper-V Server 2022 gen2 UFSHyper-V Server 2022 gen2 ZFS #1400800120016002000SE +/- 19.81, N = 3SE +/- 48.63, N = 917774361. (CC) gcc options: -fno-strict-aliasing -fwrapv -O2 -lpgcommon -lpgport -lpq -lpthread -lexecinfo -lm

OpenSSL

Algorithm: RSA4096

OpenBenchmarking.orgverify/s, More Is BetterOpenSSL 3.0Algorithm: RSA4096PhysicalXCP-ng 8.2 ZFSHyper-V Server 2019 gen2 UFSHyper-V Server 2019 gen2 ZFSProxmox 7 ZFSHyper-V Server 2022 gen2 UFSHyper-V Server 2022 gen2 ZFS #120K40K60K80K100KSE +/- 40.47, N = 3SE +/- 728.16, N = 13SE +/- 695.73, N = 3SE +/- 84.84, N = 3SE +/- 104.96, N = 3SE +/- 74.87, N = 3SE +/- 37.37, N = 3113146.472872.121469.320817.251093.420691.520825.11. (CC) gcc options: -pthread -O3 -lssl -lcrypto

PostgreSQL pgbench

Scaling Factor: 1 - Clients: 100 - Mode: Read Only - Average Latency

OpenBenchmarking.orgms, Fewer Is BetterPostgreSQL pgbench 13.0Scaling Factor: 1 - Clients: 100 - Mode: Read Only - Average LatencyPhysicalXCP-ng 8.2 ZFSHyper-V Server 2019 gen2 UFSHyper-V Server 2019 gen2 ZFSProxmox 7 ZFS0.5311.0621.5932.1242.655SE +/- 0.004, N = 3SE +/- 0.000, N = 3SE +/- 0.025, N = 3SE +/- 0.008, N = 3SE +/- 0.003, N = 30.4891.1912.2792.3601.492-lpq-lpq-lpq1. (CC) gcc options: -fno-strict-aliasing -fwrapv -O2 -lpgcommon -lpgport -lexecinfo -lm

PostgreSQL pgbench

Scaling Factor: 1 - Clients: 100 - Mode: Read Write - Average Latency

OpenBenchmarking.orgms, Fewer Is BetterPostgreSQL pgbench 13.0Scaling Factor: 1 - Clients: 100 - Mode: Read Write - Average LatencyPhysicalXCP-ng 8.2 ZFSHyper-V Server 2019 gen2 UFSHyper-V Server 2019 gen2 ZFSProxmox 7 ZFS4080120160200SE +/- 0.13, N = 3SE +/- 1.27, N = 15SE +/- 0.41, N = 3SE +/- 1.59, N = 3SE +/- 12.35, N = 1562.1848.8159.83130.36169.75-lpq-lpq-lpq1. (CC) gcc options: -fno-strict-aliasing -fwrapv -O2 -lpgcommon -lpgport -lexecinfo -lm

PostgreSQL pgbench

Scaling Factor: 1 - Clients: 100 - Mode: Read Only - Average Latency

OpenBenchmarking.orgms, Fewer Is BetterPostgreSQL pgbench 14.0Scaling Factor: 1 - Clients: 100 - Mode: Read Only - Average LatencyHyper-V Server 2022 gen2 UFSHyper-V Server 2022 gen2 ZFS #10.53821.07641.61462.15282.691SE +/- 0.007, N = 3SE +/- 0.007, N = 32.3922.2631. (CC) gcc options: -fno-strict-aliasing -fwrapv -O2 -lpgcommon -lpgport -lpq -lpthread -lexecinfo -lm

PostgreSQL pgbench

Scaling Factor: 1 - Clients: 100 - Mode: Read Write - Average Latency

OpenBenchmarking.orgms, Fewer Is BetterPostgreSQL pgbench 14.0Scaling Factor: 1 - Clients: 100 - Mode: Read Write - Average LatencyHyper-V Server 2022 gen2 UFSHyper-V Server 2022 gen2 ZFS #150100150200250SE +/- 0.62, N = 3SE +/- 22.82, N = 956.29249.651. (CC) gcc options: -fno-strict-aliasing -fwrapv -O2 -lpgcommon -lpgport -lpq -lpthread -lexecinfo -lm

t-test1

Threads: 1

OpenBenchmarking.orgSeconds, Fewer Is Bettert-test1 2017-01-13Threads: 1PhysicalXCP-ng 8.2 ZFSHyper-V Server 2019 gen2 UFSHyper-V Server 2019 gen2 ZFSProxmox 7 ZFSHyper-V Server 2022 gen2 UFSHyper-V Server 2022 gen2 ZFS #148121620SE +/- 0.030, N = 3SE +/- 0.053, N = 3SE +/- 0.033, N = 3SE +/- 0.069, N = 3SE +/- 0.037, N = 3SE +/- 0.121, N = 3SE +/- 0.029, N = 38.25016.14816.44716.3879.16116.77416.0191. (CC) gcc options: -pthread

t-test1

Threads: 2

OpenBenchmarking.orgSeconds, Fewer Is Bettert-test1 2017-01-13Threads: 2PhysicalXCP-ng 8.2 ZFSHyper-V Server 2019 gen2 UFSHyper-V Server 2019 gen2 ZFSProxmox 7 ZFSHyper-V Server 2022 gen2 UFSHyper-V Server 2022 gen2 ZFS #11.34242.68484.02725.36966.712SE +/- 0.025, N = 7SE +/- 0.067, N = 15SE +/- 0.058, N = 15SE +/- 0.071, N = 15SE +/- 0.006, N = 3SE +/- 0.020, N = 3SE +/- 0.063, N = 32.6914.6425.6215.7223.2955.9665.7051. (CC) gcc options: -pthread

SQLite Speedtest

Timed Time - Size 1,000

OpenBenchmarking.orgSeconds, Fewer Is BetterSQLite Speedtest 3.30Timed Time - Size 1,000PhysicalXCP-ng 8.2 ZFSHyper-V Server 2019 gen2 UFSHyper-V Server 2019 gen2 ZFSProxmox 7 ZFSHyper-V Server 2022 gen2 UFSHyper-V Server 2022 gen2 ZFS #1140280420560700SE +/- 0.04, N = 3SE +/- 5.15, N = 3SE +/- 0.45, N = 3SE +/- 1.48, N = 3SE +/- 1.39, N = 3SE +/- 0.17, N = 3SE +/- 0.40, N = 388.01401.76125.22567.08607.51125.61658.191. (CC) gcc options: -O2 -lz -lpthread

Phoronix Test Suite v10.8.5