FreeBSD ZFS Hypervisor

AMD Ryzen 5 3600 6-Core testing on FreeBSD via the Phoronix Test Suite.

HTML result view exported from: https://openbenchmarking.org/result/2110053-IB-FREEBSDZF95&grw.

FreeBSD ZFS HypervisorProcessorChipsetMemoryDiskGraphicsAudioMotherboardOSKernelCompilerFile-SystemScreen ResolutionPhysicalHyper-V Server 2022 gen2 ZFS #1Hyper-V Server 2019 gen2 UFSHyper-V Server 2019 gen2 ZFSHyper-V Server 2022 gen2 UFSHyper-V Server 2022 gen2 ZFS #2Proxmox 7 ZFSXCP-ng 8.2 ZFSAMD Ryzen 5 3600 6-Core @ 3.60GHz (12 Cores)AMD [AMD] Starship/Matisse Root Complex64GBSamsung SSD 980 1TBASPEEDAMD X570 HDAFreeBSD13.0-RELEASE (x86_64)GCC 10.3.0 + Clang 11.0.1zfs800x600AMD Ryzen 5 3600 6-Core (4 Cores)Microsoft Virtual Machine Hyper-V UEFI v4.1 (Hyper-V BIOS)4096MBMsft Virtual Disk 1.013.0-RELEASE-p4 (x86_64)1024x768Microsoft Virtual Machine Hyper-V UEFI v4.0 (Hyper-V BIOS)ufs13.0-RELEASE (x86_64)zfsMicrosoft Virtual Machine Hyper-V UEFI v4.1 (Hyper-V BIOS)ufs13.0-RELEASE-p4 (x86_64)zfsCommon KVM (4 Cores)QEMU Standard PC (i440FX + PIIX 1996) pc-i440fx-6.0Intel 440FX 82441FX PMC [Natoma]47GBIntel 82371AB/EB/MB PIIX4 ACPI13.0-RELEASE (x86_64)AMD Ryzen 5 3600 6-Core (4 Cores)Xen HVM domU 4.13Cirrus Logic GD 544613.0-RELEASE-p4 (x86_64)OpenBenchmarking.orgCompiler Details- --build=x86_64-portbld-freebsd13.0 --disable-nls --enable-gnu-indirect-function --enable-languages=c,c++,objc,fortran --enable-multilib --enable-plugin --localstatedir=/var --mandir=/usr/local/man --with-as=/usr/local/bin/as --with-build-config=bootstrap-debug --with-gmp=/usr/local --with-ld=/usr/local/bin/ld --without-zstd

FreeBSD ZFS Hypervisort-test1: 1t-test1: 2iperf: 192.168.1.27 - 5201 - 360 Seconds - TCP - 20compress-7zip: Compress Speed Testjohn-the-ripper: Blowfishjohn-the-ripper: MD5blogbench: Readblogbench: Writenginx: 100apache: 100openssl: SHA256openssl: RSA4096openssl: RSA4096phpbench: PHP Benchmark Suitesqlite-speedtest: Timed Time - Size 1,000pgbench: 1 - 100 - Read Onlypgbench: 1 - 100 - Read Only - Average Latencypgbench: 1 - 100 - Read Writepgbench: 1 - 100 - Read Write - Average Latencypgbench: 1 - 100 - Read Onlypgbench: 1 - 100 - Read Only - Average Latencypgbench: 1 - 100 - Read Writepgbench: 1 - 100 - Read Write - Average LatencyPhysicalHyper-V Server 2022 gen2 ZFS #1Hyper-V Server 2019 gen2 UFSHyper-V Server 2019 gen2 ZFSHyper-V Server 2022 gen2 UFSHyper-V Server 2022 gen2 ZFS #2Proxmox 7 ZFSXCP-ng 8.2 ZFS8.2502.691885433918867310526192772413102224.70104396.2791849355101748.8113146.445704688.0082046490.489160862.18016.0195.705941882818381206108789941498203.7726738.701696724313318.520825.1238950658.185441922.263436249.64516.4475.62193985241822119774754487671203.9230532.741733037107323.621469.3239316125.224438932.279167259.83216.3875.722940845818121206869398011199203.7526539.341818648249326.320817.2259131567.076423882.360767130.36316.7745.966940835918181199806366463050204.2231019.811695300020318.820691.5263094125.612418122.392177756.2879.1613.29594120333651017593511324561267206.4622418.651074437310780.851093.4321349607.512670171.492635169.74816.1484.64294119548619536283113557015398205.5149812.8636943991001095.472872.1509500401.757839881.191206748.805OpenBenchmarking.org

t-test1

Threads: 1

OpenBenchmarking.orgSeconds, Fewer Is Bettert-test1 2017-01-13Threads: 1PhysicalHyper-V Server 2022 gen2 ZFS #1Hyper-V Server 2019 gen2 UFSHyper-V Server 2019 gen2 ZFSHyper-V Server 2022 gen2 UFSProxmox 7 ZFSXCP-ng 8.2 ZFS48121620SE +/- 0.030, N = 3SE +/- 0.029, N = 3SE +/- 0.033, N = 3SE +/- 0.069, N = 3SE +/- 0.121, N = 3SE +/- 0.037, N = 3SE +/- 0.053, N = 38.25016.01916.44716.38716.7749.16116.1481. (CC) gcc options: -pthread

t-test1

Threads: 2

OpenBenchmarking.orgSeconds, Fewer Is Bettert-test1 2017-01-13Threads: 2PhysicalHyper-V Server 2022 gen2 ZFS #1Hyper-V Server 2019 gen2 UFSHyper-V Server 2019 gen2 ZFSHyper-V Server 2022 gen2 UFSProxmox 7 ZFSXCP-ng 8.2 ZFS1.34242.68484.02725.36966.712SE +/- 0.025, N = 7SE +/- 0.063, N = 3SE +/- 0.058, N = 15SE +/- 0.071, N = 15SE +/- 0.020, N = 3SE +/- 0.006, N = 3SE +/- 0.067, N = 152.6915.7055.6215.7225.9663.2954.6421. (CC) gcc options: -pthread

iPerf

Server Address: 192.168.1.27 - Server Port: 5201 - Duration: 360 Seconds - Test: TCP - Parallel: 20

OpenBenchmarking.orgMbits/sec, More Is BetteriPerf 3.7Server Address: 192.168.1.27 - Server Port: 5201 - Duration: 360 Seconds - Test: TCP - Parallel: 20PhysicalHyper-V Server 2022 gen2 ZFS #1Hyper-V Server 2019 gen2 UFSHyper-V Server 2019 gen2 ZFSHyper-V Server 2022 gen2 UFSProxmox 7 ZFSXCP-ng 8.2 ZFS2004006008001000SE +/- 36.26, N = 9SE +/- 0.33, N = 38859419399409409419411. (CC) gcc options: -O3 -march=native -lssl -lcrypto -lm

7-Zip Compression

Compress Speed Test

OpenBenchmarking.orgMIPS, More Is Better7-Zip Compression 16.02Compress Speed TestPhysicalHyper-V Server 2022 gen2 ZFS #1Hyper-V Server 2019 gen2 UFSHyper-V Server 2019 gen2 ZFSHyper-V Server 2022 gen2 UFSProxmox 7 ZFSXCP-ng 8.2 ZFS9K18K27K36K45KSE +/- 413.92, N = 6SE +/- 9.61, N = 3SE +/- 54.58, N = 3SE +/- 45.92, N = 3SE +/- 55.19, N = 3SE +/- 4.91, N = 3SE +/- 139.45, N = 343391882885248458835920333195481. (CXX) g++ options: -pipe -lpthread

John The Ripper

Test: Blowfish

OpenBenchmarking.orgReal C/S, More Is BetterJohn The Ripper 1.9.0-jumbo-1Test: BlowfishPhysicalHyper-V Server 2022 gen2 ZFS #1Hyper-V Server 2019 gen2 UFSHyper-V Server 2019 gen2 ZFSHyper-V Server 2022 gen2 UFSProxmox 7 ZFSXCP-ng 8.2 ZFS2K4K6K8K10KSE +/- 82.40, N = 15SE +/- 3.33, N = 3SE +/- 4.36, N = 3SE +/- 2.19, N = 3SE +/- 13.86, N = 3SE +/- 7.22, N = 3SE +/- 45.34, N = 38867183818221812181865106195-lssl -lcrypto -lgmp -pthread -lm -lz -ldl -lcrypt -lbz2-lssl -lcrypto -lgmp -pthread -lm -lz -ldl -lcrypt -lbz2-lpcap-lpcap-lssl -lcrypto -lgmp -pthread -lm -lz -ldl -lcrypt -lbz2-lssl -lcrypto -lgmp -pthread -lm -lz -ldl -lcrypt -lbz2-lssl -lcrypto -lgmp -pthread -lm -lz -ldl -lcrypt -lbz21. (CC) gcc options: -m64 -fopenmp

John The Ripper

Test: MD5

OpenBenchmarking.orgReal C/S, More Is BetterJohn The Ripper 1.9.0-jumbo-1Test: MD5PhysicalHyper-V Server 2022 gen2 ZFS #1Hyper-V Server 2019 gen2 UFSHyper-V Server 2019 gen2 ZFSHyper-V Server 2022 gen2 UFSProxmox 7 ZFSXCP-ng 8.2 ZFS80K160K240K320K400KSE +/- 3778.81, N = 3SE +/- 579.82, N = 3SE +/- 313.45, N = 3SE +/- 394.82, N = 3SE +/- 628.29, N = 3SE +/- 863.36, N = 3SE +/- 348.38, N = 3310526120610119774120686119980175935362831-lssl -lcrypto -lgmp -pthread -lm -lz -ldl -lcrypt -lbz2-lssl -lcrypto -lgmp -pthread -lm -lz -ldl -lcrypt -lbz2-lpcap-lpcap-lssl -lcrypto -lgmp -pthread -lm -lz -ldl -lcrypt -lbz2-lssl -lcrypto -lgmp -pthread -lm -lz -ldl -lcrypt -lbz2-lssl -lcrypto -lgmp -pthread -lm -lz -ldl -lcrypt -lbz21. (CC) gcc options: -m64 -fopenmp

BlogBench

Test: Read

OpenBenchmarking.orgFinal Score, More Is BetterBlogBench 1.1Test: ReadPhysicalHyper-V Server 2022 gen2 ZFS #1Hyper-V Server 2019 gen2 UFSHyper-V Server 2019 gen2 ZFSHyper-V Server 2022 gen2 UFSProxmox 7 ZFSXCP-ng 8.2 ZFS400K800K1200K1600K2000KSE +/- 20896.13, N = 5SE +/- 68010.49, N = 6SE +/- 37468.76, N = 12SE +/- 51240.74, N = 9SE +/- 62232.41, N = 6SE +/- 119452.26, N = 9SE +/- 69015.49, N = 91927724878994754487939801636646113245613557011. (CC) gcc options: -O2 -pthread

BlogBench

Test: Write

OpenBenchmarking.orgFinal Score, More Is BetterBlogBench 1.1Test: WritePhysicalHyper-V Server 2022 gen2 ZFS #1Hyper-V Server 2019 gen2 UFSHyper-V Server 2019 gen2 ZFSHyper-V Server 2022 gen2 UFSProxmox 7 ZFSXCP-ng 8.2 ZFS3K6K9K12K15KSE +/- 1135.65, N = 3SE +/- 495.20, N = 3SE +/- 211.50, N = 2SE +/- 383.67, N = 3SE +/- 2643.32, N = 313102149867111993050126753981. (CC) gcc options: -O2 -pthread

nginx

Concurrent Requests: 100

OpenBenchmarking.orgRequests Per Second, More Is Betternginx 1.21.1Concurrent Requests: 100PhysicalHyper-V Server 2022 gen2 ZFS #1Hyper-V Server 2019 gen2 UFSHyper-V Server 2019 gen2 ZFSHyper-V Server 2022 gen2 UFSProxmox 7 ZFSXCP-ng 8.2 ZFS50100150200250SE +/- 6.73, N = 12SE +/- 0.99, N = 3SE +/- 1.15, N = 3SE +/- 0.96, N = 3SE +/- 0.63, N = 3SE +/- 1.11, N = 3SE +/- 0.85, N = 3224.70203.77203.92203.75204.22206.46205.511. (CC) gcc options: -lcrypt -lz -O3 -march=native

Apache HTTP Server

Concurrent Requests: 100

OpenBenchmarking.orgRequests Per Second, More Is BetterApache HTTP Server 2.4.48Concurrent Requests: 100PhysicalHyper-V Server 2022 gen2 ZFS #1Hyper-V Server 2019 gen2 UFSHyper-V Server 2019 gen2 ZFSHyper-V Server 2022 gen2 UFSProxmox 7 ZFSXCP-ng 8.2 ZFS20K40K60K80K100KSE +/- 357.41, N = 3SE +/- 284.16, N = 5SE +/- 367.83, N = 4SE +/- 363.45, N = 3SE +/- 345.94, N = 4SE +/- 46.86, N = 3SE +/- 615.93, N = 4104396.2726738.7030532.7426539.3431019.8122418.6549812.861. (CC) gcc options: -shared -fPIC -O2

OpenSSL

Algorithm: SHA256

OpenBenchmarking.orgbyte/s, More Is BetterOpenSSL 3.0Algorithm: SHA256PhysicalHyper-V Server 2022 gen2 ZFS #1Hyper-V Server 2019 gen2 UFSHyper-V Server 2019 gen2 ZFSHyper-V Server 2022 gen2 UFSProxmox 7 ZFSXCP-ng 8.2 ZFS2000M4000M6000M8000M10000MSE +/- 4517236.92, N = 3SE +/- 7574201.44, N = 3SE +/- 14460466.96, N = 3SE +/- 30814745.08, N = 12SE +/- 5593250.35, N = 3SE +/- 664193.75, N = 3SE +/- 4064000.36, N = 391849355101696724313173303710718186482491695300020107443731036943991001. (CC) gcc options: -pthread -O3 -lssl -lcrypto

OpenSSL

Algorithm: RSA4096

OpenBenchmarking.orgsign/s, More Is BetterOpenSSL 3.0Algorithm: RSA4096PhysicalHyper-V Server 2022 gen2 ZFS #1Hyper-V Server 2019 gen2 UFSHyper-V Server 2019 gen2 ZFSHyper-V Server 2022 gen2 UFSProxmox 7 ZFSXCP-ng 8.2 ZFS400800120016002000SE +/- 0.03, N = 3SE +/- 1.22, N = 3SE +/- 1.89, N = 3SE +/- 2.80, N = 3SE +/- 1.59, N = 3SE +/- 0.74, N = 3SE +/- 11.14, N = 131748.8318.5323.6326.3318.8780.81095.41. (CC) gcc options: -pthread -O3 -lssl -lcrypto

OpenSSL

Algorithm: RSA4096

OpenBenchmarking.orgverify/s, More Is BetterOpenSSL 3.0Algorithm: RSA4096PhysicalHyper-V Server 2022 gen2 ZFS #1Hyper-V Server 2019 gen2 UFSHyper-V Server 2019 gen2 ZFSHyper-V Server 2022 gen2 UFSProxmox 7 ZFSXCP-ng 8.2 ZFS20K40K60K80K100KSE +/- 40.47, N = 3SE +/- 37.37, N = 3SE +/- 695.73, N = 3SE +/- 84.84, N = 3SE +/- 74.87, N = 3SE +/- 104.96, N = 3SE +/- 728.16, N = 13113146.420825.121469.320817.220691.551093.472872.11. (CC) gcc options: -pthread -O3 -lssl -lcrypto

PHPBench

PHP Benchmark Suite

OpenBenchmarking.orgScore, More Is BetterPHPBench 0.8.1PHP Benchmark SuitePhysicalHyper-V Server 2022 gen2 ZFS #1Hyper-V Server 2019 gen2 UFSHyper-V Server 2019 gen2 ZFSHyper-V Server 2022 gen2 UFSProxmox 7 ZFSXCP-ng 8.2 ZFS110K220K330K440K550KSE +/- 360.51, N = 3SE +/- 953.42, N = 3SE +/- 668.88, N = 3SE +/- 814.65, N = 3SE +/- 1108.50, N = 3SE +/- 44.94, N = 3SE +/- 319.82, N = 3457046238950239316259131263094321349509500

SQLite Speedtest

Timed Time - Size 1,000

OpenBenchmarking.orgSeconds, Fewer Is BetterSQLite Speedtest 3.30Timed Time - Size 1,000PhysicalHyper-V Server 2022 gen2 ZFS #1Hyper-V Server 2019 gen2 UFSHyper-V Server 2019 gen2 ZFSHyper-V Server 2022 gen2 UFSProxmox 7 ZFSXCP-ng 8.2 ZFS140280420560700SE +/- 0.04, N = 3SE +/- 0.40, N = 3SE +/- 0.45, N = 3SE +/- 1.48, N = 3SE +/- 0.17, N = 3SE +/- 1.39, N = 3SE +/- 5.15, N = 388.01658.19125.22567.08125.61607.51401.761. (CC) gcc options: -O2 -lz -lpthread

PostgreSQL pgbench

Scaling Factor: 1 - Clients: 100 - Mode: Read Only

OpenBenchmarking.orgTPS, More Is BetterPostgreSQL pgbench 13.0Scaling Factor: 1 - Clients: 100 - Mode: Read OnlyPhysicalHyper-V Server 2019 gen2 UFSHyper-V Server 2019 gen2 ZFSProxmox 7 ZFSXCP-ng 8.2 ZFS40K80K120K160K200KSE +/- 1738.86, N = 3SE +/- 475.16, N = 3SE +/- 144.69, N = 3SE +/- 120.61, N = 3SE +/- 24.29, N = 320464943893423886701783988-lpq-lpq-lpq1. (CC) gcc options: -fno-strict-aliasing -fwrapv -O2 -lpgcommon -lpgport -lexecinfo -lm

PostgreSQL pgbench

Scaling Factor: 1 - Clients: 100 - Mode: Read Only - Average Latency

OpenBenchmarking.orgms, Fewer Is BetterPostgreSQL pgbench 13.0Scaling Factor: 1 - Clients: 100 - Mode: Read Only - Average LatencyPhysicalHyper-V Server 2019 gen2 UFSHyper-V Server 2019 gen2 ZFSProxmox 7 ZFSXCP-ng 8.2 ZFS0.5311.0621.5932.1242.655SE +/- 0.004, N = 3SE +/- 0.025, N = 3SE +/- 0.008, N = 3SE +/- 0.003, N = 3SE +/- 0.000, N = 30.4892.2792.3601.4921.191-lpq-lpq-lpq1. (CC) gcc options: -fno-strict-aliasing -fwrapv -O2 -lpgcommon -lpgport -lexecinfo -lm

PostgreSQL pgbench

Scaling Factor: 1 - Clients: 100 - Mode: Read Write

OpenBenchmarking.orgTPS, More Is BetterPostgreSQL pgbench 13.0Scaling Factor: 1 - Clients: 100 - Mode: Read WritePhysicalHyper-V Server 2019 gen2 UFSHyper-V Server 2019 gen2 ZFSProxmox 7 ZFSXCP-ng 8.2 ZFS400800120016002000SE +/- 3.35, N = 3SE +/- 11.36, N = 3SE +/- 9.40, N = 3SE +/- 48.39, N = 15SE +/- 50.23, N = 15160816727676352067-lpq-lpq-lpq1. (CC) gcc options: -fno-strict-aliasing -fwrapv -O2 -lpgcommon -lpgport -lexecinfo -lm

PostgreSQL pgbench

Scaling Factor: 1 - Clients: 100 - Mode: Read Write - Average Latency

OpenBenchmarking.orgms, Fewer Is BetterPostgreSQL pgbench 13.0Scaling Factor: 1 - Clients: 100 - Mode: Read Write - Average LatencyPhysicalHyper-V Server 2019 gen2 UFSHyper-V Server 2019 gen2 ZFSProxmox 7 ZFSXCP-ng 8.2 ZFS4080120160200SE +/- 0.13, N = 3SE +/- 0.41, N = 3SE +/- 1.59, N = 3SE +/- 12.35, N = 15SE +/- 1.27, N = 1562.1859.83130.36169.7548.81-lpq-lpq-lpq1. (CC) gcc options: -fno-strict-aliasing -fwrapv -O2 -lpgcommon -lpgport -lexecinfo -lm

PostgreSQL pgbench

Scaling Factor: 1 - Clients: 100 - Mode: Read Only

OpenBenchmarking.orgTPS, More Is BetterPostgreSQL pgbench 14.0Scaling Factor: 1 - Clients: 100 - Mode: Read OnlyHyper-V Server 2022 gen2 ZFS #1Hyper-V Server 2022 gen2 UFS9K18K27K36K45KSE +/- 141.62, N = 3SE +/- 125.66, N = 344192418121. (CC) gcc options: -fno-strict-aliasing -fwrapv -O2 -lpgcommon -lpgport -lpq -lpthread -lexecinfo -lm

PostgreSQL pgbench

Scaling Factor: 1 - Clients: 100 - Mode: Read Only - Average Latency

OpenBenchmarking.orgms, Fewer Is BetterPostgreSQL pgbench 14.0Scaling Factor: 1 - Clients: 100 - Mode: Read Only - Average LatencyHyper-V Server 2022 gen2 ZFS #1Hyper-V Server 2022 gen2 UFS0.53821.07641.61462.15282.691SE +/- 0.007, N = 3SE +/- 0.007, N = 32.2632.3921. (CC) gcc options: -fno-strict-aliasing -fwrapv -O2 -lpgcommon -lpgport -lpq -lpthread -lexecinfo -lm

PostgreSQL pgbench

Scaling Factor: 1 - Clients: 100 - Mode: Read Write

OpenBenchmarking.orgTPS, More Is BetterPostgreSQL pgbench 14.0Scaling Factor: 1 - Clients: 100 - Mode: Read WriteHyper-V Server 2022 gen2 ZFS #1Hyper-V Server 2022 gen2 UFS400800120016002000SE +/- 48.63, N = 9SE +/- 19.81, N = 343617771. (CC) gcc options: -fno-strict-aliasing -fwrapv -O2 -lpgcommon -lpgport -lpq -lpthread -lexecinfo -lm

PostgreSQL pgbench

Scaling Factor: 1 - Clients: 100 - Mode: Read Write - Average Latency

OpenBenchmarking.orgms, Fewer Is BetterPostgreSQL pgbench 14.0Scaling Factor: 1 - Clients: 100 - Mode: Read Write - Average LatencyHyper-V Server 2022 gen2 ZFS #1Hyper-V Server 2022 gen2 UFS50100150200250SE +/- 22.82, N = 9SE +/- 0.62, N = 3249.6556.291. (CC) gcc options: -fno-strict-aliasing -fwrapv -O2 -lpgcommon -lpgport -lpq -lpthread -lexecinfo -lm

Phoronix Test Suite v10.8.5