AMD EPYC 7R13 48-Core testing with a Supermicro H12SSL-I v1.02 (2.7 BIOS) and NVIDIA GeForce RTX 4090 24GB on EndeavourOS rolling via the Phoronix Test Suite.
Compare your own system(s) to this result file with the
Phoronix Test Suite by running the command:
phoronix-test-suite benchmark 2402177-NE-HWMIG553791 hw-mig - Phoronix Test Suite hw-mig AMD EPYC 7R13 48-Core testing with a Supermicro H12SSL-I v1.02 (2.7 BIOS) and NVIDIA GeForce RTX 4090 24GB on EndeavourOS rolling via the Phoronix Test Suite.
HTML result view exported from: https://openbenchmarking.org/result/2402177-NE-HWMIG553791&grt&sor .
hw-mig Processor Motherboard Chipset Memory Disk Graphics Audio Monitor Network OS Kernel Desktop Display Server Display Driver OpenGL Compiler File-System Screen Resolution AMD EPYC 7R13 48-Core - NVIDIA GeForce RTX 4090 24GB AMD EPYC 7R13 48-Core AMD EPYC 7R13 48-Core @ 3.73GHz (48 Cores / 96 Threads) Supermicro H12SSL-I v1.02 (2.7 BIOS) AMD Starship/Matisse 256GB 15363GB Micron_7450_MTFDKCC15T3TFR NVIDIA GeForce RTX 4090 24GB NVIDIA AD102 HD Audio 38GN950 2 x Intel X710 for 10GbE SFP+ EndeavourOS rolling 6.7.4-zen1-1-zen (x86_64) Xfce 4.18 X Server 1.21.1.11 NVIDIA 545.29.06 4.6.0 GCC 13.2.1 20230801 + Clang 16.0.6 + LLVM 16.0.6 + CUDA 12.3 btrfs 3840x1600 OpenBenchmarking.org Kernel Details - Transparent Huge Pages: always Environment Details - NVCC_PREPEND_FLAGS="-ccbin /opt/cuda/bin" Compiler Details - --disable-libssp --disable-libstdcxx-pch --disable-werror --enable-__cxa_atexit --enable-bootstrap --enable-cet=auto --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-default-ssp --enable-gnu-indirect-function --enable-gnu-unique-object --enable-languages=ada,c,c++,d,fortran,go,lto,m2,objc,obj-c++ --enable-libstdcxx-backtrace --enable-link-serialization=1 --enable-lto --enable-multilib --enable-plugin --enable-shared --enable-threads=posix --mandir=/usr/share/man --with-build-config=bootstrap-lto --with-linker-hash-style=gnu Processor Details - Scaling Governor: amd-pstate-epp performance (EPP: performance) - CPU Microcode: 0xa0011d1 Security Details - gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_rstack_overflow: Mitigation of Safe RET + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional IBRS_FW STIBP: always-on RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected
hw-mig llama-cpp: llama-2-7b.Q4_0.gguf llama-cpp: llama-2-13b.Q4_0.gguf llama-cpp: llama-2-70b-chat.Q5_0.gguf llamafile: llava-v1.5-7b-q4 - CPU llamafile: mistral-7b-instruct-v0.2.Q8_0 - CPU llamafile: wizardcoder-python-34b-v1.0.Q6_K - CPU povray: Trace Time redis: GET - 50 redis: GET - 1000 redis: SET - 1000 redis: LPOP - 1000 redis: SADD - 1000 redis: LPUSH - 1000 AMD EPYC 7R13 48-Core - NVIDIA GeForce RTX 4090 24GB AMD EPYC 7R13 48-Core 23.17 14.45 2.86 22.30 13.34 4.73 2792000.50 2505995.40 2055844.83 1679590.47 2268077.83 1744859.50 10.616 OpenBenchmarking.org
Llama.cpp Model: llama-2-7b.Q4_0.gguf OpenBenchmarking.org Tokens Per Second, More Is Better Llama.cpp b1808 Model: llama-2-7b.Q4_0.gguf AMD EPYC 7R13 48-Core - NVIDIA GeForce RTX 4090 24GB 6 12 18 24 30 SE +/- 0.03, N = 3 23.17 1. (CXX) g++ options: -std=c++11 -fPIC -O3 -pthread -march=native -mtune=native -fopenmp -lopenblas
Llama.cpp Model: llama-2-13b.Q4_0.gguf OpenBenchmarking.org Tokens Per Second, More Is Better Llama.cpp b1808 Model: llama-2-13b.Q4_0.gguf AMD EPYC 7R13 48-Core - NVIDIA GeForce RTX 4090 24GB 4 8 12 16 20 SE +/- 0.01, N = 3 14.45 1. (CXX) g++ options: -std=c++11 -fPIC -O3 -pthread -march=native -mtune=native -fopenmp -lopenblas
Llama.cpp Model: llama-2-70b-chat.Q5_0.gguf OpenBenchmarking.org Tokens Per Second, More Is Better Llama.cpp b1808 Model: llama-2-70b-chat.Q5_0.gguf AMD EPYC 7R13 48-Core - NVIDIA GeForce RTX 4090 24GB 0.6435 1.287 1.9305 2.574 3.2175 SE +/- 0.01, N = 3 2.86 1. (CXX) g++ options: -std=c++11 -fPIC -O3 -pthread -march=native -mtune=native -fopenmp -lopenblas
Llamafile Test: llava-v1.5-7b-q4 - Acceleration: CPU OpenBenchmarking.org Tokens Per Second, More Is Better Llamafile 0.6 Test: llava-v1.5-7b-q4 - Acceleration: CPU AMD EPYC 7R13 48-Core - NVIDIA GeForce RTX 4090 24GB 5 10 15 20 25 SE +/- 0.05, N = 3 22.30
Llamafile Test: mistral-7b-instruct-v0.2.Q8_0 - Acceleration: CPU OpenBenchmarking.org Tokens Per Second, More Is Better Llamafile 0.6 Test: mistral-7b-instruct-v0.2.Q8_0 - Acceleration: CPU AMD EPYC 7R13 48-Core - NVIDIA GeForce RTX 4090 24GB 3 6 9 12 15 SE +/- 0.03, N = 3 13.34
Llamafile Test: wizardcoder-python-34b-v1.0.Q6_K - Acceleration: CPU OpenBenchmarking.org Tokens Per Second, More Is Better Llamafile 0.6 Test: wizardcoder-python-34b-v1.0.Q6_K - Acceleration: CPU AMD EPYC 7R13 48-Core - NVIDIA GeForce RTX 4090 24GB 1.0643 2.1286 3.1929 4.2572 5.3215 SE +/- 0.01, N = 3 4.73
POV-Ray Trace Time OpenBenchmarking.org Seconds, Fewer Is Better POV-Ray 3.7.0.7 Trace Time AMD EPYC 7R13 48-Core 3 6 9 12 15 SE +/- 0.02, N = 3 10.62 1. (CXX) g++ options: -pipe -O3 -ffast-math -march=native -lXpm -lSM -lICE -lX11 -ltiff -ljpeg -lpng -lz -lrt -lm -lboost_thread -lboost_system
Redis Test: GET - Parallel Connections: 50 OpenBenchmarking.org Requests Per Second, More Is Better Redis 7.0.4 Test: GET - Parallel Connections: 50 AMD EPYC 7R13 48-Core - NVIDIA GeForce RTX 4090 24GB 600K 1200K 1800K 2400K 3000K SE +/- 19715.89, N = 3 2792000.50 1. (CXX) g++ options: -MM -MT -g3 -fvisibility=hidden -O3
Redis Test: GET - Parallel Connections: 1000 OpenBenchmarking.org Requests Per Second, More Is Better Redis 7.0.4 Test: GET - Parallel Connections: 1000 AMD EPYC 7R13 48-Core - NVIDIA GeForce RTX 4090 24GB 500K 1000K 1500K 2000K 2500K SE +/- 38739.75, N = 15 2505995.40 1. (CXX) g++ options: -MM -MT -g3 -fvisibility=hidden -O3
Redis Test: SET - Parallel Connections: 1000 OpenBenchmarking.org Requests Per Second, More Is Better Redis 7.0.4 Test: SET - Parallel Connections: 1000 AMD EPYC 7R13 48-Core - NVIDIA GeForce RTX 4090 24GB 400K 800K 1200K 1600K 2000K SE +/- 20463.94, N = 12 2055844.83 1. (CXX) g++ options: -MM -MT -g3 -fvisibility=hidden -O3
Redis Test: LPOP - Parallel Connections: 1000 OpenBenchmarking.org Requests Per Second, More Is Better Redis 7.0.4 Test: LPOP - Parallel Connections: 1000 AMD EPYC 7R13 48-Core - NVIDIA GeForce RTX 4090 24GB 400K 800K 1200K 1600K 2000K SE +/- 11821.21, N = 12 1679590.47 1. (CXX) g++ options: -MM -MT -g3 -fvisibility=hidden -O3
Redis Test: SADD - Parallel Connections: 1000 OpenBenchmarking.org Requests Per Second, More Is Better Redis 7.0.4 Test: SADD - Parallel Connections: 1000 AMD EPYC 7R13 48-Core - NVIDIA GeForce RTX 4090 24GB 500K 1000K 1500K 2000K 2500K SE +/- 33076.13, N = 15 2268077.83 1. (CXX) g++ options: -MM -MT -g3 -fvisibility=hidden -O3
Redis Test: LPUSH - Parallel Connections: 1000 OpenBenchmarking.org Requests Per Second, More Is Better Redis 7.0.4 Test: LPUSH - Parallel Connections: 1000 AMD EPYC 7R13 48-Core - NVIDIA GeForce RTX 4090 24GB 400K 800K 1200K 1600K 2000K SE +/- 22632.71, N = 15 1744859.50 1. (CXX) g++ options: -MM -MT -g3 -fvisibility=hidden -O3
Phoronix Test Suite v10.8.4