llama

AMD Ryzen 7 7840U testing with a Framework FRANMDCP07 (03.03 BIOS) and AMD Phoenix1 512MB on Ubuntu 23.10 via the Phoronix Test Suite.

HTML result view exported from: https://openbenchmarking.org/result/2401119-NE-LLAMA444079&grr.

llamaProcessorMotherboardChipsetMemoryDiskGraphicsAudioNetworkOSKernelDesktopDisplay ServerOpenGLCompilerFile-SystemScreen ResolutionabcdAMD Ryzen 7 7840U @ 5.13GHz (8 Cores / 16 Threads)Framework FRANMDCP07 (03.03 BIOS)AMD Device 14e82 x 8 GB DRAM-5600MT/s AD5S56008G-B512GB Western Digital WD PC SN740 SDDPNQD-512GAMD Phoenix1 512MB (2700/2800MHz)AMD Rembrandt Radeon HD AudioMEDIATEK MT7922 802.11ax PCIUbuntu 23.106.7.0-060700rc5-generic (x86_64)GNOME Shell 45.1X Server 1.21.1.7 + Wayland4.6 Mesa 24.0~git2312160600.5d937f~oibaf~m (git-5d937f0 2023-12-16 mantic-oibaf-ppa) (LLVM 16.0.6 DRM 3.56)GCC 13.2.0ext42256x1504OpenBenchmarking.orgKernel Details- Transparent Huge Pages: madviseCompiler Details- --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-link-serialization=2 --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-defaulted --enable-offload-targets=nvptx-none=/build/gcc-13-XYspKM/gcc-13-13.2.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-13-XYspKM/gcc-13-13.2.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-build-config=bootstrap-lto-lean --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v Processor Details- Scaling Governor: amd-pstate-epp powersave (EPP: performance) - Platform Profile: balanced - CPU Microcode: 0xa704103 - ACPI Profile: balanced Security Details- gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_rstack_overflow: Vulnerable: Safe RET no microcode + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Enhanced / Automatic IBRS IBPB: conditional STIBP: always-on RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected

llamacachebench: Readcachebench: Read / Modify / Writecachebench: Writellama-cpp: llama-2-13b.Q4_0.ggufllama-cpp: llama-2-7b.Q4_0.ggufabcd12396.27548131556.50111672497.8562957.1113.3212270.096071131367.67014272913.1620707.1513.5912319.498226130628.2741772157.679857.1813.4812381.073447131333.36296172501.8283017.2913.5OpenBenchmarking.org

CacheBench

Test: Read

OpenBenchmarking.orgMB/s, More Is BetterCacheBenchTest: Readabcd3K6K9K12K15KSE +/- 18.01, N = 312396.2812270.1012319.5012381.07MIN: 12367.08 / MAX: 12496.62MIN: 12226.09 / MAX: 12316.73MIN: 12310.3 / MAX: 12325.17MIN: 12375.59 / MAX: 12381.241. (CC) gcc options: -O3 -lrt

CacheBench

Test: Read / Modify / Write

OpenBenchmarking.orgMB/s, More Is BetterCacheBenchTest: Read / Modify / Writeabcd30K60K90K120K150KSE +/- 334.41, N = 3131556.50131367.67130628.27131333.36MIN: 104085.06 / MAX: 142061.03MIN: 103002.69 / MAX: 142719.87MIN: 103222.92 / MAX: 141269.42MIN: 105232.15 / MAX: 142024.311. (CC) gcc options: -O3 -lrt

CacheBench

Test: Write

OpenBenchmarking.orgMB/s, More Is BetterCacheBenchTest: Writeabcd16K32K48K64K80KSE +/- 19.79, N = 372497.8672913.1672157.6872501.83MIN: 72039.23 / MAX: 72756.09MIN: 72419.43 / MAX: 73142.21MIN: 71728.42 / MAX: 72349.37MIN: 72040.49 / MAX: 72760.161. (CC) gcc options: -O3 -lrt

Llama.cpp

Model: llama-2-13b.Q4_0.gguf

OpenBenchmarking.orgTokens Per Second, More Is BetterLlama.cpp b1808Model: llama-2-13b.Q4_0.ggufabcd246810SE +/- 0.01, N = 37.117.157.187.291. (CXX) g++ options: -std=c++11 -fPIC -O3 -pthread -march=native -mtune=native -lopenblas

Llama.cpp

Model: llama-2-7b.Q4_0.gguf

OpenBenchmarking.orgTokens Per Second, More Is BetterLlama.cpp b1808Model: llama-2-7b.Q4_0.ggufabcd3691215SE +/- 0.15, N = 313.3213.5913.4813.501. (CXX) g++ options: -std=c++11 -fPIC -O3 -pthread -march=native -mtune=native -lopenblas

Phoronix Test Suite v10.8.5