m7g.8xlarge

amazon testing on Ubuntu 22.04 via the Phoronix Test Suite.

HTML result view exported from: https://openbenchmarking.org/result/2407011-NE-M7G8XLARG85&grw.

m7g.8xlargeProcessorMotherboardChipsetMemoryDiskNetworkOSKernelVulkanCompilerFile-SystemSystem Layerm7g.8xlargeARMv8 Neoverse-V1 (32 Cores)Amazon EC2 m7g.8xlarge (1.0 BIOS)Amazon Device 0200128GB537GB Amazon Elastic Block StoreAmazon ElasticUbuntu 22.046.5.0-1017-aws (aarch64)1.3.255GCC 11.4.0ext4amazonOpenBenchmarking.org- Transparent Huge Pages: madvise- --build=aarch64-linux-gnu --disable-libquadmath --disable-libquadmath-support --disable-werror --enable-bootstrap --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-fix-cortex-a53-843419 --enable-gnu-unique-object --enable-languages=c,ada,c++,go,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-link-serialization=2 --enable-multiarch --enable-nls --enable-objc-gc=auto --enable-plugin --enable-shared --enable-threads=posix --host=aarch64-linux-gnu --program-prefix=aarch64-linux-gnu- --target=aarch64-linux-gnu --with-build-config=bootstrap-lto-lean --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-target-system-zlib=auto -v - gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_rstack_overflow: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of __user pointer sanitization + spectre_v2: Mitigation of CSV2 BHB + srbds: Not affected + tsx_async_abort: Not affected

m7g.8xlargewhisper-cpp: ggml-base.en - 2016 State of the Unionwhisper-cpp: ggml-small.en - 2016 State of the Unionwhisper-cpp: ggml-medium.en - 2016 State of the Unionllama-cpp: Meta-Llama-3-8B-Instruct-Q8_0.ggufm7g.8xlarge81.60950179.73014439.6117022.47OpenBenchmarking.org

Whisper.cpp

Model: ggml-base.en - Input: 2016 State of the Union

OpenBenchmarking.orgSeconds, Fewer Is BetterWhisper.cpp 1.6.2Model: ggml-base.en - Input: 2016 State of the Unionm7g.8xlarge20406080100SE +/- 0.69, N = 1581.611. (CXX) g++ options: -O3 -std=c++11 -fPIC -pthread -mcpu=native

Whisper.cpp

Model: ggml-small.en - Input: 2016 State of the Union

OpenBenchmarking.orgSeconds, Fewer Is BetterWhisper.cpp 1.6.2Model: ggml-small.en - Input: 2016 State of the Unionm7g.8xlarge4080120160200SE +/- 2.02, N = 4179.731. (CXX) g++ options: -O3 -std=c++11 -fPIC -pthread -mcpu=native

Whisper.cpp

Model: ggml-medium.en - Input: 2016 State of the Union

OpenBenchmarking.orgSeconds, Fewer Is BetterWhisper.cpp 1.6.2Model: ggml-medium.en - Input: 2016 State of the Unionm7g.8xlarge100200300400500SE +/- 5.49, N = 9439.611. (CXX) g++ options: -O3 -std=c++11 -fPIC -pthread -mcpu=native

Llama.cpp

Model: Meta-Llama-3-8B-Instruct-Q8_0.gguf

OpenBenchmarking.orgTokens Per Second, More Is BetterLlama.cpp b3067Model: Meta-Llama-3-8B-Instruct-Q8_0.ggufm7g.8xlarge510152025SE +/- 0.14, N = 322.471. (CXX) g++ options: -std=c++11 -fPIC -O3 -pthread -mcpu=native -lopenblas

Phoronix Test Suite v10.8.5