m7g.8xlarge

amazon testing on Ubuntu 22.04 via the Phoronix Test Suite.

HTML result view exported from: https://openbenchmarking.org/result/2407011-NE-M7G8XLARG93&gru.

OpenVINO

Model: Face Detection FP16 - Device: CPU

OpenVINO

Model: Person Detection FP16 - Device: CPU

OpenVINO

Model: Person Detection FP32 - Device: CPU

OpenVINO

Model: Vehicle Detection FP16 - Device: CPU

OpenVINO

Model: Face Detection FP16-INT8 - Device: CPU

OpenVINO

Model: Face Detection Retail FP16 - Device: CPU

OpenVINO

Model: Road Segmentation ADAS FP16 - Device: CPU

OpenVINO

Model: Vehicle Detection FP16-INT8 - Device: CPU

OpenVINO

Model: Weld Porosity Detection FP16 - Device: CPU

OpenVINO

Model: Face Detection Retail FP16-INT8 - Device: CPU

OpenVINO

Model: Road Segmentation ADAS FP16-INT8 - Device: CPU

OpenVINO

Model: Machine Translation EN To DE FP16 - Device: CPU

OpenVINO

Model: Weld Porosity Detection FP16-INT8 - Device: CPU

OpenVINO

Model: Person Vehicle Bike Detection FP16 - Device: CPU

OpenVINO

Model: Noise Suppression Poconet-Like FP16 - Device: CPU

OpenVINO

Model: Handwritten English Recognition FP16 - Device: CPU

OpenVINO

Model: Person Re-Identification Retail FP16 - Device: CPU

OpenVINO

Model: Age Gender Recognition Retail 0013 FP16 - Device: CPU

OpenVINO

Model: Handwritten English Recognition FP16-INT8 - Device: CPU

OpenVINO

Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPU

ONNX Runtime

Model: GPT-2 - Device: CPU - Executor: Parallel

ONNX Runtime

Model: GPT-2 - Device: CPU - Executor: Standard

ONNX Runtime

Model: yolov4 - Device: CPU - Executor: Parallel

ONNX Runtime

Model: yolov4 - Device: CPU - Executor: Standard

ONNX Runtime

Model: T5 Encoder - Device: CPU - Executor: Parallel

ONNX Runtime

Model: T5 Encoder - Device: CPU - Executor: Standard

ONNX Runtime

Model: bertsquad-12 - Device: CPU - Executor: Parallel

ONNX Runtime

Model: bertsquad-12 - Device: CPU - Executor: Standard

ONNX Runtime

Model: CaffeNet 12-int8 - Device: CPU - Executor: Parallel

ONNX Runtime

Model: CaffeNet 12-int8 - Device: CPU - Executor: Standard

ONNX Runtime

Model: fcn-resnet101-11 - Device: CPU - Executor: Parallel

ONNX Runtime

Model: fcn-resnet101-11 - Device: CPU - Executor: Standard

ONNX Runtime

Model: ArcFace ResNet-100 - Device: CPU - Executor: Parallel

ONNX Runtime

Model: ArcFace ResNet-100 - Device: CPU - Executor: Standard

ONNX Runtime

Model: ResNet50 v1-12-int8 - Device: CPU - Executor: Parallel

ONNX Runtime

Model: ResNet50 v1-12-int8 - Device: CPU - Executor: Standard

ONNX Runtime

Model: super-resolution-10 - Device: CPU - Executor: Parallel

ONNX Runtime

Model: super-resolution-10 - Device: CPU - Executor: Standard

ONNX Runtime

Model: Faster R-CNN R-50-FPN-int8 - Device: CPU - Executor: Parallel

ONNX Runtime

Model: Faster R-CNN R-50-FPN-int8 - Device: CPU - Executor: Standard

Llama.cpp

Model: Meta-Llama-3-8B-Instruct-Q8_0.gguf

ONNX Runtime

Model: GPT-2 - Device: CPU - Executor: Parallel

ONNX Runtime

Model: GPT-2 - Device: CPU - Executor: Standard

ONNX Runtime

Model: yolov4 - Device: CPU - Executor: Parallel

ONNX Runtime

Model: yolov4 - Device: CPU - Executor: Standard

ONNX Runtime

Model: T5 Encoder - Device: CPU - Executor: Parallel

ONNX Runtime

Model: T5 Encoder - Device: CPU - Executor: Standard

ONNX Runtime

Model: bertsquad-12 - Device: CPU - Executor: Parallel

ONNX Runtime

Model: bertsquad-12 - Device: CPU - Executor: Standard

ONNX Runtime

Model: CaffeNet 12-int8 - Device: CPU - Executor: Parallel

ONNX Runtime

Model: CaffeNet 12-int8 - Device: CPU - Executor: Standard

ONNX Runtime

Model: fcn-resnet101-11 - Device: CPU - Executor: Parallel

ONNX Runtime

Model: fcn-resnet101-11 - Device: CPU - Executor: Standard

ONNX Runtime

Model: ArcFace ResNet-100 - Device: CPU - Executor: Parallel

ONNX Runtime

Model: ArcFace ResNet-100 - Device: CPU - Executor: Standard

ONNX Runtime

Model: ResNet50 v1-12-int8 - Device: CPU - Executor: Parallel

ONNX Runtime

Model: ResNet50 v1-12-int8 - Device: CPU - Executor: Standard

ONNX Runtime

Model: super-resolution-10 - Device: CPU - Executor: Parallel

ONNX Runtime

Model: super-resolution-10 - Device: CPU - Executor: Standard

ONNX Runtime

Model: Faster R-CNN R-50-FPN-int8 - Device: CPU - Executor: Parallel

ONNX Runtime

Model: Faster R-CNN R-50-FPN-int8 - Device: CPU - Executor: Standard

OpenVINO

Model: Face Detection FP16 - Device: CPU

OpenVINO

Model: Person Detection FP16 - Device: CPU

OpenVINO

Model: Person Detection FP32 - Device: CPU

OpenVINO

Model: Vehicle Detection FP16 - Device: CPU

OpenVINO

Model: Face Detection FP16-INT8 - Device: CPU

OpenVINO

Model: Face Detection Retail FP16 - Device: CPU

OpenVINO

Model: Road Segmentation ADAS FP16 - Device: CPU

OpenVINO

Model: Vehicle Detection FP16-INT8 - Device: CPU

OpenVINO

Model: Weld Porosity Detection FP16 - Device: CPU

OpenVINO

Model: Face Detection Retail FP16-INT8 - Device: CPU

OpenVINO

Model: Road Segmentation ADAS FP16-INT8 - Device: CPU

OpenVINO

Model: Machine Translation EN To DE FP16 - Device: CPU

OpenVINO

Model: Weld Porosity Detection FP16-INT8 - Device: CPU

OpenVINO

Model: Person Vehicle Bike Detection FP16 - Device: CPU

OpenVINO

Model: Noise Suppression Poconet-Like FP16 - Device: CPU

OpenVINO

Model: Handwritten English Recognition FP16 - Device: CPU

OpenVINO

Model: Person Re-Identification Retail FP16 - Device: CPU

OpenVINO

Model: Age Gender Recognition Retail 0013 FP16 - Device: CPU

OpenVINO

Model: Handwritten English Recognition FP16-INT8 - Device: CPU

OpenVINO

Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPU

Whisper.cpp

Model: ggml-base.en - Input: 2016 State of the Union

Whisper.cpp

Model: ggml-small.en - Input: 2016 State of the Union

Whisper.cpp

Model: ggml-medium.en - Input: 2016 State of the Union

Phoronix Test Suite v10.8.5