nlp-benchmarks
c6i.2xlarge NLP benchmarking on AL2023


c6i.2xlarge: 

	Processor: Intel Xeon Platinum 8375C (4 Cores / 8 Threads), Motherboard: Amazon EC2 c6i.2xlarge (1.0 BIOS), Chipset: Intel 440FX 82441FX PMC, Memory: 1 x 16GB DDR4-3200MT/s, Disk: 215GB Amazon Elastic Block Store, Network: Amazon Elastic

	OS: Amazon Linux 2023, Kernel: 6.1.61-85.141.amzn2023.x86_64 (x86_64), Compiler: GCC 11.4.1 20230605, File-System: xfs, System Layer: amazon


oneDNN 3.3
Harness: Convolution Batch Shapes Auto - Data Type: f32 - Engine: CPU
ms < Lower Is Better
c6i.2xlarge . 5.93669 |========================================================


oneDNN 3.3
Harness: Deconvolution Batch shapes_1d - Data Type: f32 - Engine: CPU
ms < Lower Is Better
c6i.2xlarge . 12.66 |==========================================================


oneDNN 3.3
Harness: Deconvolution Batch shapes_3d - Data Type: f32 - Engine: CPU
ms < Lower Is Better
c6i.2xlarge . 8.09803 |========================================================


oneDNN 3.3
Harness: Convolution Batch Shapes Auto - Data Type: u8s8f32 - Engine: CPU
ms < Lower Is Better
c6i.2xlarge . 6.24391 |========================================================


oneDNN 3.3
Harness: Deconvolution Batch shapes_1d - Data Type: u8s8f32 - Engine: CPU
ms < Lower Is Better
c6i.2xlarge . 2.03836 |========================================================


oneDNN 3.3
Harness: Deconvolution Batch shapes_3d - Data Type: u8s8f32 - Engine: CPU
ms < Lower Is Better
c6i.2xlarge . 1.77286 |========================================================


oneDNN 3.3
Harness: Recurrent Neural Network Inference - Data Type: f32 - Engine: CPU
ms < Lower Is Better
c6i.2xlarge . 2496.84 |========================================================


oneDNN 3.3
Harness: Convolution Batch Shapes Auto - Data Type: bf16bf16bf16 - Engine: CPU
ms < Lower Is Better
c6i.2xlarge . 33.19 |==========================================================


oneDNN 3.3
Harness: Deconvolution Batch shapes_1d - Data Type: bf16bf16bf16 - Engine: CPU
ms < Lower Is Better
c6i.2xlarge . 46.70 |==========================================================


oneDNN 3.3
Harness: Deconvolution Batch shapes_3d - Data Type: bf16bf16bf16 - Engine: CPU
ms < Lower Is Better
c6i.2xlarge . 34.72 |==========================================================


oneDNN 3.3
Harness: Recurrent Neural Network Inference - Data Type: u8s8f32 - Engine: CPU
ms < Lower Is Better
c6i.2xlarge . 2492.29 |========================================================


oneDNN 3.3
Harness: Recurrent Neural Network Inference - Data Type: bf16bf16bf16 - Engine: CPU
ms < Lower Is Better
c6i.2xlarge . 2501.50 |========================================================


Numpy Benchmark

Score > Higher Is Better
c6i.2xlarge . 374.99 |=========================================================


PyTorch 2.1
Device: CPU - Batch Size: 1 - Model: ResNet-50
batches/sec > Higher Is Better
c6i.2xlarge . 26.78 |==========================================================


PyTorch 2.1
Device: CPU - Batch Size: 1 - Model: ResNet-152
batches/sec > Higher Is Better
c6i.2xlarge . 10.57 |==========================================================


PyTorch 2.1
Device: CPU - Batch Size: 16 - Model: ResNet-50
batches/sec > Higher Is Better
c6i.2xlarge . 15.96 |==========================================================


PyTorch 2.1
Device: CPU - Batch Size: 32 - Model: ResNet-50
batches/sec > Higher Is Better
c6i.2xlarge . 15.81 |==========================================================


PyTorch 2.1
Device: CPU - Batch Size: 16 - Model: ResNet-152
batches/sec > Higher Is Better
c6i.2xlarge . 6.36 |===========================================================


PyTorch 2.1
Device: CPU - Batch Size: 32 - Model: ResNet-152
batches/sec > Higher Is Better
c6i.2xlarge . 6.38 |===========================================================


PyTorch 2.1
Device: CPU - Batch Size: 1 - Model: Efficientnet_v2_l
batches/sec > Higher Is Better
c6i.2xlarge . 7.99 |===========================================================


PyTorch 2.1
Device: CPU - Batch Size: 16 - Model: Efficientnet_v2_l
batches/sec > Higher Is Better
c6i.2xlarge . 4.06 |===========================================================


PyTorch 2.1
Device: CPU - Batch Size: 32 - Model: Efficientnet_v2_l
batches/sec > Higher Is Better
c6i.2xlarge . 4.04 |===========================================================


OpenVINO 2023.2.dev
Model: Face Detection FP16 - Device: CPU
FPS > Higher Is Better
c6i.2xlarge . 1.77 |===========================================================


OpenVINO 2023.2.dev
Model: Face Detection FP16 - Device: CPU
ms < Lower Is Better
c6i.2xlarge . 2252.04 |========================================================


OpenVINO 2023.2.dev
Model: Face Detection FP16-INT8 - Device: CPU
FPS > Higher Is Better
c6i.2xlarge . 6.53 |===========================================================


OpenVINO 2023.2.dev
Model: Face Detection FP16-INT8 - Device: CPU
ms < Lower Is Better
c6i.2xlarge . 610.59 |=========================================================


OpenVINO 2023.2.dev
Model: Machine Translation EN To DE FP16 - Device: CPU
FPS > Higher Is Better
c6i.2xlarge . 22.26 |==========================================================


OpenVINO 2023.2.dev
Model: Machine Translation EN To DE FP16 - Device: CPU
ms < Lower Is Better
c6i.2xlarge . 179.53 |=========================================================


PyBench 2018-02-16
Total For Average Test Times
Milliseconds < Lower Is Better
c6i.2xlarge . 1000 |===========================================================