9575F smoke Benchmarks for a future article. AMD EPYC 9575F 64-Core testing with a Supermicro Super Server H13SSL-N v1.01 (3.0 BIOS) and ASPEED on Ubuntu 24.10 via the Phoronix Test Suite. a: Processor: AMD EPYC 9575F 64-Core @ 3.30GHz (64 Cores / 128 Threads), Motherboard: Supermicro Super Server H13SSL-N v1.01 (3.0 BIOS), Chipset: AMD 1Ah, Memory: 12 x 64GB DDR5-6000MT/s Micron MTC40F2046S1RC64BDY QSFF, Disk: 3201GB Micron_7450_MTFDKCB3T2TFS, Graphics: ASPEED, Network: 2 x Broadcom NetXtreme BCM5720 PCIe OS: Ubuntu 24.10, Kernel: 6.12.0-rc7-linux-pm-next-phx (x86_64), Desktop: GNOME Shell 47.0, Display Server: X Server, Compiler: GCC 14.2.0, File-System: ext4, Screen Resolution: 1024x768 b: Processor: AMD EPYC 9575F 64-Core @ 3.30GHz (64 Cores / 128 Threads), Motherboard: Supermicro Super Server H13SSL-N v1.01 (3.0 BIOS), Chipset: AMD 1Ah, Memory: 12 x 64GB DDR5-6000MT/s Micron MTC40F2046S1RC64BDY QSFF, Disk: 3201GB Micron_7450_MTFDKCB3T2TFS, Graphics: ASPEED, Network: 2 x Broadcom NetXtreme BCM5720 PCIe OS: Ubuntu 24.10, Kernel: 6.12.0-rc7-linux-pm-next-phx (x86_64), Desktop: GNOME Shell 47.0, Display Server: X Server, Compiler: GCC 14.2.0, File-System: ext4, Screen Resolution: 1024x768 LiteRT 2024-10-15 Model: DeepLab V3 Microseconds < Lower Is Better a . 16007.5 |============================================================ b . 17473.4 |================================================================== LiteRT 2024-10-15 Model: Mobilenet Quant Microseconds < Lower Is Better a . 19415.0 |================================================================== b . 18266.4 |============================================================== Apache Cassandra 5.0 Test: Writes Op/s > Higher Is Better a . 464181 |=============================================================== b . 491661 |=================================================================== Llama.cpp b4154 Backend: CPU BLAS - Model: Llama-3.1-Tulu-3-8B-Q8_0 - Test: Prompt Processing 512 Tokens Per Second > Higher Is Better a . 313.79 |=================================================================== b . 296.41 |=============================================================== ASTC Encoder 5.0 Preset: Fast MT/s > Higher Is Better a . 1077.78 |=============================================================== b . 1135.12 |================================================================== Laghos 3.1 Test: Triple Point Problem Major Kernels Total Rate > Higher Is Better a . 308.92 |=================================================================== b . 295.65 |================================================================ LiteRT 2024-10-15 Model: Quantized COCO SSD MobileNet v1 Microseconds < Lower Is Better a . 8744.91 |=============================================================== b . 9116.47 |================================================================== OpenVINO GenAI 2024.5 Model: Falcon-7b-instruct-int4-ov - Device: CPU tokens/s > Higher Is Better a . 57.82 |==================================================================== b . 55.78 |================================================================== LiteRT 2024-10-15 Model: Inception ResNet V2 Microseconds < Lower Is Better a . 39371.7 |================================================================== b . 38276.5 |================================================================ Llama.cpp b4154 Backend: CPU BLAS - Model: Llama-3.1-Tulu-3-8B-Q8_0 - Test: Prompt Processing 1024 Tokens Per Second > Higher Is Better a . 334.84 |=================================================================== b . 325.90 |================================================================= Llama.cpp b4154 Backend: CPU BLAS - Model: Llama-3.1-Tulu-3-8B-Q8_0 - Test: Prompt Processing 2048 Tokens Per Second > Higher Is Better a . 319.54 |================================================================= b . 328.07 |=================================================================== XNNPACK b7b048 Model: FP16MobileNetV2 us < Lower Is Better a . 4692 |=================================================================== b . 4809 |===================================================================== OpenVINO 2024.5 Model: Road Segmentation ADAS FP16 - Device: CPU ms < Lower Is Better a . 17.60 |==================================================================== b . 17.18 |================================================================== OpenVINO 2024.5 Model: Road Segmentation ADAS FP16 - Device: CPU FPS > Higher Is Better a . 1812.32 |================================================================ b . 1856.61 |================================================================== Laghos 3.1 Test: Sedov Blast Wave, ube_922_hex.mesh Major Kernels Total Rate > Higher Is Better a . 579.09 |=================================================================== b . 567.84 |================================================================== Llama.cpp b4154 Backend: CPU BLAS - Model: Mistral-7B-Instruct-v0.3-Q8_0 - Test: Prompt Processing 2048 Tokens Per Second > Higher Is Better a . 321.82 |================================================================== b . 327.81 |=================================================================== Llama.cpp b4154 Backend: CPU BLAS - Model: granite-3.0-3b-a800m-instruct-Q8_0 - Test: Prompt Processing 512 Tokens Per Second > Higher Is Better a . 1013.03 |================================================================== b . 998.20 |================================================================= LiteRT 2024-10-15 Model: NASNet Mobile Microseconds < Lower Is Better a . 218197 |================================================================== b . 221184 |=================================================================== SVT-AV1 2.3 Encoder Mode: Preset 13 - Input: Bosphorus 1080p Frames Per Second > Higher Is Better a . 1299.73 |================================================================= b . 1316.95 |================================================================== Llama.cpp b4154 Backend: CPU BLAS - Model: Mistral-7B-Instruct-v0.3-Q8_0 - Test: Prompt Processing 1024 Tokens Per Second > Higher Is Better a . 335.62 |=================================================================== b . 331.38 |================================================================== OpenVINO 2024.5 Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPU FPS > Higher Is Better a . 166477.75 |================================================================ b . 164424.12 |=============================================================== NAMD 3.0 Input: STMV with 1,066,628 Atoms ns/day > Higher Is Better a . 3.75360 |================================================================= b . 3.79737 |================================================================== LiteRT 2024-10-15 Model: SqueezeNet Microseconds < Lower Is Better a . 3814.42 |================================================================= b . 3857.68 |================================================================== Palabos 2.3 Grid Size: 100 Mega Site Updates Per Second > Higher Is Better a . 812.29 |=================================================================== b . 804.12 |================================================================== Blender 4.3 Blend File: BMW27 - Compute: CPU-Only Seconds < Lower Is Better a . 15.15 |==================================================================== b . 15.01 |=================================================================== OpenVINO GenAI 2024.5 Model: Gemma-7b-int4-ov - Device: CPU tokens/s > Higher Is Better a . 49.16 |=================================================================== b . 49.61 |==================================================================== SVT-AV1 2.3 Encoder Mode: Preset 8 - Input: Bosphorus 1080p Frames Per Second > Higher Is Better a . 529.47 |================================================================== b . 534.21 |=================================================================== OpenVINO GenAI 2024.5 Model: Phi-3-mini-128k-instruct-int4-ov - Device: CPU tokens/s > Higher Is Better a . 65.33 |=================================================================== b . 65.91 |==================================================================== OpenVINO 2024.5 Model: Person Detection FP16 - Device: CPU FPS > Higher Is Better a . 656.29 |=================================================================== b . 651.86 |=================================================================== OpenVINO 2024.5 Model: Person Detection FP16 - Device: CPU ms < Lower Is Better a . 48.68 |==================================================================== b . 49.01 |==================================================================== OpenVINO 2024.5 Model: Face Detection Retail FP16 - Device: CPU FPS > Higher Is Better a . 15364.86 |================================================================= b . 15466.24 |================================================================= SVT-AV1 2.3 Encoder Mode: Preset 13 - Input: Bosphorus 4K Frames Per Second > Higher Is Better a . 481.97 |=================================================================== b . 478.84 |=================================================================== XNNPACK b7b048 Model: FP32MobileNetV1 us < Lower Is Better a . 2417 |===================================================================== b . 2432 |===================================================================== Blender 4.3 Blend File: Classroom - Compute: CPU-Only Seconds < Lower Is Better a . 41.71 |==================================================================== b . 41.46 |==================================================================== Blender 4.3 Blend File: Junkshop - Compute: CPU-Only Seconds < Lower Is Better a . 20.13 |==================================================================== b . 20.25 |==================================================================== OpenVINO 2024.5 Model: Vehicle Detection FP16 - Device: CPU ms < Lower Is Better a . 6.86 |===================================================================== b . 6.90 |===================================================================== OpenVINO 2024.5 Model: Vehicle Detection FP16 - Device: CPU FPS > Higher Is Better a . 4634.75 |================================================================== b . 4609.70 |================================================================== XNNPACK b7b048 Model: FP16MobileNetV3Small us < Lower Is Better a . 5446 |===================================================================== b . 5417 |===================================================================== BYTE Unix Benchmark 5.1.3-git Computational Test: Dhrystone 2 LPS > Higher Is Better a . 6028934961.9 |============================================================= b . 6060068455.1 |============================================================= OpenVINO 2024.5 Model: Face Detection Retail FP16 - Device: CPU ms < Lower Is Better a . 2.03 |===================================================================== b . 2.02 |===================================================================== Palabos 2.3 Grid Size: 500 Mega Site Updates Per Second > Higher Is Better a . 776.31 |=================================================================== b . 772.50 |=================================================================== NAMD 3.0 Input: ATPase with 327,506 Atoms ns/day > Higher Is Better a . 12.59 |==================================================================== b . 12.65 |==================================================================== OpenVINO 2024.5 Model: Road Segmentation ADAS FP16-INT8 - Device: CPU FPS > Higher Is Better a . 2316.75 |================================================================== b . 2326.58 |================================================================== OpenVINO 2024.5 Model: Face Detection Retail FP16-INT8 - Device: CPU FPS > Higher Is Better a . 22034.49 |================================================================= b . 21941.81 |================================================================= Llama.cpp b4154 Backend: CPU BLAS - Model: Mistral-7B-Instruct-v0.3-Q8_0 - Test: Text Generation 128 Tokens Per Second > Higher Is Better a . 54.23 |==================================================================== b . 54.01 |==================================================================== XNNPACK b7b048 Model: FP32MobileNetV2 us < Lower Is Better a . 4725 |===================================================================== b . 4743 |===================================================================== Llama.cpp b4154 Backend: CPU BLAS - Model: Llama-3.1-Tulu-3-8B-Q8_0 - Test: Text Generation 128 Tokens Per Second > Higher Is Better a . 51.84 |==================================================================== b . 51.65 |==================================================================== OpenVINO 2024.5 Model: Road Segmentation ADAS FP16-INT8 - Device: CPU ms < Lower Is Better a . 13.76 |==================================================================== b . 13.71 |==================================================================== Llama.cpp b4154 Backend: CPU BLAS - Model: granite-3.0-3b-a800m-instruct-Q8_0 - Test: Prompt Processing 2048 Tokens Per Second > Higher Is Better a . 965.44 |=================================================================== b . 962.07 |=================================================================== Timed Eigen Compilation 3.4.0 Time To Compile Seconds < Lower Is Better a . 28.47 |==================================================================== b . 28.37 |==================================================================== Llama.cpp b4154 Backend: CPU BLAS - Model: granite-3.0-3b-a800m-instruct-Q8_0 - Test: Text Generation 128 Tokens Per Second > Higher Is Better a . 120.87 |=================================================================== b . 121.27 |=================================================================== XNNPACK b7b048 Model: QS8MobileNetV2 us < Lower Is Better a . 5041 |===================================================================== b . 5057 |===================================================================== Blender 4.3 Blend File: Pabellon Barcelona - Compute: CPU-Only Seconds < Lower Is Better a . 47.43 |==================================================================== b . 47.58 |==================================================================== XNNPACK b7b048 Model: FP32MobileNetV3Large us < Lower Is Better a . 7242 |===================================================================== b . 7220 |===================================================================== OpenVINO 2024.5 Model: Age Gender Recognition Retail 0013 FP16 - Device: CPU FPS > Higher Is Better a . 137029.68 |================================================================ b . 137431.92 |================================================================ OpenVINO 2024.5 Model: Person Re-Identification Retail FP16 - Device: CPU ms < Lower Is Better a . 3.41 |===================================================================== b . 3.42 |===================================================================== OpenVINO 2024.5 Model: Vehicle Detection FP16-INT8 - Device: CPU FPS > Higher Is Better a . 7561.85 |================================================================== b . 7540.16 |================================================================== BYTE Unix Benchmark 5.1.3-git Computational Test: Pipe LPS > Higher Is Better a . 379708473.5 |============================================================== b . 378638216.8 |============================================================== Palabos 2.3 Grid Size: 400 Mega Site Updates Per Second > Higher Is Better a . 734.24 |=================================================================== b . 732.41 |=================================================================== XNNPACK b7b048 Model: FP16MobileNetV3Large us < Lower Is Better a . 7099 |===================================================================== b . 7082 |===================================================================== OpenVINO 2024.5 Model: Vehicle Detection FP16-INT8 - Device: CPU ms < Lower Is Better a . 4.21 |===================================================================== b . 4.22 |===================================================================== LiteRT 2024-10-15 Model: Inception V4 Microseconds < Lower Is Better a . 24673.3 |================================================================== b . 24727.4 |================================================================== OpenVINO 2024.5 Model: Weld Porosity Detection FP16-INT8 - Device: CPU ms < Lower Is Better a . 4.64 |===================================================================== b . 4.63 |===================================================================== Primesieve 12.6 Length: 1e13 Seconds < Lower Is Better a . 24.78 |==================================================================== b . 24.83 |==================================================================== OpenVINO 2024.5 Model: Handwritten English Recognition FP16-INT8 - Device: CPU ms < Lower Is Better a . 20.55 |==================================================================== b . 20.59 |==================================================================== SVT-AV1 2.3 Encoder Mode: Preset 3 - Input: Bosphorus 1080p Frames Per Second > Higher Is Better a . 44.81 |==================================================================== b . 44.72 |==================================================================== LiteRT 2024-10-15 Model: Mobilenet Float Microseconds < Lower Is Better a . 2429.06 |================================================================== b . 2424.82 |================================================================== OpenVINO 2024.5 Model: Handwritten English Recognition FP16-INT8 - Device: CPU FPS > Higher Is Better a . 3111.82 |================================================================== b . 3106.45 |================================================================== SVT-AV1 2.3 Encoder Mode: Preset 5 - Input: Bosphorus 1080p Frames Per Second > Higher Is Better a . 147.93 |=================================================================== b . 148.16 |=================================================================== OpenVINO 2024.5 Model: Weld Porosity Detection FP16-INT8 - Device: CPU FPS > Higher Is Better a . 13617.58 |================================================================= b . 13596.14 |================================================================= OpenVINO 2024.5 Model: Person Re-Identification Retail FP16 - Device: CPU FPS > Higher Is Better a . 9322.35 |================================================================== b . 9309.66 |================================================================== SVT-AV1 2.3 Encoder Mode: Preset 8 - Input: Bosphorus 4K Frames Per Second > Higher Is Better a . 202.15 |=================================================================== b . 202.43 |=================================================================== XNNPACK b7b048 Model: FP32MobileNetV3Small us < Lower Is Better a . 5413 |===================================================================== b . 5406 |===================================================================== OpenVINO 2024.5 Model: Face Detection FP16 - Device: CPU FPS > Higher Is Better a . 70.11 |==================================================================== b . 70.02 |==================================================================== OpenVINO 2024.5 Model: Face Detection FP16-INT8 - Device: CPU FPS > Higher Is Better a . 140.59 |=================================================================== b . 140.42 |=================================================================== ASTC Encoder 5.0 Preset: Exhaustive MT/s > Higher Is Better a . 6.2161 |=================================================================== b . 6.2233 |=================================================================== OpenVINO 2024.5 Model: Noise Suppression Poconet-Like FP16 - Device: CPU ms < Lower Is Better a . 9.56 |===================================================================== b . 9.55 |===================================================================== Primesieve 12.6 Length: 1e12 Seconds < Lower Is Better a . 1.973 |==================================================================== b . 1.975 |==================================================================== OpenVINO 2024.5 Model: Noise Suppression Poconet-Like FP16 - Device: CPU FPS > Higher Is Better a . 6485.97 |================================================================== b . 6492.43 |================================================================== ASTC Encoder 5.0 Preset: Thorough MT/s > Higher Is Better a . 72.41 |==================================================================== b . 72.34 |==================================================================== OpenVINO 2024.5 Model: Handwritten English Recognition FP16 - Device: CPU ms < Lower Is Better a . 20.93 |==================================================================== b . 20.95 |==================================================================== Llama.cpp b4154 Backend: CPU BLAS - Model: granite-3.0-3b-a800m-instruct-Q8_0 - Test: Prompt Processing 1024 Tokens Per Second > Higher Is Better a . 1055.54 |================================================================== b . 1056.46 |================================================================== Llama.cpp b4154 Backend: CPU BLAS - Model: Mistral-7B-Instruct-v0.3-Q8_0 - Test: Prompt Processing 512 Tokens Per Second > Higher Is Better a . 317.19 |=================================================================== b . 317.46 |=================================================================== OpenVINO 2024.5 Model: Weld Porosity Detection FP16 - Device: CPU FPS > Higher Is Better a . 6826.87 |================================================================== b . 6821.09 |================================================================== OpenVINO 2024.5 Model: Face Detection FP16-INT8 - Device: CPU ms < Lower Is Better a . 227.21 |=================================================================== b . 227.40 |=================================================================== ASTC Encoder 5.0 Preset: Medium MT/s > Higher Is Better a . 514.67 |=================================================================== b . 514.24 |=================================================================== OpenVINO 2024.5 Model: Person Detection FP32 - Device: CPU ms < Lower Is Better a . 48.88 |==================================================================== b . 48.92 |==================================================================== OpenVINO 2024.5 Model: Person Detection FP32 - Device: CPU FPS > Higher Is Better a . 653.60 |=================================================================== b . 653.07 |=================================================================== OpenVINO 2024.5 Model: Face Detection FP16 - Device: CPU ms < Lower Is Better a . 455.23 |=================================================================== b . 455.56 |=================================================================== SVT-AV1 2.3 Encoder Mode: Preset 3 - Input: Bosphorus 4K Frames Per Second > Higher Is Better a . 17.03 |==================================================================== b . 17.04 |==================================================================== SVT-AV1 2.3 Encoder Mode: Preset 5 - Input: Bosphorus 4K Frames Per Second > Higher Is Better a . 60.73 |==================================================================== b . 60.77 |==================================================================== OpenVINO 2024.5 Model: Handwritten English Recognition FP16 - Device: CPU FPS > Higher Is Better a . 3054.87 |================================================================== b . 3053.00 |================================================================== ACES DGEMM 1.0 Sustained Floating-Point Rate GFLOP/s > Higher Is Better a . 5201.06 |================================================================== b . 5203.83 |================================================================== Blender 4.3 Blend File: Fishy Cat - Compute: CPU-Only Seconds < Lower Is Better a . 21.26 |==================================================================== b . 21.27 |==================================================================== XNNPACK b7b048 Model: FP16MobileNetV1 us < Lower Is Better a . 2470 |===================================================================== b . 2471 |===================================================================== OpenVINO 2024.5 Model: Person Vehicle Bike Detection FP16 - Device: CPU FPS > Higher Is Better a . 6219.36 |================================================================== b . 6221.54 |================================================================== OpenVINO 2024.5 Model: Machine Translation EN To DE FP16 - Device: CPU FPS > Higher Is Better a . 781.09 |=================================================================== b . 781.32 |=================================================================== ASTC Encoder 5.0 Preset: Very Thorough MT/s > Higher Is Better a . 10.12 |==================================================================== b . 10.13 |==================================================================== OpenVINO GenAI 2024.5 Model: TinyLlama-1.1B-Chat-v1.0 - Device: CPU tokens/s > Higher Is Better a . 77.27 |==================================================================== b . 77.25 |==================================================================== OpenVINO 2024.5 Model: Machine Translation EN To DE FP16 - Device: CPU ms < Lower Is Better a . 40.94 |==================================================================== b . 40.93 |==================================================================== BYTE Unix Benchmark 5.1.3-git Computational Test: System Call LPS > Higher Is Better a . 360905676.5 |============================================================== b . 360867935.4 |============================================================== Blender 4.3 Blend File: Barbershop - Compute: CPU-Only Seconds < Lower Is Better a . 148.59 |=================================================================== b . 148.60 |=================================================================== BYTE Unix Benchmark 5.1.3-git Computational Test: Whetstone Double MWIPS > Higher Is Better a . 1440313.3 |================================================================ b . 1440301.8 |================================================================ OpenVINO 2024.5 Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPU ms < Lower Is Better a . 0.26 |===================================================================== b . 0.26 |===================================================================== OpenVINO 2024.5 Model: Age Gender Recognition Retail 0013 FP16 - Device: CPU ms < Lower Is Better a . 0.34 |===================================================================== b . 0.34 |===================================================================== OpenVINO 2024.5 Model: Person Vehicle Bike Detection FP16 - Device: CPU ms < Lower Is Better a . 5.09 |===================================================================== b . 5.09 |===================================================================== OpenVINO 2024.5 Model: Face Detection Retail FP16-INT8 - Device: CPU ms < Lower Is Better a . 2.83 |===================================================================== b . 2.83 |===================================================================== OpenVINO 2024.5 Model: Weld Porosity Detection FP16 - Device: CPU ms < Lower Is Better a . 9.31 |===================================================================== b . 9.31 |===================================================================== OpenVINO GenAI 2024.5 Model: Phi-3-mini-128k-instruct-int4-ov - Device: CPU - Time Per Output Token ms < Lower Is Better a . 15.31 |==================================================================== b . 15.17 |=================================================================== OpenVINO GenAI 2024.5 Model: Phi-3-mini-128k-instruct-int4-ov - Device: CPU - Time To First Token ms < Lower Is Better a . 23.34 |==================================================================== b . 21.91 |================================================================ OpenVINO GenAI 2024.5 Model: Falcon-7b-instruct-int4-ov - Device: CPU - Time Per Output Token ms < Lower Is Better a . 17.30 |================================================================== b . 17.93 |==================================================================== OpenVINO GenAI 2024.5 Model: Falcon-7b-instruct-int4-ov - Device: CPU - Time To First Token ms < Lower Is Better a . 31.28 |==================================================================== b . 29.27 |================================================================ OpenVINO GenAI 2024.5 Model: TinyLlama-1.1B-Chat-v1.0 - Device: CPU - Time Per Output Token ms < Lower Is Better a . 12.94 |==================================================================== b . 12.95 |==================================================================== OpenVINO GenAI 2024.5 Model: TinyLlama-1.1B-Chat-v1.0 - Device: CPU - Time To First Token ms < Lower Is Better a . 15.82 |==================================================================== b . 15.90 |==================================================================== OpenVINO GenAI 2024.5 Model: Gemma-7b-int4-ov - Device: CPU - Time Per Output Token ms < Lower Is Better a . 20.34 |==================================================================== b . 20.16 |=================================================================== OpenVINO GenAI 2024.5 Model: Gemma-7b-int4-ov - Device: CPU - Time To First Token ms < Lower Is Better a . 30.36 |==================================================================== b . 30.16 |====================================================================