2888 Intel Xeon E E-2488 testing with a Supermicro Super Server X13SCL-F v0123456789 (1.1 BIOS) and llvmpipe on Ubuntu 22.04 via the Phoronix Test Suite. a: Processor: Intel Xeon E E-2488 @ 3.20GHz (8 Cores / 16 Threads), Motherboard: Supermicro Super Server X13SCL-F v0123456789 (1.1 BIOS), Chipset: Intel Device 7a27, Memory: 2 x 32GB DRAM-4400MT/s Micron MTC20C2085S1EC48BA1, Disk: 960GB Micron_7450_MTFDKBA960TFR + 3201GB Micron_7450_MTFDKCC3T2TFS, Graphics: llvmpipe, Monitor: VE228, Network: 2 x Intel I210 OS: Ubuntu 22.04, Kernel: 6.2.0-26-generic (x86_64), Desktop: GNOME Shell 42.9, Display Server: X Server 1.21.1.4, OpenGL: 4.5 Mesa 23.0.4-0ubuntu1~22.04.1 (LLVM 15.0.7 256 bits), Vulkan: 1.3.238, Compiler: GCC 11.4.0, File-System: ext4, Screen Resolution: 1920x1080 b: Processor: Intel Xeon E E-2488 @ 3.20GHz (8 Cores / 16 Threads), Motherboard: Supermicro Super Server X13SCL-F v0123456789 (1.1 BIOS), Chipset: Intel Device 7a27, Memory: 2 x 32GB DRAM-4400MT/s Micron MTC20C2085S1EC48BA1, Disk: 960GB Micron_7450_MTFDKBA960TFR + 3201GB Micron_7450_MTFDKCC3T2TFS, Graphics: llvmpipe, Monitor: VE228, Network: 2 x Intel I210 OS: Ubuntu 22.04, Kernel: 6.2.0-26-generic (x86_64), Desktop: GNOME Shell 42.9, Display Server: X Server 1.21.1.4, OpenGL: 4.5 Mesa 23.0.4-0ubuntu1~22.04.1 (LLVM 15.0.7 256 bits), Vulkan: 1.3.238, Compiler: GCC 11.4.0, File-System: ext4, Screen Resolution: 1920x1080 c: Processor: Intel Xeon E E-2488 @ 3.20GHz (8 Cores / 16 Threads), Motherboard: Supermicro Super Server X13SCL-F v0123456789 (1.1 BIOS), Chipset: Intel Device 7a27, Memory: 2 x 32GB DRAM-4400MT/s Micron MTC20C2085S1EC48BA1, Disk: 960GB Micron_7450_MTFDKBA960TFR + 3201GB Micron_7450_MTFDKCC3T2TFS, Graphics: llvmpipe, Monitor: VE228, Network: 2 x Intel I210 OS: Ubuntu 22.04, Kernel: 6.2.0-26-generic (x86_64), Desktop: GNOME Shell 42.9, Display Server: X Server 1.21.1.4, OpenGL: 4.5 Mesa 23.0.4-0ubuntu1~22.04.1 (LLVM 15.0.7 256 bits), Vulkan: 1.3.238, Compiler: GCC 11.4.0, File-System: ext4, Screen Resolution: 1920x1080 d: Processor: Intel Xeon E E-2488 @ 3.20GHz (8 Cores / 16 Threads), Motherboard: Supermicro Super Server X13SCL-F v0123456789 (1.1 BIOS), Chipset: Intel Device 7a27, Memory: 2 x 32GB DRAM-4400MT/s Micron MTC20C2085S1EC48BA1, Disk: 960GB Micron_7450_MTFDKBA960TFR + 3201GB Micron_7450_MTFDKCC3T2TFS, Graphics: llvmpipe, Monitor: VE228, Network: 2 x Intel I210 OS: Ubuntu 22.04, Kernel: 6.2.0-26-generic (x86_64), Desktop: GNOME Shell 42.9, Display Server: X Server 1.21.1.4, OpenGL: 4.5 Mesa 23.0.4-0ubuntu1~22.04.1 (LLVM 15.0.7 256 bits), Vulkan: 1.3.238, Compiler: GCC 11.4.0, File-System: ext4, Screen Resolution: 1920x1080 Quicksilver 20230818 Input: CTS2 Figure Of Merit > Higher Is Better a . 12930000 |================================================================= b . 12920000 |================================================================= c . 12900000 |================================================================= d . 12680000 |================================================================ Quicksilver 20230818 Input: CORAL2 P1 Figure Of Merit > Higher Is Better a . 13320000 |================================================================= b . 12430000 |============================================================ c . 13410000 |================================================================= d . 13400000 |================================================================= Quicksilver 20230818 Input: CORAL2 P2 Figure Of Merit > Higher Is Better a . 19000000 |================================================================= b . 18700000 |================================================================ c . 18460000 |=============================================================== d . 18730000 |================================================================ OpenVINO 2024.0 Model: Face Detection FP16 - Device: CPU FPS > Higher Is Better a . 2.67 |===================================================================== b . 2.67 |===================================================================== c . 2.66 |===================================================================== d . 2.66 |===================================================================== OpenVINO 2024.0 Model: Person Detection FP16 - Device: CPU FPS > Higher Is Better a . 29.03 |=================================================================== b . 28.91 |================================================================== c . 29.64 |==================================================================== d . 28.96 |================================================================== OpenVINO 2024.0 Model: Person Detection FP32 - Device: CPU FPS > Higher Is Better a . 29.09 |=================================================================== b . 29.74 |==================================================================== c . 29.17 |=================================================================== d . 28.76 |================================================================== OpenVINO 2024.0 Model: Vehicle Detection FP16 - Device: CPU FPS > Higher Is Better a . 242.67 |=================================================================== b . 238.98 |================================================================== c . 239.66 |================================================================== d . 239.78 |================================================================== OpenVINO 2024.0 Model: Face Detection FP16-INT8 - Device: CPU FPS > Higher Is Better a . 10.78 |==================================================================== b . 10.71 |==================================================================== c . 10.59 |=================================================================== d . 10.62 |=================================================================== OpenVINO 2024.0 Model: Face Detection Retail FP16 - Device: CPU FPS > Higher Is Better a . 815.19 |=================================================================== b . 816.66 |=================================================================== c . 809.39 |================================================================== d . 778.95 |================================================================ OpenVINO 2024.0 Model: Road Segmentation ADAS FP16 - Device: CPU FPS > Higher Is Better a . 93.39 |================================================================= b . 97.98 |==================================================================== c . 91.07 |=============================================================== d . 86.33 |============================================================ OpenVINO 2024.0 Model: Vehicle Detection FP16-INT8 - Device: CPU FPS > Higher Is Better a . 502.11 |=================================================================== b . 502.15 |=================================================================== c . 499.31 |=================================================================== d . 494.26 |================================================================== OpenVINO 2024.0 Model: Weld Porosity Detection FP16 - Device: CPU FPS > Higher Is Better a . 272.34 |================================================================== b . 271.84 |================================================================== c . 275.68 |=================================================================== d . 273.91 |=================================================================== OpenVINO 2024.0 Model: Face Detection Retail FP16-INT8 - Device: CPU FPS > Higher Is Better a . 1638.80 |================================================================== b . 1635.40 |================================================================= c . 1626.27 |================================================================= d . 1649.85 |================================================================== OpenVINO 2024.0 Model: Road Segmentation ADAS FP16-INT8 - Device: CPU FPS > Higher Is Better a . 217.38 |================================================================== b . 213.64 |================================================================= c . 219.22 |=================================================================== d . 214.62 |================================================================== OpenVINO 2024.0 Model: Machine Translation EN To DE FP16 - Device: CPU FPS > Higher Is Better a . 33.55 |==================================================================== b . 33.51 |==================================================================== c . 33.72 |==================================================================== d . 33.55 |==================================================================== OpenVINO 2024.0 Model: Weld Porosity Detection FP16-INT8 - Device: CPU FPS > Higher Is Better a . 1036.92 |================================================================= b . 1052.53 |================================================================== c . 1039.07 |================================================================= d . 1047.18 |================================================================== OpenVINO 2024.0 Model: Person Vehicle Bike Detection FP16 - Device: CPU FPS > Higher Is Better a . 384.46 |=================================================================== b . 383.87 |=================================================================== c . 385.24 |=================================================================== d . 384.29 |=================================================================== OpenVINO 2024.0 Model: Noise Suppression Poconet-Like FP16 - Device: CPU FPS > Higher Is Better a . 523.93 |================================================================== b . 528.03 |=================================================================== c . 524.06 |================================================================== d . 522.81 |================================================================== OpenVINO 2024.0 Model: Handwritten English Recognition FP16 - Device: CPU FPS > Higher Is Better a . 154.02 |=================================================================== b . 153.23 |================================================================== c . 154.34 |=================================================================== d . 154.54 |=================================================================== OpenVINO 2024.0 Model: Person Re-Identification Retail FP16 - Device: CPU FPS > Higher Is Better a . 441.47 |=================================================================== b . 440.94 |=================================================================== c . 437.87 |================================================================== d . 433.48 |================================================================== OpenVINO 2024.0 Model: Age Gender Recognition Retail 0013 FP16 - Device: CPU FPS > Higher Is Better a . 8604.26 |================================================================== b . 8497.74 |================================================================= c . 8581.04 |================================================================== d . 8570.68 |================================================================== OpenVINO 2024.0 Model: Handwritten English Recognition FP16-INT8 - Device: CPU FPS > Higher Is Better a . 180.54 |================================================================== b . 181.28 |================================================================== c . 180.48 |================================================================== d . 183.06 |=================================================================== OpenVINO 2024.0 Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPU FPS > Higher Is Better a . 21993.11 |================================================================= b . 21852.43 |================================================================= c . 21956.68 |================================================================= d . 21874.68 |================================================================= SVT-AV1 2.0 Encoder Mode: Preset 4 - Input: Bosphorus 4K Frames Per Second > Higher Is Better a . 4.314 |=================================================================== b . 4.296 |================================================================== c . 4.317 |=================================================================== d . 4.400 |==================================================================== SVT-AV1 2.0 Encoder Mode: Preset 8 - Input: Bosphorus 4K Frames Per Second > Higher Is Better a . 35.45 |=================================================================== b . 34.86 |================================================================== c . 35.32 |=================================================================== d . 35.93 |==================================================================== SVT-AV1 2.0 Encoder Mode: Preset 12 - Input: Bosphorus 4K Frames Per Second > Higher Is Better a . 115.91 |=================================================================== b . 115.02 |================================================================== c . 106.98 |============================================================== d . 116.37 |=================================================================== SVT-AV1 2.0 Encoder Mode: Preset 13 - Input: Bosphorus 4K Frames Per Second > Higher Is Better a . 116.36 |=================================================================== b . 103.49 |============================================================ c . 116.12 |=================================================================== d . 109.27 |=============================================================== SVT-AV1 2.0 Encoder Mode: Preset 4 - Input: Bosphorus 1080p Frames Per Second > Higher Is Better a . 16.20 |==================================================================== b . 15.97 |=================================================================== c . 16.23 |==================================================================== d . 15.93 |=================================================================== SVT-AV1 2.0 Encoder Mode: Preset 8 - Input: Bosphorus 1080p Frames Per Second > Higher Is Better a . 134.86 |================================================================== b . 135.94 |=================================================================== c . 136.52 |=================================================================== d . 120.66 |=========================================================== SVT-AV1 2.0 Encoder Mode: Preset 12 - Input: Bosphorus 1080p Frames Per Second > Higher Is Better a . 531.29 |=================================================================== b . 531.97 |=================================================================== c . 534.97 |=================================================================== d . 530.37 |================================================================== SVT-AV1 2.0 Encoder Mode: Preset 13 - Input: Bosphorus 1080p Frames Per Second > Higher Is Better a . 644.33 |=================================================================== b . 642.67 |=================================================================== c . 643.95 |=================================================================== d . 639.40 |================================================================== OSPRay 3.1 Benchmark: particle_volume/ao/real_time Items Per Second > Higher Is Better a . 6.00500 |================================================================== b . 5.98988 |================================================================== c . 5.98920 |================================================================== d . 5.99903 |================================================================== OSPRay 3.1 Benchmark: particle_volume/scivis/real_time Items Per Second > Higher Is Better a . 5.96546 |================================================================== b . 5.95732 |================================================================== c . 5.93355 |================================================================== d . 5.96271 |================================================================== OSPRay 3.1 Benchmark: particle_volume/pathtracer/real_time Items Per Second > Higher Is Better a . 163.10 |=================================================================== b . 163.00 |=================================================================== c . 162.49 |=================================================================== d . 163.09 |=================================================================== OSPRay 3.1 Benchmark: gravity_spheres_volume/dim_512/ao/real_time Items Per Second > Higher Is Better a . 2.78974 |================================================================= b . 2.80995 |================================================================== c . 2.80782 |================================================================== d . 2.82308 |================================================================== OSPRay 3.1 Benchmark: gravity_spheres_volume/dim_512/scivis/real_time Items Per Second > Higher Is Better a . 2.65458 |================================================================= b . 2.68163 |================================================================= c . 2.70994 |================================================================== d . 2.71491 |================================================================== OSPRay 3.1 Benchmark: gravity_spheres_volume/dim_512/pathtracer/real_time Items Per Second > Higher Is Better a . 3.87077 |================================================================== b . 3.87307 |================================================================== c . 3.86452 |================================================================== d . 3.86617 |================================================================== Neural Magic DeepSparse 1.7 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 9.1491 |=================================================================== b . 8.9375 |================================================================= c . 9.0093 |================================================================== d . 9.0640 |================================================================== Neural Magic DeepSparse 1.7 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 9.2711 |=================================================================== b . 9.1535 |================================================================== c . 9.1718 |================================================================== d . 9.2635 |=================================================================== Neural Magic DeepSparse 1.7 Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 341.04 |=================================================================== b . 335.62 |================================================================== c . 334.36 |================================================================== d . 339.19 |=================================================================== Neural Magic DeepSparse 1.7 Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 259.54 |================================================================== b . 260.95 |================================================================== c . 260.45 |================================================================== d . 264.12 |=================================================================== Neural Magic DeepSparse 1.7 Model: ResNet-50, Baseline - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 110.53 |=================================================================== b . 110.13 |=================================================================== c . 109.65 |================================================================== d . 110.53 |=================================================================== Neural Magic DeepSparse 1.7 Model: ResNet-50, Baseline - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 99.59 |==================================================================== b . 99.53 |==================================================================== c . 99.58 |==================================================================== d . 99.68 |==================================================================== Neural Magic DeepSparse 1.7 Model: ResNet-50, Sparse INT8 - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 814.33 |================================================================== b . 817.87 |=================================================================== c . 805.79 |================================================================== d . 822.16 |=================================================================== Neural Magic DeepSparse 1.7 Model: ResNet-50, Sparse INT8 - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 686.16 |=================================================================== b . 687.06 |=================================================================== c . 676.40 |================================================================== d . 683.35 |=================================================================== Neural Magic DeepSparse 1.7 Model: Llama2 Chat 7b Quantized - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 5.0827 |=================================================================== b . 5.0978 |=================================================================== c . 5.1016 |=================================================================== d . 5.0893 |=================================================================== Neural Magic DeepSparse 1.7 Model: Llama2 Chat 7b Quantized - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 8.0241 |=================================================================== b . 8.0350 |=================================================================== c . 8.0302 |=================================================================== d . 8.0335 |=================================================================== Neural Magic DeepSparse 1.7 Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 111.26 |=================================================================== b . 109.51 |================================================================== c . 109.03 |================================================================== d . 110.86 |=================================================================== Neural Magic DeepSparse 1.7 Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 99.55 |==================================================================== b . 99.55 |==================================================================== c . 99.51 |==================================================================== d . 99.66 |==================================================================== Neural Magic DeepSparse 1.7 Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 55.24 |==================================================================== b . 55.35 |==================================================================== c . 54.92 |=================================================================== d . 55.33 |==================================================================== Neural Magic DeepSparse 1.7 Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 53.62 |==================================================================== b . 53.92 |==================================================================== c . 53.59 |==================================================================== d . 53.77 |==================================================================== Neural Magic DeepSparse 1.7 Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 79.56 |==================================================================== b . 79.42 |==================================================================== c . 79.75 |==================================================================== d . 79.49 |==================================================================== Neural Magic DeepSparse 1.7 Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 68.67 |==================================================================== b . 68.97 |==================================================================== c . 68.85 |==================================================================== d . 68.89 |==================================================================== Neural Magic DeepSparse 1.7 Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 11.22 |==================================================================== b . 11.19 |==================================================================== c . 11.23 |==================================================================== d . 11.23 |==================================================================== Neural Magic DeepSparse 1.7 Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 10.64 |==================================================================== b . 10.67 |==================================================================== c . 10.67 |==================================================================== d . 10.62 |==================================================================== Neural Magic DeepSparse 1.7 Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 156.14 |=================================================================== b . 151.15 |================================================================= c . 152.44 |================================================================= d . 153.72 |================================================================== Neural Magic DeepSparse 1.7 Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 102.80 |================================================================== b . 102.61 |================================================================== c . 102.38 |================================================================== d . 104.35 |=================================================================== Neural Magic DeepSparse 1.7 Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 9.0749 |=================================================================== b . 8.9956 |================================================================== c . 9.0183 |=================================================================== d . 8.9861 |================================================================== Neural Magic DeepSparse 1.7 Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 9.2610 |================================================================== b . 9.1867 |================================================================== c . 9.2712 |=================================================================== d . 9.3361 |=================================================================== 7-Zip Compression 22.01 Test: Compression Rating MIPS > Higher Is Better a . 96642 |==================================================================== b . 97323 |==================================================================== c . 97159 |==================================================================== d . 97044 |==================================================================== 7-Zip Compression 22.01 Test: Decompression Rating MIPS > Higher Is Better a . 67998 |==================================================================== b . 67818 |==================================================================== c . 67661 |==================================================================== d . 67622 |==================================================================== Stockfish 16.1 Chess Benchmark Nodes Per Second > Higher Is Better a . 17776692 |================================================================= b . 16895156 |============================================================== c . 17408986 |================================================================ d . 17015035 |============================================================== GROMACS 2024 Implementation: MPI CPU - Input: water_GMX50_bare Ns Per Day > Higher Is Better a . 1.405 |==================================================================== b . 1.406 |==================================================================== c . 1.408 |==================================================================== d . 1.405 |==================================================================== NAMD 3.0b6 Input: ATPase with 327,506 Atoms ns/day > Higher Is Better a . 0.67383 |================================================================== b . 0.67382 |================================================================== c . 0.67334 |================================================================== d . 0.65844 |================================================================ NAMD 3.0b6 Input: STMV with 1,066,628 Atoms ns/day > Higher Is Better a . 0.20897 |================================================================== b . 0.20952 |================================================================== c . 0.20891 |================================================================== d . 0.20937 |================================================================== RocksDB 9.0 Test: Overwrite Op/s > Higher Is Better a . 1485724 |================================================================== b . 1468149 |================================================================= c . 1452056 |================================================================= d . 1455423 |================================================================= RocksDB 9.0 Test: Random Fill Op/s > Higher Is Better a . 1449625 |================================================================= b . 1468080 |================================================================== c . 1466579 |================================================================== d . 1447290 |================================================================= RocksDB 9.0 Test: Random Read Op/s > Higher Is Better a . 56724663 |================================================================ b . 57366733 |================================================================= c . 57492080 |================================================================= d . 56385081 |================================================================ RocksDB 9.0 Test: Update Random Op/s > Higher Is Better a . 940801 |=================================================================== b . 928889 |================================================================== c . 929396 |================================================================== d . 939218 |=================================================================== RocksDB 9.0 Test: Sequential Fill Op/s > Higher Is Better a . 2001345 |================================================================ b . 2076954 |================================================================== c . 2042355 |================================================================= d . 2008846 |================================================================ RocksDB 9.0 Test: Random Fill Sync Op/s > Higher Is Better a . 206255 |=================================================================== b . 205948 |=================================================================== c . 206635 |=================================================================== d . 206350 |=================================================================== RocksDB 9.0 Test: Read While Writing Op/s > Higher Is Better a . 2258480 |============================================================== b . 2252397 |============================================================== c . 2374402 |================================================================= d . 2392670 |================================================================== RocksDB 9.0 Test: Read Random Write Random Op/s > Higher Is Better a . 2236460 |================================================================== b . 2219780 |================================================================== c . 2202547 |================================================================= d . 2202307 |================================================================= Speedb 2.7 Test: Random Fill Op/s > Higher Is Better a . 883180 |=================================================================== b . 883375 |=================================================================== c . 875583 |================================================================== d . 880777 |=================================================================== Speedb 2.7 Test: Random Read Op/s > Higher Is Better a . 60086996 |================================================================= b . 60307089 |================================================================= c . 59761930 |================================================================ d . 58376227 |=============================================================== Speedb 2.7 Test: Update Random Op/s > Higher Is Better a . 741879 |=================================================================== b . 736698 |================================================================== c . 743285 |=================================================================== d . 734814 |================================================================== Speedb 2.7 Test: Sequential Fill Op/s > Higher Is Better a . 1057203 |================================================================== b . 863184 |====================================================== c . 1045594 |================================================================= d . 1064769 |================================================================== Speedb 2.7 Test: Random Fill Sync Op/s > Higher Is Better a . 232803 |=================================================================== b . 231177 |=================================================================== c . 231664 |=================================================================== d . 232392 |=================================================================== Speedb 2.7 Test: Read While Writing Op/s > Higher Is Better a . 2986868 |============================================================== b . 2992041 |============================================================== c . 3165497 |================================================================== d . 3036556 |=============================================================== Speedb 2.7 Test: Read Random Write Random Op/s > Higher Is Better a . 2449782 |================================================================== b . 2396499 |================================================================= c . 2405977 |================================================================= d . 2421096 |================================================================= BRL-CAD 7.38.2 VGR Performance Metric VGR Performance Metric > Higher Is Better a . 187668 |=================================================================== b . 188736 |=================================================================== c . 188971 |=================================================================== d . 188430 |=================================================================== Chaos Group V-RAY 6.0 Mode: CPU vsamples > Higher Is Better a . 20309 |=================================================================== b . 20474 |==================================================================== c . 20359 |==================================================================== d . 20326 |==================================================================== oneDNN 3.4 Harness: IP Shapes 1D - Engine: CPU ms < Lower Is Better a . 2.89586 |================================================================== b . 2.80998 |================================================================ c . 2.90020 |================================================================== d . 2.81256 |================================================================ oneDNN 3.4 Harness: IP Shapes 3D - Engine: CPU ms < Lower Is Better a . 7.84533 |================================================================== b . 7.83920 |================================================================== c . 7.84377 |================================================================== d . 7.84734 |================================================================== oneDNN 3.4 Harness: Convolution Batch Shapes Auto - Engine: CPU ms < Lower Is Better a . 6.42439 |================================================================== b . 6.42538 |================================================================== c . 6.43553 |================================================================== d . 6.40691 |================================================================== oneDNN 3.4 Harness: Deconvolution Batch shapes_1d - Engine: CPU ms < Lower Is Better a . 7.18796 |================================================================== b . 6.44395 |=========================================================== c . 6.03052 |======================================================= d . 6.67469 |============================================================= oneDNN 3.4 Harness: Deconvolution Batch shapes_3d - Engine: CPU ms < Lower Is Better a . 4.54099 |================================================================== b . 4.53763 |================================================================== c . 4.54541 |================================================================== d . 4.53315 |================================================================== oneDNN 3.4 Harness: Recurrent Neural Network Training - Engine: CPU ms < Lower Is Better a . 2817.81 |================================================================== b . 2811.37 |================================================================== c . 2808.20 |================================================================== d . 2814.22 |================================================================== oneDNN 3.4 Harness: Recurrent Neural Network Inference - Engine: CPU ms < Lower Is Better a . 1487.64 |================================================================== b . 1485.18 |================================================================== c . 1487.05 |================================================================== d . 1487.19 |================================================================== OSPRay Studio 1.0 Camera: 1 - Resolution: 4K - Samples Per Pixel: 1 - Renderer: Path Tracer - Acceleration: CPU ms < Lower Is Better a . 9569 |===================================================================== b . 9578 |===================================================================== c . 9568 |===================================================================== d . 9578 |===================================================================== OSPRay Studio 1.0 Camera: 2 - Resolution: 4K - Samples Per Pixel: 1 - Renderer: Path Tracer - Acceleration: CPU ms < Lower Is Better a . 9730 |===================================================================== b . 9720 |===================================================================== c . 9722 |===================================================================== d . 9727 |===================================================================== OSPRay Studio 1.0 Camera: 3 - Resolution: 4K - Samples Per Pixel: 1 - Renderer: Path Tracer - Acceleration: CPU ms < Lower Is Better a . 11351 |==================================================================== b . 11328 |==================================================================== c . 11328 |==================================================================== d . 11327 |==================================================================== OSPRay Studio 1.0 Camera: 1 - Resolution: 4K - Samples Per Pixel: 16 - Renderer: Path Tracer - Acceleration: CPU ms < Lower Is Better a . 155844 |=================================================================== b . 156271 |=================================================================== c . 156366 |=================================================================== d . 156265 |=================================================================== OSPRay Studio 1.0 Camera: 1 - Resolution: 4K - Samples Per Pixel: 32 - Renderer: Path Tracer - Acceleration: CPU ms < Lower Is Better a . 310533 |=================================================================== b . 310594 |=================================================================== c . 310458 |=================================================================== d . 310409 |=================================================================== OSPRay Studio 1.0 Camera: 2 - Resolution: 4K - Samples Per Pixel: 16 - Renderer: Path Tracer - Acceleration: CPU ms < Lower Is Better a . 157835 |=================================================================== b . 158659 |=================================================================== c . 157963 |=================================================================== d . 158320 |=================================================================== OSPRay Studio 1.0 Camera: 2 - Resolution: 4K - Samples Per Pixel: 32 - Renderer: Path Tracer - Acceleration: CPU ms < Lower Is Better a . 315995 |=================================================================== b . 314965 |=================================================================== c . 315016 |=================================================================== d . 315601 |=================================================================== OSPRay Studio 1.0 Camera: 3 - Resolution: 4K - Samples Per Pixel: 16 - Renderer: Path Tracer - Acceleration: CPU ms < Lower Is Better a . 184602 |=================================================================== b . 184046 |=================================================================== c . 184423 |=================================================================== d . 184006 |=================================================================== OSPRay Studio 1.0 Camera: 3 - Resolution: 4K - Samples Per Pixel: 32 - Renderer: Path Tracer - Acceleration: CPU ms < Lower Is Better a . 367733 |=================================================================== b . 367376 |=================================================================== c . 367235 |=================================================================== d . 366681 |=================================================================== OSPRay Studio 1.0 Camera: 1 - Resolution: 1080p - Samples Per Pixel: 1 - Renderer: Path Tracer - Acceleration: CPU ms < Lower Is Better a . 2412 |===================================================================== b . 2415 |===================================================================== c . 2413 |===================================================================== d . 2414 |===================================================================== OSPRay Studio 1.0 Camera: 2 - Resolution: 1080p - Samples Per Pixel: 1 - Renderer: Path Tracer - Acceleration: CPU ms < Lower Is Better a . 2454 |===================================================================== b . 2448 |===================================================================== c . 2448 |===================================================================== d . 2448 |===================================================================== OSPRay Studio 1.0 Camera: 3 - Resolution: 1080p - Samples Per Pixel: 1 - Renderer: Path Tracer - Acceleration: CPU ms < Lower Is Better a . 2858 |===================================================================== b . 2860 |===================================================================== c . 2857 |===================================================================== d . 2858 |===================================================================== OSPRay Studio 1.0 Camera: 1 - Resolution: 1080p - Samples Per Pixel: 16 - Renderer: Path Tracer - Acceleration: CPU ms < Lower Is Better a . 40686 |==================================================================== b . 40849 |==================================================================== c . 40759 |==================================================================== d . 40641 |==================================================================== OSPRay Studio 1.0 Camera: 1 - Resolution: 1080p - Samples Per Pixel: 32 - Renderer: Path Tracer - Acceleration: CPU ms < Lower Is Better a . 79824 |==================================================================== b . 79976 |==================================================================== c . 79629 |==================================================================== d . 79324 |=================================================================== OSPRay Studio 1.0 Camera: 2 - Resolution: 1080p - Samples Per Pixel: 16 - Renderer: Path Tracer - Acceleration: CPU ms < Lower Is Better a . 41235 |=================================================================== b . 41382 |==================================================================== c . 41560 |==================================================================== d . 40969 |=================================================================== OSPRay Studio 1.0 Camera: 2 - Resolution: 1080p - Samples Per Pixel: 32 - Renderer: Path Tracer - Acceleration: CPU ms < Lower Is Better a . 80830 |==================================================================== b . 80544 |==================================================================== c . 80517 |==================================================================== d . 80395 |==================================================================== OSPRay Studio 1.0 Camera: 3 - Resolution: 1080p - Samples Per Pixel: 16 - Renderer: Path Tracer - Acceleration: CPU ms < Lower Is Better a . 48103 |==================================================================== b . 48081 |==================================================================== c . 47805 |==================================================================== d . 47474 |=================================================================== OSPRay Studio 1.0 Camera: 3 - Resolution: 1080p - Samples Per Pixel: 32 - Renderer: Path Tracer - Acceleration: CPU ms < Lower Is Better a . 93632 |==================================================================== b . 94309 |==================================================================== c . 93566 |=================================================================== d . 93543 |=================================================================== OpenVINO 2024.0 Model: Face Detection FP16 - Device: CPU ms < Lower Is Better a . 1500.07 |================================================================== b . 1499.89 |================================================================== c . 1502.74 |================================================================== d . 1501.44 |================================================================== OpenVINO 2024.0 Model: Person Detection FP16 - Device: CPU ms < Lower Is Better a . 137.78 |=================================================================== b . 138.36 |=================================================================== c . 134.90 |================================================================= d . 138.10 |=================================================================== OpenVINO 2024.0 Model: Person Detection FP32 - Device: CPU ms < Lower Is Better a . 137.47 |================================================================== b . 134.46 |================================================================= c . 137.07 |================================================================== d . 139.07 |=================================================================== OpenVINO 2024.0 Model: Vehicle Detection FP16 - Device: CPU ms < Lower Is Better a . 16.47 |=================================================================== b . 16.73 |==================================================================== c . 16.68 |==================================================================== d . 16.67 |==================================================================== OpenVINO 2024.0 Model: Face Detection FP16-INT8 - Device: CPU ms < Lower Is Better a . 370.26 |================================================================== b . 372.98 |================================================================== c . 377.20 |=================================================================== d . 376.51 |=================================================================== OpenVINO 2024.0 Model: Face Detection Retail FP16 - Device: CPU ms < Lower Is Better a . 9.80 |================================================================= b . 9.79 |================================================================= c . 9.87 |================================================================= d . 10.26 |==================================================================== OpenVINO 2024.0 Model: Road Segmentation ADAS FP16 - Device: CPU ms < Lower Is Better a . 42.81 |=============================================================== b . 40.80 |============================================================ c . 43.90 |================================================================ d . 46.30 |==================================================================== OpenVINO 2024.0 Model: Vehicle Detection FP16-INT8 - Device: CPU ms < Lower Is Better a . 15.92 |=================================================================== b . 15.92 |=================================================================== c . 16.01 |=================================================================== d . 16.18 |==================================================================== OpenVINO 2024.0 Model: Weld Porosity Detection FP16 - Device: CPU ms < Lower Is Better a . 29.36 |==================================================================== b . 29.42 |==================================================================== c . 29.00 |=================================================================== d . 29.19 |=================================================================== OpenVINO 2024.0 Model: Face Detection Retail FP16-INT8 - Device: CPU ms < Lower Is Better a . 4.88 |==================================================================== b . 4.89 |===================================================================== c . 4.92 |===================================================================== d . 4.84 |==================================================================== OpenVINO 2024.0 Model: Road Segmentation ADAS FP16-INT8 - Device: CPU ms < Lower Is Better a . 36.78 |=================================================================== b . 37.43 |==================================================================== c . 36.47 |================================================================== d . 37.26 |==================================================================== OpenVINO 2024.0 Model: Machine Translation EN To DE FP16 - Device: CPU ms < Lower Is Better a . 119.20 |=================================================================== b . 119.34 |=================================================================== c . 118.60 |=================================================================== d . 119.22 |=================================================================== OpenVINO 2024.0 Model: Weld Porosity Detection FP16-INT8 - Device: CPU ms < Lower Is Better a . 7.71 |===================================================================== b . 7.60 |==================================================================== c . 7.69 |===================================================================== d . 7.63 |==================================================================== OpenVINO 2024.0 Model: Person Vehicle Bike Detection FP16 - Device: CPU ms < Lower Is Better a . 10.39 |==================================================================== b . 10.41 |==================================================================== c . 10.37 |==================================================================== d . 10.40 |==================================================================== OpenVINO 2024.0 Model: Noise Suppression Poconet-Like FP16 - Device: CPU ms < Lower Is Better a . 15.20 |==================================================================== b . 15.08 |=================================================================== c . 15.20 |==================================================================== d . 15.23 |==================================================================== OpenVINO 2024.0 Model: Handwritten English Recognition FP16 - Device: CPU ms < Lower Is Better a . 51.92 |==================================================================== b . 52.19 |==================================================================== c . 51.81 |==================================================================== d . 51.74 |=================================================================== OpenVINO 2024.0 Model: Person Re-Identification Retail FP16 - Device: CPU ms < Lower Is Better a . 18.11 |=================================================================== b . 18.13 |=================================================================== c . 18.26 |=================================================================== d . 18.44 |==================================================================== OpenVINO 2024.0 Model: Age Gender Recognition Retail 0013 FP16 - Device: CPU ms < Lower Is Better a . 0.93 |==================================================================== b . 0.94 |===================================================================== c . 0.93 |==================================================================== d . 0.93 |==================================================================== OpenVINO 2024.0 Model: Handwritten English Recognition FP16-INT8 - Device: CPU ms < Lower Is Better a . 44.30 |==================================================================== b . 44.11 |==================================================================== c . 44.30 |==================================================================== d . 43.68 |=================================================================== OpenVINO 2024.0 Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPU ms < Lower Is Better a . 0.36 |===================================================================== b . 0.36 |===================================================================== c . 0.36 |===================================================================== d . 0.36 |===================================================================== Neural Magic DeepSparse 1.7 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 437.19 |================================================================= b . 447.54 |=================================================================== c . 443.97 |================================================================== d . 441.29 |================================================================== Neural Magic DeepSparse 1.7 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 107.86 |================================================================== b . 109.24 |=================================================================== c . 109.02 |=================================================================== d . 107.95 |================================================================== Neural Magic DeepSparse 1.7 Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 11.72 |=================================================================== b . 11.91 |==================================================================== c . 11.95 |==================================================================== d . 11.78 |=================================================================== Neural Magic DeepSparse 1.7 Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 3.8485 |=================================================================== b . 3.8279 |=================================================================== c . 3.8345 |=================================================================== d . 3.7815 |================================================================== Neural Magic DeepSparse 1.7 Model: ResNet-50, Baseline - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 36.18 |=================================================================== b . 36.31 |==================================================================== c . 36.47 |==================================================================== d . 36.18 |=================================================================== Neural Magic DeepSparse 1.7 Model: ResNet-50, Baseline - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 10.04 |==================================================================== b . 10.04 |==================================================================== c . 10.04 |==================================================================== d . 10.03 |==================================================================== Neural Magic DeepSparse 1.7 Model: ResNet-50, Sparse INT8 - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 4.8981 |================================================================== b . 4.8763 |================================================================== c . 4.9502 |=================================================================== d . 4.8511 |================================================================== Neural Magic DeepSparse 1.7 Model: ResNet-50, Sparse INT8 - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 1.4526 |================================================================== b . 1.4499 |================================================================== c . 1.4736 |=================================================================== d . 1.4580 |================================================================== Neural Magic DeepSparse 1.7 Model: Llama2 Chat 7b Quantized - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 779.36 |=================================================================== b . 777.02 |=================================================================== c . 776.52 |=================================================================== d . 778.33 |=================================================================== Neural Magic DeepSparse 1.7 Model: Llama2 Chat 7b Quantized - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 124.61 |=================================================================== b . 124.44 |=================================================================== c . 124.51 |=================================================================== d . 124.46 |=================================================================== Neural Magic DeepSparse 1.7 Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 35.94 |=================================================================== b . 36.52 |==================================================================== c . 36.67 |==================================================================== d . 36.07 |=================================================================== Neural Magic DeepSparse 1.7 Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 10.04 |==================================================================== b . 10.04 |==================================================================== c . 10.04 |==================================================================== d . 10.03 |==================================================================== Neural Magic DeepSparse 1.7 Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 72.40 |==================================================================== b . 72.26 |=================================================================== c . 72.82 |==================================================================== d . 72.28 |=================================================================== Neural Magic DeepSparse 1.7 Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 18.64 |==================================================================== b . 18.54 |==================================================================== c . 18.66 |==================================================================== d . 18.59 |==================================================================== Neural Magic DeepSparse 1.7 Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 50.27 |==================================================================== b . 50.35 |==================================================================== c . 50.15 |==================================================================== d . 50.31 |==================================================================== Neural Magic DeepSparse 1.7 Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 14.56 |==================================================================== b . 14.50 |==================================================================== c . 14.52 |==================================================================== d . 14.51 |==================================================================== Neural Magic DeepSparse 1.7 Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 356.63 |=================================================================== b . 357.55 |=================================================================== c . 356.29 |=================================================================== d . 356.15 |=================================================================== Neural Magic DeepSparse 1.7 Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 93.93 |==================================================================== b . 93.67 |==================================================================== c . 93.73 |==================================================================== d . 94.16 |==================================================================== Neural Magic DeepSparse 1.7 Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 25.60 |================================================================== b . 26.45 |==================================================================== c . 26.22 |=================================================================== d . 26.01 |=================================================================== Neural Magic DeepSparse 1.7 Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 9.7178 |=================================================================== b . 9.7363 |=================================================================== c . 9.7584 |=================================================================== d . 9.5744 |================================================================== Neural Magic DeepSparse 1.7 Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 440.76 |================================================================== b . 444.65 |=================================================================== c . 443.53 |=================================================================== d . 445.12 |=================================================================== Neural Magic DeepSparse 1.7 Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 107.97 |================================================================== b . 108.85 |=================================================================== c . 107.85 |================================================================== d . 107.11 |================================================================== Timed Linux Kernel Compilation 6.8 Build: defconfig Seconds < Lower Is Better a . 93.03 |==================================================================== b . 93.23 |==================================================================== c . 92.62 |==================================================================== d . 91.84 |=================================================================== Timed Linux Kernel Compilation 6.8 Build: allmodconfig Seconds < Lower Is Better a . 1310.08 |================================================================== b . 1310.86 |================================================================== c . 1308.28 |================================================================== d . 1308.98 |================================================================== Primesieve 12.1 Length: 1e12 Seconds < Lower Is Better a . 20.27 |================================================================= b . 21.06 |==================================================================== c . 20.54 |================================================================== d . 19.96 |================================================================ Primesieve 12.1 Length: 1e13 Seconds < Lower Is Better a . 245.05 |=================================================================== b . 246.01 |=================================================================== c . 242.44 |================================================================== d . 242.19 |================================================================== Blender 4.0 Blend File: BMW27 - Compute: CPU-Only Seconds < Lower Is Better a . 123.28 |=================================================================== b . 124.06 |=================================================================== c . 123.13 |================================================================== d . 122.49 |================================================================== Blender 4.0 Blend File: Classroom - Compute: CPU-Only Seconds < Lower Is Better a . 336.89 |=================================================================== b . 335.93 |=================================================================== c . 336.35 |=================================================================== d . 336.10 |=================================================================== Blender 4.0 Blend File: Fishy Cat - Compute: CPU-Only Seconds < Lower Is Better a . 165.69 |=================================================================== b . 166.18 |=================================================================== c . 164.96 |================================================================== d . 166.34 |=================================================================== Blender 4.0 Blend File: Barbershop - Compute: CPU-Only Seconds < Lower Is Better a . 1295.39 |================================================================== b . 1300.83 |================================================================== c . 1294.93 |================================================================== d . 1294.63 |================================================================== Blender 4.0 Blend File: Pabellon Barcelona - Compute: CPU-Only Seconds < Lower Is Better a . 414.08 |=================================================================== b . 413.33 |=================================================================== c . 414.52 |=================================================================== d . 413.90 |===================================================================