ddddx AMD Ryzen Threadripper PRO 5965WX 24-Cores testing with a ASUS Pro WS WRX80E-SAGE SE WIFI (1201 BIOS) and ASUS NVIDIA NV106 2GB on Ubuntu 23.10 via the Phoronix Test Suite. a: Processor: AMD Ryzen Threadripper PRO 5965WX 24-Cores @ 3.80GHz (24 Cores / 48 Threads), Motherboard: ASUS Pro WS WRX80E-SAGE SE WIFI (1201 BIOS), Chipset: AMD Starship/Matisse, Memory: 8 x 16GB DDR4-2133MT/s Corsair CMK32GX4M2E3200C16, Disk: 2048GB SOLIDIGM SSDPFKKW020X7, Graphics: ASUS NVIDIA NV106 2GB, Audio: AMD Starship/Matisse, Monitor: VA2431, Network: 2 x Intel X550 + Intel Wi-Fi 6 AX200 OS: Ubuntu 23.10, Kernel: 6.5.0-15-generic (x86_64), Desktop: GNOME Shell 45.0, Display Server: X Server + Wayland, Display Driver: nouveau, OpenGL: 4.3 Mesa 23.2.1-1ubuntu3, Compiler: GCC 13.2.0, File-System: ext4, Screen Resolution: 1920x1080 b: Processor: AMD Ryzen Threadripper PRO 5965WX 24-Cores @ 3.80GHz (24 Cores / 48 Threads), Motherboard: ASUS Pro WS WRX80E-SAGE SE WIFI (1201 BIOS), Chipset: AMD Starship/Matisse, Memory: 8 x 16GB DDR4-2133MT/s Corsair CMK32GX4M2E3200C16, Disk: 2048GB SOLIDIGM SSDPFKKW020X7, Graphics: ASUS NVIDIA NV106 2GB, Audio: AMD Starship/Matisse, Monitor: VA2431, Network: 2 x Intel X550 + Intel Wi-Fi 6 AX200 OS: Ubuntu 23.10, Kernel: 6.5.0-15-generic (x86_64), Desktop: GNOME Shell 45.0, Display Server: X Server + Wayland, Display Driver: nouveau, OpenGL: 4.3 Mesa 23.2.1-1ubuntu3, Compiler: GCC 13.2.0, File-System: ext4, Screen Resolution: 1920x1080 c: Processor: AMD Ryzen Threadripper PRO 5965WX 24-Cores @ 3.80GHz (24 Cores / 48 Threads), Motherboard: ASUS Pro WS WRX80E-SAGE SE WIFI (1201 BIOS), Chipset: AMD Starship/Matisse, Memory: 8 x 16GB DDR4-2133MT/s Corsair CMK32GX4M2E3200C16, Disk: 2048GB SOLIDIGM SSDPFKKW020X7, Graphics: ASUS NVIDIA NV106 2GB, Audio: AMD Starship/Matisse, Monitor: VA2431, Network: 2 x Intel X550 + Intel Wi-Fi 6 AX200 OS: Ubuntu 23.10, Kernel: 6.5.0-15-generic (x86_64), Desktop: GNOME Shell 45.0, Display Server: X Server + Wayland, Display Driver: nouveau, OpenGL: 4.3 Mesa 23.2.1-1ubuntu3, Compiler: GCC 13.2.0, File-System: ext4, Screen Resolution: 1920x1080 d: Processor: AMD Ryzen Threadripper PRO 5965WX 24-Cores @ 3.80GHz (24 Cores / 48 Threads), Motherboard: ASUS Pro WS WRX80E-SAGE SE WIFI (1201 BIOS), Chipset: AMD Starship/Matisse, Memory: 8 x 16GB DDR4-2133MT/s Corsair CMK32GX4M2E3200C16, Disk: 2048GB SOLIDIGM SSDPFKKW020X7, Graphics: ASUS NVIDIA NV106 2GB, Audio: AMD Starship/Matisse, Monitor: VA2431, Network: 2 x Intel X550 + Intel Wi-Fi 6 AX200 OS: Ubuntu 23.10, Kernel: 6.5.0-15-generic (x86_64), Desktop: GNOME Shell 45.0, Display Server: X Server + Wayland, Display Driver: nouveau, OpenGL: 4.3 Mesa 23.2.1-1ubuntu3, Compiler: GCC 13.2.0, File-System: ext4, Screen Resolution: 1920x1080 JPEG-XL libjxl 0.10.1 Input: PNG - Quality: 80 MP/s > Higher Is Better a . 44.75 |==================================================================== b . 43.25 |================================================================== c . 39.90 |============================================================= d . 42.18 |================================================================ JPEG-XL libjxl 0.10.1 Input: PNG - Quality: 90 MP/s > Higher Is Better a . 39.40 |==================================================================== b . 37.44 |================================================================= c . 38.51 |================================================================== d . 38.27 |================================================================== JPEG-XL libjxl 0.10.1 Input: JPEG - Quality: 80 MP/s > Higher Is Better a . 46.58 |==================================================================== b . 42.66 |============================================================== c . 42.35 |============================================================== d . 43.50 |================================================================ JPEG-XL libjxl 0.10.1 Input: JPEG - Quality: 90 MP/s > Higher Is Better a . 42.50 |==================================================================== b . 39.89 |================================================================ c . 40.76 |================================================================= d . 39.60 |=============================================================== JPEG-XL libjxl 0.10.1 Input: PNG - Quality: 100 MP/s > Higher Is Better a . 27.65 |==================================================================== b . 27.57 |==================================================================== c . 27.49 |==================================================================== d . 27.34 |=================================================================== JPEG-XL libjxl 0.10.1 Input: JPEG - Quality: 100 MP/s > Higher Is Better a . 27.53 |==================================================================== b . 27.32 |=================================================================== c . 27.32 |=================================================================== d . 27.32 |=================================================================== JPEG-XL Decoding libjxl 0.10.1 CPU Threads: 1 MP/s > Higher Is Better a . 64.03 |==================================================================== b . 63.30 |=================================================================== c . 63.04 |=================================================================== d . 62.82 |=================================================================== JPEG-XL Decoding libjxl 0.10.1 CPU Threads: All MP/s > Higher Is Better a . 483.42 |=================================================================== b . 482.58 |=================================================================== c . 470.16 |================================================================= d . 439.61 |============================================================= srsRAN Project 23.10.1-20240219 Test: PDSCH Processor Benchmark, Throughput Total Mbps > Higher Is Better a . 11710.3 |================================================================ b . 12063.4 |================================================================== c . 11723.5 |================================================================ d . 11654.9 |================================================================ srsRAN Project 23.10.1-20240219 Test: PUSCH Processor Benchmark, Throughput Total Mbps > Higher Is Better a . 1888.2 |=================================================================== b . 1889.2 |=================================================================== c . 1886.9 |=================================================================== d . 1857.0 |================================================================== srsRAN Project 23.10.1-20240219 Test: PDSCH Processor Benchmark, Throughput Thread Mbps > Higher Is Better a . 603.7 |==================================================================== b . 600.2 |=================================================================== c . 605.5 |==================================================================== d . 607.8 |==================================================================== srsRAN Project 23.10.1-20240219 Test: PUSCH Processor Benchmark, Throughput Thread Mbps > Higher Is Better a . 177.6 |==================================================================== b . 178.7 |==================================================================== c . 178.6 |==================================================================== d . 177.9 |==================================================================== SVT-AV1 2.0 Encoder Mode: Preset 4 - Input: Bosphorus 4K Frames Per Second > Higher Is Better a . 6.647 |=================================================================== b . 6.787 |==================================================================== c . 6.711 |=================================================================== d . 6.660 |=================================================================== SVT-AV1 2.0 Encoder Mode: Preset 8 - Input: Bosphorus 4K Frames Per Second > Higher Is Better a . 62.95 |=================================================================== b . 63.89 |==================================================================== c . 62.50 |=================================================================== d . 62.93 |=================================================================== SVT-AV1 2.0 Encoder Mode: Preset 12 - Input: Bosphorus 4K Frames Per Second > Higher Is Better a . 154.17 |=================================================================== b . 150.81 |================================================================== c . 150.13 |================================================================= d . 151.64 |================================================================== SVT-AV1 2.0 Encoder Mode: Preset 13 - Input: Bosphorus 4K Frames Per Second > Higher Is Better a . 151.96 |=================================================================== b . 145.97 |================================================================ c . 150.52 |================================================================== d . 152.47 |=================================================================== SVT-AV1 2.0 Encoder Mode: Preset 4 - Input: Bosphorus 1080p Frames Per Second > Higher Is Better a . 19.21 |==================================================================== b . 18.92 |=================================================================== c . 19.17 |=================================================================== d . 19.32 |==================================================================== SVT-AV1 2.0 Encoder Mode: Preset 8 - Input: Bosphorus 1080p Frames Per Second > Higher Is Better a . 131.72 |================================================================== b . 131.93 |================================================================== c . 133.27 |=================================================================== d . 133.78 |=================================================================== SVT-AV1 2.0 Encoder Mode: Preset 12 - Input: Bosphorus 1080p Frames Per Second > Higher Is Better a . 489.43 |================================================================== b . 499.14 |=================================================================== c . 487.66 |================================================================= d . 488.52 |================================================================== SVT-AV1 2.0 Encoder Mode: Preset 13 - Input: Bosphorus 1080p Frames Per Second > Higher Is Better a . 587.48 |================================================================= b . 602.97 |=================================================================== c . 606.36 |=================================================================== d . 587.97 |================================================================= VVenC 1.11 Video Input: Bosphorus 4K - Video Preset: Fast Frames Per Second > Higher Is Better a . 7.060 |==================================================================== b . 7.075 |==================================================================== c . 7.066 |==================================================================== d . 7.062 |==================================================================== VVenC 1.11 Video Input: Bosphorus 4K - Video Preset: Faster Frames Per Second > Higher Is Better a . 14.57 |=================================================================== b . 14.82 |==================================================================== c . 14.78 |==================================================================== d . 14.86 |==================================================================== VVenC 1.11 Video Input: Bosphorus 1080p - Video Preset: Fast Frames Per Second > Higher Is Better a . 19.62 |==================================================================== b . 19.49 |==================================================================== c . 19.46 |=================================================================== d . 19.53 |==================================================================== VVenC 1.11 Video Input: Bosphorus 1080p - Video Preset: Faster Frames Per Second > Higher Is Better a . 41.28 |==================================================================== b . 40.46 |=================================================================== c . 40.92 |=================================================================== d . 41.00 |==================================================================== OSPRay 3.1 Benchmark: particle_volume/ao/real_time Items Per Second > Higher Is Better a . 10.28 |==================================================================== b . 10.25 |==================================================================== c . 10.28 |==================================================================== d . 10.30 |==================================================================== OSPRay 3.1 Benchmark: particle_volume/scivis/real_time Items Per Second > Higher Is Better a . 10.14 |==================================================================== b . 10.14 |==================================================================== c . 10.15 |==================================================================== d . 10.18 |==================================================================== OSPRay 3.1 Benchmark: particle_volume/pathtracer/real_time Items Per Second > Higher Is Better a . 156.40 |=================================================================== b . 155.37 |=================================================================== c . 155.83 |=================================================================== d . 155.05 |================================================================== OSPRay 3.1 Benchmark: gravity_spheres_volume/dim_512/ao/real_time Items Per Second > Higher Is Better a . 4.90257 |================================================================== b . 4.88053 |================================================================== c . 4.88608 |================================================================== d . 4.89928 |================================================================== OSPRay 3.1 Benchmark: gravity_spheres_volume/dim_512/scivis/real_time Items Per Second > Higher Is Better a . 4.59882 |================================================================== b . 4.57706 |================================================================== c . 4.59266 |================================================================== d . 4.58081 |================================================================== OSPRay 3.1 Benchmark: gravity_spheres_volume/dim_512/pathtracer/real_time Items Per Second > Higher Is Better a . 7.46614 |================================================================== b . 7.44837 |================================================================== c . 7.43821 |================================================================== d . 7.45220 |================================================================== Stockfish 16.1 Chess Benchmark Nodes Per Second > Higher Is Better a . 52607528 |======================================================== b . 61008270 |================================================================= c . 55237573 |=========================================================== d . 53082452 |========================================================= Timed Linux Kernel Compilation 6.8 Build: defconfig Seconds < Lower Is Better a . 54.13 |==================================================================== b . 54.21 |==================================================================== c . 52.81 |================================================================== d . 54.16 |==================================================================== Timed Linux Kernel Compilation 6.8 Build: allmodconfig Seconds < Lower Is Better a . 597.03 |=================================================================== b . 597.90 |=================================================================== c . 596.51 |=================================================================== d . 597.83 |=================================================================== Parallel BZIP2 Compression 1.1.13 FreeBSD-13.0-RELEASE-amd64-memstick.img Compression Seconds < Lower Is Better a . 3.230305 |=============================================================== b . 3.314749 |================================================================= c . 3.285082 |================================================================ d . 3.221993 |=============================================================== Primesieve 12.1 Length: 1e12 Seconds < Lower Is Better a . 6.244 |==================================================================== b . 6.081 |================================================================== c . 6.113 |=================================================================== d . 6.123 |=================================================================== Primesieve 12.1 Length: 1e13 Seconds < Lower Is Better a . 77.15 |==================================================================== b . 76.92 |==================================================================== c . 77.21 |==================================================================== d . 77.06 |==================================================================== oneDNN 3.4 Harness: IP Shapes 1D - Engine: CPU ms < Lower Is Better a . 1.31474 |================================================================= b . 1.31101 |================================================================= c . 1.32743 |================================================================== d . 1.31333 |================================================================= oneDNN 3.4 Harness: IP Shapes 3D - Engine: CPU ms < Lower Is Better a . 3.52569 |================================================================== b . 3.52584 |================================================================== c . 3.53323 |================================================================== d . 3.53851 |================================================================== oneDNN 3.4 Harness: Convolution Batch Shapes Auto - Engine: CPU ms < Lower Is Better a . 2.73804 |================================================================== b . 2.71135 |================================================================= c . 2.73192 |================================================================== d . 2.73352 |================================================================== oneDNN 3.4 Harness: Deconvolution Batch shapes_1d - Engine: CPU ms < Lower Is Better a . 5.51082 |================================================================== b . 5.33929 |================================================================ c . 5.37367 |================================================================ d . 5.03641 |============================================================ oneDNN 3.4 Harness: Deconvolution Batch shapes_3d - Engine: CPU ms < Lower Is Better a . 2.36511 |================================================================== b . 2.34602 |================================================================= c . 2.36138 |================================================================== d . 2.34346 |================================================================= oneDNN 3.4 Harness: Recurrent Neural Network Training - Engine: CPU ms < Lower Is Better a . 1254.68 |================================================================== b . 1255.67 |================================================================== c . 1256.43 |================================================================== d . 1254.43 |================================================================== oneDNN 3.4 Harness: Recurrent Neural Network Inference - Engine: CPU ms < Lower Is Better a . 638.13 |=================================================================== b . 636.77 |================================================================== c . 637.17 |================================================================== d . 642.19 |=================================================================== OSPRay Studio 1.0 Camera: 1 - Resolution: 4K - Samples Per Pixel: 1 - Renderer: Path Tracer - Acceleration: CPU ms < Lower Is Better a . 4528 |===================================================================== b . 4521 |===================================================================== c . 4535 |===================================================================== d . 4528 |===================================================================== OSPRay Studio 1.0 Camera: 2 - Resolution: 4K - Samples Per Pixel: 1 - Renderer: Path Tracer - Acceleration: CPU ms < Lower Is Better a . 4616 |===================================================================== b . 4609 |===================================================================== c . 4615 |===================================================================== d . 4618 |===================================================================== OSPRay Studio 1.0 Camera: 3 - Resolution: 4K - Samples Per Pixel: 1 - Renderer: Path Tracer - Acceleration: CPU ms < Lower Is Better a . 5330 |===================================================================== b . 5342 |===================================================================== c . 5341 |===================================================================== d . 5331 |===================================================================== OSPRay Studio 1.0 Camera: 1 - Resolution: 4K - Samples Per Pixel: 16 - Renderer: Path Tracer - Acceleration: CPU ms < Lower Is Better a . 78061 |==================================================================== b . 76995 |=================================================================== c . 77511 |==================================================================== d . 77073 |=================================================================== OSPRay Studio 1.0 Camera: 1 - Resolution: 4K - Samples Per Pixel: 32 - Renderer: Path Tracer - Acceleration: CPU ms < Lower Is Better a . 150375 |=================================================================== b . 150040 |=================================================================== c . 150381 |=================================================================== d . 150694 |=================================================================== OSPRay Studio 1.0 Camera: 2 - Resolution: 4K - Samples Per Pixel: 16 - Renderer: Path Tracer - Acceleration: CPU ms < Lower Is Better a . 78788 |==================================================================== b . 78304 |=================================================================== c . 78919 |==================================================================== d . 78354 |==================================================================== OSPRay Studio 1.0 Camera: 2 - Resolution: 4K - Samples Per Pixel: 32 - Renderer: Path Tracer - Acceleration: CPU ms < Lower Is Better a . 152653 |=================================================================== b . 153552 |=================================================================== c . 152510 |=================================================================== d . 151620 |================================================================== OSPRay Studio 1.0 Camera: 3 - Resolution: 4K - Samples Per Pixel: 16 - Renderer: Path Tracer - Acceleration: CPU ms < Lower Is Better a . 90362 |==================================================================== b . 90131 |==================================================================== c . 90083 |==================================================================== d . 90380 |==================================================================== OSPRay Studio 1.0 Camera: 3 - Resolution: 4K - Samples Per Pixel: 32 - Renderer: Path Tracer - Acceleration: CPU ms < Lower Is Better a . 177445 |=================================================================== b . 175496 |================================================================== c . 175767 |================================================================== d . 175129 |================================================================== OSPRay Studio 1.0 Camera: 1 - Resolution: 1080p - Samples Per Pixel: 1 - Renderer: Path Tracer - Acceleration: CPU ms < Lower Is Better a . 1138 |===================================================================== b . 1138 |===================================================================== c . 1138 |===================================================================== d . 1134 |===================================================================== OSPRay Studio 1.0 Camera: 2 - Resolution: 1080p - Samples Per Pixel: 1 - Renderer: Path Tracer - Acceleration: CPU ms < Lower Is Better a . 1158 |===================================================================== b . 1150 |===================================================================== c . 1152 |===================================================================== d . 1153 |===================================================================== OSPRay Studio 1.0 Camera: 3 - Resolution: 1080p - Samples Per Pixel: 1 - Renderer: Path Tracer - Acceleration: CPU ms < Lower Is Better a . 1336 |===================================================================== b . 1338 |===================================================================== c . 1333 |===================================================================== d . 1330 |===================================================================== OSPRay Studio 1.0 Camera: 1 - Resolution: 1080p - Samples Per Pixel: 16 - Renderer: Path Tracer - Acceleration: CPU ms < Lower Is Better a . 18203 |==================================================================== b . 18181 |==================================================================== c . 18229 |==================================================================== d . 18177 |==================================================================== OSPRay Studio 1.0 Camera: 1 - Resolution: 1080p - Samples Per Pixel: 32 - Renderer: Path Tracer - Acceleration: CPU ms < Lower Is Better a . 41167 |==================================================================== b . 41125 |==================================================================== c . 41189 |==================================================================== d . 41111 |==================================================================== OSPRay Studio 1.0 Camera: 2 - Resolution: 1080p - Samples Per Pixel: 16 - Renderer: Path Tracer - Acceleration: CPU ms < Lower Is Better a . 18543 |==================================================================== b . 18471 |==================================================================== c . 18579 |==================================================================== d . 18488 |==================================================================== OSPRay Studio 1.0 Camera: 2 - Resolution: 1080p - Samples Per Pixel: 32 - Renderer: Path Tracer - Acceleration: CPU ms < Lower Is Better a . 41839 |==================================================================== b . 41598 |=================================================================== c . 42050 |==================================================================== d . 41959 |==================================================================== OSPRay Studio 1.0 Camera: 3 - Resolution: 1080p - Samples Per Pixel: 16 - Renderer: Path Tracer - Acceleration: CPU ms < Lower Is Better a . 21449 |==================================================================== b . 21398 |==================================================================== c . 21409 |==================================================================== d . 21385 |==================================================================== OSPRay Studio 1.0 Camera: 3 - Resolution: 1080p - Samples Per Pixel: 32 - Renderer: Path Tracer - Acceleration: CPU ms < Lower Is Better a . 47495 |==================================================================== b . 47827 |==================================================================== c . 47686 |==================================================================== d . 47597 |==================================================================== Neural Magic DeepSparse 1.7 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 26.69 |==================================================================== b . 26.60 |==================================================================== c . 26.62 |==================================================================== d . 26.69 |==================================================================== Neural Magic DeepSparse 1.7 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 448.71 |=================================================================== b . 447.65 |=================================================================== c . 449.12 |=================================================================== d . 448.93 |=================================================================== Neural Magic DeepSparse 1.7 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 18.37 |=================================================================== b . 18.51 |==================================================================== c . 18.40 |==================================================================== d . 18.40 |==================================================================== Neural Magic DeepSparse 1.7 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 54.42 |==================================================================== b . 54.01 |=================================================================== c . 54.33 |==================================================================== d . 54.34 |==================================================================== Neural Magic DeepSparse 1.7 Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 684.57 |=================================================================== b . 685.45 |=================================================================== c . 684.02 |=================================================================== d . 685.82 |=================================================================== Neural Magic DeepSparse 1.7 Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 17.51 |==================================================================== b . 17.49 |==================================================================== c . 17.52 |==================================================================== d . 17.48 |==================================================================== Neural Magic DeepSparse 1.7 Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 194.29 |=================================================================== b . 191.76 |================================================================== c . 192.98 |=================================================================== d . 194.38 |=================================================================== Neural Magic DeepSparse 1.7 Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 5.1435 |================================================================== b . 5.2115 |=================================================================== c . 5.1787 |=================================================================== d . 5.1408 |================================================================== Neural Magic DeepSparse 1.7 Model: ResNet-50, Baseline - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 307.00 |=================================================================== b . 307.02 |=================================================================== c . 306.92 |=================================================================== d . 306.94 |=================================================================== Neural Magic DeepSparse 1.7 Model: ResNet-50, Baseline - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 39.05 |==================================================================== b . 39.05 |==================================================================== c . 39.07 |==================================================================== d . 39.06 |==================================================================== Neural Magic DeepSparse 1.7 Model: ResNet-50, Baseline - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 156.21 |=================================================================== b . 155.34 |=================================================================== c . 155.68 |=================================================================== d . 156.04 |=================================================================== Neural Magic DeepSparse 1.7 Model: ResNet-50, Baseline - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 6.3927 |=================================================================== b . 6.4286 |=================================================================== c . 6.4155 |=================================================================== d . 6.4001 |=================================================================== Neural Magic DeepSparse 1.7 Model: ResNet-50, Sparse INT8 - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 1994.39 |================================================================= b . 2032.96 |================================================================== c . 1998.61 |================================================================= d . 2014.58 |================================================================= Neural Magic DeepSparse 1.7 Model: ResNet-50, Sparse INT8 - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 6.0030 |=================================================================== b . 5.8898 |================================================================== c . 5.9888 |=================================================================== d . 5.9417 |================================================================== Neural Magic DeepSparse 1.7 Model: ResNet-50, Sparse INT8 - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 798.47 |=================================================================== b . 800.05 |=================================================================== c . 799.42 |=================================================================== d . 794.39 |=================================================================== Neural Magic DeepSparse 1.7 Model: ResNet-50, Sparse INT8 - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 1.2491 |=================================================================== b . 1.2468 |=================================================================== c . 1.2478 |=================================================================== d . 1.2559 |=================================================================== Neural Magic DeepSparse 1.7 Model: Llama2 Chat 7b Quantized - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 1.8772 |=================================================================== b . 1.8677 |=================================================================== c . 1.8705 |=================================================================== d . 1.8701 |=================================================================== Neural Magic DeepSparse 1.7 Model: Llama2 Chat 7b Quantized - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 5873.27 |================================================================== b . 5904.28 |================================================================== c . 5896.59 |================================================================== d . 5897.28 |================================================================== Neural Magic DeepSparse 1.7 Model: Llama2 Chat 7b Quantized - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 6.0448 |=================================================================== b . 6.0393 |=================================================================== c . 6.0425 |=================================================================== d . 6.0373 |=================================================================== Neural Magic DeepSparse 1.7 Model: Llama2 Chat 7b Quantized - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 165.40 |=================================================================== b . 165.55 |=================================================================== c . 165.47 |=================================================================== d . 165.61 |=================================================================== Neural Magic DeepSparse 1.7 Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 306.97 |=================================================================== b . 307.09 |=================================================================== c . 306.95 |=================================================================== d . 306.90 |=================================================================== Neural Magic DeepSparse 1.7 Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 39.05 |==================================================================== b . 39.03 |==================================================================== c . 39.06 |==================================================================== d . 39.07 |==================================================================== Neural Magic DeepSparse 1.7 Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 156.43 |=================================================================== b . 155.31 |=================================================================== c . 155.79 |=================================================================== d . 155.90 |=================================================================== Neural Magic DeepSparse 1.7 Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 6.3846 |=================================================================== b . 6.4293 |=================================================================== c . 6.4103 |=================================================================== d . 6.4059 |=================================================================== Neural Magic DeepSparse 1.7 Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 150.83 |================================================================== b . 151.98 |=================================================================== c . 151.26 |=================================================================== d . 150.73 |================================================================== Neural Magic DeepSparse 1.7 Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 79.45 |==================================================================== b . 78.82 |=================================================================== c . 79.25 |==================================================================== d . 79.51 |==================================================================== Neural Magic DeepSparse 1.7 Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 99.33 |==================================================================== b . 99.22 |==================================================================== c . 98.80 |==================================================================== d . 99.05 |==================================================================== Neural Magic DeepSparse 1.7 Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 10.06 |==================================================================== b . 10.07 |==================================================================== c . 10.12 |==================================================================== d . 10.09 |==================================================================== Neural Magic DeepSparse 1.7 Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 224.16 |=================================================================== b . 224.20 |=================================================================== c . 223.00 |=================================================================== d . 223.55 |=================================================================== Neural Magic DeepSparse 1.7 Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 53.48 |==================================================================== b . 53.47 |==================================================================== c . 53.76 |==================================================================== d . 53.63 |==================================================================== Neural Magic DeepSparse 1.7 Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 112.33 |=================================================================== b . 111.84 |=================================================================== c . 112.00 |=================================================================== d . 111.45 |================================================================== Neural Magic DeepSparse 1.7 Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 8.8921 |================================================================== b . 8.9317 |=================================================================== c . 8.9189 |=================================================================== d . 8.9625 |=================================================================== Neural Magic DeepSparse 1.7 Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 30.47 |==================================================================== b . 30.49 |==================================================================== c . 30.39 |==================================================================== d . 30.49 |==================================================================== Neural Magic DeepSparse 1.7 Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 393.66 |=================================================================== b . 393.08 |=================================================================== c . 393.82 |=================================================================== d . 393.39 |=================================================================== Neural Magic DeepSparse 1.7 Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 21.70 |==================================================================== b . 21.75 |==================================================================== c . 21.69 |==================================================================== d . 21.72 |==================================================================== Neural Magic DeepSparse 1.7 Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 46.06 |==================================================================== b . 45.96 |==================================================================== c . 46.08 |==================================================================== d . 46.02 |==================================================================== Neural Magic DeepSparse 1.7 Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 334.39 |================================================================== b . 337.45 |=================================================================== c . 334.75 |================================================================== d . 334.72 |================================================================== Neural Magic DeepSparse 1.7 Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 35.85 |==================================================================== b . 35.54 |=================================================================== c . 35.81 |==================================================================== d . 35.81 |==================================================================== Neural Magic DeepSparse 1.7 Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 75.88 |==================================================================== b . 76.12 |==================================================================== c . 76.06 |==================================================================== d . 75.70 |==================================================================== Neural Magic DeepSparse 1.7 Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 13.17 |==================================================================== b . 13.13 |==================================================================== c . 13.14 |==================================================================== d . 13.20 |==================================================================== Neural Magic DeepSparse 1.7 Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 26.74 |==================================================================== b . 26.86 |==================================================================== c . 26.70 |==================================================================== d . 26.81 |==================================================================== Neural Magic DeepSparse 1.7 Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 446.84 |=================================================================== b . 446.24 |=================================================================== c . 447.58 |=================================================================== d . 447.17 |=================================================================== Neural Magic DeepSparse 1.7 Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 18.49 |==================================================================== b . 18.52 |==================================================================== c . 18.49 |==================================================================== d . 18.48 |==================================================================== Neural Magic DeepSparse 1.7 Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 54.07 |==================================================================== b . 53.97 |==================================================================== c . 54.08 |==================================================================== d . 54.10 |==================================================================== Google Draco 1.5.6 Model: Lion ms < Lower Is Better a . 5328 |===================================================================== b . 5365 |===================================================================== c . 5363 |===================================================================== d . 5347 |===================================================================== Google Draco 1.5.6 Model: Church Facade ms < Lower Is Better a . 7023 |==================================================================== b . 7034 |==================================================================== c . 7092 |===================================================================== d . 7096 |===================================================================== OpenVINO 2024.0 Model: Face Detection FP16 - Device: CPU FPS > Higher Is Better a . 7.60 |==================================================================== b . 7.68 |===================================================================== c . 7.59 |==================================================================== d . 7.64 |===================================================================== OpenVINO 2024.0 Model: Face Detection FP16 - Device: CPU ms < Lower Is Better a . 1558.81 |================================================================== b . 1547.31 |================================================================= c . 1563.15 |================================================================== d . 1553.54 |================================================================== OpenVINO 2024.0 Model: Person Detection FP16 - Device: CPU FPS > Higher Is Better a . 70.01 |==================================================================== b . 69.95 |==================================================================== c . 69.83 |==================================================================== d . 69.61 |==================================================================== OpenVINO 2024.0 Model: Person Detection FP16 - Device: CPU ms < Lower Is Better a . 171.22 |=================================================================== b . 171.43 |=================================================================== c . 171.67 |=================================================================== d . 172.18 |=================================================================== OpenVINO 2024.0 Model: Person Detection FP32 - Device: CPU FPS > Higher Is Better a . 69.73 |==================================================================== b . 70.01 |==================================================================== c . 69.99 |==================================================================== d . 69.86 |==================================================================== OpenVINO 2024.0 Model: Person Detection FP32 - Device: CPU ms < Lower Is Better a . 171.92 |=================================================================== b . 171.16 |=================================================================== c . 171.29 |=================================================================== d . 171.54 |=================================================================== OpenVINO 2024.0 Model: Vehicle Detection FP16 - Device: CPU FPS > Higher Is Better a . 600.94 |=================================================================== b . 598.03 |================================================================== c . 592.96 |================================================================== d . 603.96 |=================================================================== OpenVINO 2024.0 Model: Vehicle Detection FP16 - Device: CPU ms < Lower Is Better a . 19.94 |=================================================================== b . 20.04 |=================================================================== c . 20.21 |==================================================================== d . 19.84 |=================================================================== OpenVINO 2024.0 Model: Face Detection FP16-INT8 - Device: CPU FPS > Higher Is Better a . 16.66 |==================================================================== b . 16.73 |==================================================================== c . 16.66 |==================================================================== d . 16.74 |==================================================================== OpenVINO 2024.0 Model: Face Detection FP16-INT8 - Device: CPU ms < Lower Is Better a . 715.22 |=================================================================== b . 713.72 |=================================================================== c . 716.16 |=================================================================== d . 713.64 |=================================================================== OpenVINO 2024.0 Model: Face Detection Retail FP16 - Device: CPU FPS > Higher Is Better a . 2192.09 |================================================================== b . 2197.50 |================================================================== c . 2197.98 |================================================================== d . 2198.11 |================================================================== OpenVINO 2024.0 Model: Face Detection Retail FP16 - Device: CPU ms < Lower Is Better a . 5.46 |===================================================================== b . 5.45 |===================================================================== c . 5.45 |===================================================================== d . 5.45 |===================================================================== OpenVINO 2024.0 Model: Road Segmentation ADAS FP16 - Device: CPU FPS > Higher Is Better a . 170.10 |=================================================================== b . 170.68 |=================================================================== c . 169.65 |=================================================================== d . 169.41 |=================================================================== OpenVINO 2024.0 Model: Road Segmentation ADAS FP16 - Device: CPU ms < Lower Is Better a . 70.48 |==================================================================== b . 70.23 |=================================================================== c . 70.66 |==================================================================== d . 70.76 |==================================================================== OpenVINO 2024.0 Model: Vehicle Detection FP16-INT8 - Device: CPU FPS > Higher Is Better a . 1104.03 |================================================================== b . 1106.44 |================================================================== c . 1107.64 |================================================================== d . 1103.60 |================================================================== OpenVINO 2024.0 Model: Vehicle Detection FP16-INT8 - Device: CPU ms < Lower Is Better a . 10.86 |==================================================================== b . 10.83 |==================================================================== c . 10.82 |==================================================================== d . 10.86 |==================================================================== OpenVINO 2024.0 Model: Weld Porosity Detection FP16 - Device: CPU FPS > Higher Is Better a . 718.20 |=================================================================== b . 719.93 |=================================================================== c . 715.28 |=================================================================== d . 716.84 |=================================================================== OpenVINO 2024.0 Model: Weld Porosity Detection FP16 - Device: CPU ms < Lower Is Better a . 16.69 |==================================================================== b . 16.65 |==================================================================== c . 16.76 |==================================================================== d . 16.72 |==================================================================== OpenVINO 2024.0 Model: Face Detection Retail FP16-INT8 - Device: CPU FPS > Higher Is Better a . 3224.64 |================================================================== b . 3241.35 |================================================================== c . 3228.19 |================================================================== d . 3230.25 |================================================================== OpenVINO 2024.0 Model: Face Detection Retail FP16-INT8 - Device: CPU ms < Lower Is Better a . 3.71 |===================================================================== b . 3.69 |===================================================================== c . 3.71 |===================================================================== d . 3.71 |===================================================================== OpenVINO 2024.0 Model: Road Segmentation ADAS FP16-INT8 - Device: CPU FPS > Higher Is Better a . 428.43 |=================================================================== b . 428.18 |=================================================================== c . 427.02 |=================================================================== d . 428.55 |=================================================================== OpenVINO 2024.0 Model: Road Segmentation ADAS FP16-INT8 - Device: CPU ms < Lower Is Better a . 27.99 |==================================================================== b . 28.00 |==================================================================== c . 28.08 |==================================================================== d . 27.98 |==================================================================== OpenVINO 2024.0 Model: Machine Translation EN To DE FP16 - Device: CPU FPS > Higher Is Better a . 88.13 |==================================================================== b . 88.33 |==================================================================== c . 88.05 |==================================================================== d . 88.14 |==================================================================== OpenVINO 2024.0 Model: Machine Translation EN To DE FP16 - Device: CPU ms < Lower Is Better a . 136.00 |=================================================================== b . 135.70 |=================================================================== c . 136.15 |=================================================================== d . 135.96 |=================================================================== OpenVINO 2024.0 Model: Weld Porosity Detection FP16-INT8 - Device: CPU FPS > Higher Is Better a . 1652.05 |================================================================== b . 1654.72 |================================================================== c . 1647.13 |================================================================== d . 1652.91 |================================================================== OpenVINO 2024.0 Model: Weld Porosity Detection FP16-INT8 - Device: CPU ms < Lower Is Better a . 14.52 |==================================================================== b . 14.49 |==================================================================== c . 14.56 |==================================================================== d . 14.51 |==================================================================== OpenVINO 2024.0 Model: Person Vehicle Bike Detection FP16 - Device: CPU FPS > Higher Is Better a . 895.25 |=================================================================== b . 889.36 |=================================================================== c . 894.57 |=================================================================== d . 894.41 |=================================================================== OpenVINO 2024.0 Model: Person Vehicle Bike Detection FP16 - Device: CPU ms < Lower Is Better a . 13.38 |==================================================================== b . 13.47 |==================================================================== c . 13.39 |==================================================================== d . 13.40 |==================================================================== OpenVINO 2024.0 Model: Noise Suppression Poconet-Like FP16 - Device: CPU FPS > Higher Is Better a . 1033.67 |================================================================= b . 1046.30 |================================================================== c . 1041.67 |================================================================== d . 1043.82 |================================================================== OpenVINO 2024.0 Model: Noise Suppression Poconet-Like FP16 - Device: CPU ms < Lower Is Better a . 11.54 |==================================================================== b . 11.41 |=================================================================== c . 11.46 |==================================================================== d . 11.44 |=================================================================== OpenVINO 2024.0 Model: Handwritten English Recognition FP16 - Device: CPU FPS > Higher Is Better a . 377.84 |=================================================================== b . 380.22 |=================================================================== c . 378.34 |=================================================================== d . 377.49 |=================================================================== OpenVINO 2024.0 Model: Handwritten English Recognition FP16 - Device: CPU ms < Lower Is Better a . 63.46 |==================================================================== b . 63.08 |==================================================================== c . 63.40 |==================================================================== d . 63.54 |==================================================================== OpenVINO 2024.0 Model: Person Re-Identification Retail FP16 - Device: CPU FPS > Higher Is Better a . 1198.64 |================================================================== b . 1201.22 |================================================================== c . 1202.09 |================================================================== d . 1202.22 |================================================================== OpenVINO 2024.0 Model: Person Re-Identification Retail FP16 - Device: CPU ms < Lower Is Better a . 10.00 |==================================================================== b . 9.98 |==================================================================== c . 9.97 |==================================================================== d . 9.97 |==================================================================== OpenVINO 2024.0 Model: Age Gender Recognition Retail 0013 FP16 - Device: CPU FPS > Higher Is Better a . 24478.31 |================================================================= b . 24434.51 |================================================================= c . 24329.21 |================================================================= d . 24496.84 |================================================================= OpenVINO 2024.0 Model: Age Gender Recognition Retail 0013 FP16 - Device: CPU ms < Lower Is Better a . 0.97 |==================================================================== b . 0.97 |==================================================================== c . 0.98 |===================================================================== d . 0.97 |==================================================================== OpenVINO 2024.0 Model: Handwritten English Recognition FP16-INT8 - Device: CPU FPS > Higher Is Better a . 421.72 |================================================================== b . 426.03 |=================================================================== c . 423.46 |=================================================================== d . 421.54 |================================================================== OpenVINO 2024.0 Model: Handwritten English Recognition FP16-INT8 - Device: CPU ms < Lower Is Better a . 56.87 |==================================================================== b . 56.30 |=================================================================== c . 56.64 |==================================================================== d . 56.90 |==================================================================== OpenVINO 2024.0 Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPU FPS > Higher Is Better a . 44518.20 |================================================================= b . 44461.02 |================================================================= c . 44306.10 |================================================================= d . 44596.35 |================================================================= OpenVINO 2024.0 Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPU ms < Lower Is Better a . 0.53 |===================================================================== b . 0.53 |===================================================================== c . 0.53 |===================================================================== d . 0.53 |===================================================================== RocksDB 9.0 Test: Overwrite Op/s > Higher Is Better a . 781166 |=================================================================== b . 779236 |=================================================================== c . 777804 |=================================================================== d . 762175 |================================================================= RocksDB 9.0 Test: Random Fill Op/s > Higher Is Better a . 790603 |=================================================================== b . 783967 |================================================================== c . 774384 |================================================================== d . 777115 |================================================================== RocksDB 9.0 Test: Random Read Op/s > Higher Is Better a . 145962891 |================================================================ b . 145893753 |================================================================ c . 144921763 |================================================================ d . 145516874 |================================================================ RocksDB 9.0 Test: Update Random Op/s > Higher Is Better a . 671121 |=================================================================== b . 666715 |=================================================================== c . 660978 |================================================================== d . 658520 |================================================================== RocksDB 9.0 Test: Sequential Fill Op/s > Higher Is Better a . 920430 |=================================================================== b . 908177 |================================================================== c . 908882 |================================================================== d . 919148 |=================================================================== RocksDB 9.0 Test: Random Fill Sync Op/s > Higher Is Better a . 45528 |================================================================== b . 45974 |=================================================================== c . 46701 |==================================================================== d . 46299 |=================================================================== RocksDB 9.0 Test: Read While Writing Op/s > Higher Is Better a . 5546801 |================================================================= b . 5586539 |================================================================= c . 5592760 |================================================================== d . 5633613 |================================================================== RocksDB 9.0 Test: Read Random Write Random Op/s > Higher Is Better a . 2879787 |================================================================== b . 2859339 |================================================================== c . 2868681 |================================================================== d . 2814513 |================================================================= WavPack Audio Encoding 5.7 WAV To WavPack Seconds < Lower Is Better a . 4.433 |==================================================================== b . 4.438 |==================================================================== c . 4.435 |==================================================================== d . 4.442 |==================================================================== BRL-CAD 7.38.2 VGR Performance Metric VGR Performance Metric > Higher Is Better a . 430387 |=================================================================== b . 424641 |================================================================== c . 420528 |================================================================= d . 425778 |================================================================== Chaos Group V-RAY 6.0 Mode: CPU vsamples > Higher Is Better a . 44287 |=================================================================== b . 44634 |==================================================================== c . 44375 |==================================================================== d . 44633 |====================================================================