AMD Ryzen Threadripper PRO 5965WX 24-Cores testing with a ASUS Pro WS WRX80E-SAGE SE WIFI (1201 BIOS) and ASUS NVIDIA NV106 2GB on Ubuntu 23.10 via the Phoronix Test Suite.
Compare your own system(s) to this result file with the
Phoronix Test Suite by running the command:
phoronix-test-suite benchmark 2402040-NE-2024YEAR116
2024 year
AMD Ryzen Threadripper PRO 5965WX 24-Cores testing with a ASUS Pro WS WRX80E-SAGE SE WIFI (1201 BIOS) and ASUS NVIDIA NV106 2GB on Ubuntu 23.10 via the Phoronix Test Suite.
a:
Processor: AMD Ryzen Threadripper PRO 5965WX 24-Cores @ 3.80GHz (24 Cores / 48 Threads), Motherboard: ASUS Pro WS WRX80E-SAGE SE WIFI (1201 BIOS), Chipset: AMD Starship/Matisse, Memory: 8 x 16GB DDR4-2133MT/s Corsair CMK32GX4M2E3200C16, Disk: 2048GB SOLIDIGM SSDPFKKW020X7, Graphics: ASUS NVIDIA NV106 2GB, Audio: AMD Starship/Matisse, Monitor: VA2431, Network: 2 x Intel X550 + Intel Wi-Fi 6 AX200
OS: Ubuntu 23.10, Kernel: 6.5.0-13-generic (x86_64), Desktop: GNOME Shell 45.0, Display Server: X Server + Wayland, Display Driver: nouveau, OpenGL: 4.3 Mesa 23.2.1-1ubuntu3, Compiler: GCC 13.2.0, File-System: ext4, Screen Resolution: 1920x1080
b:
Processor: AMD Ryzen Threadripper PRO 5965WX 24-Cores @ 3.80GHz (24 Cores / 48 Threads), Motherboard: ASUS Pro WS WRX80E-SAGE SE WIFI (1201 BIOS), Chipset: AMD Starship/Matisse, Memory: 8 x 16GB DDR4-2133MT/s Corsair CMK32GX4M2E3200C16, Disk: 2048GB SOLIDIGM SSDPFKKW020X7, Graphics: ASUS NVIDIA NV106 2GB, Audio: AMD Starship/Matisse, Monitor: VA2431, Network: 2 x Intel X550 + Intel Wi-Fi 6 AX200
OS: Ubuntu 23.10, Kernel: 6.5.0-13-generic (x86_64), Desktop: GNOME Shell 45.0, Display Server: X Server + Wayland, Display Driver: nouveau, OpenGL: 4.3 Mesa 23.2.1-1ubuntu3, Compiler: GCC 13.2.0, File-System: ext4, Screen Resolution: 1920x1080
c:
Processor: AMD Ryzen Threadripper PRO 5965WX 24-Cores @ 3.80GHz (24 Cores / 48 Threads), Motherboard: ASUS Pro WS WRX80E-SAGE SE WIFI (1201 BIOS), Chipset: AMD Starship/Matisse, Memory: 8 x 16GB DDR4-2133MT/s Corsair CMK32GX4M2E3200C16, Disk: 2048GB SOLIDIGM SSDPFKKW020X7, Graphics: ASUS NVIDIA NV106 2GB, Audio: AMD Starship/Matisse, Monitor: VA2431, Network: 2 x Intel X550 + Intel Wi-Fi 6 AX200
OS: Ubuntu 23.10, Kernel: 6.5.0-13-generic (x86_64), Desktop: GNOME Shell 45.0, Display Server: X Server + Wayland, Display Driver: nouveau, OpenGL: 4.3 Mesa 23.2.1-1ubuntu3, Compiler: GCC 13.2.0, File-System: ext4, Screen Resolution: 1920x1080
d:
Processor: AMD Ryzen Threadripper PRO 5965WX 24-Cores @ 3.80GHz (24 Cores / 48 Threads), Motherboard: ASUS Pro WS WRX80E-SAGE SE WIFI (1201 BIOS), Chipset: AMD Starship/Matisse, Memory: 8 x 16GB DDR4-2133MT/s Corsair CMK32GX4M2E3200C16, Disk: 2048GB SOLIDIGM SSDPFKKW020X7, Graphics: ASUS NVIDIA NV106 2GB, Audio: AMD Starship/Matisse, Monitor: VA2431, Network: 2 x Intel X550 + Intel Wi-Fi 6 AX200
OS: Ubuntu 23.10, Kernel: 6.5.0-13-generic (x86_64), Desktop: GNOME Shell 45.0, Display Server: X Server + Wayland, Display Driver: nouveau, OpenGL: 4.3 Mesa 23.2.1-1ubuntu3, Compiler: GCC 13.2.0, File-System: ext4, Screen Resolution: 1920x1080
LZ4 Compression 1.9.4
Compression Level: 1 - Decompression Speed
MB/s > Higher Is Better
a . 5019.5 |===================================================================
b . 5020.0 |===================================================================
c . 5023.2 |===================================================================
d . 5019.5 |===================================================================
Quicksilver 20230818
Input: CORAL2 P2
Figure Of Merit > Higher Is Better
a . 24030000 |=================================================================
b . 24026667 |=================================================================
c . 23890000 |=================================================================
d . 23840000 |================================================================
LZ4 Compression 1.9.4
Compression Level: 1 - Compression Speed
MB/s > Higher Is Better
a . 828.78 |===================================================================
b . 829.36 |===================================================================
c . 829.15 |===================================================================
d . 830.37 |===================================================================
LZ4 Compression 1.9.4
Compression Level: 3 - Decompression Speed
MB/s > Higher Is Better
a . 4595.9 |===================================================================
b . 4597.9 |===================================================================
c . 4598.0 |===================================================================
d . 4596.7 |===================================================================
LZ4 Compression 1.9.4
Compression Level: 9 - Compression Speed
MB/s > Higher Is Better
a . 44.28 |==================================================================
b . 44.48 |==================================================================
c . 45.49 |====================================================================
d . 44.52 |===================================================================
LZ4 Compression 1.9.4
Compression Level: 9 - Decompression Speed
MB/s > Higher Is Better
a . 4840.5 |===================================================================
b . 4841.4 |===================================================================
c . 4842.4 |===================================================================
d . 4844.5 |===================================================================
Quicksilver 20230818
Input: CORAL2 P1
Figure Of Merit > Higher Is Better
a . 24210000 |=================================================================
b . 24230000 |=================================================================
c . 24240000 |=================================================================
d . 24290000 |=================================================================
Quicksilver 20230818
Input: CTS2
Figure Of Merit > Higher Is Better
a . 20680000 |=================================================================
b . 20646667 |=================================================================
c . 20620000 |=================================================================
d . 20600000 |=================================================================
LZ4 Compression 1.9.4
Compression Level: 3 - Compression Speed
MB/s > Higher Is Better
a . 131.24 |===================================================================
b . 131.10 |===================================================================
c . 131.40 |===================================================================
d . 131.61 |===================================================================
CacheBench
Test: Read
MB/s > Higher Is Better
a . 11543.37 |=================================================================
b . 11543.16 |=================================================================
c . 11543.10 |=================================================================
d . 11543.49 |=================================================================
CacheBench
Test: Write
MB/s > Higher Is Better
a . 69134.68 |=================================================================
b . 69140.53 |=================================================================
c . 69142.44 |=================================================================
d . 69142.19 |=================================================================
CacheBench
Test: Read / Modify / Write
MB/s > Higher Is Better
a . 130857.58 |================================================================
b . 130069.85 |================================================================
c . 130806.25 |================================================================
d . 130851.31 |================================================================
TensorFlow 2.12
Device: CPU - Batch Size: 1 - Model: VGG-16
images/sec > Higher Is Better
a . 2.72 |=====================================================================
b . 2.70 |====================================================================
c . 2.72 |=====================================================================
d . 2.69 |====================================================================
TensorFlow 2.12
Device: CPU - Batch Size: 1 - Model: AlexNet
images/sec > Higher Is Better
a . 6.26 |=====================================================================
b . 6.23 |=====================================================================
c . 6.23 |=====================================================================
d . 6.21 |====================================================================
TensorFlow 2.12
Device: CPU - Batch Size: 16 - Model: VGG-16
images/sec > Higher Is Better
a . 8.51 |=====================================================================
b . 8.46 |=====================================================================
c . 8.48 |=====================================================================
d . 8.51 |=====================================================================
TensorFlow 2.12
Device: CPU - Batch Size: 16 - Model: AlexNet
images/sec > Higher Is Better
a . 100.44 |===================================================================
b . 100.01 |===================================================================
c . 100.08 |===================================================================
d . 99.90 |===================================================================
TensorFlow 2.12
Device: CPU - Batch Size: 1 - Model: GoogLeNet
images/sec > Higher Is Better
a . 9.94 |=========================================
b . 9.74 |========================================
c . 16.49 |====================================================================
d . 9.73 |========================================
TensorFlow 2.12
Device: CPU - Batch Size: 1 - Model: ResNet-50
images/sec > Higher Is Better
a . 8.85 |=====================================================================
b . 8.79 |====================================================================
c . 8.85 |=====================================================================
d . 8.89 |=====================================================================
TensorFlow 2.12
Device: CPU - Batch Size: 16 - Model: GoogLeNet
images/sec > Higher Is Better
a . 60.85 |====================================================================
b . 60.18 |===================================================================
c . 60.04 |===================================================================
d . 59.82 |===================================================================
TensorFlow 2.12
Device: CPU - Batch Size: 16 - Model: ResNet-50
images/sec > Higher Is Better
a . 19.87 |====================================================================
b . 19.45 |===================================================================
c . 19.61 |===================================================================
d . 19.81 |====================================================================
LeelaChessZero 0.30
Backend: BLAS
Nodes Per Second > Higher Is Better
a . 173 |======================================================
b . 219 |====================================================================
c . 225 |======================================================================
d . 213 |==================================================================
Y-Cruncher 0.8.3
Pi Digits To Calculate: 1B
Seconds < Lower Is Better
a . 15.55 |====================================================================
b . 15.50 |====================================================================
c . 15.53 |====================================================================
d . 15.50 |====================================================================
LeelaChessZero 0.30
Backend: Eigen
Nodes Per Second > Higher Is Better
a . 121 |=======================================================
b . 146 |==================================================================
c . 154 |======================================================================
d . 151 |=====================================================================
Y-Cruncher 0.8.3
Pi Digits To Calculate: 500M
Seconds < Lower Is Better
a . 7.325 |====================================================================
b . 7.349 |====================================================================
c . 7.301 |====================================================================
d . 7.297 |====================================================================
Neural Magic DeepSparse 1.6
Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream
items/sec > Higher Is Better
a . 26.84 |====================================================================
b . 26.65 |====================================================================
c . 26.67 |====================================================================
d . 26.69 |====================================================================
Neural Magic DeepSparse 1.6
Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream
ms/batch < Lower Is Better
a . 446.63 |===================================================================
b . 448.91 |===================================================================
c . 449.30 |===================================================================
d . 449.16 |===================================================================
Neural Magic DeepSparse 1.6
Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Stream
items/sec > Higher Is Better
a . 18.39 |====================================================================
b . 18.33 |====================================================================
c . 18.36 |====================================================================
d . 18.40 |====================================================================
Neural Magic DeepSparse 1.6
Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Stream
ms/batch < Lower Is Better
a . 54.37 |====================================================================
b . 54.54 |====================================================================
c . 54.46 |====================================================================
d . 54.35 |====================================================================
Neural Magic DeepSparse 1.6
Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Asynchronous Multi-Stream
items/sec > Higher Is Better
a . 685.12 |===================================================================
b . 681.00 |===================================================================
c . 683.25 |===================================================================
d . 681.53 |===================================================================
Neural Magic DeepSparse 1.6
Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Asynchronous Multi-Stream
ms/batch < Lower Is Better
a . 17.50 |====================================================================
b . 17.60 |====================================================================
c . 17.54 |====================================================================
d . 17.59 |====================================================================
Neural Magic DeepSparse 1.6
Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Synchronous Single-Stream
items/sec > Higher Is Better
a . 189.91 |==================================================================
b . 193.11 |===================================================================
c . 192.98 |===================================================================
d . 189.27 |==================================================================
Neural Magic DeepSparse 1.6
Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Synchronous Single-Stream
ms/batch < Lower Is Better
a . 5.2624 |===================================================================
b . 5.1753 |==================================================================
c . 5.1787 |==================================================================
d . 5.2804 |===================================================================
Neural Magic DeepSparse 1.6
Model: ResNet-50, Baseline - Scenario: Asynchronous Multi-Stream
items/sec > Higher Is Better
a . 307.01 |===================================================================
b . 304.77 |===================================================================
c . 306.50 |===================================================================
d . 305.86 |===================================================================
Neural Magic DeepSparse 1.6
Model: ResNet-50, Baseline - Scenario: Asynchronous Multi-Stream
ms/batch < Lower Is Better
a . 39.04 |===================================================================
b . 39.33 |====================================================================
c . 39.12 |====================================================================
d . 39.19 |====================================================================
Neural Magic DeepSparse 1.6
Model: ResNet-50, Baseline - Scenario: Synchronous Single-Stream
items/sec > Higher Is Better
a . 157.23 |===================================================================
b . 155.48 |==================================================================
c . 155.40 |==================================================================
d . 155.20 |==================================================================
Neural Magic DeepSparse 1.6
Model: ResNet-50, Baseline - Scenario: Synchronous Single-Stream
ms/batch < Lower Is Better
a . 6.3518 |==================================================================
b . 6.4230 |===================================================================
c . 6.4266 |===================================================================
d . 6.4345 |===================================================================
Neural Magic DeepSparse 1.6
Model: ResNet-50, Sparse INT8 - Scenario: Asynchronous Multi-Stream
items/sec > Higher Is Better
a . 2012.21 |==================================================================
b . 1962.70 |================================================================
c . 1970.61 |=================================================================
d . 1962.16 |================================================================
Neural Magic DeepSparse 1.6
Model: ResNet-50, Sparse INT8 - Scenario: Asynchronous Multi-Stream
ms/batch < Lower Is Better
a . 5.9507 |=================================================================
b . 6.1010 |===================================================================
c . 6.0747 |===================================================================
d . 6.1024 |===================================================================
Neural Magic DeepSparse 1.6
Model: ResNet-50, Sparse INT8 - Scenario: Synchronous Single-Stream
items/sec > Higher Is Better
a . 757.16 |==================================================================
b . 752.78 |==================================================================
c . 765.99 |===================================================================
d . 757.27 |==================================================================
Neural Magic DeepSparse 1.6
Model: ResNet-50, Sparse INT8 - Scenario: Synchronous Single-Stream
ms/batch < Lower Is Better
a . 1.3175 |===================================================================
b . 1.3252 |===================================================================
c . 1.3025 |==================================================================
d . 1.3175 |===================================================================
Neural Magic DeepSparse 1.6
Model: CV Detection, YOLOv5s COCO - Scenario: Asynchronous Multi-Stream
items/sec > Higher Is Better
a . 150.04 |===================================================================
b . 148.24 |==================================================================
c . 148.76 |==================================================================
d . 149.27 |===================================================================
Neural Magic DeepSparse 1.6
Model: CV Detection, YOLOv5s COCO - Scenario: Asynchronous Multi-Stream
ms/batch < Lower Is Better
a . 79.89 |===================================================================
b . 80.85 |====================================================================
c . 80.57 |====================================================================
d . 80.27 |====================================================================
Neural Magic DeepSparse 1.6
Model: CV Detection, YOLOv5s COCO - Scenario: Synchronous Single-Stream
items/sec > Higher Is Better
a . 98.52 |====================================================================
b . 98.56 |====================================================================
c . 98.65 |====================================================================
d . 98.77 |====================================================================
Neural Magic DeepSparse 1.6
Model: CV Detection, YOLOv5s COCO - Scenario: Synchronous Single-Stream
ms/batch < Lower Is Better
a . 10.14 |====================================================================
b . 10.14 |====================================================================
c . 10.13 |====================================================================
d . 10.12 |====================================================================
Neural Magic DeepSparse 1.6
Model: BERT-Large, NLP Question Answering - Scenario: Asynchronous Multi-Stream
items/sec > Higher Is Better
a . 32.52 |====================================================================
b . 32.36 |====================================================================
c . 32.42 |====================================================================
d . 32.38 |====================================================================
Neural Magic DeepSparse 1.6
Model: BERT-Large, NLP Question Answering - Scenario: Asynchronous Multi-Stream
ms/batch < Lower Is Better
a . 368.31 |===================================================================
b . 369.77 |===================================================================
c . 369.62 |===================================================================
d . 369.53 |===================================================================
Neural Magic DeepSparse 1.6
Model: BERT-Large, NLP Question Answering - Scenario: Synchronous Single-Stream
items/sec > Higher Is Better
a . 17.16 |====================================================================
b . 17.13 |====================================================================
c . 17.24 |====================================================================
d . 17.12 |====================================================================
Neural Magic DeepSparse 1.6
Model: BERT-Large, NLP Question Answering - Scenario: Synchronous Single-Stream
ms/batch < Lower Is Better
a . 58.25 |====================================================================
b . 58.35 |====================================================================
c . 58.00 |====================================================================
d . 58.41 |====================================================================
Neural Magic DeepSparse 1.6
Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream
items/sec > Higher Is Better
a . 306.99 |===================================================================
b . 305.38 |===================================================================
c . 306.52 |===================================================================
d . 299.83 |=================================================================
Neural Magic DeepSparse 1.6
Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream
ms/batch < Lower Is Better
a . 39.05 |==================================================================
b . 39.26 |===================================================================
c . 39.12 |===================================================================
d . 39.98 |====================================================================
Neural Magic DeepSparse 1.6
Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Stream
items/sec > Higher Is Better
a . 156.97 |===================================================================
b . 155.54 |==================================================================
c . 155.62 |==================================================================
d . 155.19 |==================================================================
Neural Magic DeepSparse 1.6
Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Stream
ms/batch < Lower Is Better
a . 6.3619 |==================================================================
b . 6.4208 |===================================================================
c . 6.4178 |===================================================================
d . 6.4346 |===================================================================
Neural Magic DeepSparse 1.6
Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Asynchronous Multi-Stream
items/sec > Higher Is Better
a . 151.47 |===================================================================
b . 149.78 |==================================================================
c . 149.95 |==================================================================
d . 149.76 |==================================================================
Neural Magic DeepSparse 1.6
Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Asynchronous Multi-Stream
ms/batch < Lower Is Better
a . 79.16 |===================================================================
b . 80.05 |====================================================================
c . 79.89 |====================================================================
d . 80.05 |====================================================================
Neural Magic DeepSparse 1.6
Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Synchronous Single-Stream
items/sec > Higher Is Better
a . 98.91 |====================================================================
b . 98.75 |====================================================================
c . 98.50 |====================================================================
d . 98.89 |====================================================================
Neural Magic DeepSparse 1.6
Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Synchronous Single-Stream
ms/batch < Lower Is Better
a . 10.10 |====================================================================
b . 10.12 |====================================================================
c . 10.15 |====================================================================
d . 10.11 |====================================================================
Neural Magic DeepSparse 1.6
Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Stream
items/sec > Higher Is Better
a . 224.18 |===================================================================
b . 221.71 |==================================================================
c . 223.05 |===================================================================
d . 222.72 |===================================================================
Neural Magic DeepSparse 1.6
Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Stream
ms/batch < Lower Is Better
a . 53.47 |===================================================================
b . 54.06 |====================================================================
c . 53.74 |====================================================================
d . 53.82 |====================================================================
Neural Magic DeepSparse 1.6
Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Stream
items/sec > Higher Is Better
a . 112.09 |===================================================================
b . 111.29 |===================================================================
c . 111.59 |===================================================================
d . 111.50 |===================================================================
Neural Magic DeepSparse 1.6
Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Stream
ms/batch < Lower Is Better
a . 8.9109 |===================================================================
b . 8.9755 |===================================================================
c . 8.9520 |===================================================================
d . 8.9586 |===================================================================
Neural Magic DeepSparse 1.6
Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Stream
items/sec > Higher Is Better
a . 30.49 |====================================================================
b . 30.28 |====================================================================
c . 30.45 |====================================================================
d . 30.32 |====================================================================
Neural Magic DeepSparse 1.6
Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Stream
ms/batch < Lower Is Better
a . 392.96 |===================================================================
b . 394.78 |===================================================================
c . 393.83 |===================================================================
d . 394.38 |===================================================================
Neural Magic DeepSparse 1.6
Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Synchronous Single-Stream
items/sec > Higher Is Better
a . 21.73 |====================================================================
b . 21.47 |===================================================================
c . 21.69 |====================================================================
d . 21.67 |====================================================================
Neural Magic DeepSparse 1.6
Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Synchronous Single-Stream
ms/batch < Lower Is Better
a . 46.00 |===================================================================
b . 46.55 |====================================================================
c . 46.09 |===================================================================
d . 46.13 |===================================================================
Neural Magic DeepSparse 1.6
Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Asynchronous Multi-Stream
items/sec > Higher Is Better
a . 335.35 |===================================================================
b . 333.37 |===================================================================
c . 333.64 |===================================================================
d . 334.64 |===================================================================
Neural Magic DeepSparse 1.6
Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Asynchronous Multi-Stream
ms/batch < Lower Is Better
a . 35.76 |====================================================================
b . 35.97 |====================================================================
c . 35.93 |====================================================================
d . 35.84 |====================================================================
Neural Magic DeepSparse 1.6
Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Synchronous Single-Stream
items/sec > Higher Is Better
a . 76.14 |====================================================================
b . 75.51 |===================================================================
c . 75.99 |====================================================================
d . 75.71 |====================================================================
Neural Magic DeepSparse 1.6
Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Synchronous Single-Stream
ms/batch < Lower Is Better
a . 13.13 |===================================================================
b . 13.24 |====================================================================
c . 13.15 |====================================================================
d . 13.20 |====================================================================
Neural Magic DeepSparse 1.6
Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream
items/sec > Higher Is Better
a . 26.91 |====================================================================
b . 26.67 |===================================================================
c . 26.71 |===================================================================
d . 26.54 |===================================================================
Neural Magic DeepSparse 1.6
Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream
ms/batch < Lower Is Better
a . 445.26 |==================================================================
b . 448.04 |===================================================================
c . 448.90 |===================================================================
d . 448.03 |===================================================================
Neural Magic DeepSparse 1.6
Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Stream
items/sec > Higher Is Better
a . 18.47 |====================================================================
b . 18.40 |====================================================================
c . 18.46 |====================================================================
d . 18.44 |====================================================================
Neural Magic DeepSparse 1.6
Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Stream
ms/batch < Lower Is Better
a . 54.12 |====================================================================
b . 54.33 |====================================================================
c . 54.15 |====================================================================
d . 54.20 |====================================================================
PyTorch 2.1
Device: CPU - Batch Size: 1 - Model: ResNet-50
batches/sec > Higher Is Better
a . 40.68 |====================================================================
b . 40.42 |===================================================================
c . 40.82 |====================================================================
d . 40.35 |===================================================================
PyTorch 2.1
Device: CPU - Batch Size: 16 - Model: ResNet-50
batches/sec > Higher Is Better
a . 32.50 |====================================================================
b . 32.10 |===================================================================
c . 31.65 |==================================================================
d . 32.21 |===================================================================
PyTorch 2.1
Device: CPU - Batch Size: 256 - Model: ResNet-50
batches/sec > Higher Is Better
a . 31.92 |====================================================================
b . 32.15 |====================================================================
c . 31.59 |===================================================================
d . 32.07 |====================================================================
Llama.cpp b1808
Model: llama-2-7b.Q4_0.gguf
Tokens Per Second > Higher Is Better
a . 20.76 |===================================================================
b . 20.95 |====================================================================
c . 20.74 |===================================================================
d . 20.68 |===================================================================
Llama.cpp b1808
Model: llama-2-13b.Q4_0.gguf
Tokens Per Second > Higher Is Better
a . 11.64 |====================================================================
b . 11.32 |==================================================================
c . 11.27 |==================================================================
d . 11.25 |==================================================================
Llama.cpp b1808
Model: llama-2-70b-chat.Q5_0.gguf
Tokens Per Second > Higher Is Better
a . 1.94 |=====================================================================
b . 1.94 |=====================================================================
c . 1.95 |=====================================================================
d . 1.95 |=====================================================================
Llamafile 0.6
Test: llava-v1.5-7b-q4 - Acceleration: CPU
Tokens Per Second > Higher Is Better
a . 17.22 |====================================================================
b . 17.26 |====================================================================
c . 17.30 |====================================================================
d . 17.25 |====================================================================
Llamafile 0.6
Test: mistral-7b-instruct-v0.2.Q8_0 - Acceleration: CPU
Tokens Per Second > Higher Is Better
a . 10.13 |====================================================================
b . 10.15 |====================================================================
c . 10.14 |====================================================================
d . 10.13 |====================================================================
Llamafile 0.6
Test: wizardcoder-python-34b-v1.0.Q6_K - Acceleration: CPU
Tokens Per Second > Higher Is Better
a . 3.25 |=====================================================================
b . 3.25 |=====================================================================
c . 3.25 |=====================================================================
d . 3.25 |=====================================================================
SVT-AV1 1.8
Encoder Mode: Preset 4 - Input: Bosphorus 4K
Frames Per Second > Higher Is Better
a . 6.677 |====================================================================
b . 6.669 |====================================================================
c . 6.678 |====================================================================
d . 6.633 |====================================================================
SVT-AV1 1.8
Encoder Mode: Preset 8 - Input: Bosphorus 4K
Frames Per Second > Higher Is Better
a . 61.53 |====================================================================
b . 61.83 |====================================================================
c . 61.47 |====================================================================
d . 60.95 |===================================================================
SVT-AV1 1.8
Encoder Mode: Preset 12 - Input: Bosphorus 4K
Frames Per Second > Higher Is Better
a . 190.92 |==================================================================
b . 190.40 |==================================================================
c . 192.16 |===================================================================
d . 192.68 |===================================================================
SVT-AV1 1.8
Encoder Mode: Preset 13 - Input: Bosphorus 4K
Frames Per Second > Higher Is Better
a . 190.79 |==================================================================
b . 192.73 |===================================================================
c . 189.25 |==================================================================
d . 192.60 |===================================================================
SVT-AV1 1.8
Encoder Mode: Preset 4 - Input: Bosphorus 1080p
Frames Per Second > Higher Is Better
a . 18.80 |====================================================================
b . 18.68 |===================================================================
c . 18.41 |==================================================================
d . 18.92 |====================================================================
SVT-AV1 1.8
Encoder Mode: Preset 8 - Input: Bosphorus 1080p
Frames Per Second > Higher Is Better
a . 122.95 |===================================================================
b . 123.29 |===================================================================
c . 122.95 |===================================================================
d . 122.62 |===================================================================
SVT-AV1 1.8
Encoder Mode: Preset 12 - Input: Bosphorus 1080p
Frames Per Second > Higher Is Better
a . 501.42 |==================================================================
b . 506.60 |===================================================================
c . 501.16 |==================================================================
d . 506.17 |===================================================================
SVT-AV1 1.8
Encoder Mode: Preset 13 - Input: Bosphorus 1080p
Frames Per Second > Higher Is Better
a . 543.55 |===============================================================
b . 573.04 |==================================================================
c . 580.47 |===================================================================
d . 565.91 |=================================================================
rav1e 0.7
Speed: 1
Frames Per Second > Higher Is Better
a . 1.044 |===================================================================
b . 1.048 |====================================================================
c . 1.044 |===================================================================
d . 1.054 |====================================================================
rav1e 0.7
Speed: 5
Frames Per Second > Higher Is Better
a . 3.747 |=================================================================
b . 3.791 |==================================================================
c . 3.769 |==================================================================
d . 3.891 |====================================================================
rav1e 0.7
Speed: 6
Frames Per Second > Higher Is Better
a . 5.261 |====================================================================
b . 5.292 |====================================================================
c . 5.282 |====================================================================
d . 5.191 |===================================================================
rav1e 0.7
Speed: 10
Frames Per Second > Higher Is Better
a . 10.63 |==================================================================
b . 10.89 |===================================================================
c . 10.96 |====================================================================
d . 11.02 |====================================================================
Speedb 2.7
Test: Random Fill
Op/s > Higher Is Better
a . 558330 |===================================================================
b . 554997 |===================================================================
c . 557348 |===================================================================
d . 556675 |===================================================================
Speedb 2.7
Test: Random Read
Op/s > Higher Is Better
a . 148134848 |================================================================
b . 147432214 |================================================================
c . 146285738 |===============================================================
d . 146473036 |===============================================================
Speedb 2.7
Test: Update Random
Op/s > Higher Is Better
a . 431692 |===================================================================
b . 418848 |=================================================================
c . 423788 |==================================================================
d . 417457 |=================================================================
Speedb 2.7
Test: Sequential Fill
Op/s > Higher Is Better
a . 620607 |===================================================================
b . 618776 |===================================================================
c . 604758 |=================================================================
d . 612428 |==================================================================
Speedb 2.7
Test: Random Fill Sync
Op/s > Higher Is Better
a . 47488 |====================================================================
b . 47708 |====================================================================
c . 47373 |====================================================================
d . 47267 |===================================================================
Speedb 2.7
Test: Read While Writing
Op/s > Higher Is Better
a . 7004007 |=================================================================
b . 7070407 |==================================================================
c . 7047502 |==================================================================
d . 6896007 |================================================================
Speedb 2.7
Test: Read Random Write Random
Op/s > Higher Is Better
a . 2327911 |==================================================================
b . 2307686 |=================================================================
c . 2320670 |==================================================================
d . 2316804 |==================================================================