Intel Core i9-14900K testing with a ASUS PRIME Z790-P WIFI (1402 BIOS) and ASUS Intel RPL-S 31GB on Ubuntu 23.10 via the Phoronix Test Suite.
Compare your own system(s) to this result file with the
Phoronix Test Suite by running the command:
phoronix-test-suite benchmark 2403152-PTS-DEEPSPAR00
deepsparse 1.7 raptor lake
Intel Core i9-14900K testing with a ASUS PRIME Z790-P WIFI (1402 BIOS) and ASUS Intel RPL-S 31GB on Ubuntu 23.10 via the Phoronix Test Suite.
a:
Processor: Intel Core i9-14900K @ 5.70GHz (24 Cores / 32 Threads), Motherboard: ASUS PRIME Z790-P WIFI (1402 BIOS), Chipset: Intel Device 7a27, Memory: 2 x 16GB DRAM-6000MT/s Corsair CMK32GX5M2B6000C36, Disk: Western Digital WD_BLACK SN850X 1000GB, Graphics: ASUS Intel RPL-S 31GB (1650MHz), Audio: Realtek ALC897, Monitor: ASUS VP28U
OS: Ubuntu 23.10, Kernel: 6.8.0-phx (x86_64), Desktop: GNOME Shell 45.1, Display Server: X Server 1.21.1.7, OpenGL: 4.6 Mesa 24.0~git2312240600.c05261~oibaf~m (git-c05261a 2023-12-24 mantic-oibaf-ppa), Compiler: GCC 13.2.0, File-System: ext4, Screen Resolution: 3840x2160
b:
Processor: Intel Core i9-14900K @ 5.70GHz (24 Cores / 32 Threads), Motherboard: ASUS PRIME Z790-P WIFI (1402 BIOS), Chipset: Intel Device 7a27, Memory: 2 x 16GB DRAM-6000MT/s Corsair CMK32GX5M2B6000C36, Disk: Western Digital WD_BLACK SN850X 1000GB, Graphics: ASUS Intel RPL-S 31GB (1650MHz), Audio: Realtek ALC897, Monitor: ASUS VP28U
OS: Ubuntu 23.10, Kernel: 6.8.0-phx (x86_64), Desktop: GNOME Shell 45.1, Display Server: X Server 1.21.1.7, OpenGL: 4.6 Mesa 24.0~git2312240600.c05261~oibaf~m (git-c05261a 2023-12-24 mantic-oibaf-ppa), Compiler: GCC 13.2.0, File-System: ext4, Screen Resolution: 3840x2160
c:
Processor: Intel Core i9-14900K @ 5.70GHz (24 Cores / 32 Threads), Motherboard: ASUS PRIME Z790-P WIFI (1402 BIOS), Chipset: Intel Device 7a27, Memory: 2 x 16GB DRAM-6000MT/s Corsair CMK32GX5M2B6000C36, Disk: Western Digital WD_BLACK SN850X 1000GB, Graphics: ASUS Intel RPL-S 31GB (1650MHz), Audio: Realtek ALC897, Monitor: ASUS VP28U
OS: Ubuntu 23.10, Kernel: 6.8.0-phx (x86_64), Desktop: GNOME Shell 45.1, Display Server: X Server 1.21.1.7, OpenGL: 4.6 Mesa 24.0~git2312240600.c05261~oibaf~m (git-c05261a 2023-12-24 mantic-oibaf-ppa), Compiler: GCC 13.2.0, File-System: ext4, Screen Resolution: 3840x2160
d:
Processor: Intel Core i9-14900K @ 5.70GHz (24 Cores / 32 Threads), Motherboard: ASUS PRIME Z790-P WIFI (1402 BIOS), Chipset: Intel Device 7a27, Memory: 2 x 16GB DRAM-6000MT/s Corsair CMK32GX5M2B6000C36, Disk: Western Digital WD_BLACK SN850X 1000GB, Graphics: ASUS Intel RPL-S 31GB (1650MHz), Audio: Realtek ALC897, Monitor: ASUS VP28U
OS: Ubuntu 23.10, Kernel: 6.8.0-phx (x86_64), Desktop: GNOME Shell 45.1, Display Server: X Server 1.21.1.7, OpenGL: 4.6 Mesa 24.0~git2312240600.c05261~oibaf~m (git-c05261a 2023-12-24 mantic-oibaf-ppa), Compiler: GCC 13.2.0, File-System: ext4, Screen Resolution: 3840x2160
e:
Processor: Intel Core i9-14900K @ 5.70GHz (24 Cores / 32 Threads), Motherboard: ASUS PRIME Z790-P WIFI (1402 BIOS), Chipset: Intel Device 7a27, Memory: 2 x 16GB DRAM-6000MT/s Corsair CMK32GX5M2B6000C36, Disk: Western Digital WD_BLACK SN850X 1000GB, Graphics: ASUS Intel RPL-S 31GB (1650MHz), Audio: Realtek ALC897, Monitor: ASUS VP28U
OS: Ubuntu 23.10, Kernel: 6.8.0-phx (x86_64), Desktop: GNOME Shell 45.1, Display Server: X Server 1.21.1.7, OpenGL: 4.6 Mesa 24.0~git2312240600.c05261~oibaf~m (git-c05261a 2023-12-24 mantic-oibaf-ppa), Compiler: GCC 13.2.0, File-System: ext4, Screen Resolution: 3840x2160
Neural Magic DeepSparse 1.7
Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream
items/sec > Higher Is Better
a . 10.68 |===================================================================
b . 10.85 |====================================================================
c . 10.72 |===================================================================
d . 10.65 |===================================================================
e . 10.86 |====================================================================
Neural Magic DeepSparse 1.7
Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream
ms/batch < Lower Is Better
a . 374.66 |===================================================================
b . 368.59 |==================================================================
c . 373.12 |===================================================================
d . 375.63 |===================================================================
e . 368.23 |==================================================================
Neural Magic DeepSparse 1.7
Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Stream
items/sec > Higher Is Better
a . 9.6041 |===================================================================
b . 9.5789 |===================================================================
c . 9.5059 |==================================================================
d . 9.4750 |==================================================================
e . 9.5673 |===================================================================
Neural Magic DeepSparse 1.7
Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Stream
ms/batch < Lower Is Better
a . 104.12 |==================================================================
b . 104.39 |==================================================================
c . 105.19 |===================================================================
d . 105.54 |===================================================================
e . 104.52 |==================================================================
Neural Magic DeepSparse 1.7
Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Asynchronous Multi-Stream
items/sec > Higher Is Better
a . 436.25 |===================================================================
b . 434.22 |===================================================================
c . 436.12 |===================================================================
d . 435.85 |===================================================================
e . 433.88 |===================================================================
Neural Magic DeepSparse 1.7
Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Asynchronous Multi-Stream
ms/batch < Lower Is Better
a . 9.1560 |===================================================================
b . 9.1990 |===================================================================
c . 9.1596 |===================================================================
d . 9.1642 |===================================================================
e . 9.2062 |===================================================================
Neural Magic DeepSparse 1.7
Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Synchronous Single-Stream
items/sec > Higher Is Better
a . 324.56 |==================================================================
b . 327.19 |===================================================================
c . 326.89 |===================================================================
d . 327.29 |===================================================================
e . 328.56 |===================================================================
Neural Magic DeepSparse 1.7
Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Synchronous Single-Stream
ms/batch < Lower Is Better
a . 3.0776 |===================================================================
b . 3.0531 |==================================================================
c . 3.0560 |===================================================================
d . 3.0521 |==================================================================
e . 3.0404 |==================================================================
Neural Magic DeepSparse 1.7
Model: ResNet-50, Baseline - Scenario: Asynchronous Multi-Stream
items/sec > Higher Is Better
a . 143.04 |===================================================================
b . 143.19 |===================================================================
c . 143.39 |===================================================================
d . 143.00 |===================================================================
e . 143.57 |===================================================================
Neural Magic DeepSparse 1.7
Model: ResNet-50, Baseline - Scenario: Asynchronous Multi-Stream
ms/batch < Lower Is Better
a . 27.95 |====================================================================
b . 27.92 |====================================================================
c . 27.88 |====================================================================
d . 27.96 |====================================================================
e . 27.85 |====================================================================
Neural Magic DeepSparse 1.7
Model: ResNet-50, Baseline - Scenario: Synchronous Single-Stream
items/sec > Higher Is Better
a . 124.99 |==================================================================
b . 126.07 |===================================================================
c . 125.77 |===================================================================
d . 125.00 |==================================================================
e . 125.80 |===================================================================
Neural Magic DeepSparse 1.7
Model: ResNet-50, Baseline - Scenario: Synchronous Single-Stream
ms/batch < Lower Is Better
a . 7.9967 |===================================================================
b . 7.9282 |==================================================================
c . 7.9467 |===================================================================
d . 7.9961 |===================================================================
e . 7.9452 |===================================================================
Neural Magic DeepSparse 1.7
Model: ResNet-50, Sparse INT8 - Scenario: Asynchronous Multi-Stream
items/sec > Higher Is Better
a . 1152.25 |=================================================================
b . 1156.18 |==================================================================
c . 1129.73 |================================================================
d . 1152.75 |==================================================================
e . 1161.30 |==================================================================
Neural Magic DeepSparse 1.7
Model: ResNet-50, Sparse INT8 - Scenario: Asynchronous Multi-Stream
ms/batch < Lower Is Better
a . 3.4607 |==================================================================
b . 3.4488 |=================================================================
c . 3.5300 |===================================================================
d . 3.4601 |==================================================================
e . 3.4336 |=================================================================
Neural Magic DeepSparse 1.7
Model: ResNet-50, Sparse INT8 - Scenario: Synchronous Single-Stream
items/sec > Higher Is Better
a . 975.81 |================================================================
b . 941.79 |=============================================================
c . 927.77 |=============================================================
d . 1011.71 |==================================================================
e . 994.77 |=================================================================
Neural Magic DeepSparse 1.7
Model: ResNet-50, Sparse INT8 - Scenario: Synchronous Single-Stream
ms/batch < Lower Is Better
a . 1.0225 |================================================================
b . 1.0591 |==================================================================
c . 1.0746 |===================================================================
d . 0.9856 |=============================================================
e . 1.0024 |==============================================================
Neural Magic DeepSparse 1.7
Model: Llama2 Chat 7b Quantized - Scenario: Asynchronous Multi-Stream
items/sec > Higher Is Better
a . 7.6693 |==================================================================
b . 7.7518 |===================================================================
c . 7.5635 |=================================================================
d . 7.6435 |==================================================================
e . 7.6128 |==================================================================
Neural Magic DeepSparse 1.7
Model: Llama2 Chat 7b Quantized - Scenario: Asynchronous Multi-Stream
ms/batch < Lower Is Better
a . 518.25 |==================================================================
b . 512.69 |=================================================================
c . 525.39 |===================================================================
d . 520.01 |==================================================================
e . 522.00 |===================================================================
Neural Magic DeepSparse 1.7
Model: Llama2 Chat 7b Quantized - Scenario: Synchronous Single-Stream
items/sec > Higher Is Better
a . 11.25 |====================================================================
b . 11.25 |====================================================================
c . 11.24 |====================================================================
d . 11.24 |====================================================================
e . 11.24 |====================================================================
Neural Magic DeepSparse 1.7
Model: Llama2 Chat 7b Quantized - Scenario: Synchronous Single-Stream
ms/batch < Lower Is Better
a . 88.87 |====================================================================
b . 88.87 |====================================================================
c . 88.94 |====================================================================
d . 88.93 |====================================================================
e . 88.98 |====================================================================
Neural Magic DeepSparse 1.7
Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream
items/sec > Higher Is Better
a . 142.92 |==================================================================
b . 143.84 |===================================================================
c . 143.29 |==================================================================
d . 143.24 |==================================================================
e . 144.75 |===================================================================
Neural Magic DeepSparse 1.7
Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream
ms/batch < Lower Is Better
a . 27.97 |====================================================================
b . 27.79 |====================================================================
c . 27.90 |====================================================================
d . 27.91 |====================================================================
e . 27.62 |===================================================================
Neural Magic DeepSparse 1.7
Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Stream
items/sec > Higher Is Better
a . 124.33 |==================================================================
b . 126.67 |===================================================================
c . 125.87 |===================================================================
d . 125.35 |==================================================================
e . 126.11 |===================================================================
Neural Magic DeepSparse 1.7
Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Stream
ms/batch < Lower Is Better
a . 8.0391 |===================================================================
b . 7.8904 |==================================================================
c . 7.9405 |==================================================================
d . 7.9734 |==================================================================
e . 7.9248 |==================================================================
Neural Magic DeepSparse 1.7
Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Asynchronous Multi-Stream
items/sec > Higher Is Better
a . 71.70 |====================================================================
b . 71.75 |====================================================================
c . 71.76 |====================================================================
d . 71.76 |====================================================================
e . 71.78 |====================================================================
Neural Magic DeepSparse 1.7
Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Asynchronous Multi-Stream
ms/batch < Lower Is Better
a . 55.77 |====================================================================
b . 55.74 |====================================================================
c . 55.72 |====================================================================
d . 55.73 |====================================================================
e . 55.71 |====================================================================
Neural Magic DeepSparse 1.7
Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Synchronous Single-Stream
items/sec > Higher Is Better
a . 67.26 |====================================================================
b . 67.62 |====================================================================
c . 67.38 |====================================================================
d . 67.41 |====================================================================
e . 67.32 |====================================================================
Neural Magic DeepSparse 1.7
Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Synchronous Single-Stream
ms/batch < Lower Is Better
a . 14.86 |====================================================================
b . 14.78 |====================================================================
c . 14.84 |====================================================================
d . 14.83 |====================================================================
e . 14.85 |====================================================================
Neural Magic DeepSparse 1.7
Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Stream
items/sec > Higher Is Better
a . 81.49 |====================================================================
b . 81.50 |====================================================================
c . 81.51 |====================================================================
d . 81.68 |====================================================================
e . 81.17 |====================================================================
Neural Magic DeepSparse 1.7
Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Stream
ms/batch < Lower Is Better
a . 49.07 |====================================================================
b . 49.07 |====================================================================
c . 49.06 |====================================================================
d . 48.96 |====================================================================
e . 49.26 |====================================================================
Neural Magic DeepSparse 1.7
Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Stream
items/sec > Higher Is Better
a . 49.79 |===================================================================
b . 50.85 |====================================================================
c . 50.89 |====================================================================
d . 50.44 |===================================================================
e . 50.09 |===================================================================
Neural Magic DeepSparse 1.7
Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Stream
ms/batch < Lower Is Better
a . 20.08 |====================================================================
b . 19.66 |===================================================================
c . 19.65 |===================================================================
d . 19.82 |===================================================================
e . 19.96 |====================================================================
Neural Magic DeepSparse 1.7
Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Stream
items/sec > Higher Is Better
a . 14.82 |====================================================================
b . 14.92 |====================================================================
c . 14.92 |====================================================================
d . 14.83 |====================================================================
e . 14.89 |====================================================================
Neural Magic DeepSparse 1.7
Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Stream
ms/batch < Lower Is Better
a . 269.97 |===================================================================
b . 268.08 |===================================================================
c . 268.00 |===================================================================
d . 269.62 |===================================================================
e . 268.55 |===================================================================
Neural Magic DeepSparse 1.7
Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Synchronous Single-Stream
items/sec > Higher Is Better
a . 13.70 |===================================================================
b . 14.00 |====================================================================
c . 14.00 |====================================================================
d . 13.87 |===================================================================
e . 13.90 |====================================================================
Neural Magic DeepSparse 1.7
Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Synchronous Single-Stream
ms/batch < Lower Is Better
a . 72.99 |====================================================================
b . 71.43 |===================================================================
c . 71.44 |===================================================================
d . 72.07 |===================================================================
e . 71.94 |===================================================================
Neural Magic DeepSparse 1.7
Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Asynchronous Multi-Stream
items/sec > Higher Is Better
a . 196.62 |===================================================================
b . 197.48 |===================================================================
c . 197.28 |===================================================================
d . 197.20 |===================================================================
e . 197.57 |===================================================================
Neural Magic DeepSparse 1.7
Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Asynchronous Multi-Stream
ms/batch < Lower Is Better
a . 20.33 |====================================================================
b . 20.24 |====================================================================
c . 20.26 |====================================================================
d . 20.27 |====================================================================
e . 20.23 |====================================================================
Neural Magic DeepSparse 1.7
Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Synchronous Single-Stream
items/sec > Higher Is Better
a . 125.92 |===================================================================
b . 125.45 |===================================================================
c . 125.63 |===================================================================
d . 125.57 |===================================================================
e . 125.41 |===================================================================
Neural Magic DeepSparse 1.7
Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Synchronous Single-Stream
ms/batch < Lower Is Better
a . 7.9348 |===================================================================
b . 7.9643 |===================================================================
c . 7.9518 |===================================================================
d . 7.9571 |===================================================================
e . 7.9673 |===================================================================
Neural Magic DeepSparse 1.7
Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream
items/sec > Higher Is Better
a . 10.53 |====================================================================
b . 10.55 |====================================================================
c . 10.57 |====================================================================
d . 10.58 |====================================================================
e . 10.60 |====================================================================
Neural Magic DeepSparse 1.7
Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream
ms/batch < Lower Is Better
a . 379.71 |===================================================================
b . 379.14 |===================================================================
c . 378.26 |===================================================================
d . 378.17 |===================================================================
e . 377.36 |===================================================================
Neural Magic DeepSparse 1.7
Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Stream
items/sec > Higher Is Better
a . 9.4444 |===================================================================
b . 9.4148 |===================================================================
c . 9.4838 |===================================================================
d . 9.3639 |==================================================================
e . 9.3782 |==================================================================
Neural Magic DeepSparse 1.7
Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Stream
ms/batch < Lower Is Better
a . 105.88 |==================================================================
b . 106.21 |===================================================================
c . 105.44 |==================================================================
d . 106.80 |===================================================================
e . 106.63 |===================================================================