deepsparse 1.7 raptor lake

Intel Core i9-14900K testing with a ASUS PRIME Z790-P WIFI (1402 BIOS) and ASUS Intel RPL-S 31GB on Ubuntu 23.10 via the Phoronix Test Suite.

Compare your own system(s) to this result file with the Phoronix Test Suite by running the command: phoronix-test-suite benchmark 2403152-PTS-DEEPSPAR00
Jump To Table - Results

View

Do Not Show Noisy Results
Do Not Show Results With Incomplete Data
Do Not Show Results With Little Change/Spread
List Notable Results

Statistics

Show Overall Harmonic Mean(s)
Show Overall Geometric Mean
Show Wins / Losses Counts (Pie Chart)
Normalize Results
Remove Outliers Before Calculating Averages

Graph Settings

Force Line Graphs Where Applicable
Convert To Scalar Where Applicable
Prefer Vertical Bar Graphs

Multi-Way Comparison

Condense Multi-Option Tests Into Single Result Graphs

Table

Show Detailed System Result Table

Run Management

Highlight
Result
Hide
Result
Result
Identifier
View Logs
Performance Per
Dollar
Date
Run
  Test
  Duration
a
March 15
  1 Hour, 30 Minutes
b
March 15
  30 Minutes
c
March 15
  30 Minutes
d
March 15
  1 Hour, 28 Minutes
e
March 15
  30 Minutes
Invert Hiding All Results Option
  54 Minutes

Only show results where is faster than
Only show results matching title/arguments (delimit multiple options with a comma):
Do not show results matching title/arguments (delimit multiple options with a comma):

deepsparse 1.7 raptor lake, "Neural Magic DeepSparse 1.7 - Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream", Higher Results Are Better "a",10.8535,10.5674,10.6111 "b", "c", "d",10.8205,10.5774,10.5508 "e", "Neural Magic DeepSparse 1.7 - Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream", Lower Results Are Better "a",368.5289,378.5006,376.9424 "b", "c", "d",369.6486,378.1402,379.0992 "e", "Neural Magic DeepSparse 1.7 - Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Stream", Higher Results Are Better "a",9.6193,9.6291,9.564 "b", "c", "d",9.4483,9.4971,9.4795 "e", "Neural Magic DeepSparse 1.7 - Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Stream", Lower Results Are Better "a",103.9535,103.8477,104.5551 "b", "c", "d",105.8346,105.2914,105.4873 "e", "Neural Magic DeepSparse 1.7 - Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Asynchronous Multi-Stream", Higher Results Are Better "a",438.4461,433.0494,437.2494 "b", "c", "d",436.7894,435.6648,435.1081 "e", "Neural Magic DeepSparse 1.7 - Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Asynchronous Multi-Stream", Lower Results Are Better "a",9.1096,9.2236,9.1348 "b", "c", "d",9.1443,9.1686,9.1797 "e", "Neural Magic DeepSparse 1.7 - Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Synchronous Single-Stream", Higher Results Are Better "a",324.1779,325.4956,324.0154 "b", "c", "d",326.4409,326.9762,328.455 "e", "Neural Magic DeepSparse 1.7 - Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Synchronous Single-Stream", Lower Results Are Better "a",3.081,3.069,3.0829 "b", "c", "d",3.06,3.055,3.0414 "e", "Neural Magic DeepSparse 1.7 - Model: ResNet-50, Baseline - Scenario: Asynchronous Multi-Stream", Higher Results Are Better "a",143.5489,142.9199,142.6446 "b", "c", "d",143.029,143.0366,142.929 "e", "Neural Magic DeepSparse 1.7 - Model: ResNet-50, Baseline - Scenario: Asynchronous Multi-Stream", Lower Results Are Better "a",27.8495,27.9719,28.0263 "b", "c", "d",27.9511,27.9498,27.9707 "e", "Neural Magic DeepSparse 1.7 - Model: ResNet-50, Baseline - Scenario: Synchronous Single-Stream", Higher Results Are Better "a",125.5649,124.5289,124.8817 "b", "c", "d",126.1297,124.9999,123.8796 "e", "Neural Magic DeepSparse 1.7 - Model: ResNet-50, Baseline - Scenario: Synchronous Single-Stream", Lower Results Are Better "a",7.96,8.0264,8.0038 "b", "c", "d",7.9244,7.9958,8.068 "e", "Neural Magic DeepSparse 1.7 - Model: ResNet-50, Sparse INT8 - Scenario: Asynchronous Multi-Stream", Higher Results Are Better "a",1153.323,1145.7325,1157.7002 "b", "c", "d",1166.3533,1159.95,1131.9537 "e", "Neural Magic DeepSparse 1.7 - Model: ResNet-50, Sparse INT8 - Scenario: Asynchronous Multi-Stream", Lower Results Are Better "a",3.4573,3.4809,3.4439 "b", "c", "d",3.4189,3.4383,3.5231 "e", "Neural Magic DeepSparse 1.7 - Model: ResNet-50, Sparse INT8 - Scenario: Synchronous Single-Stream", Higher Results Are Better "a",967.3955,992.9423,937.9863,990.3089,990.4057 "b", "c", "d",1013.7906,1009.8664,1011.4869 "e", "Neural Magic DeepSparse 1.7 - Model: ResNet-50, Sparse INT8 - Scenario: Synchronous Single-Stream", Lower Results Are Better "a",1.0311,1.0045,1.0631,1.0069,1.007 "b", "c", "d",0.9838,0.9874,0.9857 "e", "Neural Magic DeepSparse 1.7 - Model: Llama2 Chat 7b Quantized - Scenario: Asynchronous Multi-Stream", Higher Results Are Better "a",7.5298,7.7403,7.7379 "b", "c", "d",7.6099,7.5366,7.7841 "e", "Neural Magic DeepSparse 1.7 - Model: Llama2 Chat 7b Quantized - Scenario: Asynchronous Multi-Stream", Lower Results Are Better "a",527.696,513.4357,513.6056 "b", "c", "d",522.1978,527.252,510.5847 "e", "Neural Magic DeepSparse 1.7 - Model: Llama2 Chat 7b Quantized - Scenario: Synchronous Single-Stream", Higher Results Are Better "a",11.2453,11.2568,11.2503 "b", "c", "d",11.2472,11.2446,11.2382 "e", "Neural Magic DeepSparse 1.7 - Model: Llama2 Chat 7b Quantized - Scenario: Synchronous Single-Stream", Lower Results Are Better "a",88.9134,88.8192,88.8713 "b", "c", "d",88.8979,88.9172,88.9689 "e", "Neural Magic DeepSparse 1.7 - Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream", Higher Results Are Better "a",143.2492,142.7386,142.7574 "b", "c", "d",143.668,142.9666,143.0754 "e", "Neural Magic DeepSparse 1.7 - Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream", Lower Results Are Better "a",27.9091,28.0091,28.0052 "b", "c", "d",27.8269,27.9637,27.9425 "e", "Neural Magic DeepSparse 1.7 - Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Stream", Higher Results Are Better "a",124.681,124.5633,123.7514 "b", "c", "d",126.1382,124.9035,125.0119 "e", "Neural Magic DeepSparse 1.7 - Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Stream", Lower Results Are Better "a",8.0168,8.0238,8.0767 "b", "c", "d",7.9236,8.0016,7.9951 "e", "Neural Magic DeepSparse 1.7 - Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Asynchronous Multi-Stream", Higher Results Are Better "a",71.7594,71.7,71.6332 "b", "c", "d",71.8093,71.7061,71.766 "e", "Neural Magic DeepSparse 1.7 - Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Asynchronous Multi-Stream", Lower Results Are Better "a",55.7261,55.7728,55.8249 "b", "c", "d",55.6874,55.7679,55.7214 "e", "Neural Magic DeepSparse 1.7 - Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Synchronous Single-Stream", Higher Results Are Better "a",67.2246,67.253,67.3097 "b", "c", "d",67.3716,67.6645,67.1968 "e", "Neural Magic DeepSparse 1.7 - Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Synchronous Single-Stream", Lower Results Are Better "a",14.8715,14.8641,14.8527 "b", "c", "d",14.839,14.7751,14.8774 "e", "Neural Magic DeepSparse 1.7 - Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Stream", Higher Results Are Better "a",81.514,81.6105,81.3492 "b", "c", "d",81.6232,82.0929,81.3308 "e", "Neural Magic DeepSparse 1.7 - Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Stream", Lower Results Are Better "a",49.0581,48.9987,49.1573 "b", "c", "d",48.9914,48.7106,49.1672 "e", "Neural Magic DeepSparse 1.7 - Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Stream", Higher Results Are Better "a",49.8306,49.5314,49.9971 "b", "c", "d",50.0572,50.398,50.8663 "e", "Neural Magic DeepSparse 1.7 - Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Stream", Lower Results Are Better "a",20.0645,20.1856,19.9974 "b", "c", "d",19.9736,19.8384,19.656 "e", "Neural Magic DeepSparse 1.7 - Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Stream", Higher Results Are Better "a",14.9213,14.7109,14.8147 "b", "c", "d",14.8981,14.7418,14.8646 "e", "Neural Magic DeepSparse 1.7 - Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Stream", Lower Results Are Better "a",268.0533,271.8841,269.9805 "b", "c", "d",268.4708,271.3167,269.0749 "e", "Neural Magic DeepSparse 1.7 - Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Synchronous Single-Stream", Higher Results Are Better "a",13.6244,13.7882,13.6846 "b", "c", "d",13.9902,13.868,13.7656 "e", "Neural Magic DeepSparse 1.7 - Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Synchronous Single-Stream", Lower Results Are Better "a",73.3876,72.5154,73.0642 "b", "c", "d",71.4681,72.0977,72.6343 "e", "Neural Magic DeepSparse 1.7 - Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Asynchronous Multi-Stream", Higher Results Are Better "a",196.3445,196.7321,196.7841 "b", "c", "d",197.434,197.3255,196.829 "e", "Neural Magic DeepSparse 1.7 - Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Asynchronous Multi-Stream", Lower Results Are Better "a",20.3587,20.3187,20.313 "b", "c", "d",20.2455,20.2571,20.3084 "e", "Neural Magic DeepSparse 1.7 - Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Synchronous Single-Stream", Higher Results Are Better "a",126.1684,125.6046,125.9853 "b", "c", "d",125.7858,125.9614,124.9582 "e", "Neural Magic DeepSparse 1.7 - Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Synchronous Single-Stream", Lower Results Are Better "a",7.9188,7.9547,7.9308 "b", "c", "d",7.9434,7.932,7.9958 "e", "Neural Magic DeepSparse 1.7 - Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream", Higher Results Are Better "a",10.5963,10.504,10.5012 "b", "c", "d",10.6337,10.5707,10.5264 "e", "Neural Magic DeepSparse 1.7 - Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream", Lower Results Are Better "a",377.4672,380.7866,380.8884 "b", "c", "d",376.1408,378.3833,379.9787 "e", "Neural Magic DeepSparse 1.7 - Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Stream", Higher Results Are Better "a",9.4863,9.4176,9.4293 "b", "c", "d",9.4673,9.3378,9.2866 "e", "Neural Magic DeepSparse 1.7 - Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Stream", Lower Results Are Better "a",105.4113,106.1805,106.0481 "b", "c", "d",105.6228,107.0871,107.6774 "e",