Intel Core i9-14900K testing with a ASUS PRIME Z790-P WIFI (1402 BIOS) and ASUS Intel RPL-S 31GB on Ubuntu 23.10 via the Phoronix Test Suite.
Compare your own system(s) to this result file with the
Phoronix Test Suite by running the command:
phoronix-test-suite benchmark 2403152-PTS-DEEPSPAR00
deepsparse 1.7 raptor lake,
"Neural Magic DeepSparse 1.7 - Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream",
Higher Results Are Better
"a",10.8535,10.5674,10.6111
"b",
"c",
"d",10.8205,10.5774,10.5508
"e",
"Neural Magic DeepSparse 1.7 - Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream",
Lower Results Are Better
"a",368.5289,378.5006,376.9424
"b",
"c",
"d",369.6486,378.1402,379.0992
"e",
"Neural Magic DeepSparse 1.7 - Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Stream",
Higher Results Are Better
"a",9.6193,9.6291,9.564
"b",
"c",
"d",9.4483,9.4971,9.4795
"e",
"Neural Magic DeepSparse 1.7 - Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Stream",
Lower Results Are Better
"a",103.9535,103.8477,104.5551
"b",
"c",
"d",105.8346,105.2914,105.4873
"e",
"Neural Magic DeepSparse 1.7 - Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Asynchronous Multi-Stream",
Higher Results Are Better
"a",438.4461,433.0494,437.2494
"b",
"c",
"d",436.7894,435.6648,435.1081
"e",
"Neural Magic DeepSparse 1.7 - Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Asynchronous Multi-Stream",
Lower Results Are Better
"a",9.1096,9.2236,9.1348
"b",
"c",
"d",9.1443,9.1686,9.1797
"e",
"Neural Magic DeepSparse 1.7 - Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Synchronous Single-Stream",
Higher Results Are Better
"a",324.1779,325.4956,324.0154
"b",
"c",
"d",326.4409,326.9762,328.455
"e",
"Neural Magic DeepSparse 1.7 - Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Synchronous Single-Stream",
Lower Results Are Better
"a",3.081,3.069,3.0829
"b",
"c",
"d",3.06,3.055,3.0414
"e",
"Neural Magic DeepSparse 1.7 - Model: ResNet-50, Baseline - Scenario: Asynchronous Multi-Stream",
Higher Results Are Better
"a",143.5489,142.9199,142.6446
"b",
"c",
"d",143.029,143.0366,142.929
"e",
"Neural Magic DeepSparse 1.7 - Model: ResNet-50, Baseline - Scenario: Asynchronous Multi-Stream",
Lower Results Are Better
"a",27.8495,27.9719,28.0263
"b",
"c",
"d",27.9511,27.9498,27.9707
"e",
"Neural Magic DeepSparse 1.7 - Model: ResNet-50, Baseline - Scenario: Synchronous Single-Stream",
Higher Results Are Better
"a",125.5649,124.5289,124.8817
"b",
"c",
"d",126.1297,124.9999,123.8796
"e",
"Neural Magic DeepSparse 1.7 - Model: ResNet-50, Baseline - Scenario: Synchronous Single-Stream",
Lower Results Are Better
"a",7.96,8.0264,8.0038
"b",
"c",
"d",7.9244,7.9958,8.068
"e",
"Neural Magic DeepSparse 1.7 - Model: ResNet-50, Sparse INT8 - Scenario: Asynchronous Multi-Stream",
Higher Results Are Better
"a",1153.323,1145.7325,1157.7002
"b",
"c",
"d",1166.3533,1159.95,1131.9537
"e",
"Neural Magic DeepSparse 1.7 - Model: ResNet-50, Sparse INT8 - Scenario: Asynchronous Multi-Stream",
Lower Results Are Better
"a",3.4573,3.4809,3.4439
"b",
"c",
"d",3.4189,3.4383,3.5231
"e",
"Neural Magic DeepSparse 1.7 - Model: ResNet-50, Sparse INT8 - Scenario: Synchronous Single-Stream",
Higher Results Are Better
"a",967.3955,992.9423,937.9863,990.3089,990.4057
"b",
"c",
"d",1013.7906,1009.8664,1011.4869
"e",
"Neural Magic DeepSparse 1.7 - Model: ResNet-50, Sparse INT8 - Scenario: Synchronous Single-Stream",
Lower Results Are Better
"a",1.0311,1.0045,1.0631,1.0069,1.007
"b",
"c",
"d",0.9838,0.9874,0.9857
"e",
"Neural Magic DeepSparse 1.7 - Model: Llama2 Chat 7b Quantized - Scenario: Asynchronous Multi-Stream",
Higher Results Are Better
"a",7.5298,7.7403,7.7379
"b",
"c",
"d",7.6099,7.5366,7.7841
"e",
"Neural Magic DeepSparse 1.7 - Model: Llama2 Chat 7b Quantized - Scenario: Asynchronous Multi-Stream",
Lower Results Are Better
"a",527.696,513.4357,513.6056
"b",
"c",
"d",522.1978,527.252,510.5847
"e",
"Neural Magic DeepSparse 1.7 - Model: Llama2 Chat 7b Quantized - Scenario: Synchronous Single-Stream",
Higher Results Are Better
"a",11.2453,11.2568,11.2503
"b",
"c",
"d",11.2472,11.2446,11.2382
"e",
"Neural Magic DeepSparse 1.7 - Model: Llama2 Chat 7b Quantized - Scenario: Synchronous Single-Stream",
Lower Results Are Better
"a",88.9134,88.8192,88.8713
"b",
"c",
"d",88.8979,88.9172,88.9689
"e",
"Neural Magic DeepSparse 1.7 - Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream",
Higher Results Are Better
"a",143.2492,142.7386,142.7574
"b",
"c",
"d",143.668,142.9666,143.0754
"e",
"Neural Magic DeepSparse 1.7 - Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream",
Lower Results Are Better
"a",27.9091,28.0091,28.0052
"b",
"c",
"d",27.8269,27.9637,27.9425
"e",
"Neural Magic DeepSparse 1.7 - Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Stream",
Higher Results Are Better
"a",124.681,124.5633,123.7514
"b",
"c",
"d",126.1382,124.9035,125.0119
"e",
"Neural Magic DeepSparse 1.7 - Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Stream",
Lower Results Are Better
"a",8.0168,8.0238,8.0767
"b",
"c",
"d",7.9236,8.0016,7.9951
"e",
"Neural Magic DeepSparse 1.7 - Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Asynchronous Multi-Stream",
Higher Results Are Better
"a",71.7594,71.7,71.6332
"b",
"c",
"d",71.8093,71.7061,71.766
"e",
"Neural Magic DeepSparse 1.7 - Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Asynchronous Multi-Stream",
Lower Results Are Better
"a",55.7261,55.7728,55.8249
"b",
"c",
"d",55.6874,55.7679,55.7214
"e",
"Neural Magic DeepSparse 1.7 - Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Synchronous Single-Stream",
Higher Results Are Better
"a",67.2246,67.253,67.3097
"b",
"c",
"d",67.3716,67.6645,67.1968
"e",
"Neural Magic DeepSparse 1.7 - Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Synchronous Single-Stream",
Lower Results Are Better
"a",14.8715,14.8641,14.8527
"b",
"c",
"d",14.839,14.7751,14.8774
"e",
"Neural Magic DeepSparse 1.7 - Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Stream",
Higher Results Are Better
"a",81.514,81.6105,81.3492
"b",
"c",
"d",81.6232,82.0929,81.3308
"e",
"Neural Magic DeepSparse 1.7 - Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Stream",
Lower Results Are Better
"a",49.0581,48.9987,49.1573
"b",
"c",
"d",48.9914,48.7106,49.1672
"e",
"Neural Magic DeepSparse 1.7 - Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Stream",
Higher Results Are Better
"a",49.8306,49.5314,49.9971
"b",
"c",
"d",50.0572,50.398,50.8663
"e",
"Neural Magic DeepSparse 1.7 - Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Stream",
Lower Results Are Better
"a",20.0645,20.1856,19.9974
"b",
"c",
"d",19.9736,19.8384,19.656
"e",
"Neural Magic DeepSparse 1.7 - Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Stream",
Higher Results Are Better
"a",14.9213,14.7109,14.8147
"b",
"c",
"d",14.8981,14.7418,14.8646
"e",
"Neural Magic DeepSparse 1.7 - Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Stream",
Lower Results Are Better
"a",268.0533,271.8841,269.9805
"b",
"c",
"d",268.4708,271.3167,269.0749
"e",
"Neural Magic DeepSparse 1.7 - Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Synchronous Single-Stream",
Higher Results Are Better
"a",13.6244,13.7882,13.6846
"b",
"c",
"d",13.9902,13.868,13.7656
"e",
"Neural Magic DeepSparse 1.7 - Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Synchronous Single-Stream",
Lower Results Are Better
"a",73.3876,72.5154,73.0642
"b",
"c",
"d",71.4681,72.0977,72.6343
"e",
"Neural Magic DeepSparse 1.7 - Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Asynchronous Multi-Stream",
Higher Results Are Better
"a",196.3445,196.7321,196.7841
"b",
"c",
"d",197.434,197.3255,196.829
"e",
"Neural Magic DeepSparse 1.7 - Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Asynchronous Multi-Stream",
Lower Results Are Better
"a",20.3587,20.3187,20.313
"b",
"c",
"d",20.2455,20.2571,20.3084
"e",
"Neural Magic DeepSparse 1.7 - Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Synchronous Single-Stream",
Higher Results Are Better
"a",126.1684,125.6046,125.9853
"b",
"c",
"d",125.7858,125.9614,124.9582
"e",
"Neural Magic DeepSparse 1.7 - Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Synchronous Single-Stream",
Lower Results Are Better
"a",7.9188,7.9547,7.9308
"b",
"c",
"d",7.9434,7.932,7.9958
"e",
"Neural Magic DeepSparse 1.7 - Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream",
Higher Results Are Better
"a",10.5963,10.504,10.5012
"b",
"c",
"d",10.6337,10.5707,10.5264
"e",
"Neural Magic DeepSparse 1.7 - Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream",
Lower Results Are Better
"a",377.4672,380.7866,380.8884
"b",
"c",
"d",376.1408,378.3833,379.9787
"e",
"Neural Magic DeepSparse 1.7 - Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Stream",
Higher Results Are Better
"a",9.4863,9.4176,9.4293
"b",
"c",
"d",9.4673,9.3378,9.2866
"e",
"Neural Magic DeepSparse 1.7 - Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Stream",
Lower Results Are Better
"a",105.4113,106.1805,106.0481
"b",
"c",
"d",105.6228,107.0871,107.6774
"e",