sdfa

AMD Ryzen Threadripper 3990X 64-Core testing with a Gigabyte TRX40 AORUS PRO WIFI (F6 BIOS) and AMD Radeon RX 5700 8GB on Ubuntu 23.10 via the Phoronix Test Suite.

HTML result view exported from: https://openbenchmarking.org/result/2312122-PTS-SDFA911983&sro&grr.

sdfaProcessorMotherboardChipsetMemoryDiskGraphicsAudioMonitorNetworkOSKernelDesktopDisplay ServerOpenGLCompilerFile-SystemScreen ResolutionabcdAMD Ryzen Threadripper 3990X 64-Core @ 2.90GHz (64 Cores / 128 Threads)Gigabyte TRX40 AORUS PRO WIFI (F6 BIOS)AMD Starship/Matisse128GBSamsung SSD 970 EVO Plus 500GBAMD Radeon RX 5700 8GB (1750/875MHz)AMD Navi 10 HDMI AudioDELL P2415QIntel I211 + Intel Wi-Fi 6 AX200Ubuntu 23.106.5.0-13-generic (x86_64)GNOME Shell 45.0X Server + Wayland4.6 Mesa 23.2.1-1ubuntu3 (LLVM 15.0.7 DRM 3.54)GCC 13.2.0ext43840x2160OpenBenchmarking.orgKernel Details- Transparent Huge Pages: madviseProcessor Details- Scaling Governor: acpi-cpufreq schedutil (Boost: Enabled) - CPU Microcode: 0x830107aJava Details- OpenJDK Runtime Environment (build 11.0.21+9-post-Ubuntu-0ubuntu123.10)Python Details- Python 3.11.6Security Details- gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Mitigation of untrained return thunk; SMT enabled with STIBP protection + spec_rstack_overflow: Mitigation of safe RET + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional STIBP: always-on RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected

sdfaspark-tpch: 50 - Q22spark-tpch: 50 - Q21spark-tpch: 50 - Q20spark-tpch: 50 - Q19spark-tpch: 50 - Q18spark-tpch: 50 - Q17spark-tpch: 50 - Q16spark-tpch: 50 - Q15spark-tpch: 50 - Q14spark-tpch: 50 - Q13spark-tpch: 50 - Q12spark-tpch: 50 - Q11spark-tpch: 50 - Q10spark-tpch: 50 - Q09spark-tpch: 50 - Q08spark-tpch: 50 - Q07spark-tpch: 50 - Q06spark-tpch: 50 - Q05spark-tpch: 50 - Q04spark-tpch: 50 - Q03spark-tpch: 50 - Q02spark-tpch: 50 - Q01spark-tpch: 50 - Geometric Mean Of All Queriesspark-tpch: 10 - Q22spark-tpch: 10 - Q21spark-tpch: 10 - Q20spark-tpch: 10 - Q19spark-tpch: 10 - Q18spark-tpch: 10 - Q17spark-tpch: 10 - Q16spark-tpch: 10 - Q15spark-tpch: 10 - Q14spark-tpch: 10 - Q13spark-tpch: 10 - Q12spark-tpch: 10 - Q11spark-tpch: 10 - Q10spark-tpch: 10 - Q09spark-tpch: 10 - Q08spark-tpch: 10 - Q07spark-tpch: 10 - Q06spark-tpch: 10 - Q05spark-tpch: 10 - Q04spark-tpch: 10 - Q03spark-tpch: 10 - Q02spark-tpch: 10 - Q01spark-tpch: 10 - Geometric Mean Of All Queriesdeepsparse: BERT-Large, NLP Question Answering - Synchronous Single-Streamdeepsparse: BERT-Large, NLP Question Answering - Synchronous Single-Streamdeepsparse: BERT-Large, NLP Question Answering, Sparse INT8 - Synchronous Single-Streamdeepsparse: BERT-Large, NLP Question Answering, Sparse INT8 - Synchronous Single-Streamdeepsparse: BERT-Large, NLP Question Answering - Asynchronous Multi-Streamdeepsparse: BERT-Large, NLP Question Answering - Asynchronous Multi-Streamspark-tpch: 1 - Q22spark-tpch: 1 - Q21spark-tpch: 1 - Q20spark-tpch: 1 - Q19spark-tpch: 1 - Q18spark-tpch: 1 - Q17spark-tpch: 1 - Q16spark-tpch: 1 - Q15spark-tpch: 1 - Q14spark-tpch: 1 - Q13spark-tpch: 1 - Q12spark-tpch: 1 - Q11spark-tpch: 1 - Q10spark-tpch: 1 - Q09spark-tpch: 1 - Q08spark-tpch: 1 - Q07spark-tpch: 1 - Q06spark-tpch: 1 - Q05spark-tpch: 1 - Q04spark-tpch: 1 - Q03spark-tpch: 1 - Q02spark-tpch: 1 - Q01spark-tpch: 1 - Geometric Mean Of All Queriesdeepsparse: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Synchronous Single-Streamdeepsparse: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Synchronous Single-Streamdeepsparse: NLP Document Classification, oBERT base uncased on IMDB - Asynchronous Multi-Streamdeepsparse: NLP Document Classification, oBERT base uncased on IMDB - Asynchronous Multi-Streamdeepsparse: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Asynchronous Multi-Streamdeepsparse: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Asynchronous Multi-Streamdeepsparse: BERT-Large, NLP Question Answering, Sparse INT8 - Asynchronous Multi-Streamdeepsparse: BERT-Large, NLP Question Answering, Sparse INT8 - Asynchronous Multi-Streamdeepsparse: NLP Document Classification, oBERT base uncased on IMDB - Synchronous Single-Streamdeepsparse: NLP Document Classification, oBERT base uncased on IMDB - Synchronous Single-Streamdeepsparse: NLP Token Classification, BERT base uncased conll2003 - Asynchronous Multi-Streamdeepsparse: NLP Token Classification, BERT base uncased conll2003 - Asynchronous Multi-Streamdeepsparse: NLP Token Classification, BERT base uncased conll2003 - Synchronous Single-Streamdeepsparse: NLP Token Classification, BERT base uncased conll2003 - Synchronous Single-Streamdeepsparse: CV Segmentation, 90% Pruned YOLACT Pruned - Asynchronous Multi-Streamdeepsparse: CV Segmentation, 90% Pruned YOLACT Pruned - Asynchronous Multi-Streamdeepsparse: CV Segmentation, 90% Pruned YOLACT Pruned - Synchronous Single-Streamdeepsparse: CV Segmentation, 90% Pruned YOLACT Pruned - Synchronous Single-Streamdeepsparse: NLP Text Classification, DistilBERT mnli - Asynchronous Multi-Streamdeepsparse: NLP Text Classification, DistilBERT mnli - Asynchronous Multi-Streamdeepsparse: NLP Text Classification, DistilBERT mnli - Synchronous Single-Streamdeepsparse: NLP Text Classification, DistilBERT mnli - Synchronous Single-Streamdeepsparse: CV Detection, YOLOv5s COCO - Synchronous Single-Streamdeepsparse: CV Detection, YOLOv5s COCO - Synchronous Single-Streamdeepsparse: CV Detection, YOLOv5s COCO, Sparse INT8 - Synchronous Single-Streamdeepsparse: CV Detection, YOLOv5s COCO, Sparse INT8 - Synchronous Single-Streamdeepsparse: CV Detection, YOLOv5s COCO - Asynchronous Multi-Streamdeepsparse: CV Detection, YOLOv5s COCO - Asynchronous Multi-Streamdeepsparse: CV Detection, YOLOv5s COCO, Sparse INT8 - Asynchronous Multi-Streamdeepsparse: CV Detection, YOLOv5s COCO, Sparse INT8 - Asynchronous Multi-Streamdeepsparse: ResNet-50, Sparse INT8 - Asynchronous Multi-Streamdeepsparse: ResNet-50, Sparse INT8 - Asynchronous Multi-Streamdeepsparse: ResNet-50, Baseline - Asynchronous Multi-Streamdeepsparse: ResNet-50, Baseline - Asynchronous Multi-Streamdeepsparse: ResNet-50, Baseline - Synchronous Single-Streamdeepsparse: ResNet-50, Baseline - Synchronous Single-Streamdeepsparse: CV Classification, ResNet-50 ImageNet - Synchronous Single-Streamdeepsparse: CV Classification, ResNet-50 ImageNet - Synchronous Single-Streamdeepsparse: CV Classification, ResNet-50 ImageNet - Asynchronous Multi-Streamdeepsparse: CV Classification, ResNet-50 ImageNet - Asynchronous Multi-Streamdeepsparse: ResNet-50, Sparse INT8 - Synchronous Single-Streamdeepsparse: ResNet-50, Sparse INT8 - Synchronous Single-Streamspark-tpch: 100abcd10.67486382130.8656412830.7076950124.7187226662.5085741755.6091779110.5865411823.0454743725.4870529212.3371191030.430392599.0623906535.9260393846.8971544938.3594220535.6305478420.3811728241.1654268930.5848776536.6144994110.4835224226.0223490427.975628233.5358621329.314933788.841609647.2162753814.8622325312.450447404.320506265.837094315.796152754.457083078.034650014.5493162510.2655518814.8994369510.9939940810.631187444.4013136212.476073588.6368297010.244067194.850995228.678337428.1709066788.467511.308815.451064.7072649.934849.21801.085022847.395054702.755551761.028470384.003722492.661922221.427312911.994915871.832315361.400586642.149872691.308498562.837899394.400907402.290912693.162965000.758707523.461568062.960263913.158902592.199165114.695118432.203287347.7066129.7041836.917237.929432.6019980.449763.4770503.413271.937513.8987841.425937.735171.039014.0751736.773943.190658.551117.074595.0151336.191717.597856.790211.561186.376910.963591.1432158.1544201.9130147.9620215.829712.48682555.423272.4193441.44079.6462103.56439.3853106.443271.1719449.30791.5897627.753910.52690792131.4552612330.5381336225.0755310161.955020955.2799911510.5323028622.8892459925.3969287912.1311054230.407165538.6932077436.2720794746.3175430337.9715690635.3421478320.2972335843.2500114430.2687091836.0291671810.4503164325.9903545427.786828933.4011077928.798635488.876638416.042502414.8768053112.441077234.155917645.680895335.828408724.036980158.108592994.2676134110.1844549214.180296910.6704559310.5317444.6216292412.918885238.7080907810.321226124.884950168.701518067.8939945586.150511.605715.52264.3985648.52449.33280.988863237.703692442.587625981.17394594.04589512.722641941.439398532.174607751.781570551.458663232.02941991.364477993.079596284.515100482.327807663.303481340.732237643.37768653.092572213.23912432.207261324.730920792.245314157.6192131.167806.392839.383332.1567993.603762.8112508.590269.417414.4029837.458137.945669.878414.3078726.250443.838455.562117.991294.8818337.082116.935559.008211.327988.16511.126589.8018154.3035206.9627150.2259212.755212.10952635.572970.9424450.80629.3424106.92959.377106.538771.2672448.76031.6656599.186310.63667170131.4217529330.7896194424.8944753062.2995541955.3342908210.8244365122.9463895225.4562212613.1364854230.885828658.7526063936.8214238547.1084391337.8074277235.4553070120.3408381139.7004280130.2234160137.0584462510.5609486926.0016663928.112194133.6996615829.033676788.912171366.5432815516.1145203912.418603904.175729755.810214365.861774134.097770068.171160384.3896342910.4197120714.8189811710.8527135910.620092714.3003606812.663379358.5045766810.264451034.928970818.666814488.1114257584.458711.838715.592664.1154649.755049.08051.058803247.461975892.752848231.208379113.993240672.585270721.494487602.143637741.838365991.399916812.040395381.289742673.029143574.329672182.357799213.184302170.710825563.524186532.944285953.345568972.278467664.840341092.237059117.5689132.1047841.343537.784532.3392988.490563.1295506.133072.027313.8813837.141837.938070.898914.1048737.977743.045458.852616.986594.6789337.781317.477957.179311.357687.930610.961991.1565160.2780199.2162147.4889216.391412.51992548.377171.8091445.16239.4215106.03339.3277107.104270.7399452.04011.6887591.257010.26505438131.0061238630.5178140025.0289204962.7336400355.4218126910.9140424723.0023676625.4237861613.6845792130.041802729.0176893935.9282480946.9971860238.3807525735.4003842720.3820470238.9946009330.5570793135.9920043910.4435723626.1190382728.123942263.5968260728.908061358.824525838.0279534714.9112904912.400583594.346012605.855655995.735224724.613405868.094197754.4805746010.3587230014.9209750510.8172769510.745416004.3671096212.656619078.3395834010.152245524.794374308.732316978.2456727385.157211.742315.600164.0842650.693149.16871.061138477.407735512.747976620.994358203.869211362.577730501.539288202.277157311.832191391.392255512.069544161.291904693.142105904.558725042.225297373.253300430.742623983.309953692.858560403.154473222.275128684.846229872.223564467.4402134.3975820.120838.718832.3108989.058963.2965504.842071.919213.9020845.443837.453471.416513.9997737.852443.056958.501117.087895.4458335.096817.576156.859911.417287.465411.153989.5918158.1065201.9444148.2827215.290912.54832543.113872.4572441.39419.6084103.97919.5802104.293071.0506450.05981.6090620.2412OpenBenchmarking.org

Apache Spark TPC-H

Scale Factor: 50 - Q22

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark TPC-H 3.5Scale Factor: 50 - Q22abcd3691215SE +/- 0.26, N = 3SE +/- 0.11, N = 3SE +/- 0.04, N = 310.6710.5310.6410.27

Apache Spark TPC-H

Scale Factor: 50 - Q21

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark TPC-H 3.5Scale Factor: 50 - Q21abcd306090120150SE +/- 0.28, N = 3SE +/- 0.77, N = 3SE +/- 0.96, N = 3130.87131.46131.42131.01

Apache Spark TPC-H

Scale Factor: 50 - Q20

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark TPC-H 3.5Scale Factor: 50 - Q20abcd714212835SE +/- 0.07, N = 3SE +/- 0.24, N = 3SE +/- 0.01, N = 330.7130.5430.7930.52

Apache Spark TPC-H

Scale Factor: 50 - Q19

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark TPC-H 3.5Scale Factor: 50 - Q19abcd612182430SE +/- 0.11, N = 3SE +/- 0.11, N = 3SE +/- 0.24, N = 324.7225.0824.8925.03

Apache Spark TPC-H

Scale Factor: 50 - Q18

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark TPC-H 3.5Scale Factor: 50 - Q18abcd1428425670SE +/- 0.41, N = 3SE +/- 0.13, N = 3SE +/- 0.54, N = 362.5161.9662.3062.73

Apache Spark TPC-H

Scale Factor: 50 - Q17

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark TPC-H 3.5Scale Factor: 50 - Q17abcd1224364860SE +/- 0.08, N = 3SE +/- 0.19, N = 3SE +/- 0.55, N = 355.6155.2855.3355.42

Apache Spark TPC-H

Scale Factor: 50 - Q16

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark TPC-H 3.5Scale Factor: 50 - Q16abcd3691215SE +/- 0.09, N = 3SE +/- 0.13, N = 3SE +/- 0.47, N = 310.5910.5310.8210.91

Apache Spark TPC-H

Scale Factor: 50 - Q15

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark TPC-H 3.5Scale Factor: 50 - Q15abcd612182430SE +/- 0.04, N = 3SE +/- 0.07, N = 3SE +/- 0.03, N = 323.0522.8922.9523.00

Apache Spark TPC-H

Scale Factor: 50 - Q14

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark TPC-H 3.5Scale Factor: 50 - Q14abcd612182430SE +/- 0.11, N = 3SE +/- 0.03, N = 3SE +/- 0.20, N = 325.4925.4025.4625.42

Apache Spark TPC-H

Scale Factor: 50 - Q13

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark TPC-H 3.5Scale Factor: 50 - Q13abcd48121620SE +/- 0.04, N = 3SE +/- 0.87, N = 3SE +/- 0.76, N = 312.3412.1313.1413.68

Apache Spark TPC-H

Scale Factor: 50 - Q12

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark TPC-H 3.5Scale Factor: 50 - Q12abcd714212835SE +/- 0.31, N = 3SE +/- 0.47, N = 3SE +/- 0.21, N = 330.4330.4130.8930.04

Apache Spark TPC-H

Scale Factor: 50 - Q11

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark TPC-H 3.5Scale Factor: 50 - Q11abcd3691215SE +/- 0.17117283, N = 3SE +/- 0.09627546, N = 3SE +/- 0.14619367, N = 39.062390658.693207748.752606399.01768939

Apache Spark TPC-H

Scale Factor: 50 - Q10

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark TPC-H 3.5Scale Factor: 50 - Q10abcd816243240SE +/- 0.66, N = 3SE +/- 0.34, N = 3SE +/- 0.22, N = 335.9336.2736.8235.93

Apache Spark TPC-H

Scale Factor: 50 - Q09

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark TPC-H 3.5Scale Factor: 50 - Q09abcd1122334455SE +/- 0.20, N = 3SE +/- 0.54, N = 3SE +/- 0.25, N = 346.9046.3247.1147.00

Apache Spark TPC-H

Scale Factor: 50 - Q08

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark TPC-H 3.5Scale Factor: 50 - Q08abcd918273645SE +/- 0.55, N = 3SE +/- 0.08, N = 3SE +/- 0.24, N = 338.3637.9737.8138.38

Apache Spark TPC-H

Scale Factor: 50 - Q07

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark TPC-H 3.5Scale Factor: 50 - Q07abcd816243240SE +/- 0.46, N = 3SE +/- 0.13, N = 3SE +/- 0.15, N = 335.6335.3435.4635.40

Apache Spark TPC-H

Scale Factor: 50 - Q06

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark TPC-H 3.5Scale Factor: 50 - Q06abcd510152025SE +/- 0.03, N = 3SE +/- 0.06, N = 3SE +/- 0.07, N = 320.3820.3020.3420.38

Apache Spark TPC-H

Scale Factor: 50 - Q05

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark TPC-H 3.5Scale Factor: 50 - Q05abcd1020304050SE +/- 0.81, N = 3SE +/- 1.29, N = 3SE +/- 0.23, N = 341.1743.2539.7038.99

Apache Spark TPC-H

Scale Factor: 50 - Q04

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark TPC-H 3.5Scale Factor: 50 - Q04abcd714212835SE +/- 0.26, N = 3SE +/- 0.12, N = 3SE +/- 0.40, N = 330.5830.2730.2230.56

Apache Spark TPC-H

Scale Factor: 50 - Q03

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark TPC-H 3.5Scale Factor: 50 - Q03abcd918273645SE +/- 0.49, N = 3SE +/- 0.13, N = 3SE +/- 0.42, N = 336.6136.0337.0635.99

Apache Spark TPC-H

Scale Factor: 50 - Q02

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark TPC-H 3.5Scale Factor: 50 - Q02abcd3691215SE +/- 0.18, N = 3SE +/- 0.09, N = 3SE +/- 0.12, N = 310.4810.4510.5610.44

Apache Spark TPC-H

Scale Factor: 50 - Q01

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark TPC-H 3.5Scale Factor: 50 - Q01abcd612182430SE +/- 0.08, N = 3SE +/- 0.09, N = 3SE +/- 0.18, N = 326.0225.9926.0026.12

Apache Spark TPC-H

Scale Factor: 50 - Geometric Mean Of All Queries

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark TPC-H 3.5Scale Factor: 50 - Geometric Mean Of All Queriesabcd714212835SE +/- 0.09, N = 3SE +/- 0.14, N = 3SE +/- 0.15, N = 327.9827.7928.1128.12MIN: 8.84 / MAX: 131.32MIN: 8.69 / MAX: 131.46MIN: 8.58 / MAX: 132.82MIN: 8.73 / MAX: 132.42

Apache Spark TPC-H

Scale Factor: 10 - Q22

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark TPC-H 3.5Scale Factor: 10 - Q22abcd0.83241.66482.49723.32964.162SE +/- 0.10541256, N = 3SE +/- 0.18733559, N = 3SE +/- 0.01543789, N = 33.535862133.401107793.699661583.59682607

Apache Spark TPC-H

Scale Factor: 10 - Q21

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark TPC-H 3.5Scale Factor: 10 - Q21abcd714212835SE +/- 0.25, N = 3SE +/- 0.12, N = 3SE +/- 0.20, N = 329.3128.8029.0328.91

Apache Spark TPC-H

Scale Factor: 10 - Q20

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark TPC-H 3.5Scale Factor: 10 - Q20abcd246810SE +/- 0.06394497, N = 3SE +/- 0.01360476, N = 3SE +/- 0.08793082, N = 38.841609648.876638418.912171368.82452583

Apache Spark TPC-H

Scale Factor: 10 - Q19

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark TPC-H 3.5Scale Factor: 10 - Q19abcd246810SE +/- 0.61100502, N = 3SE +/- 0.63365910, N = 3SE +/- 0.02955483, N = 37.216275386.042502406.543281558.02795347

Apache Spark TPC-H

Scale Factor: 10 - Q18

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark TPC-H 3.5Scale Factor: 10 - Q18abcd48121620SE +/- 0.12, N = 3SE +/- 0.79, N = 3SE +/- 0.10, N = 314.8614.8816.1114.91

Apache Spark TPC-H

Scale Factor: 10 - Q17

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark TPC-H 3.5Scale Factor: 10 - Q17abcd3691215SE +/- 0.10, N = 3SE +/- 0.04, N = 3SE +/- 0.13, N = 312.4512.4412.4212.40

Apache Spark TPC-H

Scale Factor: 10 - Q16

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark TPC-H 3.5Scale Factor: 10 - Q16abcd0.97791.95582.93373.91164.8895SE +/- 0.04048184, N = 3SE +/- 0.08541494, N = 3SE +/- 0.00250658, N = 34.320506264.155917644.175729754.34601260

Apache Spark TPC-H

Scale Factor: 10 - Q15

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark TPC-H 3.5Scale Factor: 10 - Q15abcd1.31752.6353.95255.276.5875SE +/- 0.03384145, N = 3SE +/- 0.03487590, N = 3SE +/- 0.04663640, N = 35.837094315.680895335.810214365.85565599

Apache Spark TPC-H

Scale Factor: 10 - Q14

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark TPC-H 3.5Scale Factor: 10 - Q14abcd1.31892.63783.95675.27566.5945SE +/- 0.02725146, N = 3SE +/- 0.02997105, N = 3SE +/- 0.04489420, N = 35.796152755.828408725.861774135.73522472

Apache Spark TPC-H

Scale Factor: 10 - Q13

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark TPC-H 3.5Scale Factor: 10 - Q13abcd1.0382.0763.1144.1525.19SE +/- 0.30907989, N = 3SE +/- 0.05603992, N = 3SE +/- 0.25379241, N = 34.457083074.036980154.097770064.61340586

Apache Spark TPC-H

Scale Factor: 10 - Q12

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark TPC-H 3.5Scale Factor: 10 - Q12abcd246810SE +/- 0.04371011, N = 3SE +/- 0.01728126, N = 3SE +/- 0.05148904, N = 38.034650018.108592998.171160388.09419775

Apache Spark TPC-H

Scale Factor: 10 - Q11

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark TPC-H 3.5Scale Factor: 10 - Q11abcd1.02362.04723.07084.09445.118SE +/- 0.03938644, N = 3SE +/- 0.02158043, N = 3SE +/- 0.00931035, N = 34.549316254.267613414.389634294.48057460

Apache Spark TPC-H

Scale Factor: 10 - Q10

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark TPC-H 3.5Scale Factor: 10 - Q10abcd3691215SE +/- 0.12, N = 3SE +/- 0.03, N = 3SE +/- 0.01, N = 310.2710.1810.4210.36

Apache Spark TPC-H

Scale Factor: 10 - Q09

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark TPC-H 3.5Scale Factor: 10 - Q09abcd48121620SE +/- 0.27, N = 3SE +/- 0.07, N = 3SE +/- 0.31, N = 314.9014.1814.8214.92

Apache Spark TPC-H

Scale Factor: 10 - Q08

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark TPC-H 3.5Scale Factor: 10 - Q08abcd3691215SE +/- 0.13, N = 3SE +/- 0.15, N = 3SE +/- 0.08, N = 310.9910.6710.8510.82

Apache Spark TPC-H

Scale Factor: 10 - Q07

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark TPC-H 3.5Scale Factor: 10 - Q07abcd3691215SE +/- 0.11, N = 3SE +/- 0.05, N = 3SE +/- 0.17, N = 310.6310.5310.6210.75

Apache Spark TPC-H

Scale Factor: 10 - Q06

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark TPC-H 3.5Scale Factor: 10 - Q06abcd1.03992.07983.11974.15965.1995SE +/- 0.04431026, N = 3SE +/- 0.01079337, N = 3SE +/- 0.03080177, N = 34.401313624.621629244.300360684.36710962

Apache Spark TPC-H

Scale Factor: 10 - Q05

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark TPC-H 3.5Scale Factor: 10 - Q05abcd3691215SE +/- 0.23, N = 3SE +/- 0.33, N = 3SE +/- 0.18, N = 312.4812.9212.6612.66

Apache Spark TPC-H

Scale Factor: 10 - Q04

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark TPC-H 3.5Scale Factor: 10 - Q04abcd246810SE +/- 0.24804165, N = 3SE +/- 0.08517991, N = 3SE +/- 0.07436974, N = 38.636829708.708090788.504576688.33958340

Apache Spark TPC-H

Scale Factor: 10 - Q03

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark TPC-H 3.5Scale Factor: 10 - Q03abcd3691215SE +/- 0.28, N = 3SE +/- 0.27, N = 3SE +/- 0.00, N = 310.2410.3210.2610.15

Apache Spark TPC-H

Scale Factor: 10 - Q02

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark TPC-H 3.5Scale Factor: 10 - Q02abcd1.1092.2183.3274.4365.545SE +/- 0.02777661, N = 3SE +/- 0.06191683, N = 3SE +/- 0.02874460, N = 34.850995224.884950164.928970814.79437430

Apache Spark TPC-H

Scale Factor: 10 - Q01

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark TPC-H 3.5Scale Factor: 10 - Q01abcd246810SE +/- 0.07219277, N = 3SE +/- 0.10886176, N = 3SE +/- 0.05844458, N = 38.678337428.701518068.666814488.73231697

Apache Spark TPC-H

Scale Factor: 10 - Geometric Mean Of All Queries

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark TPC-H 3.5Scale Factor: 10 - Geometric Mean Of All Queriesabcd246810SE +/- 0.00516651, N = 3SE +/- 0.05385571, N = 3SE +/- 0.01945474, N = 38.170906677.893994558.111425758.24567273MIN: 3.42 / MAX: 29.75MIN: 3.4 / MAX: 28.8MIN: 3.43 / MAX: 29.17MIN: 3.57 / MAX: 29.28

Neural Magic DeepSparse

Model: BERT-Large, NLP Question Answering - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.6Model: BERT-Large, NLP Question Answering - Scenario: Synchronous Single-Streamabcd20406080100SE +/- 0.60, N = 15SE +/- 0.42, N = 3SE +/- 0.61, N = 388.4786.1584.4685.16

Neural Magic DeepSparse

Model: BERT-Large, NLP Question Answering - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.6Model: BERT-Large, NLP Question Answering - Scenario: Synchronous Single-Streamabcd3691215SE +/- 0.07, N = 15SE +/- 0.06, N = 3SE +/- 0.08, N = 311.3111.6111.8411.74

Neural Magic DeepSparse

Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.6Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Synchronous Single-Streamabcd48121620SE +/- 0.16, N = 3SE +/- 0.12, N = 3SE +/- 0.14, N = 315.4515.5215.5915.60

Neural Magic DeepSparse

Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.6Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Synchronous Single-Streamabcd1428425670SE +/- 0.66, N = 3SE +/- 0.51, N = 3SE +/- 0.57, N = 364.7164.4064.1264.08

Neural Magic DeepSparse

Model: BERT-Large, NLP Question Answering - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.6Model: BERT-Large, NLP Question Answering - Scenario: Asynchronous Multi-Streamabcd140280420560700SE +/- 0.29, N = 3SE +/- 1.51, N = 3SE +/- 1.27, N = 3649.93648.52649.76650.69

Neural Magic DeepSparse

Model: BERT-Large, NLP Question Answering - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.6Model: BERT-Large, NLP Question Answering - Scenario: Asynchronous Multi-Streamabcd1122334455SE +/- 0.03, N = 3SE +/- 0.26, N = 3SE +/- 0.10, N = 349.2249.3349.0849.17

Apache Spark TPC-H

Scale Factor: 1 - Q22

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark TPC-H 3.5Scale Factor: 1 - Q22abcd0.24410.48820.73230.97641.2205SE +/- 0.00972834, N = 4SE +/- 0.00495537, N = 3SE +/- 0.01300436, N = 31.085022840.988863231.058803241.06113847

Apache Spark TPC-H

Scale Factor: 1 - Q21

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark TPC-H 3.5Scale Factor: 1 - Q21abcd246810SE +/- 0.08062589, N = 4SE +/- 0.11898636, N = 3SE +/- 0.17170189, N = 37.395054707.703692447.461975897.40773551

Apache Spark TPC-H

Scale Factor: 1 - Q20

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark TPC-H 3.5Scale Factor: 1 - Q20abcd0.621.241.862.483.1SE +/- 0.09583966, N = 4SE +/- 0.13759046, N = 3SE +/- 0.14206756, N = 32.755551762.587625982.752848232.74797662

Apache Spark TPC-H

Scale Factor: 1 - Q19

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark TPC-H 3.5Scale Factor: 1 - Q19abcd0.27190.54380.81571.08761.3595SE +/- 0.02636608, N = 4SE +/- 0.07878807, N = 3SE +/- 0.01874751, N = 31.028470381.173945901.208379110.99435820

Apache Spark TPC-H

Scale Factor: 1 - Q18

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark TPC-H 3.5Scale Factor: 1 - Q18abcd0.91031.82062.73093.64124.5515SE +/- 0.10303949, N = 4SE +/- 0.10019808, N = 3SE +/- 0.13038086, N = 34.003722494.045895103.993240673.86921136

Apache Spark TPC-H

Scale Factor: 1 - Q17

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark TPC-H 3.5Scale Factor: 1 - Q17abcd0.61261.22521.83782.45043.063SE +/- 0.02035298, N = 4SE +/- 0.01865956, N = 3SE +/- 0.03392729, N = 32.661922222.722641942.585270722.57773050

Apache Spark TPC-H

Scale Factor: 1 - Q16

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark TPC-H 3.5Scale Factor: 1 - Q16abcd0.34630.69261.03891.38521.7315SE +/- 0.01144007, N = 4SE +/- 0.05940937, N = 3SE +/- 0.04799671, N = 31.427312911.439398531.494487601.53928820

Apache Spark TPC-H

Scale Factor: 1 - Q15

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark TPC-H 3.5Scale Factor: 1 - Q15abcd0.51241.02481.53722.04962.562SE +/- 0.04340326, N = 4SE +/- 0.20579960, N = 3SE +/- 0.09001964, N = 31.994915872.174607752.143637742.27715731

Apache Spark TPC-H

Scale Factor: 1 - Q14

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark TPC-H 3.5Scale Factor: 1 - Q14abcd0.41360.82721.24081.65442.068SE +/- 0.03540160, N = 4SE +/- 0.01014717, N = 3SE +/- 0.02463739, N = 31.832315361.781570551.838365991.83219139

Apache Spark TPC-H

Scale Factor: 1 - Q13

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark TPC-H 3.5Scale Factor: 1 - Q13abcd0.32820.65640.98461.31281.641SE +/- 0.03676867, N = 4SE +/- 0.04981792, N = 3SE +/- 0.02733553, N = 31.400586641.458663231.399916811.39225551

Apache Spark TPC-H

Scale Factor: 1 - Q12

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark TPC-H 3.5Scale Factor: 1 - Q12abcd0.48370.96741.45111.93482.4185SE +/- 0.15031140, N = 4SE +/- 0.04470734, N = 3SE +/- 0.02411619, N = 32.149872692.029419902.040395382.06954416

Apache Spark TPC-H

Scale Factor: 1 - Q11

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark TPC-H 3.5Scale Factor: 1 - Q11abcd0.3070.6140.9211.2281.535SE +/- 0.07929998, N = 4SE +/- 0.15013623, N = 3SE +/- 0.10958276, N = 31.308498561.364477991.289742671.29190469

Apache Spark TPC-H

Scale Factor: 1 - Q10

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark TPC-H 3.5Scale Factor: 1 - Q10abcd0.7071.4142.1212.8283.535SE +/- 0.04789727, N = 4SE +/- 0.03625533, N = 3SE +/- 0.04198555, N = 32.837899393.079596283.029143573.14210590

Apache Spark TPC-H

Scale Factor: 1 - Q09

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark TPC-H 3.5Scale Factor: 1 - Q09abcd1.02572.05143.07714.10285.1285SE +/- 0.13363421, N = 4SE +/- 0.11520840, N = 3SE +/- 0.17668063, N = 34.400907404.515100484.329672184.55872504

Apache Spark TPC-H

Scale Factor: 1 - Q08

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark TPC-H 3.5Scale Factor: 1 - Q08abcd0.53051.0611.59152.1222.6525SE +/- 0.02917526, N = 4SE +/- 0.06735723, N = 3SE +/- 0.07315965, N = 32.290912692.327807662.357799212.22529737

Apache Spark TPC-H

Scale Factor: 1 - Q07

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark TPC-H 3.5Scale Factor: 1 - Q07abcd0.74331.48662.22992.97323.7165SE +/- 0.07379513, N = 4SE +/- 0.13350733, N = 3SE +/- 0.03596564, N = 33.162965003.303481343.184302173.25330043

Apache Spark TPC-H

Scale Factor: 1 - Q06

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark TPC-H 3.5Scale Factor: 1 - Q06abcd0.17070.34140.51210.68280.8535SE +/- 0.03196645, N = 4SE +/- 0.00867146, N = 3SE +/- 0.01813313, N = 30.758707520.732237640.710825560.74262398

Apache Spark TPC-H

Scale Factor: 1 - Q05

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark TPC-H 3.5Scale Factor: 1 - Q05abcd0.79291.58582.37873.17163.9645SE +/- 0.06297205, N = 4SE +/- 0.08593638, N = 3SE +/- 0.06648494, N = 33.461568063.377686503.524186533.30995369

Apache Spark TPC-H

Scale Factor: 1 - Q04

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark TPC-H 3.5Scale Factor: 1 - Q04abcd0.69581.39162.08742.78323.479SE +/- 0.02567628, N = 4SE +/- 0.05472939, N = 3SE +/- 0.09994253, N = 32.960263913.092572212.944285952.85856040

Apache Spark TPC-H

Scale Factor: 1 - Q03

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark TPC-H 3.5Scale Factor: 1 - Q03abcd0.75281.50562.25843.01123.764SE +/- 0.03420168, N = 4SE +/- 0.05826206, N = 3SE +/- 0.04118145, N = 33.158902593.239124303.345568973.15447322

Apache Spark TPC-H

Scale Factor: 1 - Q02

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark TPC-H 3.5Scale Factor: 1 - Q02abcd0.51271.02541.53812.05082.5635SE +/- 0.05229177, N = 4SE +/- 0.08231744, N = 3SE +/- 0.07468701, N = 32.199165112.207261322.278467662.27512868

Apache Spark TPC-H

Scale Factor: 1 - Q01

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark TPC-H 3.5Scale Factor: 1 - Q01abcd1.09042.18083.27124.36165.452SE +/- 0.06902799, N = 4SE +/- 0.05428360, N = 3SE +/- 0.08250324, N = 34.695118434.730920794.840341094.84622987

Apache Spark TPC-H

Scale Factor: 1 - Geometric Mean Of All Queries

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark TPC-H 3.5Scale Factor: 1 - Geometric Mean Of All Queriesabcd0.50521.01041.51562.02082.526SE +/- 0.02461970, N = 4SE +/- 0.00573499, N = 3SE +/- 0.02256519, N = 32.203287342.245314152.237059112.22356446MIN: 0.99 / MAX: 7.56MIN: 0.99 / MAX: 7.7MIN: 1.05 / MAX: 7.66MIN: 0.97 / MAX: 7.7

Neural Magic DeepSparse

Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.6Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Synchronous Single-Streamabcd246810SE +/- 0.0410, N = 3SE +/- 0.0865, N = 4SE +/- 0.0842, N = 57.70667.61927.56897.4402

Neural Magic DeepSparse

Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.6Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Synchronous Single-Streamabcd306090120150SE +/- 0.69, N = 3SE +/- 1.51, N = 4SE +/- 1.48, N = 5129.70131.17132.10134.40

Neural Magic DeepSparse

Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.6Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Streamabcd2004006008001000SE +/- 12.88, N = 3SE +/- 8.13, N = 6SE +/- 7.99, N = 3836.92806.39841.34820.12

Neural Magic DeepSparse

Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.6Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Streamabcd918273645SE +/- 0.52, N = 3SE +/- 0.38, N = 6SE +/- 0.36, N = 337.9339.3837.7838.72

Neural Magic DeepSparse

Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.6Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Asynchronous Multi-Streamabcd816243240SE +/- 0.11, N = 3SE +/- 0.04, N = 3SE +/- 0.04, N = 332.6032.1632.3432.31

Neural Magic DeepSparse

Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.6Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Asynchronous Multi-Streamabcd2004006008001000SE +/- 3.44, N = 3SE +/- 1.16, N = 3SE +/- 1.17, N = 3980.45993.60988.49989.06

Neural Magic DeepSparse

Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.6Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Asynchronous Multi-Streamabcd1428425670SE +/- 0.05, N = 3SE +/- 0.06, N = 3SE +/- 0.08, N = 363.4862.8163.1363.30

Neural Magic DeepSparse

Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.6Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Asynchronous Multi-Streamabcd110220330440550SE +/- 0.37, N = 3SE +/- 0.45, N = 3SE +/- 0.62, N = 3503.41508.59506.13504.84

Neural Magic DeepSparse

Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.6Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Streamabcd1632486480SE +/- 0.30, N = 3SE +/- 0.25, N = 3SE +/- 0.18, N = 371.9469.4272.0371.92

Neural Magic DeepSparse

Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.6Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Streamabcd48121620SE +/- 0.06, N = 3SE +/- 0.05, N = 3SE +/- 0.03, N = 313.9014.4013.8813.90

Neural Magic DeepSparse

Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.6Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Streamabcd2004006008001000SE +/- 2.31, N = 3SE +/- 6.31, N = 3SE +/- 2.60, N = 3841.43837.46837.14845.44

Neural Magic DeepSparse

Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.6Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Streamabcd918273645SE +/- 0.10, N = 3SE +/- 0.25, N = 3SE +/- 0.10, N = 337.7437.9537.9437.45

Neural Magic DeepSparse

Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.6Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Streamabcd1632486480SE +/- 0.43, N = 3SE +/- 0.72, N = 3SE +/- 0.16, N = 371.0469.8870.9071.42

Neural Magic DeepSparse

Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.6Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Streamabcd48121620SE +/- 0.08, N = 3SE +/- 0.14, N = 3SE +/- 0.03, N = 314.0814.3114.1014.00

Neural Magic DeepSparse

Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.6Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Streamabcd160320480640800SE +/- 6.44, N = 3SE +/- 4.42, N = 3SE +/- 2.01, N = 3736.77726.25737.98737.85

Neural Magic DeepSparse

Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.6Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Streamabcd1020304050SE +/- 0.30, N = 3SE +/- 0.28, N = 3SE +/- 0.06, N = 343.1943.8443.0543.06

Neural Magic DeepSparse

Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.6Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Synchronous Single-Streamabcd1326395265SE +/- 0.38, N = 3SE +/- 0.30, N = 3SE +/- 0.11, N = 358.5555.5658.8558.50

Neural Magic DeepSparse

Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.6Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Synchronous Single-Streamabcd48121620SE +/- 0.11, N = 3SE +/- 0.09, N = 3SE +/- 0.03, N = 317.0717.9916.9917.09

Neural Magic DeepSparse

Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.6Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Streamabcd20406080100SE +/- 0.08, N = 3SE +/- 0.12, N = 3SE +/- 0.28, N = 395.0294.8894.6895.45

Neural Magic DeepSparse

Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.6Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Streamabcd70140210280350SE +/- 0.53, N = 3SE +/- 0.44, N = 3SE +/- 1.00, N = 3336.19337.08337.78335.10

Neural Magic DeepSparse

Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.6Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Streamabcd48121620SE +/- 0.05, N = 3SE +/- 0.06, N = 3SE +/- 0.06, N = 317.6016.9417.4817.58

Neural Magic DeepSparse

Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.6Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Streamabcd1326395265SE +/- 0.15, N = 3SE +/- 0.19, N = 3SE +/- 0.19, N = 356.7959.0157.1856.86

Neural Magic DeepSparse

Model: CV Detection, YOLOv5s COCO - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.6Model: CV Detection, YOLOv5s COCO - Scenario: Synchronous Single-Streamabcd3691215SE +/- 0.08, N = 3SE +/- 0.04, N = 3SE +/- 0.03, N = 311.5611.3311.3611.42

Neural Magic DeepSparse

Model: CV Detection, YOLOv5s COCO - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.6Model: CV Detection, YOLOv5s COCO - Scenario: Synchronous Single-Streamabcd20406080100SE +/- 0.58, N = 3SE +/- 0.32, N = 3SE +/- 0.19, N = 386.3888.1787.9387.47

Neural Magic DeepSparse

Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.6Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Synchronous Single-Streamabcd3691215SE +/- 0.04, N = 3SE +/- 0.02, N = 3SE +/- 0.04, N = 310.9611.1310.9611.15

Neural Magic DeepSparse

Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.6Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Synchronous Single-Streamabcd20406080100SE +/- 0.36, N = 3SE +/- 0.17, N = 3SE +/- 0.36, N = 391.1489.8091.1689.59

Neural Magic DeepSparse

Model: CV Detection, YOLOv5s COCO - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.6Model: CV Detection, YOLOv5s COCO - Scenario: Asynchronous Multi-Streamabcd4080120160200SE +/- 0.94, N = 3SE +/- 1.36, N = 3SE +/- 1.02, N = 3158.15154.30160.28158.11

Neural Magic DeepSparse

Model: CV Detection, YOLOv5s COCO - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.6Model: CV Detection, YOLOv5s COCO - Scenario: Asynchronous Multi-Streamabcd50100150200250SE +/- 1.28, N = 3SE +/- 1.60, N = 3SE +/- 1.27, N = 3201.91206.96199.22201.94

Neural Magic DeepSparse

Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.6Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Asynchronous Multi-Streamabcd306090120150SE +/- 0.45, N = 3SE +/- 0.12, N = 3SE +/- 0.06, N = 3147.96150.23147.49148.28

Neural Magic DeepSparse

Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.6Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Asynchronous Multi-Streamabcd50100150200250SE +/- 0.62, N = 3SE +/- 0.14, N = 3SE +/- 0.09, N = 3215.83212.76216.39215.29

Neural Magic DeepSparse

Model: ResNet-50, Sparse INT8 - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.6Model: ResNet-50, Sparse INT8 - Scenario: Asynchronous Multi-Streamabcd3691215SE +/- 0.07, N = 3SE +/- 0.06, N = 3SE +/- 0.08, N = 312.4912.1112.5212.55

Neural Magic DeepSparse

Model: ResNet-50, Sparse INT8 - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.6Model: ResNet-50, Sparse INT8 - Scenario: Asynchronous Multi-Streamabcd6001200180024003000SE +/- 14.52, N = 3SE +/- 12.01, N = 3SE +/- 15.60, N = 32555.422635.572548.382543.11

Neural Magic DeepSparse

Model: ResNet-50, Baseline - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.6Model: ResNet-50, Baseline - Scenario: Asynchronous Multi-Streamabcd1632486480SE +/- 0.35, N = 3SE +/- 0.24, N = 3SE +/- 0.19, N = 372.4270.9471.8172.46

Neural Magic DeepSparse

Model: ResNet-50, Baseline - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.6Model: ResNet-50, Baseline - Scenario: Asynchronous Multi-Streamabcd100200300400500SE +/- 2.10, N = 3SE +/- 1.47, N = 3SE +/- 1.15, N = 3441.44450.81445.16441.39

Neural Magic DeepSparse

Model: ResNet-50, Baseline - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.6Model: ResNet-50, Baseline - Scenario: Synchronous Single-Streamabcd3691215SE +/- 0.0493, N = 3SE +/- 0.0132, N = 3SE +/- 0.0758, N = 39.64629.34249.42159.6084

Neural Magic DeepSparse

Model: ResNet-50, Baseline - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.6Model: ResNet-50, Baseline - Scenario: Synchronous Single-Streamabcd20406080100SE +/- 0.53, N = 3SE +/- 0.15, N = 3SE +/- 0.81, N = 3103.56106.93106.03103.98

Neural Magic DeepSparse

Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.6Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Streamabcd3691215SE +/- 0.0142, N = 3SE +/- 0.0294, N = 3SE +/- 0.0916, N = 39.38539.37709.32779.5802

Neural Magic DeepSparse

Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.6Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Streamabcd20406080100SE +/- 0.16, N = 3SE +/- 0.34, N = 3SE +/- 1.00, N = 3106.44106.54107.10104.29

Neural Magic DeepSparse

Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.6Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Streamabcd1632486480SE +/- 0.17, N = 3SE +/- 0.10, N = 3SE +/- 0.25, N = 371.1771.2770.7471.05

Neural Magic DeepSparse

Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.6Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Streamabcd100200300400500SE +/- 1.01, N = 3SE +/- 0.60, N = 3SE +/- 1.54, N = 3449.31448.76452.04450.06

Neural Magic DeepSparse

Model: ResNet-50, Sparse INT8 - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.6Model: ResNet-50, Sparse INT8 - Scenario: Synchronous Single-Streamabcd0.380.761.141.521.9SE +/- 0.0051, N = 3SE +/- 0.0240, N = 3SE +/- 0.0037, N = 31.58971.66561.68871.6090

Neural Magic DeepSparse

Model: ResNet-50, Sparse INT8 - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.6Model: ResNet-50, Sparse INT8 - Scenario: Synchronous Single-Streamabcd140280420560700SE +/- 2.01, N = 3SE +/- 8.33, N = 3SE +/- 1.41, N = 3627.75599.19591.26620.24


Phoronix Test Suite v10.8.5