deepsparse 1.7 raptor lake

Intel Core i9-14900K testing with a ASUS PRIME Z790-P WIFI (1402 BIOS) and ASUS Intel RPL-S 31GB on Ubuntu 23.10 via the Phoronix Test Suite.

Compare your own system(s) to this result file with the Phoronix Test Suite by running the command: phoronix-test-suite benchmark 2403152-PTS-DEEPSPAR00
Jump To Table - Results

View

Do Not Show Noisy Results
Do Not Show Results With Incomplete Data
Do Not Show Results With Little Change/Spread
List Notable Results

Statistics

Show Overall Harmonic Mean(s)
Show Overall Geometric Mean
Show Wins / Losses Counts (Pie Chart)
Normalize Results
Remove Outliers Before Calculating Averages

Graph Settings

Force Line Graphs Where Applicable
Convert To Scalar Where Applicable
Prefer Vertical Bar Graphs

Multi-Way Comparison

Condense Multi-Option Tests Into Single Result Graphs

Table

Show Detailed System Result Table

Run Management

Highlight
Result
Hide
Result
Result
Identifier
View Logs
Performance Per
Dollar
Date
Run
  Test
  Duration
a
March 15
  1 Hour, 30 Minutes
b
March 15
  30 Minutes
c
March 15
  30 Minutes
d
March 15
  1 Hour, 28 Minutes
e
March 15
  30 Minutes
Invert Hiding All Results Option
  54 Minutes

Only show results where is faster than
Only show results matching title/arguments (delimit multiple options with a comma):
Do not show results matching title/arguments (delimit multiple options with a comma):

{ "title": "deepsparse 1.7 raptor lake", "last_modified": "2024-03-15 20:41:39", "description": "Intel Core i9-14900K testing with a ASUS PRIME Z790-P WIFI (1402 BIOS) and ASUS Intel RPL-S 31GB on Ubuntu 23.10 via the Phoronix Test Suite.", "systems": { "a": { "identifier": "a", "hardware": { "Processor": "Intel Core i9-14900K @ 5.70GHz (24 Cores \/ 32 Threads)", "Motherboard": "ASUS PRIME Z790-P WIFI (1402 BIOS)", "Chipset": "Intel Device 7a27", "Memory": "2 x 16GB DRAM-6000MT\/s Corsair CMK32GX5M2B6000C36", "Disk": "Western Digital WD_BLACK SN850X 1000GB", "Graphics": "ASUS Intel RPL-S 31GB (1650MHz)", "Audio": "Realtek ALC897", "Monitor": "ASUS VP28U" }, "software": { "OS": "Ubuntu 23.10", "Kernel": "6.8.0-phx (x86_64)", "Desktop": "GNOME Shell 45.1", "Display Server": "X Server 1.21.1.7", "OpenGL": "4.6 Mesa 24.0~git2312240600.c05261~oibaf~m (git-c05261a 2023-12-24 mantic-oibaf-ppa)", "Compiler": "GCC 13.2.0", "File-System": "ext4", "Screen Resolution": "3840x2160" }, "user": "pts", "timestamp": "2024-03-15 17:18:49", "client_version": "10.8.4", "data": { "cpu-scaling-governor": "intel_pstate powersave (EPP: performance)", "cpu-microcode": "0x122", "cpu-thermald": "2.5.4", "kernel-extra-details": "Transparent Huge Pages: madvise", "python": "Python 3.11.6", "security": "gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + reg_file_data_sampling: Mitigation of Clear Register File + retbleed: Not affected + spec_rstack_overflow: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy\/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Enhanced \/ Automatic IBRS IBPB: conditional RSB filling PBRSB-eIBRS: SW sequence + srbds: Not affected + tsx_async_abort: Not affected" } }, "b": { "identifier": "b", "hardware": { "Processor": "Intel Core i9-14900K @ 5.70GHz (24 Cores \/ 32 Threads)", "Motherboard": "ASUS PRIME Z790-P WIFI (1402 BIOS)", "Chipset": "Intel Device 7a27", "Memory": "2 x 16GB DRAM-6000MT\/s Corsair CMK32GX5M2B6000C36", "Disk": "Western Digital WD_BLACK SN850X 1000GB", "Graphics": "ASUS Intel RPL-S 31GB (1650MHz)", "Audio": "Realtek ALC897", "Monitor": "ASUS VP28U" }, "software": { "OS": "Ubuntu 23.10", "Kernel": "6.8.0-phx (x86_64)", "Desktop": "GNOME Shell 45.1", "Display Server": "X Server 1.21.1.7", "OpenGL": "4.6 Mesa 24.0~git2312240600.c05261~oibaf~m (git-c05261a 2023-12-24 mantic-oibaf-ppa)", "Compiler": "GCC 13.2.0", "File-System": "ext4", "Screen Resolution": "3840x2160" }, "user": "pts", "timestamp": "2024-03-15 18:14:29", "client_version": "10.8.4", "data": { "cpu-scaling-governor": "intel_pstate powersave (EPP: performance)", "cpu-microcode": "0x122", "cpu-thermald": "2.5.4", "kernel-extra-details": "Transparent Huge Pages: madvise", "python": "Python 3.11.6", "security": "gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + reg_file_data_sampling: Mitigation of Clear Register File + retbleed: Not affected + spec_rstack_overflow: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy\/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Enhanced \/ Automatic IBRS IBPB: conditional RSB filling PBRSB-eIBRS: SW sequence + srbds: Not affected + tsx_async_abort: Not affected" } }, "c": { "identifier": "c", "hardware": { "Processor": "Intel Core i9-14900K @ 5.70GHz (24 Cores \/ 32 Threads)", "Motherboard": "ASUS PRIME Z790-P WIFI (1402 BIOS)", "Chipset": "Intel Device 7a27", "Memory": "2 x 16GB DRAM-6000MT\/s Corsair CMK32GX5M2B6000C36", "Disk": "Western Digital WD_BLACK SN850X 1000GB", "Graphics": "ASUS Intel RPL-S 31GB (1650MHz)", "Audio": "Realtek ALC897", "Monitor": "ASUS VP28U" }, "software": { "OS": "Ubuntu 23.10", "Kernel": "6.8.0-phx (x86_64)", "Desktop": "GNOME Shell 45.1", "Display Server": "X Server 1.21.1.7", "OpenGL": "4.6 Mesa 24.0~git2312240600.c05261~oibaf~m (git-c05261a 2023-12-24 mantic-oibaf-ppa)", "Compiler": "GCC 13.2.0", "File-System": "ext4", "Screen Resolution": "3840x2160" }, "user": "pts", "timestamp": "2024-03-15 18:34:16", "client_version": "10.8.4", "data": { "cpu-scaling-governor": "intel_pstate powersave (EPP: performance)", "cpu-microcode": "0x122", "cpu-thermald": "2.5.4", "kernel-extra-details": "Transparent Huge Pages: madvise", "python": "Python 3.11.6", "security": "gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + reg_file_data_sampling: Mitigation of Clear Register File + retbleed: Not affected + spec_rstack_overflow: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy\/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Enhanced \/ Automatic IBRS IBPB: conditional RSB filling PBRSB-eIBRS: SW sequence + srbds: Not affected + tsx_async_abort: Not affected" } }, "d": { "identifier": "d", "hardware": { "Processor": "Intel Core i9-14900K @ 5.70GHz (24 Cores \/ 32 Threads)", "Motherboard": "ASUS PRIME Z790-P WIFI (1402 BIOS)", "Chipset": "Intel Device 7a27", "Memory": "2 x 16GB DRAM-6000MT\/s Corsair CMK32GX5M2B6000C36", "Disk": "Western Digital WD_BLACK SN850X 1000GB", "Graphics": "ASUS Intel RPL-S 31GB (1650MHz)", "Audio": "Realtek ALC897", "Monitor": "ASUS VP28U" }, "software": { "OS": "Ubuntu 23.10", "Kernel": "6.8.0-phx (x86_64)", "Desktop": "GNOME Shell 45.1", "Display Server": "X Server 1.21.1.7", "OpenGL": "4.6 Mesa 24.0~git2312240600.c05261~oibaf~m (git-c05261a 2023-12-24 mantic-oibaf-ppa)", "Compiler": "GCC 13.2.0", "File-System": "ext4", "Screen Resolution": "3840x2160" }, "user": "pts", "timestamp": "2024-03-15 18:59:42", "client_version": "10.8.4", "data": { "cpu-scaling-governor": "intel_pstate powersave (EPP: performance)", "cpu-microcode": "0x122", "cpu-thermald": "2.5.4", "kernel-extra-details": "Transparent Huge Pages: madvise", "python": "Python 3.11.6", "security": "gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + reg_file_data_sampling: Mitigation of Clear Register File + retbleed: Not affected + spec_rstack_overflow: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy\/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Enhanced \/ Automatic IBRS IBPB: conditional RSB filling PBRSB-eIBRS: SW sequence + srbds: Not affected + tsx_async_abort: Not affected" } }, "e": { "identifier": "e", "hardware": { "Processor": "Intel Core i9-14900K @ 5.70GHz (24 Cores \/ 32 Threads)", "Motherboard": "ASUS PRIME Z790-P WIFI (1402 BIOS)", "Chipset": "Intel Device 7a27", "Memory": "2 x 16GB DRAM-6000MT\/s Corsair CMK32GX5M2B6000C36", "Disk": "Western Digital WD_BLACK SN850X 1000GB", "Graphics": "ASUS Intel RPL-S 31GB (1650MHz)", "Audio": "Realtek ALC897", "Monitor": "ASUS VP28U" }, "software": { "OS": "Ubuntu 23.10", "Kernel": "6.8.0-phx (x86_64)", "Desktop": "GNOME Shell 45.1", "Display Server": "X Server 1.21.1.7", "OpenGL": "4.6 Mesa 24.0~git2312240600.c05261~oibaf~m (git-c05261a 2023-12-24 mantic-oibaf-ppa)", "Compiler": "GCC 13.2.0", "File-System": "ext4", "Screen Resolution": "3840x2160" }, "user": "pts", "timestamp": "2024-03-15 20:00:01", "client_version": "10.8.4", "data": { "cpu-scaling-governor": "intel_pstate powersave (EPP: performance)", "cpu-microcode": "0x122", "cpu-thermald": "2.5.4", "kernel-extra-details": "Transparent Huge Pages: madvise", "python": "Python 3.11.6", "security": "gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + reg_file_data_sampling: Mitigation of Clear Register File + retbleed: Not affected + spec_rstack_overflow: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy\/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Enhanced \/ Automatic IBRS IBPB: conditional RSB filling PBRSB-eIBRS: SW sequence + srbds: Not affected + tsx_async_abort: Not affected" } } }, "results": { "947610139beb46c152f74ada08833bfe9728062d": { "identifier": "pts\/deepsparse-1.7.0", "title": "Neural Magic DeepSparse", "app_version": "1.7", "arguments": "zoo:nlp\/document_classification\/obert-base\/pytorch\/huggingface\/imdb\/base-none --scenario async", "description": "Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream", "scale": "items\/sec", "proportion": "HIB", "display_format": "BAR_GRAPH", "results": { "a": { "value": 10.6773000000000006792788553866557776927947998046875, "raw_values": [ 10.853500000000000369482222595252096652984619140625, 10.56739999999999923829818726517260074615478515625, 10.61110000000000042064129956997931003570556640625 ], "test_run_times": [ 40.10000000000000142108547152020037174224853515625, 39.81000000000000227373675443232059478759765625, 39.78999999999999914734871708787977695465087890625 ] }, "b": { "value": 10.8515999999999994685140336514450609683990478515625, "test_run_times": [ 40.18999999999999772626324556767940521240234375 ] }, "c": { "value": 10.719699999999999562305674771778285503387451171875, "test_run_times": [ 40 ] }, "d": { "value": 10.64959999999999951114659779705107212066650390625, "raw_values": [ 10.820499999999999118927007657475769519805908203125, 10.577400000000000801492205937393009662628173828125, 10.5508000000000006224354365258477628231048583984375 ], "test_run_times": [ 40.07000000000000028421709430404007434844970703125, 40.35000000000000142108547152020037174224853515625, 40.11999999999999744204615126363933086395263671875 ] }, "e": { "value": 10.8622999999999994003019310184754431247711181640625, "test_run_times": [ 39.99000000000000198951966012828052043914794921875 ] } } }, "6f6400821c5c8b1463564cdf0dcc6fc2fc0a66f6": { "identifier": "pts\/deepsparse-1.7.0", "title": "Neural Magic DeepSparse", "app_version": "1.7", "arguments": "zoo:nlp\/document_classification\/obert-base\/pytorch\/huggingface\/imdb\/base-none --scenario async", "description": "Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream", "scale": "ms\/batch", "proportion": "LIB", "display_format": "BAR_GRAPH", "results": { "a": { "value": 374.65730000000002064552973024547100067138671875, "raw_values": [ 368.528900000000021464074961841106414794921875, 378.50060000000001991793396882712841033935546875, 376.9424000000000205545802600681781768798828125 ], "test_run_times": [ 40.10000000000000142108547152020037174224853515625, 39.81000000000000227373675443232059478759765625, 39.78999999999999914734871708787977695465087890625 ] }, "b": { "value": 368.5855000000000245563569478690624237060546875, "test_run_times": [ 40.18999999999999772626324556767940521240234375 ] }, "c": { "value": 373.123299999999971987563185393810272216796875, "test_run_times": [ 40 ] }, "d": { "value": 375.62930000000000063664629124104976654052734375, "raw_values": [ 369.64859999999998763087205588817596435546875, 378.14019999999999299689079634845256805419921875, 379.0991999999999961801222525537014007568359375 ], "test_run_times": [ 40.07000000000000028421709430404007434844970703125, 40.35000000000000142108547152020037174224853515625, 40.11999999999999744204615126363933086395263671875 ] }, "e": { "value": 368.225599999999985811882652342319488525390625, "test_run_times": [ 39.99000000000000198951966012828052043914794921875 ] } } }, "5a36b450b1f8e7b3be90e33289141057ce6391e4": { "identifier": "pts\/deepsparse-1.7.0", "title": "Neural Magic DeepSparse", "app_version": "1.7", "arguments": "zoo:nlp\/document_classification\/obert-base\/pytorch\/huggingface\/imdb\/base-none --scenario sync", "description": "Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Stream", "scale": "items\/sec", "proportion": "HIB", "display_format": "BAR_GRAPH", "results": { "a": { "value": 9.6041000000000007474909580196253955364227294921875, "raw_values": [ 9.6193000000000008498091119690798223018646240234375, 9.6290999999999993264054864994250237941741943359375, 9.56400000000000005684341886080801486968994140625 ], "test_run_times": [ 39.61999999999999744204615126363933086395263671875, 39.75, 39.840000000000003410605131648480892181396484375 ] }, "b": { "value": 9.578900000000000858335624798201024532318115234375, "test_run_times": [ 39.77000000000000312638803734444081783294677734375 ] }, "c": { "value": 9.505900000000000460431692772544920444488525390625, "test_run_times": [ 39.74000000000000198951966012828052043914794921875 ] }, "d": { "value": 9.4749999999999996447286321199499070644378662109375, "raw_values": [ 9.448299999999999698729880037717521190643310546875, 9.497099999999999653255144949071109294891357421875, 9.4794999999999998152588887023739516735076904296875 ], "test_run_times": [ 39.68999999999999772626324556767940521240234375, 39.7999999999999971578290569595992565155029296875, 39.7000000000000028421709430404007434844970703125 ] }, "e": { "value": 9.567299999999999471356204594485461711883544921875, "test_run_times": [ 39.780000000000001136868377216160297393798828125 ] } } }, "fa0e1332a3568488061a24e2db603659f1c6501e": { "identifier": "pts\/deepsparse-1.7.0", "title": "Neural Magic DeepSparse", "app_version": "1.7", "arguments": "zoo:nlp\/document_classification\/obert-base\/pytorch\/huggingface\/imdb\/base-none --scenario sync", "description": "Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Stream", "scale": "ms\/batch", "proportion": "LIB", "display_format": "BAR_GRAPH", "results": { "a": { "value": 104.118799999999993133315001614391803741455078125, "raw_values": [ 103.9535000000000053432813729159533977508544921875, 103.847700000000003228706191293895244598388671875, 104.5550999999999959300112095661461353302001953125 ], "test_run_times": [ 39.61999999999999744204615126363933086395263671875, 39.75, 39.840000000000003410605131648480892181396484375 ] }, "b": { "value": 104.39199999999999590727384202182292938232421875, "test_run_times": [ 39.77000000000000312638803734444081783294677734375 ] }, "c": { "value": 105.1933999999999969077180139720439910888671875, "test_run_times": [ 39.74000000000000198951966012828052043914794921875 ] }, "d": { "value": 105.53780000000000427462509833276271820068359375, "raw_values": [ 105.834599999999994679455994628369808197021484375, 105.2913999999999958845364744774997234344482421875, 105.487300000000004729372449219226837158203125 ], "test_run_times": [ 39.68999999999999772626324556767940521240234375, 39.7999999999999971578290569595992565155029296875, 39.7000000000000028421709430404007434844970703125 ] }, "e": { "value": 104.5182999999999964302332955412566661834716796875, "test_run_times": [ 39.780000000000001136868377216160297393798828125 ] } } }, "a5ba11a554d6383c4486b0661fdd16e9399e857f": { "identifier": "pts\/deepsparse-1.7.0", "title": "Neural Magic DeepSparse", "app_version": "1.7", "arguments": "zoo:nlp\/sentiment_analysis\/oberta-base\/pytorch\/huggingface\/sst2\/pruned90_quant-none --input_shapes='[1,128]' --scenario async", "description": "Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Asynchronous Multi-Stream", "scale": "items\/sec", "proportion": "HIB", "display_format": "BAR_GRAPH", "results": { "a": { "value": 436.248299999999971987563185393810272216796875, "raw_values": [ 438.4461000000000012732925824820995330810546875, 433.0493999999999914507498033344745635986328125, 437.24939999999998008206603117287158966064453125 ], "test_run_times": [ 43.81000000000000227373675443232059478759765625, 44.030000000000001136868377216160297393798828125, 43.8599999999999994315658113919198513031005859375 ] }, "b": { "value": 434.2205000000000154614099301397800445556640625, "test_run_times": [ 46.9500000000000028421709430404007434844970703125 ] }, "c": { "value": 436.11579999999997880877344869077205657958984375, "test_run_times": [ 43.909999999999996589394868351519107818603515625 ] }, "d": { "value": 435.85410000000001673470251262187957763671875, "raw_values": [ 436.7894000000000005456968210637569427490234375, 435.66480000000001382431946694850921630859375, 435.10809999999997899067238904535770416259765625 ], "test_run_times": [ 43.78999999999999914734871708787977695465087890625, 43.86999999999999744204615126363933086395263671875, 44.92999999999999971578290569595992565155029296875 ] }, "e": { "value": 433.8822000000000116415321826934814453125, "test_run_times": [ 45.8299999999999982946974341757595539093017578125 ] } } }, "bda84ddf23206af7a77b4a03ef51ef2f11c11b45": { "identifier": "pts\/deepsparse-1.7.0", "title": "Neural Magic DeepSparse", "app_version": "1.7", "arguments": "zoo:nlp\/sentiment_analysis\/oberta-base\/pytorch\/huggingface\/sst2\/pruned90_quant-none --input_shapes='[1,128]' --scenario async", "description": "Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Asynchronous Multi-Stream", "scale": "ms\/batch", "proportion": "LIB", "display_format": "BAR_GRAPH", "results": { "a": { "value": 9.1560000000000005826450433232821524143218994140625, "raw_values": [ 9.109600000000000363797880709171295166015625, 9.2235999999999993548271959298290312290191650390625, 9.1348000000000002529532139305956661701202392578125 ], "test_run_times": [ 43.81000000000000227373675443232059478759765625, 44.030000000000001136868377216160297393798828125, 43.8599999999999994315658113919198513031005859375 ] }, "b": { "value": 9.1989999999999998436805981327779591083526611328125, "test_run_times": [ 46.9500000000000028421709430404007434844970703125 ] }, "c": { "value": 9.1595999999999992979837770690210163593292236328125, "test_run_times": [ 43.909999999999996589394868351519107818603515625 ] }, "d": { "value": 9.1641999999999992354560163221322000026702880859375, "raw_values": [ 9.1442999999999994287236404488794505596160888671875, 9.1685999999999996390442902338691055774688720703125, 9.179700000000000414956957683898508548736572265625 ], "test_run_times": [ 43.78999999999999914734871708787977695465087890625, 43.86999999999999744204615126363933086395263671875, 44.92999999999999971578290569595992565155029296875 ] }, "e": { "value": 9.2062000000000008270717444247566163539886474609375, "test_run_times": [ 45.8299999999999982946974341757595539093017578125 ] } } }, "855ea67bb65a4f10b8a649ffcc43fbeb7227cae3": { "identifier": "pts\/deepsparse-1.7.0", "title": "Neural Magic DeepSparse", "app_version": "1.7", "arguments": "zoo:nlp\/sentiment_analysis\/oberta-base\/pytorch\/huggingface\/sst2\/pruned90_quant-none --input_shapes='[1,128]' --scenario sync", "description": "Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Synchronous Single-Stream", "scale": "items\/sec", "proportion": "HIB", "display_format": "BAR_GRAPH", "results": { "a": { "value": 324.5629999999999881765688769519329071044921875, "raw_values": [ 324.1779000000000223735696636140346527099609375, 325.49560000000002446540747769176959991455078125, 324.015399999999999636202119290828704833984375 ], "test_run_times": [ 43.07000000000000028421709430404007434844970703125, 42.52000000000000312638803734444081783294677734375, 42.36999999999999744204615126363933086395263671875 ] }, "b": { "value": 327.191199999999980718712322413921356201171875, "test_run_times": [ 42.6400000000000005684341886080801486968994140625 ] }, "c": { "value": 326.88799999999997680788510479032993316650390625, "test_run_times": [ 42.38000000000000255795384873636066913604736328125 ] }, "d": { "value": 327.29070000000001527951098978519439697265625, "raw_values": [ 326.44089999999999918145476840436458587646484375, 326.97620000000000572981662116944789886474609375, 328.45499999999998408384271897375583648681640625 ], "test_run_times": [ 42.530000000000001136868377216160297393798828125, 42.60000000000000142108547152020037174224853515625, 42.67999999999999971578290569595992565155029296875 ] }, "e": { "value": 328.56049999999999045030563138425350189208984375, "test_run_times": [ 42.469999999999998863131622783839702606201171875 ] } } }, "5bbdba8bdb353a59e581a45b7f7943e3030de63a": { "identifier": "pts\/deepsparse-1.7.0", "title": "Neural Magic DeepSparse", "app_version": "1.7", "arguments": "zoo:nlp\/sentiment_analysis\/oberta-base\/pytorch\/huggingface\/sst2\/pruned90_quant-none --input_shapes='[1,128]' --scenario sync", "description": "Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Synchronous Single-Stream", "scale": "ms\/batch", "proportion": "LIB", "display_format": "BAR_GRAPH", "results": { "a": { "value": 3.077599999999999891286961428704671561717987060546875, "raw_values": [ 3.080999999999999960920149533194489777088165283203125, 3.06899999999999995026200849679298698902130126953125, 3.082899999999999973709918776876293122768402099609375 ], "test_run_times": [ 43.07000000000000028421709430404007434844970703125, 42.52000000000000312638803734444081783294677734375, 42.36999999999999744204615126363933086395263671875 ] }, "b": { "value": 3.053100000000000147082346302340738475322723388671875, "test_run_times": [ 42.6400000000000005684341886080801486968994140625 ] }, "c": { "value": 3.05600000000000004973799150320701301097869873046875, "test_run_times": [ 42.38000000000000255795384873636066913604736328125 ] }, "d": { "value": 3.052099999999999813127260495093651115894317626953125, "raw_values": [ 3.060000000000000053290705182007513940334320068359375, 3.055000000000000159872115546022541821002960205078125, 3.041399999999999881339363128063268959522247314453125 ], "test_run_times": [ 42.530000000000001136868377216160297393798828125, 42.60000000000000142108547152020037174224853515625, 42.67999999999999971578290569595992565155029296875 ] }, "e": { "value": 3.0403999999999999914734871708787977695465087890625, "test_run_times": [ 42.469999999999998863131622783839702606201171875 ] } } }, "9b57be456e41aaad993f0ce7481a5c9c9a04a9ff": { "identifier": "pts\/deepsparse-1.7.0", "title": "Neural Magic DeepSparse", "app_version": "1.7", "arguments": "zoo:cv\/classification\/resnet_v1-50\/pytorch\/sparseml\/imagenet\/base-none --scenario async", "description": "Model: ResNet-50, Baseline - Scenario: Asynchronous Multi-Stream", "scale": "items\/sec", "proportion": "HIB", "display_format": "BAR_GRAPH", "results": { "a": { "value": 143.03780000000000427462509833276271820068359375, "raw_values": [ 143.548900000000003274180926382541656494140625, 142.919900000000012596501619555056095123291015625, 142.644599999999996953192749060690402984619140625 ], "test_run_times": [ 36.909999999999996589394868351519107818603515625, 36.9500000000000028421709430404007434844970703125, 36.89999999999999857891452847979962825775146484375 ] }, "b": { "value": 143.1865999999999985448084771633148193359375, "test_run_times": [ 36.8900000000000005684341886080801486968994140625 ] }, "c": { "value": 143.391400000000004411049303598701953887939453125, "test_run_times": [ 37.030000000000001136868377216160297393798828125 ] }, "d": { "value": 142.998199999999997089616954326629638671875, "raw_values": [ 143.02899999999999636202119290828704833984375, 143.036599999999992860466591082513332366943359375, 142.929000000000002046363078989088535308837890625 ], "test_run_times": [ 36.97999999999999687361196265555918216705322265625, 36.9200000000000017053025658242404460906982421875, 36.97999999999999687361196265555918216705322265625 ] }, "e": { "value": 143.570099999999996498445398174226284027099609375, "test_run_times": [ 36.96000000000000085265128291212022304534912109375 ] } } }, "21a28bcec532bfa9a162a2718b6bd205c8afdb1e": { "identifier": "pts\/deepsparse-1.7.0", "title": "Neural Magic DeepSparse", "app_version": "1.7", "arguments": "zoo:cv\/classification\/resnet_v1-50\/pytorch\/sparseml\/imagenet\/base-none --scenario async", "description": "Model: ResNet-50, Baseline - Scenario: Asynchronous Multi-Stream", "scale": "ms\/batch", "proportion": "LIB", "display_format": "BAR_GRAPH", "results": { "a": { "value": 27.949200000000001153921402874402701854705810546875, "raw_values": [ 27.84949999999999903366187936626374721527099609375, 27.971900000000001540456651127897202968597412109375, 28.02629999999999910187398199923336505889892578125 ], "test_run_times": [ 36.909999999999996589394868351519107818603515625, 36.9500000000000028421709430404007434844970703125, 36.89999999999999857891452847979962825775146484375 ] }, "b": { "value": 27.9207999999999998408384271897375583648681640625, "test_run_times": [ 36.8900000000000005684341886080801486968994140625 ] }, "c": { "value": 27.88029999999999830606611794792115688323974609375, "test_run_times": [ 37.030000000000001136868377216160297393798828125 ] }, "d": { "value": 27.95720000000000027284841053187847137451171875, "raw_values": [ 27.951100000000000278532752417959272861480712890625, 27.949799999999999755573298898525536060333251953125, 27.97070000000000078443918027915060520172119140625 ], "test_run_times": [ 36.97999999999999687361196265555918216705322265625, 36.9200000000000017053025658242404460906982421875, 36.97999999999999687361196265555918216705322265625 ] }, "e": { "value": 27.84530000000000171667124959640204906463623046875, "test_run_times": [ 36.96000000000000085265128291212022304534912109375 ] } } }, "1da2e9cb8d7a6eb8e64ede4d748a5c63dbec8559": { "identifier": "pts\/deepsparse-1.7.0", "title": "Neural Magic DeepSparse", "app_version": "1.7", "arguments": "zoo:cv\/classification\/resnet_v1-50\/pytorch\/sparseml\/imagenet\/base-none --scenario sync", "description": "Model: ResNet-50, Baseline - Scenario: Synchronous Single-Stream", "scale": "items\/sec", "proportion": "HIB", "display_format": "BAR_GRAPH", "results": { "a": { "value": 124.9917999999999977944753482006490230560302734375, "raw_values": [ 125.564899999999994406607584096491336822509765625, 124.528899999999993042365531437098979949951171875, 124.881699999999995043253875337541103363037109375 ], "test_run_times": [ 37.10000000000000142108547152020037174224853515625, 36.8900000000000005684341886080801486968994140625, 36.909999999999996589394868351519107818603515625 ] }, "b": { "value": 126.0683999999999969077180139720439910888671875, "test_run_times": [ 37.18999999999999772626324556767940521240234375 ] }, "c": { "value": 125.7690999999999945657691569067537784576416015625, "test_run_times": [ 37 ] }, "d": { "value": 125.0031000000000034333424991928040981292724609375, "raw_values": [ 126.1296999999999997044142219237983226776123046875, 124.999899999999996680344338528811931610107421875, 123.8795999999999963847585604526102542877197265625 ], "test_run_times": [ 36.86999999999999744204615126363933086395263671875, 36.8599999999999994315658113919198513031005859375, 36.96000000000000085265128291212022304534912109375 ] }, "e": { "value": 125.798100000000005138645065017044544219970703125, "test_run_times": [ 36.8599999999999994315658113919198513031005859375 ] } } }, "02cfacf2ca4fbbb6b2413969891ed99032c6c05e": { "identifier": "pts\/deepsparse-1.7.0", "title": "Neural Magic DeepSparse", "app_version": "1.7", "arguments": "zoo:cv\/classification\/resnet_v1-50\/pytorch\/sparseml\/imagenet\/base-none --scenario sync", "description": "Model: ResNet-50, Baseline - Scenario: Synchronous Single-Stream", "scale": "ms\/batch", "proportion": "LIB", "display_format": "BAR_GRAPH", "results": { "a": { "value": 7.99669999999999969730879456619732081890106201171875, "raw_values": [ 7.95999999999999996447286321199499070644378662109375, 8.0264000000000006451728040701709687709808349609375, 8.0038000000000000255795384873636066913604736328125 ], "test_run_times": [ 37.10000000000000142108547152020037174224853515625, 36.8900000000000005684341886080801486968994140625, 36.909999999999996589394868351519107818603515625 ] }, "b": { "value": 7.928200000000000358113538823090493679046630859375, "test_run_times": [ 37.18999999999999772626324556767940521240234375 ] }, "c": { "value": 7.94669999999999987494447850622236728668212890625, "test_run_times": [ 37 ] }, "d": { "value": 7.9961000000000002074784788419492542743682861328125, "raw_values": [ 7.9244000000000003325340003357268869876861572265625, 7.99580000000000001847411112976260483264923095703125, 8.0679999999999996163069226895458996295928955078125 ], "test_run_times": [ 36.86999999999999744204615126363933086395263671875, 36.8599999999999994315658113919198513031005859375, 36.96000000000000085265128291212022304534912109375 ] }, "e": { "value": 7.9451999999999998181010596454143524169921875, "test_run_times": [ 36.8599999999999994315658113919198513031005859375 ] } } }, "4c16c458467e4713990cbf508dda0ad418740729": { "identifier": "pts\/deepsparse-1.7.0", "title": "Neural Magic DeepSparse", "app_version": "1.7", "arguments": "zoo:cv\/classification\/resnet_v1-50\/pytorch\/sparseml\/imagenet\/pruned95_uniform_quant-none --scenario async", "description": "Model: ResNet-50, Sparse INT8 - Scenario: Asynchronous Multi-Stream", "scale": "items\/sec", "proportion": "HIB", "display_format": "BAR_GRAPH", "results": { "a": { "value": 1152.251899999999977808329276740550994873046875, "raw_values": [ 1153.323000000000092768459580838680267333984375, 1145.732500000000072759576141834259033203125, 1157.700199999999995270627550780773162841796875 ], "test_run_times": [ 37.10000000000000142108547152020037174224853515625, 37.32000000000000028421709430404007434844970703125, 37.21000000000000085265128291212022304534912109375 ] }, "b": { "value": 1156.176600000000007639755494892597198486328125, "test_run_times": [ 37.2000000000000028421709430404007434844970703125 ] }, "c": { "value": 1129.727000000000089130480773746967315673828125, "test_run_times": [ 42.27000000000000312638803734444081783294677734375 ] }, "d": { "value": 1152.7523000000001047737896442413330078125, "raw_values": [ 1166.353299999999990177457220852375030517578125, 1159.950000000000045474735088646411895751953125, 1131.953700000000026193447411060333251953125 ], "test_run_times": [ 37.219999999999998863131622783839702606201171875, 37.14999999999999857891452847979962825775146484375, 37.2000000000000028421709430404007434844970703125 ] }, "e": { "value": 1161.297800000000052023096941411495208740234375, "test_run_times": [ 37.2999999999999971578290569595992565155029296875 ] } } }, "893f2428b81e1c66c35110b1e39867b98d7ffa5b": { "identifier": "pts\/deepsparse-1.7.0", "title": "Neural Magic DeepSparse", "app_version": "1.7", "arguments": "zoo:cv\/classification\/resnet_v1-50\/pytorch\/sparseml\/imagenet\/pruned95_uniform_quant-none --scenario async", "description": "Model: ResNet-50, Sparse INT8 - Scenario: Asynchronous Multi-Stream", "scale": "ms\/batch", "proportion": "LIB", "display_format": "BAR_GRAPH", "results": { "a": { "value": 3.46070000000000010942358130705542862415313720703125, "raw_values": [ 3.457300000000000039790393202565610408782958984375, 3.4809000000000001051603248924948275089263916015625, 3.44390000000000018332002582610584795475006103515625 ], "test_run_times": [ 37.10000000000000142108547152020037174224853515625, 37.32000000000000028421709430404007434844970703125, 37.21000000000000085265128291212022304534912109375 ] }, "b": { "value": 3.448799999999999865707422941341064870357513427734375, "test_run_times": [ 37.2000000000000028421709430404007434844970703125 ] }, "c": { "value": 3.529999999999999804600747665972448885440826416015625, "test_run_times": [ 42.27000000000000312638803734444081783294677734375 ] }, "d": { "value": 3.460100000000000175504055732744745910167694091796875, "raw_values": [ 3.41889999999999982804865794605575501918792724609375, 3.4382999999999999118927007657475769519805908203125, 3.523099999999999898392388786305673420429229736328125 ], "test_run_times": [ 37.219999999999998863131622783839702606201171875, 37.14999999999999857891452847979962825775146484375, 37.2000000000000028421709430404007434844970703125 ] }, "e": { "value": 3.4336000000000002074784788419492542743682861328125, "test_run_times": [ 37.2999999999999971578290569595992565155029296875 ] } } }, "3b395af9b9d076fb9bff8d47a249ec2fe8331f6e": { "identifier": "pts\/deepsparse-1.7.0", "title": "Neural Magic DeepSparse", "app_version": "1.7", "arguments": "zoo:cv\/classification\/resnet_v1-50\/pytorch\/sparseml\/imagenet\/pruned95_uniform_quant-none --scenario sync", "description": "Model: ResNet-50, Sparse INT8 - Scenario: Synchronous Single-Stream", "scale": "items\/sec", "proportion": "HIB", "display_format": "BAR_GRAPH", "results": { "a": { "value": 975.8076999999999543433659709990024566650390625, "raw_values": [ 967.3954999999999699866748414933681488037109375, 992.9423000000000456566340290009975433349609375, 937.98630000000002837623469531536102294921875, 990.30889999999999417923390865325927734375, 990.405700000000024374458007514476776123046875 ], "test_run_times": [ 36.96000000000000085265128291212022304534912109375, 37.06000000000000227373675443232059478759765625, 36.92999999999999971578290569595992565155029296875, 36.96000000000000085265128291212022304534912109375, 37.06000000000000227373675443232059478759765625 ] }, "b": { "value": 941.786299999999982901499606668949127197265625, "test_run_times": [ 37.02000000000000312638803734444081783294677734375 ] }, "c": { "value": 927.772600000000011277734301984310150146484375, "test_run_times": [ 37.36999999999999744204615126363933086395263671875 ] }, "d": { "value": 1011.714600000000018553691916167736053466796875, "raw_values": [ 1013.790600000000040381564758718013763427734375, 1009.8663999999999987267074175179004669189453125, 1011.4868999999999914507498033344745635986328125 ], "test_run_times": [ 36.92999999999999971578290569595992565155029296875, 37, 37 ] }, "e": { "value": 994.7676999999999907231540419161319732666015625, "test_run_times": [ 37.090000000000003410605131648480892181396484375 ] } } }, "314827a2e497fa3e23a149d2aba87047cb19e7e3": { "identifier": "pts\/deepsparse-1.7.0", "title": "Neural Magic DeepSparse", "app_version": "1.7", "arguments": "zoo:cv\/classification\/resnet_v1-50\/pytorch\/sparseml\/imagenet\/pruned95_uniform_quant-none --scenario sync", "description": "Model: ResNet-50, Sparse INT8 - Scenario: Synchronous Single-Stream", "scale": "ms\/batch", "proportion": "LIB", "display_format": "BAR_GRAPH", "results": { "a": { "value": 1.02249999999999996447286321199499070644378662109375, "raw_values": [ 1.031099999999999905497816143906675279140472412109375, 1.0044999999999999484856516573927365243434906005859375, 1.063099999999999933919525574310682713985443115234375, 1.0068999999999999062083588796667754650115966796875, 1.0069999999999998951949464753852225840091705322265625 ], "test_run_times": [ 36.96000000000000085265128291212022304534912109375, 37.06000000000000227373675443232059478759765625, 36.92999999999999971578290569595992565155029296875, 36.96000000000000085265128291212022304534912109375, 37.06000000000000227373675443232059478759765625 ] }, "b": { "value": 1.05909999999999993036681189551018178462982177734375, "test_run_times": [ 37.02000000000000312638803734444081783294677734375 ] }, "c": { "value": 1.0745999999999999996447286321199499070644378662109375, "test_run_times": [ 37.36999999999999744204615126363933086395263671875 ] }, "d": { "value": 0.9856000000000000316191517413244582712650299072265625, "raw_values": [ 0.983800000000000007815970093361102044582366943359375, 0.98740000000000005542233338928781449794769287109375, 0.985700000000000020605739337042905390262603759765625 ], "test_run_times": [ 36.92999999999999971578290569595992565155029296875, 37, 37 ] }, "e": { "value": 1.0023999999999999577227072222740389406681060791015625, "test_run_times": [ 37.090000000000003410605131648480892181396484375 ] } } }, "20f489ce3ebeafb9d92bbb4838ca4c0390d3dd4d": { "identifier": "pts\/deepsparse-1.7.0", "title": "Neural Magic DeepSparse", "app_version": "1.7", "arguments": "zoo:llama2-7b-llama2_chat_llama2_pretrain-base_quantized --scenario async", "description": "Model: Llama2 Chat 7b Quantized - Scenario: Asynchronous Multi-Stream", "scale": "items\/sec", "proportion": "HIB", "display_format": "BAR_GRAPH", "results": { "a": { "value": 7.66929999999999978399500832892954349517822265625, "raw_values": [ 7.5297999999999998266275724745355546474456787109375, 7.74030000000000040216718844021670520305633544921875, 7.737899999999999778310666442848742008209228515625 ], "test_run_times": [ 47.5499999999999971578290569595992565155029296875, 46.6400000000000005684341886080801486968994140625, 46.67999999999999971578290569595992565155029296875 ] }, "b": { "value": 7.75180000000000024584778657299466431140899658203125, "test_run_times": [ 46.52000000000000312638803734444081783294677734375 ] }, "c": { "value": 7.56350000000000033395508580724708735942840576171875, "test_run_times": [ 47.78999999999999914734871708787977695465087890625 ] }, "d": { "value": 7.64350000000000040500935938325710594654083251953125, "raw_values": [ 7.609899999999999664623828721232712268829345703125, 7.53659999999999996589394868351519107818603515625, 7.78409999999999957509544401546008884906768798828125 ], "test_run_times": [ 46.77000000000000312638803734444081783294677734375, 46.840000000000003410605131648480892181396484375, 46.469999999999998863131622783839702606201171875 ] }, "e": { "value": 7.61280000000000001136868377216160297393798828125, "test_run_times": [ 46.52000000000000312638803734444081783294677734375 ] } } }, "d7626651777606a7730e9a1546ff6fd717d306f3": { "identifier": "pts\/deepsparse-1.7.0", "title": "Neural Magic DeepSparse", "app_version": "1.7", "arguments": "zoo:llama2-7b-llama2_chat_llama2_pretrain-base_quantized --scenario async", "description": "Model: Llama2 Chat 7b Quantized - Scenario: Asynchronous Multi-Stream", "scale": "ms\/batch", "proportion": "LIB", "display_format": "BAR_GRAPH", "results": { "a": { "value": 518.2458000000000311047188006341457366943359375, "raw_values": [ 527.6960000000000263753463514149188995361328125, 513.435699999999997089616954326629638671875, 513.6055999999999812644091434776782989501953125 ], "test_run_times": [ 47.5499999999999971578290569595992565155029296875, 46.6400000000000005684341886080801486968994140625, 46.67999999999999971578290569595992565155029296875 ] }, "b": { "value": 512.689300000000002910383045673370361328125, "test_run_times": [ 46.52000000000000312638803734444081783294677734375 ] }, "c": { "value": 525.3862000000000307409209199249744415283203125, "test_run_times": [ 47.78999999999999914734871708787977695465087890625 ] }, "d": { "value": 520.0114999999999554347596131265163421630859375, "raw_values": [ 522.1978000000000292857293970882892608642578125, 527.25199999999995270627550780773162841796875, 510.5846999999999979991116560995578765869140625 ], "test_run_times": [ 46.77000000000000312638803734444081783294677734375, 46.840000000000003410605131648480892181396484375, 46.469999999999998863131622783839702606201171875 ] }, "e": { "value": 521.999199999999973442754708230495452880859375, "test_run_times": [ 46.52000000000000312638803734444081783294677734375 ] } } }, "4ad67f37de90d07a80a48a47c3595fdb13a68be8": { "identifier": "pts\/deepsparse-1.7.0", "title": "Neural Magic DeepSparse", "app_version": "1.7", "arguments": "zoo:llama2-7b-llama2_chat_llama2_pretrain-base_quantized --scenario sync", "description": "Model: Llama2 Chat 7b Quantized - Scenario: Synchronous Single-Stream", "scale": "items\/sec", "proportion": "HIB", "display_format": "BAR_GRAPH", "results": { "a": { "value": 11.2507999999999999118927007657475769519805908203125, "raw_values": [ 11.2453000000000002955857780762016773223876953125, 11.2568000000000001392663762089796364307403564453125, 11.250299999999999300825948012061417102813720703125 ], "test_run_times": [ 45.469999999999998863131622783839702606201171875, 45.469999999999998863131622783839702606201171875, 45.469999999999998863131622783839702606201171875 ] }, "b": { "value": 11.2506000000000003780087354243732988834381103515625, "test_run_times": [ 45.47999999999999687361196265555918216705322265625 ] }, "c": { "value": 11.242000000000000881072992342524230480194091796875, "test_run_times": [ 45.4500000000000028421709430404007434844970703125 ] }, "d": { "value": 11.2432999999999996276756064617075026035308837890625, "raw_values": [ 11.24719999999999942019712761975824832916259765625, 11.2446000000000001506350599811412394046783447265625, 11.2382000000000008554934538551606237888336181640625 ], "test_run_times": [ 45.46000000000000085265128291212022304534912109375, 45.67999999999999971578290569595992565155029296875, 45.57000000000000028421709430404007434844970703125 ] }, "e": { "value": 11.2370999999999998664179656771011650562286376953125, "test_run_times": [ 45.52000000000000312638803734444081783294677734375 ] } } }, "27ad8c70cf928aa19cccf637f113d2e489a4c94a": { "identifier": "pts\/deepsparse-1.7.0", "title": "Neural Magic DeepSparse", "app_version": "1.7", "arguments": "zoo:llama2-7b-llama2_chat_llama2_pretrain-base_quantized --scenario sync", "description": "Model: Llama2 Chat 7b Quantized - Scenario: Synchronous Single-Stream", "scale": "ms\/batch", "proportion": "LIB", "display_format": "BAR_GRAPH", "results": { "a": { "value": 88.86799999999999499777914024889469146728515625, "raw_values": [ 88.913399999999995770849636755883693695068359375, 88.819199999999995043253875337541103363037109375, 88.8713000000000050704329623840749263763427734375 ], "test_run_times": [ 45.469999999999998863131622783839702606201171875, 45.469999999999998863131622783839702606201171875, 45.469999999999998863131622783839702606201171875 ] }, "b": { "value": 88.8700999999999936562744551338255405426025390625, "test_run_times": [ 45.47999999999999687361196265555918216705322265625 ] }, "c": { "value": 88.9381000000000057070792536251246929168701171875, "test_run_times": [ 45.4500000000000028421709430404007434844970703125 ] }, "d": { "value": 88.9279999999999972715158946812152862548828125, "raw_values": [ 88.8979000000000070258465711958706378936767578125, 88.9171999999999940200723358429968357086181640625, 88.9689000000000049794834922067821025848388671875 ], "test_run_times": [ 45.46000000000000085265128291212022304534912109375, 45.67999999999999971578290569595992565155029296875, 45.57000000000000028421709430404007434844970703125 ] }, "e": { "value": 88.9779999999999944293449516408145427703857421875, "test_run_times": [ 45.52000000000000312638803734444081783294677734375 ] } } }, "329837c092e993e81c947bb88e5010ebc9891b31": { "identifier": "pts\/deepsparse-1.7.0", "title": "Neural Magic DeepSparse", "app_version": "1.7", "arguments": "zoo:cv\/classification\/resnet_v1-50\/pytorch\/sparseml\/imagenet\/base-none --scenario async", "description": "Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream", "scale": "items\/sec", "proportion": "HIB", "display_format": "BAR_GRAPH", "results": { "a": { "value": 142.91509999999999536157702095806598663330078125, "raw_values": [ 143.249200000000001864464138634502887725830078125, 142.738599999999991041477187536656856536865234375, 142.757399999999989859134075231850147247314453125 ], "test_run_times": [ 37.030000000000001136868377216160297393798828125, 36.99000000000000198951966012828052043914794921875, 37.17999999999999971578290569595992565155029296875 ] }, "b": { "value": 143.83580000000000609361450187861919403076171875, "test_run_times": [ 36.89999999999999857891452847979962825775146484375 ] }, "c": { "value": 143.288399999999995770849636755883693695068359375, "test_run_times": [ 36.96000000000000085265128291212022304534912109375 ] }, "d": { "value": 143.236700000000013233147910796105861663818359375, "raw_values": [ 143.6680000000000063664629124104976654052734375, 142.966599999999999681676854379475116729736328125, 143.07540000000000190993887372314929962158203125 ], "test_run_times": [ 37.030000000000001136868377216160297393798828125, 37.02000000000000312638803734444081783294677734375, 36.97999999999999687361196265555918216705322265625 ] }, "e": { "value": 144.74569999999999936335370875895023345947265625, "test_run_times": [ 37 ] } } }, "638213f946adb5ae9135ab4b8b446f7d58664d7f": { "identifier": "pts\/deepsparse-1.7.0", "title": "Neural Magic DeepSparse", "app_version": "1.7", "arguments": "zoo:cv\/classification\/resnet_v1-50\/pytorch\/sparseml\/imagenet\/base-none --scenario async", "description": "Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream", "scale": "ms\/batch", "proportion": "LIB", "display_format": "BAR_GRAPH", "results": { "a": { "value": 27.97449999999999903366187936626374721527099609375, "raw_values": [ 27.909099999999998686917024315334856510162353515625, 28.009100000000000108002495835535228252410888671875, 28.005199999999998539124135277234017848968505859375 ], "test_run_times": [ 37.030000000000001136868377216160297393798828125, 36.99000000000000198951966012828052043914794921875, 37.17999999999999971578290569595992565155029296875 ] }, "b": { "value": 27.7941000000000002501110429875552654266357421875, "test_run_times": [ 36.89999999999999857891452847979962825775146484375 ] }, "c": { "value": 27.90050000000000096633812063373625278472900390625, "test_run_times": [ 36.96000000000000085265128291212022304534912109375 ] }, "d": { "value": 27.91100000000000136424205265939235687255859375, "raw_values": [ 27.826899999999998414068613783456385135650634765625, 27.963699999999999334931999328546226024627685546875, 27.942499999999999005240169935859739780426025390625 ], "test_run_times": [ 37.030000000000001136868377216160297393798828125, 37.02000000000000312638803734444081783294677734375, 36.97999999999999687361196265555918216705322265625 ] }, "e": { "value": 27.61880000000000023874235921539366245269775390625, "test_run_times": [ 37 ] } } }, "f23135dde731648112460c357dc2312eab3ce978": { "identifier": "pts\/deepsparse-1.7.0", "title": "Neural Magic DeepSparse", "app_version": "1.7", "arguments": "zoo:cv\/classification\/resnet_v1-50\/pytorch\/sparseml\/imagenet\/base-none --scenario sync", "description": "Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Stream", "scale": "items\/sec", "proportion": "HIB", "display_format": "BAR_GRAPH", "results": { "a": { "value": 124.3319000000000045247361413203179836273193359375, "raw_values": [ 124.6809999999999973852027324028313159942626953125, 124.563299999999998135535861365497112274169921875, 123.7514000000000038426151149906218051910400390625 ], "test_run_times": [ 36.8299999999999982946974341757595539093017578125, 37.090000000000003410605131648480892181396484375, 36.92999999999999971578290569595992565155029296875 ] }, "b": { "value": 126.6727000000000060708771343342959880828857421875, "test_run_times": [ 36.8900000000000005684341886080801486968994140625 ] }, "c": { "value": 125.87069999999999936335370875895023345947265625, "test_run_times": [ 36.909999999999996589394868351519107818603515625 ] }, "d": { "value": 125.35120000000000572981662116944789886474609375, "raw_values": [ 126.1381999999999976580511429347097873687744140625, 124.9034999999999939745976007543504238128662109375, 125.011899999999997135091689415276050567626953125 ], "test_run_times": [ 36.8900000000000005684341886080801486968994140625, 37.030000000000001136868377216160297393798828125, 36.9200000000000017053025658242404460906982421875 ] }, "e": { "value": 126.112899999999996225596987642347812652587890625, "test_run_times": [ 36.9200000000000017053025658242404460906982421875 ] } } }, "948f96bcbf119b23cb418fb75b2bd04da31f599f": { "identifier": "pts\/deepsparse-1.7.0", "title": "Neural Magic DeepSparse", "app_version": "1.7", "arguments": "zoo:cv\/classification\/resnet_v1-50\/pytorch\/sparseml\/imagenet\/base-none --scenario sync", "description": "Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Stream", "scale": "ms\/batch", "proportion": "LIB", "display_format": "BAR_GRAPH", "results": { "a": { "value": 8.0390999999999994685140336514450609683990478515625, "raw_values": [ 8.016799999999999926103555480949580669403076171875, 8.0237999999999995992538970313034951686859130859375, 8.0767000000000006565414878423325717449188232421875 ], "test_run_times": [ 36.8299999999999982946974341757595539093017578125, 37.090000000000003410605131648480892181396484375, 36.92999999999999971578290569595992565155029296875 ] }, "b": { "value": 7.890399999999999636202119290828704833984375, "test_run_times": [ 36.8900000000000005684341886080801486968994140625 ] }, "c": { "value": 7.9405000000000001136868377216160297393798828125, "test_run_times": [ 36.909999999999996589394868351519107818603515625 ] }, "d": { "value": 7.9733999999999998209432305884547531604766845703125, "raw_values": [ 7.92360000000000042064129956997931003570556640625, 8.001599999999999823785401531495153903961181640625, 7.99509999999999987352339303470216691493988037109375 ], "test_run_times": [ 36.8900000000000005684341886080801486968994140625, 37.030000000000001136868377216160297393798828125, 36.9200000000000017053025658242404460906982421875 ] }, "e": { "value": 7.92480000000000028848035071860067546367645263671875, "test_run_times": [ 36.9200000000000017053025658242404460906982421875 ] } } }, "9f3ae2bc1f10bc938757c19ae97a957a6a9ee97a": { "identifier": "pts\/deepsparse-1.7.0", "title": "Neural Magic DeepSparse", "app_version": "1.7", "arguments": "zoo:cv\/detection\/yolov5-s\/pytorch\/ultralytics\/coco\/pruned85-none --scenario async", "description": "Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Asynchronous Multi-Stream", "scale": "items\/sec", "proportion": "HIB", "display_format": "BAR_GRAPH", "results": { "a": { "value": 71.6975000000000051159076974727213382720947265625, "raw_values": [ 71.759399999999999408828443847596645355224609375, 71.7000000000000028421709430404007434844970703125, 71.6332000000000022055246517993509769439697265625 ], "test_run_times": [ 37.090000000000003410605131648480892181396484375, 37.03999999999999914734871708787977695465087890625, 37.0499999999999971578290569595992565155029296875 ] }, "b": { "value": 71.74569999999999936335370875895023345947265625, "test_run_times": [ 37.159999999999996589394868351519107818603515625 ] }, "c": { "value": 71.762699999999995270627550780773162841796875, "test_run_times": [ 37.14999999999999857891452847979962825775146484375 ] }, "d": { "value": 71.7604999999999932924765744246542453765869140625, "raw_values": [ 71.8092999999999932470018393360078334808349609375, 71.7061000000000063892002799548208713531494140625, 71.7660000000000053432813729159533977508544921875 ], "test_run_times": [ 37.030000000000001136868377216160297393798828125, 36.96000000000000085265128291212022304534912109375, 37.13000000000000255795384873636066913604736328125 ] }, "e": { "value": 71.7753000000000014324541552923619747161865234375, "test_run_times": [ 37.1400000000000005684341886080801486968994140625 ] } } }, "7c1c5406d5fd0bc7f453370348b492b3d54aaa76": { "identifier": "pts\/deepsparse-1.7.0", "title": "Neural Magic DeepSparse", "app_version": "1.7", "arguments": "zoo:cv\/detection\/yolov5-s\/pytorch\/ultralytics\/coco\/pruned85-none --scenario async", "description": "Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Asynchronous Multi-Stream", "scale": "ms\/batch", "proportion": "LIB", "display_format": "BAR_GRAPH", "results": { "a": { "value": 55.77459999999999951114659779705107212066650390625, "raw_values": [ 55.726100000000002410160959698259830474853515625, 55.77279999999999660076355212368071079254150390625, 55.8248999999999995225152815692126750946044921875 ], "test_run_times": [ 37.090000000000003410605131648480892181396484375, 37.03999999999999914734871708787977695465087890625, 37.0499999999999971578290569595992565155029296875 ] }, "b": { "value": 55.7374999999999971578290569595992565155029296875, "test_run_times": [ 37.159999999999996589394868351519107818603515625 ] }, "c": { "value": 55.72420000000000328554961015470325946807861328125, "test_run_times": [ 37.14999999999999857891452847979962825775146484375 ] }, "d": { "value": 55.7256000000000000227373675443232059478759765625, "raw_values": [ 55.687399999999996680344338528811931610107421875, 55.76789999999999736246536485850811004638671875, 55.7214000000000027057467377744615077972412109375 ], "test_run_times": [ 37.030000000000001136868377216160297393798828125, 36.96000000000000085265128291212022304534912109375, 37.13000000000000255795384873636066913604736328125 ] }, "e": { "value": 55.71419999999999816964191268198192119598388671875, "test_run_times": [ 37.1400000000000005684341886080801486968994140625 ] } } }, "b2a594486c92f0c4e4b93b0a9dbe5a0be062f298": { "identifier": "pts\/deepsparse-1.7.0", "title": "Neural Magic DeepSparse", "app_version": "1.7", "arguments": "zoo:cv\/detection\/yolov5-s\/pytorch\/ultralytics\/coco\/pruned85-none --scenario sync", "description": "Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Synchronous Single-Stream", "scale": "items\/sec", "proportion": "HIB", "display_format": "BAR_GRAPH", "results": { "a": { "value": 67.2623999999999995225152815692126750946044921875, "raw_values": [ 67.2245999999999952478901832364499568939208984375, 67.2530000000000001136868377216160297393798828125, 67.3097000000000065256244852207601070404052734375 ], "test_run_times": [ 37.0499999999999971578290569595992565155029296875, 36.9500000000000028421709430404007434844970703125, 36.909999999999996589394868351519107818603515625 ] }, "b": { "value": 67.6240999999999985448084771633148193359375, "test_run_times": [ 36.97999999999999687361196265555918216705322265625 ] }, "c": { "value": 67.381699999999995043253875337541103363037109375, "test_run_times": [ 36.96000000000000085265128291212022304534912109375 ] }, "d": { "value": 67.41100000000000136424205265939235687255859375, "raw_values": [ 67.37160000000000081854523159563541412353515625, 67.664500000000003865352482534945011138916015625, 67.19679999999999608917278237640857696533203125 ], "test_run_times": [ 36.99000000000000198951966012828052043914794921875, 36.9500000000000028421709430404007434844970703125, 37.13000000000000255795384873636066913604736328125 ] }, "e": { "value": 67.31950000000000500222085975110530853271484375, "test_run_times": [ 37.03999999999999914734871708787977695465087890625 ] } } }, "a77140031a418d03f49e6debb4e03cbe7a03a678": { "identifier": "pts\/deepsparse-1.7.0", "title": "Neural Magic DeepSparse", "app_version": "1.7", "arguments": "zoo:cv\/detection\/yolov5-s\/pytorch\/ultralytics\/coco\/pruned85-none --scenario sync", "description": "Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Synchronous Single-Stream", "scale": "ms\/batch", "proportion": "LIB", "display_format": "BAR_GRAPH", "results": { "a": { "value": 14.86280000000000001136868377216160297393798828125, "raw_values": [ 14.8714999999999992752464095246978104114532470703125, 14.86410000000000053432813729159533977508544921875, 14.8527000000000004575895218295045197010040283203125 ], "test_run_times": [ 37.0499999999999971578290569595992565155029296875, 36.9500000000000028421709430404007434844970703125, 36.909999999999996589394868351519107818603515625 ] }, "b": { "value": 14.7837999999999993860910763032734394073486328125, "test_run_times": [ 36.97999999999999687361196265555918216705322265625 ] }, "c": { "value": 14.8368999999999999772626324556767940521240234375, "test_run_times": [ 36.96000000000000085265128291212022304534912109375 ] }, "d": { "value": 14.830500000000000682121026329696178436279296875, "raw_values": [ 14.8390000000000004121147867408581078052520751953125, 14.7751000000000001222133505507372319698333740234375, 14.8773999999999997356781022972427308559417724609375 ], "test_run_times": [ 36.99000000000000198951966012828052043914794921875, 36.9500000000000028421709430404007434844970703125, 37.13000000000000255795384873636066913604736328125 ] }, "e": { "value": 14.85040000000000048885340220294892787933349609375, "test_run_times": [ 37.03999999999999914734871708787977695465087890625 ] } } }, "4b08a77993201615e7cdd48a86e15cf72e9b3352": { "identifier": "pts\/deepsparse-1.7.0", "title": "Neural Magic DeepSparse", "app_version": "1.7", "arguments": "zoo:nlp\/text_classification\/distilbert-none\/pytorch\/huggingface\/mnli\/base-none --scenario async", "description": "Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Stream", "scale": "items\/sec", "proportion": "HIB", "display_format": "BAR_GRAPH", "results": { "a": { "value": 81.4912000000000062982508097775280475616455078125, "raw_values": [ 81.5139999999999957935870043002068996429443359375, 81.610500000000001818989403545856475830078125, 81.3491999999999961801222525537014007568359375 ], "test_run_times": [ 37.78999999999999914734871708787977695465087890625, 37.85000000000000142108547152020037174224853515625, 37.82000000000000028421709430404007434844970703125 ] }, "b": { "value": 81.5020999999999986584953148849308490753173828125, "test_run_times": [ 37.97999999999999687361196265555918216705322265625 ] }, "c": { "value": 81.5086000000000012732925824820995330810546875, "test_run_times": [ 43.0499999999999971578290569595992565155029296875 ] }, "d": { "value": 81.68229999999999790816218592226505279541015625, "raw_values": [ 81.623199999999997089616954326629638671875, 82.0929000000000002046363078989088535308837890625, 81.3307999999999964302332955412566661834716796875 ], "test_run_times": [ 37.780000000000001136868377216160297393798828125, 37.81000000000000227373675443232059478759765625, 37.7999999999999971578290569595992565155029296875 ] }, "e": { "value": 81.174499999999994770405464805662631988525390625, "test_run_times": [ 37.78999999999999914734871708787977695465087890625 ] } } }, "7108c1bde12eb0d9c6ff634d37ff517773086268": { "identifier": "pts\/deepsparse-1.7.0", "title": "Neural Magic DeepSparse", "app_version": "1.7", "arguments": "zoo:nlp\/text_classification\/distilbert-none\/pytorch\/huggingface\/mnli\/base-none --scenario async", "description": "Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Stream", "scale": "ms\/batch", "proportion": "LIB", "display_format": "BAR_GRAPH", "results": { "a": { "value": 49.0713999999999970214048516936600208282470703125, "raw_values": [ 49.05810000000000314912540488876402378082275390625, 48.9986999999999994770405464805662631988525390625, 49.15729999999999932924765744246542453765869140625 ], "test_run_times": [ 37.78999999999999914734871708787977695465087890625, 37.85000000000000142108547152020037174224853515625, 37.82000000000000028421709430404007434844970703125 ] }, "b": { "value": 49.066000000000002501110429875552654266357421875, "test_run_times": [ 37.97999999999999687361196265555918216705322265625 ] }, "c": { "value": 49.0611999999999994770405464805662631988525390625, "test_run_times": [ 43.0499999999999971578290569595992565155029296875 ] }, "d": { "value": 48.956400000000002137312549166381359100341796875, "raw_values": [ 48.9913999999999987267074175179004669189453125, 48.7105999999999994543031789362430572509765625, 49.16720000000000112549969344399869441986083984375 ], "test_run_times": [ 37.780000000000001136868377216160297393798828125, 37.81000000000000227373675443232059478759765625, 37.7999999999999971578290569595992565155029296875 ] }, "e": { "value": 49.261899999999997135091689415276050567626953125, "test_run_times": [ 37.78999999999999914734871708787977695465087890625 ] } } }, "ec149e1403247f5016fb1f96ce6614000e37c266": { "identifier": "pts\/deepsparse-1.7.0", "title": "Neural Magic DeepSparse", "app_version": "1.7", "arguments": "zoo:nlp\/text_classification\/distilbert-none\/pytorch\/huggingface\/mnli\/base-none --scenario sync", "description": "Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Stream", "scale": "items\/sec", "proportion": "HIB", "display_format": "BAR_GRAPH", "results": { "a": { "value": 49.7864000000000004320099833421409130096435546875, "raw_values": [ 49.83059999999999689634933019988238811492919921875, 49.53139999999999787405613460578024387359619140625, 49.9971000000000032059688237495720386505126953125 ], "test_run_times": [ 37.719999999999998863131622783839702606201171875, 37.9200000000000017053025658242404460906982421875, 37.78999999999999914734871708787977695465087890625 ] }, "b": { "value": 50.85439999999999827196006663143634796142578125, "test_run_times": [ 37.8299999999999982946974341757595539093017578125 ] }, "c": { "value": 50.89200000000000301270119962282478809356689453125, "test_run_times": [ 37.68999999999999772626324556767940521240234375 ] }, "d": { "value": 50.4405000000000001136868377216160297393798828125, "raw_values": [ 50.05720000000000169393388205207884311676025390625, 50.39800000000000324007487506605684757232666015625, 50.86630000000000251247911364771425724029541015625 ], "test_run_times": [ 37.75999999999999801048033987171947956085205078125, 37.8599999999999994315658113919198513031005859375, 37.8900000000000005684341886080801486968994140625 ] }, "e": { "value": 50.09420000000000072759576141834259033203125, "test_run_times": [ 37.75999999999999801048033987171947956085205078125 ] } } }, "1968a2c97ff6ff373d30fe5697bcd3ea3912ae50": { "identifier": "pts\/deepsparse-1.7.0", "title": "Neural Magic DeepSparse", "app_version": "1.7", "arguments": "zoo:nlp\/text_classification\/distilbert-none\/pytorch\/huggingface\/mnli\/base-none --scenario sync", "description": "Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Stream", "scale": "ms\/batch", "proportion": "LIB", "display_format": "BAR_GRAPH", "results": { "a": { "value": 20.082499999999999573674358543939888477325439453125, "raw_values": [ 20.064499999999998891553332214243710041046142578125, 20.18560000000000087538865045644342899322509765625, 19.997399999999998954081092961132526397705078125 ], "test_run_times": [ 37.719999999999998863131622783839702606201171875, 37.9200000000000017053025658242404460906982421875, 37.78999999999999914734871708787977695465087890625 ] }, "b": { "value": 19.66069999999999851070242584683001041412353515625, "test_run_times": [ 37.8299999999999982946974341757595539093017578125 ] }, "c": { "value": 19.64620000000000032969182939268648624420166015625, "test_run_times": [ 37.68999999999999772626324556767940521240234375 ] }, "d": { "value": 19.822700000000001097077984013594686985015869140625, "raw_values": [ 19.973600000000001131184035330079495906829833984375, 19.83840000000000003410605131648480892181396484375, 19.65599999999999880628820392303168773651123046875 ], "test_run_times": [ 37.75999999999999801048033987171947956085205078125, 37.8599999999999994315658113919198513031005859375, 37.8900000000000005684341886080801486968994140625 ] }, "e": { "value": 19.95870000000000032969182939268648624420166015625, "test_run_times": [ 37.75999999999999801048033987171947956085205078125 ] } } }, "029547d9862776523c5601395a6358610a352681": { "identifier": "pts\/deepsparse-1.7.0", "title": "Neural Magic DeepSparse", "app_version": "1.7", "arguments": "zoo:cv\/segmentation\/yolact-darknet53\/pytorch\/dbolya\/coco\/pruned90-none --scenario async", "description": "Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Stream", "scale": "items\/sec", "proportion": "HIB", "display_format": "BAR_GRAPH", "results": { "a": { "value": 14.815599999999999880628820392303168773651123046875, "raw_values": [ 14.9213000000000004519051799434237182140350341796875, 14.7109000000000005314859663485549390316009521484375, 14.8147000000000002017941369558684527873992919921875 ], "test_run_times": [ 40.49000000000000198951966012828052043914794921875, 40.38000000000000255795384873636066913604736328125, 43.85000000000000142108547152020037174224853515625 ] }, "b": { "value": 14.919499999999999317878973670303821563720703125, "test_run_times": [ 43.89999999999999857891452847979962825775146484375 ] }, "c": { "value": 14.9245000000000000994759830064140260219573974609375, "test_run_times": [ 40.25 ] }, "d": { "value": 14.8347999999999995424104781704954802989959716796875, "raw_values": [ 14.8980999999999994543031789362430572509765625, 14.7417999999999995708321876008994877338409423828125, 14.864599999999999369038050645031034946441650390625 ], "test_run_times": [ 41.11999999999999744204615126363933086395263671875, 41.67999999999999971578290569595992565155029296875, 44.63000000000000255795384873636066913604736328125 ] }, "e": { "value": 14.893499999999999516830939683131873607635498046875, "test_run_times": [ 40.469999999999998863131622783839702606201171875 ] } } }, "a6893edb09ab37f3c2a30cc382eee443bef51652": { "identifier": "pts\/deepsparse-1.7.0", "title": "Neural Magic DeepSparse", "app_version": "1.7", "arguments": "zoo:cv\/segmentation\/yolact-darknet53\/pytorch\/dbolya\/coco\/pruned90-none --scenario async", "description": "Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Stream", "scale": "ms\/batch", "proportion": "LIB", "display_format": "BAR_GRAPH", "results": { "a": { "value": 269.97259999999999990905052982270717620849609375, "raw_values": [ 268.05329999999997880877344869077205657958984375, 271.884099999999989449861459434032440185546875, 269.9805000000000063664629124104976654052734375 ], "test_run_times": [ 40.49000000000000198951966012828052043914794921875, 40.38000000000000255795384873636066913604736328125, 43.85000000000000142108547152020037174224853515625 ] }, "b": { "value": 268.08249999999998181010596454143524169921875, "test_run_times": [ 43.89999999999999857891452847979962825775146484375 ] }, "c": { "value": 267.99520000000001118678483180701732635498046875, "test_run_times": [ 40.25 ] }, "d": { "value": 269.62079999999997426129993982613086700439453125, "raw_values": [ 268.47079999999999699866748414933681488037109375, 271.31670000000002573870006017386913299560546875, 269.07490000000001373336999677121639251708984375 ], "test_run_times": [ 41.11999999999999744204615126363933086395263671875, 41.67999999999999971578290569595992565155029296875, 44.63000000000000255795384873636066913604736328125 ] }, "e": { "value": 268.55189999999998917701304890215396881103515625, "test_run_times": [ 40.469999999999998863131622783839702606201171875 ] } } }, "2e09524a2afac98ccf359361f18a9bf744359ec7": { "identifier": "pts\/deepsparse-1.7.0", "title": "Neural Magic DeepSparse", "app_version": "1.7", "arguments": "zoo:cv\/segmentation\/yolact-darknet53\/pytorch\/dbolya\/coco\/pruned90-none --scenario sync", "description": "Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Synchronous Single-Stream", "scale": "items\/sec", "proportion": "HIB", "display_format": "BAR_GRAPH", "results": { "a": { "value": 13.6990999999999996106225808034650981426239013671875, "raw_values": [ 13.6243999999999996219912645756267011165618896484375, 13.788199999999999789679350215010344982147216796875, 13.684599999999999653255144949071109294891357421875 ], "test_run_times": [ 39.96000000000000085265128291212022304534912109375, 40.7000000000000028421709430404007434844970703125, 39.840000000000003410605131648480892181396484375 ] }, "b": { "value": 13.998300000000000409272615797817707061767578125, "test_run_times": [ 39.8599999999999994315658113919198513031005859375 ] }, "c": { "value": 13.9961000000000002074784788419492542743682861328125, "test_run_times": [ 39.840000000000003410605131648480892181396484375 ] }, "d": { "value": 13.8745999999999991558752299170009791851043701171875, "raw_values": [ 13.9901999999999997470467860694043338298797607421875, 13.8680000000000003268496584496460855007171630859375, 13.76559999999999917008608463220298290252685546875 ], "test_run_times": [ 40.43999999999999772626324556767940521240234375, 40.469999999999998863131622783839702606201171875, 40.46000000000000085265128291212022304534912109375 ] }, "e": { "value": 13.8987999999999995992538970313034951686859130859375, "test_run_times": [ 40.3900000000000005684341886080801486968994140625 ] } } }, "4c1cc90e3ea0d3504b4076fbf87308d1ee648506": { "identifier": "pts\/deepsparse-1.7.0", "title": "Neural Magic DeepSparse", "app_version": "1.7", "arguments": "zoo:cv\/segmentation\/yolact-darknet53\/pytorch\/dbolya\/coco\/pruned90-none --scenario sync", "description": "Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Synchronous Single-Stream", "scale": "ms\/batch", "proportion": "LIB", "display_format": "BAR_GRAPH", "results": { "a": { "value": 72.9890999999999934289007796905934810638427734375, "raw_values": [ 73.3876000000000061618266045115888118743896484375, 72.515399999999999636202119290828704833984375, 73.064199999999999590727384202182292938232421875 ], "test_run_times": [ 39.96000000000000085265128291212022304534912109375, 40.7000000000000028421709430404007434844970703125, 39.840000000000003410605131648480892181396484375 ] }, "b": { "value": 71.4266999999999967485564411617815494537353515625, "test_run_times": [ 39.8599999999999994315658113919198513031005859375 ] }, "c": { "value": 71.438299999999998135535861365497112274169921875, "test_run_times": [ 39.840000000000003410605131648480892181396484375 ] }, "d": { "value": 72.066699999999997316990629769861698150634765625, "raw_values": [ 71.4681000000000068439476308412849903106689453125, 72.097700000000003228706191293895244598388671875, 72.63429999999999608917278237640857696533203125 ], "test_run_times": [ 40.43999999999999772626324556767940521240234375, 40.469999999999998863131622783839702606201171875, 40.46000000000000085265128291212022304534912109375 ] }, "e": { "value": 71.938299999999998135535861365497112274169921875, "test_run_times": [ 40.3900000000000005684341886080801486968994140625 ] } } }, "83ca1e360507a59226001525682d066a1eee5e84": { "identifier": "pts\/deepsparse-1.7.0", "title": "Neural Magic DeepSparse", "app_version": "1.7", "arguments": "zoo:nlp\/question_answering\/obert-large\/pytorch\/huggingface\/squad\/pruned97_quant-none --input_shapes='[1,128]' --scenario async", "description": "Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Asynchronous Multi-Stream", "scale": "items\/sec", "proportion": "HIB", "display_format": "BAR_GRAPH", "results": { "a": { "value": 196.62020000000001118678483180701732635498046875, "raw_values": [ 196.344500000000010686562745831906795501708984375, 196.73210000000000263753463514149188995361328125, 196.784099999999995134203345514833927154541015625 ], "test_run_times": [ 41.6099999999999994315658113919198513031005859375, 41.5799999999999982946974341757595539093017578125, 41.35000000000000142108547152020037174224853515625 ] }, "b": { "value": 197.483800000000002228262019343674182891845703125, "test_run_times": [ 41.18999999999999772626324556767940521240234375 ] }, "c": { "value": 197.27719999999999345163814723491668701171875, "test_run_times": [ 41.46000000000000085265128291212022304534912109375 ] }, "d": { "value": 197.196200000000004592948243953287601470947265625, "raw_values": [ 197.433999999999997498889570124447345733642578125, 197.325500000000005229594535194337368011474609375, 196.82900000000000773070496506989002227783203125 ], "test_run_times": [ 41.35000000000000142108547152020037174224853515625, 46.5499999999999971578290569595992565155029296875, 41.32000000000000028421709430404007434844970703125 ] }, "e": { "value": 197.5747000000000070940586738288402557373046875, "test_run_times": [ 41.43999999999999772626324556767940521240234375 ] } } }, "356fd68833e3222975779732b39790139f819b22": { "identifier": "pts\/deepsparse-1.7.0", "title": "Neural Magic DeepSparse", "app_version": "1.7", "arguments": "zoo:nlp\/question_answering\/obert-large\/pytorch\/huggingface\/squad\/pruned97_quant-none --input_shapes='[1,128]' --scenario async", "description": "Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Asynchronous Multi-Stream", "scale": "ms\/batch", "proportion": "LIB", "display_format": "BAR_GRAPH", "results": { "a": { "value": 20.3301000000000016143530956469476222991943359375, "raw_values": [ 20.358699999999998908606357872486114501953125, 20.31869999999999976125764078460633754730224609375, 20.312999999999998834709913353435695171356201171875 ], "test_run_times": [ 41.6099999999999994315658113919198513031005859375, 41.5799999999999982946974341757595539093017578125, 41.35000000000000142108547152020037174224853515625 ] }, "b": { "value": 20.240700000000000358113538823090493679046630859375, "test_run_times": [ 41.18999999999999772626324556767940521240234375 ] }, "c": { "value": 20.261900000000000687805368215776979923248291015625, "test_run_times": [ 41.46000000000000085265128291212022304534912109375 ] }, "d": { "value": 20.27029999999999887450030655600130558013916015625, "raw_values": [ 20.24549999999999982946974341757595539093017578125, 20.25710000000000121644916362129151821136474609375, 20.30839999999999889723767410032451152801513671875 ], "test_run_times": [ 41.35000000000000142108547152020037174224853515625, 46.5499999999999971578290569595992565155029296875, 41.32000000000000028421709430404007434844970703125 ] }, "e": { "value": 20.231500000000000483169060316868126392364501953125, "test_run_times": [ 41.43999999999999772626324556767940521240234375 ] } } }, "56185e1b61cd8aa824fb78f1d7d102710de33b94": { "identifier": "pts\/deepsparse-1.7.0", "title": "Neural Magic DeepSparse", "app_version": "1.7", "arguments": "zoo:nlp\/question_answering\/obert-large\/pytorch\/huggingface\/squad\/pruned97_quant-none --input_shapes='[1,128]' --scenario sync", "description": "Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Synchronous Single-Stream", "scale": "items\/sec", "proportion": "HIB", "display_format": "BAR_GRAPH", "results": { "a": { "value": 125.919399999999995998223312199115753173828125, "raw_values": [ 126.1684000000000054342308430932462215423583984375, 125.6046000000000049112713895738124847412109375, 125.98529999999999517967808060348033905029296875 ], "test_run_times": [ 41.159999999999996589394868351519107818603515625, 41.42999999999999971578290569595992565155029296875, 41.28999999999999914734871708787977695465087890625 ] }, "b": { "value": 125.448700000000002319211489520967006683349609375, "test_run_times": [ 41.22999999999999687361196265555918216705322265625 ] }, "c": { "value": 125.63339999999999463398125953972339630126953125, "test_run_times": [ 41.280000000000001136868377216160297393798828125 ] }, "d": { "value": 125.568500000000000227373675443232059478759765625, "raw_values": [ 125.7857999999999947249307297170162200927734375, 125.961399999999997589839040301740169525146484375, 124.958200000000005047695594839751720428466796875 ], "test_run_times": [ 41.27000000000000312638803734444081783294677734375, 41.1099999999999994315658113919198513031005859375, 41.28999999999999914734871708787977695465087890625 ] }, "e": { "value": 125.4107999999999947249307297170162200927734375, "test_run_times": [ 41.2999999999999971578290569595992565155029296875 ] } } }, "d4b9f05f0f69b018a9d07117bd621c6a7f7953e4": { "identifier": "pts\/deepsparse-1.7.0", "title": "Neural Magic DeepSparse", "app_version": "1.7", "arguments": "zoo:nlp\/question_answering\/obert-large\/pytorch\/huggingface\/squad\/pruned97_quant-none --input_shapes='[1,128]' --scenario sync", "description": "Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Synchronous Single-Stream", "scale": "ms\/batch", "proportion": "LIB", "display_format": "BAR_GRAPH", "results": { "a": { "value": 7.93480000000000007531752999057061970233917236328125, "raw_values": [ 7.91880000000000006110667527536861598491668701171875, 7.95469999999999988204990586382336914539337158203125, 7.9307999999999996276756064617075026035308837890625 ], "test_run_times": [ 41.159999999999996589394868351519107818603515625, 41.42999999999999971578290569595992565155029296875, 41.28999999999999914734871708787977695465087890625 ] }, "b": { "value": 7.9642999999999997129407347529195249080657958984375, "test_run_times": [ 41.22999999999999687361196265555918216705322265625 ] }, "c": { "value": 7.9518000000000004234834705130197107791900634765625, "test_run_times": [ 41.280000000000001136868377216160297393798828125 ] }, "d": { "value": 7.95709999999999961772800816106610000133514404296875, "raw_values": [ 7.94339999999999957225327307241968810558319091796875, 7.9320000000000003836930773104541003704071044921875, 7.99580000000000001847411112976260483264923095703125 ], "test_run_times": [ 41.27000000000000312638803734444081783294677734375, 41.1099999999999994315658113919198513031005859375, 41.28999999999999914734871708787977695465087890625 ] }, "e": { "value": 7.9672999999999998266275724745355546474456787109375, "test_run_times": [ 41.2999999999999971578290569595992565155029296875 ] } } }, "93d0edcc5ea127a7ae76411e7c930bccd559b10d": { "identifier": "pts\/deepsparse-1.7.0", "title": "Neural Magic DeepSparse", "app_version": "1.7", "arguments": "zoo:nlp\/token_classification\/bert-base\/pytorch\/huggingface\/conll2003\/base-none --scenario async", "description": "Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream", "scale": "items\/sec", "proportion": "HIB", "display_format": "BAR_GRAPH", "results": { "a": { "value": 10.5337999999999993860910763032734394073486328125, "raw_values": [ 10.5962999999999993860910763032734394073486328125, 10.5039999999999995594635038287378847599029541015625, 10.501200000000000756017470848746597766876220703125 ], "test_run_times": [ 40.07000000000000028421709430404007434844970703125, 39.9500000000000028421709430404007434844970703125, 39.909999999999996589394868351519107818603515625 ] }, "b": { "value": 10.5495000000000000994759830064140260219573974609375, "test_run_times": [ 40.1700000000000017053025658242404460906982421875 ] }, "c": { "value": 10.5739000000000000767386154620908200740814208984375, "test_run_times": [ 40.02000000000000312638803734444081783294677734375 ] }, "d": { "value": 10.5769000000000001904254531837068498134613037109375, "raw_values": [ 10.6336999999999992638777257525362074375152587890625, 10.5707000000000004291678123991005122661590576171875, 10.5264000000000006451728040701709687709808349609375 ], "test_run_times": [ 39.85000000000000142108547152020037174224853515625, 39.9500000000000028421709430404007434844970703125, 39.72999999999999687361196265555918216705322265625 ] }, "e": { "value": 10.599299999999999499777914024889469146728515625, "test_run_times": [ 39.9500000000000028421709430404007434844970703125 ] } } }, "1ef7bb431fa28fd7b9e986fd0cb9552c75643fd9": { "identifier": "pts\/deepsparse-1.7.0", "title": "Neural Magic DeepSparse", "app_version": "1.7", "arguments": "zoo:nlp\/token_classification\/bert-base\/pytorch\/huggingface\/conll2003\/base-none --scenario async", "description": "Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream", "scale": "ms\/batch", "proportion": "LIB", "display_format": "BAR_GRAPH", "results": { "a": { "value": 379.71409999999997353370417840778827667236328125, "raw_values": [ 377.46719999999999117790139280259609222412109375, 380.7866000000000212821760214865207672119140625, 380.88839999999999008650775067508220672607421875 ], "test_run_times": [ 40.07000000000000028421709430404007434844970703125, 39.9500000000000028421709430404007434844970703125, 39.909999999999996589394868351519107818603515625 ] }, "b": { "value": 379.142200000000002546585164964199066162109375, "test_run_times": [ 40.1700000000000017053025658242404460906982421875 ] }, "c": { "value": 378.262600000000020372681319713592529296875, "test_run_times": [ 40.02000000000000312638803734444081783294677734375 ] }, "d": { "value": 378.167599999999993087840266525745391845703125, "raw_values": [ 376.1408000000000129148247651755809783935546875, 378.38330000000001973603502847254276275634765625, 379.9787000000000034560798667371273040771484375 ], "test_run_times": [ 39.85000000000000142108547152020037174224853515625, 39.9500000000000028421709430404007434844970703125, 39.72999999999999687361196265555918216705322265625 ] }, "e": { "value": 377.36390000000000100044417195022106170654296875, "test_run_times": [ 39.9500000000000028421709430404007434844970703125 ] } } }, "8810fe9271cfde46003041442cc253185eb44e1c": { "identifier": "pts\/deepsparse-1.7.0", "title": "Neural Magic DeepSparse", "app_version": "1.7", "arguments": "zoo:nlp\/token_classification\/bert-base\/pytorch\/huggingface\/conll2003\/base-none --scenario sync", "description": "Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Stream", "scale": "items\/sec", "proportion": "HIB", "display_format": "BAR_GRAPH", "results": { "a": { "value": 9.4443999999999999062083588796667754650115966796875, "raw_values": [ 9.486299999999999954525264911353588104248046875, 9.41760000000000019326762412674725055694580078125, 9.4292999999999995708321876008994877338409423828125 ], "test_run_times": [ 39.60000000000000142108547152020037174224853515625, 39.74000000000000198951966012828052043914794921875, 39.719999999999998863131622783839702606201171875 ] }, "b": { "value": 9.4147999999999996134647517465054988861083984375, "test_run_times": [ 39.60000000000000142108547152020037174224853515625 ] }, "c": { "value": 9.4838000000000004519051799434237182140350341796875, "test_run_times": [ 39.60000000000000142108547152020037174224853515625 ] }, "d": { "value": 9.3638999999999992240873325499705970287322998046875, "raw_values": [ 9.4672999999999998266275724745355546474456787109375, 9.3377999999999996560973158921115100383758544921875, 9.28659999999999996589394868351519107818603515625 ], "test_run_times": [ 39.659999999999996589394868351519107818603515625, 39.6700000000000017053025658242404460906982421875, 39.46000000000000085265128291212022304534912109375 ] }, "e": { "value": 9.37819999999999964757080306299030780792236328125, "test_run_times": [ 39.719999999999998863131622783839702606201171875 ] } } }, "78aa5fcf7f9028bdaf1287fd59d1b0d907e4aca2": { "identifier": "pts\/deepsparse-1.7.0", "title": "Neural Magic DeepSparse", "app_version": "1.7", "arguments": "zoo:nlp\/token_classification\/bert-base\/pytorch\/huggingface\/conll2003\/base-none --scenario sync", "description": "Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Stream", "scale": "ms\/batch", "proportion": "LIB", "display_format": "BAR_GRAPH", "results": { "a": { "value": 105.8799999999999954525264911353588104248046875, "raw_values": [ 105.4112999999999971123543218709528446197509765625, 106.18049999999999499777914024889469146728515625, 106.048100000000005138645065017044544219970703125 ], "test_run_times": [ 39.60000000000000142108547152020037174224853515625, 39.74000000000000198951966012828052043914794921875, 39.719999999999998863131622783839702606201171875 ] }, "b": { "value": 106.210700000000002773958840407431125640869140625, "test_run_times": [ 39.60000000000000142108547152020037174224853515625 ] }, "c": { "value": 105.439300000000002910383045673370361328125, "test_run_times": [ 39.60000000000000142108547152020037174224853515625 ] }, "d": { "value": 106.7957999999999998408384271897375583648681640625, "raw_values": [ 105.6227999999999980218490236438810825347900390625, 107.0871000000000066165739553980529308319091796875, 107.677400000000005775291356258094310760498046875 ], "test_run_times": [ 39.659999999999996589394868351519107818603515625, 39.6700000000000017053025658242404460906982421875, 39.46000000000000085265128291212022304534912109375 ] }, "e": { "value": 106.6261999999999972033037920482456684112548828125, "test_run_times": [ 39.719999999999998863131622783839702606201171875 ] } } } } }