decking AMD Custom APU 0405 testing with a Valve Jupiter v1 (F7A0110 BIOS) and AMD Custom GPU 0405 1GB on SteamOS rolling via the Phoronix Test Suite. a: Processor: AMD Custom APU 0405 @ 2.80GHz (4 Cores / 8 Threads), Motherboard: Valve Jupiter v1 (F7A0110 BIOS), Chipset: AMD VanGogh Root Complex, Memory: 16GB, Disk: 512GB Phison ESMP512GKB4C3-E13TS + 1000GB RTL9210B-CG, Graphics: AMD Custom GPU 0405 1GB (1600/400MHz), Audio: AMD Rembrandt Radeon HD Audio, Monitor: ANX7530 U, Network: Realtek RTL8822CE 802.11ac PCIe OS: SteamOS rolling, Kernel: 5.13.0-valve36-1-neptune (x86_64), Desktop: KDE Plasma 5.26.1, Display Server: X Server 1.21.1.3, OpenGL: 4.6 Mesa 22.2.0 (git-17e5312102) (LLVM 14.0.6 DRM 3.45), Vulkan: 1.3.238, Compiler: GCC 12.2.0, File-System: ext4, Screen Resolution: 1280x800 b: Processor: AMD Custom APU 0405 @ 2.80GHz (4 Cores / 8 Threads), Motherboard: Valve Jupiter v1 (F7A0110 BIOS), Chipset: AMD VanGogh Root Complex, Memory: 16GB, Disk: 512GB Phison ESMP512GKB4C3-E13TS + 1000GB RTL9210B-CG, Graphics: AMD Custom GPU 0405 1GB (1600/400MHz), Audio: AMD Rembrandt Radeon HD Audio, Monitor: ANX7530 U, Network: Realtek RTL8822CE 802.11ac PCIe OS: SteamOS rolling, Kernel: 5.13.0-valve36-1-neptune (x86_64), Desktop: KDE Plasma 5.26.1, Display Server: X Server 1.21.1.3, OpenGL: 4.6 Mesa 22.2.0 (git-17e5312102) (LLVM 14.0.6 DRM 3.45), Vulkan: 1.3.238, Compiler: GCC 12.2.0, File-System: ext4, Screen Resolution: 1280x800 c: Processor: AMD Custom APU 0405 @ 2.80GHz (4 Cores / 8 Threads), Motherboard: Valve Jupiter v1 (F7A0110 BIOS), Chipset: AMD VanGogh Root Complex, Memory: 16GB, Disk: 512GB Phison ESMP512GKB4C3-E13TS + 1000GB RTL9210B-CG, Graphics: AMD Custom GPU 0405 1GB (1600/400MHz), Audio: AMD Rembrandt Radeon HD Audio, Monitor: ANX7530 U, Network: Realtek RTL8822CE 802.11ac PCIe OS: SteamOS rolling, Kernel: 5.13.0-valve36-1-neptune (x86_64), Desktop: KDE Plasma 5.26.1, Display Server: X Server 1.21.1.3, OpenGL: 4.6 Mesa 22.2.0 (git-17e5312102) (LLVM 14.0.6 DRM 3.45), Vulkan: 1.3.238, Compiler: GCC 12.2.0, File-System: ext4, Screen Resolution: 1280x800 vkpeak 20230730 fp32-scalar GFLOPS > Higher Is Better a . 1638.24 |================================================================== b . 1638.72 |================================================================== c . 1639.27 |================================================================== vkpeak 20230730 fp32-vec4 GFLOPS > Higher Is Better a . 1626.08 |================================================================== b . 1626.60 |================================================================== c . 1628.05 |================================================================== vkpeak 20230730 fp16-scalar GFLOPS > Higher Is Better a . 1638.60 |================================================================== b . 1639.31 |================================================================== c . 1639.19 |================================================================== vkpeak 20230730 fp16-vec4 GFLOPS > Higher Is Better a . 2618.33 |================================================================== b . 2618.49 |================================================================== c . 2619.64 |================================================================== vkpeak 20230730 fp64-scalar GFLOPS > Higher Is Better a . 102.02 |=================================================================== b . 102.01 |=================================================================== c . 102.03 |=================================================================== vkpeak 20230730 int32-scalar GIOPS > Higher Is Better a . 279.05 |=================================================================== b . 279.13 |=================================================================== c . 279.16 |=================================================================== vkpeak 20230730 int32-vec4 GIOPS > Higher Is Better a . 326.93 |=================================================================== b . 326.95 |=================================================================== c . 327.27 |=================================================================== vkpeak 20230730 int16-scalar GIOPS > Higher Is Better a . 1638.31 |================================================================== b . 1638.91 |================================================================== c . 1639.04 |================================================================== vkpeak 20230730 int16-vec4 GIOPS > Higher Is Better a . 2620.00 |================================================================== b . 2619.45 |================================================================== c . 2621.89 |================================================================== Apache IoTDB 1.1.2 Device Count: 100 - Batch Size Per Write: 1 - Sensor Count: 200 Apache IoTDB 1.1.2 Device Count: 100 - Batch Size Per Write: 1 - Sensor Count: 500 point/sec > Higher Is Better a . 711045.2 |================================================================= Apache IoTDB 1.1.2 Device Count: 100 - Batch Size Per Write: 1 - Sensor Count: 500 Average Latency < Lower Is Better a . 56.1 |===================================================================== Apache IoTDB 1.1.2 Device Count: 200 - Batch Size Per Write: 1 - Sensor Count: 200 Apache IoTDB 1.1.2 Device Count: 200 - Batch Size Per Write: 1 - Sensor Count: 500 Apache IoTDB 1.1.2 Device Count: 500 - Batch Size Per Write: 1 - Sensor Count: 200 Apache IoTDB 1.1.2 Device Count: 500 - Batch Size Per Write: 1 - Sensor Count: 500 Apache IoTDB 1.1.2 Device Count: 100 - Batch Size Per Write: 100 - Sensor Count: 200 Apache IoTDB 1.1.2 Device Count: 100 - Batch Size Per Write: 100 - Sensor Count: 500 Apache IoTDB 1.1.2 Device Count: 200 - Batch Size Per Write: 100 - Sensor Count: 200 Apache IoTDB 1.1.2 Device Count: 200 - Batch Size Per Write: 100 - Sensor Count: 500 Apache IoTDB 1.1.2 Device Count: 500 - Batch Size Per Write: 100 - Sensor Count: 200 Apache IoTDB 1.1.2 Device Count: 500 - Batch Size Per Write: 100 - Sensor Count: 500 Neural Magic DeepSparse 1.5 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 2.1122 |============================================================= b . 2.1213 |============================================================= c . 2.3264 |=================================================================== Neural Magic DeepSparse 1.5 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 940.70 |=================================================================== b . 940.47 |=================================================================== c . 853.08 |============================================================= Neural Magic DeepSparse 1.5 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 2.1732 |============================================================= b . 2.1968 |============================================================== c . 2.3847 |=================================================================== Neural Magic DeepSparse 1.5 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 460.14 |=================================================================== b . 455.19 |================================================================== c . 419.32 |============================================================= Neural Magic DeepSparse 1.5 Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 53.70 |=============================================================== b . 53.32 |============================================================== c . 58.14 |==================================================================== Neural Magic DeepSparse 1.5 Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 37.19 |==================================================================== b . 37.46 |==================================================================== c . 34.35 |============================================================== Neural Magic DeepSparse 1.5 Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 50.18 |============================================================= b . 50.23 |============================================================= c . 55.62 |==================================================================== Neural Magic DeepSparse 1.5 Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 19.91 |==================================================================== b . 19.89 |==================================================================== c . 17.96 |============================================================= Neural Magic DeepSparse 1.5 Model: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 21.31 |================================================================= b . 21.09 |================================================================= c . 22.18 |==================================================================== Neural Magic DeepSparse 1.5 Model: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 93.82 |=================================================================== b . 94.75 |==================================================================== c . 90.14 |================================================================= Neural Magic DeepSparse 1.5 Model: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 18.18 |=============================================================== b . 18.18 |=============================================================== c . 19.50 |==================================================================== Neural Magic DeepSparse 1.5 Model: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 55.00 |==================================================================== b . 55.00 |==================================================================== c . 51.27 |=============================================================== Neural Magic DeepSparse 1.5 Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 6.4230 |========================================================== b . 6.4754 |=========================================================== c . 7.3794 |=================================================================== Neural Magic DeepSparse 1.5 Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 311.13 |=================================================================== b . 308.56 |================================================================== c . 270.32 |========================================================== Neural Magic DeepSparse 1.5 Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 6.0942 |============================================================= b . 6.1143 |============================================================= c . 6.6652 |=================================================================== Neural Magic DeepSparse 1.5 Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 164.08 |=================================================================== b . 163.53 |=================================================================== c . 150.01 |============================================================= Neural Magic DeepSparse 1.5 Model: ResNet-50, Baseline - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 26.99 |============================================================== b . 26.65 |============================================================= c . 29.47 |==================================================================== Neural Magic DeepSparse 1.5 Model: ResNet-50, Baseline - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 74.06 |=================================================================== b . 75.00 |==================================================================== c . 67.83 |============================================================= Neural Magic DeepSparse 1.5 Model: ResNet-50, Baseline - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 25.14 |=============================================================== b . 24.91 |=============================================================== c . 27.07 |==================================================================== Neural Magic DeepSparse 1.5 Model: ResNet-50, Baseline - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 39.76 |=================================================================== b . 40.13 |==================================================================== c . 36.93 |=============================================================== Neural Magic DeepSparse 1.5 Model: ResNet-50, Sparse INT8 - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 188.96 |============================================================== b . 188.31 |============================================================== c . 203.55 |=================================================================== Neural Magic DeepSparse 1.5 Model: ResNet-50, Sparse INT8 - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 10.5418 |================================================================== b . 10.5794 |================================================================== c . 9.7887 |============================================================= Neural Magic DeepSparse 1.5 Model: ResNet-50, Sparse INT8 - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 170.87 |============================================================== b . 170.41 |============================================================== c . 184.00 |=================================================================== Neural Magic DeepSparse 1.5 Model: ResNet-50, Sparse INT8 - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 5.8323 |=================================================================== b . 5.8481 |=================================================================== c . 5.4147 |============================================================== Neural Magic DeepSparse 1.5 Model: CV Detection, YOLOv5s COCO - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 12.90 |=============================================================== b . 12.96 |=============================================================== c . 13.98 |==================================================================== Neural Magic DeepSparse 1.5 Model: CV Detection, YOLOv5s COCO - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 154.78 |=================================================================== b . 154.12 |=================================================================== c . 143.06 |============================================================== Neural Magic DeepSparse 1.5 Model: CV Detection, YOLOv5s COCO - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 12.33 |================================================================ b . 12.38 |================================================================ c . 13.18 |==================================================================== Neural Magic DeepSparse 1.5 Model: CV Detection, YOLOv5s COCO - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 81.05 |==================================================================== b . 80.76 |==================================================================== c . 75.88 |================================================================ Neural Magic DeepSparse 1.5 Model: BERT-Large, NLP Question Answering - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 2.7457 |=================================================================== Neural Magic DeepSparse 1.5 Model: BERT-Large, NLP Question Answering - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 724.89 |=================================================================== Neural Magic DeepSparse 1.5 Model: BERT-Large, NLP Question Answering - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 2.6313 |=================================================================== Neural Magic DeepSparse 1.5 Model: BERT-Large, NLP Question Answering - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 380.02 |=================================================================== Neural Magic DeepSparse 1.5 Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 26.58 |============================================================= b . 28.87 |=================================================================== c . 29.51 |==================================================================== Neural Magic DeepSparse 1.5 Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 75.18 |==================================================================== b . 69.21 |=============================================================== c . 67.74 |============================================================= Neural Magic DeepSparse 1.5 Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 24.57 |============================================================== b . 26.97 |==================================================================== c . 26.97 |==================================================================== Neural Magic DeepSparse 1.5 Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 40.67 |==================================================================== b . 37.06 |============================================================== c . 37.06 |============================================================== Neural Magic DeepSparse 1.5 Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 13.13 |=============================================================== b . 14.22 |==================================================================== c . 14.28 |==================================================================== Neural Magic DeepSparse 1.5 Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 152.07 |=================================================================== b . 140.63 |============================================================== c . 140.01 |============================================================== Neural Magic DeepSparse 1.5 Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 12.52 |=============================================================== b . 13.51 |==================================================================== c . 13.50 |==================================================================== Neural Magic DeepSparse 1.5 Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 79.85 |==================================================================== b . 74.01 |=============================================================== c . 74.03 |=============================================================== Neural Magic DeepSparse 1.5 Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 18.11 |=============================================================== b . 19.28 |=================================================================== c . 19.68 |==================================================================== Neural Magic DeepSparse 1.5 Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 110.31 |=================================================================== b . 103.59 |=============================================================== c . 101.61 |============================================================== Neural Magic DeepSparse 1.5 Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 15.64 |================================================================= b . 16.38 |==================================================================== c . 16.48 |==================================================================== Neural Magic DeepSparse 1.5 Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 63.92 |==================================================================== b . 61.04 |================================================================= c . 60.65 |================================================================= Neural Magic DeepSparse 1.5 Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 2.2637 |========================================================== b . 2.6289 |=================================================================== c . 2.5672 |================================================================= Neural Magic DeepSparse 1.5 Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 878.31 |=================================================================== b . 760.72 |========================================================== c . 778.47 |=========================================================== Neural Magic DeepSparse 1.5 Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 2.3648 |============================================================== b . 2.5560 |=================================================================== c . 2.5558 |=================================================================== Neural Magic DeepSparse 1.5 Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 422.84 |=================================================================== b . 391.21 |============================================================== c . 391.23 |============================================================== Neural Magic DeepSparse 1.5 Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 27.72 |=============================================================== b . 29.84 |==================================================================== c . 29.81 |==================================================================== Neural Magic DeepSparse 1.5 Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 72.11 |==================================================================== b . 66.96 |=============================================================== c . 67.04 |=============================================================== Neural Magic DeepSparse 1.5 Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 25.77 |============================================================= b . 28.55 |==================================================================== c . 28.71 |==================================================================== Neural Magic DeepSparse 1.5 Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 38.79 |==================================================================== b . 35.01 |============================================================= c . 34.82 |============================================================= Neural Magic DeepSparse 1.5 Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 9.9683 |============================================================ b . 11.0097 |================================================================== c . 10.9375 |================================================================== Neural Magic DeepSparse 1.5 Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 200.46 |=================================================================== b . 181.62 |============================================================= c . 182.70 |============================================================= Neural Magic DeepSparse 1.5 Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 8.7687 |============================================================== b . 9.4427 |=================================================================== c . 9.4377 |=================================================================== Neural Magic DeepSparse 1.5 Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 114.03 |=================================================================== b . 105.89 |============================================================== c . 105.94 |============================================================== Neural Magic DeepSparse 1.5 Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 2.1364 |============================================================= b . 2.2798 |================================================================= c . 2.3420 |=================================================================== Neural Magic DeepSparse 1.5 Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 932.31 |=================================================================== b . 877.23 |=============================================================== c . 850.67 |============================================================= Neural Magic DeepSparse 1.5 Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 2.2291 |============================================================== b . 2.3896 |=================================================================== c . 2.3825 |=================================================================== Neural Magic DeepSparse 1.5 Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 448.61 |=================================================================== b . 418.47 |============================================================== c . 419.71 |=============================================================== Apache Cassandra 4.1.3 Test: Writes Op/s > Higher Is Better a . 31081 |==================================================================== b . 29616 |================================================================= vkpeak 20230730 fp64-vec4 GFLOPS > Higher Is Better b . 102.03 |=================================================================== c . 102.06 |===================================================================