xeon-8490h-deepsparse

2 x Intel Xeon Platinum 8490H testing with a Quanta Cloud S6Q-MB-MPS (3A10.uh BIOS) and ASPEED on Ubuntu 22.04 via the Phoronix Test Suite.

HTML result view exported from: https://openbenchmarking.org/result/2307122-NE-XEON8490H02.

xeon-8490h-deepsparseProcessorMotherboardChipsetMemoryDiskGraphicsNetworkOSKernelDisplay ServerVulkanCompilerFile-SystemScreen Resolutionbert-all2 x Intel Xeon Platinum 8490H @ 3.50GHz (120 Cores / 240 Threads)Quanta Cloud S6Q-MB-MPS (3A10.uh BIOS)Intel Device 1bce1008GB8 x 1920GB Dell Ent NVMe AGN RI U.2 1.92TB + 1920GB INTEL SSDSC2KG01 + 800GB INTEL SSDSC2BA80 + 800GB INTEL SSDSC2BB80ASPEED4 x Intel E810-C for QSFP + 2 x Intel X710 for 10GBASE-TUbuntu 22.045.15.0-76-generic (x86_64)X Server1.3.224GCC 11.3.0ext41024x768OpenBenchmarking.org- Transparent Huge Pages: madvise- Scaling Governor: intel_pstate performance (EPP: performance) - CPU Microcode: 0x2b000181- Python 3.10.6- itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl and seccomp + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Enhanced IBRS IBPB: conditional RSB filling PBRSB-eIBRS: SW sequence + srbds: Not affected + tsx_async_abort: Not affected

xeon-8490h-deepsparsedeepsparse: NLP Text Classification, BERT base uncased SST2 - Asynchronous Multi-Streamdeepsparse: NLP Text Classification, BERT base uncased SST2 - Asynchronous Multi-Streamdeepsparse: NLP Text Classification, BERT base uncased SST2 - Synchronous Single-Streamdeepsparse: NLP Text Classification, BERT base uncased SST2 - Synchronous Single-Streambert-all435.6587137.507058.976516.9498OpenBenchmarking.org

Neural Magic DeepSparse

Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.5Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Asynchronous Multi-Streambert-all90180270360450SE +/- 5.96, N = 3435.66

Neural Magic DeepSparse

Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.5Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Asynchronous Multi-Streambert-all306090120150SE +/- 1.91, N = 3137.51

Neural Magic DeepSparse

Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.5Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Synchronous Single-Streambert-all1326395265SE +/- 0.21, N = 358.98

Neural Magic DeepSparse

Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.5Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Synchronous Single-Streambert-all48121620SE +/- 0.06, N = 316.95

Phoronix Test Suite v10.8.5