dgga AMD Ryzen Threadripper 3970X 32-Core testing with a ASUS ROG ZENITH II EXTREME (1802 BIOS) and AMD Radeon RX 5700 8GB on Ubuntu 22.04 via the Phoronix Test Suite.
HTML result view exported from: https://openbenchmarking.org/result/2312245-NE-DGGA0402643&sro&gru .
dgga Processor Motherboard Chipset Memory Disk Graphics Audio Monitor Network OS Kernel Desktop Display Server OpenGL Vulkan Compiler File-System Screen Resolution AMD Ryzen Threadripper 3970X 32-Core b c AMD Ryzen Threadripper 3970X 32-Core @ 3.70GHz (32 Cores / 64 Threads) ASUS ROG ZENITH II EXTREME (1802 BIOS) AMD Starship/Matisse 64GB Samsung SSD 980 PRO 500GB AMD Radeon RX 5700 8GB (1750/875MHz) AMD Navi 10 HDMI Audio ASUS VP28U Aquantia AQC107 NBase-T/IEEE + Intel I211 + Intel Wi-Fi 6 AX200 Ubuntu 22.04 6.2.0-36-generic (x86_64) GNOME Shell 42.2 X Server + Wayland 4.6 Mesa 22.0.1 (LLVM 13.0.1 DRM 3.49) 1.2.204 GCC 11.4.0 ext4 3840x2160 OpenBenchmarking.org Kernel Details - Transparent Huge Pages: madvise Compiler Details - --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-link-serialization=2 --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-targets=nvptx-none=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-build-config=bootstrap-lto-lean --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v Processor Details - Scaling Governor: acpi-cpufreq schedutil (Boost: Enabled) - CPU Microcode: 0x830107a Java Details - OpenJDK Runtime Environment (build 11.0.20.1+1-post-Ubuntu-0ubuntu122.04) Python Details - Python 3.10.12 Security Details - gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Mitigation of untrained return thunk; SMT enabled with STIBP protection + spec_rstack_overflow: Mitigation of safe RET + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional STIBP: always-on RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected
dgga pytorch: CPU - 1 - ResNet-50 pytorch: CPU - 1 - ResNet-152 pytorch: CPU - 16 - ResNet-50 pytorch: CPU - 32 - ResNet-50 pytorch: CPU - 64 - ResNet-50 pytorch: CPU - 16 - ResNet-152 pytorch: CPU - 32 - ResNet-152 pytorch: CPU - 64 - ResNet-152 pytorch: CPU - 1 - Efficientnet_v2_l pytorch: CPU - 16 - Efficientnet_v2_l pytorch: CPU - 32 - Efficientnet_v2_l pytorch: CPU - 64 - Efficientnet_v2_l openssl: SHA256 openssl: SHA512 openssl: ChaCha20 openssl: AES-128-GCM openssl: AES-256-GCM openssl: ChaCha20-Poly1305 embree: Pathtracer - Crown embree: Pathtracer ISPC - Crown embree: Pathtracer - Asian Dragon embree: Pathtracer - Asian Dragon Obj embree: Pathtracer ISPC - Asian Dragon embree: Pathtracer ISPC - Asian Dragon Obj svt-av1: Preset 4 - Bosphorus 4K svt-av1: Preset 8 - Bosphorus 4K svt-av1: Preset 12 - Bosphorus 4K svt-av1: Preset 13 - Bosphorus 4K svt-av1: Preset 4 - Bosphorus 1080p svt-av1: Preset 8 - Bosphorus 1080p svt-av1: Preset 12 - Bosphorus 1080p svt-av1: Preset 13 - Bosphorus 1080p xmrig: KawPow - 1M xmrig: Monero - 1M xmrig: Wownero - 1M xmrig: GhostRider - 1M xmrig: CryptoNight-Heavy - 1M xmrig: CryptoNight-Femto UPX2 - 1M deepsparse: NLP Document Classification, oBERT base uncased on IMDB - Asynchronous Multi-Stream deepsparse: NLP Document Classification, oBERT base uncased on IMDB - Synchronous Single-Stream deepsparse: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Asynchronous Multi-Stream deepsparse: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Synchronous Single-Stream deepsparse: ResNet-50, Baseline - Asynchronous Multi-Stream deepsparse: ResNet-50, Baseline - Synchronous Single-Stream deepsparse: ResNet-50, Sparse INT8 - Asynchronous Multi-Stream deepsparse: ResNet-50, Sparse INT8 - Synchronous Single-Stream deepsparse: CV Detection, YOLOv5s COCO - Asynchronous Multi-Stream deepsparse: CV Detection, YOLOv5s COCO - Synchronous Single-Stream deepsparse: BERT-Large, NLP Question Answering - Asynchronous Multi-Stream deepsparse: BERT-Large, NLP Question Answering - Synchronous Single-Stream deepsparse: CV Classification, ResNet-50 ImageNet - Asynchronous Multi-Stream deepsparse: CV Classification, ResNet-50 ImageNet - Synchronous Single-Stream deepsparse: CV Detection, YOLOv5s COCO, Sparse INT8 - Asynchronous Multi-Stream deepsparse: CV Detection, YOLOv5s COCO, Sparse INT8 - Synchronous Single-Stream deepsparse: NLP Text Classification, DistilBERT mnli - Asynchronous Multi-Stream deepsparse: NLP Text Classification, DistilBERT mnli - Synchronous Single-Stream deepsparse: CV Segmentation, 90% Pruned YOLACT Pruned - Asynchronous Multi-Stream deepsparse: CV Segmentation, 90% Pruned YOLACT Pruned - Synchronous Single-Stream deepsparse: BERT-Large, NLP Question Answering, Sparse INT8 - Asynchronous Multi-Stream deepsparse: BERT-Large, NLP Question Answering, Sparse INT8 - Synchronous Single-Stream deepsparse: NLP Token Classification, BERT base uncased conll2003 - Asynchronous Multi-Stream deepsparse: NLP Token Classification, BERT base uncased conll2003 - Synchronous Single-Stream java-scimark2: Composite java-scimark2: Monte Carlo java-scimark2: Fast Fourier Transform java-scimark2: Sparse Matrix Multiply java-scimark2: Dense LU Matrix Factorization java-scimark2: Jacobi Successive Over-Relaxation webp2: Default webp2: Quality 75, Compression Effort 7 webp2: Quality 95, Compression Effort 7 webp2: Quality 100, Compression Effort 5 webp2: Quality 100, Lossless Compression openssl: RSA4096 openssl: RSA4096 deepsparse: NLP Document Classification, oBERT base uncased on IMDB - Asynchronous Multi-Stream deepsparse: NLP Document Classification, oBERT base uncased on IMDB - Synchronous Single-Stream deepsparse: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Asynchronous Multi-Stream deepsparse: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Synchronous Single-Stream deepsparse: ResNet-50, Baseline - Asynchronous Multi-Stream deepsparse: ResNet-50, Baseline - Synchronous Single-Stream deepsparse: ResNet-50, Sparse INT8 - Asynchronous Multi-Stream deepsparse: ResNet-50, Sparse INT8 - Synchronous Single-Stream deepsparse: CV Detection, YOLOv5s COCO - Asynchronous Multi-Stream deepsparse: CV Detection, YOLOv5s COCO - Synchronous Single-Stream deepsparse: BERT-Large, NLP Question Answering - Asynchronous Multi-Stream deepsparse: BERT-Large, NLP Question Answering - Synchronous Single-Stream deepsparse: CV Classification, ResNet-50 ImageNet - Asynchronous Multi-Stream deepsparse: CV Classification, ResNet-50 ImageNet - Synchronous Single-Stream deepsparse: CV Detection, YOLOv5s COCO, Sparse INT8 - Asynchronous Multi-Stream deepsparse: CV Detection, YOLOv5s COCO, Sparse INT8 - Synchronous Single-Stream deepsparse: NLP Text Classification, DistilBERT mnli - Asynchronous Multi-Stream deepsparse: NLP Text Classification, DistilBERT mnli - Synchronous Single-Stream deepsparse: CV Segmentation, 90% Pruned YOLACT Pruned - Asynchronous Multi-Stream deepsparse: CV Segmentation, 90% Pruned YOLACT Pruned - Synchronous Single-Stream deepsparse: BERT-Large, NLP Question Answering, Sparse INT8 - Asynchronous Multi-Stream deepsparse: BERT-Large, NLP Question Answering, Sparse INT8 - Synchronous Single-Stream deepsparse: NLP Token Classification, BERT base uncased conll2003 - Asynchronous Multi-Stream deepsparse: NLP Token Classification, BERT base uncased conll2003 - Synchronous Single-Stream AMD Ryzen Threadripper 3970X 32-Core b c 35.38 14.04 30.97 30.97 30.33 11.76 11.79 11.85 7.89 5.98 5.97 6.00 42506964290 14922955600 132709569060 152254146970 139151548420 86074896380 38.6634 34.8277 41.9049 37.7699 39.4881 33.989 4.712 44.813 139.049 141.072 12.238 77.466 302.256 361.391 16276.6 16368.8 27698.5 4771.3 16450.1 16317.7 29.8842 17.5286 674.1167 169.4233 323.1865 144.8781 1820.1556 766.4844 154.4159 92.4279 37.0667 21.412 325.3167 152.0352 158.0562 94.279 247.0871 90.0923 33.9103 21.7159 345.4208 85.8602 30.5082 17.7819 3628.54 1662.78 460.52 3259.85 10937.37 1822.2 9.74 0.36 0.18 7.89 0.04 9381 609998 530.7568 57.041 23.712 5.896 49.4836 6.894 8.7652 1.3008 103.4836 10.8073 431.396 46.6936 49.16 6.5694 101.1265 10.5994 64.7241 11.0925 469.2415 46.0342 46.2963 11.6407 522.4351 56.2287 36.22 14.26 31.08 30.44 29.85 11.79 11.65 11.59 7.95 5.92 5.96 6.09 42475560230 14877687680 132535967470 152243466510 138940289020 86178622410 38.6826 34.6107 41.9584 37.5554 39.3303 33.9219 4.699 44.26 140.202 143.719 12.17 77.744 303.201 358.894 16472.3 16429.8 28017.5 4833.4 16398.3 16315.6 29.5264 17.1783 671.9488 167.5851 320.683 145.6431 1805.6704 752.7752 149.1863 91.6072 37.1773 21.2394 324.6279 147.051 157.0528 94.009 245.4336 83.5111 33.5294 21.5157 344.6466 86.1225 29.9828 17.4838 3500.03 1664.07 452.84 2552.03 11007.91 1823.29 9.47 0.36 0.18 7.99 0.04 9368.1 609586.6 539.6021 58.2042 23.7852 5.9607 49.8705 6.8579 8.8357 1.324 107.1318 10.904 427.7464 47.0734 49.2645 6.7922 101.838 10.6301 65.1634 11.9675 476.8669 46.4632 46.3817 11.6045 532.7014 57.1874 35.95 14.30 31.36 31.11 30.87 11.71 11.95 11.69 7.93 6.06 6.00 6.01 42542937860 14860394950 132480014470 152248762370 139048226820 85685137680 38.788 34.7267 41.9227 37.5529 39.3408 33.9649 4.797 44.897 141.265 137.23 12.173 77.763 303.029 353.489 16425.2 16527 28103.3 4784.7 16312.4 16555.2 29.6241 17.2364 670.8731 167.2022 320.2104 142.5611 1808.3307 766.4 149.2703 91.5797 36.9162 21.2359 324.1138 150.8915 158.4441 93.7414 246.1586 83.9665 33.4894 21.284 344.6426 86.7163 29.6746 17.5075 3630.68 1603.8 447.63 3252.08 11024.98 1824.92 9.77 0.36 0.18 7.78 0.04 9368 609364.3 536.858 58.0081 23.8232 5.9747 49.9441 7.0064 8.821 1.3006 107.1516 10.9073 431.935 47.081 49.3431 6.6191 100.8112 10.6602 64.9519 11.9026 477.2306 46.9683 46.4012 11.5257 536.3063 57.1103 OpenBenchmarking.org
PyTorch Device: CPU - Batch Size: 1 - Model: ResNet-50 OpenBenchmarking.org batches/sec, More Is Better PyTorch 2.1 Device: CPU - Batch Size: 1 - Model: ResNet-50 AMD Ryzen Threadripper 3970X 32-Core b c 8 16 24 32 40 35.38 36.22 35.95 MIN: 31.54 / MAX: 36.03 MIN: 35.77 / MAX: 36.63 MIN: 32.12 / MAX: 36.39
PyTorch Device: CPU - Batch Size: 1 - Model: ResNet-152 OpenBenchmarking.org batches/sec, More Is Better PyTorch 2.1 Device: CPU - Batch Size: 1 - Model: ResNet-152 AMD Ryzen Threadripper 3970X 32-Core b c 4 8 12 16 20 14.04 14.26 14.30 MIN: 13.32 / MAX: 14.21 MIN: 13.64 / MAX: 14.35 MIN: 13.48 / MAX: 14.43
PyTorch Device: CPU - Batch Size: 16 - Model: ResNet-50 OpenBenchmarking.org batches/sec, More Is Better PyTorch 2.1 Device: CPU - Batch Size: 16 - Model: ResNet-50 AMD Ryzen Threadripper 3970X 32-Core b c 7 14 21 28 35 30.97 31.08 31.36 MIN: 30.46 / MAX: 31.35 MIN: 30.3 / MAX: 31.41 MIN: 30.9 / MAX: 31.58
PyTorch Device: CPU - Batch Size: 32 - Model: ResNet-50 OpenBenchmarking.org batches/sec, More Is Better PyTorch 2.1 Device: CPU - Batch Size: 32 - Model: ResNet-50 AMD Ryzen Threadripper 3970X 32-Core b c 7 14 21 28 35 30.97 30.44 31.11 MIN: 28.08 / MAX: 31.41 MIN: 27.31 / MAX: 30.86 MIN: 28.26 / MAX: 31.39
PyTorch Device: CPU - Batch Size: 64 - Model: ResNet-50 OpenBenchmarking.org batches/sec, More Is Better PyTorch 2.1 Device: CPU - Batch Size: 64 - Model: ResNet-50 AMD Ryzen Threadripper 3970X 32-Core b c 7 14 21 28 35 30.33 29.85 30.87 MIN: 29.76 / MAX: 30.69 MIN: 26.63 / MAX: 30.71 MIN: 30.38 / MAX: 31.05
PyTorch Device: CPU - Batch Size: 16 - Model: ResNet-152 OpenBenchmarking.org batches/sec, More Is Better PyTorch 2.1 Device: CPU - Batch Size: 16 - Model: ResNet-152 AMD Ryzen Threadripper 3970X 32-Core b c 3 6 9 12 15 11.76 11.79 11.71 MIN: 11.35 / MAX: 11.85 MIN: 11.36 / MAX: 11.88 MIN: 11.21 / MAX: 11.83
PyTorch Device: CPU - Batch Size: 32 - Model: ResNet-152 OpenBenchmarking.org batches/sec, More Is Better PyTorch 2.1 Device: CPU - Batch Size: 32 - Model: ResNet-152 AMD Ryzen Threadripper 3970X 32-Core b c 3 6 9 12 15 11.79 11.65 11.95 MIN: 11.35 / MAX: 11.86 MIN: 11.19 / MAX: 11.78 MIN: 11.46 / MAX: 12.05
PyTorch Device: CPU - Batch Size: 64 - Model: ResNet-152 OpenBenchmarking.org batches/sec, More Is Better PyTorch 2.1 Device: CPU - Batch Size: 64 - Model: ResNet-152 AMD Ryzen Threadripper 3970X 32-Core b c 3 6 9 12 15 11.85 11.59 11.69 MIN: 11.36 / MAX: 11.93 MIN: 11.15 / MAX: 11.75 MIN: 11.19 / MAX: 11.76
PyTorch Device: CPU - Batch Size: 1 - Model: Efficientnet_v2_l OpenBenchmarking.org batches/sec, More Is Better PyTorch 2.1 Device: CPU - Batch Size: 1 - Model: Efficientnet_v2_l AMD Ryzen Threadripper 3970X 32-Core b c 2 4 6 8 10 7.89 7.95 7.93 MIN: 7.7 / MAX: 7.97 MIN: 7.74 / MAX: 8.01 MIN: 7.7 / MAX: 8.02
PyTorch Device: CPU - Batch Size: 16 - Model: Efficientnet_v2_l OpenBenchmarking.org batches/sec, More Is Better PyTorch 2.1 Device: CPU - Batch Size: 16 - Model: Efficientnet_v2_l AMD Ryzen Threadripper 3970X 32-Core b c 2 4 6 8 10 5.98 5.92 6.06 MIN: 5.87 / MAX: 6.02 MIN: 5.73 / MAX: 6 MIN: 5.92 / MAX: 6.09
PyTorch Device: CPU - Batch Size: 32 - Model: Efficientnet_v2_l OpenBenchmarking.org batches/sec, More Is Better PyTorch 2.1 Device: CPU - Batch Size: 32 - Model: Efficientnet_v2_l AMD Ryzen Threadripper 3970X 32-Core b c 2 4 6 8 10 5.97 5.96 6.00 MIN: 5.85 / MAX: 6 MIN: 5.84 / MAX: 6 MIN: 5.87 / MAX: 6.05
PyTorch Device: CPU - Batch Size: 64 - Model: Efficientnet_v2_l OpenBenchmarking.org batches/sec, More Is Better PyTorch 2.1 Device: CPU - Batch Size: 64 - Model: Efficientnet_v2_l AMD Ryzen Threadripper 3970X 32-Core b c 2 4 6 8 10 6.00 6.09 6.01 MIN: 5.87 / MAX: 6.03 MIN: 5.63 / MAX: 6.21 MIN: 5.89 / MAX: 6.04
OpenSSL Algorithm: SHA256 OpenBenchmarking.org byte/s, More Is Better OpenSSL Algorithm: SHA256 AMD Ryzen Threadripper 3970X 32-Core b c 9000M 18000M 27000M 36000M 45000M 42506964290 42475560230 42542937860 1. OpenSSL 3.0.2 15 Mar 2022 (Library: OpenSSL 3.0.2 15 Mar 2022)
OpenSSL Algorithm: SHA512 OpenBenchmarking.org byte/s, More Is Better OpenSSL Algorithm: SHA512 AMD Ryzen Threadripper 3970X 32-Core b c 3000M 6000M 9000M 12000M 15000M 14922955600 14877687680 14860394950 1. OpenSSL 3.0.2 15 Mar 2022 (Library: OpenSSL 3.0.2 15 Mar 2022)
OpenSSL Algorithm: ChaCha20 OpenBenchmarking.org byte/s, More Is Better OpenSSL Algorithm: ChaCha20 AMD Ryzen Threadripper 3970X 32-Core b c 30000M 60000M 90000M 120000M 150000M 132709569060 132535967470 132480014470 1. OpenSSL 3.0.2 15 Mar 2022 (Library: OpenSSL 3.0.2 15 Mar 2022)
OpenSSL Algorithm: AES-128-GCM OpenBenchmarking.org byte/s, More Is Better OpenSSL Algorithm: AES-128-GCM AMD Ryzen Threadripper 3970X 32-Core b c 30000M 60000M 90000M 120000M 150000M 152254146970 152243466510 152248762370 1. OpenSSL 3.0.2 15 Mar 2022 (Library: OpenSSL 3.0.2 15 Mar 2022)
OpenSSL Algorithm: AES-256-GCM OpenBenchmarking.org byte/s, More Is Better OpenSSL Algorithm: AES-256-GCM AMD Ryzen Threadripper 3970X 32-Core b c 30000M 60000M 90000M 120000M 150000M 139151548420 138940289020 139048226820 1. OpenSSL 3.0.2 15 Mar 2022 (Library: OpenSSL 3.0.2 15 Mar 2022)
OpenSSL Algorithm: ChaCha20-Poly1305 OpenBenchmarking.org byte/s, More Is Better OpenSSL Algorithm: ChaCha20-Poly1305 AMD Ryzen Threadripper 3970X 32-Core b c 20000M 40000M 60000M 80000M 100000M 86074896380 86178622410 85685137680 1. OpenSSL 3.0.2 15 Mar 2022 (Library: OpenSSL 3.0.2 15 Mar 2022)
Embree Binary: Pathtracer - Model: Crown OpenBenchmarking.org Frames Per Second, More Is Better Embree 4.3 Binary: Pathtracer - Model: Crown AMD Ryzen Threadripper 3970X 32-Core b c 9 18 27 36 45 38.66 38.68 38.79 MIN: 38.32 / MAX: 39.41 MIN: 38.31 / MAX: 39.27 MIN: 38.44 / MAX: 39.41
Embree Binary: Pathtracer ISPC - Model: Crown OpenBenchmarking.org Frames Per Second, More Is Better Embree 4.3 Binary: Pathtracer ISPC - Model: Crown AMD Ryzen Threadripper 3970X 32-Core b c 8 16 24 32 40 34.83 34.61 34.73 MIN: 34.51 / MAX: 35.43 MIN: 34.23 / MAX: 35.36 MIN: 34.36 / MAX: 35.34
Embree Binary: Pathtracer - Model: Asian Dragon OpenBenchmarking.org Frames Per Second, More Is Better Embree 4.3 Binary: Pathtracer - Model: Asian Dragon AMD Ryzen Threadripper 3970X 32-Core b c 10 20 30 40 50 41.90 41.96 41.92 MIN: 41.66 / MAX: 42.52 MIN: 41.73 / MAX: 42.57 MIN: 41.68 / MAX: 42.7
Embree Binary: Pathtracer - Model: Asian Dragon Obj OpenBenchmarking.org Frames Per Second, More Is Better Embree 4.3 Binary: Pathtracer - Model: Asian Dragon Obj AMD Ryzen Threadripper 3970X 32-Core b c 9 18 27 36 45 37.77 37.56 37.55 MIN: 37.52 / MAX: 38.28 MIN: 37.33 / MAX: 38.01 MIN: 37.34 / MAX: 38.21
Embree Binary: Pathtracer ISPC - Model: Asian Dragon OpenBenchmarking.org Frames Per Second, More Is Better Embree 4.3 Binary: Pathtracer ISPC - Model: Asian Dragon AMD Ryzen Threadripper 3970X 32-Core b c 9 18 27 36 45 39.49 39.33 39.34 MIN: 39.27 / MAX: 39.89 MIN: 39.12 / MAX: 39.81 MIN: 39.13 / MAX: 39.88
Embree Binary: Pathtracer ISPC - Model: Asian Dragon Obj OpenBenchmarking.org Frames Per Second, More Is Better Embree 4.3 Binary: Pathtracer ISPC - Model: Asian Dragon Obj AMD Ryzen Threadripper 3970X 32-Core b c 8 16 24 32 40 33.99 33.92 33.96 MIN: 33.79 / MAX: 34.43 MIN: 33.73 / MAX: 34.41 MIN: 33.77 / MAX: 34.36
SVT-AV1 Encoder Mode: Preset 4 - Input: Bosphorus 4K OpenBenchmarking.org Frames Per Second, More Is Better SVT-AV1 1.8 Encoder Mode: Preset 4 - Input: Bosphorus 4K AMD Ryzen Threadripper 3970X 32-Core b c 1.0793 2.1586 3.2379 4.3172 5.3965 4.712 4.699 4.797 1. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq
SVT-AV1 Encoder Mode: Preset 8 - Input: Bosphorus 4K OpenBenchmarking.org Frames Per Second, More Is Better SVT-AV1 1.8 Encoder Mode: Preset 8 - Input: Bosphorus 4K AMD Ryzen Threadripper 3970X 32-Core b c 10 20 30 40 50 44.81 44.26 44.90 1. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq
SVT-AV1 Encoder Mode: Preset 12 - Input: Bosphorus 4K OpenBenchmarking.org Frames Per Second, More Is Better SVT-AV1 1.8 Encoder Mode: Preset 12 - Input: Bosphorus 4K AMD Ryzen Threadripper 3970X 32-Core b c 30 60 90 120 150 139.05 140.20 141.27 1. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq
SVT-AV1 Encoder Mode: Preset 13 - Input: Bosphorus 4K OpenBenchmarking.org Frames Per Second, More Is Better SVT-AV1 1.8 Encoder Mode: Preset 13 - Input: Bosphorus 4K AMD Ryzen Threadripper 3970X 32-Core b c 30 60 90 120 150 141.07 143.72 137.23 1. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq
SVT-AV1 Encoder Mode: Preset 4 - Input: Bosphorus 1080p OpenBenchmarking.org Frames Per Second, More Is Better SVT-AV1 1.8 Encoder Mode: Preset 4 - Input: Bosphorus 1080p AMD Ryzen Threadripper 3970X 32-Core b c 3 6 9 12 15 12.24 12.17 12.17 1. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq
SVT-AV1 Encoder Mode: Preset 8 - Input: Bosphorus 1080p OpenBenchmarking.org Frames Per Second, More Is Better SVT-AV1 1.8 Encoder Mode: Preset 8 - Input: Bosphorus 1080p AMD Ryzen Threadripper 3970X 32-Core b c 20 40 60 80 100 77.47 77.74 77.76 1. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq
SVT-AV1 Encoder Mode: Preset 12 - Input: Bosphorus 1080p OpenBenchmarking.org Frames Per Second, More Is Better SVT-AV1 1.8 Encoder Mode: Preset 12 - Input: Bosphorus 1080p AMD Ryzen Threadripper 3970X 32-Core b c 70 140 210 280 350 302.26 303.20 303.03 1. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq
SVT-AV1 Encoder Mode: Preset 13 - Input: Bosphorus 1080p OpenBenchmarking.org Frames Per Second, More Is Better SVT-AV1 1.8 Encoder Mode: Preset 13 - Input: Bosphorus 1080p AMD Ryzen Threadripper 3970X 32-Core b c 80 160 240 320 400 361.39 358.89 353.49 1. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq
Xmrig Variant: KawPow - Hash Count: 1M OpenBenchmarking.org H/s, More Is Better Xmrig 6.21 Variant: KawPow - Hash Count: 1M AMD Ryzen Threadripper 3970X 32-Core b c 4K 8K 12K 16K 20K 16276.6 16472.3 16425.2 1. (CXX) g++ options: -fexceptions -fno-rtti -maes -O3 -Ofast -static-libgcc -static-libstdc++ -rdynamic -lssl -lcrypto -luv -lpthread -lrt -ldl -lhwloc
Xmrig Variant: Monero - Hash Count: 1M OpenBenchmarking.org H/s, More Is Better Xmrig 6.21 Variant: Monero - Hash Count: 1M AMD Ryzen Threadripper 3970X 32-Core b c 4K 8K 12K 16K 20K 16368.8 16429.8 16527.0 1. (CXX) g++ options: -fexceptions -fno-rtti -maes -O3 -Ofast -static-libgcc -static-libstdc++ -rdynamic -lssl -lcrypto -luv -lpthread -lrt -ldl -lhwloc
Xmrig Variant: Wownero - Hash Count: 1M OpenBenchmarking.org H/s, More Is Better Xmrig 6.21 Variant: Wownero - Hash Count: 1M AMD Ryzen Threadripper 3970X 32-Core b c 6K 12K 18K 24K 30K 27698.5 28017.5 28103.3 1. (CXX) g++ options: -fexceptions -fno-rtti -maes -O3 -Ofast -static-libgcc -static-libstdc++ -rdynamic -lssl -lcrypto -luv -lpthread -lrt -ldl -lhwloc
Xmrig Variant: GhostRider - Hash Count: 1M OpenBenchmarking.org H/s, More Is Better Xmrig 6.21 Variant: GhostRider - Hash Count: 1M AMD Ryzen Threadripper 3970X 32-Core b c 1000 2000 3000 4000 5000 4771.3 4833.4 4784.7 1. (CXX) g++ options: -fexceptions -fno-rtti -maes -O3 -Ofast -static-libgcc -static-libstdc++ -rdynamic -lssl -lcrypto -luv -lpthread -lrt -ldl -lhwloc
Xmrig Variant: CryptoNight-Heavy - Hash Count: 1M OpenBenchmarking.org H/s, More Is Better Xmrig 6.21 Variant: CryptoNight-Heavy - Hash Count: 1M AMD Ryzen Threadripper 3970X 32-Core b c 4K 8K 12K 16K 20K 16450.1 16398.3 16312.4 1. (CXX) g++ options: -fexceptions -fno-rtti -maes -O3 -Ofast -static-libgcc -static-libstdc++ -rdynamic -lssl -lcrypto -luv -lpthread -lrt -ldl -lhwloc
Xmrig Variant: CryptoNight-Femto UPX2 - Hash Count: 1M OpenBenchmarking.org H/s, More Is Better Xmrig 6.21 Variant: CryptoNight-Femto UPX2 - Hash Count: 1M AMD Ryzen Threadripper 3970X 32-Core b c 4K 8K 12K 16K 20K 16317.7 16315.6 16555.2 1. (CXX) g++ options: -fexceptions -fno-rtti -maes -O3 -Ofast -static-libgcc -static-libstdc++ -rdynamic -lssl -lcrypto -luv -lpthread -lrt -ldl -lhwloc
Neural Magic DeepSparse Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.6 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream AMD Ryzen Threadripper 3970X 32-Core b c 7 14 21 28 35 29.88 29.53 29.62
Neural Magic DeepSparse Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.6 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Stream AMD Ryzen Threadripper 3970X 32-Core b c 4 8 12 16 20 17.53 17.18 17.24
Neural Magic DeepSparse Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.6 Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Asynchronous Multi-Stream AMD Ryzen Threadripper 3970X 32-Core b c 150 300 450 600 750 674.12 671.95 670.87
Neural Magic DeepSparse Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Synchronous Single-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.6 Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Synchronous Single-Stream AMD Ryzen Threadripper 3970X 32-Core b c 40 80 120 160 200 169.42 167.59 167.20
Neural Magic DeepSparse Model: ResNet-50, Baseline - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.6 Model: ResNet-50, Baseline - Scenario: Asynchronous Multi-Stream AMD Ryzen Threadripper 3970X 32-Core b c 70 140 210 280 350 323.19 320.68 320.21
Neural Magic DeepSparse Model: ResNet-50, Baseline - Scenario: Synchronous Single-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.6 Model: ResNet-50, Baseline - Scenario: Synchronous Single-Stream AMD Ryzen Threadripper 3970X 32-Core b c 30 60 90 120 150 144.88 145.64 142.56
Neural Magic DeepSparse Model: ResNet-50, Sparse INT8 - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.6 Model: ResNet-50, Sparse INT8 - Scenario: Asynchronous Multi-Stream AMD Ryzen Threadripper 3970X 32-Core b c 400 800 1200 1600 2000 1820.16 1805.67 1808.33
Neural Magic DeepSparse Model: ResNet-50, Sparse INT8 - Scenario: Synchronous Single-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.6 Model: ResNet-50, Sparse INT8 - Scenario: Synchronous Single-Stream AMD Ryzen Threadripper 3970X 32-Core b c 170 340 510 680 850 766.48 752.78 766.40
Neural Magic DeepSparse Model: CV Detection, YOLOv5s COCO - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.6 Model: CV Detection, YOLOv5s COCO - Scenario: Asynchronous Multi-Stream AMD Ryzen Threadripper 3970X 32-Core b c 30 60 90 120 150 154.42 149.19 149.27
Neural Magic DeepSparse Model: CV Detection, YOLOv5s COCO - Scenario: Synchronous Single-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.6 Model: CV Detection, YOLOv5s COCO - Scenario: Synchronous Single-Stream AMD Ryzen Threadripper 3970X 32-Core b c 20 40 60 80 100 92.43 91.61 91.58
Neural Magic DeepSparse Model: BERT-Large, NLP Question Answering - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.6 Model: BERT-Large, NLP Question Answering - Scenario: Asynchronous Multi-Stream AMD Ryzen Threadripper 3970X 32-Core b c 9 18 27 36 45 37.07 37.18 36.92
Neural Magic DeepSparse Model: BERT-Large, NLP Question Answering - Scenario: Synchronous Single-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.6 Model: BERT-Large, NLP Question Answering - Scenario: Synchronous Single-Stream AMD Ryzen Threadripper 3970X 32-Core b c 5 10 15 20 25 21.41 21.24 21.24
Neural Magic DeepSparse Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.6 Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream AMD Ryzen Threadripper 3970X 32-Core b c 70 140 210 280 350 325.32 324.63 324.11
Neural Magic DeepSparse Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.6 Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Stream AMD Ryzen Threadripper 3970X 32-Core b c 30 60 90 120 150 152.04 147.05 150.89
Neural Magic DeepSparse Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.6 Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Asynchronous Multi-Stream AMD Ryzen Threadripper 3970X 32-Core b c 40 80 120 160 200 158.06 157.05 158.44
Neural Magic DeepSparse Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Synchronous Single-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.6 Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Synchronous Single-Stream AMD Ryzen Threadripper 3970X 32-Core b c 20 40 60 80 100 94.28 94.01 93.74
Neural Magic DeepSparse Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.6 Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Stream AMD Ryzen Threadripper 3970X 32-Core b c 50 100 150 200 250 247.09 245.43 246.16
Neural Magic DeepSparse Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.6 Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Stream AMD Ryzen Threadripper 3970X 32-Core b c 20 40 60 80 100 90.09 83.51 83.97
Neural Magic DeepSparse Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.6 Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Stream AMD Ryzen Threadripper 3970X 32-Core b c 8 16 24 32 40 33.91 33.53 33.49
Neural Magic DeepSparse Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Synchronous Single-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.6 Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Synchronous Single-Stream AMD Ryzen Threadripper 3970X 32-Core b c 5 10 15 20 25 21.72 21.52 21.28
Neural Magic DeepSparse Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.6 Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Asynchronous Multi-Stream AMD Ryzen Threadripper 3970X 32-Core b c 80 160 240 320 400 345.42 344.65 344.64
Neural Magic DeepSparse Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Synchronous Single-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.6 Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Synchronous Single-Stream AMD Ryzen Threadripper 3970X 32-Core b c 20 40 60 80 100 85.86 86.12 86.72
Neural Magic DeepSparse Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.6 Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream AMD Ryzen Threadripper 3970X 32-Core b c 7 14 21 28 35 30.51 29.98 29.67
Neural Magic DeepSparse Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.6 Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Stream AMD Ryzen Threadripper 3970X 32-Core b c 4 8 12 16 20 17.78 17.48 17.51
Java SciMark Computational Test: Composite OpenBenchmarking.org Mflops, More Is Better Java SciMark 2.2 Computational Test: Composite AMD Ryzen Threadripper 3970X 32-Core b c 800 1600 2400 3200 4000 3628.54 3500.03 3630.68
Java SciMark Computational Test: Monte Carlo OpenBenchmarking.org Mflops, More Is Better Java SciMark 2.2 Computational Test: Monte Carlo AMD Ryzen Threadripper 3970X 32-Core b c 400 800 1200 1600 2000 1662.78 1664.07 1603.80
Java SciMark Computational Test: Fast Fourier Transform OpenBenchmarking.org Mflops, More Is Better Java SciMark 2.2 Computational Test: Fast Fourier Transform AMD Ryzen Threadripper 3970X 32-Core b c 100 200 300 400 500 460.52 452.84 447.63
Java SciMark Computational Test: Sparse Matrix Multiply OpenBenchmarking.org Mflops, More Is Better Java SciMark 2.2 Computational Test: Sparse Matrix Multiply AMD Ryzen Threadripper 3970X 32-Core b c 700 1400 2100 2800 3500 3259.85 2552.03 3252.08
Java SciMark Computational Test: Dense LU Matrix Factorization OpenBenchmarking.org Mflops, More Is Better Java SciMark 2.2 Computational Test: Dense LU Matrix Factorization AMD Ryzen Threadripper 3970X 32-Core b c 2K 4K 6K 8K 10K 10937.37 11007.91 11024.98
Java SciMark Computational Test: Jacobi Successive Over-Relaxation OpenBenchmarking.org Mflops, More Is Better Java SciMark 2.2 Computational Test: Jacobi Successive Over-Relaxation AMD Ryzen Threadripper 3970X 32-Core b c 400 800 1200 1600 2000 1822.20 1823.29 1824.92
WebP2 Image Encode Encode Settings: Default OpenBenchmarking.org MP/s, More Is Better WebP2 Image Encode 20220823 Encode Settings: Default AMD Ryzen Threadripper 3970X 32-Core b c 3 6 9 12 15 9.74 9.47 9.77 1. (CXX) g++ options: -msse4.2 -fno-rtti -O3 -ldl
WebP2 Image Encode Encode Settings: Quality 75, Compression Effort 7 OpenBenchmarking.org MP/s, More Is Better WebP2 Image Encode 20220823 Encode Settings: Quality 75, Compression Effort 7 AMD Ryzen Threadripper 3970X 32-Core b c 0.081 0.162 0.243 0.324 0.405 0.36 0.36 0.36 1. (CXX) g++ options: -msse4.2 -fno-rtti -O3 -ldl
WebP2 Image Encode Encode Settings: Quality 95, Compression Effort 7 OpenBenchmarking.org MP/s, More Is Better WebP2 Image Encode 20220823 Encode Settings: Quality 95, Compression Effort 7 AMD Ryzen Threadripper 3970X 32-Core b c 0.0405 0.081 0.1215 0.162 0.2025 0.18 0.18 0.18 1. (CXX) g++ options: -msse4.2 -fno-rtti -O3 -ldl
WebP2 Image Encode Encode Settings: Quality 100, Compression Effort 5 OpenBenchmarking.org MP/s, More Is Better WebP2 Image Encode 20220823 Encode Settings: Quality 100, Compression Effort 5 AMD Ryzen Threadripper 3970X 32-Core b c 2 4 6 8 10 7.89 7.99 7.78 1. (CXX) g++ options: -msse4.2 -fno-rtti -O3 -ldl
WebP2 Image Encode Encode Settings: Quality 100, Lossless Compression OpenBenchmarking.org MP/s, More Is Better WebP2 Image Encode 20220823 Encode Settings: Quality 100, Lossless Compression AMD Ryzen Threadripper 3970X 32-Core b c 0.009 0.018 0.027 0.036 0.045 0.04 0.04 0.04 1. (CXX) g++ options: -msse4.2 -fno-rtti -O3 -ldl
OpenSSL Algorithm: RSA4096 OpenBenchmarking.org sign/s, More Is Better OpenSSL Algorithm: RSA4096 AMD Ryzen Threadripper 3970X 32-Core b c 2K 4K 6K 8K 10K 9381.0 9368.1 9368.0 1. OpenSSL 3.0.2 15 Mar 2022 (Library: OpenSSL 3.0.2 15 Mar 2022)
OpenSSL Algorithm: RSA4096 OpenBenchmarking.org verify/s, More Is Better OpenSSL Algorithm: RSA4096 AMD Ryzen Threadripper 3970X 32-Core b c 130K 260K 390K 520K 650K 609998.0 609586.6 609364.3 1. OpenSSL 3.0.2 15 Mar 2022 (Library: OpenSSL 3.0.2 15 Mar 2022)
Neural Magic DeepSparse Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.6 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream AMD Ryzen Threadripper 3970X 32-Core b c 120 240 360 480 600 530.76 539.60 536.86
Neural Magic DeepSparse Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.6 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Stream AMD Ryzen Threadripper 3970X 32-Core b c 13 26 39 52 65 57.04 58.20 58.01
Neural Magic DeepSparse Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.6 Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Asynchronous Multi-Stream AMD Ryzen Threadripper 3970X 32-Core b c 6 12 18 24 30 23.71 23.79 23.82
Neural Magic DeepSparse Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Synchronous Single-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.6 Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Synchronous Single-Stream AMD Ryzen Threadripper 3970X 32-Core b c 1.3443 2.6886 4.0329 5.3772 6.7215 5.8960 5.9607 5.9747
Neural Magic DeepSparse Model: ResNet-50, Baseline - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.6 Model: ResNet-50, Baseline - Scenario: Asynchronous Multi-Stream AMD Ryzen Threadripper 3970X 32-Core b c 11 22 33 44 55 49.48 49.87 49.94
Neural Magic DeepSparse Model: ResNet-50, Baseline - Scenario: Synchronous Single-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.6 Model: ResNet-50, Baseline - Scenario: Synchronous Single-Stream AMD Ryzen Threadripper 3970X 32-Core b c 2 4 6 8 10 6.8940 6.8579 7.0064
Neural Magic DeepSparse Model: ResNet-50, Sparse INT8 - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.6 Model: ResNet-50, Sparse INT8 - Scenario: Asynchronous Multi-Stream AMD Ryzen Threadripper 3970X 32-Core b c 2 4 6 8 10 8.7652 8.8357 8.8210
Neural Magic DeepSparse Model: ResNet-50, Sparse INT8 - Scenario: Synchronous Single-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.6 Model: ResNet-50, Sparse INT8 - Scenario: Synchronous Single-Stream AMD Ryzen Threadripper 3970X 32-Core b c 0.2979 0.5958 0.8937 1.1916 1.4895 1.3008 1.3240 1.3006
Neural Magic DeepSparse Model: CV Detection, YOLOv5s COCO - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.6 Model: CV Detection, YOLOv5s COCO - Scenario: Asynchronous Multi-Stream AMD Ryzen Threadripper 3970X 32-Core b c 20 40 60 80 100 103.48 107.13 107.15
Neural Magic DeepSparse Model: CV Detection, YOLOv5s COCO - Scenario: Synchronous Single-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.6 Model: CV Detection, YOLOv5s COCO - Scenario: Synchronous Single-Stream AMD Ryzen Threadripper 3970X 32-Core b c 3 6 9 12 15 10.81 10.90 10.91
Neural Magic DeepSparse Model: BERT-Large, NLP Question Answering - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.6 Model: BERT-Large, NLP Question Answering - Scenario: Asynchronous Multi-Stream AMD Ryzen Threadripper 3970X 32-Core b c 90 180 270 360 450 431.40 427.75 431.94
Neural Magic DeepSparse Model: BERT-Large, NLP Question Answering - Scenario: Synchronous Single-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.6 Model: BERT-Large, NLP Question Answering - Scenario: Synchronous Single-Stream AMD Ryzen Threadripper 3970X 32-Core b c 11 22 33 44 55 46.69 47.07 47.08
Neural Magic DeepSparse Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.6 Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream AMD Ryzen Threadripper 3970X 32-Core b c 11 22 33 44 55 49.16 49.26 49.34
Neural Magic DeepSparse Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.6 Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Stream AMD Ryzen Threadripper 3970X 32-Core b c 2 4 6 8 10 6.5694 6.7922 6.6191
Neural Magic DeepSparse Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.6 Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Asynchronous Multi-Stream AMD Ryzen Threadripper 3970X 32-Core b c 20 40 60 80 100 101.13 101.84 100.81
Neural Magic DeepSparse Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Synchronous Single-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.6 Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Synchronous Single-Stream AMD Ryzen Threadripper 3970X 32-Core b c 3 6 9 12 15 10.60 10.63 10.66
Neural Magic DeepSparse Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.6 Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Stream AMD Ryzen Threadripper 3970X 32-Core b c 15 30 45 60 75 64.72 65.16 64.95
Neural Magic DeepSparse Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.6 Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Stream AMD Ryzen Threadripper 3970X 32-Core b c 3 6 9 12 15 11.09 11.97 11.90
Neural Magic DeepSparse Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.6 Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Stream AMD Ryzen Threadripper 3970X 32-Core b c 100 200 300 400 500 469.24 476.87 477.23
Neural Magic DeepSparse Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Synchronous Single-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.6 Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Synchronous Single-Stream AMD Ryzen Threadripper 3970X 32-Core b c 11 22 33 44 55 46.03 46.46 46.97
Neural Magic DeepSparse Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.6 Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Asynchronous Multi-Stream AMD Ryzen Threadripper 3970X 32-Core b c 11 22 33 44 55 46.30 46.38 46.40
Neural Magic DeepSparse Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Synchronous Single-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.6 Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Synchronous Single-Stream AMD Ryzen Threadripper 3970X 32-Core b c 3 6 9 12 15 11.64 11.60 11.53
Neural Magic DeepSparse Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.6 Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream AMD Ryzen Threadripper 3970X 32-Core b c 120 240 360 480 600 522.44 532.70 536.31
Neural Magic DeepSparse Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.6 Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Stream AMD Ryzen Threadripper 3970X 32-Core b c 13 26 39 52 65 56.23 57.19 57.11
Phoronix Test Suite v10.8.5