litert xnnpack AMD Ryzen Threadripper 3990X 64-Core testing with a Gigabyte TRX40 AORUS PRO WIFI (F6 BIOS) and AMD Radeon RX 5700 8GB on Pop 22.04 via the Phoronix Test Suite.
HTML result view exported from: https://openbenchmarking.org/result/2410157-PTS-LITERTXN77&grw&sor .
litert xnnpack Processor Motherboard Chipset Memory Disk Graphics Audio Monitor Network OS Kernel Desktop Display Server OpenGL Vulkan Compiler File-System Screen Resolution a b d AMD Ryzen Threadripper 3990X 64-Core @ 2.90GHz (64 Cores / 128 Threads) Gigabyte TRX40 AORUS PRO WIFI (F6 BIOS) AMD Starship/Matisse 4 x 32GB DDR4-3000MT/s CMK64GX4M2D3000C16 Samsung SSD 970 EVO Plus 500GB AMD Radeon RX 5700 8GB AMD Navi 10 HDMI Audio DELL P2415Q Intel I211 + Intel Wi-Fi 6 AX200 Pop 22.04 6.8.0-76060800daily20240311-generic (x86_64) GNOME Shell 42.5 X Server 1.21.1.4 4.6 Mesa 24.0.3-1pop1~1711635559~22.04~7a9f319 (LLVM 15.0.7 DRM 3.57) 1.3.274 GCC 11.4.0 ext4 1920x1080 OpenBenchmarking.org Kernel Details - Transparent Huge Pages: madvise Compiler Details - --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-link-serialization=2 --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-targets=nvptx-none=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-build-config=bootstrap-lto-lean --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v Processor Details - Scaling Governor: acpi-cpufreq schedutil (Boost: Enabled) - CPU Microcode: 0x830107a Security Details - gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Mitigation of untrained return thunk; SMT enabled with STIBP protection + spec_rstack_overflow: Mitigation of Safe RET + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional STIBP: always-on RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected
litert xnnpack xnnpack: FP32MobileNetV2 xnnpack: FP32MobileNetV3Large xnnpack: FP32MobileNetV3Small xnnpack: FP16MobileNetV1 xnnpack: FP16MobileNetV2 xnnpack: FP16MobileNetV3Large litert: Quantized COCO SSD MobileNet v1 litert: SqueezeNet litert: Mobilenet Quant litert: Inception ResNet V2 litert: Mobilenet Float litert: NASNet Mobile litert: Inception V4 xnnpack: FP32MobileNetV1 xnnpack: FP16MobileNetV3Small xnnpack: QS8MobileNetV2 litert: DeepLab V3 a b d 6964 9298 5398 3621 5908 9055 5332.12 6423.64 3191.88 59856.0 4243.70 54458.3 56528.6 4280 6357 5696 10506.6 6934 9174 5543 3649 5983 7559 5384.80 6349.14 3089.77 58681.4 4187.56 54646.9 57577.7 4238 5842 5691 10343.9 6889 11231 7369 3825 6324 7944 5207.07 6274.44 3084.06 59960.4 4173.74 53315.5 56815.6 4243 6057 6038 10484.8 OpenBenchmarking.org
XNNPACK Model: FP32MobileNetV2 OpenBenchmarking.org us, Fewer Is Better XNNPACK b7b048 Model: FP32MobileNetV2 d b a 1500 3000 4500 6000 7500 SE +/- 15.63, N = 3 SE +/- 41.70, N = 3 SE +/- 15.96, N = 4 6889 6934 6964 1. (CXX) g++ options: -O3 -lrt -lm
XNNPACK Model: FP32MobileNetV3Large OpenBenchmarking.org us, Fewer Is Better XNNPACK b7b048 Model: FP32MobileNetV3Large b a d 2K 4K 6K 8K 10K SE +/- 54.24, N = 3 SE +/- 69.99, N = 4 SE +/- 1046.96, N = 3 9174 9298 11231 1. (CXX) g++ options: -O3 -lrt -lm
XNNPACK Model: FP32MobileNetV3Small OpenBenchmarking.org us, Fewer Is Better XNNPACK b7b048 Model: FP32MobileNetV3Small a b d 1600 3200 4800 6400 8000 SE +/- 47.52, N = 4 SE +/- 121.75, N = 3 SE +/- 1111.98, N = 3 5398 5543 7369 1. (CXX) g++ options: -O3 -lrt -lm
XNNPACK Model: FP16MobileNetV1 OpenBenchmarking.org us, Fewer Is Better XNNPACK b7b048 Model: FP16MobileNetV1 a b d 800 1600 2400 3200 4000 SE +/- 16.80, N = 4 SE +/- 11.67, N = 3 SE +/- 169.19, N = 3 3621 3649 3825 1. (CXX) g++ options: -O3 -lrt -lm
XNNPACK Model: FP16MobileNetV2 OpenBenchmarking.org us, Fewer Is Better XNNPACK b7b048 Model: FP16MobileNetV2 a b d 1400 2800 4200 5600 7000 SE +/- 39.38, N = 4 SE +/- 27.67, N = 3 SE +/- 312.09, N = 3 5908 5983 6324 1. (CXX) g++ options: -O3 -lrt -lm
XNNPACK Model: FP16MobileNetV3Large OpenBenchmarking.org us, Fewer Is Better XNNPACK b7b048 Model: FP16MobileNetV3Large b d a 2K 4K 6K 8K 10K SE +/- 47.34, N = 3 SE +/- 426.05, N = 3 SE +/- 1200.15, N = 4 7559 7944 9055 1. (CXX) g++ options: -O3 -lrt -lm
LiteRT Model: Quantized COCO SSD MobileNet v1 OpenBenchmarking.org Microseconds, Fewer Is Better LiteRT 2024-10-15 Model: Quantized COCO SSD MobileNet v1 d a b 1200 2400 3600 4800 6000 SE +/- 72.45, N = 3 SE +/- 47.15, N = 15 SE +/- 72.48, N = 3 5207.07 5332.12 5384.80
LiteRT Model: SqueezeNet OpenBenchmarking.org Microseconds, Fewer Is Better LiteRT 2024-10-15 Model: SqueezeNet d b a 1400 2800 4200 5600 7000 SE +/- 24.67, N = 3 SE +/- 97.62, N = 15 SE +/- 148.43, N = 15 6274.44 6349.14 6423.64
LiteRT Model: Mobilenet Quant OpenBenchmarking.org Microseconds, Fewer Is Better LiteRT 2024-10-15 Model: Mobilenet Quant d b a 700 1400 2100 2800 3500 SE +/- 99.43, N = 12 SE +/- 81.43, N = 12 SE +/- 108.69, N = 15 3084.06 3089.77 3191.88
LiteRT Model: Inception ResNet V2 OpenBenchmarking.org Microseconds, Fewer Is Better LiteRT 2024-10-15 Model: Inception ResNet V2 b a d 13K 26K 39K 52K 65K SE +/- 317.66, N = 3 SE +/- 665.58, N = 3 SE +/- 319.38, N = 3 58681.4 59856.0 59960.4
LiteRT Model: Mobilenet Float OpenBenchmarking.org Microseconds, Fewer Is Better LiteRT 2024-10-15 Model: Mobilenet Float d b a 900 1800 2700 3600 4500 SE +/- 6.16, N = 3 SE +/- 24.52, N = 3 SE +/- 22.62, N = 3 4173.74 4187.56 4243.70
LiteRT Model: NASNet Mobile OpenBenchmarking.org Microseconds, Fewer Is Better LiteRT 2024-10-15 Model: NASNet Mobile d a b 12K 24K 36K 48K 60K SE +/- 390.67, N = 3 SE +/- 347.90, N = 3 SE +/- 560.60, N = 3 53315.5 54458.3 54646.9
LiteRT Model: Inception V4 OpenBenchmarking.org Microseconds, Fewer Is Better LiteRT 2024-10-15 Model: Inception V4 a d b 12K 24K 36K 48K 60K SE +/- 639.55, N = 3 SE +/- 464.00, N = 3 SE +/- 670.44, N = 15 56528.6 56815.6 57577.7
XNNPACK Model: FP32MobileNetV1 OpenBenchmarking.org us, Fewer Is Better XNNPACK b7b048 Model: FP32MobileNetV1 b d a 900 1800 2700 3600 4500 SE +/- 34.33, N = 3 SE +/- 41.77, N = 3 SE +/- 46.30, N = 4 4238 4243 4280 1. (CXX) g++ options: -O3 -lrt -lm
XNNPACK Model: FP16MobileNetV3Small OpenBenchmarking.org us, Fewer Is Better XNNPACK b7b048 Model: FP16MobileNetV3Small b d a 1400 2800 4200 5600 7000 SE +/- 43.47, N = 3 SE +/- 252.57, N = 3 SE +/- 506.60, N = 4 5842 6057 6357 1. (CXX) g++ options: -O3 -lrt -lm
XNNPACK Model: QS8MobileNetV2 OpenBenchmarking.org us, Fewer Is Better XNNPACK b7b048 Model: QS8MobileNetV2 b a d 1300 2600 3900 5200 6500 SE +/- 27.75, N = 3 SE +/- 22.24, N = 4 SE +/- 292.13, N = 3 5691 5696 6038 1. (CXX) g++ options: -O3 -lrt -lm
LiteRT Model: DeepLab V3 OpenBenchmarking.org Microseconds, Fewer Is Better LiteRT 2024-10-15 Model: DeepLab V3 b d a 2K 4K 6K 8K 10K SE +/- 84.63, N = 3 SE +/- 84.63, N = 3 SE +/- 163.08, N = 15 10343.9 10484.8 10506.6
Phoronix Test Suite v10.8.5