litert xnnpack

AMD Ryzen Threadripper 3990X 64-Core testing with a Gigabyte TRX40 AORUS PRO WIFI (F6 BIOS) and AMD Radeon RX 5700 8GB on Pop 22.04 via the Phoronix Test Suite.

HTML result view exported from: https://openbenchmarking.org/result/2410157-PTS-LITERTXN77&grs&sro.

litert xnnpackProcessorMotherboardChipsetMemoryDiskGraphicsAudioMonitorNetworkOSKernelDesktopDisplay ServerOpenGLVulkanCompilerFile-SystemScreen ResolutionabdAMD Ryzen Threadripper 3990X 64-Core @ 2.90GHz (64 Cores / 128 Threads)Gigabyte TRX40 AORUS PRO WIFI (F6 BIOS)AMD Starship/Matisse4 x 32GB DDR4-3000MT/s CMK64GX4M2D3000C16Samsung SSD 970 EVO Plus 500GBAMD Radeon RX 5700 8GBAMD Navi 10 HDMI AudioDELL P2415QIntel I211 + Intel Wi-Fi 6 AX200Pop 22.046.8.0-76060800daily20240311-generic (x86_64)GNOME Shell 42.5X Server 1.21.1.44.6 Mesa 24.0.3-1pop1~1711635559~22.04~7a9f319 (LLVM 15.0.7 DRM 3.57)1.3.274GCC 11.4.0ext41920x1080OpenBenchmarking.orgKernel Details- Transparent Huge Pages: madviseCompiler Details- --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-link-serialization=2 --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-targets=nvptx-none=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-build-config=bootstrap-lto-lean --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v Processor Details- Scaling Governor: acpi-cpufreq schedutil (Boost: Enabled) - CPU Microcode: 0x830107aSecurity Details- gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Mitigation of untrained return thunk; SMT enabled with STIBP protection + spec_rstack_overflow: Mitigation of Safe RET + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional STIBP: always-on RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected

litert xnnpacklitert: Quantized COCO SSD MobileNet v1litert: NASNet Mobilelitert: Inception ResNet V2litert: Inception V4litert: Mobilenet Floatxnnpack: FP32MobileNetV2xnnpack: FP32MobileNetV1xnnpack: QS8MobileNetV2xnnpack: FP16MobileNetV3Smallxnnpack: FP16MobileNetV3Largexnnpack: FP16MobileNetV2xnnpack: FP16MobileNetV1xnnpack: FP32MobileNetV3Smallxnnpack: FP32MobileNetV3Largelitert: Mobilenet Quantlitert: SqueezeNetlitert: DeepLab V3abd5332.1254458.359856.056528.64243.706964428056966357905559083621539892983191.886423.6410506.65384.8054646.958681.457577.74187.566934423856915842755959833649554391743089.776349.1410343.95207.0753315.559960.456815.64173.7468894243603860577944632438257369112313084.066274.4410484.8OpenBenchmarking.org

LiteRT

Model: Quantized COCO SSD MobileNet v1

OpenBenchmarking.orgMicroseconds, Fewer Is BetterLiteRT 2024-10-15Model: Quantized COCO SSD MobileNet v1abd12002400360048006000SE +/- 47.15, N = 15SE +/- 72.48, N = 3SE +/- 72.45, N = 35332.125384.805207.07

LiteRT

Model: NASNet Mobile

OpenBenchmarking.orgMicroseconds, Fewer Is BetterLiteRT 2024-10-15Model: NASNet Mobileabd12K24K36K48K60KSE +/- 347.90, N = 3SE +/- 560.60, N = 3SE +/- 390.67, N = 354458.354646.953315.5

LiteRT

Model: Inception ResNet V2

OpenBenchmarking.orgMicroseconds, Fewer Is BetterLiteRT 2024-10-15Model: Inception ResNet V2abd13K26K39K52K65KSE +/- 665.58, N = 3SE +/- 317.66, N = 3SE +/- 319.38, N = 359856.058681.459960.4

LiteRT

Model: Inception V4

OpenBenchmarking.orgMicroseconds, Fewer Is BetterLiteRT 2024-10-15Model: Inception V4abd12K24K36K48K60KSE +/- 639.55, N = 3SE +/- 670.44, N = 15SE +/- 464.00, N = 356528.657577.756815.6

LiteRT

Model: Mobilenet Float

OpenBenchmarking.orgMicroseconds, Fewer Is BetterLiteRT 2024-10-15Model: Mobilenet Floatabd9001800270036004500SE +/- 22.62, N = 3SE +/- 24.52, N = 3SE +/- 6.16, N = 34243.704187.564173.74

XNNPACK

Model: FP32MobileNetV2

OpenBenchmarking.orgus, Fewer Is BetterXNNPACK b7b048Model: FP32MobileNetV2abd15003000450060007500SE +/- 15.96, N = 4SE +/- 41.70, N = 3SE +/- 15.63, N = 36964693468891. (CXX) g++ options: -O3 -lrt -lm

XNNPACK

Model: FP32MobileNetV1

OpenBenchmarking.orgus, Fewer Is BetterXNNPACK b7b048Model: FP32MobileNetV1abd9001800270036004500SE +/- 46.30, N = 4SE +/- 34.33, N = 3SE +/- 41.77, N = 34280423842431. (CXX) g++ options: -O3 -lrt -lm

XNNPACK

Model: QS8MobileNetV2

OpenBenchmarking.orgus, Fewer Is BetterXNNPACK b7b048Model: QS8MobileNetV2abd13002600390052006500SE +/- 22.24, N = 4SE +/- 27.75, N = 3SE +/- 292.13, N = 35696569160381. (CXX) g++ options: -O3 -lrt -lm

XNNPACK

Model: FP16MobileNetV3Small

OpenBenchmarking.orgus, Fewer Is BetterXNNPACK b7b048Model: FP16MobileNetV3Smallabd14002800420056007000SE +/- 506.60, N = 4SE +/- 43.47, N = 3SE +/- 252.57, N = 36357584260571. (CXX) g++ options: -O3 -lrt -lm

XNNPACK

Model: FP16MobileNetV3Large

OpenBenchmarking.orgus, Fewer Is BetterXNNPACK b7b048Model: FP16MobileNetV3Largeabd2K4K6K8K10KSE +/- 1200.15, N = 4SE +/- 47.34, N = 3SE +/- 426.05, N = 39055755979441. (CXX) g++ options: -O3 -lrt -lm

XNNPACK

Model: FP16MobileNetV2

OpenBenchmarking.orgus, Fewer Is BetterXNNPACK b7b048Model: FP16MobileNetV2abd14002800420056007000SE +/- 39.38, N = 4SE +/- 27.67, N = 3SE +/- 312.09, N = 35908598363241. (CXX) g++ options: -O3 -lrt -lm

XNNPACK

Model: FP16MobileNetV1

OpenBenchmarking.orgus, Fewer Is BetterXNNPACK b7b048Model: FP16MobileNetV1abd8001600240032004000SE +/- 16.80, N = 4SE +/- 11.67, N = 3SE +/- 169.19, N = 33621364938251. (CXX) g++ options: -O3 -lrt -lm

XNNPACK

Model: FP32MobileNetV3Small

OpenBenchmarking.orgus, Fewer Is BetterXNNPACK b7b048Model: FP32MobileNetV3Smallabd16003200480064008000SE +/- 47.52, N = 4SE +/- 121.75, N = 3SE +/- 1111.98, N = 35398554373691. (CXX) g++ options: -O3 -lrt -lm

XNNPACK

Model: FP32MobileNetV3Large

OpenBenchmarking.orgus, Fewer Is BetterXNNPACK b7b048Model: FP32MobileNetV3Largeabd2K4K6K8K10KSE +/- 69.99, N = 4SE +/- 54.24, N = 3SE +/- 1046.96, N = 392989174112311. (CXX) g++ options: -O3 -lrt -lm

LiteRT

Model: Mobilenet Quant

OpenBenchmarking.orgMicroseconds, Fewer Is BetterLiteRT 2024-10-15Model: Mobilenet Quantabd7001400210028003500SE +/- 108.69, N = 15SE +/- 81.43, N = 12SE +/- 99.43, N = 123191.883089.773084.06

LiteRT

Model: SqueezeNet

OpenBenchmarking.orgMicroseconds, Fewer Is BetterLiteRT 2024-10-15Model: SqueezeNetabd14002800420056007000SE +/- 148.43, N = 15SE +/- 97.62, N = 15SE +/- 24.67, N = 36423.646349.146274.44

LiteRT

Model: DeepLab V3

OpenBenchmarking.orgMicroseconds, Fewer Is BetterLiteRT 2024-10-15Model: DeepLab V3abd2K4K6K8K10KSE +/- 163.08, N = 15SE +/- 84.63, N = 3SE +/- 84.63, N = 310506.610343.910484.8


Phoronix Test Suite v10.8.5