litert xnnpack

AMD Ryzen Threadripper 3990X 64-Core testing with a Gigabyte TRX40 AORUS PRO WIFI (F6 BIOS) and AMD Radeon RX 5700 8GB on Pop 22.04 via the Phoronix Test Suite.

HTML result view exported from: https://openbenchmarking.org/result/2410157-PTS-LITERTXN77&sor&gru.

litert xnnpackProcessorMotherboardChipsetMemoryDiskGraphicsAudioMonitorNetworkOSKernelDesktopDisplay ServerOpenGLVulkanCompilerFile-SystemScreen ResolutionabdAMD Ryzen Threadripper 3990X 64-Core @ 2.90GHz (64 Cores / 128 Threads)Gigabyte TRX40 AORUS PRO WIFI (F6 BIOS)AMD Starship/Matisse4 x 32GB DDR4-3000MT/s CMK64GX4M2D3000C16Samsung SSD 970 EVO Plus 500GBAMD Radeon RX 5700 8GBAMD Navi 10 HDMI AudioDELL P2415QIntel I211 + Intel Wi-Fi 6 AX200Pop 22.046.8.0-76060800daily20240311-generic (x86_64)GNOME Shell 42.5X Server 1.21.1.44.6 Mesa 24.0.3-1pop1~1711635559~22.04~7a9f319 (LLVM 15.0.7 DRM 3.57)1.3.274GCC 11.4.0ext41920x1080OpenBenchmarking.orgKernel Details- Transparent Huge Pages: madviseCompiler Details- --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-link-serialization=2 --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-targets=nvptx-none=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-build-config=bootstrap-lto-lean --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v Processor Details- Scaling Governor: acpi-cpufreq schedutil (Boost: Enabled) - CPU Microcode: 0x830107aSecurity Details- gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Mitigation of untrained return thunk; SMT enabled with STIBP protection + spec_rstack_overflow: Mitigation of Safe RET + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional STIBP: always-on RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected

litert xnnpacklitert: DeepLab V3litert: SqueezeNetlitert: Inception V4litert: NASNet Mobilelitert: Mobilenet Floatlitert: Mobilenet Quantlitert: Inception ResNet V2litert: Quantized COCO SSD MobileNet v1xnnpack: FP32MobileNetV1xnnpack: FP32MobileNetV2xnnpack: FP32MobileNetV3Largexnnpack: FP32MobileNetV3Smallxnnpack: FP16MobileNetV1xnnpack: FP16MobileNetV2xnnpack: FP16MobileNetV3Largexnnpack: FP16MobileNetV3Smallxnnpack: QS8MobileNetV2abd10506.66423.6456528.654458.34243.703191.8859856.05332.1242806964929853983621590890556357569610343.96349.1457577.754646.94187.563089.7758681.45384.8042386934917455433649598375595842569110484.86274.4456815.653315.54173.743084.0659960.45207.074243688911231736938256324794460576038OpenBenchmarking.org

LiteRT

Model: DeepLab V3

OpenBenchmarking.orgMicroseconds, Fewer Is BetterLiteRT 2024-10-15Model: DeepLab V3bda2K4K6K8K10KSE +/- 84.63, N = 3SE +/- 84.63, N = 3SE +/- 163.08, N = 1510343.910484.810506.6

LiteRT

Model: SqueezeNet

OpenBenchmarking.orgMicroseconds, Fewer Is BetterLiteRT 2024-10-15Model: SqueezeNetdba14002800420056007000SE +/- 24.67, N = 3SE +/- 97.62, N = 15SE +/- 148.43, N = 156274.446349.146423.64

LiteRT

Model: Inception V4

OpenBenchmarking.orgMicroseconds, Fewer Is BetterLiteRT 2024-10-15Model: Inception V4adb12K24K36K48K60KSE +/- 639.55, N = 3SE +/- 464.00, N = 3SE +/- 670.44, N = 1556528.656815.657577.7

LiteRT

Model: NASNet Mobile

OpenBenchmarking.orgMicroseconds, Fewer Is BetterLiteRT 2024-10-15Model: NASNet Mobiledab12K24K36K48K60KSE +/- 390.67, N = 3SE +/- 347.90, N = 3SE +/- 560.60, N = 353315.554458.354646.9

LiteRT

Model: Mobilenet Float

OpenBenchmarking.orgMicroseconds, Fewer Is BetterLiteRT 2024-10-15Model: Mobilenet Floatdba9001800270036004500SE +/- 6.16, N = 3SE +/- 24.52, N = 3SE +/- 22.62, N = 34173.744187.564243.70

LiteRT

Model: Mobilenet Quant

OpenBenchmarking.orgMicroseconds, Fewer Is BetterLiteRT 2024-10-15Model: Mobilenet Quantdba7001400210028003500SE +/- 99.43, N = 12SE +/- 81.43, N = 12SE +/- 108.69, N = 153084.063089.773191.88

LiteRT

Model: Inception ResNet V2

OpenBenchmarking.orgMicroseconds, Fewer Is BetterLiteRT 2024-10-15Model: Inception ResNet V2bad13K26K39K52K65KSE +/- 317.66, N = 3SE +/- 665.58, N = 3SE +/- 319.38, N = 358681.459856.059960.4

LiteRT

Model: Quantized COCO SSD MobileNet v1

OpenBenchmarking.orgMicroseconds, Fewer Is BetterLiteRT 2024-10-15Model: Quantized COCO SSD MobileNet v1dab12002400360048006000SE +/- 72.45, N = 3SE +/- 47.15, N = 15SE +/- 72.48, N = 35207.075332.125384.80

XNNPACK

Model: FP32MobileNetV1

OpenBenchmarking.orgus, Fewer Is BetterXNNPACK b7b048Model: FP32MobileNetV1bda9001800270036004500SE +/- 34.33, N = 3SE +/- 41.77, N = 3SE +/- 46.30, N = 44238424342801. (CXX) g++ options: -O3 -lrt -lm

XNNPACK

Model: FP32MobileNetV2

OpenBenchmarking.orgus, Fewer Is BetterXNNPACK b7b048Model: FP32MobileNetV2dba15003000450060007500SE +/- 15.63, N = 3SE +/- 41.70, N = 3SE +/- 15.96, N = 46889693469641. (CXX) g++ options: -O3 -lrt -lm

XNNPACK

Model: FP32MobileNetV3Large

OpenBenchmarking.orgus, Fewer Is BetterXNNPACK b7b048Model: FP32MobileNetV3Largebad2K4K6K8K10KSE +/- 54.24, N = 3SE +/- 69.99, N = 4SE +/- 1046.96, N = 391749298112311. (CXX) g++ options: -O3 -lrt -lm

XNNPACK

Model: FP32MobileNetV3Small

OpenBenchmarking.orgus, Fewer Is BetterXNNPACK b7b048Model: FP32MobileNetV3Smallabd16003200480064008000SE +/- 47.52, N = 4SE +/- 121.75, N = 3SE +/- 1111.98, N = 35398554373691. (CXX) g++ options: -O3 -lrt -lm

XNNPACK

Model: FP16MobileNetV1

OpenBenchmarking.orgus, Fewer Is BetterXNNPACK b7b048Model: FP16MobileNetV1abd8001600240032004000SE +/- 16.80, N = 4SE +/- 11.67, N = 3SE +/- 169.19, N = 33621364938251. (CXX) g++ options: -O3 -lrt -lm

XNNPACK

Model: FP16MobileNetV2

OpenBenchmarking.orgus, Fewer Is BetterXNNPACK b7b048Model: FP16MobileNetV2abd14002800420056007000SE +/- 39.38, N = 4SE +/- 27.67, N = 3SE +/- 312.09, N = 35908598363241. (CXX) g++ options: -O3 -lrt -lm

XNNPACK

Model: FP16MobileNetV3Large

OpenBenchmarking.orgus, Fewer Is BetterXNNPACK b7b048Model: FP16MobileNetV3Largebda2K4K6K8K10KSE +/- 47.34, N = 3SE +/- 426.05, N = 3SE +/- 1200.15, N = 47559794490551. (CXX) g++ options: -O3 -lrt -lm

XNNPACK

Model: FP16MobileNetV3Small

OpenBenchmarking.orgus, Fewer Is BetterXNNPACK b7b048Model: FP16MobileNetV3Smallbda14002800420056007000SE +/- 43.47, N = 3SE +/- 252.57, N = 3SE +/- 506.60, N = 45842605763571. (CXX) g++ options: -O3 -lrt -lm

XNNPACK

Model: QS8MobileNetV2

OpenBenchmarking.orgus, Fewer Is BetterXNNPACK b7b048Model: QS8MobileNetV2bad13002600390052006500SE +/- 27.75, N = 3SE +/- 22.24, N = 4SE +/- 292.13, N = 35691569660381. (CXX) g++ options: -O3 -lrt -lm


Phoronix Test Suite v10.8.5