tensorflow-onnx

microsoft testing on Ubuntu 22.04 via the Phoronix Test Suite.

HTML result view exported from: https://openbenchmarking.org/result/2310314-NE-TENSORFLO66&grt.

tensorflow-onnxProcessorMotherboardMemoryDiskGraphicsOSKernelVulkanCompilerFile-SystemScreen ResolutionSystem LayerIntel Xeon Platinum 8272CL - hyperv_drmdrmfb -Intel Xeon Platinum 8272CL (4 Cores / 8 Threads)Microsoft Virtual Machine v7.0 (090008 BIOS)32GB32GB Virtual Diskhyperv_drmdrmfbUbuntu 22.046.2.0-1015-azure (x86_64)1.3.238GCC 11.4.0ext41024x768microsoftOpenBenchmarking.org- Transparent Huge Pages: always- --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-link-serialization=2 --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-targets=nvptx-none=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-build-config=bootstrap-lto-lean --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v - CPU Microcode: 0xffffffff- Python 3.10.12- gather_data_sampling: Unknown: Dependent on hypervisor status + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Vulnerable: Clear buffers attempted no microcode; SMT Host state unknown + retbleed: Vulnerable + spec_rstack_overflow: Not affected + spec_store_bypass: Vulnerable + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines STIBP: disabled RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Vulnerable: Clear buffers attempted no microcode; SMT Host state unknown

tensorflow-onnxonnx: GPT-2 - CPU - Standardonnx: super-resolution-10 - CPU - Standardtensorflow: CPU - 32 - VGG-16tensorflow: CPU - 32 - ResNet-50Intel Xeon Platinum 8272CL - hyperv_drmdrmfb -348818615.5414.81OpenBenchmarking.org

ONNX Runtime

Model: GPT-2 - Device: CPU - Executor: Standard

OpenBenchmarking.orgInferences Per Minute, More Is BetterONNX Runtime 1.11Model: GPT-2 - Device: CPU - Executor: StandardIntel Xeon Platinum 8272CL - hyperv_drmdrmfb -7001400210028003500SE +/- 8.38, N = 334881. (CXX) g++ options: -ffunction-sections -fdata-sections -march=native -mtune=native -O3 -flto -fno-fat-lto-objects -ldl -lrt

ONNX Runtime

Model: super-resolution-10 - Device: CPU - Executor: Standard

OpenBenchmarking.orgInferences Per Minute, More Is BetterONNX Runtime 1.11Model: super-resolution-10 - Device: CPU - Executor: StandardIntel Xeon Platinum 8272CL - hyperv_drmdrmfb -400800120016002000SE +/- 1.48, N = 318611. (CXX) g++ options: -ffunction-sections -fdata-sections -march=native -mtune=native -O3 -flto -fno-fat-lto-objects -ldl -lrt

TensorFlow

Device: CPU - Batch Size: 32 - Model: VGG-16

OpenBenchmarking.orgimages/sec, More Is BetterTensorFlow 2.12Device: CPU - Batch Size: 32 - Model: VGG-16Intel Xeon Platinum 8272CL - hyperv_drmdrmfb -1.24652.4933.73954.9866.2325SE +/- 0.01, N = 35.54

TensorFlow

Device: CPU - Batch Size: 32 - Model: ResNet-50

OpenBenchmarking.orgimages/sec, More Is BetterTensorFlow 2.12Device: CPU - Batch Size: 32 - Model: ResNet-50Intel Xeon Platinum 8272CL - hyperv_drmdrmfb -48121620SE +/- 0.06, N = 314.81

Phoronix Test Suite v10.8.5