Tests
Suites
Latest Results
Search
Register
Login
Popular Tests
Timed Linux Kernel Compilation
7-Zip Compression
SVT-AV1
Stockfish
FFmpeg
PostgreSQL
Newest Tests
WarpX
Epoch
Valkey
Whisperfile
XNNPACK
GROMACS
Recently Updated Tests
CP2K Molecular Dynamics
Apache CouchDB
PostgreSQL
NAMD
Apache Cassandra
Opus Codec Encoding
New & Recently Updated Tests
Recently Updated Suites
Database Test Suite
Machine Learning
Steam
New & Recently Updated Suites
Component Benchmarks
CPUs / Processors
GPUs / Graphics
OpenGL
Disks / Storage
Motherboards
File-Systems
Operating Systems
OpenBenchmarking.org
Corporate / Organization Info
Bug Reports / Feature Requests
ONNX Runtime 1.6.0
pts/onnx-1.6.0
- 11 February 2023 -
Update against ONNX Runtime 1.14 upstream.
downloads.xml
<?xml version="1.0"?> <!--Phoronix Test Suite v10.8.4--> <PhoronixTestSuite> <Downloads> <Package> <URL>https://github.com/onnx/models/blob/main/vision/object_detection_segmentation/yolov4/model/yolov4.tar.gz</URL> <FileName>yolov4.tar.gz</FileName> <FileSize>248665735</FileSize> <Optional>TRUE</Optional> </Package> <Package> <URL>https://github.com/onnx/models/raw/main/vision/object_detection_segmentation/fcn/model/fcn-resnet101-11.tar.gz</URL> <FileName>fcn-resnet101-11.tar.gz</FileName> <FileSize>294729465</FileSize> <Optional>TRUE</Optional> </Package> <Package> <URL>https://github.com/onnx/models/raw/main/vision/super_resolution/sub_pixel_cnn_2016/model/super-resolution-10.tar.gz</URL> <FileName>super-resolution-10.tar.gz</FileName> <FileSize>1815286</FileSize> <Optional>TRUE</Optional> </Package> <Package> <URL>https://github.com/onnx/models/raw/main/text/machine_comprehension/bert-squad/model/bertsquad-12.tar.gz</URL> <FileName>bertsquad-12.tar.gz</FileName> <FileSize>403082198</FileSize> <Optional>TRUE</Optional> </Package> <Package> <URL>https://github.com/onnx/models/raw/main/text/machine_comprehension/gpt-2/model/gpt2-10.tar.gz</URL> <FileName>gpt2-10.tar.gz</FileName> <FileSize>463131530</FileSize> <Optional>TRUE</Optional> </Package> <Package> <URL>https://github.com/onnx/models/raw/main/vision/body_analysis/arcface/model/arcfaceresnet100-8.tar.gz</URL> <FileName>arcfaceresnet100-8.tar.gz</FileName> <FileSize>237564370</FileSize> <Optional>TRUE</Optional> </Package> <Package> <URL>https://github.com/onnx/models/raw/main/vision/classification/resnet/model/resnet50-v1-12-int8.tar.gz</URL> <FileName>resnet50-v1-12-int8.tar.gz</FileName> <FileSize>22355322</FileSize> <Optional>TRUE</Optional> </Package> <Package> <URL>https://github.com/onnx/models/raw/main/vision/classification/caffenet/model/caffenet-12-int8.tar.gz</URL> <FileName>caffenet-12-int8.tar.gz</FileName> <FileSize>40718510</FileSize> <Optional>TRUE</Optional> </Package> <Package> <URL>https://github.com/onnx/models/raw/main/vision/object_detection_segmentation/faster-rcnn/model/FasterRCNN-12-int8.tar.gz</URL> <FileName>FasterRCNN-12-int8.tar.gz</FileName> <FileSize>38019008</FileSize> <Optional>TRUE</Optional> </Package> </Downloads> </PhoronixTestSuite>
install.sh
#!/bin/bash rm -rf onnxruntime git clone https://github.com/microsoft/onnxruntime cd onnxruntime git checkout v1.14.0 ./build.sh --config Release --build_shared_lib --parallel --skip_tests --enable_lto --cmake_extra_defines onnxruntime_BUILD_FOR_NATIVE_MACHINE=ON echo $? > ~/install-exit-status cd ~ tar -xf yolov4.tar.gz tar -xf fcn-resnet101-11.tar.gz tar -xf super-resolution-10.tar.gz tar -xf bertsquad-12.tar.gz tar -xf gpt2-10.tar.gz tar -xf arcfaceresnet100-8.tar.gz tar -xf resnet50-v1-12-int8.tar.gz tar -xf caffenet-12-int8.tar.gz tar -xf FasterRCNN-12-int8.tar.gz echo "#!/bin/bash ./onnxruntime/build/Linux/Release/onnxruntime_perf_test \$@ > \$LOG_FILE 2>&1 echo \$? > ~/test-exit-status" > onnx chmod +x onnx
results-definition.xml
<?xml version="1.0"?> <!--Phoronix Test Suite v10.8.4--> <PhoronixTestSuite> <ResultsParser> <OutputTemplate>Number of inferences per second: #_RESULT_#</OutputTemplate> </ResultsParser> <ResultsParser> <OutputTemplate>Average inference time cost: #_RESULT_# ms</OutputTemplate> <ResultScale>Inference Time Cost (ms)</ResultScale> <ResultProportion>LIB</ResultProportion> <Importance>Secondary</Importance> </ResultsParser> </PhoronixTestSuite>
test-definition.xml
<?xml version="1.0"?> <!--Phoronix Test Suite v10.8.4--> <PhoronixTestSuite> <TestInformation> <Title>ONNX Runtime</Title> <AppVersion>1.14</AppVersion> <Description>ONNX Runtime is developed by Microsoft and partners as a open-source, cross-platform, high performance machine learning inferencing and training accelerator. This test profile runs the ONNX Runtime with various models available from the ONNX Model Zoo.</Description> <ResultScale>Inferences Per Second</ResultScale> <Proportion>HIB</Proportion> <TimesToRun>3</TimesToRun> </TestInformation> <TestProfile> <Version>1.6.0</Version> <SupportedPlatforms>Linux</SupportedPlatforms> <SoftwareType>Utility</SoftwareType> <TestType>System</TestType> <License>Free</License> <Status>Verified</Status> <ExternalDependencies>python, git, build-utilities, cmake</ExternalDependencies> <InstallRequiresInternet>TRUE</InstallRequiresInternet> <EnvironmentSize>4900</EnvironmentSize> <ProjectURL>https://www.onnxruntime.ai/</ProjectURL> <RepositoryURL>https://github.com/microsoft/onnxruntime</RepositoryURL> <InternalTags>SMP</InternalTags> <Maintainer>Michael Larabel</Maintainer> <SystemDependencies>gmock/gmock.h</SystemDependencies> </TestProfile> <TestSettings> <Default> <PostArguments>-t 90 </PostArguments> </Default> <Option> <DisplayName>Model</DisplayName> <Identifier>model</Identifier> <Menu> <Entry> <Name>yolov4</Name> <Value>yolov4/yolov4.onnx</Value> </Entry> <Entry> <Name>fcn-resnet101-11</Name> <Value>fcn-resnet101-11/model.onnx</Value> </Entry> <Entry> <Name>super-resolution-10</Name> <Value>super_resolution/super_resolution.onnx</Value> </Entry> <Entry> <Name>bertsquad-12</Name> <Value>bertsquad-12/bertsquad-12.onnx</Value> </Entry> <Entry> <Name>GPT-2</Name> <Value>GPT2/model.onnx</Value> </Entry> <Entry> <Name>ArcFace ResNet-100</Name> <Value>resnet100/resnet100.onnx</Value> </Entry> <Entry> <Name>ResNet50 v1-12-int8</Name> <Value>resnet50-v1-12-int8/resnet50-v1-12-int8.onnx</Value> </Entry> <Entry> <Name>CaffeNet 12-int8</Name> <Value>caffenet-12-int8/caffenet-12-int8.onnx</Value> </Entry> <Entry> <Name>Faster R-CNN R-50-FPN-int8</Name> <Value>FasterRCNN-12-int8/FasterRCNN-12-int8.onnx</Value> </Entry> </Menu> </Option> <Option> <DisplayName>Device</DisplayName> <Identifier>device</Identifier> <ArgumentPrefix>-e </ArgumentPrefix> <Menu> <Entry> <Name>CPU</Name> <Value>cpu</Value> </Entry> </Menu> </Option> <Option> <DisplayName>Executor</DisplayName> <Identifier>executor</Identifier> <ArgumentPrefix> </ArgumentPrefix> <Menu> <Entry> <Name>Standard</Name> </Entry> <Entry> <Name>Parallel</Name> <Value>-P</Value> </Entry> </Menu> </Option> </TestSettings> </PhoronixTestSuite>