For Users

Repeatable Results

Easily take a result set and reproduce the exact execution on a different system.

Compare Globally

OpenBenchmarking Performance Classifications (OPC) and other comparison features provide a concise view of any result against the thousands of other results.

Shopping Assistance

When shopping for new computer hardware, users can leverage the vast databases of hardware and software information to find products of interest to them that meets their performance and functional requirements.

Phoronix Test Suite

OpenBenchmarking.org is built around the Phoronix Test Suite, thereby taking advantage of its vast community of testers and years of benchmarking research by Phoronix Media and its partners.

FOR PROJECTS & ORGANIZATIONS

Crowd-Sourced Results

Leveraging OpenBenchmarking.org allows for benchmarks and functional tests from hundreds or thousands of users in a very easy, direct, and real-time manner.

Aggregated Results

Collect historical results for a software/hardware project or product. By aggregating results on OpenBenchmarking.org you can better determine where regressions or quirks occur as well as spot trends in performance.

Embedded To Cloud Scale

OpenBenchmarking.org and the Phoronix Test Suite run on everything from tiny ARM-based plug computers to global cloud-based deployments.

Cross-Platform

OpenBenchmarking.org and the Phoronix Test Suite support a wide variety of platforms including Linux, Mac OS X, Solaris, *BSD, and Windows.

Free Public Use

Free use of OpenBenchmarking.org is encouraged. However, all results are required to be made public or have a short embargo time for which they can be set private. Commercial options are available for organizations wishing to deploy custom or private installations.

Related Products & Services

Phoronix Media offers a portfolio of extra products and services that provide increased and targeted support for benchmarking and functional testing. These include:

Phoronix Test Suite

Phoromatic

PTS Commercial


The Full Feature Set

OpenBenchmarking.org is a centralized testing ecosystem. OpenBenchmarking.org provides a collaborative, open test platform with a standardized test profile and suite management system for distributing and standardizing benchmarks.

Users can search for a given software or hardware component and find detailed information on its Linux/BSD/OSX/Solaris performance results, find current pricing information, and compare the performance to like-hardware.

OpenBenchmarking.org not only makes it efficient to gather new tests and suites and to find test results -- whether it be quantitative or qualitative based -- for a given hardware or software configuration, but it also provides a robust feature-set for exploring and discovering new tests and hardware, analyzing results in great detail, communicating over result findings, and to collaborate over future software and hardware tests.

With OpenBenchmarking.org, the user can now easily access updated and/or new test profiles and suites without having to update the test client (assuming the test/suite isn’t dependent upon an XML schema or specification change only found in a later client release).

Access third-party test profiles and suites, that can be created and uploaded by any individual or organization -- there is no "gate keeper" for letting tests in nor are there manual steps any longer to distributing independent tests and suites.

Users can directly compare the performance of multiple test results, find similar hardware components, and discover new hardware that may suit their needs.

Via the OpenBenchmarking.org web interface are a number of analytical features for interpreting multi-way comparisons, finding geometric / aggregate means, calculate performance per price ratios, and examine system log files for given configurations, among other capabilities.

From OpenBenchmarking.org Statistics, view the popularity of difference hardware/software components, different tested kernel/X/distribution configurations for given hardware components, and popular and minimum software versions used in conjunction with selected hardware.

When uploading results to OpenBenchmarking.org, users can optionally attach system logs such as the dmesg, Xorg.0.log, and lspci output. Prominent configuration files such as the xorg.conf can also be uploaded so that others can collaborate and find potential problems or configuration options that may lead to greater performance or other benefits.

Primary focus is obviously on CPU processors, graphics processors, and motherboards, but OpenBenchmarking.org supports many devices, including disk drives, network adapters, and audio adapters. There is also support for analyzing software components such as a particular operating system release, compiler, or kernel version.

OpenBenchmarking.org is completely extensible. Any free software project can take advantage of any of these features for their own testing purposes by simply writing a test profile / suite that compiles with our public specifications. Technically, any benchmarking client can also implement this benchmarking platform and take advantage of this entire framework as well.

Advanced commenting abilities. Users can comment on test results, test profiles, test suites, and even search results. Where this search though differs greatly from forums or Hardware Compatibility Lists is the precise focus and detail. With a forum or HCL-type databases you are dependent upon a user posting a question/comment in a particular section and then finding yourself lucky that a owner of the said hardware will come across that question. With OpenBenchmarking.org, if a potential customer of "Graphics Card XYZ" posts a question asking if "Feature 123" is implemented in the open-source driver for this graphics card, depending upon each user’s notification settings, OpenBenchmarking.org can proactively send a notification to those users that have or had "Graphics Card XYZ" informing them another user has a question, comment, or suggestion based upon their settings. This is leading to the largest global test farm.

OpenBenchmarking.org provides custom user pages to registered account users so they can optionally show off their test systems and results as well as their own custom test profiles and suites.

When the Phoronix Test Suite is installed with .OpenBenchmarking file extension support with MIME recognition, simply clicking on the OpenBenchmarking.org icon from a test result, profile, or suite will automatically launch the Phoronix Test Suite with the given test configuration.

With OpenBenchmarking.org Performance Classifications (OPC), users can see how their own system's performance compares to that of any other system results hosted on OpenBenchmarking.org.

Future work on OpenBenchmarking.org includes analyzing the most common hardware / software configurations, autonomous spotting of performance / compatibility anomalies or quirks, analyzing real-world upgrade trends, auto-parsing and suggestions based upon uploaded configuration files for reaching optimal performance while being self-learning based upon existing uploaded logs and configuration files, embeddable graphs / results onto other web-pages in a rich web format, web-based test profile and suite creation, and an external OpenBenchmarking.org API.