LeelaChessZero

LeelaChessZero (lc0 / lczero) is a chess engine automated vian neural networks. This test profile can be used for OpenCL, CUDA + cuDNN, and BLAS (CPU-based) benchmarking.

To run this test with the Phoronix Test Suite, the basic command is: phoronix-test-suite benchmark lczero.

Project Site

github.com

Test Created

12 January 2019

Last Updated

27 September 2020

Test Maintainer

Michael Larabel 

Test Type

Processor

Average Install Time

1 Minute, 20 Seconds

Average Run Time

4 Minutes, 22 Seconds

Test Dependencies

C/C++ Compiler Toolchain + BLAS (Basic Linear Algebra Sub-Routine) + OpenCL + Meson Build System + Eigen + Zlib

Accolades

40k+ Downloads

Supported Platforms


Public Result UploadsReported Installs*Test Completions*OpenBenchmarking.orgEventsLeelaChessZero Popularity Statisticspts/lczero2019.012019.022019.032019.042019.052019.062019.072019.082019.092019.102019.112019.122020.012020.022020.032020.042020.052020.062020.072020.082020.092020.102020.112020.122021.012021.022021.032021.045001000150020002500
* Data based on those opting to upload their test results to OpenBenchmarking.org and users enabling the opt-in anonymous statistics reporting while running benchmarks from an Internet-connected platform.
Data current as of Sun, 18 Apr 2021 18:26:16 GMT.
Random13.2%OpenCL26.4%BLAS29.6%Eigen30.8%Backend Option PopularityOpenBenchmarking.org

Revision History

pts/lczero-1.5.1   [View Source]   Sun, 27 Sep 2020 17:41:46 GMT
Limit max CPU threads to 64 to workaround upstream issue with lc0 bailing out otherwise.

pts/lczero-1.5.0   [View Source]   Sun, 06 Sep 2020 14:18:27 GMT
Update against latest upstream along with updated network, add eigen as possible external dependency.

pts/lczero-1.4.0   [View Source]   Thu, 30 Apr 2020 09:06:47 GMT
Update against lc0 0.25, use new network as old one was removed.

pts/lczero-1.3.0   [View Source]   Fri, 10 Jan 2020 20:28:53 GMT
Update against latest lc0 v0.23.2 upstream, other test improvements.

pts/lczero-1.2.1   [View Source]   Thu, 03 Oct 2019 14:07:11 GMT
Add Windows support.

pts/lczero-1.2.0   [View Source]   Thu, 26 Sep 2019 16:49:44 GMT
Update against lczero upstream

pts/lczero-1.1.1   [View Source]   Wed, 16 Jan 2019 05:41:40 GMT
Add zlib to external dependency list.

pts/lczero-1.1.0   [View Source]   Tue, 15 Jan 2019 10:11:47 GMT
Set threads option always for CPU testing.

pts/lczero-1.0.1   [View Source]   Sun, 13 Jan 2019 14:22:54 GMT
Allow CUDA and BLAS benchmarking back-end support.

pts/lczero-1.0.0   [View Source]   Sat, 12 Jan 2019 20:10:43 GMT
Initial commit of lc0 / lczero chess benchmark using neural networks with GPU compute.

Suites Using This Test

Chess Test Suite

Machine Learning

CPU Massive

HPC - High Performance Computing

NVIDIA GPU Compute


Performance Metrics

Analyze Test Configuration:

LeelaChessZero 0.26

Backend: OpenCL

OpenBenchmarking.org metrics for this test profile configuration based on 559 public results since 6 September 2020 with the latest data as of 8 April 2021.

Below is an overview of the generalized performance for components where there is sufficient statistically significant data based upon user-uploaded results. It is important to keep in mind particularly in the Linux/open-source space there can be vastly different OS configurations, with this overview intended to offer just general guidance as to the performance expectations.

Component
Percentile Rank
# Matching Public Results
Nodes Per Second (Average)
100th
7
38880 +/- 248
98th
5
29131 +/- 128
88th
14
17706 +/- 2187
Mid-Tier
75th
< 16705
63rd
15
15779 +/- 1740
62nd
14
14842 +/- 1578
57th
14
12735 +/- 1204
55th
14
12456 +/- 1215
55th
9
12383 +/- 686
Median
50th
11399
50th
13
10611 +/- 959
40th
5
6267 +/- 4
34th
3
4152 +/- 19
27th
7
588 +/- 30
Low-Tier
25th
< 495
25th
3
381 +/- 4
24th
119
344
2nd
4
308 +/- 3
OpenBenchmarking.orgDistribution Of Public Results - Backend: OpenCL559 Results Range From 18 To 39148 Nodes Per Second18801158423673150393347165499628270657848863194141019710980117631254613329141121489515678164611724418027188101959320376211592194222725235082429125074258572664027423282062898929772305553133832121329043368734470352533603636819376023838539168306090120150

Based on OpenBenchmarking.org data, the selected test / test configuration (LeelaChessZero 0.26 - Backend: OpenCL) has an average run-time of 19 minutes. By default this test profile is set to run at least 3 times but may increase if the standard deviation exceeds pre-defined defaults or other calculations deem additional runs necessary for greater statistical accuracy of the result.

OpenBenchmarking.orgMinutesTime Required To Complete BenchmarkBackend: OpenCLRun-Time612182430Min: 17 / Avg: 18.76 / Max: 26

Based on public OpenBenchmarking.org results, the selected test / test configuration has an average standard deviation of 0.2%.

OpenBenchmarking.orgPercent, Fewer Is BetterAverage Deviation Between RunsBackend: OpenCLDeviation246810Min: 0 / Avg: 0.25 / Max: 2

Recent Test Results

OpenBenchmarking.org Results Compare

3 Systems - 99 Benchmark Results

AMD Ryzen 9 5900X 12-Core - ASUS ROG CROSSHAIR VIII HERO - AMD Starship

Ubuntu 20.04 - 5.8.0-48-generic - GNOME Shell 3.36.7

3 Systems - 100 Benchmark Results

AMD Ryzen 9 5900X 12-Core - ASUS ROG CROSSHAIR VIII HERO - AMD Starship

Ubuntu 20.04 - 5.8.0-48-generic - GNOME Shell 3.36.7

1 System - 100 Benchmark Results

AMD Ryzen 9 5900X 12-Core - ASUS ROG CROSSHAIR VIII HERO - AMD Starship

Ubuntu 20.04 - 5.8.0-48-generic - GNOME Shell 3.36.7

4 Systems - 138 Benchmark Results

Intel Core i9-10900K - Gigabyte Z490 AORUS MASTER - Intel Comet Lake PCH

Ubuntu 21.04 - 5.12.0-051200rc3daily20210315-generic - GNOME Shell 3.38.3

2 Systems - 243 Benchmark Results

Intel Core i9-10885H - HP 8736 - Intel Comet Lake PCH

Ubuntu 21.04 - 5.10.0-14-generic - GNOME Shell 3.38.3

1 System - 238 Benchmark Results

Intel Core i9-10885H - HP 8736 - Intel Comet Lake PCH

Ubuntu 20.10 - 5.8.0-43-generic - GNOME Shell 3.38.2

1 System - 131 Benchmark Results

Intel Core i9-10885H - HP 8736 - Intel Comet Lake PCH

Ubuntu 20.04 - 5.6.0-1047-oem - GNOME Shell 3.36.4

2 Systems - 8 Benchmark Results

AMD Ryzen 7 1700 Eight-Core - ASUS PRIME A320M-K - AMD 17h

ManjaroLinux 20.2.1 - 5.9.16-1-MANJARO - GNOME Shell 3.38.3

1 System - 8 Benchmark Results

AMD Ryzen 7 1700 Eight-Core - ASUS PRIME A320M-K - AMD 17h

ManjaroLinux 20.2.1 - 5.9.16-1-MANJARO - GNOME Shell 3.38.3

Most Popular Test Results

OpenBenchmarking.org Results Compare

3 Systems - 75 Benchmark Results

Intel Core i9-10980XE - ASRock X299 Steel Legend - Intel Sky Lake-E DMI3 Registers

Ubuntu 20.04 - 5.4.0-48-generic - GNOME Shell 3.36.3

2 Systems - 243 Benchmark Results

Intel Core i9-10885H - HP 8736 - Intel Comet Lake PCH

Ubuntu 21.04 - 5.10.0-14-generic - GNOME Shell 3.38.3

1 System - 132 Benchmark Results

AMD Ryzen Threadripper 3970X 32-Core - ASUS ROG ZENITH II EXTREME - AMD Starship

Ubuntu 20.04 - 5.4.0-48-generic - GNOME Shell 3.36.4

3 Systems - 52 Benchmark Results

Intel Core i7-5775C - CompuLab v1.0 - Intel Broadwell-U DMI

Ubuntu 20.10 - 5.8.0-26-generic - GNOME Shell 3.38.1

4 Systems - 65 Benchmark Results

Intel Core i9-9900K - ASRock Z390M Pro4 - Intel Cannon Lake PCH

Ubuntu 20.04 - 5.9.0-050900rc1daily20200819-generic - GNOME Shell 3.36.4

23 Systems - 123 Benchmark Results

AMD Ryzen 9 3950X 16-Core - ASUS ROG CROSSHAIR VIII HERO - AMD Starship

Ubuntu 20.04 - 5.4.0-45-generic - GNOME Shell 3.36.4

2 Systems - 190 Benchmark Results

Intel Core i7-1065G7 - Dell 06CDVY - Intel Device 34ef

Ubuntu 20.10 - 5.9.1-050901-generic - GNOME Shell 3.38.1

1 System - 55 Benchmark Results

AMD Ryzen 9 3950X 16-Core - ASUS ROG CROSSHAIR VIII HERO - AMD Starship

Ubuntu 20.04 - 5.4.0-48-generic - GNOME Shell 3.36.4

Find More Test Results