Memtier_benchmark

Memtier_benchmark is a NoSQL Redis/Memcache traffic generation plus benchmarking tool. This current test profile currently just stresses the Redis protocol and basic options exposed wotj a 1:1 Set/Get ratio, 30 pipeline, 100 clients per thread, and thread count equal to the number of CPU cores/threads present. Patches to extend the test are welcome as always.

To run this test with the Phoronix Test Suite, the basic command is: phoronix-test-suite benchmark memtier-benchmark.

Project Site

github.com

Test Created

23 September 2019

Last Updated

24 September 2019

Test Maintainer

Michael Larabel 

Test Type

System

Average Install Time

1 Minute, 10 Seconds

Average Run Time

2 Minutes, 20 Seconds

Test Dependencies

C/C++ Compiler Toolchain + Libevent + Perl Compatible Regular Expressions + OpenSSL + Libtool

Accolades

10k+ Downloads

Supported Platforms


Public Result UploadsReported Installs*Test Completions*OpenBenchmarking.orgEventsMemtier_benchmark Popularity Statisticspts/memtier-benchmark2019.092019.102019.112019.122020.012020.022020.032020.042020.052020.062020.072020.082020.092020.102020.112020.122021.012021.022021.032021.045001000150020002500
* Data based on those opting to upload their test results to OpenBenchmarking.org and users enabling the opt-in anonymous statistics reporting while running benchmarks from an Internet-connected platform.
Data current as of Mon, 12 Apr 2021 20:24:33 GMT.

Revision History

pts/memtier-benchmark-1.0.1   [View Source]   Tue, 24 Sep 2019 15:20:16 GMT
Add libtool as possible external dependency.

pts/memtier-benchmark-1.0.0   [View Source]   Mon, 23 Sep 2019 17:41:24 GMT
Initial commit of Redis/memcached memtier_benchmark program.

Suites Using This Test

Database Test Suite

Server


Performance Metrics

Analyze Test Configuration:

Memtier_benchmark 1.2.17

Protocol: Redis

OpenBenchmarking.org metrics for this test profile configuration based on 248 public results since 23 September 2019 with the latest data as of 22 March 2021.

Below is an overview of the generalized performance for components where there is sufficient statistically significant data based upon user-uploaded results. It is important to keep in mind particularly in the Linux/open-source space there can be vastly different OS configurations, with this overview intended to offer just general guidance as to the performance expectations.

Component
Percentile Rank
# Matching Public Results
Ops/sec (Average)
98th
22
4145403 +/- 338761
93rd
3
3715581 +/- 187539
86th
3
2890309 +/- 377632
85th
4
2885454 +/- 69442
82nd
4
2792857 +/- 102621
82nd
10
2784259 +/- 113870
78th
7
2743850 +/- 113220
Mid-Tier
75th
< 2678700
74th
6
2653786 +/- 121145
67th
27
2562052 +/- 136376
55th
4
2361418 +/- 38584
Median
50th
2300474
49th
6
2285680 +/- 42825
40th
5
2044125 +/- 49334
38th
3
1997000 +/- 71726
35th
3
1950393 +/- 29150
30th
3
1874687 +/- 6438
29th
5
1866427 +/- 106880
Low-Tier
25th
< 1762266
24th
3
1645381 +/- 145270
22nd
7
1596694 +/- 15939
11th
10
1212755 +/- 76940
7th
6
1120827 +/- 18922
OpenBenchmarking.orgDistribution Of Public Results - Protocol: Redis248 Results Range From 176928 To 4800891 Ops/sec176928269408361888454368546848639328731808824288916768100924811017281194208128668813791681471648156412816566081749088184156819340482026528211900822114882303968239644824889282581408267388827663682858848295132830438083136288322876833212483413728350620835986883691168378364838761283968608406108841535684246048433852844310084523488461596847084484800928510152025

Based on OpenBenchmarking.org data, the selected test / test configuration (Memtier_benchmark 1.2.17 - Protocol: Redis) has an average run-time of 10 minutes. By default this test profile is set to run at least 3 times but may increase if the standard deviation exceeds pre-defined defaults or other calculations deem additional runs necessary for greater statistical accuracy of the result.

OpenBenchmarking.orgMinutesTime Required To Complete BenchmarkProtocol: RedisRun-Time816243240Min: 4 / Avg: 10.08 / Max: 40

Based on public OpenBenchmarking.org results, the selected test / test configuration has an average standard deviation of 2.3%.

OpenBenchmarking.orgPercent, Fewer Is BetterAverage Deviation Between RunsProtocol: RedisDeviation3691215Min: 0 / Avg: 2.31 / Max: 11

Notable Instruction Set Usage

Notable instruction set extensions supported by this test, based on an automatic analysis by the Phoronix Test Suite / OpenBenchmarking.org analytics engine.

Instruction Set
Support
Instructions Detected
SSE2 (SSE2)
Used by default on supported hardware.
 
CVTSI2SD COMISD UCOMISD MOVAPD DIVSD MULSD CVTTSD2SI ADDSD CVTSS2SD SUBSD MOVDQU MOVDQA MOVD CVTSD2SS SQRTSD
Last automated analysis: 30 January 2021

This test profile binary relies on the shared libraries libevent-2.1.so.7, libpthread.so.0, libz.so.1, libpcre.so.3, libm.so.6, libc.so.6.

Recent Test Results

OpenBenchmarking.org Results Compare

1 System - 171 Benchmark Results

2 x Intel Xeon Gold 5218R - Supermicro X11DPX-T - Intel Device 2020

Ubuntu 16.04 - 4.15.0-139-generic - Unity 7.4.0

1 System - 87 Benchmark Results

2 x Intel Xeon E5-2680 0 - Cisco UCSC-C220-M3S - Intel Xeon E5

CentOS 7.9.2009 - 3.10.0-1160.6.1.el7.x86_64 - X Server

1 System - 107 Benchmark Results

2 x Intel Xeon Gold 6240 - Intel 440BX - Intel 440BX

OracleServer 8.3 - 5.4.17-2036.104.4.el8uek.x86_64 - vmware

1 System - 119 Benchmark Results

2 x Intel Xeon E5-2680 v3 - Cisco UCSB-B200-M4 - Intel Xeon E7 v3

OracleServer 7.9 - 5.4.17-2011.6.2.el7uek.x86_64 - matrox

1 System - 94 Benchmark Results

2 x Intel Xeon Gold 6240 - Intel 440BX - Intel 440BX

Amazon Linux 2 - 4.14.214-160.339.amzn2.x86_64 - vmware

1 System - 119 Benchmark Results

2 x Intel Xeon E5-2680 v3 - Cisco UCSB-B200-M4 - Intel Xeon E7 v3

OracleServer 7.9 - 5.4.17-2011.6.2.el7uek.x86_64 - matrox

1 System - 100 Benchmark Results

2 x Intel Xeon Gold 6240 - Intel 440BX - Intel 440BX

CentOS 7.9.2009 - 3.10.0-1160.11.1.el7.x86_64 - X Server

1 System - 118 Benchmark Results

2 x Intel Xeon E5-2680 0 - Cisco UCSC-C220-M3S - Intel Xeon E5

CentOS 7.9.2009 - 3.10.0-1160.11.1.el7.x86_64 - X Server

1 System - 117 Benchmark Results

2 x Intel Xeon E5-2680 0 - Cisco UCSC-C220-M3S - Intel Xeon E5

CentOS 7.4.1708 - 3.10.0-693.2.2.el7.x86_64 - X Server

1 System - 91 Benchmark Results

2 x Intel Xeon E5-2680 0 - Cisco UCSC-C220-M3S - Intel Xeon E5

CentOS 7.4.1708 - 3.10.0-693.2.2.el7.x86_64 - X Server

1 System - 118 Benchmark Results

2 x Intel Xeon E5-2680 0 - Cisco UCSC-C220-M3S - Intel Xeon E5

CentOS 7.9.2009 - 3.10.0-1160.11.1.el7.x86_64 - X Server

1 System - 140 Benchmark Results

2 x Intel Xeon E5-2680 v3 - Cisco UCSB-B200-M4 - Intel Xeon E7 v3

OracleServer 8.3 - 5.4.17-2011.7.4.el8uek.x86_64 - matrox

Most Popular Test Results

OpenBenchmarking.org Results Compare

2 Systems - 403 Benchmark Results

Intel Core i9-10900K - Gigabyte Z490 AORUS MASTER - Intel Comet Lake PCH

Ubuntu 20.10 - 5.8.0-22-generic - GNOME Shell 3.38.0

3 Systems - 406 Benchmark Results

AMD Ryzen 9 3900XT 12-Core - MSI MEG X570 GODLIKE - AMD Starship

Ubuntu 20.10 - 5.8.0-20-generic - GNOME Shell 3.38.0

2 Systems - 475 Benchmark Results

AMD Ryzen Threadripper 3970X 32-Core - ASUS ROG ZENITH II EXTREME - AMD Starship

Ubuntu 19.10 - 5.3.0-40-generic - GNOME Shell 3.34.1

1 System - 17 Benchmark Results

Intel Core i5-8400 - MSI Z370M MORTAR - Intel 8th Gen Core

Ubuntu 19.04 - 5.0.0-29-generic - GNOME Shell 3.32.1

2 Systems - 345 Benchmark Results

AMD Ryzen Threadripper 3960X 24-Core - MSI Creator TRX40 - AMD Starship

Ubuntu 19.10 - 5.3.0-23-generic - GNOME Shell 3.34.1

1 System - 376 Benchmark Results

AMD Ryzen 9 3900X 12-Core - ASUS ROG CROSSHAIR VIII HERO - AMD Starship

Ubuntu 19.10 - 5.3.0-10-generic - GNOME Shell 3.34.0

3 Systems - 25 Benchmark Results

2 x Intel Core - QEMU Standard PC - Intel 440FX 82441FX PMC

Ubuntu 20.04 - 5.4.0-52-generic - GCC 9.3.0

1 System - 10 Benchmark Results

AMD Ryzen 9 3900X 12-Core - ASUS ROG CROSSHAIR VIII HERO - AMD Starship

Ubuntu 19.10 - 5.3.0-10-generic - GNOME Shell 3.34.0

3 Systems - 335 Benchmark Results

2 x Intel Xeon E5-2687W v4 - Dell 0NK5PH - Intel Xeon E7 v4

Ubuntu 19.04 - 5.0.0-36-generic - GNOME Shell 3.32.2

2 Systems - 406 Benchmark Results

AMD Ryzen 9 3900XT 12-Core - MSI MEG X570 GODLIKE - AMD Starship

Ubuntu 20.10 - 5.8.0-20-generic - GNOME Shell 3.38.0

Find More Test Results