Apache Spark AMD EPYC

2 x AMD EPYC 7713 64-Core testing with a AMD DAYTONA_X (RYM1009B BIOS) and ASPEED on Ubuntu 22.04 via the Phoronix Test Suite.

HTML result view exported from: https://openbenchmarking.org/result/2208047-NE-APACHESPA20&sro&grs.

Apache Spark AMD EPYCProcessorMotherboardChipsetMemoryDiskGraphicsMonitorNetworkOSKernelDesktopDisplay ServerVulkanCompilerFile-SystemScreen ResolutionABC2 x AMD EPYC 7713 64-Core @ 2.00GHz (128 Cores / 256 Threads)AMD DAYTONA_X (RYM1009B BIOS)AMD Starship/Matisse512GB3841GB Micron_9300_MTFDHAL3T8TDPASPEEDVE2282 x Mellanox MT27710Ubuntu 22.045.19.0-051900daily20220803-generic (x86_64)GNOME Shell 42.2X Server 1.21.1.31.2.204GCC 11.2.0ext41920x1080OpenBenchmarking.orgKernel Details- Transparent Huge Pages: madviseProcessor Details- Scaling Governor: acpi-cpufreq performance (Boost: Enabled) - CPU Microcode: 0xa001173 Java Details- OpenJDK Runtime Environment (build 11.0.15+10-Ubuntu-0ubuntu0.22.04.1)Python Details- Python 3.10.4Security Details- itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional IBRS_FW STIBP: always-on RSB filling + srbds: Not affected + tsx_async_abort: Not affected

Apache Spark AMD EPYCspark: 10000000 - 2000 - Calculate Pi Benchmark Using Dataframespark: 1000000 - 500 - Calculate Pi Benchmark Using Dataframespark: 20000000 - 100 - Inner Join Test Timespark: 20000000 - 100 - Broadcast Inner Join Test Timespark: 1000000 - 500 - SHA-512 Benchmark Timespark: 1000000 - 100 - Inner Join Test Timespark: 10000000 - 2000 - SHA-512 Benchmark Timespark: 1000000 - 1000 - SHA-512 Benchmark Timespark: 20000000 - 100 - SHA-512 Benchmark Timespark: 10000000 - 100 - SHA-512 Benchmark Timespark: 20000000 - 2000 - SHA-512 Benchmark Timespark: 1000000 - 100 - Group By Test Timespark: 10000000 - 1000 - SHA-512 Benchmark Timespark: 10000000 - 500 - SHA-512 Benchmark Timespark: 20000000 - 100 - Group By Test Timespark: 1000000 - 2000 - SHA-512 Benchmark Timespark: 10000000 - 2000 - Calculate Pi Benchmarkspark: 1000000 - 100 - SHA-512 Benchmark Timespark: 20000000 - 1000 - SHA-512 Benchmark Timespark: 40000000 - 100 - SHA-512 Benchmark Timespark: 40000000 - 1000 - Calculate Pi Benchmarkspark: 40000000 - 1000 - SHA-512 Benchmark Timespark: 1000000 - 500 - Calculate Pi Benchmarkspark: 20000000 - 100 - Repartition Test Timespark: 40000000 - 500 - Calculate Pi Benchmarkspark: 20000000 - 500 - Calculate Pi Benchmarkspark: 40000000 - 2000 - SHA-512 Benchmark Timespark: 1000000 - 1000 - Calculate Pi Benchmarkspark: 40000000 - 2000 - Calculate Pi Benchmarkspark: 20000000 - 2000 - Calculate Pi Benchmarkspark: 1000000 - 2000 - Calculate Pi Benchmarkspark: 1000000 - 100 - Calculate Pi Benchmarkspark: 20000000 - 100 - Calculate Pi Benchmarkspark: 40000000 - 100 - Calculate Pi Benchmarkspark: 20000000 - 500 - SHA-512 Benchmark Timespark: 10000000 - 1000 - Calculate Pi Benchmarkspark: 40000000 - 500 - SHA-512 Benchmark Timespark: 10000000 - 100 - Calculate Pi Benchmarkspark: 20000000 - 1000 - Calculate Pi Benchmarkspark: 10000000 - 500 - Calculate Pi Benchmarkspark: 10000000 - 2000 - Broadcast Inner Join Test Timespark: 10000000 - 2000 - Inner Join Test Timespark: 10000000 - 2000 - Repartition Test Timespark: 10000000 - 2000 - Group By Test Timespark: 1000000 - 500 - Broadcast Inner Join Test Timespark: 1000000 - 500 - Inner Join Test Timespark: 1000000 - 500 - Repartition Test Timespark: 1000000 - 500 - Group By Test Timespark: 40000000 - 2000 - Calculate Pi Benchmark Using Dataframespark: 40000000 - 1000 - Calculate Pi Benchmark Using Dataframespark: 20000000 - 2000 - Calculate Pi Benchmark Using Dataframespark: 20000000 - 1000 - Calculate Pi Benchmark Using Dataframespark: 10000000 - 1000 - Calculate Pi Benchmark Using Dataframespark: 40000000 - 500 - Calculate Pi Benchmark Using Dataframespark: 40000000 - 100 - Calculate Pi Benchmark Using Dataframespark: 20000000 - 500 - Calculate Pi Benchmark Using Dataframespark: 20000000 - 100 - Calculate Pi Benchmark Using Dataframespark: 10000000 - 500 - Calculate Pi Benchmark Using Dataframespark: 10000000 - 100 - Broadcast Inner Join Test Timespark: 10000000 - 100 - Inner Join Test Timespark: 10000000 - 100 - Repartition Test Timespark: 10000000 - 100 - Group By Test Timespark: 10000000 - 100 - Calculate Pi Benchmark Using Dataframespark: 1000000 - 2000 - Broadcast Inner Join Test Timespark: 1000000 - 2000 - Inner Join Test Timespark: 1000000 - 2000 - Repartition Test Timespark: 1000000 - 2000 - Group By Test Timespark: 1000000 - 2000 - Calculate Pi Benchmark Using Dataframespark: 1000000 - 1000 - Broadcast Inner Join Test Timespark: 1000000 - 1000 - Inner Join Test Timespark: 1000000 - 1000 - Repartition Test Timespark: 1000000 - 1000 - Group By Test Timespark: 1000000 - 1000 - Calculate Pi Benchmark Using Dataframespark: 1000000 - 100 - Broadcast Inner Join Test Timespark: 1000000 - 100 - Repartition Test Timespark: 1000000 - 100 - Calculate Pi Benchmark Using DataframeABC3.343.185.932.9213.195.9118.8311.6020.966.5312.9113.295.9117.484.5420.9534.9617.3335.1817.4717.6717.4535.7817.4517.6317.4217.5217.5817.3817.5420.3717.4435.2617.4817.4717.452.042.903.673.343.543.383.443.444.253.805.818.517.2812.323.514.427.186.0411.053.213.906.585.4911.953.381.862.943.673.223.2214.2512.026.013.1713.096.3619.2011.6320.476.7813.0912.8537.616.1617.424.6220.6535.1217.3135.5517.4913.4817.4117.6435.3617.5917.5417.3017.5517.5317.4017.5120.4617.5435.0317.3717.5017.498.1812.7614.4418.194.495.594.8715.623.433.553.623.573.223.283.093.823.393.316.239.128.3713.263.684.047.345.9810.963.643.976.936.2514.252.461.842.753.124.092.6912.7610.876.063.2112.626.0819.3312.2421.746.9112.5312.72350245839.265.9117.804.4521.4233.9417.8436.2117.1713.8617.2817.2635.0717.8017.3117.6117.6917.7517.1917.7120.2617.6135.2817.37897353517.4317.484.367029263.803.332.562.893.972.533.403.403.086.039.278.1612.672.044.447.478.5111.863.254.137.016.1611.053.031.992.913.48OpenBenchmarking.org

Apache Spark

Row Count: 10000000 - Partitions: 2000 - Calculate Pi Benchmark Using Dataframe

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 2000 - Calculate Pi Benchmark Using DataframeABC0.92031.84062.76093.68124.6015SE +/- 0.22, N = 15SE +/- 0.18, N = 153.343.222.56

Apache Spark

Row Count: 1000000 - Partitions: 500 - Calculate Pi Benchmark Using Dataframe

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 500 - Calculate Pi Benchmark Using DataframeABC0.85281.70562.55843.41124.264SE +/- 0.16, N = 15SE +/- 0.16, N = 153.183.222.69

Apache Spark

Row Count: 20000000 - Partitions: 100 - Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 100 - Inner Join Test TimeBC48121620SE +/- 0.76, N = 214.2512.76

Apache Spark

Row Count: 20000000 - Partitions: 100 - Broadcast Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 100 - Broadcast Inner Join Test TimeBC3691215SE +/- 0.81, N = 212.0210.87

Apache Spark

Row Count: 1000000 - Partitions: 500 - SHA-512 Benchmark Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 500 - SHA-512 Benchmark TimeABC246810SE +/- 0.09, N = 15SE +/- 0.09, N = 155.936.015.54

Apache Spark

Row Count: 1000000 - Partitions: 100 - Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 100 - Inner Join Test TimeABC0.72231.44462.16692.88923.6115SE +/- 0.04, N = 3SE +/- 0.03, N = 42.923.173.21

Apache Spark

Row Count: 10000000 - Partitions: 2000 - SHA-512 Benchmark Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 2000 - SHA-512 Benchmark TimeABC48121620SE +/- 0.15, N = 15SE +/- 0.12, N = 1513.1913.0912.62

Apache Spark

Row Count: 1000000 - Partitions: 1000 - SHA-512 Benchmark Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 1000 - SHA-512 Benchmark TimeABC246810SE +/- 0.06, N = 15SE +/- 0.06, N = 35.916.366.08

Apache Spark

Row Count: 20000000 - Partitions: 100 - SHA-512 Benchmark Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 100 - SHA-512 Benchmark TimeABC510152025SE +/- 0.20, N = 3SE +/- 0.17, N = 1218.8319.2019.33

Apache Spark

Row Count: 10000000 - Partitions: 100 - SHA-512 Benchmark Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 100 - SHA-512 Benchmark TimeABC3691215SE +/- 0.09, N = 3SE +/- 0.11, N = 1511.6011.6311.50

Apache Spark

Row Count: 20000000 - Partitions: 2000 - SHA-512 Benchmark Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 2000 - SHA-512 Benchmark TimeABC510152025SE +/- 0.18, N = 12SE +/- 0.25, N = 320.9620.4721.74

Apache Spark

Row Count: 1000000 - Partitions: 100 - Group By Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 100 - Group By Test TimeABC246810SE +/- 0.04, N = 3SE +/- 0.02, N = 46.536.786.91

Apache Spark

Row Count: 10000000 - Partitions: 1000 - SHA-512 Benchmark Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 1000 - SHA-512 Benchmark TimeABC3691215SE +/- 0.10, N = 12SE +/- 0.15, N = 1212.9113.0912.53

Apache Spark

Row Count: 10000000 - Partitions: 500 - SHA-512 Benchmark Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 500 - SHA-512 Benchmark TimeABC3691215SE +/- 0.15, N = 3SE +/- 0.13, N = 513.2912.8512.72

Apache Spark

Row Count: 20000000 - Partitions: 100 - Group By Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 100 - Group By Test TimeBC918273645SE +/- 3.13, N = 237.6139.26

Apache Spark

Row Count: 1000000 - Partitions: 2000 - SHA-512 Benchmark Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 2000 - SHA-512 Benchmark TimeABC246810SE +/- 0.07, N = 15SE +/- 0.05, N = 85.916.165.91

Apache Spark

Row Count: 10000000 - Partitions: 2000 - Calculate Pi Benchmark

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 2000 - Calculate Pi BenchmarkABC48121620SE +/- 0.05, N = 15SE +/- 0.07, N = 1517.4817.4217.11

Apache Spark

Row Count: 1000000 - Partitions: 100 - SHA-512 Benchmark Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 100 - SHA-512 Benchmark TimeABC1.03952.0793.11854.1585.1975SE +/- 0.03, N = 3SE +/- 0.06, N = 44.544.624.45

Apache Spark

Row Count: 20000000 - Partitions: 1000 - SHA-512 Benchmark Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 1000 - SHA-512 Benchmark TimeABC510152025SE +/- 0.23, N = 5SE +/- 0.28, N = 320.9520.6521.42

Apache Spark

Row Count: 40000000 - Partitions: 100 - SHA-512 Benchmark Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 100 - SHA-512 Benchmark TimeABC816243240SE +/- 0.27, N = 12SE +/- 0.38, N = 934.9635.1233.94

Apache Spark

Row Count: 40000000 - Partitions: 1000 - Calculate Pi Benchmark

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 1000 - Calculate Pi BenchmarkABC48121620SE +/- 0.07, N = 5SE +/- 0.01, N = 317.3317.3117.84

Apache Spark

Row Count: 40000000 - Partitions: 1000 - SHA-512 Benchmark Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 1000 - SHA-512 Benchmark TimeABC816243240SE +/- 0.38, N = 5SE +/- 0.50, N = 335.1835.5536.21

Apache Spark

Row Count: 1000000 - Partitions: 500 - Calculate Pi Benchmark

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 500 - Calculate Pi BenchmarkABC48121620SE +/- 0.07, N = 15SE +/- 0.07, N = 1517.4717.4917.17

Apache Spark

Row Count: 20000000 - Partitions: 100 - Repartition Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 100 - Repartition Test TimeBC48121620SE +/- 0.08, N = 213.4813.86

Apache Spark

Row Count: 40000000 - Partitions: 500 - Calculate Pi Benchmark

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 500 - Calculate Pi BenchmarkABC48121620SE +/- 0.09, N = 3SE +/- 0.07, N = 517.6717.4117.28

Apache Spark

Row Count: 20000000 - Partitions: 500 - Calculate Pi Benchmark

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 500 - Calculate Pi BenchmarkABC48121620SE +/- 0.18, N = 3SE +/- 0.10, N = 317.4517.6417.26

Apache Spark

Row Count: 40000000 - Partitions: 2000 - SHA-512 Benchmark Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 2000 - SHA-512 Benchmark TimeABC816243240SE +/- 0.09, N = 3SE +/- 0.39, N = 535.7835.3635.07

Apache Spark

Row Count: 1000000 - Partitions: 1000 - Calculate Pi Benchmark

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 1000 - Calculate Pi BenchmarkABC48121620SE +/- 0.06, N = 15SE +/- 0.19, N = 317.4517.5917.80

Apache Spark

Row Count: 40000000 - Partitions: 2000 - Calculate Pi Benchmark

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 2000 - Calculate Pi BenchmarkABC48121620SE +/- 0.16, N = 3SE +/- 0.10, N = 517.6317.5417.31

Apache Spark

Row Count: 20000000 - Partitions: 2000 - Calculate Pi Benchmark

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 2000 - Calculate Pi BenchmarkABC48121620SE +/- 0.06, N = 12SE +/- 0.06, N = 317.4217.3017.61

Apache Spark

Row Count: 1000000 - Partitions: 2000 - Calculate Pi Benchmark

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 2000 - Calculate Pi BenchmarkABC48121620SE +/- 0.07, N = 15SE +/- 0.09, N = 817.5217.5517.46

Apache Spark

Row Count: 1000000 - Partitions: 100 - Calculate Pi Benchmark

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 100 - Calculate Pi BenchmarkABC48121620SE +/- 0.20, N = 3SE +/- 0.13, N = 417.5817.5317.75

Apache Spark

Row Count: 20000000 - Partitions: 100 - Calculate Pi Benchmark

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 100 - Calculate Pi BenchmarkABC48121620SE +/- 0.16, N = 3SE +/- 0.06, N = 1217.3817.4017.19

Apache Spark

Row Count: 40000000 - Partitions: 100 - Calculate Pi Benchmark

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 100 - Calculate Pi BenchmarkABC48121620SE +/- 0.06, N = 12SE +/- 0.10, N = 917.5417.5117.71

Apache Spark

Row Count: 20000000 - Partitions: 500 - SHA-512 Benchmark Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 500 - SHA-512 Benchmark TimeABC510152025SE +/- 0.21, N = 3SE +/- 0.13, N = 320.3720.4620.26

Apache Spark

Row Count: 10000000 - Partitions: 1000 - Calculate Pi Benchmark

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 1000 - Calculate Pi BenchmarkABC48121620SE +/- 0.07, N = 12SE +/- 0.13, N = 1217.4417.5417.61

Apache Spark

Row Count: 40000000 - Partitions: 500 - SHA-512 Benchmark Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 500 - SHA-512 Benchmark TimeABC816243240SE +/- 0.33, N = 3SE +/- 0.38, N = 535.2635.0335.28

Apache Spark

Row Count: 10000000 - Partitions: 100 - Calculate Pi Benchmark

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 100 - Calculate Pi BenchmarkABC48121620SE +/- 0.11, N = 3SE +/- 0.05, N = 1517.4817.3717.38

Apache Spark

Row Count: 20000000 - Partitions: 1000 - Calculate Pi Benchmark

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 1000 - Calculate Pi BenchmarkABC48121620SE +/- 0.11, N = 5SE +/- 0.12, N = 317.4717.5017.43

Apache Spark

Row Count: 10000000 - Partitions: 500 - Calculate Pi Benchmark

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 500 - Calculate Pi BenchmarkABC48121620SE +/- 0.11, N = 3SE +/- 0.13, N = 517.4517.4917.48

Apache Spark

Row Count: 10000000 - Partitions: 2000 - Broadcast Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 2000 - Broadcast Inner Join Test TimeB246810SE +/- 0.39, N = 28.18

Apache Spark

Row Count: 10000000 - Partitions: 2000 - Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 2000 - Inner Join Test TimeB3691215SE +/- 1.47, N = 212.76

Apache Spark

Row Count: 10000000 - Partitions: 2000 - Repartition Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 2000 - Repartition Test TimeB48121620SE +/- 0.97, N = 214.44

Apache Spark

Row Count: 10000000 - Partitions: 2000 - Group By Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 2000 - Group By Test TimeB48121620SE +/- 0.52, N = 218.19

Apache Spark

Row Count: 1000000 - Partitions: 500 - Broadcast Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 500 - Broadcast Inner Join Test TimeB1.01032.02063.03094.04125.0515SE +/- 0.46, N = 24.49

Apache Spark

Row Count: 1000000 - Partitions: 500 - Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 500 - Inner Join Test TimeB1.25782.51563.77345.03126.289SE +/- 0.17, N = 25.59

Apache Spark

Row Count: 1000000 - Partitions: 500 - Repartition Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 500 - Repartition Test TimeB1.09582.19163.28744.38325.479SE +/- 0.07, N = 24.87

Apache Spark

Row Count: 1000000 - Partitions: 500 - Group By Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 500 - Group By Test TimeB48121620SE +/- 4.11, N = 215.62

Apache Spark

Row Count: 40000000 - Partitions: 2000 - Calculate Pi Benchmark Using Dataframe

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 2000 - Calculate Pi Benchmark Using DataframeABC0.98261.96522.94783.93044.913SE +/- 0.01771345, N = 3SE +/- 0.23677806, N = 52.040000003.430000004.36702926

Apache Spark

Row Count: 40000000 - Partitions: 1000 - Calculate Pi Benchmark Using Dataframe

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 1000 - Calculate Pi Benchmark Using DataframeABC0.8551.712.5653.424.275SE +/- 0.35, N = 5SE +/- 0.36, N = 32.903.553.80

Apache Spark

Row Count: 20000000 - Partitions: 2000 - Calculate Pi Benchmark Using Dataframe

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 2000 - Calculate Pi Benchmark Using DataframeABC0.82581.65162.47743.30324.129SE +/- 0.23, N = 12SE +/- 0.57, N = 33.673.623.33

Apache Spark

Row Count: 20000000 - Partitions: 1000 - Calculate Pi Benchmark Using Dataframe

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 1000 - Calculate Pi Benchmark Using DataframeABC0.80331.60662.40993.21324.0165SE +/- 0.33, N = 5SE +/- 0.29, N = 33.343.572.56

Apache Spark

Row Count: 10000000 - Partitions: 1000 - Calculate Pi Benchmark Using Dataframe

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 1000 - Calculate Pi Benchmark Using DataframeABC0.79651.5932.38953.1863.9825SE +/- 0.19, N = 12SE +/- 0.15, N = 123.543.222.89

Apache Spark

Row Count: 40000000 - Partitions: 500 - Calculate Pi Benchmark Using Dataframe

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 500 - Calculate Pi Benchmark Using DataframeABC0.89331.78662.67993.57324.4665SE +/- 0.34, N = 3SE +/- 0.36, N = 53.383.283.97

Apache Spark

Row Count: 40000000 - Partitions: 100 - Calculate Pi Benchmark Using Dataframe

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 100 - Calculate Pi Benchmark Using DataframeABC0.7741.5482.3223.0963.87SE +/- 0.20, N = 12SE +/- 0.24, N = 93.443.092.53

Apache Spark

Row Count: 20000000 - Partitions: 500 - Calculate Pi Benchmark Using Dataframe

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 500 - Calculate Pi Benchmark Using DataframeABC0.85951.7192.57853.4384.2975SE +/- 0.33, N = 3SE +/- 0.64, N = 33.443.823.40

Apache Spark

Row Count: 20000000 - Partitions: 100 - Calculate Pi Benchmark Using Dataframe

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 100 - Calculate Pi Benchmark Using DataframeABC0.95631.91262.86893.82524.7815SE +/- 0.03, N = 3SE +/- 0.21, N = 124.253.392.90

Apache Spark

Row Count: 10000000 - Partitions: 500 - Calculate Pi Benchmark Using Dataframe

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 500 - Calculate Pi Benchmark Using DataframeABC0.8551.712.5653.424.275SE +/- 0.30, N = 3SE +/- 0.46, N = 53.803.313.08

Apache Spark

Row Count: 10000000 - Partitions: 100 - Broadcast Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 100 - Broadcast Inner Join Test TimeABC246810SE +/- 0.24, N = 3SE +/- 0.13, N = 115.816.236.03

Apache Spark

Row Count: 10000000 - Partitions: 100 - Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 100 - Inner Join Test TimeABC3691215SE +/- 0.09, N = 3SE +/- 0.18, N = 118.519.129.27

Apache Spark

Row Count: 10000000 - Partitions: 100 - Repartition Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 100 - Repartition Test TimeABC246810SE +/- 0.41, N = 3SE +/- 0.16, N = 117.288.378.16

Apache Spark

Row Count: 10000000 - Partitions: 100 - Group By Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 100 - Group By Test TimeABC3691215SE +/- 0.23, N = 3SE +/- 0.27, N = 1112.3213.2612.67

Apache Spark

Row Count: 10000000 - Partitions: 100 - Calculate Pi Benchmark Using Dataframe

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 100 - Calculate Pi Benchmark Using DataframeABC0.8281.6562.4843.3124.14SE +/- 0.13, N = 3SE +/- 0.13, N = 153.513.682.04

Apache Spark

Row Count: 1000000 - Partitions: 2000 - Broadcast Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 2000 - Broadcast Inner Join Test TimeABC0.9991.9982.9973.9964.995SE +/- 0.11, N = 11SE +/- 0.12, N = 64.424.044.44

Apache Spark

Row Count: 1000000 - Partitions: 2000 - Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 2000 - Inner Join Test TimeABC246810SE +/- 0.22, N = 11SE +/- 0.23, N = 67.187.347.47

Apache Spark

Row Count: 1000000 - Partitions: 2000 - Repartition Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 2000 - Repartition Test TimeABC246810SE +/- 0.22, N = 11SE +/- 0.25, N = 66.045.988.51

Apache Spark

Row Count: 1000000 - Partitions: 2000 - Group By Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 2000 - Group By Test TimeABC3691215SE +/- 0.24, N = 11SE +/- 0.32, N = 611.0510.9611.86

Apache Spark

Row Count: 1000000 - Partitions: 2000 - Calculate Pi Benchmark Using Dataframe

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 2000 - Calculate Pi Benchmark Using DataframeABC0.8191.6382.4573.2764.095SE +/- 0.18, N = 15SE +/- 0.27, N = 83.213.643.25

Apache Spark

Row Count: 1000000 - Partitions: 1000 - Broadcast Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 1000 - Broadcast Inner Join Test TimeABC0.92931.85862.78793.71724.6465SE +/- 0.10, N = 9SE +/- 0.30, N = 23.903.974.13

Apache Spark

Row Count: 1000000 - Partitions: 1000 - Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 1000 - Inner Join Test TimeABC246810SE +/- 0.25, N = 9SE +/- 0.77, N = 26.586.937.01

Apache Spark

Row Count: 1000000 - Partitions: 1000 - Repartition Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 1000 - Repartition Test TimeABC246810SE +/- 0.16, N = 9SE +/- 0.43, N = 25.496.256.16

Apache Spark

Row Count: 1000000 - Partitions: 1000 - Group By Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 1000 - Group By Test TimeABC48121620SE +/- 0.70, N = 9SE +/- 2.86, N = 211.9514.2511.05

Apache Spark

Row Count: 1000000 - Partitions: 1000 - Calculate Pi Benchmark Using Dataframe

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 1000 - Calculate Pi Benchmark Using DataframeABC0.76051.5212.28153.0423.8025SE +/- 0.21, N = 15SE +/- 0.30, N = 33.382.463.03

Apache Spark

Row Count: 1000000 - Partitions: 100 - Broadcast Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 100 - Broadcast Inner Join Test TimeABC0.44780.89561.34341.79122.239SE +/- 0.08, N = 3SE +/- 0.05, N = 41.861.841.99

Apache Spark

Row Count: 1000000 - Partitions: 100 - Repartition Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 100 - Repartition Test TimeABC0.66151.3231.98452.6463.3075SE +/- 0.12, N = 3SE +/- 0.07, N = 42.942.752.91

Apache Spark

Row Count: 1000000 - Partitions: 100 - Calculate Pi Benchmark Using Dataframe

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 100 - Calculate Pi Benchmark Using DataframeABC0.82581.65162.47743.30324.129SE +/- 0.32, N = 3SE +/- 0.31, N = 43.673.123.48

Phoronix Test Suite v10.8.5