spark 12900K Intel Intel Core i9-12900K testing with a ASUS ROG STRIX Z690-E GAMING WIFI (1601 BIOS) and AMD Radeon RX 6800 XT 16GB on Ubuntu 22.04 via the Phoronix Test Suite.
HTML result view exported from: https://openbenchmarking.org/result/2208053-NE-SPARK129078&grs .
spark 12900K Intel Processor Motherboard Chipset Memory Disk Graphics Audio Monitor Network OS Kernel Desktop Display Server OpenGL Vulkan Compiler File-System Screen Resolution A B C D Intel Core i9-12900K @ 5.20GHz (16 Cores / 24 Threads) ASUS ROG STRIX Z690-E GAMING WIFI (1601 BIOS) Intel Alder Lake-S PCH 32GB 1000GB Western Digital WDS100T1X0E-00AFY0 + 2000GB AMD Radeon RX 6800 XT 16GB (2575/1000MHz) Intel Alder Lake-S HD Audio ASUS VP28U Intel I225-V + Intel Wi-Fi 6 AX210/AX211/AX411 Ubuntu 22.04 5.19.0-051900-generic (x86_64) GNOME Shell 42.2 X Server 1.21.1.3 + Wayland 4.6 Mesa 22.3.0-devel (git-42a52a8 2022-08-04 jammy-oibaf-ppa) (LLVM 14.0.6 DRM 3.47) 1.3.219 GCC 11.2.0 ext4 3840x2160 OpenBenchmarking.org Kernel Details - Transparent Huge Pages: madvise Processor Details - Scaling Governor: intel_pstate performance (EPP: performance) - CPU Microcode: 0x1f - Thermald 2.4.9 Java Details - OpenJDK Runtime Environment (build 11.0.15+10-Ubuntu-0ubuntu0.22.04.1) Python Details - Python 3.10.4 Security Details - itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Enhanced IBRS IBPB: conditional RSB filling + srbds: Not affected + tsx_async_abort: Not affected
spark 12900K Intel spark: 20000000 - 2000 - Repartition Test Time spark: 1000000 - 2000 - SHA-512 Benchmark Time spark: 1000000 - 2000 - Group By Test Time spark: 1000000 - 2000 - Broadcast Inner Join Test Time spark: 20000000 - 2000 - SHA-512 Benchmark Time spark: 20000000 - 2000 - Broadcast Inner Join Test Time spark: 1000000 - 2000 - Inner Join Test Time spark: 20000000 - 2000 - Inner Join Test Time spark: 20000000 - 2000 - Group By Test Time spark: 1000000 - 1000 - Broadcast Inner Join Test Time spark: 1000000 - 2000 - Repartition Test Time spark: 40000000 - 2000 - Group By Test Time spark: 20000000 - 500 - Calculate Pi Benchmark Using Dataframe spark: 1000000 - 100 - Broadcast Inner Join Test Time spark: 10000000 - 500 - Inner Join Test Time spark: 20000000 - 1000 - Broadcast Inner Join Test Time spark: 1000000 - 1000 - Calculate Pi Benchmark Using Dataframe spark: 20000000 - 100 - Inner Join Test Time spark: 10000000 - 2000 - Calculate Pi Benchmark Using Dataframe spark: 10000000 - 100 - Broadcast Inner Join Test Time spark: 40000000 - 100 - Calculate Pi Benchmark Using Dataframe spark: 1000000 - 1000 - SHA-512 Benchmark Time spark: 10000000 - 100 - Inner Join Test Time spark: 40000000 - 500 - Calculate Pi Benchmark Using Dataframe spark: 1000000 - 500 - Calculate Pi Benchmark Using Dataframe spark: 1000000 - 100 - Inner Join Test Time spark: 1000000 - 2000 - Calculate Pi Benchmark Using Dataframe spark: 20000000 - 500 - SHA-512 Benchmark Time spark: 40000000 - 1000 - Calculate Pi Benchmark Using Dataframe spark: 10000000 - 100 - SHA-512 Benchmark Time spark: 10000000 - 1000 - Calculate Pi Benchmark Using Dataframe spark: 10000000 - 500 - Broadcast Inner Join Test Time spark: 20000000 - 100 - Calculate Pi Benchmark Using Dataframe spark: 1000000 - 1000 - Inner Join Test Time spark: 10000000 - 100 - Calculate Pi Benchmark Using Dataframe spark: 10000000 - 500 - Repartition Test Time spark: 20000000 - 500 - Group By Test Time spark: 40000000 - 2000 - Repartition Test Time spark: 10000000 - 100 - Repartition Test Time spark: 20000000 - 500 - Repartition Test Time spark: 10000000 - 1000 - Broadcast Inner Join Test Time spark: 40000000 - 500 - Inner Join Test Time spark: 10000000 - 1000 - Inner Join Test Time spark: 1000000 - 1000 - Repartition Test Time spark: 20000000 - 100 - SHA-512 Benchmark Time spark: 10000000 - 500 - Group By Test Time spark: 1000000 - 500 - Inner Join Test Time spark: 1000000 - 100 - SHA-512 Benchmark Time spark: 40000000 - 1000 - Repartition Test Time spark: 20000000 - 2000 - Calculate Pi Benchmark Using Dataframe spark: 10000000 - 2000 - Repartition Test Time spark: 10000000 - 100 - Group By Test Time spark: 10000000 - 1000 - Repartition Test Time spark: 40000000 - 1000 - Inner Join Test Time spark: 40000000 - 500 - SHA-512 Benchmark Time spark: 10000000 - 2000 - Inner Join Test Time spark: 20000000 - 1000 - Repartition Test Time spark: 20000000 - 500 - Broadcast Inner Join Test Time spark: 20000000 - 1000 - Group By Test Time spark: 1000000 - 500 - Repartition Test Time spark: 20000000 - 500 - Inner Join Test Time spark: 10000000 - 2000 - SHA-512 Benchmark Time spark: 40000000 - 2000 - SHA-512 Benchmark Time spark: 40000000 - 1000 - Group By Test Time spark: 20000000 - 1000 - Inner Join Test Time spark: 1000000 - 500 - Broadcast Inner Join Test Time spark: 40000000 - 2000 - Calculate Pi Benchmark Using Dataframe spark: 10000000 - 1000 - Group By Test Time spark: 10000000 - 500 - Calculate Pi Benchmark Using Dataframe spark: 40000000 - 100 - Inner Join Test Time spark: 40000000 - 500 - Group By Test Time spark: 1000000 - 500 - SHA-512 Benchmark Time spark: 10000000 - 500 - Calculate Pi Benchmark spark: 20000000 - 100 - Repartition Test Time spark: 1000000 - 2000 - Calculate Pi Benchmark spark: 1000000 - 100 - Repartition Test Time spark: 40000000 - 500 - Repartition Test Time spark: 20000000 - 1000 - SHA-512 Benchmark Time spark: 40000000 - 100 - Broadcast Inner Join Test Time spark: 40000000 - 2000 - Calculate Pi Benchmark spark: 40000000 - 2000 - Inner Join Test Time spark: 1000000 - 500 - Group By Test Time spark: 10000000 - 500 - SHA-512 Benchmark Time spark: 40000000 - 1000 - Calculate Pi Benchmark spark: 1000000 - 1000 - Group By Test Time spark: 40000000 - 500 - Broadcast Inner Join Test Time spark: 40000000 - 100 - Group By Test Time spark: 10000000 - 2000 - Broadcast Inner Join Test Time spark: 20000000 - 100 - Broadcast Inner Join Test Time spark: 1000000 - 100 - Calculate Pi Benchmark spark: 40000000 - 1000 - Broadcast Inner Join Test Time spark: 1000000 - 100 - Group By Test Time spark: 10000000 - 2000 - Calculate Pi Benchmark spark: 1000000 - 100 - Calculate Pi Benchmark Using Dataframe spark: 10000000 - 1000 - Calculate Pi Benchmark spark: 1000000 - 1000 - Calculate Pi Benchmark spark: 10000000 - 1000 - SHA-512 Benchmark Time spark: 20000000 - 1000 - Calculate Pi Benchmark Using Dataframe spark: 10000000 - 100 - Calculate Pi Benchmark spark: 10000000 - 2000 - Group By Test Time spark: 40000000 - 100 - Calculate Pi Benchmark spark: 40000000 - 1000 - SHA-512 Benchmark Time spark: 20000000 - 500 - Calculate Pi Benchmark spark: 20000000 - 100 - Calculate Pi Benchmark spark: 40000000 - 100 - Repartition Test Time spark: 1000000 - 500 - Calculate Pi Benchmark spark: 20000000 - 100 - Group By Test Time spark: 20000000 - 1000 - Calculate Pi Benchmark spark: 40000000 - 500 - Calculate Pi Benchmark spark: 20000000 - 2000 - Calculate Pi Benchmark spark: 40000000 - 100 - SHA-512 Benchmark Time spark: 40000000 - 2000 - Broadcast Inner Join Test Time A B C D 1.16 2.63 3.21 1.28 0.54 0.06 1.54 0.11 0.69 1.49 1.63 14.768565164 5.03 0.82 6.21 10.55 4.59 12.67 5.09 6.28 4.67 2.56 6.13 4.94 5.07 0.97 4.75 14.207804304 5.01 8.16 5.06 5.82 5.02 1.22 5.08 5.10 7.74 18.42 5.40 9.88 5.62 21.138411296 5.97 1.45 14.326420697 4.65 1.01 2.12 18.684138822 4.82 5.19 4.98 5.06 21.878095867 25.264777819 6.08 9.46 12.24 6.80 1.31 11.799657537 7.84 24.786136683 15.40 11.371578033 0.88 4.99 4.96 5.08 23.04 17.12 2.29 87.28 10.098177409 84.51 1.25 18.475092188 13.43 23.81 85.35 21.83 2.79 7.40 85.271494834 2.94 21.252051853 19.14 6.07 12.40159508 81.45 22.41 2.71 86.821172323 5.04 86.28 84.361830344 7.73 5.00 85.75 5.00 84.931547925 24.94 85.69 85.139703903 19.24 84.800545422 7.65 84.83 84.63 85.122994156 27.24 21.99 9.79 0.53 0.67 0.05 13.68 11.60 0.11 11.96 7.32 1.12 1.16 14.72 4.97 0.84 5.79 11.72 4.96 11.890685683 4.66 6.85 5.04 2.48 6.523776991 5.05 4.763385925 1.05 5.13 15.29 5.06 8.59 5.080638849 5.85 4.70 1.27 5.03 4.81 7.84 17.55 5.25 9.62 5.33 20.691751412 5.68 1.42 14.55 4.43 1.000558709 2.21 18.13 4.93 5.17 4.99 5.02 21.867596614 24.356403173 6.12 9.21 12.32 6.95 1.36 11.847774087 7.80 24.712695848 15.775853179 11.15 0.91 4.92 4.92 5.03 23.13 16.84 2.26 86.093697249 10.20 85.68 1.22 18.16 13.24 24.26 84.85 22.01 2.75 7.46 86.57 2.92 21.19 18.842812858 5.95 12.64129401 82.77 22.35 2.74 85.28 5.01 86.02 85.47 7.77 5.07 85.312196129 4.97 85.669825169 24.97 85.090602077 85.48 19.299186496 84.4113598 7.66 85.352606409 85.03 84.888997196 27.097468226 21.94 9.57 2.65 3.05 1.33 13.70550217 11.80 1.54 11.67 7.31 0.99 1.59 16.90 4.48 0.86 5.58 10.74 4.94 12.25 5.03 6.55 4.72 2.58 6.33 4.70 4.71 0.98 5.02 14.34 4.69 8.00 4.98 5.49 4.80 1.28 4.78 4.96 7.52 18.5559665 5.39 10.05 5.35 21.78 5.74 1.38 14.37 4.63 1.05 2.21 18.08 5.03 5.39 5.04 5.23 21.260827487 24.30 6.32 9.47 12.49 7.06 1.34 11.43 7.69 24.70 15.24 11.073977287 0.89 4.89 5.00 4.96 22.91 16.989188878 2.32 85.04 10.07 85.772644515 1.25 18.600499237 13.12 24.27 85.55 22.09 2.81 7.42 85.71 2.98 21.30 18.76 6.05 12.51 82.472849923 22.00 2.76 86.04 5.10 85.82 84.46 7.78 4.99 86.346556539 4.98 84.93 24.97 85.02 84.84 19.27 85.013937598 7.61 84.81 85.02 85.131798659 27.10 21.98 9.39 2.54 3.14 1.31 13.68 10.75 1.52 11.16 7.14 1.04 1.62 15.25 4.94 0.92 6.17 10.60 4.50 11.58 4.92 6.45 5.08 2.38 6.02 5.09 5.10 1.02 5.06 14.17 4.80 7.97 4.72 5.90 5.03 1.20 5.07 4.96 7.41 18.33 5.11 9.52 5.39 21.55 5.79 1.43 15.04 4.50 1.02 2.22 17.87 4.97 5.21 5.19 5.16 22.14 24.31 6.17 9.57 12.03 6.82 1.36 11.75 7.97 25.579009038 15.58 10.99 0.91 5.04 5.07 5.11 23.55 16.67 2.30 85.675618513 9.95 86.593576543 1.24 18.28 13.28 23.71 86.843310376 22.33 2.77 7.56 84.79 2.96 21.62 18.87 6.02 12.43 82.97 22.38 2.72 85.51 5.10 84.81 85.81 7.65 4.99 86.067847462 5.03 85.22 25.156112784 85.14 85.094043096 19.16 84.44 7.65 85.07 85.13 85.345414321 27.11 21.93 OpenBenchmarking.org
Apache Spark Row Count: 20000000 - Partitions: 2000 - Repartition Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 2000 - Repartition Test Time A B C D 3 6 9 12 15 1.16 9.79 9.57 9.39
Apache Spark Row Count: 1000000 - Partitions: 2000 - SHA-512 Benchmark Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 2000 - SHA-512 Benchmark Time A B C D 0.5963 1.1926 1.7889 2.3852 2.9815 2.63 0.53 2.65 2.54
Apache Spark Row Count: 1000000 - Partitions: 2000 - Group By Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 2000 - Group By Test Time A B C D 0.7223 1.4446 2.1669 2.8892 3.6115 3.21 0.67 3.05 3.14
Apache Spark Row Count: 1000000 - Partitions: 2000 - Broadcast Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 2000 - Broadcast Inner Join Test Time A B C D 0.2993 0.5986 0.8979 1.1972 1.4965 1.28 0.05 1.33 1.31
Apache Spark Row Count: 20000000 - Partitions: 2000 - SHA-512 Benchmark Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 2000 - SHA-512 Benchmark Time A B C D 4 8 12 16 20 0.54000000 13.68000000 13.70550217 13.68000000
Apache Spark Row Count: 20000000 - Partitions: 2000 - Broadcast Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 2000 - Broadcast Inner Join Test Time A B C D 3 6 9 12 15 0.06 11.72 11.80 10.75
Apache Spark Row Count: 1000000 - Partitions: 2000 - Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 2000 - Inner Join Test Time A B C D 0.3465 0.693 1.0395 1.386 1.7325 1.54 0.11 1.54 1.52
Apache Spark Row Count: 20000000 - Partitions: 2000 - Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 2000 - Inner Join Test Time A B C D 3 6 9 12 15 0.11 11.83 11.67 11.16
Apache Spark Row Count: 20000000 - Partitions: 2000 - Group By Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 2000 - Group By Test Time A B C D 2 4 6 8 10 0.69 7.36 7.31 7.14
Apache Spark Row Count: 1000000 - Partitions: 1000 - Broadcast Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 1000 - Broadcast Inner Join Test Time A B C D 0.3353 0.6706 1.0059 1.3412 1.6765 1.49 1.12 0.99 1.04
Apache Spark Row Count: 1000000 - Partitions: 2000 - Repartition Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 2000 - Repartition Test Time A B C D 0.3668 0.7336 1.1004 1.4672 1.834 1.63 1.16 1.59 1.62
Apache Spark Row Count: 40000000 - Partitions: 2000 - Group By Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 2000 - Group By Test Time A B C D 4 8 12 16 20 14.77 14.72 16.90 15.25
Apache Spark Row Count: 20000000 - Partitions: 500 - Calculate Pi Benchmark Using Dataframe OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 500 - Calculate Pi Benchmark Using Dataframe A B C D 1.1318 2.2636 3.3954 4.5272 5.659 5.03 4.97 4.48 4.94
Apache Spark Row Count: 1000000 - Partitions: 100 - Broadcast Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 100 - Broadcast Inner Join Test Time A B C D 0.207 0.414 0.621 0.828 1.035 0.82 0.84 0.86 0.92
Apache Spark Row Count: 10000000 - Partitions: 500 - Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 500 - Inner Join Test Time A B C D 2 4 6 8 10 6.21 5.79 5.58 6.17
Apache Spark Row Count: 20000000 - Partitions: 1000 - Broadcast Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 1000 - Broadcast Inner Join Test Time A B C D 3 6 9 12 15 10.55 11.72 10.74 10.60
Apache Spark Row Count: 1000000 - Partitions: 1000 - Calculate Pi Benchmark Using Dataframe OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 1000 - Calculate Pi Benchmark Using Dataframe A B C D 1.116 2.232 3.348 4.464 5.58 4.59 4.96 4.94 4.50
Apache Spark Row Count: 20000000 - Partitions: 100 - Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 100 - Inner Join Test Time A B C D 3 6 9 12 15 12.67 11.89 12.25 11.58
Apache Spark Row Count: 10000000 - Partitions: 2000 - Calculate Pi Benchmark Using Dataframe OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 2000 - Calculate Pi Benchmark Using Dataframe A B C D 1.1453 2.2906 3.4359 4.5812 5.7265 5.09 4.66 5.03 4.92
Apache Spark Row Count: 10000000 - Partitions: 100 - Broadcast Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 100 - Broadcast Inner Join Test Time A B C D 2 4 6 8 10 6.28 6.85 6.55 6.45
Apache Spark Row Count: 40000000 - Partitions: 100 - Calculate Pi Benchmark Using Dataframe OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 100 - Calculate Pi Benchmark Using Dataframe A B C D 1.143 2.286 3.429 4.572 5.715 4.67 5.04 4.72 5.08
Apache Spark Row Count: 1000000 - Partitions: 1000 - SHA-512 Benchmark Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 1000 - SHA-512 Benchmark Time A B C D 0.5805 1.161 1.7415 2.322 2.9025 2.56 2.48 2.58 2.38
Apache Spark Row Count: 10000000 - Partitions: 100 - Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 100 - Inner Join Test Time A B C D 2 4 6 8 10 6.130000000 6.523776991 6.330000000 6.020000000
Apache Spark Row Count: 40000000 - Partitions: 500 - Calculate Pi Benchmark Using Dataframe OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 500 - Calculate Pi Benchmark Using Dataframe A B C D 1.1453 2.2906 3.4359 4.5812 5.7265 4.94 5.05 4.70 5.09
Apache Spark Row Count: 1000000 - Partitions: 500 - Calculate Pi Benchmark Using Dataframe OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 500 - Calculate Pi Benchmark Using Dataframe A B C D 1.1475 2.295 3.4425 4.59 5.7375 5.070000000 4.763385925 4.710000000 5.100000000
Apache Spark Row Count: 1000000 - Partitions: 100 - Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 100 - Inner Join Test Time A B C D 0.2363 0.4726 0.7089 0.9452 1.1815 0.97 1.05 0.98 1.02
Apache Spark Row Count: 1000000 - Partitions: 2000 - Calculate Pi Benchmark Using Dataframe OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 2000 - Calculate Pi Benchmark Using Dataframe A B C D 1.1543 2.3086 3.4629 4.6172 5.7715 4.75 5.13 5.02 5.06
Apache Spark Row Count: 20000000 - Partitions: 500 - SHA-512 Benchmark Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 500 - SHA-512 Benchmark Time A B C D 4 8 12 16 20 14.21 15.29 14.34 14.17
Apache Spark Row Count: 40000000 - Partitions: 1000 - Calculate Pi Benchmark Using Dataframe OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 1000 - Calculate Pi Benchmark Using Dataframe A B C D 1.1385 2.277 3.4155 4.554 5.6925 5.01 5.06 4.69 4.80
Apache Spark Row Count: 10000000 - Partitions: 100 - SHA-512 Benchmark Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 100 - SHA-512 Benchmark Time A B C D 2 4 6 8 10 8.16 8.59 8.00 7.97
Apache Spark Row Count: 10000000 - Partitions: 1000 - Calculate Pi Benchmark Using Dataframe OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 1000 - Calculate Pi Benchmark Using Dataframe A B C D 1.1431 2.2862 3.4293 4.5724 5.7155 5.060000000 5.080638849 4.980000000 4.720000000
Apache Spark Row Count: 10000000 - Partitions: 500 - Broadcast Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 500 - Broadcast Inner Join Test Time A B C D 1.3275 2.655 3.9825 5.31 6.6375 5.82 5.85 5.49 5.90
Apache Spark Row Count: 20000000 - Partitions: 100 - Calculate Pi Benchmark Using Dataframe OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 100 - Calculate Pi Benchmark Using Dataframe A B C D 1.1318 2.2636 3.3954 4.5272 5.659 5.02 4.70 4.80 5.03
Apache Spark Row Count: 1000000 - Partitions: 1000 - Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 1000 - Inner Join Test Time A B C D 0.288 0.576 0.864 1.152 1.44 1.22 1.27 1.28 1.20
Apache Spark Row Count: 10000000 - Partitions: 100 - Calculate Pi Benchmark Using Dataframe OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 100 - Calculate Pi Benchmark Using Dataframe A B C D 1.143 2.286 3.429 4.572 5.715 5.08 5.03 4.78 5.07
Apache Spark Row Count: 10000000 - Partitions: 500 - Repartition Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 500 - Repartition Test Time A B C D 1.1475 2.295 3.4425 4.59 5.7375 5.10 4.81 4.96 4.96
Apache Spark Row Count: 20000000 - Partitions: 500 - Group By Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 500 - Group By Test Time A B C D 2 4 6 8 10 7.74 7.84 7.52 7.41
Apache Spark Row Count: 40000000 - Partitions: 2000 - Repartition Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 2000 - Repartition Test Time A B C D 5 10 15 20 25 18.42 17.55 18.56 18.33
Apache Spark Row Count: 10000000 - Partitions: 100 - Repartition Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 100 - Repartition Test Time A B C D 1.215 2.43 3.645 4.86 6.075 5.40 5.25 5.39 5.11
Apache Spark Row Count: 20000000 - Partitions: 500 - Repartition Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 500 - Repartition Test Time A B C D 3 6 9 12 15 9.88 9.62 10.05 9.52
Apache Spark Row Count: 10000000 - Partitions: 1000 - Broadcast Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 1000 - Broadcast Inner Join Test Time A B C D 1.2645 2.529 3.7935 5.058 6.3225 5.62 5.33 5.35 5.39
Apache Spark Row Count: 40000000 - Partitions: 500 - Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 500 - Inner Join Test Time A B C D 5 10 15 20 25 21.14 20.69 21.78 21.55
Apache Spark Row Count: 10000000 - Partitions: 1000 - Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 1000 - Inner Join Test Time A B C D 1.3433 2.6866 4.0299 5.3732 6.7165 5.97 5.68 5.74 5.79
Apache Spark Row Count: 1000000 - Partitions: 1000 - Repartition Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 1000 - Repartition Test Time A B C D 0.3263 0.6526 0.9789 1.3052 1.6315 1.45 1.42 1.38 1.43
Apache Spark Row Count: 20000000 - Partitions: 100 - SHA-512 Benchmark Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 100 - SHA-512 Benchmark Time A B C D 4 8 12 16 20 14.33 14.55 14.37 15.04
Apache Spark Row Count: 10000000 - Partitions: 500 - Group By Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 500 - Group By Test Time A B C D 1.0463 2.0926 3.1389 4.1852 5.2315 4.65 4.43 4.63 4.50
Apache Spark Row Count: 1000000 - Partitions: 500 - Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 500 - Inner Join Test Time A B C D 0.2363 0.4726 0.7089 0.9452 1.1815 1.010000000 1.000558709 1.050000000 1.020000000
Apache Spark Row Count: 1000000 - Partitions: 100 - SHA-512 Benchmark Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 100 - SHA-512 Benchmark Time A B C D 0.4995 0.999 1.4985 1.998 2.4975 2.12 2.21 2.21 2.22
Apache Spark Row Count: 40000000 - Partitions: 1000 - Repartition Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 1000 - Repartition Test Time A B C D 5 10 15 20 25 18.68 18.13 18.08 17.87
Apache Spark Row Count: 20000000 - Partitions: 2000 - Calculate Pi Benchmark Using Dataframe OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 2000 - Calculate Pi Benchmark Using Dataframe A B C D 1.1318 2.2636 3.3954 4.5272 5.659 4.82 4.93 5.03 4.97
Apache Spark Row Count: 10000000 - Partitions: 2000 - Repartition Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 2000 - Repartition Test Time A B C D 1.2128 2.4256 3.6384 4.8512 6.064 5.19 5.17 5.39 5.21
Apache Spark Row Count: 10000000 - Partitions: 100 - Group By Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 100 - Group By Test Time A B C D 1.1678 2.3356 3.5034 4.6712 5.839 4.98 4.99 5.04 5.19
Apache Spark Row Count: 10000000 - Partitions: 1000 - Repartition Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 1000 - Repartition Test Time A B C D 1.1768 2.3536 3.5304 4.7072 5.884 5.06 5.02 5.23 5.16
Apache Spark Row Count: 40000000 - Partitions: 1000 - Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 1000 - Inner Join Test Time A B C D 5 10 15 20 25 21.88 21.87 21.26 22.14
Apache Spark Row Count: 40000000 - Partitions: 500 - SHA-512 Benchmark Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 500 - SHA-512 Benchmark Time A B C D 6 12 18 24 30 25.26 24.36 24.30 24.31
Apache Spark Row Count: 10000000 - Partitions: 2000 - Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 2000 - Inner Join Test Time A B C D 2 4 6 8 10 6.08 6.12 6.32 6.17
Apache Spark Row Count: 20000000 - Partitions: 1000 - Repartition Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 1000 - Repartition Test Time A B C D 3 6 9 12 15 9.46 9.21 9.47 9.57
Apache Spark Row Count: 20000000 - Partitions: 500 - Broadcast Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 500 - Broadcast Inner Join Test Time A B C D 3 6 9 12 15 12.24 12.32 12.49 12.03
Apache Spark Row Count: 20000000 - Partitions: 1000 - Group By Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 1000 - Group By Test Time A B C D 2 4 6 8 10 6.80 6.95 7.06 6.82
Apache Spark Row Count: 1000000 - Partitions: 500 - Repartition Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 500 - Repartition Test Time A B C D 0.306 0.612 0.918 1.224 1.53 1.31 1.36 1.34 1.36
Apache Spark Row Count: 20000000 - Partitions: 500 - Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 500 - Inner Join Test Time A B C D 3 6 9 12 15 11.80 11.85 11.43 11.75
Apache Spark Row Count: 10000000 - Partitions: 2000 - SHA-512 Benchmark Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 2000 - SHA-512 Benchmark Time A B C D 2 4 6 8 10 7.84 7.80 7.69 7.97
Apache Spark Row Count: 40000000 - Partitions: 2000 - SHA-512 Benchmark Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 2000 - SHA-512 Benchmark Time A B C D 6 12 18 24 30 24.79 24.71 24.70 25.58
Apache Spark Row Count: 40000000 - Partitions: 1000 - Group By Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 1000 - Group By Test Time A B C D 4 8 12 16 20 15.40 15.78 15.24 15.58
Apache Spark Row Count: 20000000 - Partitions: 1000 - Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 1000 - Inner Join Test Time A B C D 3 6 9 12 15 11.37 11.15 11.07 10.99
Apache Spark Row Count: 1000000 - Partitions: 500 - Broadcast Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 500 - Broadcast Inner Join Test Time A B C D 0.2048 0.4096 0.6144 0.8192 1.024 0.88 0.91 0.89 0.91
Apache Spark Row Count: 40000000 - Partitions: 2000 - Calculate Pi Benchmark Using Dataframe OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 2000 - Calculate Pi Benchmark Using Dataframe A B C D 1.134 2.268 3.402 4.536 5.67 4.99 4.92 4.89 5.04
Apache Spark Row Count: 10000000 - Partitions: 1000 - Group By Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 1000 - Group By Test Time A B C D 1.1408 2.2816 3.4224 4.5632 5.704 4.96 4.92 5.00 5.07
Apache Spark Row Count: 10000000 - Partitions: 500 - Calculate Pi Benchmark Using Dataframe OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 500 - Calculate Pi Benchmark Using Dataframe A B C D 1.1498 2.2996 3.4494 4.5992 5.749 5.08 5.03 4.96 5.11
Apache Spark Row Count: 40000000 - Partitions: 100 - Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 100 - Inner Join Test Time A B C D 6 12 18 24 30 23.04 23.13 22.91 23.55
Apache Spark Row Count: 40000000 - Partitions: 500 - Group By Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 500 - Group By Test Time A B C D 4 8 12 16 20 17.12 16.84 16.99 16.67
Apache Spark Row Count: 1000000 - Partitions: 500 - SHA-512 Benchmark Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 500 - SHA-512 Benchmark Time A B C D 0.522 1.044 1.566 2.088 2.61 2.29 2.26 2.32 2.30
Apache Spark Row Count: 10000000 - Partitions: 500 - Calculate Pi Benchmark OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 500 - Calculate Pi Benchmark A B C D 20 40 60 80 100 87.28 86.09 85.04 85.68
Apache Spark Row Count: 20000000 - Partitions: 100 - Repartition Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 100 - Repartition Test Time A B C D 3 6 9 12 15 10.098177409 10.200000000 10.070000000 9.950000000
Apache Spark Row Count: 1000000 - Partitions: 2000 - Calculate Pi Benchmark OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 2000 - Calculate Pi Benchmark A B C D 20 40 60 80 100 84.51 85.68 85.77 86.59
Apache Spark Row Count: 1000000 - Partitions: 100 - Repartition Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 100 - Repartition Test Time A B C D 0.2813 0.5626 0.8439 1.1252 1.4065 1.25 1.22 1.25 1.24
Apache Spark Row Count: 40000000 - Partitions: 500 - Repartition Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 500 - Repartition Test Time A B C D 5 10 15 20 25 18.48 18.16 18.60 18.28
Apache Spark Row Count: 20000000 - Partitions: 1000 - SHA-512 Benchmark Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 1000 - SHA-512 Benchmark Time A B C D 3 6 9 12 15 13.43 13.24 13.12 13.28
Apache Spark Row Count: 40000000 - Partitions: 100 - Broadcast Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 100 - Broadcast Inner Join Test Time A B C D 6 12 18 24 30 23.81 24.26 24.27 23.71
Apache Spark Row Count: 40000000 - Partitions: 2000 - Calculate Pi Benchmark OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 2000 - Calculate Pi Benchmark A B C D 20 40 60 80 100 85.35 84.85 85.55 86.84
Apache Spark Row Count: 40000000 - Partitions: 2000 - Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 2000 - Inner Join Test Time A B C D 5 10 15 20 25 21.83 22.01 22.09 22.33
Apache Spark Row Count: 1000000 - Partitions: 500 - Group By Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 500 - Group By Test Time A B C D 0.6323 1.2646 1.8969 2.5292 3.1615 2.79 2.75 2.81 2.77
Apache Spark Row Count: 10000000 - Partitions: 500 - SHA-512 Benchmark Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 500 - SHA-512 Benchmark Time A B C D 2 4 6 8 10 7.40 7.46 7.42 7.56
Apache Spark Row Count: 40000000 - Partitions: 1000 - Calculate Pi Benchmark OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 1000 - Calculate Pi Benchmark A B C D 20 40 60 80 100 85.27 86.57 85.71 84.79
Apache Spark Row Count: 1000000 - Partitions: 1000 - Group By Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 1000 - Group By Test Time A B C D 0.6705 1.341 2.0115 2.682 3.3525 2.94 2.92 2.98 2.96
Apache Spark Row Count: 40000000 - Partitions: 500 - Broadcast Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 500 - Broadcast Inner Join Test Time A B C D 5 10 15 20 25 21.25 21.19 21.30 21.62
Apache Spark Row Count: 40000000 - Partitions: 100 - Group By Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 100 - Group By Test Time A B C D 5 10 15 20 25 19.14 18.84 18.76 18.87
Apache Spark Row Count: 10000000 - Partitions: 2000 - Broadcast Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 2000 - Broadcast Inner Join Test Time A B C D 2 4 6 8 10 6.07 5.95 6.05 6.02
Apache Spark Row Count: 20000000 - Partitions: 100 - Broadcast Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 100 - Broadcast Inner Join Test Time A B C D 3 6 9 12 15 12.40 12.64 12.51 12.43
Apache Spark Row Count: 1000000 - Partitions: 100 - Calculate Pi Benchmark OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 100 - Calculate Pi Benchmark A B C D 20 40 60 80 100 81.45 82.77 82.47 82.97
Apache Spark Row Count: 40000000 - Partitions: 1000 - Broadcast Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 1000 - Broadcast Inner Join Test Time A B C D 5 10 15 20 25 22.41 22.35 22.00 22.38
Apache Spark Row Count: 1000000 - Partitions: 100 - Group By Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 100 - Group By Test Time A B C D 0.621 1.242 1.863 2.484 3.105 2.71 2.74 2.76 2.72
Apache Spark Row Count: 10000000 - Partitions: 2000 - Calculate Pi Benchmark OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 2000 - Calculate Pi Benchmark A B C D 20 40 60 80 100 86.82 85.28 86.04 85.51
Apache Spark Row Count: 1000000 - Partitions: 100 - Calculate Pi Benchmark Using Dataframe OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 100 - Calculate Pi Benchmark Using Dataframe A B C D 1.1475 2.295 3.4425 4.59 5.7375 5.04 5.01 5.10 5.10
Apache Spark Row Count: 10000000 - Partitions: 1000 - Calculate Pi Benchmark OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 1000 - Calculate Pi Benchmark A B C D 20 40 60 80 100 86.28 86.02 85.82 84.81
Apache Spark Row Count: 1000000 - Partitions: 1000 - Calculate Pi Benchmark OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 1000 - Calculate Pi Benchmark A B C D 20 40 60 80 100 84.36 85.47 84.46 85.81
Apache Spark Row Count: 10000000 - Partitions: 1000 - SHA-512 Benchmark Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 1000 - SHA-512 Benchmark Time A B C D 2 4 6 8 10 7.73 7.77 7.78 7.65
Apache Spark Row Count: 20000000 - Partitions: 1000 - Calculate Pi Benchmark Using Dataframe OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 1000 - Calculate Pi Benchmark Using Dataframe A B C D 1.1408 2.2816 3.4224 4.5632 5.704 5.00 5.07 4.99 4.99
Apache Spark Row Count: 10000000 - Partitions: 100 - Calculate Pi Benchmark OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 100 - Calculate Pi Benchmark A B C D 20 40 60 80 100 85.75 85.31 86.35 86.07
Apache Spark Row Count: 10000000 - Partitions: 2000 - Group By Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 2000 - Group By Test Time A B C D 1.1318 2.2636 3.3954 4.5272 5.659 5.00 4.97 4.98 5.03
Apache Spark Row Count: 40000000 - Partitions: 100 - Calculate Pi Benchmark OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 100 - Calculate Pi Benchmark A B C D 20 40 60 80 100 84.93 85.67 84.93 85.22
Apache Spark Row Count: 40000000 - Partitions: 1000 - SHA-512 Benchmark Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 1000 - SHA-512 Benchmark Time A B C D 6 12 18 24 30 24.94 24.97 24.97 25.16
Apache Spark Row Count: 20000000 - Partitions: 500 - Calculate Pi Benchmark OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 500 - Calculate Pi Benchmark A B C D 20 40 60 80 100 85.69 85.09 85.02 85.14
Apache Spark Row Count: 20000000 - Partitions: 100 - Calculate Pi Benchmark OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 100 - Calculate Pi Benchmark A B C D 20 40 60 80 100 85.14 85.48 84.84 85.09
Apache Spark Row Count: 40000000 - Partitions: 100 - Repartition Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 100 - Repartition Test Time A B C D 5 10 15 20 25 19.24 19.30 19.27 19.16
Apache Spark Row Count: 1000000 - Partitions: 500 - Calculate Pi Benchmark OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 500 - Calculate Pi Benchmark A B C D 20 40 60 80 100 84.80 84.41 85.01 84.44
Apache Spark Row Count: 20000000 - Partitions: 100 - Group By Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 100 - Group By Test Time A B C D 2 4 6 8 10 7.65 7.66 7.61 7.65
Apache Spark Row Count: 20000000 - Partitions: 1000 - Calculate Pi Benchmark OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 1000 - Calculate Pi Benchmark A B C D 20 40 60 80 100 84.83 85.35 84.81 85.07
Apache Spark Row Count: 40000000 - Partitions: 500 - Calculate Pi Benchmark OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 500 - Calculate Pi Benchmark A B C D 20 40 60 80 100 84.63 85.03 85.02 85.13
Apache Spark Row Count: 20000000 - Partitions: 2000 - Calculate Pi Benchmark OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 2000 - Calculate Pi Benchmark A B C D 20 40 60 80 100 85.12 84.89 85.13 85.35
Apache Spark Row Count: 40000000 - Partitions: 100 - SHA-512 Benchmark Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 100 - SHA-512 Benchmark Time A B C D 6 12 18 24 30 27.24 27.10 27.10 27.11
Apache Spark Row Count: 40000000 - Partitions: 2000 - Broadcast Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 2000 - Broadcast Inner Join Test Time A B C D 5 10 15 20 25 21.99 21.94 21.98 21.93
Phoronix Test Suite v10.8.4