spark tr AMD Ryzen Threadripper 2990WX 32-Core testing with a ASUS ROG ZENITH EXTREME (2201 BIOS) and Gigabyte AMD Radeon RX 570 4GB on Ubuntu 22.04 via the Phoronix Test Suite.
HTML result view exported from: https://openbenchmarking.org/result/2208040-NE-SPARKTR8101&rdt&grs .
spark tr Processor Motherboard Chipset Memory Disk Graphics Audio Monitor Network OS Kernel Desktop Display Server OpenGL Vulkan Compiler File-System Screen Resolution A B AMD Ryzen Threadripper 2990WX 32-Core @ 3.00GHz (32 Cores / 64 Threads) ASUS ROG ZENITH EXTREME (2201 BIOS) AMD 17h 32GB Samsung SSD 970 EVO 500GB + 250GB Western Digital WDS250G2X0C-00L350 Gigabyte AMD Radeon RX 570 4GB (1244/1750MHz) Realtek ALC1220 MX279 Intel I211 + Qualcomm Atheros QCA6174 802.11ac + Wilocity Wil6200 802.11ad Ubuntu 22.04 5.19.0-051900rc7-generic (x86_64) GNOME Shell 42.2 X Server + Wayland 4.6 Mesa 22.0.1 (LLVM 13.0.1 DRM 3.47) 1.2.204 GCC 11.2.0 ext4 1920x1080 OpenBenchmarking.org Kernel Details - Transparent Huge Pages: madvise Compiler Details - --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-link-serialization=2 --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-targets=nvptx-none=/build/gcc-11-gBFGDP/gcc-11-11.2.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-11-gBFGDP/gcc-11-11.2.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-build-config=bootstrap-lto-lean --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v Processor Details - Scaling Governor: acpi-cpufreq schedutil (Boost: Enabled) - CPU Microcode: 0x800820d Java Details - OpenJDK Runtime Environment (build 11.0.15+10-Ubuntu-0ubuntu0.22.04.1) Python Details - Python 3.10.4 Security Details - itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Mitigation of untrained return thunk; SMT vulnerable + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional STIBP: disabled RSB filling + srbds: Not affected + tsx_async_abort: Not affected
spark tr spark: 10000000 - 100 - Inner Join Test Time spark: 10000000 - 100 - Repartition Test Time spark: 10000000 - 100 - Broadcast Inner Join Test Time spark: 20000000 - 100 - Inner Join Test Time spark: 20000000 - 100 - Broadcast Inner Join Test Time spark: 10000000 - 500 - Broadcast Inner Join Test Time spark: 10000000 - 500 - Repartition Test Time spark: 20000000 - 100 - Repartition Test Time spark: 10000000 - 500 - Inner Join Test Time spark: 1000000 - 2000 - Repartition Test Time spark: 10000000 - 100 - SHA-512 Benchmark Time spark: 20000000 - 500 - Inner Join Test Time spark: 20000000 - 500 - Repartition Test Time spark: 20000000 - 500 - Broadcast Inner Join Test Time spark: 40000000 - 100 - Broadcast Inner Join Test Time spark: 40000000 - 100 - Repartition Test Time spark: 10000000 - 500 - SHA-512 Benchmark Time spark: 1000000 - 1000 - Repartition Test Time spark: 20000000 - 500 - SHA-512 Benchmark Time spark: 1000000 - 100 - Broadcast Inner Join Test Time spark: 1000000 - 2000 - Broadcast Inner Join Test Time spark: 1000000 - 100 - Repartition Test Time spark: 40000000 - 100 - Inner Join Test Time spark: 1000000 - 500 - Repartition Test Time spark: 20000000 - 100 - SHA-512 Benchmark Time spark: 1000000 - 500 - Inner Join Test Time svt-av1: Preset 12 - Bosphorus 1080p spark: 40000000 - 500 - Repartition Test Time spark: 40000000 - 500 - Inner Join Test Time spark: 10000000 - 1000 - Broadcast Inner Join Test Time spark: 20000000 - 2000 - Group By Test Time spark: 1000000 - 500 - SHA-512 Benchmark Time spark: 10000000 - 1000 - Repartition Test Time spark: 1000000 - 2000 - Inner Join Test Time spark: 1000000 - 100 - Inner Join Test Time spark: 1000000 - 1000 - Inner Join Test Time spark: 1000000 - 1000 - SHA-512 Benchmark Time spark: 1000000 - 500 - Broadcast Inner Join Test Time spark: 40000000 - 100 - Group By Test Time spark: 40000000 - 1000 - Broadcast Inner Join Test Time svt-av1: Preset 8 - Bosphorus 1080p spark: 20000000 - 100 - Group By Test Time spark: 20000000 - 1000 - Repartition Test Time spark: 1000000 - 500 - Group By Test Time spark: 40000000 - 500 - Calculate Pi Benchmark Using Dataframe spark: 40000000 - 2000 - Calculate Pi Benchmark Using Dataframe spark: 10000000 - 500 - Group By Test Time svt-av1: Preset 10 - Bosphorus 1080p spark: 10000000 - 2000 - Broadcast Inner Join Test Time svt-av1: Preset 10 - Bosphorus 4K spark: 40000000 - 100 - SHA-512 Benchmark Time spark: 40000000 - 500 - SHA-512 Benchmark Time spark: 40000000 - 500 - Broadcast Inner Join Test Time spark: 1000000 - 2000 - SHA-512 Benchmark Time spark: 10000000 - 2000 - SHA-512 Benchmark Time spark: 1000000 - 100 - SHA-512 Benchmark Time spark: 20000000 - 1000 - Broadcast Inner Join Test Time spark: 40000000 - 1000 - Repartition Test Time spark: 40000000 - 2000 - Group By Test Time spark: 40000000 - 2000 - Broadcast Inner Join Test Time spark: 20000000 - 2000 - Repartition Test Time spark: 20000000 - 2000 - Inner Join Test Time spark: 1000000 - 2000 - Calculate Pi Benchmark Using Dataframe spark: 1000000 - 1000 - Broadcast Inner Join Test Time spark: 20000000 - 1000 - Group By Test Time spark: 10000000 - 2000 - Repartition Test Time spark: 10000000 - 100 - Group By Test Time spark: 20000000 - 1000 - SHA-512 Benchmark Time spark: 1000000 - 100 - Group By Test Time spark: 10000000 - 2000 - Inner Join Test Time spark: 10000000 - 1000 - Calculate Pi Benchmark Using Dataframe svt-av1: Preset 8 - Bosphorus 4K spark: 20000000 - 1000 - Inner Join Test Time spark: 10000000 - 1000 - Group By Test Time spark: 20000000 - 2000 - Broadcast Inner Join Test Time spark: 40000000 - 500 - Group By Test Time spark: 10000000 - 2000 - Calculate Pi Benchmark Using Dataframe svt-av1: Preset 12 - Bosphorus 4K spark: 10000000 - 100 - Calculate Pi Benchmark Using Dataframe spark: 40000000 - 2000 - Repartition Test Time spark: 20000000 - 100 - Calculate Pi Benchmark spark: 20000000 - 2000 - SHA-512 Benchmark Time spark: 1000000 - 2000 - Group By Test Time spark: 20000000 - 500 - Group By Test Time spark: 1000000 - 1000 - Calculate Pi Benchmark spark: 40000000 - 2000 - Inner Join Test Time svt-av1: Preset 4 - Bosphorus 4K spark: 1000000 - 100 - Calculate Pi Benchmark Using Dataframe spark: 10000000 - 500 - Calculate Pi Benchmark Using Dataframe spark: 20000000 - 2000 - Calculate Pi Benchmark Using Dataframe spark: 10000000 - 1000 - Inner Join Test Time spark: 10000000 - 2000 - Group By Test Time spark: 20000000 - 500 - Calculate Pi Benchmark Using Dataframe spark: 1000000 - 1000 - Calculate Pi Benchmark Using Dataframe spark: 20000000 - 2000 - Calculate Pi Benchmark spark: 10000000 - 2000 - Calculate Pi Benchmark spark: 1000000 - 2000 - Calculate Pi Benchmark spark: 40000000 - 2000 - SHA-512 Benchmark Time svt-av1: Preset 4 - Bosphorus 1080p spark: 10000000 - 1000 - SHA-512 Benchmark Time spark: 40000000 - 1000 - Calculate Pi Benchmark spark: 40000000 - 1000 - SHA-512 Benchmark Time spark: 1000000 - 100 - Calculate Pi Benchmark spark: 20000000 - 1000 - Calculate Pi Benchmark spark: 10000000 - 1000 - Calculate Pi Benchmark spark: 20000000 - 1000 - Calculate Pi Benchmark Using Dataframe spark: 1000000 - 1000 - Group By Test Time spark: 40000000 - 100 - Calculate Pi Benchmark spark: 1000000 - 500 - Calculate Pi Benchmark Using Dataframe spark: 1000000 - 500 - Calculate Pi Benchmark spark: 40000000 - 1000 - Calculate Pi Benchmark Using Dataframe spark: 40000000 - 100 - Calculate Pi Benchmark Using Dataframe spark: 40000000 - 2000 - Calculate Pi Benchmark spark: 10000000 - 500 - Calculate Pi Benchmark spark: 10000000 - 100 - Calculate Pi Benchmark spark: 40000000 - 1000 - Group By Test Time spark: 40000000 - 500 - Calculate Pi Benchmark spark: 20000000 - 500 - Calculate Pi Benchmark spark: 40000000 - 1000 - Inner Join Test Time spark: 20000000 - 100 - Calculate Pi Benchmark Using Dataframe A B 17.083292218 16.178241522 15.102118238 28.857376046 26.429071267 14.623652287 15.315967879 27.337910713 15.825329803 6.15 20.06391253 24.462975514 23.552140763 23.545246024 43.02298204 40.979517765 19.504123346 4.77 30.738066039 2.36 4.015974333 4.01 43.76988138 4.448490734 31.155512825 3.75 233.906 35.936626267 38.805176603 10.538025619 16.642311492 5.77 10.664056294 4.231091785 3.07470943 3.47 5.707415972 2.475242862 31.884101243 31.340923075 72.267 14.753288032 17.444436259 7.00 4.53 4.62 11.477514675 150.077 11.376763629 48.462 43.910825271 46.704938593 35.542473101 5.99405918 15.295615927 5.13080919 17.533151598 31.12755649 32.790285393 33.406302575 18.129978776 20.098796966 4.50417597 3.07 14.545897933 10.668073982 11.039362384 24.55720381 6.479692439 12.691831458 4.54 28.875 18.129485143 10.691886696 18.885806515 33.81815958 4.444747603 60.767 4.41 32.367778177 61.889646244 25.218516804 7.03 14.918612745 62.428705739 36.89 1.472 4.33 4.44 4.69 10.950003509 11.740644063 4.52 4.42 62.508160211 61.554074426 62.439872636 44.327354258 3.568 15.225220123 61.930554898 44.735944448 61.131040206 62.23774266 61.57736862 4.635858138 6.69 62.136693946 4.442555283 61.739942656 4.43 4.43 61.990397429 61.824250186 61.957279936 33.856953727 62.40252554 62.025743606 34.819192588 4.46 10.053629991 9.740366086 9.20 18.178785652 16.81 9.45 9.93 18.18 10.68 4.25 14.42 17.636233312 17.014284711 17.11 31.92 30.69 14.86 3.69 24.016552673 1.85 3.16 3.20 35.77 3.65 25.802573232 3.11 201.802 31.040781082 34.60 9.43 15.041605602 5.30 9.85 3.91 2.85 3.22 5.30 2.30 29.730571283 33.61 67.743 13.83 18.60 6.58 4.79 4.38 10.89 142.495 11.97 50.882 46.03 48.940567345 33.950674349 5.74 15.97 5.35 18.23 32.348191608 31.62 34.63 18.775430078 19.49 4.37 2.98 14.164335829 10.94 11.30869129 25.13 6.63 12.98 4.64 28.259 17.744465328 10.913284688 19.261739304 33.168941005 4.53 61.899 4.49 31.86 62.87 25.614620798 7.14 14.69 61.558668692 37.377857332 1.453 4.38 4.39 4.64 11.04 11.83 4.488093463 4.45 62.105153355 61.941561169 62.06 44.058118018 3.547 15.31529798 62.279023502 44.49 61.43 62.53 61.86 4.65 6.67 61.95799001 4.43 61.906716893 4.42 4.44 62.13 61.70 61.836282332 33.810705774 62.332106635 62.074873258 34.82076963 4.46 OpenBenchmarking.org
Apache Spark Row Count: 10000000 - Partitions: 100 - Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 100 - Inner Join Test Time A B 4 8 12 16 20 17.08 10.05
Apache Spark Row Count: 10000000 - Partitions: 100 - Repartition Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 100 - Repartition Test Time A B 4 8 12 16 20 16.178241522 9.740366086
Apache Spark Row Count: 10000000 - Partitions: 100 - Broadcast Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 100 - Broadcast Inner Join Test Time A B 4 8 12 16 20 15.102118238 9.200000000
Apache Spark Row Count: 20000000 - Partitions: 100 - Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 100 - Inner Join Test Time A B 7 14 21 28 35 28.86 18.18
Apache Spark Row Count: 20000000 - Partitions: 100 - Broadcast Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 100 - Broadcast Inner Join Test Time A B 6 12 18 24 30 26.43 16.81
Apache Spark Row Count: 10000000 - Partitions: 500 - Broadcast Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 500 - Broadcast Inner Join Test Time A B 4 8 12 16 20 14.623652287 9.450000000
Apache Spark Row Count: 10000000 - Partitions: 500 - Repartition Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 500 - Repartition Test Time A B 4 8 12 16 20 15.315967879 9.930000000
Apache Spark Row Count: 20000000 - Partitions: 100 - Repartition Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 100 - Repartition Test Time A B 6 12 18 24 30 27.34 18.18
Apache Spark Row Count: 10000000 - Partitions: 500 - Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 500 - Inner Join Test Time A B 4 8 12 16 20 15.83 10.68
Apache Spark Row Count: 1000000 - Partitions: 2000 - Repartition Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 2000 - Repartition Test Time A B 2 4 6 8 10 6.15 4.25
Apache Spark Row Count: 10000000 - Partitions: 100 - SHA-512 Benchmark Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 100 - SHA-512 Benchmark Time A B 5 10 15 20 25 20.06 14.42
Apache Spark Row Count: 20000000 - Partitions: 500 - Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 500 - Inner Join Test Time A B 6 12 18 24 30 24.46 17.64
Apache Spark Row Count: 20000000 - Partitions: 500 - Repartition Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 500 - Repartition Test Time A B 6 12 18 24 30 23.55 17.01
Apache Spark Row Count: 20000000 - Partitions: 500 - Broadcast Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 500 - Broadcast Inner Join Test Time A B 6 12 18 24 30 23.55 17.11
Apache Spark Row Count: 40000000 - Partitions: 100 - Broadcast Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 100 - Broadcast Inner Join Test Time A B 10 20 30 40 50 43.02 31.92
Apache Spark Row Count: 40000000 - Partitions: 100 - Repartition Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 100 - Repartition Test Time A B 9 18 27 36 45 40.98 30.69
Apache Spark Row Count: 10000000 - Partitions: 500 - SHA-512 Benchmark Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 500 - SHA-512 Benchmark Time A B 5 10 15 20 25 19.50 14.86
Apache Spark Row Count: 1000000 - Partitions: 1000 - Repartition Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 1000 - Repartition Test Time A B 1.0733 2.1466 3.2199 4.2932 5.3665 4.77 3.69
Apache Spark Row Count: 20000000 - Partitions: 500 - SHA-512 Benchmark Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 500 - SHA-512 Benchmark Time A B 7 14 21 28 35 30.74 24.02
Apache Spark Row Count: 1000000 - Partitions: 100 - Broadcast Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 100 - Broadcast Inner Join Test Time A B 0.531 1.062 1.593 2.124 2.655 2.36 1.85
Apache Spark Row Count: 1000000 - Partitions: 2000 - Broadcast Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 2000 - Broadcast Inner Join Test Time A B 0.9036 1.8072 2.7108 3.6144 4.518 4.015974333 3.160000000
Apache Spark Row Count: 1000000 - Partitions: 100 - Repartition Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 100 - Repartition Test Time A B 0.9023 1.8046 2.7069 3.6092 4.5115 4.01 3.20
Apache Spark Row Count: 40000000 - Partitions: 100 - Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 100 - Inner Join Test Time A B 10 20 30 40 50 43.77 35.77
Apache Spark Row Count: 1000000 - Partitions: 500 - Repartition Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 500 - Repartition Test Time A B 1.0009 2.0018 3.0027 4.0036 5.0045 4.448490734 3.650000000
Apache Spark Row Count: 20000000 - Partitions: 100 - SHA-512 Benchmark Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 100 - SHA-512 Benchmark Time A B 7 14 21 28 35 31.16 25.80
Apache Spark Row Count: 1000000 - Partitions: 500 - Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 500 - Inner Join Test Time A B 0.8438 1.6876 2.5314 3.3752 4.219 3.75 3.11
SVT-AV1 Encoder Mode: Preset 12 - Input: Bosphorus 1080p OpenBenchmarking.org Frames Per Second, More Is Better SVT-AV1 1.2 Encoder Mode: Preset 12 - Input: Bosphorus 1080p A B 50 100 150 200 250 233.91 201.80 1. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq
Apache Spark Row Count: 40000000 - Partitions: 500 - Repartition Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 500 - Repartition Test Time A B 8 16 24 32 40 35.94 31.04
Apache Spark Row Count: 40000000 - Partitions: 500 - Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 500 - Inner Join Test Time A B 9 18 27 36 45 38.81 34.60
Apache Spark Row Count: 10000000 - Partitions: 1000 - Broadcast Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 1000 - Broadcast Inner Join Test Time A B 3 6 9 12 15 10.538025619 9.430000000
Apache Spark Row Count: 20000000 - Partitions: 2000 - Group By Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 2000 - Group By Test Time A B 4 8 12 16 20 16.64 15.04
Apache Spark Row Count: 1000000 - Partitions: 500 - SHA-512 Benchmark Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 500 - SHA-512 Benchmark Time A B 1.2983 2.5966 3.8949 5.1932 6.4915 5.77 5.30
Apache Spark Row Count: 10000000 - Partitions: 1000 - Repartition Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 1000 - Repartition Test Time A B 3 6 9 12 15 10.664056294 9.850000000
Apache Spark Row Count: 1000000 - Partitions: 2000 - Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 2000 - Inner Join Test Time A B 0.952 1.904 2.856 3.808 4.76 4.231091785 3.910000000
Apache Spark Row Count: 1000000 - Partitions: 100 - Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 100 - Inner Join Test Time A B 0.6918 1.3836 2.0754 2.7672 3.459 3.07470943 2.85000000
Apache Spark Row Count: 1000000 - Partitions: 1000 - Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 1000 - Inner Join Test Time A B 0.7808 1.5616 2.3424 3.1232 3.904 3.47 3.22
Apache Spark Row Count: 1000000 - Partitions: 1000 - SHA-512 Benchmark Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 1000 - SHA-512 Benchmark Time A B 1.2842 2.5684 3.8526 5.1368 6.421 5.707415972 5.300000000
Apache Spark Row Count: 1000000 - Partitions: 500 - Broadcast Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 500 - Broadcast Inner Join Test Time A B 0.5569 1.1138 1.6707 2.2276 2.7845 2.475242862 2.300000000
Apache Spark Row Count: 40000000 - Partitions: 100 - Group By Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 100 - Group By Test Time A B 7 14 21 28 35 31.88 29.73
Apache Spark Row Count: 40000000 - Partitions: 1000 - Broadcast Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 1000 - Broadcast Inner Join Test Time A B 8 16 24 32 40 31.34 33.61
SVT-AV1 Encoder Mode: Preset 8 - Input: Bosphorus 1080p OpenBenchmarking.org Frames Per Second, More Is Better SVT-AV1 1.2 Encoder Mode: Preset 8 - Input: Bosphorus 1080p A B 16 32 48 64 80 72.27 67.74 1. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq
Apache Spark Row Count: 20000000 - Partitions: 100 - Group By Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 100 - Group By Test Time A B 4 8 12 16 20 14.75 13.83
Apache Spark Row Count: 20000000 - Partitions: 1000 - Repartition Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 1000 - Repartition Test Time A B 5 10 15 20 25 17.44 18.60
Apache Spark Row Count: 1000000 - Partitions: 500 - Group By Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 500 - Group By Test Time A B 2 4 6 8 10 7.00 6.58
Apache Spark Row Count: 40000000 - Partitions: 500 - Calculate Pi Benchmark Using Dataframe OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 500 - Calculate Pi Benchmark Using Dataframe A B 1.0778 2.1556 3.2334 4.3112 5.389 4.53 4.79
Apache Spark Row Count: 40000000 - Partitions: 2000 - Calculate Pi Benchmark Using Dataframe OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 2000 - Calculate Pi Benchmark Using Dataframe A B 1.0395 2.079 3.1185 4.158 5.1975 4.62 4.38
Apache Spark Row Count: 10000000 - Partitions: 500 - Group By Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 500 - Group By Test Time A B 3 6 9 12 15 11.48 10.89
SVT-AV1 Encoder Mode: Preset 10 - Input: Bosphorus 1080p OpenBenchmarking.org Frames Per Second, More Is Better SVT-AV1 1.2 Encoder Mode: Preset 10 - Input: Bosphorus 1080p A B 30 60 90 120 150 150.08 142.50 1. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq
Apache Spark Row Count: 10000000 - Partitions: 2000 - Broadcast Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 2000 - Broadcast Inner Join Test Time A B 3 6 9 12 15 11.38 11.97
SVT-AV1 Encoder Mode: Preset 10 - Input: Bosphorus 4K OpenBenchmarking.org Frames Per Second, More Is Better SVT-AV1 1.2 Encoder Mode: Preset 10 - Input: Bosphorus 4K A B 11 22 33 44 55 48.46 50.88 1. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq
Apache Spark Row Count: 40000000 - Partitions: 100 - SHA-512 Benchmark Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 100 - SHA-512 Benchmark Time A B 10 20 30 40 50 43.91 46.03
Apache Spark Row Count: 40000000 - Partitions: 500 - SHA-512 Benchmark Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 500 - SHA-512 Benchmark Time A B 11 22 33 44 55 46.70 48.94
Apache Spark Row Count: 40000000 - Partitions: 500 - Broadcast Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 500 - Broadcast Inner Join Test Time A B 8 16 24 32 40 35.54 33.95
Apache Spark Row Count: 1000000 - Partitions: 2000 - SHA-512 Benchmark Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 2000 - SHA-512 Benchmark Time A B 1.3487 2.6974 4.0461 5.3948 6.7435 5.99405918 5.74000000
Apache Spark Row Count: 10000000 - Partitions: 2000 - SHA-512 Benchmark Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 2000 - SHA-512 Benchmark Time A B 4 8 12 16 20 15.30 15.97
Apache Spark Row Count: 1000000 - Partitions: 100 - SHA-512 Benchmark Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 100 - SHA-512 Benchmark Time A B 1.2038 2.4076 3.6114 4.8152 6.019 5.13080919 5.35000000
Apache Spark Row Count: 20000000 - Partitions: 1000 - Broadcast Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 1000 - Broadcast Inner Join Test Time A B 4 8 12 16 20 17.53 18.23
Apache Spark Row Count: 40000000 - Partitions: 1000 - Repartition Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 1000 - Repartition Test Time A B 8 16 24 32 40 31.13 32.35
Apache Spark Row Count: 40000000 - Partitions: 2000 - Group By Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 2000 - Group By Test Time A B 8 16 24 32 40 32.79 31.62
Apache Spark Row Count: 40000000 - Partitions: 2000 - Broadcast Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 2000 - Broadcast Inner Join Test Time A B 8 16 24 32 40 33.41 34.63
Apache Spark Row Count: 20000000 - Partitions: 2000 - Repartition Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 2000 - Repartition Test Time A B 5 10 15 20 25 18.13 18.78
Apache Spark Row Count: 20000000 - Partitions: 2000 - Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 2000 - Inner Join Test Time A B 5 10 15 20 25 20.10 19.49
Apache Spark Row Count: 1000000 - Partitions: 2000 - Calculate Pi Benchmark Using Dataframe OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 2000 - Calculate Pi Benchmark Using Dataframe A B 1.0134 2.0268 3.0402 4.0536 5.067 4.50417597 4.37000000
Apache Spark Row Count: 1000000 - Partitions: 1000 - Broadcast Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 1000 - Broadcast Inner Join Test Time A B 0.6908 1.3816 2.0724 2.7632 3.454 3.07 2.98
Apache Spark Row Count: 20000000 - Partitions: 1000 - Group By Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 1000 - Group By Test Time A B 4 8 12 16 20 14.55 14.16
Apache Spark Row Count: 10000000 - Partitions: 2000 - Repartition Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 2000 - Repartition Test Time A B 3 6 9 12 15 10.67 10.94
Apache Spark Row Count: 10000000 - Partitions: 100 - Group By Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 100 - Group By Test Time A B 3 6 9 12 15 11.04 11.31
Apache Spark Row Count: 20000000 - Partitions: 1000 - SHA-512 Benchmark Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 1000 - SHA-512 Benchmark Time A B 6 12 18 24 30 24.56 25.13
Apache Spark Row Count: 1000000 - Partitions: 100 - Group By Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 100 - Group By Test Time A B 2 4 6 8 10 6.479692439 6.630000000
Apache Spark Row Count: 10000000 - Partitions: 2000 - Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 2000 - Inner Join Test Time A B 3 6 9 12 15 12.69 12.98
Apache Spark Row Count: 10000000 - Partitions: 1000 - Calculate Pi Benchmark Using Dataframe OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 1000 - Calculate Pi Benchmark Using Dataframe A B 1.044 2.088 3.132 4.176 5.22 4.54 4.64
SVT-AV1 Encoder Mode: Preset 8 - Input: Bosphorus 4K OpenBenchmarking.org Frames Per Second, More Is Better SVT-AV1 1.2 Encoder Mode: Preset 8 - Input: Bosphorus 4K A B 7 14 21 28 35 28.88 28.26 1. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq
Apache Spark Row Count: 20000000 - Partitions: 1000 - Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 1000 - Inner Join Test Time A B 4 8 12 16 20 18.13 17.74
Apache Spark Row Count: 10000000 - Partitions: 1000 - Group By Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 1000 - Group By Test Time A B 3 6 9 12 15 10.69 10.91
Apache Spark Row Count: 20000000 - Partitions: 2000 - Broadcast Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 2000 - Broadcast Inner Join Test Time A B 5 10 15 20 25 18.89 19.26
Apache Spark Row Count: 40000000 - Partitions: 500 - Group By Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 500 - Group By Test Time A B 8 16 24 32 40 33.82 33.17
Apache Spark Row Count: 10000000 - Partitions: 2000 - Calculate Pi Benchmark Using Dataframe OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 2000 - Calculate Pi Benchmark Using Dataframe A B 1.0193 2.0386 3.0579 4.0772 5.0965 4.444747603 4.530000000
SVT-AV1 Encoder Mode: Preset 12 - Input: Bosphorus 4K OpenBenchmarking.org Frames Per Second, More Is Better SVT-AV1 1.2 Encoder Mode: Preset 12 - Input: Bosphorus 4K A B 14 28 42 56 70 60.77 61.90 1. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq
Apache Spark Row Count: 10000000 - Partitions: 100 - Calculate Pi Benchmark Using Dataframe OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 100 - Calculate Pi Benchmark Using Dataframe A B 1.0103 2.0206 3.0309 4.0412 5.0515 4.41 4.49
Apache Spark Row Count: 40000000 - Partitions: 2000 - Repartition Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 2000 - Repartition Test Time A B 8 16 24 32 40 32.37 31.86
Apache Spark Row Count: 20000000 - Partitions: 100 - Calculate Pi Benchmark OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 100 - Calculate Pi Benchmark A B 14 28 42 56 70 61.89 62.87
Apache Spark Row Count: 20000000 - Partitions: 2000 - SHA-512 Benchmark Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 2000 - SHA-512 Benchmark Time A B 6 12 18 24 30 25.22 25.61
Apache Spark Row Count: 1000000 - Partitions: 2000 - Group By Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 2000 - Group By Test Time A B 2 4 6 8 10 7.03 7.14
Apache Spark Row Count: 20000000 - Partitions: 500 - Group By Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 500 - Group By Test Time A B 4 8 12 16 20 14.92 14.69
Apache Spark Row Count: 1000000 - Partitions: 1000 - Calculate Pi Benchmark OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 1000 - Calculate Pi Benchmark A B 14 28 42 56 70 62.43 61.56
Apache Spark Row Count: 40000000 - Partitions: 2000 - Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 2000 - Inner Join Test Time A B 9 18 27 36 45 36.89 37.38
SVT-AV1 Encoder Mode: Preset 4 - Input: Bosphorus 4K OpenBenchmarking.org Frames Per Second, More Is Better SVT-AV1 1.2 Encoder Mode: Preset 4 - Input: Bosphorus 4K A B 0.3312 0.6624 0.9936 1.3248 1.656 1.472 1.453 1. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq
Apache Spark Row Count: 1000000 - Partitions: 100 - Calculate Pi Benchmark Using Dataframe OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 100 - Calculate Pi Benchmark Using Dataframe A B 0.9855 1.971 2.9565 3.942 4.9275 4.33 4.38
Apache Spark Row Count: 10000000 - Partitions: 500 - Calculate Pi Benchmark Using Dataframe OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 500 - Calculate Pi Benchmark Using Dataframe A B 0.999 1.998 2.997 3.996 4.995 4.44 4.39
Apache Spark Row Count: 20000000 - Partitions: 2000 - Calculate Pi Benchmark Using Dataframe OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 2000 - Calculate Pi Benchmark Using Dataframe A B 1.0553 2.1106 3.1659 4.2212 5.2765 4.69 4.64
Apache Spark Row Count: 10000000 - Partitions: 1000 - Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 1000 - Inner Join Test Time A B 3 6 9 12 15 10.95 11.04
Apache Spark Row Count: 10000000 - Partitions: 2000 - Group By Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 2000 - Group By Test Time A B 3 6 9 12 15 11.74 11.83
Apache Spark Row Count: 20000000 - Partitions: 500 - Calculate Pi Benchmark Using Dataframe OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 500 - Calculate Pi Benchmark Using Dataframe A B 1.017 2.034 3.051 4.068 5.085 4.520000000 4.488093463
Apache Spark Row Count: 1000000 - Partitions: 1000 - Calculate Pi Benchmark Using Dataframe OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 1000 - Calculate Pi Benchmark Using Dataframe A B 1.0013 2.0026 3.0039 4.0052 5.0065 4.42 4.45
Apache Spark Row Count: 20000000 - Partitions: 2000 - Calculate Pi Benchmark OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 2000 - Calculate Pi Benchmark A B 14 28 42 56 70 62.51 62.11
Apache Spark Row Count: 10000000 - Partitions: 2000 - Calculate Pi Benchmark OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 2000 - Calculate Pi Benchmark A B 14 28 42 56 70 61.55 61.94
Apache Spark Row Count: 1000000 - Partitions: 2000 - Calculate Pi Benchmark OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 2000 - Calculate Pi Benchmark A B 14 28 42 56 70 62.44 62.06
Apache Spark Row Count: 40000000 - Partitions: 2000 - SHA-512 Benchmark Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 2000 - SHA-512 Benchmark Time A B 10 20 30 40 50 44.33 44.06
SVT-AV1 Encoder Mode: Preset 4 - Input: Bosphorus 1080p OpenBenchmarking.org Frames Per Second, More Is Better SVT-AV1 1.2 Encoder Mode: Preset 4 - Input: Bosphorus 1080p A B 0.8028 1.6056 2.4084 3.2112 4.014 3.568 3.547 1. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq
Apache Spark Row Count: 10000000 - Partitions: 1000 - SHA-512 Benchmark Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 1000 - SHA-512 Benchmark Time A B 4 8 12 16 20 15.23 15.32
Apache Spark Row Count: 40000000 - Partitions: 1000 - Calculate Pi Benchmark OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 1000 - Calculate Pi Benchmark A B 14 28 42 56 70 61.93 62.28
Apache Spark Row Count: 40000000 - Partitions: 1000 - SHA-512 Benchmark Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 1000 - SHA-512 Benchmark Time A B 10 20 30 40 50 44.74 44.49
Apache Spark Row Count: 1000000 - Partitions: 100 - Calculate Pi Benchmark OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 100 - Calculate Pi Benchmark A B 14 28 42 56 70 61.13 61.43
Apache Spark Row Count: 20000000 - Partitions: 1000 - Calculate Pi Benchmark OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 1000 - Calculate Pi Benchmark A B 14 28 42 56 70 62.24 62.53
Apache Spark Row Count: 10000000 - Partitions: 1000 - Calculate Pi Benchmark OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 1000 - Calculate Pi Benchmark A B 14 28 42 56 70 61.58 61.86
Apache Spark Row Count: 20000000 - Partitions: 1000 - Calculate Pi Benchmark Using Dataframe OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 1000 - Calculate Pi Benchmark Using Dataframe A B 1.0463 2.0926 3.1389 4.1852 5.2315 4.635858138 4.650000000
Apache Spark Row Count: 1000000 - Partitions: 1000 - Group By Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 1000 - Group By Test Time A B 2 4 6 8 10 6.69 6.67
Apache Spark Row Count: 40000000 - Partitions: 100 - Calculate Pi Benchmark OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 100 - Calculate Pi Benchmark A B 14 28 42 56 70 62.14 61.96
Apache Spark Row Count: 1000000 - Partitions: 500 - Calculate Pi Benchmark Using Dataframe OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 500 - Calculate Pi Benchmark Using Dataframe A B 0.9996 1.9992 2.9988 3.9984 4.998 4.442555283 4.430000000
Apache Spark Row Count: 1000000 - Partitions: 500 - Calculate Pi Benchmark OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 500 - Calculate Pi Benchmark A B 14 28 42 56 70 61.74 61.91
Apache Spark Row Count: 40000000 - Partitions: 1000 - Calculate Pi Benchmark Using Dataframe OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 1000 - Calculate Pi Benchmark Using Dataframe A B 0.9968 1.9936 2.9904 3.9872 4.984 4.43 4.42
Apache Spark Row Count: 40000000 - Partitions: 100 - Calculate Pi Benchmark Using Dataframe OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 100 - Calculate Pi Benchmark Using Dataframe A B 0.999 1.998 2.997 3.996 4.995 4.43 4.44
Apache Spark Row Count: 40000000 - Partitions: 2000 - Calculate Pi Benchmark OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 2000 - Calculate Pi Benchmark A B 14 28 42 56 70 61.99 62.13
Apache Spark Row Count: 10000000 - Partitions: 500 - Calculate Pi Benchmark OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 500 - Calculate Pi Benchmark A B 14 28 42 56 70 61.82 61.70
Apache Spark Row Count: 10000000 - Partitions: 100 - Calculate Pi Benchmark OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 100 - Calculate Pi Benchmark A B 14 28 42 56 70 61.96 61.84
Apache Spark Row Count: 40000000 - Partitions: 1000 - Group By Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 1000 - Group By Test Time A B 8 16 24 32 40 33.86 33.81
Apache Spark Row Count: 40000000 - Partitions: 500 - Calculate Pi Benchmark OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 500 - Calculate Pi Benchmark A B 14 28 42 56 70 62.40 62.33
Apache Spark Row Count: 20000000 - Partitions: 500 - Calculate Pi Benchmark OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 500 - Calculate Pi Benchmark A B 14 28 42 56 70 62.03 62.07
Apache Spark Row Count: 40000000 - Partitions: 1000 - Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 1000 - Inner Join Test Time A B 8 16 24 32 40 34.82 34.82
Apache Spark Row Count: 20000000 - Partitions: 100 - Calculate Pi Benchmark Using Dataframe OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 100 - Calculate Pi Benchmark Using Dataframe A B 1.0035 2.007 3.0105 4.014 5.0175 4.46 4.46
Phoronix Test Suite v10.8.5