tr 2950X AMD Ryzen Threadripper 2950X 16-Core testing with a MSI MEG X399 CREATION (MS-7B92) v1.0 (1.30 BIOS) and AMD FirePro V4900 1GB on Ubuntu 22.04 via the Phoronix Test Suite.
HTML result view exported from: https://openbenchmarking.org/result/2208075-NE-TR2950X6958&grr&sor .
tr 2950X Processor Motherboard Chipset Memory Disk Graphics Audio Monitor Network OS Kernel Desktop Display Server OpenGL Vulkan Compiler File-System Screen Resolution A BN C AMD Ryzen Threadripper 2950X 16-Core @ 3.50GHz (16 Cores / 32 Threads) MSI MEG X399 CREATION (MS-7B92) v1.0 (1.30 BIOS) AMD 17h 32GB Samsung SSD 970 EVO 250GB AMD FirePro V4900 1GB Realtek ALC1220 DELL S2409W 2 x Intel I211 + Intel-AC 9260 Ubuntu 22.04 5.19.0-051900rc7-generic (x86_64) GNOME Shell 42.2 X Server + Wayland 3.3 Mesa 22.0.1 (LLVM 13.0.1) 1.2.204 GCC 11.2.0 ext4 1920x1080 OpenBenchmarking.org Kernel Details - Transparent Huge Pages: madvise Compiler Details - --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-link-serialization=2 --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-targets=nvptx-none=/build/gcc-11-gBFGDP/gcc-11-11.2.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-11-gBFGDP/gcc-11-11.2.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-build-config=bootstrap-lto-lean --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v Processor Details - Scaling Governor: acpi-cpufreq schedutil (Boost: Enabled) - CPU Microcode: 0x800820d Java Details - OpenJDK Runtime Environment (build 11.0.15+10-Ubuntu-0ubuntu0.22.04.1) Python Details - Python 3.10.4 Security Details - itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Mitigation of untrained return thunk; SMT vulnerable + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional STIBP: disabled RSB filling + srbds: Not affected + tsx_async_abort: Not affected
tr 2950X spark: 40000000 - 500 - Broadcast Inner Join Test Time spark: 40000000 - 500 - Inner Join Test Time spark: 40000000 - 500 - Repartition Test Time spark: 40000000 - 500 - Group By Test Time spark: 40000000 - 500 - Calculate Pi Benchmark Using Dataframe spark: 40000000 - 500 - Calculate Pi Benchmark spark: 40000000 - 500 - SHA-512 Benchmark Time spark: 10000000 - 1000 - Broadcast Inner Join Test Time spark: 10000000 - 1000 - Inner Join Test Time spark: 10000000 - 1000 - Repartition Test Time spark: 10000000 - 1000 - Group By Test Time spark: 10000000 - 1000 - Calculate Pi Benchmark Using Dataframe spark: 10000000 - 1000 - Calculate Pi Benchmark spark: 10000000 - 1000 - SHA-512 Benchmark Time spark: 40000000 - 100 - Broadcast Inner Join Test Time spark: 40000000 - 100 - Inner Join Test Time spark: 40000000 - 100 - Repartition Test Time spark: 40000000 - 100 - Group By Test Time spark: 40000000 - 100 - Calculate Pi Benchmark Using Dataframe spark: 40000000 - 100 - Calculate Pi Benchmark spark: 40000000 - 100 - SHA-512 Benchmark Time spark: 40000000 - 1000 - Broadcast Inner Join Test Time spark: 40000000 - 1000 - Inner Join Test Time spark: 40000000 - 1000 - Repartition Test Time spark: 40000000 - 1000 - Group By Test Time spark: 40000000 - 1000 - Calculate Pi Benchmark Using Dataframe spark: 40000000 - 1000 - Calculate Pi Benchmark spark: 40000000 - 1000 - SHA-512 Benchmark Time spark: 40000000 - 2000 - Broadcast Inner Join Test Time spark: 40000000 - 2000 - Inner Join Test Time spark: 40000000 - 2000 - Repartition Test Time spark: 40000000 - 2000 - Group By Test Time spark: 40000000 - 2000 - Calculate Pi Benchmark Using Dataframe spark: 40000000 - 2000 - Calculate Pi Benchmark spark: 40000000 - 2000 - SHA-512 Benchmark Time spark: 20000000 - 2000 - Broadcast Inner Join Test Time spark: 20000000 - 2000 - Inner Join Test Time spark: 20000000 - 2000 - Repartition Test Time spark: 20000000 - 2000 - Group By Test Time spark: 20000000 - 2000 - Calculate Pi Benchmark Using Dataframe spark: 20000000 - 2000 - Calculate Pi Benchmark spark: 20000000 - 2000 - SHA-512 Benchmark Time spark: 10000000 - 500 - Broadcast Inner Join Test Time spark: 10000000 - 500 - Inner Join Test Time spark: 10000000 - 500 - Repartition Test Time spark: 10000000 - 500 - Group By Test Time spark: 10000000 - 500 - Calculate Pi Benchmark Using Dataframe spark: 10000000 - 500 - Calculate Pi Benchmark spark: 10000000 - 500 - SHA-512 Benchmark Time spark: 1000000 - 500 - Broadcast Inner Join Test Time spark: 1000000 - 500 - Inner Join Test Time spark: 1000000 - 500 - Repartition Test Time spark: 1000000 - 500 - Group By Test Time spark: 1000000 - 500 - Calculate Pi Benchmark Using Dataframe spark: 1000000 - 500 - Calculate Pi Benchmark spark: 1000000 - 500 - SHA-512 Benchmark Time spark: 20000000 - 100 - Broadcast Inner Join Test Time spark: 20000000 - 100 - Inner Join Test Time spark: 20000000 - 100 - Repartition Test Time spark: 20000000 - 100 - Group By Test Time spark: 20000000 - 100 - Calculate Pi Benchmark Using Dataframe spark: 20000000 - 100 - Calculate Pi Benchmark spark: 20000000 - 100 - SHA-512 Benchmark Time spark: 20000000 - 500 - Broadcast Inner Join Test Time spark: 20000000 - 500 - Inner Join Test Time spark: 20000000 - 500 - Repartition Test Time spark: 20000000 - 500 - Group By Test Time spark: 20000000 - 500 - Calculate Pi Benchmark Using Dataframe spark: 20000000 - 500 - Calculate Pi Benchmark spark: 20000000 - 500 - SHA-512 Benchmark Time spark: 1000000 - 2000 - Broadcast Inner Join Test Time spark: 1000000 - 2000 - Inner Join Test Time spark: 1000000 - 2000 - Repartition Test Time spark: 1000000 - 2000 - Group By Test Time spark: 1000000 - 2000 - Calculate Pi Benchmark Using Dataframe spark: 1000000 - 2000 - Calculate Pi Benchmark spark: 1000000 - 2000 - SHA-512 Benchmark Time spark: 10000000 - 2000 - Broadcast Inner Join Test Time spark: 10000000 - 2000 - Inner Join Test Time spark: 10000000 - 2000 - Repartition Test Time spark: 10000000 - 2000 - Group By Test Time spark: 10000000 - 2000 - Calculate Pi Benchmark Using Dataframe spark: 10000000 - 2000 - Calculate Pi Benchmark spark: 10000000 - 2000 - SHA-512 Benchmark Time spark: 10000000 - 100 - Broadcast Inner Join Test Time spark: 10000000 - 100 - Inner Join Test Time spark: 10000000 - 100 - Repartition Test Time spark: 10000000 - 100 - Group By Test Time spark: 10000000 - 100 - Calculate Pi Benchmark Using Dataframe spark: 10000000 - 100 - Calculate Pi Benchmark spark: 10000000 - 100 - SHA-512 Benchmark Time spark: 1000000 - 100 - Broadcast Inner Join Test Time spark: 1000000 - 100 - Inner Join Test Time spark: 1000000 - 100 - Repartition Test Time spark: 1000000 - 100 - Group By Test Time spark: 1000000 - 100 - Calculate Pi Benchmark Using Dataframe spark: 1000000 - 100 - Calculate Pi Benchmark spark: 1000000 - 100 - SHA-512 Benchmark Time spark: 20000000 - 1000 - Broadcast Inner Join Test Time spark: 20000000 - 1000 - Inner Join Test Time spark: 20000000 - 1000 - Repartition Test Time spark: 20000000 - 1000 - Group By Test Time spark: 20000000 - 1000 - Calculate Pi Benchmark Using Dataframe spark: 20000000 - 1000 - Calculate Pi Benchmark spark: 20000000 - 1000 - SHA-512 Benchmark Time spark: 1000000 - 1000 - Broadcast Inner Join Test Time spark: 1000000 - 1000 - Inner Join Test Time spark: 1000000 - 1000 - Repartition Test Time spark: 1000000 - 1000 - Group By Test Time spark: 1000000 - 1000 - Calculate Pi Benchmark Using Dataframe spark: 1000000 - 1000 - Calculate Pi Benchmark spark: 1000000 - 1000 - SHA-512 Benchmark Time svt-av1: Preset 4 - Bosphorus 4K svt-av1: Preset 4 - Bosphorus 1080p svt-av1: Preset 8 - Bosphorus 4K svt-av1: Preset 10 - Bosphorus 4K svt-av1: Preset 8 - Bosphorus 1080p svt-av1: Preset 12 - Bosphorus 4K svt-av1: Preset 10 - Bosphorus 1080p svt-av1: Preset 12 - Bosphorus 1080p A BN C 37.86012772 38.916631644 31.759709462 31.80822044 6.72 109.231835166 45.815407438 10.412153908 10.226908103 9.35 9.56 6.70 109.522356865 14.004523565 40.089747144 40.835276383 34.185559705 32.423043313 6.84 110.523657566 46.523112355 36.818880624 37.542600102 31.448323516 29.126483464 6.86 109.896421862 44.777573634 36.697244482 37.991943155 30.648054198 25.818363399 6.86 110.085285547 43.859768868 19.056877299 20.393716972 16.431636403 13.22601236 6.77 109.43976691 23.247350909 10.016525853 10.444853075 9.09 8.621025209 6.759258954 109.230311687 13.198824567 1.70007341 2.170995055 2.65 5.504610213 6.77685602 108.225955158 4.37 20.754541595 20.683734622 17.154390228 13.329626392 6.80 109.108538715 24.428886208 18.428674686 18.785819235 16.640636577 11.890085005 6.812744629 109.612467287 22.754111465 2.32 3.09 2.97 6.17 6.84 108.597289108 4.904759563 9.96 10.997072357 9.30 9.80 6.76 109.91105171 14.139350937 10.662243237 10.764697533 9.47 9.439752516 6.81 108.688367026 13.931141663 1.972517798 2.10 2.700264051 5.876942244 6.783271558 108.419781632 4.530063798 19.237627995 19.682438668 15.892958165 13.180563735 6.875336993 110.730128235 23.235006524 1.92 2.43 2.87 5.37 6.799198247 109.807174541 4.34 1.384 3.593 31.995 61.388 75.963 89.752 162.279 221.104 38.99 37.94 32.56 31.26 6.86 109.940453650 46.27 10.66 10.78 9.59 9.67 6.80 109.968195448 14.34 41.27 39.98 32.99 32.11 6.79 109.746997331 46.40 36.73 36.52 31.51 29.34 6.87 110.394615796 43.90 35.82 38.80 31.54 27.22 6.84 110.551039063 43.42 19.27 20.55 16.69 12.90 6.79 109.896984405 23.69 9.75 9.73 8.93 8.49 6.78 109.329856012 13.09 1.77 2.25 2.72 5.57 6.81 109.907407989 4.54 20.80 20.35 17.38 13.40 6.82 109.576906446 24.63 18.37 19.29 16.16 11.99 6.80 109.971942948 22.58 2.41 3.17 3.38 6.27 6.79 110.039838817 4.99 10.83 10.83 9.45 9.89 6.78 110.250973756 14.04 10.69 10.47 9.31 9.59 6.80 109.357033407 14.19 1.88 2.37 2.81 6.01 6.89 110.166207147 4.62 19.61 19.95 16.40 13.20 6.80 110.258647571 24.89 2.05 2.45 2.94 5.48 6.82 109.680205103 4.65 1.379 3.530 32.129 60.846 76.005 89.460 158.929 223.442 39.13 39.06 32.55 31.45 6.84 110.39 46.14 10.87 10.80 9.46 9.87 6.81 110.105280590 14.30 39.81 40.12 33.29 32.49 6.92 111.43 48.05 35.45 36.58 31.02 29.43 6.74 110.273490935 41.003127286 36.74 39.31 31.27 27.267906089 6.76 110.588119919 44.72 19.62 20.30 16.30 13.05 6.77 109.91605971 23.55 9.88 10.19 9.12 8.50 6.82 110.539073220 13.29 1.79 2.26 2.74 5.44 6.81 110.357863946 4.43 21.22 20.60 17.37 13.35 6.77 109.616776140 25.02 18.69 19.06 16.33 11.77 6.86 109.977979218 22.64 2.48 3.12 3.39 6.31 6.81 110.12 4.92 10.54 10.66 9.51 10.21 6.84 109.96 14.22 10.72 10.41 9.30 9.68 6.85 110.411376802 14.01 1.75 2.13 2.73 5.97 6.86 109.870024098 4.50 20.55 19.85 16.67 12.99 6.88 109.959690974 25.45 2.03 2.43 2.82 5.55 6.84 110.057783068 4.64 1.379 3.566 32.021 60.942 75.225 89.366 156.492 219.616 OpenBenchmarking.org
Apache Spark Row Count: 40000000 - Partitions: 500 - Broadcast Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 500 - Broadcast Inner Join Test Time A BN C 9 18 27 36 45 SE +/- 0.14, N = 5 SE +/- 0.24, N = 9 37.86 38.99 39.13
Apache Spark Row Count: 40000000 - Partitions: 500 - Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 500 - Inner Join Test Time BN A C 9 18 27 36 45 SE +/- 0.49, N = 5 SE +/- 0.39, N = 9 37.94 38.92 39.06
Apache Spark Row Count: 40000000 - Partitions: 500 - Repartition Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 500 - Repartition Test Time A C BN 8 16 24 32 40 SE +/- 0.04, N = 9 SE +/- 0.16, N = 5 31.76 32.55 32.56
Apache Spark Row Count: 40000000 - Partitions: 500 - Group By Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 500 - Group By Test Time BN C A 7 14 21 28 35 SE +/- 0.10, N = 5 SE +/- 0.11, N = 9 31.26 31.45 31.81
Apache Spark Row Count: 40000000 - Partitions: 500 - Calculate Pi Benchmark Using Dataframe OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 500 - Calculate Pi Benchmark Using Dataframe A C BN 2 4 6 8 10 SE +/- 0.04, N = 9 SE +/- 0.02, N = 5 6.72 6.84 6.86
Apache Spark Row Count: 40000000 - Partitions: 500 - Calculate Pi Benchmark OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 500 - Calculate Pi Benchmark A BN C 20 40 60 80 100 SE +/- 0.12, N = 5 SE +/- 0.09, N = 9 109.23 109.94 110.39
Apache Spark Row Count: 40000000 - Partitions: 500 - SHA-512 Benchmark Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 500 - SHA-512 Benchmark Time A C BN 10 20 30 40 50 SE +/- 0.47, N = 9 SE +/- 0.47, N = 5 45.82 46.14 46.27
Apache Spark Row Count: 10000000 - Partitions: 1000 - Broadcast Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 1000 - Broadcast Inner Join Test Time A BN C 3 6 9 12 15 SE +/- 0.11, N = 12 SE +/- 0.11, N = 12 10.41 10.66 10.87
Apache Spark Row Count: 10000000 - Partitions: 1000 - Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 1000 - Inner Join Test Time A BN C 3 6 9 12 15 SE +/- 0.13, N = 12 SE +/- 0.13, N = 12 10.23 10.78 10.80
Apache Spark Row Count: 10000000 - Partitions: 1000 - Repartition Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 1000 - Repartition Test Time A C BN 3 6 9 12 15 SE +/- 0.06, N = 12 SE +/- 0.07, N = 12 9.35 9.46 9.59
Apache Spark Row Count: 10000000 - Partitions: 1000 - Group By Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 1000 - Group By Test Time A BN C 3 6 9 12 15 SE +/- 0.05, N = 12 SE +/- 0.06, N = 12 9.56 9.67 9.87
Apache Spark Row Count: 10000000 - Partitions: 1000 - Calculate Pi Benchmark Using Dataframe OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 1000 - Calculate Pi Benchmark Using Dataframe A BN C 2 4 6 8 10 SE +/- 0.01, N = 12 SE +/- 0.01, N = 12 6.70 6.80 6.81
Apache Spark Row Count: 10000000 - Partitions: 1000 - Calculate Pi Benchmark OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 1000 - Calculate Pi Benchmark A BN C 20 40 60 80 100 SE +/- 0.13, N = 12 SE +/- 0.15, N = 12 109.52 109.97 110.11
Apache Spark Row Count: 10000000 - Partitions: 1000 - SHA-512 Benchmark Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 1000 - SHA-512 Benchmark Time A C BN 4 8 12 16 20 SE +/- 0.11, N = 12 SE +/- 0.14, N = 12 14.00 14.30 14.34
Apache Spark Row Count: 40000000 - Partitions: 100 - Broadcast Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 100 - Broadcast Inner Join Test Time C A BN 9 18 27 36 45 SE +/- 0.52, N = 3 SE +/- 0.37, N = 9 39.81 40.09 41.27
Apache Spark Row Count: 40000000 - Partitions: 100 - Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 100 - Inner Join Test Time BN C A 9 18 27 36 45 SE +/- 0.28, N = 9 SE +/- 0.17, N = 3 39.98 40.12 40.84
Apache Spark Row Count: 40000000 - Partitions: 100 - Repartition Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 100 - Repartition Test Time BN C A 8 16 24 32 40 SE +/- 0.04, N = 9 SE +/- 0.48, N = 3 32.99 33.29 34.19
Apache Spark Row Count: 40000000 - Partitions: 100 - Group By Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 100 - Group By Test Time BN A C 8 16 24 32 40 SE +/- 0.07, N = 9 SE +/- 0.61, N = 3 32.11 32.42 32.49
Apache Spark Row Count: 40000000 - Partitions: 100 - Calculate Pi Benchmark Using Dataframe OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 100 - Calculate Pi Benchmark Using Dataframe BN A C 2 4 6 8 10 SE +/- 0.01, N = 9 SE +/- 0.06, N = 3 6.79 6.84 6.92
Apache Spark Row Count: 40000000 - Partitions: 100 - Calculate Pi Benchmark OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 100 - Calculate Pi Benchmark BN A C 20 40 60 80 100 SE +/- 0.15, N = 9 SE +/- 0.92, N = 3 109.75 110.52 111.43
Apache Spark Row Count: 40000000 - Partitions: 100 - SHA-512 Benchmark Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 100 - SHA-512 Benchmark Time BN A C 11 22 33 44 55 SE +/- 0.41, N = 9 SE +/- 0.33, N = 3 46.40 46.52 48.05
Apache Spark Row Count: 40000000 - Partitions: 1000 - Broadcast Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 1000 - Broadcast Inner Join Test Time C BN A 8 16 24 32 40 SE +/- 0.21, N = 9 35.45 36.73 36.82
Apache Spark Row Count: 40000000 - Partitions: 1000 - Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 1000 - Inner Join Test Time BN C A 9 18 27 36 45 SE +/- 0.26, N = 9 36.52 36.58 37.54
Apache Spark Row Count: 40000000 - Partitions: 1000 - Repartition Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 1000 - Repartition Test Time C A BN 7 14 21 28 35 SE +/- 0.18, N = 9 31.02 31.45 31.51
Apache Spark Row Count: 40000000 - Partitions: 1000 - Group By Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 1000 - Group By Test Time A BN C 7 14 21 28 35 SE +/- 0.28, N = 9 29.13 29.34 29.43
Apache Spark Row Count: 40000000 - Partitions: 1000 - Calculate Pi Benchmark Using Dataframe OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 1000 - Calculate Pi Benchmark Using Dataframe C A BN 2 4 6 8 10 SE +/- 0.03, N = 9 6.74 6.86 6.87
Apache Spark Row Count: 40000000 - Partitions: 1000 - Calculate Pi Benchmark OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 1000 - Calculate Pi Benchmark A C BN 20 40 60 80 100 SE +/- 0.19, N = 9 109.90 110.27 110.39
Apache Spark Row Count: 40000000 - Partitions: 1000 - SHA-512 Benchmark Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 1000 - SHA-512 Benchmark Time C BN A 10 20 30 40 50 SE +/- 0.53, N = 9 41.00 43.90 44.78
Apache Spark Row Count: 40000000 - Partitions: 2000 - Broadcast Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 2000 - Broadcast Inner Join Test Time BN A C 8 16 24 32 40 SE +/- 0.15, N = 9 35.82 36.70 36.74
Apache Spark Row Count: 40000000 - Partitions: 2000 - Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 2000 - Inner Join Test Time A BN C 9 18 27 36 45 SE +/- 0.21, N = 9 37.99 38.80 39.31
Apache Spark Row Count: 40000000 - Partitions: 2000 - Repartition Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 2000 - Repartition Test Time A C BN 7 14 21 28 35 SE +/- 0.09, N = 9 30.65 31.27 31.54
Apache Spark Row Count: 40000000 - Partitions: 2000 - Group By Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 2000 - Group By Test Time A BN C 6 12 18 24 30 SE +/- 0.38, N = 9 25.82 27.22 27.27
Apache Spark Row Count: 40000000 - Partitions: 2000 - Calculate Pi Benchmark Using Dataframe OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 2000 - Calculate Pi Benchmark Using Dataframe C BN A 2 4 6 8 10 SE +/- 0.02, N = 9 6.76 6.84 6.86
Apache Spark Row Count: 40000000 - Partitions: 2000 - Calculate Pi Benchmark OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 2000 - Calculate Pi Benchmark A BN C 20 40 60 80 100 SE +/- 0.21, N = 9 110.09 110.55 110.59
Apache Spark Row Count: 40000000 - Partitions: 2000 - SHA-512 Benchmark Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 2000 - SHA-512 Benchmark Time BN A C 10 20 30 40 50 SE +/- 0.49, N = 9 43.42 43.86 44.72
Apache Spark Row Count: 20000000 - Partitions: 2000 - Broadcast Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 2000 - Broadcast Inner Join Test Time A BN C 5 10 15 20 25 SE +/- 0.17, N = 12 19.06 19.27 19.62
Apache Spark Row Count: 20000000 - Partitions: 2000 - Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 2000 - Inner Join Test Time C A BN 5 10 15 20 25 SE +/- 0.19, N = 12 20.30 20.39 20.55
Apache Spark Row Count: 20000000 - Partitions: 2000 - Repartition Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 2000 - Repartition Test Time C A BN 4 8 12 16 20 SE +/- 0.10, N = 12 16.30 16.43 16.69
Apache Spark Row Count: 20000000 - Partitions: 2000 - Group By Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 2000 - Group By Test Time BN C A 3 6 9 12 15 SE +/- 0.06, N = 12 12.90 13.05 13.23
Apache Spark Row Count: 20000000 - Partitions: 2000 - Calculate Pi Benchmark Using Dataframe OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 2000 - Calculate Pi Benchmark Using Dataframe A C BN 2 4 6 8 10 SE +/- 0.01, N = 12 6.77 6.77 6.79
Apache Spark Row Count: 20000000 - Partitions: 2000 - Calculate Pi Benchmark OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 2000 - Calculate Pi Benchmark A BN C 20 40 60 80 100 SE +/- 0.12, N = 12 109.44 109.90 109.92
Apache Spark Row Count: 20000000 - Partitions: 2000 - SHA-512 Benchmark Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 2000 - SHA-512 Benchmark Time A C BN 6 12 18 24 30 SE +/- 0.16, N = 12 23.25 23.55 23.69
Apache Spark Row Count: 10000000 - Partitions: 500 - Broadcast Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 500 - Broadcast Inner Join Test Time BN C A 3 6 9 12 15 SE +/- 0.085017131, N = 3 SE +/- 0.094306283, N = 6 9.750000000 9.880000000 10.016525853
Apache Spark Row Count: 10000000 - Partitions: 500 - Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 500 - Inner Join Test Time BN C A 3 6 9 12 15 SE +/- 0.118870708, N = 3 SE +/- 0.144346648, N = 6 9.730000000 10.190000000 10.444853075
Apache Spark Row Count: 10000000 - Partitions: 500 - Repartition Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 500 - Repartition Test Time BN A C 3 6 9 12 15 SE +/- 0.19, N = 3 SE +/- 0.10, N = 6 8.93 9.09 9.12
Apache Spark Row Count: 10000000 - Partitions: 500 - Group By Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 500 - Group By Test Time BN C A 2 4 6 8 10 SE +/- 0.069704950, N = 3 SE +/- 0.074190780, N = 6 8.490000000 8.500000000 8.621025209
Apache Spark Row Count: 10000000 - Partitions: 500 - Calculate Pi Benchmark Using Dataframe OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 500 - Calculate Pi Benchmark Using Dataframe A BN C 2 4 6 8 10 SE +/- 0.003533182, N = 3 SE +/- 0.025564809, N = 6 6.759258954 6.780000000 6.820000000
Apache Spark Row Count: 10000000 - Partitions: 500 - Calculate Pi Benchmark OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 500 - Calculate Pi Benchmark A BN C 20 40 60 80 100 SE +/- 0.30, N = 3 SE +/- 0.25, N = 6 109.23 109.33 110.54
Apache Spark Row Count: 10000000 - Partitions: 500 - SHA-512 Benchmark Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 500 - SHA-512 Benchmark Time BN A C 3 6 9 12 15 SE +/- 0.14, N = 3 SE +/- 0.13, N = 6 13.09 13.20 13.29
Apache Spark Row Count: 1000000 - Partitions: 500 - Broadcast Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 500 - Broadcast Inner Join Test Time A BN C 0.4028 0.8056 1.2084 1.6112 2.014 SE +/- 0.05474021, N = 8 SE +/- 0.04087612, N = 3 1.70007341 1.77000000 1.79000000
Apache Spark Row Count: 1000000 - Partitions: 500 - Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 500 - Inner Join Test Time A BN C 0.5085 1.017 1.5255 2.034 2.5425 SE +/- 0.028445371, N = 8 SE +/- 0.110292282, N = 3 2.170995055 2.250000000 2.260000000
Apache Spark Row Count: 1000000 - Partitions: 500 - Repartition Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 500 - Repartition Test Time A BN C 0.6165 1.233 1.8495 2.466 3.0825 SE +/- 0.04, N = 8 SE +/- 0.05, N = 3 2.65 2.72 2.74
Apache Spark Row Count: 1000000 - Partitions: 500 - Group By Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 500 - Group By Test Time C A BN 1.2533 2.5066 3.7599 5.0132 6.2665 SE +/- 0.043659720, N = 3 SE +/- 0.029315415, N = 8 5.440000000 5.504610213 5.570000000
Apache Spark Row Count: 1000000 - Partitions: 500 - Calculate Pi Benchmark Using Dataframe OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 500 - Calculate Pi Benchmark Using Dataframe A BN C 2 4 6 8 10 SE +/- 0.02582423, N = 8 SE +/- 0.02772713, N = 3 6.77685602 6.81000000 6.81000000
Apache Spark Row Count: 1000000 - Partitions: 500 - Calculate Pi Benchmark OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 500 - Calculate Pi Benchmark A BN C 20 40 60 80 100 SE +/- 0.12, N = 8 SE +/- 0.32, N = 3 108.23 109.91 110.36
Apache Spark Row Count: 1000000 - Partitions: 500 - SHA-512 Benchmark Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 500 - SHA-512 Benchmark Time A C BN 1.0215 2.043 3.0645 4.086 5.1075 SE +/- 0.05, N = 3 SE +/- 0.04, N = 8 4.37 4.43 4.54
Apache Spark Row Count: 20000000 - Partitions: 100 - Broadcast Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 100 - Broadcast Inner Join Test Time A BN C 5 10 15 20 25 SE +/- 0.24, N = 3 SE +/- 0.25, N = 3 20.75 20.80 21.22
Apache Spark Row Count: 20000000 - Partitions: 100 - Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 100 - Inner Join Test Time BN C A 5 10 15 20 25 SE +/- 0.71, N = 3 SE +/- 0.24, N = 3 20.35 20.60 20.68
Apache Spark Row Count: 20000000 - Partitions: 100 - Repartition Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 100 - Repartition Test Time A C BN 4 8 12 16 20 SE +/- 0.07, N = 3 SE +/- 0.11, N = 3 17.15 17.37 17.38
Apache Spark Row Count: 20000000 - Partitions: 100 - Group By Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 100 - Group By Test Time A C BN 3 6 9 12 15 SE +/- 0.07, N = 3 SE +/- 0.20, N = 3 13.33 13.35 13.40
Apache Spark Row Count: 20000000 - Partitions: 100 - Calculate Pi Benchmark Using Dataframe OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 100 - Calculate Pi Benchmark Using Dataframe C A BN 2 4 6 8 10 SE +/- 0.03, N = 3 SE +/- 0.01, N = 3 6.77 6.80 6.82
Apache Spark Row Count: 20000000 - Partitions: 100 - Calculate Pi Benchmark OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 100 - Calculate Pi Benchmark A BN C 20 40 60 80 100 SE +/- 0.21, N = 3 SE +/- 0.14, N = 3 109.11 109.58 109.62
Apache Spark Row Count: 20000000 - Partitions: 100 - SHA-512 Benchmark Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 100 - SHA-512 Benchmark Time A BN C 6 12 18 24 30 SE +/- 0.03, N = 3 SE +/- 0.27, N = 3 24.43 24.63 25.02
Apache Spark Row Count: 20000000 - Partitions: 500 - Broadcast Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 500 - Broadcast Inner Join Test Time BN A C 5 10 15 20 25 SE +/- 0.30, N = 3 SE +/- 0.12, N = 3 18.37 18.43 18.69
Apache Spark Row Count: 20000000 - Partitions: 500 - Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 500 - Inner Join Test Time A C BN 5 10 15 20 25 SE +/- 0.15, N = 3 SE +/- 0.35, N = 3 18.79 19.06 19.29
Apache Spark Row Count: 20000000 - Partitions: 500 - Repartition Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 500 - Repartition Test Time BN C A 4 8 12 16 20 SE +/- 0.26, N = 3 SE +/- 0.17, N = 3 16.16 16.33 16.64
Apache Spark Row Count: 20000000 - Partitions: 500 - Group By Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 500 - Group By Test Time C A BN 3 6 9 12 15 SE +/- 0.07, N = 3 SE +/- 0.11, N = 3 11.77 11.89 11.99
Apache Spark Row Count: 20000000 - Partitions: 500 - Calculate Pi Benchmark Using Dataframe OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 500 - Calculate Pi Benchmark Using Dataframe BN A C 2 4 6 8 10 SE +/- 0.034639869, N = 3 SE +/- 0.012838149, N = 3 6.800000000 6.812744629 6.860000000
Apache Spark Row Count: 20000000 - Partitions: 500 - Calculate Pi Benchmark OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 500 - Calculate Pi Benchmark A BN C 20 40 60 80 100 SE +/- 0.13, N = 3 SE +/- 0.17, N = 3 109.61 109.97 109.98
Apache Spark Row Count: 20000000 - Partitions: 500 - SHA-512 Benchmark Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 500 - SHA-512 Benchmark Time BN C A 5 10 15 20 25 SE +/- 0.30, N = 3 SE +/- 0.27, N = 3 22.58 22.64 22.75
Apache Spark Row Count: 1000000 - Partitions: 2000 - Broadcast Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 2000 - Broadcast Inner Join Test Time A BN C 0.558 1.116 1.674 2.232 2.79 SE +/- 0.05, N = 6 SE +/- 0.07, N = 3 2.32 2.41 2.48
Apache Spark Row Count: 1000000 - Partitions: 2000 - Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 2000 - Inner Join Test Time A C BN 0.7133 1.4266 2.1399 2.8532 3.5665 SE +/- 0.09, N = 3 SE +/- 0.04, N = 6 3.09 3.12 3.17
Apache Spark Row Count: 1000000 - Partitions: 2000 - Repartition Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 2000 - Repartition Test Time A BN C 0.7628 1.5256 2.2884 3.0512 3.814 SE +/- 0.04, N = 6 SE +/- 0.02, N = 3 2.97 3.38 3.39
Apache Spark Row Count: 1000000 - Partitions: 2000 - Group By Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 2000 - Group By Test Time A BN C 2 4 6 8 10 SE +/- 0.04, N = 6 SE +/- 0.10, N = 3 6.17 6.27 6.31
Apache Spark Row Count: 1000000 - Partitions: 2000 - Calculate Pi Benchmark Using Dataframe OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 2000 - Calculate Pi Benchmark Using Dataframe BN C A 2 4 6 8 10 SE +/- 0.02, N = 6 SE +/- 0.05, N = 3 6.79 6.81 6.84
Apache Spark Row Count: 1000000 - Partitions: 2000 - Calculate Pi Benchmark OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 2000 - Calculate Pi Benchmark A BN C 20 40 60 80 100 SE +/- 0.17, N = 6 SE +/- 0.24, N = 3 108.60 110.04 110.12
Apache Spark Row Count: 1000000 - Partitions: 2000 - SHA-512 Benchmark Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 2000 - SHA-512 Benchmark Time A C BN 1.1228 2.2456 3.3684 4.4912 5.614 SE +/- 0.049353750, N = 3 SE +/- 0.048557793, N = 6 4.904759563 4.920000000 4.990000000
Apache Spark Row Count: 10000000 - Partitions: 2000 - Broadcast Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 2000 - Broadcast Inner Join Test Time A C BN 3 6 9 12 15 SE +/- 0.50, N = 3 SE +/- 0.35, N = 3 9.96 10.54 10.83
Apache Spark Row Count: 10000000 - Partitions: 2000 - Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 2000 - Inner Join Test Time C BN A 3 6 9 12 15 SE +/- 0.14, N = 3 SE +/- 0.15, N = 3 10.66 10.83 11.00
Apache Spark Row Count: 10000000 - Partitions: 2000 - Repartition Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 2000 - Repartition Test Time A BN C 3 6 9 12 15 SE +/- 0.06, N = 3 SE +/- 0.06, N = 3 9.30 9.45 9.51
Apache Spark Row Count: 10000000 - Partitions: 2000 - Group By Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 2000 - Group By Test Time A BN C 3 6 9 12 15 SE +/- 0.02, N = 3 SE +/- 0.19, N = 3 9.80 9.89 10.21
Apache Spark Row Count: 10000000 - Partitions: 2000 - Calculate Pi Benchmark Using Dataframe OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 2000 - Calculate Pi Benchmark Using Dataframe A BN C 2 4 6 8 10 SE +/- 0.02, N = 3 SE +/- 0.07, N = 3 6.76 6.78 6.84
Apache Spark Row Count: 10000000 - Partitions: 2000 - Calculate Pi Benchmark OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 2000 - Calculate Pi Benchmark A C BN 20 40 60 80 100 SE +/- 0.22, N = 3 SE +/- 0.63, N = 3 109.91 109.96 110.25
Apache Spark Row Count: 10000000 - Partitions: 2000 - SHA-512 Benchmark Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 2000 - SHA-512 Benchmark Time BN A C 4 8 12 16 20 SE +/- 0.02, N = 3 SE +/- 0.07, N = 3 14.04 14.14 14.22
Apache Spark Row Count: 10000000 - Partitions: 100 - Broadcast Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 100 - Broadcast Inner Join Test Time A BN C 3 6 9 12 15 SE +/- 0.24, N = 3 SE +/- 0.11, N = 3 10.66 10.69 10.72
Apache Spark Row Count: 10000000 - Partitions: 100 - Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 100 - Inner Join Test Time C BN A 3 6 9 12 15 SE +/- 0.18, N = 3 SE +/- 0.26, N = 3 10.41 10.47 10.76
Apache Spark Row Count: 10000000 - Partitions: 100 - Repartition Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 100 - Repartition Test Time C BN A 3 6 9 12 15 SE +/- 0.09, N = 3 SE +/- 0.16, N = 3 9.30 9.31 9.47
Apache Spark Row Count: 10000000 - Partitions: 100 - Group By Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 100 - Group By Test Time A BN C 3 6 9 12 15 SE +/- 0.023359039, N = 3 SE +/- 0.055018199, N = 3 9.439752516 9.590000000 9.680000000
Apache Spark Row Count: 10000000 - Partitions: 100 - Calculate Pi Benchmark Using Dataframe OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 100 - Calculate Pi Benchmark Using Dataframe BN A C 2 4 6 8 10 SE +/- 0.00, N = 3 SE +/- 0.03, N = 3 6.80 6.81 6.85
Apache Spark Row Count: 10000000 - Partitions: 100 - Calculate Pi Benchmark OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 100 - Calculate Pi Benchmark A BN C 20 40 60 80 100 SE +/- 0.17, N = 3 SE +/- 0.10, N = 3 108.69 109.36 110.41
Apache Spark Row Count: 10000000 - Partitions: 100 - SHA-512 Benchmark Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 100 - SHA-512 Benchmark Time A C BN 4 8 12 16 20 SE +/- 0.10, N = 3 SE +/- 0.07, N = 3 13.93 14.01 14.19
Apache Spark Row Count: 1000000 - Partitions: 100 - Broadcast Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 100 - Broadcast Inner Join Test Time C BN A 0.4438 0.8876 1.3314 1.7752 2.219 SE +/- 0.055547846, N = 4 SE +/- 0.059135782, N = 3 1.750000000 1.880000000 1.972517798
Apache Spark Row Count: 1000000 - Partitions: 100 - Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 100 - Inner Join Test Time A C BN 0.5333 1.0666 1.5999 2.1332 2.6665 SE +/- 0.05, N = 4 SE +/- 0.07, N = 3 2.10 2.13 2.37
Apache Spark Row Count: 1000000 - Partitions: 100 - Repartition Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 100 - Repartition Test Time A C BN 0.6323 1.2646 1.8969 2.5292 3.1615 SE +/- 0.052181616, N = 4 SE +/- 0.018430805, N = 3 2.700264051 2.730000000 2.810000000
Apache Spark Row Count: 1000000 - Partitions: 100 - Group By Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 100 - Group By Test Time A C BN 2 4 6 8 10 SE +/- 0.080220571, N = 4 SE +/- 0.060621777, N = 3 5.876942244 5.970000000 6.010000000
Apache Spark Row Count: 1000000 - Partitions: 100 - Calculate Pi Benchmark Using Dataframe OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 100 - Calculate Pi Benchmark Using Dataframe A C BN 2 4 6 8 10 SE +/- 0.022438723, N = 4 SE +/- 0.068811394, N = 3 6.783271558 6.860000000 6.890000000
Apache Spark Row Count: 1000000 - Partitions: 100 - Calculate Pi Benchmark OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 100 - Calculate Pi Benchmark A C BN 20 40 60 80 100 SE +/- 0.10, N = 4 SE +/- 0.21, N = 3 108.42 109.87 110.17
Apache Spark Row Count: 1000000 - Partitions: 100 - SHA-512 Benchmark Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 100 - SHA-512 Benchmark Time C A BN 1.0395 2.079 3.1185 4.158 5.1975 SE +/- 0.055483981, N = 4 SE +/- 0.034411464, N = 3 4.500000000 4.530063798 4.620000000
Apache Spark Row Count: 20000000 - Partitions: 1000 - Broadcast Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 1000 - Broadcast Inner Join Test Time A BN C 5 10 15 20 25 SE +/- 0.21, N = 3 19.24 19.61 20.55
Apache Spark Row Count: 20000000 - Partitions: 1000 - Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 1000 - Inner Join Test Time A C BN 5 10 15 20 25 SE +/- 0.31, N = 3 19.68 19.85 19.95
Apache Spark Row Count: 20000000 - Partitions: 1000 - Repartition Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 1000 - Repartition Test Time A BN C 4 8 12 16 20 SE +/- 0.18, N = 3 15.89 16.40 16.67
Apache Spark Row Count: 20000000 - Partitions: 1000 - Group By Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 1000 - Group By Test Time C A BN 3 6 9 12 15 SE +/- 0.15, N = 3 12.99 13.18 13.20
Apache Spark Row Count: 20000000 - Partitions: 1000 - Calculate Pi Benchmark Using Dataframe OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 1000 - Calculate Pi Benchmark Using Dataframe BN A C 2 4 6 8 10 SE +/- 0.046089856, N = 3 6.800000000 6.875336993 6.880000000
Apache Spark Row Count: 20000000 - Partitions: 1000 - Calculate Pi Benchmark OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 1000 - Calculate Pi Benchmark C BN A 20 40 60 80 100 SE +/- 0.19, N = 3 109.96 110.26 110.73
Apache Spark Row Count: 20000000 - Partitions: 1000 - SHA-512 Benchmark Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 1000 - SHA-512 Benchmark Time A BN C 6 12 18 24 30 SE +/- 0.11, N = 3 23.24 24.89 25.45
Apache Spark Row Count: 1000000 - Partitions: 1000 - Broadcast Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 1000 - Broadcast Inner Join Test Time A C BN 0.4613 0.9226 1.3839 1.8452 2.3065 SE +/- 0.06, N = 3 SE +/- 0.06, N = 3 1.92 2.03 2.05
Apache Spark Row Count: 1000000 - Partitions: 1000 - Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 1000 - Inner Join Test Time A C BN 0.5513 1.1026 1.6539 2.2052 2.7565 SE +/- 0.04, N = 3 SE +/- 0.05, N = 3 2.43 2.43 2.45
Apache Spark Row Count: 1000000 - Partitions: 1000 - Repartition Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 1000 - Repartition Test Time C A BN 0.6615 1.323 1.9845 2.646 3.3075 SE +/- 0.04, N = 3 SE +/- 0.03, N = 3 2.82 2.87 2.94
Apache Spark Row Count: 1000000 - Partitions: 1000 - Group By Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 1000 - Group By Test Time A BN C 1.2488 2.4976 3.7464 4.9952 6.244 SE +/- 0.04, N = 3 SE +/- 0.02, N = 3 5.37 5.48 5.55
Apache Spark Row Count: 1000000 - Partitions: 1000 - Calculate Pi Benchmark Using Dataframe OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 1000 - Calculate Pi Benchmark Using Dataframe A BN C 2 4 6 8 10 SE +/- 0.031303906, N = 3 SE +/- 0.025947993, N = 3 6.799198247 6.820000000 6.840000000
Apache Spark Row Count: 1000000 - Partitions: 1000 - Calculate Pi Benchmark OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 1000 - Calculate Pi Benchmark BN A C 20 40 60 80 100 SE +/- 0.38, N = 3 SE +/- 0.15, N = 3 109.68 109.81 110.06
Apache Spark Row Count: 1000000 - Partitions: 1000 - SHA-512 Benchmark Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 1000 - SHA-512 Benchmark Time A C BN 1.0463 2.0926 3.1389 4.1852 5.2315 SE +/- 0.06, N = 3 SE +/- 0.05, N = 3 4.34 4.64 4.65
SVT-AV1 Encoder Mode: Preset 4 - Input: Bosphorus 4K OpenBenchmarking.org Frames Per Second, More Is Better SVT-AV1 1.2 Encoder Mode: Preset 4 - Input: Bosphorus 4K A C BN 0.3114 0.6228 0.9342 1.2456 1.557 SE +/- 0.001, N = 3 SE +/- 0.004, N = 3 1.384 1.379 1.379 1. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq
SVT-AV1 Encoder Mode: Preset 4 - Input: Bosphorus 1080p OpenBenchmarking.org Frames Per Second, More Is Better SVT-AV1 1.2 Encoder Mode: Preset 4 - Input: Bosphorus 1080p A C BN 0.8084 1.6168 2.4252 3.2336 4.042 SE +/- 0.014, N = 3 SE +/- 0.015, N = 3 3.593 3.566 3.530 1. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq
SVT-AV1 Encoder Mode: Preset 8 - Input: Bosphorus 4K OpenBenchmarking.org Frames Per Second, More Is Better SVT-AV1 1.2 Encoder Mode: Preset 8 - Input: Bosphorus 4K BN C A 7 14 21 28 35 SE +/- 0.10, N = 3 SE +/- 0.08, N = 3 32.13 32.02 32.00 1. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq
SVT-AV1 Encoder Mode: Preset 10 - Input: Bosphorus 4K OpenBenchmarking.org Frames Per Second, More Is Better SVT-AV1 1.2 Encoder Mode: Preset 10 - Input: Bosphorus 4K A C BN 14 28 42 56 70 SE +/- 0.53, N = 3 SE +/- 0.21, N = 3 61.39 60.94 60.85 1. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq
SVT-AV1 Encoder Mode: Preset 8 - Input: Bosphorus 1080p OpenBenchmarking.org Frames Per Second, More Is Better SVT-AV1 1.2 Encoder Mode: Preset 8 - Input: Bosphorus 1080p BN A C 20 40 60 80 100 SE +/- 0.37, N = 3 SE +/- 0.10, N = 3 76.01 75.96 75.23 1. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq
SVT-AV1 Encoder Mode: Preset 12 - Input: Bosphorus 4K OpenBenchmarking.org Frames Per Second, More Is Better SVT-AV1 1.2 Encoder Mode: Preset 12 - Input: Bosphorus 4K A BN C 20 40 60 80 100 SE +/- 0.46, N = 3 SE +/- 0.47, N = 3 89.75 89.46 89.37 1. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq
SVT-AV1 Encoder Mode: Preset 10 - Input: Bosphorus 1080p OpenBenchmarking.org Frames Per Second, More Is Better SVT-AV1 1.2 Encoder Mode: Preset 10 - Input: Bosphorus 1080p A BN C 40 80 120 160 200 SE +/- 1.56, N = 3 SE +/- 1.66, N = 4 162.28 158.93 156.49 1. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq
SVT-AV1 Encoder Mode: Preset 12 - Input: Bosphorus 1080p OpenBenchmarking.org Frames Per Second, More Is Better SVT-AV1 1.2 Encoder Mode: Preset 12 - Input: Bosphorus 1080p BN A C 50 100 150 200 250 SE +/- 1.68, N = 3 SE +/- 1.25, N = 3 223.44 221.10 219.62 1. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq
Phoronix Test Suite v10.8.4