tr 2950X AMD Ryzen Threadripper 2950X 16-Core testing with a MSI MEG X399 CREATION (MS-7B92) v1.0 (1.30 BIOS) and AMD FirePro V4900 1GB on Ubuntu 22.04 via the Phoronix Test Suite.
HTML result view exported from: https://openbenchmarking.org/result/2208075-NE-TR2950X6958&sor .
tr 2950X Processor Motherboard Chipset Memory Disk Graphics Audio Monitor Network OS Kernel Desktop Display Server OpenGL Vulkan Compiler File-System Screen Resolution A BN C AMD Ryzen Threadripper 2950X 16-Core @ 3.50GHz (16 Cores / 32 Threads) MSI MEG X399 CREATION (MS-7B92) v1.0 (1.30 BIOS) AMD 17h 32GB Samsung SSD 970 EVO 250GB AMD FirePro V4900 1GB Realtek ALC1220 DELL S2409W 2 x Intel I211 + Intel-AC 9260 Ubuntu 22.04 5.19.0-051900rc7-generic (x86_64) GNOME Shell 42.2 X Server + Wayland 3.3 Mesa 22.0.1 (LLVM 13.0.1) 1.2.204 GCC 11.2.0 ext4 1920x1080 OpenBenchmarking.org Kernel Details - Transparent Huge Pages: madvise Compiler Details - --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-link-serialization=2 --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-targets=nvptx-none=/build/gcc-11-gBFGDP/gcc-11-11.2.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-11-gBFGDP/gcc-11-11.2.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-build-config=bootstrap-lto-lean --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v Processor Details - Scaling Governor: acpi-cpufreq schedutil (Boost: Enabled) - CPU Microcode: 0x800820d Java Details - OpenJDK Runtime Environment (build 11.0.15+10-Ubuntu-0ubuntu0.22.04.1) Python Details - Python 3.10.4 Security Details - itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Mitigation of untrained return thunk; SMT vulnerable + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional STIBP: disabled RSB filling + srbds: Not affected + tsx_async_abort: Not affected
tr 2950X svt-av1: Preset 4 - Bosphorus 4K svt-av1: Preset 8 - Bosphorus 4K svt-av1: Preset 10 - Bosphorus 4K svt-av1: Preset 12 - Bosphorus 4K svt-av1: Preset 4 - Bosphorus 1080p svt-av1: Preset 8 - Bosphorus 1080p svt-av1: Preset 10 - Bosphorus 1080p svt-av1: Preset 12 - Bosphorus 1080p spark: 1000000 - 100 - SHA-512 Benchmark Time spark: 1000000 - 100 - Calculate Pi Benchmark spark: 1000000 - 100 - Calculate Pi Benchmark Using Dataframe spark: 1000000 - 100 - Group By Test Time spark: 1000000 - 100 - Repartition Test Time spark: 1000000 - 100 - Inner Join Test Time spark: 1000000 - 100 - Broadcast Inner Join Test Time spark: 1000000 - 500 - SHA-512 Benchmark Time spark: 1000000 - 500 - Calculate Pi Benchmark spark: 1000000 - 500 - Calculate Pi Benchmark Using Dataframe spark: 1000000 - 500 - Group By Test Time spark: 1000000 - 500 - Repartition Test Time spark: 1000000 - 500 - Inner Join Test Time spark: 1000000 - 500 - Broadcast Inner Join Test Time spark: 1000000 - 1000 - SHA-512 Benchmark Time spark: 1000000 - 1000 - Calculate Pi Benchmark spark: 1000000 - 1000 - Calculate Pi Benchmark Using Dataframe spark: 1000000 - 1000 - Group By Test Time spark: 1000000 - 1000 - Repartition Test Time spark: 1000000 - 1000 - Inner Join Test Time spark: 1000000 - 1000 - Broadcast Inner Join Test Time spark: 1000000 - 2000 - SHA-512 Benchmark Time spark: 1000000 - 2000 - Calculate Pi Benchmark spark: 1000000 - 2000 - Calculate Pi Benchmark Using Dataframe spark: 1000000 - 2000 - Group By Test Time spark: 1000000 - 2000 - Repartition Test Time spark: 1000000 - 2000 - Inner Join Test Time spark: 1000000 - 2000 - Broadcast Inner Join Test Time spark: 10000000 - 100 - SHA-512 Benchmark Time spark: 10000000 - 100 - Calculate Pi Benchmark spark: 10000000 - 100 - Calculate Pi Benchmark Using Dataframe spark: 10000000 - 100 - Group By Test Time spark: 10000000 - 100 - Repartition Test Time spark: 10000000 - 100 - Inner Join Test Time spark: 10000000 - 100 - Broadcast Inner Join Test Time spark: 10000000 - 500 - SHA-512 Benchmark Time spark: 10000000 - 500 - Calculate Pi Benchmark spark: 10000000 - 500 - Calculate Pi Benchmark Using Dataframe spark: 10000000 - 500 - Group By Test Time spark: 10000000 - 500 - Repartition Test Time spark: 10000000 - 500 - Inner Join Test Time spark: 10000000 - 500 - Broadcast Inner Join Test Time spark: 20000000 - 100 - SHA-512 Benchmark Time spark: 20000000 - 100 - Calculate Pi Benchmark spark: 20000000 - 100 - Calculate Pi Benchmark Using Dataframe spark: 20000000 - 100 - Group By Test Time spark: 20000000 - 100 - Repartition Test Time spark: 20000000 - 100 - Inner Join Test Time spark: 20000000 - 100 - Broadcast Inner Join Test Time spark: 20000000 - 500 - SHA-512 Benchmark Time spark: 20000000 - 500 - Calculate Pi Benchmark spark: 20000000 - 500 - Calculate Pi Benchmark Using Dataframe spark: 20000000 - 500 - Group By Test Time spark: 20000000 - 500 - Repartition Test Time spark: 20000000 - 500 - Inner Join Test Time spark: 20000000 - 500 - Broadcast Inner Join Test Time spark: 40000000 - 100 - SHA-512 Benchmark Time spark: 40000000 - 100 - Calculate Pi Benchmark spark: 40000000 - 100 - Calculate Pi Benchmark Using Dataframe spark: 40000000 - 100 - Group By Test Time spark: 40000000 - 100 - Repartition Test Time spark: 40000000 - 100 - Inner Join Test Time spark: 40000000 - 100 - Broadcast Inner Join Test Time spark: 40000000 - 500 - SHA-512 Benchmark Time spark: 40000000 - 500 - Calculate Pi Benchmark spark: 40000000 - 500 - Calculate Pi Benchmark Using Dataframe spark: 40000000 - 500 - Group By Test Time spark: 40000000 - 500 - Repartition Test Time spark: 40000000 - 500 - Inner Join Test Time spark: 40000000 - 500 - Broadcast Inner Join Test Time spark: 10000000 - 1000 - SHA-512 Benchmark Time spark: 10000000 - 1000 - Calculate Pi Benchmark spark: 10000000 - 1000 - Calculate Pi Benchmark Using Dataframe spark: 10000000 - 1000 - Group By Test Time spark: 10000000 - 1000 - Repartition Test Time spark: 10000000 - 1000 - Inner Join Test Time spark: 10000000 - 1000 - Broadcast Inner Join Test Time spark: 10000000 - 2000 - SHA-512 Benchmark Time spark: 10000000 - 2000 - Calculate Pi Benchmark spark: 10000000 - 2000 - Calculate Pi Benchmark Using Dataframe spark: 10000000 - 2000 - Group By Test Time spark: 10000000 - 2000 - Repartition Test Time spark: 10000000 - 2000 - Inner Join Test Time spark: 10000000 - 2000 - Broadcast Inner Join Test Time spark: 20000000 - 1000 - SHA-512 Benchmark Time spark: 20000000 - 1000 - Calculate Pi Benchmark spark: 20000000 - 1000 - Calculate Pi Benchmark Using Dataframe spark: 20000000 - 1000 - Group By Test Time spark: 20000000 - 1000 - Repartition Test Time spark: 20000000 - 1000 - Inner Join Test Time spark: 20000000 - 1000 - Broadcast Inner Join Test Time spark: 20000000 - 2000 - SHA-512 Benchmark Time spark: 20000000 - 2000 - Calculate Pi Benchmark spark: 20000000 - 2000 - Calculate Pi Benchmark Using Dataframe spark: 20000000 - 2000 - Group By Test Time spark: 20000000 - 2000 - Repartition Test Time spark: 20000000 - 2000 - Inner Join Test Time spark: 20000000 - 2000 - Broadcast Inner Join Test Time spark: 40000000 - 1000 - SHA-512 Benchmark Time spark: 40000000 - 1000 - Calculate Pi Benchmark spark: 40000000 - 1000 - Calculate Pi Benchmark Using Dataframe spark: 40000000 - 1000 - Group By Test Time spark: 40000000 - 1000 - Repartition Test Time spark: 40000000 - 1000 - Inner Join Test Time spark: 40000000 - 1000 - Broadcast Inner Join Test Time spark: 40000000 - 2000 - SHA-512 Benchmark Time spark: 40000000 - 2000 - Calculate Pi Benchmark spark: 40000000 - 2000 - Calculate Pi Benchmark Using Dataframe spark: 40000000 - 2000 - Group By Test Time spark: 40000000 - 2000 - Repartition Test Time spark: 40000000 - 2000 - Inner Join Test Time spark: 40000000 - 2000 - Broadcast Inner Join Test Time A BN C 1.384 31.995 61.388 89.752 3.593 75.963 162.279 221.104 4.530063798 108.419781632 6.783271558 5.876942244 2.700264051 2.10 1.972517798 4.37 108.225955158 6.77685602 5.504610213 2.65 2.170995055 1.70007341 4.34 109.807174541 6.799198247 5.37 2.87 2.43 1.92 4.904759563 108.597289108 6.84 6.17 2.97 3.09 2.32 13.931141663 108.688367026 6.81 9.439752516 9.47 10.764697533 10.662243237 13.198824567 109.230311687 6.759258954 8.621025209 9.09 10.444853075 10.016525853 24.428886208 109.108538715 6.80 13.329626392 17.154390228 20.683734622 20.754541595 22.754111465 109.612467287 6.812744629 11.890085005 16.640636577 18.785819235 18.428674686 46.523112355 110.523657566 6.84 32.423043313 34.185559705 40.835276383 40.089747144 45.815407438 109.231835166 6.72 31.80822044 31.759709462 38.916631644 37.86012772 14.004523565 109.522356865 6.70 9.56 9.35 10.226908103 10.412153908 14.139350937 109.91105171 6.76 9.80 9.30 10.997072357 9.96 23.235006524 110.730128235 6.875336993 13.180563735 15.892958165 19.682438668 19.237627995 23.247350909 109.43976691 6.77 13.22601236 16.431636403 20.393716972 19.056877299 44.777573634 109.896421862 6.86 29.126483464 31.448323516 37.542600102 36.818880624 43.859768868 110.085285547 6.86 25.818363399 30.648054198 37.991943155 36.697244482 1.379 32.129 60.846 89.460 3.530 76.005 158.929 223.442 4.62 110.166207147 6.89 6.01 2.81 2.37 1.88 4.54 109.907407989 6.81 5.57 2.72 2.25 1.77 4.65 109.680205103 6.82 5.48 2.94 2.45 2.05 4.99 110.039838817 6.79 6.27 3.38 3.17 2.41 14.19 109.357033407 6.80 9.59 9.31 10.47 10.69 13.09 109.329856012 6.78 8.49 8.93 9.73 9.75 24.63 109.576906446 6.82 13.40 17.38 20.35 20.80 22.58 109.971942948 6.80 11.99 16.16 19.29 18.37 46.40 109.746997331 6.79 32.11 32.99 39.98 41.27 46.27 109.940453650 6.86 31.26 32.56 37.94 38.99 14.34 109.968195448 6.80 9.67 9.59 10.78 10.66 14.04 110.250973756 6.78 9.89 9.45 10.83 10.83 24.89 110.258647571 6.80 13.20 16.40 19.95 19.61 23.69 109.896984405 6.79 12.90 16.69 20.55 19.27 43.90 110.394615796 6.87 29.34 31.51 36.52 36.73 43.42 110.551039063 6.84 27.22 31.54 38.80 35.82 1.379 32.021 60.942 89.366 3.566 75.225 156.492 219.616 4.50 109.870024098 6.86 5.97 2.73 2.13 1.75 4.43 110.357863946 6.81 5.44 2.74 2.26 1.79 4.64 110.057783068 6.84 5.55 2.82 2.43 2.03 4.92 110.12 6.81 6.31 3.39 3.12 2.48 14.01 110.411376802 6.85 9.68 9.30 10.41 10.72 13.29 110.539073220 6.82 8.50 9.12 10.19 9.88 25.02 109.616776140 6.77 13.35 17.37 20.60 21.22 22.64 109.977979218 6.86 11.77 16.33 19.06 18.69 48.05 111.43 6.92 32.49 33.29 40.12 39.81 46.14 110.39 6.84 31.45 32.55 39.06 39.13 14.30 110.105280590 6.81 9.87 9.46 10.80 10.87 14.22 109.96 6.84 10.21 9.51 10.66 10.54 25.45 109.959690974 6.88 12.99 16.67 19.85 20.55 23.55 109.91605971 6.77 13.05 16.30 20.30 19.62 41.003127286 110.273490935 6.74 29.43 31.02 36.58 35.45 44.72 110.588119919 6.76 27.267906089 31.27 39.31 36.74 OpenBenchmarking.org
SVT-AV1 Encoder Mode: Preset 4 - Input: Bosphorus 4K OpenBenchmarking.org Frames Per Second, More Is Better SVT-AV1 1.2 Encoder Mode: Preset 4 - Input: Bosphorus 4K A C BN 0.3114 0.6228 0.9342 1.2456 1.557 SE +/- 0.001, N = 3 SE +/- 0.004, N = 3 1.384 1.379 1.379 1. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq
SVT-AV1 Encoder Mode: Preset 8 - Input: Bosphorus 4K OpenBenchmarking.org Frames Per Second, More Is Better SVT-AV1 1.2 Encoder Mode: Preset 8 - Input: Bosphorus 4K BN C A 7 14 21 28 35 SE +/- 0.10, N = 3 SE +/- 0.08, N = 3 32.13 32.02 32.00 1. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq
SVT-AV1 Encoder Mode: Preset 10 - Input: Bosphorus 4K OpenBenchmarking.org Frames Per Second, More Is Better SVT-AV1 1.2 Encoder Mode: Preset 10 - Input: Bosphorus 4K A C BN 14 28 42 56 70 SE +/- 0.53, N = 3 SE +/- 0.21, N = 3 61.39 60.94 60.85 1. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq
SVT-AV1 Encoder Mode: Preset 12 - Input: Bosphorus 4K OpenBenchmarking.org Frames Per Second, More Is Better SVT-AV1 1.2 Encoder Mode: Preset 12 - Input: Bosphorus 4K A BN C 20 40 60 80 100 SE +/- 0.46, N = 3 SE +/- 0.47, N = 3 89.75 89.46 89.37 1. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq
SVT-AV1 Encoder Mode: Preset 4 - Input: Bosphorus 1080p OpenBenchmarking.org Frames Per Second, More Is Better SVT-AV1 1.2 Encoder Mode: Preset 4 - Input: Bosphorus 1080p A C BN 0.8084 1.6168 2.4252 3.2336 4.042 SE +/- 0.014, N = 3 SE +/- 0.015, N = 3 3.593 3.566 3.530 1. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq
SVT-AV1 Encoder Mode: Preset 8 - Input: Bosphorus 1080p OpenBenchmarking.org Frames Per Second, More Is Better SVT-AV1 1.2 Encoder Mode: Preset 8 - Input: Bosphorus 1080p BN A C 20 40 60 80 100 SE +/- 0.37, N = 3 SE +/- 0.10, N = 3 76.01 75.96 75.23 1. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq
SVT-AV1 Encoder Mode: Preset 10 - Input: Bosphorus 1080p OpenBenchmarking.org Frames Per Second, More Is Better SVT-AV1 1.2 Encoder Mode: Preset 10 - Input: Bosphorus 1080p A BN C 40 80 120 160 200 SE +/- 1.56, N = 3 SE +/- 1.66, N = 4 162.28 158.93 156.49 1. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq
SVT-AV1 Encoder Mode: Preset 12 - Input: Bosphorus 1080p OpenBenchmarking.org Frames Per Second, More Is Better SVT-AV1 1.2 Encoder Mode: Preset 12 - Input: Bosphorus 1080p BN A C 50 100 150 200 250 SE +/- 1.68, N = 3 SE +/- 1.25, N = 3 223.44 221.10 219.62 1. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq
Apache Spark Row Count: 1000000 - Partitions: 100 - SHA-512 Benchmark Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 100 - SHA-512 Benchmark Time C A BN 1.0395 2.079 3.1185 4.158 5.1975 SE +/- 0.055483981, N = 4 SE +/- 0.034411464, N = 3 4.500000000 4.530063798 4.620000000
Apache Spark Row Count: 1000000 - Partitions: 100 - Calculate Pi Benchmark OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 100 - Calculate Pi Benchmark A C BN 20 40 60 80 100 SE +/- 0.10, N = 4 SE +/- 0.21, N = 3 108.42 109.87 110.17
Apache Spark Row Count: 1000000 - Partitions: 100 - Calculate Pi Benchmark Using Dataframe OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 100 - Calculate Pi Benchmark Using Dataframe A C BN 2 4 6 8 10 SE +/- 0.022438723, N = 4 SE +/- 0.068811394, N = 3 6.783271558 6.860000000 6.890000000
Apache Spark Row Count: 1000000 - Partitions: 100 - Group By Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 100 - Group By Test Time A C BN 2 4 6 8 10 SE +/- 0.080220571, N = 4 SE +/- 0.060621777, N = 3 5.876942244 5.970000000 6.010000000
Apache Spark Row Count: 1000000 - Partitions: 100 - Repartition Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 100 - Repartition Test Time A C BN 0.6323 1.2646 1.8969 2.5292 3.1615 SE +/- 0.052181616, N = 4 SE +/- 0.018430805, N = 3 2.700264051 2.730000000 2.810000000
Apache Spark Row Count: 1000000 - Partitions: 100 - Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 100 - Inner Join Test Time A C BN 0.5333 1.0666 1.5999 2.1332 2.6665 SE +/- 0.05, N = 4 SE +/- 0.07, N = 3 2.10 2.13 2.37
Apache Spark Row Count: 1000000 - Partitions: 100 - Broadcast Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 100 - Broadcast Inner Join Test Time C BN A 0.4438 0.8876 1.3314 1.7752 2.219 SE +/- 0.055547846, N = 4 SE +/- 0.059135782, N = 3 1.750000000 1.880000000 1.972517798
Apache Spark Row Count: 1000000 - Partitions: 500 - SHA-512 Benchmark Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 500 - SHA-512 Benchmark Time A C BN 1.0215 2.043 3.0645 4.086 5.1075 SE +/- 0.05, N = 3 SE +/- 0.04, N = 8 4.37 4.43 4.54
Apache Spark Row Count: 1000000 - Partitions: 500 - Calculate Pi Benchmark OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 500 - Calculate Pi Benchmark A BN C 20 40 60 80 100 SE +/- 0.12, N = 8 SE +/- 0.32, N = 3 108.23 109.91 110.36
Apache Spark Row Count: 1000000 - Partitions: 500 - Calculate Pi Benchmark Using Dataframe OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 500 - Calculate Pi Benchmark Using Dataframe A BN C 2 4 6 8 10 SE +/- 0.02582423, N = 8 SE +/- 0.02772713, N = 3 6.77685602 6.81000000 6.81000000
Apache Spark Row Count: 1000000 - Partitions: 500 - Group By Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 500 - Group By Test Time C A BN 1.2533 2.5066 3.7599 5.0132 6.2665 SE +/- 0.043659720, N = 3 SE +/- 0.029315415, N = 8 5.440000000 5.504610213 5.570000000
Apache Spark Row Count: 1000000 - Partitions: 500 - Repartition Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 500 - Repartition Test Time A BN C 0.6165 1.233 1.8495 2.466 3.0825 SE +/- 0.04, N = 8 SE +/- 0.05, N = 3 2.65 2.72 2.74
Apache Spark Row Count: 1000000 - Partitions: 500 - Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 500 - Inner Join Test Time A BN C 0.5085 1.017 1.5255 2.034 2.5425 SE +/- 0.028445371, N = 8 SE +/- 0.110292282, N = 3 2.170995055 2.250000000 2.260000000
Apache Spark Row Count: 1000000 - Partitions: 500 - Broadcast Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 500 - Broadcast Inner Join Test Time A BN C 0.4028 0.8056 1.2084 1.6112 2.014 SE +/- 0.05474021, N = 8 SE +/- 0.04087612, N = 3 1.70007341 1.77000000 1.79000000
Apache Spark Row Count: 1000000 - Partitions: 1000 - SHA-512 Benchmark Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 1000 - SHA-512 Benchmark Time A C BN 1.0463 2.0926 3.1389 4.1852 5.2315 SE +/- 0.06, N = 3 SE +/- 0.05, N = 3 4.34 4.64 4.65
Apache Spark Row Count: 1000000 - Partitions: 1000 - Calculate Pi Benchmark OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 1000 - Calculate Pi Benchmark BN A C 20 40 60 80 100 SE +/- 0.38, N = 3 SE +/- 0.15, N = 3 109.68 109.81 110.06
Apache Spark Row Count: 1000000 - Partitions: 1000 - Calculate Pi Benchmark Using Dataframe OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 1000 - Calculate Pi Benchmark Using Dataframe A BN C 2 4 6 8 10 SE +/- 0.031303906, N = 3 SE +/- 0.025947993, N = 3 6.799198247 6.820000000 6.840000000
Apache Spark Row Count: 1000000 - Partitions: 1000 - Group By Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 1000 - Group By Test Time A BN C 1.2488 2.4976 3.7464 4.9952 6.244 SE +/- 0.04, N = 3 SE +/- 0.02, N = 3 5.37 5.48 5.55
Apache Spark Row Count: 1000000 - Partitions: 1000 - Repartition Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 1000 - Repartition Test Time C A BN 0.6615 1.323 1.9845 2.646 3.3075 SE +/- 0.04, N = 3 SE +/- 0.03, N = 3 2.82 2.87 2.94
Apache Spark Row Count: 1000000 - Partitions: 1000 - Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 1000 - Inner Join Test Time A C BN 0.5513 1.1026 1.6539 2.2052 2.7565 SE +/- 0.04, N = 3 SE +/- 0.05, N = 3 2.43 2.43 2.45
Apache Spark Row Count: 1000000 - Partitions: 1000 - Broadcast Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 1000 - Broadcast Inner Join Test Time A C BN 0.4613 0.9226 1.3839 1.8452 2.3065 SE +/- 0.06, N = 3 SE +/- 0.06, N = 3 1.92 2.03 2.05
Apache Spark Row Count: 1000000 - Partitions: 2000 - SHA-512 Benchmark Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 2000 - SHA-512 Benchmark Time A C BN 1.1228 2.2456 3.3684 4.4912 5.614 SE +/- 0.049353750, N = 3 SE +/- 0.048557793, N = 6 4.904759563 4.920000000 4.990000000
Apache Spark Row Count: 1000000 - Partitions: 2000 - Calculate Pi Benchmark OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 2000 - Calculate Pi Benchmark A BN C 20 40 60 80 100 SE +/- 0.17, N = 6 SE +/- 0.24, N = 3 108.60 110.04 110.12
Apache Spark Row Count: 1000000 - Partitions: 2000 - Calculate Pi Benchmark Using Dataframe OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 2000 - Calculate Pi Benchmark Using Dataframe BN C A 2 4 6 8 10 SE +/- 0.02, N = 6 SE +/- 0.05, N = 3 6.79 6.81 6.84
Apache Spark Row Count: 1000000 - Partitions: 2000 - Group By Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 2000 - Group By Test Time A BN C 2 4 6 8 10 SE +/- 0.04, N = 6 SE +/- 0.10, N = 3 6.17 6.27 6.31
Apache Spark Row Count: 1000000 - Partitions: 2000 - Repartition Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 2000 - Repartition Test Time A BN C 0.7628 1.5256 2.2884 3.0512 3.814 SE +/- 0.04, N = 6 SE +/- 0.02, N = 3 2.97 3.38 3.39
Apache Spark Row Count: 1000000 - Partitions: 2000 - Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 2000 - Inner Join Test Time A C BN 0.7133 1.4266 2.1399 2.8532 3.5665 SE +/- 0.09, N = 3 SE +/- 0.04, N = 6 3.09 3.12 3.17
Apache Spark Row Count: 1000000 - Partitions: 2000 - Broadcast Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 2000 - Broadcast Inner Join Test Time A BN C 0.558 1.116 1.674 2.232 2.79 SE +/- 0.05, N = 6 SE +/- 0.07, N = 3 2.32 2.41 2.48
Apache Spark Row Count: 10000000 - Partitions: 100 - SHA-512 Benchmark Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 100 - SHA-512 Benchmark Time A C BN 4 8 12 16 20 SE +/- 0.10, N = 3 SE +/- 0.07, N = 3 13.93 14.01 14.19
Apache Spark Row Count: 10000000 - Partitions: 100 - Calculate Pi Benchmark OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 100 - Calculate Pi Benchmark A BN C 20 40 60 80 100 SE +/- 0.17, N = 3 SE +/- 0.10, N = 3 108.69 109.36 110.41
Apache Spark Row Count: 10000000 - Partitions: 100 - Calculate Pi Benchmark Using Dataframe OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 100 - Calculate Pi Benchmark Using Dataframe BN A C 2 4 6 8 10 SE +/- 0.00, N = 3 SE +/- 0.03, N = 3 6.80 6.81 6.85
Apache Spark Row Count: 10000000 - Partitions: 100 - Group By Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 100 - Group By Test Time A BN C 3 6 9 12 15 SE +/- 0.023359039, N = 3 SE +/- 0.055018199, N = 3 9.439752516 9.590000000 9.680000000
Apache Spark Row Count: 10000000 - Partitions: 100 - Repartition Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 100 - Repartition Test Time C BN A 3 6 9 12 15 SE +/- 0.09, N = 3 SE +/- 0.16, N = 3 9.30 9.31 9.47
Apache Spark Row Count: 10000000 - Partitions: 100 - Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 100 - Inner Join Test Time C BN A 3 6 9 12 15 SE +/- 0.18, N = 3 SE +/- 0.26, N = 3 10.41 10.47 10.76
Apache Spark Row Count: 10000000 - Partitions: 100 - Broadcast Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 100 - Broadcast Inner Join Test Time A BN C 3 6 9 12 15 SE +/- 0.24, N = 3 SE +/- 0.11, N = 3 10.66 10.69 10.72
Apache Spark Row Count: 10000000 - Partitions: 500 - SHA-512 Benchmark Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 500 - SHA-512 Benchmark Time BN A C 3 6 9 12 15 SE +/- 0.14, N = 3 SE +/- 0.13, N = 6 13.09 13.20 13.29
Apache Spark Row Count: 10000000 - Partitions: 500 - Calculate Pi Benchmark OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 500 - Calculate Pi Benchmark A BN C 20 40 60 80 100 SE +/- 0.30, N = 3 SE +/- 0.25, N = 6 109.23 109.33 110.54
Apache Spark Row Count: 10000000 - Partitions: 500 - Calculate Pi Benchmark Using Dataframe OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 500 - Calculate Pi Benchmark Using Dataframe A BN C 2 4 6 8 10 SE +/- 0.003533182, N = 3 SE +/- 0.025564809, N = 6 6.759258954 6.780000000 6.820000000
Apache Spark Row Count: 10000000 - Partitions: 500 - Group By Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 500 - Group By Test Time BN C A 2 4 6 8 10 SE +/- 0.069704950, N = 3 SE +/- 0.074190780, N = 6 8.490000000 8.500000000 8.621025209
Apache Spark Row Count: 10000000 - Partitions: 500 - Repartition Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 500 - Repartition Test Time BN A C 3 6 9 12 15 SE +/- 0.19, N = 3 SE +/- 0.10, N = 6 8.93 9.09 9.12
Apache Spark Row Count: 10000000 - Partitions: 500 - Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 500 - Inner Join Test Time BN C A 3 6 9 12 15 SE +/- 0.118870708, N = 3 SE +/- 0.144346648, N = 6 9.730000000 10.190000000 10.444853075
Apache Spark Row Count: 10000000 - Partitions: 500 - Broadcast Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 500 - Broadcast Inner Join Test Time BN C A 3 6 9 12 15 SE +/- 0.085017131, N = 3 SE +/- 0.094306283, N = 6 9.750000000 9.880000000 10.016525853
Apache Spark Row Count: 20000000 - Partitions: 100 - SHA-512 Benchmark Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 100 - SHA-512 Benchmark Time A BN C 6 12 18 24 30 SE +/- 0.03, N = 3 SE +/- 0.27, N = 3 24.43 24.63 25.02
Apache Spark Row Count: 20000000 - Partitions: 100 - Calculate Pi Benchmark OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 100 - Calculate Pi Benchmark A BN C 20 40 60 80 100 SE +/- 0.21, N = 3 SE +/- 0.14, N = 3 109.11 109.58 109.62
Apache Spark Row Count: 20000000 - Partitions: 100 - Calculate Pi Benchmark Using Dataframe OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 100 - Calculate Pi Benchmark Using Dataframe C A BN 2 4 6 8 10 SE +/- 0.03, N = 3 SE +/- 0.01, N = 3 6.77 6.80 6.82
Apache Spark Row Count: 20000000 - Partitions: 100 - Group By Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 100 - Group By Test Time A C BN 3 6 9 12 15 SE +/- 0.07, N = 3 SE +/- 0.20, N = 3 13.33 13.35 13.40
Apache Spark Row Count: 20000000 - Partitions: 100 - Repartition Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 100 - Repartition Test Time A C BN 4 8 12 16 20 SE +/- 0.07, N = 3 SE +/- 0.11, N = 3 17.15 17.37 17.38
Apache Spark Row Count: 20000000 - Partitions: 100 - Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 100 - Inner Join Test Time BN C A 5 10 15 20 25 SE +/- 0.71, N = 3 SE +/- 0.24, N = 3 20.35 20.60 20.68
Apache Spark Row Count: 20000000 - Partitions: 100 - Broadcast Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 100 - Broadcast Inner Join Test Time A BN C 5 10 15 20 25 SE +/- 0.24, N = 3 SE +/- 0.25, N = 3 20.75 20.80 21.22
Apache Spark Row Count: 20000000 - Partitions: 500 - SHA-512 Benchmark Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 500 - SHA-512 Benchmark Time BN C A 5 10 15 20 25 SE +/- 0.30, N = 3 SE +/- 0.27, N = 3 22.58 22.64 22.75
Apache Spark Row Count: 20000000 - Partitions: 500 - Calculate Pi Benchmark OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 500 - Calculate Pi Benchmark A BN C 20 40 60 80 100 SE +/- 0.13, N = 3 SE +/- 0.17, N = 3 109.61 109.97 109.98
Apache Spark Row Count: 20000000 - Partitions: 500 - Calculate Pi Benchmark Using Dataframe OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 500 - Calculate Pi Benchmark Using Dataframe BN A C 2 4 6 8 10 SE +/- 0.034639869, N = 3 SE +/- 0.012838149, N = 3 6.800000000 6.812744629 6.860000000
Apache Spark Row Count: 20000000 - Partitions: 500 - Group By Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 500 - Group By Test Time C A BN 3 6 9 12 15 SE +/- 0.07, N = 3 SE +/- 0.11, N = 3 11.77 11.89 11.99
Apache Spark Row Count: 20000000 - Partitions: 500 - Repartition Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 500 - Repartition Test Time BN C A 4 8 12 16 20 SE +/- 0.26, N = 3 SE +/- 0.17, N = 3 16.16 16.33 16.64
Apache Spark Row Count: 20000000 - Partitions: 500 - Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 500 - Inner Join Test Time A C BN 5 10 15 20 25 SE +/- 0.15, N = 3 SE +/- 0.35, N = 3 18.79 19.06 19.29
Apache Spark Row Count: 20000000 - Partitions: 500 - Broadcast Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 500 - Broadcast Inner Join Test Time BN A C 5 10 15 20 25 SE +/- 0.30, N = 3 SE +/- 0.12, N = 3 18.37 18.43 18.69
Apache Spark Row Count: 40000000 - Partitions: 100 - SHA-512 Benchmark Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 100 - SHA-512 Benchmark Time BN A C 11 22 33 44 55 SE +/- 0.41, N = 9 SE +/- 0.33, N = 3 46.40 46.52 48.05
Apache Spark Row Count: 40000000 - Partitions: 100 - Calculate Pi Benchmark OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 100 - Calculate Pi Benchmark BN A C 20 40 60 80 100 SE +/- 0.15, N = 9 SE +/- 0.92, N = 3 109.75 110.52 111.43
Apache Spark Row Count: 40000000 - Partitions: 100 - Calculate Pi Benchmark Using Dataframe OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 100 - Calculate Pi Benchmark Using Dataframe BN A C 2 4 6 8 10 SE +/- 0.01, N = 9 SE +/- 0.06, N = 3 6.79 6.84 6.92
Apache Spark Row Count: 40000000 - Partitions: 100 - Group By Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 100 - Group By Test Time BN A C 8 16 24 32 40 SE +/- 0.07, N = 9 SE +/- 0.61, N = 3 32.11 32.42 32.49
Apache Spark Row Count: 40000000 - Partitions: 100 - Repartition Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 100 - Repartition Test Time BN C A 8 16 24 32 40 SE +/- 0.04, N = 9 SE +/- 0.48, N = 3 32.99 33.29 34.19
Apache Spark Row Count: 40000000 - Partitions: 100 - Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 100 - Inner Join Test Time BN C A 9 18 27 36 45 SE +/- 0.28, N = 9 SE +/- 0.17, N = 3 39.98 40.12 40.84
Apache Spark Row Count: 40000000 - Partitions: 100 - Broadcast Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 100 - Broadcast Inner Join Test Time C A BN 9 18 27 36 45 SE +/- 0.52, N = 3 SE +/- 0.37, N = 9 39.81 40.09 41.27
Apache Spark Row Count: 40000000 - Partitions: 500 - SHA-512 Benchmark Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 500 - SHA-512 Benchmark Time A C BN 10 20 30 40 50 SE +/- 0.47, N = 9 SE +/- 0.47, N = 5 45.82 46.14 46.27
Apache Spark Row Count: 40000000 - Partitions: 500 - Calculate Pi Benchmark OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 500 - Calculate Pi Benchmark A BN C 20 40 60 80 100 SE +/- 0.12, N = 5 SE +/- 0.09, N = 9 109.23 109.94 110.39
Apache Spark Row Count: 40000000 - Partitions: 500 - Calculate Pi Benchmark Using Dataframe OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 500 - Calculate Pi Benchmark Using Dataframe A C BN 2 4 6 8 10 SE +/- 0.04, N = 9 SE +/- 0.02, N = 5 6.72 6.84 6.86
Apache Spark Row Count: 40000000 - Partitions: 500 - Group By Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 500 - Group By Test Time BN C A 7 14 21 28 35 SE +/- 0.10, N = 5 SE +/- 0.11, N = 9 31.26 31.45 31.81
Apache Spark Row Count: 40000000 - Partitions: 500 - Repartition Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 500 - Repartition Test Time A C BN 8 16 24 32 40 SE +/- 0.04, N = 9 SE +/- 0.16, N = 5 31.76 32.55 32.56
Apache Spark Row Count: 40000000 - Partitions: 500 - Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 500 - Inner Join Test Time BN A C 9 18 27 36 45 SE +/- 0.49, N = 5 SE +/- 0.39, N = 9 37.94 38.92 39.06
Apache Spark Row Count: 40000000 - Partitions: 500 - Broadcast Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 500 - Broadcast Inner Join Test Time A BN C 9 18 27 36 45 SE +/- 0.14, N = 5 SE +/- 0.24, N = 9 37.86 38.99 39.13
Apache Spark Row Count: 10000000 - Partitions: 1000 - SHA-512 Benchmark Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 1000 - SHA-512 Benchmark Time A C BN 4 8 12 16 20 SE +/- 0.11, N = 12 SE +/- 0.14, N = 12 14.00 14.30 14.34
Apache Spark Row Count: 10000000 - Partitions: 1000 - Calculate Pi Benchmark OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 1000 - Calculate Pi Benchmark A BN C 20 40 60 80 100 SE +/- 0.13, N = 12 SE +/- 0.15, N = 12 109.52 109.97 110.11
Apache Spark Row Count: 10000000 - Partitions: 1000 - Calculate Pi Benchmark Using Dataframe OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 1000 - Calculate Pi Benchmark Using Dataframe A BN C 2 4 6 8 10 SE +/- 0.01, N = 12 SE +/- 0.01, N = 12 6.70 6.80 6.81
Apache Spark Row Count: 10000000 - Partitions: 1000 - Group By Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 1000 - Group By Test Time A BN C 3 6 9 12 15 SE +/- 0.05, N = 12 SE +/- 0.06, N = 12 9.56 9.67 9.87
Apache Spark Row Count: 10000000 - Partitions: 1000 - Repartition Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 1000 - Repartition Test Time A C BN 3 6 9 12 15 SE +/- 0.06, N = 12 SE +/- 0.07, N = 12 9.35 9.46 9.59
Apache Spark Row Count: 10000000 - Partitions: 1000 - Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 1000 - Inner Join Test Time A BN C 3 6 9 12 15 SE +/- 0.13, N = 12 SE +/- 0.13, N = 12 10.23 10.78 10.80
Apache Spark Row Count: 10000000 - Partitions: 1000 - Broadcast Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 1000 - Broadcast Inner Join Test Time A BN C 3 6 9 12 15 SE +/- 0.11, N = 12 SE +/- 0.11, N = 12 10.41 10.66 10.87
Apache Spark Row Count: 10000000 - Partitions: 2000 - SHA-512 Benchmark Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 2000 - SHA-512 Benchmark Time BN A C 4 8 12 16 20 SE +/- 0.02, N = 3 SE +/- 0.07, N = 3 14.04 14.14 14.22
Apache Spark Row Count: 10000000 - Partitions: 2000 - Calculate Pi Benchmark OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 2000 - Calculate Pi Benchmark A C BN 20 40 60 80 100 SE +/- 0.22, N = 3 SE +/- 0.63, N = 3 109.91 109.96 110.25
Apache Spark Row Count: 10000000 - Partitions: 2000 - Calculate Pi Benchmark Using Dataframe OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 2000 - Calculate Pi Benchmark Using Dataframe A BN C 2 4 6 8 10 SE +/- 0.02, N = 3 SE +/- 0.07, N = 3 6.76 6.78 6.84
Apache Spark Row Count: 10000000 - Partitions: 2000 - Group By Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 2000 - Group By Test Time A BN C 3 6 9 12 15 SE +/- 0.02, N = 3 SE +/- 0.19, N = 3 9.80 9.89 10.21
Apache Spark Row Count: 10000000 - Partitions: 2000 - Repartition Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 2000 - Repartition Test Time A BN C 3 6 9 12 15 SE +/- 0.06, N = 3 SE +/- 0.06, N = 3 9.30 9.45 9.51
Apache Spark Row Count: 10000000 - Partitions: 2000 - Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 2000 - Inner Join Test Time C BN A 3 6 9 12 15 SE +/- 0.14, N = 3 SE +/- 0.15, N = 3 10.66 10.83 11.00
Apache Spark Row Count: 10000000 - Partitions: 2000 - Broadcast Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 2000 - Broadcast Inner Join Test Time A C BN 3 6 9 12 15 SE +/- 0.50, N = 3 SE +/- 0.35, N = 3 9.96 10.54 10.83
Apache Spark Row Count: 20000000 - Partitions: 1000 - SHA-512 Benchmark Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 1000 - SHA-512 Benchmark Time A BN C 6 12 18 24 30 SE +/- 0.11, N = 3 23.24 24.89 25.45
Apache Spark Row Count: 20000000 - Partitions: 1000 - Calculate Pi Benchmark OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 1000 - Calculate Pi Benchmark C BN A 20 40 60 80 100 SE +/- 0.19, N = 3 109.96 110.26 110.73
Apache Spark Row Count: 20000000 - Partitions: 1000 - Calculate Pi Benchmark Using Dataframe OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 1000 - Calculate Pi Benchmark Using Dataframe BN A C 2 4 6 8 10 SE +/- 0.046089856, N = 3 6.800000000 6.875336993 6.880000000
Apache Spark Row Count: 20000000 - Partitions: 1000 - Group By Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 1000 - Group By Test Time C A BN 3 6 9 12 15 SE +/- 0.15, N = 3 12.99 13.18 13.20
Apache Spark Row Count: 20000000 - Partitions: 1000 - Repartition Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 1000 - Repartition Test Time A BN C 4 8 12 16 20 SE +/- 0.18, N = 3 15.89 16.40 16.67
Apache Spark Row Count: 20000000 - Partitions: 1000 - Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 1000 - Inner Join Test Time A C BN 5 10 15 20 25 SE +/- 0.31, N = 3 19.68 19.85 19.95
Apache Spark Row Count: 20000000 - Partitions: 1000 - Broadcast Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 1000 - Broadcast Inner Join Test Time A BN C 5 10 15 20 25 SE +/- 0.21, N = 3 19.24 19.61 20.55
Apache Spark Row Count: 20000000 - Partitions: 2000 - SHA-512 Benchmark Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 2000 - SHA-512 Benchmark Time A C BN 6 12 18 24 30 SE +/- 0.16, N = 12 23.25 23.55 23.69
Apache Spark Row Count: 20000000 - Partitions: 2000 - Calculate Pi Benchmark OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 2000 - Calculate Pi Benchmark A BN C 20 40 60 80 100 SE +/- 0.12, N = 12 109.44 109.90 109.92
Apache Spark Row Count: 20000000 - Partitions: 2000 - Calculate Pi Benchmark Using Dataframe OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 2000 - Calculate Pi Benchmark Using Dataframe A C BN 2 4 6 8 10 SE +/- 0.01, N = 12 6.77 6.77 6.79
Apache Spark Row Count: 20000000 - Partitions: 2000 - Group By Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 2000 - Group By Test Time BN C A 3 6 9 12 15 SE +/- 0.06, N = 12 12.90 13.05 13.23
Apache Spark Row Count: 20000000 - Partitions: 2000 - Repartition Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 2000 - Repartition Test Time C A BN 4 8 12 16 20 SE +/- 0.10, N = 12 16.30 16.43 16.69
Apache Spark Row Count: 20000000 - Partitions: 2000 - Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 2000 - Inner Join Test Time C A BN 5 10 15 20 25 SE +/- 0.19, N = 12 20.30 20.39 20.55
Apache Spark Row Count: 20000000 - Partitions: 2000 - Broadcast Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 2000 - Broadcast Inner Join Test Time A BN C 5 10 15 20 25 SE +/- 0.17, N = 12 19.06 19.27 19.62
Apache Spark Row Count: 40000000 - Partitions: 1000 - SHA-512 Benchmark Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 1000 - SHA-512 Benchmark Time C BN A 10 20 30 40 50 SE +/- 0.53, N = 9 41.00 43.90 44.78
Apache Spark Row Count: 40000000 - Partitions: 1000 - Calculate Pi Benchmark OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 1000 - Calculate Pi Benchmark A C BN 20 40 60 80 100 SE +/- 0.19, N = 9 109.90 110.27 110.39
Apache Spark Row Count: 40000000 - Partitions: 1000 - Calculate Pi Benchmark Using Dataframe OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 1000 - Calculate Pi Benchmark Using Dataframe C A BN 2 4 6 8 10 SE +/- 0.03, N = 9 6.74 6.86 6.87
Apache Spark Row Count: 40000000 - Partitions: 1000 - Group By Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 1000 - Group By Test Time A BN C 7 14 21 28 35 SE +/- 0.28, N = 9 29.13 29.34 29.43
Apache Spark Row Count: 40000000 - Partitions: 1000 - Repartition Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 1000 - Repartition Test Time C A BN 7 14 21 28 35 SE +/- 0.18, N = 9 31.02 31.45 31.51
Apache Spark Row Count: 40000000 - Partitions: 1000 - Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 1000 - Inner Join Test Time BN C A 9 18 27 36 45 SE +/- 0.26, N = 9 36.52 36.58 37.54
Apache Spark Row Count: 40000000 - Partitions: 1000 - Broadcast Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 1000 - Broadcast Inner Join Test Time C BN A 8 16 24 32 40 SE +/- 0.21, N = 9 35.45 36.73 36.82
Apache Spark Row Count: 40000000 - Partitions: 2000 - SHA-512 Benchmark Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 2000 - SHA-512 Benchmark Time BN A C 10 20 30 40 50 SE +/- 0.49, N = 9 43.42 43.86 44.72
Apache Spark Row Count: 40000000 - Partitions: 2000 - Calculate Pi Benchmark OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 2000 - Calculate Pi Benchmark A BN C 20 40 60 80 100 SE +/- 0.21, N = 9 110.09 110.55 110.59
Apache Spark Row Count: 40000000 - Partitions: 2000 - Calculate Pi Benchmark Using Dataframe OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 2000 - Calculate Pi Benchmark Using Dataframe C BN A 2 4 6 8 10 SE +/- 0.02, N = 9 6.76 6.84 6.86
Apache Spark Row Count: 40000000 - Partitions: 2000 - Group By Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 2000 - Group By Test Time A BN C 6 12 18 24 30 SE +/- 0.38, N = 9 25.82 27.22 27.27
Apache Spark Row Count: 40000000 - Partitions: 2000 - Repartition Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 2000 - Repartition Test Time A C BN 7 14 21 28 35 SE +/- 0.09, N = 9 30.65 31.27 31.54
Apache Spark Row Count: 40000000 - Partitions: 2000 - Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 2000 - Inner Join Test Time A BN C 9 18 27 36 45 SE +/- 0.21, N = 9 37.99 38.80 39.31
Apache Spark Row Count: 40000000 - Partitions: 2000 - Broadcast Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 2000 - Broadcast Inner Join Test Time BN A C 8 16 24 32 40 SE +/- 0.15, N = 9 35.82 36.70 36.74
Phoronix Test Suite v10.8.4