tr 2950X AMD Ryzen Threadripper 2950X 16-Core testing with a MSI MEG X399 CREATION (MS-7B92) v1.0 (1.30 BIOS) and AMD FirePro V4900 1GB on Ubuntu 22.04 via the Phoronix Test Suite.
HTML result view exported from: https://openbenchmarking.org/result/2208075-NE-TR2950X6958&grs&sor .
tr 2950X Processor Motherboard Chipset Memory Disk Graphics Audio Monitor Network OS Kernel Desktop Display Server OpenGL Vulkan Compiler File-System Screen Resolution A BN C AMD Ryzen Threadripper 2950X 16-Core @ 3.50GHz (16 Cores / 32 Threads) MSI MEG X399 CREATION (MS-7B92) v1.0 (1.30 BIOS) AMD 17h 32GB Samsung SSD 970 EVO 250GB AMD FirePro V4900 1GB Realtek ALC1220 DELL S2409W 2 x Intel I211 + Intel-AC 9260 Ubuntu 22.04 5.19.0-051900rc7-generic (x86_64) GNOME Shell 42.2 X Server + Wayland 3.3 Mesa 22.0.1 (LLVM 13.0.1) 1.2.204 GCC 11.2.0 ext4 1920x1080 OpenBenchmarking.org Kernel Details - Transparent Huge Pages: madvise Compiler Details - --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-link-serialization=2 --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-targets=nvptx-none=/build/gcc-11-gBFGDP/gcc-11-11.2.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-11-gBFGDP/gcc-11-11.2.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-build-config=bootstrap-lto-lean --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v Processor Details - Scaling Governor: acpi-cpufreq schedutil (Boost: Enabled) - CPU Microcode: 0x800820d Java Details - OpenJDK Runtime Environment (build 11.0.15+10-Ubuntu-0ubuntu0.22.04.1) Python Details - Python 3.10.4 Security Details - itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Mitigation of untrained return thunk; SMT vulnerable + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional STIBP: disabled RSB filling + srbds: Not affected + tsx_async_abort: Not affected
tr 2950X spark: 1000000 - 2000 - Repartition Test Time spark: 1000000 - 100 - Inner Join Test Time spark: 20000000 - 1000 - SHA-512 Benchmark Time spark: 40000000 - 1000 - SHA-512 Benchmark Time spark: 10000000 - 500 - Inner Join Test Time spark: 1000000 - 1000 - SHA-512 Benchmark Time spark: 1000000 - 2000 - Broadcast Inner Join Test Time spark: 20000000 - 1000 - Broadcast Inner Join Test Time spark: 1000000 - 1000 - Broadcast Inner Join Test Time spark: 40000000 - 2000 - Group By Test Time spark: 10000000 - 1000 - Inner Join Test Time spark: 20000000 - 1000 - Repartition Test Time spark: 10000000 - 1000 - Broadcast Inner Join Test Time spark: 1000000 - 1000 - Repartition Test Time spark: 10000000 - 2000 - Group By Test Time spark: 1000000 - 100 - Repartition Test Time spark: 1000000 - 500 - SHA-512 Benchmark Time spark: 40000000 - 1000 - Broadcast Inner Join Test Time svt-av1: Preset 10 - Bosphorus 1080p spark: 40000000 - 100 - Broadcast Inner Join Test Time spark: 40000000 - 100 - Repartition Test Time spark: 40000000 - 100 - SHA-512 Benchmark Time spark: 40000000 - 2000 - Inner Join Test Time spark: 10000000 - 100 - Inner Join Test Time spark: 1000000 - 500 - Repartition Test Time spark: 40000000 - 500 - Broadcast Inner Join Test Time spark: 1000000 - 1000 - Group By Test Time spark: 10000000 - 1000 - Group By Test Time spark: 10000000 - 2000 - Inner Join Test Time spark: 40000000 - 2000 - SHA-512 Benchmark Time spark: 20000000 - 500 - Repartition Test Time spark: 20000000 - 2000 - Broadcast Inner Join Test Time spark: 40000000 - 500 - Inner Join Test Time spark: 40000000 - 2000 - Repartition Test Time spark: 40000000 - 1000 - Inner Join Test Time spark: 10000000 - 500 - Broadcast Inner Join Test Time spark: 20000000 - 500 - Inner Join Test Time spark: 1000000 - 100 - SHA-512 Benchmark Time spark: 1000000 - 2000 - Inner Join Test Time spark: 40000000 - 2000 - Broadcast Inner Join Test Time spark: 10000000 - 1000 - Repartition Test Time spark: 10000000 - 100 - Group By Test Time spark: 20000000 - 2000 - Group By Test Time spark: 40000000 - 500 - Repartition Test Time spark: 20000000 - 100 - SHA-512 Benchmark Time spark: 10000000 - 1000 - SHA-512 Benchmark Time spark: 20000000 - 2000 - Repartition Test Time spark: 1000000 - 500 - Group By Test Time spark: 1000000 - 2000 - Group By Test Time spark: 1000000 - 100 - Group By Test Time spark: 10000000 - 2000 - Repartition Test Time spark: 20000000 - 100 - Broadcast Inner Join Test Time spark: 40000000 - 100 - Inner Join Test Time spark: 10000000 - 500 - Repartition Test Time spark: 40000000 - 500 - Calculate Pi Benchmark Using Dataframe spark: 1000000 - 500 - Calculate Pi Benchmark spark: 40000000 - 1000 - Calculate Pi Benchmark Using Dataframe spark: 40000000 - 100 - Calculate Pi Benchmark Using Dataframe spark: 20000000 - 2000 - SHA-512 Benchmark Time spark: 20000000 - 500 - Group By Test Time spark: 10000000 - 100 - SHA-512 Benchmark Time spark: 10000000 - 100 - Repartition Test Time svt-av1: Preset 4 - Bosphorus 1080p spark: 40000000 - 500 - Group By Test Time svt-av1: Preset 12 - Bosphorus 1080p spark: 20000000 - 500 - Broadcast Inner Join Test Time spark: 1000000 - 2000 - SHA-512 Benchmark Time spark: 10000000 - 1000 - Calculate Pi Benchmark Using Dataframe spark: 20000000 - 1000 - Group By Test Time spark: 1000000 - 100 - Calculate Pi Benchmark spark: 10000000 - 100 - Calculate Pi Benchmark spark: 40000000 - 1000 - Repartition Test Time spark: 1000000 - 100 - Calculate Pi Benchmark Using Dataframe spark: 10000000 - 500 - Group By Test Time spark: 40000000 - 100 - Calculate Pi Benchmark spark: 10000000 - 500 - SHA-512 Benchmark Time spark: 40000000 - 2000 - Calculate Pi Benchmark Using Dataframe spark: 1000000 - 2000 - Calculate Pi Benchmark spark: 20000000 - 1000 - Inner Join Test Time spark: 20000000 - 100 - Repartition Test Time spark: 10000000 - 2000 - SHA-512 Benchmark Time spark: 20000000 - 2000 - Inner Join Test Time spark: 10000000 - 500 - Calculate Pi Benchmark spark: 10000000 - 2000 - Calculate Pi Benchmark Using Dataframe spark: 40000000 - 100 - Group By Test Time spark: 20000000 - 1000 - Calculate Pi Benchmark Using Dataframe spark: 40000000 - 500 - Calculate Pi Benchmark spark: 40000000 - 1000 - Group By Test Time svt-av1: Preset 8 - Bosphorus 1080p spark: 40000000 - 500 - SHA-512 Benchmark Time spark: 10000000 - 500 - Calculate Pi Benchmark Using Dataframe svt-av1: Preset 10 - Bosphorus 4K spark: 20000000 - 500 - Calculate Pi Benchmark Using Dataframe spark: 1000000 - 1000 - Inner Join Test Time spark: 20000000 - 500 - SHA-512 Benchmark Time spark: 20000000 - 100 - Calculate Pi Benchmark Using Dataframe spark: 1000000 - 2000 - Calculate Pi Benchmark Using Dataframe spark: 10000000 - 100 - Calculate Pi Benchmark Using Dataframe spark: 20000000 - 1000 - Calculate Pi Benchmark spark: 1000000 - 1000 - Calculate Pi Benchmark Using Dataframe spark: 10000000 - 100 - Broadcast Inner Join Test Time spark: 10000000 - 1000 - Calculate Pi Benchmark spark: 20000000 - 100 - Group By Test Time spark: 1000000 - 500 - Calculate Pi Benchmark Using Dataframe spark: 20000000 - 100 - Calculate Pi Benchmark spark: 40000000 - 2000 - Calculate Pi Benchmark spark: 40000000 - 1000 - Calculate Pi Benchmark spark: 20000000 - 2000 - Calculate Pi Benchmark svt-av1: Preset 12 - Bosphorus 4K svt-av1: Preset 8 - Bosphorus 4K svt-av1: Preset 4 - Bosphorus 4K spark: 1000000 - 1000 - Calculate Pi Benchmark spark: 20000000 - 500 - Calculate Pi Benchmark spark: 10000000 - 2000 - Calculate Pi Benchmark spark: 20000000 - 2000 - Calculate Pi Benchmark Using Dataframe spark: 10000000 - 2000 - Broadcast Inner Join Test Time spark: 20000000 - 100 - Inner Join Test Time spark: 1000000 - 500 - Broadcast Inner Join Test Time spark: 1000000 - 500 - Inner Join Test Time spark: 1000000 - 100 - Broadcast Inner Join Test Time A BN C 2.97 2.10 23.235006524 44.777573634 10.444853075 4.34 2.32 19.237627995 1.92 25.818363399 10.226908103 15.892958165 10.412153908 2.87 9.80 2.700264051 4.37 36.818880624 162.279 40.089747144 34.185559705 46.523112355 37.991943155 10.764697533 2.65 37.86012772 5.37 9.56 10.997072357 43.859768868 16.640636577 19.056877299 38.916631644 30.648054198 37.542600102 10.016525853 18.785819235 4.530063798 3.09 36.697244482 9.35 9.439752516 13.22601236 31.759709462 24.428886208 14.004523565 16.431636403 5.504610213 6.17 5.876942244 9.30 20.754541595 40.835276383 9.09 6.72 108.225955158 6.86 6.84 23.247350909 11.890085005 13.931141663 9.47 3.593 31.80822044 221.104 18.428674686 4.904759563 6.70 13.180563735 108.419781632 108.688367026 31.448323516 6.783271558 8.621025209 110.523657566 13.198824567 6.86 108.597289108 19.682438668 17.154390228 14.139350937 20.393716972 109.230311687 6.76 32.423043313 6.875336993 109.231835166 29.126483464 75.963 45.815407438 6.759258954 61.388 6.812744629 2.43 22.754111465 6.80 6.84 6.81 110.730128235 6.799198247 10.662243237 109.522356865 13.329626392 6.77685602 109.108538715 110.085285547 109.896421862 109.43976691 89.752 31.995 1.384 109.807174541 109.612467287 109.91105171 6.77 9.96 20.683734622 1.70007341 2.170995055 1.972517798 3.38 2.37 24.89 43.90 9.73 4.65 2.41 19.61 2.05 27.22 10.78 16.40 10.66 2.94 9.89 2.81 4.54 36.73 158.929 41.27 32.99 46.40 38.80 10.47 2.72 38.99 5.48 9.67 10.83 43.42 16.16 19.27 37.94 31.54 36.52 9.75 19.29 4.62 3.17 35.82 9.59 9.59 12.90 32.56 24.63 14.34 16.69 5.57 6.27 6.01 9.45 20.80 39.98 8.93 6.86 109.907407989 6.87 6.79 23.69 11.99 14.19 9.31 3.530 31.26 223.442 18.37 4.99 6.80 13.20 110.166207147 109.357033407 31.51 6.89 8.49 109.746997331 13.09 6.84 110.039838817 19.95 17.38 14.04 20.55 109.329856012 6.78 32.11 6.80 109.940453650 29.34 76.005 46.27 6.78 60.846 6.80 2.45 22.58 6.82 6.79 6.80 110.258647571 6.82 10.69 109.968195448 13.40 6.81 109.576906446 110.551039063 110.394615796 109.896984405 89.460 32.129 1.379 109.680205103 109.971942948 110.250973756 6.79 10.83 20.35 1.77 2.25 1.88 3.39 2.13 25.45 41.003127286 10.19 4.64 2.48 20.55 2.03 27.267906089 10.80 16.67 10.87 2.82 10.21 2.73 4.43 35.45 156.492 39.81 33.29 48.05 39.31 10.41 2.74 39.13 5.55 9.87 10.66 44.72 16.33 19.62 39.06 31.27 36.58 9.88 19.06 4.50 3.12 36.74 9.46 9.68 13.05 32.55 25.02 14.30 16.30 5.44 6.31 5.97 9.51 21.22 40.12 9.12 6.84 110.357863946 6.74 6.92 23.55 11.77 14.01 9.30 3.566 31.45 219.616 18.69 4.92 6.81 12.99 109.870024098 110.411376802 31.02 6.86 8.50 111.43 13.29 6.76 110.12 19.85 17.37 14.22 20.30 110.539073220 6.84 32.49 6.88 110.39 29.43 75.225 46.14 6.82 60.942 6.86 2.43 22.64 6.77 6.81 6.85 109.959690974 6.84 10.72 110.105280590 13.35 6.81 109.616776140 110.588119919 110.273490935 109.91605971 89.366 32.021 1.379 110.057783068 109.977979218 109.96 6.77 10.54 20.60 1.79 2.26 1.75 OpenBenchmarking.org
Apache Spark Row Count: 1000000 - Partitions: 2000 - Repartition Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 2000 - Repartition Test Time A BN C 0.7628 1.5256 2.2884 3.0512 3.814 SE +/- 0.04, N = 6 SE +/- 0.02, N = 3 2.97 3.38 3.39
Apache Spark Row Count: 1000000 - Partitions: 100 - Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 100 - Inner Join Test Time A C BN 0.5333 1.0666 1.5999 2.1332 2.6665 SE +/- 0.05, N = 4 SE +/- 0.07, N = 3 2.10 2.13 2.37
Apache Spark Row Count: 20000000 - Partitions: 1000 - SHA-512 Benchmark Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 1000 - SHA-512 Benchmark Time A BN C 6 12 18 24 30 SE +/- 0.11, N = 3 23.24 24.89 25.45
Apache Spark Row Count: 40000000 - Partitions: 1000 - SHA-512 Benchmark Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 1000 - SHA-512 Benchmark Time C BN A 10 20 30 40 50 SE +/- 0.53, N = 9 41.00 43.90 44.78
Apache Spark Row Count: 10000000 - Partitions: 500 - Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 500 - Inner Join Test Time BN C A 3 6 9 12 15 SE +/- 0.118870708, N = 3 SE +/- 0.144346648, N = 6 9.730000000 10.190000000 10.444853075
Apache Spark Row Count: 1000000 - Partitions: 1000 - SHA-512 Benchmark Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 1000 - SHA-512 Benchmark Time A C BN 1.0463 2.0926 3.1389 4.1852 5.2315 SE +/- 0.06, N = 3 SE +/- 0.05, N = 3 4.34 4.64 4.65
Apache Spark Row Count: 1000000 - Partitions: 2000 - Broadcast Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 2000 - Broadcast Inner Join Test Time A BN C 0.558 1.116 1.674 2.232 2.79 SE +/- 0.05, N = 6 SE +/- 0.07, N = 3 2.32 2.41 2.48
Apache Spark Row Count: 20000000 - Partitions: 1000 - Broadcast Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 1000 - Broadcast Inner Join Test Time A BN C 5 10 15 20 25 SE +/- 0.21, N = 3 19.24 19.61 20.55
Apache Spark Row Count: 1000000 - Partitions: 1000 - Broadcast Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 1000 - Broadcast Inner Join Test Time A C BN 0.4613 0.9226 1.3839 1.8452 2.3065 SE +/- 0.06, N = 3 SE +/- 0.06, N = 3 1.92 2.03 2.05
Apache Spark Row Count: 40000000 - Partitions: 2000 - Group By Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 2000 - Group By Test Time A BN C 6 12 18 24 30 SE +/- 0.38, N = 9 25.82 27.22 27.27
Apache Spark Row Count: 10000000 - Partitions: 1000 - Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 1000 - Inner Join Test Time A BN C 3 6 9 12 15 SE +/- 0.13, N = 12 SE +/- 0.13, N = 12 10.23 10.78 10.80
Apache Spark Row Count: 20000000 - Partitions: 1000 - Repartition Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 1000 - Repartition Test Time A BN C 4 8 12 16 20 SE +/- 0.18, N = 3 15.89 16.40 16.67
Apache Spark Row Count: 10000000 - Partitions: 1000 - Broadcast Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 1000 - Broadcast Inner Join Test Time A BN C 3 6 9 12 15 SE +/- 0.11, N = 12 SE +/- 0.11, N = 12 10.41 10.66 10.87
Apache Spark Row Count: 1000000 - Partitions: 1000 - Repartition Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 1000 - Repartition Test Time C A BN 0.6615 1.323 1.9845 2.646 3.3075 SE +/- 0.04, N = 3 SE +/- 0.03, N = 3 2.82 2.87 2.94
Apache Spark Row Count: 10000000 - Partitions: 2000 - Group By Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 2000 - Group By Test Time A BN C 3 6 9 12 15 SE +/- 0.02, N = 3 SE +/- 0.19, N = 3 9.80 9.89 10.21
Apache Spark Row Count: 1000000 - Partitions: 100 - Repartition Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 100 - Repartition Test Time A C BN 0.6323 1.2646 1.8969 2.5292 3.1615 SE +/- 0.052181616, N = 4 SE +/- 0.018430805, N = 3 2.700264051 2.730000000 2.810000000
Apache Spark Row Count: 1000000 - Partitions: 500 - SHA-512 Benchmark Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 500 - SHA-512 Benchmark Time A C BN 1.0215 2.043 3.0645 4.086 5.1075 SE +/- 0.05, N = 3 SE +/- 0.04, N = 8 4.37 4.43 4.54
Apache Spark Row Count: 40000000 - Partitions: 1000 - Broadcast Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 1000 - Broadcast Inner Join Test Time C BN A 8 16 24 32 40 SE +/- 0.21, N = 9 35.45 36.73 36.82
SVT-AV1 Encoder Mode: Preset 10 - Input: Bosphorus 1080p OpenBenchmarking.org Frames Per Second, More Is Better SVT-AV1 1.2 Encoder Mode: Preset 10 - Input: Bosphorus 1080p A BN C 40 80 120 160 200 SE +/- 1.56, N = 3 SE +/- 1.66, N = 4 162.28 158.93 156.49 1. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq
Apache Spark Row Count: 40000000 - Partitions: 100 - Broadcast Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 100 - Broadcast Inner Join Test Time C A BN 9 18 27 36 45 SE +/- 0.52, N = 3 SE +/- 0.37, N = 9 39.81 40.09 41.27
Apache Spark Row Count: 40000000 - Partitions: 100 - Repartition Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 100 - Repartition Test Time BN C A 8 16 24 32 40 SE +/- 0.04, N = 9 SE +/- 0.48, N = 3 32.99 33.29 34.19
Apache Spark Row Count: 40000000 - Partitions: 100 - SHA-512 Benchmark Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 100 - SHA-512 Benchmark Time BN A C 11 22 33 44 55 SE +/- 0.41, N = 9 SE +/- 0.33, N = 3 46.40 46.52 48.05
Apache Spark Row Count: 40000000 - Partitions: 2000 - Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 2000 - Inner Join Test Time A BN C 9 18 27 36 45 SE +/- 0.21, N = 9 37.99 38.80 39.31
Apache Spark Row Count: 10000000 - Partitions: 100 - Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 100 - Inner Join Test Time C BN A 3 6 9 12 15 SE +/- 0.18, N = 3 SE +/- 0.26, N = 3 10.41 10.47 10.76
Apache Spark Row Count: 1000000 - Partitions: 500 - Repartition Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 500 - Repartition Test Time A BN C 0.6165 1.233 1.8495 2.466 3.0825 SE +/- 0.04, N = 8 SE +/- 0.05, N = 3 2.65 2.72 2.74
Apache Spark Row Count: 40000000 - Partitions: 500 - Broadcast Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 500 - Broadcast Inner Join Test Time A BN C 9 18 27 36 45 SE +/- 0.14, N = 5 SE +/- 0.24, N = 9 37.86 38.99 39.13
Apache Spark Row Count: 1000000 - Partitions: 1000 - Group By Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 1000 - Group By Test Time A BN C 1.2488 2.4976 3.7464 4.9952 6.244 SE +/- 0.04, N = 3 SE +/- 0.02, N = 3 5.37 5.48 5.55
Apache Spark Row Count: 10000000 - Partitions: 1000 - Group By Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 1000 - Group By Test Time A BN C 3 6 9 12 15 SE +/- 0.05, N = 12 SE +/- 0.06, N = 12 9.56 9.67 9.87
Apache Spark Row Count: 10000000 - Partitions: 2000 - Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 2000 - Inner Join Test Time C BN A 3 6 9 12 15 SE +/- 0.14, N = 3 SE +/- 0.15, N = 3 10.66 10.83 11.00
Apache Spark Row Count: 40000000 - Partitions: 2000 - SHA-512 Benchmark Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 2000 - SHA-512 Benchmark Time BN A C 10 20 30 40 50 SE +/- 0.49, N = 9 43.42 43.86 44.72
Apache Spark Row Count: 20000000 - Partitions: 500 - Repartition Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 500 - Repartition Test Time BN C A 4 8 12 16 20 SE +/- 0.26, N = 3 SE +/- 0.17, N = 3 16.16 16.33 16.64
Apache Spark Row Count: 20000000 - Partitions: 2000 - Broadcast Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 2000 - Broadcast Inner Join Test Time A BN C 5 10 15 20 25 SE +/- 0.17, N = 12 19.06 19.27 19.62
Apache Spark Row Count: 40000000 - Partitions: 500 - Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 500 - Inner Join Test Time BN A C 9 18 27 36 45 SE +/- 0.49, N = 5 SE +/- 0.39, N = 9 37.94 38.92 39.06
Apache Spark Row Count: 40000000 - Partitions: 2000 - Repartition Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 2000 - Repartition Test Time A C BN 7 14 21 28 35 SE +/- 0.09, N = 9 30.65 31.27 31.54
Apache Spark Row Count: 40000000 - Partitions: 1000 - Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 1000 - Inner Join Test Time BN C A 9 18 27 36 45 SE +/- 0.26, N = 9 36.52 36.58 37.54
Apache Spark Row Count: 10000000 - Partitions: 500 - Broadcast Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 500 - Broadcast Inner Join Test Time BN C A 3 6 9 12 15 SE +/- 0.085017131, N = 3 SE +/- 0.094306283, N = 6 9.750000000 9.880000000 10.016525853
Apache Spark Row Count: 20000000 - Partitions: 500 - Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 500 - Inner Join Test Time A C BN 5 10 15 20 25 SE +/- 0.15, N = 3 SE +/- 0.35, N = 3 18.79 19.06 19.29
Apache Spark Row Count: 1000000 - Partitions: 100 - SHA-512 Benchmark Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 100 - SHA-512 Benchmark Time C A BN 1.0395 2.079 3.1185 4.158 5.1975 SE +/- 0.055483981, N = 4 SE +/- 0.034411464, N = 3 4.500000000 4.530063798 4.620000000
Apache Spark Row Count: 1000000 - Partitions: 2000 - Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 2000 - Inner Join Test Time A C BN 0.7133 1.4266 2.1399 2.8532 3.5665 SE +/- 0.09, N = 3 SE +/- 0.04, N = 6 3.09 3.12 3.17
Apache Spark Row Count: 40000000 - Partitions: 2000 - Broadcast Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 2000 - Broadcast Inner Join Test Time BN A C 8 16 24 32 40 SE +/- 0.15, N = 9 35.82 36.70 36.74
Apache Spark Row Count: 10000000 - Partitions: 1000 - Repartition Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 1000 - Repartition Test Time A C BN 3 6 9 12 15 SE +/- 0.06, N = 12 SE +/- 0.07, N = 12 9.35 9.46 9.59
Apache Spark Row Count: 10000000 - Partitions: 100 - Group By Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 100 - Group By Test Time A BN C 3 6 9 12 15 SE +/- 0.023359039, N = 3 SE +/- 0.055018199, N = 3 9.439752516 9.590000000 9.680000000
Apache Spark Row Count: 20000000 - Partitions: 2000 - Group By Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 2000 - Group By Test Time BN C A 3 6 9 12 15 SE +/- 0.06, N = 12 12.90 13.05 13.23
Apache Spark Row Count: 40000000 - Partitions: 500 - Repartition Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 500 - Repartition Test Time A C BN 8 16 24 32 40 SE +/- 0.04, N = 9 SE +/- 0.16, N = 5 31.76 32.55 32.56
Apache Spark Row Count: 20000000 - Partitions: 100 - SHA-512 Benchmark Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 100 - SHA-512 Benchmark Time A BN C 6 12 18 24 30 SE +/- 0.03, N = 3 SE +/- 0.27, N = 3 24.43 24.63 25.02
Apache Spark Row Count: 10000000 - Partitions: 1000 - SHA-512 Benchmark Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 1000 - SHA-512 Benchmark Time A C BN 4 8 12 16 20 SE +/- 0.11, N = 12 SE +/- 0.14, N = 12 14.00 14.30 14.34
Apache Spark Row Count: 20000000 - Partitions: 2000 - Repartition Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 2000 - Repartition Test Time C A BN 4 8 12 16 20 SE +/- 0.10, N = 12 16.30 16.43 16.69
Apache Spark Row Count: 1000000 - Partitions: 500 - Group By Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 500 - Group By Test Time C A BN 1.2533 2.5066 3.7599 5.0132 6.2665 SE +/- 0.043659720, N = 3 SE +/- 0.029315415, N = 8 5.440000000 5.504610213 5.570000000
Apache Spark Row Count: 1000000 - Partitions: 2000 - Group By Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 2000 - Group By Test Time A BN C 2 4 6 8 10 SE +/- 0.04, N = 6 SE +/- 0.10, N = 3 6.17 6.27 6.31
Apache Spark Row Count: 1000000 - Partitions: 100 - Group By Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 100 - Group By Test Time A C BN 2 4 6 8 10 SE +/- 0.080220571, N = 4 SE +/- 0.060621777, N = 3 5.876942244 5.970000000 6.010000000
Apache Spark Row Count: 10000000 - Partitions: 2000 - Repartition Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 2000 - Repartition Test Time A BN C 3 6 9 12 15 SE +/- 0.06, N = 3 SE +/- 0.06, N = 3 9.30 9.45 9.51
Apache Spark Row Count: 20000000 - Partitions: 100 - Broadcast Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 100 - Broadcast Inner Join Test Time A BN C 5 10 15 20 25 SE +/- 0.24, N = 3 SE +/- 0.25, N = 3 20.75 20.80 21.22
Apache Spark Row Count: 40000000 - Partitions: 100 - Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 100 - Inner Join Test Time BN C A 9 18 27 36 45 SE +/- 0.28, N = 9 SE +/- 0.17, N = 3 39.98 40.12 40.84
Apache Spark Row Count: 10000000 - Partitions: 500 - Repartition Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 500 - Repartition Test Time BN A C 3 6 9 12 15 SE +/- 0.19, N = 3 SE +/- 0.10, N = 6 8.93 9.09 9.12
Apache Spark Row Count: 40000000 - Partitions: 500 - Calculate Pi Benchmark Using Dataframe OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 500 - Calculate Pi Benchmark Using Dataframe A C BN 2 4 6 8 10 SE +/- 0.04, N = 9 SE +/- 0.02, N = 5 6.72 6.84 6.86
Apache Spark Row Count: 1000000 - Partitions: 500 - Calculate Pi Benchmark OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 500 - Calculate Pi Benchmark A BN C 20 40 60 80 100 SE +/- 0.12, N = 8 SE +/- 0.32, N = 3 108.23 109.91 110.36
Apache Spark Row Count: 40000000 - Partitions: 1000 - Calculate Pi Benchmark Using Dataframe OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 1000 - Calculate Pi Benchmark Using Dataframe C A BN 2 4 6 8 10 SE +/- 0.03, N = 9 6.74 6.86 6.87
Apache Spark Row Count: 40000000 - Partitions: 100 - Calculate Pi Benchmark Using Dataframe OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 100 - Calculate Pi Benchmark Using Dataframe BN A C 2 4 6 8 10 SE +/- 0.01, N = 9 SE +/- 0.06, N = 3 6.79 6.84 6.92
Apache Spark Row Count: 20000000 - Partitions: 2000 - SHA-512 Benchmark Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 2000 - SHA-512 Benchmark Time A C BN 6 12 18 24 30 SE +/- 0.16, N = 12 23.25 23.55 23.69
Apache Spark Row Count: 20000000 - Partitions: 500 - Group By Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 500 - Group By Test Time C A BN 3 6 9 12 15 SE +/- 0.07, N = 3 SE +/- 0.11, N = 3 11.77 11.89 11.99
Apache Spark Row Count: 10000000 - Partitions: 100 - SHA-512 Benchmark Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 100 - SHA-512 Benchmark Time A C BN 4 8 12 16 20 SE +/- 0.10, N = 3 SE +/- 0.07, N = 3 13.93 14.01 14.19
Apache Spark Row Count: 10000000 - Partitions: 100 - Repartition Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 100 - Repartition Test Time C BN A 3 6 9 12 15 SE +/- 0.09, N = 3 SE +/- 0.16, N = 3 9.30 9.31 9.47
SVT-AV1 Encoder Mode: Preset 4 - Input: Bosphorus 1080p OpenBenchmarking.org Frames Per Second, More Is Better SVT-AV1 1.2 Encoder Mode: Preset 4 - Input: Bosphorus 1080p A C BN 0.8084 1.6168 2.4252 3.2336 4.042 SE +/- 0.014, N = 3 SE +/- 0.015, N = 3 3.593 3.566 3.530 1. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq
Apache Spark Row Count: 40000000 - Partitions: 500 - Group By Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 500 - Group By Test Time BN C A 7 14 21 28 35 SE +/- 0.10, N = 5 SE +/- 0.11, N = 9 31.26 31.45 31.81
SVT-AV1 Encoder Mode: Preset 12 - Input: Bosphorus 1080p OpenBenchmarking.org Frames Per Second, More Is Better SVT-AV1 1.2 Encoder Mode: Preset 12 - Input: Bosphorus 1080p BN A C 50 100 150 200 250 SE +/- 1.68, N = 3 SE +/- 1.25, N = 3 223.44 221.10 219.62 1. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq
Apache Spark Row Count: 20000000 - Partitions: 500 - Broadcast Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 500 - Broadcast Inner Join Test Time BN A C 5 10 15 20 25 SE +/- 0.30, N = 3 SE +/- 0.12, N = 3 18.37 18.43 18.69
Apache Spark Row Count: 1000000 - Partitions: 2000 - SHA-512 Benchmark Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 2000 - SHA-512 Benchmark Time A C BN 1.1228 2.2456 3.3684 4.4912 5.614 SE +/- 0.049353750, N = 3 SE +/- 0.048557793, N = 6 4.904759563 4.920000000 4.990000000
Apache Spark Row Count: 10000000 - Partitions: 1000 - Calculate Pi Benchmark Using Dataframe OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 1000 - Calculate Pi Benchmark Using Dataframe A BN C 2 4 6 8 10 SE +/- 0.01, N = 12 SE +/- 0.01, N = 12 6.70 6.80 6.81
Apache Spark Row Count: 20000000 - Partitions: 1000 - Group By Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 1000 - Group By Test Time C A BN 3 6 9 12 15 SE +/- 0.15, N = 3 12.99 13.18 13.20
Apache Spark Row Count: 1000000 - Partitions: 100 - Calculate Pi Benchmark OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 100 - Calculate Pi Benchmark A C BN 20 40 60 80 100 SE +/- 0.10, N = 4 SE +/- 0.21, N = 3 108.42 109.87 110.17
Apache Spark Row Count: 10000000 - Partitions: 100 - Calculate Pi Benchmark OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 100 - Calculate Pi Benchmark A BN C 20 40 60 80 100 SE +/- 0.17, N = 3 SE +/- 0.10, N = 3 108.69 109.36 110.41
Apache Spark Row Count: 40000000 - Partitions: 1000 - Repartition Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 1000 - Repartition Test Time C A BN 7 14 21 28 35 SE +/- 0.18, N = 9 31.02 31.45 31.51
Apache Spark Row Count: 1000000 - Partitions: 100 - Calculate Pi Benchmark Using Dataframe OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 100 - Calculate Pi Benchmark Using Dataframe A C BN 2 4 6 8 10 SE +/- 0.022438723, N = 4 SE +/- 0.068811394, N = 3 6.783271558 6.860000000 6.890000000
Apache Spark Row Count: 10000000 - Partitions: 500 - Group By Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 500 - Group By Test Time BN C A 2 4 6 8 10 SE +/- 0.069704950, N = 3 SE +/- 0.074190780, N = 6 8.490000000 8.500000000 8.621025209
Apache Spark Row Count: 40000000 - Partitions: 100 - Calculate Pi Benchmark OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 100 - Calculate Pi Benchmark BN A C 20 40 60 80 100 SE +/- 0.15, N = 9 SE +/- 0.92, N = 3 109.75 110.52 111.43
Apache Spark Row Count: 10000000 - Partitions: 500 - SHA-512 Benchmark Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 500 - SHA-512 Benchmark Time BN A C 3 6 9 12 15 SE +/- 0.14, N = 3 SE +/- 0.13, N = 6 13.09 13.20 13.29
Apache Spark Row Count: 40000000 - Partitions: 2000 - Calculate Pi Benchmark Using Dataframe OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 2000 - Calculate Pi Benchmark Using Dataframe C BN A 2 4 6 8 10 SE +/- 0.02, N = 9 6.76 6.84 6.86
Apache Spark Row Count: 1000000 - Partitions: 2000 - Calculate Pi Benchmark OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 2000 - Calculate Pi Benchmark A BN C 20 40 60 80 100 SE +/- 0.17, N = 6 SE +/- 0.24, N = 3 108.60 110.04 110.12
Apache Spark Row Count: 20000000 - Partitions: 1000 - Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 1000 - Inner Join Test Time A C BN 5 10 15 20 25 SE +/- 0.31, N = 3 19.68 19.85 19.95
Apache Spark Row Count: 20000000 - Partitions: 100 - Repartition Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 100 - Repartition Test Time A C BN 4 8 12 16 20 SE +/- 0.07, N = 3 SE +/- 0.11, N = 3 17.15 17.37 17.38
Apache Spark Row Count: 10000000 - Partitions: 2000 - SHA-512 Benchmark Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 2000 - SHA-512 Benchmark Time BN A C 4 8 12 16 20 SE +/- 0.02, N = 3 SE +/- 0.07, N = 3 14.04 14.14 14.22
Apache Spark Row Count: 20000000 - Partitions: 2000 - Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 2000 - Inner Join Test Time C A BN 5 10 15 20 25 SE +/- 0.19, N = 12 20.30 20.39 20.55
Apache Spark Row Count: 10000000 - Partitions: 500 - Calculate Pi Benchmark OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 500 - Calculate Pi Benchmark A BN C 20 40 60 80 100 SE +/- 0.30, N = 3 SE +/- 0.25, N = 6 109.23 109.33 110.54
Apache Spark Row Count: 10000000 - Partitions: 2000 - Calculate Pi Benchmark Using Dataframe OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 2000 - Calculate Pi Benchmark Using Dataframe A BN C 2 4 6 8 10 SE +/- 0.02, N = 3 SE +/- 0.07, N = 3 6.76 6.78 6.84
Apache Spark Row Count: 40000000 - Partitions: 100 - Group By Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 100 - Group By Test Time BN A C 8 16 24 32 40 SE +/- 0.07, N = 9 SE +/- 0.61, N = 3 32.11 32.42 32.49
Apache Spark Row Count: 20000000 - Partitions: 1000 - Calculate Pi Benchmark Using Dataframe OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 1000 - Calculate Pi Benchmark Using Dataframe BN A C 2 4 6 8 10 SE +/- 0.046089856, N = 3 6.800000000 6.875336993 6.880000000
Apache Spark Row Count: 40000000 - Partitions: 500 - Calculate Pi Benchmark OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 500 - Calculate Pi Benchmark A BN C 20 40 60 80 100 SE +/- 0.12, N = 5 SE +/- 0.09, N = 9 109.23 109.94 110.39
Apache Spark Row Count: 40000000 - Partitions: 1000 - Group By Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 1000 - Group By Test Time A BN C 7 14 21 28 35 SE +/- 0.28, N = 9 29.13 29.34 29.43
SVT-AV1 Encoder Mode: Preset 8 - Input: Bosphorus 1080p OpenBenchmarking.org Frames Per Second, More Is Better SVT-AV1 1.2 Encoder Mode: Preset 8 - Input: Bosphorus 1080p BN A C 20 40 60 80 100 SE +/- 0.37, N = 3 SE +/- 0.10, N = 3 76.01 75.96 75.23 1. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq
Apache Spark Row Count: 40000000 - Partitions: 500 - SHA-512 Benchmark Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 500 - SHA-512 Benchmark Time A C BN 10 20 30 40 50 SE +/- 0.47, N = 9 SE +/- 0.47, N = 5 45.82 46.14 46.27
Apache Spark Row Count: 10000000 - Partitions: 500 - Calculate Pi Benchmark Using Dataframe OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 500 - Calculate Pi Benchmark Using Dataframe A BN C 2 4 6 8 10 SE +/- 0.003533182, N = 3 SE +/- 0.025564809, N = 6 6.759258954 6.780000000 6.820000000
SVT-AV1 Encoder Mode: Preset 10 - Input: Bosphorus 4K OpenBenchmarking.org Frames Per Second, More Is Better SVT-AV1 1.2 Encoder Mode: Preset 10 - Input: Bosphorus 4K A C BN 14 28 42 56 70 SE +/- 0.53, N = 3 SE +/- 0.21, N = 3 61.39 60.94 60.85 1. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq
Apache Spark Row Count: 20000000 - Partitions: 500 - Calculate Pi Benchmark Using Dataframe OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 500 - Calculate Pi Benchmark Using Dataframe BN A C 2 4 6 8 10 SE +/- 0.034639869, N = 3 SE +/- 0.012838149, N = 3 6.800000000 6.812744629 6.860000000
Apache Spark Row Count: 1000000 - Partitions: 1000 - Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 1000 - Inner Join Test Time A C BN 0.5513 1.1026 1.6539 2.2052 2.7565 SE +/- 0.04, N = 3 SE +/- 0.05, N = 3 2.43 2.43 2.45
Apache Spark Row Count: 20000000 - Partitions: 500 - SHA-512 Benchmark Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 500 - SHA-512 Benchmark Time BN C A 5 10 15 20 25 SE +/- 0.30, N = 3 SE +/- 0.27, N = 3 22.58 22.64 22.75
Apache Spark Row Count: 20000000 - Partitions: 100 - Calculate Pi Benchmark Using Dataframe OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 100 - Calculate Pi Benchmark Using Dataframe C A BN 2 4 6 8 10 SE +/- 0.03, N = 3 SE +/- 0.01, N = 3 6.77 6.80 6.82
Apache Spark Row Count: 1000000 - Partitions: 2000 - Calculate Pi Benchmark Using Dataframe OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 2000 - Calculate Pi Benchmark Using Dataframe BN C A 2 4 6 8 10 SE +/- 0.02, N = 6 SE +/- 0.05, N = 3 6.79 6.81 6.84
Apache Spark Row Count: 10000000 - Partitions: 100 - Calculate Pi Benchmark Using Dataframe OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 100 - Calculate Pi Benchmark Using Dataframe BN A C 2 4 6 8 10 SE +/- 0.00, N = 3 SE +/- 0.03, N = 3 6.80 6.81 6.85
Apache Spark Row Count: 20000000 - Partitions: 1000 - Calculate Pi Benchmark OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 1000 - Calculate Pi Benchmark C BN A 20 40 60 80 100 SE +/- 0.19, N = 3 109.96 110.26 110.73
Apache Spark Row Count: 1000000 - Partitions: 1000 - Calculate Pi Benchmark Using Dataframe OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 1000 - Calculate Pi Benchmark Using Dataframe A BN C 2 4 6 8 10 SE +/- 0.031303906, N = 3 SE +/- 0.025947993, N = 3 6.799198247 6.820000000 6.840000000
Apache Spark Row Count: 10000000 - Partitions: 100 - Broadcast Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 100 - Broadcast Inner Join Test Time A BN C 3 6 9 12 15 SE +/- 0.24, N = 3 SE +/- 0.11, N = 3 10.66 10.69 10.72
Apache Spark Row Count: 10000000 - Partitions: 1000 - Calculate Pi Benchmark OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 1000 - Calculate Pi Benchmark A BN C 20 40 60 80 100 SE +/- 0.13, N = 12 SE +/- 0.15, N = 12 109.52 109.97 110.11
Apache Spark Row Count: 20000000 - Partitions: 100 - Group By Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 100 - Group By Test Time A C BN 3 6 9 12 15 SE +/- 0.07, N = 3 SE +/- 0.20, N = 3 13.33 13.35 13.40
Apache Spark Row Count: 1000000 - Partitions: 500 - Calculate Pi Benchmark Using Dataframe OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 500 - Calculate Pi Benchmark Using Dataframe A BN C 2 4 6 8 10 SE +/- 0.02582423, N = 8 SE +/- 0.02772713, N = 3 6.77685602 6.81000000 6.81000000
Apache Spark Row Count: 20000000 - Partitions: 100 - Calculate Pi Benchmark OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 100 - Calculate Pi Benchmark A BN C 20 40 60 80 100 SE +/- 0.21, N = 3 SE +/- 0.14, N = 3 109.11 109.58 109.62
Apache Spark Row Count: 40000000 - Partitions: 2000 - Calculate Pi Benchmark OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 2000 - Calculate Pi Benchmark A BN C 20 40 60 80 100 SE +/- 0.21, N = 9 110.09 110.55 110.59
Apache Spark Row Count: 40000000 - Partitions: 1000 - Calculate Pi Benchmark OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 40000000 - Partitions: 1000 - Calculate Pi Benchmark A C BN 20 40 60 80 100 SE +/- 0.19, N = 9 109.90 110.27 110.39
Apache Spark Row Count: 20000000 - Partitions: 2000 - Calculate Pi Benchmark OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 2000 - Calculate Pi Benchmark A BN C 20 40 60 80 100 SE +/- 0.12, N = 12 109.44 109.90 109.92
SVT-AV1 Encoder Mode: Preset 12 - Input: Bosphorus 4K OpenBenchmarking.org Frames Per Second, More Is Better SVT-AV1 1.2 Encoder Mode: Preset 12 - Input: Bosphorus 4K A BN C 20 40 60 80 100 SE +/- 0.46, N = 3 SE +/- 0.47, N = 3 89.75 89.46 89.37 1. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq
SVT-AV1 Encoder Mode: Preset 8 - Input: Bosphorus 4K OpenBenchmarking.org Frames Per Second, More Is Better SVT-AV1 1.2 Encoder Mode: Preset 8 - Input: Bosphorus 4K BN C A 7 14 21 28 35 SE +/- 0.10, N = 3 SE +/- 0.08, N = 3 32.13 32.02 32.00 1. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq
SVT-AV1 Encoder Mode: Preset 4 - Input: Bosphorus 4K OpenBenchmarking.org Frames Per Second, More Is Better SVT-AV1 1.2 Encoder Mode: Preset 4 - Input: Bosphorus 4K A C BN 0.3114 0.6228 0.9342 1.2456 1.557 SE +/- 0.001, N = 3 SE +/- 0.004, N = 3 1.384 1.379 1.379 1. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq
Apache Spark Row Count: 1000000 - Partitions: 1000 - Calculate Pi Benchmark OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 1000 - Calculate Pi Benchmark BN A C 20 40 60 80 100 SE +/- 0.38, N = 3 SE +/- 0.15, N = 3 109.68 109.81 110.06
Apache Spark Row Count: 20000000 - Partitions: 500 - Calculate Pi Benchmark OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 500 - Calculate Pi Benchmark A BN C 20 40 60 80 100 SE +/- 0.13, N = 3 SE +/- 0.17, N = 3 109.61 109.97 109.98
Apache Spark Row Count: 10000000 - Partitions: 2000 - Calculate Pi Benchmark OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 2000 - Calculate Pi Benchmark A C BN 20 40 60 80 100 SE +/- 0.22, N = 3 SE +/- 0.63, N = 3 109.91 109.96 110.25
Apache Spark Row Count: 20000000 - Partitions: 2000 - Calculate Pi Benchmark Using Dataframe OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 2000 - Calculate Pi Benchmark Using Dataframe A C BN 2 4 6 8 10 SE +/- 0.01, N = 12 6.77 6.77 6.79
Apache Spark Row Count: 10000000 - Partitions: 2000 - Broadcast Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 10000000 - Partitions: 2000 - Broadcast Inner Join Test Time A C BN 3 6 9 12 15 SE +/- 0.50, N = 3 SE +/- 0.35, N = 3 9.96 10.54 10.83
Apache Spark Row Count: 20000000 - Partitions: 100 - Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 20000000 - Partitions: 100 - Inner Join Test Time BN C A 5 10 15 20 25 SE +/- 0.71, N = 3 SE +/- 0.24, N = 3 20.35 20.60 20.68
Apache Spark Row Count: 1000000 - Partitions: 500 - Broadcast Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 500 - Broadcast Inner Join Test Time A BN C 0.4028 0.8056 1.2084 1.6112 2.014 SE +/- 0.05474021, N = 8 SE +/- 0.04087612, N = 3 1.70007341 1.77000000 1.79000000
Apache Spark Row Count: 1000000 - Partitions: 500 - Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 500 - Inner Join Test Time A BN C 0.5085 1.017 1.5255 2.034 2.5425 SE +/- 0.028445371, N = 8 SE +/- 0.110292282, N = 3 2.170995055 2.250000000 2.260000000
Apache Spark Row Count: 1000000 - Partitions: 100 - Broadcast Inner Join Test Time OpenBenchmarking.org Seconds, Fewer Is Better Apache Spark 3.3 Row Count: 1000000 - Partitions: 100 - Broadcast Inner Join Test Time C BN A 0.4438 0.8876 1.3314 1.7752 2.219 SE +/- 0.055547846, N = 4 SE +/- 0.059135782, N = 3 1.750000000 1.880000000 1.972517798
Phoronix Test Suite v10.8.5