spark svt AMD EPYC 7551 32-Core testing with a GIGABYTE MZ31-AR0-00 v01010101 (F10 BIOS) and ASPEED on Debian 11 via the Phoronix Test Suite. A: Processor: AMD EPYC 7551 32-Core @ 2.00GHz (32 Cores / 64 Threads), Motherboard: GIGABYTE MZ31-AR0-00 v01010101 (F10 BIOS), Chipset: AMD 17h, Memory: 8 x 4 GB DDR4-2667MT/s 9ASF51272PZ-2G6E1, Disk: Samsung SSD 960 EVO 500GB + 31GB SanDisk 3.2Gen1, Graphics: ASPEED, Network: Realtek RTL8111/8168/8411 + 2 x Broadcom NetXtreme II BCM57810 10 OS: Debian 11, Kernel: 5.10.0-9-amd64 (x86_64), Compiler: GCC 10.2.1 20210110, File-System: ext4, Screen Resolution: 1024x768 B: Processor: AMD EPYC 7551 32-Core @ 2.00GHz (32 Cores / 64 Threads), Motherboard: GIGABYTE MZ31-AR0-00 v01010101 (F10 BIOS), Chipset: AMD 17h, Memory: 8 x 4 GB DDR4-2667MT/s 9ASF51272PZ-2G6E1, Disk: Samsung SSD 960 EVO 500GB + 31GB SanDisk 3.2Gen1, Graphics: ASPEED, Network: Realtek RTL8111/8168/8411 + 2 x Broadcom NetXtreme II BCM57810 10 OS: Debian 11, Kernel: 5.10.0-9-amd64 (x86_64), Compiler: GCC 10.2.1 20210110, File-System: ext4, Screen Resolution: 1024x768 C: Processor: AMD EPYC 7551 32-Core @ 2.00GHz (32 Cores / 64 Threads), Motherboard: GIGABYTE MZ31-AR0-00 v01010101 (F10 BIOS), Chipset: AMD 17h, Memory: 8 x 4 GB DDR4-2667MT/s 9ASF51272PZ-2G6E1, Disk: Samsung SSD 960 EVO 500GB + 31GB SanDisk 3.2Gen1, Graphics: ASPEED, Network: Realtek RTL8111/8168/8411 + 2 x Broadcom NetXtreme II BCM57810 10 OS: Debian 11, Kernel: 5.10.0-9-amd64 (x86_64), Compiler: GCC 10.2.1 20210110, File-System: ext4, Screen Resolution: 1024x768 D: Processor: AMD EPYC 7551 32-Core @ 2.00GHz (32 Cores / 64 Threads), Motherboard: GIGABYTE MZ31-AR0-00 v01010101 (F10 BIOS), Chipset: AMD 17h, Memory: 8 x 4 GB DDR4-2667MT/s 9ASF51272PZ-2G6E1, Disk: Samsung SSD 960 EVO 500GB + 31GB SanDisk 3.2Gen1, Graphics: ASPEED, Network: Realtek RTL8111/8168/8411 + 2 x Broadcom NetXtreme II BCM57810 10 OS: Debian 11, Kernel: 5.10.0-9-amd64 (x86_64), Compiler: GCC 10.2.1 20210110, File-System: ext4, Screen Resolution: 1024x768 SVT-AV1 1.2 Encoder Mode: Preset 4 - Input: Bosphorus 4K Frames Per Second > Higher Is Better A . 0.945 |================================================================ B . 1.008 |==================================================================== C . 0.974 |================================================================== D . 0.984 |================================================================== SVT-AV1 1.2 Encoder Mode: Preset 8 - Input: Bosphorus 4K Frames Per Second > Higher Is Better A . 28.88 |==================================================================== B . 24.24 |========================================================= C . 25.17 |=========================================================== D . 25.42 |============================================================ SVT-AV1 1.2 Encoder Mode: Preset 10 - Input: Bosphorus 4K Frames Per Second > Higher Is Better A . 49.17 |==================================================================== B . 40.57 |======================================================== C . 42.51 |=========================================================== D . 46.59 |================================================================ SVT-AV1 1.2 Encoder Mode: Preset 12 - Input: Bosphorus 4K Frames Per Second > Higher Is Better A . 62.04 |==================================================================== B . 60.31 |================================================================== C . 54.82 |============================================================ D . 55.32 |============================================================= SVT-AV1 1.2 Encoder Mode: Preset 4 - Input: Bosphorus 1080p Frames Per Second > Higher Is Better A . 2.432 |================================================================= B . 2.539 |==================================================================== C . 2.492 |=================================================================== D . 2.529 |==================================================================== SVT-AV1 1.2 Encoder Mode: Preset 8 - Input: Bosphorus 1080p Frames Per Second > Higher Is Better A . 61.78 |=================================================================== B . 57.43 |=============================================================== C . 62.24 |==================================================================== D . 60.95 |=================================================================== SVT-AV1 1.2 Encoder Mode: Preset 10 - Input: Bosphorus 1080p Frames Per Second > Higher Is Better A . 129.46 |=============================================================== B . 119.50 |========================================================== C . 131.75 |================================================================ D . 138.69 |=================================================================== SVT-AV1 1.2 Encoder Mode: Preset 12 - Input: Bosphorus 1080p Frames Per Second > Higher Is Better A . 193.67 |================================================================== B . 170.28 |========================================================== C . 196.23 |=================================================================== D . 195.51 |=================================================================== Apache Spark 3.3 Row Count: 1000000 - Partitions: 100 - SHA-512 Benchmark Time Seconds < Lower Is Better A . 7.62 |==================================================================== B . 7.16 |================================================================ C . 7.69 |===================================================================== D . 7.55 |==================================================================== Apache Spark 3.3 Row Count: 1000000 - Partitions: 100 - Calculate Pi Benchmark Seconds < Lower Is Better A . 78.68 |=================================================================== B . 80.09 |==================================================================== C . 79.56 |==================================================================== D . 78.79 |=================================================================== Apache Spark 3.3 Row Count: 1000000 - Partitions: 100 - Calculate Pi Benchmark Using Dataframe Seconds < Lower Is Better A . 6.05 |================================================================== B . 6.36 |===================================================================== C . 6.26 |==================================================================== D . 6.26 |==================================================================== Apache Spark 3.3 Row Count: 1000000 - Partitions: 100 - Group By Test Time Seconds < Lower Is Better A . 9.24 |=================================================================== B . 8.93 |================================================================ C . 9.25 |=================================================================== D . 9.56 |===================================================================== Apache Spark 3.3 Row Count: 1000000 - Partitions: 100 - Repartition Test Time Seconds < Lower Is Better A . 4.64 |======================================== B . 3.87 |================================== C . 4.77 |========================================= D . 7.94 |===================================================================== Apache Spark 3.3 Row Count: 1000000 - Partitions: 100 - Inner Join Test Time Seconds < Lower Is Better A . 3.65 |================================================== B . 5.01 |===================================================================== C . 3.97 |======================================================= D . 3.52 |================================================ Apache Spark 3.3 Row Count: 1000000 - Partitions: 100 - Broadcast Inner Join Test Time Seconds < Lower Is Better A . 3.10 |===================================================================== B . 2.56 |========================================================= C . 2.05 |============================================== D . 2.91 |================================================================= Apache Spark 3.3 Row Count: 1000000 - Partitions: 500 - SHA-512 Benchmark Time Seconds < Lower Is Better A . 8.57 |===================================================================== B . 7.57 |============================================================= C . 7.71 |============================================================== D . 7.82 |=============================================================== Apache Spark 3.3 Row Count: 1000000 - Partitions: 500 - Calculate Pi Benchmark Seconds < Lower Is Better A . 78.34 |=================================================================== B . 79.11 |==================================================================== C . 78.54 |==================================================================== D . 78.64 |==================================================================== Apache Spark 3.3 Row Count: 1000000 - Partitions: 500 - Calculate Pi Benchmark Using Dataframe Seconds < Lower Is Better A . 6.02 |================================================================= B . 6.42 |===================================================================== C . 6.26 |=================================================================== D . 6.30 |==================================================================== Apache Spark 3.3 Row Count: 1000000 - Partitions: 500 - Group By Test Time Seconds < Lower Is Better A . 9.85 |==================================================================== B . 9.30 |================================================================ C . 9.53 |================================================================== D . 9.99 |===================================================================== Apache Spark 3.3 Row Count: 1000000 - Partitions: 500 - Repartition Test Time Seconds < Lower Is Better A . 5.09 |================================================ B . 5.98 |========================================================= C . 7.29 |===================================================================== D . 5.83 |======================================================= Apache Spark 3.3 Row Count: 1000000 - Partitions: 500 - Inner Join Test Time Seconds < Lower Is Better A . 4.40 |====================================================== B . 4.58 |======================================================== C . 4.43 |====================================================== D . 5.62 |===================================================================== Apache Spark 3.3 Row Count: 1000000 - Partitions: 500 - Broadcast Inner Join Test Time Seconds < Lower Is Better A . 3.16 |================================================= B . 3.56 |======================================================= C . 4.47 |===================================================================== D . 3.11 |================================================ Apache Spark 3.3 Row Count: 1000000 - Partitions: 1000 - SHA-512 Benchmark Time Seconds < Lower Is Better A . 7.76 |============================================================= B . 7.43 |=========================================================== C . 8.75 |===================================================================== D . 7.81 |============================================================== Apache Spark 3.3 Row Count: 1000000 - Partitions: 1000 - Calculate Pi Benchmark Seconds < Lower Is Better A . 78.88 |==================================================================== B . 78.84 |==================================================================== C . 78.56 |==================================================================== D . 78.62 |==================================================================== Apache Spark 3.3 Row Count: 1000000 - Partitions: 1000 - Calculate Pi Benchmark Using Dataframe Seconds < Lower Is Better A . 6.08 |================================================================ B . 6.40 |==================================================================== C . 6.54 |===================================================================== D . 6.11 |================================================================ Apache Spark 3.3 Row Count: 1000000 - Partitions: 1000 - Group By Test Time Seconds < Lower Is Better A . 9.99 |=============================================================== B . 10.81 |==================================================================== C . 10.42 |================================================================== D . 10.15 |================================================================ Apache Spark 3.3 Row Count: 1000000 - Partitions: 1000 - Repartition Test Time Seconds < Lower Is Better A . 5.43 |========================================= B . 5.68 |=========================================== C . 5.30 |======================================== D . 9.22 |===================================================================== Apache Spark 3.3 Row Count: 1000000 - Partitions: 1000 - Inner Join Test Time Seconds < Lower Is Better A . 4.53 |========================================================= B . 5.01 |=============================================================== C . 5.47 |===================================================================== D . 4.33 |======================================================= Apache Spark 3.3 Row Count: 1000000 - Partitions: 1000 - Broadcast Inner Join Test Time Seconds < Lower Is Better A . 3.02 |=================================================== B . 4.06 |===================================================================== C . 3.25 |======================================================= D . 3.28 |======================================================== Apache Spark 3.3 Row Count: 1000000 - Partitions: 2000 - SHA-512 Benchmark Time Seconds < Lower Is Better A . 8.31 |=================================================================== B . 8.61 |===================================================================== C . 7.71 |============================================================== D . 8.56 |===================================================================== Apache Spark 3.3 Row Count: 1000000 - Partitions: 2000 - Calculate Pi Benchmark Seconds < Lower Is Better A . 78.84 |=================================================================== B . 79.73 |==================================================================== C . 78.70 |=================================================================== D . 78.50 |=================================================================== Apache Spark 3.3 Row Count: 1000000 - Partitions: 2000 - Calculate Pi Benchmark Using Dataframe Seconds < Lower Is Better A . 6.57 |===================================================================== B . 6.20 |================================================================= C . 6.21 |================================================================= D . 6.32 |================================================================== Apache Spark 3.3 Row Count: 1000000 - Partitions: 2000 - Group By Test Time Seconds < Lower Is Better A . 10.05 |=============================================================== B . 10.77 |==================================================================== C . 10.13 |================================================================ D . 10.01 |=============================================================== Apache Spark 3.3 Row Count: 1000000 - Partitions: 2000 - Repartition Test Time Seconds < Lower Is Better A . 6.19 |=========================================================== B . 7.19 |===================================================================== C . 5.76 |======================================================= D . 6.97 |=================================================================== Apache Spark 3.3 Row Count: 1000000 - Partitions: 2000 - Inner Join Test Time Seconds < Lower Is Better A . 5.74 |============================================================= B . 5.87 |============================================================== C . 6.52 |===================================================================== D . 5.75 |============================================================= Apache Spark 3.3 Row Count: 1000000 - Partitions: 2000 - Broadcast Inner Join Test Time Seconds < Lower Is Better A . 5.62 |================================================== B . 7.74 |===================================================================== C . 4.57 |========================================= D . 5.71 |=================================================== Apache Spark 3.3 Row Count: 10000000 - Partitions: 100 - SHA-512 Benchmark Time Seconds < Lower Is Better A . 22.38 |============================================================== B . 21.43 |============================================================ C . 24.07 |=================================================================== D . 24.39 |==================================================================== Apache Spark 3.3 Row Count: 10000000 - Partitions: 100 - Calculate Pi Benchmark Seconds < Lower Is Better A . 78.80 |=================================================================== B . 79.50 |==================================================================== C . 78.44 |=================================================================== D . 78.89 |=================================================================== Apache Spark 3.3 Row Count: 10000000 - Partitions: 100 - Calculate Pi Benchmark Using Dataframe Seconds < Lower Is Better A . 6.40 |===================================================================== B . 6.27 |=================================================================== C . 6.31 |==================================================================== D . 6.43 |===================================================================== Apache Spark 3.3 Row Count: 10000000 - Partitions: 100 - Group By Test Time Seconds < Lower Is Better A . 14.55 |========================================================= B . 15.10 |=========================================================== C . 15.18 |=========================================================== D . 17.51 |==================================================================== Apache Spark 3.3 Row Count: 10000000 - Partitions: 100 - Repartition Test Time Seconds < Lower Is Better A . 12.31 |================================================== B . 16.72 |==================================================================== C . 15.71 |================================================================ D . 16.34 |================================================================== Apache Spark 3.3 Row Count: 10000000 - Partitions: 100 - Inner Join Test Time Seconds < Lower Is Better A . 13.51 |==================================================================== B . 11.79 |=========================================================== C . 13.11 |================================================================== D . 13.35 |=================================================================== Apache Spark 3.3 Row Count: 10000000 - Partitions: 100 - Broadcast Inner Join Test Time Seconds < Lower Is Better A . 11.67 |==================================================================== B . 10.56 |============================================================== C . 11.12 |================================================================= D . 10.93 |================================================================ Apache Spark 3.3 Row Count: 10000000 - Partitions: 500 - SHA-512 Benchmark Time Seconds < Lower Is Better A . 21.91 |================================================================== B . 22.51 |==================================================================== C . 22.00 |================================================================== D . 21.95 |================================================================== Apache Spark 3.3 Row Count: 10000000 - Partitions: 500 - Calculate Pi Benchmark Seconds < Lower Is Better A . 79.68 |==================================================================== B . 79.44 |==================================================================== C . 79.27 |==================================================================== D . 79.22 |==================================================================== Apache Spark 3.3 Row Count: 10000000 - Partitions: 500 - Calculate Pi Benchmark Using Dataframe Seconds < Lower Is Better A . 6.24 |================================================================== B . 6.14 |================================================================= C . 6.22 |================================================================= D . 6.56 |===================================================================== Apache Spark 3.3 Row Count: 10000000 - Partitions: 500 - Group By Test Time Seconds < Lower Is Better A . 15.30 |===================================================== B . 19.54 |==================================================================== C . 15.00 |==================================================== D . 15.50 |====================================================== Apache Spark 3.3 Row Count: 10000000 - Partitions: 500 - Repartition Test Time Seconds < Lower Is Better A . 15.13 |=================================================================== B . 13.72 |============================================================= C . 12.64 |======================================================== D . 15.26 |==================================================================== Apache Spark 3.3 Row Count: 10000000 - Partitions: 500 - Inner Join Test Time Seconds < Lower Is Better A . 12.56 |============================================================ B . 12.17 |========================================================== C . 14.15 |==================================================================== D . 13.23 |================================================================ Apache Spark 3.3 Row Count: 10000000 - Partitions: 500 - Broadcast Inner Join Test Time Seconds < Lower Is Better A . 12.28 |==================================================================== B . 11.23 |============================================================== C . 12.29 |==================================================================== D . 10.81 |============================================================ Apache Spark 3.3 Row Count: 20000000 - Partitions: 100 - SHA-512 Benchmark Time Seconds < Lower Is Better A . 36.15 |======================================================== B . 35.84 |======================================================= C . 34.03 |===================================================== D . 44.06 |==================================================================== Apache Spark 3.3 Row Count: 20000000 - Partitions: 100 - Calculate Pi Benchmark Seconds < Lower Is Better A . 79.16 |==================================================================== B . 79.73 |==================================================================== C . 79.19 |==================================================================== D . 79.25 |==================================================================== Apache Spark 3.3 Row Count: 20000000 - Partitions: 100 - Calculate Pi Benchmark Using Dataframe Seconds < Lower Is Better A . 6.03 |===================================================================== B . 6.03 |===================================================================== C . 5.97 |==================================================================== D . 6.02 |===================================================================== Apache Spark 3.3 Row Count: 20000000 - Partitions: 100 - Group By Test Time Seconds < Lower Is Better A . 21.05 |==================================================================== B . 20.29 |================================================================== C . 18.53 |============================================================ D . 18.98 |============================================================= Apache Spark 3.3 Row Count: 20000000 - Partitions: 100 - Repartition Test Time Seconds < Lower Is Better A . 23.16 |==================================================================== B . 20.08 |=========================================================== C . 21.41 |=============================================================== D . 19.57 |========================================================= Apache Spark 3.3 Row Count: 20000000 - Partitions: 100 - Inner Join Test Time Seconds < Lower Is Better A . 21.76 |================================================================= B . 20.00 |=========================================================== C . 22.90 |==================================================================== D . 22.59 |=================================================================== Apache Spark 3.3 Row Count: 20000000 - Partitions: 100 - Broadcast Inner Join Test Time Seconds < Lower Is Better A . 19.70 |============================================================= B . 19.17 |============================================================ C . 21.90 |==================================================================== D . 20.34 |=============================================================== Apache Spark 3.3 Row Count: 20000000 - Partitions: 500 - SHA-512 Benchmark Time Seconds < Lower Is Better A . 37.11 |=============================================================== B . 37.95 |================================================================ C . 40.03 |==================================================================== D . 36.68 |============================================================== Apache Spark 3.3 Row Count: 20000000 - Partitions: 500 - Calculate Pi Benchmark Seconds < Lower Is Better A . 80.53 |==================================================================== B . 79.91 |=================================================================== C . 80.02 |==================================================================== D . 79.76 |=================================================================== Apache Spark 3.3 Row Count: 20000000 - Partitions: 500 - Calculate Pi Benchmark Using Dataframe Seconds < Lower Is Better A . 6.29 |===================================================================== B . 6.17 |==================================================================== C . 6.07 |=================================================================== D . 5.91 |================================================================= Apache Spark 3.3 Row Count: 20000000 - Partitions: 500 - Group By Test Time Seconds < Lower Is Better A . 21.82 |==================================================================== B . 19.57 |============================================================= C . 20.64 |================================================================ D . 21.59 |=================================================================== Apache Spark 3.3 Row Count: 20000000 - Partitions: 500 - Repartition Test Time Seconds < Lower Is Better A . 19.18 |======================================================== B . 23.49 |==================================================================== C . 22.41 |================================================================= D . 19.20 |======================================================== Apache Spark 3.3 Row Count: 20000000 - Partitions: 500 - Inner Join Test Time Seconds < Lower Is Better A . 19.96 |================================================================= B . 19.46 |=============================================================== C . 20.95 |==================================================================== D . 20.17 |================================================================= Apache Spark 3.3 Row Count: 20000000 - Partitions: 500 - Broadcast Inner Join Test Time Seconds < Lower Is Better A . 18.50 |=============================================================== B . 18.12 |============================================================= C . 20.08 |==================================================================== D . 18.79 |================================================================ Apache Spark 3.3 Row Count: 40000000 - Partitions: 100 - SHA-512 Benchmark Time Seconds < Lower Is Better A . 79.48 |==================================================================== B . 61.73 |===================================================== C . 60.38 |==================================================== D . 71.77 |============================================================= Apache Spark 3.3 Row Count: 40000000 - Partitions: 100 - Calculate Pi Benchmark Seconds < Lower Is Better A . 80.39 |==================================================================== B . 80.88 |==================================================================== C . 79.71 |=================================================================== D . 79.72 |=================================================================== Apache Spark 3.3 Row Count: 40000000 - Partitions: 100 - Calculate Pi Benchmark Using Dataframe Seconds < Lower Is Better A . 6.06 |==================================================================== B . 6.09 |===================================================================== C . 5.90 |=================================================================== D . 6.11 |===================================================================== Apache Spark 3.3 Row Count: 40000000 - Partitions: 100 - Group By Test Time Seconds < Lower Is Better A . 42.58 |==================================================================== B . 42.59 |==================================================================== C . 38.98 |============================================================== D . 40.39 |================================================================ Apache Spark 3.3 Row Count: 40000000 - Partitions: 100 - Repartition Test Time Seconds < Lower Is Better A . 37.61 |==================================================================== B . 33.43 |============================================================ C . 34.68 |=============================================================== D . 36.69 |================================================================== Apache Spark 3.3 Row Count: 40000000 - Partitions: 100 - Inner Join Test Time Seconds < Lower Is Better A . 38.22 |==================================================================== B . 36.85 |================================================================== C . 36.68 |================================================================= D . 37.70 |=================================================================== Apache Spark 3.3 Row Count: 40000000 - Partitions: 100 - Broadcast Inner Join Test Time Seconds < Lower Is Better A . 39.41 |==================================================================== B . 35.04 |============================================================ C . 37.29 |================================================================ D . 36.87 |================================================================ Apache Spark 3.3 Row Count: 40000000 - Partitions: 500 - SHA-512 Benchmark Time Seconds < Lower Is Better A . 63.08 |============================================================= B . 65.06 |=============================================================== C . 66.57 |================================================================ D . 70.38 |==================================================================== Apache Spark 3.3 Row Count: 40000000 - Partitions: 500 - Calculate Pi Benchmark Seconds < Lower Is Better A . 80.69 |==================================================================== B . 81.20 |==================================================================== C . 80.43 |=================================================================== D . 80.07 |=================================================================== Apache Spark 3.3 Row Count: 40000000 - Partitions: 500 - Calculate Pi Benchmark Using Dataframe Seconds < Lower Is Better A . 6.07 |=================================================================== B . 6.26 |===================================================================== C . 6.18 |==================================================================== D . 6.04 |=================================================================== Apache Spark 3.3 Row Count: 40000000 - Partitions: 500 - Group By Test Time Seconds < Lower Is Better A . 46.76 |==================================================================== B . 45.47 |================================================================== C . 44.61 |================================================================= D . 45.73 |=================================================================== Apache Spark 3.3 Row Count: 40000000 - Partitions: 500 - Repartition Test Time Seconds < Lower Is Better A . 35.70 |==================================================================== B . 34.16 |================================================================= C . 35.64 |==================================================================== D . 34.35 |================================================================= Apache Spark 3.3 Row Count: 40000000 - Partitions: 500 - Inner Join Test Time Seconds < Lower Is Better A . 35.66 |================================================================ B . 35.86 |================================================================= C . 37.65 |==================================================================== D . 36.94 |=================================================================== Apache Spark 3.3 Row Count: 40000000 - Partitions: 500 - Broadcast Inner Join Test Time Seconds < Lower Is Better A . 35.72 |================================================================== B . 36.89 |==================================================================== C . 36.39 |=================================================================== D . 36.09 |=================================================================== Apache Spark 3.3 Row Count: 10000000 - Partitions: 1000 - SHA-512 Benchmark Time Seconds < Lower Is Better A . 22.56 |==================================================================== B . 22.14 |=================================================================== C . 21.85 |================================================================== D . 21.02 |=============================================================== Apache Spark 3.3 Row Count: 10000000 - Partitions: 1000 - Calculate Pi Benchmark Seconds < Lower Is Better A . 80.86 |==================================================================== B . 80.06 |=================================================================== C . 79.27 |=================================================================== D . 79.84 |=================================================================== Apache Spark 3.3 Row Count: 10000000 - Partitions: 1000 - Calculate Pi Benchmark Using Dataframe Seconds < Lower Is Better A . 6.18 |================================================================== B . 6.28 |=================================================================== C . 6.50 |===================================================================== D . 6.37 |==================================================================== Apache Spark 3.3 Row Count: 10000000 - Partitions: 1000 - Group By Test Time Seconds < Lower Is Better A . 17.09 |==================================================================== B . 16.50 |================================================================== C . 15.06 |============================================================ D . 14.20 |========================================================= Apache Spark 3.3 Row Count: 10000000 - Partitions: 1000 - Repartition Test Time Seconds < Lower Is Better A . 15.37 |==================================================================== B . 13.65 |============================================================ C . 14.85 |================================================================== D . 14.34 |=============================================================== Apache Spark 3.3 Row Count: 10000000 - Partitions: 1000 - Inner Join Test Time Seconds < Lower Is Better A . 13.87 |================================================================= B . 13.76 |================================================================= C . 14.40 |==================================================================== D . 13.19 |============================================================== Apache Spark 3.3 Row Count: 10000000 - Partitions: 1000 - Broadcast Inner Join Test Time Seconds < Lower Is Better A . 12.97 |==================================================================== B . 11.13 |========================================================== C . 11.95 |=============================================================== D . 11.71 |============================================================= Apache Spark 3.3 Row Count: 10000000 - Partitions: 2000 - SHA-512 Benchmark Time Seconds < Lower Is Better A . 23.00 |============================================================== B . 25.03 |==================================================================== C . 22.31 |============================================================= D . 22.89 |============================================================== Apache Spark 3.3 Row Count: 10000000 - Partitions: 2000 - Calculate Pi Benchmark Seconds < Lower Is Better A . 80.74 |==================================================================== B . 80.67 |==================================================================== C . 79.91 |=================================================================== D . 80.06 |=================================================================== Apache Spark 3.3 Row Count: 10000000 - Partitions: 2000 - Calculate Pi Benchmark Using Dataframe Seconds < Lower Is Better A . 6.19 |===================================================================== B . 5.93 |================================================================== C . 6.16 |===================================================================== D . 6.18 |===================================================================== Apache Spark 3.3 Row Count: 10000000 - Partitions: 2000 - Group By Test Time Seconds < Lower Is Better A . 19.09 |==================================================================== B . 17.98 |================================================================ C . 18.47 |================================================================== D . 16.30 |========================================================== Apache Spark 3.3 Row Count: 10000000 - Partitions: 2000 - Repartition Test Time Seconds < Lower Is Better A . 13.87 |================================================================== B . 14.26 |==================================================================== C . 13.46 |================================================================ D . 13.26 |=============================================================== Apache Spark 3.3 Row Count: 10000000 - Partitions: 2000 - Inner Join Test Time Seconds < Lower Is Better A . 17.08 |==================================================================== B . 14.95 |============================================================ C . 15.38 |============================================================= D . 14.55 |========================================================== Apache Spark 3.3 Row Count: 10000000 - Partitions: 2000 - Broadcast Inner Join Test Time Seconds < Lower Is Better A . 14.71 |==================================================================== B . 12.97 |============================================================ C . 11.82 |======================================================= D . 11.75 |====================================================== Apache Spark 3.3 Row Count: 20000000 - Partitions: 1000 - SHA-512 Benchmark Time Seconds < Lower Is Better A . 38.63 |================================================================ B . 41.15 |==================================================================== C . 35.63 |=========================================================== D . 36.37 |============================================================ Apache Spark 3.3 Row Count: 20000000 - Partitions: 1000 - Calculate Pi Benchmark Seconds < Lower Is Better A . 80.72 |==================================================================== B . 79.73 |=================================================================== C . 79.44 |=================================================================== D . 79.04 |=================================================================== Apache Spark 3.3 Row Count: 20000000 - Partitions: 1000 - Calculate Pi Benchmark Using Dataframe Seconds < Lower Is Better A . 6.04 |================================================================== B . 6.09 |=================================================================== C . 6.11 |=================================================================== D . 6.27 |===================================================================== Apache Spark 3.3 Row Count: 20000000 - Partitions: 1000 - Group By Test Time Seconds < Lower Is Better A . 21.57 |============================================================== B . 22.62 |================================================================= C . 23.78 |==================================================================== D . 19.23 |======================================================= Apache Spark 3.3 Row Count: 20000000 - Partitions: 1000 - Repartition Test Time Seconds < Lower Is Better A . 22.42 |============================================================= B . 23.28 |================================================================ C . 22.84 |============================================================== D . 24.90 |==================================================================== Apache Spark 3.3 Row Count: 20000000 - Partitions: 1000 - Inner Join Test Time Seconds < Lower Is Better A . 21.88 |==================================================================== B . 21.32 |================================================================== C . 21.06 |================================================================= D . 21.83 |==================================================================== Apache Spark 3.3 Row Count: 20000000 - Partitions: 1000 - Broadcast Inner Join Test Time Seconds < Lower Is Better A . 20.17 |==================================================================== B . 19.47 |================================================================== C . 19.45 |================================================================== D . 19.77 |=================================================================== Apache Spark 3.3 Row Count: 20000000 - Partitions: 2000 - SHA-512 Benchmark Time Seconds < Lower Is Better A . 39.28 |==================================================================== B . 39.19 |==================================================================== C . 36.55 |=============================================================== D . 37.41 |================================================================= Apache Spark 3.3 Row Count: 20000000 - Partitions: 2000 - Calculate Pi Benchmark Seconds < Lower Is Better A . 80.42 |==================================================================== B . 79.81 |=================================================================== C . 78.65 |=================================================================== D . 79.09 |=================================================================== Apache Spark 3.3 Row Count: 20000000 - Partitions: 2000 - Calculate Pi Benchmark Using Dataframe Seconds < Lower Is Better A . 6.10 |===================================================================== B . 6.01 |==================================================================== C . 5.92 |=================================================================== D . 6.08 |===================================================================== Apache Spark 3.3 Row Count: 20000000 - Partitions: 2000 - Group By Test Time Seconds < Lower Is Better A . 23.47 |=================================================================== B . 23.71 |==================================================================== C . 22.79 |================================================================= D . 22.55 |================================================================= Apache Spark 3.3 Row Count: 20000000 - Partitions: 2000 - Repartition Test Time Seconds < Lower Is Better A . 23.43 |==================================================================== B . 20.30 |=========================================================== C . 20.31 |=========================================================== D . 22.03 |================================================================ Apache Spark 3.3 Row Count: 20000000 - Partitions: 2000 - Inner Join Test Time Seconds < Lower Is Better A . 23.02 |=================================================================== B . 22.88 |=================================================================== C . 22.15 |================================================================= D . 23.34 |==================================================================== Apache Spark 3.3 Row Count: 20000000 - Partitions: 2000 - Broadcast Inner Join Test Time Seconds < Lower Is Better A . 21.60 |================================================================== B . 22.24 |==================================================================== C . 21.42 |================================================================= D . 22.39 |==================================================================== Apache Spark 3.3 Row Count: 40000000 - Partitions: 1000 - SHA-512 Benchmark Time Seconds < Lower Is Better A . 71.67 |==================================================================== B . 63.03 |============================================================ C . 63.36 |============================================================ D . 65.19 |============================================================== Apache Spark 3.3 Row Count: 40000000 - Partitions: 1000 - Calculate Pi Benchmark Seconds < Lower Is Better A . 80.96 |==================================================================== B . 80.79 |==================================================================== C . 79.91 |=================================================================== D . 80.39 |==================================================================== Apache Spark 3.3 Row Count: 40000000 - Partitions: 1000 - Calculate Pi Benchmark Using Dataframe Seconds < Lower Is Better A . 6.33 |===================================================================== B . 6.04 |================================================================== C . 6.24 |==================================================================== D . 6.26 |==================================================================== Apache Spark 3.3 Row Count: 40000000 - Partitions: 1000 - Group By Test Time Seconds < Lower Is Better A . 45.87 |==================================================================== B . 43.70 |================================================================= C . 41.48 |============================================================= D . 42.19 |=============================================================== Apache Spark 3.3 Row Count: 40000000 - Partitions: 1000 - Repartition Test Time Seconds < Lower Is Better A . 35.82 |==================================================================== B . 35.27 |=================================================================== C . 34.77 |================================================================== D . 34.94 |================================================================== Apache Spark 3.3 Row Count: 40000000 - Partitions: 1000 - Inner Join Test Time Seconds < Lower Is Better A . 37.92 |==================================================================== B . 37.54 |=================================================================== C . 36.42 |================================================================= D . 36.82 |================================================================== Apache Spark 3.3 Row Count: 40000000 - Partitions: 1000 - Broadcast Inner Join Test Time Seconds < Lower Is Better A . 37.36 |==================================================================== B . 36.63 |=================================================================== C . 36.12 |================================================================== D . 35.58 |================================================================= Apache Spark 3.3 Row Count: 40000000 - Partitions: 2000 - SHA-512 Benchmark Time Seconds < Lower Is Better A . 63.54 |=============================================================== B . 68.38 |==================================================================== C . 65.00 |================================================================= D . 63.32 |=============================================================== Apache Spark 3.3 Row Count: 40000000 - Partitions: 2000 - Calculate Pi Benchmark Seconds < Lower Is Better A . 80.61 |==================================================================== B . 80.46 |==================================================================== C . 80.01 |=================================================================== D . 79.77 |=================================================================== Apache Spark 3.3 Row Count: 40000000 - Partitions: 2000 - Calculate Pi Benchmark Using Dataframe Seconds < Lower Is Better A . 6.03 |================================================================= B . 6.41 |===================================================================== C . 6.09 |================================================================== D . 6.02 |================================================================= Apache Spark 3.3 Row Count: 40000000 - Partitions: 2000 - Group By Test Time Seconds < Lower Is Better A . 45.70 |================================================================== B . 46.88 |==================================================================== C . 43.42 |=============================================================== D . 45.96 |=================================================================== Apache Spark 3.3 Row Count: 40000000 - Partitions: 2000 - Repartition Test Time Seconds < Lower Is Better A . 34.29 |=============================================================== B . 36.69 |=================================================================== C . 34.33 |=============================================================== D . 37.23 |==================================================================== Apache Spark 3.3 Row Count: 40000000 - Partitions: 2000 - Inner Join Test Time Seconds < Lower Is Better A . 37.30 |=========================================================== B . 42.99 |==================================================================== C . 41.55 |================================================================== D . 40.05 |=============================================================== Apache Spark 3.3 Row Count: 40000000 - Partitions: 2000 - Broadcast Inner Join Test Time Seconds < Lower Is Better A . 38.32 |================================================================ B . 39.54 |================================================================== C . 39.50 |================================================================== D . 40.69 |====================================================================