tr5 spark svt av1

Benchmarks for a future article.

HTML result view exported from: https://openbenchmarking.org/result/2208075-NE-TR5SPARKS30&grs&sor.

Apache Spark

Row Count: 20000000 - Partitions: 1000 - Inner Join Test Time

Apache Spark

Row Count: 40000000 - Partitions: 500 - Inner Join Test Time

Apache Spark

Row Count: 40000000 - Partitions: 2000 - Repartition Test Time

Apache Spark

Row Count: 20000000 - Partitions: 100 - Inner Join Test Time

Apache Spark

Row Count: 40000000 - Partitions: 500 - Repartition Test Time

Apache Spark

Row Count: 10000000 - Partitions: 500 - Broadcast Inner Join Test Time

Apache Spark

Row Count: 20000000 - Partitions: 2000 - Broadcast Inner Join Test Time

Apache Spark

Row Count: 40000000 - Partitions: 1000 - Broadcast Inner Join Test Time

Apache Spark

Row Count: 40000000 - Partitions: 1000 - Repartition Test Time

Apache Spark

Row Count: 20000000 - Partitions: 500 - Inner Join Test Time

Apache Spark

Row Count: 40000000 - Partitions: 100 - Repartition Test Time

Apache Spark

Row Count: 20000000 - Partitions: 2000 - Repartition Test Time

Apache Spark

Row Count: 20000000 - Partitions: 100 - Broadcast Inner Join Test Time

Apache Spark

Row Count: 20000000 - Partitions: 2000 - Inner Join Test Time

Apache Spark

Row Count: 20000000 - Partitions: 1000 - Broadcast Inner Join Test Time

Apache Spark

Row Count: 20000000 - Partitions: 1000 - Repartition Test Time

Apache Spark

Row Count: 40000000 - Partitions: 500 - Broadcast Inner Join Test Time

Apache Spark

Row Count: 40000000 - Partitions: 2000 - Broadcast Inner Join Test Time

Apache Spark

Row Count: 20000000 - Partitions: 100 - Repartition Test Time

Apache Spark

Row Count: 40000000 - Partitions: 2000 - Inner Join Test Time

Apache Spark

Row Count: 20000000 - Partitions: 500 - Broadcast Inner Join Test Time

Apache Spark

Row Count: 20000000 - Partitions: 500 - Repartition Test Time

Apache Spark

Row Count: 10000000 - Partitions: 500 - Inner Join Test Time

Apache Spark

Row Count: 40000000 - Partitions: 1000 - Inner Join Test Time

Apache Spark

Row Count: 40000000 - Partitions: 2000 - SHA-512 Benchmark Time

Apache Spark

Row Count: 40000000 - Partitions: 500 - SHA-512 Benchmark Time

Apache Spark

Row Count: 10000000 - Partitions: 100 - Broadcast Inner Join Test Time

Apache Spark

Row Count: 40000000 - Partitions: 100 - Inner Join Test Time

Apache Spark

Row Count: 40000000 - Partitions: 100 - Broadcast Inner Join Test Time

Apache Spark

Row Count: 40000000 - Partitions: 100 - SHA-512 Benchmark Time

Apache Spark

Row Count: 20000000 - Partitions: 100 - SHA-512 Benchmark Time

Apache Spark

Row Count: 10000000 - Partitions: 100 - Inner Join Test Time

Apache Spark

Row Count: 20000000 - Partitions: 1000 - SHA-512 Benchmark Time

Apache Spark

Row Count: 40000000 - Partitions: 1000 - SHA-512 Benchmark Time

Apache Spark

Row Count: 10000000 - Partitions: 2000 - Repartition Test Time

Apache Spark

Row Count: 10000000 - Partitions: 100 - Repartition Test Time

Apache Spark

Row Count: 10000000 - Partitions: 2000 - Broadcast Inner Join Test Time

Apache Spark

Row Count: 20000000 - Partitions: 500 - SHA-512 Benchmark Time

Apache Spark

Row Count: 10000000 - Partitions: 500 - Repartition Test Time

Apache Spark

Row Count: 10000000 - Partitions: 2000 - Inner Join Test Time

Apache Spark

Row Count: 10000000 - Partitions: 1000 - Broadcast Inner Join Test Time

Apache Spark

Row Count: 20000000 - Partitions: 2000 - SHA-512 Benchmark Time

Apache Spark

Row Count: 10000000 - Partitions: 1000 - Repartition Test Time

Apache Spark

Row Count: 10000000 - Partitions: 1000 - Inner Join Test Time

Apache Spark

Row Count: 40000000 - Partitions: 2000 - Group By Test Time

Apache Spark

Row Count: 1000000 - Partitions: 1000 - Broadcast Inner Join Test Time

Apache Spark

Row Count: 10000000 - Partitions: 2000 - SHA-512 Benchmark Time

Apache Spark

Row Count: 10000000 - Partitions: 1000 - SHA-512 Benchmark Time

Apache Spark

Row Count: 10000000 - Partitions: 500 - SHA-512 Benchmark Time

Apache Spark

Row Count: 40000000 - Partitions: 500 - Group By Test Time

Apache Spark

Row Count: 40000000 - Partitions: 100 - Group By Test Time

Apache Spark

Row Count: 40000000 - Partitions: 1000 - Group By Test Time

Apache Spark

Row Count: 10000000 - Partitions: 100 - SHA-512 Benchmark Time

Apache Spark

Row Count: 1000000 - Partitions: 500 - Broadcast Inner Join Test Time

SVT-AV1

Encoder Mode: Preset 8 - Input: Bosphorus 4K

SVT-AV1

Encoder Mode: Preset 10 - Input: Bosphorus 4K

SVT-AV1

Encoder Mode: Preset 12 - Input: Bosphorus 4K

Apache Spark

Row Count: 20000000 - Partitions: 1000 - Group By Test Time

Apache Spark

Row Count: 20000000 - Partitions: 2000 - Group By Test Time

Apache Spark

Row Count: 1000000 - Partitions: 500 - Inner Join Test Time

Apache Spark

Row Count: 1000000 - Partitions: 1000 - Inner Join Test Time

Apache Spark

Row Count: 20000000 - Partitions: 100 - Group By Test Time

Apache Spark

Row Count: 1000000 - Partitions: 1000 - Repartition Test Time

Apache Spark

Row Count: 20000000 - Partitions: 500 - Group By Test Time

Apache Spark

Row Count: 1000000 - Partitions: 500 - Repartition Test Time

Apache Spark

Row Count: 10000000 - Partitions: 1000 - Group By Test Time

Apache Spark

Row Count: 10000000 - Partitions: 2000 - Group By Test Time

Apache Spark

Row Count: 1000000 - Partitions: 2000 - Inner Join Test Time

Apache Spark

Row Count: 10000000 - Partitions: 100 - Group By Test Time

Apache Spark

Row Count: 1000000 - Partitions: 2000 - Repartition Test Time

Apache Spark

Row Count: 10000000 - Partitions: 500 - Group By Test Time

Apache Spark

Row Count: 1000000 - Partitions: 100 - Inner Join Test Time

Apache Spark

Row Count: 1000000 - Partitions: 100 - Repartition Test Time

Apache Spark

Row Count: 1000000 - Partitions: 2000 - Broadcast Inner Join Test Time

Apache Spark

Row Count: 1000000 - Partitions: 100 - Broadcast Inner Join Test Time

SVT-AV1

Encoder Mode: Preset 10 - Input: Bosphorus 1080p

SVT-AV1

Encoder Mode: Preset 12 - Input: Bosphorus 1080p

Apache Spark

Row Count: 1000000 - Partitions: 2000 - SHA-512 Benchmark Time

Apache Spark

Row Count: 1000000 - Partitions: 1000 - SHA-512 Benchmark Time

Apache Spark

Row Count: 1000000 - Partitions: 1000 - Group By Test Time

Apache Spark

Row Count: 1000000 - Partitions: 500 - SHA-512 Benchmark Time

Apache Spark

Row Count: 1000000 - Partitions: 100 - Group By Test Time

SVT-AV1

Encoder Mode: Preset 8 - Input: Bosphorus 1080p

Apache Spark

Row Count: 1000000 - Partitions: 500 - Group By Test Time

Apache Spark

Row Count: 1000000 - Partitions: 100 - SHA-512 Benchmark Time

Apache Spark

Row Count: 1000000 - Partitions: 2000 - Group By Test Time

Apache Spark

Row Count: 40000000 - Partitions: 500 - Calculate Pi Benchmark Using Dataframe

Apache Spark

Row Count: 1000000 - Partitions: 100 - Calculate Pi Benchmark

Apache Spark

Row Count: 1000000 - Partitions: 500 - Calculate Pi Benchmark Using Dataframe

Apache Spark

Row Count: 10000000 - Partitions: 500 - Calculate Pi Benchmark Using Dataframe

Apache Spark

Row Count: 20000000 - Partitions: 100 - Calculate Pi Benchmark Using Dataframe

Apache Spark

Row Count: 40000000 - Partitions: 2000 - Calculate Pi Benchmark

Apache Spark

Row Count: 10000000 - Partitions: 1000 - Calculate Pi Benchmark

Apache Spark

Row Count: 40000000 - Partitions: 2000 - Calculate Pi Benchmark Using Dataframe

Apache Spark

Row Count: 20000000 - Partitions: 1000 - Calculate Pi Benchmark Using Dataframe

Apache Spark

Row Count: 20000000 - Partitions: 500 - Calculate Pi Benchmark Using Dataframe

SVT-AV1

Encoder Mode: Preset 4 - Input: Bosphorus 1080p

Apache Spark

Row Count: 20000000 - Partitions: 500 - Calculate Pi Benchmark

Apache Spark

Row Count: 40000000 - Partitions: 1000 - Calculate Pi Benchmark

Apache Spark

Row Count: 40000000 - Partitions: 1000 - Calculate Pi Benchmark Using Dataframe

Apache Spark

Row Count: 10000000 - Partitions: 500 - Calculate Pi Benchmark

Apache Spark

Row Count: 1000000 - Partitions: 500 - Calculate Pi Benchmark

Apache Spark

Row Count: 10000000 - Partitions: 2000 - Calculate Pi Benchmark Using Dataframe

Apache Spark

Row Count: 40000000 - Partitions: 100 - Calculate Pi Benchmark

Apache Spark

Row Count: 10000000 - Partitions: 100 - Calculate Pi Benchmark Using Dataframe

Apache Spark

Row Count: 20000000 - Partitions: 2000 - Calculate Pi Benchmark Using Dataframe

Apache Spark

Row Count: 1000000 - Partitions: 2000 - Calculate Pi Benchmark

Apache Spark

Row Count: 20000000 - Partitions: 2000 - Calculate Pi Benchmark

Apache Spark

Row Count: 1000000 - Partitions: 1000 - Calculate Pi Benchmark

Apache Spark

Row Count: 10000000 - Partitions: 2000 - Calculate Pi Benchmark

Apache Spark

Row Count: 40000000 - Partitions: 500 - Calculate Pi Benchmark

SVT-AV1

Encoder Mode: Preset 4 - Input: Bosphorus 4K

Apache Spark

Row Count: 20000000 - Partitions: 1000 - Calculate Pi Benchmark

Apache Spark

Row Count: 10000000 - Partitions: 100 - Calculate Pi Benchmark

Apache Spark

Row Count: 1000000 - Partitions: 2000 - Calculate Pi Benchmark Using Dataframe

Apache Spark

Row Count: 1000000 - Partitions: 100 - Calculate Pi Benchmark Using Dataframe

Apache Spark

Row Count: 10000000 - Partitions: 1000 - Calculate Pi Benchmark Using Dataframe

Apache Spark

Row Count: 20000000 - Partitions: 100 - Calculate Pi Benchmark

Apache Spark

Row Count: 1000000 - Partitions: 1000 - Calculate Pi Benchmark Using Dataframe

Apache Spark

Row Count: 40000000 - Partitions: 100 - Calculate Pi Benchmark Using Dataframe

Phoronix Test Suite v10.8.4