Llama.cpp NVIDIA GeForce RTX 5090 Benchmarks by Michael Larabel for a future article on Phoronix. RTX 3090: Processor: Intel Core Ultra 9 285K @ 5.10GHz (24 Cores), Motherboard: ASUS ROG MAXIMUS Z890 HERO (1203 BIOS), Chipset: Intel Device ae7f, Memory: 2 x 16GB DDR5-6400MT/s Micron CP16G64C38U5B.M8D1, Disk: 4001GB Western Digital WD_BLACK SN850X 4000GB + 1000GB Western Digital WDS100T1X0E-00AFY0, Graphics: ASUS NVIDIA GeForce RTX 3090 24GB, Audio: Intel Device 7f50, Monitor: ASUS VP28U, Network: Realtek Device 8126 + Intel I226-V + Intel Wi-Fi 7 OS: Ubuntu 24.10, Kernel: 6.11.0-13-generic (x86_64), Desktop: GNOME Shell 47.0, Display Server: X Server 1.21.1.13, Display Driver: NVIDIA 570.86.10, OpenGL: 4.6.0, OpenCL: OpenCL 3.0 CUDA 12.8.51 + OpenCL 3.0, Compiler: GCC 14.2.0 + CUDA 12.8, File-System: ext4, Screen Resolution: 3840x2160 RTX 4070: Processor: Intel Core Ultra 9 285K @ 5.10GHz (24 Cores), Motherboard: ASUS ROG MAXIMUS Z890 HERO (1203 BIOS), Chipset: Intel Device ae7f, Memory: 2 x 16GB DDR5-6400MT/s Micron CP16G64C38U5B.M8D1, Disk: 4001GB Western Digital WD_BLACK SN850X 4000GB + 1000GB Western Digital WDS100T1X0E-00AFY0, Graphics: ASUS NVIDIA GeForce RTX 4070 12GB, Audio: Intel Device 7f50, Monitor: ASUS VP28U, Network: Realtek Device 8126 + Intel I226-V + Intel Wi-Fi 7 OS: Ubuntu 24.10, Kernel: 6.11.0-13-generic (x86_64), Desktop: GNOME Shell 47.0, Display Server: X Server 1.21.1.13, Display Driver: NVIDIA 570.86.10, OpenGL: 4.6.0, OpenCL: OpenCL 3.0 CUDA 12.8.51 + OpenCL 3.0, Compiler: GCC 14.2.0 + CUDA 12.8, File-System: ext4, Screen Resolution: 3840x2160 RTX 4070 SUPER: Processor: Intel Core Ultra 9 285K @ 5.10GHz (24 Cores), Motherboard: ASUS ROG MAXIMUS Z890 HERO (1203 BIOS), Chipset: Intel Device ae7f, Memory: 2 x 16GB DDR5-6400MT/s Micron CP16G64C38U5B.M8D1, Disk: 4001GB Western Digital WD_BLACK SN850X 4000GB + 1000GB Western Digital WDS100T1X0E-00AFY0, Graphics: ASUS NVIDIA GeForce RTX 4070 SUPER 12GB, Audio: Intel Device 7f50, Monitor: ASUS VP28U, Network: Realtek Device 8126 + Intel I226-V + Intel Wi-Fi 7 OS: Ubuntu 24.10, Kernel: 6.11.0-13-generic (x86_64), Desktop: GNOME Shell 47.0, Display Server: X Server 1.21.1.13, Display Driver: NVIDIA 570.86.10, OpenGL: 4.6.0, OpenCL: OpenCL 3.0 CUDA 12.8.51 + OpenCL 3.0, Compiler: GCC 14.2.0 + CUDA 12.8, File-System: ext4, Screen Resolution: 3840x2160 RTX 4070 Ti SUPER: Processor: Intel Core Ultra 9 285K @ 5.10GHz (24 Cores), Motherboard: ASUS ROG MAXIMUS Z890 HERO (1203 BIOS), Chipset: Intel Device ae7f, Memory: 2 x 16GB DDR5-6400MT/s Micron CP16G64C38U5B.M8D1, Disk: 4001GB Western Digital WD_BLACK SN850X 4000GB + 1000GB Western Digital WDS100T1X0E-00AFY0, Graphics: ASUS NVIDIA GeForce RTX 4070 Ti SUPER 16GB, Audio: Intel Device 7f50, Monitor: ASUS VP28U, Network: Realtek Device 8126 + Intel I226-V + Intel Wi-Fi 7 OS: Ubuntu 24.10, Kernel: 6.11.0-13-generic (x86_64), Desktop: GNOME Shell 47.0, Display Server: X Server 1.21.1.13, Display Driver: NVIDIA 570.86.10, OpenGL: 4.6.0, OpenCL: OpenCL 3.0 CUDA 12.8.51 + OpenCL 3.0, Compiler: GCC 14.2.0 + CUDA 12.8, File-System: ext4, Screen Resolution: 3840x2160 RTX 4080: Processor: Intel Core Ultra 9 285K @ 5.10GHz (24 Cores), Motherboard: ASUS ROG MAXIMUS Z890 HERO (1203 BIOS), Chipset: Intel Device ae7f, Memory: 2 x 16GB DDR5-6400MT/s Micron CP16G64C38U5B.M8D1, Disk: 4001GB Western Digital WD_BLACK SN850X 4000GB + 1000GB Western Digital WDS100T1X0E-00AFY0, Graphics: ASUS NVIDIA GeForce RTX 4080 16GB, Audio: Intel Device 7f50, Monitor: ASUS VP28U, Network: Realtek Device 8126 + Intel I226-V + Intel Wi-Fi 7 OS: Ubuntu 24.10, Kernel: 6.11.0-13-generic (x86_64), Desktop: GNOME Shell 47.0, Display Server: X Server 1.21.1.13, Display Driver: NVIDIA 570.86.10, OpenGL: 4.6.0, OpenCL: OpenCL 3.0 CUDA 12.8.51 + OpenCL 3.0, Compiler: GCC 14.2.0 + CUDA 12.8, File-System: ext4, Screen Resolution: 3840x2160 RTX 4080 SUPER: Processor: Intel Core Ultra 9 285K @ 5.10GHz (24 Cores), Motherboard: ASUS ROG MAXIMUS Z890 HERO (1203 BIOS), Chipset: Intel Device ae7f, Memory: 2 x 16GB DDR5-6400MT/s Micron CP16G64C38U5B.M8D1, Disk: 4001GB Western Digital WD_BLACK SN850X 4000GB + 1000GB Western Digital WDS100T1X0E-00AFY0, Graphics: ASUS NVIDIA GeForce RTX 4080 SUPER 16GB, Audio: Intel Device 7f50, Monitor: ASUS VP28U, Network: Realtek Device 8126 + Intel I226-V + Intel Wi-Fi 7 OS: Ubuntu 24.10, Kernel: 6.11.0-13-generic (x86_64), Desktop: GNOME Shell 47.0, Display Server: X Server 1.21.1.13, Display Driver: NVIDIA 570.86.10, OpenGL: 4.6.0, OpenCL: OpenCL 3.0 CUDA 12.8.51 + OpenCL 3.0, Compiler: GCC 14.2.0 + CUDA 12.8, File-System: ext4, Screen Resolution: 3840x2160 RTX 4090: Processor: Intel Core Ultra 9 285K @ 5.10GHz (24 Cores), Motherboard: ASUS ROG MAXIMUS Z890 HERO (1203 BIOS), Chipset: Intel Device ae7f, Memory: 2 x 16GB DDR5-6400MT/s Micron CP16G64C38U5B.M8D1, Disk: 4001GB Western Digital WD_BLACK SN850X 4000GB + 1000GB Western Digital WDS100T1X0E-00AFY0, Graphics: ASUS NVIDIA GeForce RTX 4090 24GB, Audio: Intel Device 7f50, Monitor: ASUS VP28U, Network: Realtek Device 8126 + Intel I226-V + Intel Wi-Fi 7 OS: Ubuntu 24.10, Kernel: 6.11.0-13-generic (x86_64), Desktop: GNOME Shell 47.0, Display Server: X Server 1.21.1.13, Display Driver: NVIDIA 570.86.10, OpenGL: 4.6.0, OpenCL: OpenCL 3.0 CUDA 12.8.51 + OpenCL 3.0, Compiler: GCC 14.2.0 + CUDA 12.8, File-System: ext4, Screen Resolution: 3840x2160 RTX 5090: Processor: Intel Core Ultra 9 285K @ 5.10GHz (24 Cores), Motherboard: ASUS ROG MAXIMUS Z890 HERO (1203 BIOS), Chipset: Intel Device ae7f, Memory: 2 x 16GB DDR5-6400MT/s Micron CP16G64C38U5B.M8D1, Disk: 4001GB Western Digital WD_BLACK SN850X 4000GB + 1000GB Western Digital WDS100T1X0E-00AFY0, Graphics: ASUS NVIDIA GeForce RTX 5090 32GB, Audio: Intel Device 7f50, Monitor: ASUS VP28U, Network: Realtek Device 8126 + Intel I226-V + Intel Wi-Fi 7 OS: Ubuntu 24.10, Kernel: 6.11.0-13-generic (x86_64), Desktop: GNOME Shell 47.0, Display Server: X Server 1.21.1.13, Display Driver: NVIDIA 570.86.10, OpenGL: 4.6.0, OpenCL: OpenCL 3.0 CUDA 12.8.51 + OpenCL 3.0, Compiler: GCC 14.2.0 + CUDA 12.8, File-System: ext4, Screen Resolution: 3840x2160 Llama.cpp b4397 Backend: NVIDIA CUDA - Model: Llama-3.1-Tulu-3-8B-Q8_0 - Test: Text Generation 128 Tokens Per Second > Higher Is Better RTX 3090 .......... 90.88 |============================= RTX 4070 .......... 54.47 |================= RTX 4070 SUPER .... 54.59 |================== RTX 4070 Ti SUPER . 70.83 |======================= RTX 4080 .......... 74.56 |======================== RTX 4080 SUPER .... 77.08 |========================= RTX 4090 .......... 100.51 |================================ RTX 5090 .......... 158.77 |=================================================== Llama.cpp b4397 Backend: NVIDIA CUDA - Model: Llama-3.1-Tulu-3-8B-Q8_0 - Test: Prompt Processing 512 Tokens Per Second > Higher Is Better RTX 3090 .......... 5030.78 |=================== RTX 4070 .......... 4502.64 |================= RTX 4070 SUPER .... 5419.57 |===================== RTX 4070 Ti SUPER . 6328.82 |======================== RTX 4080 .......... 7606.74 |============================= RTX 4080 SUPER .... 7782.93 |============================== RTX 4090 .......... 11675.08 |============================================= RTX 5090 .......... 12780.54 |================================================= Llama.cpp b4397 Backend: NVIDIA CUDA - Model: Llama-3.1-Tulu-3-8B-Q8_0 - Test: Prompt Processing 1024 Tokens Per Second > Higher Is Better RTX 3090 .......... 4784.57 |=================== RTX 4070 .......... 4198.29 |================= RTX 4070 SUPER .... 4997.77 |==================== RTX 4070 Ti SUPER . 5889.85 |======================= RTX 4080 .......... 7034.85 |============================ RTX 4080 SUPER .... 7201.82 |============================ RTX 4090 .......... 10889.08 |=========================================== RTX 5090 .......... 12385.34 |================================================= Llama.cpp b4397 Backend: NVIDIA CUDA - Model: Llama-3.1-Tulu-3-8B-Q8_0 - Test: Prompt Processing 2048 Tokens Per Second > Higher Is Better RTX 3090 .......... 4437.57 |=================== RTX 4070 .......... 3798.06 |================= RTX 4070 SUPER .... 4436.29 |=================== RTX 4070 Ti SUPER . 5301.35 |======================= RTX 4080 .......... 6228.28 |=========================== RTX 4080 SUPER .... 6380.34 |============================ RTX 4090 .......... 9573.68 |========================================== RTX 5090 .......... 11211.19 |================================================= Llama.cpp b4397 Backend: NVIDIA CUDA - Model: Mistral-7B-Instruct-v0.3-Q8_0 - Test: Text Generation 128 Tokens Per Second > Higher Is Better RTX 3090 .......... 95.29 |============================= RTX 4070 .......... 57.36 |================== RTX 4070 SUPER .... 57.48 |================== RTX 4070 Ti SUPER . 74.55 |======================= RTX 4080 .......... 78.50 |======================== RTX 4080 SUPER .... 81.17 |========================= RTX 4090 .......... 105.68 |================================ RTX 5090 .......... 166.12 |=================================================== Llama.cpp b4397 Backend: NVIDIA CUDA - Model: Mistral-7B-Instruct-v0.3-Q8_0 - Test: Prompt Processing 512 Tokens Per Second > Higher Is Better RTX 3090 .......... 5046.02 |=================== RTX 4070 .......... 4533.01 |================= RTX 4070 SUPER .... 5475.92 |===================== RTX 4070 Ti SUPER . 6391.67 |======================== RTX 4080 .......... 7685.77 |============================= RTX 4080 SUPER .... 7862.60 |============================== RTX 4090 .......... 11826.41 |============================================= RTX 5090 .......... 12884.46 |================================================= Llama.cpp b4397 Backend: NVIDIA CUDA - Model: Mistral-7B-Instruct-v0.3-Q8_0 - Test: Prompt Processing 1024 Tokens Per Second > Higher Is Better RTX 3090 .......... 4783.19 |=================== RTX 4070 .......... 4212.67 |================= RTX 4070 SUPER .... 5024.48 |==================== RTX 4070 Ti SUPER . 5924.06 |======================= RTX 4080 .......... 7067.75 |============================ RTX 4080 SUPER .... 7236.30 |============================ RTX 4090 .......... 10961.06 |=========================================== RTX 5090 .......... 12462.03 |================================================= Llama.cpp b4397 Backend: NVIDIA CUDA - Model: Mistral-7B-Instruct-v0.3-Q8_0 - Test: Prompt Processing 2048 Tokens Per Second > Higher Is Better RTX 3090 .......... 4437.49 |=================== RTX 4070 .......... 3803.46 |================= RTX 4070 SUPER .... 4452.75 |=================== RTX 4070 Ti SUPER . 5313.40 |======================= RTX 4080 .......... 6246.69 |=========================== RTX 4080 SUPER .... 6410.73 |============================ RTX 4090 .......... 9593.96 |========================================== RTX 5090 .......... 11240.15 |================================================= Llama.cpp b4397 GPU Temperature Monitor Celsius < Lower Is Better RTX 3090 .......... MIN: 34.0 AVG: 59.6 MAX: 65.0 RTX 4070 .......... MIN: 35.0 AVG: 54.1 MAX: 57.0 RTX 4070 SUPER .... MIN: 33.0 AVG: 57.4 MAX: 60.0 RTX 4070 Ti SUPER . MIN: 25.0 AVG: 53.8 MAX: 58.0 RTX 4080 .......... MIN: 32.0 AVG: 50.2 MAX: 53.0 RTX 4080 SUPER .... MIN: 35.0 AVG: 50.7 MAX: 53.0 RTX 4090 .......... MIN: 32.0 AVG: 51.6 MAX: 56.0 RTX 5090 .......... MIN: 37.0 AVG: 61.9 MAX: 69.0 Llama.cpp b4397 GPU Temperature Monitor Celsius < Lower Is Better RTX 3090 .......... MIN: 48.0 AVG: 56.7 MAX: 61.0 RTX 4070 .......... MIN: 42.0 AVG: 53.6 MAX: 61.0 RTX 4070 SUPER .... MIN: 44.0 AVG: 56.0 MAX: 64.0 RTX 4070 Ti SUPER . MIN: 42.0 AVG: 51.9 MAX: 60.0 RTX 4080 .......... MIN: 39.0 AVG: 49.5 MAX: 59.0 RTX 4080 SUPER .... MIN: 39.0 AVG: 49.6 MAX: 60.0 RTX 4090 .......... MIN: 43.0 AVG: 51.1 MAX: 62.0 RTX 5090 .......... MIN: 50.0 AVG: 56.9 MAX: 66.0 Llama.cpp b4397 GPU Temperature Monitor Celsius < Lower Is Better RTX 3090 .......... MIN: 51.0 AVG: 60.1 MAX: 64.0 RTX 4070 .......... MIN: 40.0 AVG: 56.0 MAX: 62.0 RTX 4070 SUPER .... MIN: 41.0 AVG: 58.5 MAX: 66.0 RTX 4070 Ti SUPER . MIN: 39.0 AVG: 54.5 MAX: 62.0 RTX 4080 .......... MIN: 37.0 AVG: 52.5 MAX: 61.0 RTX 4080 SUPER .... MIN: 37.0 AVG: 53.0 MAX: 61.0 RTX 4090 .......... MIN: 40.0 AVG: 53.8 MAX: 63.0 RTX 5090 .......... MIN: 46.0 AVG: 58.7 MAX: 68.0 Llama.cpp b4397 GPU Temperature Monitor Celsius < Lower Is Better RTX 3090 .......... MIN: 53.0 AVG: 63.3 MAX: 66.0 RTX 4070 .......... MIN: 41.0 AVG: 60.4 MAX: 65.0 RTX 4070 SUPER .... MIN: 42.0 AVG: 63.0 MAX: 69.0 RTX 4070 Ti SUPER . MIN: 40.0 AVG: 58.3 MAX: 64.0 RTX 4080 .......... MIN: 38.0 AVG: 56.6 MAX: 63.0 RTX 4080 SUPER .... MIN: 38.0 AVG: 56.9 MAX: 62.0 RTX 4090 .......... MIN: 41.0 AVG: 58.4 MAX: 66.0 RTX 5090 .......... MIN: 48.0 AVG: 63.9 MAX: 71.0 Llama.cpp b4397 GPU Temperature Monitor Celsius < Lower Is Better RTX 3090 .......... MIN: 52.0 AVG: 62.4 MAX: 66.0 RTX 4070 .......... MIN: 36.0 AVG: 54.1 MAX: 56.0 RTX 4070 SUPER .... MIN: 37.0 AVG: 57.8 MAX: 60.0 RTX 4070 Ti SUPER . MIN: 36.0 AVG: 55.5 MAX: 58.0 RTX 4080 .......... MIN: 34.0 AVG: 50.1 MAX: 53.0 RTX 4080 SUPER .... MIN: 34.0 AVG: 50.1 MAX: 52.0 RTX 4090 .......... MIN: 35.0 AVG: 52.4 MAX: 56.0 RTX 5090 .......... MIN: 41.0 AVG: 62.3 MAX: 69.0 Llama.cpp b4397 GPU Temperature Monitor Celsius < Lower Is Better RTX 3090 .......... MIN: 49.0 AVG: 57.5 MAX: 62.0 RTX 4070 .......... MIN: 42.0 AVG: 53.8 MAX: 61.0 RTX 4070 SUPER .... MIN: 44.0 AVG: 56.2 MAX: 65.0 RTX 4070 Ti SUPER . MIN: 42.0 AVG: 52.1 MAX: 60.0 RTX 4080 .......... MIN: 39.0 AVG: 49.6 MAX: 59.0 RTX 4080 SUPER .... MIN: 39.0 AVG: 50.2 MAX: 60.0 RTX 4090 .......... MIN: 43.0 AVG: 51.5 MAX: 62.0 RTX 5090 .......... MIN: 50.0 AVG: 57.2 MAX: 66.0 Llama.cpp b4397 GPU Temperature Monitor Celsius < Lower Is Better RTX 3090 .......... MIN: 52.0 AVG: 60.7 MAX: 65.0 RTX 4070 .......... MIN: 40.0 AVG: 56.2 MAX: 62.0 RTX 4070 SUPER .... MIN: 41.0 AVG: 58.8 MAX: 66.0 RTX 4070 Ti SUPER . MIN: 39.0 AVG: 54.9 MAX: 62.0 RTX 4080 .......... MIN: 37.0 AVG: 53.0 MAX: 61.0 RTX 4080 SUPER .... MIN: 37.0 AVG: 53.1 MAX: 62.0 RTX 4090 .......... MIN: 40.0 AVG: 54.3 MAX: 63.0 RTX 5090 .......... MIN: 46.0 AVG: 58.7 MAX: 68.0 Llama.cpp b4397 GPU Temperature Monitor Celsius < Lower Is Better RTX 3090 .......... MIN: 53.0 AVG: 63.6 MAX: 66.0 RTX 4070 .......... MIN: 41.0 AVG: 60.3 MAX: 65.0 RTX 4070 SUPER .... MIN: 42.0 AVG: 63.2 MAX: 68.0 RTX 4070 Ti SUPER . MIN: 40.0 AVG: 58.5 MAX: 63.0 RTX 4080 .......... MIN: 38.0 AVG: 57.0 MAX: 62.0 RTX 4080 SUPER .... MIN: 38.0 AVG: 57.1 MAX: 62.0 RTX 4090 .......... MIN: 41.0 AVG: 58.5 MAX: 65.0 RTX 5090 .......... MIN: 48.0 AVG: 64.1 MAX: 71.0 Llama.cpp b4397 GPU Power Consumption Monitor Watts < Lower Is Better RTX 3090 .......... MIN: 25 AVG: 327 MAX: 343 RTX 4070 .......... MIN: 6 AVG: 154 MAX: 160 RTX 4070 SUPER .... MIN: 8 AVG: 164 MAX: 170 RTX 4070 Ti SUPER . MIN: 5 AVG: 214 MAX: 226 RTX 4080 .......... MIN: 5 AVG: 204 MAX: 214 RTX 4080 SUPER .... MIN: 5 AVG: 200 MAX: 210 RTX 4090 .......... MIN: 14 AVG: 257 MAX: 273 RTX 5090 .......... MIN: 17 AVG: 415 MAX: 458 Llama.cpp b4397 GPU Power Consumption Monitor Watts < Lower Is Better RTX 3090 .......... MIN: 22 AVG: 238 MAX: 341 RTX 4070 .......... MIN: 8 AVG: 133 MAX: 200 RTX 4070 SUPER .... MIN: 7 AVG: 137 MAX: 221 RTX 4070 Ti SUPER . MIN: 14 AVG: 170 MAX: 286 RTX 4080 .......... MIN: 9 AVG: 175 MAX: 319 RTX 4080 SUPER .... MIN: 5 AVG: 168 MAX: 320 RTX 4090 .......... MIN: 15 AVG: 193 MAX: 435 RTX 5090 .......... MIN: 28 AVG: 257 MAX: 561 Llama.cpp b4397 GPU Power Consumption Monitor Watts < Lower Is Better RTX 3090 .......... MIN: 24 AVG: 276 MAX: 345 RTX 4070 .......... MIN: 8 AVG: 159 MAX: 200 RTX 4070 SUPER .... MIN: 8 AVG: 169 MAX: 220 RTX 4070 Ti SUPER . MIN: 14 AVG: 212 MAX: 285 RTX 4080 .......... MIN: 4 AVG: 223 MAX: 317 RTX 4080 SUPER .... MIN: 5 AVG: 217 MAX: 314 RTX 4090 .......... MIN: 14 AVG: 265 MAX: 430 RTX 5090 .......... MIN: 19 AVG: 346 MAX: 570 Llama.cpp b4397 GPU Power Consumption Monitor Watts < Lower Is Better RTX 3090 .......... MIN: 23 AVG: 306 MAX: 344 RTX 4070 .......... MIN: 8 AVG: 178 MAX: 200 RTX 4070 SUPER .... MIN: 7 AVG: 193 MAX: 221 RTX 4070 Ti SUPER . MIN: 14 AVG: 245 MAX: 286 RTX 4080 .......... MIN: 9 AVG: 261 MAX: 312 RTX 4080 SUPER .... MIN: 5 AVG: 257 MAX: 311 RTX 4090 .......... MIN: 14 AVG: 321 MAX: 418 RTX 5090 .......... MIN: 19 AVG: 433 MAX: 575 Llama.cpp b4397 GPU Power Consumption Monitor Watts < Lower Is Better RTX 3090 .......... MIN: 24 AVG: 329 MAX: 344 RTX 4070 .......... MIN: 7 AVG: 154 MAX: 159 RTX 4070 SUPER .... MIN: 7 AVG: 164 MAX: 170 RTX 4070 Ti SUPER . MIN: 13 AVG: 216 MAX: 226 RTX 4080 .......... MIN: 7 AVG: 203 MAX: 212 RTX 4080 SUPER .... MIN: 5 AVG: 199 MAX: 209 RTX 4090 .......... MIN: 14 AVG: 258 MAX: 274 RTX 5090 .......... MIN: 17 AVG: 414 MAX: 455 Llama.cpp b4397 GPU Power Consumption Monitor Watts < Lower Is Better RTX 3090 .......... MIN: 21 AVG: 240 MAX: 345 RTX 4070 .......... MIN: 8 AVG: 134 MAX: 202 RTX 4070 SUPER .... MIN: 8 AVG: 139 MAX: 220 RTX 4070 Ti SUPER . MIN: 14 AVG: 172 MAX: 285 RTX 4080 .......... MIN: 8 AVG: 175 MAX: 320 RTX 4080 SUPER .... MIN: 5 AVG: 169 MAX: 319 RTX 4090 .......... MIN: 14 AVG: 196 MAX: 438 RTX 5090 .......... MIN: 27 AVG: 258 MAX: 561 Llama.cpp b4397 GPU Power Consumption Monitor Watts < Lower Is Better RTX 3090 .......... MIN: 25 AVG: 278 MAX: 342 RTX 4070 .......... MIN: 8 AVG: 160 MAX: 200 RTX 4070 SUPER .... MIN: 8 AVG: 170 MAX: 220 RTX 4070 Ti SUPER . MIN: 14 AVG: 214 MAX: 285 RTX 4080 .......... MIN: 4 AVG: 226 MAX: 317 RTX 4080 SUPER .... MIN: 5 AVG: 219 MAX: 314 RTX 4090 .......... MIN: 15 AVG: 265 MAX: 432 RTX 5090 .......... MIN: 19 AVG: 350 MAX: 571 Llama.cpp b4397 GPU Power Consumption Monitor Watts < Lower Is Better RTX 3090 .......... MIN: 24 AVG: 308 MAX: 344 RTX 4070 .......... MIN: 8 AVG: 179 MAX: 200 RTX 4070 SUPER .... MIN: 8 AVG: 193 MAX: 221 RTX 4070 Ti SUPER . MIN: 14 AVG: 247 MAX: 285 RTX 4080 .......... MIN: 6 AVG: 262 MAX: 313 RTX 4080 SUPER .... MIN: 5 AVG: 258 MAX: 310 RTX 4090 .......... MIN: 14 AVG: 324 MAX: 418 RTX 5090 .......... MIN: 21 AVG: 437 MAX: 575