MBP M1 Max Machine Learning, sys76-kudu-ML Apple M1 Max testing with a Apple MacBook Pro and Apple M1 Max on macOS 12.1 via the Phoronix Test Suite. sys76-kudu-ML: AMD Ryzen 9 5900HX testing with a System76 Kudu (1.07.09RSA1 BIOS) and AMD Cezanne on Pop 21.10 via the Phoronix Test Suite. MBP M1 Max Machine Learning: Processor: Apple M1 Max (10 Cores), Motherboard: Apple MacBook Pro, Memory: 64GB, Disk: 1859GB, Graphics: Apple M1 Max, Monitor: Color LCD OS: macOS 12.1, Kernel: 21.2.0 (arm64), OpenCL: OpenCL 1.2 (Nov 13 2021 00:45:09), Compiler: GCC 13.0.0 + Clang 13.0.0, File-System: APFS, Screen Resolution: 3456x2234 ML Tests: Processor: AMD Ryzen 9 5900HX @ 3.30GHz (8 Cores / 16 Threads), Motherboard: System76 Kudu (1.07.09RSA1 BIOS), Chipset: AMD Renoir/Cezanne, Memory: 16GB, Disk: Samsung SSD 970 EVO Plus 500GB, Graphics: AMD Cezanne (2100/400MHz), Audio: AMD Renoir Radeon HD Audio, Network: Realtek RTL8125 2.5GbE + Intel Wi-Fi 6 AX200 OS: Pop 21.10, Kernel: 5.15.15-76051515-generic (x86_64), Desktop: GNOME Shell 40.5, Display Server: X Server 1.20.13, OpenGL: 4.6 Mesa 21.2.2 (LLVM 12.0.1), Vulkan: 1.2.182, Compiler: GCC 11.2.0, File-System: ext4, Screen Resolution: 1920x1080 PlaidML FP16: No - Mode: Inference - Network: VGG16 - Device: CPU FPS > Higher Is Better ML Tests . 12.47 |============================================================= PlaidML FP16: No - Mode: Inference - Network: ResNet 50 - Device: CPU FPS > Higher Is Better ML Tests . 6.88 |============================================================== OpenVINO 2021.1 Model: Face Detection 0106 FP16 - Device: CPU FPS > Higher Is Better OpenVINO 2021.1 Model: Face Detection 0106 FP32 - Device: CPU FPS > Higher Is Better OpenVINO 2021.1 Model: Person Detection 0106 FP16 - Device: CPU FPS > Higher Is Better OpenVINO 2021.1 Model: Person Detection 0106 FP32 - Device: CPU FPS > Higher Is Better OpenVINO 2021.1 Model: Face Detection 0106 FP16 - Device: Intel GPU FPS > Higher Is Better OpenVINO 2021.1 Model: Face Detection 0106 FP32 - Device: Intel GPU FPS > Higher Is Better OpenVINO 2021.1 Model: Person Detection 0106 FP16 - Device: Intel GPU FPS > Higher Is Better OpenVINO 2021.1 Model: Person Detection 0106 FP32 - Device: Intel GPU FPS > Higher Is Better OpenVINO 2021.1 Model: Age Gender Recognition Retail 0013 FP16 - Device: CPU FPS > Higher Is Better OpenVINO 2021.1 Model: Age Gender Recognition Retail 0013 FP32 - Device: CPU FPS > Higher Is Better OpenVINO 2021.1 Model: Age Gender Recognition Retail 0013 FP16 - Device: Intel GPU FPS > Higher Is Better OpenVINO 2021.1 Model: Age Gender Recognition Retail 0013 FP32 - Device: Intel GPU FPS > Higher Is Better ONNX Runtime 1.10 Model: yolov4 - Device: CPU Inferences Per Minute > Higher Is Better ONNX Runtime 1.10 Model: fcn-resnet101-11 - Device: CPU Inferences Per Minute > Higher Is Better ONNX Runtime 1.10 Model: shufflenet-v2-10 - Device: CPU Inferences Per Minute > Higher Is Better ONNX Runtime 1.10 Model: super-resolution-10 - Device: CPU Inferences Per Minute > Higher Is Better LeelaChessZero 0.28 Backend: BLAS Nodes Per Second > Higher Is Better ML Tests . 563 |=============================================================== Numpy Benchmark Score > Higher Is Better ML Tests .................... 422.45 |========================================= AI Benchmark Alpha 0.1.2 Score > Higher Is Better TensorFlow Lite 2020-08-23 Model: SqueezeNet Microseconds < Lower Is Better ML Tests . 189764 |============================================================ TensorFlow Lite 2020-08-23 Model: Inception V4 Microseconds < Lower Is Better ML Tests . 2749623 |=========================================================== TensorFlow Lite 2020-08-23 Model: NASNet Mobile Microseconds < Lower Is Better ML Tests . 152186 |============================================================ TensorFlow Lite 2020-08-23 Model: Mobilenet Float Microseconds < Lower Is Better ML Tests . 127818 |============================================================ TensorFlow Lite 2020-08-23 Model: Mobilenet Quant Microseconds < Lower Is Better ML Tests . 141174 |============================================================ TensorFlow Lite 2020-08-23 Model: Inception ResNet V2 Microseconds < Lower Is Better ML Tests . 2479080 |=========================================================== Caffe 2020-02-13 Model: AlexNet - Acceleration: CPU - Iterations: 100 Milli-Seconds < Lower Is Better ML Tests . 33496 |============================================================= Caffe 2020-02-13 Model: AlexNet - Acceleration: CPU - Iterations: 200 Milli-Seconds < Lower Is Better ML Tests . 65986 |============================================================= Caffe 2020-02-13 Model: AlexNet - Acceleration: CPU - Iterations: 1000 Milli-Seconds < Lower Is Better ML Tests . 325884 |============================================================ Caffe 2020-02-13 Model: GoogleNet - Acceleration: CPU - Iterations: 100 Milli-Seconds < Lower Is Better ML Tests . 86567 |============================================================= Caffe 2020-02-13 Model: GoogleNet - Acceleration: CPU - Iterations: 200 Milli-Seconds < Lower Is Better ML Tests . 173671 |============================================================ Caffe 2020-02-13 Model: GoogleNet - Acceleration: CPU - Iterations: 1000 Milli-Seconds < Lower Is Better ML Tests . 868758 |============================================================ oneDNN 2.1.2 Harness: IP Shapes 1D - Data Type: f32 - Engine: CPU ms < Lower Is Better ML Tests .................... 4.25855 |======================================== oneDNN 2.1.2 Harness: IP Shapes 3D - Data Type: f32 - Engine: CPU ms < Lower Is Better ML Tests .................... 12.09 |========================================== oneDNN 2.1.2 Harness: IP Shapes 1D - Data Type: u8s8f32 - Engine: CPU ms < Lower Is Better ML Tests .................... 1.62985 |======================================== oneDNN 2.1.2 Harness: IP Shapes 3D - Data Type: u8s8f32 - Engine: CPU ms < Lower Is Better ML Tests .................... 2.69210 |======================================== oneDNN 2.1.2 Harness: IP Shapes 1D - Data Type: bf16bf16bf16 - Engine: CPU ms < Lower Is Better oneDNN 2.1.2 Harness: IP Shapes 3D - Data Type: bf16bf16bf16 - Engine: CPU ms < Lower Is Better oneDNN 2.1.2 Harness: Convolution Batch Shapes Auto - Data Type: f32 - Engine: CPU ms < Lower Is Better ML Tests .................... 22.79 |========================================== oneDNN 2.1.2 Harness: Deconvolution Batch shapes_1d - Data Type: f32 - Engine: CPU ms < Lower Is Better ML Tests .................... 8.34789 |======================================== oneDNN 2.1.2 Harness: Deconvolution Batch shapes_3d - Data Type: f32 - Engine: CPU ms < Lower Is Better ML Tests .................... 6.74559 |======================================== oneDNN 2.1.2 Harness: Convolution Batch Shapes Auto - Data Type: u8s8f32 - Engine: CPU ms < Lower Is Better ML Tests .................... 23.77 |========================================== oneDNN 2.1.2 Harness: Deconvolution Batch shapes_1d - Data Type: u8s8f32 - Engine: CPU ms < Lower Is Better ML Tests .................... 2.11771 |======================================== oneDNN 2.1.2 Harness: Deconvolution Batch shapes_3d - Data Type: u8s8f32 - Engine: CPU ms < Lower Is Better ML Tests .................... 3.24784 |======================================== oneDNN 2.1.2 Harness: Recurrent Neural Network Training - Data Type: f32 - Engine: CPU ms < Lower Is Better ML Tests .................... 3579.00 |======================================== oneDNN 2.1.2 Harness: Recurrent Neural Network Inference - Data Type: f32 - Engine: CPU ms < Lower Is Better ML Tests .................... 2219.13 |======================================== oneDNN 2.1.2 Harness: Recurrent Neural Network Training - Data Type: u8s8f32 - Engine: CPU ms < Lower Is Better ML Tests .................... 3587.17 |======================================== oneDNN 2.1.2 Harness: Convolution Batch Shapes Auto - Data Type: bf16bf16bf16 - Engine: CPU ms < Lower Is Better oneDNN 2.1.2 Harness: Deconvolution Batch shapes_1d - Data Type: bf16bf16bf16 - Engine: CPU ms < Lower Is Better oneDNN 2.1.2 Harness: Deconvolution Batch shapes_3d - Data Type: bf16bf16bf16 - Engine: CPU ms < Lower Is Better oneDNN 2.1.2 Harness: Recurrent Neural Network Inference - Data Type: u8s8f32 - Engine: CPU ms < Lower Is Better ML Tests .................... 2228.17 |======================================== oneDNN 2.1.2 Harness: Matrix Multiply Batch Shapes Transformer - Data Type: f32 - Engine: CPU ms < Lower Is Better ML Tests .................... 4.59343 |======================================== oneDNN 2.1.2 Harness: Recurrent Neural Network Training - Data Type: bf16bf16bf16 - Engine: CPU ms < Lower Is Better ML Tests .................... 3577.00 |======================================== oneDNN 2.1.2 Harness: Recurrent Neural Network Inference - Data Type: bf16bf16bf16 - Engine: CPU ms < Lower Is Better ML Tests .................... 2237.65 |======================================== oneDNN 2.1.2 Harness: Matrix Multiply Batch Shapes Transformer - Data Type: u8s8f32 - Engine: CPU ms < Lower Is Better ML Tests .................... 2.98659 |======================================== oneDNN 2.1.2 Harness: Matrix Multiply Batch Shapes Transformer - Data Type: bf16bf16bf16 - Engine: CPU ms < Lower Is Better Mobile Neural Network 1.2 Model: mobilenetV3 ms < Lower Is Better MBP M1 Max Machine Learning . 9.152 |========================================== ML Tests .................... 1.202 |====== Mobile Neural Network 1.2 Model: squeezenetv1.1 ms < Lower Is Better MBP M1 Max Machine Learning . 7.274 |========================================== ML Tests .................... 2.803 |================ Mobile Neural Network 1.2 Model: resnet-v2-50 ms < Lower Is Better MBP M1 Max Machine Learning . 42.43 |========================================== ML Tests .................... 22.44 |====================== Mobile Neural Network 1.2 Model: SqueezeNetV1.0 ms < Lower Is Better MBP M1 Max Machine Learning . 9.967 |========================================== ML Tests .................... 4.546 |=================== Mobile Neural Network 1.2 Model: MobileNetV2_224 ms < Lower Is Better MBP M1 Max Machine Learning . 10.677 |========================================= ML Tests .................... 2.387 |========= Mobile Neural Network 1.2 Model: mobilenet-v1-1.0 ms < Lower Is Better MBP M1 Max Machine Learning . 8.205 |========================================== ML Tests .................... 2.440 |============ Mobile Neural Network 1.2 Model: inception-v3 ms < Lower Is Better MBP M1 Max Machine Learning . 58.25 |========================================== ML Tests .................... 31.58 |======================= NCNN 20210720 Target: CPU - Model: mobilenet ms < Lower Is Better MBP M1 Max Machine Learning . 20.32 |========================================== ML Tests .................... 15.95 |================================= NCNN 20210720 Target: CPU-v2-v2 - Model: mobilenet-v2 ms < Lower Is Better MBP M1 Max Machine Learning . 5.33 |=========================================== ML Tests .................... 3.99 |================================ NCNN 20210720 Target: CPU-v3-v3 - Model: mobilenet-v3 ms < Lower Is Better MBP M1 Max Machine Learning . 4.36 |=========================================== ML Tests .................... 3.41 |================================== NCNN 20210720 Target: CPU - Model: shufflenet-v2 ms < Lower Is Better MBP M1 Max Machine Learning . 3.47 |=========================================== ML Tests .................... 2.75 |================================== NCNN 20210720 Target: CPU - Model: mnasnet ms < Lower Is Better MBP M1 Max Machine Learning . 5.40 |=========================================== ML Tests .................... 3.25 |========================== NCNN 20210720 Target: CPU - Model: efficientnet-b0 ms < Lower Is Better MBP M1 Max Machine Learning . 8.69 |=========================================== ML Tests .................... 5.22 |========================== NCNN 20210720 Target: CPU - Model: blazeface ms < Lower Is Better MBP M1 Max Machine Learning . 1.65 |=========================================== ML Tests .................... 1.20 |=============================== NCNN 20210720 Target: CPU - Model: googlenet ms < Lower Is Better MBP M1 Max Machine Learning . 24.96 |========================================== ML Tests .................... 13.74 |======================= NCNN 20210720 Target: CPU - Model: vgg16 ms < Lower Is Better MBP M1 Max Machine Learning . 71.01 |========================================= ML Tests .................... 71.97 |========================================== NCNN 20210720 Target: CPU - Model: resnet18 ms < Lower Is Better MBP M1 Max Machine Learning . 16.82 |========================================== ML Tests .................... 15.78 |======================================= NCNN 20210720 Target: CPU - Model: alexnet ms < Lower Is Better MBP M1 Max Machine Learning . 29.93 |========================================== ML Tests .................... 14.55 |==================== NCNN 20210720 Target: CPU - Model: resnet50 ms < Lower Is Better MBP M1 Max Machine Learning . 43.16 |========================================== ML Tests .................... 25.17 |======================== NCNN 20210720 Target: CPU - Model: yolov4-tiny ms < Lower Is Better MBP M1 Max Machine Learning . 30.24 |========================================== ML Tests .................... 24.97 |=================================== NCNN 20210720 Target: CPU - Model: squeezenet_ssd ms < Lower Is Better MBP M1 Max Machine Learning . 20.53 |========================================== ML Tests .................... 18.56 |====================================== NCNN 20210720 Target: CPU - Model: regnety_400m ms < Lower Is Better MBP M1 Max Machine Learning . 7.18 |=========================================== ML Tests .................... 6.90 |========================================= NCNN 20210720 Target: Vulkan GPU - Model: mobilenet ms < Lower Is Better MBP M1 Max Machine Learning . 20.30 |========================================== ML Tests .................... 10.27 |===================== NCNN 20210720 Target: Vulkan GPU-v2-v2 - Model: mobilenet-v2 ms < Lower Is Better MBP M1 Max Machine Learning . 5.30 |=========================================== ML Tests .................... 3.88 |=============================== NCNN 20210720 Target: Vulkan GPU-v3-v3 - Model: mobilenet-v3 ms < Lower Is Better MBP M1 Max Machine Learning . 4.35 |======================================== ML Tests .................... 4.69 |=========================================== NCNN 20210720 Target: Vulkan GPU - Model: shufflenet-v2 ms < Lower Is Better MBP M1 Max Machine Learning . 3.46 |=========================================== ML Tests .................... 3.02 |====================================== NCNN 20210720 Target: Vulkan GPU - Model: mnasnet ms < Lower Is Better MBP M1 Max Machine Learning . 5.37 |=========================================== ML Tests .................... 3.89 |=============================== NCNN 20210720 Target: Vulkan GPU - Model: efficientnet-b0 ms < Lower Is Better MBP M1 Max Machine Learning . 8.71 |==================================== ML Tests .................... 10.07 |========================================== NCNN 20210720 Target: Vulkan GPU - Model: blazeface ms < Lower Is Better MBP M1 Max Machine Learning . 1.64 |=========================================== ML Tests .................... 1.35 |=================================== NCNN 20210720 Target: Vulkan GPU - Model: googlenet ms < Lower Is Better MBP M1 Max Machine Learning . 24.90 |========================================== ML Tests .................... 8.73 |=============== NCNN 20210720 Target: Vulkan GPU - Model: vgg16 ms < Lower Is Better MBP M1 Max Machine Learning . 70.89 |========================================== ML Tests .................... 43.99 |========================== NCNN 20210720 Target: Vulkan GPU - Model: resnet18 ms < Lower Is Better MBP M1 Max Machine Learning . 16.80 |========================================== ML Tests .................... 6.09 |=============== NCNN 20210720 Target: Vulkan GPU - Model: alexnet ms < Lower Is Better MBP M1 Max Machine Learning . 29.89 |========================================== ML Tests .................... 6.32 |========= NCNN 20210720 Target: Vulkan GPU - Model: resnet50 ms < Lower Is Better MBP M1 Max Machine Learning . 43.08 |========================================== ML Tests .................... 13.12 |============= NCNN 20210720 Target: Vulkan GPU - Model: yolov4-tiny ms < Lower Is Better MBP M1 Max Machine Learning . 30.33 |========================================== ML Tests .................... 18.82 |========================== NCNN 20210720 Target: Vulkan GPU - Model: squeezenet_ssd ms < Lower Is Better MBP M1 Max Machine Learning . 20.55 |========================================== ML Tests .................... 15.36 |=============================== NCNN 20210720 Target: Vulkan GPU - Model: regnety_400m ms < Lower Is Better MBP M1 Max Machine Learning . 7.19 |=========================================== ML Tests .................... 5.28 |================================ TNN 0.3 Target: CPU - Model: DenseNet ms < Lower Is Better ML Tests . 2736.17 |=========================================================== TNN 0.3 Target: CPU - Model: MobileNet v2 ms < Lower Is Better ML Tests . 249.48 |============================================================ TNN 0.3 Target: CPU - Model: SqueezeNet v2 ms < Lower Is Better ML Tests . 55.43 |============================================================= TNN 0.3 Target: CPU - Model: SqueezeNet v1.1 ms < Lower Is Better ML Tests . 222.33 |============================================================ OpenCV 4.5.4 Test: DNN - Deep Neural Network ms < Lower Is Better ML Tests . 13787 |============================================================= DeepSpeech 0.6 Acceleration: CPU Seconds < Lower Is Better ML Tests .................... 74.44 |========================================== Numenta Anomaly Benchmark 1.1 Detector: EXPoSE Seconds < Lower Is Better Numenta Anomaly Benchmark 1.1 Detector: Relative Entropy Seconds < Lower Is Better Numenta Anomaly Benchmark 1.1 Detector: Windowed Gaussian Seconds < Lower Is Better Numenta Anomaly Benchmark 1.1 Detector: Earthgecko Skyline Seconds < Lower Is Better Numenta Anomaly Benchmark 1.1 Detector: Bayesian Changepoint Seconds < Lower Is Better R Benchmark Seconds < Lower Is Better ML Tests . 0.1293 |============================================================ RNNoise 2020-06-28 Seconds < Lower Is Better ML Tests . 16.14 |============================================================= Tensorflow Build: Cifar10 Seconds < Lower Is Better ECP-CANDLE 0.4 Benchmark: P1B2 Seconds < Lower Is Better ML Tests . 37.51 |============================================================= ECP-CANDLE 0.4 Benchmark: P3B1 Seconds < Lower Is Better ML Tests . 1463.72 |=========================================================== ECP-CANDLE 0.4 Benchmark: P3B2 Seconds < Lower Is Better ML Tests . 730.74 |============================================================ Mlpack Benchmark Benchmark: scikit_ica Seconds < Lower Is Better ML Tests . 48.40 |============================================================= Mlpack Benchmark Benchmark: scikit_qda Seconds < Lower Is Better ML Tests . 65.69 |============================================================= Mlpack Benchmark Benchmark: scikit_svm Seconds < Lower Is Better ML Tests . 17.60 |============================================================= Mlpack Benchmark Benchmark: scikit_linearridgeregression Seconds < Lower Is Better ML Tests . 2.10 |==============================================================