AmirHossein Sojoodi

AmirHossein Sojoodi

PhD student in Computer Engineering at Queen's University

Reporting Prediction Errors

Published:
1 minute read

There are several types of reporting errors in prediction. There is a whole research behind this topic, especially for the purpose of training the ML models. However, I put some of them that I usually use here.

Given that $f \rightarrow forcast$ and $y \rightarrow value$, we can calculate these errros:

  1. Raw Error (RE)

    \[RE = f - y\]

  2. Percentage Error (PE)

    \[PE = {(f - y) \over y}\]

  3. Symmetric Percentage Error (sPE)

    \[sPE = {(f - y) \over (f + y) / 2}\]

  4. Log Error (LE)

    \(LE = log(f) - log(y)\) or

    \[LE = log({f \over y})\]

And some of the associated performance metrics are:

  1. Mean Symmetric Error (MSE)

    \[{1 \over n} \sum_{k=1}^n (f_k - y_k)^2\]

  2. Mean Absolute Percentage Error (MAPE)

    \[{1 \over n} \sum_{k=1}^n |f_k - y_k| / y_k\]

  3. Symmetric Mean Absolute Percentage Error (sMAPE)

    \[{1 \over n} \sum_{k=1}^n |f_k - y_k| / y_k\]

  4. Mean Absolute Log Error (MALE)

    \[{1 \over n} \sum_{k=1}^n |log(f_k/y_k)|\]

  5. Root Mean Square Log Error (RMSLE)

    \[sqrt({1 \over n} \sum_{k=1}^n |log(f_k/y_k)|^2)\]

  6. Exponential Mean Absolute MALE (EMALE)

    \[\exp({1 \over n} \sum_{k=1}^n |log(f_k/y_k)|)\]

  7. Exponential Root Mean Square Log Error (ERMSLE)

    \[\exp(sqrt({1 \over n} \sum_{k=1}^n |log(f_k/y_k)|^2))\]

Python version

# Given:
# f -> forecast
# y -> value

# RAW Error (RE)
RE = f - y

# Percentage Error (PE)
PE = (f - y) / y

# Symmetric Percentage Error (sPE)
sPE = (f - y) / ((f + y) / 2)

# Log Error (LE)
LE = log(f) - log(y) 
# or
LE = log(f/y)
# LE = log(1 + PE)

And the respective performance metrics of the model/prediction:

# Mean Absolute Percentage Error
MAPE = mean(abs(PE))

# Symmetric Mean Absolute Percentage Error
sMAPE = mean(abs(sPE))

# Mean Absolute Log Error
MALE = mean(abs(LE))

# Root Mean Square Log Error
RMSLE = sqrt(mean(LE ** 2))

# Exponential Mean Absolute MALE
EMALE = exp(MALE)

# Exponential Root Mean Square Log Error
ERMSLE = expo(RMSLE)

There are tons of resources out there. If you want to see which one suits your usecase, take a look at here.

You May Also Enjoy

Setup LAMMPS

4 minute read

Published:

Large-scale Atomic/Molecular Massively Parallel Simulator (LAMMPS) is a classical molecular dynamics code that can be used to model atoms or, more generally, as a parallel particle simulator at various scales. The complete documentation of LAMMPS can be found here. In this post, I will provide a guide on how to setup LAMMPS on a Linux machine. My setup is on a cluster with NVIDIA GPUs, UCX, and OpenMPI. Also, we have a built-in module system to load the necessary modules.

MPS on Multi-Instance GPU

9 minute read

Published:

In previous posts, (MPS and MIG), I have explained how to enable MPS and MIG on NVIDIA GPUs. In this post, I will explain how to use both technologies at the same time. In more detail, I would like to enable MPS on all of the MIG instances. For more information, you can refer to the NVIDIA document.

Wasm + WebGPU example on DCP

12 minute read

Published:

This example is a follow-up to my previous post on how to write a cross-platform WebGPU example. In this one, I’ll demonstrate how to deploy a matmult example written in C/C++ and WebGPU in a DCP worker using WASM. Note that for verification purposes, I provide dawn-based native test, too, but this example doesn’t need to build/install dawn in order to work.

Cross Platform WebGPU Example - Compute API

5 minute read

Published:

As a part of my work at Distributive, I developed an example providing a simple matrix multiplication using WebGPU Compute API in C++. This example is written in C++ and can be compiled to WebAssembly using Emscripten, native executable file using Dawn. The example is based on here, and its complete source can be found there.