When a Machine Learning (ML) model is deployed, we may perform inference on a wide variety of data, in particular data the model has not been trained to predict on, thus producing erroneous or even nonsense predictions. Out-of-Distribution (OOD) Detection seeks precisely to detect this data, so that we avoid performing inference on it, or we warn the user about the untrustworhiness of the model predictions. Post-hoc OOD detection is a series of OOD detection techniques that use a trained ML model in order to detect OOD data, without changing the predictive model's weights or training auxiliar model.

This repository gathers materials from the DEEL master class to acquire theoretical and practical knowledge about post-hoc OOD detection. Through these resources, you will discover the main post-hoc OOD scores and methods as well as the metrics used to assess their performance. You will learn to apply the different methods and to assess them using the OODEEL library across various examples, and develop the skills to interpret the results and select suitable algorithms for real-world applications.

This repository contains the tutorial notebooks for the following topics:

You can either work locally by cloning the project or open the notebooks following the colab links.

To get started with this tutorial, you have two options:

If you have access to Google Colab, you can run this project directly in your web browser without needing to install anything locally.

1. Sign in to Google Colab: Ensure you have a Google account to sign in.
2. Open the Colab Notebook: Open the Colab notebook links (see below).
3. Install Dependencies in Colab: The notebook will install all required packages in Colab’s environment.

This option is ideal if Colab is accessible to you.

If access to Google Colab is restricted or unavailable, follow these steps to install the project locally:

1. Clone the repository:

git clone https://github.com/deel-ai/uq-masterclass.git
cd uq-masterclass

2. Create a virtual environment:

python -m virtualenv venv-masterclass

3. Activate the virtual environment:

   • On windows:

   venv-masterclass\Scripts\activate

   • On macOS/Linux:

   source venv-masterclass/bin/activate

4. Install the required packages:

pip install -r requirements.txt

5. Add the virtual environment to Jupyter as a new kernel:

python -m ipykernel install --user --name=venv-masterclass"

6. Launch Jupyter Notebook:

jupyter notebook

In Jupyter, you can select the newly created kernel Python (venv-masterclass) from the kernel options.

If you want to check your work, you can find all solutions in the "completed_notebooks" folder. Each file in this folder corresponds to the respective tutorial notebook. Feel free to review these solutions as you progress through the exercises.

• OODEEL Github
• 📘 OODEEL Documentation