Repository for the course with all material. The course is organized in two days, notebooks and presentations can be found in the respective folders.

If possible, work with uv. Clone the repository and run uv sync.

Create an venv or conda environment and install the following packages:

• accelerate
• autoawq
• bitsandbytes
• device-smi
• exllamav2
• flash-attn
• flashinfer-python
• ipykernel
• ipython
• ipywidgets
• jupyter
• langchain
• langchain-community
• langchain-huggingface
• logbar
• nvidia-ml-py
• openai
• optimum
• pandas
• pip
• pymilvus
• sentence-transformers
• spacy
• tantivy
• tokenicer
• tqdm
• transformers
• txtai
• usearch
• vllm

flash-attn should be installed with the option --no-build-isolation.

Of course, you can also Use the supplied requirements.txt, but some dependencies might be outdated.

You can also use runpod. uv is already preinstalled there.

You can either try to run the notebooks directly or try to follow how I run them and use it as a documentation (or run it later).