MT3 is a multi-instrument automatic music transcription model that uses the T5X framework.

This is not an officially supported Google product.

This project uses Poetry for dependency management.

# Install Poetry if you haven't already
curl -sSL https://install.python-poetry.org | python3 -

# Clone the repository
git clone https://github.com/magenta/mt3.git
cd mt3

# Install dependencies using Poetry (including git dependencies)
poetry install

# Activate the virtual environment
poetry shell

For consistent and reproducible environments, we recommend using Docker:

# Clone the repository
git clone https://github.com/magenta/mt3.git
cd mt3

# Build and start the Docker container
docker-compose up -d

# Connect to the running container
docker-compose exec mt3 bash

# Once inside the container, you can run commands directly

Use our colab notebook to transcribe audio files of your choosing. You can use a pretrained checkpoint from either a) the piano transcription model described in our ISMIR 2021 paper or b) the multi-instrument transcription model described in our ICLR 2022 paper.

For now, we do not (easily) support training. If you like, you can try to follow the T5X training instructions and use one of the tasks defined in tasks.py.

This project is configured with Python 3.11 as the target version. We use:

• Poetry for dependency management
• pyenv or mise for Python version management
• pytest for testing

All dependencies, including git dependencies (flax, note-seq, seqio, and t5x), are managed through Poetry in the pyproject.toml file.

Due to the complex dependency tree of TensorFlow and T5X, some compatibility issues may arise when installing dependencies locally. Using Docker is recommended for a more consistent environment.