Skip to content
View mars-sep's full-sized avatar

Block or report mars-sep

Block user

Prevent this user from interacting with your repositories and sending you notifications. Learn more about blocking users.

You must be logged in to block users.

Maximum 250 characters. Please don't include any personal information such as legal names or email addresses. Markdown supported. This note will be visible to only you.
Report abuse

Contact GitHub support about this user’s behavior. Learn more about reporting abuse.

Report abuse
MARS-Sep/README.md

πŸ“¦ Data Preparation

🎡 MUSIC Dataset

Please refer to the script under dataset/music.

πŸ”Š VGGSound Dataset

Please refer to the script under dataset/vggsound.

πŸš€ Installation

Clone the repository and set up the environment:

git clone https://anonymous.4open.science/r/ImageBind.git
cd Imagebind
pip install .

git clone https://anonymous.4open.science/r/MARS-Sep.git
cd MARS-Sep/

conda create -n marssep python=3.10
conda activate marssep

pip install -r requirements.txt

Training

python train.py \
    -o exp/vggsound/marssep \
    -c conf/mars.yaml
    -t data/vggsound/train.csv \
    -v data/vggsound/val.csv \
    --batch_size 128 \
    --workers 20 \
    --emb_dim 1024 \
    --train_mode image text audio \
    --is_feature \
    --feature_mode imagebind

Evaluate

Evaluate on MUSIC and VGGSound.

OMP_NUM_THREADS=1 python evaluate.py -o exp/vggsound/marssep/ -c conf/mars.yaml -l exp/vggsound/marssep/eval_MUSIC_VGGS.txt -t data/MUSIC/solo/test.csv -t2 data/vggsound/test-good-no-music.csv --no-pit --prompt_ens --audio_source ./MUSIC-aq.npy

Evaluate on VGGSoundClean + VGGSound.

OMP_NUM_THREADS=1 python evaluate.py -o exp/vggsound/marssep/ -c conf/mars.yaml -l exp/vggsound/marssep/eval_VGGS_VGGSN.txt -t data/vggsound/test-good.csv -t2 data/vggsound/test-no-music.csv --no-pit --prompt_ens --audio_source ./VGGSOUND-aq.npy

Inference

OMP_NUM_THREADS=1 python infer3.py -o exp/vggsound/marssep/  -i "demo/audio/hvCj8Dk0Su4.wav" --text_query "playing bagpipes" -f "exp/vggsound/marssep/hvCj8Dk0Su4/playing bagpipes.wav"

Popular repositories Loading

  1. mars-sep.github.io mars-sep.github.io Public

    HTML

  2. ImageBind ImageBind Public

    Forked from facebookresearch/ImageBind

    ImageBind One Embedding Space to Bind Them All

    Python

  3. MARS-Sep MARS-Sep Public

    Python 1

  4. Conv-TasNet-MARS Conv-TasNet-MARS Public

    MARS-Sep implementation on Conv-TasNet(PyTorch only).

    Python