2024/07: Our paper has been accepted by Pattern Recogintion(PR).2023/06: Code and models are released.
- Enhanced Decoder: A multi-query and multi-level enhanced network for semantic segmentation, which aims to exploit diverse information at different feature map levels in plain transformer backbone.
- Stronger performance: We got state-of-the-art performance mIoU 56.2% on ADE20K, mIoU 51.0% on COCOStuff10K, and mIoU 60.2% on PASCAL-Context datasets using ViT backbone.
- Install the mmsegmentation library and some required packages.
pip install mmcv-full==1.7.1 mmsegmentation==0.30.0
pip install scipy timmsh tools/dist_train.sh configs/qlseg/qlseg_vit-l_jax_640x640_160k_ade20k.py
sh tools/dist_test.sh configs/qlseg/qlseg_vit-l_jax_640x640_160k_ade20k.py {path_to_ckpt}
Please follow the instructions of mmsegmentation data preparation
| Model backbone | datasets | mIoU | mIoU (ms) | ckpt |
|---|---|---|---|---|
| Vit-Base | ADE20k | 52.9 | 53.6 | Baidu Drive, Google Drive |
| Vit-Large | ADE20k | 55.3 | 56.2 | Baidu Drive, Google Drive |
| Vit-Large | COCOStuff10K | 50.5 | 51.0 | Baidu Drive |
| Vit-Large | PASCAL-Context (59cls) | 65.5 | 66.4 | Baidu Drive |
| Vit-Large | PASCAL-Context (60cls) | 59.3 | 60.2 | Baidu Drive |
For academic use, this project is licensed under the 2-clause BSD License - see the LICENSE file for details. For commercial use, please contact the authors.
@article{xie2024multi,
title={Multi-query and multi-level enhanced network for semantic segmentation},
author={Xie, Bin and Cao, Jiale and Anwer, Rao Muhammad and Xie, Jin and Nie, Jing and Yang, Aiping and Pang, Yanwei},
journal={Pattern Recognition},
pages={110777},
year={2024},
publisher={Elsevier}
}
Thanks to previous open-sourced repo: SegVit.