Underwater Object Detection in the Era of Artificial Intelligence: Current, Challenges and Future (UODReview)
This website provides the source code to reproduce the experiments in our work UODReview, we publish the well-trained models, the detection results of some advanced detection networks trained on RUOD and DUO. Both xml and json annotations are provided for RUOD and DUO.
We also introduce two useful detection analysis tools (i.e., Diagnosis and Tide) to analyse the errors made by underwater detectors. This work provides the researchers with an off-the-shelf platform for comparing their underwater object detectors with previous works.
Put the downloaded datasets into ./UODReview/RUOD and ./UODReview/DUO
Our RUOD and DUO datasets contain both .xml format and .json format annotations for the test set.
The .json format are used for training and testing, while the .xml format annotations are used for the detection analysis tool Diagnosis in Section 3.2.
Put the downloaded models into ./UODReview/models/RUOD and ./UODReview/models/DUO
For example,
python ./tools/test.py ./configs/reppoints/reppoints-moment_r101_fpn-gn_head-gn_2x_coco_RUOD.py ./models/RUOD/modelsReppoints/modelsReppoints.pth --work-dir ./models/RUOD/modelsReppoints
The source code is based on the MMDetection toolbox, which a part of the OpenMMLab project. Please refer to Installation to set the running environment.
- Python: 3.8.19
- CUDA: 11.3
- PyTorch: 1.12.1
- CuDNN: 8.3.2
- TorchVision: 0.13.1
- OpenCV: 4.9.0
- Magma: 2.5.2
- MMEngine: 0.10.4
python ./tools/train.py ./configs/xxx/xxx_RUOD.py --work-dir ./models/RUOD/modelsxxx #On RUOD
python ./tools/train.py ./configs/xxx/xxx_DUO.py --work-dir ./models/DUO/modelsxxx #On DUO
For example,
python ./tools/train.py ./configs/reppoints/reppoints-moment_r101_fpn-gn_head-gn_2x_coco_RUOD.py --work-dir ./RUOD/modelsReppoints
Notice: when changing between RUOD and DUO, please change the dataset setting in:
./mmdet/datasets/coco.py
./configs/_base_/datasets/coco_detection.py
./configs/xxx/xxx_RUOD.py or ./configs/xxx/xxx_DUO.py #set num_classes=10 for RUOD and num_classes=4 for DUO
python ./tools/test.py ./configs/xxx/xxx_RUOD.py ./models/RUOD/modelsxxx/modelsxxx.pth --work-dir ./models/RUOD/modelsxxx #On RUOD
For example,
python ./tools/test.py ./configs/reppoints/reppoints-moment_r101_fpn-gn_head-gn_2x_coco_RUOD.py ./models/RUOD/modelsReppoints/modelsReppoints.pth --work-dir ./models/RUOD/modelsReppoints
Notice: when changing between RUOD and DUO, please change the dataset setting in:
./mmdet/datasets/coco.py
./configs/_base_/datasets/coco_detection.py
./configs/xxx/xxx_RUOD.py or ./configs/xxx/xxx_DUO.py #set num_classes=10 for RUOD and num_classes=4 for DUO
Set "classwise=True" in ./configs/_base_/datasets/coco_detection.py as follows:
val_evaluator = dict(
type='CocoMetric',
ann_file=data_root + 'annotations_json/instances_test.json',
metric='bbox',
classwise=True, # Set
format_only=False,
backend_args=backend_args)
Add the following statement in ./configs/_base_/datasets/coco_detection.py as follows:
test_evaluator = dict(
type='CocoMetric',
metric='bbox',
format_only=True,
classwise= True,
ann_file=data_root + 'annotations_json/instances_val.json',
outfile_prefix='./UODReview/models/RUOD/modelsReppoints/detectionresults',
backend_args=backend_args)
The detection results are saved in './UODReview/models/RUOD/modelsxxx/detectionresultsxxx.bbox.json'
python tools/analysis_tools/get_flops.py ./configs/reppoints/reppoints-moment_r101_fpn-gn_head-gn_2x_coco_RUOD.py #On RUOD
python tools/analysis_tools/get_flops.py ./configs/reppoints/reppoints-moment_r101_fpn-gn_head-gn_2x_coco_DUO.py #On DUO
| Methods | Backbones | Params | FLOPs | AP | AP_50 | AP_75 | AP_s | AP_m | AP_l | Models | Detections |
|---|---|---|---|---|---|---|---|---|---|---|---|
| FoveaBox | ResNet101 | 56.68M | 268.29G | 44.8 | 80.2 | 45.2 | 18.0 | 37.5 | 49.1 | model | Detections |
| GuideAnchor | ResNet101 | 60.98M | 258.05G | 46.6 | 80.8 | 48.3 | 21.5 | 41.1 | 50.6 | model | Detections |
| CascadeRCNN | ResNet101 | 88.17M | 301.06G | 49.8 | 80.5 | 54.2 | 18.7 | 43.1 | 54.3 | model | Detections |
| RetinaNet | ResNet101 | 55.51M | 273.41G | 52.8 | 81.8 | 57.9 | 14.6 | 39.9 | 58.3 | model | Detections |
| RepPoints | ResNet101 | 55.82M | 256.00G | 53.2 | 82.2 | 60.1 | 28.2 | 44.9 | 57.8 | model | Detections |
| DetectoRS | DResNet50 | 123.23M | 90.05G | 53.3 | 84.1 | 58.7 | 30.8 | 46.6 | 57.8 | model | Detections |
| ATSS | ResNet101 | 51.13M | 267.26G | 54.0 | 80.3 | 60.2 | 18.0 | 40.0 | 59.5 | model | Detections |
| GridRCNN | ResNet101 | 129.63M | 365.60G | 54.2 | 81.6 | 60.0 | 25.6 | 46.1 | 59.8 | model | Detections |
| Methods | Backbones | Params | FLOPs | AP | AP_50 | AP_75 | AP_s | AP_m | AP_l | Models | Detections |
|---|---|---|---|---|---|---|---|---|---|---|---|
| FoveaBox | ResNet101 | 55.24M | 286.72G | 53.7 | 78.4 | 63.9 | 55.3 | 54.3 | 54.6 | model | Detections |
| GuideAnchor | ResNet101 | 60.94M | 276.48G | 61.4 | 83.8 | 72.0 | 58.9 | 62.4 | 61.3 | model | Detections |
| CascadeRCNN | ResNet101 | 88.15M | 319.49G | 60.6 | 80.9 | 70.5 | 54.4 | 61.4 | 61.2 | model | Detections |
| RetinaNet | ResNet101 | 55.38M | 289.79G | 50.9 | 73.1 | 59.7 | 51.0 | 50.1 | 53.1 | model | Detections |
| RepPoints | ResNet101 | 55.82M | 256.00G | 59.4 | 80.4 | 70.1 | 55.5 | 59.6 | 60.1 | model | Detections |
| DetectoRS | DResNet50 | 123.23M | 90.05G | 58.9 | 81.4 | 68.3 | 49.6 | 57.6 | 61.8 | model | Detections |
| ATSS | ResNet101 | 51.11M | 286.72G | 55.4 | 79.2 | 63.2 | 55.7 | 55.7 | 56.0 | model | Detections |
| GridRCNN | ResNet101 | 129.63M | 365.60G | 56.6 | 78.9 | 67.2 | 50.3 | 56.7 | 57.4 | model | Detections |
Our TIDE source code is custom-designed for RUOD and UOD datasets, located in UODRevew/diagnosis/Tide/
pip3 install tidecv
python ./diagnosis/tide-master/examples/TIDE_RUOD_DUO.py
Set the Path of Ground Truth and Detections
annFile = './RUOD/annotations/instances_val2017.json' # Ground Truth
resFile = './RUOD/modelsxxx/detectionresultsxxx.bbox.json' # Detections
And a summary plot for the model's errors are saved in UODReview/RUOD/modelsxxx/xxx_bbox_summary.json, one example as follows:
Our Diagnosis source code is custom-designed for RUOD and UOD datasets, located in UODRevew/diagnosis/Diagnosis/
Diagnosis needs to read the txt format detection results saved in per txt per class, such as:
comp3_det_test_holothurian.txt
So we need to transfer the json format detection results (produced in Section 2.3.2) into txt per class format using the python script:
python ./UODReview/models/convert_json2txt_per_cls.py
The txt format results will be saved in ./UODReview/diagnosis/Diagnosis/detections/xxx
Change between DUO and RUOD datasets in UODReview/diagnosis/Diagnosis/src/setDatasetParameters.m and UODReview/diagnosis/Diagnosis/src/VOCcode/VOCinit.m
In UODReview/diagnosis/Diagnosis/src/setDetectorInfo.m, for example, the ATSS detector was set as:
case 'ATSS'
detpath = '../detections/ATSS/comp3_det_test_%s.txt';
resultdir = '../results/ATSS';
detname = 'ATSS';
UODReview/diagnosis/Diagnosis/src/detectionAnalysisScript.m
We appreciate all contributions in the field of underwater object detection to improve UODReview. Welcome community users to participate in these projects.
If you use this toolbox or benchmark in your research, please cite this project.
@article{UODReview,
title = {Underwater Object Detection in the Era of Artificial Intelligence: Current, Challenges and Future},
author = {Long Chen, Yuzhi Huang, Junyu Dong, Qi xu, Sam Kwong, Huimin Lu, Chongyi Li},
journal= {},
year={2024}
}
@article{hoiem2012diagnosing,
title={Diagnosing error in object detectors},
author={Hoiem, Derek and Chodpathumwan, Yodsawalai and Dai, Qieyun},
journal={Computer Vision--ECCV 2012},
pages={340--353},
year={2012},
publisher={Springer}
}
@inproceedings{tide-eccv2020,
author = {Daniel Bolya and Sean Foley and James Hays and Judy Hoffman},
title = {TIDE: A General Toolbox for Identifying Object Detection Errors},
booktitle = {ECCV},
year = {2020},
}