This is the official code of the CVPR 2024 paper Sat2Scene: 3D Urban Scene Generation from Satellite Images with Diffusion. For more details, please refer to our paper and project page.

For 3D generation part

• PyTorch 2.0.0+cu117
• MinkowskiEngine 0.5.4 Minkowski Engine
• ema_pytorch

For rendering part

• Coming soon

Please refer to this page for the dataset preparation.

The inference process can be run on NVIDIA TITAN RTX with 24GB memory. The trained model checkpoint can be downloaded here. Place the checkpoint file in the folder checkpoint_folder.

Exemplary point cloud files can be downloaded here. Place the point cloud .txt file in the folder point_cloud_files. (IMPORTANT!) For customized point clouds, it is required that the unit of the coordinates is the meter and the point density is ~400 points per square meter sampled with Poisson Disk.

After running the below script, the result files will be saved in the folder checkpoint_folder/result_files.

CUDA_VISIBLE_DEVICES=0 python3 denoising_diffusion_pytorch/denoising_diffusion_minkowski.py \
  --dataset_folder point_cloud_files \
  --dataset_mode test \
  --work_folder checkpoint_folder \
  --sampling_steps 1000 \
  --use_ema \
  --num_sample 1 \
  --point_scale 15 \
  --ckpt 218 \
  --save_folder result_files

MeshLab can visualize the exemplary point clouds (left) and the textures produced by the 3D sparse diffusion model (right).

Instruction coming soon.

Training and inference code coming soon.

@InProceedings{li2024sat2scene,
    author    = {Li, Zuoyue and Li, Zhenqiang and Cui, Zhaopeng and Pollefeys, Marc and Oswald, Martin R.},
    title     = {Sat2Scene: 3D Urban Scene Generation from Satellite Images with Diffusion},
    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
    month     = {June},
    year      = {2024},
    pages     = {7141-7150}
}