This is a project to detect holes or under sampled regions in 3D point clouds of indoor spaces using the following pipeline:

In order to run the pipeline you need some external libraries:

• PCL version: 1.10
• Eigen version: 3.3
• OpenCV version: 4.2
• Pkg-Config version: 0.29.1 (should be preinstalled)

You need the following data in order to run the pipeline:

• A pcd file of the indoor space in which you want to search for holes. (e.g. hololens.pcd)
• An image of the floor plan of your indoor space (e.g. floorplan.jpg)
• or a pcd file containing the vertices of your floor plan which are aligned to the main space (e.g. floorplan.pcd)
• (optional) A pcd file of each 3d point where data was collected (e.g. trajectory.pcd)
• (optional) An obj file containing the heading of each data collection point (e.g. gaze_directions.obj)
• (optional) A text file containing the length of each coherent trajectory in trajectory.pcd (e.g. trajectory_lengths.txt)

Download the example data and copy it into the data folder then execute (at the moment it is not possible to use the pipeline in Release mode because some external libraries used will behave different):

mkdir build
cd build
cmake .. -DCMAKE_BUILD_TYPE=Debug
make -j 4
./HoleDet

First the floor plan jpg is opened. Here you can click on all corners of your indoor space. It is very important that you remember in which order you click them. If you have clicked them all, close the picture with the esc button (do not just close the window).
Next, the projected point cloud of the floor plane is opened. Here you have to hold shift and click on the same corners (in the same order as before). After you have finished, do not just close window, instead hold shift again and click anywhere. Unfortunately the viewer cannot be closed in a different way, which is a known PCL issue. A first transformation based on your selection of corners is then applied. This is then refined using RANSAC. This point cloud is then saved, so you do not have to do this step again (remember to change the config parameter).

The pipeline generates multiple output files in the output/ directory. First there is an overview file which shows for each hole:

• Coordinates of the centroid
• Number of boundary points
• Area of hole
• Score of the hole

Furthermore, for each hole there is a csv file generated in output/poses containing the computed poses to resample the hole. The poses are saved as row major flattened matrices. Before rerunning the pipeline you should delete all the files from the output folder (but not the folder).

The pipeline uses a lot of parameters. These can be found and changed in the cfg/config.yaml file.

• use_existing_floorplan : This should be true if you already have a pcd file of your floor plan. If this is false the pipeline will create it for the next use
• use_gaze : True if you want to use the optional gaze and trajectory data. This makes the pipeline a lot slower, but incorporates gaze data into the scoring mechanism
• input : The path to all the input file. If you use the example data this should not change
• poisson_depth : The depth of the poisson reconstruction of the mesh
• normal_search_radius : The search radius used for the normal estimation
• outlier_radius : Max radius of the radial outlier removal used for pre-filtering of the point cloud
• outlier_min_neighbours : The minimum neighbours needed to be an inlier in the outlier removal
• image_resolution : The resolution of the floor plan image in m/px. This is only needed if use_existing_floorplan is false
• max_iteration : Maximum number of RANSAC iterations used to match the floor plan to the original space
• max_translation : Maximum translation in m used to translate the floor plan in the ransac step
• max_angle : Maximum angle in deg used to rotate the floor plan in the ransac step
• boundary_search_radius : The max search radius used for boundary estimation
• angle_tresh : An angle threshold in rad which is used in clustering. Only boundary points with an angle between the normals smaller than this threshold can belong to the same cluster
• vert_score_threshold : The max covariance in height of the points inside a point cloud. If it is higher than this threshold the score will be penalised.
• min_score : The minimum score a hole has to have to be written out and visualized
• step_back : The desire distance between a generated pose and its corrensponding hole's boundary