Time and Robustness efficient method for SFM of ordered and unordered images
|Förderung durch:||CSC (Chinese Scholarship Council)|
In close-range photogrammetry untrained users typically do not capture images in a pre-planned pattern, and even experts often need significantly more time for data acquisition when strict recording protocols must be followed. For some years, following the development of sensors, images can be accessed in a much easier way, e.g. crowd-sourced data from the Internet. Different from images taken sequentially (e.g. by unmanned aerial vehicles or UAV), crowd-sourced images are unordered images, i.e. the overlap relationships first need to be established. Fig. 1 is the image overlap graph of UAV images (left) and Internet images (right), respectively, where the horizontal and vertical axes are the image ID and a white spot means that the corresponding image pairs overlap. We first determine the overlap graph and in this way significantly improve the speed of subsequent image matching, since only overlapping image pairs need to be processed. Then, we solve the SFM (Structure from Motion) problem, referred to as image orientation in photogrammetry. A typical solution is based on incremental methods, in which new images are added into the block sequentially, and a global method. In the incremental method bundle adjustment needs to be repeatedly performed to take care of block deformations, which reduce the time efficiency. We want to improve on these deficiencies.
|(a) UAV images||(b) Internet images|
Fig. 1: Image overlap graph.
We mainly work on the two most time consuming problems: image matching and image orientation. Currently, we came up two methods to solve these two problems, see Fig. 2.
- Image matching: We develop an efficient method to detect mutual overlap of a large set of unordered images. Features extracted from all images separately are split by several random k-d trees to efficiently obtain the nearest neighbours. Then, several strategies are applied to refine the results.
- Image orientation: We use global pose estimation instead of an incremental approach. We present several methods to detect outliers first, and then separate rotation and translation determination into separate steps. Finally, only one bundle adjustment is used to optimize the result.
Fig. 2: Overview of our SFM method.
The experimental results are available: https://github.com/wx7531774/SFM_results
Benchmark data are available for download at: https://www.ipi.uni-hannover.de/CVPR-PCV-Benchmark
Wang, X.; Zhan, Z.Q.; Heipke, C. (2017): AN EFFICIENT METHOD TO DETECT MUTUAL OVERLAP OF A LARGE SET OF UNORDERED IMAGES FOR STRUCTURE-FROM-MOTION. In: ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences IV-1/W1, pp. 191-198
Wang, X.; Rottensteiner, F.; Heipke, C. (2018): Robust image orientation based on relative rotations and tie points. In: ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences IV-2, pp. 295-302.