3D image processing in the context of reliable maps
|Duration:||2017 - 2019|
|Funded by:||Niedersächsisches Ministerium für Wissenschaft und Kultur (As part of the Wissenschaftsallianz Hannover-Braunschweig in the part Mobilize/Mobile Human)|
Reliable and up-to-date maps received much attention in the field of mobility in recent times. Especially the sector of assisted and autonomous driving strongly depends on such maps in order to be able to accurately determine the vehicle's pose and compute a safe trajectory. With regard to the importance of the currency of the information contained in such a map, data acquisition must be done continuously.
For the purpose of data acquisition, mobile mapping using stereo cameras is well suited. In comparison to laser scanners, stereo cameras are less expensive and furthermore, they do not only provide geometric, but also multi-spectral radiometric information. Hence, additional information for later image segmentation and classification is provided. But, special attention has to be paid to the fusion of current information and data captured earlier to obtain a consistent, reliable and accurate result. Depending on the temporal distance between two recordings, changes in illumination can lead to significant differences in the acquired data. Not only the color of illumination or the geometric relationship between illumination, scene and sensor but also the number of light sources might have changed, making the fusion task a potentially difficult challenge.
Figure 1: Example for simulated illumination variation (The brightness of the right image is reduced by the respective value.)
The goal of this project is to develop new approaches for illumination invariant dense image matching. For this purpose, we are currently working on a cost function, which is robust against changes in illumination, utilizing census transformation and Phase Congruency. The process of cost computation is supported by a triangle-based depth prediction approach using a set of matched feature points. These points can reliably be detected even under drastic changes in illumination. The results are validated on the challenging KITTI stereo dataset and a set of images with simulated changes.
Figure 2: Qualitative results for our approach. In descending order: Left image, estimated disparity, ground truth disparity (from large disparities in yellow to small ones in blue), deviation between estimation and ground truth (from small deviations in white/light blue to large ones in orange/red).
Overall research project: “Mobile Human"
Within the framework of the joint master plan "MOBILISE - Mobility in Engineering and Science" the two universities of Lower Saxony, Leibniz Universität Hannover and TU Braunschweig cooperate in the field of "Digitization". The field "Mobile Human: Intelligent Mobility in the Balance of Autonomy, Linkage and Security" under the direction of Prof. Kurt Schneider aims to install an expanded junior research group on a seminal, previously unestablished topic of social relevance.
A total of 13 professorships from five LUH faculties with their specific, complementary areas of focus are involved in the project. In addition to the Faculty of Civil Engineering and Geodetic Science, these are the Faculty of Electrical Engineering and Computer Science, the Faculty of Architecture and Landscape Sciences, the Faculty of Law and the Faculty of Humanities.
Mehltretter, M., Heipke, C. (2018): Illumination Invariant Dense Image Matching based on Sparse Features. 38. Wissenschaftlich-Technische Jahrestagung der DGPF und PFGK18 Tagung in München, Band 27, 584-596. more
Behmann, N.; Mehltretter, M.; Kleinschmidt, S. P.; Wagner, B.; Heipke, C.; Blume, H. (2018): GPU-enhanced Multimodal Dense Matching. 2018 IEEE Nordic Circuits and Systems Conference (NORCAS): NORCHIP and International Symposium of System-on-Chip (SoC). | file | more
Mehltretter, M.; Kleinschmidt, S.P.; Wagner, B.; Heipke, C. (2018): Multimodal dense stereo matching. In: Bronx T., Bruhn A. (Eds.): Pattern recognition – 40th German Conference GCPR Stuttgart, LNCS 11269, Springer, 407-421. | file |