Toggle Main Menu Toggle Search

Open Access padlockePrints

Automatic registration of panoramic image sequence and mobile laser scanning data using semantic features

Lookup NU author(s): Dr Wen Xiao



This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License (CC BY-NC-ND).


© 2017 International Society for Photogrammetry and Remote Sensing, Inc. (ISPRS) Inaccurate exterior orientation parameters (EoPs) between sensors obtained by pre-calibration leads to failure of registration between panoramic image sequence and mobile laser scanning data. To address this challenge, this paper proposes an automatic registration method based on semantic features extracted from panoramic images and point clouds. Firstly, accurate rotation parameters between the panoramic camera and the laser scanner are estimated using GPS and IMU aided structure from motion (SfM). The initial EoPs of panoramic images are obtained at the same time. Secondly, vehicles in panoramic images are extracted by the Faster-RCNN as candidate primitives to be matched with potential corresponding primitives in point clouds according to the initial EoPs. Finally, translation between the panoramic camera and the laser scanner is refined by maximizing the overlapping area of corresponding primitive pairs based on the Particle Swarm Optimization (PSO), resulting in a finer registration between panoramic image sequences and point clouds. Two challenging urban scenes were experimented to assess the proposed method, and the final registration errors of these two scenes were both less than three pixels, which demonstrates a high level of automation, robustness and accuracy.

Publication metadata

Author(s): Li J, Yang B, Chen C, Huang R, Dong Z, Xiao W

Publication type: Article

Publication status: Published

Journal: ISPRS Journal of Photogrammetry and Remote Sensing

Year: 2018

Volume: 136

Pages: 41-57

Print publication date: 01/02/2018

Online publication date: 19/12/2017

Acceptance date: 09/12/2017

Date deposited: 27/02/2018

ISSN (print): 0924-2716

ISSN (electronic): 1872-8235

Publisher: Elsevier BV


DOI: 10.1016/j.isprsjprs.2017.12.005


Altmetrics provided by Altmetric