HAUTModel-based tracking

The various points studied in this theme are listed below as links leading to sections with their details and illustrations:

Pose direct estimation | 3D model-based tracking | Pose estimation and calibration

HAUTPose direct estimation

  • Mobile robot localization in urban environnement:

    International conference

    N. Crombez, G. Caron and E. Mouaddib, Using dense point clouds as environment model for visual localization of mobile robot, Best runner-up paper award, IEEE Int. Conf. on Ubiquitous Robots and Ambiant Intelligence, URAI'15, pp.40-45, Goyang, South Korea, Oct 2015. PDF

  • Photo-realistic recoloring of 3D point clouds:
    Here is a first result in order to cover a heritage building as the Amiens cathedral for coloring a preliminary 3D model (point cloud) with a high definition. To do so, photos are registered over point clouds got from laser scanners by implementing a pose direct computation.
    Recolored south portal Zoom on the Golden Virgin

    French journal

    Nathan Crombez, Guillaume Caron and El Mustapha Mouaddib, Colorisation de nuages de points 3D par recalage dense d'images numériques, Traitement du Signal, TS, 31(1-2):81-106, 2014.

    International workshop

    N. Crombez, G. Caron and E. Mouaddib, 3D point cloud model colorization by dense registration of digital images, 3D Virtual Reconstruction and Visualization of Complex Architectures, 3DARCH, ISPRS workshop, Avila, Spain, February 2015.

    French conference

    Nathan Crombez, Guillaume Caron and El Mustapha Mouaddib, Colorisation photo-réaliste de nuages de points 3D, Congrès des jeunes chercheurs en vision par ordinateur, ORASIS, Cluny, France, June 2013. PDF


  • 3D model based pose estimation exploiting mutual information:
    This work was made during my postdoc, from october 2010 to august 2011, in the Lagadic team of INRIA Rennes/IRISA. It focused on the achievement of a new method of model-based pose estimation using non-linear optimization. The pose is optimized maximizing the mutual information shared by a real image of a video/camera stream and the synthetic image obtained as the projection of the textured 3D model. The mutual information similarity criterion between images, being highly robust to various visual perturbations, has been developed in the PhD thesis of Amaury Dame for visual servoing, 2D plane tracking and introduced the mutual information based pose estimation (3D tracking).

    International journal

    Guillaume Caron, Amaury Dame, Eric Marchand, Direct model based visual tracking and pose estimation using mutual information, Image and Vision Computing, to appear in 2014. PDF

    French speaking conference

    Guillaume Caron, Amaury Dame, Eric Marchand, L'information mutuelle pour l'estimation visuelle directe de pose, french speaking congress on Pattern Recognition and Artificial Intelligence, RFIA'12, Lyon, France, january 2012. PDF

    International workshop

    Antoine Petit, Guillaume Caron, Hideaki Uchiyama, Eric Marchand, Evaluation of Model based Tracking with TrakMark Dataset, Workshop on Benchmark Test Schemes for AR/MR Geometric Registration and Tracking Method, TrakMark'11 at IEEE Int. Symp. on Mixed and Augmented Reality, ISMAR'11, Basel, Switzerland, october 2011. PDF

HAUT3D model-based tracking for omnidirectional stereovision

This is a result of calculating the pose using stereo omnidirectional sensor. This time, the information used is richer because it uses a 3D model composed of faces, edges and vertices to calculate the pose of the camera. This type of work was carried out in monocular omnidirectional vision by the Lagadic research team. We extended it to omnidirectional stereovision. Here is a video of tracking and pose computation of a box:
Other videos can also be downloaded: static sensor, mobile sensor.

International journal

Guillaume Caron, El Mustapha Mouaddib and Eric Marchand, 3D model based tracking for omnidirectional vision: a new spherical approach, Robotics and Autonomous Systems, 60(8):1056-1068, August 2012. PDF

International Conference

Guillaume Caron, Eric Marchand and El Mustapha Mouaddib, 3D Model Based Pose Estimation For Omnidirectional Stereovision, IEEE/RSJ International Conference on Intelligent RObots and Systems, IROS'09, St. Louis, Missouri, USA, October 2009. PDF

National Communication

Guillaume Caron, Eric Marchand and El Mustapha Mouaddib, Estimation non-linéaire de pose d’objet 3D par stéréovision , National Days on Robotics Research (Journées Nationales de la Recherche en Robotique) JNRR'09, Neuvy-sur-Barangeon, France, November 2009.

HAUTPoints-based pose estimation and calibration

  • In November 2008, I made a stay in the Lagadic team from IRISA/INRIA in Rennes and I worked with Eric Marchand. During this stay, I worked in the calibration and sensor pose calculation with a monocular omnidirectional sensor and then with the omnidirectional stereovision sensor. Here is a video for each case.
    The monocular pose computation was applied to the 3D external localization of a mobile robot. We consider a fisheye camera and equip the mobile robot with a target made of four infrared LEDs. The target pose, and hence the one of the robot, is then computed from these points. La pose de la balise, et donc celle du robot, est ensuite calculée à partir de ces points. This video shows the experimental setup and localization results:
    This video can also be downloaded.

  • Then, in 2009, I generalized the calibration to hybrid stereo rigs with Damien Eynard, particularly combining a perspective camera and a fisheye, the first modelled by the so called pinhole model and the other by the central unified projection model of Geyer/Barreto. The method allows to calibrate N cameras with N different models, intrinsically and extrinsically, simultaneously. Since then, we also integrated the ah hoc projection model of the paracatadioptric camera.

    International Conferences

    Guillaume Caron et Damien Eynard, Multiple Camera Types Simultaneous Stereo Calibration, IEEE International Conference on Robotics and Automation, ICRA'11, Shangai, Chine, Mai 2011. PDF
    Guillaume Caron, Eric Marchand and El Mustapha Mouaddib, Single Viewpoint Stereoscopic Sensor Calibration, International Symposium on Image/Video Communications over fixed and mobile networks, ISIVC'10, Rabat, Morocco, September 2010. PDF

    International publication

    Guillaume Caron and Damien Eynard, Hybrid stereoscopic calibration, SPIE newsroom in Electronic Imaging & Signal Processing, June 2011. article | PDF

    National Conference

    Guillaume Caron and Damien Eynard, Etalonnage simultané de systèmes stéréoscopiques hybrides, Young researchers in computer vision congress (Congrès des jeunes chercheurs en vision par ordinateur) ORASIS'11, Praz-sur-Arly, France, June 2011. PDF

    Software design

    HySCaS : Hybrid Stereoscopic Calibration Software

    This software is the implementation of the simultaneous intrinsic and extrinsic stereo calibration method that I proposed during my PhD thesis. This software has been developed with Damien Eynard. Stereo rigs with different camera models are calibrated by this cross-platform software (Linux, Mac, Windows) and developed in C++ with Qt, ViSP and OpenCV library.
    HySCaS is available from the software section of this website.