A Robust Inside-Out Approach for 3D Interaction with Large Displays

David Scherfgen, Rainer Herpers, Timur Saitov
10th IEEE Symposium on 3D User Interfaces (3DUI), page 137-140 - March 2015


This contribution presents several improvements over previous "inside-out" techniques for pointing interaction targeted at large and high-resolution displays. Fiducial markers are projected from an interaction device's built-in camera onto the displays, where they are transparently superimposed on the display content. We reconstruct the camera's full 6-DoF pose by detecting and tracking these markers in the camera images. For increased robustness, the marker pattern is dynamically adaptable. We address display lag and high pixel response times by precisely timing image captures. An image-based display calibration procedure enables the system to work with multiple non-coplanar displays. Results show that pointing locations are measured with sub-millimeter precision and the camera's 3D position with sub-centimeter precision. Update rates of up to 60 Hz and a 24 ms latency were achieved. Our technique performed comparably to an OptiTrack system in 2D target selection tasks.

This paper won the 2015 3DUI Best Technote Award.

Images and movies


BibTex references

  author       = {Scherfgen, David and Herpers, Rainer and Saitov, Timur},
  title        = {A Robust Inside-Out Approach for 3D Interaction with Large Displays},
  booktitle    = {10th IEEE Symposium on 3D User Interfaces (3DUI)},
  pages        = {137-140},
  month        = {March},
  year         = {2015},

Other publications in the database

» David Scherfgen
» Rainer Herpers
» Timur Saitov