Digital Image Compositing Using Stereo

pj_rendering005_fixed — Re-illumination of an image based on 3-D structure estimated from a stereopair

Team: P. McNerney, J. Konrad, M. Betke
Funding: Boston University
Status: completed (2001-2003)

Background: The perception of depth permits new modalities in human-computer communication. For example, traditional image compositing methods use exclusively photometric scene properties (luminance, color attributes, etc.) and disregard scene structure. This poses problems when combining two images with different illuminations. However, should the scene structure be known, a more accurate and transparent compositing should be possible.

Summary: In this project, we developed digital compositing methods based on depth queues. The main idea was to combine digital matting (segmentation) with depth recovery (disparity estimation) in order to obtain as accurate foreground object representation as possible. Having recovered the geometric (3-D) and photometric description of the object, it is possible to insert it into 3-D imagery with correct illumination, shading, etc. Possible applications of this research are in virtual studios, virtual walkthroughs, etc.

Publications:

P. McNerney, J. Konrad, and M. Betke, “Block-based MAP disparity estimation under alpha-channel constraints,” IEEE Trans. Circuits Syst. Video Technol., vol. 17, pp. 785-789, June 2007.