Image Processing and Compression 3-D Visual Communications

This research thrust focuses on enhancement, view generation and compression algorithms for automultiscopic 3-D displays, i.e., displays that do not require glasses while providing multiple perspectives (multiple views) to the viewer (look-around). Such displays have recently been introduced on the market and hold a great promise for the future of 3-D visual communications. Following are the current projects in this thrust:

Digital signal processing for enhanced automultiscopic 3-D visualization (2002-2006)
Team: P. Agniel, J. Konrad
Funding: National Science Foundation (ITR)
Next-generation display systems will include depth in order to invoke a “being there” experience. This can be achieved by presenting two views on a 3-D stereoscopic display (anaglyph, polarized, shuttered, autostereoscopic). Our research in this project is stimulated by a new generation of advanced automultiscopic (multiview, no glasses) 3-D displays that have been recently introduced on the market, such as monitors from Stereographics Corp. and 4-D Vision GmbH (currently X3D Corp.). We are working on optimal multiplexing of views in order to invoke as natural 3-D perception as possible. One of the issues is anti-alias pre-filtering of each view. Since individual views are sampled irregularly during multiplexing, we use 2-D lattice approximations and derive anti-alias filter specifications. Application of such filters results in reduced aliasing and improved 3-D perception.

Reconstruction and compression of views for eyewear-free 3-D image communication (2002-2007)
Team: S. Ince, J. Konrad
Funding: National Science Foundation (ITR)
As new automultiscopic displays begin to deliver acceptable 3-D experience, difficulties arise with data generation and delivery. In order to take full advantage of such displays, multiple views (as many as 10-15) need to be generated, but typical acquisition systems use 2-3, perhaps 5, cameras. The issue is of intermediate view reconstruction (generation) which is a difficult, ill-posed problem. Once all the views are generated they may need to be delivered remotely via a communications channel and thus compression may be needed. We are currently working on both problems with the ultimate goal of posing and solving a joint problem of view reconstruction and compression.

Digital image compositing using stereo (2001-2003)
Team: P. McNerney, J. Konrad, M. Betke
Funding: Boston University
The perception of depth permits new modalities in human-computer communication. In this project, we develop digital compositing methods based on stereoscopic imagery. The main idea is to combine digital matting (segmentation) with depth recovery (disparity estimation) in order to obtain as accurate foreground object representation as possible. Having recovered the geometric (3-D) and photometric description of the object, it is possible to insert it into 3-D imagery with correct illumination, shading, etc. Possible applications of this research are in virtual studios, virtual walkthroughs, etc.