3-D Video Processing, Compression and Display

Effortless, 3-D perception of surroundings is a natural ability for about 90% of the population, namely those who benefit from stereopsis, i.e., the capacity to fuse two viewpoints captured by the eyes.  However, transmission and display systems to date reproduce almost exclusively two-dimensional (2-D) depth cues, such as perspective, oin_action_smallcclusions, shading, etc.  Although various 3-D system designs have attempted to take advantage of stereopsis, none have attained equal acceptance to their 2-D counterparts. Today, novel electronic display technologies, digital signal processing algorithms, and high-bandwidth networks have opened a new era for 3-D experience. 3-D cinema has taken a solid footing in the entertainment market. In scientific applications, a traditional 3-D market, 3-D hardware is becoming a commodity item. Perhaps most  importantly, 3-D home entertainment is no more an elusive mirage, but more a clear target within our reach, and at its forefront is 3-D video gaming.

Yet challenges remain in the quest for a realistic, effortless “being there” experience. One challenge is in the development of new display technologies to allow high-quality, high-resolution 3-D experience without glasses. The second challenge is in focusing signal processing methods on the unique problems of 3-D visualization. The third challenge is in the area of compression for 3-D displays that require massive amounts of visual data. This research thrust focuses on the last two challenges, and particularly on enhancement, view generation and compression algorithms for automultiscopic 3-D displays, i.e., displays that do not require glasses while providing multiple perspectives (multiple views) to the viewer (look-around). Such displays hold a great promise for the future of 3-D visual communications. Below are listed some of the projects in this thrust:

The video below shows a sequence of 9 frames, where the middle 7 frames were reconstructed from the first and last frames by means of intermediate view reconstruction.