3-D Video Processing, Compression and Display
Effortless, 3-D perception of surroundings is a natural ability for about 90% of the population, namely those who benefit from stereopsis, i.e., the capacity to fuse two viewpoints captured by the eyes. However, transmission and display systems to date reproduce almost exclusively two-dimensional (2-D) depth cues, such as perspective, o
cclusions, shading, etc. Although various 3-D system designs have attempted to take advantage of stereopsis, none have attained equal acceptance to their 2-D counterparts. Today, novel electronic display technologies, digital signal processing algorithms, and high-bandwidth networks have opened a new era for 3-D experience. 3-D cinema has taken a solid footing in the entertainment market. In scientific applications, a traditional 3-D market, 3-D hardware is becoming a commodity item. Perhaps most importantly, 3-D home entertainment is no more an elusive mirage, but more a clear target within our reach, and at its forefront is 3-D video gaming.
Yet challenges remain in the quest for a realistic, effortless “being there” experience. One challenge is in the development of new display technologies to allow high-quality, high-resolution 3-D experience without glasses. The second challenge is in focusing signal processing methods on the unique problems of 3-D visualization. The third challenge is in the area of compression for 3-D displays that require massive amounts of visual data. This research thrust focuses on the last two challenges, and particularly on enhancement, view generation and compression algorithms for automultiscopic 3-D displays, i.e., displays that do not require glasses while providing multiple perspectives (multiple views) to the viewer (look-around). Such displays hold a great promise for the future of 3-D visual communications. Below are listed some of the projects in this thrust:
- Automatic 2D-to-3D Image Conversion: Learning of 2D image depth from on-line examples
- Digital Signal Processing for Enhanced Automultiscopic 3-D Visualization: Enhancement methods for eyewear-free 3-D displays
- View Reconstruction and Compression for Eyewear-Free 3-D: Rendering of virtual views to allow multview display
- 3Tube – An Immersive Tube for You: 3-D video system without glasses on commodity hardware
- Digital Image Compositing Using Stereo: Novel image compositing that accounts for scene structure to permit illumination change
- Crosstalk Cancellation in Stereoscopic 3-D Displays: Real-time, low-complexity, low-memory crosstalk cancellation algorithm for liquid-crystal shutters
The video below shows a sequence of 9 frames, where the middle 7 frames were reconstructed from the first and last frames by means of intermediate view reconstruction.