View Reconstruction and Compression for Eyewear-Free 3-D

Team: S. Ince, J. Konrad
Funding: National Science Foundation (ITR)
Status: Completed (2002-2007)

Background: As new automultiscopic displays begin to deliver acceptable 3-D experience, difficulties arise with data generation and delivery. In order to take full advantage of such displays, multiple views (as many as 9-15) need to be generated, but typical acquisition systems typically use 2-3 cameras. The issue is of intermediate view reconstruction (generation) which is a difficult, ill-posed problem. Once all the views are generated they may need to be delivered remotely via a communications channel and thus compression may be needed.

Summary: We have developed a number of solutions to the problem of view reconstruction, and, in the process, to the problem of disparity estimation in presence of occlusions. Our solutions range from simple approaches based on matching to more complex variational solutions. We have also studied multiview image compression for automultiscopic 9-view 3-D displays.

The videos below contain 9 frames computed from the first and last frames by means of intermediate view reconstruction. Played as a video sequence this intermediate frames result in a “look-around” sensation.

Publications:

• S. Ince and J. Konrad, “Recovery of a missing color component in stereo images (or helping NASA find little green martians),” in Proc. SPIE Stereoscopic Displays and Virtual Reality Systems, vol. 5664, pp. 127-138, Jan. 2005.
• S. Ince and J. Konrad, “Geometry-based estimation of occlusions from video frame pairs,” in Proc. IEEE Int. Conf. Acoustics Speech Signal Processing, vol. II, pp. 933-936, Mar. 2005.
• S. Ince, J. Konrad, and C. Vázquez, “Spline-based intermediate view reconstruction,” in Proc. SPIE Stereoscopic Displays and Virtual Reality Systems, vol. 6490, pp. 0F.1-0F.12, Jan. 2007.
• R. Lau, S. Ince, and J. Konrad, “Compression of still multi-view images for 3-D automultiscopic spatially-multiplexed displays,” in Proc. SPIE Stereoscopic Displays and Virtual Reality Systems, vol. 6490, pp. 0O.1-0O.9, Jan. 2007.
• S. Ince and J. Konrad, “Occlusion-aware optical flow estimation,” IEEE Trans. Image Process., vol. 17, pp. 1443-1451, Aug. 2008.
• S. Ince and J. Konrad, “Occlusion-aware view interpolation,” EURASIP J. Image and Video Process., vol. doi:10.1155/2008/803231, 2008.