|
Next-Generation Methods for Image Sequence
Analysis, Processing, and Transmission
This research thrust concentrates around the idea that by processing multiple frames of an image sequence jointly, one can achieve not only better results,
e.g., more accurate segmentation or improved compression ratio, but also extract qualitatively new information, such as occlusion and newly-exposed area information. Following are the current projects in this thrust:
Joint space-time segmentation and analysis of video sequences
Team: M. Ristivojevic, J. Konrad, collaborators from the University of Nice, France
Funding: National Science Foundation (CISE-CCR-SPS, Intern. Collab. USA-Frrance)
Traditional
video processing methods use two image frames at a time to analyze such dynamics as motion, occlusions, etc. We explore new framework that is based on joint treatment of many image frames (e.g., 20-30). A form of
joint space-time processing, this framework is essentially three-dimensional (3-D) since its domain is the x-y-t space of image sequences. It is expected to result in more reliable video segmentation, detection of
occlusion effects and identification of various dynamic events. To date, we have developed a video segmentation method within this framework that is based on an active-surface model and level-set solution. Applied
to both synthetic and natural image sequences this method results in “object tunnels” in the x-y-t space, that we have used successfully to identify certain occlusion events and measure time instants of object
occlusions, disappearance, entry, etc. See the above link for more details and results.
Video sequence compression based on spatio-temporal transformations
Team: N. Bozinovic, J. Konrad, collaborators from the University of Nice, France
Funding: National Science Foundation (CISE-CCR-SPS, Intern. Collab. USA-France)
It is widely believed
in the research community today that the next significant video coding gains will come from a joint compression of multiple video frames, such as offered by 3D wavelet coding. We have been studying the behavior of
video data subject to various 3-D (x-y-t) transformations. We have developed a multiple-frame characterization of linear motion in the DCT domain. An extension of the fundamental result in Fourier domain, this
outcome allows us to better understand spectral composition of 3D-DCT-transformed video data. Based on this, we have developed 3D DCT coefficient scanning patterns that are more efficient than scans used to date,
and may lead to efficient 3D transform-based video coding techniques.
|
