Team: P.-M. Jodoin, V. Saligrama, J. Konrad
Funding: National Science Foundation (CISE-CCF-CIF)
Status: Ongoing (2008-…)
Background: In order to realize the goal of autonomous visual sensor network, i.e., camera network that does not require round-the-clock human operator involvement, video analysis methods need to be developed capable of detecting various activities, especially abnormalities. This is often referred to video as analytics. The simplest form of video analytics, background subtraction, may be sufficient for restricted-area access control where a mere detection of movement is sufficient. However, it may not be sufficient for the detection of unusual behavior. Thus, the question one needs to answer is: How to distinguish normal motion patterns from abnormal ones?
Summary: Using the background detection results, we have developed an abnormal behavior detection framework that can detect abnormally high activity, departure from typical activity, etc. all within the same formulation. By aggregating motion labels for a given pixel from all frames of a training sequence, we build a low-dimensionality representation against which we test an aggregate computed from the observed sequence. If the observed aggregate eitehr exceeds or departs from the training aggregate, abnormality is declared. This simple strategy leads to remarkably robust results as demonstrated in video sequences below.
Results: Below are shown some experimental results produced by our method. Each video below shows the original sequence, the result of background substraction, and the result of behavior subtraction tuned to detect abnormally high activity. Click on any of the images below to play a video file in Windows Media format (.wmv). Mac users may need to download Windows Media® Components for QuickTime.
- P.-M. Jodoin, V. Saligrama, and J. Konrad, “Behavior subtraction,” in Proc. SPIE Visual Communications and Image Process., vol. 6822, pp. 10.1-10.12, Jan. 2008.
- P.-M. Jodoin, J. Konrad, and V. Saligrama, “Modeling background activity for behavior subtraction,” in ACM/IEEE Int. Conf. Distributed Smart Cameras, Sept. 2008.
- P.-M. Jodoin, J. Konrad, V. Saligrama, and V. Veilleux-Gaboury, “Motion detection with an unstable camera,” in Proc. IEEE Int. Conf. Image Processing, Oct. 2008.
- V. Saligrama, J. Konrad, and P.-M. Jodoin, “Video anomaly detection: A statistical approach,” in IEEE Signal Process. Mag., vol. 27, pp. 18-33, Sept. 2010.
- P.-M. Jodoin, V. Saligrama, and J. Konrad, “Behavior subtraction,” IEEE Trans. Image Process., vol. 21, pp. 4244-4255, Sept. 2012.