Spatio Temporal Background Modeling With Gaussian Mixture Model

Abstract

Background modeling algorithms are very popular in vision-based systems of both research and casual activities. Their range of applications are wide and have included not only video inpainting, video privacy protection, and computational photography, but also foreground extractions and motion analytic tasks such as those in human behavior monitoring systems, or observations of animal or insect behaviors. This prevalence can be clearly observed among statistical background modeling methods such as those employing the Gaussian Mixture Models (GMM). Not only can their advantages be attributed to their efficient processing on clear mathematical foundations for fair context modeling, but their formulation also entails pixel-wise generalistic adaptation to scene dynamics, which has been demonstrably successful across many cases. However, by the lack of knowledge about scene semantics, their indiscriminately generalistic adaptiveness is plauged by a dilemma between incorporation of true background variations and avoidance of long-lived foreground corruptions. Furthermore, while many modern technological advances have been proposed regarding parallel paradigms of computing on data-driven learning, statistical GMM methods generally follow only sequential computing models even on high image resolution. In this thesis, we propose a tensordriven framework of statistical GMM background modeling called TensorMoG, along with its extensions in considerations of the aforementioned issues. Specifically, from TensorMoG’s explicit parallelism for background modeling on videos, we introduce High-Variational Removal (HVR) and Adaptive Learning (ADA) plug-ins to maintain model robustness against signal noises and allow for adaptiveness in different manners. Then, in order to explicitly institute scene semantics with spatio-temporal consistency to the domain for context selective update, we adopt into TensorMoG a generalizable, data-driven maintenance component called Neural Poisson Inpainter (NPI). NPI is used to inpaint error regions by solving Poisson problems on images, and thus maintain a semi-supervised, spatio-temporally stable background modeling process on GMM. Our results with indicate decent quality in background modeling with our proposed approaches with real-time processing on GPU.