WO2018058854A1 - 一种视频的背景去除方法 - Google Patents

一种视频的背景去除方法 Download PDF

Info

Publication number
WO2018058854A1
WO2018058854A1 PCT/CN2017/070219 CN2017070219W WO2018058854A1 WO 2018058854 A1 WO2018058854 A1 WO 2018058854A1 CN 2017070219 W CN2017070219 W CN 2017070219W WO 2018058854 A1 WO2018058854 A1 WO 2018058854A1
Authority
WO
WIPO (PCT)
Prior art keywords
background
block
foreground
model
blocks
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2017/070219
Other languages
English (en)
French (fr)
Inventor
李革
臧祥浩
王文敏
王荣刚
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Peking University Shenzhen Graduate School
Original Assignee
Peking University Shenzhen Graduate School
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Peking University Shenzhen Graduate School filed Critical Peking University Shenzhen Graduate School
Priority to US16/089,493 priority Critical patent/US10297016B2/en
Publication of WO2018058854A1 publication Critical patent/WO2018058854A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N7/00—Computing arrangements based on specific mathematical models
    • G06N7/01—Probabilistic graphical models, e.g. probabilistic networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/11—Region-based segmentation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/174—Segmentation; Edge detection involving the use of two or more images
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/194—Segmentation; Edge detection involving foreground-background segmentation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20021—Dividing image into blocks, subimages or windows
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20212—Image combination
    • G06T2207/20224—Image subtraction
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30232—Surveillance

Definitions

  • the present invention relates to the field of video analysis technologies, and in particular, to a background removal method based on an image block, a Gaussian mixture model, and a random process.
  • Background removal of the video refers to separating the foreground from the background object from the video.
  • video is mainly divided into two categories: one is a video with a fixed background, and the other is a video with a constantly changing background.
  • the video of the fixed picture mainly includes video fixed by the camera such as surveillance video.
  • the ever-changing video in the background mainly includes TV programs, DVR videos and so on.
  • the present invention is mainly directed to a first type of video, that is, a video of a fixed picture background.
  • the biggest application of this type of video is surveillance video that is widely distributed in streets and elevators. Because of its 24-hour recording, the amount of data is extremely large, but the useful information is very small. Therefore, it has always been a very important step in the field of video analysis.
  • the existing technologies mainly include the following:
  • the method based on the frame difference method. This is the easiest way to judge the appearance of foreground objects by comparing the differences between the frames before and after. Therefore, the biggest problem of the frame difference method is the background update method, and the difference between the frame and the previous frame difference, and only the frame difference method is used to achieve a very good algorithm.
  • the present invention provides a new video background removal method, which implements video background removal based on an image block, a Gaussian mixture model, and a random process.
  • the method of the invention can be applied to video pre-processing of a fixed picture, thereby obtaining a foreground picture, which is convenient for subsequent video analysis.
  • the principle of the invention is that the method of the invention combines the frame difference method idea, the Gaussian mixture model and the stochastic process, and utilizes the tradition
  • the idea of the frame difference method is to obtain the foreground background by the difference between frames, the difference is that the previous methods are to compare the differences between pixels.
  • the method of the present invention employs the difference between pixel sums, i.e., image blocks. We model each image block using a Gaussian mixture model and automatically generate thresholds. We use the idea of a random process to randomly update the background of the image to eliminate small noise.
  • the invention mainly adopts an image block-based frame difference method, and automatically adjusts the threshold of the frame difference by using a Gaussian mixture model, and updates the background by means of a random process.
  • the method of the present invention is tested on the real video of the BMC (Background Modeling Challenge) data set, and the effect is beyond most current methods.
  • a method for removing a video background the image is divided into blocks of the same size, and the value of the block is the sum of the pixel values in the block; the Gaussian mixture model is modeled by the history of the blocks in the video, and the standard deviation is automatically adjusted.
  • the threshold is used to compare the differences between the blocks to obtain the foreground and the background; in addition, the background is randomly updated to eliminate the false object of the foreground object; the specific steps include the following steps:
  • the background can be updated in three ways; firstly, the first frame image is used to initialize the background; the background here and the background mentioned below refer to the value of a block; the first way is to use adjacent blocks The difference between the differences is less than the threshold and the length is exceeded
  • the mean of the blocks is the background, where a set value of the length of a continuous block sequence having similar values, in the present invention Set to 5, indicating the length of the block with similar values; the second way is to use continuous over The mean of the block is the background; the third way is to use a random piece around the background, if this block is foreground and continuously exceeds Then the mean of the consecutive foregrounds in the history of the block is represented as the background;
  • the foreground can be updated in two ways; the first way is to compare the difference between adjacent blocks before and after, if the difference is greater than the threshold in step 2), the current block is a potential foreground, where the potential prospect is not necessarily the foreground Second, the difference between the current block and the background is compared. If the difference is greater than the threshold, the current block is the potential foreground. Then, the pixels in all the processed blocks are averaged, and the obtained values are obtained. The pixels in the potential foreground block are subtracted, and the difference is ⁇ , and the difference is passed through low-pass filtering (LPF). If the passable value is less than half of the pixels, It is the foreground F.
  • LPF low-pass filtering
  • the invention provides a novel background removal method, which can freely set the size of the block, especially when the video size is large, and the acceleration effect is very obvious.
  • different rectangles can be set. This better adapts to the object of interest.
  • the invention adopts a Gaussian mixture model to automatically generate a threshold, so that the updated threshold can better adapt to changes in the environment.
  • the invention can be applied to the video pre-processing of the fixed picture, thereby obtaining the foreground picture, and laying a good foundation for the future video analysis work.
  • the present invention achieves the best effect that the current method can achieve in the real data portion of the BMC data set.
  • the real data includes parking lots, highways, highways, railways, subways, home gardens, etc., including day and night, rain, snow and wind. This demonstrates that the invention has broad applicability and achieves very high accuracy.
  • FIG. 1 is a flow chart of a video background removal method provided by the present invention.
  • 1 - use the block to quickly establish the background; 2 - use the foreground to update the background; 3 - use the background to update the foreground.
  • FIG. 2 is a diagram showing an example of removing a background from an original video in an embodiment of the present invention.
  • Background subtraction has very important applications in the field of video analytics, especially video surveillance. Compared to other video resources, surveillance video has some unique features, such as stable images and similar textures. Based on this, the present invention proposes a novel background removal method.
  • the concept of a block is defined, the image is divided into blocks of the same size, the value of the block is the sum of the pixel values in the block; the foreground and the background are determined by comparing the differences between the blocks; the threshold is automatically adjusted using the Gaussian mixture model At the same time, the idea of random process is used to update the background.
  • experiments on the BMC (Background Models Challenge) dataset show that our method surpasses most of the current advanced algorithms.
  • the invention provides a novel background removal method, which can freely set the size of the block, especially when the video size is large, and the acceleration effect is very obvious.
  • different rectangles can be set. This better adapts to the object of interest.
  • the invention adopts a Gaussian mixture model to automatically generate a threshold, so that the updated threshold can better adapt to changes in the environment.
  • 1 is a block diagram of a video background removal method provided by the present invention; the method includes dividing an image into blocks of the same size, the value of the block is the sum of pixel values in the block; and modeling the Gaussian mixture model by the history of blocks in the video. And use its standard deviation to automatically adjust the threshold; use the threshold to compare the differences between the blocks to get the foreground background.
  • the background is randomly updated to eliminate false objects in the foreground; specifically, the following steps are included:
  • each frame of image into a grayscale image.
  • the size of each block is w*h
  • the size of each frame image is W*H
  • m W/w
  • the value of the block ⁇ as the sum of the pixels p i,j in the block, expressed as Equation 1:
  • ⁇ i,t represents the estimated weight of the i-th Gaussian model at time t;
  • ⁇ i,t is the mean, ie the center of the model;
  • ⁇ i,t is the standard deviation.
  • each model in the hybrid model was sort by ⁇ / ⁇ .
  • Each value v t is used to match the mixed model; matching is to compare each value v t with the center of each model, that is, the weight will increase when the model gets more matches; the more the match, the variance The smaller it will be, the higher the front row will be.
  • V k,t is 1 when the match is successful, and 0 otherwise.
  • the updated weights need to be re-regularized to ensure that the sum of all weights is 1.
  • the biggest advantage of this method is that when a new thing appears, the original Gaussian mixture model will not be destroyed but only updated.
  • the first model tends to have higher weights and lower variance, and we use three times the standard deviation of the first Gaussian model (3 ⁇ 1 ) as the threshold. This ensures that our algorithm is highly sensitive to new things.
  • l refers to the length of the connected foreground block. If there is more than here, we use the foreground average Update the background. In the actual scene, there will be some objects to stop after the movement, such as cars, animals, people, etc., our algorithm will treat these objects as background processing.
  • the current block is regarded as the potential foreground F c .
  • the current block is also considered as the potential foreground F c .
  • Potential prospects here may not be identified as prospects. The reason for this is the sudden change of light in the video, such as the opening of the car lights, the lighting of the street lights, so that each pixel value in the block will increase, the value of the block will increase a lot, and the light is not the foreground. object.
  • LPF can even filter out shadows at the foot of pedestrians.
  • 2 is a diagram showing an example of removing the background from the original video in the embodiment of the present invention, wherein the upper image is the original video image, and the lower image is the foreground image obtained by removing the background by the method of the present invention.
  • Table 1 is the test results of the method of the present invention and the other six more advanced algorithms compared to the BMC dataset.
  • the leftmost cell of the table is [7] (Andrews Sobral and Antoine Vacavant, "A comprehensive review of background subtraction algorithms evaluated with synthetic and real videos," Computer Vision and Image Understanding, vol. 122, pp. 4-21,
  • the six methods mentioned in 2014) including the five methods of Top5 on the BMC dataset and one method of Top1 in the Change Detection Challenge 2014
  • the method proposed by the present invention the top row of cells is in the BMC dataset.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Computational Mathematics (AREA)
  • Mathematical Optimization (AREA)
  • Artificial Intelligence (AREA)
  • Probability & Statistics with Applications (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Mathematical Analysis (AREA)
  • Algebra (AREA)
  • Pure & Applied Mathematics (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Image Analysis (AREA)

Abstract

公布了一种视频的背景去除方法，涉及视频分析技术领域，尤其涉及一种基于图像块、高斯混合模型、随机过程的背景去除方法；首先定义了块的概念，通过比较块之间的不同来确定前景与背景；使用高斯混合模型来自动的调整阈值，同时采用随机过程的思想来更新背景；最后在BMC数据集上做实验，结果显示本方法超越了目前大多数先进算法，准确度非常高。本方法具有广泛的适用性，可适用于监控视频的背景减除，在视频分析领域具有非常重要的应用。

Description

一种视频的背景去除方法 技术领域
本发明涉及视频分析技术领域，尤其涉及一种基于图像块、高斯混合模型、随机过程的背景去除方法。
背景技术
视频的背景去除指将视频中的前景与背景物体分别开来。目前视频主要分为两类：一类是固定画面背景的视频，一类是背景不断变化的视频。固定画面的视频主要包括监控视频等摄像头固定的视频。背景不断变化的视频主要包括电视节目、DVR录像等等。本发明主要针对第一类视频，即固定画面背景的视频。这一类视频最大的应用是广泛分布于大街小巷、电梯路口的监控视频。因为其24小时摄录的原因，导致数据量极大，但是有用的信息却少之又少。因而一直以来是视频分析领域非常关键的一个步骤。
现有的技术主要包含以下几种：
一、基于帧差法的方法。这是最简单的方式，通过比较前后帧画面的差异，判断前景物体的出现。因此帧差法最大的问题在于背景的更新方法，与前后帧差异的差别，仅利用帧差法鲜有达到很好效果的算法。
二、基于高斯混合模型的方法。这是上世纪90年代提出的方法，通过高斯混合模型对每一个像素点进行建模，并利用期望最大化算法对模型在线更新，最后通过比较当前像素与模型的差异来决定该像素是否为前景。该方法由于对每个像素的三通道分别进行建模，造成较大的计算量，难以应对当前越来越大的图像尺寸。
三、基于随机过程的方法。21世纪初，有学者提出了基于随机采样的方法。对每一帧图像像素随机的采样作为背景，并随机的更新背景，这种算法利用随机过程的思想，可以得到较好的效果，但这种算法依然有可提高的余地。
发明内容
为了克服上述现有技术的不足，本发明提供一种新的视频背景去除方法，基于图像块、高斯混合模型、随机过程，实现视频背景去除。本发明方法可应用于固定画面的视频前处理，以此得到前景画面，便于后续进行视频分析。
本发明的原理是：本发明方法融合了帧差法思想、高斯混合模型和随机过程，利用传统 的帧差法的思想，通过帧之间的不同来得到前景背景，不同之处在于，之前的方法都是通过比较像素之间的不同。而本发明方法是采用像素和，即图像块之间的不同。我们采用高斯混合模型对每一个图像块进行建模，并自动产生阈值。我们采用随机过程的思想，对图像的背景进行随机更新，这样可以消除掉小的噪声。
本发明主要采用基于图像块的帧差法，并利用高斯混合模型对帧差的阈值进行自动调节，利用随机过程的方式对背景进行更新。本发明方法在BMC(Background Modeling Challenge)数据集的真实视频上做测试，其效果超出了当前大多数方法。
本发明提供的技术方案是：
一种视频背景的去除方法，将图像分为相同大小的块，块的值为块内像素值的加和；通过视频中块的历史来进行高斯混合模型建模，并利用其标准差自动调节阈值；利用阈值比较块之间的差别，以此得到前景和背景；除此之外，还对背景进行随机的更新，以此来消除误检的前景物体；具体包括如下步骤：
1)设定块的大小为w*h，然后对每一帧图像进行灰度化，图像大小为W*H，这样灰度化的图像可分为m*n块。每一块的值定义为块内像素的加和；
2)将连续帧中同一位置的块看做是块的历史{ν1，ν2，…，νn}，以下所有讨论仅限于块及块的历史；用高斯混合模型对块的历史进行建模，对混合模型中的各个模型，用各个模型的ω/σ对模型进行排序，采用排名最高的模型的三倍标准差3σ1为阈值，保持阈值的不断更新；
3)更新背景：可采用三种方式来更新背景；首先用第一帧图像来初始化背景；这里的背景及下文中提到背景指一个块的值；第一种方式是，采用相邻前后块之间差值小于阈值且长度超过
Figure PCTCN2017070219-appb-000001
的块的均值为背景，其中，
Figure PCTCN2017070219-appb-000002
为值相近的连续块序列的长度的设定值，在本发明中
Figure PCTCN2017070219-appb-000003
设定为5，表示值相近的块的长度；第二种方式是，采用连续超过
Figure PCTCN2017070219-appb-000004
的块的均值为背景；第三种方式是，采用背景周围随机的一块，若这一块为前景，且连续超过
Figure PCTCN2017070219-appb-000005
则将该块的历史中连续的前景的均值表示为背景；
4)更新前景：首先设置潜在前景，再更新为最终前景；
可采用两种方式来更新前景；第一种方式是，通过比较相邻前后块的差异，若差值大于步骤2)所述阈值，则当前块为潜在前景，这里潜在前景指不一定是前景，需要进一步判断；第二种方式，通过比较当前块与背景的差异，若差值大于阈值，则当前块为潜在前景；然后，平均化所有处理过的块内的像素，将得到的值与潜在前景块内的像素做减法，得到的差值为ψ，将差值通过低通滤波(LPF)，若可通过的值少于一半的像素，即
Figure PCTCN2017070219-appb-000006
即为前景F。
与现有技术相比，本发明的有益效果是：
本发明提供了一种新颖的背景去除方法，可以自由设置块的大小，尤其在视频尺寸较大时加速效果非常明显，除此之外，对于不同形状的感兴趣物体，可以设置不同的矩形，这样更好的适应了感兴趣物体。本发明采用高斯混合模型可以自动生成阈值，这样更新的阈值可以更好的适应环境的变化。我们还利用随机过程的思想，更新背景，这样背景会不断的侵蚀前景物体，直到前景物体消失，对于小的噪声前景可以很快的将其消除。本发明可应用于固定画面的视频前处理，以此得到前景画面，为以后的视频分析工作打下一个好的基础。
与现有技术相比，本发明在BMC数据集的真实数据部分，取得了目前方法所能够达到的最好效果。真实数据包含了停车场、高速公路、公路、铁路、地铁、家庭院落等，包含白天夜晚、下雨下雪大风等天气。这就说明了本发明具有广泛的适用性，并且取得了非常高的准确度。
附图说明
图1是本发明提供的视频背景去除方法的流程框图；
其中，1—使用块快速建立背景；2—使用前景更新背景；3—使用背景更新前景。
图2是本发明实施例中的由原视频去除背景后的的示例图。
具体实施方式
下面结合附图，通过实施例进一步描述本发明，但不以任何方式限制本发明的范围。
背景减除在视频分析领域具有非常重要的应用，尤其是视频监控。相比其他视频资源，监控视频具有一些独特的特征，比如画面稳定，纹理相似等。基于此，本发明提出了一种新颖的背景去除方法。首先定义了块的概念，将图像分为相同大小的块，块的值为块内像素值的加和；通过比较块之间的不同来确定前景与背景；使用高斯混合模型来自动的调整阈值，同时采用随机过程的思想来更新背景；最后在BMC(Background Models Challenge)数据集上做实验，结果显示我们的方法超越了目前大多数先进算法。
本发明提供了一种新颖的背景去除方法，可以自由设置块的大小，尤其在视频尺寸较大时加速效果非常明显，除此之外，对于不同形状的感兴趣物体，可以设置不同的矩形，这样更好的适应了感兴趣物体。本发明采用高斯混合模型可以自动生成阈值，这样更新的阈值可以更好的适应环境的变化。我们还利用随机过程的思想，更新背景，这样背景会不断的侵蚀前景物体，直到前景物体消失，对于小的噪声前景可以很快的将其消除。
图1是本发明提供的视频背景去除方法的流程框图；包括将图像分为相同大小的块，块的值为块内像素值的加和；通过视频中块的历史来进行高斯混合模型建模，并利用其标准差自动调节阈值；利用阈值比较块之间的差别，以此得到前景背景。除此之外，还对背景进行随机的更新，以此来消除误检的前景物体；具体包括如下步骤：
第一，我们将每一帧图像都转化为灰度图。我们假定每一个块的大小为w*h，每一帧图像的大小为W*H，那么每一帧由m*n块组成，其中m＝W/w，n＝H/h。我们使用一个矩阵表示每一帧，矩阵的每一个元素表示一个块的值。我们定义块的值ν为块内像素pi,j的加和，表示为式1：
Figure PCTCN2017070219-appb-000007
第二，建立高斯混合模型。我们使用一系列的块{ν1，ν2，…，νn}建立K个高斯模型的混合。K个高斯模型的加权得到混合高斯模型。以高斯混合模型表示当前块值的概率，记为P(vt)，用式2表示：
Figure PCTCN2017070219-appb-000008
这里，ωi,t表示第i个高斯模型在时间t的估计权重；μi,t是均值，即模型的中心；σi,t是标准差。
我们对混合模型中的各个模型以ω/σ进行排序。每一个值vt都会用来与混合模型进行匹配；匹配是将每一个值vt与各模型的中心即均值进行比较，当模型得到更多匹配时权重会增大；而越匹配，方差就会越小，这样排的就会越靠前。我们以混合模型中的某个模型的3σ作为阈值，来判断是否匹配成功，即当|vt-μi,t|＜3σ时表示vt与该模型匹配成功。若与所有模型都匹配不成功，则删除排名最后的模型，并新建以vt为中心的模型，添加到混合模型的最后面。当匹配成功时，更新以下三个参数：权重ωi,t、均值μ和标准差σ。
对于权重ωi,t，我们用如下等式(式3)进行更新：
ωi,t＝(1-α)ωi,t-1+αVk,t          (式3)
其中，α是学习率，当匹配成功时Vk,t为1，否则为0。经过更新后的权重需要重新规则化，以保证全部权重的加和为1。
对于没有匹配到的模型，参数μ，σ都保持不变。若匹配到，则两个参数需要按如下方式进行更新：
μt＝μt-1+ρt(vt-μt-1)          (式4)
Figure PCTCN2017070219-appb-000009
其中
ρt＝α/ωt-1          (式6)
这里采用式6这种近似的方法，而没有采用ρt＝P(vt)，可以简化计算，同时获得了与之前相近的性能。
这种方法最大的优势在于，当一个新的事物出现时，原有高斯混合模型并不会被破坏而仅仅是被更新。除此之外，最前面的模型往往具有较高的权重和较低的方差，我们使用第一个高斯模型的标准差的三倍(3σ1)作为阈值。这样就保证了我们的算法对新的事物具有较高的敏感度。
第三，更新背景。我们首先用第一帧图像初始化背景。这里的背景以及下面所讨论的背景均指一个图像块的值。我们可通过下面这三种方式来更新背景。
31)快速的建立背景。通过块序列中，长度超过
Figure PCTCN2017070219-appb-000010
的连续子序列(连续的块序列)，且前后块之间的差值小于阈值，则用这些块的均值来更新背景。如式7所示：
Figure PCTCN2017070219-appb-000011
这里，di-1,i＝|vi-vi-1|。这样更新之后，我们将
Figure PCTCN2017070219-appb-000012
变为
Figure PCTCN2017070219-appb-000013
的20倍，即
Figure PCTCN2017070219-appb-000014
这就意味着，我们再次更新需要超过
Figure PCTCN2017070219-appb-000015
个相近的连续块。我们这样做的目的是为了在得到背景后保持一个较低的更新速度。
32)使用前景更新背景。这里的前景Ft同背景一样，仅指一个块值。我们更新前景均值
Figure PCTCN2017070219-appb-000016
如式8所示：
Figure PCTCN2017070219-appb-000017
这里，l指连接前景块的长度。如果这里有超过
Figure PCTCN2017070219-appb-000018
的连续前景块，我们就用前景均值
Figure PCTCN2017070219-appb-000019
更新背景。在实际的场景中，会有一些物体运动之后停下来，比如汽车、动物、人等，我们的算法会把停下来的这些物体当做背景处理。
33)使用背景更新背景。我们采用随机过程的思想。对于每一个背景块，随机选择其中的一个相邻块，如图所示。假设这个相邻块是前景块，并且长度已经超过
Figure PCTCN2017070219-appb-000020
这里
Figure PCTCN2017070219-appb-000021
是
Figure PCTCN2017070219-appb-000022
的4倍大小。那么使用前景均值更新背景。这就是意味着们可以更快的更新背景。一些小的噪声就可以被背景逐渐吃掉。因此这个方法可以保留大的物体，而放弃小的。
第四，更新前景。我们采用与帧差法类似的思想，如果相邻前后块的差异di-1,i＜3σ1，则将当前块视为潜在前景Fc。除此之外，如果当前块与背景之间的差异超过3σ1，则同样将当前块视为潜在前景Fc。这里潜在前景指可能不被认定为前景。出现这种情况的原因是视频中突变的光照，比如汽车车灯的开启，路灯的亮起，这样块内的每个像素值都会增大，块的值一下增大很多，而光并不是前景物体。
为解决这个问题，我们采用基于像素级别的运算。对于所有运算过的像素，我们将他们块的像素平均化，然后得到当前块内像素与这些平均像素的差值ψ，将这个差值通过低通滤波器LPF，如果有少于一半的像素通过，我们就认为当前块是前景。
Figure PCTCN2017070219-appb-000023
通过调整LPF，我们可以改变算法对光照的敏感度。比如，通过LPF甚至可以过滤掉行人脚下的影子。图2是本发明实施例中的由原视频去除背景后的示例图，其中上图为原视频图像，下图为采用本发明方法去除背景后得到的前景图。
表1 不同方法在BMC数据集上的测试结果
Figure PCTCN2017070219-appb-000024
最后，本方法在BMC数据集的9个实际视频中做实验，表1是本发明方法和与之对比的其他六个比较先进的算法在BMC数据集上的测试结果。其中，表格最左侧单元格为文献[7](Andrews Sobral and Antoine Vacavant,“A comprehensivereview of background subtraction algorithms evaluated with synthetic and real videos,”Computer Visionand Image Understanding,vol.122,pp.4–21,2014)中提到的六种方法(包括在BMC数据集上Top5的五种方法和在Change Detection Challenge 2014上Top1的一种方法)及本发明提出的方法，最上一行单元格为BMC数据集中的9个真实场景监控视频的编号1～9和平均FSD。这里所有的分值都是FSD计分，FSD是表示背景去除算法中的评估方法，
Figure PCTCN2017070219-appb-000025
表示FSD的加权均值。结果表明，本发明方法的效果超过了绝大多数最佳方法。在BMC数据集上Top5的五种方法和在Change Detection Challenge 2014上Top1的一种方法分别记载在以下相应文献中：
[1]Martin Hofmann,Philipp Tiefenbacher,and GerhardRigoll,“Background segmentation with feedback:Thepixel-based adaptive segmenter,”in 2012IEEE ComputerSociety Conference on Computer Vision and PatternRecognition Workshops.IEEE,2012,pp.38–43.
[2]Jian Yao and Jean-Marc Odobez,“Multi-layer backgroundsubtraction based on color and texture,”in2007IEEE Conference on Computer Vision and PatternRecognition.IEEE,2007,pp.1–8.
[3]Lucia Maddalena and Alfredo Petrosino,“A selforganizingapproach to background subtraction for visualsurveillance applications,”IEEE Transactions onImage Processing,vol.17,no.7,pp.1168–1177,2008.
[4]Christopher Richard Wren,Ali Azarbayejani,TrevorDarrell,and Alex Paul Pentland,“Pfinder:Real-timetracking of the human body,”IEEE Transactions onpattern analysis and machine intelligence,vol.19,no.7,pp.780–785,1997.
[5]Pakorn KaewTraKulPong and Richard Bowden,“Animproved adaptive background mixture model for realtimetracking with shadow detection,”in Video-basedsurveillance systems,pp.135–144.Springer,2002.
[6]Rui Wang,Filiz Bunyak,Guna Seetharaman,and KannappanPalaniappan,“Static and moving object detectionusing flux tensor with split gaussian models,”in Proceedings of the IEEE 
Conference on ComputerVision and Pattern Recognition Workshops,2014,pp.414–418.
需要注意的是，公布实施例的目的在于帮助进一步理解本发明，但是本领域的技术人员可以理解：在不脱离本发明及所附权利要求的精神和范围内，各种替换和修改都是可能的。因此，本发明不应局限于实施例所公开的内容，本发明要求保护的范围以权利要求书界定的范围为准。

Claims (8)

  1. 一种视频背景的去除方法，将图像分为相同大小的块，块的值为块内像素值的加和；通过视频中块的历史来进行高斯混合模型建模，并利用其标准差自动调节阈值；利用阈值比较块之间的差别，得到前景和背景；还对背景进行随机的更新，以消除误检的前景物体；具体包括如下步骤：
    1)设定块的大小为w*h，对每一帧图像进行灰度化，所述图像大小为W*H，将灰度化的图像分为m*n块；每一块的值定义为块内像素的加和；
    2)将连续帧图像中同一位置的块记作块的历史{ν1，ν2，…，νn}，用高斯混合模型对块的历史进行建模，对混合模型中的各个模型进行排序，设定阈值并利用标准差保持阈值的不断更新；
    3)用第一帧图像来初始化背景，再更新背景；所述背景指一个块的值；所述更新背景的方法包括：通过块序列更新背景；通过前景更新背景；通过背景更新背景；
    4)利用所述阈值比较相邻块之间、当前块与背景之间的差别，得到潜在前景，再根据潜在前景得到前景。
  2. 如权利要求1所述去除方法，其特征是，步骤2)用高斯混合模型对块的历史进行建模，具体使用一系列的块{ν1，ν2，…，νn}建立K个高斯模型的混合，K个高斯模型的加权得到高斯混合模型，以高斯混合模型表示当前块值的概率，记为P(vt)，用式2表示：
    Figure PCTCN2017070219-appb-100001
    其中，ωi，t表示第i个高斯模型在时间t的估计权重，μi，t是均值，σi，t是标准差。
  3. 如权利要求2所述去除方法，其特征是，对所述高斯混合模型中的各个模型具体用各个模型的ω/σ对模型进行排序，所述ω为各个模型的估计权重，所述σ为标准差；再设定排名最高的模型的三倍标准差作为阈值，由此保持阈值的不断更新。
  4. 如权利要求1所述去除方法，其特征是，步骤3)更新背景方法，所述通过块序列更新背景具体将前后块之间差值小于阈值且长度超过l的块的均值作为背景；所述通过前景更新背景具体将连续超过20l的前景的均值作为背景；所述通过背景更新背景；具体采用背景周围随机的一块，当这一块为前景且在块的历史中连续超过4l时，将所述前景的均值表示为背景；所述l为值相近的连续块序列的长度的设定值。
  5. 如权利要求4所述去除方法，其特征是，所述l的值为5。
  6. 如权利要求1所述去除方法，其特征是，步骤4)设置潜在前景采用以下两种方法中的一种：
    第一种方法是，通过比较相邻前后块，当所述相邻前后块的差值大于所述阈值时，将当前块设为潜在前景；
    第二种方式是，通过比较当前块与背景，当当前块与背景的差值大于所述阈值时，将当前块设为潜在前景。
  7. 如权利要求1所述去除方法，其特征是，步骤4)所述根据潜在前景得到前景，具体是：平均化所有处理过的块内的像素，将得到的值与潜在前景块内的像素做减法，得到的差值为ψ，将差值ψ通过低通滤波，当可通过的值少于一半的像素时，将潜在前景作为前景。
  8. 如权利要求7所述去除方法，其特征是，通过调整低通滤波改变所述去除方法对光照的敏感度。
PCT/CN2017/070219 2016-09-30 2017-01-05 一种视频的背景去除方法 Ceased WO2018058854A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US16/089,493 US10297016B2 (en) 2016-09-30 2017-01-05 Video background removal method

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201610873997.0 2016-09-30
CN201610873997.0A CN106447674B (zh) 2016-09-30 2016-09-30 一种视频的背景去除方法

Publications (1)

Publication Number Publication Date
WO2018058854A1 true WO2018058854A1 (zh) 2018-04-05

Family

ID=58171589

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2017/070219 Ceased WO2018058854A1 (zh) 2016-09-30 2017-01-05 一种视频的背景去除方法

Country Status (3)

Country Link
US (1) US10297016B2 (zh)
CN (1) CN106447674B (zh)
WO (1) WO2018058854A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113822161A (zh) * 2021-08-23 2021-12-21 河北大学 肤色与三级背景更新模型融合的动态手部检测方法及装置
CN116475089A (zh) * 2023-04-25 2023-07-25 西安科技大学 矿井带式输送机视频图像智能识别的联动控制装置及方法

Families Citing this family (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107633493B (zh) * 2017-09-28 2021-03-23 珠海博明视觉科技有限公司 一种适用于工业检测的自适应背景扣除的方法
CN110248207B (zh) * 2018-03-08 2021-03-16 株式会社理光 图像现实感展示服务器、展示方法和记录介质及展示系统
JP7233873B2 (ja) * 2018-09-19 2023-03-07 キヤノン株式会社 画像処理装置、画像処理方法、およびプログラム
CN111260695A (zh) * 2020-01-17 2020-06-09 桂林理工大学 一种扔杂物识别算法、系统、服务器及介质
CN111429472B (zh) * 2020-03-16 2023-11-03 阿波罗智联(北京)科技有限公司 图像的识别方法、装置、电子设备和存储介质
CN112734791B (zh) * 2021-01-18 2022-11-29 烟台南山学院 一种基于正则误差建模的在线视频前景背景分离方法
WO2023282662A1 (en) * 2021-07-07 2023-01-12 Samsung Electronics Co., Ltd. Method and electronic device for producing media file with blur effect
US11935299B2 (en) * 2022-04-08 2024-03-19 Cisco Technology, Inc. Decoding light-based messages to support device maintenance
CN116165353B (zh) * 2023-04-26 2023-07-25 江西拓荒者科技有限公司 一种工业污染物监测数据处理方法及系统

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120051631A1 (en) * 2010-08-30 2012-03-01 The Board Of Trustees Of The University Of Illinois System for background subtraction with 3d camera
CN103077520A (zh) * 2013-01-04 2013-05-01 浙江大学 一种针对移动摄像机的背景减除方法
CN103530886A (zh) * 2013-10-22 2014-01-22 上海安奎拉信息技术有限公司 一种视频分析低运算量背景去除方法

Family Cites Families (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2003036557A1 (en) * 2001-10-22 2003-05-01 Intel Zao Method and apparatus for background segmentation based on motion localization
US7254268B2 (en) * 2002-04-11 2007-08-07 Arcsoft, Inc. Object extraction
US8264542B2 (en) * 2007-12-31 2012-09-11 Industrial Technology Research Institute Methods and systems for image processing in a multiview video system
US8599255B2 (en) * 2010-12-07 2013-12-03 Qnap Systems, Inc. Video surveillance system based on Gaussian mixture modeling with two-type learning rate control scheme
CN102592125A (zh) * 2011-12-20 2012-07-18 福建省华大数码科技有限公司 基于标准差特征的运动目标检测方法
TWI499291B (zh) * 2012-01-05 2015-09-01 Univ Nat Taiwan 視訊資料之雜訊估測方法及系統
JP6348368B2 (ja) * 2014-08-12 2018-06-27 セコム株式会社 対象検出装置
US9449230B2 (en) * 2014-11-26 2016-09-20 Zepp Labs, Inc. Fast object tracking framework for sports video recognition
CN106327520B (zh) * 2016-08-19 2020-04-07 苏州大学 一种运动目标检测方法及系统

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120051631A1 (en) * 2010-08-30 2012-03-01 The Board Of Trustees Of The University Of Illinois System for background subtraction with 3d camera
CN103077520A (zh) * 2013-01-04 2013-05-01 浙江大学 一种针对移动摄像机的背景减除方法
CN103530886A (zh) * 2013-10-22 2014-01-22 上海安奎拉信息技术有限公司 一种视频分析低运算量背景去除方法

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
ZHANG, SHUNMIAO ET AL.: "A Motion Detection Algorithm based on Surendra Background Updating and Background Subtraction", JOURNAL OF NANJING INSTITUTE OF TECHNOLOGY, vol. 9, no. 2, 30 June 2011 (2011-06-30), ISSN: 1672-2558 *

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113822161A (zh) * 2021-08-23 2021-12-21 河北大学 肤色与三级背景更新模型融合的动态手部检测方法及装置
CN113822161B (zh) * 2021-08-23 2023-07-25 河北大学 肤色与三级背景更新模型融合的动态手部检测方法及装置
CN116475089A (zh) * 2023-04-25 2023-07-25 西安科技大学 矿井带式输送机视频图像智能识别的联动控制装置及方法

Also Published As

Publication number Publication date
US10297016B2 (en) 2019-05-21
US20190114753A1 (en) 2019-04-18
CN106447674B (zh) 2020-01-14
CN106447674A (zh) 2017-02-22

Similar Documents

Publication Publication Date Title
WO2018058854A1 (zh) 一种视频的背景去除方法
CN103971386B (zh) 一种动态背景场景下的前景检测方法
CN105678803B (zh) 基于w4算法和帧间差分的视频监控目标检测方法
CN107967695B (zh) 一种基于深度光流和形态学方法的运动目标检测方法
CN110688987A (zh) 一种行人位置检测与跟踪方法及系统
CN107451553B (zh) 一种基于超图转变的视频中暴力事件检测方法
Yi et al. Moving object detection based on running average background and temporal difference
CN106846359A (zh) 基于视频序列的运动目标快速检测方法
Boukhriss et al. Moving object detection under different weather conditions using full-spectrum light sources
CN103530893A (zh) 相机抖动场景下基于背景差和运动信息的前景检测方法
CN101221663A (zh) 一种基于运动目标检测的智能监控报警方法
TWI441096B (zh) 適用複雜場景的移動偵測方法
CN105550999A (zh) 一种基于背景复用的视频图像增强处理方法
CN104715480B (zh) 一种基于统计背景模型的目标检测方法
CN107360344A (zh) 监控视频快速去雾方法
CN1266656C (zh) 视频监控系统的智能化报警处理方法
Roy et al. A comprehensive survey on computer vision based approaches for moving object detection
CN111627047A (zh) 水下鱼类动态视觉序列运动目标检测方法
CN114821441A (zh) 联合ads-b信息的基于深度学习的机场场面运动目标识别方法
CN103218829B (zh) 一种适应于动态背景的前景提取方法
CN103500456A (zh) 一种基于动态贝叶斯模型网络的对象跟踪方法和设备
CN104200455B (zh) 一种基于运动统计特征分析的关键姿态提取方法
Nikolov et al. Motion detection using adaptive temporal averaging method
Li et al. Advanced multiple linear regression based dark channel prior applied on dehazing image and generating synthetic haze
Shi et al. Moving cast shadow detection in video based on new chromatic criteria and statistical modeling

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17854316

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 17854316

Country of ref document: EP

Kind code of ref document: A1