WO2018000882A1 - 基于监控视频的视频前景提取方法 - Google Patents

基于监控视频的视频前景提取方法 Download PDF

Info

Publication number
WO2018000882A1
WO2018000882A1 PCT/CN2017/079520 CN2017079520W WO2018000882A1 WO 2018000882 A1 WO2018000882 A1 WO 2018000882A1 CN 2017079520 W CN2017079520 W CN 2017079520W WO 2018000882 A1 WO2018000882 A1 WO 2018000882A1
Authority
WO
WIPO (PCT)
Prior art keywords
block
background
foreground
video
image
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2017/079520
Other languages
English (en)
French (fr)
Inventor
李革
臧祥浩
王文敏
王荣刚
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Peking University Shenzhen Graduate School
Original Assignee
Peking University Shenzhen Graduate School
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Peking University Shenzhen Graduate School filed Critical Peking University Shenzhen Graduate School
Priority to US16/089,517 priority Critical patent/US10395374B2/en
Publication of WO2018000882A1 publication Critical patent/WO2018000882A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/194—Segmentation; Edge detection involving foreground-background segmentation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/11—Region-based segmentation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/136—Segmentation; Edge detection involving thresholding
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/215—Motion-based segmentation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20016—Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20021—Dividing image into blocks, subimages or windows
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30232—Surveillance

Definitions

  • the present invention relates to the field of digital image processing, and in particular, to a video foreground extraction method based on surveillance video, which can be applied to preprocessing of surveillance video for further analysis of surveillance video.
  • the foreground extraction of a video refers to separating the foreground objects in the video from the background.
  • surveillance video has the characteristics of long time, complex foreground and increasing resolution.
  • analysis of some cases, accidents, etc. requires the use of previously monitored surveillance videos to provide more evidence for the development of the case.
  • the existing foreground extraction technology mainly extracts foreground objects by Gaussian mixture model, expectation maximization (EM) algorithm, saliency detection, Trimap algorithm, GrabCut algorithm, super pixel method and the like.
  • EM expectation maximization
  • Trimap Trimap
  • GrabCut super pixel method
  • the above method can partially extract the saliency region in the video, but the extraction process does not consider the difference between the static state and the motion state foreground, resulting in a globally consistent background, while the foreground object in the stationary state is regarded as a new background. ;
  • the present invention provides a video foreground extraction method for monitoring video. Based on the image block processing method, the size of the block is adjusted to adapt to different video resolutions; and then the background block model is established to extract The foreground object in motion state; the method of the invention can eliminate the influence of increasing monitoring video size on video processing analysis.
  • block image I' block image I'; temporary block background ⁇ ; block background Background block weights; foreground block weights The foreground block end point ⁇ .
  • the thresholds ⁇ 1 , ⁇ 2 , and ⁇ 3 are set to determine the magnitude of the weight.
  • the principle of the present invention is that a piece of surveillance video contains multiple frames of images, and the background of the surveillance video is fixed.
  • the present invention processes each frame of the surveillance video based on the block method, and adapts the size of the block to different video resolutions.
  • By creating a background block model extracting foreground objects in motion state; performing dimensionality reduction on each frame image in the surveillance video, eliminating the influence of illumination on the block method through pixel-based comparison; and simultaneously eliminating the surveillance video size continuously increasing the video The impact of the analysis.
  • the method of the present invention is capable of extracting dynamic foreground objects, adapting to larger surveillance video sizes, and having lower algorithmic complexity.
  • a video foreground extraction method for monitoring video based on a block processing method, adapting different block resolutions by adjusting a block size; and then acquiring a foreground object in a moving state by establishing a background block model; specifically comprising the following steps:
  • the image I is grayed out for each frame and the integral map is calculated.
  • the integration graph can conveniently calculate the sum of the pixels in different size blocks, and the pixels in the block add up and represent all the pixels in the block to be replaced.
  • the present invention uses the sum of m*n pixel values in the block to represent the block, that is, the original m*n dimension becomes the current 1*1 dimension.
  • the second step initialization, initializes the background with the first frame image of the video for the first frame image
  • the background here refers to the remainder of the image of each frame except the foreground object.
  • Initialize the foreground with a zero matrix The foreground here refers to the foreground object in each frame of the image.
  • the intermediate result in the process is saved with the temporary background ⁇ and the temporary foreground ⁇ .
  • the weight refers to the duration of a block in the image in the video stream.
  • the initialization value of the ownership value is 0, and the dimension is consistent with the dimension of the block background.
  • the foreground block end point ⁇ is initialized with a zero matrix which is used to determine the continuity of the block stream, the dimension being identical to the block image I'.
  • a third step the current frame block of the image I 'i in each element in the image block of the previous frame I' i-1 corresponding to the elements compared, if the absolute difference between the two exceeds the threshold ⁇ 1, the front and rear frame represents the corresponding The block difference is large and the current block may be foreground.
  • the background block weight is incremented by 1, and the weighted average frame block image elements are used to find the temporary block background ⁇ . Otherwise, the background block weight ⁇ is reset to 0, and the temporary block background ⁇ is reset to the current element.
  • the temporary block background means that the current block may be part of the background, and it is determined whether it is a background or not.
  • the block background is updated with the element in the temporary block background ⁇
  • a fifth step a case, the comparison block image I 'I elements in the image block of the previous frame I' i-1 corresponding element, if the difference between the two is greater than the threshold value ⁇ 1; Case 2, the comparison block image I 'in Element and block background Corresponding elements, if the difference between the two is greater than the threshold ⁇ 1 , both of them indicate that the current block may be a potential foreground block. If one of the above conditions is met, the sixth step is performed, otherwise the third step is performed.
  • the seventh step comparing the pixels and the background in the original image I corresponding to the current block image I' element Corresponding pixels, if the number of pixels whose pixel difference exceeds the threshold ⁇ 3 exceeds half of the number of pixels in the block, the seventh step is performed, otherwise the third step is performed.
  • the seventh step the pixel value and background in the original image I
  • the difference of the corresponding pixel value is saved as the foreground
  • the foreground block is judged to be continuous according to the foreground block end point ⁇ . If it is continuous, the background block weight ⁇ is not 0, indicating that the current foreground is likely to be a new background, and the foreground block weight is Increase by 1. And the weighted average before and after frames correspond to the pixels in the block to obtain the temporary foreground ⁇ .
  • the temporary foreground here means that the current block may be part of the background, to determine whether it is a background, and to see the eighth step.
  • the eighth step if the foreground block weights If the threshold value ⁇ 2 is exceeded, the temporary foreground ⁇ is set as the background The background of the temporary block is calculated, the background block weight is reset to ⁇ 2 , and the temporary foreground ⁇ is set to zero.
  • the ninth step the prospects of the seventh step Into the binarization process, the current frame number is saved as the foreground block end point ⁇ .
  • the third to ninth steps are repeated until all the frames in the video have been processed.
  • the invention provides a video foreground extraction method for monitoring video, which is adapted to different video resolutions by adjusting the size of the block based on the image block processing method; and then extracting a foreground object in a moving state by establishing a background block model; Each frame of the image is dimension-reduced, and the effect of illumination on the block method is eliminated through pixel-based comparison; at the same time, the influence of increasing monitoring video size on video analysis is eliminated. Therefore, the method of the invention can extract dynamic foreground objects, adapt to large monitoring video size, and has low algorithm complexity, can be applied to preprocessing of monitoring video, and provides support for further analysis of monitoring video, and more and more in surveillance cameras. More today is of great significance. Compared with the prior art, the present invention has the following advantages:
  • the present invention is based on a block processing method, changing the comparison of the prior art for a single pixel to a comparison of pixel summation, liberating most of the computational pressure in practical applications, greatly reducing computational complexity, and high execution efficiency;
  • the present invention realizes stable extraction of dynamic objects in the surveillance video by updating the background by establishing a background block model (ie, extracting the foreground);
  • the present invention eliminates the influence of the sudden change of illumination on the block method by introducing the comparison of the pixels in the block, and the method has high precision.
  • FIG. 1 is a flow chart of a video foreground extraction method for monitoring video provided by the present invention.
  • FIG. 2 is a schematic diagram of an image I of each frame in the video as a block image I′ in the embodiment of the present invention
  • the resolution of the image is 1920*1080, and the block size is 60*60, and the original image I can be represented by a block image I' of 32*18.
  • FIG. 3 is a frame image of a surveillance video processed in an embodiment of the present invention.
  • FIG. 4 is a background obtained by processing a video by using the method of the present invention in an embodiment of the present invention.
  • FIG. 5 is a perspective of the video processed by the method of the present invention in the embodiment of the present invention. Binarized image.
  • the present invention provides a video foreground extraction method for monitoring video, which is adapted to different video resolutions by adjusting the size of the block based on the image block processing method; and then extracts a foreground object in a moving state by establishing a background block model; Eliminates the impact of increasing surveillance video size on video processing analytics.
  • variables defined in the following methods or formulas that are identical to the original image dimensions include: temporary background ⁇ ; background Temporary prospect ⁇ ; prospect
  • variables that define the dimensionality reduction of the original image include: temporary block background ⁇ ; block background Background block weights; foreground block weights The foreground block end point ⁇ ; threshold 1, threshold 2 and threshold 3 are used to determine the magnitude of the weight.
  • FIG. 3 is a frame image of a piece of surveillance video processed in an embodiment of the present invention, and a segment of the surveillance video includes a plurality of frames of images.
  • FIG. 1 is a block diagram of a method for extracting video for monitoring video. The specific implementation of the present invention includes the following steps:
  • each frame of the video is represented as a block.
  • the video resolution is P*Q (the surveillance video includes the resolution of each frame in the multi-frame image is P*Q)
  • the grayscale image of each frame image is obtained.
  • I gray the size of the block is determined to be p*q.
  • the video resolution is 320*240, so a smaller block is used, and the block size is 2*4.
  • each block uses the sum of the pixels in the block to use b s, and t is expressed as Equation 1:
  • Equation 2 the original image can be represented by a matrix I' as shown in Equation 2:
  • Step 2 Initialize the temporary background ⁇ and background with I
  • Temporary foreground is represented by a zero matrix ⁇ of P*Q size; matrix Representing the foreground; initializing the block background with matrix I' And the temporary block background ⁇ ; the background block weight is represented by a zero matrix m of m*n size; Indicates the foreground block weight; ⁇ represents the foreground block end point; as in Equations 3 and 4:
  • the third step the elements in the next frame image I' i Compare it with the corresponding element in the previous frame image Difference Set the threshold ⁇ 1 if Then the block background weight ⁇ s, t is incremented by 1, otherwise the block background weight ⁇ s, t is 0, and the block temporary background and temporary background are updated as shown in Equation 5:
  • ⁇ st represents all pixels in the image block corresponding to the current block
  • ⁇ 1 represents a threshold value of the difference between pixel summation (ie, the block herein),
  • the threshold ⁇ 2 indicates the length of time during which the block stream remains unchanged; if the value of the block background image weight exceeds the threshold ⁇ 2 , the block background is updated with the temporary block background, and the background is updated by the temporary background, as shown in Equation 6:
  • Step 5 When the difference between the current block and the background block exceeds the threshold ⁇ 1 or the weight of the background block is 0, the sixth step is performed; otherwise, the third step is performed;
  • Case 1 Comparing the elements and block backgrounds in the block image I' Corresponding elements, if the difference between the two is greater than the threshold ⁇ 1 , as shown in Equation 7, proceed to the sixth step:
  • Step 6 Compare the pixels in the current block with the pixels in the block corresponding to the background frame. If the pixel difference exceeds the threshold ⁇ 3 , the number of pixels exceeds half of the number of pixels in the block, as shown in Equation 9:
  • f() represents a count of pixels that meet the conditions within the block
  • Step 7 Pixel values and background in the original image I
  • the difference of the corresponding pixel value is saved as the foreground
  • For the fifth step case 1 the difference between the current block and the background block exceeds the threshold ⁇ 1 , it is determined whether the foreground block is continuous; if continuous, and the difference between the front and rear foreground blocks is less than the threshold ⁇ 1 , the foreground block weight is incremented by 1, and The weighted average foreground block, as shown in Equation 10:
  • the background is updated with the temporary foreground
  • updating the block background and the temporary block background by summing the pixel values in each block of the updated background according to Equation 11, where the block background and the temporary block background take the same value, and the block background weight is reset to the threshold ⁇ 2 , that is, 11:
  • ⁇ represents the temporary foreground
  • Is the block background
  • ⁇ is the temporary block background
  • ⁇ is the background block weight
  • This step causes the background to be updated in real time, for example, when a car is moving, the car is foreground, but when the car stops, the car is updated to the background.
  • What you get in the seventh step is the foreground, which controls the real-time update of the background by setting the threshold. Judging according to the time when the foreground is not moving, if the time is not moving, the foreground is treated as a background.
  • the ninth step the prospects of the seventh step Perform binarization, which is the final foreground result, and save the current frame number as the foreground end point ⁇ s,t ;
  • the third to the ninth steps are repeatedly executed until all the frames in the video are processed, that is, the video foreground is extracted.
  • FIG 4 is a background obtained by processing a video by using the method of the present invention in this embodiment.
  • Figure 5 is a perspective of the present embodiment after processing the video by using the method of the present invention. Binarized image.

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Multimedia (AREA)
  • Image Analysis (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)
  • Closed-Circuit Television Systems (AREA)

Abstract

本发明公布了一种针对监控视频的视频前景提取方法，基于图像块处理方法，通过调节块的大小适应不同的视频分辨率；再通过建立背景块模型，提取处于运动态的前景物体；包括：将监控视频中每一帧图像I表示为块；进行初始化；更新块背景权重、块临时背景和临时背景；更新块背景和背景；保存前景、更新前景块权重和前景块；将前景进行二值化处理即为最终前景结果。本发明针对监控视频中每一帧图像进行降维，通过基于像素的比较消除光照对于块方法的影响；同时消除监控视频尺寸不断增大对视频分析带来的影响。本发明能够提取动态前景物体、适应较大监控视频尺寸、且具有较低算法复杂度。

Description

基于监控视频的视频前景提取方法 技术领域
本发明涉及数字图像处理领域，尤其涉及一种基于监控视频的视频前景提取方法，可应用于监控视频的预处理，以便对监控视频做进一步分析。
背景技术
视频的前景提取是指将视频中的前景对象从背景中分离出来。监控视频作为一种越来越常见的视频类型，具有时间长、前景复杂、分辨率不断增大的特点。现实生活中对一些案件、事故等的分析都需要借助于之前拍摄的监控视频来为案情的发展提供更多的证据。
现有前景提取技术主要通过高斯混合模型、期望最大化(EM)算法、显著性检测、Trimap算法、GrabCut算法、超像素方法等进行前景物体的提取。但是，这些视频前景提取方法存在以下缺点：
(一)上述方法可部分提取出视频中的显著性区域，但提取过程没有考虑静止态与运动态前景的不同，导致形成全局一致的背景，而对静止态的前景对象要看作为新的背景；
(二)对于尺寸越来越大的监控视频，上述方法的处理速度越来越慢，针对大尺寸视频的处理效率低；
(三)上述方法大多具有较高的算法复杂度，处理性能不佳。
发明内容
为了克服上述现有技术的不足，本发明提供一种针对监控视频的视频前景提取方法，基于图像块处理方法，通过调节块的大小适应不同的视频分辨率；再通过建立背景块模型，提取处于运动态的前景物体；本发明方法可消除监控视频尺寸不断增大对视频处理分析带来的影响。
为方便说明，本说明书约定以下术语和参数表示：
(一)定义与原图像维度相同的变量：原图像I；临时背景β；背景
Figure PCTCN2017079520-appb-000001
临时前景γ；前景
Figure PCTCN2017079520-appb-000002
(二)定义与降维后图像维度相同的变量：块图像I′；临时块背景α；块背景
Figure PCTCN2017079520-appb-000003
背景块权重ψ；前景块权重
Figure PCTCN2017079520-appb-000004
前景块终点ω。
设定阈值ε1、ε2、ε3，均用来判断权重的大小。
本发明的原理是：一段监控视频包含多帧图像，而监控视频的背景固定不动，本发明基于块方法对监控视频中每帧图像进行处理，通过调节块的大小适应不同的视频分辨率。通过建立背景块模型，提取处于运动态的前景物体；针对监控视频中每一帧图像进行降维，通过基于像素的比较，消除光照对于块方法的影响；同时消除监控视频尺寸不断增大对视频分析带来的影响。因而，本发明方法能够提取动态前景物体、适应较大监控视频尺寸、且具有较低算法复杂度。
本发明提供的技术方案是：
一种针对监控视频的视频前景提取方法，基于块处理方法，通过调节块的大小适应不同的视频分辨率；再通过建立背景块模型，提取处于运动态的前景物体；具体包括如下步骤：
第一步，对每一帧图像I灰度化并计算积分图，通过积分图可以方便的计算不同大小块内像素的加和，以此块内像素加和代表块内的所有像素，得到代替原图的具有更小维度的图像矩阵块图像I′。一个块内有m*n个像素值，本发明用这一个块内m*n个像素值的加和表示这一个块，即原来的m*n维变为了现在的1*1维。
第二步，初始化，对于第一帧图像，用视频第一帧图像初始化背景
Figure PCTCN2017079520-appb-000005
这里背景指每一帧图像中除了前景对象之外的剩余部分。用零矩阵初始化前景
Figure PCTCN2017079520-appb-000006
这里前景指每一帧图像中的前景对象。用临时背景β和临时前景γ保存处理过程中的中间结果。用块图像I′初始化块背景
Figure PCTCN2017079520-appb-000007
和临时块背景α。用零矩阵初始化背景块权重ψ和前景块权重
Figure PCTCN2017079520-appb-000008
这里权重指图像中某一块在视频流中持续的时长，所有权值的初始化值为0，维度与块背景的维度一致。用零矩阵初始化前景块终点ω，此终点用来判断块流的连续性，维度与块图像I′一致。
第三步，将当前帧块图像I′i中的每一个元素与前一帧块图像I′i-1对应元素进行比较，如果两者差值的绝对值超过阈值ε1，表示前后帧对应块差别较大，当前块可能为前景。将背景块权重ψ自增1，并且加权平均前后帧块图像元素求得临时块背景α。否则将背景块权重ψ重置为0，并且重置临时块背景α为当前元素，这里临时块背景指当前块可能为背景的一部分，判断它是不是背景，要看第四步。
第四步，若背景块权重ψ中元素超过阈值ε2，说明当前块的持续时间够长，则用临时块背景α中元素更新块背景
Figure PCTCN2017079520-appb-000009
第五步，情况一，比较块图像I′i中的元素与前一帧块图像I′i-1对应元素，若两者差值大于阈值ε1；情况二，比较块图像I′中的元素与块背景
Figure PCTCN2017079520-appb-000010
对应元素，若两者差值大于阈值ε1，以 上均表示当前块可能是潜在前景块。满足上述情况之一则执行第六步，否则执行第三步。
第六步，比较当前块图像I′元素对应的原图像I中的像素与背景
Figure PCTCN2017079520-appb-000011
对应的像素，若像素差值超过阈值ε3的像素个数，超过了块内像素个数的一半，执行第七步，否则执行第三步。
第七步，将原图像I中的像素值与背景
Figure PCTCN2017079520-appb-000012
对应的像素值的差值保存为前景
Figure PCTCN2017079520-appb-000013
对于第五步情况二，根据前景块终点ω判断前景块是否连续，如果连续，背景块权重ψ不为0，说明当前前景很有可能是新的背景，对前景块权重
Figure PCTCN2017079520-appb-000014
自增1。并且加权平均前后帧对应块内像素求得临时前景γ。这里临时前景指当前块可能为背景的一部分，判断它是不是背景，要看第八步。
第八步，若前景块权重
Figure PCTCN2017079520-appb-000015
超过阈值值ε2，则把临时前景γ置为背景
Figure PCTCN2017079520-appb-000016
计算得临时块背景，背景块权重置为ε2，置临时前景γ为0。
第九步，将第七步所得前景
Figure PCTCN2017079520-appb-000017
进二值化处理，保存当前帧号为前景块终点ω。
第十步，重复执行第三到第九步，直到视频内所有帧处理完毕。
与现有技术相比，本发明的有益效果是：
本发明提供一种针对监控视频的视频前景提取方法，基于图像块处理方法，通过调节块的大小适应不同的视频分辨率；再通过建立背景块模型，提取处于运动态的前景物体；针对监控视频中每一帧图像进行降维，通过基于像素的比较，消除光照对于块方法的影响；同时消除监控视频尺寸不断增大对视频分析带来的影响。因而，本发明方法能够提取动态前景物体、适应较大监控视频尺寸、且具有较低算法复杂度，可以应用于监控视频的预处理，为监控视频的进一步分析提供支持，在监控摄像头越来越多的今天具有重要的意义。与现有技术相比，本发明具有如下优点：
(一)本发明基于块处理方法，将现有技术针对单个像素的比较改变为对像素加和的比较，解放了实际应用中大多数的计算压力，大大降低计算复杂度，执行效率高；
(二)本发明通过建立背景块模型，通过更新背景，实现稳定地提取监控视频中动态的物体(即提取前景)；
(三)本发明通过引入块内像素的比较，消除了由于光照的突变对块方法带来的影响，方法精度高。
附图说明
图1是本发明提供的针对监控视频的视频前景提取方法的流程框图。
图2是本发明实施例中将视频中每一帧图像I为块图像I′示意图；
图中，本图像分辨率为1920*1080，块大小为60*60，则原图像I可用32*18的块图像I′表示。
图3是本发明实施例中处理的监控视频中的一帧图像。
图4是本发明实施例通过采用本发明方法对视频进行处理后得到的背景
Figure PCTCN2017079520-appb-000018
图5是本发明实施例通过采用本发明方法对视频进行处理后得到的前景
Figure PCTCN2017079520-appb-000019
二值化图像。
具体实施方式
下面结合附图，通过实施例进一步描述本发明，但不以任何方式限制本发明的范围。
本发明提供一种针对监控视频的视频前景提取方法，基于图像块处理方法，通过调节块的大小适应不同的视频分辨率；再通过建立背景块模型，提取处于运动态的前景物体；本发明方法可消除监控视频尺寸不断增大对视频处理分析带来的影响。
以下方法或公式中定义与原图像维度相同的变量包括：临时背景β；背景
Figure PCTCN2017079520-appb-000020
临时前景γ；前景
Figure PCTCN2017079520-appb-000021
定义原图像降维后的变量包括：临时块背景α；块背景
Figure PCTCN2017079520-appb-000022
背景块权重ψ；前景块权重
Figure PCTCN2017079520-appb-000023
前景块终点ω；阈值1、阈值2和阈值3均用来判断权重的大小。
图3是本发明实施例中处理的一段监控视频中的一帧图像，一段监控视频包含多帧图像。针对监控视频提取视频前景，图1是本发明方法的流程框图，本发明具体实施包括如下步骤：
第一步：如图2所示，将视频中每一帧图像表示为块。假设视频分辨率为P*Q(监控视频包括多帧图像中的每一帧图像的分辨率为P*Q)，对于视频序列的每一帧图像I，求得每一帧图像的灰度图Igray，确定块的大小为p*q，在本实施例中，视频分辨率为320*240，因此采用较小的块，块大小为2*4。则每一个块用块内像素的加和采用bs,t表示为式1：
Figure PCTCN2017079520-appb-000024
其中，s、t分别表示对应新矩阵的行与列；pi,j表示灰度图像的像素值；
这样，原图像可以用一个如式2所示的矩阵I′表示：
Figure PCTCN2017079520-appb-000025
其中，
Figure PCTCN2017079520-appb-000026
第二步：用I初始化临时背景β和背景
Figure PCTCN2017079520-appb-000027
用P*Q大小的零矩阵γ表示临时前景；矩阵
Figure PCTCN2017079520-appb-000028
表 示前景；用矩阵I′初始化块背景
Figure PCTCN2017079520-appb-000029
和临时块背景α；用m*n大小的零矩阵ψ表示背景块权重；
Figure PCTCN2017079520-appb-000030
表示前景块权重；ω表示前景块终点；如式3和式4：
Figure PCTCN2017079520-appb-000031
Figure PCTCN2017079520-appb-000032
第三步：对下一帧图像I′i中的元素
Figure PCTCN2017079520-appb-000033
比较它与前一帧图像中对应元素
Figure PCTCN2017079520-appb-000034
的差值
Figure PCTCN2017079520-appb-000035
设定阈值ε1，如果
Figure PCTCN2017079520-appb-000036
则块背景权重ψs,t自增1，否则重置块背景权重ψs,t为0，并更新块临时背景和临时背景如式5所示：
Figure PCTCN2017079520-appb-000037
其中，βs.t表示当前块对应的图像块内的所有像素；
在本实施例中，ε1表示像素加和(即本文中的块)之间差值的阈值，
Figure PCTCN2017079520-appb-000038
其中阈值ε3用来比较像素之间的差值；ε3取值为15时，则ε1＝60，采用公式分母为2的原因是为了抵消块方法对精度的影响；
第四步：阈值ε2表示块流保持不变的时间长度；若块背景图像权重的值超过阈值ε2，则用临时块背景更新块背景，临时背景更新背景，即如式6所示：
Figure PCTCN2017079520-appb-000039
在本实施例中，取值ε2＝15，表示块流的大小至少在15帧以上，此时我们认为图像符合象监控视频中图像背景是不动的假设。
第五步：当当前块与背景块的差值超过阈值ε1或背景块的权重为0时，执行第六步；否则执行第三步；
具体是，情况一：比较块图像I′中的元素与块背景
Figure PCTCN2017079520-appb-000040
对应元素，若两者差值大于阈值ε1，如式7所示，则继续执行第六步：
Figure PCTCN2017079520-appb-000041
情况二：若当背景块权重ψs,t为0，如式8所示，则继续执行第六步：
ψs,t＝0          (式8)
否则执行第三步；
第六步：比较当前块内的像素与背景帧对应块内的像素，若像素差值超过阈值ε3的像素个数，超过了块内像素个数的一半，如式9所示：
Figure PCTCN2017079520-appb-000042
其中，f()表示对块内符合条件的像素计数；
Figure PCTCN2017079520-appb-000043
分别表示bs，t对应的的原图像I中的像素和背景
Figure PCTCN2017079520-appb-000044
对应的像素，bs,t是块内共计m*n个像素的加和；若符合条件(式9成立)则执行第七步，否则执行第三步；
第七步：将原图像I中的像素值与背景
Figure PCTCN2017079520-appb-000045
对应的像素值的差值保存为前景
Figure PCTCN2017079520-appb-000046
对于第五步情况一(当前块与背景块的差值超过阈值ε1)，判断前景块是否连续；若连续，且前后前景块的差值小于阈值ε1，前景块权重自增1，并加权平均前景块，即如式10所示：
Figure PCTCN2017079520-appb-000047
第八步，若块前景图像权重的值超过阈值ε2，则用临时前景更新背景
Figure PCTCN2017079520-appb-000048
并根据式11由更新后的背景每块内像素值求和来更新块背景和临时块背景，这里块背景和临时块背景取相同的值，并重置块背景权重为阈值ε2，即式11：
Figure PCTCN2017079520-appb-000049
其中，
Figure PCTCN2017079520-appb-000050
为背景；s、t分别表示对应新矩阵的行与列；γ表示临时前景；
Figure PCTCN2017079520-appb-000051
为块背景；α为临时块背景；ψ为背景块权重；
Figure PCTCN2017079520-appb-000052
为前景块权重；
此步骤使得背景会实时更新，例如，一辆车在运动时该车是前景，但当车停下来时该车更新为背景。第七步中得到的是前景，此步骤通过设定阈值控制背景的实时更新。根据前景不动的时间进行判断，如果长时间不动，就把前景当作背景看待。
第九步，将第七步得到的前景
Figure PCTCN2017079520-appb-000053
进行二值化处理，即为最终前景结果，保存当前帧号为前景终点ωs,t；
上述第八步和第九步的次序可以交换。
第十步，重复执行第三到第九步，直到视频内所有帧处理完毕，即提取得到视频前景。
图4是本实施例通过采用本发明方法对视频进行处理后得到的背景
Figure PCTCN2017079520-appb-000054
图5是本实施例通过采用本发明方法对视频进行处理后得到的前景
Figure PCTCN2017079520-appb-000055
二值化图像。
需要注意的是，公布实施例的目的在于帮助进一步理解本发明，但是本领域的技术人员 可以理解：在不脱离本发明及所附权利要求的精神和范围内，各种替换和修改都是可能的。因此，本发明不应局限于实施例所公开的内容，本发明要求保护的范围以权利要求书界定的范围为准。

Claims (6)

  1. 一种基于监控视频的视频前景提取方法，采用图像块处理方法，通过调节块的大小适应不同的视频分辨率；再通过建立背景块模型，提取处于运动态的前景物体；包括如下步骤：
    第一步：将监控视频中每一帧图像I表示为块，将图像I表示为矩阵块I′；
    第二步：进行初始化：对第一帧图像I，用图像I初始化临时背景β和背景
    Figure PCTCN2017079520-appb-100001
    用P*Q大小的零矩阵γ表示临时前景；矩阵
    Figure PCTCN2017079520-appb-100002
    表示前景，用零矩阵初始化；用矩阵I’初始化块背景
    Figure PCTCN2017079520-appb-100003
    和临时块背景α；用m*n大小的零矩阵ψ表示背景块权重；
    Figure PCTCN2017079520-appb-100004
    表示前景块权重，用零矩阵初始化；ω表示前景块终点，用零矩阵初始化；
    第三步：设定s、t分别表示矩阵块的行与列，对下一帧图像的矩阵块I’i中的元素
    Figure PCTCN2017079520-appb-100005
    比较它与前一帧图像中对应元素
    Figure PCTCN2017079520-appb-100006
    的差值
    Figure PCTCN2017079520-appb-100007
    设定阈值ε1，如果
    Figure PCTCN2017079520-appb-100008
    则块背景权重ψs,t自增1；否则重置块背景权重ψs,t为0，并更新块临时背景和临时背景；
    第四步：设定阈值ε2表示块流保持不变的时间长度；若块背景图像权重的值超过阈值ε2，则用临时块背景更新块背景，用临时背景更新背景；
    第五步：当当前块与背景块的差值超过阈值ε1或背景块的权重为0时，执行第六步；否则执行第三步；
    第六步：比较当前块内的像素与背景帧对应块内的像素，当像素差值超过阈值ε3的像素个数超过了块内像素个数的一半时，执行第七步；否则执行第三步；
    第七步：将原图像I中的像素值与背景
    Figure PCTCN2017079520-appb-100009
    对应的像素值的差值保存为前景
    Figure PCTCN2017079520-appb-100010
    当前景块连续且前后前景块的差值小于阈值ε1时，前景块权重自增1，并加权平均前景块；
    第八步：当块前景图像权重的值超过阈值ε2时，用临时前景更新背景块背景
    Figure PCTCN2017079520-appb-100012
    和临时块背景α，并重置块背景权重为阈值ε2；置临时前景γ为0；
    第九步：将第七步得到的前景
    Figure PCTCN2017079520-appb-100013
    进行二值化处理，即为最终前景结果，保存当前帧号为前景终点ωs,t；
    第十步：重复执行第三到第九步，直到视频内所有帧图像处理完毕，即提取得到视频前景。
  2. 如权利要求1所述基于监控视频的视频前景提取方法，其特征是，第一步所述将图 像I表示为矩阵块图像I′，具体执行如下操作：
    首先将图像I灰度化并计算积分图；
    再通过积分图计算不同大小块内像素的加和，代表块内的所有像素，得到代替原图像的具有更小维度的图像矩阵块图像I′。
  3. 如权利要求1所述基于监控视频的视频前景提取方法，其特征是，设定块的大小为p*q；ε1为表示块之间差值的阈值；阈值ε3为像素之间的差值的阈值；设置
    Figure PCTCN2017079520-appb-100014
  4. 如权利要求1所述基于监控视频的视频前景提取方法，其特征是，阈值ε2取值为15。
  5. 如权利要求1所述基于监控视频的视频前景提取方法，其特征是，阈值ε3取值为30。
  6. 如权利要求1所述基于监控视频的视频前景提取方法，其特征是，第八步所述用临时前景更新背景
    Figure PCTCN2017079520-appb-100015
    块背景
    Figure PCTCN2017079520-appb-100016
    和临时块背景α，具体通过如下公式计算：
    Figure PCTCN2017079520-appb-100017
    式11中，
    Figure PCTCN2017079520-appb-100018
    为背景；s、t分别表示对应新矩阵的行与列；γ表示临时前景；
    Figure PCTCN2017079520-appb-100019
    为块背景；α为临时块背景；ψ为背景块权重；
    Figure PCTCN2017079520-appb-100020
    为前景块权重。
PCT/CN2017/079520 2016-06-27 2017-04-06 基于监控视频的视频前景提取方法 Ceased WO2018000882A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US16/089,517 US10395374B2 (en) 2016-06-27 2017-04-06 Surveillance video based video foreground extraction method

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201610478505.8 2016-06-27
CN201610478505.8A CN106204636B (zh) 2016-06-27 2016-06-27 基于监控视频的视频前景提取方法

Publications (1)

Publication Number Publication Date
WO2018000882A1 true WO2018000882A1 (zh) 2018-01-04

Family

ID=57462006

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2017/079520 Ceased WO2018000882A1 (zh) 2016-06-27 2017-04-06 基于监控视频的视频前景提取方法

Country Status (3)

Country Link
US (1) US10395374B2 (zh)
CN (1) CN106204636B (zh)
WO (1) WO2018000882A1 (zh)

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106204636B (zh) 2016-06-27 2019-03-22 北京大学深圳研究生院 基于监控视频的视频前景提取方法
CN108805878A (zh) * 2018-05-22 2018-11-13 深圳腾视科技有限公司 一种基于计算机视觉的前景物体检测解决方法
CN111008579A (zh) * 2019-11-22 2020-04-14 华中师范大学 专注度识别方法、装置和电子设备
CN111462155B (zh) * 2020-03-26 2023-10-24 深圳万拓科技创新有限公司 移动侦测方法、装置、计算机设备和存储介质
CN112802054B (zh) * 2021-02-04 2023-09-01 重庆大学 一种融合图像分割的混合高斯模型前景检测方法
CN113542531B (zh) * 2021-06-16 2023-11-21 星宸科技股份有限公司 侦测影像中物件移动的装置与方法
CN115426525B (zh) * 2022-09-05 2023-05-26 北京拙河科技有限公司 一种基于高速动帧联动图像拆分方法及装置

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101216888A (zh) * 2008-01-14 2008-07-09 浙江大学 基于快速图像配准的视角变化条件下视频前景提取方法
CN102103751A (zh) * 2009-12-18 2011-06-22 华为技术有限公司 前景图像提取方法及装置
CN103077520A (zh) * 2013-01-04 2013-05-01 浙江大学 一种针对移动摄像机的背景减除方法
WO2014022170A1 (en) * 2012-07-30 2014-02-06 Intel Corporation Video communication with three dimensional perception
CN106204636A (zh) * 2016-06-27 2016-12-07 北京大学深圳研究生院 基于监控视频的视频前景提取方法

Family Cites Families (37)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4896446A (en) 1989-07-10 1990-01-30 G. Squared, Inc. Buttplate and comb assembly for shoulder firearms
US5410833A (en) 1993-07-16 1995-05-02 Paterson; Douglas F. Recoil absorbing firearm stock
US5392553A (en) 1994-01-11 1995-02-28 Carey; Donald C. Gun stock assembly with universally adjustable comb piece
US5970642A (en) 1998-01-29 1999-10-26 Martin; Billy B. Ergonomic adjustable gun stock
US20060254111A1 (en) 2005-05-11 2006-11-16 Giauque Timothy D Adjustable firearm stock
EP1748272A3 (en) 2005-07-26 2007-03-21 T.D.I. Arms Ltd. Telescoping firearm stock
US7930849B2 (en) 2006-03-11 2011-04-26 Dick Abraham Adjustable butt stock
US7640688B2 (en) 2006-06-16 2010-01-05 Command Arms Accessories Adjustable cheek rest and accessory rail for firearms
US7647719B2 (en) 2007-01-11 2010-01-19 Magpul Industries Corp. Gunstocks and adapters
US8141288B2 (en) 2009-01-16 2012-03-27 Prototype Productions, Inc. Rugged low light reflectivity electrical contact
US8402683B2 (en) 2009-01-16 2013-03-26 Prototype Productions Incorporated Ventures Two, Llc Rifle accessory rail, communication, and power transfer system-battery pack
DE102009010768B4 (de) 2009-02-26 2013-07-11 Wilfried NILL Schaftsystem für eine schultergestützte Waffe
US8061072B1 (en) 2009-03-16 2011-11-22 Crose Dinora M Retractable stock firearm system
US8186090B1 (en) 2009-06-29 2012-05-29 The United States Of America As Represented By The Secretary Of The Army Adjustable buttstock assembly
US20120036757A1 (en) 2010-08-12 2012-02-16 Larue Mark C Gun-stock riser for AR15 type tactical firearm
US8950099B2 (en) 2010-09-29 2015-02-10 Rogers Holster Co., Llc Collapsible stock assembly
US8453365B1 (en) 2012-01-23 2013-06-04 Brookshire Tool & Mfg Co., Inc. Firearm with movable cheek riser
US8844185B2 (en) 2012-08-27 2014-09-30 Ra Brands, L.L.C. Buttstock assembly
USD704294S1 (en) 2012-09-19 2014-05-06 Ra Brands, L.L.C. Buttstock
US9664478B2 (en) 2012-10-13 2017-05-30 Rmdi, Llc Adjustable firearm stock
US8763296B1 (en) 2013-03-15 2014-07-01 Ati Ip, Llc Adjustable cheek rest assembly for a firearm stock
US8955245B2 (en) 2013-03-15 2015-02-17 Ati Ip, Llc Adjustable stock for a firearm
US8720099B1 (en) 2013-05-07 2014-05-13 Charles H. Sisk Multi-axis adjustable buttstock
CN103617632B (zh) * 2013-11-19 2017-06-13 浙江工业大学 一种结合相邻帧差法和混合高斯模型的运动目标检测方法
EP3120100A1 (en) 2014-03-19 2017-01-25 Robert Irvin Recoil mitigation and buttstock floating system, method and apparatus
USD736336S1 (en) 2014-03-27 2015-08-11 Magpul Industries Corporation Firearm stock
US9360272B2 (en) 2014-05-20 2016-06-07 Sig Sauer, Inc. Adjustable buttstock clamp
EP3213026B1 (en) 2014-10-28 2019-08-28 Sturm, Ruger & Company, Inc. Firearm with folding buttstock
EP3245473B1 (en) 2015-01-13 2019-09-25 Sturm, Ruger & Company, Inc. Adjustable buttstock for firearm
US9404708B1 (en) 2015-06-30 2016-08-02 Magpul Industries Corp. Stock for a firearm
US9791238B2 (en) 2015-08-13 2017-10-17 Daniel Lucian Bowen Rifle butt stock adjustment actuator
EP3255585B1 (en) * 2016-06-08 2018-05-09 Axis AB Method and apparatus for updating a background model
US9664476B1 (en) 2016-08-23 2017-05-30 Rmdi, Llc Adjustable firearm stock
US9664479B1 (en) 2016-08-23 2017-05-30 Rmdi, Llc Adjustable firearm stock
US10095932B2 (en) * 2016-12-22 2018-10-09 Sap Se Video abstract using signed foreground extraction and fusion
US10269123B2 (en) * 2017-01-09 2019-04-23 Qualcomm Incorporated Methods and apparatus for video background subtraction
US10121093B2 (en) * 2017-04-11 2018-11-06 Sony Corporation System and method for background subtraction in video content

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101216888A (zh) * 2008-01-14 2008-07-09 浙江大学 基于快速图像配准的视角变化条件下视频前景提取方法
CN102103751A (zh) * 2009-12-18 2011-06-22 华为技术有限公司 前景图像提取方法及装置
WO2014022170A1 (en) * 2012-07-30 2014-02-06 Intel Corporation Video communication with three dimensional perception
CN103077520A (zh) * 2013-01-04 2013-05-01 浙江大学 一种针对移动摄像机的背景减除方法
CN106204636A (zh) * 2016-06-27 2016-12-07 北京大学深圳研究生院 基于监控视频的视频前景提取方法

Also Published As

Publication number Publication date
CN106204636B (zh) 2019-03-22
US20190108642A1 (en) 2019-04-11
US10395374B2 (en) 2019-08-27
CN106204636A (zh) 2016-12-07

Similar Documents

Publication Publication Date Title
WO2018000882A1 (zh) 基于监控视频的视频前景提取方法
CN106447674B (zh) 一种视频的背景去除方法
CN107016691B (zh) 基于超像素特征的运动目标检测方法
CN105303581B (zh) 一种自适应参数的运动目标检测方法
CN103258332B (zh) 一种抗光照变化的运动目标的检测方法
JP2018508875A5 (zh)
CN102542571A (zh) 一种运动目标检测方法及装置
CN106462975A (zh) 用于对象跟踪和经由背景跟踪进行分割的方法和装置
JP2021182320A5 (zh)
CN103942812A (zh) 基于混合高斯与边缘检测的运动目标检测方法
CN104715480B (zh) 一种基于统计背景模型的目标检测方法
CN103729828B (zh) 视频去雨方法
CN105205791A (zh) 一种基于高斯混合模型的视频雨滴去除方法及其系统
CN104680521A (zh) 一种改进的背景建模及前景检测方法
CN101916449A (zh) 建立图像处理中基于运动信息的背景模型的方法
CN110490899A (zh) 一种结合目标跟踪的可变形施工机械的实时检测方法
CN101533511B (zh) 更新背景图像的方法和装置
CN105825234A (zh) 一种融合超像素和背景模型的前景检测方法
CN108010050B (zh) 一种基于自适应背景更新和选择性背景更新的前景检测方法
CN105139358A (zh) 一种结合形态学和模糊c聚类的视频雨滴去除方法及其系统
CN107292910B (zh) 一种基于像素建模的移动相机下运动目标检测方法
CN111079516A (zh) 基于深度神经网络的行人步态分割方法
CN105741277A (zh) 基于ViBe算法与SLIC超像素的背景差分法
CN104899557B (zh) 一种基于视频的交叉路口背景图像提取方法
CN110136164B (zh) 基于在线透射变换、低秩稀疏矩阵分解去除动态背景的方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17818881

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 17818881

Country of ref document: EP

Kind code of ref document: A1