CN101916449A - Method for establishing background model based on motion information during image processing - Google Patents
Method for establishing background model based on motion information during image processing Download PDFInfo
- Publication number
- CN101916449A CN101916449A CN 201010259541 CN201010259541A CN101916449A CN 101916449 A CN101916449 A CN 101916449A CN 201010259541 CN201010259541 CN 201010259541 CN 201010259541 A CN201010259541 A CN 201010259541A CN 101916449 A CN101916449 A CN 101916449A
- Authority
- CN
- China
- Prior art keywords
- pixel
- background model
- frame
- image
- energy
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Abstract
The invention relates to a method for establishing a background model based on motion information during image processing, belonging to the technical field of image processing. The method comprises the following steps of: (1) estimating a moving field between adjacent frames; estimating the moving field of each pixel point of an integral image by utilizing the invariance of pixel gray before and after motion for each pair of the adjacent frames input to videos, wherein the pixel point of which the speed is higher than a certain threshold value is used as a foreground point so as to obtain an initial foreground segmentation; and (2) estimating the background model by utilizing missing data: each frame obtained from the step (1) can be viewed as a sample point missing partial data in an original background space, the original background space is reestablishes through a series of sample points missing the partial data in the videos, and then the background model of each frame can be obtained through reprojection.
Description
Technical field
What the present invention relates to is a kind of modeling method of technical field of image processing, and being specifically related to is a kind of method of setting up in the Flame Image Process based on the background model of movable information.
Background technology
Along with being extensive use of of video monitoring, the treatment technology of video data has obtained very big concern, for example pedestrian detection, tracking, the research of aspects such as object modeling and attitude estimation.And background subtraction is used for extracting the prospect part of video by general being applied in the various technology, and this part generally includes interested research object, for example people or vehicle and other article etc.For traditional background subtraction method, its background model usually needs given in advance, for example takes the video of one section background, sets up the model of background according to the statistical nature---average and variance---of the color of each pixel then.There are two problems in this quadrat method, and the one, need extra step to set up background model, the 2nd, the background model of being set up is static, promptly loses efficacy after background environment changes.Therefore, can directly set up background model from the successive video frames that comprises sport foreground and background simultaneously, will be the significant improvement to classic method.
In the prior art document, background modeling method commonly used is the method based on gauss hybrid models that Stauffer and Grimson proposed in the paper Adaptive background mixturemodels for real-time tracking (the adaptive background mixture model in real-time follow-up) of Proc.IEEEConf.on Computer Vision and Pattern Recognition in 1999.Their method can adapt to illumination variation slowly and by for example branch rocks or shade causes multi-modal color distribution.But owing to do not utilize the movable information of prospect, smear problem can appear in this method, and promptly slowly moving of object can stay historical motion vestige in the foreground segmentation that obtains.Therefore, this area need a kind ofly can utilize the movable information of prospect to set up the method for background model.
Summary of the invention
The objective of the invention is at the deficiencies in the prior art, propose a kind of method of setting up in the Flame Image Process based on the background model of movable information.The present invention need not extra background acquisition step, utilize movable information to come the prospect part of image according to a preliminary estimate, utilize in the multiple image the imperfect background after the removal prospect that each frame is set up background model again, utilize background subtraction can obtain accurate foreground segmentation at last.Has practical application widely at aspects such as object and pedestrian's tracking, motion analysiss.
The present invention is achieved by the following technical solutions:
The present invention to the consecutive frame image, estimates the sports ground (light stream) of each pixel in the input video each by condition random field.Then with speed greater than the pixel of certain threshold value as the foreground point, will obtain each two field picture is obtained the initial segmentation of a prospect like this.For each frame in the list entries, can obtain such initial segmentation.If regard background as a space that changes in low dimension, so wherein each width of cloth can be counted as a sample point with missing data through the image of initial segmentation.Wherein Que Shi part is exactly the difference of being blocked by prospect.For these a series of sample points with missing data, this method utilizes the principal component analytical method under the missing data to reconstruct the spatial context of this low-dimensional.The background re-projection of excalation in each frame in this lower dimensional space, can be recovered the background of present frame.
The present invention includes following steps:
1. the estimation of sports ground between consecutive frame: to consecutive frame, the unchangeability of pixel grey scale is estimated the sports ground of each pixel of entire image before and after this method utilization campaign to each of input video.Its medium velocity is used as the foreground point greater than the pixel of certain threshold value, obtains an initial foreground segmentation.
2. utilize missing data to set up background model: in step 1 to each frame all can be regarded as a sample point that lacks partial data in the former spatial context.Sample point by a series of disappearance partial data in the video reconstructs former spatial context.By re-projection, obtain the background model of each frame subsequently.
Compared with prior art, method proposed by the invention need not extra background acquisition step, can adapt to simultaneously the linear change of background, for utilizing background subtraction to come the extraction prospect to have actual use value, and is better than original method.
Description of drawings
Fig. 1 is adjacent two frames in the input video.
Fig. 2 is the size of the sports ground that obtains according to the consecutive frame among Fig. 1.
Fig. 3 is the size according to sports ground, and the point that speed is non-vanishing is made as the segmentation result of foreground point.
The background model of Fig. 4 for obtaining according to a series of imperfect background image.
Embodiment
Below in conjunction with accompanying drawing embodiments of the invention are elaborated: following examples have provided detailed embodiment and process being to implement under the prerequisite with the technical solution of the present invention, but protection scope of the present invention is not limited to following embodiment.
Embodiment
1. the estimation of sports ground between consecutive frame.Fig. 1 is the image of two width of cloth frames in the input video among the embodiment.The unchangeability of pixel grey scale is meant for a pairing object point of pixel before and after the motion, when it moves to second frame by first frame, because the interval on time and space is all very short, can suppose that the gray scale of this point remains unchanged.In this condition, utilize condition random field can solve the motion vector of each pixel correspondence.
The sports ground of k frame is designated as v
k, suppose motion vector v at pixel x place
k(x) only with the motion vector v of its neighbor pixel y
k(y) and the image g of adjacent this two frame
K-1, g
kRelevant, that is:
p[v
k(x)|v
k(y)，g
k-1，g
k，y≠x]＝p[v
k(x)|v
k(y)，g
k-1，g
k，y∈N(x)] (1)
Wherein N (x) is meant the set with the x neighbor pixel.
Under such prerequisite, can separate and become the estimation minimum energy problem finding the solution maximum a posteriori probability.The energy of system is defined as:
Wherein two energy terms are defined as follows respectively: E
DataExpression current pixel assignment is v
k(x) energy is used for weighing the motion gray scale unchangeability of object pixel down.When a point from x
K-1Put the v that moved
k(x) arrive x
kBehind the point, if grey scale change is more little, this energy term is more little so, and vice versa.This energy term is defined as follows:
E
data[v
k(x)|g
k-1，g
k]＝λmin{|g
k-1[x-v
k(x)]-g
k(x)|，α} (3)
Here added one by α, made that its energy no longer increases when grey scale change during greater than α.This is used to handle the situation of blocking when between consecutive frame.λ is a zoom factor of regulating the energy size.
Another energy term E
SmoothThe motion vector of expression neighbor pixel x, y is respectively v
k(x) and v
k(y) energy the time is used for weighing the motion continuity between neighbor pixel, is defined as follows:
E
smooth[v
k(x)，v
k(y)|g
k-1，g
k]＝d(g
k，x，y)min[|v
k(x)-v
k(y)|，β] (4)
Here β be and formula (3) in α similarly by.D (g
k, x y) is one and describes image g
kThe function of changing features on middle x, the y pixel, when when x, y place image change are big, the value of this function reduces, and when variation was mild, functional value increased.Like this reason of Chu Liing be because for the motion continuity between neighbor pixel at the object edge place and be false, and the image change at object edge place is generally bigger, therefore the energy term that reduces this moment.Here choose with minor function:
Here Δ (x), Δ (y) is the Laplace transform of x, y place image.
At last, calculate the feasible value that is defined in the energy term minimum of sports ground between consecutive frame by belief propagation algorithm.This value is exactly to estimate for the maximum a posteriori of sports ground.For two consecutive frames that Fig. 1 among the embodiment provides, the size of the sports ground that its estimation obtains as shown in Figure 2.Wherein bright more point represents that movement velocity is big more, and the arm of personage's swing is the fastest part of movement velocity among the visible figure.
On this sports ground, setting speed is the foreground point greater than zero point (i.e. all points that move), and the segmentation result that obtains so as shown in Figure 3.Can see,, but effectively remove the part of motion though this preliminary segmentation result is not accurately cutting apart of a foreground object.Like this, Sheng Xia part just can be regarded as an incomplete background.
2. utilize missing data to set up background model
The principal component analysis method is as the common method of graphical analysis, and for a two field picture, its grey scale pixel value can sequence turn to a big vector.The main use of principal component analysis (PCA) is to reduce dimension, for a n-dimensional vector x, can it be projected as a k dimensional vector y by a matrix W:
y＝W
Tx (6)
The W here is the matrix of a k * n dimension.The best value W that can make y approach x in the k space promptly is called main composition.
For background modeling, though the enormous amount of pixel exists very strong relevance between pixel grey scale, the variation that is to say background is in a low-down dimension.For the image M of width of cloth m * n size, each row is regarded a n-dimensional vector as.Therefore, in the time of on projecting to main composition, each is worth Y
iBe a m dimensional vector:
Y＝MW (7)
For a series of incomplete background image that provides in the step 1, can find the solution this principal component analysis problem by expecting maximum algorithm, the result on embodiment is as shown in Figure 4.So image average M that is obtained
0With main composition C
i(i=1,2 ..., d) can be used for background modeling.Other is P
iThe projection of expression original image on i main composition, background can be redeveloped into so:
Claims (5)
1. a method of setting up in the Flame Image Process based on the background model of movable information is characterized in that, may further comprise the steps:
Step 1. the estimation of sports ground between consecutive frame: each of input video to consecutive frame, is estimated the sports ground of each pixel of entire image with the unchangeability of pixel grey scale before and after the motion;
Its medium velocity is used as the foreground point greater than the pixel of certain threshold value, obtains an initial foreground segmentation;
Step 2. utilize missing data to set up background model: in step 1 to each frame all can be regarded as a sample point that lacks partial data in the former spatial context;
Sample point by a series of disappearance partial data in the video reconstructs former spatial context;
By re-projection, obtain the background model of each frame subsequently.
2. the method for setting up in the Flame Image Process based on the background model of movable information according to claim 1, it is characterized in that, the unchangeability of pixel grey scale is meant for a pairing object point of pixel before and after the described motion, when it moves to second frame by first frame, because the interval on time and space is all very short, the gray scale of supposing this point remains unchanged, and in this condition, utilizes condition random to solve the motion vector of each pixel correspondence.
3. the method for setting up in the Flame Image Process based on the background model of movable information according to claim 1 is characterized in that described motion vector obtains by the following method:
The sports ground of k frame is designated as v
k, suppose motion vector v at pixel x place
k(x) only with the motion vector v of its neighbor pixel y
k(y) and the image g of adjacent this two frame
K-1, g
kRelevant, that is:
p[v
k(x)|v
k(y)，g
k-1，g
k，y≠x]＝p[v
k(x)|v
k(y)，g
k-1，g
k，y∈N(x)]
Wherein N (x) is meant the set with the x neighbor pixel;
Under such prerequisite, can separate and become the estimation minimum energy problem finding the solution maximum a posteriori probability:
The energy of system is defined as:
Wherein two energy terms are defined as follows respectively: E
DataExpression current pixel assignment is v
k(x) energy is used for weighing the motion gray scale unchangeability of object pixel down;
When a point from x
K-1Put the v that moved
k(x) arrive x
kBehind the point, if grey scale change is more little, this energy term is more little so, and vice versa.
4. the method for setting up in the Flame Image Process based on the background model of movable information according to claim 1 is characterized in that described energy term is defined as follows:
E
data[v
k(x)|g
k-1，g
k]＝λmin{|g
k-1[x-v
k(x)]-g
k(x)|，α} (3)
Here added one by α, made that its energy no longer increases when grey scale change during greater than α, this is used to handle the situation of blocking when between consecutive frame, and λ is the zoom factor of an adjusting energy size;
Another energy term E
SmoothThe motion vector of expression neighbor pixel x, y is respectively v
k(x) and v
k(y) energy the time is used for weighing the motion continuity between neighbor pixel, is defined as follows:
E
smooth[v
k(x)，v
k(y)|g
k-1，g
k]＝d(g
k，x，y)min[|v
k(x)-v
k(y)|，β]
Here β be and formula (3) in α similarly by.D (g
k, x y) is one and describes image g
kThe function of changing features on middle x, the y pixel, when when x, y place image change are big, the value of this function reduces, and when variation was mild, functional value increased; Therefore reduce energy term at this moment, choose with minor function:
Here Δ (x), Δ (y) is the Laplace transform of x, y place image;
At last, calculate the feasible value that is defined in the energy term minimum of sports ground between consecutive frame by the belief propagation method.
5. the method for setting up in the Flame Image Process based on the background model of movable information according to claim 1, it is characterized in that, described step 2 specifically is meant: for a two field picture, its grey scale pixel value can sequence turn to a big vector, for a n-dimensional vector x, it is projected as a k dimensional vector y by a matrix W:
v＝W
Tx
The W here is the matrix of a k * n dimension, and the best value W that can make y approach x in the k space promptly is called main composition;
Exist relevance between pixel grey scale, for the image M of width of cloth m * n size, each row is regarded a n-dimensional vector as, and therefore, in the time of on projecting to main composition, each is worth Y
iBe a m dimensional vector:
Y＝MW
For a series of incomplete background image that provides in the step 1, can find the solution this main composition, the image average M that is obtained
0With main composition C
i(i=1,2 ..., d) can be used for, in addition P background modeling
iThe projection of expression original image on i main composition, background can be redeveloped into so:
Be the background model that obtains.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 201010259541 CN101916449A (en) | 2010-08-21 | 2010-08-21 | Method for establishing background model based on motion information during image processing |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 201010259541 CN101916449A (en) | 2010-08-21 | 2010-08-21 | Method for establishing background model based on motion information during image processing |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN101916449A true CN101916449A (en) | 2010-12-15 |
Family
ID=43323953
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN 201010259541 Pending CN101916449A (en) | 2010-08-21 | 2010-08-21 | Method for establishing background model based on motion information during image processing |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN101916449A (en) |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102629383A (en) * | 2012-02-28 | 2012-08-08 | 湖南大学 | Motion object detection method based on random strategy |
| CN102833464A (en) * | 2012-07-24 | 2012-12-19 | 常州泰宇信息科技有限公司 | Method for structurally reconstructing background for intelligent video monitoring |
| CN104135612A (en) * | 2014-07-11 | 2014-11-05 | 深圳市中兴移动通信有限公司 | A shooting method and a shooting device with an adjustable location of a shot object |
| CN104680124A (en) * | 2013-11-28 | 2015-06-03 | 现代摩比斯株式会社 | Device And Method For Detecting Pedestrains |
| WO2015117464A1 (en) * | 2014-08-20 | 2015-08-13 | 中兴通讯股份有限公司 | Device and method for processing video image |
| CN107968917A (en) * | 2017-12-05 | 2018-04-27 | 广东欧珀移动通信有限公司 | Image processing method and device, computer equipment, computer-readable recording medium |
| CN111080674A (en) * | 2019-12-18 | 2020-04-28 | 上海无线电设备研究所 | Multi-target ISAR key point extraction method based on Gaussian mixture model |
| CN113111842A (en) * | 2021-04-26 | 2021-07-13 | 浙江商汤科技开发有限公司 | Action recognition method, device, equipment and computer readable storage medium |
| CN114240788A (en) * | 2021-12-21 | 2022-03-25 | 西南石油大学 | Robustness and self-adaptability background restoration method for complex scene |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1471694A (en) * | 2001-06-27 | 2004-01-28 | ���ṫ˾ | Image processing apparatus and method, and image pickup apparatus |
| US6766037B1 (en) * | 1998-10-02 | 2004-07-20 | Canon Kabushiki Kaisha | Segmenting moving objects and determining their motion |
| US20050104964A1 (en) * | 2001-10-22 | 2005-05-19 | Bovyrin Alexandr V. | Method and apparatus for background segmentation based on motion localization |
-
2010
- 2010-08-21 CN CN 201010259541 patent/CN101916449A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6766037B1 (en) * | 1998-10-02 | 2004-07-20 | Canon Kabushiki Kaisha | Segmenting moving objects and determining their motion |
| CN1471694A (en) * | 2001-06-27 | 2004-01-28 | ���ṫ˾ | Image processing apparatus and method, and image pickup apparatus |
| US20050104964A1 (en) * | 2001-10-22 | 2005-05-19 | Bovyrin Alexandr V. | Method and apparatus for background segmentation based on motion localization |
Non-Patent Citations (1)
| Title |
|---|
| 《Optical Engineering》 20091231 Ming Zhang etc. Motion-based background subtraction 第127004-1-127004-5页 1-5 第8卷, 第12期 2 * |
Cited By (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102629383A (en) * | 2012-02-28 | 2012-08-08 | 湖南大学 | Motion object detection method based on random strategy |
| CN102833464A (en) * | 2012-07-24 | 2012-12-19 | 常州泰宇信息科技有限公司 | Method for structurally reconstructing background for intelligent video monitoring |
| CN102833464B (en) * | 2012-07-24 | 2015-06-17 | 常州展华机器人有限公司 | Method for structurally reconstructing background for intelligent video monitoring |
| CN104680124A (en) * | 2013-11-28 | 2015-06-03 | 现代摩比斯株式会社 | Device And Method For Detecting Pedestrains |
| CN104135612A (en) * | 2014-07-11 | 2014-11-05 | 深圳市中兴移动通信有限公司 | A shooting method and a shooting device with an adjustable location of a shot object |
| WO2015117464A1 (en) * | 2014-08-20 | 2015-08-13 | 中兴通讯股份有限公司 | Device and method for processing video image |
| CN107968917A (en) * | 2017-12-05 | 2018-04-27 | 广东欧珀移动通信有限公司 | Image processing method and device, computer equipment, computer-readable recording medium |
| CN107968917B (en) * | 2017-12-05 | 2019-10-18 | Oppo广东移动通信有限公司 | Image processing method and device, computer equipment, computer readable storage medium |
| CN111080674A (en) * | 2019-12-18 | 2020-04-28 | 上海无线电设备研究所 | Multi-target ISAR key point extraction method based on Gaussian mixture model |
| CN111080674B (en) * | 2019-12-18 | 2023-11-14 | 上海无线电设备研究所 | Multi-target ISAR key point extraction method based on Gaussian mixture model |
| CN113111842A (en) * | 2021-04-26 | 2021-07-13 | 浙江商汤科技开发有限公司 | Action recognition method, device, equipment and computer readable storage medium |
| CN114240788A (en) * | 2021-12-21 | 2022-03-25 | 西南石油大学 | Robustness and self-adaptability background restoration method for complex scene |
| CN114240788B (en) * | 2021-12-21 | 2023-09-08 | 西南石油大学 | Complex scene-oriented robustness and adaptive background restoration method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN101916449A (en) | Method for establishing background model based on motion information during image processing | |
| Han et al. | Density-based multifeature background subtraction with support vector machine | |
| CN106846359A (en) | Moving target method for quick based on video sequence | |
| Yi et al. | Moving object detection based on running average background and temporal difference | |
| Tavakkoli et al. | Non-parametric statistical background modeling for efficient foreground region detection | |
| WO2022027931A1 (en) | Video image-based foreground detection method for vehicle in motion | |
| CN105261037A (en) | Moving object detection method capable of automatically adapting to complex scenes | |
| CN105427626A (en) | Vehicle flow statistics method based on video analysis | |
| CN103077539A (en) | Moving object tracking method under complicated background and sheltering condition | |
| CN103530893A (en) | Foreground detection method in camera shake scene based on background subtraction and motion information | |
| CN101493944A (en) | Moving target detecting and tracking method and system | |
| CN102156995A (en) | Video movement foreground dividing method in moving camera | |
| CN105205791A (en) | Gaussian-mixture-model-based video raindrop removing method and system | |
| Yoshinaga et al. | Background model based on intensity change similarity among pixels | |
| Mayr et al. | Self-supervised learning of the drivable area for autonomous vehicles | |
| CN113763427A (en) | Multi-target tracking method based on coarse-fine shielding processing | |
| Roy et al. | A comprehensive survey on computer vision based approaches for moving object detection | |
| Devi et al. | A survey on different background subtraction method for moving object detection | |
| CN105139358A (en) | Video raindrop removing method and system based on combination of morphology and fuzzy C clustering | |
| Hsieh et al. | Grid-based template matching for people counting | |
| Chen et al. | Real-time robust hand tracking based on camshift and motion velocity | |
| CN114821441A (en) | Deep learning-based airport scene moving target identification method combined with ADS-B information | |
| Valiere et al. | Robust vehicle counting with severe shadows and occlusions | |
| Nguyen et al. | M2gan: A multi-stage self-attention network for image rain removal on autonomous vehicles | |
| Dan et al. | A multi-object motion-tracking method for video surveillance |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C02 | Deemed withdrawal of patent application after publication (patent law 2001) | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Open date: 20101215 |