CN104240267A - Moving object detection method based on three-dimensional structure coordinate constraint - Google Patents
Moving object detection method based on three-dimensional structure coordinate constraint Download PDFInfo
- Publication number
- CN104240267A CN104240267A CN201410481725.7A CN201410481725A CN104240267A CN 104240267 A CN104240267 A CN 104240267A CN 201410481725 A CN201410481725 A CN 201410481725A CN 104240267 A CN104240267 A CN 104240267A
- Authority
- CN
- China
- Prior art keywords
- constraint
- point
- match point
- formula
- meeting
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Abstract
The invention provides a moving object detection method based on three-dimensional structure coordinate constraint. The method comprises the steps that firstly, matching points in three adjacent frames of images are distinguished into matching points meeting epipolar constraint and matching points not meeting the epipolar constraint through an epipolar constraint method; then, the three-dimensional structure coordinates of the matching points meeting the epipolar constraint are figured out through a Plane and Parallax method, and a three-dimensional structure coordinate constraint method is used for distinguishing the matching points meeting the structure coordinate constraint and the matching points not meeting the structure coordinate constraint; finally, the matching points not meeting the epipolar constraint and the matching points not meeting the structure coordinate constraint are combined, and a moving object is obtained. According to the moving object detection method, the problem that the detection result false alarm rate is high on the high parallax condition is solved, and the movement degradation condition under two views is improved.
Description
Technical field
The invention belongs to image detection and process field, be specifically related to a kind of moving target detecting method based on three-dimensional structure coordinates restriction.
Background technology
Under mobile platform, how to carry out effective moving object detection is a key issue in machine vision and intelligent monitoring.Under mobile platform, all there is motion in the picture in moving target and background, the classic method solving this kind of problem is mainly divided into two kinds, a kind of method is ignored by the parallax existed in video image, think that adjacent two two field pictures meet the conversion of simple homography matrix or affined transformation, utilize homography matrix to estimate the global motion of background in two width images; Another kind method is then analyze the internal and external parameter of camera, utilizes the method for Geometric Modeling to extrapolate the optical flow field in image, finds the optical flow field restriction relation that background dot meets.But, these two kinds of methods can only be applicable to the inapparent situation of image parallactic, such as to take photo by plane video or the shorter occasion of camera focus, and it is helpless for there is the moving object detection of (such as public arena is closely monitored) in larger parallax situation in video image, often easy background dot larger for parallax is also sentenced make moving target, formed false-alarm.
The people such as Yan Zhang demonstrate under pure flat condition of shifting one's love, background light stream will converge in image certain a bit, i.e. FOE, thus detect moving target; Amnon Shashua proposes the affine structure of class in camera imaging, proposes famous Plane & Parallax method.But these two kinds of methods all with two view geometry for foundation, be unable to do without basis matrix (Fundamental Matrix), the motor deterioration situation be directed under two view geometry is then helpless, namely when target is consistent with camera moving direction, said method cannot detect moving target, thus there is undetected situation, false alarm rate is high.
The image compensation method proposed in document Moving Object Localization in Thermal Imagery by Forward-backward MHI, homography matrix satisfied between consecutive frame image is estimated by robustness, eliminate the global motion of the image background caused because of camera motion to a certain extent, but the parallax that cannot exist in removal of images, thus by background dot larger for parallax in image also as moving target, appoint and so there is the high problem of false alarm rate.
Summary of the invention
The present invention proposes a kind of moving target detecting method based on three-dimensional structure coordinates restriction, solves the problem that testing result false alarm rate in strong parallax situation is high, also makes moderate progress simultaneously to the motor deterioration situation under two views.
In order to solve the problems of the technologies described above, the invention provides a kind of moving target detecting method based on three-dimensional structure coordinates restriction, first to the match point in adjacent three two field pictures, the match point meeting epipolar-line constraint and the match point not meeting epipolar-line constraint is distinguished by the method for epipolar-line constraint; Then for the match point meeting epipolar-line constraint, utilize Plane & Parallax method to calculate the three-dimensional structure coordinate of match point, use three-dimensional structure coordinates restriction method to distinguish the match point meeting structure coordinate constraint and the match point not meeting structure coordinate constraint; Finally will not meet the match point of epipolar-line constraint and not meet the match point merging acquisition moving target of structure coordinate constraint.
The present invention compared with prior art, its remarkable advantage is: (1) the method in video image adjacent three two field pictures as analytic target, for moving object detection provides more multi-constraint condition, thus its degenerative conditions is very harsh, effectively improve the verification and measurement ratio of moving object detection.(2) the method utilizes the structure coordinate restriction relation of three dimensions point, consider the depth information of three dimensions point, thus be applicable to the situation that there is strong parallax in image, effectively effectively can classify to moving target, strong parallax background dot and general background dot, thus greatly reduce false alarm rate.
Accompanying drawing explanation
Fig. 1 is the theory diagram of the inventive method.
Fig. 2 is the Plane & Parallax Method And Principle figure used in the present invention.
Fig. 3 is adjacent three two field pictures of the pending sequence of video images used in emulation experiment of the present invention, and wherein, (a) is previous frame image, and (b) is intermediate frame image, and (c) is a rear two field picture.
Fig. 4 is the testing result using existing method to obtain in emulation experiment of the present invention.
Fig. 5 is the testing result using the inventive method to obtain in emulation experiment of the present invention.
Embodiment
The present invention by find a large amount of match point robustness ground estimation background dot in video image the various visual angles geometrical-restriction relation that meets, thus all match points to be classified, find out match point that discontented instep scape retrains as moving target.
Concrete as shown in Figure 1, the present invention in video image adjacent three two field pictures as analytic target,
First by the method for epipolar-line constraint, a classification and Detection is carried out to the match point in adjacent three two field pictures, distinguish the match point meeting epipolar-line constraint and the match point not meeting epipolar-line constraint, the match point meeting epipolar-line constraint is classified as a class, be classified as another kind of by the match point not meeting epipolar-line constraint, the match point wherein not meeting epipolar-line constraint is exactly the moving target obtained after a classification and Detection;
Then, for the match point meeting epipolar-line constraint, Plane & Parallax (principal plane+parallax) method is utilized to calculate the three-dimensional structure coordinate of match point, three-dimensional structure coordinates restriction is used to carry out secondary classification detection to the match point meeting epipolar-line constraint, distinguish the match point meeting structure coordinate constraint and the match point not meeting structure coordinate constraint, the match point wherein not meeting structure coordinate constraint is exactly the moving target obtained after secondary classification detects;
Finally, double classification is detected the moving target merging obtained and namely obtain the moving target finally detected.
Aforementioned Plane & Parallax (principal plane+parallax) method can see see document Relative Affine Structure:Canonical Model for3D From2D Geometry and Applications.
In the present invention, the described method by epipolar-line constraint to the process that the match point in adjacent three two field pictures carries out a classification and Detection is:
Step 101, the matching double points asked in adjacent three two field pictures.Adjacent adjacent three two field pictures are respectively previous frame image, intermediate frame image and a rear two field picture.First detect the unique point (such as Harris unique point etc.) of intermediate frame image, then the method (see document Good Features to Track) utilizing KLT point to follow the tracks of follows the tracks of two groups of match points of unique point respectively in the two field picture of front and back two obtaining intermediate frame image, gets the match point of common factor as adjacent three two field pictures of two groups of match points.
Step 102, basis matrix (Fundamental Matrix) F estimating between previous frame image and intermediate frame image
12, try to achieve the basis matrix residual error d of each match point
i, the basis matrix residual error d of each match point
ias shown in formula (1),
In formula (1), x
iwith x '
ibe illustrated respectively in i-th unique point corresponding in previous frame image and intermediate frame image, T is transpose operator;
Whether step 103, judgment formula (2) are set up, if formula (2) is set up, then this unique point meets epipolar-line constraint; If formula (2) is false, then illustrates that this unique point does not meet epipolar-line constraint, the set of the match point not meeting epipolar-line constraint is denoted as R
1, the moving target namely obtained after a classification and Detection,
d
i＜α·med{d
1,d
2,...} (4)
In formula (2), α is given coefficient threshold, and med is for getting median.
In the present invention, described use three-dimensional structure coordinates restriction to the process that the match point secondary classification meeting epipolar-line constraint detects is:
Step 201, the match point meeting epipolar-line constraint obtained for step 103, utilize Plane & Parallax method to calculate the three-dimensional structure coordinate of match point.
Plane & Parallax Method And Principle as shown in Figure 2.In three dimensions, spatial point X imaging in the two-phase mechanical coke plane of left and right is respectively left view x and x ', and π plane, induction of the homograph matrix H of left view to right view, puts x '
πfor x passes through π Planar Mapping to the point on right view, i.e. x '
π=Hx, according to the known right limit e ' of geometric relationship, x '
πwith x ' conllinear, thus the available formula of x ' (3) represents,
x′＝Hx+ke′ (5)
In formula (3), k characterizes the departure degree of X and π plane, and even k is more close to 0, then illustrate that X distance π plane is nearer, therefore, k represents " degree of depth " of X in three dimensions.
Make s=(x
t, k)
t, x is the picture point of spatial point X on left camera focal plane, and s is the structure coordinate of an X.
tfor transposition.
The present invention is according to Plane & Parallax method, the structure coordinate s of spatial point X between previous frame image and intermediate frame image is obtained by previous frame image and intermediate frame image, in like manner, the structure coordinate s ' between intermediate frame image and a rear two field picture of X is obtained by intermediate frame image and a rear two field picture, now, if X is background dot, then s and s ' meets the restriction relation as shown in formula (4)
s
′TGs＝0 (6)
In formula (4), G is the matrix of 4 × 4, and its order is 2.
The residual epsilon of step 202, calculating unique point
i, account form as shown in formula (5),
Whether step 203, judgment formula (6) are set up, if formula (6) is set up, then characterization point meets structure coordinate constraint; If formula (6) is false, then characterization point does not meet structure coordinate constraint, and the unique point set not meeting structure coordinate constraint is denoted as R
2, namely secondary classification detects the rear moving target obtained,
ε
i＜β·med{ε
1,ε
2,...} (8)
In formula (6), β is given coefficient threshold, and med is for getting median.
Step 4, by double classification result merge, by subseries detect after obtain moving target R
1the moving target R obtained after detecting with secondary classification
2merge, thus obtain final moving object detection result R, as shown in formula (7),
R＝R
1∪R
2 (9)
Effect of the present invention can be described further by following emulation experiment:
Fig. 3 is that this tests continuous three two field pictures used, wherein, a () is previous frame image, b () is intermediate frame image, (c) is a rear two field picture, in three two field pictures, tank model is moving target, books, bottle and paper handkerchief are background, and because scene distance camera is comparatively near, video image exists larger parallax.
Fig. 4 adopts the image compensation method proposed in document Moving Object Localization in Thermal Imagery by Forward-backward MHI, three two field pictures continuous in Fig. 2 are processed, the impact point detected, in figure, every a pair circle and cross are the moving target point detected.As can be seen from the figure, because bottle is nearer apart from camera, parallax reason causes background well not compensated, thus is treated as moving target.
Fig. 5 processes three two field pictures continuous in Fig. 2 for have employed the inventive method, the impact point detected, and in figure, every a pair circle and cross are the moving target point detected.As can be seen from the figure, impact point is all on the tank of motion, and the impact point on static bottle is all by filtering, and false alarm rate reduces greatly.
Claims (6)
1., based on a moving target detecting method for three-dimensional structure coordinates restriction, it is characterized in that,
First, to the match point in adjacent three two field pictures, distinguish the match point meeting epipolar-line constraint and the match point not meeting epipolar-line constraint by the method for epipolar-line constraint;
Then, for the match point meeting epipolar-line constraint, utilize Plane & Parallax method to calculate the three-dimensional structure coordinate of match point, use three-dimensional structure coordinates restriction method to distinguish the match point meeting structure coordinate constraint and the match point not meeting structure coordinate constraint;
Finally, will the match point of epipolar-line constraint do not met and not meet the match point merging acquisition moving target of structure coordinate constraint.
2. as claimed in claim 1 based on the moving target detecting method of three-dimensional structure coordinates restriction, it is characterized in that, the method for the match point that described differentiation meets epipolar-line constraint and the match point that do not meet epipolar-line constraint is:
Step 101, the matching double points asked in adjacent three two field pictures;
Step 102, the basis matrix F estimating between previous frame image and intermediate frame image
12, try to achieve the basis matrix residual error d of each match point
i, the basis matrix residual error d of each match point
icalculating as shown in formula (1),
In formula (1), x
iwith x '
ibe illustrated respectively in i-th unique point corresponding in previous frame image and intermediate frame image, T is transpose operator;
Whether step 103, judgment formula (2) are set up, if formula (2) is set up, then this unique point meets epipolar-line constraint; If formula (2) is false, then this unique point does not meet epipolar-line constraint,
d
i＜α·med{d
1,d
2,...}(2)
In formula (2), α is given coefficient threshold, and med is for getting median.
3. as claimed in claim 2 based on the moving target detecting method of three-dimensional structure coordinates restriction, it is characterized in that, the method asking for the matching double points in adjacent three two field pictures is:
First detect the Harris unique point of intermediate frame image, the method then utilizing KLT point to follow the tracks of follows the tracks of the two group match points of unique point respectively in the two field picture of front and back two obtaining intermediate frame image, gets the match point of common factor as adjacent three two field pictures of two groups of match points.
4. as claimed in claim 2 based on the moving target detecting method of three-dimensional structure coordinates restriction, it is characterized in that, the three-dimensional structure coordinate s=(x of described match point
t, k)
t, wherein, x is the picture point of spatial point in adjacent two two field pictures in previous frame image.
5. as claimed in claim 4 based on the moving target detecting method of three-dimensional structure coordinates restriction, it is characterized in that, the method distinguishing the match point meeting structure coordinate constraint and the match point not meeting structure coordinate constraint is:
The residual epsilon of step 201, calculating unique point
i, account form as shown in formula (3),
In formula (3), s obtains the structure coordinate of spatial point between previous frame image and intermediate frame image by previous frame image and intermediate frame image; The structure coordinate intermediate frame image and a rear two field picture between of s ' for being obtained by intermediate frame image and a rear two field picture; G is the matrix of 4 × 4, and its order is 2; T is transpose operator;
Whether step 202, judgment formula (4) are set up, if formula (4) is set up, then this unique point meets structure coordinate constraint; If formula (4) is false, then this unique point does not meet structure coordinate constraint,
ε
i＜β·med{ε
1,ε
2,...} (4)
In formula (4), β is given coefficient threshold, and med is for getting median.
6., as claimed in claim 5 based on the moving target detecting method of three-dimensional structure coordinates restriction, it is characterized in that, by do not meet epipolar-line constraint match point and do not meet structure coordinate constraint match point merge mode as shown in formula (5),
R＝R
1∪R
2 (5)
Formula (5), R is the final moving target detected, R
1for not meeting the set of the match point of epipolar-line constraint, R
2for not meeting the set of the match point of structure coordinate constraint.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410481725.7A CN104240267B (en) | 2014-09-19 | 2014-09-19 | A kind of moving target detecting method based on three-dimensional structure coordinates restriction |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410481725.7A CN104240267B (en) | 2014-09-19 | 2014-09-19 | A kind of moving target detecting method based on three-dimensional structure coordinates restriction |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN104240267A true CN104240267A (en) | 2014-12-24 |
| CN104240267B CN104240267B (en) | 2017-06-27 |
Family
ID=52228273
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410481725.7A Expired - Fee Related CN104240267B (en) | 2014-09-19 | 2014-09-19 | A kind of moving target detecting method based on three-dimensional structure coordinates restriction |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104240267B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104680516A (en) * | 2015-01-08 | 2015-06-03 | 南京邮电大学 | Acquisition method for high-quality feature matching set of images |
| CN109902725A (en) * | 2019-01-31 | 2019-06-18 | 北京达佳互联信息技术有限公司 | Mobile mesh object detection method, device and electronic equipment and storage medium |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080273751A1 (en) * | 2006-10-16 | 2008-11-06 | Chang Yuan | Detection and Tracking of Moving Objects from a Moving Platform in Presence of Strong Parallax |
| CN203397395U (en) * | 2013-07-19 | 2014-01-15 | 南京理工大学 | Moving object detection device in platform based on DSP + FPGA |
-
2014
- 2014-09-19 CN CN201410481725.7A patent/CN104240267B/en not_active Expired - Fee Related
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080273751A1 (en) * | 2006-10-16 | 2008-11-06 | Chang Yuan | Detection and Tracking of Moving Objects from a Moving Platform in Presence of Strong Parallax |
| CN203397395U (en) * | 2013-07-19 | 2014-01-15 | 南京理工大学 | Moving object detection device in platform based on DSP + FPGA |
Non-Patent Citations (3)
| Title |
|---|
| A. SHASHUA ET AL.: "Relative Affine Structure:Canonical Model for 3D From 2D Geometry and Applications", 《IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE》 * |
| 易科: "动态场景下运动目标检测与跟踪", 《中国优秀硕士论文全文数据库 信息科技辑》 * |
| 陈海欣 等: "一种快速的单目移动相机下运动目标检测方法", 《光子学报》 * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104680516A (en) * | 2015-01-08 | 2015-06-03 | 南京邮电大学 | Acquisition method for high-quality feature matching set of images |
| CN104680516B (en) * | 2015-01-08 | 2017-09-29 | 南京邮电大学 | A kind of acquisition methods of image quality features set of matches |
| CN109902725A (en) * | 2019-01-31 | 2019-06-18 | 北京达佳互联信息技术有限公司 | Mobile mesh object detection method, device and electronic equipment and storage medium |
| US11176687B2 (en) | 2019-01-31 | 2021-11-16 | Beijing Dajia Internet Information Technology Co., Ltd | Method and apparatus for detecting moving target, and electronic equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN104240267B (en) | 2017-06-27 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Hu et al. | Moving object detection and tracking from video captured by moving camera | |
| Black et al. | Multi camera image tracking | |
| Resendiz et al. | Automated visual inspection of railroad tracks | |
| Klappstein et al. | Moving object segmentation using optical flow and depth information | |
| Rougier et al. | Fall detection from depth map video sequences | |
| CN105260719B (en) | Railroad platform gets over line detecting method | |
| CN103021177B (en) | Method and system for processing traffic monitoring video image in foggy day | |
| CN105225482A (en) | Based on vehicle detecting system and the method for binocular stereo vision | |
| CN103458261B (en) | Video scene variation detection method based on stereoscopic vision | |
| CN101876535B (en) | A kind of height measurement method, device and supervisory system | |
| CN105915846A (en) | Monocular and binocular multiplexed invading object monitoring method and system | |
| CN105069751A (en) | Depth image missing data interpolation method | |
| EP2476999B1 (en) | Method for measuring displacement, device for measuring displacement, and program for measuring displacement | |
| CN106033614A (en) | Moving object detection method of mobile camera under high parallax | |
| CN112836634A (en) | Multi-sensor information fusion gate trailing prevention method, device, equipment and medium | |
| CN104240267A (en) | Moving object detection method based on three-dimensional structure coordinate constraint | |
| Zhou et al. | On modeling ego-motion uncertainty for moving object detection from a mobile platform | |
| Tan et al. | Vehicle speed measurement for accident scene investigation | |
| Ziran et al. | A contactless solution for monitoring social distancing: A stereo vision enabled real-time human distance measuring system | |
| CN102142148B (en) | Video space-time feature extraction method | |
| Sato et al. | Efficient hundreds-baseline stereo by counting interest points for moving omni-directional multi-camera system | |
| Xiong et al. | Crowd density estimation based on image potential energy model | |
| CN101685538A (en) | Method and device for tracking object | |
| Joshi | Vehicle speed determination using image processing | |
| CN104299224B (en) | Method for property protection based on video image background matching |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20170627 Termination date: 20190919 |