CN104240267B - A kind of moving target detecting method based on three-dimensional structure coordinates restriction - Google Patents
A kind of moving target detecting method based on three-dimensional structure coordinates restriction Download PDFInfo
- Publication number
- CN104240267B CN104240267B CN201410481725.7A CN201410481725A CN104240267B CN 104240267 B CN104240267 B CN 104240267B CN 201410481725 A CN201410481725 A CN 201410481725A CN 104240267 B CN104240267 B CN 104240267B
- Authority
- CN
- China
- Prior art keywords
- match point
- constraint
- point
- epipolar
- field picture
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Landscapes
- Image Processing (AREA)
- Image Analysis (AREA)
Abstract
The present invention proposes a kind of moving target detecting method based on three-dimensional structure coordinates restriction.First to the match point in adjacent three two field picture, the match point for meeting epipolar-line constraint and the match point for being unsatisfactory for epipolar-line constraint are distinguished by the method for epipolar-line constraint；Then for the match point for meeting epipolar-line constraint, the three-dimensional structure coordinate of match point is calculated using Plane＆Parallax methods, the match point for meeting structure coordinate constraint and the match point for being unsatisfactory for structure coordinate constraint are distinguished using three-dimensional structure coordinates restriction method；The match point that epipolar-line constraint will be finally unsatisfactory for and the match point for being unsatisfactory for structure coordinate constraint merge acquisition moving target.Testing result false alarm rate is high in the case of the present invention solves the problems, such as strong parallax, while also being made moderate progress to the motor deterioration situation under two views.
Description
Technical field
The invention belongs to image detection and process field, and in particular to a kind of motion mesh based on three-dimensional structure coordinates restriction
Mark detection method.
Background technology
It is that machine vision is closed with intelligent monitoring that effective moving object detection how is carried out under mobile platform
Key problem.Under mobile platform, to there is the conventional method of such issues that motion, solution in moving target and background main in the picture
It is divided into two kinds, a kind of method is to ignore parallax present in video image, it is believed that adjacent two field pictures meet simple list should
Matrixing or affine transformation, the global motion of background in two images is estimated using homography matrix；Another method is then
The internal and external parameter of camera is analyzed, the optical flow field in image is extrapolated using the method for Geometric Modeling, found background dot and meet
Optical flow field restriction relation.However, both approaches may be only available for the inapparent situation of image parallactic, for example take photo by plane video or
The shorter occasion of camera focus, and (for example public arena closely monitors in the case of there is larger parallax in being directed to video image
Deng) moving object detection it is helpless, tend to also sentence the larger background dot of parallax and make moving target, formed empty
It is alert.
Yan Zhang et al. are demonstrated in the case of pure flat shifting, and background light stream will converge to the certain point in image, i.e.,
FOE, so as to detect moving target；Amnon Shashua propose the affine structure of class in camera imaging, it is proposed that famous
Plane＆Parallax methods.But both approaches are foundation with two view geometries, basis matrix is be unable to do without
(Fundamental Matrix), and it is then helpless to be directed to the motor deterioration situation under two view geometries, i.e., when target with
When camera moving direction is consistent, the above method will be unable to detect moving target, and so as to missing inspection situation occur, false alarm rate is high.
Document Moving Object Localization in Thermal Imagery by Forward-backward
The image compensation method proposed in MHI, the homography matrix met between adjacent two field picture is estimated by robustness, to a certain degree
On eliminate because of the global motion of the image background that camera motion causes, but parallax present in image cannot be eliminated so that
The larger background dot of parallax in image is also regarded into moving target, is appointed and so be there is a problem of that false alarm rate is high.
The content of the invention
The present invention proposes a kind of moving target detecting method based on three-dimensional structure coordinates restriction, solves strong parallax situation
Lower testing result false alarm rate problem high, while also being made moderate progress to the motor deterioration situation under two views.
In order to solve the above-mentioned technical problem, the present invention provides a kind of moving object detection based on three-dimensional structure coordinates restriction
Method, first to the match point in adjacent three two field picture, the matching for meeting epipolar-line constraint is distinguished by the method for epipolar-line constraint
Put and be unsatisfactory for the match point of epipolar-line constraint；Then for the match point for meeting epipolar-line constraint, using Plane＆Parallax side
Method calculates the three-dimensional structure coordinate of match point, is distinguished using three-dimensional structure coordinates restriction method and meets structure coordinate constraint
Match point and the match point for being unsatisfactory for structure coordinate constraint；The match point of epipolar-line constraint will be finally unsatisfactory for and structure seat is unsatisfactory for
The match point for marking constraint merges acquisition moving target.
Compared with prior art, its remarkable advantage is the present invention：(1) the method is with adjacent three two field picture in video image
As analysis object, for moving object detection provides more multi-constraint condition, so that its degenerative conditions is extremely harsh, effectively improve
The verification and measurement ratio of moving object detection.(2) the method utilizes the structure coordinate restriction relation of three dimensions point, it is contemplated that three-dimensional space
Between the depth information put, so as to there is the situation of strong parallax suitable for image, can effectively to moving target, strong parallax background
Point and general background dot are effectively classified, so as to substantially reduce false alarm rate.
Brief description of the drawings
Fig. 1 is the theory diagram of the inventive method.
Fig. 2 is the Plane＆Parallax method schematics used in the present invention.
Fig. 3 is adjacent three two field picture of the pending sequence of video images used in emulation experiment of the present invention, wherein,
A () is previous frame image, (b) is middle two field picture, and (c) is latter two field picture.
Fig. 4 is the testing result for using in emulation experiment of the present invention existing method to obtain.
Fig. 5 is the testing result for using in emulation experiment of the present invention the inventive method to obtain.
Specific embodiment
The present invention estimates that the various visual angles that background dot is met are several to a large amount of match point robustness by finding in video image
What restriction relation, so as to classify to all of match point, finds out the match point for being unsatisfactory for background constraint as moving target.
Specifically as shown in figure 1, the present invention is using adjacent three two field picture in video image as analysis object,
A classification and Detection is carried out to the match point in adjacent three two field picture by the method for epipolar-line constraint first, is distinguished
The match point for meeting epipolar-line constraint and the match point for being unsatisfactory for epipolar-line constraint, the match point that will meet epipolar-line constraint are classified as a class,
To be unsatisfactory for epipolar-line constraint match point be classified as it is another kind of, wherein the match point for being unsatisfactory for epipolar-line constraint is exactly a classification and Detection
The moving target for obtaining afterwards；
Then, for the match point for meeting epipolar-line constraint, calculated using Plane＆Parallax (principal plane+parallax) method
Go out the three-dimensional structure coordinate of match point, secondary classification is carried out to the match point for meeting epipolar-line constraint using three-dimensional structure coordinates restriction
Detection, distinguishes and meets the match point of structure coordinate constraint and be unsatisfactory for the match point of structure coordinate constraint, wherein being unsatisfactory for knot
The match point of structure coordinates restriction is exactly the moving target of acquisition after secondary classification detection；
Finally, the moving target that double classification detection is obtained is merged the moving target for obtaining and eventually detecting.
Foregoing Plane＆Parallax (principal plane+parallax) method may refer to referring to document Relative Affine
Structure:Canonical Model for3D From2D Geometry and Applications。
In the present invention, the method by epipolar-line constraint carries out a subseries inspection to the match point in adjacent three two field pictures
The process of survey is：
Step 101, the matching double points asked in adjacent three two field picture.Adjacent adjacent three two field picture be respectively previous frame image,
Middle two field picture and latter two field picture.The characteristic point (such as Harris characteristic points etc.) of two field picture, then utilizes in the middle of first detecting
Method (referring to the document Good Features to Track) tracking of KLT points tracking obtains the characteristic point difference of middle two field picture
Two groups of match points in front and rear two field pictures, take match point of two groups of common factors of match point as adjacent three two field picture.
Basis matrix (Fundamental Matrix) between step 102, estimation previous frame image and middle two field picture
F12, try to achieve the basis matrix residual error d of each match pointi, the basis matrix residual error d of each match pointiAs shown in formula (1),
In formula (1), xiWith x 'iIt is illustrated respectively in corresponding ith feature point, T in previous frame image and middle two field picture
It is transposition operator；
Whether step 103, judgment formula (2) are set up, if formula (2) is set up, this feature point meets epipolar-line constraint；If public
Formula (2) is invalid, then illustrate that this feature point is unsatisfactory for epipolar-line constraint, and the set that will be unsatisfactory for the match point of epipolar-line constraint is denoted as
R1, i.e., the moving target obtained after one time classification and Detection,
di＜ α med { d1,d2,...} (4)
In formula (2), α is given coefficient threshold, and med is to take median function.
In the present invention, the use three-dimensional structure coordinates restriction is detected to the match point secondary classification for meeting epipolar-line constraint
Process is：
Step 201, the match point for meeting epipolar-line constraint obtained for step 103, using Plane＆Parallax methods
Calculate the three-dimensional structure coordinate of match point.
Plane＆Parallax Method And Principles are as shown in Figure 2.In three dimensions, a spatial point X is burnt in two cameras of left and right
Imaging is respectively a homograph matrix H of left view x and x ', the π plane induction of left view to right view, point in plane
x′πIt is x by the point in π Planar Mappings to right view, i.e. x 'π=Hx, right limit e ', x ' are understood according to geometrical relationshipπWith x ' altogether
Line, so that x ' can be represented with formula (3),
X '=Hx+ke ' (5)
In formula (3), k characterizes the departure degree of X and π planes, and even k then illustrates that X gets over apart from π planes closer to 0
Closely, therefore, k represents " depth " of X in three dimensions.
Make s=(xT,k)T, x is picture points of the spatial point X in left camera focal plane, and s is the structure coordinate of point X.TTo turn
Put.
The present invention is obtained spatial point X and is existed according to Plane＆Parallax methods by previous frame image and middle two field picture
Structure coordinate s between previous frame image and middle two field picture, similarly, obtains X's by middle two field picture and latter two field picture
Structure coordinate s ' between middle two field picture and latter two field picture, now, if X is background dot, s and s ' meets such as formula
(4) restriction relation shown in,
s′TGs=0 (6)
In formula (4), G is 4 × 4 matrix, and its order is 2.
Step 202, the residual epsilon for calculating characteristic pointi, shown in calculation such as formula (5),
Whether step 203, judgment formula (6) are set up, if formula (6) is set up, illustrate that characteristic point meets structure coordinate about
Beam；If formula (6) is invalid, illustrate that characteristic point is unsatisfactory for structure coordinate constraint, the feature of structure coordinate constraint will be unsatisfactory for
Point set is denoted as R2, i.e., the moving target for being obtained after secondary classification detection,
εi＜ β med { ε1,ε2,...} (8)
In formula (6), β is given coefficient threshold, and med is to take median function.
Step 4, by double classification result merge, will subseries detection after obtain moving target R1And secondary classification
The moving target R obtained after detection2Merge, so as to obtain final moving object detection result R, such as shown in formula (7),
R=R1∪R2 (9)
Effect of the invention can be described further by following emulation experiment：
Fig. 3 is this experiment continuous three two field picture used, wherein, (a) is previous frame image, and (b) is middle two field picture,
C () is latter two field picture, in three two field pictures, tank model is moving target, and books, bottle and paper handkerchief are background, due to field
Scape is nearer apart from camera, and video image has larger parallax.
Fig. 4 is using document Moving Object Localization in Thermal Imagery by
The image compensation method proposed in Forward-backward MHI, is processed continuous three two field picture in Fig. 2, is detected
Impact point, in figure it is every a pair of circle and cross be the moving target point for detecting.It can be seen that because bottle is away from camera
Closer to parallax reason causes background not compensated well, so as to be treated as moving target.
Fig. 5 is processed continuous three two field picture in Fig. 2 to employ the inventive method, the impact point for detecting, figure
In it is every a pair of circle and cross be the moving target point for detecting.It can be seen that impact point is on the tank of motion, it is quiet
Only the impact point on bottle is filtered out, and false alarm rate is substantially reduced.
Claims (2)
1. a kind of moving target detecting method based on three-dimensional structure coordinates restriction, it is characterised in that
First, to the match point in adjacent three two field picture, the matching for meeting epipolar-line constraint is distinguished by the method for epipolar-line constraint
Put and be unsatisfactory for the match point of epipolar-line constraint；
Then, for the match point for meeting epipolar-line constraint, the three-dimensional knot of match point is calculated using Plane＆Parallax methods
Structure coordinate, is distinguished using three-dimensional structure coordinates restriction method and meets the match point of structure coordinate constraint and be unsatisfactory for structure coordinate
The match point of constraint；
Finally, the match point that will be unsatisfactory for epipolar-line constraint and the match point for being unsatisfactory for structure coordinate constraint merge acquisition motion mesh
Mark；
The match point that the differentiation meets epipolar-line constraint and the method for the match point for being unsatisfactory for epipolar-line constraint are：
Step 101, the matching double points asked in adjacent three two field picture；
Basis matrix F between step 102, estimation previous frame image and middle two field picture12, try to achieve the basic square of each match point
Battle array residual error di, the basis matrix residual error d of each match pointiCalculating such as formula (1) shown in,
In formula (1), xiAnd xi' corresponding ith feature point in previous frame image and middle two field picture is illustrated respectively in, T is to turn
Put operator；
Whether step 103, judgment formula (2) are set up, if formula (2) is set up, this feature point meets epipolar-line constraint；If formula
(2) invalid, then this feature point is unsatisfactory for epipolar-line constraint,
di＜ α med { d1,d2,...} (2)
In formula (2), α is given coefficient threshold, and med is to take median function；
The method for asking for the matching double points in adjacent three two field picture is：The Harris characteristic points of two field picture in the middle of first detecting, then
The two groups of matchings respectively in front and rear two field pictures of the characteristic point of the method middle two field picture of tracking acquisition tracked using KLT points
Point, takes match point of two groups of common factors of match point as adjacent three two field picture；
The three-dimensional structure coordinate s=(x of the match pointT,k)T, wherein, x is spatial point former frame figure in adjacent two field pictures
Picture point as in；
Differentiation meets the match point of structure coordinate constraint and the method for the match point for being unsatisfactory for structure coordinate constraint is：
Step 201, the residual epsilon for calculating characteristic pointi, calculation as shown by the equation,
In formula, s be by previous frame image and middle two field picture obtain spatial point previous frame image and middle two field picture it
Between structure coordinate；S' be by middle two field picture and latter two field picture obtain between middle two field picture and latter two field picture
Structure coordinate；G is 4 × 4 matrix, and its order is 2；T is transposition operator；
Whether step 202, judgment formula are set up, if formula is set up, this feature point meets structure coordinate constraint；If formula not into
Vertical, then this feature point is unsatisfactory for structure coordinate constraint,
εi＜ β med { ε1,ε2,...} (4)
In formula, β is given coefficient threshold, and med is to take median function.
2. the moving target detecting method of three-dimensional structure coordinates restriction is based on as claimed in claim 1, it is characterised in that will be discontented
Shown in the match point of sufficient epipolar-line constraint and the mode such as formula (5) of the match point merging for being unsatisfactory for structure coordinate constraint,
R=R1∪R2 (5)
Formula (5), R is the final moving target for detecting, R1To be unsatisfactory for the set of the match point of epipolar-line constraint, R2It is discontented
The set of the match point of sufficient structure coordinate constraint.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410481725.7A CN104240267B (en) | 2014-09-19 | 2014-09-19 | A kind of moving target detecting method based on three-dimensional structure coordinates restriction |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410481725.7A CN104240267B (en) | 2014-09-19 | 2014-09-19 | A kind of moving target detecting method based on three-dimensional structure coordinates restriction |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN104240267A CN104240267A (en) | 2014-12-24 |
| CN104240267B true CN104240267B (en) | 2017-06-27 |
Family
ID=52228273
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410481725.7A Expired - Fee Related CN104240267B (en) | 2014-09-19 | 2014-09-19 | A kind of moving target detecting method based on three-dimensional structure coordinates restriction |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104240267B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104680516B (en) * | 2015-01-08 | 2017-09-29 | 南京邮电大学 | A kind of acquisition methods of image quality features set of matches |
| CN109902725A (en) * | 2019-01-31 | 2019-06-18 | 北京达佳互联信息技术有限公司 | Mobile mesh object detection method, device and electronic equipment and storage medium |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN203397395U (en) * | 2013-07-19 | 2014-01-15 | 南京理工大学 | Moving object detection device in platform based on DSP + FPGA |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8073196B2 (en) * | 2006-10-16 | 2011-12-06 | University Of Southern California | Detection and tracking of moving objects from a moving platform in presence of strong parallax |
-
2014
- 2014-09-19 CN CN201410481725.7A patent/CN104240267B/en not_active Expired - Fee Related
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN203397395U (en) * | 2013-07-19 | 2014-01-15 | 南京理工大学 | Moving object detection device in platform based on DSP + FPGA |
Non-Patent Citations (3)
| Title |
|---|
| Relative Affine Structure:Canonical Model for 3D From 2D Geometry and Applications;A. Shashua et al.;《IEEE Transactions on Pattern Analysis and Machine Intelligence》;19960930;第18卷(第9期);第873-883页 * |
| 一种快速的单目移动相机下运动目标检测方法;陈海欣 等;《光子学报》;20140331;第43卷(第3期);第3页第3.1节，第4页图2 * |
| 动态场景下运动目标检测与跟踪;易科;《中国优秀硕士论文全文数据库 信息科技辑》;20120115;第28页第3.2.2节-第32页第3.3.3节，图3-10 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN104240267A (en) | 2014-12-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9607227B2 (en) | Boundary detection apparatus and boundary detection method | |
| EP2757524B1 (en) | Depth sensing method and system for autonomous vehicles | |
| CN108028023A (en) | Information processor, information processing method and program | |
| JP6171593B2 (en) | Object tracking method and system from parallax map | |
| CN106033614B (en) | A kind of mobile camera motion object detection method under strong parallax | |
| WO2011067788A3 (en) | A cost effective and robust system and method for eye tracking and driver drowsiness identification | |
| CN103458261A (en) | Video scene variation detection method based on stereoscopic vision | |
| CN108021849B (en) | Pedestrian early warning method and device | |
| CN112115889B (en) | Intelligent vehicle moving target detection method based on vision | |
| CN104240267B (en) | A kind of moving target detecting method based on three-dimensional structure coordinates restriction | |
| Dubey | Stereo vision-Facing the challenges and seeing the opportunities for ADAS applications | |
| Geiger et al. | Object flow: A descriptor for classifying traffic motion | |
| JP2011232965A (en) | Plane area detection device and stereo camera system | |
| TWI499999B (en) | The 3D ring car image system based on probability calculation and its obtaining method | |
| Zhou et al. | On modeling ego-motion uncertainty for moving object detection from a mobile platform | |
| CN102567992B (en) | Image matching method of occluded area | |
| CN108090930A (en) | Barrier vision detection system and method based on binocular solid camera | |
| WO2012116350A2 (en) | Figure-ground organization of 3-d scenes | |
| Chowdhury et al. | Fast window based stereo matching for 3D scene reconstruction. | |
| Suganuma et al. | Fast dynamic object extraction using stereovision based on occupancy grid maps and optical flow | |
| CN102142148A (en) | Video space-time feature extraction method | |
| CN116151320A (en) | Visual odometer method and device for resisting dynamic target interference | |
| Sato et al. | Efficient hundreds-baseline stereo by counting interest points for moving omni-directional multi-camera system | |
| CN106303501A (en) | Stereo-picture reconstructing method based on image sparse characteristic matching and device | |
| TWM486116U (en) | 3D vehicle surrounding image system based on probability calculation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20170627 Termination date: 20190919 |