CN108010055A - The tracking system and its tracking of three-dimensional body - Google Patents
The tracking system and its tracking of three-dimensional body Download PDFInfo
- Publication number
- CN108010055A CN108010055A CN201711183555.4A CN201711183555A CN108010055A CN 108010055 A CN108010055 A CN 108010055A CN 201711183555 A CN201711183555 A CN 201711183555A CN 108010055 A CN108010055 A CN 108010055A
- Authority
- CN
- China
- Prior art keywords
- characteristic point
- frame
- video frame
- data
- module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
- G06T7/248—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments involving reference images or patches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
Abstract
The invention discloses the tracking system and its tracking of a three-dimensional body,The tracking system of wherein described three-dimensional body includes a key frame and forms unit,The outer parameter analysis unit of one video frame and a tracking judging unit,Wherein described key frame forms unit by analyzing the data of a template frame and the data of a video frame to form a key frame data,Wherein described key frame data includes the outer parameter of the key frame,The outer parameter analysis unit of wherein described video frame is communicatively coupled to the key frame and forms unit,The outer parameter analysis unit of wherein described video frame can obtain the outer parameter of the key frame,And the outer parameter of the video frame is calculated according to the outer parameter of the key frame,Wherein described tracking judging unit is communicatively coupled to parameter analysis unit outside the video frame,After wherein described video frame analysis module obtains the data of the key frame and the data of the video frame,Calculate tracked object corresponding pose in the video frame.
Description
Technical field
The present invention relates to the tracking system and its method of a three-dimensional body, more particularly to one without by reality scene
Tracking system and its tracking of the middle mode for placing mark to the three-dimensional body into line trace of three-dimensional body.
Background technology
Tradition, it is necessary to place predetermined mark in real scene in advance, passes through detection to the tracking of three-dimensional body
Mark come realize track and determine camera parameters, further renders three-dimensional data model, with mark occur scaling, translation
Or the change such as rotation, by recalculating the outer parameter of the video camera, and then realize the tracking to three-dimensional body.And
In actual life, it is cumbersome to place mark, on the one hand, needs the object for avoiding the signs blocking placed from being traced, separately
On the one hand, some tracked objects are mobile, therefore video frame at this time obtains equipment and also correspondingly moving, and by
The mark of placement is really static, if the video frame obtains equipment, movement is very rapid, it is likely that causes to be placed on existing
The mark in real field scape removes the video frame and obtains outside the field range that equipment can obtain, and so may result in whole
The failure of a tracking process.To solve problems, traditional tracking to three-dimensional body is by using flat image as mark
Into line trace, and using two-dimensional image as indicating into line trace, usually so that the direction of motion of tracked object by
To limitation, meanwhile, the freedom of motion for the threedimensional model being further superimposed also is restricted.
During modern development in science and technology, mark-free view-based access control model feature has wider application range, but at present
CAD model requirement of the relevant technology to being traced object is higher, and the process tracked is sufficiently complex, while to corresponding
Operational outfit also have higher requirement, with a large amount of popularizations of mobile terminal AR applications, real-time becomes algorithm research
Important Problems, wherein crucial is some the stability of tracking, traditional tracking to three-dimensional body answers texture
The speed of miscellaneous object tracking is slower, to simple Object Extraction less than effective feature and sufficient amount of feature, moreover,
When traditional tracking is to tracked Object Extraction feature, very sensitive to intensity of illumination, this causes to a certain extent
Whole tracking process continuity is poor, often occurs that superposition virtual three-dimensional object is acutely joyous dynamic in the scene, does not link up
Visual perception.
The content of the invention
It is an object of the present invention to provide the tracking system and its tracking of a three-dimensional body, wherein according to institute
State the tracking system of three-dimensional body to object into during line trace, without placed in reality scene specific mark in kind and
Plane landmark.
It is another object of the present invention to provide the tracking system and its tracking of a three-dimensional body, wherein described
The tracking system of three-dimensional body can be different to texture-rich degree object all, can stably be tracked.
It is another object of the present invention to provide the tracking system and its tracking of a three-dimensional body, wherein described
The tracking system of the three-dimensional body object complex to texture can be tracked rapidly.
It is another object of the present invention to provide the tracking system and its tracking of a three-dimensional body, wherein described
The tracking system of the three-dimensional body object complex to texture can be tracked stably.
It is another object of the present invention to provide the tracking system and its tracking of a three-dimensional body, wherein described
The tracking system of three-dimensional body can extract simple object enough characteristic points, so carry out effectively, stably with
Track.
It is another object of the present invention to provide the tracking system and its tracking of a three-dimensional body, wherein described
The tracking system of the three-dimensional body into during line trace, will not influenced three-dimensional body be subject to light intensity.
It is another object of the present invention to provide the tracking system and its tracking of a three-dimensional body, wherein described
The tracking system of three-dimensional body can by way of updating template key frame in real time to object into line trace so that even in
When video frame is come back to after tracked object out-of-bounds obtaining the visual field of equipment, also it can continue to object into line trace.
That is after tracked object causes tracking to fail because of out-of-bounds, if tracked object comes back to video again
After frame obtains the visual field of equipment, the tracking system of the three-dimensional body can be automatically again to being traced object into line trace.
To realize more than the present invention at least one purpose, the present invention provides the tracking system of a three-dimensional body, including：
One key frame forms unit, is regarded wherein the key frame forms unit by analyzing the data and one of a template frame
The data of frequency frame are to form a key frame data, wherein the key frame data includes the outer parameter of the key frame；
The outer parameter analysis unit of one video frame, wherein described in parameter analysis unit is communicatively coupled to outside the video frame
Key frame forms unit, wherein the outer parameter analysis unit of the video frame can obtain the outer parameter of the key frame, and root
The outer parameter of the video frame is calculated according to the outer parameter of the key frame；And
One tracking judging unit, wherein the tracking judging unit is communicatively coupled to Parameter analysis outside the video frame
Unit, wherein after the video frame analysis module obtains the data of the key frame and the data of the video frame, calculates place
State the locus in video frame.
An embodiment according to the present invention, wherein the tracking system of the three-dimensional body further comprises a data acquisition list
Member, wherein the data capture unit further comprises a template frame acquisition module and a video frame acquisition module, wherein described
Template frame acquisition module and the video frame acquisition module are communicatively coupled to the key frame and form unit respectively.
An embodiment according to the present invention, wherein the data capture unit further comprises a characteristic point processing module, its
Described in characteristic point processing module include a characteristic point judgment module and a feature point extraction module, wherein the characteristic point is sentenced
Disconnected module is communicatively coupled to the template frame acquisition module and the video frame acquisition module, wherein the feature point extraction
Module is communicatively coupled to the characteristic point judgment module, the key frame forms parameter point outside unit and the video frame
Analyse unit.
An embodiment according to the present invention, wherein the characteristic point processing module further comprises that a texture-rich degree judges
Module, wherein the texture-rich degree judgment module includes a characteristic point analysis module, wherein the characteristic point analysis module quilt
It is communicatively coupled to the characteristic point judgment module.
An embodiment according to the present invention, wherein the texture-rich degree judgment module further comprises that a characteristic point changes
Judgment module and a Homogenization Treatments module, wherein characteristic point change judgment module is communicatively coupled to the characteristic point
Extraction module, wherein the Homogenization Treatments module is communicatively coupled to the characteristic point judgment module.
An embodiment according to the present invention, wherein the characteristic point processing module further comprises that a Feature Points Matching judges
Module, wherein the Feature Points Matching judgment module is communicatively coupled to the feature point extraction module.
To realize more than the present invention at least one purpose, the present invention provides the tracking of a three-dimensional body, including following
Step：
(A) a matched pass of flat image with tracking object is obtained from video frame according to the data of tracked object
The data of key frame, wherein the data of the key frame include the outer parameter of the key frame；
(B) data of the data for the key frame that analysis obtains and the video frame obtained, and then calculate current institute
State the corresponding outer parameter of video frame；And
(C) calculated in the video frame and be traced in object and the template frame according to the outer parameter of the video frame
Tracked object between pose.
An embodiment according to the present invention, the tracking of the three-dimensional body further comprise：
(D) whether the difference between the outer parameter of the video frame and the outer parameter of the key frame meets specific
Threshold value；And
(E) when the difference is unsatisfactory for the threshold value, the template frame is changed to the last template used
Frame.
An embodiment according to the present invention, wherein the step (A) includes：
(F) according to a characteristic point judgment threshold, judge point in current image date whether characteristic point；
(G) according to judgement as a result, extraction characteristic point；
(H) relatively described in (G) described characteristic point of extraction with the characteristic point in a reference data to judge the feature
Point meets that the reference data requires characteristic point accordingly；And
(I) if satisfied, the characteristic point of extraction is as the characteristic point finally extracted in described in (G), if not satisfied, passing through
The characteristic point judgment threshold in (F) described in change, continues to execute (F).
An embodiment according to the present invention, wherein the step (A) includes：
(J) whether the quantity for the characteristic point for judging to be extracted meets a characteristic point amount threshold scope；With
(K) when the discrepancy of quantity for the characteristic point being extracted unifies characteristic point amount threshold scope, described in Homogenization Treatments
Video frame；And
(L) characteristic point of the video frame after extraction is homogenized.
Brief description of the drawings
Fig. 1 is the schematic diagram of the tracking system of a three-dimensional body of the invention.
Fig. 2 is the tracking schematic diagram of the tracking system of a three-dimensional body of the invention.
Fig. 3 is the feature point extraction schematic diagram of the tracking system of a three-dimensional body of the invention.
Fig. 4 is the flow chart of the tracking of a three-dimensional body of the invention.
Embodiment
It is described below to be disclosed for so that those skilled in the art manufacture and using the present invention.Middle offer described below
Only as the example and modification that will be apparent to those skilled in the art, it is not formed to the scope of the invention for preferred embodiment
Limitation.General Principle defined in described below can be applied to other realities without departing substantially from spirit of the invention and invention scope
Apply example, optional replacement, modification, equivalent implementation and application.
It will be understood by those skilled in the art that the present invention exposure in, term " longitudinal direction ", " transverse direction ", " on ",
" under ", "front", "rear", "left", "right", " vertical ", " level ", " top ", " bottom " " interior ", the orientation or position of the instruction such as " outer "
Relation is to be based on orientation shown in the drawings or position relationship, it is for only for ease of the description present invention and simplifies description, rather than
Indicate or imply signified device or element there must be specific orientation, with specific azimuth configuration and operation, thus it is above-mentioned
Term is not considered as limiting the invention.
The invention discloses a three-dimensional body tracking system and its tracking will be elaborated in herein below,
The tracking system of wherein described three-dimensional body includes a data capture unit 10, a key frame forms unit 20, outside a video frame
Parameter analysis unit 30 and one tracks judging unit 40, wherein the data capture unit 10 can obtain tracked object
Tracking data, wherein the key frame, which forms unit 20, is communicatively coupled to the data capture unit 10 so as to from institute
State data capture unit 10 and obtain the tracking data, and key frame formation unit 20 can correspondingly form one and have
The primary data of the key frame is closed, wherein the outer parameter comprising the key frame, intrinsic parameter in the primary data, wherein
The outer parameter analysis unit 30 of the video frame is communicatively coupled to the key frame and forms unit 20 so as to according to described initial
Key frame forms the primary data that unit 20 is formed and obtains the outer parameter Mi of presently described video frame, wherein the tracking
Judging unit 40 is communicatively coupled to parameter analysis unit 30 and the key frame outside the video frame and forms unit 20, wherein
The tracking judging unit 40 can be by the outer parameter Mi of the obtained video frame and the outer parameter M of the key frame0
Contrasted, wherein as the Mi and the M0Between difference when exceeding certain scope, tracking 40 phase of judging unit
Answer ground to form one to update the data, wherein described update the data is transferred to the key frame formation unit 20, by the key
The primary data of frame replaces with the related data of template frame described in a width, so as to track again.
It will be appreciated to those of skill in the art that by key frame described in real-time update, and then even if the thing being traced
When body removes the visual field of the video frame acquisition equipment and comes back in the visual field, the three-dimensional article based on CAD models
The tracking system of body also can be to object into line trace.
In the present invention, the data capture unit 10 includes a template frame acquisition module 11 and a video frame obtains mould
Block 12, wherein the template frame acquisition module 11 and the video frame acquisition module 12 are communicatively coupled to the key frame shape
Into unit 20, wherein the key frame, which forms unit 20, passes through the template frame that forms the template frame acquisition module 11
The data for the video frame that data are formed with the video frame acquisition module 12 carry out analysis contrast, so as to judge the video frame
Whether template key frame can be used as.
Specifically, in an embodiment of the present invention, the template frame acquisition module 11 can obtain tracked object object
CAD model data, and parse the CAD model data, the data after parsing stored, wherein the CAD model data
Format Type include but not limited to OBJ either 2DS, wherein the CAD after being parsed by the template frame acquisition module 11
Model data includes Triangular object model data, vertex data, normal vector data, data texturing, material quality data, Lighting information number
According to etc..
It will be appreciated to those of skill in the art that the template frame acquisition module 11 can be at the same time to the CAD models
Data carry out vertex redundancy processing, reduce the redundancy of the CAD model data, and then improve CAD moulds described in subsequent treatment
The time of type data.
Further, the template frame acquisition module 11 can carry out wash with watercolours using OpenGL to the CAD model data
Dye is matched with forming the template frame for the data characteristics that follow-up tracking uses, skilled artisans appreciate that
, in the present invention, the mode rendered to the tracked object can be other methods, and the present invention is from this
The limitation of aspect.
The video frame acquisition module 12 can be communicatively coupled to the video frame and obtain the shooting of an equipment such as monocular
Head, wherein the video frame acquisition module 12 can obtain an at least width video frame, divides wherein the key frame forms unit 20
The data of the template frame and described can not be obtained from the template frame acquisition module 11 and the video frame acquisition module 12
The data of video frame, so that judge whether the current video requency frame data can be used as a key frame, if it can,
Then can be using the video frame as the key frame, and data using the data of the template frame as the key frame.
In the present invention, the data of the initial template frame are implemented as rendering tracked object by OpenGL
Image T, and the data of the template frame include giving at this time by the outer parameter M of the OpenGL images rendered0=[R0
t0], wherein the key frame formed unit 20 can contrast presently described video frame whether the data phase with the template frame
Match somebody with somebody, if matched, the video frame is made by as the key frame, and it is corresponding to demarcate the key frame at this time
The outer parameter that the video frame images obtain equipment is that the outer parameter of the template frame is M0=[R0 t0]。
Specifically, the CAD model for tracked thing being rendered using OpenGL can obtain CAD model under current pose
Two-dimensional projection image T, and the given video frame obtains the intrinsic parameter K and outer parameter M of equipment such as monocular-camera, leads at the same time
Cross the video frame acquisition equipment and obtain the video frame in real time, wherein key frame formation unit 20 can be to template frame
Quick initial matching is carried out with the video frame, successful match, which is believed that in present frame, includes tracked object, and present frame
The outer parameter is equal with the outer parameter for rendering image, i.e. M ≈ M', and presently described video frame is by as the pass
Key frame.
Quick initial matching detailed process is in embodiments of the present invention：
The intrinsic parameter K and outer parameter M of the CAD model image rendered by OpenGL given first, secondly using adaptive
Threshold value, image and the contour feature and binaryzation of the video frame are rendered described in extraction；The key frame forms unit 20 to place
Image after reason is matched using the similarity matrix that conventional minimum square difference method obtains, and D (i, j) is similitude
Matrix, l (i, j) are the top left co-ordinate of similitude highest zone：
Optimum Matching position, that is, element smallest point ε in D (i, j)1=min { dij, experiment proves that match point falls in ε1~2 ε1
The probability of scope is up to 98%, therefore uses ε2=2 ε1As the match point similitude upper bound, then revised similarity matrix
For d'ijFor matrix element value after amendment:
Revised similarity matrix accelerates matching speed, when the template frame and the figure of the video frame to be matched
As it is close when, be worth rapid convergence；And when difference is very big, this value is amplified rapidly, therefore is carried out slightly using characteristic distributions
Scanning can then determine rapidly approximate region, then shorten step-size in search progress close scanning again, matched time-consuming to be decreased obviously.
It will be appreciated to those of skill in the art that in embodiments of the present invention, by approach described above, so as to improve
State the matching speed between template frame and the video frame, thus to object into during line trace, can keeping tracking faster
Speed.
After the template frame is combined quick initial matching success with video frame progress thickness, the video frame is preserved
For the key frame, and the outer parameter M of the external parameters of cameras as the key frame that CAD model will be rendered0=[R0
t0]。
Further, the data capture unit 10 further comprises a characteristic point processing module 13, wherein the feature
Point processing module 13 is communicatively coupled to the template frame acquisition module 11 and the video frame acquisition module 12, wherein described
Characteristic point processing module 13 can extract the characteristic point of the template frame and the characteristic point of the video frame respectively to form phase
The characteristic point data answered.
Specifically, the characteristic point processing module 13 utilizes ORB feature extractions, to the template frame and the video
Feature descriptor of the frame extraction with Scale invariant and rotational invariance, and the two is matched using distance metric, obtain
Rough matching point set.Then error hiding is rejected to obtained matching point set, reduces wrong correspondence to the shadow that subsequently calculates
Ring, concretely comprise the following steps：
ORB methods feature point extraction is carried out with the video frame to the template frame and builds the corresponding feature point set
To obtain the descriptor of characteristic point, the characteristic point of the template frame and the feature of the video frame are clicked through using Hamming distance
Row matching, comprises the following steps that：
For the characteristic point p on the template frame, if its simple description subset is FP, on another video frame I
Feature point description is FS={ F1, F2 …Fn}；
Calculate FPWith FIIn all characteristic points Hamming distance：D={ d1,d2,…dn, wherein：
dn=Fp^Fn
The minimum value D of Hamming distancemin=min { d1,d2,…dnCorresponding to point be then point p nearest neighbor points, t is matching
Threshold value, if Dmin<T, then judge that p is matched with the point, otherwise p does not have match point；
Above-mentioned identical operation is performed to remaining point in the template frame P, obtains all Feature Points Matching point sets.
In an embodiment of the present invention, the characteristic point processing module 13 further comprises a characteristic point judgment module
131st, a feature point extraction module 132 and a texture-rich degree judgment module 133, wherein the feature point extraction module 132
The characteristic point judgment module 131 and the texture-rich degree judgment module 133 are communicatively coupled to, wherein the feature
Point judgment module 131 is communicatively coupled to the template frame acquisition module 11 of the data capture unit 10 and the video
Frame acquisition module 12 from the template frame acquisition module 11 and the video frame acquisition module 12 so as to obtain corresponding described
The data of the data of template frame and the video frame, wherein the characteristic point judgment module 131 can interpolate that out the template
Data in the data of frame in relation to vertex and whether the data in relation to vertex meet the characteristic point in the data of the video frame
Requirement, wherein the feature point extraction module 132 can will meet that the point of characteristic point requirement is extracted to form one respectively
Template frame feature point set and a video frame feature point set, wherein the texture-rich degree judgment module 133 is communicatively coupled to
The feature point extraction module 132 from the texture-rich degree judgment module 133 so as to obtain the template frame feature point set
With the video frame feature point set and the characteristic point judgment module 131, and can analyze in the video frame feature point set
The quantity of the characteristic point whether meet the requirements, if undesirable, 133 phase of texture-rich degree judgment module
Ground is answered to form a replacement data, wherein the characteristic point judgment module 131 can correspondingly obtain the replacement data, and energy
Enough requirements that the characteristic point is changed according to the replacement data.
Specifically, in embodiments of the present invention, a characteristic point judgment threshold is preset in the characteristic point judgment module 131
δ, it is preferable that the characteristic point judgment threshold δ be implemented as using certain pixel radius be upper 16 pixel point values of the circle of R with
The absolute value of heart point pixel difference, if having more than det (P on circlenum) difference between a pixel and central point exceedes characteristic point
Judgment threshold δ, then the central point be as characteristic point, its decision condition：
The feature point extraction module 132 can correspondingly extract that to meet the characteristic points of above-mentioned requirements each described to be formed
Video features point set.
In the present invention, the texture-rich degree judgment module 133 includes a characteristic point analysis module 1331, wherein institute
State characteristic point analysis module 1331 and be communicatively coupled to the feature point extraction module 132 and the characteristic point judgment module
131, wherein characteristic point analysis module 1331 can obtain the video features point set from the feature point extraction module 132, and
The quantity and a default a reference value (δ for the characteristic point that the video features point can be concentrated1-δ2It is interior) contrasted, wherein
When the video features point that the characteristic point analysis module 1331 analysis obtains concentrates the quantity of characteristic point not in the benchmark
When being worth in corresponding scope, the characteristic point analysis module 1331 correspondingly forms the replacement data, wherein the characteristic point
After judgment module 131 obtains the replacement data, the replacement data can be correspondingly performed to adjust the feature point extraction
Condition.
By above description, it will be appreciated to those of skill in the art that in embodiments of the present invention, by the line
The characteristic point analysis module 1331 of richness judgment module 133 is managed, so as to make the tracking system of the three-dimensional body
Enough special video features point sets can be also extracted for the different object of texture-rich degree, and then make the three-dimensional article
The tracking system of body can be suitable for the different object of texture-rich degree.
Further, the texture-rich degree judgment module 133 further comprises that a characteristic point changes judgment module
1332, wherein characteristic point change judgment module 1332 is communicatively coupled to the feature point extraction module 132 to monitor quilt
Extraction the video features point the adjacent characteristic point extracted twice number differences, wherein the characteristic point change judge
Module 1332 is communicatively coupled to the characteristic point analysis module 1331, if the number differences are less than a given change
Threshold value, then it represents that be currently at normal condition, the characteristic point analysis module 1331 continues to extract the spy of the video frame
Point is levied, if the number differences are more than the change threshold, then it represents that the characteristic point is there occurs violent change, accordingly
Ground represents the object out-of-bounds being currently traced or is blocked, and needs to carry out Homogenization Treatments to described image at this time.
More specifically, the characteristic point processing module 13 further comprises a Homogenization Treatments module 134, wherein described
Homogenization Treatments module 134 is communicatively coupled to the characteristic point change judgment module 1332 and the characteristic point judgment module
131, wherein when the characteristic point analysis module 1331 analysis obtains the number differences and is more than the change threshold, it is described
Homogenization Treatments module 134 will form a Homogenization Treatments data, wherein the characteristic point judgment module 131 correspondingly can
The Homogenization Treatments data are performed, and decile can be carried out to the video frame according to the Homogenization Treatments data, and then
The feature point extraction module 132 is set to carry out the extraction of characteristic point to the video frame after decile.
Skilled artisans appreciate that, change judgment module 1332 and the homogenization by the characteristic point
Processing module 134, so that the either out-of-bounds even if the object in the image of analysis is blocked, described image feature point extraction system
System can equally extract the characteristic point of suitable number.
The characteristic point processing module 13 further comprises the wherein described characteristic point of a Feature Points Matching judgment module 135
Matching judgment module 135 is communicatively coupled to parameter analysis unit outside the feature point extraction module 132 and the video frame
30, wherein after the feature point extraction module 132 has extracted the video frame characteristic point, wherein the Feature Points Matching is sentenced
Disconnected module 135 can calculate each characteristic point of each characteristic point and video frame described in every width of the template frame
Between Hamming distance D={ d1,d2,...,dn, wherein Hamming distance minimum value Dmin=min { d1,d2,...,dnCorresponding
Point be p nearest neighbor points, if Dmin<T then assert 2 points of matchings, and otherwise p is without match point.
The Feature Points Matching judgment module 135 is according to described matched as a result, and then to the spy of the video frame
Sign point rearranges, so that the feature point set after matching is formed, specifically, on the basis of the template two field picture, with institute
It is target to state video template two field picture, ratio testing is carried out, first, using the characteristic point of the template frame to the figure
As characteristic point carries out 2 neighbouring inquiries, thus to the screen frame each characteristic point to the template frame each institute
The distance between characteristic point is stated, correspondingly, can also arrive to obtain each characteristic point arest neighbors bn of the screen frame and time near
Adjacent bn ', the distance to arest neighbors and time neighbour is respectively dn and dn ' secondly, threshold testing is carried out, if matching threshold is t,
If dn>T, rejects the matching pair；Ratio testing is finally carried out, if rate threshold is ε, ifAt this time it is considered that bn and
Bn ' be likely to be query set match point, therefore reject the matching pair.Cross-beta then is carried out to obtained point set again, is looked into
The set of characteristic points for asking collection and object set matching pair is { s respectivelynAnd { pn, wherein n=1,2,3 ..., invert query set and mesh
Mark collection, asks for { pnMatch point { s'n, correctly matching is { s to corresponding query setn}∩{s'n}。
It will be appreciated to those of skill in the art that also there is substantial amounts of mistake due to matching obtained matching point set for the first time
Error hiding, very big error may be caused to follow-up result and computing, therefore by the feature after matching double points are obtained
Point matching judgment module 135 can be realized to error hiding to rejecting.
Further, the outer parameter analysis unit 30 of the video frame is communicatively coupled to the key frame and forms unit 20
With the Feature Points Matching judgment module 135, wherein the outer parameter analysis unit 30 of the video frame can obtain the template frame
Data and the video frame related data so that the video demonstrate,prove outer Parameter analysis module 30 can be according to known
The initial outer ginseng M0=[R0 t0], the internal reference K and current match point correspondence obtain the described outer of the video frame
Parameter Mi=[Ri ti], and concretely comprised the following steps using Kalman prediction with ensuring the stability of parameter：
Video frame obtains the initial outer ginseng M of equipment0=[R0 t0] with the internal reference K be a series of world coordinate systems
Under non-coplanar 3D points pi=(xi,yi,zi)tWith under two dimensional surface under corresponding point gi=(ui',vi',1)tBetween relation：
gi=KMpi
During the tracking of follow-up feature based, the template frame I has been obtained0Characteristic point g0With the video frame IiFeature
Point giCorresponding matching point set, i.e. 2D-2D correspondences:
gi=H0ig0
According to the initial outer current 2D-2D correspondences of participation, new 2D-3D correspondences are obtained, and calculate current
The described image for obtaining the video frame obtains the outer ginseng M of equipmentiFor: gi=H0iKMip0, then using Kalman filtering to estimating
Meter result optimizes.
The tracking judging unit 40 can obtain the video that the outer parameter analysis unit 30 of the video frame calculates
The outer parameter of frame, and space bit of the tracking of the tracked object in the video frame is calculated according to the outer parameter
Put, to realize the tracking to the tracked object.
It is noted that the outer parameter analysis unit 30 of the video frame is communicatively coupled to the characteristic point processing mould
Block 13 and the Feature Points Matching judgment module 135, so as to the characteristic point data is obtained, and according to the spy of acquisition
Sign point data calculates the outer parameter M of the video framei, so that corresponding analysis data are formed, wherein the tracking judges list
Member 40 is communicatively coupled to parameter analysis unit 30 outside the video frame so as to judging to work as forward sight according to the analysis data
Whether tracked object is had in frequency frame, to judge whether current tracking succeeds, if tracking failure, the tracking judge single
Member 40 correspondingly forms one and updates the data, wherein the tracking judging unit 40 is communicatively coupled to the initial module and is formed
Unit 20, so as to update the related data in the template frame.
Specifically, the tracking judging unit 40 can obtain the institute that the outer parameter analysis unit 30 of the video frame is formed
Analysis result is stated, wherein a threshold value is set in the tracking judging unit 40, wherein described tracking judging unit 40 can be sentenced
It is big between difference and the threshold value between the outer parameter for presently described video frame of breaking and the outer parameter of the template frame
Small relation, wherein when the difference is more than the threshold value, illustrates in current video frame not by the object of a tracking, then institute
State tracking judging unit 40 formation one to update the data, wherein the initial key frame, which forms unit 20, can obtain the renewal
Data, and the template frame data can be updated the data according to described；Wherein when the difference is less than institute's threshold value, say
There is the tracked tracking object in bright current video frame, to ensure under different visual angles to the tenacious tracking of three-dimensional body,
The current tracking mode of real-time judgment simultaneously carries out template key image update.By above-mentioned steps, complete to the steady of three-dimensional body
It is fixed to track and export external parameters of cameras in real time, concretely comprise the following steps：
Judge whether the matching point set quantity of present frame and template key frame is less than given threshold value；
Judge that the external parameters of cameras that present frame is tried to achieve exceedes a certain range, then say that camera moving range becomes larger, it is full
First the preserving certain video frame of the two condition of foot is done new template key frame and is subsequently tracked, selection video frame plan
Slightly：
Obtaining the frame posture information by matrix computations is：
{rx,ry,rz,tx,ty,tz}
By going the posture information after singular value and Kalman prediction to be:
{rx',ry',rz',t'x,t'y,t'z}
Frame tracking is scored at:
G=(rx'-rx)2+(ry'-ry)2+(rz'-rz)2+(t'x-tx)2+(t'y-ty)2+(t'z-tz)2
If g<G, then judge the frame for the optional frame of template replacement.
According to another aspect of the present invention, the present invention provides the tracking of a three-dimensional body, wherein the method bag
Include following steps：
(A) a matched pass of flat image with tracking object is obtained from video frame according to the data of tracked object
The data of key frame, wherein the data of the key frame include the outer parameter of the key frame；
(B) data of the data for the key frame that analysis obtains and the video frame obtained, and then calculate current institute
State the corresponding outer parameter of video frame；And
(C) calculated in the video frame and be traced in object and the template frame according to the outer parameter of the video frame
Tracked object between pose.
An embodiment according to the present invention, the tracking of the three-dimensional body further comprise：
(D) whether the difference between the outer parameter of the video frame and the outer parameter of the key frame meets specific
Threshold value；And
(E) when the difference is unsatisfactory for the threshold value, the template frame is changed to the last template used
Frame.
In the present invention, the step (A) includes：
(F) according to a characteristic point judgment threshold, judge point in current image date whether characteristic point；
(G) according to judgement as a result, extraction characteristic point；
(H) relatively described in (G) described characteristic point of extraction with the characteristic point in a reference data to judge the feature
Point meets that the reference data requires characteristic point accordingly；And
(I) if satisfied, the characteristic point of extraction is as the characteristic point finally extracted in described in (G), if not satisfied, passing through
The characteristic point judgment threshold in (F) described in change, continues to execute (F).
Further, the step (A) includes：
(J) whether the quantity for the characteristic point for judging to be extracted meets a characteristic point amount threshold scope；With
(K) when the discrepancy of quantity for the characteristic point being extracted unifies characteristic point amount threshold scope, described in Homogenization Treatments
Video frame；And
(L) characteristic point of the video frame after extraction is homogenized.
It can thus be seen that the object of the invention can be efficiently accomplished fully.For explaining function and structure principle of the present invention
The embodiment absolutely proved and described, and the present invention from the limit based on the change on these embodiment basis
System.Therefore, the present invention includes covering all modifications within appended claims claimed range and spirit.
Claims (10)
1. the tracking system of a three-dimensional body, it is characterised in that including：
One key frame forms unit, wherein the key frame forms unit by analyzing the data and a video frame of a template frame
Data are to form a key frame data, wherein the key frame data includes the outer parameter of the key frame；
The outer parameter analysis unit of one video frame, wherein the outer parameter analysis unit of the video frame is communicatively coupled to the key frame
Unit is formed, wherein the outer parameter analysis unit of the video frame can obtain the outer parameter of the key frame, and according to the pass
The outer parameter of key frame calculates the outer parameter of the video frame；And
One tracking judging unit, wherein the tracking judging unit is communicatively coupled to parameter analysis unit outside the video frame,
After wherein described video frame analysis module obtains the data of the key frame and the data of the video frame, tracked thing is calculated
Body corresponding pose in the video frame.
2. the tracking system of three-dimensional body according to claim 1, wherein the tracking system of the three-dimensional body is further
Including a data capture unit, wherein the data capture unit further comprises that a template frame acquisition module and a video frame obtain
Modulus block, wherein the template frame acquisition module and the video frame acquisition module are communicatively coupled to the key frame shape respectively
Into unit.
3. the tracking system of three-dimensional body according to claim 2, wherein the data capture unit further comprises one
Characteristic point processing module, wherein the characteristic point processing module includes a characteristic point judgment module and a feature point extraction module,
Wherein described characteristic point judgment module is communicatively coupled to the template frame acquisition module and the video frame acquisition module, wherein
The feature point extraction module is communicatively coupled to the characteristic point judgment module, the key frame forms unit and described is regarded
The outer parameter analysis unit of frequency frame.
4. the tracking of three-dimensional body according to claim 3, wherein the characteristic point processing module further comprises
One texture-rich degree judgment module, wherein the texture-rich degree judgment module includes a characteristic point analysis module, wherein described
Characteristic point analysis module is communicatively coupled to the characteristic point judgment module.
5. the tracking of three-dimensional body according to claim 4, wherein the texture-rich degree judgment module is further
Change judgment module and a Homogenization Treatments module including a characteristic point, wherein the characteristic point changes judgment module by communication link
The feature point extraction module is connected to, wherein the Homogenization Treatments module is communicatively coupled to the characteristic point judgment module.
6. according to the tracking of any three-dimensional body in claim 3 to 5, wherein the characteristic point processing module into
One step includes a Feature Points Matching judgment module, wherein the Feature Points Matching judgment module is communicatively coupled to the characteristic point
Extraction module.
7. the tracking of a three-dimensional body, it is characterised in that comprise the following steps：
(A) one and the matched key frame of flat image of tracking object are obtained from video frame according to the data of tracked object
Data, wherein the data of the key frame include the outer parameter of the key frame；
(B) data of the data for the key frame that analysis obtains and the video frame obtained, and then calculate presently described regard
The corresponding outer parameter of frequency frame；And
(C) calculated according to the outer parameter of the video frame be traced in the video frame in object and the template frame by with
Pose between the object of track.
8. the tracking of three-dimensional body according to claim 7, the tracking of the three-dimensional body further comprises：
(D) whether the difference between the outer parameter of the video frame and the outer parameter of the key frame meets specific threshold
Value；And
(E) when the difference is unsatisfactory for the threshold value, the template frame is changed to the last template frame used.
9. the tracking of the three-dimensional body according to claim 7 or 8, wherein the step (A) includes：
(F) according to a characteristic point judgment threshold, judge point in current image date whether characteristic point；
(G) according to judgement as a result, extraction characteristic point；
(H) relatively described in (G) extraction described characteristic point with the characteristic point in a reference data to judge that the characteristic point expires
The foot reference data requires characteristic point accordingly；And
(I) if satisfied, the characteristic point of extraction is as the characteristic point finally extracted in described in (G), if not satisfied, by varying institute
The characteristic point judgment threshold in (F) is stated, is continued to execute (F).
10. the tracking of three-dimensional body according to claim 9, wherein the step (A) includes：
(J) whether the quantity for the characteristic point for judging to be extracted meets a characteristic point amount threshold scope；With
(K) when the discrepancy of quantity for the characteristic point being extracted unifies characteristic point amount threshold scope, video described in Homogenization Treatments
Frame；And
(L) characteristic point of the video frame after extraction is homogenized.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711183555.4A CN108010055B (en) | 2017-11-23 | 2017-11-23 | Tracking system and tracking method for three-dimensional object |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711183555.4A CN108010055B (en) | 2017-11-23 | 2017-11-23 | Tracking system and tracking method for three-dimensional object |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108010055A true CN108010055A (en) | 2018-05-08 |
| CN108010055B CN108010055B (en) | 2022-07-12 |
Family
ID=62053343
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711183555.4A Active CN108010055B (en) | 2017-11-23 | 2017-11-23 | Tracking system and tracking method for three-dimensional object |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108010055B (en) |
Citations (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2001155164A (en) * | 1999-11-26 | 2001-06-08 | Ntt Communications Kk | Device for tracing mobile object |
| US20070122001A1 (en) * | 2005-11-30 | 2007-05-31 | Microsoft Corporation | Real-time Bayesian 3D pose tracking |
| EP2017769A2 (en) * | 2007-07-19 | 2009-01-21 | Honeywell International Inc. | Multi-pose face tracking using multiple appearance models |
| CN101739686A (en) * | 2009-02-11 | 2010-06-16 | 北京智安邦科技有限公司 | Moving object tracking method and system thereof |
| CN102073864A (en) * | 2010-12-01 | 2011-05-25 | 北京邮电大学 | Football item detecting system with four-layer structure in sports video and realization method thereof |
| CN106342332B (en) * | 2008-07-04 | 2012-10-03 | 中国航空工业集团公司洛阳电光设备研究所 | Target following keeping method when switch visual field under airborne moving condition |
| CN103198488A (en) * | 2013-04-16 | 2013-07-10 | 北京天睿空间科技有限公司 | PTZ surveillance camera realtime posture rapid estimation method |
| CN103839277A (en) * | 2014-02-21 | 2014-06-04 | 北京理工大学 | Mobile augmented reality registration method of outdoor wide-range natural scene |
| US20140226854A1 (en) * | 2013-02-13 | 2014-08-14 | Lsi Corporation | Three-Dimensional Region of Interest Tracking Based on Key Frame Matching |
| CN104145294A (en) * | 2012-03-02 | 2014-11-12 | 高通股份有限公司 | Scene structure-based self-pose estimation |
| CN104778697A (en) * | 2015-04-13 | 2015-07-15 | 清华大学 | Three-dimensional tracking method and system based on fast positioning of image dimension and area |
| US20160110885A1 (en) * | 2014-10-21 | 2016-04-21 | Government Of The United States As Represented By The Secretary Of The Air Force | Cloud based video detection and tracking system |
| CN105578034A (en) * | 2015-12-10 | 2016-05-11 | 深圳市道通智能航空技术有限公司 | Control method, control device and system for carrying out tracking shooting for object |
| CN107122770A (en) * | 2017-06-13 | 2017-09-01 | 驭势（上海）汽车科技有限公司 | Many mesh camera systems, intelligent driving system, automobile, method and storage medium |
| CN107122782A (en) * | 2017-03-16 | 2017-09-01 | 成都通甲优博科技有限责任公司 | A kind of half intensive solid matching method in a balanced way |
| CN107248169A (en) * | 2016-03-29 | 2017-10-13 | 中兴通讯股份有限公司 | Image position method and device |
-
2017
- 2017-11-23 CN CN201711183555.4A patent/CN108010055B/en active Active
Patent Citations (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2001155164A (en) * | 1999-11-26 | 2001-06-08 | Ntt Communications Kk | Device for tracing mobile object |
| US20070122001A1 (en) * | 2005-11-30 | 2007-05-31 | Microsoft Corporation | Real-time Bayesian 3D pose tracking |
| EP2017769A2 (en) * | 2007-07-19 | 2009-01-21 | Honeywell International Inc. | Multi-pose face tracking using multiple appearance models |
| CN106342332B (en) * | 2008-07-04 | 2012-10-03 | 中国航空工业集团公司洛阳电光设备研究所 | Target following keeping method when switch visual field under airborne moving condition |
| CN101739686A (en) * | 2009-02-11 | 2010-06-16 | 北京智安邦科技有限公司 | Moving object tracking method and system thereof |
| CN102073864A (en) * | 2010-12-01 | 2011-05-25 | 北京邮电大学 | Football item detecting system with four-layer structure in sports video and realization method thereof |
| CN104145294A (en) * | 2012-03-02 | 2014-11-12 | 高通股份有限公司 | Scene structure-based self-pose estimation |
| US20140226854A1 (en) * | 2013-02-13 | 2014-08-14 | Lsi Corporation | Three-Dimensional Region of Interest Tracking Based on Key Frame Matching |
| CN103198488A (en) * | 2013-04-16 | 2013-07-10 | 北京天睿空间科技有限公司 | PTZ surveillance camera realtime posture rapid estimation method |
| CN103839277A (en) * | 2014-02-21 | 2014-06-04 | 北京理工大学 | Mobile augmented reality registration method of outdoor wide-range natural scene |
| US20160110885A1 (en) * | 2014-10-21 | 2016-04-21 | Government Of The United States As Represented By The Secretary Of The Air Force | Cloud based video detection and tracking system |
| CN104778697A (en) * | 2015-04-13 | 2015-07-15 | 清华大学 | Three-dimensional tracking method and system based on fast positioning of image dimension and area |
| CN105578034A (en) * | 2015-12-10 | 2016-05-11 | 深圳市道通智能航空技术有限公司 | Control method, control device and system for carrying out tracking shooting for object |
| CN107248169A (en) * | 2016-03-29 | 2017-10-13 | 中兴通讯股份有限公司 | Image position method and device |
| CN107122782A (en) * | 2017-03-16 | 2017-09-01 | 成都通甲优博科技有限责任公司 | A kind of half intensive solid matching method in a balanced way |
| CN107122770A (en) * | 2017-06-13 | 2017-09-01 | 驭势（上海）汽车科技有限公司 | Many mesh camera systems, intelligent driving system, automobile, method and storage medium |
Non-Patent Citations (4)
| Title |
|---|
| MARTIN HOSSBACH 等: "Design and analysis of a calibrationmethod for stereo-optical motion tracking in MRI using a virtual calibration phantom", 《MEDICAL IMAGING 2013: PHYSICS OF MEDICAL IMAGING》 * |
| SHU CHEN 等: "3D Pose Tracking With Multitemplate Warping and SIFT Correspondences", 《 IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY》 * |
| 岳思聪 等: "基于奇异值分解的宽基线图像匹配算法", 《计算机科学》 * |
| 顾照鹏、董秋雷: "基于部分惯性传感器信息的单目视觉同步定位与地图创建方法", 《计算机辅助设计与图形学学报》 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108010055B (en) | 2022-07-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107392964B (en) | The indoor SLAM method combined based on indoor characteristic point and structure lines | |
| CN105701820B (en) | A kind of point cloud registration method based on matching area | |
| CN112505065B (en) | Method for detecting surface defects of large part by indoor unmanned aerial vehicle | |
| CN109685913B (en) | Augmented reality implementation method based on computer vision positioning | |
| CN106709436A (en) | Cross-camera suspicious pedestrian target tracking system for rail transit panoramic monitoring | |
| CN107481279B (en) | Monocular video depth map calculation method | |
| CN109584251A (en) | A kind of tongue body image partition method based on single goal region segmentation | |
| CN108427871A (en) | 3D faces rapid identity authentication method and device | |
| CN109544456A (en) | The panorama environment perception method merged based on two dimensional image and three dimensional point cloud | |
| CN104463859B (en) | A kind of real-time video joining method based on tracking specified point | |
| CN110390640A (en) | Graph cut image split-joint method, system, equipment and medium based on template | |
| CN107239748A (en) | Robot target identification and localization method based on gridiron pattern calibration technique | |
| CN110232389A (en) | A kind of stereoscopic vision air navigation aid based on green crop feature extraction invariance | |
| CN110176016B (en) | Virtual fitting method based on human body contour segmentation and skeleton recognition | |
| CN111046856B (en) | Parallel pose tracking and map creating method based on dynamic and static feature extraction | |
| CN110189294B (en) | RGB-D image significance detection method based on depth reliability analysis | |
| Zhang et al. | Application of migration image registration algorithm based on improved SURF in remote sensing image mosaic | |
| CN110516639B (en) | Real-time figure three-dimensional position calculation method based on video stream natural scene | |
| CN109711267A (en) | A kind of pedestrian identifies again, pedestrian movement's orbit generation method and device | |
| CN111199556A (en) | Indoor pedestrian detection and tracking method based on camera | |
| CN113744315B (en) | Semi-direct vision odometer based on binocular vision | |
| CN110298860A (en) | A kind of high bar silk ball detection number system based on machine vision | |
| CN105741326B (en) | A kind of method for tracking target of the video sequence based on Cluster-Fusion | |
| CN108010055A (en) | The tracking system and its tracking of three-dimensional body | |
| CN110430400A (en) | A kind of ground level method for detecting area of the movable video camera of binocular |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information |
Address after: 202177 room 493-61, building 3, No. 2111, Beiyan highway, Chongming District, Shanghai Applicant after: TAPUYIHAI (SHANGHAI) INTELLIGENT TECHNOLOGY Co.,Ltd. Address before: 201802 room 412, building 5, No. 1082, Huyi Road, Jiading District, Shanghai Applicant before: TAPUYIHAI (SHANGHAI) INTELLIGENT TECHNOLOGY Co.,Ltd. |
|
| CB02 | Change of applicant information | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |