CN105225233B - A kind of stereopsis dense Stereo Matching method and system based on the expansion of two classes - Google Patents
A kind of stereopsis dense Stereo Matching method and system based on the expansion of two classes Download PDFInfo
- Publication number
- CN105225233B CN105225233B CN201510586875.9A CN201510586875A CN105225233B CN 105225233 B CN105225233 B CN 105225233B CN 201510586875 A CN201510586875 A CN 201510586875A CN 105225233 B CN105225233 B CN 105225233B
- Authority
- CN
- China
- Prior art keywords
- matched
- point
- image
- expansion
- matching
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
- G06T2207/10012—Stereo images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20016—Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform
Landscapes
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
A kind of stereopsis dense Stereo Matching method and system based on the expansion of two classes, including notable feature extraction is carried out to reference images, the image formed to reference images and image to be matched is to carrying out rough matching as initial known point；First kind expansion is carried out with each known point in current known point set respectively and searches for characteristic point to be matched, carry out the expansion of the second class respectively to each characteristic point to be matched and search the closest some known points of surrounding, distance weighted average computation goes out same place initial position, and root determines hunting zone；Carry out Gray-scale Matching, reject insecure known point, using coefficient correlation it is big as newly-increased known point；Dense Stereo Matching result is obtained after meeting iteration termination condition.The present invention can effectively solve the problems, such as that texture is not notable and the error hiding of parallax discontinuity zone, while this matching operation amount is very small, and treatment effeciency is high, be adapted to the image of various imaging types, and not need the geometrical condition of any priori.
Description
Technical field
It is more particularly to a kind of to be stood using two class inflation policies the present invention relates to stereopsis dense Stereo Matching technical field
The method and system of body image dense Stereo Matching.
Background technology
Stereopsis dense Stereo Matching is computation vision and a basis in photogrammetric field and key issue.Dense Stereo Matching
Concept most early in photogrammetric field propose, for solve the problems, such as digital aerial surveying automate mapping.Intensive
With the key issue for being also computer vision field, be related to 3D models establish, robot navigation and operation and in a computer
Form mixing outdoor scene action etc..
The target of three-dimensional dense Stereo Matching is that three-dimensional scene models are rebuild from bidimensional image.By under scene different points of view
Two width images establish matching corresponding relation, with the three-dimensional information of restoration scenario, the problem of its is crucial is how to utilize image
With the reliable same place of acquisition, therefore the essence of Image Matching is to measure or obtain same place or spy of the same name on different images
Sign.This that three-dimensional scenic inherently ill-conditioning problem is rebuild from bidimensional image, marking area and parallax be not discontinuous for texture
Error hiding problem be difficult to be resolved comprehensively always.To obtain more preferable matching result, it is necessary to rationally include more priori
Constraint.Such as the matching process based on area grayscale, improve Window match using image local information in matching process,
The raisings such as Pyramid technology matching strategy matching stability and accuracy are utilized simultaneously.But this method generally require it is larger
Match window, matching result are difficult to ensure that the recovery to image local detail.Also certain methods using Image Segmentation as constraint,
But there is very strong dependence for image segmentation result.In a word, traditional dense Stereo Matching method is often directly with various constraints
Condition reduces disparity search scope, therefore too strong for the dependence of priori conditions, it is desirable to which priori conditions have higher precision, no
Then easily produce error hiding.
Occur some in recent years by improving cost agglomeration approach to realize the method for matching, for example with adaptive cost
Agglomeration approach improves local window matching process, the matching problem for solving to repeat texture using multiple dimensioned cost agglomeration approach, adopts
Solves the discontinuous matching problem of parallax with half global cost agglomeration approach.The common ground of these methods is all in cost matrix
Respective constraint is realized, but the constraint due to including is not abundant enough, and therefore, it is difficult to more fully solve the problems, such as error hiding.In addition this
A little method amounts of calculation using half global cost accumulation are very huge, current survey production requirement can not be met in efficiency, more
It can not meet real-time demand.
In particular, current image matching method, it is most of to all rely on " geometrical condition of photography ", such as
" constraint of core line " condition etc., therefore when the photogrammetric distortion of video camera is larger, Image Matching will be affected.Meanwhile to current
For most of image matching methods, difference (such as light of geometrical condition different (such as face battle array, line are old), imaging mechanism when photography
Study as being " angle imaging ", and radar image is then " range Imaging ") when, it is impossible to accomplish to adapt to automatically.And the present invention with
Imaging geometry, imaging mechanism are unrelated, as long as two images have " parallax ", can be achieved with matching operation, determine same place.Essence
On, it is closer to " stereovision of people ".
The content of the invention
Present invention is generally directed to constrain (the i.e. easily production of the too strong problem of dependence for priori conditions present in prior art
Raw error hiding) and inefficient problem, it is proposed that it is a kind of using approximate same place on a small quantity as seed point, using two class inflation policies,
Expanding matching is carried out to all characteristic points on image, can effectively solve that texture is not notable and the mistake of parallax discontinuity zone
Matching problem.
Technical solution of the present invention provides a kind of stereopsis dense Stereo Matching method based on the expansion of two classes, including following step
Suddenly：
Step 1, a width image is taken as reference images, and notable feature extraction is carried out to local image according to predetermined interval,
Form the set of characteristic point to be matched；
Step 2, seed point of the same name is obtained to carrying out rough matching to the image that reference images and image to be matched are formed, made
For initial known point, initial known point set is formed；
Step 3, based on reference images, first kind expansion is carried out with each known point in current known point set respectively,
Characteristic point to be matched is searched in first kind expansion；
Step 4, based on reference images, it is swollen that the second class is carried out respectively to each characteristic point to be matched that must be found in step 3
It is swollen, the second class expansion in search some known points closest around the characteristic point to be matched, by find it is all
Know and click through row distance weighted average, calculate the initial position of same place of the point feature point to be matched on image to be matched, and
All known points according to finding determine hunting zone,
If certain feature point coordinates to be matched is p (x in reference imagesp,yp), the second class expansion in find it is all
Know that points are N, the collection for obtaining all known points pair is combined into Q, it is described it is distance weighted it is average be defined as follows,
Wherein, p'(xp',yp') represent characteristic point p (x to be matchedp,yp) on image to be matched possibility matching point coordinates,
With p'(xp',yp') be same place of the point feature point to be matched on image to be matched initial position；
A known point pair in set Q is represented,
Represent a known point pair in set Q, i, j=1,2,3..., N；
Represent p (xp,yp) andThe distance between；
Step 5, based on reference images and image to be matched, characteristic point to be matched is in image to be matched according to obtained by step 4
On same place initial position and hunting zone carry out Gray-scale Matching, including reference images and image to be matched are made respectively
For the left and right image of image pair, respectively to the left and right of image pair centered on the initial position of characteristic point to be matched and its same place
Image establishes multistage pyramid image block, then intensity correlation matching is carried out on pyramid top layer image, according to initial position
And hunting zone obtains the same place on image to be matched, then transmit downwards and match step by step by pyramid series, finally return
To raw video, the coordinate and coefficient correlation of same place of the characteristic point to be matched on image to be matched are obtained；
Step 6, to each known point in current known point set, counted respectively according to step 5 gained matching result,
Including being carried out in one known point current iteration of statistics in all characteristic points to be matched that first kind expansion is found, matching gained phase
Relation number accounts for the ratio of all characteristic points to be matched less than the characteristic point to be matched of predetermined coefficient threshold value, when the ratio is in default ratio
Rate threshold value then rejects this known point from current known point set；
Step 7, to remaining each known point in current known point set, according to step 5 gained matching result, when needing
Coefficient correlation obtained by the matching of matching characteristic point is more than or equal to predetermined coefficient threshold value, as newly-increased known point, is added to known
In point set；Judge whether the newly-increased known point sum of current iteration is less than default amount threshold, if otherwise return to step 3,
If then terminating flow, current known point set is last dense Stereo Matching result.
Moreover, expansion algorithm used in first kind expansion and the expansion of the second class is defined as follows,
Wherein,
X represents the finite element in finite element set X,
B [] represents to increase operation to the eight neighborhood of element；
D (X) represents result element set.
Moreover, in step 3, when certain known point in current known point set carries out first kind expansion, if this is known
Search border has been arrived in the hunting zone of point, then skips the point without expansion.
Moreover, in step 5, intensity correlation matching is using the covariance of the standardization of left and right match window gray scale as related
Coefficient, calculated according to equation below,
Wherein,
U, ν represent two points of coefficient correlation to be calculated on the image of left and right respectively；
R (u, ν) represents u, the coefficient correlation that ν at 2 points；
M, N represent to calculate width, the height of the left and right match window of coefficient correlation；
Represent the gray value of pixel inside the match window of left and right；
Represent the gray average of left and right match window.
The present invention accordingly provides a kind of stereopsis dense Stereo Matching system based on the expansion of two classes, including with lower module：
Characteristic point search module to be matched, for taking a width image as reference images, according to predetermined interval to local shadow
As carrying out notable feature extraction, the set of characteristic point to be matched is formed；
Seed point extraction module, for reference images and image to be matched composition image to carry out rough matching acquisition
Seed point of the same name, as initial known point, form initial known point set；
First kind expansion module, for based on reference images, being carried out respectively with each known point in current known point set
The first kind is expanded, and characteristic point to be matched is searched in first kind expansion；
Second class expansion module, it is each to be matched to what must be found in first kind expansion module for based on reference images
Characteristic point carries out the second class expansion respectively, searched in the expansion of the second class around the characteristic point to be matched it is closest it is some
Know a little, it is distance weighted average by all known points progress found, point feature point to be matched is calculated on image to be matched
Same place initial position, and determine hunting zone according to all known points found,
If certain feature point coordinates to be matched is p (x in reference imagesp,yp), the second class expansion in find it is all
Know that points are N, the collection for obtaining all known points pair is combined into Q, it is described it is distance weighted it is average be defined as follows,
Wherein,
p'(xp',yp') represent characteristic point p (x to be matchedp,yp) on image to be matched possibility matching point coordinates, with p'
(xp',yp') be same place of the point feature point to be matched on image to be matched initial position；
A known point pair in set Q is represented,
Represent a known point pair in set Q, i, j=1,2,3..., N；
Represent p (xp,yp) andThe distance between；
Gray-scale Matching module, for based on reference images and image to be matched, being treated according to obtained by the second class expansion module
The initial position of same place with characteristic point on image to be matched and hunting zone carry out Gray-scale Matching, including by benchmark shadow
Picture and image to be matched respectively as image pair left and right image, using the initial position of characteristic point to be matched and its same place in
The heart establishes multistage pyramid image block to the left and right image of image pair respectively, and gray scale phase is then carried out on pyramid top layer image
Matching is closed, the same place on image to be matched is obtained according to initial position and hunting zone, then by pyramid series step by step
Downwards transmit matching, eventually pass back to raw video, obtain same place of the characteristic point to be matched on image to be matched coordinate and
Coefficient correlation；
Screening module, for each known point in current known point set, respectively according to obtained by Gray-scale Matching module
Counted with result, including all features to be matched that first kind expansion is found are carried out in one known point current iteration of statistics
In point, matching gained coefficient correlation accounts for the ratio of all characteristic points to be matched less than the characteristic point to be matched of predetermined coefficient threshold value,
When the ratio then rejects this known point in pre-set ratio threshold value from current known point set；
Result judgement module, for remaining each known point in current known point set, according to Gray-scale Matching module
Gained matching result, the coefficient correlation obtained by need the matching of matching characteristic point are more than or equal to predetermined coefficient threshold value, as increasing newly
Known point, be added in known point set；Judge whether the newly-increased known point sum of current iteration is less than default quantity threshold
Value, if otherwise it is added in known point set, and order first kind expansion module works on, if then power cut-off, currently
Known point set be last dense Stereo Matching result.
Moreover, expansion algorithm used in first kind expansion and the expansion of the second class is defined as follows,
Wherein,
X represents the finite element in finite element set X,
B [] represents to increase operation to the eight neighborhood of element；
D (X) represents result element set.
Moreover, in first kind expansion module, when certain known point in current known point set carries out first kind expansion,
If search border has been arrived in the hunting zone of the known point, the point is skipped without expansion.
Moreover, in Gray-scale Matching module, intensity correlation matching uses the covariance of the standardization of left and right match window gray scale
As coefficient correlation, calculated according to equation below,
Wherein,
U, ν represent two points of coefficient correlation to be calculated on the image of left and right respectively；
R (u, ν) represents u, the coefficient correlation that ν at 2 points；
M, N represent to calculate width, the height of the left and right match window of coefficient correlation；
Represent the gray value of pixel inside the match window of left and right；
Represent the gray average of left and right match window.
The invention has the advantages that：
The present invention proposes one kind using a small amount of approximate same place as seed point, using two class inflation policies, to institute on image
There is the technical scheme that characteristic point carries out expanding matching.Due to taken into full account during dense Stereo Matching each match point and around
The correlation of point, is enhanced the accuracy of the prediction of same place, is disobeyed using the precision of this method technical scheme acquired results
Rely the same place positional precision obtained in characteristic matching, in the matching process can the relatively low kind of precision in automatic rejection characteristic matching
It is sub-, ensure the reliability of result.The present invention for present in prior art for priori conditions constrain dependence it is too strong and
Inefficient problem and propose, be diffused according to the reliable matching result of previous stage, can effectively solve texture not significantly with
And the error hiding problem of parallax discontinuity zone, while ensure with the operand of very little the high efficiency of algorithm.
Can effectively solve the problems, such as that texture is not notable and the error hiding of parallax discontinuity zone using the present invention, while this
Kind matching algorithm operand is very small, and treatment effeciency is high, is adapted to the image of various imaging types, and does not need any priori
" geometrical condition ".
Brief description of the drawings
Fig. 1 is the overview flow chart of the embodiment of the present invention；
Fig. 2 be the embodiment of the present invention certain expansion when local image on seed point, known point, the distribution situation of point to be matched
Schematic diagram；
Fig. 3 is the schematic diagram for the first kind inflation policy that the embodiment of the present invention is looked for point to be matched by known point；
Fig. 4 is that the embodiment of the present invention looks for known point to determine that the second class of initial position and hunting zone expands by point to be matched
The schematic diagram of strategy.
Embodiment
Technical scheme is described in detail below in conjunction with drawings and examples.
Technical scheme provided by the invention is a kind of method that image dense Stereo Matching is carried out using two class inflation policies, its
In two class inflation policies be this Image Matching key technology.When it is implemented, computer software technology can be used to realize certainly
Dynamic operational process.As shown in figure 1, embodiment is expanded using a small amount of approximate same place as seed point using two classes, to owning on image
Characteristic point carries out expanding matching, and flow comprises the following steps：
Step 1, select any one width image that it is notable to carry out Harris to local image according to interval cx as reference images
Feature extraction, form set of characteristic points to be matched.Point in Fig. 2 represented by crosshair is characteristic point to be matched (pf).Tool
When body is implemented, those skilled in the art can voluntarily predetermined interval cx value.
Step 2, seed point of the same name is obtained (such as to carrying out rough matching to the image that reference images and image to be matched are formed
Point Pseed represented by Fig. 2 intermediate cam shapes), seed point set is obtained, while these seed points also serve as initial known point (such as
Point Pknown in Fig. 2 represented by square), form initial known matching point set, referred to as initial known point set.It is known
Same place point position need not be very accurate in matching point set.When rough matching, the spy of some comparative maturities can be used
Sign matching algorithm is handled (such as SIFT matchings, SURF matchings, HarrisSIFT matchings), can also be used and reduced image
The method of the accurate Gray-scale Matching based on window of enterprising rower obtains a small amount of same place.Special instruction, even if known
Subsequent match is not interfered with yet with the Mismatching point for having a small amount of in point set.When it is implemented, typically require initial seed
Point at least three, if rough matching obtains seed point of the same name and do not reach three, need separately to match searching.
Step 3, based on reference images, expanded and searched for respectively with each known point in current known point set and treated
With characteristic point：
Centered on pending known point, first kind expansion is carried out, finds the characteristic point to be matched in expansion path, and
Record these characteristic points.If the hunting zone of the point has arrived it and has searched for border, then skips the point, progress is next
Know search a little, and return to step 3 can all be skipped no longer to point progress first kind expansion afterwards.When it is implemented,
Each known point can be extracted and operated successively in this way.When performing step 3 for the first time, to step 2 gained seed point institute structure
Each point performs expansion respectively as known point into initial known matching point set, during subsequent execution step 3, is changed according to last round of
All known points are performed expansion by each newly-increased known point respectively obtained by the step 7 in generation.When it is implemented, search border may
There are two kinds of situations, first refers to edge that image has been arrived when expansion；Second is, the first kind of two adjacent known points is swollen
Swollen scope connects.
In practical operation of the present invention, expand, including each known point may be carried out more for the first kind of known point
The expansion of secondary expanding, find the characteristic point to be matched around it.Model can be expanded to any known point Pknown, respectively search
Enclose interior all characteristic point Pf to be matched.
Expansion algorithm is defined as follows：
Wherein:
Finite element x in x ∈ X expression finite element set X is known point Pknown in of the invention (to step 3
Speech) or feature Pf to be matched (for step 4)；
B [] represents certain operation to element, represents that eight neighborhood increases operation herein；
D (X) represents, to finite element set X operation acquired results element sets, it is to be matched to represent that expansion obtains herein
Characteristic point.
As shown in Figure 3, it is assumed that now Pseed (i) is the known point for needing expanded search, while is assumed this time for the point
Expand for the first time, then the expansion path that numbering is 1 along in Fig. 3, can obtain including f1, f2, f3, f4, f5, f6 and Pf
(j) 7 characteristic points to be matched are amounted to including.When being back to step 3 next time Pseed (i) is scanned for, enter again
According to the expansion path that numbering is 2 in Fig. 3 handle during capable expansion, similarly again next iteration when, to Pseed
(i) expansion process is carried out according to the expansion path that numbering is 3 in Fig. 3.
Step 4, based on reference images, the second class expansion is carried out to each characteristic point to be matched that must be found in step 3, looked into
Look for multiple known points closest around the point, the threshold value at least counted can be set during specific implementation, find it is each
Know a little and respective point forms known point pair on image to be matched.It is distance weighted average by the known point progress found, calculate
The initial position of point feature point same place to be matched, and determine hunting zone.
First kind expansion is the expansion carried out to known point；The expansion of second class is to treat the expansion of match point progress.This hair
Bright that the second class expansion is carried out to the characteristic point to be matched obtained in step 3 in step 4, specific implementation and the expansion of step 3 are calculated
The definition of method is consistent, searches out the known point of the certain amount around characteristic point to be matched.
It is distance weighted average to be defined as follows：
Wherein:
I, j=1,2,3..., N, N represent the number of known point pair；
p(xp,yp) represent feature point coordinates to be matched in reference images；
p'(xp',yp') represent that the possibility of point to be matched matches point coordinates (on image to be matched), i.e., characteristic point to be matched
The initial position of same place；
Q represents the set of all known points pair,Represent a matching in the set
Point pair, qi,q'iPoint respectively in reference images and image to be matched；Equally,Representing should
A matching double points in set, qj,q'jPoint respectively in reference images and image to be matched；
Represent p, qiThe distance between point, qiBe known point in set Q appoint
Left point (in reference images) of meaning known point centering.
In step 4, the initial matching position of its same place and matching are calculated by the known point around characteristic point to be matched
Hunting zone.The correlation of each match point and surrounding point has been taken into full account, has enhanced the accuracy of the prediction of same place, has been improved
Rate that the match is successful.
It is assumed that search is now extended to Pf (j), then searched along expansion path shown in figure
Rope, it can find including Pseed (i), s1, s2, s3, s4, s5, s6 and n1, n2, n3, n4, n5,11 known points including n6,
Assuming that the known point quantity threshold needed around the point to be matched set has currently reached requirement, then can passes through this as 10
11 known points, the rough matching position of the same place of characteristic point Pf (j) to be matched is estimated by distance weighted average method
Put (Qf_p (j), i.e. p'(xp',yp')) and matching hunting zone (Rang (j)), i.e., the initial position of to be matched same place with
And hunting zone.The hunting zone matched herein can be e.g. swollen by the second class by the default structure rule of those skilled in the art
The outsourcing rectangle of matching point coordinates corresponding to the known point (i.e. known point in set Q) of swollen middle lookup and determine.
Step 5, based on reference images and image to be matched, the initial position of to be matched same place is obtained according to step 4
And hunting zone carries out Gray-scale Matching.To accelerate matching speed in matching process, using single-point multistage pyramid matching algorithm.
Specific algorithm is：Using reference images and image to be matched as the left and right image of image pair, with point to be matched and its of the same name
Multistage pyramid image block is established to the left and right image of image pair respectively centered on the initial position of point, then in pyramid top layer
Intensity correlation matching is carried out on image, is obtained according to step 4 gained initial position and hunting zone same on image to be matched
Famous cake, then transmit downwards and match step by step by pyramid series, the raw video acquisition point to be matched for eventually passing back to the bottom is same
The coordinate (Pf (j) optimal match point position Qf (j)) and coefficient correlation (Cf (j)) of famous cake.
The related criterion of gray scale is using the coefficient correlation between two windows established on the image of left and right, coefficient correlation
The covariance of the standardization of actually two window gray scales.Calculate according to equation below：
Wherein：
U, ν represent two points of coefficient correlation to be calculated on the image of left and right respectively；
R (u, ν) represents u, the coefficient correlation that ν at 2 points；
M, N represent that the width for calculating the left and right match window of coefficient correlation is high；
Represent the gray value of pixel inside the match window of left and right；
Represent the gray average of left and right match window.
When calculating each time, to two points u, ν to be calculated, M × N size is established on the image of left and right respectively
The window of (those skilled in the art voluntarily can set value according to image size during specific implementation), then according to above formula meter
The coefficient correlation of two windows is calculated, obtained value is considered u, the coefficient correlation between two pixels of ν.Finally in position to be matched
Hunting zone in, find coefficient correlation maximum when corresponding two points position, it is believed that this position is exactly the position of same place
Put.
Step 6, to each known point in current known point set, counted respectively according to step 5 gained matching result,
Including being carried out in one known point current iteration of statistics in all characteristic points to be matched that first kind expansion is found, matching gained phase
Relation number accounts for the ratios of all characteristic points to be matched less than the characteristic point to be matched of pre-determined factor threshold value, when the ratio is in given
Rate threshold then rejects this known point from current known point set.
In embodiment, after the completion of each characteristic point to be matched found in step 3 is matched, it is by each known point
Packet object counts to matching result.If matching result in the multiple characteristic points found in some known point expansion process
Coefficient correlation (such as R that step 5 calculates (u, v)) is less than given coefficient threshold (when it is implemented, those skilled in the art can be voluntarily
Predetermined threshold value, such as Thresh_C take empirical value to be more than given rate threshold for the ratio (Per (i)) of characteristic point 0.8)
(when it is implemented, those skilled in the art can voluntarily predetermined threshold value, such as it is 60%) then to think this that Thresh_P, which takes empirical value,
Known node failure, rejected as erroneous point.By the step, reduce this method and wanted for seed point positional accuracy
Ask, improve the reliability of this method matching.
Step 7, to remaining each known point in current known point set, according to step 5 gained matching result, when needing
Coefficient correlation obtained by the matching of matching characteristic point is more than or equal to predetermined coefficient threshold value, as newly-increased known point, is added to known
In point set；Judge whether the newly-increased known point of current iteration total (Nnknown) is less than default amount threshold, if otherwise returning
Step 3 is returned, if then terminating flow, current known point set is last dense Stereo Matching result.
If in the judgement of step 6, it is believed that certain known point is unreliable, then think of the corresponding packet of the known point
It is also unreliable, it is necessary to abandon the matching result of the packet with result.If thinking, the known point is still reliable, will be obtained in step 5
To coefficient correlation be more than the reliable matching point of certain coefficient threshold (Thresh_C) as newly-increased known point and be put into known to
In point set, and return to step 3, step 7 is arrived according to current known point set repeat step 3, until newfound known point
Number be less than certain amount threshold (when it is implemented, those skilled in the art can voluntarily preset respective threshold Thresh_N,
Such as take Thresh_N=10) when terminate whole matching process.
When it is implemented, those skilled in the art can also use modular mode to provide corresponding system.The present invention provides one
Stereopsis dense Stereo Matching system of the kind based on the expansion of two classes, including with lower module：
Characteristic point search module to be matched, for taking a width image as reference images, according to predetermined interval to local shadow
As carrying out notable feature extraction, the set of characteristic point to be matched is formed；
Seed point extraction module, for reference images and image to be matched composition image to carry out rough matching acquisition
Seed point of the same name, as initial known point, form initial known point set；
First kind expansion module, for based on reference images, being carried out respectively with each known point in current known point set
The first kind is expanded, and characteristic point to be matched is searched in first kind expansion；
Second class expansion module, it is each to be matched to what must be found in first kind expansion module for based on reference images
Characteristic point carries out the second class expansion respectively, searched in the expansion of the second class around the characteristic point to be matched it is closest it is some
Know a little, it is distance weighted average by all known points progress found, point feature point to be matched is calculated on image to be matched
Same place initial position, and determine hunting zone according to all known points found,
If certain feature point coordinates to be matched is p (x in reference imagesp,yp), the second class expansion in find it is all
Know that points are N, the collection for obtaining all known points pair is combined into Q, it is described it is distance weighted it is average be defined as follows,
Wherein,
p'(xp',yp') represent characteristic point p (x to be matchedp,yp) on image to be matched possibility matching point coordinates, with p'
(xp',yp') be same place of the point feature point to be matched on image to be matched initial position；
A known point pair in set Q is represented,
Represent a known point pair in set Q, i, j=1,2,3..., N；
Represent p (xp,yp) andThe distance between；
Gray-scale Matching module, for based on reference images and image to be matched, being treated according to obtained by the second class expansion module
The initial position of same place with characteristic point on image to be matched and hunting zone carry out Gray-scale Matching, including by benchmark shadow
Picture and image to be matched respectively as image pair left and right image, using the initial position of characteristic point to be matched and its same place in
The heart establishes multistage pyramid image block to the left and right image of image pair respectively, and gray scale phase is then carried out on pyramid top layer image
Matching is closed, the same place on image to be matched is obtained according to initial position and hunting zone, then by pyramid series step by step
Downwards transmit matching, eventually pass back to raw video, obtain same place of the characteristic point to be matched on image to be matched coordinate and
Coefficient correlation；
Screening module, for each known point in current known point set, respectively according to obtained by Gray-scale Matching module
Counted with result, including all features to be matched that first kind expansion is found are carried out in one known point current iteration of statistics
In point, matching gained coefficient correlation accounts for the ratio of all characteristic points to be matched less than the characteristic point to be matched of predetermined coefficient threshold value,
When the ratio then rejects this known point in pre-set ratio threshold value from current known point set；
Result judgement module, for remaining each known point in current known point set, according to Gray-scale Matching module
Gained matching result, the coefficient correlation obtained by need the matching of matching characteristic point are more than or equal to predetermined coefficient threshold value, as increasing newly
Known point, be added in known point set；Judge whether the newly-increased known point sum of current iteration is less than default quantity threshold
Value, if otherwise it is added in known point set, and order first kind expansion module works on, if then power cut-off, currently
Known point set be last dense Stereo Matching result.
Specific module realization is corresponding to each step, and it will not go into details by the present invention.
Specific embodiment described herein is only to spirit explanation for example of the invention.Technology belonging to the present invention is led
The technical staff in domain can be made various modifications or supplement to described specific embodiment or be replaced using similar mode
Generation, but without departing from the spiritual of the present invention or surmount scope defined in appended claims.
Claims (10)
- A kind of 1. stereopsis dense Stereo Matching method based on the expansion of two classes, it is characterised in that comprise the following steps：Step 1, a width image is taken as reference images, and notable feature extraction is locally carried out to reference images according to predetermined interval, Form the set of characteristic point to be matched；Step 2, seed point of the same name is obtained to carrying out rough matching to the image that reference images and image to be matched are formed, as first The known point of beginning, form initial known point set；Step 3, based on reference images, first kind expansion is carried out with each known point in current known point set respectively, first Characteristic point to be matched is searched in class expansion；Step 4, based on reference images, the second class expansion is carried out respectively to each characteristic point to be matched obtained in step 3, Some known points closest around the characteristic point to be matched are searched in the expansion of two classes, are carried out by all known points found It is distance weighted average, calculate the initial position of same place of the characteristic point to be matched on image to be matched, and according to finding All known points determine hunting zone,If certain feature point coordinates to be matched is p (x in reference imagesp,yp), all known points found in the expansion of the second class Number is N, searches and obtains the collection of all known points pair and be combined into Q, it is described it is distance weighted it is average be defined as follows,Wherein,p'(xp',yp') represent characteristic point p (x to be matchedp,yp) on image to be matched possibility matching point coordinates, with p'(xp', yp') be same place of the characteristic point to be matched on image to be matched initial position；A known point pair in set Q is represented,Represent collection Close a known point pair in Q, i, j=1,2,3..., N；Represent p (xp,yp) andThe distance between；Step 5, based on reference images and image to be matched, characteristic point to be matched is on image to be matched according to obtained by step 4 The initial position of same place and hunting zone carry out Gray-scale Matching, including using reference images and image to be matched as shadow The left and right image of picture pair, respectively to the left and right image of image pair centered on the initial position of characteristic point to be matched and its same place Establish multistage pyramid image block, then carry out intensity correlation matching on pyramid top layer image, according to initial position and Hunting zone obtains the same place on image to be matched, then transmits downwards and matches step by step by pyramid series, eventually passes back to original Beginning image, obtain the coordinate and coefficient correlation of same place of the characteristic point to be matched on image to be matched；Step 6, to each known point in current known point set, counted respectively according to step 5 gained matching result, including Carried out in one known point current iteration of statistics in all characteristic points to be matched that first kind expansion is found, matching gained phase relation Number accounts for the ratio of all characteristic points to be matched less than the characteristic point to be matched of predetermined coefficient threshold value, when the ratio is more than pre-set ratio Threshold value then rejects this known point from current known point set；Step 7, to remaining each known point in current known point set, according to step 5 gained matching result, treated when there is certain Coefficient correlation obtained by matching with characteristic point is more than or equal to predetermined coefficient threshold value, using the characteristic point to be matched as known to increasing newly Point, it is added in known point set；Judge whether the newly-increased known point sum of current iteration is less than default amount threshold, if not Then return to step 3, if then terminating flow, current known point set is last dense Stereo Matching result.
- 2. the stereopsis dense Stereo Matching method according to claim 1 based on the expansion of two classes, it is characterised in that：The first kind is swollen Expansion algorithm used in the expansion of swollen and the second class is defined as follows,Wherein,X represents the finite element in finite element set X,B [] represents to increase operation to the eight neighborhood of element；D (X) represents result element set.
- 3. the stereopsis dense Stereo Matching method according to claim 1 or claim 2 based on the expansion of two classes, it is characterised in that：Step 3 In, when certain known point in current known point set carries out first kind expansion, if the hunting zone of the known point has been arrived Search border, then the point is skipped without expansion.
- 4. the stereopsis dense Stereo Matching method according to claim 1 or claim 2 based on the expansion of two classes, it is characterised in that：Step 5 In, intensity correlation matching uses the covariance of the standardization of left and right match window gray scale as coefficient correlation, according to equation below Calculate,Wherein,U, ν represent two points of coefficient correlation to be calculated on the image of left and right respectively；R (u, ν) represents u, the coefficient correlation that ν at 2 points；M, N represent to calculate width, the height of the left and right match window of coefficient correlation；Represent the gray value of pixel inside the match window of left and right；Represent the gray average of left and right match window.
- 5. the stereopsis dense Stereo Matching method according to claim 3 based on the expansion of two classes, it is characterised in that：In step 5, Intensity correlation matching uses the covariance of the standardization of left and right match window gray scale as coefficient correlation, according to equation below meter Calculate,Wherein,U, ν represent two points of coefficient correlation to be calculated on the image of left and right respectively；R (u, ν) represents u, the coefficient correlation that ν at 2 points；M, N represent to calculate width, the height of the left and right match window of coefficient correlation；Represent the gray value of pixel inside the match window of left and right；Represent the gray average of left and right match window.
- 6. a kind of stereopsis dense Stereo Matching system based on the expansion of two classes, it is characterised in that including with lower module：Characteristic point search module to be matched, for taking a width image as reference images, according to predetermined interval to reference images office Portion carries out notable feature extraction, forms the set of characteristic point to be matched；Seed point extraction module, for of the same name to carrying out rough matching acquisition to the image of reference images and image to be matched composition Seed point, as initial known point, form initial known point set；First kind expansion module, for based on reference images, first to be carried out respectively with each known point in current known point set Class is expanded, and characteristic point to be matched is searched in first kind expansion；Second class expansion module, for based on reference images, to each characteristic point to be matched obtained in first kind expansion module The second class expansion is carried out respectively, and some known points closest around the characteristic point to be matched are searched in the expansion of the second class, It is distance weighted average by all known points progress found, calculate same place of the characteristic point to be matched on image to be matched Initial position, and determine hunting zone according to all known points found,If certain feature point coordinates to be matched is p (x in reference imagesp,yp), all known points found in the expansion of the second class Number is N, searches and obtains the collection of all known points pair and be combined into Q, it is described it is distance weighted it is average be defined as follows,Wherein,p'(xp',yp') represent characteristic point p (x to be matchedp,yp) on image to be matched possibility matching point coordinates, with p'(xp', yp') be same place of the characteristic point to be matched on image to be matched initial position；A known point pair in set Q is represented,Represent collection Close a known point pair in Q, i, j=1,2,3..., N；Represent p (xp,yp) andThe distance between；Gray-scale Matching module, for based on reference images and image to be matched, the spy to be matched according to obtained by the second class expansion module The initial position of same place of the sign point on image to be matched and hunting zone carry out Gray-scale Matching, including by reference images and Image to be matched respectively as image pair left and right image, by centered on the initial position of characteristic point to be matched and its same place point The other left and right image to image pair establishes multistage pyramid image block, and related of gray scale is then carried out on pyramid top layer image Match somebody with somebody, the same place on image to be matched is obtained according to initial position and hunting zone, it is then downward step by step by pyramid series Matching is transmitted, raw video is eventually passed back to, obtains the coordinate and correlation of same place of the characteristic point to be matched on image to be matched Coefficient；Screening module, for each known point in current known point set, matching to be tied according to obtained by Gray-scale Matching module respectively Fruit is counted, including all characteristic points to be matched that first kind expansion is found are carried out in one known point current iteration of statistics In, matching gained coefficient correlation accounts for the ratio of all characteristic points to be matched less than the characteristic point to be matched of predetermined coefficient threshold value, when The ratio then rejects this known point more than pre-set ratio threshold value from current known point set；Result judgement module, for remaining each known point in current known point set, according to obtained by Gray-scale Matching module Matching result, when have certain characteristic point to be matched matching obtained by coefficient correlation be more than or equal to predetermined coefficient threshold value, this is to be matched Characteristic point is added in known point set as newly-increased known point；Judge whether the newly-increased known point sum of current iteration is small In default amount threshold, if otherwise it is added in known point set, and order first kind expansion module works on, if then Power cut-off, current known point set is last dense Stereo Matching result.
- 7. the stereopsis dense Stereo Matching system according to claim 6 based on the expansion of two classes, it is characterised in that：The first kind is swollen Expansion algorithm used in the expansion of swollen and the second class is defined as follows,Wherein,X represents the finite element in finite element set X,B [] represents to increase operation to the eight neighborhood of element；D (X) represents result element set.
- 8. the stereopsis dense Stereo Matching system based on the expansion of two classes according to claim 6 or 7, it is characterised in that：First In class expansion module, when certain known point in current known point set carries out first kind expansion, if the search of the known point Scope has arrived search border, then skips the point without expansion.
- 9. the stereopsis dense Stereo Matching system based on the expansion of two classes according to claim 6 or 7, it is characterised in that：Gray scale In matching module, intensity correlation matching using the covariance of the standardization of left and right match window gray scale as coefficient correlation, according to Equation below calculates,Wherein,U, ν represent two points of coefficient correlation to be calculated on the image of left and right respectively；R (u, ν) represents u, the coefficient correlation that ν at 2 points；M, N represent to calculate width, the height of the left and right match window of coefficient correlation；Represent the gray value of pixel inside the match window of left and right；Represent the gray average of left and right match window.
- 10. the stereopsis dense Stereo Matching system according to claim 8 based on the expansion of two classes, it is characterised in that：Gray scale With in module, intensity correlation matching uses the covariance of the standardization of left and right match window gray scale as coefficient correlation, according to such as Lower formula calculates,Wherein,U, ν represent two points of coefficient correlation to be calculated on the image of left and right respectively；R (u, ν) represents u, the coefficient correlation that ν at 2 points；M, N represent to calculate width, the height of the left and right match window of coefficient correlation；Represent the gray value of pixel inside the match window of left and right；Represent the gray average of left and right match window.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510586875.9A CN105225233B (en) | 2015-09-15 | 2015-09-15 | A kind of stereopsis dense Stereo Matching method and system based on the expansion of two classes |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510586875.9A CN105225233B (en) | 2015-09-15 | 2015-09-15 | A kind of stereopsis dense Stereo Matching method and system based on the expansion of two classes |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN105225233A CN105225233A (en) | 2016-01-06 |
| CN105225233B true CN105225233B (en) | 2018-01-26 |
Family
ID=54994182
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510586875.9A Active CN105225233B (en) | 2015-09-15 | 2015-09-15 | A kind of stereopsis dense Stereo Matching method and system based on the expansion of two classes |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105225233B (en) |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107194334B (en) * | 2017-05-10 | 2019-09-10 | 武汉大学 | Video satellite image dense Stereo Matching method and system based on optical flow estimation |
| CN110060283B (en) * | 2019-04-17 | 2020-10-30 | 武汉大学 | Multi-measure semi-global dense matching method |
| CN110942102B (en) * | 2019-12-03 | 2022-04-01 | 武汉大学 | Probability relaxation epipolar matching method and system |
| CN112161609A (en) * | 2020-09-07 | 2021-01-01 | 武汉大学 | Internal and external integrated control point measurement and automatic thorn turning method |
| CN112233246B (en) * | 2020-09-24 | 2024-04-16 | 中山大学 | Satellite image dense matching method and system based on SRTM constraint |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103115614A (en) * | 2013-01-21 | 2013-05-22 | 武汉大学 | Associated parallel matching method for multi-source multi-track long-strip satellite remote sensing images |
| CN104299228A (en) * | 2014-09-23 | 2015-01-21 | 中国人民解放军信息工程大学 | Remote-sensing image dense matching method based on accurate point location prediction model |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP2335220A2 (en) * | 2008-07-06 | 2011-06-22 | Sergei Startchik | Method for distributed and minimum-support point matching in two or more images of 3d scene taken with video or stereo camera. |
-
2015
- 2015-09-15 CN CN201510586875.9A patent/CN105225233B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103115614A (en) * | 2013-01-21 | 2013-05-22 | 武汉大学 | Associated parallel matching method for multi-source multi-track long-strip satellite remote sensing images |
| CN104299228A (en) * | 2014-09-23 | 2015-01-21 | 中国人民解放军信息工程大学 | Remote-sensing image dense matching method based on accurate point location prediction model |
Also Published As
| Publication number | Publication date |
|---|---|
| CN105225233A (en) | 2016-01-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105225233B (en) | A kind of stereopsis dense Stereo Matching method and system based on the expansion of two classes | |
| CN104484648B (en) | Robot variable visual angle obstacle detection method based on outline identification | |
| CN110310320B (en) | Binocular vision matching cost aggregation optimization method | |
| Costea et al. | Creating roadmaps in aerial images with generative adversarial networks and smoothing-based optimization | |
| CN109934862A (en) | A kind of binocular vision SLAM method that dotted line feature combines | |
| CN104318570A (en) | Self-adaptation camouflage design method based on background | |
| CN106157323B (en) | A kind of insulator division and extracting method of dynamic division threshold value and block search combination | |
| CN108010081A (en) | A kind of RGB-D visual odometry methods based on Census conversion and Local map optimization | |
| CN107833239B (en) | Optimization matching target tracking method based on weighting model constraint | |
| CN105046701B (en) | Multi-scale salient target detection method based on construction graph | |
| CN111480183A (en) | Light field image rendering method and system for generating perspective effect | |
| CN105004337B (en) | Agricultural unmanned plane autonomous navigation method based on matching line segments | |
| CN107560592A (en) | A kind of precision ranging method for optronic tracker linkage target | |
| CN110490913A (en) | Feature based on angle point and the marshalling of single line section describes operator and carries out image matching method | |
| CN101727654A (en) | Method realized by parallel pipeline for performing real-time marking and identification on connected domains of point targets | |
| CN112907573B (en) | Depth completion method based on 3D convolution | |
| CN114882222A (en) | Improved YOLOv5 target detection model construction method and tea tender shoot identification and picking point positioning method | |
| CN116449384A (en) | Radar inertial tight coupling positioning mapping method based on solid-state laser radar | |
| Stucker et al. | ResDepth: Learned residual stereo reconstruction | |
| CN106780309A (en) | A kind of diameter radar image joining method | |
| CN109708627A (en) | A kind of moving platform down space dynamic point target rapid detection method | |
| Hirner et al. | FC-DCNN: A densely connected neural network for stereo estimation | |
| CN107221007A (en) | A kind of unmanned vehicle monocular visual positioning method based on characteristics of image dimensionality reduction | |
| CN114689038A (en) | Fruit detection positioning and orchard map construction method based on machine vision | |
| CN104463896B (en) | Image corner point detection method and system based on kernel similar region distribution characteristics |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |