CN108876861A - A kind of solid matching method of objects outside Earth rover - Google Patents

A kind of solid matching method of objects outside Earth rover Download PDF

Info

Publication number
CN108876861A
CN108876861A CN201810513673.5A CN201810513673A CN108876861A CN 108876861 A CN108876861 A CN 108876861A CN 201810513673 A CN201810513673 A CN 201810513673A CN 108876861 A CN108876861 A CN 108876861A
Authority
CN
China
Prior art keywords
pyramid
tomographic image
point
highest
pixels point
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201810513673.5A
Other languages
Chinese (zh)
Other versions
CN108876861B (en
Inventor
李海超
李莹
陈亮
顾征
李峰
辛蕾
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
China Academy of Space Technology CAST
Original Assignee
China Academy of Space Technology CAST
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by China Academy of Space Technology CAST filed Critical China Academy of Space Technology CAST
Priority to CN201810513673.5A priority Critical patent/CN108876861B/en
Publication of CN108876861A publication Critical patent/CN108876861A/en
Application granted granted Critical
Publication of CN108876861B publication Critical patent/CN108876861B/en
Expired - Fee Related legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
    • G06T7/85—Stereo camera calibration
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/50—Depth or shape recovery
    • G06T7/55—Depth or shape recovery from multiple images
    • G06T7/593—Depth or shape recovery from multiple images from stereo images
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10004—Still image; Photographic image
    • G06T2207/10012—Stereo images
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20016—Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20228—Disparity calculation for image-based rendering
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30248—Vehicle exterior or interior
    • G06T2207/30252—Vehicle exterior; Vicinity of vehicle

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Image Processing (AREA)

Abstract

The present invention relates to a kind of solid matching methods of objects outside Earth rover, belong to stereoscopic vision matching technique.Step of the invention is as follows：The left and right camera of objects outside Earth rover obtains left images；Polar curve correction is carried out to left images, the left images after polar curve correction are constructed with L layers of left and right pyramid respectively；Two-way Stereo matching is carried out to L layers of left and right pyramid of two images, obtains horizontal parallax figure；The disparity search range that L-1 layers of each pixel are determined using L layers of disparity map is carried out two-way Stereo matching to L-1 layers of two images, obtains L-1 layers of horizontal parallax figure；Circulation carries out later, the disparity search range until determining the 1st layer of each pixel using the 2nd layer of disparity map, carries out two-way Stereo matching to the 1st layer of two images, obtains the 1st layer of horizontal parallax figure.

Description

A kind of solid matching method of objects outside Earth rover
Technical field
The present invention relates to a kind of solid matching methods of objects outside Earth rover, and in particular to a kind of objects outside Earth rover Multiresolution quick stereo matching process, belong to computer vision and field of deep space exploration.
Background technique
Objects outside Earth rover detects the important content for having become deep space exploration.Since objects outside Earth has working environment The features such as unknown, unstructured, with a varied topography, working range is big, remote apart from the earth, therefore, it is necessary to emphasis to solve rover A series of problems, such as autonomous environment sensing, path planning.The landform quick three-dimensional reconstruct on objects outside Earth surface is to guarantee rover In the critical issue that objects outside Earth is smoothly advanced.
Objects outside Earth rover mainly passes through laser radar method and stereo vision method realization to the three-dimensional reconstruction of landform at present. Laser radar method can produce high accuracy depth information, and the advantage relative to stereoscopic vision is that calculation amount is small, but exist The disadvantages of equipment power dissipation is big, volume and weight is larger.For example, due to the limitation of the processor processing speed of radiation hardened at that time, Laser stripe instrument is used in the U.S. " Suo Jiena " Marsokhod that in July, 1997 lands carries out detection of obstacles.But according to survey It calculates, 1 kilogram of weight of every increase enters space, needs tens of thousands of dollars of costs of pay this extra.And stereo visual system has that visual field is big, function Consumption, the features such as volume and quality are smaller, at low cost meet contained equipment quality, energy consumption and volume transmitting as small as possible and make With requiring.The progress of raising and stereovision technique especially as processor processing speed, stereoscopic vision are visited in objects outside Earth Survey field is with a wide range of applications.For example, the U.S.'s courage number landed in 2004 and Opportunity Rover Marsokhod just utilize solid Vision carrys out drawing three-dimensional topographic map；The curiosity Marsokhod of the landing Mars of in August, 2012 also uses stereoscopic vision as main Avoidance, path planning, navigator fix technical way；" Jade Hare " of the goddess in the moon No. three of China carryings of in December, 2013 number month Ball vehicle, and use stereovision technique realize the three-dimensional reconstruction to lunar surface circumstances not known, realize oneself based on stereoscopic vision Main avoidance.
Method based on stereoscopic vision is that two width obtained to binocular solid camera or multiple image carry out Stereo matching, is obtained Take the three-dimensional information of ground surface or terrain.Scharstein etc. by solid matching method be broadly divided into four steps (referring to D.Scharstein,R.Szeliski.A taxonomy and evaluation of dense two-frame stereo correspondence algorithms.IJCV,47(1-3):7-42,2002)：(1) calculating of pixel matching cost；(2) The aggregation of matching cost；(3) disparity computation and optimization (sub-pix)；(4) parallax refinement, and according to optimum theory by match party Method is divided into local matching method and global optimization method.Local matching method calculates matching using the match window around pixel Cost finds one the smallest matching cost value by the optimization thought of " the victor is a king (Winner Take All-WTA) " As parallax value.This method is mainly from being considered locally problem, so it is easy to produce error hiding, but this method is calculating speed There is advantage on degree, can adapt to the high occasion of requirement of real-time.And global optimization method is based primarily upon energy minimization philosophy, What is obtained is the anaglyph of global optimum.Usual Global Optimal Problem can pass through markov random file (Markov Random Field, MRF) expression, still, MRF model field is two dimensional image, therefore the energy function overall situation based on MRF is most Optimization is NP problem, although there are figures to cut some near-optimal methods such as algorithm, belief propagation algorithm, its computational efficiency Still lower, the binocular solid of objects outside Earth rover is matched and is not suitable with.And Dynamic Programming is carried out to single scanning line Optimization, dynamic programming algorithm is a kind of algorithm that real-time is relatively high in Global Algorithm.But due to the one of dynamic programming algorithm Characteristic is tieed up, there are fault-layer-phenomenons between different scanning line.Therefore, in order to overcome the above problem, SGM (Semi-global Matching) Dynamic Programming of single scanning line is generalized to multiple directions by algorithm, so that holotopy is approximatively expressed, and Disparity continuity constraint is introduced in Dynamic Programming (referring to Hirschmuller H., Stereo processing by semi- global matching and mutual information.IEEE Transactions and Pattern Analysis and Machine Intelligence,30(2):328-341,2008.).In order to overcome SGM algorithm may in weak texture region The ribbon grain of generation, Facciolo et al. improve SGM algorithm, propose MGM (More Global Matching) algorithm is (referring to Facciolo G., Franchis C., MGM:A Significantly More Global Matching for Stereo Vision.British Machine Vision Conference,September, 2015.)。
Although SGM and MGM algorithm all obtains preferable effect, on the one hand, in the objects outside Earth of circumstances not known Under environment, since we can not predict the disparity search range of scene, if using traditional SGM or MGM algorithm, entire image All pixels all use a bigger disparity search range.However, the setting of disparity search range is excessive, can consume more More calculating time occupies more memory headrooms, while may cause the matching of mistake；If the setting of disparity search range It is too small, then it cannot get correct parallax.Meanwhile the imaging resolution of objects outside Earth rover is higher and higher, can all bring huge Calculation amount.On the other hand, no matter SGM or MGM Processing Algorithm all uses same group of punishment parameter P in entire image1With P2, lead to that there is poor matching precision in parallax discontinuity zone and occlusion area.
To solve the problems, such as above-mentioned both sides, the multiresolution that one aspect of the present invention establishes objects outside Earth rover is quickly stood Body matching strategy, the low resolution matching result that it is generated can be propagated step by step, be searched for constraining the parallax of high-definition picture Rope range, the essence accelerated is the reduction of the search range of parallax, while also reducing the probability of happening of error hiding；Another party Face proposes root to overcome traditional SGM or MGM algorithm to have poor matching precision in parallax discontinuity zone and occlusion area The strategy of punishment parameter is adaptively adjusted according to marginal information.
Summary of the invention
Technology of the invention solves the problems, such as：Overcome the deficiencies of the prior art and provide a kind of standing for objects outside Earth rover The disparity search range of body matching process, this method is small, and calculating speed is fast, and parallax discontinuity zone and occlusion area have preferably Matching precision.
The technical solution of the invention is as follows：
A kind of the step of solid matching method of objects outside Earth rover, this method includes：
S1, the stereo pairs that objects outside Earth scene is obtained using the left and right camera of objects outside Earth rover, left camera are obtained The image definition obtained is left image, and the image definition that right camera obtains is right image；
S2, respectively the left image structure to the left image and right image progress polar curve correction in step S1, after being corrected to polar curve L layers of pyramid are built, referred to as left pyramid constructs L layers of pyramid, referred to as right pyramid to the right image after polar curve correction；
The L be natural number, preferably 2~5；
The pyramid is preferably gaussian pyramid；
S3, to top i.e. L layers of the image of left pyramid constructed in step S2 and top i.e. L layers of right pyramid Image carry out two-way Stereo matching, obtain the disparity map of L layers of left pyramid and the disparity map of L layers of right pyramid；
It is described to top i.e. L layers of the image of left pyramid and the top i.e. L of right pyramid that are constructed in step S2 The method that the image of layer carries out two-way Stereo matching, specific step is as follows：
S31：The SIFT feature in left pyramid highest tomographic image is extracted, referred to as left SIFT feature extracts right golden word SIFT feature in tower highest tomographic image, referred to as right SIFT feature, and left SIFT feature and right SIFT spy to extraction Sign point is matched, and obtains n to match point；N should be in left pyramid highest tomographic image to the i-th pair matching double points in match point Characteristic point coordinate be denoted as (xi,yi), n should be in the spy in right pyramid highest tomographic image to the i-th pair matching double points in match point Sign point coordinate is denoted as (xi',yi'), i=0,1 ..., n-1, n >=3；SIFT (the Scale Invariant Feature Transform, i.e. Scale invariant features transform)；
S32：D is calculated to the i-th pair match point in match point according to n obtained in step S31i=xi'-xi, obtain left gold The corresponding spatial point of ith feature point in word tower highest tomographic image, coordinate are denoted as (xi,yi,di), left pyramid is top N characteristic point in image obtains n spatial point, the n space obtained to n characteristic point in left pyramid highest tomographic image Point carries out space plane fitting, and the disparity search range of all pixels point in left pyramid highest tomographic image is calculated；
D is calculated to the i-th pair match point in match point according to n obtained in step S31i'=xi-xi', obtain right golden word The corresponding spatial point of ith feature point in tower highest tomographic image, coordinate are denoted as (xi',yi',di'), right pyramid is top N characteristic point in image obtains n spatial point, the n space obtained to n characteristic point in right pyramid highest tomographic image Point carries out space plane fitting, and the disparity search range of all pixels point in right pyramid highest tomographic image is calculated；
It is quasi- that the n spatial point obtained to n characteristic point in left pyramid highest tomographic image carries out space plane It closes, being calculated in left pyramid highest tomographic image has the method for the disparity search range of pixel, and step includes：
It is quasi- to carry out space plane to the n spatial point that n characteristic point in left pyramid highest tomographic image obtains by S321 It closes, the equation of obtained space plane Π is expressed as D=aX+bY+c；
Wherein, the coefficient in a, b, c representation space plane equation；(X, Y, D) is the seat of any point on space plane Π Mark, D is parallax；
N spatial point is fitted using least square method, to determine the coefficient in spatial plane equation, i.e.,
According toObtain space plane coefficient a, b, c；
S322, for the coordinate (x of the ith feature point in n characteristic point in left pyramid highest tomographic imagei,yi), The corresponding parallax d of ith feature point is calculated according to the equation D=aX+bY+c of space plane Πi Π, calculate ith feature point Disparity difference di Π-di；For the corresponding n disparity difference of n characteristic point in left pyramid highest tomographic image, n view is counted Maximum value D in differencemaxWith minimum value Dmin；
S323, for any pixel point p (x, y) in left pyramid highest tomographic image, according to the equation D of space plane Π =aX+bY+c calculates the corresponding parallax d of pixel p (x, y)Π(x, y) calculates dmin(x, y)=dΠ(x,y)+DminΔ d, dmax(x, y)=dΠ(x,y)+Dmax+ Δ d, wherein Δ d is reserved parallax surplus, preferably 2~5, determine pixel p (x, y) Disparity search range be [dmin(x,y),dmax(x, y)] in integer, obtain having pixel in left pyramid highest tomographic image Disparity search range；
The side of space plane fitting is carried out to the n spatial point that n characteristic point in right pyramid highest tomographic image obtains Method repeats step S321~S323 according to identical principle, obtains the parallax of all pixels point in right pyramid highest tomographic image Search range.
S33：Census transformation is carried out to all pixels point in left pyramid highest tomographic image, obtains left pyramid highest The Census sequence of all pixels point in tomographic image carries out Census change to all pixels point in right pyramid highest tomographic image It changes, obtains the Census sequence of all pixels point in right pyramid highest tomographic image；According to the obtained top figure of left pyramid As in the Census sequence of all pixels point, in right pyramid highest tomographic image all pixels point Census sequence, in left gold In word tower highest tomographic image within the scope of the disparity search of all pixels point, all pictures in left pyramid highest tomographic image are calculated The matching cost of vegetarian refreshments；According to the Census sequence of all pixels point, right pyramid in obtained left pyramid highest tomographic image The Census sequence of all pixels point in highest tomographic image, the disparity search of all pixels point in right pyramid highest tomographic image In range, the matching cost of all pixels point in right pyramid highest tomographic image is calculated；
Described carries out Census transformation to all pixels point in left pyramid highest tomographic image, obtains left pyramid most The method of the Census sequence of all pixels point is in high-rise image：
S331 establishes Census window centered on any pixel point in left pyramid highest tomographic image, compares foundation Census window in pixel and central point in addition to central point gray value size relation, gray value is less than or is waited It is labeled as 0 in the pixel of central point gray value, 1 is otherwise labeled as, comparison result step-by-step is finally connected as 0/1 binary code Stream, referred to as the Census sequence of current pixel point；
S332 repeats step S331, until all pixels point in left pyramid highest tomographic image is all disposed, obtains The Census sequence of all pixels point in left pyramid highest tomographic image；
Census transformation is carried out to all pixels point in right pyramid highest tomographic image, obtains the top figure of right pyramid The method of the Census sequence of all pixels point repeats step S331~S332 according to same principle as in, obtains right pyramid most The Census sequence of all pixels point in high-rise image.
The Census sequence of all pixels point, right pyramid be most in the left pyramid highest tomographic image that the described basis obtains The Census sequence of all pixels point in high-rise image, the disparity search model of all pixels point in left pyramid highest tomographic image In enclosing, the method that the matching cost of all pixels point in left pyramid highest tomographic image is calculated is：
S333 calculates the picture in the pixel p (x, y) and right pyramid highest tomographic image in left pyramid highest tomographic image The Hamming distance of the Census sequence of vegetarian refreshments q (x-d, y), d are parallax, d ∈ [dmin(x,y),dmax(x, y)], and d is integer, Obtain matching cost C (p, d) of the pixel p (x, y) when parallax is d in left pyramid highest tomographic image；
S334 repeats step S333, obtains the pixel p (x, y) in left pyramid highest tomographic image in different parallax d When matching cost；
S335 repeats step S333~S334, until all pixels point in left pyramid highest tomographic image has all been handled Finish, obtains the matching cost of all pixels point in left pyramid highest tomographic image；
Left pyramid highest is obtained according to the disparity search range computation of all pixels point in left pyramid highest tomographic image The method of the matching cost of all pixels point repeats step S333~S335 according to same principle in tomographic image, obtains right pyramid The matching cost of all pixels point in highest tomographic image.
S34：The matching cost of all pixels point is to left golden word in the left pyramid highest tomographic image obtained according to step S33 All pixels point carries out matching cost polymerization in tower highest tomographic image, obtains the matching cost polymerization of left pyramid highest tomographic image Value, obtains the initial view of left pyramid highest tomographic image according to the matching cost polymerizing value of obtained left pyramid highest tomographic image Difference figure；The matching cost of all pixels point is to right pyramid highest in the right pyramid highest tomographic image obtained according to step S33 All pixels point carries out matching cost polymerization in tomographic image, obtains the matching cost polymerizing value of right pyramid highest tomographic image, root The initial parallax figure of right pyramid highest tomographic image is obtained according to the matching cost polymerizing value of obtained right pyramid highest tomographic image；
The matching cost of all pixels point is to left golden word in the left pyramid highest tomographic image obtained according to step S33 All pixels point carries out matching cost polymerization in tower highest tomographic image, matching cost polymerizing value is obtained, according to obtained matching generation The method that valence polymerizing value obtains the initial parallax figure of left pyramid highest tomographic image is：
S341 carries out the detection of Canny edge feature to left pyramid highest tomographic image, obtains several edges；By edge On point be known as marginal point, for any edge point, according to the disparity search range of pixel corresponding with the marginal point [dmin(x,y),dmax(x, y)], calculating difference dmax(x,y)-dmin(x, y), referred to as the parallax span of the marginal point calculate every The average value of the parallax span of all marginal points on edge will be gone if average value is less than the threshold value of setting when preceding article edge Fall, if average value retains not less than the threshold value of setting and works as preceding article edge；The top image detection of left pyramid is obtained After several edge processings, qualified edge in left pyramid highest tomographic image is obtained；The threshold value of the setting is preferred For the real number in 2~8；
S342, according to the matching cost of all pixels point in left pyramid highest tomographic image to left pyramid highest tomographic image Middle all pixels point carries out matching cost polymerization, obtains matching cost polymerizing value；
Calculate the matching cost polymerizing value S (p, d) of pixel p (x, y) when parallax is d：
In formula, r indicates the direction of propagation of pixel p (x, y)；Lr(p, d) indicates pixel p (x, y) along the matching of direction r Cost；
In formula, R indicates the m direction derived from by direction of propagation r, R ∈ { r1,…,rm, m >=1, r1It is identical as the direction r, i.e., r1=r；P-R indicates previous pixel of the pixel p on the R of direction；Lr(p-R, d) indicate pixel p-R when parallax be d when Matching cost, Lr(p-R, d+1) indicates matching cost of the pixel p-R when parallax is d+1, Lr(p-R, d-1) indicates pixel p- Matching cost of the R when parallax is d-1；D' is the parallax of pixel p-R, the disparity search range d' ∈ [d of pixel p-Rmin(p- R),dmax(p-R)],Indicate pixel p-R in disparity search range [dmin(p-R),dmax(p-R)] minimum in Route matching cost；P1 R、P2 RIndicate punishment parameter, P2 R>P1 R；
The direction of propagation r is 8 directions,
R ∈ { (- 1,0), (1,0), (0,1), (0, -1), (- 1, -1), (1, -1), (1,1), (- 1,1) },
Or 4 directions,
r∈{(-1,0),(1,0),(0,1),(0,-1)}；
The m is 4,2 or 1；
When m is 4, direction of propagation r is 8, then it is represented sequentially as by the R that 8 direction of propagation r derive from：
R∈{(-1,0),(0,-1),(-1,-1),(1,-1)}、R∈{(1,0),(0,1),(1,1),(-1,1)}、
R∈{(0,1),(-1,0),(-1,1),(-1,-1)}、R∈{(0,-1),(1,0),(1,-1),(1,1)}
R∈{(-1,-1),(1,-1),(0,-1),(1,0)}、R∈{(1,-1),(1,1),(1,0),(0,1)}、
R∈{(1,1),(-1,1),(0,1),(-1,0)}、R∈{(-1,1),(-1,-1),(-1,0),(0,-1)}
When m is 4, direction of propagation r is 4, then it is represented sequentially as by the R that 4 direction of propagation r derive from：
R∈{(-1,0),(0,-1),(-1,-1),(1,-1)}、R∈{(1,0),(0,1),(1,1),(-1,1)}、
R∈{(0,1),(-1,0),(-1,1),(-1,-1)}、R∈{(0,-1),(1,0),(1,-1),(1,1)}
When m is 2, direction of propagation r is 8, then it is represented sequentially as by the R that 8 directions of propagation are derived from：
R∈{(-1,0),(0,-1)}、R∈{(1,0),(0,1)}、R∈{(0,1),(-1,0)}、R∈{(0,-1),(1, 0)}、
R∈{(-1,-1),(1,-1)}、R∈{(1,-1),(1,1)}、R∈{(1,1),(-1,1)}、R∈{(-1,1),(- 1,-1)}
When m is 2, direction of propagation r is 4, then it is represented sequentially as by the R that 4 directions of propagation are derived from：R∈{(-1,0), (0,-1)},R∈{(1,0),(0,1)},R∈{(0,1),(-1,0)},R∈{(0,-1),(1,0)}；
When m is 1, direction of propagation r is 8, then it is represented sequentially as by the R that 8 directions of propagation are derived from：
R∈{(-1,0)}、R∈{(1,0)}、R∈{(0,1)}、R∈{(0,-1)}、
R∈{(-1,-1)}、R∈{(1,-1)}、R∈{(1,1)}、R∈{(1,1)}
When m is 1, direction of propagation r is 4, then it is represented sequentially as by the R that 4 directions of propagation are derived from：R∈{(-1,0)},R ∈{(1,0)},R∈{(0,1)},R∈{(0,-1)}；
If current pixel point p (x, y) is the marginal point in left pyramid highest tomographic image on qualified edge, The corresponding punishment parameter of p (x, y) is：
If current pixel point p (x, y) is not the marginal point in left pyramid highest tomographic image on qualified edge, Then the corresponding punishment parameter of p (x, y) is：
Wherein, P1Preferably 8, P2Preferably 32；
Other pixels in left pyramid highest tomographic image are handled according to step S342, obtain the top figure of left pyramid The matching cost polymerizing value of all pixels point as in；
S343 calculates the initial parallax figure of left pyramid highest tomographic image：It is tactful according to the victor is a king, it calculates point p (x, y) The parallax value at place：
Other pixels in left pyramid highest tomographic image are handled according to step S343, obtain the top figure of left pyramid The parallax value of all pixels point, forms the initial parallax figure of left pyramid highest tomographic image as in；
In the right pyramid highest tomographic image obtained according to step S33 the matching cost of all pixels point to right pyramid most All pixels point carries out matching cost polymerization in high-rise image, obtains matching cost polymerizing value, poly- according to obtained matching cost The method that conjunction is worth to obtain right initial parallax figure repeats step S341~S343 according to identical same principle, obtains right pyramid most The initial parallax figure of high-rise image.
S35：Sub-pix disparity computation is carried out to the initial parallax figure of the obtained left pyramid highest tomographic image of step S34, Obtain the sub-pix disparity map of left pyramid highest tomographic image；To the initial of the obtained right pyramid highest tomographic image of step S34 Disparity map carries out sub-pix disparity computation, obtains the sub-pix disparity map of right pyramid highest tomographic image；Then to an obtained left side The sub-pix disparity map of pyramid highest tomographic image is consistent by left and right with the sub-pix disparity map of right pyramid highest tomographic image Property detection processing, obtains the disparity map of left pyramid highest tomographic image and the disparity map of right pyramid highest tomographic image.
The sub-pix disparity computation uses conic fitting method；
The method of the left and right consistency detection processing：
The sub-pix disparity map for setting left pyramid highest tomographic image is denoted as Dlr(x, y), right pyramid highest tomographic image Sub-pix disparity map is denoted as Drl(x, y), for any pixel point in the sub-pix disparity map of left pyramid highest tomographic image, such as Fruit meets | Dlr(x,y)-Drl(x+Dlr(x,y),y)|<T, T 1, then the pixel is left and right consistency point, otherwise the pixel For left and right inconsistency point.
S4, determine in left pyramid L-1 tomographic image own using L layers of left pyramid of the disparity map that step S3 is obtained The disparity search range of pixel；L-1 layers of right pyramid is determined using L layers of right pyramid of the disparity map that step S3 is obtained The disparity search range of all pixels point in image；According to the view of all pixels point in determining left pyramid L-1 tomographic image Poor search range carries out Stereo matching from left to right, root to left pyramid L-1 tomographic image and right pyramid L-1 tomographic image According to the disparity search range of all pixels point in determining right pyramid L-1 tomographic image to left pyramid L-1 tomographic image and Right pyramid L-1 tomographic image carries out Stereo matching from right to left, obtains the disparity map and right golden word of L-1 layers of left pyramid The disparity map that L-1 layers of tower；
Described is determined in left pyramid L-1 tomographic image using L layers of left pyramid of the disparity map that step S3 is obtained The method of the disparity search range of all pixels point is：
S41：To any pixel point p in left pyramid L-1 tomographic imageL-1(xL-1,yL-1), it calculatesL layers of left pyramid of the disparity map obtained according to step S3 is by left pyramid L Pixel p in tomographic imageL(xL,yL) corresponding parallax is denoted as dL(xL,yL), with pL(xL,yL) centered on establish size be 3 × 3 The window of pixel；
S42：If all pixels in the window are all left and right consistency points, all pixels pair in the window are calculated The parallax maximum value d answeredL maxWith parallax minimum value dL min：
dL max=max { dL(xL+i,yL+ j) | i=-1,0,1, j=-1,0,1 }
dL min=min { dL(xL+i,yL+ j) | i=-1,0,1, j=-1,0,1 }
By the pixel p in left pyramid L-1 tomographic imageL-1(xL-1,yL-1) disparity search range be set as [2dL min-δ,2dL max+ δ] in integer, δ is surplus, preferably 1~3；
S43：If there are left and right inconsistency points for the pixel in the window, in yLRow, with pL(xL,yL) it is starting point, It is searched for the left until obtaining left and right consistency point pl(xL-xl,yL), it searches for the right until obtaining left and right consistency point pr(xL+xr, yL)；In yL- 1 row, with pL(xL,yL- 1) it is starting point, is searched for the left until obtaining left and right consistency point pul(xL-xul,yL- 1), It searches for the right until obtaining left and right consistency point pur(xL+xur,yL-1)；In yL+ 1 row, with pd(xL,yLIt+1) is starting point, to the left Search is until obtaining left and right consistency point pdl(xL-xdl,yL+ 1) it, searches for the right until obtaining left and right consistency point pdr(xL+xdr, yL+1)；
L layers of left pyramid of the disparity map obtained according to step S3 obtains above-mentioned point pul,pur,pl,pr,pdl,pdrInstitute Maximum value d' in corresponding parallaxLmaxWith minimum value d'Lmin, by the pixel p in left pyramid L-1 tomographic imageL-1(xL-1, yL-1) disparity search range be set asIn integer；
S44：Step S41~S43 is repeated, until all pixels point in left pyramid L-1 tomographic image is all disposed, Obtain the disparity search range of all pixels point in left pyramid L-1 tomographic image；
All pictures in right pyramid L-1 tomographic image are determined using L layers of right pyramid of the disparity map that step S3 is obtained The method of the disparity search range of vegetarian refreshments repeats step S41~S44 according to same principle, obtains right pyramid L-1 tomographic image The disparity search range of middle all pixels point.
The disparity search range according to all pixels point in determining left pyramid L-1 tomographic image is to left golden word Tower L-1 tomographic image and right pyramid L-1 tomographic image carry out Stereo matching from left to right, according to determining right pyramid the The disparity search range of all pixels point is to left pyramid L-1 tomographic image and right pyramid L-1 tomographic image in L-1 tomographic image Stereo matching from right to left is carried out, the left disparity map of L-1 layers of left pyramid and the right parallax of L-1 layers of right pyramid are obtained The method of figure, specific step is as follows：
S45：Census transformation is carried out to all pixels point in left pyramid L-1 tomographic image, obtains left pyramid the The Census sequence of all pixels point in L-1 tomographic image carries out all pixels point in right pyramid L-1 tomographic image Census transformation, obtains the Census sequence of all pixels point in right pyramid L-1 tomographic image；According to obtained left pyramid The Census sequence of all pixels point in L-1 tomographic image, in right pyramid L-1 tomographic image all pixels point Census sequence Column, in left pyramid L-1 tomographic image within the scope of the disparity search of all pixels point, are calculated L-1 layers of left pyramid The matching cost of all pixels point in image；According to the Census of all pixels point in obtained left pyramid L-1 tomographic image The Census sequence of all pixels point in sequence, right pyramid L-1 tomographic image owns in right pyramid L-1 tomographic image Within the scope of the disparity search of pixel, the matching cost of all pixels point in right pyramid L-1 tomographic image is calculated；
S46：The matching cost of all pixels point is to left gold in the left pyramid L-1 tomographic image obtained according to step S45 All pixels point carries out matching cost polymerization in word tower L-1 tomographic image, obtains the matching cost of left pyramid L-1 tomographic image Polymerizing value obtains left pyramid L-1 tomographic image according to the matching cost polymerizing value of obtained left pyramid L-1 tomographic image Initial parallax figure；The matching cost of all pixels point is to right golden word in the right pyramid L-1 tomographic image obtained according to step S45 All pixels point carries out matching cost polymerization in tower L-1 tomographic image, and the matching cost for obtaining right pyramid L-1 tomographic image is poly- Conjunction value obtains the first of right pyramid L-1 tomographic image according to the matching cost polymerizing value of obtained right pyramid L-1 tomographic image Beginning disparity map；
S47：Sub-pix disparity computation is carried out to the initial parallax figure of the obtained left pyramid L-1 tomographic image of step S46, Obtain the sub-pix disparity map of left pyramid L-1 tomographic image；To the first of the obtained right pyramid L-1 tomographic image of step S46 Beginning disparity map carries out sub-pix disparity computation, obtains the sub-pix disparity map of right pyramid L-1 tomographic image；Then to obtaining The sub-pix disparity map of left pyramid L-1 tomographic image and the sub-pix disparity map of right pyramid L-1 tomographic image are by left and right Consistency detection processing, obtains the disparity map of left pyramid L-1 tomographic image and the disparity map of right pyramid L-1 tomographic image.
S5, institute in left pyramid L-2 tomographic image is determined using L-1 layers of left pyramid of the disparity map that step S4 is obtained There is the disparity search range of pixel；The disparity map of L-1 layers of the right pyramid obtained using step S4 determines right pyramid The disparity search range of all pixels point in L-2 tomographic image；According to all pixels point in determining left pyramid L-2 tomographic image Disparity search range from left to right three-dimensional is carried out to left pyramid L-2 tomographic image and right pyramid L-2 tomographic image Match, L-2 layers of left pyramid are schemed according to the disparity search range of all pixels point in determining right pyramid L-2 tomographic image Picture and right pyramid L-2 tomographic image carry out Stereo matching from right to left, obtain disparity map and the right side of L-2 layers of left pyramid The disparity map that L-2 layers of pyramid；
……
S6, left the 1st tomographic image of the pyramid (left figure i.e. after polar curve correction is determined using the 2nd layer of left pyramid of disparity map Picture) in all pixels point disparity search range；Right the 1st tomographic image of pyramid is determined using the 2nd layer of disparity map of right pyramid The disparity search range of all pixels point in (right image i.e. after polar curve correction)；According to determining the 1st tomographic image of left pyramid The disparity search range of middle all pixels point carries out from left to right left the 1st tomographic image of pyramid and the 1st tomographic image of right pyramid Stereo matching, according to the disparity search range of all pixels point in determining the 1st tomographic image of right pyramid to left pyramid the 1st Tomographic image and the 1st tomographic image of right pyramid carry out Stereo matching from right to left, obtain the 1st layer of left pyramid of disparity map and the right side The disparity map that the 1st layer of pyramid.
The present invention has the beneficial effect that compared with prior art：
(1) left images that this characteristic of the invention that is inversely proportional at a distance from observer according to parallax, i.e. rover obtain The middle point parallax value close apart from rover is larger, relatively small apart from remote point parallax value, proposes the side using space plane fitting Method estimating disparity search range greatly reduces disparity search range, and does not need that disparity search range is manually specified, and is suitble to The unknown objects outside Earth application of environment.
(2) present invention is proposed by slightly to smart pyramid Stereo matching strategy, according to the parallax of the high tomographic image of pyramid Value determines the disparity search range of each pixel in a low tomographic image, greatly reduces parallax and searches plain range, matching primitives time, Faster processing speed can be obtained to high-resolution rover image.
(3) present invention for overcome conventional method parallax discontinuity zone, occlusion area have poor matching precision this One problem, picture edge characteristic is extracted in proposition, and selects penalty coefficient for the region adaptivity for including edge, as a result table It is bright to obtain higher matching precision in parallax discontinuity zone and occlusion area.
Detailed description of the invention
Fig. 1 is the flow chart of the solid matching method of objects outside Earth rover of the present invention；
Fig. 2A is the left image after polar curve corrects that the left camera of lunar rover of the present invention obtains, image size is 947 × 696 pixels；
Fig. 2 B is the right image after polar curve corrects that the right camera of lunar rover of the present invention obtains, image size is 947 × 696 pixels；
Fig. 3 is the flow chart that pyramid highest tomographic image in left and right of the present invention obtains left disparity map and right disparity map；
Fig. 4 A is the SIFT matching result of embodiment of the present invention or so pyramid highest tomographic image；
Fig. 4 B is that the embodiment of the present invention is quasi- to the spatial point of SIFT point and parallax composition in left pyramid highest tomographic image Close obtained space plane；
Fig. 4 C is the Canny edge detection graph of left pyramid highest tomographic image；
Fig. 4 D is the corresponding left disparity map of the left pyramid highest tomographic image of the embodiment of the present invention；
Fig. 4 E is the disparity map after the disparity map that Fig. 4 D is indicated is superimposed with edge；
Fig. 5 A is the signal for the disparity search range that the present invention determines a low tomographic image according to the disparity map of a high tomographic image Figure；
Fig. 5 B is the present invention when there are pixel disparity search range schematic diagrames when left and right inconsistency point in 3 × 3 regions；
Fig. 6 A is the edge graph that left the 2nd tomographic image of pyramid of the embodiment of the present invention extracts, and image size is 473 × 348 pictures Element；
Fig. 6 B is the corresponding disparity map of left the 2nd tomographic image of pyramid of the embodiment of the present invention；
Fig. 6 C is the disparity map after the disparity map that Fig. 6 B is indicated is superimposed with edge；
Fig. 7 A is the edge graph that left the 1st tomographic image of pyramid of the embodiment of the present invention extracts；
Fig. 7 B is the corresponding disparity map of left the 1st tomographic image of pyramid of the embodiment of the present invention；
Fig. 7 C is the parallax after the disparity map that Fig. 7 B is indicated is superimposed with edge；
Fig. 7 D is the disparity map after the disparity map that traditional MGM method obtains is superimposed with edge.
Specific embodiment
Embodiment in order to provide objects outside Earth rover images match, the embodiment of the invention provides days outside a kind of ground The solid matching method of body rover, below in conjunction with Figure of description, embodiments of the present invention will be described, it should be understood that Embodiment described herein is merely to illustrate and explain the present invention, and is not intended to restrict the invention.
The present invention provides a kind of solid matching method of objects outside Earth rover, as shown in Figure 1, including：
S1, the stereo pairs that objects outside Earth scene is obtained using the left and right camera of objects outside Earth rover, left camera are obtained The image definition obtained is left image, and the image definition that right camera obtains is right image.
S2, respectively the left image structure to the left image and right image progress polar curve correction in step S1, after being corrected to polar curve L layers of pyramid are built, referred to as left pyramid constructs L layers of pyramid, referred to as right pyramid to the right image after polar curve correction, wherein L is preferably 2~5, and the pyramid in the present invention is preferably gaussian pyramid.
Fig. 2A and Fig. 2 B is the left image and right image after polar curve corrects that lunar rover or so camera obtains, and image is big Small is all 947 × 696 pixels, and the embodiment of the present invention is by the left gaussian pyramid for constructing 3 layers respectively to image to this and right Gauss The image size of pyramid, then top (i.e. the 3rd layer) is all 236 × 174 pixels.
S3, i.e. L layers of image top to left pyramid and top i.e. L layers of the image of right pyramid carry out two-way Stereo matching obtains the disparity map of L layers of left pyramid and the disparity map of L layers of right pyramid, main flow as shown in figure 3, The present embodiment provides that specific step is as follows：
S31：SIFT (Scale Invariant Feature Transform, the i.e. Scale invariant features transform) present invention The extracting and matching feature points of top two images are realized using SIFT algorithm, are mainly included scale space extremum extracting, are closed Key point location, key point local direction calculate, description generates, SIFT is matched, RANSAC eliminates several steps such as error hiding.
The embodiment of the present invention extracts SIFT feature (left SIFT feature) and right gold in left pyramid highest tomographic image SIFT feature (right SIFT feature) in word tower highest tomographic image, and to the left SIFT feature and right SIFT feature of extraction Point is matched, and obtains n to match point, feature of the n to the i-th pair match point in match point in left pyramid highest tomographic image Its coordinate of point is denoted as (xi,yi), n is to its seat of characteristic point of the i-th pair match point in match point in right pyramid highest tomographic image Labeled as (xi',yi'), i=0,1 ..., n-1, n >=3.Fig. 4 A is that the SIFT feature of left and right pyramid highest tomographic image is extracted As a result and matching result, including 260 pairs of match points.
S32：After above-mentioned SIFT feature matching, conventional way is that the view of entire image is determined according to SIFT matching result Poor search range.Under binocular solid camera imaging device, parallax is inversely proportional at a distance from observer.Objects outside Earth is maked an inspection tour Front is imaged in device, and the two images of acquisition have following features：Apart from the parallax of the closer scene point of rover in the picture It is worth larger, apart from the farther away scene point of rover, parallax value is relatively small in the picture.Therefore, if conventionally, will be whole Each pixel in width image specifies identical disparity search range, and it is huge to be bound to cause calculation amount, however most meter Calculation is all redundancy, it is also possible to bring error hiding.
D is calculated to the i-th pair match point in match point according to n obtained in step S31i=xi'-xi, obtain left pyramid The corresponding spatial point of ith feature point in highest tomographic image, coordinate are denoted as (xi,yi,di), left pyramid highest tomographic image In n characteristic point obtain n spatial point.Then, n sky n characteristic point in left pyramid highest tomographic image obtained Between point carry out space plane fitting, the disparity search range of left pyramid highest tomographic image all pixels point is calculated, specifically Step includes：
It is quasi- to carry out space plane to the n spatial point that n characteristic point in left pyramid highest tomographic image obtains by S321 It closes, the equation for obtaining space plane Π is expressed as D=aX+bY+c.
Wherein, the coefficient in a, b, c representation space plane equation；(X, Y, D) is the seat of any point on space plane Π Mark, D are coordinate (X, Y) corresponding parallax.
N spatial point is fitted using least square method, to determine the coefficient in spatial plane equation, i.e.,
Make S minimum, should meet：Space plane coefficient can be obtained a、b、c。
Fig. 4 B is that 260 SIFT features with 260 pairs of match points in left pyramid highest tomographic image are 260 corresponding The space plane that spatial point is fitted, floor coefficient a=-0.0052, b=-0.226, c=30.5.
S322, for the coordinate (x of the ith feature point in n characteristic point in left pyramid highest tomographic imagei,yi), The corresponding parallax d of ith feature point is calculated according to the equation D=aX+bY+c of space plane Πi Π, calculate ith feature point Disparity difference is di Π-di；For the corresponding n disparity difference of n characteristic point in left pyramid highest tomographic image, n are counted Maximum value D in disparity differencemaxWith minimum Dmin。
S323, for any pixel point p (x, y) in left pyramid highest tomographic image, according to the equation D of space plane Π =aX+bY+c calculates the parallax d of pixel p (x, y)Π(x, y) calculates dmin(x, y)=dΠ(x,y)+DminΔ d, dmax(x, Y)=dΠ(x,y)+Dmax+ Δ d, wherein Δ d is reserved parallax surplus, preferably 2~5, determine the parallax of pixel p (x, y) Search range is [dmin(x,y),dmax(x, y)] in integer, obtain the parallax for having pixel in left pyramid highest tomographic image Search range.
The side of space plane fitting is carried out to the n spatial point that n characteristic point in right pyramid highest tomographic image obtains Method repeats step S321~S323 according to identical principle, obtains the parallax of all pixels point in right pyramid highest tomographic image Search range.
D is calculated to the i-th pair match point in match point according to n obtained in step S31i'=xi-xi', obtain right golden word The corresponding spatial point of ith feature point in tower highest tomographic image, coordinate are denoted as (xi',yi',di'), right pyramid is top N characteristic point in image obtains n spatial point, clicks through to the n space that n point in right pyramid highest tomographic image obtains The fitting of row space plane, is calculated the disparity search range of right pyramid highest tomographic image all pixels point.
S33：Census transformation is carried out to all pixels point in left pyramid highest tomographic image, obtains left pyramid highest The Census sequence of all pixels point in tomographic image carries out Census change to all pixels point in right pyramid highest tomographic image It changes, obtains the Census sequence of all pixels point in right pyramid highest tomographic image；According to the obtained top figure of left pyramid As in the Census sequence of all pixels point, in right pyramid highest tomographic image all pixels point Census sequence, in left gold In word tower highest tomographic image within the scope of the disparity search of all pixels point, all pictures in left pyramid highest tomographic image are calculated The matching cost of vegetarian refreshments；According to the Census sequence of all pixels point, right pyramid in obtained left pyramid highest tomographic image The Census sequence of all pixels point in highest tomographic image, the disparity search of all pixels point in right pyramid highest tomographic image In range, the matching cost of all pixels point in right pyramid highest tomographic image is calculated.
Census transformation is carried out to all pixels point in left pyramid highest tomographic image, obtains the top figure of left pyramid The method of the Census sequence of all pixels point is as in：
S331, establishes Census window centered on any pixel point in left pyramid highest tomographic image, window it is big Small preferably 3 × 3 pixels or 5 × 5 pixels compare pixel in the Census window of foundation in addition to central point and central point The pixel that gray value is less than or equal to central point gray value is labeled as 0, is otherwise labeled as 1, most by the size relation of gray value Comparison result step-by-step is connected as 0/1 binary code stream, referred to as the Census sequence of current pixel point afterwards.
S332 repeats step S331, until all pixels point in left pyramid highest tomographic image is all disposed, obtains The Census sequence of all pixels point in left pyramid highest tomographic image.
Census transformation is carried out to all pixels point in right pyramid highest tomographic image, obtains the top figure of right pyramid The method of the Census sequence of all pixels point repeats step S331~S332 according to same principle as in, obtains right pyramid most The Census sequence of all pixels point in high-rise image.
According to the Census sequence of all pixels point, the top figure of right pyramid in obtained left pyramid highest tomographic image The Census sequence of all pixels point as in, in left pyramid highest tomographic image within the scope of the disparity search of all pixels point, The method that the matching cost of all pixels point in left pyramid highest tomographic image is calculated is：
S333 calculates the picture in the pixel p (x, y) and right pyramid highest tomographic image in left pyramid highest tomographic image The Hamming distance of the Census sequence of vegetarian refreshments q (x-d, y), d are parallax, d ∈ [dmin(x,y),dmax(x, y)], and d is integer, Obtain matching cost C (p, d) of the pixel p (x, y) when parallax is d in left pyramid highest tomographic image.
S334 repeats step S333, obtains the pixel p (x, y) in left pyramid highest tomographic image in different parallax d When matching cost.
S335 repeats step S333~S334, until all pixels point in left pyramid highest tomographic image has all been handled Finish, obtains the matching cost of all pixels point in left pyramid highest tomographic image；
Left pyramid highest is obtained according to the disparity search range computation of all pixels point in left pyramid highest tomographic image The method of the matching cost of all pixels point repeats step S333~S335 according to same principle in tomographic image, obtains right pyramid The matching cost of all pixels point in highest tomographic image.
S34：The matching cost of all pixels point is to left golden word in the left pyramid highest tomographic image obtained according to step S33 All pixels point carries out matching cost polymerization in tower highest tomographic image, obtains the matching cost polymerization of left pyramid highest tomographic image Value, obtains the initial view of left pyramid highest tomographic image according to the matching cost polymerizing value of obtained left pyramid highest tomographic image Difference figure.
Based on the matching cost of calculating, building energy function is as follows：
In above-mentioned energy function, p is the pixel in image, NpFor the neighborhood of pixel p, P1、P2For in pixel p neighborhood The penalty coefficient and P of parallax variation2>P1>0, P1Punish lesser parallax variation, P2Punish the variation of biggish parallax, D be to Dense disparity map with image, if the parameter in T [] function is if true, return 1, otherwise returns to 0.
For solve the problems, such as E (D) global minimization this, global issue is converted multiple sides by SGM algorithm and MGM algorithm To Dynamic Programming optimize calculative strategy, they all show outstanding effect.But the two methods to parallax mutation at There is no considerations, have all used the punishment parameter of steady state value in all directions of all pixels point, have caused both algorithms in parallax The disparity computation for being mutated near zone is incorrect.Generally, due to which objects outside Earth often has a large amount of rock, parallax is usually deposited Acutely it is being mutated, and the mutated site of parallax occurs in edge in disparity map.Therefore, the embodiment of the present invention first extracts left and right figure The marginal information of picture；Then, in the calculating process of matching cost polymerization, changing marginal position greatly in parallax, we are used not Same punishment parameter.
S341, since Canny edge detection operator has the edge detection of low error rate, oplimal Location, appointing in image Meaning edge should only be labeled the advantages that primary and picture noise should not generate pseudo-edge, and therefore, the present invention uses Canny first Edge detection operator detects the edge of left images, mainly includes with Gaussian filter smoothed image, limited with single order local derviation Difference come calculate gradient amplitude and direction, to gradient magnitude application non-maxima suppression, with dual threashold value-based algorithm detect and connect Edge and etc..
The detection of Canny edge feature is carried out to left pyramid highest tomographic image, obtains several edges；By the point on edge Referred to as marginal point, for any edge point, according to the disparity search range [d of pixel corresponding with the marginal pointmin(x, y),dmax(x, y)], calculating difference dmax(x,y)-dmin(x, y), referred to as the parallax span of the marginal point are calculated and are worked as preceding article edge The average value of the parallax span of upper all marginal points will remove, such as if average value is less than the threshold value of setting when preceding article edge Fruit average value then retains not less than the threshold value of setting and works as preceding article edge；Several sides that left pyramid highest tomographic image is obtained After edge processing, qualified edge in left pyramid highest tomographic image is obtained.The threshold value of setting is preferably the reality in 2~8 Number.
Fig. 4 C is the Canny edge detection graph of left pyramid highest tomographic image, it can be seen that rock and lunar soil are handed on the moon Parallax occurs at boundary and is mutated violent place, corresponding marginal position in the picture.It, can be with when the mean value calculation of parallax span According to the pyramidal layer where image, different threshold values is set, in the embodiment of the present invention, the threshold value set in the 3rd tomographic image is taken Value is 2.5.
S342, according to the matching cost of all pixels point in left pyramid highest tomographic image to left pyramid highest tomographic image Middle all pixels point carries out matching cost polymerization, obtains matching cost polymerizing value.
Due to will appear in Dynamic Programming in one direction calculating process to the very strong correlation constraint in this direction, in reality It is easy to generate striped in existing, therefore the sum of matching cost on multiple directions path optimizing of the present invention calculates matching cost Polymerization.Calculate the matching cost polymerizing value S (p, d) of pixel p (x, y) when parallax is d：
In formula, r indicates the direction of propagation of pixel p (x, y), when direction of propagation r takes 8 directions, r ∈ (- 1,0), (1, 0), (0,1), (0, -1), (- 1, -1), (1, -1), (1,1), (- 1,1) } or direction of propagation r when taking 4 directions, r ∈ (- 1, 0),(1,0),(0,1),(0,-1)}。
Lr(p, d) indicates pixel p (x, y) along the matching cost of direction r；
In formula, R indicates the m direction derived from by direction of propagation r, R ∈ { r1,…,rm, m >=1, r1It is identical as the direction r, i.e., r1=r, wherein m of the present invention can be chosen for 4,2 or 1.
When m is 4, direction of propagation r is 8, then it is represented sequentially as by the R that 8 direction of propagation r derive from：
R∈{(-1,0),(0,-1),(-1,-1),(1,-1)}、R∈{(1,0),(0,1),(1,1),(-1,1)}、
R∈{(0,1),(-1,0),(-1,1),(-1,-1)}、R∈{(0,-1),(1,0),(1,-1),(1,1)}
R∈{(-1,-1),(1,-1),(0,-1),(1,0)}、R∈{(1,-1),(1,1),(1,0),(0,1)}、
R∈{(1,1),(-1,1),(0,1),(-1,0)}、R∈{(-1,1),(-1,-1),(-1,0),(0,-1)}
When m is 4, direction of propagation r is 4, then it is represented sequentially as by the R that 4 direction of propagation r derive from：
R∈{(-1,0),(0,-1),(-1,-1),(1,-1)}、R∈{(1,0),(0,1),(1,1),(-1,1)}、
R∈{(0,1),(-1,0),(-1,1),(-1,-1)}、R∈{(0,-1),(1,0),(1,-1),(1,1)}
When m is 2, direction of propagation r is 8, then it is represented sequentially as by the R that 8 directions of propagation are derived from：
R∈{(-1,0),(0,-1)}、R∈{(1,0),(0,1)}、R∈{(0,1),(-1,0)}、R∈{(0,-1),(1, 0)}、
R∈{(-1,-1),(1,-1)}、R∈{(1,-1),(1,1)}、R∈{(1,1),(-1,1)}、R∈{(-1,1),(- 1,-1)}
When m is 2, direction of propagation r is 4, then it is represented sequentially as by the R that 4 directions of propagation are derived from：
R∈{(-1,0),(0,-1)}、R∈{(1,0),(0,1)}、R∈{(0,1),(-1,0)}、R∈{(0,-1),(1, 0)}；
When m is 1, direction of propagation r is 8, then it is represented sequentially as by the R that 8 directions of propagation are derived from：
R∈{(-1,0)}、R∈{(1,0)}、R∈{(0,1)}、R∈{(0,-1)}、
R∈{(-1,-1)}、R∈{(1,-1)}、R∈{(1,1)}、R∈{(1,1)}
When m is 1, direction of propagation r is 4, then it is represented sequentially as by the R that 4 directions of propagation are derived from：R∈{(-1,0)},R ∈{(1,0)}、R∈{(0,1)}、R∈{(0,-1)}。
P-R indicates previous pixel of the pixel p on the R of direction；Lr(p-R, d) indicates pixel p-R when parallax is d Matching cost, Lr(p-R, d+1) indicates matching cost of the pixel p-R when parallax is d+1, Lr(p-R, d-1) indicates pixel Matching cost of the p-R when parallax is d-1；D' is the parallax of pixel p-R, the disparity search range d' ∈ [d of pixel p-Rmin(p- R),dmax(p-R)],Indicate pixel p-R in disparity search range [dmin(p-R),dmax(p-R)] minimum in Route matching cost；P1 R、P2 RIndicate punishment parameter, P2 R>P1 R。
Parameter P1 RWith P2 RSelection have a larger impact to final matching results, size determines the smoothness of parallax.Such as When fruit punishment parameter is larger, parallax is smooth, but can not retain edge details in parallax sudden change region, and the corresponding edge of parallax is fixed Position precision is lower.The present invention is to ensure the edge details of barrier, obtains accurate edge, selects punishment ginseng according to marginal information Number.Region not mutated for flat site, that is, parallax will reinforce disparity smoothness constraint using biggish punishment parameter；And for Non-planar regions at parallax mutation will retain the parallax mutation of edge using lesser punishment parameter, improve break edge Positional accuracy.
If current pixel point p (x, y) is the marginal point in left pyramid highest tomographic image on qualified edge, The corresponding punishment parameter of p (x, y) is：
If current pixel point p (x, y) is not the marginal point in left pyramid highest tomographic image on qualified edge, Then the corresponding punishment parameter of p (x, y) is：
Wherein, P1With P2Value refer to MGM method value, P1Preferably 8, P2Preferably 32.
Other pixels in left pyramid highest tomographic image are handled according to step S342, obtain the top figure of left pyramid The matching cost polymerizing value of all pixels point as in.
S343 calculates the initial parallax figure of left pyramid highest tomographic image：It is tactful according to the victor is a king, it calculates point p (x, y) The parallax value at place：
Other pixels in left pyramid highest tomographic image are handled according to step S343, obtain the top figure of left pyramid The parallax value of all pixels point, forms the initial parallax figure of left pyramid highest tomographic image as in.
In the right pyramid highest tomographic image obtained according to step S33 the matching cost of all pixels point to right pyramid most All pixels point carries out matching cost polymerization in high-rise image, obtains matching cost polymerizing value, poly- according to obtained matching cost The method that conjunction is worth to obtain right initial parallax figure repeats step S341~S343 according to identical same principle, obtains right pyramid most The initial parallax figure of high-rise image.
S35：Sub-pix disparity computation is carried out to the initial parallax figure of the obtained left pyramid highest tomographic image of step S34, Obtain the sub-pix disparity map of left pyramid highest tomographic image；To the initial of the obtained right pyramid highest tomographic image of step S34 Disparity map carries out sub-pix disparity computation, obtains the sub-pix disparity map of right pyramid highest tomographic image；Then to an obtained left side The sub-pix disparity map of pyramid highest tomographic image is consistent by left and right with the sub-pix disparity map of right pyramid highest tomographic image Property detection processing, obtains the disparity map of left pyramid highest tomographic image and the disparity map of right pyramid highest tomographic image.
Sub-pix disparity computation：The pyramidal initial parallax figure in left and right described above is all pixel scale, this is resulted in There is more serious crenellated phenomena in some continuous planes, especially shows big-inclination plane, curved surface etc. and camera shooting In the non-scene faced of machine.Therefore, it is necessary to be increased to sub-pixel from Pixel-level to matching precision, so that on target surface Naturally smooth transition is presented in parallax.The present invention obtains the disparity map of sub-pixel precision using conic fitting method.
Consistency detection initial horizontal parallax figure in left and right realizes the detection of occlusion area and Mismatching point, it requires left The parallax of right disparity map is consistent.
The sub-pix disparity map for setting left pyramid highest tomographic image is denoted as Dlr(x, y), right pyramid highest tomographic image Sub-pix disparity map is denoted as Drl(x, y), for any pixel point in the sub-pix disparity map of left pyramid highest tomographic image, such as Fruit meets | Dlr(x,y)-Drl(x+Dlr(x,y),y)|<T, T 1, then the pixel is left and right consistency point, otherwise the pixel For left and right inconsistency point.
The embodiment of the present invention is obtained in the case where direction of propagation r takes 8 directions, derives from the m=4 in the R of direction.Figure 4D is the corresponding disparity map of the left pyramid highest tomographic image of the method for the present invention, and Fig. 4 E is the disparity map and meet parallax that Fig. 4 D is indicated Disparity map after a plurality of edge superposition of span condition.
S4, determine in left pyramid L-1 tomographic image own using L layers of left pyramid of the disparity map that step S3 is obtained The disparity search range of pixel；L-1 layers of right pyramid is determined using L layers of right pyramid of the disparity map that step S3 is obtained The disparity search range of all pixels point in image；According to the view of all pixels point in determining left pyramid L-1 tomographic image Poor search range carries out Stereo matching from left to right, root to left pyramid L-1 tomographic image and right pyramid L-1 tomographic image According to the disparity search range of all pixels point in determining right pyramid L-1 tomographic image to left pyramid L-1 tomographic image and Right pyramid L-1 tomographic image carries out Stereo matching from right to left, obtains the disparity map and right golden word of L-1 layers of left pyramid The disparity map that L-1 layers of tower.
The view of all pixels point in left pyramid L-1 tomographic image is determined using L layers of the disparity map that step S3 is obtained The method of poor search range is：
S41：To any pixel point p in left pyramid L-1 tomographic imageL-1(xL-1,yL-1), it calculatesL layers of left pyramid of the disparity map obtained according to step S3 is by left pyramid L Pixel p in tomographic imageL(xL,yL) corresponding parallax is denoted as dL(xL,yL), with pL(xL,yL) centered on establish size be 3 × 3 The window of pixel；
S42：If all pixels in the window are all left and right consistency points, as shown in Figure 5A, then calculate in the window The corresponding parallax maximum value d of all pixelsL maxWith parallax minimum value dL min：
dL max=max { dL(xL+i,yL+ j) | i=-1,0,1, j=-1,0,1 }
dL min=min { dL(xL+i,yL+ j) | i=-1,0,1, j=-1,0,1 }
By the pixel p in left pyramid L-1 tomographic imageL-1(xL-1,yL-1) disparity search range be set as [2dL min-δ,2dL max+ δ] in integer, δ is surplus, preferably 1~3.
S43：If there are left and right inconsistency points for the pixel in the window, as shown in Figure 5 B, then in yLRow, with pL (xL,yL) it is starting point, it is searched for the left until obtaining left and right consistency point pl(xL-xl,yL), it searches for the right consistent until obtaining left and right Property point pr(xL+xr,yL)；In yL- 1 row, with pL(xL,yL- 1) it is starting point, is searched for the left until obtaining left and right consistency point pul (xL-xul,yL- 1) it, searches for the right until obtaining left and right consistency point pur(xL+xur,yL-1)；In yL+ 1 row, with pd(xL,yL+ 1) it is starting point, is searched for the left until obtaining left and right consistency point pdl(xL-xdl,yL+ 1) it, searches for the right consistent until obtaining left and right Property point pdr(xL+xdr,yL+1)。
L layers of left pyramid of the disparity map obtained according to step S3 obtains above-mentioned point pul,pur,pl,pr,pdl,pdrInstitute Maximum value d' in corresponding parallaxLmaxWith minimum value d'Lmin, by the pixel p in left pyramid L-1 tomographic imageL-1(xL-1, yL-1) disparity search range be set asIn integer.
S44：Step S41~S43 is repeated, until all pixels point in left pyramid L-1 tomographic image is all disposed, Obtain the disparity search range of all pixels point in left pyramid L-1 tomographic image.
All pictures in right pyramid L-1 tomographic image are determined using L layers of right pyramid of the disparity map that step S3 is obtained The method of the disparity search range of vegetarian refreshments repeats step S41~S44 according to same principle, obtains right pyramid L-1 tomographic image The disparity search range of middle all pixels point.
According to the disparity search range of all pixels point in determining left pyramid L-1 tomographic image to left pyramid L- 1 tomographic image and right pyramid L-1 tomographic image carry out Stereo matching from left to right, according to determining L-1 layers of right pyramid The disparity search range of all pixels point carries out left pyramid L-1 tomographic image and right pyramid L-1 tomographic image in image Stereo matching from right to left obtains the left disparity map of L-1 layers of left pyramid and the right disparity map of L-1 layers of right pyramid Method, specific step is as follows：
S45：Census transformation is carried out to all pixels point in left pyramid L-1 tomographic image, obtains left pyramid the The Census sequence of all pixels point in L-1 tomographic image carries out all pixels point in right pyramid L-1 tomographic image Census transformation, obtains the Census sequence of all pixels point in right pyramid L-1 tomographic image；According to obtained left pyramid The Census sequence of all pixels point in L-1 tomographic image, in right pyramid L-1 tomographic image all pixels point Census sequence Column, in left pyramid L-1 tomographic image within the scope of the disparity search of all pixels point, are calculated L-1 layers of left pyramid The matching cost of all pixels point in image；According to the Census of all pixels point in obtained left pyramid L-1 tomographic image The Census sequence of all pixels point in sequence, right pyramid L-1 tomographic image owns in right pyramid L-1 tomographic image Within the scope of the disparity search of pixel, the matching cost of all pixels point in right pyramid L-1 tomographic image is calculated；
S46：The matching cost of all pixels point is to left gold in the left pyramid L-1 tomographic image obtained according to step S45 All pixels point carries out matching cost polymerization in word tower L-1 tomographic image, obtains the matching cost of left pyramid L-1 tomographic image Polymerizing value obtains left pyramid L-1 tomographic image according to the matching cost polymerizing value of obtained left pyramid L-1 tomographic image Initial parallax figure；The matching cost of all pixels point is to right golden word in the right pyramid L-1 tomographic image obtained according to step S45 All pixels point carries out matching cost polymerization in tower L-1 tomographic image, and the matching cost for obtaining right pyramid L-1 tomographic image is poly- Conjunction value obtains the first of right pyramid L-1 tomographic image according to the matching cost polymerizing value of obtained right pyramid L-1 tomographic image Beginning disparity map；
S47：Sub-pix disparity computation is carried out to the initial parallax figure of the obtained left pyramid L-1 tomographic image of step S46, Obtain the sub-pix disparity map of left pyramid L-1 tomographic image；To the first of the obtained right pyramid L-1 tomographic image of step S46 Beginning disparity map carries out sub-pix disparity computation, obtains the sub-pix disparity map of right pyramid L-1 tomographic image；Then to obtaining The sub-pix disparity map of left pyramid L-1 tomographic image and the sub-pix disparity map of right pyramid L-1 tomographic image are by left and right Consistency detection processing, obtains the disparity map of left pyramid L-1 tomographic image and the disparity map of right pyramid L-1 tomographic image.
Fig. 6 A is the edge graph that left the 2nd tomographic image of pyramid extracts, and image size is 473 × 348 pixels, and Fig. 6 B is this hair The corresponding left disparity map of the 2nd tomographic image of left pyramid that bright method obtains, Fig. 6 C are the disparity map and meet parallax that Fig. 6 B is indicated Disparity map after the edge superposition of span condition, in the average value processing of this layer of parallax span, the threshold value that sets is 5.
S5, institute in left pyramid L-2 tomographic image is determined using L-1 layers of left pyramid of the disparity map that step S4 is obtained There is the disparity search range of pixel；The disparity map of L-1 layers of the right pyramid obtained using step S4 determines right pyramid The disparity search range of all pixels point in L-2 tomographic image；According to all pixels point in determining left pyramid L-2 tomographic image Disparity search range from left to right three-dimensional is carried out to left pyramid L-2 tomographic image and right pyramid L-2 tomographic image Match, L-2 layers of left pyramid are schemed according to the disparity search range of all pixels point in determining right pyramid L-2 tomographic image Picture and right pyramid L-2 tomographic image carry out Stereo matching from right to left, obtain disparity map and the right side of L-2 layers of left pyramid The disparity map that L-2 layers of pyramid.
Circulation carries out later, until determining left the 1st tomographic image (i.e. pole of pyramid using the 2nd layer of left pyramid of disparity map Line correction after left image) in all pixels point disparity search range；Right gold is determined using the 2nd layer of disparity map of right pyramid The disparity search range of all pixels point in the 1st tomographic image of word tower (right image i.e. after polar curve correction)；According to determining left gold The disparity search range of all pixels point is to left the 1st tomographic image of pyramid and the 1st tomographic image of right pyramid in the 1st tomographic image of word tower Stereo matching from left to right is carried out, according to the disparity search range of all pixels point in determining the 1st tomographic image of right pyramid Stereo matching from right to left is carried out to left the 1st tomographic image of pyramid and the 1st tomographic image of right pyramid, obtains left pyramid the 1st The disparity map and the 1st layer of right pyramid of disparity map of layer.
Fig. 7 A is the side that left the 1st tomographic image of pyramid (left image after polar curve correction, the i.e. image that Fig. 2A is indicated) extracts Edge figure, Fig. 7 B are disparity maps corresponding to left the 1st tomographic image of pyramid of the method for the present invention, Fig. 7 C be the disparity map that indicates of Fig. 7 B with Meet the disparity map after the edge superposition of condition, wherein at the edge for obtaining meeting condition, set in the 1st tomographic image Threshold value is that 7.5, Fig. 7 D is disparity map after the disparity map that traditional MGM method obtains is superimposed with the edge for meeting parallax conditions. It is inaccurate to the disparity computation of edge environ, prominent to parallax to can be seen that traditional MGM method by comparing Fig. 7 C and Fig. 7 D Positioning at change is inaccurate, however the method for the present invention can be accurately positioned at parallax mutation, therefore can in edge environ Obtain accurate parallax.
Computation complexity analysis.Each pixel to resolution ratio for W × H original image is needed based on traditional MGM method It calculates D times, D is parallax span, i.e., computation complexity is O (W × H × D), wherein original image whole picture figure of the embodiment of the present invention The disparity search range of picture is [- 40,150], then parallax span D is 190 pixels, therefore computation complexity is expressed as O (947 ×696×190).Computation complexity analysis for the method for the present invention, needs to count all layers of all pixels in pyramid Disparity search range, wherein the matching primitives complexity of top (the 3rd layer) two images is (734903) O, the 2nd layer of two width The matching primitives complexity of image is (611766) O, and the matching primitives complexity of the 1st layer of (original) two images is O (2457204).The ratio of two methods computation complexity is：

Claims (10)

1. a kind of solid matching method of objects outside Earth rover, it is characterised in that the step of this method includes：
S1, the two images that objects outside Earth scene is obtained using the left and right camera of objects outside Earth rover, wherein left camera obtains Image definition be left image, the image definition that right camera obtains is right image；
S2, respectively L layers of the left image building to the left image and right image progress polar curve correction in step S1, after being corrected to polar curve Pyramid, referred to as left pyramid construct L layers of pyramid, referred to as right pyramid to the right image after polar curve correction；
S3, to top i.e. L layers of the image of left pyramid and top i.e. L layers of the figure of right pyramid constructed in step S2 As carrying out two-way Stereo matching, the disparity map of L layers of left pyramid and the disparity map of L layers of right pyramid are obtained；
S4, all pixels in left pyramid L-1 tomographic image are determined using L layers of left pyramid of the disparity map that step S3 is obtained The disparity search range of point；Right pyramid L-1 tomographic image is determined using L layers of right pyramid of the disparity map that step S3 is obtained The disparity search range of middle all pixels point；It is searched according to the parallax of all pixels point in determining left pyramid L-1 tomographic image Rope range carries out Stereo matching from left to right to left pyramid L-1 tomographic image and right pyramid L-1 tomographic image, according to true The disparity search range of all pixels point is to left pyramid L-1 tomographic image and right gold in fixed right pyramid L-1 tomographic image Word tower L-1 tomographic image carries out Stereo matching from right to left, obtains L-1 layers of left pyramid of disparity map and right pyramid the L-1 layers of disparity map；
S5, all pictures in left pyramid L-2 tomographic image are determined using L-1 layers of left pyramid of the disparity map that step S4 is obtained The disparity search range of vegetarian refreshments；L-2 layers of right pyramid is determined using L-1 layers of right pyramid of the disparity map that step S4 is obtained The disparity search range of all pixels point in image；According to the view of all pixels point in determining left pyramid L-2 tomographic image Poor search range carries out Stereo matching from left to right, root to left pyramid L-2 tomographic image and right pyramid L-2 tomographic image According to the disparity search range of all pixels point in determining right pyramid L-2 tomographic image to left pyramid L-2 tomographic image and Right pyramid L-2 tomographic image carries out Stereo matching from right to left, obtains the disparity map and right golden word of L-2 layers of left pyramid The disparity map that L-2 layers of tower；
……
S6, it is determined using the 2nd layer of left pyramid of disparity map in left the 1st tomographic image of pyramid (left image i.e. after polar curve correction) The disparity search range of all pixels point；Right the 1st tomographic image (i.e. pole of pyramid is determined using the 2nd layer of disparity map of right pyramid Line correction after right image) in all pixels point disparity search range；Own according in determining the 1st tomographic image of left pyramid The disparity search range of pixel carries out solid from left to right to left the 1st tomographic image of pyramid and the 1st tomographic image of right pyramid Matching, according to the disparity search range of all pixels point in determining the 1st tomographic image of right pyramid to left the 1st tomographic image of pyramid Stereo matching from right to left is carried out with right the 1st tomographic image of pyramid, obtains the 1st layer of left pyramid of disparity map and right pyramid 1st layer of disparity map.
2. a kind of solid matching method of objects outside Earth rover according to claim 1, it is characterised in that：The step In rapid S2, L is natural number, and value is 2~5；The pyramid is gaussian pyramid；
It is described to top i.e. L layers of the image of left pyramid constructed in step S2 and right pyramid in the step S3 The method that top i.e. L layers of image carries out two-way Stereo matching, specific step is as follows：
S31：The SIFT feature in left pyramid highest tomographic image is extracted, referred to as left SIFT feature extracts right pyramid most SIFT feature in high-rise image, referred to as right SIFT feature, and to the left SIFT feature and right SIFT feature of extraction It is matched, obtains n to match point；N should be in the spy in left pyramid highest tomographic image to the i-th pair matching double points in match point Sign point coordinate is denoted as (xi,yi), n should be in the characteristic point in right pyramid highest tomographic image to the i-th pair matching double points in match point Coordinate is denoted as (xi',yi'), i=0,1 ..., n-1, n >=3；
S32：D is calculated to the i-th pair match point in match point according to n obtained in step S31i=xi'-xi, obtain left pyramid The corresponding spatial point of ith feature point in highest tomographic image, coordinate are denoted as (xi,yi,di), left pyramid highest tomographic image In n characteristic point obtain n spatial point, the n space obtained to n characteristic point in left pyramid highest tomographic image clicks through The fitting of row space plane, is calculated the disparity search range of all pixels point in left pyramid highest tomographic image；
D is calculated to the i-th pair match point in match point according to n obtained in step S31i'=xi-xi', obtain right pyramid most The corresponding spatial point of ith feature point in high-rise image, coordinate are denoted as (xi',yi',di'), right pyramid highest tomographic image In n characteristic point obtain n spatial point, the n space obtained to n characteristic point in right pyramid highest tomographic image clicks through The fitting of row space plane, is calculated the disparity search range of all pixels point in right pyramid highest tomographic image；
S33：Census transformation is carried out to all pixels point in left pyramid highest tomographic image, obtains the top figure of left pyramid The Census sequence of all pixels point as in carries out Census transformation to all pixels point in right pyramid highest tomographic image, Obtain the Census sequence of all pixels point in right pyramid highest tomographic image；According in obtained left pyramid highest tomographic image The Census sequence of all pixels point in the Census sequence of all pixels point, right pyramid highest tomographic image, in left pyramid In highest tomographic image within the scope of the disparity search of all pixels point, all pixels point in left pyramid highest tomographic image is calculated Matching cost；According to the Census sequence, right pyramid highest of all pixels point in obtained left pyramid highest tomographic image The Census sequence of all pixels point in tomographic image, the disparity search range of all pixels point in right pyramid highest tomographic image It is interior, calculate the matching cost of all pixels point in right pyramid highest tomographic image；
S34：In the left pyramid highest tomographic image obtained according to step S33 the matching cost of all pixels point to left pyramid most All pixels point carries out matching cost polymerization in high-rise image, obtains the matching cost polymerizing value of left pyramid highest tomographic image, The initial parallax of left pyramid highest tomographic image is obtained according to the matching cost polymerizing value of obtained left pyramid highest tomographic image Figure；The matching cost of all pixels point is top to right pyramid in the right pyramid highest tomographic image obtained according to step S33 All pixels point carries out matching cost polymerization in image, obtains the matching cost polymerizing value of right pyramid highest tomographic image, according to The matching cost polymerizing value of obtained right pyramid highest tomographic image obtains the initial parallax figure of right pyramid highest tomographic image；
S35：Sub-pix disparity computation is carried out to the initial parallax figure of the obtained left pyramid highest tomographic image of step S34, is obtained The sub-pix disparity map of left pyramid highest tomographic image；To the initial parallax of the obtained right pyramid highest tomographic image of step S34 Figure carries out sub-pix disparity computation, obtains the sub-pix disparity map of right pyramid highest tomographic image；Then to obtained left golden word The sub-pix disparity map of the sub-pix disparity map of tower highest tomographic image and right pyramid highest tomographic image is examined by left and right consistency Survey processing, obtains the disparity map of left pyramid highest tomographic image and the disparity map of right pyramid highest tomographic image.
3. a kind of solid matching method of objects outside Earth rover according to claim 2, it is characterised in that：The step In rapid S32, it is quasi- that the n spatial point that the n characteristic point in left pyramid highest tomographic image obtains carries out space plane It closes, being calculated in left pyramid highest tomographic image has the method for the disparity search range of pixel, and step includes：
S321 carries out space plane fitting to the n spatial point that n characteristic point in left pyramid highest tomographic image obtains, obtains To the equation of space plane Π be expressed as D=aX+bY+c；
Wherein, the coefficient in a, b, c representation space plane equation；(X, Y, D) is the coordinate of any point on space plane Π, D For parallax；
N spatial point is fitted using least square method, to determine the coefficient in spatial plane equation, i.e.,
According toObtain space plane coefficient a, b, c；
S322, for the coordinate (x of the ith feature point in n characteristic point in left pyramid highest tomographic imagei,yi), according to The equation D=aX+bY+c of space plane Π calculates the corresponding parallax d of ith feature pointi Π, calculate the parallax of ith feature point Difference di Π-di；For the corresponding n disparity difference of n characteristic point in left pyramid highest tomographic image, it is poor to count n parallax Maximum value D in valuemaxWith minimum value Dmin；
S323, for any pixel point p (x, y) in left pyramid highest tomographic image, according to the equation D=aX of space plane Π + bY+c calculates the corresponding parallax d of pixel p (x, y)Π(x, y) calculates dmin(x, y)=dΠ(x,y)+DminΔ d, dmax(x, Y)=dΠ(x,y)+Dmax+ Δ d, wherein Δ d is reserved parallax surplus, determines that the disparity search range of pixel p (x, y) is [dmin(x,y),dmax(x, y)] in integer, obtain the disparity search range for having pixel in left pyramid highest tomographic image；
The method that the n spatial point obtained to n characteristic point in right pyramid highest tomographic image carries out space plane fitting, according to Step S321~S323 is repeated according to identical principle, obtains the disparity search model of all pixels point in right pyramid highest tomographic image It encloses.
4. a kind of solid matching method of objects outside Earth rover according to claim 2, it is characterised in that：The step In rapid S33, Census transformation is carried out to all pixels point in left pyramid highest tomographic image, obtains the top figure of left pyramid The method of the Census sequence of all pixels point is as in：
S331 establishes Census window centered on any pixel point in left pyramid highest tomographic image, compares foundation The size relation of the gray value of pixel and central point in Census window in addition to central point, gray value is less than or equal to The pixel of central point gray value is labeled as 0, is otherwise labeled as 1, comparison result step-by-step is finally connected as 0/1 binary code Stream, referred to as the Census sequence of current pixel point；
S332 repeats step S331, until all pixels point in left pyramid highest tomographic image is all disposed, obtains left gold The Census sequence of all pixels point in word tower highest tomographic image；
Census transformation is carried out to all pixels point in right pyramid highest tomographic image, is obtained in right pyramid highest tomographic image The method of the Census sequence of all pixels point repeats step S331~S332 according to same principle, obtains right pyramid highest The Census sequence of all pixels point in tomographic image.
5. a kind of solid matching method of objects outside Earth rover according to claim 2 or 3, it is characterised in that：It is described Step S33 in, according to the Census sequence, right pyramid highest of all pixels point in obtained left pyramid highest tomographic image The Census sequence of all pixels point in tomographic image, the disparity search range of all pixels point in left pyramid highest tomographic image Interior, the method that the matching cost of all pixels point in left pyramid highest tomographic image is calculated is：
S333 calculates the pixel in the pixel p (x, y) and right pyramid highest tomographic image in left pyramid highest tomographic image The Hamming distance of the Census sequence of q (x-d, y), d are parallax, d ∈ [dmin(x,y),dmax(x, y)], and d is integer, is obtained Matching cost C (p, d) of the pixel p (x, y) when parallax is d in left pyramid highest tomographic image；
S334 repeats step S333, obtains pixel p (x, y) in left pyramid highest tomographic image in different parallax d Matching cost；
S335 repeats step S333~S334, until all pixels point in left pyramid highest tomographic image is all disposed, obtains The matching cost of all pixels point into left pyramid highest tomographic image；
The top figure of left pyramid is obtained according to the disparity search range computation of all pixels point in left pyramid highest tomographic image The method of the matching cost of all pixels point as in repeats step S333~S335 according to same principle, obtains right pyramid most The matching cost of all pixels point in high-rise image.
6. a kind of solid matching method of objects outside Earth rover according to claim 2 or 5, it is characterised in that：It is described Step S34 in, the matching cost of all pixels point is to a left side in the left pyramid highest tomographic image obtained according to step S33 All pixels point carries out matching cost polymerization in pyramid highest tomographic image, matching cost polymerizing value is obtained, according to obtained The method for obtaining the initial parallax figure of left pyramid highest tomographic image with cost polymerizing value is：
S341 carries out the detection of Canny edge feature to left pyramid highest tomographic image, obtains several edges；It will be on edge Point is known as marginal point, for any edge point, according to the disparity search range [d of pixel corresponding with the marginal pointmin(x, y),dmax(x, y)], calculating difference dmax(x,y)-dmin(x, y), referred to as the parallax span of the marginal point are calculated in each of the edges The average value of the parallax span of all marginal points will remove if average value is less than the threshold value of setting when preceding article edge, if Average value then retains not less than the threshold value of setting and works as preceding article edge；Several that the top image detection of left pyramid is obtained After edge processing, qualified edge in left pyramid highest tomographic image is obtained；
S342, according to the matching cost of all pixels point in left pyramid highest tomographic image to institute in left pyramid highest tomographic image There is pixel to carry out matching cost polymerization, obtains matching cost polymerizing value；
Calculate the matching cost polymerizing value S (p, d) of pixel p (x, y) when parallax is d：
In formula, r indicates the direction of propagation of pixel p (x, y)；Lr(p, d) indicates pixel p (x, y) along the matching cost of direction r；
In formula, R indicates the m direction derived from by direction of propagation r, R ∈ { r1,…,rm, m >=1, r1It is identical as the direction r, i.e. r1= r；P-R indicates previous pixel of the pixel p on the R of direction；Lr(p-R, d) indicates matching of the pixel p-R when parallax is d Cost, Lr(p-R, d+1) indicates matching cost of the pixel p-R when parallax is d+1, Lr(p-R, d-1) indicates that pixel p-R works as Matching cost when parallax is d-1；D' is the parallax of pixel p-R, the disparity search range d' ∈ [d of pixel p-Rmin(p-R), dmax(p-R)], d' is integer,Indicate pixel p-R in disparity search range [dmin(p-R),dmax(p-R)] Interior the smallest route matching cost；P1 R、P2 RIndicate punishment parameter, P2 R>P1 R；
If current pixel point p (x, y) is the marginal point in left pyramid highest tomographic image on qualified edge, p (x, Y) corresponding punishment parameter is：
If current pixel point p (x, y) is not the marginal point in left pyramid highest tomographic image on qualified edge, p (x, y) corresponding punishment parameter is：
Other pixels in left pyramid highest tomographic image are handled according to step S342, are obtained in left pyramid highest tomographic image The matching cost polymerizing value of all pixels point；
S343 calculates the initial parallax figure of left pyramid highest tomographic image：It is tactful according to the victor is a king, it calculates at point p (x, y) Parallax value：
Other pixels in left pyramid highest tomographic image are handled according to step S343, are obtained in left pyramid highest tomographic image The parallax value of all pixels point forms the initial parallax figure of left pyramid highest tomographic image；
The matching cost of all pixels point is top to right pyramid in the right pyramid highest tomographic image obtained according to step S33 All pixels point carries out matching cost polymerization in image, matching cost polymerizing value is obtained, according to obtained matching cost polymerizing value The method for obtaining right initial parallax figure repeats step S341~S343 according to identical same principle, and it is top to obtain right pyramid The initial parallax figure of image.
7. a kind of solid matching method of objects outside Earth rover according to claim 6, it is characterised in that：The propagation Direction r is 8 directions,
R ∈ { (- 1,0), (1,0), (0,1), (0, -1), (- 1, -1), (1, -1), (1,1), (- 1,1) },
Or 4 directions,
r∈{(-1,0),(1,0),(0,1),(0,-1)}；
The m is 4,2 or 1；
When m is 4, direction of propagation r is 8, then it is represented sequentially as by the R that 8 direction of propagation r derive from：
R∈{(-1,0),(0,-1),(-1,-1),(1,-1)}、R∈{(1,0),(0,1),(1,1),(-1,1)}、
R∈{(0,1),(-1,0),(-1,1),(-1,-1)}、R∈{(0,-1),(1,0),(1,-1),(1,1)}
R∈{(-1,-1),(1,-1),(0,-1),(1,0)}、R∈{(1,-1),(1,1),(1,0),(0,1)}、
R∈{(1,1),(-1,1),(0,1),(-1,0)}、R∈{(-1,1),(-1,-1),(-1,0),(0,-1)}
When m is 4, direction of propagation r is 4, then it is represented sequentially as by the R that 4 direction of propagation r derive from：
R∈{(-1,0),(0,-1),(-1,-1),(1,-1)}、R∈{(1,0),(0,1),(1,1),(-1,1)}、
R∈{(0,1),(-1,0),(-1,1),(-1,-1)}、R∈{(0,-1),(1,0),(1,-1),(1,1)}
When m is 2, direction of propagation r is 8, then it is represented sequentially as by the R that 8 directions of propagation are derived from：
R∈{(-1,0),(0,-1)}、R∈{(1,0),(0,1)}、R∈{(0,1),(-1,0)}、R∈{(0,-1),(1,0)}、
R∈{(-1,-1),(1,-1)}、R∈{(1,-1),(1,1)}、R∈{(1,1),(-1,1)}、R∈{(-1,1),(-1,- 1)}
When m is 2, direction of propagation r is 4, then it is represented sequentially as by the R that 4 directions of propagation are derived from：R∈{(-1,0),(0,- 1)},R∈{(1,0),(0,1)},R∈{(0,1),(-1,0)},R∈{(0,-1),(1,0)}；
When m is 1, direction of propagation r is 8, then it is represented sequentially as by the R that 8 directions of propagation are derived from：
R∈{(-1,0)}、R∈{(1,0)}、R∈{(0,1)}、R∈{(0,-1)}、
R∈{(-1,-1)}、R∈{(1,-1)}、R∈{(1,1)}、R∈{(1,1)}
When m is 1, direction of propagation r is 4, then it is represented sequentially as by the R that 4 directions of propagation are derived from：R∈{(-1,0)},R∈ {(1,0)}、R∈{(0,1)}、R∈{(0,-1)}。
8. a kind of solid matching method of objects outside Earth rover according to claim 2, it is characterised in that：The step In rapid S35, sub-pix disparity computation uses conic fitting method；
The method of the left and right consistency detection processing：
The sub-pix disparity map for setting left pyramid highest tomographic image is denoted as Dlr(x, y), the sub- picture of right pyramid highest tomographic image Plain disparity map is denoted as Drl(x, y), for any pixel point in the sub-pix disparity map of left pyramid highest tomographic image, if full Foot | Dlr(x,y)-Drl(x+Dlr(x,y),y)|<T, T 1, then the pixel is left and right consistency point, and otherwise the pixel is a left side Right inconsistency point.
9. a kind of solid matching method of objects outside Earth rover according to claim 1, it is characterised in that：The step In rapid S4, all pixels in left pyramid L-1 tomographic image are determined using L layers of left pyramid of the disparity map that step S3 is obtained The method of disparity search range of point is：
S41：To any pixel point p in left pyramid L-1 tomographic imageL-1(xL-1,yL-1), it calculatesL layers of left pyramid of the disparity map obtained according to step S3 is by left pyramid L Pixel p in tomographic imageL(xL,yL) corresponding parallax is denoted as dL(xL,yL), with pL(xL,yL) centered on establish size be 3 × 3 The window of pixel；
S42：If all pixels in the window are all left and right consistency points, all pixels calculated in the window are corresponding Parallax maximum value dLmaxWith parallax minimum value dLmin：
dLmax=max { dL(xL+i,yL+ j) | i=-1,0,1, j=-1,0,1 }
dLmin=min { dL(xL+i,yL+ j) | i=-1,0,1, j=-1,0,1 }
By the pixel p in left pyramid L-1 tomographic imageL-1(xL-1,yL-1) disparity search range be set as [2dLmin-δ, 2dLmax+ δ] in integer, δ is surplus；
S43：If there are left and right inconsistency points for the pixel in the window, in yLRow, with pL(xL,yL) it is starting point, to the left Search is until obtaining left and right consistency point pl(xL-xl,yL), it searches for the right until obtaining left and right consistency point pr(xL+xr,yL)；? YL- 1 row, with pL(xL,yL- 1) it is starting point, is searched for the left until obtaining left and right consistency point pul(xL-xul,yL- 1) it, searches to the right Rope is until obtaining left and right consistency point pur(xL+xur,yL-1)；In yL+ 1 row, with pd(xL,yLIt+1) is starting point, search is straight to the left To obtaining left and right consistency point pdl(xL-xdl,yL+ 1) it, searches for the right until obtaining left and right consistency point pdr(xL+xdr,yL+1)；
L layers of left pyramid of the disparity map obtained according to step S3 obtains above-mentioned point pul,pur,pl,pr,pdl,pdrCorresponding Maximum value d' in parallaxLmaxWith minimum value d'Lmin, by the pixel p in left pyramid L-1 tomographic imageL-1(xL-1,yL-1) Disparity search range is set asIn integer；
S44：Step S41~S43 is repeated, until all pixels point in left pyramid L-1 tomographic image is all disposed, is obtained The disparity search range of all pixels point in left pyramid L-1 tomographic image；
All pixels point in right pyramid L-1 tomographic image is determined using L layers of right pyramid of the disparity map that step S3 is obtained Disparity search range method according to same principle repeat step S41~S44, obtain institute in right pyramid L-1 tomographic image There is the disparity search range of pixel.
10. a kind of solid matching method of objects outside Earth rover according to claim 1, it is characterised in that：Described In step S4, according to the disparity search range of all pixels point in determining left pyramid L-1 tomographic image to left pyramid L-1 tomographic image and right pyramid L-1 tomographic image carry out Stereo matching from left to right, according to determining right pyramid L-1 In tomographic image the disparity search range of all pixels point to left pyramid L-1 tomographic image and right pyramid L-1 tomographic image into The Stereo matching of row from right to left, obtains the left disparity map of L-1 layers of left pyramid and the right disparity map of L-1 layers of right pyramid Method, specific step is as follows：
S45：Census transformation is carried out to all pixels point in left pyramid L-1 tomographic image, obtains L-1 layers of left pyramid The Census sequence of all pixels point in image carries out Census change to all pixels point in right pyramid L-1 tomographic image It changes, obtains the Census sequence of all pixels point in right pyramid L-1 tomographic image；According to L-1 layers of left pyramid obtained The Census sequence of all pixels point in image, in right pyramid L-1 tomographic image all pixels point Census sequence, on a left side In pyramid L-1 tomographic image within the scope of the disparity search of all pixels point, institute in left pyramid L-1 tomographic image is calculated There is the matching cost of pixel；According to the Census sequence of all pixels point, right gold in obtained left pyramid L-1 tomographic image The Census sequence of all pixels point in word tower L-1 tomographic image, the view of all pixels point in right pyramid L-1 tomographic image In poor search range, the matching cost of all pixels point in right pyramid L-1 tomographic image is calculated；
S46：The matching cost of all pixels point is to left pyramid in the left pyramid L-1 tomographic image obtained according to step S45 All pixels point carries out matching cost polymerization in L-1 tomographic image, obtains the matching cost polymerization of left pyramid L-1 tomographic image Value, obtains the initial of left pyramid L-1 tomographic image according to the matching cost polymerizing value of obtained left pyramid L-1 tomographic image Disparity map；The matching cost of all pixels point is to right pyramid in the right pyramid L-1 tomographic image obtained according to step S45 All pixels point carries out matching cost polymerization in L-1 tomographic image, obtains the matching cost polymerization of right pyramid L-1 tomographic image Value, obtains the initial of right pyramid L-1 tomographic image according to the matching cost polymerizing value of obtained right pyramid L-1 tomographic image Disparity map；
S47：Sub-pix disparity computation is carried out to the initial parallax figure of the obtained left pyramid L-1 tomographic image of step S46, is obtained The sub-pix disparity map of left pyramid L-1 tomographic image；To the initial view of the obtained right pyramid L-1 tomographic image of step S46 Difference figure carries out sub-pix disparity computation, obtains the sub-pix disparity map of right pyramid L-1 tomographic image；Then to obtained left gold The sub-pix disparity map of word tower L-1 tomographic image is consistent by left and right with the sub-pix disparity map of right pyramid L-1 tomographic image Property detection processing, obtains the disparity map of left pyramid L-1 tomographic image and the disparity map of right pyramid L-1 tomographic image.
CN201810513673.5A 2018-05-25 2018-05-25 Stereo matching method for extraterrestrial celestial body patrolling device Expired - Fee Related CN108876861B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201810513673.5A CN108876861B (en) 2018-05-25 2018-05-25 Stereo matching method for extraterrestrial celestial body patrolling device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201810513673.5A CN108876861B (en) 2018-05-25 2018-05-25 Stereo matching method for extraterrestrial celestial body patrolling device

Publications (2)

Publication Number Publication Date
CN108876861A true CN108876861A (en) 2018-11-23
CN108876861B CN108876861B (en) 2020-10-20

Family

ID=64333054

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201810513673.5A Expired - Fee Related CN108876861B (en) 2018-05-25 2018-05-25 Stereo matching method for extraterrestrial celestial body patrolling device

Country Status (1)

Country Link
CN (1) CN108876861B (en)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111739083A (en) * 2020-07-02 2020-10-02 深兰人工智能芯片研究院（江苏）有限公司 Matching method and device
CN111768434A (en) * 2020-06-29 2020-10-13 Oppo广东移动通信有限公司 Disparity map acquisition method and device, electronic equipment and storage medium
CN111881985A (en) * 2020-07-30 2020-11-03 中国空间技术研究院 Stereo matching method, device, terminal and storage medium
CN111898396A (en) * 2019-05-06 2020-11-06 北京四维图新科技股份有限公司 Obstacle detection method and device
CN113034666A (en) * 2021-02-01 2021-06-25 中国计量大学 Stereo matching method based on pyramid parallax optimization cost calculation

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102881014A (en) * 2012-09-07 2013-01-16 航天恒星科技有限公司 Quick stereo matching method based on graph cut
CN103679680A (en) * 2012-08-31 2014-03-26 富士通株式会社 Stereo matching method and system
CN104331897A (en) * 2014-11-21 2015-02-04 天津工业大学 Polar correction based sub-pixel level phase three-dimensional matching method
CN106447601A (en) * 2016-08-31 2017-02-22 中国科学院遥感与数字地球研究所 Unmanned aerial vehicle remote image mosaicing method based on projection-similarity transformation
CN107016698A (en) * 2017-03-20 2017-08-04 深圳格兰泰克汽车电子有限公司 Based on tapered plane smooth binocular solid matching process and device

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103679680A (en) * 2012-08-31 2014-03-26 富士通株式会社 Stereo matching method and system
CN102881014A (en) * 2012-09-07 2013-01-16 航天恒星科技有限公司 Quick stereo matching method based on graph cut
CN104331897A (en) * 2014-11-21 2015-02-04 天津工业大学 Polar correction based sub-pixel level phase three-dimensional matching method
CN106447601A (en) * 2016-08-31 2017-02-22 中国科学院遥感与数字地球研究所 Unmanned aerial vehicle remote image mosaicing method based on projection-similarity transformation
CN107016698A (en) * 2017-03-20 2017-08-04 深圳格兰泰克汽车电子有限公司 Based on tapered plane smooth binocular solid matching process and device

Non-Patent Citations (6)

* Cited by examiner, † Cited by third party
Title
F. QAMAR ET AL.: ""High-level synthesis for semi-global matching: Is the juice worth the squeeze?""", 《IEEE ACCESS》 *
K. R. KIM ET AL.: ""Multiscale feature extractors for stereo matching cost computation"", 《IEEE ACCESS》 *
张晗玥: ""基于Census变换的区域立体匹配算法研究"", 《中国优秀硕士学位论文全文数据库 信息科技辑》 *
林义闽等: ""用于弱纹理三维场景重建的机器人视觉系统"", 《万方数据知识服务平台》 *
殷莉甜: ""水下双目视觉定位系统开发与应用研究"", 《中国优秀硕士学位论文全文数据库（电子期刊）》 *
郭瑞斌: ""基于图像的大尺度地面目标三维重建技术研究"", 《中国优秀硕士学位论文全文数据库（电子期刊）》 *

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111898396A (en) * 2019-05-06 2020-11-06 北京四维图新科技股份有限公司 Obstacle detection method and device
CN111898396B (en) * 2019-05-06 2024-08-09 北京四维图新科技股份有限公司 Obstacle detection method and device
CN111768434A (en) * 2020-06-29 2020-10-13 Oppo广东移动通信有限公司 Disparity map acquisition method and device, electronic equipment and storage medium
CN111739083A (en) * 2020-07-02 2020-10-02 深兰人工智能芯片研究院（江苏）有限公司 Matching method and device
CN111881985A (en) * 2020-07-30 2020-11-03 中国空间技术研究院 Stereo matching method, device, terminal and storage medium
CN111881985B (en) * 2020-07-30 2024-04-30 中国空间技术研究院 Stereo matching method, device, terminal and storage medium
CN113034666A (en) * 2021-02-01 2021-06-25 中国计量大学 Stereo matching method based on pyramid parallax optimization cost calculation
CN113034666B (en) * 2021-02-01 2023-09-12 中国计量大学 Stereo matching method based on pyramid parallax optimization cost calculation

Also Published As

Publication number Publication date
CN108876861B (en) 2020-10-20

Similar Documents

Publication Publication Date Title
CN108876861A (en) A kind of solid matching method of objects outside Earth rover
CN110058237B (en) InSAR point cloud fusion and three-dimensional deformation monitoring method for high-resolution SAR image
CN102693542B (en) Image characteristic matching method
CN104820991B (en) A kind of multiple soft-constraint solid matching method based on cost matrix
CN106683182A (en) 3D reconstruction method for weighing stereo matching and visual appearance
CN112634341A (en) Method for constructing depth estimation model of multi-vision task cooperation
CN105160702A (en) Stereoscopic image dense matching method and system based on LiDAR point cloud assistance
CN115564926B (en) Three-dimensional patch model construction method based on image building structure learning
CN103822616A (en) Remote-sensing image matching method with combination of characteristic segmentation with topographic inequality constraint
CN111462302B (en) Multi-view human body dynamic three-dimensional reconstruction method and system based on depth coding network
CN109887021A (en) Based on the random walk solid matching method across scale
CN112991420A (en) Stereo matching feature extraction and post-processing method for disparity map
CN112435267B (en) Disparity map calculation method for high-resolution urban satellite stereo image
CN111508030A (en) Stereo matching method for computer vision
CN102740096A (en) Space-time combination based dynamic scene stereo video matching method
CN105279522A (en) Scene object real-time registering method based on SIFT
CN115128628A (en) Road grid map construction method based on laser SLAM and monocular vision
CN104751451B (en) Point off density cloud extracting method based on unmanned plane low latitude high resolution image
CN115482268A (en) High-precision three-dimensional shape measurement method and system based on speckle matching network
CN116051758A (en) Height information-containing landform map construction method for outdoor robot
CN117557617B (en) Multi-view dense matching method, system and equipment based on plane priori optimization
CN111197976A (en) Three-dimensional reconstruction method considering multi-stage matching propagation of weak texture region
Yong-guo et al. The navigation of mobile robot based on stereo vision
Chen et al. Ground 3D object reconstruction based on multi-view 3D occupancy network using satellite remote sensing image
CN117315153A (en) Human body reconstruction and rendering method and device for cooperative light field and occupied field

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
CF01 Termination of patent right due to non-payment of annual fee
CF01 Termination of patent right due to non-payment of annual fee

Granted publication date: 20201020

Termination date: 20210525