CN107403448A - Cost function generation method and cost function generating means - Google Patents
Cost function generation method and cost function generating means Download PDFInfo
- Publication number
- CN107403448A CN107403448A CN201710618382.8A CN201710618382A CN107403448A CN 107403448 A CN107403448 A CN 107403448A CN 201710618382 A CN201710618382 A CN 201710618382A CN 107403448 A CN107403448 A CN 107403448A
- Authority
- CN
- China
- Prior art keywords
- pixel
- parallax
- grad
- gradient
- weights
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/50—Depth or shape recovery
- G06T7/55—Depth or shape recovery from multiple images
- G06T7/593—Depth or shape recovery from multiple images from stereo images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
- G06T7/32—Determination of transform parameters for the alignment of images, i.e. image registration using correlation-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10024—Color image
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Image Processing (AREA)
- Measurement Of Optical Distance (AREA)
Abstract
The present invention is included on a kind of cost function generation method and cost function generating means, the above method：Depth map based on two depth camera shooting Same Scenes obtains the gradient map of the scene；According to the Grad of the first pixel in the gradient map, with the ratio of the Grad sum of the intraoral all pixels of datum windows where first pixel, the weights of first pixel are calculated, wherein, the size of the weights and the size correlation of the ratio；Gray value and the weights according to the pixel in the benchmark window and Corresponding matching window where first pixel, calculate the cost function of first pixel.According to an embodiment of the invention, the pixel in window is weighted based on gradient, participated in without artificial, reduce the complexity that weights are set, ensure to set larger weights for the larger pixel of gradient, and then can more accurately calculate the parallax value of pixel, and it is applied widely.
Description
Technical field
The present invention relates to Stereo Matching Technology field, more particularly to cost function generation method and cost function generation dress
Put, terminal and computer-readable recording medium.
Background technology
During Stereo matching, in order to determine to match parallax of the image relative to benchmark image, in benchmark image
Each pixel benchmark window can be set, for matching image in each pixel window to be matched can be set, by
Mobile window to be matched in image is matched, and in the cost value determination and benchmark image of cost function according to corresponding to two windows
The corresponding pixel of pixel, and calculate the parallax between two pixels.
In order to which reasonably calculation cost value, main at present use are based on distance and based on two kinds of weighting schemes of color to window
In each pixel be weighted, but the weight relationship in both weighting schemes in window between each pixel is all artificially to set
Fixed, it is required for repeatedly being tested for different images, just can determine that rational weight relationship, ensures feature in window
Obvious pixel has a higher weight, and weighting procedure is cumbersome, and it is possible to malfunctions.
The content of the invention
The present invention provides cost function generation method and cost function generating means, terminal and computer-readable storage medium
Matter, to solve the deficiency in correlation technique.
First aspect according to embodiments of the present invention, there is provided a kind of cost function generation method, including：
Depth map based on two depth camera shooting Same Scenes obtains the gradient map of the scene；
According to the intraoral institute of datum windows where the Grad of the first pixel in the gradient map, and first pixel
There is the ratio of the Grad sum of pixel, calculate the weights of first pixel, wherein, the size of the weights and the ratio
The size correlation of value；
According to the parallax value of the pixel in the benchmark window and Corresponding matching window where first pixel and institute
Weights are stated, calculate the cost function of first pixel.
Second aspect according to embodiments of the present invention, there is provided a kind of cost function generating means, including：
Image acquisition unit, the ladder of the scene is obtained for the depth map based on two depth camera shooting Same Scenes
Degree figure；
Weight calculation unit, for the Grad according to the first pixel in the gradient map, and first pixel
The ratio of the Grad sum of the intraoral all pixels of datum windows at place, the weights of first pixel are calculated, wherein, it is described
The size of weights and the size correlation of the ratio；
Cost function generation unit, for according in the benchmark window and Corresponding matching window where first pixel
Pixel parallax value and the weights, calculate the cost function of first pixel.
The third aspect according to embodiments of the present invention, there is provided a kind of terminal, including memory, processor and it is stored in storage
On device and the computer program that can run on a processor, it is characterised in that described in the computing device during computer program
Realize following steps：
Depth map based on two depth camera shooting Same Scenes obtains the gradient map of the scene；
According to the intraoral institute of datum windows where the Grad of the first pixel in the gradient map, and first pixel
There is the ratio of the Grad sum of pixel, calculate the weights of first pixel, wherein, the size of the weights and the ratio
The size correlation of value；
According to the parallax value of the pixel in the benchmark window and Corresponding matching window where first pixel and institute
Weights are stated, calculate the cost function of first pixel.
Fourth aspect according to embodiments of the present invention, there is provided a kind of computer-readable recording medium, be stored thereon with calculating
Machine program, the computer program realize following steps when being executed by processor：
Depth map based on two depth camera shooting Same Scenes obtains the gradient map of the scene；
According to the intraoral institute of datum windows where the Grad of the first pixel in the gradient map, and first pixel
There is the ratio of the Grad sum of pixel, calculate the weights of first pixel, wherein, the size of the weights and the ratio
The size correlation of value；
According to the parallax value of the pixel in the benchmark window and Corresponding matching window where first pixel and institute
Weights are stated, calculate the cost function of first pixel.
From above-described embodiment, can be based on based on distance weighted mode, the present embodiment relative in correlation technique
Gradient is weighted to the pixel in window, can set weights, nothing automatically after the gradient of each pixel during window is determined
It need to manually participate in, reduce the complexity that weights are set, even if scene changes, the method for the present embodiment is equally applicable,
Without being manually adjusted, the scope of application is wider.
The pixel in window can be weighted based on gradient relative to based on color-weighted mode, the present embodiment,
The gradient of each pixel in window is determined can set weights automatically afterwards, can be relatively easily without carrying out color segmentation
Determine the gradient of pixel, the low complexity that weights are set, and the method for the present embodiment is applicable not only to coloured image, for
Various images are applicable, and the scope of application is more extensive.
Further, since this implementation is to set weights according to gradient, and the gradient of pixel is the gray scale according to the pixel
What the relation of the gray value of value and its surrounding pixel point was calculated, if the gradient of some pixel is bigger, then illustrate its phase
It is more obvious for the pixel point feature around it, the pixel being more likely to be on the side in image, therefore by the pixel
Weight is set larger, and in calculation cost function, it can also have larger weights so that its knot for cost function
Fruit has a great influence, and then the pixel in matching image is determined according to the result of cost function with matching in benchmark image
When, it is bigger to match the probability to match in image with the pixel in benchmark image on side, so as to calculate exactly on side
Pixel parallax value, be more beneficial for subsequently carrying out the Stereo matching based on side based on the parallax value.
It should be appreciated that the general description and following detailed description of the above are only exemplary and explanatory, not
Can the limitation present invention.
Brief description of the drawings
Accompanying drawing herein is merged in specification and forms the part of this specification, shows the implementation for meeting the present invention
Example, and for explaining principle of the invention together with specification.
Fig. 1 is a kind of schematic flow diagram of cost function generation method according to one embodiment of the invention.
Fig. 2 is a kind of schematic diagram of Filtering Template according to one embodiment of the invention.
Fig. 3 is a kind of schematic diagram of gradient template according to one embodiment of the invention.
Fig. 4 is the schematic diagram of another gradient template according to one embodiment of the invention.
Fig. 5 is a kind of schematic diagram of window according to one embodiment of the invention.
Fig. 6 is the schematic diagram of the Grad of pixel in a benchmark window according to one embodiment of the invention.
Fig. 7 is the schematic diagram of the weights of pixel in a benchmark window according to one embodiment of the invention.
Fig. 8 is the schematic diagram of the Grad of pixel in a match window according to one embodiment of the invention.
Fig. 9 is the schematic diagram based on pixel weights in distance weighted benchmark window in correlation technique.
Figure 10 is based on color-weighted schematic diagram in correlation technique.
Figure 11 is the schematic diagram of the low texture region according to one embodiment of the invention.
Figure 12 is the schematic diagram that low texture region according to one embodiment of the invention corresponds to cost function.
Figure 13 is the schematic flow diagram of another cost function generation method according to one embodiment of the invention.
Figure 14 is the schematic flow diagram of another cost function generation method according to one embodiment of the invention.
Figure 15 is the benchmark image according to one embodiment of the invention.
Figure 16 is that the Stereo matching disparity map based on side is shown according to one embodiment of the invention.
Figure 17 is a kind of schematic block diagram of cost function generating means according to one embodiment of the invention.
Figure 18 is a kind of schematic block diagram of weight calculation unit according to one embodiment of the invention.
Figure 19 is the schematic block diagram of another weight calculation unit according to one embodiment of the invention.
Embodiment
Here exemplary embodiment will be illustrated in detail, its example is illustrated in the accompanying drawings.Following description is related to
During accompanying drawing, unless otherwise indicated, the same numbers in different accompanying drawings represent same or analogous key element.Following exemplary embodiment
Described in embodiment do not represent and the consistent all embodiments of the present invention.On the contrary, they be only with it is such as appended
The example of the consistent apparatus and method of some aspects being described in detail in claims, of the invention.
In order to make it easy to understand, before to the embodiment of the present invention carrying out that explanation is explained in detail, first to the embodiment of the present invention
The noun and application scenarios being related to are introduced.
Filtering
Filtering in the present embodiment refers to the filtering for image, in particular to the bar in reservation image detail feature as far as possible
The processing method suppressed under part to the noise of image.Wherein, after the treatment effect quality of image filtering will directly influence
Continuous image procossing and the validity and reliability of analysis.Such as nonlinear filtering or medium filtering etc..
Wherein, nonlinear filtering refers to the processing method to a kind of Nonlinear Mapping relation of input signal, such as, can be with
A certain specific noise is approx mapped as the principal character of zero and stick signal.Medium filtering refers in image or sequence
A kind of processing method that the value of heart point position is substituted with the intermediate value in the region.
Gradient map
Gradient in mathematics obtains according to directional derivative, and for the depth map in the present embodiment, then can root
Be worth to according to the gray scale of depth map, the gray value yet with each pixel in depth map be it is discrete, can not be according to mathematics
The mode of middle computing differential obtains gradient, but can use with differential similar in mode, namely difference calculated, and specifically may be used
With the gray value by calculating some pixel relative to the difference of the gray value of the pixel around it, to determine the pixel
Gradient, and then the Grad of each pixel in image is calculated, you can obtain corresponding gradient map.Wherein it is possible to adopt
Above-mentioned difference is calculated with Sobel operators, or Prewitt operators.
Benchmark window and match window
The two width depth maps obtained based on two depth camera shooting Same Scenes, with any one width in this two width depth map
Figure is used as benchmark image, and another width is as matching image.Centered on some pixel in benchmark image, multiple pixels are included
Window be referred to as benchmark window, centered on matching image graph some pixel, the window comprising multiple pixels is referred to as matching
Window.For example, benchmark window can include 9 pixels, 9 pixels form 3 × 3 matrixes, can also include 25 pixels
Point, 25 pixels form 5 × 5 matrixes.Certainly, benchmark window specifically includes the quantity of pixel, can carry out as needed
Adjustment, and shape is not limited to square, and the number and shape of pixel included by match window then need to keep and datum windows
Mouth is identical.
Cost function
Cost function can be divided into otherness cost function and similitude cost function, wherein, otherness cost function, example
Such as the absolute value and function (SAD), the sum of squares function (SSD) of pixel difference etc. of pixel difference, mainly pass through calculating benchmark image
In pixel and matching image in pixel gray level otherness, the value of otherness cost function is smaller, then benchmark image
In pixel and matching image in the possibility that matches of pixel it is higher；Similitude cost function, such as zero-mean ash
Degree and correlation coefficient function (ZCC), Normalized Grey Level cross-correlation function (NCC) etc., mainly by calculating benchmark image
The similitude of pixel and the pixel gray level in matching image, the value of similitude cost function is bigger, then in benchmark image
The possibility that pixel in pixel and matching image matches is higher.
Fig. 1 is a kind of schematic flow diagram of cost function generation method according to one embodiment of the invention, the generation
Valency function generation method goes in the processor of stereo matching system, and stereo matching system can also include two-sector model picture
Collecting device, one is used to gather left image, and one is used to gather right image, and one of image is another as benchmark image
Individual image is as matching image.As shown in figure 1, the cost function generation method includes：
S1, the depth map based on two depth camera shooting Same Scenes obtain the gradient map of the scene.
In one embodiment, the image of the camera shooting in two depth cameras can be used as benchmark image, separately
The image of one camera shooting can be used as matching image., first can be to benchmark image and matching in order to obtain gradient image
Image is filtered smoothing processing, to remove the point of the noise pixel in image.Wherein it is possible to gaussian filtering is gathered respectively to benchmark
View and match views are filtered smoothing processing.
Fig. 2 is a kind of schematic diagram of Filtering Template according to one embodiment of the invention.
In one embodiment, 3 × 3 Filtering Template as shown in Figure 2 can be used, to benchmark image and movement images
In each pixel carry out gaussian filtering, for the sake of convenient, filtered benchmark image might as well be designated as imgR, will be filtered
Movement images are designated as imgL.
Second, in order to increase the robustness of later stage Stereo matching, after gaussian filtering, can respectively to benchmark image and
Movement images carry out gradient processing, and the half-tone information processing by pixel in benchmark image and movement images is pixel gradient information.
Namely the Grad of pixel in the benchmark image is determined according to the gray value of pixel in the benchmark image, according to the matching
The gray value of pixel determines the Grad of pixel in the matching image in image.
Fig. 3 is a kind of schematic diagram of gradient template according to one embodiment of the invention.Fig. 4 is according to the present invention one
The schematic diagram of the individual another gradient template for implementing to exemplify.
In one embodiment, gradient processing can use as shown in Figure 31 × 3 gradient template, can also use as schemed
3 × 3 gradient template shown in 4, wherein, Fig. 3 and gradient template shown in Fig. 4 are transverse gradients template, i.e., only calculate figure
As transverse gradients.For each pixel in imgL and imgR, carried out according to above-mentioned Fig. 3 or Fig. 4 gradient template at gradient
Reason, obtain respectively based on the gradient map of benchmark image as imGradR and based on the gradient image imGradL for matching image.
By handling two images respectively as gradient map, because the gradient of each pixel in gradient map is according to the picture
The relation of the gray value of vegetarian refreshments and the pixel around it obtains, and the relation can't be because of two depth camera inner parameters
Different or shooting angle differs greatly or ambient light difference is larger and be affected.And then according to gradient map in subsequent process
Stereo matching is carried out, can be evaded because two depth camera inner parameters are incomplete same, or shooting angle differs greatly, or clap
Ambient light both ends differ greatly when taking the photograph, and cause the gray value of same pixel in two images different, cause subsequently standing
There is error hiding when matching in body, i.e., is matched according to the Grad of pixel, is carried out with respect to the gray value of pixel
It is stronger to match robustness.
Step S2, according to the benchmark where the Grad of the first pixel in the gradient map, and first pixel
The ratio of the Grad sum of all pixels in window, the weights of first pixel are calculated, wherein, the size of the weights
With the size correlation of the ratio.
In one embodiment, the gradient map for benchmark image and the gradient map of matching image, can be in benchmark image
Gradient map in determine reference image vegetarian refreshments, centered on reference image vegetarian refreshments determine benchmark window, matching image gradient map in
Pixel to be matched is determined, window to be matched is determined centered on pixel to be matched.Benchmark window and window to be matched can be with
It is n × n window, namely includes the window of n × n pixel.
Fig. 5 is a kind of schematic diagram of window according to one embodiment of the invention.
In one embodiment, as shown in figure 5, image on the basis of left-side images, black rectangle frame therein is datum windows
Mouthful, reference image vegetarian refreshments is located at the center of benchmark window.Image right is matching image, and black rectangle frame therein is window to be matched
Mouthful, pixel to be matched is located at the center of window to be matched, and white rectangle frame therein illustrates to move in default disparity range
Move window to be matched.Certainly, in actual application, the size of window and default disparity range can be set as needed
Put, the size of window can with the difference shown in Fig. 5, and the scope of window to be matched movement can also with shown in Fig. 5 not
Together.
Fig. 6 is the schematic diagram of the Grad of pixel in a benchmark window according to one embodiment of the invention.Fig. 7
It is the schematic diagram of the weights of pixel in a benchmark window according to one embodiment of the invention.
In one embodiment, as shown in figure 5, in a benchmark window Grad of pixel can be followed successively by 3,0,1,
1、4、0、2、5、4.The Grad sum of all pixels is 3+0+1+1+4+0+2+5+4=20 in the window, then can basisThe weights of each pixel are calculated, wherein, λ(x,y)On the basis of in window coordinate for the pixel of (x, y) weights,
VsumROn the basis of in window all pixels Grad sum, V(x,y)On the basis of in window coordinate for the pixel of (x, y) gradient
Value, in the case where it is 0, it is set to 0.5.So based on the embodiment shown in Fig. 6, wherein each pixel weights such as Fig. 7
It is shown.
In one embodiment, for the first pixel in benchmark image, can using the first pixel (such as coordinate as
(x, y)) centered on take a stationary window to include n × n pixel as benchmark window, benchmark window；For in matching image
In preset disparity range (disparity range is lateral coordinates x span, such as x spans are [x+dmin, x+dmax], dmin
And dmaxBut it is default value, for example, it can be set to dmin=0, dmax=100, so that adjacent pixel differs a parallax as an example, that
Default disparity range is exactly 100 pixels) in each pixel to be matched, centered on pixel to be matched, take one it is same
The stationary window of size, namely window to be matched, window to be matched also include n × n pixel.
Step S3, according to the gray scale of the pixel in the benchmark window and Corresponding matching window where first pixel
Value and the weights, calculate the cost function of first pixel.
In one embodiment, after window is built, according to the pixel in each benchmark window, and in match window
The gray value of pixel, and the weights of each pixel, can generate cost function.Cost function can be zero-mean gray scale and phase
Pass coefficient function (ZCC), Normalized Grey Level cross-correlation function (NCC), the absolute value and function (SAD) of pixel difference, pixel difference
Sum of squares function (SSD) etc..
In the case where the cost function is zero-mean gray scale and correlation coefficient function, cost function value
In the case where the cost function is Normalized Grey Level cross-correlation function, cost function
In the case where the cost function is the absolute value and function of pixel difference, cost function
In the case where the cost function is the sum of squares function of pixel difference, cost function
Wherein, λ(x,y)On the basis of in window coordinate for the benchmark pixel of (x, y) weights,VsumRFor base
The Grad sum of all pixels, V in quasi- window(x,y)On the basis of in window coordinate for the benchmark pixel of (x, y) Grad,
In the case that it is 0, it is window to be set to 0.5, W, I1(x, y) represents in benchmark image coordinate as the pixel of (x, y)
Gray scale, I2(x+d, y) represent matching image in coordinate be the pixel of (x+d, y) gray scale, d expression parallax, avgW1Represent benchmark
Pixel grey scale average in window, avgW2Represent pixel average average in window to be matched.
It should be noted that except above-mentioned four kinds of cost functions, the embodiment of the present invention can be applicable to other cost letters
Number, such as ZNCC, ZSSD, ZSAD (Z in foregoing several cost functions refers to zeromean, namely the NCC of zero mean,
SSD, SAD algorithm) and CENSUS become the cost functions such as scaling method (be used for calculate Hamming distance).
It is mainly illustrative to the present embodiment by taking SAD as an example below.
Fig. 8 is the schematic diagram of the Grad of pixel in a match window according to one embodiment of the invention.
For window shown in Fig. 8, the value that its cost function is calculated according to above-mentioned SAD formula is | 3-4 | * 0.15+ | and 0-1
|*0.025+|1-2|*0.05+|1-2|*0.05+|4-6|*0.20+|0-1|*0.025+|2-1|*0.10+|5-3|*0.25+|
4-2 | * 0.20=1.7.
And then in this mode, the value of each first pixel calculation cost function can be directed to, and determine cost function
Value minimum when corresponding parallax.And then anaglyph can be generated according to benchmark image and the parallax determined.
In the value of calculation cost function, in order to improve the degree of accuracy of cost value calculating, it is necessary to improve the obvious picture of feature
The weight of element.The mode of weights is set in presently relevant technology mainly to be included based on distance weighted and based on color-weighted.
It is pixel in benchmark window and the different distance of benchmark window center pixel based on distance weighted mode,
Different weights is set, and distance center pixel is nearer, then weight is bigger.Fig. 9 is based on distance weighted benchmark in correlation technique
The schematic diagram of pixel weights in window.
As shown in figure 9, in 3 × 3 benchmark window, it is necessary to artificially set center pixel weight highest 0.5, and its away from
From the pixel for 1, i.e., on the left of center pixel, right side, the weight of bottom and upper segment is 0.3, and distance isPixel, i.e., four
The pixel of Angle Position, it is 0.2 to set weight.The mode of this set weights is manually carried out, and is relied solely in distance
The distance of imago element sets weight, it is therefore desirable to repeatedly test, set with reference to effect and empirical value to complete weight, it is more numerous
It is trivial.And it can not necessarily ensure that the obvious pixel of feature has higher weight, because the obvious pixel of feature is except window
The larger pixel of gradient in center pixel, usually window, but in the way of shown in Fig. 9, the pixel that weights are 0.2 is actual
On gradient might not than weights be 0.3 pixel gradient it is small.
Relative to this weighting scheme, the present embodiment can be weighted based on gradient to the pixel in window, it is determined that
In window weights can be set after the gradient of each pixel automatically, be participated in without artificial, reduce the complexity that weights are set,
Even if scene changes, the method for the present embodiment is equally applicable, and without being manually adjusted, the scope of application is wider.
Based on color-weighted mode, color-weighted Stereo matching is carried out mainly for colored binocular image, is needed simultaneously
Color segmentation first is carried out to benchmark image and matching image, then in Stereo matching, to the face where center pixel in window
The weight in color region to be set larger, and the weights of other color regions will set smaller.Figure 10 is base in correlation technique
In color-weighted schematic diagram, middle graph is artwork, and left hand view is color segmentation figure, and right part of flg is based on color-weighted figure.
As shown in Figure 10, in 7 × 7 grid matrix, window on the basis of the grid at center, the picture of benchmark window center
Element, i.e. benchmark pixel, from left hand view and middle graph, the pixel of benchmark window center is located at same face with the pixel on the right side of it
Color region, therefore the weight of its right pixel will be set larger, such as 0.5, and pixel and center pixel on the left of it
Point is not belonging to same color region, and its weight sets smaller, such as 0.1.The mode that this weights are set is, it is necessary to first to figure
As carrying out color segmentation, but the computation complexity of color segmentation is high, and the segmentation effect of algorithm is typically all poor at present；And
This kind of mode has preferable effect just for coloured image, and gray level image positive effect is reduced, and the scope of application is smaller；In addition
Weight similarly needs artificial setting, it is necessary to constantly be adjusted according to effect and empirical value, relatively complicated.
Relative to this weighting scheme, the present embodiment can be weighted based on gradient to the pixel in window, it is determined that
In window weights can be set after the gradient of each pixel automatically, without carrying out color segmentation, can relatively easily determine
The gradient of pixel, the low complexity that weights are set, and the method for the present embodiment is applicable not only to coloured image, for various
Image is applicable, and the scope of application is more extensive.
Further, since this implementation is to set weights according to gradient, and the gradient of pixel is the gray scale according to the pixel
What the relation of the gray value of value and its surrounding pixel point was calculated.If the gradient of some pixel is bigger, then illustrates its phase
Pixel more obvious for the pixel point feature around it, being more likely to be on the side in image (namely contour line of object)
Point.
Therefore by the larger of the weight setting of the pixel, in calculation cost function, it can also have larger power
Value so that it has a great influence for the result of cost function, and then according to the result of cost function determine matching image in
During the pixel to match in benchmark image, the probability to match in image with the pixel in benchmark image on side is matched more
Greatly, so as to calculate the parallax value of the pixel on side exactly, it is more beneficial for subsequently carrying out being based on side based on the parallax value
Stereo matching.
Figure 11 is the schematic diagram of the low texture region according to one embodiment of the invention.Figure 12 is according to the present invention one
The individual low texture region for implementing to exemplify corresponds to the schematic diagram of cost function.
In one embodiment, in the accompanying drawing shown in Figure 11, two pieces of a-quadrants are low texture region.
The low texture region for more than, selects a pixel in the respectively region, carries out the solid based on stationary window
Matching, obtained SSD cost function curvilinear motions are as shown in figure 12, by observing the curve shown in Figure 12 it can be found that low line
It is that pixel is generally relatively low in Stereo matching Time Value to manage regional characteristics, multiple local minizing points be present, and is occurred local
The cycle of minimum point is small.But the curve of ideal cost function, minimum number should be less, and with preferable
Monotonicity, it is clear that simultaneously condition is not satisfied for low texture region, and this can cause to be difficult to accurately determine for low texture region to regard
Difference.
Figure 13 is the schematic flow diagram of another cost function generation method according to one embodiment of the invention.Such as
Shown in Figure 13, on the basis of embodiment illustrated in fig. 1, the Grad according to the first pixel in the gradient map, and institute
The ratio of the Grad sum of the intraoral all pixels of datum windows where stating the first pixel, calculate the power of first pixel
Value, it is specially：
Step S21, if the Grad sum of the intraoral pixel of the datum windows of the first pixel is less than in the gradient map
Predetermined gradient value, it is determined that the parallax of first pixel is invalid parallax；If the base of the first pixel in the gradient map
The Grad sum of the pixel is more than or equal to predetermined gradient value in quasi- window, it is determined that the parallax of first pixel is to have
Imitate parallax；
Step S22, if the parallax of first pixel is effective parallax, according to the first pixel in the gradient map
Grad, and the ratio of the Grad sum of intraoral all pixels of datum windows where first pixel, described in calculating
The weights of first pixel.
In one embodiment, for benchmark window, if the Grad sum of wherein all pixels point is smaller (namely small
In the first predetermined gradient value), then it is larger can to determine that pixel corresponding to the window (namely pixel of window center) has
Probability belong to low texture region.
And it can be seen from Figure 11 and Figure 12 analysis, it is difficult to accurately determine parallax for low texture region, therefore counting
Before calculating cost function value, the pixel of low texture region is belonged to for determination, it may be determined that its parallax is invalid parallax, and
During follow-up calculating weights, corresponding cost function value is calculated for the first pixel of effective parallax only for parallax, is easy to follow-up
Parallax can be accurately determined for the pixel that weights are calculated, and then accurately generates the Stereo matching parallax based on side
Image.
Figure 14 is the schematic flow diagram of another cost function generation method according to one embodiment of the invention.Such as
Shown in Figure 14, on the basis of embodiment illustrated in fig. 1, the Grad according to the first pixel in the gradient map, and institute
The ratio of the Grad sum of the intraoral all pixels of datum windows where stating the first pixel, calculate the power of first pixel
Value, it is specially：
Step S23, if the intraoral Grad of the datum windows of the first pixel is more than the picture of predetermined gradient value in the gradient map
The quantity of vegetarian refreshments is less than predetermined number, and the parallax for determining first pixel is invalid parallax；If first in the gradient map
The quantity that the intraoral Grad of datum windows of pixel is more than the pixel of predetermined gradient value is more than or equal to predetermined number, it is determined that described
The parallax of first pixel is effective parallax；
Step S24, if the parallax of first pixel is effective parallax, according to the first pixel in the gradient map
Grad, and the ratio of the Grad sum of intraoral all pixels of datum windows where first pixel, described in calculating
The weights of first pixel.
In one embodiment, for benchmark window, if the ladder of wherein more (namely more than preset number) pixel
Angle value is smaller (namely no more than predetermined gradient value), then can determine (namely the window center of pixel corresponding to the window
Pixel) there is larger probability to be in low texture region.
And it can be seen from Figure 11 and Figure 12 analysis, it is difficult to accurately determine its parallax for low texture region, therefore
Before calculation cost functional value, the pixel of low texture region is belonged to for determination, it may be determined that its parallax is invalid parallax, and
When subsequently calculating weights, corresponding cost function value is calculated for the first pixel of effective parallax only for parallax, after being easy to
The continuous pixel for weights are calculated can accurately determine parallax, and then accurately generate the Stereo matching based on side and regard
Difference image.
In one embodiment, can be further right for generating anaglyph according to benchmark image and the parallax determined
The anaglyph is filtered smoothing processing.
In one embodiment, by matching corresponding parallax in each pixel of benchmark image, it can generate and regard
Difference image, for the anaglyph, can by left and right consistency detection (i.e. on the basis of left image image, using right figure as
Parallax is calculated to generate anaglyph, then the image on the basis of right image with image, and parallax is calculated by matching image of left image
To generate anaglyph, then compare the diversity factor of identical pixel in two width anaglyphs of generation), determine error hiding
The parallax of pixel (namely larger pixel of diversity factor during the consistency detection of left and right) is invalid parallax, then in
Value filtering or mean filter, smoothing processing, the picture of invalid parallax are filtered to the pixel of effective parallax in anaglyph
Vegetarian refreshments does not process, the anaglyph dispIm after being post-processed.
In one embodiment, it can also determine that filtering is handled according to the Grad of pixel in said reference image
The edge pixel in anaglyph afterwards, the Stereo matching anaglyph based on side is generated according to the edge pixel.
In one embodiment, for carrying out the image imGradR after gradient processing according to Fig. 3 or embodiment illustrated in fig. 4,
Grads threshold T can be arranged as required to, if the gradient of the pixel of a certain position is less than T in image imGradR, corresponds to and regards
In difference image dispIm, the parallax value of the position is entered as 0, otherwise, parallax value retains, and travels through all pictures in imGradR
Vegetarian refreshments, the Stereo matching anaglyph based on side is finally given, for detection identification of the later stage based on anaglyph.
Figure 15 is the benchmark image according to one embodiment of the invention.Figure 16 is shown according to one embodiment of the invention
Go out the Stereo matching disparity map based on side.
In one embodiment, according to above-described embodiment, benchmark image as shown in figure 15 can be handled, so as to
The edge pixel for wherein belonging to edge is accurately determined, and then is accurately generated according to edge pixel as shown in figure 16 based on side
Stereo matching disparity map.
With the embodiment of above-mentioned cost function generation method accordingly, the invention also provides cost function generating means
Embodiment.
Figure 17 is a kind of schematic block diagram of cost function generating means according to one embodiment of the invention.Such as Figure 17
Shown, the cost function generating means include：
Image acquisition unit 1, the scene is obtained for the depth map based on two depth camera shooting Same Scenes
Gradient map；
Weight calculation unit 2, for the Grad according to the first pixel in the gradient map, and first pixel
The ratio of the Grad sum of the intraoral all pixels of datum windows at place, the weights of first pixel are calculated, wherein, it is described
The size of weights and the size correlation of the ratio；
Cost function generation unit 3, for according to the benchmark window and Corresponding matching window where first pixel
The parallax value of interior pixel and the weights, calculate the cost function of first pixel.
Figure 18 is a kind of schematic block diagram of weight calculation unit according to one embodiment of the invention.Such as Figure 18 institutes
Show, the weight calculation unit specifically includes：
Invalid parallax determination subelement 21, if the intraoral pixel of datum windows for the first pixel in the gradient map
Grad sum be less than predetermined gradient value, it is determined that the parallax of first pixel is invalid parallax；
Effective parallax determination subelement 22, if the intraoral pixel of datum windows for the first pixel in the gradient map
Grad sum be more than or equal to predetermined gradient value, it is determined that the parallax of first pixel is effective parallax；
Weight computing subelement 23, if the parallax for first pixel is effective parallax, according to the gradient
The Grad sum of the intraoral all pixels of datum windows where the Grad of first pixel in figure, and first pixel
Ratio, calculate the weights of first pixel.
Figure 19 is the schematic block diagram of another weight calculation unit according to one embodiment of the invention.Such as Figure 19 institutes
Show, the weight calculation unit specifically includes：
Invalid parallax determination subelement 24, if the intraoral Grad of datum windows for the first pixel in the gradient map is big
It is less than predetermined number in the quantity of the pixel of predetermined gradient value, the parallax for determining first pixel is invalid parallax；
Effective parallax determination subelement 25, if the intraoral Grad of datum windows for the first pixel in the gradient map is big
It is more than or equal to predetermined number in the quantity of the pixel of predetermined gradient value, the parallax for determining first pixel is effectively to regard
Difference；
Weight computing subelement 26, if the parallax for first pixel is effective parallax, according to the gradient
The Grad sum of the intraoral all pixels of datum windows where the Grad of first pixel in figure, and first pixel
Ratio, calculate the weights of first pixel.
On the device in above-described embodiment, wherein unit performs the concrete mode of operation in relevant this method
Embodiment in be described in detail, explanation will be not set forth in detail herein.
For device embodiment, because it corresponds essentially to embodiment of the method, so related part is real referring to method
Apply the part explanation of example.Device embodiment described above is only schematical, wherein described be used as separating component
The unit of explanation can be or may not be physically separate, can be as the part that unit is shown or can also
It is not physical location, you can with positioned at a place, or can also be distributed on multiple NEs.Can be according to reality
Need to select some or all of module therein to realize the purpose of the present invention program.Those of ordinary skill in the art are not paying
In the case of going out creative work, you can to understand and implement.
The invention also provides a kind of terminal, including memory, processor and storage are on a memory and can be in processor
The computer program of upper operation, following steps are realized during computer program described in the computing device：
Depth map based on two depth camera shooting Same Scenes obtains the gradient map of the scene；
According to the intraoral institute of datum windows where the Grad of the first pixel in the gradient map, and first pixel
There is the ratio of the Grad sum of pixel, calculate the weights of first pixel, wherein, the size of the weights and the ratio
The size correlation of value；
According to the parallax value of the pixel in the benchmark window and Corresponding matching window where first pixel and institute
Weights are stated, calculate the cost function of first pixel.
The invention also provides a kind of computer-readable recording medium, is stored thereon with computer program, the computer
Following steps are realized when program is executed by processor：
Depth map based on two depth camera shooting Same Scenes obtains the gradient map of the scene；
According to the intraoral institute of datum windows where the Grad of the first pixel in the gradient map, and first pixel
There is the ratio of the Grad sum of pixel, calculate the weights of first pixel, wherein, the size of the weights and the ratio
The size correlation of value；
According to the parallax value of the pixel in the benchmark window and Corresponding matching window where first pixel and institute
Weights are stated, calculate the cost function of first pixel.
Those skilled in the art will readily occur to the present invention its after considering specification and putting into practice disclosure disclosed herein
Its embodiment.It is contemplated that cover the present invention any modification, purposes or adaptations, these modifications, purposes or
Person's adaptations follow the general principle of the present invention and including undocumented common knowledges in the art of the invention
Or conventional techniques.Description and embodiments are considered only as exemplary, and true scope and spirit of the invention are by following
Claim is pointed out.
It should be appreciated that the invention is not limited in the precision architecture for being described above and being shown in the drawings, and
And various modifications and changes can be being carried out without departing from the scope.The scope of the present invention is only limited by appended claim.
Claims (8)
- A kind of 1. cost function generation method, it is characterised in that including：Depth map based on two depth camera shooting Same Scenes obtains the gradient map of the scene；According to the intraoral all pictures of datum windows where the Grad of the first pixel in the gradient map, and first pixel The ratio of the Grad sum of element, the weights of first pixel are calculated, wherein, the size of the weights and the ratio Size correlation；Gray value and the weights according to the pixel in the benchmark window and Corresponding matching window where first pixel, Calculate the cost function of first pixel.
- 2. according to the method for claim 1, it is characterised in that the gradient according to the first pixel in the gradient map It is worth, and the ratio of the Grad sum of the intraoral all pixels of datum windows where first pixel, calculate first picture The weights of vegetarian refreshments, it is specially：If the Grad sum of the intraoral pixel of the datum windows of the first pixel is less than predetermined gradient value in the gradient map, The parallax for determining first pixel is invalid parallax；If the Grad sum of the intraoral pixel of the datum windows of the first pixel is more than or equal to predetermined gradient in the gradient map Value, it is determined that the parallax of first pixel is effective parallax；If the parallax of first pixel is effective parallax, according to the Grad of the first pixel in the gradient map, and The ratio of the Grad sum of the intraoral all pixels of datum windows where first pixel, calculate first pixel Weights.
- 3. according to the method for claim 1, it is characterised in that the gradient according to the first pixel in the gradient map It is worth, and the ratio of the Grad sum of the intraoral all pixels of datum windows where first pixel, calculate first picture The weights of vegetarian refreshments, it is specially：If the intraoral Grad of the datum windows of the first pixel is small more than the quantity of the pixel of predetermined gradient value in the gradient map In predetermined number, the parallax for determining first pixel is invalid parallax；If the intraoral Grad of the datum windows of the first pixel is big more than the quantity of the pixel of predetermined gradient value in the gradient map In equal to predetermined number, the parallax for determining first pixel is effective parallax；If the parallax of first pixel is effective parallax, according to the Grad of the first pixel in the gradient map, and The ratio of the Grad sum of the intraoral all pixels of datum windows where first pixel, calculate first pixel Weights.
- A kind of 4. cost function generating means, it is characterised in that including：Image acquisition unit, the gradient of the scene is obtained for the depth map based on two depth camera shooting Same Scenes Figure；Weight calculation unit, for where the Grad according to the first pixel in the gradient map, and first pixel The intraoral all pixels of datum windows Grad sum ratio, calculate the weights of first pixel, wherein, the weights Size and the ratio size correlation；Cost function generation unit, for according to the picture in the benchmark window and Corresponding matching window where first pixel The parallax value of vegetarian refreshments and the weights, calculate the cost function of first pixel.
- 5. device according to claim 4, it is characterised in that the weight calculation unit specifically includes：Invalid parallax determination subelement, if the gradient of the intraoral pixel of datum windows for the first pixel in the gradient map Value sum is less than predetermined gradient value, it is determined that the parallax of first pixel is invalid parallax；Effective parallax determination subelement, if the gradient of the intraoral pixel of datum windows for the first pixel in the gradient map Value sum is more than or equal to predetermined gradient value, it is determined that the parallax of first pixel is effective parallax；Weight computing subelement, if the parallax for first pixel is effective parallax, according in the gradient map The ratio of the Grad sum of the intraoral all pixels of datum windows where the Grad of one pixel, and first pixel, Calculate the weights of first pixel.
- 6. device according to claim 4, it is characterised in that the weight calculation unit specifically includes：Invalid parallax determination subelement, preset if the intraoral Grad of datum windows for the first pixel in the gradient map is more than The quantity of the pixel of Grad is less than predetermined number, and the parallax for determining first pixel is invalid parallax；Effective parallax determination subelement, preset if the intraoral Grad of datum windows for the first pixel in the gradient map is more than The quantity of the pixel of Grad is more than or equal to predetermined number, and the parallax for determining first pixel is effective parallax；Weight computing subelement, if the parallax for first pixel is effective parallax, according in the gradient map The ratio of the Grad sum of the intraoral all pixels of datum windows where the Grad of one pixel, and first pixel, Calculate the weights of first pixel.
- 7. a kind of terminal, including memory, processor and storage are on a memory and the computer journey that can run on a processor Sequence, it is characterised in that realize following steps during computer program described in the computing device：Depth map based on two depth camera shooting Same Scenes obtains the gradient map of the scene；According to the intraoral all pictures of datum windows where the Grad of the first pixel in the gradient map, and first pixel The ratio of the Grad sum of element, the weights of first pixel are calculated, wherein, the size of the weights and the ratio Size correlation；According to the parallax value of the pixel in the benchmark window and Corresponding matching window where first pixel and the power Value, calculate the cost function of first pixel.
- 8. a kind of computer-readable recording medium, is stored thereon with computer program, it is characterised in that the computer program quilt Following steps are realized during computing device：Depth map based on two depth camera shooting Same Scenes obtains the gradient map of the scene；According to the intraoral all pictures of datum windows where the Grad of the first pixel in the gradient map, and first pixel The ratio of the Grad sum of element, the weights of first pixel are calculated, wherein, the size of the weights and the ratio Size correlation；According to the parallax value of the pixel in the benchmark window and Corresponding matching window where first pixel and the power Value, calculate the cost function of first pixel.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710618382.8A CN107403448B (en) | 2017-07-26 | 2017-07-26 | Cost function generation method and cost function generation device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710618382.8A CN107403448B (en) | 2017-07-26 | 2017-07-26 | Cost function generation method and cost function generation device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107403448A true CN107403448A (en) | 2017-11-28 |
| CN107403448B CN107403448B (en) | 2020-06-30 |
Family
ID=60400963
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710618382.8A Active CN107403448B (en) | 2017-07-26 | 2017-07-26 | Cost function generation method and cost function generation device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107403448B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108254738A (en) * | 2018-01-31 | 2018-07-06 | 沈阳上博智像科技有限公司 | Obstacle-avoidance warning method, device and storage medium |
| CN109816631A (en) * | 2018-12-25 | 2019-05-28 | 河海大学 | A kind of image partition method based on new cost function |
| CN111762155A (en) * | 2020-06-09 | 2020-10-13 | 安徽奇点智能新能源汽车有限公司 | Vehicle distance measuring system and method |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102074014A (en) * | 2011-02-23 | 2011-05-25 | 山东大学 | Stereo matching method by utilizing graph theory-based image segmentation algorithm |
| CN102831601A (en) * | 2012-07-26 | 2012-12-19 | 中北大学 | Three-dimensional matching method based on union similarity measure and self-adaptive support weighting |
| CN104574391A (en) * | 2014-12-29 | 2015-04-29 | 西安交通大学 | Stereoscopic vision matching method based on adaptive feature window |
| CN105957078A (en) * | 2016-04-27 | 2016-09-21 | 浙江万里学院 | Multi-view video segmentation method based on graph cut |
| KR101714896B1 (en) * | 2015-09-09 | 2017-03-23 | 중앙대학교 산학협력단 | Robust Stereo Matching Method and Apparatus Under Radiometric Change for Advanced Driver Assistance System |
| CN106709948A (en) * | 2016-12-21 | 2017-05-24 | 浙江大学 | Quick binocular stereo matching method based on superpixel segmentation |
-
2017
- 2017-07-26 CN CN201710618382.8A patent/CN107403448B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102074014A (en) * | 2011-02-23 | 2011-05-25 | 山东大学 | Stereo matching method by utilizing graph theory-based image segmentation algorithm |
| CN102831601A (en) * | 2012-07-26 | 2012-12-19 | 中北大学 | Three-dimensional matching method based on union similarity measure and self-adaptive support weighting |
| CN104574391A (en) * | 2014-12-29 | 2015-04-29 | 西安交通大学 | Stereoscopic vision matching method based on adaptive feature window |
| KR101714896B1 (en) * | 2015-09-09 | 2017-03-23 | 중앙대학교 산학협력단 | Robust Stereo Matching Method and Apparatus Under Radiometric Change for Advanced Driver Assistance System |
| CN105957078A (en) * | 2016-04-27 | 2016-09-21 | 浙江万里学院 | Multi-view video segmentation method based on graph cut |
| CN106709948A (en) * | 2016-12-21 | 2017-05-24 | 浙江大学 | Quick binocular stereo matching method based on superpixel segmentation |
Non-Patent Citations (1)
| Title |
|---|
| 邹纪标: "双目视觉局部立体匹配算法的研究", 《中国优秀硕士学位论文全文数据库信息科技辑》 * |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108254738A (en) * | 2018-01-31 | 2018-07-06 | 沈阳上博智像科技有限公司 | Obstacle-avoidance warning method, device and storage medium |
| CN109816631A (en) * | 2018-12-25 | 2019-05-28 | 河海大学 | A kind of image partition method based on new cost function |
| CN111762155A (en) * | 2020-06-09 | 2020-10-13 | 安徽奇点智能新能源汽车有限公司 | Vehicle distance measuring system and method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107403448B (en) | 2020-06-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Bogo et al. | FAUST: Dataset and evaluation for 3D mesh registration | |
| CN104756491B (en) | Depth cue based on combination generates depth map from monoscopic image | |
| Hosni et al. | Local stereo matching using geodesic support weights | |
| Yang | Dealing with textureless regions and specular highlights-a progressive space carving scheme using a novel photo-consistency measure | |
| CN112884682B (en) | Stereo image color correction method and system based on matching and fusion | |
| WO2015188684A1 (en) | Three-dimensional model reconstruction method and system | |
| CN110909693A (en) | 3D face living body detection method and device, computer equipment and storage medium | |
| KR20170008638A (en) | Three dimensional content producing apparatus and three dimensional content producing method thereof | |
| CN107316326A (en) | Applied to disparity map computational methods of the binocular stereo vision based on side and device | |
| WO2008029345A1 (en) | Method for determining a depth map from images, device for determining a depth map | |
| CN111988593B (en) | Three-dimensional image color correction method and system based on depth residual optimization | |
| CN107864337A (en) | Sketch image processing method, device and equipment | |
| CN110263768A (en) | A kind of face identification method based on depth residual error network | |
| Vu et al. | Efficient hybrid tree-based stereo matching with applications to postcapture image refocusing | |
| CN108648264A (en) | Underwater scene method for reconstructing based on exercise recovery and storage medium | |
| CN104751407A (en) | Method and device used for blurring image | |
| CN107403448A (en) | Cost function generation method and cost function generating means | |
| CN108010075A (en) | A kind of sectional perspective matching process based on multiple features combining | |
| CN104243970A (en) | 3D drawn image objective quality evaluation method based on stereoscopic vision attention mechanism and structural similarity | |
| CN113516755B (en) | Image processing method, image processing apparatus, electronic device, and storage medium | |
| Fuentes-Jimenez et al. | Deep shape-from-template: Wide-baseline, dense and fast registration and deformable reconstruction from a single image | |
| US11475629B2 (en) | Method for 3D reconstruction of an object | |
| CN108062765A (en) | Binocular image processing method, imaging device and electronic equipment | |
| CN111899293B (en) | Virtual and real shielding processing method in AR application | |
| CN112053434B (en) | Disparity map generation method, three-dimensional reconstruction method and related device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |