CN109427043B - Method and equipment for calculating smooth item parameters of global optimization matching of stereoscopic images - Google Patents
Method and equipment for calculating smooth item parameters of global optimization matching of stereoscopic images Download PDFInfo
- Publication number
- CN109427043B CN109427043B CN201710742735.5A CN201710742735A CN109427043B CN 109427043 B CN109427043 B CN 109427043B CN 201710742735 A CN201710742735 A CN 201710742735A CN 109427043 B CN109427043 B CN 109427043B
- Authority
- CN
- China
- Prior art keywords
- pixel
- message
- dir
- matching
- value
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 62
- 238000005457 optimization Methods 0.000 title claims abstract description 50
- 230000005540 biological transmission Effects 0.000 claims abstract description 46
- 238000009499 grossing Methods 0.000 claims description 38
- 238000010586 diagram Methods 0.000 claims description 31
- 230000015654 memory Effects 0.000 claims description 21
- 238000004590 computer program Methods 0.000 claims description 14
- 238000003860 storage Methods 0.000 claims description 13
- 238000004364 calculation method Methods 0.000 claims description 10
- 230000008569 process Effects 0.000 claims description 4
- 238000004422 calculation algorithm Methods 0.000 abstract description 26
- 230000000694 effects Effects 0.000 abstract description 8
- 238000012545 processing Methods 0.000 description 10
- 230000006870 function Effects 0.000 description 5
- 238000004220 aggregation Methods 0.000 description 2
- 230000002776 aggregation Effects 0.000 description 2
- 230000006399 behavior Effects 0.000 description 2
- 239000004973 liquid crystal related substance Substances 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 239000007787 solid Substances 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000009977 dual effect Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 230000001902 propagating effect Effects 0.000 description 1
Classifications
-
- G06T5/70—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T15/00—3D [Three Dimensional] image rendering
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Abstract
The invention provides a method for calculating smooth item parameters of global optimization matching of stereoscopic images, which comprises the steps of calculating a second-order gradient value of a neighborhood direction of each pixel according to a parallax initial value of a current layer, calculating a parallax gradient of a transmission message source direction of each pixel according to the second-order gradient value of the neighborhood direction, and obtaining a smooth item parameter of each matching cost of each pixel according to the parallax gradient of the transmission message source direction of each pixel and the matching cost of a corresponding previous pixel. The method for calculating the parameters of the smooth term calculates the second-order gradient value for all neighborhood directions of each pixel, can obtain higher matching precision and better matching robustness, has particularly obvious matching effect on the inclined plane of the weak texture, and solves the problem of low matching precision of the existing stereoscopic image global optimization matching algorithm.
Description
Technical Field
The invention relates to the field of image processing and machine vision, in particular to a method and equipment for calculating smooth item parameters of global optimization matching of stereoscopic images.
Background
The stereoscopic image global optimization matching algorithm aims at global energy minimization, and overall matching is achieved through global optimization based on an energy equation. The energy equation generally includes a data item and a smoothing item. The data item refers to the similarity of a corresponding point of a certain parallax corresponding to the stereoscopic image, and can be represented by one or a combination of a plurality of gray scale distance, mutual information, correlation coefficient, CENSUS distance and the like; the smoothing term refers to a penalty parameter applied when the parallax of two adjacent pixels changes, and the smoothing term and the corresponding global optimization method are key steps different from a local matching algorithm.
Common global optimization matching algorithms are confidence propagation algorithms (Belief Propgation, BP), graph cut algorithms (GC), total Variation (TV), and generalized Total Variation (Total Generalized Variation, GTV). The conventional global optimization algorithm implies a "front-Parallel" effect, i.e. a plane that is more "liked" or "good at" matching the same parallax, and in areas of parallax variation, especially in areas of weak texture where parallax varies, stepped parallax caused by the "front-Parallel" effect occurs instead of a smooth parallax plane. The improved algorithm in recent years initializes the parallax of each point to a random plane through an over-parameterization method, and obtains a better matching effect on the inclined plane through the combined use of local optimization and global optimization. But the calculation amount is significantly increased compared with the conventional method.
The global optimization matching method aims at global energy minimization and generally adopts the following energy equation:
where E (D) represents the global energy level, D represents the "disparity map" of the entire image, q is the neighboring pixels of pixel p, and Np represents the collection of neighboring pixels of pixel p. Dp and Dq represent the parallax of these two pixels. C represents the data item of the pixel p when the parallax is Dp, the similarity degree of the pixels of the left and right slices, S represents the smooth item, and the penalty imposed by Dp and Dq is not the same.
Fig. 1 is a schematic diagram of a message propagation path of the SGM algorithm, and as shown in fig. 1, a message propagates unidirectionally in a left-right, up-down and two oblique 45-degree directions, and each pixel only accepts a message of one neighboring pixel. Fig. 2 is a schematic diagram of a message propagation path of the MGM algorithm, and as shown in fig. 2, the message propagates independently along 8 directions, but each pixel receives the message of two neighboring pixels. It is noted that the two neighborhood pixels have each accepted messages also from the two neighborhood pixels, but not the other neighborhood pixels that did not accept message propagation. Fig. 3 is a schematic diagram of a message propagation path of the BP algorithm, and as shown in fig. 3, the message propagates in four directions, i.e., up and down, left and right, and each pixel receives information from neighboring pixels in three non-propagation directions. The classical BP algorithm achieves the propagation of messages within the whole image through multiple iterations.
Fig. 4 is a schematic diagram of a smoothing term parameter setting method, where the smoothing term parameter setting method is independent of a global optimization algorithm, i.e. the same smoothing term parameter setting method can be applied to different global optimization algorithms. That is, a similar smooth term parameter setting method can be employed by both the message propagation optimization algorithm, SGM, MGM, BP, etc., and other global optimization algorithms, such as GC. The two-parameter smoothing model of the SGM is a relatively representative smoothing term model, as shown in FIG. 4, propagating from left to rightFor example, the method for receiving the pixel P1 message by the kth matching cost of the pixel P comprises the following steps: adding a larger penalty parameter Q2 to the minimum matching cost Lmin of the pixel P1, and adding the k-1 th and k+1 th matching costs L of the pixel P1 k-1 、L k+1 Plus a smaller penalty parameter Q1. The kth matching cost of pixel P1 remains unchanged. Setting Lmin+Q2, L k-1 +Q1、L k+1 +Q1、L k Is Ls, the k-th matching cost of pixel P plus Ls-Lmin is the message value of pixel P.
It can be seen that the nature of the global optimization matching algorithm is that the matching degree of the points and the parallax smoothness are balanced in the whole matching area, and the result superior to that of the local matching algorithm can be obtained under most conditions. However, the global optimization matching algorithm has the defects of large calculation amount, large memory consumption and the like, and the global optimization can form the aggregation or extension effect of parallax, and the aggregation or extension direction is not necessarily carried out along the ideal direction, so that the incorrect matching result can be caused.
Disclosure of Invention
In view of the above, the embodiment of the invention provides a method for calculating smooth term parameters of global optimization matching of stereoscopic images, so as to solve the problem of low matching precision of the existing global optimization matching algorithm of stereoscopic images.
Therefore, the embodiment of the invention provides the following technical scheme:
the embodiment of the invention provides a method for calculating a smoothing term parameter of global optimization matching of a stereoscopic image, which comprises the steps of calculating a second-order gradient value of a neighborhood direction of each pixel according to a parallax initial value of a current layer; according to the second-order gradient value of the neighborhood direction, calculating the parallax gradient of the transmission message source direction of each pixel; and obtaining a smooth term parameter of each matching cost of each pixel according to the parallax gradient of the transmission message source direction of each pixel and the matching cost of the corresponding previous pixel.
Optionally, before the step of calculating the second-order gradient value of the neighborhood direction of each pixel according to the parallax initial value of the current layer, the method further includes: obtaining a parallax image of the upper layer by using a pyramid matching method; and acquiring a parallax initial value of the current layer according to the parallax map of the upper layer.
Optionally, each pixel receives message passing values of eight neighborhood directions, and eight message propagation path combination graphs are obtained according to the message passing values of eight neighborhood directions received by each pixel, wherein each message propagation path combination comprises two message propagation directions.
Optionally, the step of calculating the parallax gradient of the message source direction of each pixel according to the second-order gradient value of the neighborhood direction includes: the parallax gradient for the eight message source directions per pixel is calculated using the following formula:
S(n) i,j +S(n 1 ) i,j at the time of > 1 the number of the holes,
S(n) i,j +S(n 1 ) i,j t is less than or equal to 1 n ＝0；
S(l) i,j +S(l 1 ) i,j At the time of > 1 the number of the holes,
S(l) i,j +S(l 1 ) i,j t is less than or equal to 1 l ＝0；
In S (n) i,j 、S(n 1 ) i,j 、S(l) i,j 、S(l 1 ) i,j Second order gradient values of message propagation directions respectively, wherein n and l are two message propagation directions in each message propagation path combination diagram, and n 1 、l 1 I, j are the row and column numbers of the image respectively for the opposite direction of the two message propagation directions, wherein,
dir 0,0 ＝0,dir 0,1 ＝1
dir 1,0 ＝0,dir 1,1 ＝-1
dir 2,0 ＝1,dir 2,1 ＝0
dir 3,0 ＝-1,dir 3,1 ＝0
dir 4,0 ＝1,dir 4,1 ＝1
dir 5,0 ＝-1,dir 5,1 ＝-1
dir 6,0 ＝-1,dir 6,1 ＝1
dir 7,0 ＝1,dir 7,1 ＝-1
dir represents 8 directions represented by 8 neighbors of each pixel, the first subscript (n, n 1 ,l,l 1 ) Represents one of specific directions, wherein n and n 1 For a pair of opposite directions, l and l 1 Also a pair of opposite directions, with directions 0 and 1, 2 and 3, 4 and 5, 6 and 7, the 2 nd subscript contains two numbers, 1 st offset in the row direction, 2 nd offset in the column direction, dir 0,0 ＝0,dir 0,1 =1 means shifting one pixel to the right, dir 1,0 ＝0,dir 1,1 = -1 represents a shift to the left by one pixel, dir 4,0 ＝1,dir 4,1 =1 means that the shift is one pixel to the upper right (45 degrees direction inclined).
Optionally, the obtaining the smoothing term parameter of each matching cost of each pixel according to the parallax gradient of the transmission message source direction of each pixel and the matching cost of the corresponding previous pixel includes: in each message propagation path combination diagram, obtaining two transmission message values of the two transmission message source directions according to the parallax gradient of the two transmission message source directions of each pixel and the matching cost of the corresponding previous pixel; obtaining an average value of the two message values according to the two transmitted message values; the average value is a smooth term parameter of each pixel in each message propagation path combination diagram, and eight smooth term parameters are obtained in total for each matching cost of each pixel; and adding the obtained eight smooth term parameters to obtain an average value, and obtaining the smooth term parameters of each matching cost of each pixel.
Optionally, the obtaining the smoothing term parameter of each matching cost of each pixel according to the parallax gradient of the transmission message source direction of each pixel and the matching cost of the corresponding previous pixel includes: in each of the message propagation path combination charts: obtaining the minimum matching cost of the previous pixel in the first direction of the current pixel message transmission, and adding a first penalty parameter to the minimum matching cost to obtain a first value; adding a second penalty parameter to the K '+1th matching cost and the K' -1th matching cost of the previous pixel to obtain a second numerical value and a third numerical value; comparing the first value, the second value, the third value and the K' th matching cost of the previous pixel to obtain a minimum value; subtracting the minimum matching cost of the previous pixel from the minimum value to obtain a message value transmitted in a first direction of message transmission received by the K-th matching cost of the current pixel; repeating the steps to obtain a message value transmitted in a second direction of message transmission received by the K-th matching cost of the current pixel; the average value of the message value transmitted in the first direction and the message value transmitted in the second direction received by the Kth matching cost is obtained, and the average value is the message transmission value received by the Kth matching cost of the current pixel; acquiring message passing values received by all matching costs of the current pixel; wherein the message passing value received by each matching cost of the current pixel is a smooth term parameter of the matching cost; and obtaining eight smooth item parameters in total for each matching cost, adding the eight obtained smooth item parameters, and obtaining an average value to obtain the smooth item parameters of each matching cost of each pixel.
The embodiment of the invention also provides a computer readable storage medium, on which a computer program is stored, and the computer program realizes the method for calculating the smooth item parameters of the global optimization matching of the stereoscopic image when being executed by a processor.
The embodiment of the invention also provides computer equipment, which comprises: at least one processor; and a memory communicatively coupled to the at least one processor; the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor, so that the at least one processor executes the above method for calculating the smoothing term parameters of global optimization matching of stereoscopic images.
The embodiment of the invention has the following advantages:
the embodiment of the invention provides a method for calculating a smooth item parameter of global optimization matching of a stereoscopic image, which comprises the steps of calculating a second-order gradient value of a neighborhood direction of each pixel according to a parallax initial value of a current layer, calculating a parallax gradient of a transmission message source direction of each pixel according to the second-order gradient value of the neighborhood direction, and obtaining a smooth item parameter of each matching cost of each pixel according to the parallax gradient of the transmission message source direction of each pixel and the matching cost of a corresponding previous pixel. Compared with the prior art, the method for calculating the parameters of the smooth term in the embodiment of the invention has the advantages that the second-order gradient value is calculated for all neighborhood directions of each pixel, the message transmission value in two propagation source directions is received for each matching cost of each pixel in the embodiment of the invention, and the message value is transmitted to the next pixel for all pixels, so that higher matching precision and better matching robustness can be obtained, the matching effect on the inclined plane of the weak texture is particularly obvious, and the problem of low matching precision of the existing stereoscopic image global optimization matching algorithm is solved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings that are needed in the description of the embodiments or the prior art will be briefly described, and it is obvious that the drawings in the description below are some embodiments of the present invention, and other drawings can be obtained according to the drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic diagram of a message propagation path of an SGM algorithm;
FIG. 2 is a schematic diagram of a message propagation path for an MGM algorithm;
FIG. 3 is a schematic diagram of the message propagation path of the BP algorithm;
FIG. 4 is a schematic diagram of a smoothing item parameter setting method;
FIG. 5 is a flowchart of a method for computing smoothing term parameters for global optimization matching of stereoscopic images according to an embodiment of the invention;
FIG. 6 is a schematic diagram of a message propagation path of a method for computing smoothing term parameters for global optimization matching of stereoscopic images according to an embodiment of the invention;
fig. 7 is a schematic hardware structure diagram of a computer device of a method for calculating parameters of smoothing terms for global optimization matching of stereoscopic images according to an embodiment of the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made apparent and fully in view of the accompanying drawings, in which some, but not all embodiments of the invention are shown. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
In the description of the present invention, it should be noted that the terms "first," "second," and "third" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance.
In addition, the technical features of the different embodiments of the present invention described below may be combined with each other as long as they do not collide with each other.
Example 1
In this embodiment, a method for calculating smoothing term parameters of global optimization matching of stereoscopic images is provided, and fig. 5 is a flowchart of a method for calculating smoothing term parameters of global optimization matching of stereoscopic images according to an embodiment of the present invention, as shown in fig. 5, the method includes the following steps:
s101: calculating a second-order gradient value of the neighborhood direction of each pixel according to the parallax initial value of the current layer; and calculating second-order gradient values of all neighborhood directions of each pixel for the stereoscopic image to obtain the enhancement effect of the image.
S102: according to the second-order gradient value of the neighborhood direction, calculating the parallax gradient of each pixel in the direction of the transmission message source; for each pixel, all neighborhood directions of the pixel can transmit messages, parallax gradient of the message source transmitting direction is calculated, and more accurate image matching is obtained.
S103: obtaining a smooth item parameter of each matching cost of each pixel according to the parallax gradient of the transmission message source direction of each pixel and the matching cost of the corresponding previous pixel; when each pixel transmits a message, the message value is transmitted along a transmission path, and the transmitted message value in all directions received by each matching cost of each pixel is averaged to obtain a smooth term parameter of each matching cost of each pixel.
Through the steps, the second-order gradient value of the neighborhood direction of each pixel is calculated according to the parallax initial value of the current layer, the parallax gradient of the transmission message source direction of each pixel is calculated according to the second-order gradient value of the neighborhood direction, the smooth term parameter of each matching cost of each pixel is obtained according to the parallax gradient of the transmission message source direction of each pixel and the matching cost of the corresponding previous pixel, and the second-order gradient value is calculated for all the neighborhood directions of each pixel.
In a specific embodiment, before step S101 calculates the second-order gradient value of the neighborhood direction of each pixel according to the parallax initial value of the current layer, the method further includes obtaining a parallax map of the previous layer by using a pyramid matching method, and obtaining the parallax initial value of the current layer according to the parallax map of the previous layer. In the embodiment of the invention, a message propagation method is used as a basic strategy of global optimization, a pyramid matching method is adopted, the parallax obtained by matching the pyramid image of the upper layer is used as the parallax initial value of the current layer and the basic data for estimating the local parallax gradient, the parallax image D0 of the upper layer is assumed to be obtained, and the parallax initial value D of the current layer is obtained through linear interpolation.
FIG. 6 is a schematic diagram of a message propagation path of a method for computing smoothing term parameters for global optimization matching of stereoscopic images according to an embodiment of the invention, as shown in FIG. 6, eachEach pixel receives message passing values in eight neighborhood directions, and obtains eight message propagation path combination graphs (M 1 ,……，M 8 ) Wherein each message propagation path combination comprises two message propagation directions.
Step S101 described above involves calculating the second-order gradient value of each pixel in the neighborhood direction according to the parallax initial value of the current layer, and assuming that the matching cost calculation of the current layer has been completed, the algorithm of the "semi-global second-order gradient map" of the current layer for calculating 8 neighborhood directions is as follows:
let the current pixel array be P i,j The next pixel of the pixel along the 8 neighborhood direction is:
P n (i+dir n,0 ,j+dir n,1 )(n＝0,1,...,7) (2)
wherein, the liquid crystal display device comprises a liquid crystal display device,
dir 0,0 ＝0,dir 0,1 ＝1
dir 1,0 ＝0,dir 1,1 ＝-1
dir 2,0 ＝1,dir 2,1 ＝0
dir 3,0 ＝-1,dir 3,1 ＝0
dir 4,0 ＝1,dir 4,1 ＝1
dir 5,0 ＝-1,dir 5,1 ＝-1
dir 6,0 ＝-1,dir 6,1 ＝1
dir 7,0 ＝1,dir 7,1 ＝-1
wherein dir represents 8 directions represented by 8 neighborhoods of each pixel, and the directions are 0 and 1, 2 and 3, 4 and 5, 6 and 7, the 2 nd subscript contains two numbers, the 1 st is offset in the row direction, the 2 nd is offset in the column direction, dir 0,0 ＝0,dir 0,1 =1 means shifting one pixel to the right, dir 1,0 ＝0,dir 1,1 = -1 represents a shift to the left by one pixel, dir 4,0 ＝1,dir 4,1 =1 means that the shift is one pixel to the upper right (45 degrees direction inclined).
The form of the semi-global second-order gradient map is 8 short integer images with the length and width being the same as the length and width of the current layer image, and the gray scale of the images is calculated as follows:
the method comprises the steps of setting a currently calculated 'semi-global second-order gradient map' S (m) in an mth direction, initializing the numerical value of each point of the 'semi-global second-order gradient map' to 0, setting the starting behavior si in the current propagation direction, the starting column to sj, the ending behavior ei, the ending column to ej, the row direction moving step length to di, the column direction moving step length to dj, and the width and the height of a current layer image to be w and h respectively. The setting method of the parameters corresponding to the mth propagation direction is as follows:
si＝1,ei＝h,sj＝1,ej＝w,di＝1,dj＝1(m＝0,2,4)
si＝h-2,ei＝-1,sj＝w-2,ej＝-1,di＝-1,dj＝-1(m＝1,3,5)
si＝1,ei＝h,sj＝w-2,ej＝-1,di＝1,dj＝-1(m＝7)
si＝h-2,ei＝-1,sj＝1,ej＝w,di＝-1,dj＝1(m＝6) (3)
the coordinates of the pixel row and column of the "next" received message can be calculated by the parameters of equation 3, and assuming that the current pixel is the kth pixel on the current propagation path, and the coordinates of the row and column are (i, j), the coordinates of the pixel row and column of the "next" received message are (i ', j'), the following relationship exists:
ei. ej represents the pixel on the current propagation path that last received the message.
For each pixel P i,j The adjacent pixels in the m direction are P i1,j1 Wherein, the method comprises the steps of, wherein,
i1＝i-dir m,0
j1＝j-dir m,1 (4)
P i,j the second order gradient along the m-direction is:
T i,j ＝2D i,j -D(i-dir m,0 ,j-dir m,1 )-D(i+dir m,0 ,j+dir m,1 ) (5)
a second order gradient threshold mx is set, such as mx=0.5. Let the current row and column be i and j, respectively. The current method of calculating the value Sm (i, j) of the "semi-global second order gradient map" is:
S(m) i,j ＝S(m) i,j +1T i1,j1 ＞mx,T i,j ＜＝mx
S(m) i,j ＝S(m) i1,j1 +1T i1,j1 ＜＝mx,T i,j ＜＝mx
S(m) i,j ＝S(m) i,j -1T i1,j1 ＜＝mx,T i,j ＞mx
S(m) i,j ＝S(m) i1,j1 -1T i1,j1 ＞mx,T i,j ＞mx (6)
after the numerical value of the "semi-global second-order gradient map" of the current pixel is calculated, i=i+di, j=j+dj, and then the numerical value of the "semi-global second-order gradient map" of the next pixel is calculated by the method, namely, the second-order gradient numerical value of each pixel is calculated.
Step S102 above involves calculating the parallax gradient of the transmission message source direction of each pixel according to the second-order gradient value of the neighborhood direction, and in a specific embodiment, the process of calculating the parallax gradient of the M-th group direction combining the two transmission source directions is as follows:
the message propagation algorithm adopted herein adopts a dual-path zigzag propagation strategy, and the dual paths are combined into 8 types, denoted by M, and as shown in fig. 6, the two paths are respectively:
where dir represents 8 directions represented by 8 neighbors of each pixel, i and j are the row and column numbers of the image, respectively, and i (or j) is used to make a remainder of 2 to distinguish two opposite directions of propagation of the word path. i.e 1 、j 1 Then the rank number from "opposite" is indicated. Let the image width and height be w and h respectively, i.e 1 ＝h-1-i，j 1 =w-1-j. i ', j' represent the row and column numbers of the image rotated 45 degrees, corresponding to the 3 rd figure of fig. 5, where the row numbers of each column are different, the first column has only pixel a, the second column has pixels b and c, and the third column has pixels d, e and f. i.e 1 '、j 1 ' represents the rank number from the "opposite" of the rotated image. It should be noted that the "rotation" is only for explaining the setting method of the direction combination, and is not true of performing the rotation operation on the image.
Taking the first diagram as an example, from the "h" pixel of the penultimate row and the second column, a message of pixel e and pixel g is accepted, so the propagation direction of the pixels of the second column is left, down to right, up. After the second column completes the message propagation, the third column accepts the messages of pixel k and pixel m from pixel n, that is, the propagation direction of the third column becomes left, up to right, down, and then the message propagation of each column is alternated.
For each pixel pointCalculating parallax gradients of two transmission message source directions, wherein the parallax gradients of the two directions are respectively T, and the n direction and the l direction are set as the directions n And T l The opposite directions of the two directions are respectively n 1 Direction sum l 1 The direction is calculated by the following steps:
S(n) i,j +S(n 1 ) i,j at the time of > 1 the number of the holes,
S(n) i,j +S(n 1 ) i,j t is less than or equal to 1 n ＝0；
S(l) i,j +S(l 1 ) i,j When > 1, (8)
S(l) i,j +S(l 1 ) i,j T is less than or equal to 1 l ＝0；
Wherein dir is defined as above, S (n) i,j 、S(n 1 ) i,j 、S(l) i,j 、S(l 1 ) i,j Second order gradient values of message propagation directions respectively, wherein n and l are two message propagation directions in each message propagation path combination diagram, and n 1 、l 1 I and j are row and column numbers of the image respectively, which are opposite directions of the two message propagation directions.
The step S103 described above involves obtaining a smoothing term parameter of each matching cost of each pixel according to the parallax gradient of the transmission message source direction of each pixel and the matching cost of the corresponding previous pixel, and in an alternative embodiment, the step includes obtaining, in each message propagation path combination diagram, two transmission message values of the two transmission message source directions according to the parallax gradient of the two transmission message source directions of each pixel and the matching cost of the corresponding previous pixel, obtaining an average value of the two message values according to the two transmission message values, where the average value is one smoothing term parameter of each matching cost of each pixel in each message propagation path combination diagram, obtaining eight smoothing term parameters for each matching cost, adding the obtained eight smoothing term parameters, and obtaining an average value to obtain the smoothing term parameter of each matching cost of each pixel. Specifically, in each message propagation path combination diagram, obtaining the minimum matching cost of a previous pixel in the first direction of the current pixel message propagation, adding a first penalty parameter to the minimum matching cost to obtain a first value, and adding a second penalty parameter to the K '+1th matching cost and the K' -1th matching cost of the previous pixel to obtain a second value and a third value; comparing the first value, the second value, the third value and the K 'matching cost of the previous pixel to obtain a minimum value, and subtracting the minimum matching cost of the previous pixel from the minimum value to obtain a message value transmitted in a first direction of message transmission received by the K' matching cost of the current pixel;
the embodiment of the invention provides a specific smoothing item parameter calculation method, according to the message propagation path combination diagram in fig. 6, for one pixel in one combination diagram, the smoothing item parameter calculation method for solving the kth matching cost n direction of the pixel is as follows:
the above equation yields the n-directional message passing value, where L' 0 、L' 1 、L' 2 K, K-1 and K+1 matching costs of the current pixel respectively, L k 、L k-1 、L k-2 、L k+1 、L k+2 And L min K, K-1, K-2, K+1, K+2 matching costs for a previous pixel in the message propagation direction and the minimum matching cost for the previous pixel, Q 1 、Q 2 For penalty parameter, T is parallax gradient, L send The value of the delivered message received for the K-th matching cost.
According to the formula, the message value and the message value transmitted in the direction of the transmission n of the message are received by the K-th matching cost of the current pixel, the average value of the message transmission values in the two directions is obtained, namely the message transmission value received by the K-th matching cost of the current pixel is a smooth item parameter of the K-th matching cost of the current pixel, because eight message transmission path combination graphs exist, each matching cost of each pixel has eight smooth item parameters, the average value of the eight smooth item parameters is obtained for each matching cost, the average value is the smooth item parameter of the matching cost, namely the received message transmission value, and after the received message transmission value of each matching cost is obtained, the matching cost of all layers of next pixels is continuously transmitted one by one.
The method for calculating the smooth term parameters of the global optimization matching of the stereoscopic image can enable the stereoscopic image optimization matching strategy to obtain higher matching precision and better matching robustness, and has an obvious matching effect on the inclined plane of the weak texture.
Example 2
The embodiment of the invention also provides a computer-readable storage medium which stores computer-executable instructions capable of executing the method for calculating the smooth item parameters of the global optimization matching of the stereoscopic image in any of the method embodiments. Wherein the storage medium may be a magnetic Disk, an optical Disk, a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM), a Flash Memory (Flash Memory), a Hard Disk (HDD), or a Solid State Drive (SSD); the storage medium may also comprise a combination of memories of the kind described above.
Example 3
Fig. 7 is a schematic hardware structure of a computer device according to a method for computing parameters of smoothing terms for global optimization matching of stereoscopic images according to an embodiment of the present invention, and as shown in fig. 7, the device includes one or more processors 710 and a memory 720, and in fig. 7, one processor 710 is taken as an example.
The apparatus for performing the smoothing term parameter calculation method for global optimization matching of stereoscopic images may further include: an input device 730 and an output device 740.
Processor 710, memory 720, input device 730, and output device 740 may be connected by a bus or other means, for example in fig. 7.
The processor 710 may be a central processing unit (Central Processing Unit, CPU). The processor 710 may also be a chip such as other general purpose processors, digital signal processors (Digital Signal Processor, DSP), application specific integrated circuits (Application Specific Integrated Circuit, ASIC), field programmable gate arrays (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or a combination thereof. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The memory 720 is used as a non-transitory computer readable storage medium, and can be used to store a non-transitory software program, a non-transitory computer executable program, and a module, such as a program instruction/module corresponding to the method for calculating the smoothing term parameter of global optimization matching of stereoscopic images in the embodiment of the present application. The processor 710 executes the non-transitory software programs, instructions, and modules stored in the memory 720 to perform various functional applications and data processing of the server, that is, to implement the smooth term parameter calculation method for global optimization matching of stereoscopic images in the above-described method embodiment.
Memory 720 may include a storage program area that may store an operating system, at least one application program required for functionality, and a storage data area; the storage data area may store data created by the use of the weak password scanning device, etc. In addition, memory 720 may include high-speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid state storage device. In some embodiments, memory 720 optionally includes memory remotely located with respect to processor 710, which may be connected via a network to a processing device for smoothing item parameter calculation for stereoscopic global optimization matching. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The input device 730 may receive input numeric or character information and key signal inputs related to user settings and function control of the processing device that generate the smoothed term parameter calculations matched to the global optimization of the stereoscopic image. The output device 740 may include a display device such as a display screen.
The one or more modules are stored in the memory 720 that, when executed by the one or more processors 710, perform the method as shown in fig. 5.
The above product may perform the method provided by the embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the performing method, and technical details that are not described in detail in this embodiment, and may be specifically referred to the related descriptions in the embodiments shown in fig. 5 to 6.
It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
It is apparent that the above examples are given by way of illustration only and are not limiting of the embodiments. Other variations or modifications of the above teachings will be apparent to those of ordinary skill in the art. It is not necessary here nor is it exhaustive of all embodiments. While still being apparent from variations or modifications that may be made by those skilled in the art are within the scope of the invention.
Claims (7)
1. A method for calculating parameters of a smoothing term of global optimization matching of a stereoscopic image is characterized by comprising the following steps:
calculating a second-order gradient value of the neighborhood direction of each pixel according to the parallax initial value of the current layer;
each pixel receives message passing values of eight neighborhood directions, and eight message propagation path combination diagrams are obtained according to the message passing values of the eight neighborhood directions received by each pixel, wherein each message propagation path combination comprises two message propagation directions;
according to the second-order gradient value of the neighborhood direction, calculating the parallax gradient of the transmission message source direction of each pixel;
and obtaining a smooth term parameter of each matching cost of each pixel according to the parallax gradient of the transmission message source direction of each pixel and the matching cost of the corresponding previous pixel.
2. The method for computing a smoothing term parameter for global optimization matching of stereoscopic images according to claim 1, further comprising, before the step of computing a second order gradient value in a neighborhood direction of each pixel from a parallax initial value of a current layer:
obtaining a parallax image of the upper layer by using a pyramid matching method;
and acquiring a parallax initial value of the current layer according to the parallax map of the upper layer.
3. The method for computing a smoothing term parameter for global optimization matching of stereoscopic images according to claim 1, wherein the step of computing a parallax gradient of a transfer message source direction of each pixel according to a second-order gradient value of the neighborhood direction comprises:
the parallax gradient for the eight message source directions per pixel is calculated using the following formula:
S(n) i,j +S(n 1 ) i,j >in the case of 1, the number of the times of the process is reduced,
S(n) i,j +S(n 1 ) i,j t is less than or equal to 1 n ＝0；
S(l) i,j +S(l 1 ) i,j >In the case of 1, the number of the times of the process is reduced,
S(l) i,j +S(l 1 ) i,j t is less than or equal to 1 l ＝0；
In S (n) i,j 、S(n 1 ) i,j 、S(l) i,j 、S(l 1 ) i,j Second steps of message propagation directions respectivelyA degree value, wherein n and l are the two message propagation directions in each message propagation path combination diagram, n 1 、l 1 I, j are the row and column numbers of the image respectively for the opposite direction of the two message propagation directions, wherein,
dir 0,0 ＝0,dir 0,1 ＝1
dir 1,0 ＝0,dir 1,1 ＝-1
dir 2,0 ＝1,dir 2,1 ＝0
dir 3,0 ＝-1,dir 3,1 ＝0
dir 4,0 ＝1,dir 4,1 ＝1
dir 5,0 ＝-1,dir 5,1 ＝-1
dir 6,0 ＝-1,dir 6,1 ＝1
dir 7,0 ＝1,dir 7,1 ＝-1
dir represents 8 directions represented by 8 neighbors of each pixel, the first subscript (n, n 1 ,l,l 1 ) Represents one of specific directions, wherein n and n 1 For a pair of opposite directions, l and l 1 Also a pair of opposite directions, with directions 0 and 1, 2 and 3, 4 and 5, 6 and 7, the 2 nd subscript contains two numbers, 1 st offset in the row direction, 2 nd offset in the column direction, dir 0,0 ＝0,dir 0,1 =1 means shifting one pixel to the right, dir 1,0 ＝0,dir 1,1 = -1 represents a shift to the left by one pixel, dir 4,0 ＝1,dir 4,1 =1 means that the direction is inclined 45 degrees upward to the right by one pixel.
4. The method for computing the smoothing term parameters for global optimization matching of stereoscopic images according to claim 1, wherein the obtaining the smoothing term parameters for each matching cost of each pixel according to the parallax gradient of the transmission message source direction of each pixel and the matching cost of the corresponding previous pixel comprises:
in each message propagation path combination diagram, obtaining two transmission message values of the two transmission message source directions according to the parallax gradient of the two transmission message source directions of each pixel and the matching cost of the corresponding previous pixel;
obtaining an average value of the two message values according to the two transmitted message values;
the average value is a smooth term parameter of each pixel in each message propagation path combination diagram, and eight smooth term parameters are obtained in total for each matching cost of each pixel;
and adding the obtained eight smooth term parameters to obtain an average value, and obtaining the smooth term parameters of each matching cost of each pixel.
5. The method for computing the smoothing term parameters for global optimization matching of stereoscopic images according to claim 4, wherein the obtaining the smoothing term parameters for each matching cost of each pixel according to the parallax gradient of the source direction of the transfer message of each pixel and the matching cost of the corresponding previous pixel comprises:
in each of the message propagation path combination charts:
obtaining the minimum matching cost of the previous pixel in the first direction of the current pixel message transmission, and adding a first penalty parameter to the minimum matching cost to obtain a first value; adding a second penalty parameter to the K '+1th matching cost and the K' -1th matching cost of the previous pixel to obtain a second numerical value and a third numerical value;
comparing the first value, the second value, the third value and the K' th matching cost of the previous pixel to obtain a minimum value;
subtracting the minimum matching cost of the previous pixel from the minimum value to obtain a message value transmitted in a first direction of message transmission received by the K-th matching cost of the current pixel;
repeating the steps to obtain a message value transmitted in a second direction of message transmission received by the K-th matching cost of the current pixel;
the average value of the message value transmitted in the first direction and the message value transmitted in the second direction received by the Kth matching cost is obtained, and the average value is the message transmission value received by the Kth matching cost of the current pixel;
acquiring message passing values received by all matching costs of the current pixel; wherein the message passing value received by each matching cost of the current pixel is a smooth term parameter of the matching cost;
and obtaining eight smooth item parameters in total for each matching cost, adding the eight obtained smooth item parameters, and obtaining an average value to obtain the smooth item parameters of each matching cost of each pixel.
6. A computer readable storage medium having stored thereon a computer program, wherein the computer program when executed by a processor implements the method for computing smooth term parameters for global optimization matching of stereoscopic images according to any one of claims 1-5.
7. A computer device, comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein the memory stores a computer program executable by the at least one processor to cause the at least one processor to perform the method of smoothing item parameter calculation for global optimization matching of stereoscopic images according to any one of claims 1-5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710742735.5A CN109427043B (en) | 2017-08-25 | 2017-08-25 | Method and equipment for calculating smooth item parameters of global optimization matching of stereoscopic images |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710742735.5A CN109427043B (en) | 2017-08-25 | 2017-08-25 | Method and equipment for calculating smooth item parameters of global optimization matching of stereoscopic images |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109427043A CN109427043A (en) | 2019-03-05 |
| CN109427043B true CN109427043B (en) | 2023-08-01 |
Family
ID=65501548
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710742735.5A Active CN109427043B (en) | 2017-08-25 | 2017-08-25 | Method and equipment for calculating smooth item parameters of global optimization matching of stereoscopic images |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109427043B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112435267B (en) * | 2020-11-17 | 2023-03-17 | 哈尔滨工程大学 | Disparity map calculation method for high-resolution urban satellite stereo image |
| CN112561832B (en) * | 2020-12-24 | 2023-12-01 | 山东省国土测绘院 | Remote sensing image data storage method and system |
| CN112765095B (en) * | 2020-12-24 | 2022-08-23 | 山东省国土测绘院 | Method and system for filing image data of stereo mapping satellite |
Family Cites Families (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| GB0125774D0 (en) * | 2001-10-26 | 2001-12-19 | Cableform Ltd | Method and apparatus for image matching |
| US8582866B2 (en) * | 2011-02-10 | 2013-11-12 | Edge 3 Technologies, Inc. | Method and apparatus for disparity computation in stereo images |
| CN102567964B (en) * | 2011-12-08 | 2014-08-27 | 北京控制工程研究所 | Filtering method for stereoscopic vision parallax image |
| TWI502544B (en) * | 2013-03-07 | 2015-10-01 | Acer Inc | Disparity estimation method of stereoscopic image |
| CN103177446B (en) * | 2013-03-13 | 2016-03-30 | 北京航空航天大学 | Based on the accurate extracting method of display foreground of neighborhood and non-neighborhood smoothing prior |
| US9681150B2 (en) * | 2014-06-13 | 2017-06-13 | Texas Instruments Incorporated | Optical flow determination using pyramidal block matching |
| CN105205808B (en) * | 2015-08-20 | 2018-01-23 | 武汉大学 | Multi-view images dense Stereo Matching fusion method and system based on multiple features multiple constraint |
-
2017
- 2017-08-25 CN CN201710742735.5A patent/CN109427043B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN109427043A (en) | 2019-03-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109427043B (en) | Method and equipment for calculating smooth item parameters of global optimization matching of stereoscopic images | |
| Wang et al. | Mvster: Epipolar transformer for efficient multi-view stereo | |
| US11170202B2 (en) | Apparatus and method for performing 3D estimation based on locally determined 3D information hypotheses | |
| US9350969B2 (en) | Target region filling involving source regions, depth information, or occlusions | |
| CN104574342B (en) | The noise recognizing method and Noise Identification device of parallax depth image | |
| Perri et al. | Adaptive Census Transform: A novel hardware-oriented stereovision algorithm | |
| CN109887008B (en) | Method, device and equipment for parallax stereo matching based on forward and backward smoothing and O (1) complexity | |
| JP2010200213A5 (en) | ||
| WO2020191731A1 (en) | Point cloud generation method and system, and computer storage medium | |
| CN103778598A (en) | Method and device for disparity map improving | |
| CN106887018A (en) | Solid matching method, controller and system | |
| CN102609936A (en) | Stereo image matching method based on belief propagation | |
| CN111553296B (en) | Two-value neural network stereo vision matching method based on FPGA | |
| TWI502544B (en) | Disparity estimation method of stereoscopic image | |
| Michailidis et al. | High performance stereo system for dense 3-D reconstruction | |
| Sheng et al. | Lfnat 2023 challenge on light field depth estimation: Methods and results | |
| CN103310446A (en) | Stereo matching method based on iterative guidance filtering | |
| US10003784B2 (en) | Stereo image matching apparatus and method requiring small calculation | |
| CN104252701B (en) | Correct the method and system of disparity map | |
| CN101877129B (en) | Minimal sum cache acceleration strategy based binocular stereo vision matching method for generalized confidence spread | |
| Tippetts et al. | Efficient stereo vision algorithms for resource-limited systems | |
| Kanaeva et al. | Camera pose and focal length estimation using regularized distance constraints | |
| CN104902260B (en) | The acquisition methods and system of a kind of image parallactic | |
| CN107332687A (en) | A kind of link prediction method based on Bayesian Estimation and common neighbours | |
| Li et al. | DRI-MVSNet: A depth residual inference network for multi-view stereo images |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information | ||
| CB02 | Change of applicant information |
Address after: 100048 Beijing city Haidian District, yum Village No. 1 Applicant after: Ministry of Natural Resources Land Satellite Remote Sensing Application Center Address before: 100048 No.1 Yushengcun Hospital, Sanhuqiao North Road, Shijingshan District, Beijing Applicant before: SATELLITE SURVEYING AND MAPPING APPLICATION CENTER, NASG |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |