CN112258635A - Three-dimensional reconstruction method and device based on improved binocular matching SAD algorithm - Google Patents

Three-dimensional reconstruction method and device based on improved binocular matching SAD algorithm Download PDF

Info

Publication number
CN112258635A
CN112258635A CN202011154642.9A CN202011154642A CN112258635A CN 112258635 A CN112258635 A CN 112258635A CN 202011154642 A CN202011154642 A CN 202011154642A CN 112258635 A CN112258635 A CN 112258635A
Authority
CN
China
Prior art keywords
disparity map
improved
sad algorithm
dimensional reconstruction
initial disparity
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202011154642.9A
Other languages
Chinese (zh)
Other versions
CN112258635B (en
Inventor
刘学君
常梦洁
晏涌
沙芸
孔祥旻
李柏灿
龚泓博
苏鹏
黎杨
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Institute of Petrochemical Technology
Original Assignee
Beijing Institute of Petrochemical Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Institute of Petrochemical Technology filed Critical Beijing Institute of Petrochemical Technology
Priority to CN202011154642.9A priority Critical patent/CN112258635B/en
Publication of CN112258635A publication Critical patent/CN112258635A/en
Application granted granted Critical
Publication of CN112258635B publication Critical patent/CN112258635B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/22—Matching criteria, e.g. proximity measures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/40—Analysis of texture
    • G06T7/49—Analysis of texture based on structural texture description, e.g. using primitives or placement rules
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/97—Determining parameters from multiple pictures
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
    • Y02T10/00—Road transport of goods or passengers
    • Y02T10/10—Internal combustion engine [ICE] based vehicles
    • Y02T10/40—Engine management systems

Abstract

The invention relates to the technical field of image processing, in particular to a three-dimensional reconstruction method and a three-dimensional reconstruction device based on an improved binocular matching SAD algorithm. The three-dimensional reconstruction method based on the improved binocular matching SAD algorithm comprises the following steps: acquiring an original picture; the original pictures comprise corresponding first pictures and second pictures; obtaining an initial disparity map through an improved SAD algorithm based on the original picture; the improved SAD algorithm is biased to the matching precision of central pixels of image blocks of two pictures in the process of generating an initial disparity map; and filling the holes of the initial disparity map to optimize the initial disparity map to obtain a final disparity map.

Description

Three-dimensional reconstruction method and device based on improved binocular matching SAD algorithm
Technical Field
The invention relates to the technical field of image processing, in particular to a three-dimensional reconstruction method and a three-dimensional reconstruction device based on an improved binocular matching SAD algorithm.
Background
The continuous increase of the production and use amount of dangerous chemicals puts higher requirements on the supervision of the storage safety of the dangerous chemicals. The three-dimensional reconstruction is carried out on the hazardous chemical substance warehouse, the environmental information around the hazardous chemical substance can be accurately acquired in real time, the visualization of the hazardous chemical substance warehouse environment is realized, and the real-time performance and the safety of monitoring and early warning are ensured. The binocular vision technology carries out three-dimensional reconstruction on the target based on the parallax principle, compared with the mode that a TOF camera, structured light, a laser radar and the like acquire scene depth information, the binocular vision technology is not limited by the material of the surface of an object, can acquire uniform and dense scene three-dimensional information, has low requirements on hardware and cost, and has a certain application prospect in the aspect of safety monitoring in a hazardous chemical library.
The effect and precision of three-dimensional reconstruction are directly determined by a matching algorithm, and the key step is to search homonymous points in two images, namely to perform stereo matching. The Sum of Absolute Differences (SAD) algorithm is a classic block matching algorithm, is used as a typical algorithm of a real-time system, is low in complexity, high in speed and good in real-time performance, and an obtained dense disparity map can be used for dense three-dimensional reconstruction. However, when a traditional SAD algorithm is applied to a complex scene, a certain error is generated by directly summing the absolute values of the differences between corresponding numerical values of two pixel blocks, the accuracy of the obtained result is insufficient, and a parallax image has a void due to the change of parallax in a matching window, so that the reconstructed three-dimensional model is affected.
Disclosure of Invention
In view of this, the present application provides a three-dimensional reconstruction method and apparatus based on an improved binocular matching SAD algorithm to solve the problems in the related art.
The invention adopts the following technical scheme:
in a first aspect, an embodiment of the present invention provides a three-dimensional reconstruction method based on an improved binocular matching SAD algorithm, including:
acquiring an original picture; the original pictures comprise corresponding first pictures and second pictures;
obtaining an initial disparity map through an improved SAD algorithm based on the original picture; the improved SAD algorithm is biased to the matching precision of central pixels of image blocks of two pictures in the process of generating an initial disparity map;
and filling the holes of the initial disparity map to optimize the initial disparity map to obtain a final disparity map.
Optionally, the improved SAD algorithm is to introduce a weighting coefficient of two-dimensional gaussian distribution with a zero mean value on the basis of the original SAD algorithm, so that when matching is performed, the matching precision of the central pixel of the image block is emphasized.
Optionally, the obtaining an initial disparity map based on the original picture by using a modified SAD algorithm includes:
and solving the template image block in the first picture and the search image block in the second picture, and measuring whether the two image blocks are matched or not based on the sum of the absolute values of the pixel value differences introduced with the two-dimensional Gaussian weighting coefficients so as to obtain an initial disparity map.
Optionally, the filling the hole of the initial disparity map by using a hole filling method includes:
determining holes existing in the initial disparity map;
and aiming at the pixel points with the holes in the disparity map, selecting a window by using texture information measured by GLCM (global Gamut-nearest neighbor) to fill the holes.
Optionally, the selecting a window by using texture information measured by GLCM to fill the hole includes:
measuring areas with different sizes of textures around the cavity by introducing a gray level co-occurrence matrix to obtain Euclidean distances between adjacent areas;
selecting a region with the minimum corresponding Euclidean distance as a target region;
and (4) solving the parallax average value of the target area for filling the hole.
Optionally, the introducing a gray level co-occurrence matrix to measure different sizes of regions of the texture around the cavity to obtain the euclidean distance includes:
sequentially selecting areas with different sizes;
selecting an angular second moment, entropy, contrast and correlation to form a four-dimensional feature vector to comprehensively describe the texture attribute of the region in the image in the selected region;
and determining Euclidean distances between adjacent regions based on the four-dimensional feature vectors.
Optionally, the adjacent areas are: and arranging the areas with different sizes according to the sizes of the areas, wherein the areas with the adjacent size relationship are adjacent areas.
In a second aspect, the present application provides a three-dimensional reconstruction apparatus based on an improved binocular matching SAD algorithm, including:
the acquisition module is used for acquiring an original picture; the original pictures comprise corresponding first pictures and second pictures;
the initial disparity map generation module is used for obtaining an initial disparity map through an improved SAD algorithm based on the original picture; the improved SAD algorithm is biased to the matching precision of central pixels of image blocks of two pictures in the process of generating an initial disparity map;
and the filling module is used for filling the holes of the initial disparity map so as to optimize the initial disparity map.
In a third aspect, the present application provides a three-dimensional reconstruction device based on an improved binocular matching SAD algorithm, including:
a processor, and a memory coupled to the processor;
the memory is configured to store a computer program for performing at least the method for three-dimensional reconstruction based on the improved binocular matching SAD algorithm according to the first aspect of the present application;
the processor is used for calling and executing the computer program in the memory.
In a fourth aspect, the present application provides a storage medium storing a computer program that, when executed by a processor, implements a three-dimensional reconstruction method based on an improved binocular matching SAD algorithm according to the first aspect of the present application.
According to the technical scheme, the first picture and the second picture are shot through the preset camera device; after the original picture is obtained; obtaining an initial disparity map through an improved SAD algorithm based on the original picture; the improved SAD algorithm is biased to the matching precision of central pixels of image blocks of two pictures in the process of generating an initial disparity map; it should be noted that, based on the flatness of the existing camera device, the matching accuracy of the central pixels of the image blocks of the two pictures is emphasized, which is in accordance with objective practice, and the overall matching accuracy can be effectively improved; filling holes of the initial disparity map so as to optimize the initial disparity map. By the arrangement, the improved SAD algorithm is used in the algorithm, a better matching result is kept in the parallax smoothing process, the texture information is used for filling the holes in the algorithm, the hole area can be well reduced, the result is superior to that of the traditional SAD algorithm, and the algorithm has better practicability.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
Fig. 1 is a flowchart of a three-dimensional reconstruction method based on an improved binocular matching SAD algorithm according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of a binocular vision model provided by an embodiment of the present invention;
FIG. 3 is a schematic diagram of a gray level co-occurrence matrix generation process according to an embodiment of the present invention;
fig. 4 is a flowchart of a three-dimensional reconstruction method based on an improved binocular matching SAD algorithm according to an embodiment of the present invention
Fig. 5 is a schematic structural diagram of a three-dimensional reconstruction device based on an improved binocular matching SAD algorithm according to an embodiment of the present invention.
Fig. 6 is a schematic structural diagram of a three-dimensional reconstruction device based on an improved binocular matching SAD algorithm according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the technical solutions of the present invention will be described in detail below. It is to be understood that the described embodiments are merely exemplary of the invention, and not restrictive of the full scope of the invention. All other embodiments, which can be derived by a person skilled in the art from the examples given herein without any inventive step, are within the scope of the present invention.
First, an application scenario of the embodiment of the invention is explained, and the continuous increase of the production and use amount of dangerous chemicals puts higher requirements on the supervision of the storage safety of the dangerous chemicals. The three-dimensional reconstruction is carried out on the hazardous chemical substance warehouse, the environmental information around the hazardous chemical substance can be accurately acquired in real time, the visualization of the hazardous chemical substance warehouse environment is realized, and the real-time performance and the safety of monitoring and early warning are ensured. The binocular vision technology carries out three-dimensional reconstruction on the target based on the parallax principle, compared with the mode that a TOF camera, structured light, a laser radar and the like acquire scene depth information, the binocular vision technology is not limited by the material of the surface of an object, can acquire uniform and dense scene three-dimensional information, has low requirements on hardware and cost, and has certain application prospect in the aspect of safety monitoring in a hazardous chemical library.
The effect and precision of three-dimensional reconstruction are directly determined by a matching algorithm, and the key step is to search homonymous points in two images, namely to perform stereo matching. The Sum of Absolute Differences (SAD) algorithm is a classic block matching algorithm, is used as a typical algorithm of a real-time system, is low in complexity, high in speed and good in real-time performance, and an obtained dense disparity map can be used for dense three-dimensional reconstruction. However, when a traditional SAD algorithm is applied to a complex scene, a certain error is generated by directly summing the absolute values of the differences between corresponding numerical values of two pixel blocks, the accuracy of the obtained result is insufficient, and a parallax image has a void due to the change of parallax in a matching window, so that the reconstructed three-dimensional model is affected.
Examples
Fig. 1 is a flowchart of a three-dimensional reconstruction method based on an improved binocular matching SAD algorithm according to an embodiment of the present invention, which may be performed by the three-dimensional reconstruction apparatus based on the improved binocular matching SAD algorithm according to the embodiment of the present invention. Referring to fig. 1, the method may specifically include the following steps:
s101, acquiring an original picture; the original pictures comprise corresponding first pictures and second pictures;
the binocular vision technology is used for three-dimensional reconstruction of a target based on the parallax principle, and compared with the mode of acquiring scene depth information by a TOF camera, structured light, a laser radar and the like, the binocular vision technology is not limited by the material of the surface of an object, can acquire uniform and dense scene three-dimensional information, has low requirements on hardware and cost, and has certain application prospect in the aspect of safety monitoring in a hazardous chemical library.
The binocular stereoscopic vision principle is as follows: the binocular stereo vision imaging model is generally two cameras arranged in parallel, as shown in FIG. 2, two-phase camera optical center OlAnd OrThe distance between the cameras is a base line distance b, and the coordinates of a point p in the space at the left camera and the right camera are respectively pl(xl,yl) And pr(xr,yr)The parallax d is defined as the difference in the position of a point in the two imaging planes: d ═ xl-xr。
From the binocular vision model, Delta pOlOr∽△pplprAccording to the principle of triangle similarity, the following results are obtained:
Figure BDA0002742380160000061
wherein ZCThe depth of point b is shown, f is the focal length of the camera, and the parallax value d and the depth value Z can be obtained by the formula (1)CInversely proportional, as the disparity value d is smaller, the depth value ZCThe larger the distance the point is from the camera. Conversely, it indicates that the closer the point is to the camera. In practical situations, two cameras cannot be strictly arranged side by side, and y needs to be corrected through an external polar linel＝yrTherefore, the searching dimension is reduced from two dimensions to one dimension, the matching complexity is reduced, and the matching efficiency is improved. In step S101, the acquired picture is a picture taken by an image pickup device corresponding to positions of two eyes in the binocular vision model as shown in fig. 2.
S102, obtaining an initial disparity map through an improved SAD algorithm based on the original picture; the improved SAD algorithm is biased to the matching precision of central pixels of image blocks of two pictures in the process of generating an initial disparity map;
note that SAD is a block matching algorithm based on gray scale in image stereo matching. The principle of the method is to calculate the sum of absolute values of pixel value differences in a template image block and a search image block, and further measure whether the two image blocks are matched, and the definition is as follows:
Figure BDA0002742380160000071
wherein d is the parallax value, PL(i, j) represents the gray level of the pixel in the template image block, PR(i, j) represents the gray scale value of the pixel within the search image block. The smaller the SAD matrix value calculated using equation (2), the more similar the two image blocks are.
For a complex scene, matching image blocks by using the similarity of pixel point difference values in two pixel blocks, wherein once similar pixel values appear in pixel points of continuous image blocks in the scene, errors are inevitably generated in matching. In order to improve the matching precision of two image blocks, aiming at the SAD target function, a weighting coefficient omega (i, j) with zero mean value and two-dimensional Gaussian distribution is introduced:
Figure BDA0002742380160000072
Figure BDA0002742380160000073
where σ 2 is a variance, σ 2 is 0.5n, and n is a block radius.
The two-dimensional Gaussian distribution is popularized from one-dimensional normal distribution to two-dimensional distribution. As shown in fig. 2, the projections of the two-dimensional gaussian distribution with zero mean on the XOZ plane and the YOZ plane are both a standard normal distribution, and the projection on the XOY plane is an ellipse.
Combining equation (3) and equation (2) results in an improved SAD algorithm, as shown in equation (5):
Figure BDA0002742380160000074
the improved SAD after Gaussian distribution is introduced is more focused on the matching precision of the central pixel of the image block, and the method accords with objective practice and can effectively improve the overall matching precision. The Gaussian weights introduced by the algorithm can keep a better matching result in the parallax smoothing process.
S103, filling the holes of the initial disparity map to optimize the initial disparity map.
Specifically, after the weighted SAD algorithm is used to obtain an initial disparity map, it is found that when the disparity of the pixels in the matching block changes, a hole appears in the disparity map. In order to fill the cavity, a Gray-level Co-occurrence Matrix (GLCM) is introduced into the scheme provided by the application to measure the texture area around the cavity, and then the appropriate window size is selected to obtain the parallax average value to fill the cavity.
Wherein filling the holes of the initial disparity map fills the holes, including:
determining holes existing in the initial disparity map;
and aiming at the pixel points with the holes in the disparity map, selecting a window by using texture information measured by GLCM (global Gamut-nearest neighbor) to fill the holes.
The method comprises the following steps: "selecting a window for hole filling using texture information measured by GLCM" includes:
measuring areas with different sizes of textures around the cavity by introducing a gray level co-occurrence matrix to obtain Euclidean distances between adjacent areas; wherein the adjacent regions are: and arranging the areas with different sizes according to the sizes of the areas, wherein the areas with the adjacent size relationship are adjacent areas.
Specifically, areas with different sizes are sequentially selected; selecting an angular second moment, entropy, contrast and correlation to form a four-dimensional feature vector to comprehensively describe the texture attribute of the region in the image in the selected region;
and determining Euclidean distances between adjacent regions based on the four-dimensional feature vectors.
Selecting a region with the minimum corresponding Euclidean distance as a target region;
and (4) solving the parallax average value of the target area for filling the hole.
Step S103 is further described below with reference to fig. 3 and 4:
GLCM describes the texture of an image by studying the spatially dependent properties of the gray levels. GLCM is denoted by G (i, j) (i, j ═ 0,1,2, … L-1), where L denotes the gray level of the image, i and j denote the gray level of the pixels, d denotes the pitch of the two pixels, and n denotes the window size. θ is the generation direction of GLCM, and is typically taken at 0 °, 45 °, 90 ° and 135 °. Fig. 3 shows the process of generating a gray matrix with θ equal to 0 ° and d equal to 1: specifically, among the 14 texture features based on the GLCM, an Angular Second Moment (ASM), an Entropy (Entropy, ENT), a Contrast (CON), and a Correlation (COR) are selected to form a four-dimensional feature vector to comprehensively describe texture attributes of a region in an image.
As shown in equation (6), Euclidean Distance (ED) is used to describe the similarity of two adjacent texture region feature vectors (ASM1, ENT1, CON1, COR1) and (ASM2, ENT2, CON2, COR 2):
Figure BDA0002742380160000091
the larger the ED of the texture feature vectors of two adjacent regions is, the larger the difference of the texture attributes of the two regions is, the higher the probability that the two regions do not belong to the same object is, and the disparity mean value cannot be directly obtained and given to the position of the cavity; if the ED of the texture feature vectors of two adjacent regions is closer to 0, it indicates that the probability that the two regions belong to the same object is higher, and the average parallax can be assigned to the hole position. The self-adaptive adjustment of the parallax filling window is carried out according to the image texture information, so that the parallax filling can be correctly and effectively carried out, and a good parallax image can be obtained. The specific steps are shown in fig. 4:
s401, solving an initial disparity map by using a weighted SAD algorithm;
s402, inspecting holes and screening;
namely: and finding the holes in the initial disparity map.
S403, determining the position of the hole;
it should be noted that a plurality of holes may exist in one initial disparity map, and in the scheme provided in the present application, each hole needs to be filled one by one, so the position of the hole that needs to be filled at this time needs to be determined before each filling.
S404, solving ASM, ENT, CON and COR by taking the cavity as a center and different radiuses as windows;
s405, solving ED (edge distribution) by the radius of adjacent windows;
s406, averaging the minimum ED window to give a cavity;
it should be noted that, for one ED corresponds to two adjacent windows, in practical application, the larger the ED of the texture feature vectors of two adjacent regions is, the larger the difference of texture attributes of the two regions is, the higher the probability that the two regions do not belong to the same object is, and the disparity mean value cannot be directly obtained and given to the hole position; if the ED of the texture feature vectors of two adjacent regions is closer to 0, it indicates that the probability that the two regions belong to the same object is higher, and the average parallax can be assigned to the hole position. Therefore, the probability that the two adjacent windows and the hole corresponding to the minimum ED belong to the same object is high, and therefore, the hole portion can be filled based on any one of the image texture information of the two adjacent windows corresponding to the minimum ED. The self-adaptive adjustment of the parallax filling window is carried out according to the image texture information, so that the parallax filling can be correctly and effectively carried out, and a good parallax image can be obtained.
S407, judging whether filling is finished;
further, if yes, go straight to step S408; if not, go to step S403. In this way, after the filling of one cavity is completed, the filling of another cavity can be performed until the filling of the cavity is completed.
And S408, obtaining a final disparity map.
In conclusion; the scheme provided by the application is as follows: weighting is carried out on the SAD matching algorithm by introducing a two-dimensional Gaussian distribution coefficient, and after an initial disparity map is obtained, a window is selected by using texture information measured by GLCM (global Gaussian mixture model) to fill the hole aiming at the pixel points with the hole in the disparity map. Compared with the SAD algorithm, the disparity map obtained by the algorithm is more accurate, and preparation can be made for three-dimensional reconstruction of a later scene.
Fig. 5 is a schematic structural diagram of a three-dimensional reconstruction device based on an improved binocular matching SAD algorithm according to an embodiment of the present invention; referring to fig. 5, the present application provides a three-dimensional reconstruction apparatus based on an improved binocular matching SAD algorithm, including:
an obtaining module 501, configured to obtain an original picture; the original pictures comprise corresponding first pictures and second pictures;
an initial disparity map generation module 502, configured to obtain an initial disparity map through an improved SAD algorithm based on the original picture; the improved SAD algorithm is biased to the matching precision of central pixels of image blocks of two pictures in the process of generating an initial disparity map;
a filling module 503, configured to fill the holes of the initial disparity map to optimize the initial disparity map.
Fig. 6 is a schematic structural diagram of a three-dimensional reconstruction based on an improved binocular matching SAD algorithm according to an embodiment of the present invention. Referring to fig. 6, the present application provides a three-dimensional reconstruction apparatus based on an improved binocular matching SAD algorithm, including:
a processor 601, and a memory 602 connected to the processor 601;
the memory 602 is used for storing a computer program for performing at least the three-dimensional reconstruction method based on the improved binocular matching SAD algorithm according to any embodiment of the present application;
the processor is used for calling and executing the computer program in the memory.
The present application further provides a storage medium storing a computer program which, when executed by a processor, implements a method for three-dimensional reconstruction based on an improved binocular matching SAD algorithm according to any of the embodiments of the present application.
It is understood that the same or similar parts in the above embodiments may be mutually referred to, and the same or similar parts in other embodiments may be referred to for the content which is not described in detail in some embodiments.
It should be noted that the terms "first," "second," and the like in the description of the present invention are used for descriptive purposes only and are not to be construed as indicating or implying relative importance. Further, in the description of the present invention, the meaning of "a plurality" means at least two unless otherwise specified.
Any process or method descriptions in flow charts or otherwise described herein may be understood as representing modules, segments, or portions of code which include one or more executable instructions for implementing specific logical functions or steps of the process, and alternate implementations are included within the scope of the preferred embodiment of the present invention in which functions may be executed out of order from that shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved, as would be understood by those reasonably skilled in the art of the present invention.
It should be understood that portions of the present invention may be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, the various steps or methods may be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any one or combination of the following techniques, which are known in the art, may be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application specific integrated circuit having an appropriate combinational logic gate circuit, a Programmable Gate Array (PGA), a Field Programmable Gate Array (FPGA), or the like.
It will be understood by those skilled in the art that all or part of the steps carried by the method for implementing the above embodiments may be implemented by hardware related to instructions of a program, which may be stored in a computer readable storage medium, and when the program is executed, the program includes one or a combination of the steps of the method embodiments.
In addition, functional units in the embodiments of the present invention may be integrated into one processing module, or each unit may exist alone physically, or two or more units are integrated into one module. The integrated module can be realized in a hardware mode, and can also be realized in a software functional module mode. The integrated module, if implemented in the form of a software functional module and sold or used as a stand-alone product, may also be stored in a computer readable storage medium.
The storage medium mentioned above may be a read-only memory, a magnetic or optical disk, etc.
In the description herein, references to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, the schematic representations of the terms used above do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention, and that variations, modifications, substitutions and alterations can be made to the above embodiments by those of ordinary skill in the art within the scope of the present invention.

Claims (10)

1. A three-dimensional reconstruction method based on an improved binocular matching SAD algorithm is characterized by comprising the following steps:
acquiring an original picture; the original pictures comprise corresponding first pictures and second pictures;
obtaining an initial disparity map through an improved SAD algorithm based on the original picture; the improved SAD algorithm is biased to the matching precision of central pixels of image blocks of two pictures in the process of generating an initial disparity map;
and filling the holes of the initial disparity map to optimize the initial disparity map to obtain a final disparity map.
2. The three-dimensional reconstruction method based on the improved binocular matching SAD algorithm as claimed in claim 1, wherein the improved SAD algorithm is based on the original SAD algorithm, and introduces a weighting coefficient of two-dimensional Gaussian distribution with a zero mean value, so that when matching is performed, the matching precision of the central pixel of the image block is emphasized.
3. The method for three-dimensional reconstruction based on the improved binocular matching SAD algorithm according to claim 2, wherein the obtaining of the initial disparity map through the improved SAD algorithm based on the original picture comprises:
and solving the template image block in the first picture and the search image block in the second picture, and measuring whether the two image blocks are matched or not based on the sum of the absolute values of the pixel value differences introduced with the two-dimensional Gaussian weighting coefficients so as to obtain an initial disparity map.
4. The improved binocular matching SAD algorithm based three-dimensional reconstruction method according to claim 2, wherein the filling of the holes of the initial disparity map fills the holes, comprising:
determining holes existing in the initial disparity map;
and aiming at the pixel points with the holes in the disparity map, selecting a window by using texture information measured by GLCM (global Gamut-nearest neighbor) to fill the holes.
5. The method for three-dimensional reconstruction based on the improved binocular matching SAD algorithm according to claim 4, wherein the selecting of the window for hole filling using the texture information of GLCM metric comprises:
measuring areas with different sizes of textures around the cavity by introducing a gray level co-occurrence matrix to obtain Euclidean distances between adjacent areas;
selecting a region with the minimum corresponding Euclidean distance as a target region;
and (4) solving the parallax average value of the target area for filling the hole.
6. The three-dimensional reconstruction method based on the improved binocular matching SAD algorithm of claim 5, wherein the step of introducing the gray level co-occurrence matrix to measure the regions with different sizes of the textures around the holes to obtain Euclidean distances comprises the following steps:
sequentially selecting areas with different sizes;
selecting an angular second moment, entropy, contrast and correlation to form a four-dimensional feature vector to comprehensively describe the texture attribute of the region in the image in the selected region;
and determining Euclidean distances between adjacent regions based on the four-dimensional feature vectors.
7. The improved binocular matching SAD algorithm-based three-dimensional reconstruction method according to claim 5, wherein the adjacent areas are: and arranging the areas with different sizes according to the sizes of the areas, wherein the areas with the adjacent size relationship are adjacent areas.
8. A three-dimensional reconstruction device based on an improved binocular matching SAD algorithm is characterized by comprising:
the acquisition module is used for acquiring an original picture; the original pictures comprise corresponding first pictures and second pictures;
the initial disparity map generation module is used for obtaining an initial disparity map through an improved SAD algorithm based on the original picture; the improved SAD algorithm is biased to the matching precision of central pixels of image blocks of two pictures in the process of generating an initial disparity map;
and the filling module is used for filling the holes of the initial disparity map so as to optimize the initial disparity map.
9. A three-dimensional reconstruction device based on an improved binocular matching SAD algorithm is characterized by comprising:
a processor, and a memory coupled to the processor;
the memory is used for storing a computer program at least for executing the improved binocular matching SAD algorithm based three-dimensional reconstruction method of any one of claims 1 to 7;
the processor is used for calling and executing the computer program in the memory.
10. A storage medium, characterized in that the storage medium stores a computer program which, when executed by a processor, implements the three-dimensional reconstruction method based on the modified binocular matching SAD algorithm according to any one of claims 1 to 7.
CN202011154642.9A 2020-10-26 2020-10-26 Three-dimensional reconstruction method and device based on improved binocular matching SAD algorithm Active CN112258635B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011154642.9A CN112258635B (en) 2020-10-26 2020-10-26 Three-dimensional reconstruction method and device based on improved binocular matching SAD algorithm

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011154642.9A CN112258635B (en) 2020-10-26 2020-10-26 Three-dimensional reconstruction method and device based on improved binocular matching SAD algorithm

Publications (2)

Publication Number Publication Date
CN112258635A true CN112258635A (en) 2021-01-22
CN112258635B CN112258635B (en) 2023-07-21

Family

ID=74262989

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011154642.9A Active CN112258635B (en) 2020-10-26 2020-10-26 Three-dimensional reconstruction method and device based on improved binocular matching SAD algorithm

Country Status (1)

Country Link
CN (1) CN112258635B (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116363219A (en) * 2023-06-02 2023-06-30 中国科学技术大学 Binocular fire source image synthesis method, device and readable storage medium

Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120293627A1 (en) * 2010-10-27 2012-11-22 Yasunori Ishii 3d image interpolation device, 3d imaging apparatus, and 3d image interpolation method
CN103868460A (en) * 2014-03-13 2014-06-18 桂林电子科技大学 Parallax optimization algorithm-based binocular stereo vision automatic measurement method
CN106340036A (en) * 2016-08-08 2017-01-18 东南大学 Binocular stereoscopic vision-based stereo matching method
CN109249930A (en) * 2018-08-10 2019-01-22 重庆大数据研究院有限公司 A kind of intelligent vehicle collision safety prediction technique
CN109815943A (en) * 2019-03-18 2019-05-28 北京石油化工学院 A kind of harmful influence storage stacking picture sample generation method and system
KR101988551B1 (en) * 2018-01-15 2019-06-12 충북대학교 산학협력단 Efficient object detection and matching system and method using stereo vision depth estimation
CN109961092A (en) * 2019-03-04 2019-07-02 北京大学深圳研究生院 A kind of binocular vision solid matching method and system based on parallax anchor point
CN110473217A (en) * 2019-07-25 2019-11-19 沈阳工业大学 A kind of binocular solid matching process based on Census transformation
CN110567441A (en) * 2019-07-29 2019-12-13 广东星舆科技有限公司 Particle filter-based positioning method, positioning device, mapping and positioning method
CN111179333A (en) * 2019-12-09 2020-05-19 天津大学 Defocus fuzzy kernel estimation method based on binocular stereo vision

Patent Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120293627A1 (en) * 2010-10-27 2012-11-22 Yasunori Ishii 3d image interpolation device, 3d imaging apparatus, and 3d image interpolation method
CN103868460A (en) * 2014-03-13 2014-06-18 桂林电子科技大学 Parallax optimization algorithm-based binocular stereo vision automatic measurement method
CN106340036A (en) * 2016-08-08 2017-01-18 东南大学 Binocular stereoscopic vision-based stereo matching method
KR101988551B1 (en) * 2018-01-15 2019-06-12 충북대학교 산학협력단 Efficient object detection and matching system and method using stereo vision depth estimation
CN109249930A (en) * 2018-08-10 2019-01-22 重庆大数据研究院有限公司 A kind of intelligent vehicle collision safety prediction technique
CN109961092A (en) * 2019-03-04 2019-07-02 北京大学深圳研究生院 A kind of binocular vision solid matching method and system based on parallax anchor point
CN109815943A (en) * 2019-03-18 2019-05-28 北京石油化工学院 A kind of harmful influence storage stacking picture sample generation method and system
CN110473217A (en) * 2019-07-25 2019-11-19 沈阳工业大学 A kind of binocular solid matching process based on Census transformation
CN110567441A (en) * 2019-07-29 2019-12-13 广东星舆科技有限公司 Particle filter-based positioning method, positioning device, mapping and positioning method
CN111179333A (en) * 2019-12-09 2020-05-19 天津大学 Defocus fuzzy kernel estimation method based on binocular stereo vision

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
YUN ZHANG等: "EFFICIENT BINOCULAR STEREO MATCHING BASED ON SAD AND IMPROVED CENSUS TRANSFORMATION", pages 1 - 6 *
张钧涵等: "基于区域增长的自适应稠密匹配算法", no. 22, pages 68 - 70 *
胡高芮等: "基于二维图像的三维重建技术研究", vol. 46, no. 8, pages 27 - 31 *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116363219A (en) * 2023-06-02 2023-06-30 中国科学技术大学 Binocular fire source image synthesis method, device and readable storage medium
CN116363219B (en) * 2023-06-02 2023-08-11 中国科学技术大学 Binocular fire source image synthesis method, device and readable storage medium

Also Published As

Publication number Publication date
CN112258635B (en) 2023-07-21

Similar Documents

Publication Publication Date Title
WO2021140886A1 (en) Three-dimensional model generation method, information processing device, and program
CN109461181A (en) Depth image acquisition method and system based on pattern light
CN113034568B (en) Machine vision depth estimation method, device and system
US20120177284A1 (en) Forming 3d models using multiple images
EP3367334B1 (en) Depth estimation method and depth estimation apparatus of multi-view images
CN111123242B (en) Combined calibration method based on laser radar and camera and computer readable storage medium
US20130002827A1 (en) Apparatus and method for capturing light field geometry using multi-view camera
CN113494893B (en) Calibration method and device of three-dimensional laser scanning system and computer equipment
CN113205592B (en) Light field three-dimensional reconstruction method and system based on phase similarity
Ahmadabadian et al. Image selection in photogrammetric multi-view stereo methods for metric and complete 3D reconstruction
CN111915723A (en) Indoor three-dimensional panorama construction method and system
Gadasin et al. Reconstruction of a Three-Dimensional Scene from its Projections in Computer Vision Systems
CN114627491A (en) Single three-dimensional attitude estimation method based on polar line convergence
KR20040091877A (en) A method for estimation of fundamental matrix in implementing a stereo vision
CN111612731A (en) Measuring method, device, system and medium based on binocular microscopic vision
CN112258635B (en) Three-dimensional reconstruction method and device based on improved binocular matching SAD algorithm
JP2009530701A (en) Method for determining depth map from image, apparatus for determining depth map
JP2009530701A5 (en)
GB2569609A (en) Method and device for digital 3D reconstruction
CN108053481A (en) Generation method, device and the storage medium of three-dimensional point cloud normal vector
CN112525106A (en) Three-phase machine cooperative laser-based 3D detection method and device
CN113160416B (en) Speckle imaging device and method for coal flow detection
JP6991700B2 (en) Information processing equipment, information processing method, program
Wong et al. 3D object model reconstruction from image sequence based on photometric consistency in volume space
Zhang et al. Passive 3D reconstruction based on binocular vision

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant