CN111553850A - Three-dimensional information acquisition method and device based on binocular stereo vision - Google Patents
Three-dimensional information acquisition method and device based on binocular stereo vision Download PDFInfo
- Publication number
- CN111553850A CN111553850A CN202010234476.7A CN202010234476A CN111553850A CN 111553850 A CN111553850 A CN 111553850A CN 202010234476 A CN202010234476 A CN 202010234476A CN 111553850 A CN111553850 A CN 111553850A
- Authority
- CN
- China
- Prior art keywords
- image
- binocular
- filtering processing
- horizontal filtering
- parallax
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 47
- 238000001914 filtration Methods 0.000 claims abstract description 199
- 238000012545 processing Methods 0.000 claims abstract description 186
- 238000004364 calculation method Methods 0.000 claims abstract description 22
- 239000011159 matrix material Substances 0.000 claims description 19
- 238000004590 computer program Methods 0.000 claims description 13
- 230000008569 process Effects 0.000 claims description 11
- 238000013507 mapping Methods 0.000 claims description 7
- 238000004891 communication Methods 0.000 claims description 4
- 238000010586 diagram Methods 0.000 description 4
- 230000002159 abnormal effect Effects 0.000 description 3
- 238000004422 calculation algorithm Methods 0.000 description 3
- 238000007664 blowing Methods 0.000 description 2
- 238000006243 chemical reaction Methods 0.000 description 2
- 238000012937 correction Methods 0.000 description 2
- 238000007781 pre-processing Methods 0.000 description 2
- 230000001360 synchronised effect Effects 0.000 description 2
- 230000009466 transformation Effects 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 230000002776 aggregation Effects 0.000 description 1
- 238000004220 aggregation Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000011084 recovery Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/70—Denoising; Smoothing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/50—Depth or shape recovery
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
- G06T7/75—Determining position or orientation of objects or cameras using feature-based methods involving models
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Processing (AREA)
Abstract
The application relates to a three-dimensional information acquisition method and device based on binocular stereo vision. The method comprises the following steps: the method for acquiring the three-dimensional information based on the binocular stereo vision comprises the following steps: acquiring a binocular image; carrying out horizontal filtering processing on the binocular image to obtain a binocular image after the horizontal filtering processing; performing assignment processing on pixel points of the binocular image after the horizontal filtering processing to obtain a target image; calculating the parallax value of each pixel point in the target image and the binocular image after the horizontal filtering processing, and generating a parallax image according to the parallax value; and carrying out depth filtering processing on the parallax image to obtain three-dimensional information corresponding to the binocular image. By adopting the method, the calculation efficiency of the three-dimensional information can be improved by reducing the calculation time of the parallax image.
Description
Technical Field
The present application relates to the field of computer technologies, and in particular, to a method and an apparatus for acquiring three-dimensional information based on binocular stereo vision, a computer device, and a storage medium.
Background
Binocular Stereo Vision (Binocular Stereo Vision) is a method for acquiring corresponding three-dimensional information by acquiring Binocular images from different viewing angles by using image acquisition equipment based on a parallax principle and calculating position deviation between corresponding pixel points of the Binocular images. The traditional binocular stereo vision-based three-dimensional information acquisition method acquires three-dimensional information through a binocular matching algorithm based on block matching. In the traditional mode, a disparity image is obtained through a binocular vision disparity semi-global matching algorithm, and the whole image needs to be subjected to filtering preprocessing, so that the computation time of the disparity image is long, and the computation efficiency of three-dimensional information is low. Therefore, how to improve the calculation efficiency of three-dimensional information by reducing the calculation time of parallax images becomes a technical problem to be solved at present.
Disclosure of Invention
Based on this, it is necessary to provide a binocular stereo vision-based three-dimensional information acquisition method, apparatus, computer device, and storage medium capable of improving the calculation efficiency of three-dimensional information by reducing the calculation time of parallax images, in view of the above technical problems.
A three-dimensional information acquisition method based on binocular stereo vision, the method comprising:
acquiring a binocular image;
carrying out horizontal filtering processing on the binocular image to obtain a binocular image after the horizontal filtering processing;
performing assignment processing on pixel points of the binocular image after the horizontal filtering processing to obtain a target image;
calculating the parallax value of each pixel point in the target image and the binocular image after the horizontal filtering processing, and generating a parallax image according to the parallax value;
and carrying out depth filtering processing on the parallax image to obtain three-dimensional information corresponding to the binocular image.
In one embodiment, the performing depth filtering processing on the parallax image to obtain three-dimensional information corresponding to a binocular image includes:
acquiring a preset weight coefficient matrix;
performing depth filtering processing on the parallax image according to the preset weight coefficient matrix to obtain the parallax image after the depth filtering processing;
and obtaining three-dimensional information corresponding to the binocular image according to the parallax image.
In one embodiment, the assigning a pixel point of the binocular image after the horizontal filtering processing to obtain the target image includes:
comparing the pixel point information in the binocular image after the horizontal filtering processing with a preset gradient threshold value to obtain a comparison result;
performing assignment processing on pixel points of pixel point information in the corresponding binocular image after horizontal filtering processing according to the comparison result;
and mapping the pixel points subjected to assignment processing to the corresponding new images to obtain the target images.
In one embodiment, the binocular image includes a first image and a second image, the target image includes a target image corresponding to the first image after horizontal filtering processing and a target image corresponding to the first image after horizontal filtering processing, and the calculating the disparity value of each pixel point in the target image and the binocular image after horizontal filtering processing includes:
calculating a difference value of the sum of pixel values of pixel points in a target image corresponding to the first image after the horizontal filtering processing and pixel values of pixel points in a target image corresponding to the second image after the horizontal filtering processing according to the target image, and taking the difference value as a first difference value;
calculating a difference value of the sum of pixel values of pixel points in the first image after the horizontal filtering processing and pixel values of pixel points in the second image after the horizontal filtering processing according to the binocular image after the horizontal filtering processing, and taking the difference value as a second difference value;
and calculating the parallax value of each pixel point in the target image and the binocular image after the horizontal filtering processing according to the first difference value and the second difference value.
In one embodiment, before the performing the horizontal filtering process on the binocular image, the method further includes:
obtaining a calibration parameter;
and correcting the binocular image according to the calibration parameters to obtain a corrected binocular image.
A binocular stereo vision-based three-dimensional information acquisition apparatus, the apparatus comprising:
the communication module is used for acquiring binocular images;
the horizontal filtering module is used for carrying out horizontal filtering processing on the binocular images to obtain the binocular images after the horizontal filtering processing;
the assignment module is used for assigning the pixel points of the binocular image subjected to the horizontal filtering processing to obtain a target image;
the calculation module is used for calculating the parallax value of each pixel point in the target image and the binocular image after the horizontal filtering processing, and generating a parallax image according to the parallax value;
and the depth filtering module is used for performing depth filtering processing on the parallax image to obtain three-dimensional information corresponding to the binocular image.
In one embodiment, the depth filtering module is further configured to obtain a preset weight coefficient matrix; performing depth filtering processing on the parallax image according to the preset weight coefficient matrix to obtain the parallax image after the depth filtering processing; and obtaining three-dimensional information corresponding to the binocular image according to the parallax image.
In one embodiment, the calculation module is further configured to calculate, according to the target image, a difference between a sum of pixel values of a pixel point in the target image corresponding to the horizontally filtered first image and a pixel value of a pixel point in the target image corresponding to the horizontally filtered second image, as a first difference; calculating a difference value of the sum of pixel values of pixel points in the first image after the horizontal filtering processing and pixel values of pixel points in the second image after the horizontal filtering processing according to the binocular image after the horizontal filtering processing, and taking the difference value as a second difference value; and calculating the parallax value of each pixel point in the target image and the binocular image after the horizontal filtering processing according to the first difference value and the second difference value.
A computer device comprising a memory and a processor, the memory storing a computer program operable on the processor, the processor implementing the steps in the various method embodiments described above when executing the computer program.
A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the respective method embodiment described above.
According to the binocular stereo vision-based three-dimensional information acquisition method and device, the computer equipment and the storage medium, the binocular image is subjected to horizontal filtering, the pixel points of the binocular image subjected to horizontal filtering are subjected to assignment processing, the target image is obtained, abnormal pixel points can be corrected, and the accuracy of subsequent parallax calculation is improved. And calculating the parallax value of each pixel point in the target image and the binocular image after the horizontal filtering processing, generating a parallax image according to the parallax value, and performing depth filtering processing on the parallax image to obtain three-dimensional information corresponding to the binocular image. Because only horizontal filtering and subsequent depth filtering are needed, the filtering processing mode is simpler and faster, the calculation time can be reduced on the basis of ensuring the calculation accuracy of the disparity map, and the calculation efficiency of the three-dimensional information is improved. Meanwhile, horizontal filtering and blowing processing are carried out on the image, so that the accuracy of the parallax information in the horizontal direction is improved. The depth filtering processing is carried out on the image, so that partial missing parallax information in the parallax image is supplemented, the continuity of the parallax information is enhanced, and the three-dimensional information of the object can be more accurately expressed.
Drawings
Fig. 1 is an application environment diagram of a three-dimensional information acquisition method based on binocular stereo vision in one embodiment;
FIG. 2 is a schematic flow chart illustrating a three-dimensional information acquisition method based on binocular stereo vision in one embodiment;
fig. 3 is a schematic flow chart illustrating a step of performing depth filtering processing on a parallax image to obtain three-dimensional information corresponding to a binocular image in one embodiment;
FIG. 4 is a block diagram showing a three-dimensional information acquisition apparatus based on binocular stereo vision in one embodiment;
FIG. 5 is a diagram illustrating an internal structure of a computer device according to an embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.
The binocular stereo vision-based three-dimensional information acquisition method can be applied to the application environment shown in fig. 1. At least one image capture device 102 and a server 104 may be included. Image capture device 102 communicates with server 104 over a network. The image capturing device 102 transmits the captured binocular images at the plurality of viewing angles to the server 104, and the binocular images include a first image and a second image. The server 104 performs horizontal filtering processing on the binocular image to obtain the binocular image after the horizontal filtering processing. The server 104 performs assignment processing on the binocular image after the horizontal filtering processing to obtain a target image. The server 104 calculates a parallax value of each pixel point in the target image and the binocular image after the horizontal filtering processing, and generates a parallax image according to the parallax value. The server 104 performs depth filtering processing on the parallax image to obtain three-dimensional information corresponding to the binocular image. The image capturing device 102 may be, but is not limited to, a binocular camera or a binocular camera. The server 104 may be implemented as a stand-alone server or as a server cluster comprised of multiple servers.
In one embodiment, as shown in fig. 2, there is provided a three-dimensional information acquisition method based on binocular stereo vision, which is described by taking the method as an example applied to the server in fig. 1, and includes the following steps:
In the three-dimensional information acquisition process, binocular images under a plurality of visual angles can be acquired through the binocular camera. The binocular image includes a first image and a second image. Wherein the first image may be a left eye image. The second image may be a right eye image. An image acquisition interface is pre-configured in the server, and the server acquires binocular images acquired by the binocular cameras by calling the image acquisition interface. The binocular image can comprise a gray level image, the gray level image comprises depth information and surface information, such as image attribute information, object depth information, object surface information, scene illumination information and the like, contained discontinuous information is more comprehensive, the omission factor of horizontal edge pixel points can be reduced, and therefore the accuracy of subsequent horizontal filtering is improved.
And 204, performing horizontal filtering processing on the binocular image to obtain the binocular image after the horizontal filtering processing.
And step 206, carrying out assignment processing on the binocular image after the horizontal filtering processing to obtain a target image.
And the server respectively performs horizontal filtering processing on the acquired binocular images. By performing horizontal filtering processing on the binocular image, edge pixel points of the binocular image, namely horizontal edge pixel points of the first image and horizontal edge pixel points of the second image, can be detected. The operator for the binocular image horizontal filtering process may be a horizontal Sobel operator. The operator for carrying out horizontal filtering processing on the binocular image is verified, and the reliability of the operation result is good.
The gradient threshold value P is preset in the server. The gradient threshold is used for judging whether the horizontal edge pixel points are normal or not. And if not, assigning the pixel point. The server can assign values to the pixel points of the binocular image after the horizontal filtering processing according to the gradient threshold value. Specifically, the server may compare the gradient threshold with pixel point information of the binocular image after the horizontal filtering, that is, compare the pixel point information of the first image after the horizontal filtering with the pixel point information of the second image after the horizontal filtering, respectively, to obtain a corresponding comparison result. And the server assigns values to the pixel points of the first image after the horizontal filtering processing according to the comparison result, and maps the pixel points after the assignment processing to a new image to obtain a target image corresponding to the first image after the horizontal filtering processing. And performing assignment processing on pixel points of the filtered second image, and mapping the pixel points subjected to assignment processing to a new image to obtain a target image corresponding to the horizontally filtered second image.
And 208, calculating the parallax value of each pixel point in the target image and the binocular image after the horizontal filtering processing, and generating the parallax image according to the parallax value.
After the server obtains the target image, the server may calculate the parallax value, and then generate a corresponding parallax image. Specifically, the server calculates costs between corresponding pixel points in the target image and costs between pixel points corresponding to the binocular image after the horizontal filtering processing, so that the costs between the corresponding pixel points in the target image and the costs between the pixel points corresponding to the binocular image after the horizontal filtering processing are aggregated to obtain an aggregated cost. And then the server calculates the disparity value of each pixel point in the target image and the binocular image after horizontal filtering processing according to the calculated aggregation cost. After the disparity values are obtained, corresponding disparity images can be generated according to the disparity values.
And step 210, performing depth filtering processing on the parallax image to obtain three-dimensional information corresponding to the binocular image.
After the server generates the parallax image, the server may continue to perform depth filtering processing on the parallax image. The depth filtering process may be depth restoration of the parallax image, and conversion of the parallax image into a depth image is realized. The depth image includes depth (distance) information of the object. By carrying out depth filtering processing on the parallax images, partial missing parallax information in the parallax images is supplemented, the continuity of the parallax information is enhanced, and the calculation amount of the parallax images in the subsequent application process can be reduced. In addition, the depth image is not influenced by the irradiation direction of the detection signal of the transmitting end and the surface reflection characteristic of the object to be detected, and shadow does not exist, so that the three-dimensional information of the object can be more accurately expressed.
In the traditional mode, a disparity image is obtained through a binocular vision disparity semi-global matching algorithm, and the whole image needs to be subjected to filtering preprocessing, so that the computation time of the disparity image is long, and the computation efficiency of three-dimensional information is low. In the embodiment, horizontal filtering processing is performed on the binocular image, assignment processing is performed on pixel points of the binocular image after the horizontal filtering processing, a target image is obtained, abnormal pixel points can be corrected, and improvement of accuracy of subsequent parallax calculation is facilitated. And the server calculates the parallax value of each pixel point in the target image and the binocular image after the horizontal filtering processing, generates a parallax image according to the parallax value, and performs depth filtering processing on the parallax image to obtain three-dimensional information corresponding to the binocular image. Because only horizontal filtering and subsequent depth filtering are needed, the filtering processing mode is simpler and faster, the calculation time can be reduced on the basis of ensuring the calculation accuracy of the disparity map, and the calculation efficiency of the three-dimensional information is improved. Meanwhile, horizontal filtering and blowing processing are carried out on the image, so that the accuracy of the parallax information in the horizontal direction is improved. The depth filtering processing is carried out on the image, so that partial missing parallax information in the parallax image is supplemented, the continuity of the parallax information is enhanced, and the three-dimensional information of the object can be more accurately expressed.
In one embodiment, as shown in fig. 3, the method further comprises: the method comprises the following steps of carrying out depth filtering processing on the parallax image to obtain three-dimensional information corresponding to a binocular image, and specifically comprises the following steps:
And 304, performing depth filtering processing on the parallax image according to the preset weight coefficient matrix to obtain the parallax image after the depth filtering processing.
And step 306, obtaining three-dimensional information corresponding to the binocular image according to the parallax image.
The server is preset with a pre-weighting coefficient matrix, and the weighting coefficient matrix is used for weighting the parallax images so as to realize depth filtering processing. The server carries out depth filtering processing on the parallax images, and can carry out depth recovery on the parallax images to obtain depth images. Specifically, the server first performs minimization processing on the parallax image to obtain the parallax image after the minimization processing. And then, carrying out weighting processing on the minimized parallax image by using a preset weight coefficient matrix, and selecting an optimal weight value for carrying out depth filtering on the parallax image by adjusting the weight value of the minimized parallax image. And the server performs depth filtering processing on the parallax image according to the optimal weight value to obtain the parallax image after the depth filtering processing. The server can further obtain distance information of the object according to the parallax image after filtering processing, and three-dimensional information corresponding to the binocular image is generated according to the distance information, so that three-dimensional reconstruction can be performed on the object.
In this embodiment, the server obtains a preset weight coefficient matrix, performs depth filtering processing on the parallax image according to the preset weight coefficient matrix to obtain the parallax image after the depth filtering processing, and further obtains three-dimensional information corresponding to the binocular image according to the parallax image. The optimal weight value for performing depth filtering on the disparity map can be directly determined through the preset weight coefficient, and the calculation time of the three-dimensional information is reduced. Meanwhile, the image is subjected to depth filtering processing, so that partial missing parallax information in the parallax image is supplemented, the continuity of the parallax information is enhanced, and the three-dimensional information of the object can be more accurately expressed.
In one embodiment, assigning the pixel points of the binocular image after the horizontal filtering processing to obtain the target image includes: comparing the pixel point information in the binocular image after the horizontal filtering processing with a preset gradient threshold value to obtain a comparison result; performing assignment processing on pixel points of pixel point information in the corresponding binocular image after horizontal filtering processing according to the comparison result; and mapping the pixel points subjected to assignment processing into a new image to obtain a target image.
After the server carries out horizontal filtering processing on the binocular images, comparing pixel point information of the first image after the horizontal filtering processing in the binocular images after the horizontal filtering processing with a preset gradient threshold value P to obtain a first comparison result. And comparing the pixel point information of the second image after the horizontal filtering processing in the binocular image after the horizontal filtering processing with a preset gradient threshold value P to obtain a second comparison result. The pixel information may be a pixel value of a pixel.
And respectively carrying out assignment processing on the pixel points of the corresponding images according to the first comparison result and the second comparison result. And when the first comparison result or the second comparison result has pixel point information smaller than-P, the server assigns 0 to a pixel point corresponding to the pixel point information smaller than-P, and maps the pixel point to a new image to obtain a target image corresponding to the first image after horizontal filtering processing or a target image corresponding to the second image after horizontal filtering processing. And when the first comparison result or the second comparison result contains pixel point information which is larger than-P and smaller than P, the server adds the pixel point information and P to assign values to corresponding pixel points, and the pixel points are mapped to corresponding target images. And when the pixel point information larger than P exists in the first comparison result and the second comparison result, the server assigns the pixel point corresponding to the pixel point information to be 2P, and maps the pixel point to the corresponding target image.
In this embodiment, the server compares the pixel point information in the binocular image after the horizontal filtering processing with a preset gradient threshold, so as to assign the pixel points of the pixel point information in the binocular image after the horizontal filtering processing, and then map the pixel points after the assignment processing into a new image, thereby obtaining a target image. The abnormal pixel points can be assigned and corrected to obtain a more accurate target image, so that the accuracy of subsequent parallax value calculation is improved.
In one embodiment, the binocular image includes a first image and a second image, the target image includes a target image corresponding to the first image after horizontal filtering processing and a target image corresponding to the second image after horizontal filtering processing, and calculating the disparity value of each pixel point in the target image and the binocular image after horizontal filtering processing includes: calculating a difference value of the sum of pixel values of pixel points in a target image corresponding to the first image after horizontal filtering processing and pixel points in a target image corresponding to the second image after horizontal filtering processing according to the target image, and taking the difference value as a first difference value; calculating a difference value of the sum of pixel values of pixel points in the first image after horizontal filtering processing and pixel values of pixel points in the second image after horizontal filtering processing according to the binocular image after horizontal filtering processing, and taking the difference value as a second difference value; and calculating the parallax value of each pixel point in the target image and the binocular image after the horizontal filtering processing according to the first difference value and the second difference value.
The binocular image after the horizontal filtering processing comprises a first image after the horizontal filtering processing and a second image after the horizontal filtering processing. The target image comprises a target image corresponding to the first image after horizontal filtering processing and a target image corresponding to the second image after horizontal filtering processing.
The method for calculating the first difference by the server may be to construct a window by centering on a pixel point (x, y) in the target image corresponding to the first image after the horizontal filtering, cover the target image corresponding to the first image after the horizontal filtering with the window, select all pixel points covered by the window, and calculate the sum of pixel values of all pixel points in the target image corresponding to the first image after the horizontal filtering. Similarly, a window with the same size is constructed in the target image corresponding to the second image after the horizontal filtering processing, the target image corresponding to the second image after the horizontal filtering processing is covered by the window, all pixel points covered by the window are selected, the sum of pixel values of all pixel points in the target image corresponding to the second image after the horizontal filtering processing is calculated, therefore, the sum of two pixel values is subtracted, a first difference value is obtained, and the absolute value of the difference value is calculated. And temporarily keeping the window of the first image subjected to the horizontal filtering processing still, continuously moving the window of the second image subjected to the horizontal filtering processing within a certain range around the window coverage area of the first image subjected to the horizontal filtering processing until the minimum value of the absolute value of the difference is obtained, and taking the offset of the coverage area as the sub-parallax value of the pixel point (x, y) in the path direction, namely the moving direction. And the server calculates and obtains parallax values in a plurality of path directions according to the mode, aggregates the parallax values in the plurality of directions, and determines the parallax value of the pixel point in the aggregated parallax values. For example, the server may perform the determination of the disparity value using a WTA (Winner take all) policy. The manner of calculating the second difference by the server is the manner of calculating the first difference as described above, and details are not repeated here.
In this embodiment, the server calculates a difference between a sum of pixel values of a pixel point in the target image corresponding to the first image after the horizontal filtering and a pixel point in the target image corresponding to the second image after the horizontal filtering, as the first difference. And calculating the difference value of the sum of the pixel values of the pixel points in the first image after the horizontal filtering processing and the pixel points in the second image after the horizontal filtering processing according to the binocular image after the horizontal filtering processing, and taking the difference value as a second difference value. And then calculating the parallax value of each pixel point in the target image and the binocular image after horizontal filtering processing according to the first difference value and the second difference value. The parallax value of each pixel point is calculated in a region matching mode, the search range is small, and the calculation efficiency of the parallax value is improved.
In one embodiment, before the horizontal filtering processing is performed on the binocular image, the method further includes: obtaining a calibration parameter; and correcting the binocular image according to the calibration parameters to obtain the corrected binocular image.
In the three-dimensional information acquisition process, binocular images under a plurality of visual angles can be acquired through the binocular camera. The binocular image includes a first image and a second image. Wherein the first image may be a left eye image. The second image may be a right eye image. The binocular camera is pre-calibrated. The server can obtain calibration parameters of the binocular camera. The calibration parameters may include distortion parameters. And the server calculates a mapping transformation matrix for distortion correction of the binocular image through the calibration parameters, and performs coordinate conversion on pixel points needing to be corrected in the binocular image according to the mapping transformation matrix, so as to obtain the corrected binocular image.
In this embodiment, the server corrects the binocular image through the calibration parameters to obtain the corrected binocular image, so that a more accurate binocular image can be obtained, and subsequent horizontal filtering processing of the binocular image is facilitated.
It should be understood that although the steps in the flowcharts of fig. 2 to 3 are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least some of the steps in fig. 2-3 may include multiple sub-steps or multiple stages that are not necessarily performed at the same time, but may be performed at different times, and the order of performing the sub-steps or stages is not necessarily sequential, but may be performed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.
In one embodiment, as shown in fig. 4, there is provided a binocular stereo-based three-dimensional information acquisition apparatus including: a communication module 402, a horizontal filtering module 404, an assignment module 406, a calculation module 408, and a depth filtering module 410, wherein:
a communication module 402 for acquiring binocular images.
And the horizontal filtering module 404 is configured to perform horizontal filtering processing on the binocular image to obtain a binocular image after the horizontal filtering processing.
And the assignment module 406 is configured to perform assignment processing on the pixel points of the binocular image after the horizontal filtering processing to obtain a target image.
The calculating module 408 is configured to calculate a disparity value of each pixel in the target image and the binocular image after the horizontal filtering processing, and generate a disparity image according to the disparity value.
And the depth filtering module 410 is configured to perform depth filtering processing on the parallax image to obtain three-dimensional information corresponding to the binocular image.
In one embodiment, the depth filtering module 410 is further configured to obtain a preset weight coefficient matrix; performing depth filtering processing on the parallax image according to a preset weight coefficient matrix to obtain the parallax image after the depth filtering processing; and obtaining three-dimensional information corresponding to the binocular image according to the parallax image.
In one embodiment, the assignment module 406 is further configured to compare pixel point information in the binocular image after the horizontal filtering processing with a preset gradient threshold, so as to obtain a comparison result; performing assignment processing on pixel points of pixel point information in the corresponding binocular image after horizontal filtering processing according to the comparison result; and mapping the pixel points subjected to assignment processing to the corresponding new images to obtain the target images.
In one embodiment, the calculating module 408 is further configured to calculate, as the first difference, a difference between a sum of pixel values of a pixel point in the target image corresponding to the first image after the horizontal filtering process and a pixel point in the target image corresponding to the second image after the horizontal filtering process according to the target image; calculating a difference value of the sum of pixel values of pixel points in the first image after horizontal filtering processing and pixel values of pixel points in the second image after horizontal filtering processing according to the binocular image after horizontal filtering processing, and taking the difference value as a second difference value; and calculating the parallax value of each pixel point in the target image and the binocular image after the horizontal filtering processing according to the first difference value and the second difference value.
In one embodiment, the above apparatus further comprises: the correction module is used for acquiring calibration parameters; and correcting the binocular image according to the calibration parameters to obtain the corrected binocular image.
Specific limitations regarding the binocular stereo vision based three-dimensional information acquisition apparatus can be found in the above limitations regarding the binocular stereo vision based three-dimensional information acquisition method, which are not described in detail herein. All or part of the modules in the binocular stereo vision-based three-dimensional information acquisition device can be realized by software, hardware and a combination thereof. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.
In one embodiment, a computer device is provided, which may be a server, the internal structure of which may be as shown in fig. 5. The computer device includes a processor, a memory, a network interface, and a database connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The database of the computer equipment is used for storing three-dimensional information corresponding to the binocular images. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement a binocular stereo-based three-dimensional information acquisition method.
Those skilled in the art will appreciate that the architecture shown in fig. 5 is merely a block diagram of some of the structures associated with the disclosed aspects and is not intended to limit the computing devices to which the disclosed aspects apply, as particular computing devices may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.
In one embodiment, a computer device is provided, comprising a memory storing a computer program and a processor implementing the steps of the various embodiments described above when the processor executes the computer program.
In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the respective embodiments described above.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in the embodiments provided herein may include non-volatile and/or volatile memory, among others. Non-volatile memory can include read-only memory (ROM), Programmable ROM (PROM), Electrically Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), Rambus Direct RAM (RDRAM), direct bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).
The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.
Claims (10)
1. A three-dimensional information acquisition method based on binocular stereo vision is characterized by comprising the following steps:
acquiring a binocular image;
carrying out horizontal filtering processing on the binocular image to obtain a binocular image after the horizontal filtering processing;
performing assignment processing on pixel points of the binocular image after the horizontal filtering processing to obtain a target image;
calculating the parallax value of each pixel point in the target image and the binocular image after the horizontal filtering processing, and generating a parallax image according to the parallax value;
and carrying out depth filtering processing on the parallax image to obtain three-dimensional information corresponding to the binocular image.
2. The method according to claim 1, wherein the depth filtering the parallax image to obtain three-dimensional information corresponding to a binocular image comprises:
acquiring a preset weight coefficient matrix;
performing depth filtering processing on the parallax image according to the preset weight coefficient matrix to obtain the parallax image after the depth filtering processing;
and obtaining three-dimensional information corresponding to the binocular image according to the parallax image.
3. The method according to claim 1, wherein the assigning the pixel points of the binocular image after the horizontal filtering process to obtain the target image comprises:
comparing the pixel point information in the binocular image after the horizontal filtering processing with a preset gradient threshold value to obtain a comparison result;
performing assignment processing on pixel points of pixel point information in the corresponding binocular image after horizontal filtering processing according to the comparison result;
and mapping the pixel points subjected to assignment processing to the corresponding new images to obtain the target images.
4. The method of claim 1, wherein the binocular images comprise a first image and a second image, the target image comprises a target image corresponding to the first image after horizontal filtering processing and a target image corresponding to the first image after horizontal filtering processing, and the calculating the disparity value of each pixel point in the target image and the binocular images after horizontal filtering processing comprises:
calculating a difference value of the sum of pixel values of pixel points in a target image corresponding to the first image after the horizontal filtering processing and pixel values of pixel points in a target image corresponding to the second image after the horizontal filtering processing according to the target image, and taking the difference value as a first difference value;
calculating a difference value of the sum of pixel values of pixel points in the first image after the horizontal filtering processing and pixel values of pixel points in the second image after the horizontal filtering processing according to the binocular image after the horizontal filtering processing, and taking the difference value as a second difference value;
and calculating the parallax value of each pixel point in the target image and the binocular image after the horizontal filtering processing according to the first difference value and the second difference value.
5. The method according to any one of claims 1 to 4, further comprising, before the horizontal filtering processing of the binocular images:
obtaining a calibration parameter;
and correcting the binocular image according to the calibration parameters to obtain a corrected binocular image.
6. A binocular stereo vision-based three-dimensional information acquisition apparatus, comprising:
the communication module is used for acquiring binocular images;
the horizontal filtering module is used for carrying out horizontal filtering processing on the binocular images to obtain the binocular images after the horizontal filtering processing;
the assignment module is used for assigning the pixel points of the binocular image subjected to the horizontal filtering processing to obtain a target image;
the calculation module is used for calculating the parallax value of each pixel point in the target image and the binocular image after the horizontal filtering processing, and generating a parallax image according to the parallax value;
and the depth filtering module is used for performing depth filtering processing on the parallax image to obtain three-dimensional information corresponding to the binocular image.
7. The apparatus of claim 6, wherein the depth filtering module is further configured to obtain a preset weight coefficient matrix; performing depth filtering processing on the parallax image according to the preset weight coefficient matrix to obtain the parallax image after the depth filtering processing; and obtaining three-dimensional information corresponding to the binocular image according to the parallax image.
8. The apparatus according to claim 6, wherein the calculating module is further configured to calculate, as the first difference, a difference between a sum of pixel values of a pixel point in the target image corresponding to the horizontally filtered first image and a pixel point in the target image corresponding to the horizontally filtered second image according to the target image; calculating a difference value of the sum of pixel values of pixel points in the first image after the horizontal filtering processing and pixel values of pixel points in the second image after the horizontal filtering processing according to the binocular image after the horizontal filtering processing, and taking the difference value as a second difference value; and calculating the parallax value of each pixel point in the target image and the binocular image after the horizontal filtering processing according to the first difference value and the second difference value.
9. A computer device comprising a memory and a processor, the memory storing a computer program operable on the processor, wherein the processor implements the steps of the method of any one of claims 1 to 5 when executing the computer program.
10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010234476.7A CN111553850B (en) | 2020-03-30 | 2020-03-30 | Three-dimensional information acquisition method and device based on binocular stereoscopic vision |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010234476.7A CN111553850B (en) | 2020-03-30 | 2020-03-30 | Three-dimensional information acquisition method and device based on binocular stereoscopic vision |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111553850A true CN111553850A (en) | 2020-08-18 |
| CN111553850B CN111553850B (en) | 2023-10-24 |
Family
ID=72002381
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010234476.7A Active CN111553850B (en) | 2020-03-30 | 2020-03-30 | Three-dimensional information acquisition method and device based on binocular stereoscopic vision |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111553850B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112767293A (en) * | 2021-01-11 | 2021-05-07 | 达闼机器人有限公司 | Method for acquiring parallax image, electronic device and storage medium |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2010113389A1 (en) * | 2009-03-31 | 2010-10-07 | パナソニック株式会社 | Stereo image processor and stereo image processing method |
| US20110044531A1 (en) * | 2007-11-09 | 2011-02-24 | Thomson Licensing | System and method for depth map extraction using region-based filtering |
| US20150124062A1 (en) * | 2013-11-04 | 2015-05-07 | Massachusetts Institute Of Technology | Joint View Expansion And Filtering For Automultiscopic 3D Displays |
| DE102015011674A1 (en) * | 2015-09-11 | 2016-03-17 | Daimler Ag | Method for depth filtering of depth images |
| CN106412560A (en) * | 2016-09-28 | 2017-02-15 | 湖南优象科技有限公司 | Three-dimensional image generating method based on depth map |
| CN107424143A (en) * | 2017-04-13 | 2017-12-01 | 中国矿业大学 | A kind of mine belt conveyor coal quantity measuring method based on binocular stereo vision depth perception |
| CN108564539A (en) * | 2018-01-04 | 2018-09-21 | 网宿科技股份有限公司 | A kind of method and apparatus of display image |
| CN110349251A (en) * | 2019-06-28 | 2019-10-18 | 深圳数位传媒科技有限公司 | A kind of three-dimensional rebuilding method and device based on binocular camera |
-
2020
- 2020-03-30 CN CN202010234476.7A patent/CN111553850B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20110044531A1 (en) * | 2007-11-09 | 2011-02-24 | Thomson Licensing | System and method for depth map extraction using region-based filtering |
| WO2010113389A1 (en) * | 2009-03-31 | 2010-10-07 | パナソニック株式会社 | Stereo image processor and stereo image processing method |
| US20150124062A1 (en) * | 2013-11-04 | 2015-05-07 | Massachusetts Institute Of Technology | Joint View Expansion And Filtering For Automultiscopic 3D Displays |
| DE102015011674A1 (en) * | 2015-09-11 | 2016-03-17 | Daimler Ag | Method for depth filtering of depth images |
| CN106412560A (en) * | 2016-09-28 | 2017-02-15 | 湖南优象科技有限公司 | Three-dimensional image generating method based on depth map |
| CN107424143A (en) * | 2017-04-13 | 2017-12-01 | 中国矿业大学 | A kind of mine belt conveyor coal quantity measuring method based on binocular stereo vision depth perception |
| CN108564539A (en) * | 2018-01-04 | 2018-09-21 | 网宿科技股份有限公司 | A kind of method and apparatus of display image |
| CN110349251A (en) * | 2019-06-28 | 2019-10-18 | 深圳数位传媒科技有限公司 | A kind of three-dimensional rebuilding method and device based on binocular camera |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112767293A (en) * | 2021-01-11 | 2021-05-07 | 达闼机器人有限公司 | Method for acquiring parallax image, electronic device and storage medium |
| CN112767293B (en) * | 2021-01-11 | 2023-04-11 | 达闼机器人股份有限公司 | Method for acquiring parallax image, electronic device and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111553850B (en) | 2023-10-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109737874B (en) | Object size measuring method and device based on three-dimensional vision technology | |
| US11010924B2 (en) | Method and device for determining external parameter of stereoscopic camera | |
| CN107316326B (en) | Edge-based disparity map calculation method and device applied to binocular stereo vision | |
| CN110493488B (en) | Video image stabilization method, video image stabilization device and computer readable storage medium | |
| CN111160232B (en) | Front face reconstruction method, device and system | |
| CN111199523B (en) | Power equipment identification method, device, computer equipment and storage medium | |
| CN110766629B (en) | CBCT system geometric correction method, device, computer equipment and storage medium | |
| CN112106111A (en) | Calibration method, calibration equipment, movable platform and storage medium | |
| CN112580434A (en) | Face false detection optimization method and system based on depth camera and face detection equipment | |
| CN111915541B (en) | Image enhancement processing method, device, equipment and medium based on artificial intelligence | |
| CN116029996A (en) | Stereo matching method and device and electronic equipment | |
| CN113902808A (en) | Camera calibration method, device, equipment and storage medium | |
| CN111445487A (en) | Image segmentation method and device, computer equipment and storage medium | |
| CN109658451B (en) | Depth sensing method and device and depth sensing equipment | |
| CN110322477B (en) | Feature point observation window setting method, tracking method, device, equipment and medium | |
| CN111553850B (en) | Three-dimensional information acquisition method and device based on binocular stereoscopic vision | |
| CN107403448B (en) | Cost function generation method and cost function generation device | |
| CN112712536B (en) | Image processing method, chip and electronic device | |
| CN112634372B (en) | Real-time binocular camera correction method, device and storage medium | |
| CN113888438A (en) | Image processing method, device and storage medium | |
| CN111967484A (en) | Point cloud clustering method and device, computer equipment and storage medium | |
| CN114331919B (en) | Depth recovery method, electronic device, and storage medium | |
| CN108389167B (en) | Image processing method, image processing device, computer equipment and storage medium | |
| CN110458768B (en) | Image processing method, computer device, and readable storage medium | |
| CN109816613B (en) | Image completion method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |