WO2021217444A1 - 深度图生成方法、电子设备、计算处理设备及存储介质 - Google Patents

深度图生成方法、电子设备、计算处理设备及存储介质 Download PDF

Info

Publication number
WO2021217444A1
WO2021217444A1 PCT/CN2020/087569 CN2020087569W WO2021217444A1 WO 2021217444 A1 WO2021217444 A1 WO 2021217444A1 CN 2020087569 W CN2020087569 W CN 2020087569W WO 2021217444 A1 WO2021217444 A1 WO 2021217444A1
Authority
WO
WIPO (PCT)
Prior art keywords
target image
image
pixel
target
mapping relationship
Prior art date
Application number
PCT/CN2020/087569
Other languages
English (en)
French (fr)
Inventor
周游
刘洁
徐彪
Original Assignee
深圳市大疆创新科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳市大疆创新科技有限公司 filed Critical 深圳市大疆创新科技有限公司
Priority to PCT/CN2020/087569 priority Critical patent/WO2021217444A1/zh
Priority to CN202080044087.6A priority patent/CN113994382A/zh
Publication of WO2021217444A1 publication Critical patent/WO2021217444A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/50—Depth or shape recovery
    • G06T7/55—Depth or shape recovery from multiple images

Definitions

  • This application relates to the technical field of data processing, in particular to a method for generating a depth map, an electronic device, a computing processing device, and a computer-readable storage medium.
  • Computer vision relies on imaging systems to replace visual organs as input-sensitive means.
  • the most commonly used camera is a camera.
  • a basic vision system can be composed of dual cameras, called Stereo Vision.
  • Stereo Vision System uses two cameras to take two photos at the same time and at different angles, and then through the difference between the two photos, and the position and angle relationship between the dual cameras, using the triangle relationship, you can Calculate the distance relationship between the scene and the camera, and draw it on a picture as a Depth Map (depth map or depth map).
  • Depth Map depth map or depth map
  • the commonly used depth calculation method is to use the Stereo Matching (binocular stereo matching) algorithm, in which it is necessary to search for the most matching block in the baseline direction of the dual cameras in the two pictures.
  • Stereo Matching binocular stereo matching
  • this application is proposed to provide a depth map generation method, electronic equipment, computing processing equipment, and computer-readable storage medium that overcome the above-mentioned problems or at least partially solve the above-mentioned problems.
  • a method for generating a depth map including:
  • the at least two matching degrees corresponding to it are fused, and the first depth corresponding to the target image is generated based on the fused matching degrees corresponding to each of the target pixels Figure;
  • the depth values are respectively determined according to the at least two matching degrees, to obtain at least two depth values corresponding to each pixel on the target image, according to at least two corresponding to each pixel on the target image Depth value, generating a second depth map corresponding to the target image.
  • the fusing the at least two matching degrees corresponding to the same target pixel includes:
  • the at least two matching degrees corresponding to the same target pixel are subjected to a sum operation to obtain the fused matching degree corresponding to the target pixel.
  • the generating a second depth map corresponding to the target image according to at least two depth values corresponding to each of the target pixel points includes:
  • the smallest depth value is selected as the first depth value of the pixel
  • a second depth map corresponding to the target image is generated.
  • the generating a second depth map corresponding to the target image based on the first depth value corresponding to each pixel on the target image includes:
  • a second depth map corresponding to the target image is generated.
  • the method further includes:
  • the target image and the image to be matched are respectively corrected to obtain the corrected target image and the image to be matched, and the mapping relationship between the corrected target image and the target image.
  • the calculating at least two matching degrees between the target pixel on the target image and the matching pixel on each of the other images in the at least three images includes:
  • the target pixel points on the corrected target image are matched with the pixel points on the corrected image to be matched on the target image, and each target pixel point on the corrected target image is calculated and corrected respectively.
  • the matching degree between the matching pixels on the matched image is a predefined range between the matching pixels on the matched image.
  • the mapping relationship includes a first mapping relationship obtained by correcting the target image and the first image and a second mapping relationship obtained by correcting the target image and the second image; Before fusing the at least two matching degrees corresponding to one of the target pixels, the method further includes:
  • the matching degree corresponding to the corrected target image corresponding to the second image is converted into the matching degree corresponding to the corrected target image corresponding to the first image.
  • the mapping relationship includes a fourth mapping relationship obtained by rectifying the target image and the third image; for the same target pixel, the at least two corresponding ones are matched
  • the method further includes:
  • the matching degree corresponding to the corrected target image corresponding to the third image is converted into the matching degree corresponding to the target image.
  • the respectively determining depth values according to the at least two matching degrees to obtain at least two depth values corresponding to each pixel on the target image includes:
  • the depth value is calculated to obtain the corresponding pixel on the target image At least two depth values.
  • the mapping relationship includes a first mapping relationship obtained by correcting the target image and the first image and a second mapping relationship obtained by correcting the target image and the second image;
  • the method further includes:
  • the depth value corresponding to the corrected target image corresponding to the second image is converted into the depth value corresponding to the corrected target image corresponding to the first image.
  • the mapping relationship includes a fourth mapping relationship obtained by rectifying the target image and the third image; in the step, the at least two depth values corresponding to each pixel on the target image are generated.
  • the method further includes:
  • the depth value corresponding to the corrected target image corresponding to the third image is converted into the depth value corresponding to the target image.
  • the positional relationship between the photographing devices includes any one of a pin shape or an L shape.
  • the at least three photographing devices are provided on an electronic device, and the electronic device includes any one of a movable platform, a mobile terminal, a virtual reality terminal, or an augmented reality terminal.
  • the at least three photographing devices are arranged on a movable platform, and the method further includes:
  • the motion trajectory of the movable platform or the operation trajectory of the mechanical arm on the movable platform is determined.
  • an electronic device including a processor, a memory, and at least three photographing devices;
  • the processor is configured to: acquire at least three images captured by at least three photographing devices; calculate at least two of the target pixel points on the target image and the matching pixel points on the other images in the at least three images Matching degree; for the same target pixel point, the at least two matching degrees corresponding to it are fused, and based on the fused matching degree corresponding to each target pixel point, the corresponding target image is generated
  • the first depth map; or, the depth values are respectively determined according to the at least two matching degrees to obtain at least two depth values corresponding to each pixel on the target image, and according to the corresponding pixel on the target image At least two depth values are used to generate a second depth map corresponding to the target image.
  • the processor when the processor merges the at least two matching degrees corresponding to the same target pixel, it is configured to:
  • the at least two matching degrees corresponding to the same target pixel are subjected to a sum operation to obtain the fused matching degree corresponding to the target pixel.
  • the processor is configured to: when generating a second depth map corresponding to the target image according to at least two depth values corresponding to each pixel on the target image:
  • the smallest depth value is selected as the first depth value of the pixel
  • a second depth map corresponding to the target image is generated.
  • the processor is configured to: when generating a second depth map corresponding to the target image based on the first depth value corresponding to each pixel on the target image:
  • a second depth map corresponding to the target image is generated.
  • the processor is further configured to :
  • the target image and the image to be matched are respectively corrected to obtain the corrected target image and the image to be matched, and the mapping relationship between the corrected target image and the target image.
  • the processor is configured to: when calculating at least two matching degrees between the target pixel on the target image and the matching pixel on each of the other images in the at least three images:
  • the target pixel points on the corrected target image are matched with the pixel points on the corrected image to be matched on the target image, and each target pixel point on the corrected target image is calculated and corrected respectively.
  • the matching degree between the matching pixels on the matched image is a predefined range between the matching pixels on the matched image.
  • the mapping relationship includes a first mapping relationship obtained by correcting the target image and the first image, and a second mapping relationship obtained by correcting the target image and the second image; in the processor Before fusing the at least two matching degrees corresponding to the same target pixel point, the processor is further configured to:
  • the matching degree corresponding to the corrected target image corresponding to the second image is converted into the matching degree corresponding to the corrected target image corresponding to the first image.
  • the mapping relationship includes a fourth mapping relationship obtained by rectifying the target image and the third image; for the same target pixel point, the processor compares the at least two corresponding target pixels. Before fusion of the two matching degrees, the processor is also used to:
  • the matching degree corresponding to the corrected target image corresponding to the third image is converted into the matching degree corresponding to the target image.
  • the processor when the processor respectively determines depth values according to the at least two matching degrees, and obtains at least two depth values corresponding to each pixel on the target image, it is used to:
  • the depth value is calculated respectively to obtain at least two depths corresponding to each pixel point on the target image value.
  • the mapping relationship includes a first mapping relationship obtained by correcting the target image and the first image, and a second mapping relationship obtained by correcting the target image and the second image; in the processor Before generating a second depth map corresponding to the target image according to at least two depth values corresponding to each pixel on the target image, the processor is further configured to:
  • the depth value corresponding to the corrected target image corresponding to the second image is converted into the depth value corresponding to the corrected target image corresponding to the first image.
  • the mapping relationship includes a fourth mapping relationship obtained by rectifying the target image and the third image; at the processor according to at least two depth values corresponding to each pixel on the target image, Before generating the second depth map corresponding to the target image, the processor is further configured to:
  • the depth value corresponding to the corrected target image corresponding to the third image is converted into the depth value corresponding to the target image.
  • the positional relationship between the photographing devices includes any one of a pin shape or an L shape.
  • the electronic device includes any one of a movable platform, a mobile terminal, a virtual reality terminal, or an augmented reality terminal.
  • the electronic device is a movable platform
  • the processor is further configured to:
  • the motion trajectory of the movable platform or the operation trajectory of the mechanical arm on the movable platform is determined.
  • the electronic device includes a display, and the display is used for:
  • the processor is further configured to:
  • a computer program including computer readable code, which when the computer readable code runs on a computing processing device, causes the computing processing device to execute the above-mentioned depth map generation method.
  • the embodiment of the present invention by acquiring at least three images captured by at least three photographing devices, at least two points between the target pixel points on the target image and the matching pixel points on the other images in the at least three images are calculated.
  • the depth map is generated after fusion, or the depth map is generated by combining at least two matching degrees corresponding to the determined depth value, which realizes the combination of at least two calculation results and solves the
  • the target image corresponding to When performing autonomous obstacle avoidance, avoid unnecessary collisions or other accidents caused by choosing a larger depth value. Choosing the smallest depth value is more secure and improves the success rate of autonomous obstacle avoidance.
  • a second depth map corresponding to the target image is generated, and a second depth map corresponding to the target image is generated.
  • the first depth value corresponding to each pixel on the image is filtered, which can filter out some wrong depth values and obtain a more accurate depth map.
  • the collision or falling of the movable platform with surrounding objects can be avoided.
  • the distance between the robot arm and surrounding objects can be determined, and the operation trajectory of the robot arm can be generated to avoid collisions between the robot arm and surrounding objects, or realize the operation of the robot arm on the surrounding target objects .
  • the depth map obtained by using the technical solutions of the embodiments of the present invention is more accurate. Therefore, the aircraft can perform better autonomous obstacle avoidance, especially rotorcraft.
  • the car can also perform better autonomous obstacle avoidance to avoid collisions with other objects or people, so that the car has Higher safety;
  • the robot can also perform better autonomous obstacle avoidance, and the robot arm can operate more accurately on objects, especially the sweeping robot can reduce the collision with furniture and moving objects, and do not leave any dead corners.
  • the reachable position is cleaned, etc., which improves the working ability of the robot.
  • the corrected target image and the image to be matched are obtained, and the search based on the baseline direction for the corrected image can obtain more accurate results.
  • the operator can timely and intuitively understand the environmental information around the electronic device, for example, what objects are around the electronic device , Which objects are closer to the electronic equipment and which objects are far away from the electronic equipment, so as to control the movement, operation, or other operations of the electronic equipment in time.
  • the electronic device including a display is not limited to that the display must be installed on the electronic device.
  • the display can also be connected to the electronic device in a communicative manner.
  • the display can be connected to the electronic device.
  • the movable platform is connected via Bluetooth, mobile network, or WiFi, and the display can be set on a remote remote control or a remote computer, which is not limited by the present invention.
  • Fig. 1 shows a flow chart of the steps of a method for generating a depth map according to an embodiment of the present invention
  • FIG. 2 shows a schematic diagram of three images taken by a photographing device in a pin-shaped configuration
  • Fig. 3 shows a schematic diagram of three images taken by another photographing device in a fringe-shaped configuration
  • FIG. 4 shows a schematic diagram of three images taken by a photographing device in an L-shaped configuration
  • Figure 5 shows a schematic diagram of the same point in the left and right images
  • FIG. 6 shows a flow chart of the steps of a method for generating a depth map according to another embodiment of the present invention.
  • Figure 7 shows a schematic diagram of image correction technology
  • FIG. 8 shows a flowchart of the steps of a method for generating a depth map according to another embodiment of the present invention.
  • Figure 9 shows a schematic diagram of depth value generation
  • Figure 10 shows a schematic diagram of depth map combination
  • FIG. 11 shows a schematic diagram of an electronic device according to still another embodiment of the present invention.
  • Fig. 12 schematically shows a block diagram of a computing processing device for executing the method according to the present invention.
  • Fig. 13 schematically shows a storage unit for holding or carrying program codes for implementing the method according to the present invention.
  • the photographing device includes but is not limited to a camera.
  • a basic vision system can be composed of two photographing devices. At least three photographing devices are required to realize the technical solution of the present invention.
  • the target image may be any one of the at least three images, and the target image may be an image taken by a certain photographing device selected in advance, or may be a randomly determined image, which is not limited in the embodiment of the present invention.
  • the feature point refers to the point where the gray value of the image changes drastically or the point with larger curvature on the edge of the image (that is, the intersection of two edges).
  • Feature points can reflect the essential characteristics of the image and can identify the target object in the image. Therefore, the matching of the feature points can complete the image matching.
  • the feature points of the extracted image can be used as target pixels, or any other applicable pixel points can be used as target pixels, which is not limited in the embodiment of the present invention.
  • corner points are generally selected.
  • the optional Corner Detection Algorithm includes: FAST (Features From Accelerated Segment Test, accelerated segmentation test features), SUSAN (a corner detection algorithm) , And Harris operator (a corner detection algorithm), etc.
  • the depth value corresponding to the pixel can be the distance between the camera and the object corresponding to the pixel, and the depth map corresponding to the image can be an image used to represent the depth value of each pixel in the image, for example, each pixel in the depth map Pixels use different colors or gray values to represent the depth value of the pixel.
  • the corresponding relationship between the pixel points is established between the target image and another image, and the depth value is calculated according to the parallax of the corresponding point.
  • the SGM (semi-global matching) algorithm can be directly used to generate the depth map after the feature points are matched.
  • the BA (Bundle Adjustment) algorithm needs to be used to estimate the pose of the camera of each image, and then the Plane-Sweeping algorithm is used to calculate the pixel point. The relative distance between the two is further adjusted by semi-global optimization, and the SGM algorithm is used to generate the depth map.
  • the matching degree between the target pixel and the matching pixel on the other image can be obtained, and the matching degree indicates the degree of similarity between the two pixels .
  • the comparison of a single pixel is very robust, and it is easily affected by changes in illumination and different viewing angles.
  • One of the common methods is to match based on a sliding window. For the target image in a window centered on the target pixel Pixels, in another image, use pixels in a sliding window of the same size from left to right to calculate Cost Values. The more similar the two windows, the smaller the cost value, and the smaller the cost value, the higher the matching degree. Therefore, the reciprocal of Cost Values can be used as the matching degree.
  • the pixel corresponding to the position with the greatest degree of matching is the best matching result, that is, the pixel matching the target pixel on other images.
  • Cost Values can also be used directly as the matching degree, so the calculation result is opposite to the matching degree, and the other steps of the matching degree judgment can be adjusted accordingly.
  • This application does not limit which parameters are specifically selected as the matching degree, and all parameters that can characterize the matching degree fall within the protection scope of this application.
  • MAD Mean Absolute Differences, mean absolute difference
  • SSD SSD
  • SAD Sum of Absolute Difference
  • NCC Normalized Cross Correlation, normalized product correlation
  • SSDA Sequential Similiarity Detection Algorithm, sequential similarity detection algorithm
  • SATD Sum of Absolute Transformed Difference
  • the present invention in the process of binocular stereo matching, if there are repeated textures or weak textures in the picture, in order to avoid the problem that it is difficult to obtain an accurate depth map.
  • the present invention provides a depth map generation mechanism. By acquiring at least three images taken by at least three photographing devices, the target pixel points on the target image in the at least three images are calculated to match the pixel points on each of the other images.
  • At least two depth values corresponding to each pixel point are generated to generate a second depth map corresponding to the target image, so that the at least two matching degrees are not only based on the result of matching the baseline direction between the two shooting devices,
  • At least two matching degrees are merged to generate a depth map, or a depth map is generated by combining at least two matching degrees corresponding to the determined depth values, which realizes the combination of at least two calculation results and solves the problem of the baseline between the two shooting devices.
  • the direction sometimes matches the wrong problem, thereby improving the accuracy of the depth map.
  • Step 101 Acquire at least three images taken by at least three photographing devices.
  • each photographing device when photographing is performed, each photographing device separately photographs and obtains an image, that is, at least three photographing devices correspond to at least three images.
  • the positional relationship between the photographing devices can be set according to actual needs.
  • the positional relationship between the four photographing devices includes parallelograms, trapezoids, etc.
  • the positional relationship between the five photographing devices includes regular pentagons, etc.
  • All of the at least three photographing devices can apply the technical solution of the present invention, which is not limited in the embodiment of the present invention.
  • the sides of different images captured by different photographing devices may be parallel or perpendicular to each other, and may also be at any angle to each other, which is not limited in the embodiment of the present invention.
  • the positional relationship between the photographing devices includes any one of a pin shape or an L shape.
  • FIG. 2 is a schematic diagram of three images taken by a camera with a pin-shaped configuration.
  • the upper image is located on the vertical line connecting the left and right images. And the long sides of the picture on the right are parallel to each other.
  • Figure 3 a schematic diagram of three images taken by another camera with a fringe-shaped configuration. The positions of the upper, left, and right images are symmetrical triangles, and the long sides of the different images are 60 degrees between them. Horn.
  • Fig. 4 a schematic diagram of three images taken by a photographing device in an L-shaped configuration, the upper image is located above the left image, the right image is located on the right side of the left image, the upper image, the left image, and the right image The long sides are parallel to each other.
  • Step 102 Calculate at least two matching degrees between the target pixel on the target image and the matching pixel on each of the other images in the at least three images.
  • a plurality of target pixel points are first determined, for example, feature points are extracted. Then, for each target pixel point, find the pixel point that best matches the target pixel point on another image, and calculate the matching degree between the target pixel point and the matching pixel point on the other image. Since pixel points are matched between the target image in at least three images and each of the other images, each target pixel point will have a corresponding matching degree that is one less than the number of images. For example, if there are three images, calculate Get two matching degrees. At least three images, corresponding to at least two matching degrees.
  • the two camera devices may not be completely parallel.
  • the same point is on the left and right images.
  • the middle may not be on the same horizontal line, and for two vertically-mounted cameras, the same point may not be on the same vertical line in the upper and lower images.
  • the five-pointed star is not on the same horizontal line in the left and right images.
  • some pixel point matching algorithms require that the same point is on the same horizontal or vertical line in two images, so that the pixel point that matches the target pixel point can be found by searching on the horizontal line or vertical line.
  • the Stereo Matching algorithm while some pixel point matching algorithms do not require the same point to be on the same horizontal or vertical line in the two images, such as the Plane-Sweeping algorithm. Not all the target image and the image to be matched need to be corrected, and the target image and the image to be matched are corrected only when the adopted pixel matching algorithm requires it.
  • the Stereo Matching algorithm is used to match the pixels between the left and right images, and the left and right images need to be matched.
  • the Plane-Sweeping algorithm is used for pixel matching between the upper image and the left image, and there is no need to correct the upper image and the left image.
  • Figure 3 a schematic diagram of three images taken by another imaging device in a fringe-shaped configuration. The Plane-Sweeping algorithm is used for pixel points between the upper and left images, and between the left and right images. Matching, the above picture, left picture and right picture do not need to be corrected. As shown in Fig.
  • the Stereo Matching algorithm is used for pixel matching between the upper and left images, and between the left and right images. It is necessary to correct the above and left images, as well as the left and right images respectively.
  • Step 103 For the same target pixel point, the at least two matching degrees corresponding to it are merged, and based on the merged matching degrees corresponding to each of the target pixels, the target image corresponding to the The first depth map; or, the depth values are respectively determined according to the at least two matching degrees to obtain at least two depth values corresponding to each pixel on the target image, and according to the corresponding pixel on the target image At least two depth values are used to generate a second depth map corresponding to the target image.
  • One strategy is a strategy for fusing matching degrees in the process, and the other strategy is a strategy for combining depth values. Either of the two strategies can combine multiple calculation results to obtain a more accurate depth map.
  • Each target pixel will have at least two matching degrees, and at least two matching degrees corresponding to the same target pixel are merged to obtain the target pixel.
  • the corresponding fusion matching degree finally obtains the fusion matching degree corresponding to each target pixel on the target image.
  • At least two matching degrees can be merged in a variety of ways, for example, performing a summation operation on at least two matching degrees, or multiplying at least two matching degrees by their respective coefficients and then performing a summation operation, or other Any applicable fusion manner, which is not limited in the embodiment of the present invention.
  • the first corresponding target image is generated.
  • Depth map so that at least two matching degrees are not only based on the results obtained by matching the baseline direction between the two camera
  • the combination solves the problem that the baseline direction between the two shooting devices is sometimes incorrectly matched, thereby improving the accuracy of the depth map.
  • an implementation manner of fusing the at least two matching degrees corresponding to the same target pixel point may include: for the same target pixel point, merging all corresponding target pixels. The at least two matching degrees are summed to obtain the fused matching degree corresponding to the target pixel.
  • each feature point has a Cost Value (cost value), and at least two Cost Values of each feature point are added together to obtain the total Cost Value.
  • a depth map corresponding to the target image is generated, which is recorded as the first depth map.
  • the SGM algorithm is used to generate the first depth map, and in the cost function constructed according to the SGBM (Semi-Global Block Matching, semi-global block matching) algorithm, the total Cost Value after the above addition is used for calculation.
  • the SGBM algorithm is to calculate the value of each cost into blocks and then use the SGM algorithm to optimize the parallax.
  • the parallax calculation adopts a winner-takes-all approach, and each target pixel selects the view corresponding to the smallest aggregate cost value. The difference is used as the final disparity.
  • the result of the disparity calculation is a disparity map of the same size as the target image.
  • the disparity value of each pixel is stored.
  • the disparity map can be converted into a depth map, and each pixel is stored.
  • the depth value of each pixel indicates the position of each pixel in space.
  • the depth values are determined according to at least two matching degrees corresponding to each target pixel, and at least two depth values corresponding to each pixel on the target image are obtained, that is, according to the target image and After obtaining the matching degree corresponding to each target pixel in another image, a depth map is directly generated according to the matching degree, and for each other image, a depth map is generated.
  • the SGM algorithm is used to generate a depth map using each matching degree of the target pixel, that is, to determine the depth value of each pixel on the target image. In this way, each pixel on the target image corresponds to at least two depth values.
  • the implementation of generating the second depth map may include multiple, for example, for the same pixel, the smallest depth value among the at least two depth values is selected as the first depth value of the pixel, Aggregate the first depth values of all pixels to obtain the second depth map; or for the same pixel, calculate the average of at least two depth values of the pixel, and use the average to generate the second depth map, or any other
  • the applicable manner is not limited in the embodiment of the present invention.
  • At least two depth values corresponding to each pixel on the target image are obtained, and according to the at least two depth values corresponding to each pixel on the target image, Generate a second depth map corresponding to the target image, so that the at least two matching degrees are not only the result of matching based on the baseline direction between the two shooting devices, but also generated by combining at least two matching degrees corresponding to the determined depth values
  • the depth map realizes the combination of at least two calculation results, solves the problem that the baseline direction between the two shooting devices is sometimes incorrectly matched, thereby improving the accuracy of the depth map.
  • an implementation manner of generating a second depth map corresponding to the target image may include: at least two depth values corresponding to the same pixel point Among the two depth values, the smallest depth value is selected as the first depth value of the pixel, and a second depth map corresponding to the target image is generated based on the first depth value corresponding to each pixel on the target image.
  • Each pixel corresponds to at least two depth values, among which the smallest depth value is selected as the first depth value of the pixel, and the first depth value corresponding to each pixel on the target image is obtained, which constitutes the target image The corresponding second depth map.
  • the benefits of choosing the smallest depth value as the first depth value of the pixel at least include: when performing autonomous obstacle avoidance, avoid unnecessary collisions or other accidents caused by choosing a larger depth value, and choosing the smallest depth value is more Insurance improves the success rate of autonomous obstacle avoidance.
  • an implementation manner of generating the second depth map corresponding to the target image may include: The depth value is filtered. According to the filtered first depth value corresponding to each pixel on the target image, a second depth map corresponding to the target image is generated, and the first depth value corresponding to each pixel on the target image is filtered. , You can filter out some wrong depth values and get a more accurate depth map.
  • the filtered first depth value corresponding to each pixel on the target image is used to generate a second depth map corresponding to the target image.
  • the movable platform when at least three photographing devices are set on the movable platform, after generating the first depth map or the second depth map, it may further include: determining the position of the movable platform according to the first depth map or the second depth map The movement track or the operation track of the mechanical arm on the movable platform.
  • the first depth map or the second depth map stores the distance between the objects around the movable platform and the shooting device on the movable platform, and then according to the position of the shooting device on the movable platform, various parts of the movable platform and surrounding objects can be determined Therefore, the trajectory of the movable platform can be determined according to the first depth map or the second depth map, so as to prevent the movable platform from colliding with surrounding objects or falling.
  • the distance between the robot arm and surrounding objects can be determined, and the operation trajectory of the robot arm can be generated to avoid collisions between the robot arm and surrounding objects, or realize the operation of the robot arm on the surrounding target objects .
  • the embodiment of the present invention by acquiring at least three images captured by at least three photographing devices, at least two points between the target pixel points on the target image and the matching pixel points on the other images in the at least three images are calculated.
  • the depth map is generated after fusion, or the depth map is generated by combining at least two matching degrees corresponding to the determined depth value, which realizes the combination of at least two calculation results and solves the
  • the electronic equipment includes a movable platform, a mobile terminal, a virtual reality terminal, an augmented reality terminal, etc., or any other applicable electronic equipment, which is not limited in the embodiment of the present invention.
  • the movable platform includes, but is not limited to, aircraft, vehicles, robots, etc.
  • the aircraft includes an unmanned aircraft, such as a rotary-wing aircraft, or a fixed-wing aircraft, or any other suitable aircraft, which is not limited in the embodiment of the present invention.
  • Vehicles include manned vehicles, unmanned vehicles, and remote-controlled vehicles, or any other applicable vehicles, which are not limited in the embodiment of the present invention.
  • the robots include sweeping robots, robots for transporting goods, robots for monitoring, etc., or any other applicable robots, which are not limited in the embodiment of the present invention.
  • the depth map obtained by the technical solution of the embodiment of the present invention is more accurate.
  • the aircraft can perform better autonomous obstacle avoidance, especially the rotorcraft requires accuracy.
  • agile autonomous obstacle avoidance to avoid accidents such as aircraft crashes or collisions, and improve the safety of the aircraft;
  • the car can also perform better autonomous obstacle avoidance to avoid collisions with other objects or people, making the car have a higher Safety;
  • the robot can also perform better autonomous obstacle avoidance, and the robot arm can operate more accurately on objects, especially the sweeping robot can reduce the collision with furniture and moving objects, leaving no dead corners for all reachable The position is cleaned, etc., which improves the working ability of the robot.
  • the mobile terminal includes a mobile phone, a tablet computer, a notebook computer, etc., or any other applicable mobile terminal, which is not limited in the embodiment of the present invention.
  • Virtual reality (VR, Virtual Reality) terminals include external wearable devices, all-in-one headsets, etc., or any other applicable virtual reality terminals, which are not limited in the embodiment of the present invention.
  • Augmented reality (AR, Augmented Reality) terminals include see-through helmets, augmented reality glasses, etc., or any other applicable augmented reality terminals, which are not limited in the embodiment of the present invention.
  • FIG. 6 there is shown a step flow chart of a depth map generating method according to another embodiment of the present invention, which may specifically include the following steps:
  • Step 201 Acquire at least three images taken by at least three photographing devices.
  • step 202 the target image and the image to be matched are respectively corrected to obtain the corrected target image and the image to be matched, and the mapping relationship between the corrected target image and the target image.
  • the target image and the image to be matched are corrected so that the same pixel on the target image and the image to be matched is on the same horizontal or vertical line, and the corrected image is searched based on the baseline direction More accurate results can be obtained.
  • the left and right cameras take images separately, and the two images are corrected so that the two images change from (1) to (2), and the two On the image, the Corresponding Point (corresponding point) should be on the parallel Epipolar Line. For example, the point on the top of the tree in (2) is on the same horizontal line, so you only need to search on the horizontal line to find the matching pixel. .
  • Correct the target image and the image to be matched to obtain the corrected target image, the corrected image to be matched, and the mapping relationship between the corrected target image and the target image, the corrected image to be matched and the desired The mapping relationship between matched images.
  • Step 203 Match the target pixel on the corrected target image with the pixel on the corrected image to be matched of the target image, and calculate each of the target pixels on the corrected target image. The degree of matching between a point and a pixel point on the image that it matches after correction.
  • the pixels in the corrected image are matched, that is, the target pixels on the corrected target image and the corrected desired image of the target image are matched.
  • the pixels on the matched image are matched. For each target pixel, find the pixel that best matches the target pixel on another image, and calculate the target pixel and the pixel that matches the other image The degree of match between points.
  • Step 204 According to the first mapping relationship and the second mapping relationship, determine a third mapping between the corrected target image corresponding to the first image and the corrected target image corresponding to the second image relation.
  • the target image is matched with the pixels of the first image and the second image respectively. Therefore, the mapping relationship between the corrected target image and the target image includes the comparison between the target image and the target image. A first mapping relationship obtained by correcting the first image and a second mapping relationship obtained by correcting the target image and the second image.
  • the corrected target image corresponding to the first image and the corrected target image corresponding to the second image are not the same, nor the original target image, it is not possible to directly base the corrected target image on the first image.
  • the corrected target image corresponding to the second image can be calculated through the first mapping relationship and the second mapping relationship. The third mapping relationship between the corresponding corrected target images.
  • Step 205 According to the third mapping relationship, the matching degree corresponding to the corrected target image corresponding to the second image is converted into the matching degree corresponding to the corrected target image corresponding to the first image.
  • the prior art since there are only two shooting devices, it is possible to directly calculate the two images captured by the two shooting devices to obtain a matching degree, and then obtain a depth value.
  • at least three photographing devices are required, and at least three photographed images are obtained by shooting at least three photographing devices. Therefore, when a target image is selected to be matched with at least two other images, the same image on the target image is selected.
  • One pixel point can obtain at least two matching degrees.
  • To perform a fusion operation on the at least two matching degrees it is required that the at least two matching degrees correspond to the same image.
  • the matching degree corresponding to the corrected target image corresponding to the second image can be converted into the matching degree corresponding to the corrected target image corresponding to the first image, so that the matching degrees of the target pixels are all And based on the same image, so that subsequent fusion can be carried out.
  • the mapping relationship includes a fourth mapping relationship obtained by rectifying the target image and the third image; before the at least two matching degrees corresponding to the same target pixel are merged, it may also include : According to the fourth mapping relationship, the matching degree corresponding to the corrected target image corresponding to the third image is converted into the matching degree corresponding to the target image.
  • the Stereo Matching algorithm is used to match the pixel points between the target image and the third image, so the target pixel and the third image need to be corrected, and the matching degree obtained corresponds to the corrected target image corresponding to the third image Matching degree, and the Plane-Sweeping algorithm is used for pixel matching between the target image and other images. No correction is required.
  • the obtained matching degree is the matching degree corresponding to the target image, so all the matching degrees that are not corresponding to the target image are converted. Is the matching degree corresponding to the target image.
  • Step 206 For the same target pixel point, the at least two matching degrees corresponding to it are merged, and based on the merged matching degrees corresponding to each target pixel point, the target image corresponding to the The first depth map.
  • the target image and the image to be matched are respectively corrected to obtain the corrected target image and the image to be matched, and the corrected image
  • the mapping relationship between the target image and the target image, the target pixel on the corrected target image and the pixel on the image to be matched by the target image are matched, and the corrected target is calculated separately
  • the degree of matching between each target pixel on the image and the matching pixel on the matched image is determined according to the first mapping relationship and the second mapping relationship, the corrected target corresponding to the first image
  • a third mapping relationship between the image and the corrected target image corresponding to the second image, and the matching degree corresponding to the corrected target image corresponding to the second image is converted according to the third mapping relationship Is the matching degree corresponding to the corrected target image corresponding to the first image, for the same target pixel, the at least two matching degrees corresponding to it are fused, based on each target pixel Corresponding to the fusion matching degree, the first depth map corresponding
  • FIG. 8 there is shown a step flow chart of a method for generating a depth map according to another embodiment of the present invention, which may specifically include the following steps:
  • Step 301 Acquire at least three images taken by at least three photographing devices.
  • step 302 the target image and the image to be matched are respectively corrected to obtain the corrected target image and the image to be matched, and the mapping relationship between the corrected target image and the target image.
  • Step 303 Match the target pixel point on the corrected target image with the pixel point on the image to be matched by the target image, and respectively calculate each target pixel point on the corrected target image and its matching The degree of matching between matching pixels on the image.
  • Step 304 According to the degree of matching between each target pixel on the corrected target image and the matched pixel on the corrected image to which it matches, the depth value is calculated to obtain each pixel on the target image. Corresponding at least two depth values.
  • the depth values are determined according to at least two matching degrees to obtain at least two depth values corresponding to each pixel on the target image.
  • the image and the image to be matched need to be corrected, and the depth value is calculated according to the degree of matching between each target pixel on the corrected target image and the corrected image to be matched to obtain the position of each pixel on the target image.
  • the method of calculating the depth value is the same as the method when correction is not required, and will not be repeated here.
  • FIG. 9 a schematic diagram of depth value generation is shown in FIG. 9.
  • the upper image is corrected to get the upper image 1
  • the left image is corrected to get the left image 1.
  • the upper image and the right image are corrected
  • the upper image is corrected to get the upper image 2
  • the right image is corrected After getting the right figure 1.
  • the depth map 1 is based on the depth map of the left image 1
  • the depth map 2 is based on the depth map of the left image 2. Therefore, the depth map 1 and the depth map 2 cannot actually be directly combined.
  • the mapping relationship needs to be found first, and then they can be combined in a one-to-one correspondence.
  • Step 305 According to the first mapping relationship and the second mapping relationship, determine a third mapping between the corrected target image corresponding to the first image and the corrected target image corresponding to the second image relation.
  • Step 306 According to the third mapping relationship, the depth value corresponding to the corrected target image corresponding to the second image is converted into the depth value corresponding to the corrected target image corresponding to the first image.
  • the depth value corresponding to the corrected target image corresponding to the second image can be converted into the depth value corresponding to the corrected target image corresponding to the first image, so that the pixel
  • the depth values of the points are all based on the same image so that they can be assembled later.
  • maphex1 represents the mapping relationship between the left image and the left image 1 (ie the first mapping relationship)
  • maphex2 represents the mapping relationship between the left image and the left image 2 (ie The second mapping relationship)
  • maphex3 from the left image 2 to the left image 1 can be calculated through maphex1 and maphex2 (that is, the third mapping relationship)
  • the depth value of the depth image 2 can be mapped to the left image 1 using maphex3
  • the depth map 1 and the depth map 2 are both based on the left map 1, and then the above-mentioned depth value combination strategy is used to finally combine to obtain the depth map 4.
  • the target image and the image to be matched are corrected only when required by the adopted pixel point matching algorithm.
  • the mapping relationship includes a fourth mapping relationship obtained by rectifying the target image and the third image; before generating the second depth map corresponding to the target image according to at least two depth values corresponding to each pixel on the target image And may further include: converting the depth value corresponding to the corrected target image corresponding to the third image into the depth value corresponding to the target image according to the fourth mapping relationship.
  • the Stereo Matching algorithm is used to match the pixel points between the target image and the third image, so the target pixel and the third image need to be corrected, and the obtained depth value corresponds to the corrected target image corresponding to the third image Depth value
  • the Plane-Sweeping algorithm is used to match the pixel points between the target image and other images without correction.
  • the matching degree obtained is the depth value corresponding to the target image, so the depth values that are not corresponding to the target image are converted
  • the above algorithm is only exemplary and does not limit the present invention. In order to implement the method of the present invention, other algorithms can also be used to implement it.
  • Step 307 Generate a second depth map corresponding to the target image according to at least two depth values corresponding to each pixel on the target image.
  • the target image and the image to be matched are respectively corrected to obtain the corrected target image and the image to be matched, and the corrected image
  • the mapping relationship between the target image and the target image, the target pixel on the corrected target image and the pixel on the image to be matched by the target image are matched, and the corrected target is calculated separately
  • the degree of matching between each target pixel on the image and the matching pixel on the matched image is based on the difference between each target pixel on the corrected target image and the matched pixel on the matched image.
  • the degree of matching is calculated separately to obtain at least two depth values corresponding to each pixel on the target image, and according to the first mapping relationship and the second mapping relationship, the corrected post-correction corresponding to the first image is determined
  • the third mapping relationship between the target image and the corrected target image corresponding to the second image, according to the third mapping relationship, the depth corresponding to the corrected target image corresponding to the second image The value is converted into a depth value corresponding to the corrected target image corresponding to the first image, and a second depth map corresponding to the target image is generated according to at least two depth values corresponding to each pixel on the target image , So that the at least two matching degrees are not only based on the result of matching the baseline direction between the two camera There is a problem that the baseline directions of the two cameras are sometimes matched incorrectly, thereby improving the accuracy of the depth map.
  • the electronic device includes a processor 401, a memory 402, and at least three photographing devices 403;
  • the processor is configured to: acquire at least three images captured by at least three photographing devices; calculate at least two of the target pixel points on the target image and the matching pixel points on the other images in the at least three images Matching degree; for the same target pixel point, the at least two matching degrees corresponding to it are fused, and based on the fused matching degree corresponding to each target pixel point, the corresponding target image is generated
  • the first depth map; or, the depth values are respectively determined according to the at least two matching degrees to obtain at least two depth values corresponding to each pixel on the target image, and according to the corresponding pixel on the target image At least two depth values are used to generate a second depth map corresponding to the target image.
  • the processor merges the at least two matching degrees corresponding to the same target pixel point, it is configured to:
  • the at least two matching degrees corresponding to the same target pixel are subjected to a sum operation to obtain the fused matching degree corresponding to the target pixel.
  • the processor When the processor generates a second depth map corresponding to the target image according to at least two depth values corresponding to each pixel on the target image, it is configured to:
  • the smallest depth value is selected as the first depth value of the pixel
  • a second depth map corresponding to the target image is generated.
  • the processor When the processor generates a second depth map corresponding to the target image based on the first depth value corresponding to each pixel on the target image, the processor is configured to:
  • a second depth map corresponding to the target image is generated.
  • the processor Before the processor calculates at least two matching degrees between the target pixel on the target image and the matching pixel on each of the other images in the at least three images, the processor is further configured to:
  • the target image and the image to be matched are respectively corrected to obtain the corrected target image and the image to be matched, and the mapping relationship between the corrected target image and the target image.
  • the processor calculates at least two matching degrees between the target pixel on the target image and the matching pixel on each of the other images in the at least three images, it is configured to:
  • the target pixel points on the corrected target image are matched with the pixel points on the corrected image to be matched on the target image, and each target pixel point on the corrected target image is calculated and corrected respectively.
  • the matching degree between the matching pixels on the matched image is a predefined range between the matching pixels on the matched image.
  • the mapping relationship includes a first mapping relationship obtained by correcting the target image and the first image and a second mapping relationship obtained by correcting the target image and the second image;
  • the processor is further configured to:
  • the matching degree corresponding to the corrected target image corresponding to the second image is converted into the matching degree corresponding to the corrected target image corresponding to the first image.
  • the mapping relationship includes a fourth mapping relationship obtained by rectifying the target image and the third image; the processor performs the at least two matching degrees corresponding to the same target pixel point Before fusion, the processor is also used to:
  • the matching degree corresponding to the corrected target image corresponding to the third image is converted into the matching degree corresponding to the target image.
  • the processor separately determines depth values according to the at least two matching degrees, and obtains at least two depth values corresponding to each pixel on the target image, it is used to:
  • the depth value is calculated respectively to obtain at least two depths corresponding to each pixel point on the target image value.
  • the mapping relationship includes a first mapping relationship obtained by correcting the target image and the first image and a second mapping relationship obtained by correcting the target image and the second image;
  • the depth value corresponding to the corrected target image corresponding to the second image is converted into the depth value corresponding to the corrected target image corresponding to the first image.
  • the mapping relationship includes a fourth mapping relationship obtained by rectifying the target image and the third image; the processor generates the target according to at least two depth values corresponding to each pixel on the target image Before the second depth map corresponding to the image, the processor is further configured to:
  • the depth value corresponding to the corrected target image corresponding to the third image is converted into the depth value corresponding to the target image.
  • the positional relationship between the photographing devices includes any one of a pin shape or an L shape.
  • the electronic device includes any one of a movable platform, a mobile terminal, a virtual reality terminal, or an augmented reality terminal.
  • the movable platform includes flying chess, rook, or robot.
  • the aircraft includes an unmanned aircraft, such as a rotary-wing aircraft, or a fixed-wing aircraft, or any other suitable aircraft, which is not limited in the embodiment of the present invention.
  • Vehicles include manned vehicles, unmanned vehicles, and remote-controlled vehicles, or any other applicable vehicles, which are not limited in the embodiment of the present invention.
  • the robots include sweeping robots, robots for transporting goods, robots for monitoring, etc., or any other applicable robots, which are not limited in the embodiment of the present invention.
  • the processor is also used for:
  • the motion trajectory of the movable platform or the operation trajectory of the mechanical arm on the movable platform is determined.
  • the electronic device includes a display, and the display is used for:
  • the electronic device including a display is not limited to that the display must be installed on the electronic device.
  • the display can also be connected to the electronic device in a communicative manner.
  • the display can be connected to the electronic device.
  • the movable platform is connected via Bluetooth, mobile network, or WiFi, and the display can be set on a remote remote control or a remote computer, which is not limited by the present invention.
  • the processor is also used for:
  • the map or the second depth map generates control instructions for the movable platform.
  • the control device can communicate with the mobile platform through Bluetooth, wireless network, 5G network, etc.
  • the control device can display the first depth map or the second depth map in real time, or the control device can display the first depth map or the second depth map in real time.
  • Two depth map generation control instructions for example, the control device determines the movement trajectory of the movable platform or the operation trajectory of the robot arm on the movable platform according to the first depth map or the second depth map, and generates it according to the movement trajectory or operation trajectory The corresponding control instruction is sent to the movable platform.
  • the embodiment of the present invention by acquiring at least three images captured by at least three photographing devices, at least two points between the target pixel points on the target image and the matching pixel points on the other images in the at least three images are calculated.
  • the depth map is generated after fusion, or the depth map is generated by combining at least two matching degrees corresponding to the determined depth value, which realizes the combination of at least two calculation results and solves the
  • the device embodiments described above are merely illustrative.
  • the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in One place, or it can be distributed to multiple network units.
  • Some or all of the modules can be selected according to actual needs to achieve the objectives of the solutions of the embodiments. Those of ordinary skill in the art can understand and implement it without creative work.
  • the various component embodiments of the present invention may be implemented by hardware, or by software modules running on one or more processors, or by a combination of them.
  • a microprocessor or a digital signal processor (DSP) may be used in practice to implement some or all of the functions of some or all of the components in the computing processing device according to the embodiments of the present invention.
  • DSP digital signal processor
  • the present invention can also be implemented as a device or device program (for example, a computer program and a computer program product) for executing part or all of the methods described herein.
  • Such a program for realizing the present invention may be stored on a computer-readable medium, or may have the form of one or more signals.
  • Such a signal can be downloaded from an Internet website, or provided on a carrier signal, or provided in any other form.
  • FIG. 12 shows a computing processing device that can implement the method according to the present invention.
  • the computing processing device traditionally includes a processor 1010 and a computer program product in the form of a memory 1020 or a computer readable medium.
  • the memory 1020 may be an electronic memory such as flash memory, EEPROM (Electrically Erasable Programmable Read Only Memory), EPROM, hard disk, or ROM.
  • the memory 1020 has a storage space 1030 for executing program codes 1031 of any method steps in the above methods.
  • the storage space 1030 for program codes may include various program codes 1031 respectively used to implement various steps in the above method. These program codes can be read from or written into one or more computer program products.
  • Such computer program products include program code carriers such as hard disks, compact disks (CDs), memory cards, or floppy disks.
  • Such a computer program product is usually a portable or fixed storage unit as described with reference to FIG. 13.
  • the storage unit may have storage segments, storage spaces, etc. arranged similarly to the memory 1020 in the computing processing device of FIG. 12.
  • the program code can be compressed in an appropriate form, for example.
  • the storage unit includes computer-readable code 1031', that is, code that can be read by a processor such as 1010, which, when run by a computing processing device, causes the computing processing device to execute the method described above. The various steps.
  • any reference signs placed between parentheses should not be constructed as a limitation to the claims.
  • the word “comprising” does not exclude the presence of elements or steps not listed in the claims.
  • the word “a” or “an” preceding an element does not exclude the presence of multiple such elements.
  • the invention can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In the unit claims listing several devices, several of these devices may be embodied in the same hardware item.
  • the use of the words first, second, and third, etc. do not indicate any order. These words can be interpreted as names.

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Image Processing (AREA)

Abstract

一种深度图生成方法、电子设备、设备和存储介质。所述方法包括：获取至少三个拍摄装置所拍摄的至少三个图像（101），计算目标图像上的目标像素点与其他图像上匹配的像素点之间的至少两个匹配度（102），将至少两个匹配度进行融合，基于各个目标像素点的融合后的匹配度，生成第一深度图；或，根据至少两个匹配度分别确定深度值，得到各个像素点对应的至少两个深度值，根据各个像素点所对应的至少两个深度值，生成第二深度图（103），使得至少两个匹配度不仅是基于基线方向进行匹配得到的结果，或者结合至少两个深度值来生成深度图，实现了对计算结果的结合，解决了两个拍摄装置之间在基线方向有时会匹配错误的问题，从而提高了深度图的准确度。

Description

深度图生成方法、电子设备、计算处理设备及存储介质 技术领域
本申请涉及数据处理技术领域，具体涉及一种深度图生成方法、一种电子设备、一种计算处理设备、一种计算机可读存储介质。
背景技术
计算机视觉是依靠成像系统代替视觉器官作为输入敏感手段，最常用的是摄像头，由双摄像头即可组成一个基础的视觉系统，称为Stereo Vision(立体视觉)。
双目摄像头系统(Stereo Vision System)通过两个摄像头，拍摄同一时刻，不同角度的两张照片，再通过两张照片的差异，以及双摄像头之间的位置、角度关系，利用三角关系，即可计算出场景与摄像头的距离关系，画在一张图上即为Depth Map(景深图或深度图)。
常用的深度计算方法是用Stereo Matching(双目立体匹配)算法，其中，需要在两张图片中在双摄像头的基线方向搜索最匹配的块。经研究发现，如果图片中有重复纹理，或是弱纹理的情况下，由于高度相似的特征点描述向量接近，扫描时难以正确判断哪一个是对应的特征点，基线方向搜索就容易出错，继而难以得到准确的Depth Map。
发明内容
鉴于上述问题，提出了本申请以便提供一种克服上述问题或者至少部分地解决上述问题的深度图生成方法、电子设备、计算处理设备、计算机可读存储介质。
依据本申请的一个方面，提供了一种深度图生成方法，包括：
获取至少三个拍摄装置所拍摄的至少三个图像；
计算所述至少三个图像中目标图像上的目标像素点与其他各个图像上匹配的像素点之间的至少两个匹配度；
对于同一个所述目标像素点，将其所对应的所述至少两个匹配度进行融合，基于各个所述目标像素点所对应的融合后的匹配度，生成所述目标图像对应的第一深度图；或者，根据所述至少两个匹配度分别确定深度值，得到 所述目标图像上各个像素点所对应的至少两个深度值，根据所述目标图像上各个像素点所对应的至少两个深度值，生成所述目标图像对应的第二深度图。
可选地，所述对于同一个所述目标像素点，将其所对应的所述至少两个匹配度进行融合包括：
对于同一个所述目标像素点，将其所对应的所述至少两个匹配度作求和运算，得到所述目标像素点所对应的融合后的匹配度。
可选地，所述根据各个所述目标像素点所对应的至少两个深度值，生成所述目标图像对应的第二深度图包括：
在同一个所述像素点所对应的所述至少两个深度值中，选择最小的深度值作为所述像素点的第一深度值；
基于各个所述目标像素点对应的第一深度值，生成所述目标图像对应的第二深度图。
可选地，所述基于所述目标图像上各个像素点对应的第一深度值，生成所述目标图像对应的第二深度图包括：
对所述目标图像上各个像素点对应的第一深度值进行滤波处理；
根据所述目标图像上各个像素点对应的滤波处理后的第一深度值，生成所述目标图像对应的第二深度图。
可选地，在所述计算所述至少三个图像中目标图像上的目标像素点与其他各个图像上匹配的像素点之间的至少两个匹配度之前，所述方法还包括：
分别对所述目标图像和所要匹配的图像进行矫正，得到矫正后的目标图像和所要匹配的图像，以及矫正后的目标图像和所述目标图像之间的映射关系。
可选地，所述计算所述至少三个图像中目标图像上的目标像素点与其他各个图像上匹配的像素点之间的至少两个匹配度包括：
将所述矫正后的目标图像上目标像素点和所述目标图像的矫正后的所要匹配的图像上的像素点进行匹配，分别计算所述矫正后的目标图像上各个所述目标像素点与矫正后的其所匹配的图像上匹配的像素点之间的匹配度。
可选地，所述映射关系包括对所述目标图像和第一图像进行矫正得到的第一映射关系和对所述目标图像和第二图像进行矫正得到的第二映射关系； 在所述对于同一个所述目标像素点，将其所对应的所述至少两个匹配度进行融合之前，所述方法还包括：
根据所述第一映射关系和第二映射关系，确定与所述第一图像对应的矫正后的目标图像和与所述第二图像对应的矫正后的目标图像之间的第三映射关系；
根据所述第三映射关系，将与所述第二图像对应的矫正后的目标图像对应的匹配度转换为与所述第一图像对应的矫正后的目标图像对应的匹配度。
可选地，所述映射关系包括对所述目标图像和第三图像进行矫正得到的第四映射关系；在所述对于同一个所述目标像素点，将其所对应的所述至少两个匹配度进行融合之前，所述方法还包括：
根据所述第四映射关系，将与所述第三图像对应的矫正后的目标图像对应的匹配度转换为所述目标图像对应的匹配度。
可选地，所述根据所述至少两个匹配度分别确定深度值，得到所述目标图像上各个像素点所对应的至少两个深度值包括：
根据所述矫正后的目标图像上各个目标像素点与矫正后的其所匹配的图像上匹配的像素点之间的匹配度，分别计算深度值，得到所述目标图像上各个像素点所对应的至少两个深度值。
可选地，所述映射关系包括对所述目标图像和第一图像进行矫正得到的第一映射关系和对所述目标图像和第二图像进行矫正得到的第二映射关系；在所述根据所述目标图像上各个像素点所对应的至少两个深度值，生成所述目标图像对应的第二深度图之前，所述方法还包括：
根据所述第一映射关系和第二映射关系，确定与所述第一图像对应的矫正后的目标图像和与所述第二图像对应的矫正后的目标图像之间的第三映射关系；
根据所述第三映射关系，将与所述第二图像对应的矫正后的目标图像对应的深度值转换为与所述第一图像对应的矫正后的目标图像对应的深度值。
可选地，所述映射关系包括对所述目标图像和第三图像进行矫正得到的第四映射关系；在所述根据所述目标图像上各个像素点所对应的至少两个深度值，生成所述目标图像对应的第二深度图之前，所述方法还包括：
根据所述第四映射关系，将与所述第三图像对应的矫正后的目标图像对 应的深度值转换为所述目标图像对应的深度值。
可选地，所述拍摄装置为三个，所述拍摄装置之间的位置关系包括品字形、或L形中的任意一种。
可选地，所述至少三个拍摄装置设置在电子设备上，所述电子设备包括可移动平台、移动终端、虚拟现实终端、或增强现实终端中的任意一种。
可选地，所述至少三个拍摄装置设置在可移动平台上，所述方法还包括：
根据所述第一深度图或第二深度图，确定所述可移动平台的运动轨迹或所述可移动平台上的机械臂的操作轨迹。
依据本申请的另一个方面，提供了一种电子设备，所述电子设备包括处理器、存储器和至少三个拍摄装置；
所述处理器用于：获取至少三个拍摄装置所拍摄的至少三个图像；计算所述至少三个图像中目标图像上的目标像素点与其他各个图像上匹配的像素点之间的至少两个匹配度；对于同一个所述目标像素点，将其所对应的所述至少两个匹配度进行融合，基于各个所述目标像素点所对应的融合后的匹配度，生成所述目标图像对应的第一深度图；或者，根据所述至少两个匹配度分别确定深度值，得到所述目标图像上各个像素点所对应的至少两个深度值，根据所述目标图像上各个像素点所对应的至少两个深度值，生成所述目标图像对应的第二深度图。
可选地，所述处理器在对于同一个所述目标像素点，将其所对应的所述至少两个匹配度进行融合时，用于：
对于同一个所述目标像素点，将其所对应的所述至少两个匹配度作求和运算，得到所述目标像素点所对应的融合后的匹配度。
可选地，所述处理器在根据所述目标图像上各个像素点所对应的至少两个深度值，生成所述目标图像对应的第二深度图时，用于：
在同一个所述像素点所对应的所述至少两个深度值中，选择最小的深度值作为所述像素点的第一深度值；
基于所述目标图像上各个像素点对应的第一深度值，生成所述目标图像对应的第二深度图。
可选地，所述处理器在基于所述目标图像上各个像素点对应的第一深度值，生成所述目标图像对应的第二深度图时，用于：
对所述目标图像上各个像素点对应的第一深度值进行滤波处理；
根据所述目标图像上各个像素点对应的滤波处理后的第一深度值，生成所述目标图像对应的第二深度图。
可选地，在所述处理器计算所述至少三个图像中目标图像上的目标像素点与其他各个图像上匹配的像素点之间的至少两个匹配度之前，所述处理器还用于：
分别对所述目标图像和所要匹配的图像进行矫正，得到矫正后的目标图像和所要匹配的图像，以及矫正后的目标图像和所述目标图像之间的映射关系。
可选地，所述处理器在计算所述至少三个图像中目标图像上的目标像素点与其他各个图像上匹配的像素点之间的至少两个匹配度时，用于：
将所述矫正后的目标图像上目标像素点和所述目标图像的矫正后的所要匹配的图像上的像素点进行匹配，分别计算所述矫正后的目标图像上各个所述目标像素点与矫正后的其所匹配的图像上匹配的像素点之间的匹配度。
可选地，所述映射关系包括对所述目标图像和第一图像进行矫正得到的第一映射关系和对所述目标图像和第二图像进行矫正得到的第二映射关系；在所述处理器对于同一个所述目标像素点，将其所对应的所述至少两个匹配度进行融合之前，所述处理器还用于：
根据所述第一映射关系和第二映射关系，确定与所述第一图像对应的矫正后的目标图像和与所述第二图像对应的矫正后的目标图像之间的第三映射关系；
根据所述第三映射关系，将与所述第二图像对应的矫正后的目标图像对应的匹配度转换为与所述第一图像对应的矫正后的目标图像对应的匹配度。
可选地，所述映射关系包括对所述目标图像和第三图像进行矫正得到的第四映射关系；在所述处理器对于同一个所述目标像素点，将其所对应的所述至少两个匹配度进行融合之前，所述处理器还用于：
根据所述第四映射关系，将与所述第三图像对应的矫正后的目标图像对应的匹配度转换为所述目标图像对应的匹配度。
可选地，所述处理器根据所述至少两个匹配度分别确定深度值，得到所述目标图像上各个像素点所对应的至少两个深度值时，用于：
根据所述矫正后的目标图像上各个目标像素点与其所匹配的图像上匹配的像素点之间的匹配度，分别计算深度值，得到所述目标图像上各个像素点所对应的至少两个深度值。
可选地，所述映射关系包括对所述目标图像和第一图像进行矫正得到的第一映射关系和对所述目标图像和第二图像进行矫正得到的第二映射关系；在所述处理器根据所述目标图像上各个像素点所对应的至少两个深度值，生成所述目标图像对应的第二深度图之前，所述处理器还用于：
根据所述第一映射关系和第二映射关系，确定与所述第一图像对应的矫正后的目标图像和与所述第二图像对应的矫正后的目标图像之间的第三映射关系；
根据所述第三映射关系，将与所述第二图像对应的矫正后的目标图像对应的深度值转换为与所述第一图像对应的矫正后的目标图像对应的深度值。
可选地，所述映射关系包括对所述目标图像和第三图像进行矫正得到的第四映射关系；在所述处理器根据所述目标图像上各个像素点所对应的至少两个深度值，生成所述目标图像对应的第二深度图之前，所述处理器还用于：
根据所述第四映射关系，将与所述第三图像对应的矫正后的目标图像对应的深度值转换为所述目标图像对应的深度值。
可选地，所述拍摄装置为三个，所述拍摄装置之间的位置关系包括品字形、或L形中的任意一种。
可选地，所述电子设备包括可移动平台、移动终端、虚拟现实终端、或增强现实终端中的任意一种。
可选地，所述电子设备为可移动平台，所述处理器还用于：
根据所述第一深度图或第二深度图，确定所述可移动平台的运动轨迹或所述可移动平台上的机械臂的操作轨迹。
可选地，所述电子设备包括显示器，所述显示器用于：
对所述第一深度图或第二深度图进行显示。
可选地，所述处理器还用于：
将所述第一深度图或第二深度图发送给所述电子设备的控制设备，以供所述控制设备对所述第一深度图或第二深度图进行显示或根据所述第一深度图或第二深度图生成对所述电子设备的控制指令。
依据本申请的另一个方面，提供了一种计算机程序，包括计算机可读代码，当所述计算机可读代码在计算处理设备上运行时，导致所述计算处理设备执行上述的深度图生成方法。
依据本申请的另一个方面，提供了一种计算机可读介质，其中存储了如上所述的计算机程序。
依据本发明实施例，通过获取至少三个拍摄装置所拍摄的至少三个图像，计算所述至少三个图像中目标图像上的目标像素点与其他各个图像上匹配的像素点之间的至少两个匹配度，对于同一个所述目标像素点，将其所对应的所述至少两个匹配度进行融合，基于各个所述目标像素点所对应的融合后的匹配度，生成所述目标图像对应的第一深度图；或者，根据所述至少两个匹配度分别确定深度值，得到所述目标图像上各个像素点所对应的至少两个深度值，根据所述目标图像上各个像素点所对应的至少两个深度值，生成所述目标图像对应的第二深度图，使得至少两个匹配度不仅仅是基于两个拍摄装置之间的基线方向进行匹配得到的结果，对至少两个匹配度进行融合后生成深度图，或者结合至少两个匹配度对应确定的深度值来生成深度图，实现了对至少两个计算结果的结合，解决了两个拍摄装置之间在基线方向有时会匹配错误的问题，从而提高了深度图的准确度。
进一步，通过在同一个像素点所对应的至少两个深度值中，选择最小的深度值作为像素点的第一深度值，基于目标图像上各个像素点对应的第一深度值，生成目标图像对应的第二深度图，在进行自主避障时，避免选择更大的深度值所导致的不必要的碰撞或其他事故，选择最小的深度值更加保险，提高了自主避障的成功率。
进一步，通过对目标图像上各个像素点对应的第一深度值进行滤波处理，根据目标图像上各个像素点对应的滤波处理后的第一深度值，生成目标图像对应的第二深度图，对目标图像上各个像素点对应的第一深度值进行滤波处理，可以滤除一些错误的深度值，获得更加精确地深度图。
进一步，通过根据第一深度图或第二深度图，确定可移动平台的运动轨迹或所述可移动平台上的机械臂的操作轨迹，避免可移动平台与周围物体发生碰撞或者发生跌落等情况。在可移动平台上有机械臂时，还可以确定出机械臂与周围物体之间的距离，生成机械臂的操作轨迹，避免机械臂与周围物体发生碰撞，或者实现机械臂对周围目标物体的操作。
进一步，通过至少三个拍摄装置设置在飞行器、车、机器人等上时，采用本发明实施例的技术方案得到的深度图更加准确，因此，飞行器可以进行更好的自主避障、尤其是旋翼飞行器非常需要准确和敏捷的自主避障，避免飞行器发生坠机或碰撞等事故，提高飞行器的安全性；车也可以进行更好的自主避障，避免车与其他物体或人发生碰撞，使得车具有更高的安全性；机器人也可以进行更好的自主避障，机器人的机械臂可以更准确的对物体进行操作，尤其是扫地机器人可以减少与家具和移动物体的碰撞，不留死角的对所有可达到的位置进行清扫等，提高了机器人的工作能力。
进一步，通过分别对所述目标图像和所要匹配的图像进行矫正，得到矫正后的目标图像和所要匹配的图像，针对矫正后的图像进行基于基线方向的搜索可以获得更加精确地结果。
进一步，通过对所述第一深度图或第二深度图进行显示，通过显示所述深度图，可以使操作人员及时地、直观地了解到电子设备周围的环境信息，例如电子设备周围有哪些物体，哪些物体距离电子设备较近，哪些物体距离电子设备较远，从而及时的电子设备的移动、作业、或其他操作进行控制。需要注意的是，所述电子设备包括显示器并不限制于所述显示器必须设置在所述电子设备上，所说显示器也可以与所述电子设备以通信方式进行连接，例如，所述显示器可以与所述可移动平台通过蓝牙、移动网络、或WiFi连接，所述显示器可以设置在远程的遥控器上，也可以设置在远程的计算机上，本发明对此不作限制。
上述说明仅是本发明技术方案的概述，为了能够更清楚了解本发明的技术手段，而可依照说明书的内容予以实施，并且为了让本发明的上述和其它目的、特征和优点能够更明显易懂，以下特举本发明的具体实施方式。
附图说明
为了更清楚地说明本发明实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图是本发明的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1示出了本发明一个实施例的一种深度图生成方法的步骤流程图；
图2示出了一种呈品字形构型的拍摄装置所拍摄的三个图像的示意图；
图3示出了另一种呈品字形构型的拍摄装置所拍摄的三个图像的示意图；
图4示出了一种呈L形构型的拍摄装置所拍摄的三个图像的示意图；
图5示出了左右两个图像中同一个点的示意图；
图6示出了本发明另一实施例的一种深度图生成方法的步骤流程图；
图7示出了图像矫正技术的示意图；
图8示出了本发明又一实施例的一种深度图生成方法的步骤流程图；
图9示出了深度值生成的示意图；
图10示出了深度图结合的示意图；
图11示出了本发明再一实施例的一种电子设备的示意图；
图12示意性地示出了用于执行根据本发明的方法的计算处理设备的框图；以及
图13示意性地示出了用于保持或者携带实现根据本发明的方法的程序代码的存储单元。
具体实施例
为使本发明实施例的目的、技术方案和优点更加清楚，下面将结合本发明实施例中的附图，对本发明实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例是本发明一部分实施例，而不是全部的实施例。基于本发明中的实施例，本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其他实施例，都属于本发明保护的范围。
为使本领域技术人员更好地理解本发明，以下对本发明涉及的概念进行说明：
拍摄装置包括但不限于摄像头，一般由双拍摄装置即可组成一个基础的视觉系统，为实现本发明的技术方案需要至少三个拍摄装置。
目标图像可以为至少三个图像中的任意一个图像，该目标图像可以为预先选定的某个拍摄装置所拍摄的图像，或者可以为随机确定的图像，本发明实施例对此不做限制。
从目标图像上选择多个像素点，记为目标像素点。在图像处理中，特征点指的是图像灰度值发生剧烈变化的点或者在图像边缘上曲率较大的点(即 两个边缘的交点)。特征点能够反映图像本质特征，能够标识图像中目标物体，因此，通过特征点的匹配能够完成图像的匹配。提取图像的特征点，可以作为目标像素点，或者其他任意适用的像素点作为目标像素点，本发明实施例对此不做限制。例如，提取图像的特征点，一般选用角点，可选的Corner Detection Algorithm(角点检测算法)包括：FAST(Features From Accelerated Segment Test，加速分割测试特征)、SUSAN(一种角点检测算法)、以及Harris operator(一种角点检测算法)等。
像素点对应的深度值可以是拍摄装置与该像素点对应的实物之间的距离，图像对应的深度图可以是用于表示图像中各个像素点的深度值的图像，例如，深度图中每个像素点利用不同的颜色或者灰度值来表示该像素点的深度值。在目标图像与另一个图像之间建立像素点之间的对应关系，在根据对应点的视差计算出深度值。
例如，在采用Stereo Matching算法时，在特征点匹配后可以直接使用SGM(semi-global matching，半全局匹配)算法来生成深度图。而在采用Plane-Sweeping(平面扫描)算法时，在特征点匹配后需要采用BA(Bundle Adjustment，集束调整)算法估算每个图像的拍摄装置的位姿，然后采用Plane-Sweeping算法计算像素点之间相对的距离，进一步来做半全局优化调整，使用SGM算法来生成深度图。
在确定其他图像上与目标图像上的目标像素点匹配的像素点时，可以得到目标像素点与其他图像上匹配的像素点之间的匹配度，匹配度表示两个像素点之间的相似程度。单个像素点进行比较鲁棒性很差，很容易受到光照变化和视角不同的影响，其中一种常用方式是基于滑动窗口来进行匹配，对于目标图像中以目标像素点为中心的一个窗口内的像素，在另一个图像中从左到右用一个同尺寸滑动窗口内的像素和它计算Cost Values(代价值)，两个窗口越相似，代价值越小，代价值越小代表匹配度越高，因此可以用Cost Values(代价值)的倒数作为匹配度。匹配度最大的位置对应的像素点就是最佳的匹配结果，即其他图像上与目标像素点匹配的像素点。需要注意的是，在实际应用中，也可以直接用Cost Values(代价值)作为匹配度，因而计算结果与匹配度的高低是相反的，在其他对匹配度判断的步骤对应调整即可。对于具体选用哪些参数作为匹配度，本申请对此不作限制，凡是能够表征匹 配程度的参数，均属于本申请的保护范围。
例如，在计算每个目标像素点与其他图像上的像素点之间的匹配度时，可以使用MAD(Mean Absolute Differences，平均绝对差)算法、SSD(Sum of Squared Differences，误差平方和)算法、SAD(Sum of Absolute Difference，绝对误差和)算法、NCC(Normalized Cross Correlation，归一化积相关)算法、SSDA(Sequential Similiarity Detection Algorithm，序贯相似性检测算法)、或者SATD(Sum of Absolute Transformed Difference，绝对变换误差和)算法等，具体可以根据实际需要选择其中任意一种算法，本发明实施例对此不做限制。
根据本发明的一种实施例，在双目立体匹配的过程中，如果图片中有重复纹理，或是弱纹理的情况下，为了避免难以得到准确的深度图的问题。本发明提供了一种深度图生成机制，通过获取至少三个拍摄装置所拍摄的至少三个图像，计算所述至少三个图像中目标图像上的目标像素点与其他各个图像上匹配的像素点之间的至少两个匹配度，对于同一个所述目标像素点，将其所对应的所述至少两个匹配度进行融合，基于各个所述目标像素点所对应的融合后的匹配度，生成所述目标图像对应的第一深度图；或者，根据所述至少两个匹配度分别确定深度值，得到所述目标图像上各个像素点所对应的至少两个深度值，根据所述目标图像上各个像素点所对应的至少两个深度值，生成所述目标图像对应的第二深度图，使得至少两个匹配度不仅仅是基于两个拍摄装置之间的基线方向进行匹配得到的结果，对至少两个匹配度进行融合后生成深度图，或者结合至少两个匹配度对应确定的深度值来生成深度图，实现了对至少两个计算结果的结合，解决了两个拍摄装置之间在基线方向有时会匹配错误的问题，从而提高了深度图的准确度。
参照图1，示出了本发明实施例的一种深度图生成方法的步骤流程图，具体可以包括如下步骤：
步骤101，获取至少三个拍摄装置所拍摄的至少三个图像。
在本发明实施例中，在进行拍摄时，每一个拍摄装置分别拍摄得到一个图像，即至少三个拍摄装置对应有至少三个图像。拍摄装置之间的位置关系可以根据实际需要设置，例如，四个拍摄装置之间的位置关系包括平行四边形、梯形等，五个拍摄装置之间的位置关系包括正五边形等，不同位置关系 的至少三个拍摄装置都可以应用本发明的技术方案，本发明实施例对此不做限制。
在本发明实施例中，不同拍摄装置拍摄得到的不同图像的边之间可以相互平行或垂直，也可以相互之间成任意的角度，本发明实施例对此不做限制。
可选地，当拍摄装置为三个时，拍摄装置之间的位置关系包括品字形、或L形中的任意一种。
例如，如图2所示的一种呈品字形构型的拍摄装置所拍摄的三个图像的示意图，上图位于左图和右图之间连线的中垂线上，上图、左图和右图的长边之间相互平行。如图3所示的另一种呈品字形构型的拍摄装置所拍摄的三个图像的示意图，上图、左图和右图的位置呈对称三角形，不同图像的长边之间成60度角。如图4所示的一种呈L形构型的拍摄装置所拍摄的三个图像的示意图，上图位于左图的上方，右图位于左图的右侧，上图、左图和右图的长边之间相互平行。
步骤102，计算所述至少三个图像中目标图像上的目标像素点与其他各个图像上匹配的像素点之间的至少两个匹配度。
在本发明实施例中，在至少三个图像中的目标图像上，先确定多个目标像素点，例如，提取特征点。然后对于每个目标像素点，在另一个图像上寻找和该目标像素点最匹配的像素点，并计算得到该目标像素点与该另一个图像上匹配的像素点之间的匹配度。由于分别对至少三个图像中的目标图像和其他各个图像之间进行像素点匹配，因此每个目标像素点会有对应的比图像数量少一个的匹配度，例如，若是三个图像，则计算得到两个匹配度。至少三个图像，对应得到的匹配度为至少两个。
在本发明实施例中，因为安装有偏差，或是制造工艺精度不够，两个拍摄装置之间可能存在不完全平行的情况，对于两个水平安装的拍摄装置，同一个点在左右两个图像中可能不会处于同一个水平线，对于两个垂直安装的拍摄装置，同一个点在上下两个图像中也可能不会处于同一个垂直线。如图5所示的左右两个图像中同一个点的示意图，五角星在左右两个图中不在同一个水平线上。
在本发明实施例中，一些像素点匹配算法需要同一个点在两个图像中处于同一个水平线或垂直线，才能在水平线或垂直线上搜索就能寻找到与目标 像素点匹配的像素点，例如，Stereo Matching算法，而一些像素点匹配算法则不需要同一个点在两个图像中处于同一个水平线或垂直线，例如Plane-Sweeping算法。并非所有的目标图像和所要匹配的图像都需要进行矫正，只有采用的像素点匹配算法需要时，才对目标图像和所要匹配的图像进行矫正。
例如，如图2所示的一种呈品字构型的三个拍摄装置所拍摄的三个图像，左图和右图之间采用Stereo Matching算法进行像素点匹配，需要对左图和右图进行矫正，上图和左图之间采用Plane-Sweeping算法进行像素点匹配，不需要对上图和左图进行矫正。如图3所示的另一种呈品字形构型的拍摄装置所拍摄的三个图像的示意图，上图和左图之间、左图和右图之间都采用Plane-Sweeping算法进行像素点匹配，上图、左图和右图都不需要进行矫正。如图4所示的一种呈L形构型的拍摄装置所拍摄的三个图像的示意图，上图和左图之间、左图和右图之间都采用Stereo Matching算法进行像素点匹配，需要对上图和左图，以及左图和右图分别进行矫正。
步骤103，对于同一个所述目标像素点，将其所对应的所述至少两个匹配度进行融合，基于各个所述目标像素点所对应的融合后的匹配度，生成所述目标图像对应的第一深度图；或者，根据所述至少两个匹配度分别确定深度值，得到所述目标图像上各个像素点所对应的至少两个深度值，根据所述目标图像上各个像素点所对应的至少两个深度值，生成所述目标图像对应的第二深度图。
在本发明实施例中，对多个计算结果的结合有两种策略，一种策略是过程中融合匹配度的策略，另一种策略是结合深度值的策略。两种策略任意一种都可以结合多个计算结果，从而得到更加准确的深度图。
下面先对过程中融合匹配度的策略进行介绍，每个目标像素点都会有对应的至少两个匹配度，将同一个目标像素点所对应的至少两个匹配度进行融合，得到该目标像素点所对应的融合后的匹配度，最终得到目标图像上的各个目标像素点所对应的融合后的匹配度。至少两个匹配度进行融合的方式可以包括多种，例如，对至少两个匹配度作求和运算，或者将至少两个匹配度分别于各自的系数相乘后再作求和运算，或者其他任意适用的融合方式，本发明实施例对此不做限制。通过对于同一个所述目标像素点，将其所对应的所述至少两个匹配度进行融合，基于各个所述目标像素点所对应的融合后的 匹配度，生成所述目标图像对应的第一深度图，使得至少两个匹配度不仅仅是基于两个拍摄装置之间的基线方向进行匹配得到的结果，对至少两个匹配度进行融合后生成深度图，实现了对至少两个计算结果的结合，解决了两个拍摄装置之间在基线方向有时会匹配错误的问题，从而提高了深度图的准确度。
可选地，对于同一个所述目标像素点，将其所对应的所述至少两个匹配度进行融合的一种实现方式可以包括：对于同一个所述目标像素点，将其所对应的所述至少两个匹配度作求和运算，得到所述目标像素点所对应的融合后的匹配度。例如，Stereo Matching算法，或是Plane-Sweeping算法，每个特征点都会有Cost Value(代价值)，将每个特征点的至少两个Cost Value相加，相加后得到总的Cost Value。
将匹配度融合之后，基于各个目标像素点所对应的融合后的匹配度，生成目标图像对应的深度图，记为第一深度图。例如，采用SGM算法生成第一深度图，在根据SGBM(Semi-Global Block Matching，半全局块匹配)算法构造的代价函数中，使用上述相加后的总的Cost Value进行计算。SGBM算法就是将每一个代价值进行成块计算后再用SGM算法进行视差优化，而在SGM算法中，视差计算采用赢家通吃的方式，每个目标像素点选择最小聚合代价值所对应的视差值作为最终视差，视差计算的结果是和目标图像相同尺寸的视差图，存储每个像素点的视差值，在图像内外参数已知的情况下，视差图可以转换为深度图，存储每个像素点的深度值，表示每个像素点在空间中的位置。
下面再对结合深度值的策略进行介绍，根据各个目标像素点所对应的至少两个匹配度分别确定深度值，得到目标图像上各个像素点所对应的至少两个深度值，即根据目标图像和另一个图像得到的各个目标像素点所对应的匹配度后，直接根据该匹配度生成深度图，对于其他各个图像，都生成一个深度图。例如，采用SGM算法，使用目标像素点的各个匹配度分别生成深度图，即确定目标图像上各个像素点的深度值。这样，目标图像上每个像素点对应有至少两个深度值。
然后根据目标图像上各个像素点所对应的至少两个深度值，生成目标图像对应的深度图，记为第二深度图。根据至少两个深度值，生成第二深度图的实现方式可以包括多种，例如，对于同一个像素点，在至少两个深度值中 选择最小的深度值作为该像素点的第一深度值，聚合所有像素点的第一深度值，得到第二深度图；或者对于同一个像素点，计算该像素点的至少两个深度值的平均值，以该平均值生成第二深度图，或者其他任意适用的方式，本发明实施例对此不做限制。通过根据所述至少两个匹配度分别确定深度值，得到所述目标图像上各个像素点所对应的至少两个深度值，根据所述目标图像上各个像素点所对应的至少两个深度值，生成所述目标图像对应的第二深度图，使得至少两个匹配度不仅仅是基于两个拍摄装置之间的基线方向进行匹配得到的结果，结合至少两个匹配度对应确定的深度值来生成深度图，实现了对至少两个计算结果的结合，解决了两个拍摄装置之间在基线方向有时会匹配错误的问题，从而提高了深度图的准确度。
可选地，根据所述目标图像上各个像素点所对应的至少两个深度值，生成所述目标图像对应的第二深度图的一种实现方式可以包括：在同一个像素点所对应的至少两个深度值中，选择最小的深度值作为像素点的第一深度值，基于目标图像上各个像素点对应的第一深度值，生成目标图像对应的第二深度图。
每个像素点都对应有至少两个深度值，在其中选择最小的深度值作为该像素点的第一深度值，得到目标图像上各个像素点对应的第一深度值，即以此组成目标图像对应的第二深度图。选择最小的深度值作为该像素点的第一深度值的好处至少包括：在进行自主避障时，避免选择更大的深度值所导致的不必要的碰撞或其他事故，选择最小的深度值更加保险，提高了自主避障的成功率。
可选地，基于所述目标图像上各个像素点对应的第一深度值，生成所述目标图像对应的第二深度图的一种实现方式可以包括：对目标图像上各个像素点对应的第一深度值进行滤波处理，根据目标图像上各个像素点对应的滤波处理后的第一深度值，生成目标图像对应的第二深度图，对目标图像上各个像素点对应的第一深度值进行滤波处理，可以滤除一些错误的深度值，获得更加精确地深度图。
为了滤除一些像素点错误的深度值，需要对目标图像上各个像素点对应的第一深度值进行滤波处理，例如，采用Speckles Filter(斑点过滤器)对各 个像素点对应的第一深度值进行滤波处理。然后采用目标图像上各个像素点对应的滤波处理后的第一深度值，生成目标图像对应的第二深度图。
可选地，至少三个拍摄装置设置在可移动平台上时，在生成第一深度图或第二深度图后，还可以包括：根据第一深度图或第二深度图，确定可移动平台的运动轨迹或所述可移动平台上的机械臂的操作轨迹。第一深度图或第二深度图存储着可移动平台周围物体与可移动平台上拍摄装置的距离，继而可以根据拍摄装置在可移动平台上的位置，确定出可移动平台的各个部分与周围物体之间的距离，因此，根据第一深度图或第二深度图可以确定出可移动平台的运动轨迹，避免可移动平台与周围物体发生碰撞或者发生跌落等情况。在可移动平台上有机械臂时，还可以确定出机械臂与周围物体之间的距离，生成机械臂的操作轨迹，避免机械臂与周围物体发生碰撞，或者实现机械臂对周围目标物体的操作。
依据本发明实施例，通过获取至少三个拍摄装置所拍摄的至少三个图像，计算所述至少三个图像中目标图像上的目标像素点与其他各个图像上匹配的像素点之间的至少两个匹配度，对于同一个所述目标像素点，将其所对应的所述至少两个匹配度进行融合，基于各个所述目标像素点所对应的融合后的匹配度，生成所述目标图像对应的第一深度图；或者，根据所述至少两个匹配度分别确定深度值，得到所述目标图像上各个像素点所对应的至少两个深度值，根据所述目标图像上各个像素点所对应的至少两个深度值，生成所述目标图像对应的第二深度图，使得至少两个匹配度不仅仅是基于两个拍摄装置之间的基线方向进行匹配得到的结果，对至少两个匹配度进行融合后生成深度图，或者结合至少两个匹配度对应确定的深度值来生成深度图，实现了对至少两个计算结果的结合，解决了两个拍摄装置之间在基线方向有时会匹配错误的问题，从而提高了深度图的准确度。
可选地，至少三个拍摄装置设置在电子设备上，所述设置包括搭载、安装、连接或其他设置方式。所述电子设备包括可移动平台、移动终端、虚拟现实终端、增强现实终端等，或者其他任意适用的电子设备，本发明实施例对此不做限制。
其中，所述可移动平台包括但不限于飞行器、车、机器人等。所述飞行器包括无人驾驶飞行器，如旋翼飞行器，或固定翼飞行器，或者其他任意适 用的飞行器，本发明实施例对此不做限制。车包括有人驾驶汽车、无人驾驶汽车、以及遥控车，或者其他任意适用的车，本发明实施例对此不做限制。机器人包括扫地机器人、用于运输的货物的机器人、以及用于监测的机器人等，或者其他任意适用的机器人，本发明实施例对此不做限制。至少三个拍摄装置设置在飞行器、车、机器人等上时，采用本发明实施例的技术方案得到的深度图更加准确，因此，飞行器可以进行更好的自主避障、尤其是旋翼飞行器非常需要准确和敏捷的自主避障，避免飞行器发生坠机或碰撞等事故，提高飞行器的安全性；车也可以进行更好的自主避障，避免车与其他物体或人发生碰撞，使得车具有更高的安全性；机器人也可以进行更好的自主避障，机器人的机械臂可以更准确的对物体进行操作，尤其是扫地机器人可以减少与家具和移动物体的碰撞，不留死角的对所有可达到的位置进行清扫等，提高了机器人的工作能力。
移动终端包括手机、平板电脑、笔记本电脑等，或者其他任意适用的移动终端，本发明实施例对此不做限制。虚拟现实(VR，Virtual Reality)终端包括外接头戴式设备、一体机头显等，或者其他任意适用的虚拟现实终端，本发明实施例对此不做限制。增强现实(AR，Augmented Reality)终端包括透视式头盔、增强现实眼镜等，或者其他任意适用的增强现实终端，本发明实施例对此不做限制。
参照图6，示出了本发明另一实施例的一种深度图生成方法的步骤流程图，具体可以包括如下步骤：
步骤201，获取至少三个拍摄装置所拍摄的至少三个图像。
步骤202，分别对所述目标图像和所要匹配的图像进行矫正，得到矫正后的目标图像和所要匹配的图像，以及矫正后的目标图像和所述目标图像之间的映射关系。
在本发明实施例中，对目标图像和所要匹配的图像进行矫正，使得目标图像和所要匹配的图像上同一个像素点处于同一个水平线或垂直线，针对矫正后的图像进行基于基线方向的搜索可以获得更加精确地结果。如图7所示的图像矫正技术的示意图，在(1)中左右两个摄像头分别拍摄图像，对两个图像进行矫正，使得两个图像发生从(1)到(2)的变化，两个图像上， Corresponding Point(对应点)应该在平行的Epipolar Line(极线)上，如(2)中树顶的点在同一个水平线上，这样只需要在水平线上搜索就能找到匹配的像素点。对目标图像和所要匹配的图像进行矫正，得到矫正后的目标图像，矫正后的所要匹配的图像，以及矫正后的目标图像和目标图像之间的映射关系，矫正后的所要匹配的图像和所要匹配的图像之间的映射关系。
步骤203，将所述矫正后的目标图像上目标像素点和所述目标图像的矫正后的所要匹配的图像上的像素点进行匹配，分别计算所述矫正后的目标图像上各个所述目标像素点与矫正后的其所匹配的图像上匹配的像素点之间的匹配度。
在本发明实施例中，在对目标图像和所要匹配的图像进行矫正后，对矫正后的图像中的像素点进行匹配，即将矫正后的目标图像上目标像素点和目标图像的矫正后的所要匹配的图像上的像素点进行匹配，对于每个目标像素点，在另一个图像上寻找和该目标像素点最匹配的像素点，并计算得到该目标像素点与该另一个图像上匹配的像素点之间的匹配度。
步骤204，根据所述第一映射关系和第二映射关系，确定与所述第一图像对应的矫正后的目标图像和与所述第二图像对应的矫正后的目标图像之间的第三映射关系。
在本发明实施例中，以三个图像为例，目标图像分别于第一图像和第二图像进行像素点匹配，因此，矫正后的目标图像与目标图像之间的映射关系包括对目标图像和第一图像进行矫正得到的第一映射关系和对目标图像和第二图像进行矫正得到的第二映射关系。
由于与第一图像对应的矫正后的目标图像和与第二图像对应的矫正后的目标图像并不相同，也不是原始的目标图像，因此无法直接将基于与第一图像对应的矫正后的目标图像的匹配度和基于与第二图像对应的矫正后的目标图像的匹配度进行融合，需要先通过映射关系，将两个匹配度对应到同一个图像上。由于第一映射关系和第二映射关系都与原始的目标图像的映射关系，因此通过第一映射关系和第二映射关系可以计算出与第一图像对应的矫正后的目标图像和与第二图像对应的矫正后的目标图像之间的第三映射关系。
步骤205，根据所述第三映射关系，将与所述第二图像对应的矫正后的 目标图像对应的匹配度转换为与所述第一图像对应的矫正后的目标图像对应的匹配度。
现有技术中由于只有两个拍摄装置，因此可以直接对两个拍摄装置拍摄的两个图像进行计算获得一个匹配度，进而获得深度值。而本发明实施例由于至少需要三个拍摄装置，至少三个拍摄装置拍摄至少获得三个拍摄图像，因此当选择一个目标图像分别与另外的至少两个图像进行匹配时，对于目标图像上的同一个像素点会获得至少两个匹配度，要想对所述至少两个匹配度进行融合操作，则要求所述至少两个匹配度对应的是同一张图像。由前述可知，为了获得更加精确的结果，对至少三个拍摄图像进行了矫正操作，由此计算获得的匹配度所对应的矫正后的图像并不是同一个图像，因此，在本发明实施例中，利用第三映射关系，可以将与第二图像对应的矫正后的目标图像对应的匹配度转换为与第一图像对应的矫正后的目标图像对应的匹配度，使得目标像素点的匹配度都与基于同一个图像，以便后续可以进行融合。
可选地，只有采用的像素点匹配算法需要矫正时，才对目标图像和所要匹配的图像进行矫正。映射关系包括对目标图像和第三图像进行矫正得到的第四映射关系；在所述对于同一个所述目标像素点，将其所对应的所述至少两个匹配度进行融合之前，还可以包括：根据第四映射关系，将与第三图像对应的矫正后的目标图像对应的匹配度转换为目标图像对应的匹配度。
例如，目标图像和第三图像之间采用Stereo Matching算法进行像素点的匹配，因此需要对目标像素和第三图像进行矫正，得到的匹配度是与第三图像对应的矫正后的目标图像对应的匹配度，而目标图像和其他图像之间采用Plane-Sweeping算法进行像素点匹配，不需要矫正，得到的匹配度是与目标图像对应的匹配度，因此将不是与目标图像对应的匹配度都转换为与目标图像对应的匹配度。
步骤206，对于同一个所述目标像素点，将其所对应的所述至少两个匹配度进行融合，基于各个所述目标像素点所对应的融合后的匹配度，生成所述目标图像对应的第一深度图。
依据本发明实施例，通过获取至少三个拍摄装置所拍摄的至少三个图像，分别对所述目标图像和所要匹配的图像进行矫正，得到矫正后的目标图像和所要匹配的图像，以及矫正后的目标图像和所述目标图像之间的映射关系，将所述矫正后的目标图像上目标像素点和所述目标图像所要匹配的图像 上的像素点进行匹配，分别计算所述矫正后的目标图像上各个所述目标像素点与其所匹配的图像上匹配的像素点之间的匹配度，根据所述第一映射关系和第二映射关系，确定与所述第一图像对应的矫正后的目标图像和与所述第二图像对应的矫正后的目标图像之间的第三映射关系，根据所述第三映射关系，将与所述第二图像对应的矫正后的目标图像对应的匹配度转换为与所述第一图像对应的矫正后的目标图像对应的匹配度，对于同一个所述目标像素点，将其所对应的所述至少两个匹配度进行融合，基于各个所述目标像素点所对应的融合后的匹配度，生成所述目标图像对应的第一深度图，使得至少两个匹配度不仅仅是基于两个拍摄装置之间的基线方向进行匹配得到的结果，对至少两个匹配度进行融合后生成深度图，实现了对至少两个计算结果的结合，解决了两个拍摄装置之间在基线方向有时会匹配错误的问题，从而提高了深度图的准确度。
参照图8，示出了本发明又一实施例的一种深度图生成方法的步骤流程图，具体可以包括如下步骤：
步骤301，获取至少三个拍摄装置所拍摄的至少三个图像。
步骤302，分别对所述目标图像和所要匹配的图像进行矫正，得到矫正后的目标图像和所要匹配的图像，以及矫正后的目标图像和所述目标图像之间的映射关系。
步骤303，将所述矫正后的目标图像上目标像素点和所述目标图像所要匹配的图像上的像素点进行匹配，分别计算所述矫正后的目标图像上各个所述目标像素点与其所匹配的图像上匹配的像素点之间的匹配度。
步骤304，根据所述矫正后的目标图像上各个目标像素点与矫正后的其所匹配的图像上匹配的像素点之间的匹配度，分别计算深度值，得到所述目标图像上各个像素点所对应的至少两个深度值。
在本发明实施例中，若目标图像和所要匹配的图像不需要进行矫正，则根据至少两个匹配度分别确定深度值，得到目标图像上各个像素点所对应的至少两个深度值，若目标图像和所要匹配的图像需要进行矫正，则根据矫正后的目标图像上各个目标像素点与矫正后的其所要匹配的图像之间的匹配度，分别计算深度值，得到目标图像上各个像素点所对应的至少两个深度值。计算深度值的方法与不需要矫正时的方式相同，此处不另赘述。
例如，如图9所示的深度值生成的示意图。对上图和左图进行矫正时，上图矫正后得到上图1，左图矫正后得到左图1，对上图和右图进行矫正时，上图矫正后得到上图2，右图矫正后得到右图1。深度图1是基于左图1的深度图，深度图2是基于左图2的深度图。所以深度图1和深度图2实际上并不能直接结合，需要先找到映射关系，一一对应起来才能结合。
步骤305，根据所述第一映射关系和第二映射关系，确定与所述第一图像对应的矫正后的目标图像和与所述第二图像对应的矫正后的目标图像之间的第三映射关系。
步骤306，根据所述第三映射关系，将与所述第二图像对应的矫正后的目标图像对应的深度值转换为与所述第一图像对应的矫正后的目标图像对应的深度值。
在本发明实施例中，利用第三映射关系，可以将与第二图像对应的矫正后的目标图像对应的深度值转换为与第一图像对应的矫正后的目标图像对应的深度值，使得像素点的深度值都与基于同一个图像，以便后续可以进行集合。
例如，如图10所示的深度图结合的示意图，maphex1代表左图与左图1之间的映射关系(即第一映射关系)，maphex2代表左图与左图2之间的映射关系(即第二映射关系)，所以通过maphex1与maphex2可以计算出左图2到左图1的映射关系maphex3(即第三映射关系)，利用maphex3就能将深度图2的深度值映射到左图1上，这样深度图1和深度图2都是基于左图1的，然后用上述的深度值结合的策略，最终结合得到深度图4。
可选地，只有采用的像素点匹配算法需要时，才对目标图像和所要匹配的图像进行矫正。映射关系包括对目标图像和第三图像进行矫正得到的第四映射关系；在根据所述目标图像上各个像素点所对应的至少两个深度值，生成所述目标图像对应的第二深度图之前，还可以包括：根据所述第四映射关系，将与所述第三图像对应的矫正后的目标图像对应的深度值转换为所述目标图像对应的深度值。
例如，目标图像和第三图像之间采用Stereo Matching算法进行像素点的匹配，因此需要对目标像素和第三图像进行矫正，得到的深度值是与第三图像对应的矫正后的目标图像对应的深度值，而目标图像和其他图像之间采用 Plane-Sweeping算法进行像素点匹配，不需要矫正，得到的匹配度是与目标图像对应的深度值，因此将不是与目标图像对应的深度值都转换为与目标图像对应的深度值，需要注意的是，以上算法仅仅是示例性的，并不对本发明构成限制，为了实现本发明所述的方法，也可以采用其他算法进行实现。
步骤307，根据所述目标图像上各个像素点所对应的至少两个深度值，生成所述目标图像对应的第二深度图。
依据本发明实施例，通过获取至少三个拍摄装置所拍摄的至少三个图像，分别对所述目标图像和所要匹配的图像进行矫正，得到矫正后的目标图像和所要匹配的图像，以及矫正后的目标图像和所述目标图像之间的映射关系，将所述矫正后的目标图像上目标像素点和所述目标图像所要匹配的图像上的像素点进行匹配，分别计算所述矫正后的目标图像上各个所述目标像素点与其所匹配的图像上匹配的像素点之间的匹配度，根据所述矫正后的目标图像上各个目标像素点与其所匹配的图像上匹配的像素点之间的匹配度，分别计算深度值，得到所述目标图像上各个像素点所对应的至少两个深度值，根据所述第一映射关系和第二映射关系，确定与所述第一图像对应的矫正后的目标图像和与所述第二图像对应的矫正后的目标图像之间的第三映射关系，根据所述第三映射关系，将与所述第二图像对应的矫正后的目标图像对应的深度值转换为与所述第一图像对应的矫正后的目标图像对应的深度值，根据所述目标图像上各个像素点所对应的至少两个深度值，生成所述目标图像对应的第二深度图，使得至少两个匹配度不仅仅是基于两个拍摄装置之间的基线方向进行匹配得到的结果，结合至少两个深度值生成深度图，实现了对至少两个计算结果的结合，解决了两个拍摄装置之间在基线方向有时会匹配错误的问题，从而提高了深度图的准确度。
参照图11，示出了本发明再一实施例的一种电子设备的示意图，所述电子设备包括处理器401、存储器402和至少三个拍摄装置403；
所述处理器用于：获取至少三个拍摄装置所拍摄的至少三个图像；计算所述至少三个图像中目标图像上的目标像素点与其他各个图像上匹配的像素点之间的至少两个匹配度；对于同一个所述目标像素点，将其所对应的所述至少两个匹配度进行融合，基于各个所述目标像素点所对应的融合后的匹 配度，生成所述目标图像对应的第一深度图；或者，根据所述至少两个匹配度分别确定深度值，得到所述目标图像上各个像素点所对应的至少两个深度值，根据所述目标图像上各个像素点所对应的至少两个深度值，生成所述目标图像对应的第二深度图。
所述处理器在对于同一个所述目标像素点，将其所对应的所述至少两个匹配度进行融合时，用于：
对于同一个所述目标像素点，将其所对应的所述至少两个匹配度作求和运算，得到所述目标像素点所对应的融合后的匹配度。
所述处理器在根据所述目标图像上各个像素点所对应的至少两个深度值，生成所述目标图像对应的第二深度图时，用于：
在同一个所述像素点所对应的所述至少两个深度值中，选择最小的深度值作为所述像素点的第一深度值；
基于所述目标图像上各个像素点对应的第一深度值，生成所述目标图像对应的第二深度图。
所述处理器在基于所述目标图像上各个像素点对应的第一深度值，生成所述目标图像对应的第二深度图时，用于：
对所述目标图像上各个像素点对应的第一深度值进行滤波处理；
根据所述目标图像上各个像素点对应的滤波处理后的第一深度值，生成所述目标图像对应的第二深度图。
在所述处理器计算所述至少三个图像中目标图像上的目标像素点与其他各个图像上匹配的像素点之间的至少两个匹配度之前，所述处理器还用于：
分别对所述目标图像和所要匹配的图像进行矫正，得到矫正后的目标图像和所要匹配的图像，以及矫正后的目标图像和所述目标图像之间的映射关系。
所述处理器在计算所述至少三个图像中目标图像上的目标像素点与其他各个图像上匹配的像素点之间的至少两个匹配度时，用于：
将所述矫正后的目标图像上目标像素点和所述目标图像的矫正后的所要匹配的图像上的像素点进行匹配，分别计算所述矫正后的目标图像上各个所述目标像素点与矫正后的其所匹配的图像上匹配的像素点之间的匹配度。
所述映射关系包括对所述目标图像和第一图像进行矫正得到的第一映射关系和对所述目标图像和第二图像进行矫正得到的第二映射关系；在所述处理器对于同一个所述目标像素点，将其所对应的所述至少两个匹配度进行融合之前，所述处理器还用于：
根据所述第一映射关系和第二映射关系，确定与所述第一图像对应的矫正后的目标图像和与所述第二图像对应的矫正后的目标图像之间的第三映射关系；
根据所述第三映射关系，将与所述第二图像对应的矫正后的目标图像对应的匹配度转换为与所述第一图像对应的矫正后的目标图像对应的匹配度。
所述映射关系包括对所述目标图像和第三图像进行矫正得到的第四映射关系；在所述处理器对于同一个所述目标像素点，将其所对应的所述至少两个匹配度进行融合之前，所述处理器还用于：
根据所述第四映射关系，将与所述第三图像对应的矫正后的目标图像对应的匹配度转换为所述目标图像对应的匹配度。
所述处理器根据所述至少两个匹配度分别确定深度值，得到所述目标图像上各个像素点所对应的至少两个深度值时，用于：
根据所述矫正后的目标图像上各个目标像素点与其所匹配的图像上匹配的像素点之间的匹配度，分别计算深度值，得到所述目标图像上各个像素点所对应的至少两个深度值。
所述映射关系包括对所述目标图像和第一图像进行矫正得到的第一映射关系和对所述目标图像和第二图像进行矫正得到的第二映射关系；在所述处理器根据所述目标图像上各个像素点所对应的至少两个深度值，生成所述目标图像对应的第二深度图之前，所述处理器还用于：
根据所述第一映射关系和第二映射关系，确定与所述第一图像对应的矫正后的目标图像和与所述第二图像对应的矫正后的目标图像之间的第三映射关系；
根据所述第三映射关系，将与所述第二图像对应的矫正后的目标图像对应的深度值转换为与所述第一图像对应的矫正后的目标图像对应的深度值。
所述映射关系包括对所述目标图像和第三图像进行矫正得到的第四映射关系；在所述处理器根据所述目标图像上各个像素点所对应的至少两个深 度值，生成所述目标图像对应的第二深度图之前，所述处理器还用于：
根据所述第四映射关系，将与所述第三图像对应的矫正后的目标图像对应的深度值转换为所述目标图像对应的深度值。
所述拍摄装置为三个，所述拍摄装置之间的位置关系包括品字形、或L形中的任意一种。
所述电子设备包括可移动平台、移动终端、虚拟现实终端、或增强现实终端中的任意一种。所述可移动平台包括飞行棋、车、或机器人等。所述飞行器包括无人驾驶飞行器，如旋翼飞行器，或固定翼飞行器，或者其他任意适用的飞行器，本发明实施例对此不做限制。车包括有人驾驶汽车、无人驾驶汽车、以及遥控车，或者其他任意适用的车，本发明实施例对此不做限制。机器人包括扫地机器人、用于运输的货物的机器人、以及用于监测的机器人等，或者其他任意适用的机器人，本发明实施例对此不做限制。
所述处理器还用于：
根据所述第一深度图或第二深度图，确定所述可移动平台的运动轨迹或所述可移动平台上的机械臂的操作轨迹。
所述电子设备包括显示器，所述显示器用于：
对所述第一深度图或第二深度图进行显示，通过显示所述深度图，可以使操作人员及时地、直观地了解到电子设备周围的环境信息，例如电子设备周围有哪些物体，哪些物体距离电子设备较近，哪些物体距离电子设备较远，从而及时的电子设备的移动、作业、或其他操作进行控制。需要注意的是，所述电子设备包括显示器并不限制于所述显示器必须设置在所述电子设备上，所说显示器也可以与所述电子设备以通信方式进行连接，例如，所述显示器可以与所述可移动平台通过蓝牙、移动网络、或WiFi连接，所述显示器可以设置在远程的遥控器上，也可以设置在远程的计算机上，本发明对此不作限制。
所述处理器还用于：
将所述第一深度图或第二深度图发送给所述可移动平台的控制设备，以供所述控制设备对所述第一深度图或第二深度图进行显示或根据所述第一深度图或第二深度图生成对所述可移动平台的控制指令。
其中，控制设备可以通过蓝牙、无线网络、5G网络等方式与可移动平 台进行通信，控制设备可以对第一深度图或第二深度图进行实时显示，或者控制设备可以根据第一深度图或第二深度图生成控制指令，例如，由控制设备根据第一深度图或第二深度图，确定可移动平台的运动轨迹或可移动平台上的机械臂的操作轨迹，并根据运动轨迹或操作轨迹生成相应的控制指令，再将控制指令发送给可移动平台。
依据本发明实施例，通过获取至少三个拍摄装置所拍摄的至少三个图像，计算所述至少三个图像中目标图像上的目标像素点与其他各个图像上匹配的像素点之间的至少两个匹配度，对于同一个所述目标像素点，将其所对应的所述至少两个匹配度进行融合，基于各个所述目标像素点所对应的融合后的匹配度，生成所述目标图像对应的第一深度图；或者，根据所述至少两个匹配度分别确定深度值，得到所述目标图像上各个像素点所对应的至少两个深度值，根据所述目标图像上各个像素点所对应的至少两个深度值，生成所述目标图像对应的第二深度图，使得至少两个匹配度不仅仅是基于两个拍摄装置之间的基线方向进行匹配得到的结果，对至少两个匹配度进行融合后生成深度图，或者结合至少两个匹配度对应确定的深度值来生成深度图，实现了对至少两个计算结果的结合，解决了两个拍摄装置之间在基线方向有时会匹配错误的问题，从而提高了深度图的准确度。
以上所描述的装置实施例仅仅是示意性的，其中所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部模块来实现本实施例方案的目的。本领域普通技术人员在不付出创造性的劳动的情况下，即可以理解并实施。
本发明的各个部件实施例可以以硬件实现，或者以在一个或者多个处理器上运行的软件模块实现，或者以它们的组合实现。本领域的技术人员应当理解，可以在实践中使用微处理器或者数字信号处理器(DSP)来实现根据本发明实施例的计算处理设备中的一些或者全部部件的一些或者全部功能。本发明还可以实现为用于执行这里所描述的方法的一部分或者全部的设备或者装置程序(例如，计算机程序和计算机程序产品)。这样的实现本发明的程序可以存储在计算机可读介质上，或者可以具有一个或者多个信号的形 式。这样的信号可以从因特网网站上下载得到，或者在载体信号上提供，或者以任何其他形式提供。
例如，图12示出了可以实现根据本发明的方法的计算处理设备。该计算处理设备传统上包括处理器1010和以存储器1020形式的计算机程序产品或者计算机可读介质。存储器1020可以是诸如闪存、EEPROM(电可擦除可编程只读存储器)、EPROM、硬盘或者ROM之类的电子存储器。存储器1020具有用于执行上述方法中的任何方法步骤的程序代码1031的存储空间1030。例如，用于程序代码的存储空间1030可以包括分别用于实现上面的方法中的各种步骤的各个程序代码1031。这些程序代码可以从一个或者多个计算机程序产品中读出或者写入到这一个或者多个计算机程序产品中。这些计算机程序产品包括诸如硬盘，紧致盘(CD)、存储卡或者软盘之类的程序代码载体。这样的计算机程序产品通常为如参考图13所述的便携式或者固定存储单元。该存储单元可以具有与图12的计算处理设备中的存储器1020类似布置的存储段、存储空间等。程序代码可以例如以适当形式进行压缩。通常，存储单元包括计算机可读代码1031’，即可以由例如诸如1010之类的处理器读取的代码，这些代码当由计算处理设备运行时，导致该计算处理设备执行上面所描述的方法中的各个步骤。
本文中所称的“一个实施例”、“实施例”或者“一个或者多个实施例”意味着，结合实施例描述的特定特征、结构或者特性包括在本发明的至少一个实施例中。此外，请注意，这里“在一个实施例中”的词语例子不一定全指同一个实施例。
在此处所提供的说明书中，说明了大量具体细节。然而，能够理解，本发明的实施例可以在没有这些具体细节的情况下被实践。在一些实例中，并未详细示出公知的方法、结构和技术，以便不模糊对本说明书的理解。
在权利要求中，不应将位于括号之间的任何参考符号构造成对权利要求的限制。单词“包含”不排除存在未列在权利要求中的元件或步骤。位于元件之前的单词“一”或“一个”不排除存在多个这样的元件。本发明可以借助于包括有若干不同元件的硬件以及借助于适当编程的计算机来实现。在列举了若干装置的单元权利要求中，这些装置中的若干个可以是通过同一个硬件项来具体体现。单词第一、第二、以及第三等的使用不表示任何顺序。可将这些单词解释为名称。
最后应说明的是：以上实施例仅用以说明本发明的技术方案，而非对其限制；尽管参照前述实施例对本发明进行了详细的说明，本领域的普通技术人员应当理解：其依然可以对前述各实施例所记载的技术方案进行修改，或者对其中部分技术特征进行等同替换；而这些修改或者替换，并不使相应技术方案的本质脱离本发明各实施例技术方案的精神和范围。

Claims (32)

  1. 一种深度图生成方法，其特征在于，包括：
    获取至少三个拍摄装置所拍摄的至少三个图像；
    计算所述至少三个图像中目标图像上的目标像素点与其他各个图像上匹配的像素点之间的至少两个匹配度；
    对于同一个所述目标像素点，将其所对应的所述至少两个匹配度进行融合，基于各个所述目标像素点所对应的融合后的匹配度，生成所述目标图像对应的第一深度图；或者，根据所述至少两个匹配度分别确定深度值，得到所述目标图像上各个像素点所对应的至少两个深度值，根据所述目标图像上各个像素点所对应的至少两个深度值，生成所述目标图像对应的第二深度图。
  2. 根据权利要求1所述的方法，其特征在于，所述对于同一个所述目标像素点，将其所对应的所述至少两个匹配度进行融合包括：
    对于同一个所述目标像素点，将其所对应的所述至少两个匹配度作求和运算，得到所述目标像素点所对应的融合后的匹配度。
  3. 根据权利要求1所述的方法，其特征在于，所述根据各个所述目标像素点所对应的至少两个深度值，生成所述目标图像对应的第二深度图包括：
    在同一个所述像素点所对应的所述至少两个深度值中，选择最小的深度值作为所述像素点的第一深度值；
    基于各个所述目标像素点对应的第一深度值，生成所述目标图像对应的第二深度图。
  4. 根据权利要求3所述的方法，其特征在于，所述基于所述目标图像上各个像素点对应的第一深度值，生成所述目标图像对应的第二深度图包括：
    对所述目标图像上各个像素点对应的第一深度值进行滤波处理；
    根据所述目标图像上各个像素点对应的滤波处理后的第一深度值，生成所述目标图像对应的第二深度图。
  5. 根据权利要求1所述的方法，其特征在于，在所述计算所述至少三个图像中目标图像上的目标像素点与其他各个图像上匹配的像素点之间的 至少两个匹配度之前，所述方法还包括：
    分别对所述目标图像和所要匹配的图像进行矫正，得到矫正后的目标图像和所要匹配的图像，以及矫正后的目标图像和所述目标图像之间的映射关系。
  6. 根据权利要求5所述的方法，其特征在于，所述计算所述至少三个图像中目标图像上的目标像素点与其他各个图像上匹配的像素点之间的至少两个匹配度包括：
    将所述矫正后的目标图像上目标像素点和所述目标图像的矫正后的所要匹配的图像上的像素点进行匹配，分别计算所述矫正后的目标图像上各个所述目标像素点与矫正后的其所匹配的图像上匹配的像素点之间的匹配度。
  7. 根据权利要求6所述的方法，其特征在于，所述映射关系包括对所述目标图像和第一图像进行矫正得到的第一映射关系和对所述目标图像和第二图像进行矫正得到的第二映射关系；在所述对于同一个所述目标像素点，将其所对应的所述至少两个匹配度进行融合之前，所述方法还包括：
    根据所述第一映射关系和第二映射关系，确定与所述第一图像对应的矫正后的目标图像和与所述第二图像对应的矫正后的目标图像之间的第三映射关系；
    根据所述第三映射关系，将与所述第二图像对应的矫正后的目标图像对应的匹配度转换为与所述第一图像对应的矫正后的目标图像对应的匹配度。
  8. 根据权利要求6所述的方法，其特征在于，所述映射关系包括对所述目标图像和第三图像进行矫正得到的第四映射关系；在所述对于同一个所述目标像素点，将其所对应的所述至少两个匹配度进行融合之前，所述方法还包括：
    根据所述第四映射关系，将与所述第三图像对应的矫正后的目标图像对应的匹配度转换为所述目标图像对应的匹配度。
  9. 根据权利要求6所述的方法，其特征在于，所述根据所述至少两个匹配度分别确定深度值，得到所述目标图像上各个像素点所对应的至少两个深度值包括：
    根据所述矫正后的目标图像上各个目标像素点与矫正后的其所匹配的图像上匹配的像素点之间的匹配度，分别计算深度值，得到所述目标图像上 各个像素点所对应的至少两个深度值。
  10. 根据权利要求9所述的方法，其特征在于，所述映射关系包括对所述目标图像和第一图像进行矫正得到的第一映射关系和对所述目标图像和第二图像进行矫正得到的第二映射关系；在所述根据所述目标图像上各个像素点所对应的至少两个深度值，生成所述目标图像对应的第二深度图之前，所述方法还包括：
    根据所述第一映射关系和第二映射关系，确定与所述第一图像对应的矫正后的目标图像和与所述第二图像对应的矫正后的目标图像之间的第三映射关系；
    根据所述第三映射关系，将与所述第二图像对应的矫正后的目标图像对应的深度值转换为与所述第一图像对应的矫正后的目标图像对应的深度值。
  11. 根据权利要求9所述的方法，其特征在于，所述映射关系包括对所述目标图像和第三图像进行矫正得到的第四映射关系；在所述根据所述目标图像上各个像素点所对应的至少两个深度值，生成所述目标图像对应的第二深度图之前，所述方法还包括：
    根据所述第四映射关系，将与所述第三图像对应的矫正后的目标图像对应的深度值转换为所述目标图像对应的深度值。
  12. 根据权利要求1所述的方法，其特征在于，所述拍摄装置为三个，所述拍摄装置之间的位置关系包括品字形、或L形中的任意一种。
  13. 根据权利要求1所述的方法，其特征在于，所述至少三个拍摄装置设置在电子设备上，所述电子设备包括可移动平台、移动终端、虚拟现实终端、或增强现实终端中的任意一种。
  14. 根据权利要求1所述的方法，其特征在于，所述至少三个拍摄装置设置在可移动平台上，所述方法还包括：
    根据所述第一深度图或第二深度图，确定所述可移动平台的运动轨迹或所述可移动平台上的机械臂的操作轨迹。
  15. 一种电子设备，其特征在于，所述电子设备包括处理器、存储器和至少三个拍摄装置；
    所述处理器用于：获取至少三个拍摄装置所拍摄的至少三个图像；计算 所述至少三个图像中目标图像上的目标像素点与其他各个图像上匹配的像素点之间的至少两个匹配度；对于同一个所述目标像素点，将其所对应的所述至少两个匹配度进行融合，基于各个所述目标像素点所对应的融合后的匹配度，生成所述目标图像对应的第一深度图；或者，根据所述至少两个匹配度分别确定深度值，得到所述目标图像上各个像素点所对应的至少两个深度值，根据所述目标图像上各个像素点所对应的至少两个深度值，生成所述目标图像对应的第二深度图。
  16. 根据权利要求15所述的电子设备，其特征在于，所述处理器在对于同一个所述目标像素点，将其所对应的所述至少两个匹配度进行融合时，用于：
    对于同一个所述目标像素点，将其所对应的所述至少两个匹配度作求和运算，得到所述目标像素点所对应的融合后的匹配度。
  17. 根据权利要求15所述的电子设备，其特征在于，所述处理器在根据所述目标图像上各个像素点所对应的至少两个深度值，生成所述目标图像对应的第二深度图时，用于：
    在同一个所述像素点所对应的所述至少两个深度值中，选择最小的深度值作为所述像素点的第一深度值；
    基于所述目标图像上各个像素点对应的第一深度值，生成所述目标图像对应的第二深度图。
  18. 根据权利要求17所述的电子设备，其特征在于，所述处理器在基于所述目标图像上各个像素点对应的第一深度值，生成所述目标图像对应的第二深度图时，用于：
    对所述目标图像上各个像素点对应的第一深度值进行滤波处理；
    根据所述目标图像上各个像素点对应的滤波处理后的第一深度值，生成所述目标图像对应的第二深度图。
  19. 根据权利要求15所述的电子设备，其特征在于，在所述处理器计算所述至少三个图像中目标图像上的目标像素点与其他各个图像上匹配的像素点之间的至少两个匹配度之前，所述处理器还用于：
    分别对所述目标图像和所要匹配的图像进行矫正，得到矫正后的目标图像和所要匹配的图像，以及矫正后的目标图像和所述目标图像之间的映射关 系。
  20. 根据权利要求19所述的电子设备，其特征在于，所述处理器在计算所述至少三个图像中目标图像上的目标像素点与其他各个图像上匹配的像素点之间的至少两个匹配度时，用于：
    将所述矫正后的目标图像上目标像素点和所述目标图像的矫正后的所要匹配的图像上的像素点进行匹配，分别计算所述矫正后的目标图像上各个所述目标像素点与矫正后的其所匹配的图像上匹配的像素点之间的匹配度。
  21. 根据权利要求20所述的电子设备，其特征在于，所述映射关系包括对所述目标图像和第一图像进行矫正得到的第一映射关系和对所述目标图像和第二图像进行矫正得到的第二映射关系；在所述处理器对于同一个所述目标像素点，将其所对应的所述至少两个匹配度进行融合之前，所述处理器还用于：
    根据所述第一映射关系和第二映射关系，确定与所述第一图像对应的矫正后的目标图像和与所述第二图像对应的矫正后的目标图像之间的第三映射关系；
    根据所述第三映射关系，将与所述第二图像对应的矫正后的目标图像对应的匹配度转换为与所述第一图像对应的矫正后的目标图像对应的匹配度。
  22. 根据权利要求20所述的电子设备，其特征在于，所述映射关系包括对所述目标图像和第三图像进行矫正得到的第四映射关系；在所述处理器对于同一个所述目标像素点，将其所对应的所述至少两个匹配度进行融合之前，所述处理器还用于：
    根据所述第四映射关系，将与所述第三图像对应的矫正后的目标图像对应的匹配度转换为所述目标图像对应的匹配度。
  23. 根据权利要求20所述的电子设备，其特征在于，所述处理器根据所述至少两个匹配度分别确定深度值，得到所述目标图像上各个像素点所对应的至少两个深度值时，用于：
    根据所述矫正后的目标图像上各个目标像素点与其所匹配的图像上匹配的像素点之间的匹配度，分别计算深度值，得到所述目标图像上各个像素点所对应的至少两个深度值。
  24. 根据权利要求23所述的电子设备，其特征在于，所述映射关系包 括对所述目标图像和第一图像进行矫正得到的第一映射关系和对所述目标图像和第二图像进行矫正得到的第二映射关系；在所述处理器根据所述目标图像上各个像素点所对应的至少两个深度值，生成所述目标图像对应的第二深度图之前，所述处理器还用于：
    根据所述第一映射关系和第二映射关系，确定与所述第一图像对应的矫正后的目标图像和与所述第二图像对应的矫正后的目标图像之间的第三映射关系；
    根据所述第三映射关系，将与所述第二图像对应的矫正后的目标图像对应的深度值转换为与所述第一图像对应的矫正后的目标图像对应的深度值。
  25. 根据权利要求23所述的电子设备，其特征在于，所述映射关系包括对所述目标图像和第三图像进行矫正得到的第四映射关系；在所述处理器根据所述目标图像上各个像素点所对应的至少两个深度值，生成所述目标图像对应的第二深度图之前，所述处理器还用于：
    根据所述第四映射关系，将与所述第三图像对应的矫正后的目标图像对应的深度值转换为所述目标图像对应的深度值。
  26. 根据权利要求15所述的电子设备，其特征在于，所述拍摄装置为三个，所述拍摄装置之间的位置关系包括品字形、或L形中的任意一种。
  27. 根据权利要求15所述的电子设备，其特征在于，所述电子设备包括可移动平台、移动终端、虚拟现实终端、或增强现实终端中的任意一种。
  28. 根据权利要求15所述的电子设备，其特征在于，所述电子设备为可移动平台，所述处理器还用于：
    根据所述第一深度图或第二深度图，确定所述可移动平台的运动轨迹或所述可移动平台上的机械臂的操作轨迹。
  29. 根据权利要求15所述的电子设备，其特征在于，所述电子设备包括显示器，所述显示器用于：
    对所述第一深度图或第二深度图进行显示。
  30. 根据权利要求15所述的电子设备，其特征在于，所述处理器还用于：
    将所述第一深度图或第二深度图发送给所述电子设备的控制设备，以供所述控制设备对所述第一深度图或第二深度图进行显示或根据所述第一深 度图或第二深度图生成对所述电子设备的控制指令。
  31. 一种计算机程序，包括计算机可读代码，当所述计算机可读代码在计算处理设备上运行时，导致所述计算处理设备执行根据权利要求1-14中的任一个所述的深度图生成方法。
  32. 一种计算机可读介质，其中存储了如权利要求31所述的计算机程序。
PCT/CN2020/087569 2020-04-28 2020-04-28 深度图生成方法、电子设备、计算处理设备及存储介质 WO2021217444A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
PCT/CN2020/087569 WO2021217444A1 (zh) 2020-04-28 2020-04-28 深度图生成方法、电子设备、计算处理设备及存储介质
CN202080044087.6A CN113994382A (zh) 2020-04-28 2020-04-28 深度图生成方法、电子设备、计算处理设备及存储介质

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2020/087569 WO2021217444A1 (zh) 2020-04-28 2020-04-28 深度图生成方法、电子设备、计算处理设备及存储介质

Publications (1)

Publication Number Publication Date
WO2021217444A1 true WO2021217444A1 (zh) 2021-11-04

Family

ID=78331582

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/087569 WO2021217444A1 (zh) 2020-04-28 2020-04-28 深度图生成方法、电子设备、计算处理设备及存储介质

Country Status (2)

Country Link
CN (1) CN113994382A (zh)
WO (1) WO2021217444A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115297249A (zh) * 2022-09-28 2022-11-04 深圳慧源创新科技有限公司 一种双目摄像头及双目避障方法

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070093945A1 (en) * 2005-10-20 2007-04-26 Grzywna Jason W System and method for onboard vision processing
CN106127788A (zh) * 2016-07-04 2016-11-16 触景无限科技（北京）有限公司 一种视觉避障方法和装置
CN106960454A (zh) * 2017-03-02 2017-07-18 武汉星巡智能科技有限公司 景深避障方法、设备及无人飞行器
CN107077741A (zh) * 2016-11-11 2017-08-18 深圳市大疆创新科技有限公司 深度图生成方法和基于该方法的无人机
CN110570468A (zh) * 2019-08-16 2019-12-13 苏州禾昆智能科技有限公司 一种基于深度学习的双目视觉深度估计方法及其系统

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070093945A1 (en) * 2005-10-20 2007-04-26 Grzywna Jason W System and method for onboard vision processing
CN106127788A (zh) * 2016-07-04 2016-11-16 触景无限科技（北京）有限公司 一种视觉避障方法和装置
CN107077741A (zh) * 2016-11-11 2017-08-18 深圳市大疆创新科技有限公司 深度图生成方法和基于该方法的无人机
CN106960454A (zh) * 2017-03-02 2017-07-18 武汉星巡智能科技有限公司 景深避障方法、设备及无人飞行器
CN110570468A (zh) * 2019-08-16 2019-12-13 苏州禾昆智能科技有限公司 一种基于深度学习的双目视觉深度估计方法及其系统

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115297249A (zh) * 2022-09-28 2022-11-04 深圳慧源创新科技有限公司 一种双目摄像头及双目避障方法
CN115297249B (zh) * 2022-09-28 2023-01-06 深圳慧源创新科技有限公司 一种双目摄像头及双目避障方法

Also Published As

Publication number Publication date
CN113994382A (zh) 2022-01-28

Similar Documents

Publication Publication Date Title
US11734846B2 (en) System and method for concurrent odometry and mapping
CN112771539B (zh) 采用使用神经网络从二维图像预测的三维数据以用于3d建模应用
US11282287B2 (en) Employing three-dimensional (3D) data predicted from two-dimensional (2D) images using neural networks for 3D modeling applications and other applications
CN110568447B (zh) 视觉定位的方法、装置及计算机可读介质
WO2019170164A1 (zh) 基于深度相机的三维重建方法、装置、设备及存储介质
US9846960B2 (en) Automated camera array calibration
CN106960454B (zh) 景深避障方法、设备及无人飞行器
JP2019536170A (ja) 仮想的に拡張された視覚的同時位置特定及びマッピングのシステム及び方法
WO2023016271A1 (zh) 位姿确定方法、电子设备及可读存储介质
US20160063705A1 (en) Systems and methods for determining a seam
WO2021139176A1 (zh) 基于双目摄像机标定的行人轨迹跟踪方法、装置、计算机设备及存储介质
US11216979B2 (en) Dual model for fisheye lens distortion and an algorithm for calibrating model parameters
Meilland et al. Dense visual mapping of large scale environments for real-time localisation
CN112115980A (zh) 基于光流跟踪和点线特征匹配的双目视觉里程计设计方法
US20200294269A1 (en) Calibrating cameras and computing point projections using non-central camera model involving axial viewpoint shift
CN112541973B (zh) 虚实叠合方法与系统
CN114022560A (zh) 标定方法及相关装置、设备
WO2024087962A1 (zh) 车厢姿态识别系统、方法、电子设备及存储介质
TW202244680A (zh) 位置姿勢獲取方法、電子設備及電腦可讀儲存媒體
JP6922348B2 (ja) 情報処理装置、方法、及びプログラム
CN110673607B (zh) 动态场景下的特征点提取方法、装置、及终端设备
WO2021217444A1 (zh) 深度图生成方法、电子设备、计算处理设备及存储介质
CN111444768A (zh) 一种用于反光地面场景的微小障碍物发现方法
CN114266823A (zh) 一种结合SuperPoint网络特征提取的单目SLAM方法
CN112802114A (zh) 多视觉传感器融合装置及其方法和电子设备

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20934067

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20934067

Country of ref document: EP

Kind code of ref document: A1