WO2021043285A1 - 一种双目立体视差确定方法、装置及无人机 - Google Patents

一种双目立体视差确定方法、装置及无人机 Download PDF

Info

Publication number
WO2021043285A1
WO2021043285A1 PCT/CN2020/113584 CN2020113584W WO2021043285A1 WO 2021043285 A1 WO2021043285 A1 WO 2021043285A1 CN 2020113584 W CN2020113584 W CN 2020113584W WO 2021043285 A1 WO2021043285 A1 WO 2021043285A1
Authority
WO
WIPO (PCT)
Prior art keywords
pixel
value
matched
disparity
extreme value
Prior art date
Application number
PCT/CN2020/113584
Other languages
English (en)
French (fr)
Inventor
代真佑
Original Assignee
深圳市道通智能航空技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳市道通智能航空技术有限公司 filed Critical 深圳市道通智能航空技术有限公司
Publication of WO2021043285A1 publication Critical patent/WO2021043285A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/80—Geometric correction
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
    • G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10004—Still image; Photographic image
    • G06T2207/10012—Stereo images
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20228—Disparity calculation for image-based rendering

Definitions

  • This application relates to the technical field of stereo matching, and in particular to a method, device and drone for determining binocular stereo disparity.
  • UAV is an unmanned aerial vehicle operated by radio remote control equipment or its own program control device.
  • the depth map of the environment in front of the UAV is generally obtained through the binocular camera device to determine the obstacle situation in the environment in front of the UAV, and then the obstacle avoidance is realized according to the determined obstacle situation .
  • the binocular camera device determines the depth map of the environment in front of the drone by using the captured binocular images for stereo matching.
  • the binocular camera device when it performs stereo matching on binocular images, it mainly determines the corresponding points of all pixels to be matched on the image to be matched in the matched image, and then determines the disparity between all pixels to be matched and their corresponding points Depth map, in this method, all the pixels to be matched on the image to be matched are searched for corresponding points. If the pixel to be matched is located in a repeated texture scene or a non-textured scene, there are multiple corresponding points for the pixel to be matched.
  • the embodiments of the present invention aim to provide a method and device for determining a binocular stereo disparity, and an unmanned aerial vehicle, which can improve the accuracy of determining the target disparity.
  • a technical solution adopted in the embodiments of the present invention is to provide a method for determining binocular stereo disparity, which is applied to a binocular camera device of a drone, and the method includes:
  • the image disparity space includes the pixel disparity space of each pixel to be matched in the image to be matched, wherein the pixel disparity space of each pixel to be matched is Both include the cost aggregation values of the pixels to be matched under different parallaxes;
  • the pixel to be matched that meets the preset matching condition is filtered out as the reference pixel;
  • the disparity corresponding to the cost aggregation value satisfying the preset aggregation condition is determined as the target disparity according to the cost aggregation value of each reference pixel under each disparity.
  • the filtering out the pixel to be matched that meets a preset matching condition as the reference pixel according to the pixel disparity space of each pixel to be matched includes:
  • the reference pixels are filtered out.
  • the cost aggregation extreme value includes the smallest extreme value, the next smallest extreme value, the next smallest extreme value, and the largest extreme value;
  • the cutoff value includes a first cutoff value, a second cutoff value, and a third cutoff value;
  • the aggregating extreme values according to the cost and calculating the boundary value of the pixel disparity space includes:
  • the third boundary value is obtained.
  • the first boundary value is one-half of the sum of the minimum extreme value and the maximum extreme value
  • the second demarcation value is one half of the sum of the first demarcation value and the minimum extreme value
  • the third boundary value is one half of the sum of the second boundary value and the minimum extreme value.
  • the filtering out reference pixels according to the boundary value and the cost aggregation extreme value includes:
  • the pixel to be matched is selected as the reference pixel.
  • the determining the disparity corresponding to the cost aggregation value satisfying a preset aggregation condition as the target disparity according to the cost aggregation value of each reference pixel under each disparity respectively includes:
  • the disparity corresponding to the minimum cost aggregation value is selected as the target disparity.
  • the method further includes:
  • the correcting the target disparity includes:
  • the target parallax is corrected by parabolic interpolation.
  • the method further includes:
  • the SAD algorithm is used to calculate the cost aggregation value of each pixel to be matched in the image to be matched under different parallaxes.
  • another technical solution adopted by the embodiments of the present invention is to provide a binocular stereo disparity determining device applied to a binocular camera device of an unmanned aerial vehicle, and the device includes:
  • the establishment module is used to establish an image disparity space according to the image to be matched and the matching image, the image disparity space includes the pixel disparity space of each pixel to be matched in the image to be matched, wherein each pixel to be matched
  • the pixel disparity space of the points all includes the cost aggregation value of the pixel to be matched under different disparity;
  • the screening module is configured to screen out the pixel to be matched that meets the preset matching condition as the reference pixel according to the pixel disparity space of each pixel to be matched;
  • the determining module is configured to determine the disparity corresponding to the cost aggregation value satisfying the preset aggregation condition according to the cost aggregation value of each reference pixel under each disparity among the reference pixels as Target parallax.
  • the screening module is specifically used for:
  • the reference pixels are filtered out.
  • the cost aggregation extreme value includes the smallest extreme value, the next smallest extreme value, the next smallest extreme value, and the largest extreme value;
  • the cutoff value includes a first cutoff value, a second cutoff value, and a third cutoff value;
  • the screening module is specifically used for:
  • the third boundary value is obtained.
  • the first boundary value is one-half of the sum of the minimum extreme value and the maximum extreme value
  • the second demarcation value is one half of the sum of the first demarcation value and the minimum extreme value
  • the third boundary value is one half of the sum of the second boundary value and the minimum extreme value.
  • the screening module is specifically used for:
  • the pixel to be matched is selected as the reference pixel.
  • the determining module is specifically configured to:
  • the disparity corresponding to the minimum cost aggregation value is selected as the target disparity.
  • the device further includes:
  • the correction module is used to correct the target parallax.
  • correction module is specifically configured to:
  • the target parallax is corrected by parabolic interpolation.
  • the establishment module is further used for:
  • the SAD algorithm is used to calculate the cost aggregation value of each pixel to be matched in the image to be matched under different parallaxes.
  • a drone including:
  • An arm connected to the fuselage
  • the power plant is arranged on the arm;
  • a binocular camera device connected to the body
  • the binocular camera device includes:
  • At least one processor At least one processor
  • the device can be used to perform the above-mentioned binocular stereo disparity determination method.
  • another technical solution adopted by the embodiments of the present invention is to provide a non-volatile computer-readable storage medium, which is characterized in that the non-volatile computer-readable storage medium stores a computer-readable storage medium. Executing instructions, the computer-executable instructions are used to make the binocular camera device of the drone execute the above-mentioned binocular stereo disparity determination method.
  • the embodiment of the present invention provides a binocular stereo disparity determination method, device, and drone.
  • the binocular stereo disparity determination method each For the pixel disparity space of the pixels to be matched, each pixel to be matched in the image to be matched is screened, and the pixel to be matched that meets the preset matching conditions is selected as the reference pixel, and then the target disparity is determined by the reference pixel.
  • the pixels to be matched that do not meet the preset matching conditions may be located in repeated texture scenes or untextured scenes, and the target disparity determined by them is prone to errors. Therefore, the pixels to be matched that meet the preset matching conditions are selected to determine the target disparity. , Which can improve the accuracy of determining the target parallax, thereby reducing the problem of false detection in repeated texture scenes or non-textured scenes.
  • FIG. 1 is a schematic structural diagram of an unmanned aerial vehicle provided by an embodiment of the present invention
  • Figure 2 is a schematic diagram of the structure of the matched image and the image to be matched
  • FIG. 3 is a schematic flowchart of a method for determining binocular stereo disparity according to an embodiment of the present invention
  • FIG. 4 is a schematic flowchart of a method for determining binocular stereo disparity according to another embodiment of the present invention.
  • FIG. 5 is a schematic structural diagram of a binocular stereo disparity determining device provided by an embodiment of the present invention.
  • FIG. 6 is a schematic structural diagram of a binocular stereo disparity determining device provided by another embodiment of the present invention.
  • Fig. 7 is a schematic diagram of the hardware structure of an unmanned aerial vehicle provided by an embodiment of the present invention.
  • the present invention provides a method and device for determining binocular stereo disparity.
  • the method and device are applied to the binocular camera device of a drone, so that the drone can accurately avoid obstacles through the binocular camera device during the autonomous obstacle avoidance process.
  • the drone can be any suitable type of high-altitude drone or low-altitude drone equipped with binocular cameras, including fixed-wing drones, rotary-wing drones, para-wing drones, or flapping-wing drones. Machine waiting.
  • FIG. 1 is an unmanned aerial vehicle 100 provided by one embodiment of the present invention.
  • the unmanned aerial vehicle 100 is a four-rotor unmanned aerial vehicle and includes a fuselage 10, an arm 20, a power unit 30, and a binocular camera.
  • the arm 20, the binocular camera device 40 and the landing gear 50 are all connected to the fuselage 10, the power unit 30 is arranged on the arm 20, and the smart battery and flight control system are arranged in the fuselage 10.
  • the power unit 30, the binocular camera device 40, and the landing gear 50 are all communicatively connected to the flight control system, so that the flight control system can control the flight of the UAV 100 through the power device 30, and the flight control system can also control the binocular camera.
  • the device 40 photographs the forward environment and controls the undercarriage 50 to open and retract.
  • the number of arms 20 is 4, evenly distributed around the fuselage 10, fixedly connected to the fuselage 10, and used to carry the power device 30.
  • the arm 20 and the body 10 are integrally formed.
  • the power unit 30 includes a motor and a propeller connected to the motor shaft.
  • the motor can drive the propeller to rotate to provide lift or thrust for the drone 100 to achieve flight; the motor can also change the flight direction of the drone 100 by changing the speed and direction of the propeller. .
  • the flight control system can control the flight of the UAV 100 by controlling the motor.
  • the power device 30 is arranged at an end of the arm 20 that is not connected to the fuselage 10, and is connected to the arm 20 through a motor.
  • a power device 30 is provided on the four arms of the unmanned aerial vehicle 100 to enable the unmanned aerial vehicle 100 to fly smoothly.
  • the binocular camera device 40 is a binocular camera, which is arranged at the bottom of the fuselage 10 and is used to photograph the front environment so as to obtain binocular images of the same scene at the same moment in different viewing angles.
  • the binocular image includes a left image and a right image, and there is a disparity between the matching points of the left image and the right image, and the binocular camera device 40 can obtain a disparity map by stereo matching the left image and the right image.
  • the binocular camera device 40 is also used to implement a method for determining the binocular stereo disparity to improve the accuracy of determining the target disparity, so that the binocular camera device 40 can obtain Accurate disparity map.
  • the binocular camera device 40 can also be installed on the bottom of the fuselage 10 through a pan/tilt to eliminate the disturbance caused to the binocular camera device 40 during the flight of the drone 100 and ensure the binocular camera device 40 The binocular images taken are clear and stable.
  • the landing gear 50 is arranged on opposite sides of the bottom of the fuselage 10 and is connected to the fuselage 10 through a driving device.
  • the landing gear 50 can be opened and retracted under the driving of the driving device.
  • the driving device controls the landing gear 50 to open so that the UAV 100 can contact the ground through the landing gear 50; during the flight of the UAV 100, the driving device controls the landing gear 50 to retract In order to avoid the landing gear 50 from affecting the flight of the UAV 100.
  • the landing gear 50 is in communication with the flight control system, the flight control system can control the opening and closing of the landing gear 50 by controlling the driving device.
  • the smart battery is used to power the drone 100 so that the power unit 30, the binocular camera device 40, the landing gear 50, and the flight control system of the drone 100 can be powered on and run.
  • smart batteries include, but are not limited to: dry batteries, lead storage batteries, and lithium batteries.
  • the flight control system communicates with the power unit 30, the binocular camera device 40, and the landing gear 50 through a wired connection or a wireless connection.
  • wireless connections include but are not limited to: WiFi, Bluetooth, ZigBee, etc.
  • the binocular camera device 40 executes the method for determining the binocular stereo disparity, which specifically includes:
  • any one of the obtained binocular images is determined as a to-be-matched image, and the other is determined as a matching image.
  • the binocular image includes a left image and a right image.
  • the left image is taken by the camera located on the left side of the binocular camera device 40
  • the right image is taken by the camera located on the right side of the binocular camera device 40
  • the left image is taken by the binocular camera device 40.
  • the size of the image and the right image are the same, both are M*N.
  • the left image may be determined as the image to be matched, and the right image may be determined as the matching image; or the left image may be determined as the matching image, and the right image may be determined as the image to be matched.
  • the left image is determined as the image to be matched, and the right image is determined as the matching image.
  • the image to be matched is composed of M*N pixels to be matched arranged in rows, and each pixel to be matched corresponds to a gray value.
  • the matching image is composed of M*N matching pixels arranged in rows and columns, and each matching pixel corresponds to a gray value.
  • the binocular camera device 40 After the binocular camera device 40 determines the image to be matched and the matched image, it establishes an image disparity space based on the image to be matched and the matched image.
  • the image disparity space includes the pixel disparity space of each pixel to be matched in the image to be matched, that is, how many pixels to be matched in the image to be matched, the image disparity space consists of how many pixel disparity spaces, and one pixel to be matched The point corresponds to a pixel disparity space.
  • the image disparity space includes M*N pixel disparity spaces.
  • the pixel disparity space of each pixel to be matched includes the cost aggregation value of the pixel to be matched under different disparity.
  • the pixel disparity space of each pixel to be matched is separately established.
  • the preset sequence includes but is not limited to: horizontal serpentine, longitudinal serpentine, horizontally parallel or longitudinally parallel, etc.
  • a matching pixel with the same position as the pixel to be matched is determined as a starting point in the matching image.
  • L is the image to be matched
  • R is the matched image.
  • the matching pixel point B0 located in the first row and first column determines the matching pixel point B0 located in the first row and first column as the starting point; similarly, if the pixel disparity space to be matched is established for the pixel to be matched A6, it is determined that the position of the pixel to be matched A6 is located in the second row and second column Therefore, in the matching image, the matching pixel point B6 located in the second row and second column is determined as the starting point.
  • the search is performed along the horizontal direction, and the matching pixel point corresponding to each disparity within the preset disparity search range is determined as the target point.
  • searching along the horizontal direction means searching along the line where the starting point is located.
  • the preset disparity search range is the range in which the pixel to be matched finds the corresponding point, that is, the range in which the corresponding point of the pixel to be matched may exist.
  • the preset disparity search range is determined by the maximum disparity between the corresponding point and the pixel to be matched and the minimum disparity between the corresponding point and the pixel to be matched, including the maximum disparity, the minimum disparity, and all disparity between the maximum disparity and the minimum disparity.
  • the disparity is a positive integer. For example, if the preset disparity search range D is [0,3], the disparity within the preset disparity search range D is 0, 1, 2, and 3, respectively.
  • the preset disparity search range is a preset empirical value.
  • the matching pixel point B0 is determined as the starting point, and the preset disparity search range is [0,3], the matching pixel point B0 is taken as the starting point, along the line where the matching pixel point B0 is located-the first Searching in one line, you can determine that the disparity between the matching pixel B0 and the pixel to be matched A0 is 0, the disparity between the matching pixel B1 and the pixel to be matched A0 is 1, and the disparity between the matching pixel B2 and the pixel to be matched A0 is 2.
  • the disparity between the matching pixel point B3 and the pixel point A0 to be matched is 3 and the disparity between the matching pixel point B4 and the pixel point A0 to be matched is 4.
  • the matching pixel point B0 corresponding to the disparity of 0 is determined as the target point 1
  • the matching pixel point B1 corresponding to the disparity of 1 is determined as the target point 2
  • the matching pixel point B2 corresponding to the disparity of 2 is determined as the target
  • the matching pixel point B3 corresponding to the disparity of 3 is determined as the target point 4.
  • the cost aggregation value of each target point is calculated, that is, the cost aggregation value of the target point is calculated separately. For example, calculating the cost aggregation value of the target point 1, calculating the cost aggregation value of the target point 2, calculating the cost aggregation value of the target point 3, and calculating the cost aggregation value of the target point 4.
  • the cost aggregation value of the target point can be calculated by the local matching algorithm.
  • Local matching algorithms include, but are not limited to: sum of absolute differences (SAD), truncated sum of absolute differences (STAD), and sum of squares of difference (SSD).
  • the cost aggregation value of the target point is calculated through the absolute difference value. For example, firstly, determine the pixel to be matched corresponding to the target point; secondly, determine the first window whose center is the pixel to be matched corresponding to the target point and the preset radius is the radius; Let the radius be the second window of the radius; then, calculate the grayscale difference of the pixels corresponding to the first window and the second window; finally, determine the sum of the absolute value of the grayscale difference as the cost aggregation value of the target point.
  • the second window P2 with a radius of 1, and at this time, the obtained second window P2
  • the size is 3*3, including matching pixels B0, B1, B5, and B6; when the first window P1 and the second window P2 overlap, the pixel to be matched A0 corresponds to the pixel to be matched B0, and the pixel to be matched A1 is
  • the pixel point B1 corresponds
  • the pixel point to be matched A5 corresponds to the matched pixel point B5
  • the pixel point to be matched A6 corresponds to the matched pixel point B6, so the grayscale difference between the pixel to be matched A0 and the matched pixel B0 is calculated, and the pixel to be matched is calculated
  • calculate the target point 2-the cost aggregation value S2 of the matching pixel point B1 calculate the target point 3-the cost aggregation value S3 of the matching pixel point B2, and calculate the target point 4-the cost aggregation value of the matching pixel point B3.
  • the cost aggregation value is S4.
  • the corresponding cost aggregation value is stored according to the disparity of the target point to form a pixel disparity space.
  • the disparity of the target point 1 is 0, so Store S1 as the cost aggregation value of the pixel to be matched A0 when the disparity is 0. Since the disparity of the target point 2 is 1, store S2 as the cost aggregation value of the pixel to be matched A0 when the disparity is 1. 3The disparity is 2, so S3 is stored as the cost aggregation value of the pixel to be matched A0 when the disparity is 3. Since the disparity of the target point 4 is 3, S4 is stored as the cost of the pixel to be matched A0 when the disparity is 3. The aggregated value forms the pixel disparity space of the pixel to be matched A0.
  • the pixel disparity spaces of the pixels to be matched A1 to A19 are respectively established by the above method, and the image disparity space is formed from the established pixel disparity spaces of the pixels to be matched A0 to A19.
  • the size of the image disparity space is related to the size of the image to be matched M*N and the number of disparity N1 in the preset disparity search range D, and the size of the image disparity space is M*N*N1.
  • the size of the image L to be matched is 4*5
  • the disparity in the preset disparity search range D is 0, 1, 2, 3, and the number is 4, so the resulting image disparity space size It is 4*5*4.
  • the binocular camera device 40 After the binocular camera device 40 establishes the image disparity space, according to the pixel disparity space of each pixel to be matched, the pixel to be matched that meets the preset matching condition is selected as the reference pixel.
  • each pixel to be matched is screened to determine whether the pixel to be matched meets the preset matching condition, and if the pixel to be matched meets the preset matching condition, Then the pixel to be matched is screened out as the reference pixel.
  • the pixel disparity space including the pixels to be matched A0 to A19 is included, and the pixel to be matched A0 is filtered according to the pixel disparity space of the pixel to be matched A0 , Determine whether the pixel to be matched A0 meets the preset matching condition, if the pixel to be matched A0 meets the preset matching condition, the pixel to be matched A0 is selected as the reference pixel, and then according to the pixel disparity of the pixel to be matched A1 Space, filter the pixel to be matched A1 to determine whether the pixel to be matched A1 meets the preset matching condition, if the pixel to be matched A1 meets the preset matching condition, filter the pixel to be matched A1 as the reference pixel; then, Then according to the pixel disparity space of the pixel to be matched A2, the pixel to be matched A2 is screened to determine
  • the preset matching condition is a preset condition for guiding the binocular camera device 40 to filter out pixels to be matched in a repetitive texture scene or a non-texture scene.
  • the benchmark pixels are filtered out.
  • the cost aggregation extreme value includes the smallest extreme value, the next smallest extreme value, the next smallest extreme value and the largest extreme value;
  • the cut-off value includes the first cut-off value, the second cut-off value and the third cut-off value.
  • the binocular camera device 40 finds the smallest extreme value, the next smallest extreme value, the next smallest extreme value, and the largest extreme value in the pixel parallax space of the pixel to be matched, the pixel is calculated according to the smallest extreme value and the largest extreme value.
  • the first boundary value of the disparity space, and the second boundary value of the pixel disparity space is calculated based on the first boundary value and the minimum extreme value, and then the third boundary of the pixel disparity space is calculated based on the second boundary value and the minimum extreme value value.
  • the first boundary value is one-half of the sum of the minimum extreme value and the maximum extreme value
  • the second boundary value is one-half of the sum of the first boundary value and the minimum extreme value
  • the third boundary value is the second One half of the sum of the cutoff value and the smallest extreme value.
  • the binocular camera device 40 calculates the first boundary value, the second boundary value and the third boundary value of the pixel parallax space, it is determined whether the second minimum extreme value is less than the third boundary value, if the second minimum extreme value is less than the third boundary value , Then filter out the pixels to be matched; if the second smallest extreme value is not less than the third cut-off value, judge whether the second smallest extreme value is less than half of the second cut-off value, if the second smallest extreme value is less than the second cut-off value If the value is one-half of the value, the pixel to be matched is filtered out; if the second smallest extreme value is not less than one-half of the second boundary value, the pixel to be matched is selected as the reference pixel.
  • the pixel to be matched A0 when the pixel to be matched A0 is filtered, it is determined that the pixel disparity space of the pixel to be matched A0 includes the cost aggregation value S1 when the disparity is 0, the cost aggregation value S2 when the disparity is 1, and the disparity is 2.
  • the imaging device 40 can find the smallest extreme value, the second smallest extreme value, the next smallest extreme value and the largest extreme value in the pixel disparity space of the pixel to be matched A0, and determine the smallest extreme value S1 and the second smallest extreme value S2, the next smallest extreme value is S3, and the largest extreme value is S4;
  • the third boundary value of the space X3, X3 (S1+X2)/2;
  • the pixel points A1 to A19 to be matched are sequentially filtered out twice to determine which of the pixels to be matched A1 to A19 can be used as reference pixels.
  • the binocular camera device 40 After screening the reference pixels, the binocular camera device 40 determines the disparity corresponding to the cost aggregation value satisfying the preset aggregation condition as the target disparity according to the cost aggregation value of each reference pixel under each disparity.
  • a target disparity is determined for each reference pixel, and the binocular camera device 40 can determine how many target disparity can be determined as many reference pixels are screened out, and each reference pixel corresponds to a target disparity.
  • the pixel to be matched A0 and the pixel to be matched A1 are reference pixels
  • first according to the cost aggregation value of the pixel to be matched A0 under each parallax, determine the corresponding cost aggregation value that meets the preset aggregation conditions
  • the disparity is used as the target disparity of the pixel to be matched A0
  • the cost aggregation value of the pixel to be matched A1 the disparity corresponding to the cost aggregation value that meets the preset aggregation conditions is determined as the target of the pixel to be matched A1 Parallax.
  • the disparity corresponding to the cost aggregation value meeting the preset aggregation condition is determined as the target disparity, which specifically includes: the cost aggregation value corresponding to each reference pixel After finding the minimum cost aggregation value, the disparity corresponding to the minimum cost aggregation value is selected as the target disparity.
  • the minimum cost aggregation value in the cost aggregation value corresponding to the pixel to be matched A0 select the disparity corresponding to the minimum cost aggregation value as the target disparity of the pixel to be matched A0, and then correspond to the pixel to be matched A1
  • find the minimum cost aggregation value find the minimum cost aggregation value, and select the disparity corresponding to the minimum cost aggregation value as the target disparity of the pixel to be matched A1.
  • the cost aggregation value corresponding to the pixel to be matched A0 includes the cost aggregation value S1 when the disparity is 0, the cost aggregation value S2 when the disparity is 1, the cost aggregation value S3 when the disparity is 2, and the cost when the disparity is 3 If the aggregation value is S4 and S1 is the smallest, S1 can be found as the minimum cost aggregation value in the cost aggregation value corresponding to the pixel to be matched A0. At this time, the disparity 0 corresponding to S1 is selected as the target disparity of the pixel to be matched A0.
  • the target disparity of the pixel to be matched A0 is 0; it is determined that the cost aggregation value corresponding to the pixel to be matched A1 includes the cost aggregation value S5 when the disparity is 0, the cost aggregation value S6 when the disparity is 1, and the cost when the disparity is 2.
  • S7 and the cost aggregation value S8 when the disparity is 3, and S7 is the smallest S7 can be found as the minimum cost aggregation value in the cost aggregation value corresponding to the pixel to be matched A1.
  • the disparity 2 corresponding to S7 is selected.
  • the target disparity of the pixel A1 to be matched that is, the target disparity of the pixel A1 to be matched is 2.
  • the target disparity is also corrected.
  • the parabolic interpolation method can be used to correct the target disparity, including: determining two reference disparity adjacent to the target disparity; extracting the cost aggregation value of each reference disparity; corresponding to the two reference disparity and the two reference disparity
  • the cost aggregation value of is constructed according to the parabolic interpolation method; the final disparity is determined according to the interpolation function and the target disparity, and the target disparity is corrected to the final disparity.
  • the interpolation function is:
  • d(x,y) is the target disparity
  • S d-1 (x,y) is the cost aggregation value of the previous disparity adjacent to the target disparity
  • S d+1 (x,y) is the adjacent target disparity The aggregate value of the cost of the latter disparity.
  • disparity 1 and disparity 3 are determined as reference disparity; because the disparity is 1, the cost aggregation value is S6, and the disparity is The cost aggregation value at time 3 is S8, so the cost aggregation value S6 of disparity 1 and the cost aggregation value S8 of disparity 3 are extracted; according to disparity 1 and its corresponding cost aggregation value S6, disparity 3 and its corresponding cost aggregation value S8,
  • the interpolation function is constructed according to the parabolic interpolation method, and the final disparity is determined according to the interpolation function and the target disparity 2.
  • the binocular camera device screens each pixel to be matched in the image to be matched through the pixel disparity space of each pixel to be matched, and screens out the pixel to be matched that meets the preset matching condition as
  • the target disparity is determined by the reference pixels, which can filter out the pixels to be matched that may be located in the repeated texture or unmanned scenes, prevent the target disparity determination error, improve the accuracy of determining the target disparity, and then make the loading
  • the drone of the binocular camera device can obtain accurate depth maps, reducing false detection problems in repeated texture scenes or non-textured scenes.
  • FIG. 3 is a schematic flow chart of a method for determining binocular stereo disparity according to one embodiment of the present invention, which is applied to a drone, and the drone is the drone 100 described in the foregoing embodiment, and
  • the method provided by the embodiment of the present invention is executed by the above-mentioned binocular camera device 40, and is used to improve the accuracy of determining the target parallax, thereby reducing the problem of false detection of drones in repeated texture or non-texture scenes.
  • the binocular stereo disparity is determined Methods include:
  • S100 Establish an image disparity space according to the image to be matched and the matched image.
  • the to-be-matched image and the matched image are binocular images of the same scene at the same time captured by the binocular camera device under different viewing angles, so the to-be-matched image and the matched image have the same size and are both M*N.
  • the image to be matched is composed of M*N pixels to be matched arranged in rows, and each pixel to be matched corresponds to a gray value.
  • the matching image is composed of M*N matching pixels arranged in rows and columns, and each matching pixel corresponds to a gray value.
  • the image disparity space contains the pixel disparity space of each pixel to be matched in the image to be matched, that is, how many pixels to be matched in the image to be matched, the image disparity space is composed of how many pixel disparity spaces, and one pixel to be matched Corresponds to a pixel parallax space. For example, when there are M*N pixels to be matched in the image to be matched, the image disparity space includes M*N pixel disparity spaces.
  • the pixel disparity space of each pixel to be matched includes the cost aggregation value of the pixel to be matched under different disparity.
  • the pixel disparity space of each pixel to be matched is separately established.
  • the preset sequence includes but is not limited to: horizontal serpentine, longitudinal serpentine, horizontally parallel or longitudinally parallel, etc.
  • a matching pixel that is the same as the pixel to be matched is determined as a starting point in the matching image.
  • L is the image to be matched
  • R is the matched image.
  • the matching pixel point B0 located in the first row and first column determines the matching pixel point B0 located in the first row and first column as the starting point; similarly, if the pixel disparity space to be matched is established for the pixel to be matched A6, it is determined that the position of the pixel to be matched A6 is located in the second row and second column Therefore, in the matching image, the matching pixel point B6 located in the second row and second column is determined as the starting point.
  • the search is performed along the horizontal direction, and the matching pixel point corresponding to each disparity within the preset disparity search range is determined as the target point.
  • searching along the horizontal direction means searching along the line where the starting point is located.
  • the preset disparity search range is the range in which the pixel to be matched finds the corresponding point, that is, the range in which the corresponding point of the pixel to be matched may exist.
  • the preset disparity search range is determined by the maximum disparity between the corresponding point and the pixel to be matched and the minimum disparity between the corresponding point and the pixel to be matched, including the maximum disparity, the minimum disparity, and all disparity between the maximum disparity and the minimum disparity.
  • the disparity is a positive integer. For example, if the preset disparity search range D is [0,3], the disparity within the preset disparity search range D is 0, 1, 2, and 3, respectively.
  • the preset disparity search range is a preset empirical value.
  • the matching pixel point B0 is determined as the starting point, and the preset disparity search range is [0,3], the matching pixel point B0 is taken as the starting point, along the line where the matching pixel point B0 is located-the first Searching in one line, you can determine that the disparity between the matching pixel B0 and the pixel to be matched A0 is 0, the disparity between the matching pixel B1 and the pixel to be matched A0 is 1, and the disparity between the matching pixel B2 and the pixel to be matched A0 is 2.
  • the disparity between the matching pixel point B3 and the pixel point A0 to be matched is 3 and the disparity between the matching pixel point B4 and the pixel point A0 to be matched is 4.
  • the matching pixel point B0 corresponding to the disparity of 0 is determined as the target point 1
  • the matching pixel point B1 corresponding to the disparity of 1 is determined as the target point 2
  • the matching pixel point B2 corresponding to the disparity of 2 is determined as the target
  • the matching pixel point B3 corresponding to the disparity of 3 is determined as the target point 4.
  • the cost aggregation value of each target point is calculated, that is, the cost aggregation value of the target point is calculated separately. For example, calculating the cost aggregation value of the target point 1, calculating the cost aggregation value of the target point 2, calculating the cost aggregation value of the target point 3, and calculating the cost aggregation value of the target point 4.
  • the cost aggregation value of the target point can be calculated by the local matching algorithm.
  • Local matching algorithms include, but are not limited to: sum of absolute differences (SAD), truncated sum of absolute differences (STAD), and sum of squares of difference (SSD).
  • the cost aggregation value of the target point is calculated through the absolute difference value. For example, firstly, determine the pixel to be matched corresponding to the target point; secondly, determine the first window whose center is the pixel to be matched corresponding to the target point and the preset radius is the radius; Let the radius be the second window of the radius; then, calculate the grayscale difference of the pixels corresponding to the first window and the second window; finally, determine the sum of the absolute value of the grayscale difference as the cost aggregation value of the target point.
  • the second window P2 with a radius of 1, and at this time, the obtained second window P2
  • the size is 3*3, including matching pixels B0, B1, B5, and B6; when the first window P1 and the second window P2 overlap, the pixel to be matched A0 corresponds to the pixel to be matched B0, and the pixel to be matched A1 is
  • the pixel point B1 corresponds
  • the pixel point to be matched A5 corresponds to the matched pixel point B5
  • the pixel point to be matched A6 corresponds to the matched pixel point B6, so the grayscale difference between the pixel to be matched A0 and the matched pixel B0 is calculated, and the pixel to be matched is calculated
  • calculate the target point 2-the cost aggregation value S2 of the matching pixel point B1 calculate the target point 3-the cost aggregation value S3 of the matching pixel point B2, and calculate the target point 4-the cost aggregation value of the matching pixel point B3.
  • the cost aggregation value is S4.
  • the corresponding cost aggregation value is stored according to the disparity of the target point to form a pixel disparity space.
  • the disparity of the target point 1 is 0, so Store S1 as the cost aggregation value of the pixel to be matched A0 when the disparity is 0. Since the disparity of the target point 2 is 1, store S2 as the cost aggregation value of the pixel to be matched A0 when the disparity is 1. 3The disparity is 2, so S3 is stored as the cost aggregation value of the pixel to be matched A0 when the disparity is 3. Since the disparity of the target point 4 is 3, S4 is stored as the cost of the pixel to be matched A0 when the disparity is 3. The aggregated value forms the pixel disparity space of the pixel to be matched A0.
  • the pixel disparity spaces of the pixel points A1 to A19 to be matched are respectively established by the above method, and the image disparity space is formed by the established pixel disparity spaces of the pixel points A0 to A19 to be matched.
  • the size of the image disparity space is related to the size of the image to be matched M*N and the number of disparity N1 in the preset disparity search range D, and the size of the image disparity space is M*N*N1.
  • the size of the image L to be matched is 4*5
  • the disparity in the preset disparity search range D is 0, 1, 2, 3, and the number is 4, so the resulting image disparity space size It is 4*5*4.
  • the preset matching condition is a preset condition for guiding the binocular camera device to filter out pixels to be matched in a repeated texture scene or a non-textured scene.
  • the pixel to be matched that meets the preset matching condition is selected as the reference pixel, that is, according to the pixel disparity space of each pixel to be matched in the image disparity space,
  • Each pixel to be matched is screened to determine whether the pixel to be matched meets the preset matching condition, and if the pixel to be matched meets the preset matching condition, the pixel to be matched is screened out as the reference pixel.
  • the pixel disparity space including the pixels to be matched A0 to A19 is included, and the pixel to be matched A0 is filtered according to the pixel disparity space of the pixel to be matched A0 , Determine whether the pixel to be matched A0 meets the preset matching condition, if the pixel to be matched A0 meets the preset matching condition, the pixel to be matched A0 is selected as the reference pixel, and then according to the pixel disparity of the pixel to be matched A1 Space, filter the pixel to be matched A1 to determine whether the pixel to be matched A1 meets the preset matching condition, if the pixel to be matched A1 meets the preset matching condition, filter the pixel to be matched A1 as the reference pixel; then, Then according to the pixel disparity space of the pixel to be matched A2, the pixel to be matched A2 is screened to determine
  • the pixels to be matched are filtered, firstly, in the pixel disparity space of the pixels to be matched, the cost aggregation extreme value is found; then, the pixel disparity space is calculated according to the cost aggregation extreme value Finally, according to the cut-off value and cost, the extreme values are aggregated to filter out the reference pixels.
  • the cost aggregation extreme value includes the smallest extreme value, the next smallest extreme value, the next smallest extreme value and the largest extreme value;
  • the cut-off value includes the first cut-off value, the second cut-off value and the third cut-off value.
  • the first of the pixel disparity space is calculated according to the smallest extremum and the largest extremum.
  • the second boundary value of the pixel disparity space is calculated according to the first boundary value and the minimum extreme value
  • the third boundary value of the pixel disparity space is calculated according to the second boundary value and the minimum extreme value.
  • the first boundary value is one-half of the sum of the minimum extreme value and the maximum extreme value
  • the second boundary value is one-half of the sum of the first boundary value and the minimum extreme value
  • the third boundary value is the second One half of the sum of the cutoff value and the smallest extreme value.
  • the second smallest extreme value is less than the third threshold. If the second smallest extreme value is less than the third threshold, filter out Matching pixels; if the second smallest extreme value is not less than the third cutoff value, judge whether the second smallest extreme value is less than half of the second cutoff value, if the second smallest extreme value is less than half of the second cutoff value 1. Filter out the pixel to be matched; if the second smallest extreme value is not less than half of the second boundary value, the pixel to be matched is selected as the reference pixel.
  • the pixel to be matched A0 when the pixel to be matched A0 is filtered, it is determined that the pixel disparity space of the pixel to be matched A0 includes the cost aggregation value S1 when the disparity is 0, the cost aggregation value S2 when the disparity is 1, and the disparity is 2.
  • the third boundary value of the space X3, X3 (S1+X2)/2;
  • the pixel points A1 to A19 to be matched are sequentially filtered out twice to determine which of the pixels to be matched A1 to A19 can be used as reference pixels.
  • a target parallax is determined for each reference pixel, and the binocular camera device filters out how many reference pixels can determine how many target parallaxes, and each reference pixel corresponds to a target parallax.
  • the pixel to be matched A0 and the pixel to be matched A1 are reference pixels
  • first according to the cost aggregation value of the pixel to be matched A0 under each parallax, determine the corresponding cost aggregation value that meets the preset aggregation conditions
  • the disparity is used as the target disparity of the pixel to be matched A0
  • the cost aggregation value of the pixel to be matched A1 the disparity corresponding to the cost aggregation value that meets the preset aggregation conditions is determined as the target of the pixel to be matched A1 Parallax.
  • determining the disparity corresponding to the cost aggregation value that meets the preset aggregation condition as the target disparity which specifically includes: Corresponding to each reference pixel In the cost aggregation value of, after finding the minimum cost aggregation value, the disparity corresponding to the minimum cost aggregation value is selected as the target disparity.
  • the minimum cost aggregation value in the cost aggregation value corresponding to the pixel to be matched A0 select the disparity corresponding to the minimum cost aggregation value as the target disparity of the pixel to be matched A0, and then correspond to the pixel to be matched A1
  • find the minimum cost aggregation value find the minimum cost aggregation value, and select the disparity corresponding to the minimum cost aggregation value as the target disparity of the pixel to be matched A1.
  • the cost aggregation value corresponding to the pixel to be matched A0 includes the cost aggregation value S1 when the disparity is 0, the cost aggregation value S2 when the disparity is 1, the cost aggregation value S3 when the disparity is 2, and the cost when the disparity is 3 If the aggregation value is S4 and S1 is the smallest, S1 can be found as the minimum cost aggregation value in the cost aggregation value corresponding to the pixel to be matched A0. At this time, the disparity 0 corresponding to S1 is selected as the target disparity of the pixel to be matched A0.
  • the target disparity of the pixel to be matched A0 is 0; it is determined that the cost aggregation value corresponding to the pixel to be matched A1 includes the cost aggregation value S5 when the disparity is 0, the cost aggregation value S6 when the disparity is 1, and the cost when the disparity is 2.
  • S7 and the cost aggregation value S8 when the disparity is 3, and S7 is the smallest S7 can be found as the minimum cost aggregation value in the cost aggregation value corresponding to the pixel to be matched A1.
  • the disparity 2 corresponding to S7 is selected.
  • the target disparity of the pixel A1 to be matched that is, the target disparity of the pixel A1 to be matched is 2.
  • the method further includes:
  • the target disparity can be corrected by the parabolic interpolation method, including: determining two reference disparity adjacent to the target disparity; extracting the cost aggregation value of each reference disparity; according to the two reference disparity and the corresponding cost of the two reference disparity Aggregate values, construct an interpolation function according to the parabolic interpolation method; determine the final disparity according to the interpolation function and the target disparity, and correct the target disparity to the final disparity.
  • the interpolation function is:
  • d(x,y) is the target disparity
  • S d-1 (x,y) is the cost aggregation value of the previous disparity adjacent to the target disparity
  • S d+1 (x,y) is the adjacent target disparity The aggregate value of the cost of the latter disparity.
  • disparity 1 and disparity 3 are determined as reference disparity; because the disparity is 1, the cost aggregation value is S6, and the disparity is The cost aggregation value at time 3 is S8, so the cost aggregation value S6 of disparity 1 and the cost aggregation value S8 of disparity 3 are extracted; according to disparity 1 and its corresponding cost aggregation value S6, disparity 3 and its corresponding cost aggregation value S8,
  • the interpolation function is constructed according to the parabolic interpolation method, and the final disparity is determined according to the interpolation function and the target disparity 2.
  • each pixel to be matched in the image to be matched is filtered, and the pixel to be matched that meets the preset matching condition is selected as the reference pixel.
  • the target disparity is determined by the reference pixel, which can filter out the pixels to be matched that may be located in the repeated texture or unmanned scene, prevent the target disparity from being determined wrongly, and improve the accuracy of determining the target disparity, so that the drone can obtain Accurate depth map reduces false detection problems in repeated texture scenes or untextured scenes.
  • module is a combination of software and/or hardware that can implement predetermined functions.
  • devices described in the following embodiments can be implemented by software, implementation by hardware or a combination of software and hardware may also be conceived.
  • FIG. 5 is a binocular stereo disparity determining device provided by one embodiment of the present invention.
  • the device is applied to a drone.
  • the drone is the drone 100 described in the above-mentioned embodiment.
  • the function of each module of the device provided by the embodiment of the invention is executed by the above-mentioned binocular camera device 40, which is used to improve the accuracy of determining the target parallax, thereby reducing the problem of false detection of drones in repeated texture or non-texture scenes.
  • the stereo disparity determining device includes:
  • the establishment module 200 is configured to establish an image disparity space according to the image to be matched and the matching image, the image disparity space includes the pixel disparity space of each pixel to be matched in the image to be matched, wherein each of the to be matched The pixel disparity space of the pixels all includes the cost aggregation value of the pixel to be matched under different disparity;
  • the screening module 300 is configured to screen out the pixel to be matched that meets the preset matching condition as the reference pixel according to the pixel disparity space of each pixel to be matched;
  • the determining module 400 is configured to determine the disparity corresponding to the cost aggregation value that satisfies the preset aggregation condition according to the cost aggregation value of each reference pixel under each disparity among the reference pixels. As the target parallax.
  • the screening module 300 is specifically configured to:
  • the reference pixels are filtered out.
  • the cost aggregation extreme value includes the smallest extreme value, the second smallest extreme value, the next smallest extreme value, and the largest extreme value;
  • the cutoff value includes a first cutoff value, a second cutoff value, and a third cutoff value. Value
  • the screening module 300 is specifically used for:
  • the third boundary value is obtained.
  • the first cutoff value is one half of the sum of the minimum extreme value and the maximum extreme value
  • the second demarcation value is one half of the sum of the first demarcation value and the minimum extreme value
  • the third boundary value is one half of the sum of the second boundary value and the minimum extreme value.
  • the screening module 300 is specifically configured to:
  • the pixel to be matched is selected as the reference pixel.
  • the determining module 400 is specifically configured to:
  • the disparity corresponding to the minimum cost aggregation value is selected as the target disparity.
  • the binocular stereo disparity determination device further includes:
  • the correction module 500 is used to correct the target disparity.
  • the correction module 500 is specifically configured to:
  • the target parallax is corrected by parabolic interpolation.
  • the establishment module 200 is further used for:
  • the SAD algorithm is used to calculate the cost aggregation value of each pixel to be matched in the image to be matched under different parallaxes.
  • the above-mentioned establishment module 200, screening module 300, determination module 400, and correction module 500 may be the image processing chip of the binocular camera 400.
  • the content of the device embodiment can be quoted from the method embodiment on the premise that the content does not conflict with each other, which will not be repeated here.
  • each pixel to be matched in the image to be matched is filtered, and the pixel to be matched that meets the preset matching condition is selected as the reference pixel.
  • the target disparity is determined by the reference pixel, which can filter out the pixels to be matched that may be located in the repeated texture or unmanned scene, prevent the target disparity from being determined wrongly, and improve the accuracy of determining the target disparity, so that the drone can obtain Accurate depth map reduces false detection problems in repeated texture scenes or untextured scenes.
  • FIG. 7 is a schematic diagram of the hardware structure of an unmanned aerial vehicle provided by one of the embodiments of the present invention.
  • the hardware modules provided by the embodiments of the present invention can be integrated into the binocular camera device 40 described in the above embodiments, so that the dual
  • the eye camera device 40 can execute the binocular stereo disparity determination method described in the above embodiment, and can also implement the functions of each module of the binocular stereo disparity determination device described in the above embodiment.
  • the drone 100 includes:
  • processors 110 and memory 120. Among them, one processor 110 is taken as an example in FIG. 7.
  • the processor 110 and the memory 120 may be connected through a bus or in other ways.
  • the connection through a bus is taken as an example.
  • the memory 120 can be used to store non-volatile software programs, non-volatile computer-executable programs and modules, such as a binocular stereoscopic parallax in the above-mentioned embodiment of the present invention.
  • the program instructions corresponding to the determination method and the modules corresponding to a binocular stereo disparity determination device for example, the establishment module 200, the screening module 300, the determination module 400, etc.
  • the processor 110 executes various functional applications and data processing of a method for determining binocular stereo disparity by running non-volatile software programs, instructions, and modules stored in the memory 120, that is, implements one of the above method embodiments.
  • the memory 120 may include a storage program area and a storage data area.
  • the storage program area may store an operating system and an application program required by at least one function;
  • the storage data area may store a binocular stereo disparity determination device created by using a binocular stereo disparity determination device. Data etc.
  • the storage data area also stores preset data, including a preset order, a preset radius, a preset disparity search range, a preset matching condition, a preset aggregation condition, and the like.
  • the memory 120 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other non-volatile solid-state storage devices.
  • the memory 120 may optionally include memories remotely provided with respect to the processor 110, and these remote memories may be connected to the processor 110 through a network. Examples of the aforementioned networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.
  • the program instructions and one or more modules are stored in the memory 120, and when executed by the one or more processors 110, each of the methods for determining binocular stereo disparity in any of the foregoing method embodiments is executed. Steps, or, realize the functions of each module of a binocular stereo disparity determining device in any of the foregoing device embodiments.
  • the above-mentioned product can execute the method provided in the above-mentioned embodiment of the present invention, and has corresponding functional modules and beneficial effects for the execution method.
  • the method provided in the foregoing embodiment of the present invention please refer to the method provided in the foregoing embodiment of the present invention.
  • the embodiment of the present invention also provides a non-volatile computer-readable storage medium, the computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are executed by one or more processors, for example, FIG. 7
  • a processor 110 in any of the foregoing method embodiments may enable a computer to execute each step of a binocular stereo disparity determination method in any of the foregoing method embodiments, or implement each of the binocular stereo disparity determining apparatus in any of the foregoing device embodiments.
  • the function of the module may be any of the foregoing method embodiments.
  • the embodiment of the present invention also provides a computer program product, the computer program product includes a computer program stored on a non-volatile computer-readable storage medium, the computer program includes program instructions, when the program instructions are Or multiple processors, such as a processor 110 in FIG. 7, can make a computer execute each step of a method for determining binocular stereo disparity in any of the foregoing method embodiments, or implement the steps in any of the foregoing device embodiments A function of each module of a binocular stereo disparity determination device.
  • the device embodiments described above are merely illustrative.
  • the modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical units, that is, they may be located in One place, or it can be distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the objectives of the solutions of the embodiments.
  • each embodiment can be implemented by software plus a general hardware platform, and of course, it can also be implemented by hardware.
  • a person of ordinary skill in the art can understand that all or part of the processes in the methods of the foregoing embodiments can be implemented by computer programs instructing relevant hardware.
  • the programs can be stored in a computer-readable storage medium, and the program can be executed during execution. At the time, it may include the flow of the implementation method of each method as described above.
  • the storage medium may be a magnetic disk, an optical disc, a read-only memory (Read-Only Memory, ROM), or a random access memory (Random Access Memory, RAM), etc.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Length Measuring Devices By Optical Means (AREA)
  • Image Processing (AREA)
  • Measurement Of Optical Distance (AREA)

Abstract

一种双目立体视差确定方法、装置及无人机, 涉及立体匹配技术领域。该方法应用于无人机的双目摄像装置，包括：根据待匹配图像与匹配图像，建立图像视差空间（S100），图像视差空间包含待匹配图像中每个待匹配像素点的像素视差空间，其中，每一个待匹配像素点的像素视差空间均包含待匹配像素点在不同视差下的代价聚合值；分别根据每一个待匹配像素点的像素视差空间，筛选出满足预设匹配条件的待匹配像素点作为基准像素点（S200）；在基准像素点中，分别根据每一个基准像素点在每个视差下的代价聚合值，确定满足预设聚合条件的代价聚合值对应的视差作为目标视差（S300）。通过上述方式，能够提高确定目标视差的准确性。

Description

一种双目立体视差确定方法、装置及无人机
本申请要求于2019年9月4日提交中国专利局、申请号为201910833064.2、申请名称为“一种双目立体视差确定方法、装置及无人机”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及立体匹配技术领域，特别是涉及一种双目立体视差确定方法、装置及无人机。
背景技术
无人机是一种由无线电遥控设备或自身程序控制装置操纵的无人驾驶飞行器。在无人机自主避障的过程中，一般通过双目摄像装置获取无人机前方环境的深度图来确定无人机前方环境存在的障碍物情况，进而根据所确定的障碍物情况实现避障。其中，双目摄像装置通过利用所拍摄的双目图像进行立体匹配来确定无人机前方环境的深度图。
目前，双目摄像装置对双目图像进行立体匹配时，主要确定待匹配图像上所有待匹配像素点在匹配图像中的对应点，进而根据所有待匹配像素点及其对应点之间的视差确定深度图，该种方式对待匹配图像上所有的待匹配像素点均进行对应点的查找，若待匹配像素点位于重复纹理场景或者无纹理场景时，则该待匹配像素点存在多个对应点，使得对应点的查找容易出错，导致待匹配像素点与对应点之间的视差确定错误，进而导致深度图错误，使得无人机判断的障碍物情况不准确，出现误检测的问题。
发明内容
本发明实施例旨在提供一种双目立体视差确定方法、装置及无人机，能够提高确定目标视差的准确性。
为解决上述技术问题，本发明实施例采用的一个技术方案是：提供一种双目立体视差确定方法，应用于无人机的双目摄像装置，所述方法包括：
根据待匹配图像与匹配图像，建立图像视差空间，所述图像视差空间包含所述待匹配图像中每个待匹配像素点的像素视差空间，其中，每一个所述待匹配像素点的像素视差空间均包含所述待匹配像素点在不同视差下的代价聚合值；
分别根据每一个所述待匹配像素点的像素视差空间，筛选出满足预设匹配条件的待匹配像素点作为基准像素点；
在所述基准像素点中，分别根据每一个所述基准像素点在每个所述视差下 的代价聚合值，确定满足预设聚合条件的代价聚合值对应的所述视差作为目标视差。
可选地，所述分别根据每一个所述待匹配像素点的像素视差空间，筛选出满足预设匹配条件的待匹配像素点作为基准像素点，包括：
在每一个所述待匹配像素点的像素视差空间中，查找出代价聚合极值；
根据所述代价聚合极值，计算出所述像素视差空间的分界值；
根据所述分界值及所述代价聚合极值，筛选出基准像素点。
可选地，所述代价聚合极值包括最小极值、次最小极值、次次最小极值以及最大极值；所述分界值包括第一分界值、第二分界值以及第三分界值；则，
所述根据所述代价聚合极值，计算出所述像素视差空间的分界值，包括：
根据所述最小极值和所述最大极值，得到所述第一分界值；
根据所述第一分界值和所述最小极值，得到所述第二分界值；
根据所述第二分界值和所述最小极值，得到所述第三分界值。
可选地，所述第一分界值为所述最小极值和所述最大极值之和的二分之一；
所述第二分界值为所述第一分界值和所述最小极值之和的二分之一；
所述第三分界值为所述第二分界值和所述最小极值之和的二分之一。
可选地，所述根据所述分界值及所述代价聚合极值，筛选出基准像素点，包括：
判断所述次最小极值是否小于所述第三分界值；
若所述次最小极值小于所述第三分界值，则滤除所述待匹配像素点；
若所述次最小极值不小于所述第三分界值，则判断所述次次最小极值是否小于所述第二分界值的二分之一；
若所述次次最小极值小于所述第二分界值的二分之一，则滤除所述待匹配像素点；
若所述次次最小极值不小于所述第二分界值的二分之一，则选择所述待匹配像素点作为基准像素点。
可选地，所述分别根据每一个所述基准像素点在每个所述视差下的代价聚合值，确定满足预设聚合条件的代价聚合值对应的所述视差作为目标视差，包括：
在每一个所述基准像素点对应的代价聚合值中，查找出最小代价聚合值；
选择所述最小代价聚合值对应的所述视差作为目标视差。
可选地，所述方法还包括：
校正所述目标视差。
可选地，所述校正所述目标视差，包括：
通过抛物线插值法校正所述目标视差。
可选地，所述方法还包括：
通过SAD算法计算所述待匹配图像中每个待匹配像素点在不同视差下的 代价聚合值。
为解决上述技术问题，本发明实施例采用的另一个技术方案是：提供一种双目立体视差确定装置，应用于无人机的双目摄像装置，所述装置包括：
建立模块，用于根据待匹配图像与匹配图像，建立图像视差空间，所述图像视差空间包含所述待匹配图像中每个待匹配像素点的像素视差空间，其中，每一个所述待匹配像素点的像素视差空间均包含所述待匹配像素点在不同视差下的代价聚合值；
筛选模块，用于分别根据每一个所述待匹配像素点的像素视差空间，筛选出满足预设匹配条件的待匹配像素点作为基准像素点；
确定模块，用于在所述基准像素点中，分别根据每一个所述基准像素点在每个所述视差下的代价聚合值，确定满足预设聚合条件的代价聚合值对应的所述视差作为目标视差。
可选地，所述筛选模块具体用于：
在每一个所述待匹配像素点的像素视差空间中，查找出代价聚合极值；
根据所述代价聚合极值，计算出所述像素视差空间的分界值；
根据所述分界值及所述代价聚合极值，筛选出基准像素点。
可选地，所述代价聚合极值包括最小极值、次最小极值、次次最小极值以及最大极值；所述分界值包括第一分界值、第二分界值以及第三分界值；则，
所述筛选模块具体用于：
根据所述最小极值和所述最大极值，得到所述第一分界值；
根据所述第一分界值和所述最小极值，得到所述第二分界值；
根据所述第二分界值和所述最小极值，得到所述第三分界值。
可选地，所述第一分界值为所述最小极值和所述最大极值之和的二分之一；
所述第二分界值为所述第一分界值和所述最小极值之和的二分之一；
所述第三分界值为所述第二分界值和所述最小极值之和的二分之一。
可选地，所述筛选模块具体用于：
判断所述次最小极值是否小于所述第三分界值；
若所述次最小极值小于所述第三分界值，则滤除所述待匹配像素点；
若所述次最小极值不小于所述第三分界值，则判断所述次次最小极值是否小于所述第二分界值的二分之一；
若所述次次最小极值小于所述第二分界值的二分之一，则滤除所述待匹配像素点；
若所述次次最小极值不小于所述第二分界值的二分之一，则选择所述待匹配像素点作为基准像素点。
可选地，所述确定模块具体用于：
在每一个所述基准像素点对应的代价聚合值中，查找出最小代价聚合值；
选择所述最小代价聚合值对应的所述视差作为目标视差。
可选地，所述装置还包括：
校正模块，用于校正所述目标视差。
可选地，所述校正模块具体用于：
通过抛物线插值法校正所述目标视差。
可选地，所述建立模块还用于：
通过SAD算法计算所述待匹配图像中每个待匹配像素点在不同视差下的代价聚合值。
为解决上述技术问题，本发明实施例采用的另一个技术方案是：提供一种无人机，包括：
机身；
机臂，与所述机身相连；
动力装置，设于所述机臂；以及
双目摄像装置，与所述机身相连；
其中，所述双目摄像装置包括：
至少一个处理器；以及
与所述至少一个处理器通信连接的存储器；其中，所述存储器存储有可被所述至少一个处理器执行的指令，所述指令被所述至少一个处理器执行，以使所述至少一个处理器能够用于执行以上所述的双目立体视差确定方法。
为解决上述技术问题，本发明实施例采用的另一个技术方案是：提供一种非易失性计算机可读存储介质，其特征在于，所述非易失性计算机可读存储介质存储有计算机可执行指令，所述计算机可执行指令用于使无人机的双目摄像装置执行以上所述的双目立体视差确定方法。
本发明实施例的有益效果是：区别于现有技术的情况下，本发明实施例提供一种双目立体视差确定方法、装置及无人机，在双目立体视差确定方法中，通过每一个待匹配像素点的像素视差空间，对待匹配图像中的每个待匹配像素点进行筛选，筛选出满足预设匹配条件的待匹配像素点作为基准像素点后，才通过基准像素点确定目标视差，其中，不满足预设匹配条件的待匹配像素点可能位于重复纹理场景或者无纹理场景中，其所确定的目标视差容易出错，故筛选出满足预设匹配条件的待匹配像素点来确定目标视差，能够提高确定目标视差的准确性，进而减少在重复纹理场景或无纹理场景中的误检测问题。
附图说明
一个或多个实施例通过与之对应的附图中的图片进行示例性说明，这些示例性说明并不构成对实施例的限定，附图中具有相同参考数字标号的元件表示为类似的元件，除非有特别申明，附图中的图不构成比例限制。
图1是本发明一实施例提供的一种无人机的结构示意图；
图2是匹配图像与待匹配图像的结构示意图；
图3是本发明一实施例提供的一种双目立体视差确定方法的流程示意图；
图4是本发明另一实施例提供的一种双目立体视差确定方法的流程示意图；
图5是本发明一实施例提供的一种双目立体视差确定装置的结构示意图；
图6是本发明另一实施例提供的一种双目立体视差确定装置的结构示意图；
图7是本发明一实施例提供的一种无人机的硬件结构示意图。
具体实施方式
为使本发明实施例的目的、技术方案和优点更加清楚，下面将结合本发明实施例中的附图，对本发明实施例中的技术方案进行清楚、完整的描述，显然，所描述的实施例是本发明一部分实施例，而不是全部的实施例。应当理解，此处所描述的具体实施例仅用以解释本发明，并不用于限定本发明。基于本发明中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本发明保护的范围。
需要说明的是，当元件被表述“固定于”另一个元件，它可以直接在另一个元件上、或者其间可以存在一个或多个居中的元件。当一个元件被表述“连接”另一个元件，它可以是直接连接到另一个元件、或者其间可以存在一个或多个居中的元件。本说明书所使用的术语“垂直的”、“水平的”、“左”、“右”以及类似的表述只是为了说明的目的。
此外，下面所描述的本发明各个实施例中所涉及到的技术特征只要彼此之间未构成冲突就可以相互组合。
本发明提供了一种双目立体视差确定方法及装置，该方法及装置应用于无人机的双目摄像装置，从而使得该无人机能够在自主避障过程中，通过双目摄像装置准确地确定目标视差，并根据目标视差获取准确的深度图后，能够根据所获取的深度图准确地确定前方环境存在的障碍物情况，减少误检测问题。其中，无人机可以是任何合适类型的搭载有双目摄像装置的高空无人机或者低空无人机，包括固定翼无人机、旋翼无人机、伞翼无人机或者扑翼无人机等。
下面，将通过具体实施例对本发明进行具体阐述。
实施例一
请参阅图1，是本发明其中一实施例提供的一种无人机100，该无人机100为四旋翼无人机，包括：机身10、机臂20、动力装置30、双目摄像装置40、起落架50、智能电池(图未示)以及飞控系统(图未示)。机臂20、双目摄像装置40以及起落架50均与机身10连接，动力装置30则设置于机臂20上，智能电池和飞控系统则设置于机身10内。其中，动力装置30、双目摄像装置40以及起落架50均与飞控系统通信连接，使得飞控系统能够通过动力装置30来控制无人机100的飞行，飞控系统还能够控制双目摄像装置40拍摄前方环 境以及控制起落架50打开与收起。
优选地，机臂20数量为4，均匀分布于机身10四周，与机身10固定连接，用于承载动力装置30。其中，机臂20与机身10一体成型。
动力装置30包括电机以及与电机轴连接的螺旋桨，电机能够带动螺旋桨旋转以为无人机100提供升力或推力，实现飞行；电机还能够通过改变螺旋桨的转速及方向来改变无人机100的飞行方向。当动力装置30与飞控系统通信连接时，飞控系统能够通过控制电机来控制无人机100的飞行。
该动力装置30设置于机臂20未与机身10连接的一端，并通过电机连接机臂20。
优选地，在无人机100的4个机臂上均设置有动力装置30，以使无人机100能够平稳飞行。
双目摄像装置40则为双目相机，设置于机身10底部，用于拍摄前方环境，以得到同一时刻的同一场景在不同视角下的双目图像。其中，双目图像包括左图像和右图像，左图像和右图像的匹配点之间存在视差，双目摄像装置40能够通过对左图像和右图像进行立体匹配得到视差图。
其中，为了减少无人机100对障碍物的误检测问题，双目摄像装置40还用于执行双目立体视差确定方法，以提高确定目标视差的准确性，进而使得双目摄像装置40能够得到准确的视差图。
在其他一些实施例中，该双目摄像装置40还能够通过云台设置于机身10底部，以消除无人机100飞行过程中对双目摄像装置40造成的扰动，保证双目摄像装置40拍摄的双目图像清晰稳定。
起落架50则设置于机身10底部相对两侧，通过驱动装置连接于机身10，起落架50在驱动装置的驱动下能够进行打开与收起。在无人机100与地面接触时，驱动装置控制起落架50打开，以使无人机100能够通过起落架50与地面接触；在无人机100飞行过程中，驱动装置控制起落架50收起，以避免起落架50影响无人机100飞行。当起落架50与飞控系统通信连接时，飞控系统能够通过控制驱动装置来控制起落架50的打开与收起。
智能电池则用于为无人机100供电，以使无人机100的动力装置30、双目摄像装置40、起落架50以及飞控系统能够通电运行。其中，智能电池包括但不限于：干电池、铅蓄电池以及锂电池等。
飞控系统则与动力装置30、双目摄像装置40以及起落架50通过有线连接或者无线连接的方式进行通信连接。其中，无线连接包括但不限于：WiFi、蓝牙、ZigBee等。
其中，双目摄像装置40执行双目立体视差确定方法，具体包括：
双目摄像装置40拍摄得到同一时刻的同一场景在不同视角下的双目图像后，将得到的双目图像中任意一个确定为待匹配图像，另一个确定为匹配图像。
其中，双目图像包括左图像和右图像，左图像由位于双目摄像装置40左侧的摄像头拍摄，右图像由位于双目摄像装置40右侧的摄像头拍摄，双目摄 像装置40拍摄的左图像和右图像大小一致，均为M*N。
可以将左图像确定为待匹配图像，右图像确定为匹配图像；也可以将左图像确定为匹配图像，右图像确定为待匹配图像。优选地，在本发明实施例中，将左图像确定为待匹配图像，将右图像确定为匹配图像。
其中，待匹配图像由M*N个排成行列的待匹配像素点组成，每个待匹配像素点均对应灰度值。
匹配图像则由M*N个排成行列的匹配像素点组成，每个匹配像素点均对应灰度值。
双目摄像装置40确定待匹配图像和匹配图像后，根据待匹配图像与匹配图像，建立图像视差空间。
其中，图像视差空间包含待匹配图像中每个待匹配像素点的像素视差空间，即待匹配图像中有多少个待匹配像素点，图像视差空间就由多少个像素视差空间组成，一个待匹配像素点对应一个像素视差空间。比如，待匹配图像中有M*N个待匹配像素点时，则图像视差空间包含M*N个像素视差空间。
其中，每一个待匹配像素点的像素视差空间均包含该待匹配像素点在不同视差下的代价聚合值。
基于此，根据待匹配图像与匹配图像，建立图像视差空间时，分别建立每一个待匹配像素点的像素视差空间。
具体地，分别建立每一个待匹配像素点的像素视差空间时，按照预设顺序在待匹配图像中提取一个待匹配像素点建立像素视差空间后，再按照预设顺序提取下一个待匹配像素点建立像素视差空间，直至按照预设顺序将待匹配图像中的待匹配像素点提取完毕。其中，预设顺序包括但不限于：横向蛇形、纵向蛇形、横向平行或者纵向平行等。比如：当预设顺序为横向蛇形时，先提取待匹配图像中第一行第一个待匹配像素点建立像素视差空间后，再提取待匹配图像中第一行第二个待匹配像素点建立像素视差空间。
对待匹配像素点建立像素视差空间时，首先，根据待匹配像素点的位置，在匹配图像中确定与待匹配像素点的位置相同的匹配像素点作为起始点。比如，请参阅图2，L为待匹配图像，R为匹配图像，当对待匹配像素点A0建立像素视差空间时，确定待匹配像素点A0的位置位于第一行第一列，因此，在匹配图像中，确定位于第一行第一列的匹配像素点B0作为起始点；同理，若对待匹配像素点A6建立像素视差空间时，确定待匹配像素点A6的位置位于第二行第二列，因此，在匹配图像中，确定位于第二行第二列的匹配像素点B6作为起始点。
其次，以起始点为起点，沿水平方向进行搜索，将预设视差搜索范围内每个视差下对应的匹配像素点确定为目标点。
其中，沿水平方向进行搜索即沿起始点所在行进行搜索。
预设视差搜索范围为待匹配像素点查找对应点的范围，亦即，待匹配像素点的对应点可能存在的范围。该预设视差搜索范围由对应点与待匹配像素点的 最大视差以及对应点与待匹配像素点的最小视差进行确定，包括最大视差、最小视差以及最大视差和最小视差之间的所有视差。其中，视差为正整数。比如，预设视差搜索范围D为[0,3]，则预设视差搜索范围D内的视差分别为0,1,2,3。
该预设视差搜索范围为预先设置的经验值。
举例而言，请参阅图2，当确定匹配像素点B0作为起始点，预设视差搜索范围为[0,3]时，以匹配像素点B0为起点，沿匹配像素点B0所在行——第一行进行搜索，可以确定匹配像素点B0与待匹配像素点A0的视差为0、匹配像素点B1与待匹配像素点A0的视差为1、匹配像素点B2与待匹配像素点A0的视差为2、匹配像素点B3与待匹配像素点A0的视差为3以及匹配像素点B4与待匹配像素点A0的视差为4，此时，因为预设视差搜索范围为0,1,2,3，因此，将视差为0时对应的匹配像素点B0确定为目标点1，将视差为1时对应的匹配像素点B1确定为目标点2，将视差为2时对应的匹配像素点B2确定为目标点3，将视差为3时对应的匹配像素点B3确定为目标点4。
然后，计算每个目标点的代价聚合值。
其中，计算每个目标点的代价聚合值，即分别计算目标点的代价聚合值。比如，计算目标点1的代价聚合值、计算目标点2的代价聚合值、计算目标点3的代价聚合值以及计算目标点4的代价聚合值。
具体地，能够通过局部匹配算法计算目标点的代价聚合值。局部匹配算法包括但不限于：绝对差值和(SAD)、截断绝对差值和(STAD)以及差值平方和(SSD)等。
优选地，在本发明实施例中，通过绝对差值和计算目标点的代价聚合值。举例而言，首先，确定目标点对应的待匹配像素点；其次，确定以目标点对应的待匹配像素点为中心，预设半径为半径的第一窗口，以及确定以目标点为中心，预设半径为半径的第二窗口；然后，计算第一窗口和第二窗口对应像素点的灰度差值；最后，将灰度差值的绝对值之和确定为目标点的代价聚合值。
比如，请参阅图2，计算目标点1——匹配像素点B0的代价聚合值时，由于目标点1是待匹配像素点A0对应的目标点，故确定目标点1对应的待匹配像素点为待匹配像素点A0；当预设半径为1时，以所确定的待匹配像素点A0为中心，确定半径为1的第一窗口P1，此时，所得到的第一窗口P1大小为3*3，包括待匹配像素点A0、A1、A5以及A6，然后，以目标点1——匹配像素点B0为中心，确定半径为1的第二窗口P2，此时，所得到的第二窗口P2大小为3*3，包括匹配像素点B0、B1、B5以及B6；当第一窗口P1和第二窗口P2重合时，待匹配像素点A0与匹配像素点B0对应，待匹配像素点A1与匹配像素点B1对应，待匹配像素点A5与匹配像素点B5对应，待匹配像素点A6与匹配像素点B6对应，故计算待匹配像素点A0与匹配像素点B0的灰度差值、待匹配像素点A1与匹配像素点B1的灰度差值、待匹配像素点A5与匹配像素点B5的灰度差值以及待匹配像素点A6与匹配像素点B6的灰度差值，然后，求取所计算出的待匹配像素点A0与匹配像素点B0的灰度差值的绝对值a1，求取所计算 出的待匹配像素点A1与匹配像素点B1的灰度差值的绝对值a2，求取所计算出的待匹配像素点A5与匹配像素点B5的灰度差值的绝对值a3以及求取所计算出的待匹配像素点A6与匹配像素点B6的灰度差值的绝对值a4，并计算a1、a2、a3以及a4之和，此时，确定a1、a2、a3以及a4之和为目标点1的代价聚合值，即目标点1的代价聚合值S1＝a1+a2+a3+a4。
同理，通过上述方法分别计算目标点2——匹配像素点B1的代价聚合值S2、计算目标点3——匹配像素点B2的代价聚合值S3以及计算目标点4——匹配像素点B3的代价聚合值S4。
最后，根据目标点的视差存储对应的代价聚合值，以形成像素视差空间。
比如，分别算出目标点1的代价聚合值S1、目标点2的代价聚合值S2、目标点3的代价聚合值S3以及目标点4的代价聚合值S4后，由于目标点1视差为0，故将S1存储为待匹配像素点A0在视差为0时的代价聚合值，由于目标点2视差为1，故将S2存储为待匹配像素点A0在视差为1时的代价聚合值，由于目标点3视差为2，故将S3存储为待匹配像素点A0在视差为3时的代价聚合值，由于目标点4视差为3，故将S4存储为待匹配像素点A0在视差为3时的代价聚合值，形成待匹配像素点A0的像素视差空间。
同理，通过上述方法分别建立待匹配像素点A1至A19的像素视差空间，由所建立的待匹配像素点A0至A19的像素视差空间形成图像视差空间。
可以理解的是，图像视差空间的大小与待匹配图像大小M*N以及预设视差搜索范围D内的视差个数N1相关，图像视差空间的大小为M*N*N1。比如，请参阅图2，由于待匹配图像L的大小为4*5，预设视差搜索范围D内的视差分别为0、1、2、3，个数为4，故形成的图像视差空间大小为4*5*4。
双目摄像装置40建立图像视差空间后，分别根据每一个待匹配像素点的像素视差空间，筛选出满足预设匹配条件的待匹配像素点作为基准像素点。
即根据图像视差空间中每一个待匹配像素点的像素视差空间，对每一个待匹配像素点进行筛选，确定待匹配像素点是否满足预设匹配条件，若待匹配像素点满足预设匹配条件，则筛选出该待匹配像素点作为基准像素点。比如，在根据待匹配图像L和匹配图像R建立的图像视差空间中，包括待匹配像素点A0至A19的像素视差空间，根据待匹配像素点A0的像素视差空间，对待匹配像素点A0进行筛选，确定待匹配像素点A0是否满足预设匹配条件，若待匹配像素点A0满足预设匹配条件，则筛选出待匹配像素点A0作为基准像素点后，再根据待匹配像素点A1的像素视差空间，对待匹配像素点A1进行筛选，确定待匹配像素点A1是否满足预设匹配条件，若待匹配像素点A1满足预设匹配条件，则筛选出待匹配像素点A1作为基准像素点；然后，再根据待匹配像素点A2的像素视差空间，对待匹配像素点A2进行筛选，确定待匹配像素点A2是否满足预设匹配条件，若待匹配像素点A2满足预设匹配条件，则筛选出待匹配像素点A2作为基准像素点，直至将待匹配像素点A0至A19全部筛选完毕。
其中，预设匹配条件为预先设置的用于指导双目摄像装置40滤除位于重 复纹理场景或无纹理场景的待匹配像素点的条件。
具体地，对待匹配像素点进行筛选时，首先，在待匹配像素点的像素视差空间中，查找出代价聚合极值；然后，根据代价聚合极值，计算出像素视差空间的分界值；最后，根据分界值及代价聚合极值，筛选出基准像素点。
其中，代价聚合极值包括最小极值、次最小极值、次次最小极值和最大极值；分界值包括第一分界值、第二分界值和第三分界值。
当双目摄像装置40在待匹配像素点的像素视差空间中，查找出最小极值、次最小极值、次次最小极值和最大极值后，根据最小极值和最大极值计算出像素视差空间的第一分界值，并根据第一分界值和最小极值，计算出像素视差空间的第二分界值，然后根据第二分界值和最小极值，计算出像素视差空间的第三分界值。
其中，第一分界值为最小极值和最大极值之和的二分之一；第二分界值为第一分界值和最小极值之和的二分之一；第三分界值为第二分界值和最小极值之和的二分之一。
当双目摄像装置40计算出像素视差空间的第一分界值、第二分界值和第三分界值后，判断次最小极值是否小于第三分界值，若次最小极值小于第三分界值，则滤除待匹配像素点；若次最小极值不小于第三分界值，则判断次次最小极值是否小于第二分界值的二分之一，若次次最小极值小于第二分界值的二分之一，则滤除待匹配像素点；若次次最小极值不小于第二分界值的二分之一，则选择待匹配像素点作为基准像素点。
举例而言，当对待匹配像素点A0进行筛选时，确定待匹配像素点A0的像素视差空间中包括视差为0时的代价聚合值S1、视差为1时的代价聚合值S2、视差为2时的代价聚合值S3以及视差为3时的代价聚合值S4，假设代价聚合值S1为最小极值、S2为次最小极值、S3为次次最小极值、S4为最大极值，则双目摄像装置40在待匹配像素点A0的像素视差空间中，能够查找出最小极值、次最小极值、次次最小极值和最大极值，并确定最小极值为S1、次最小极值为S2、次次最小极值为S3和最大极值为S4；
然后，根据最小极值S1和最大极值S4计算出待匹配像素点A0的像素视差空间的第一分界值X1，X1＝(S1+S4)/2；根据第一分界值X1和最小极值S1计算出待匹配像素点A0的像素视差空间的第二分界值X2，X2＝(S1+X1)/2；根据第二分界值X2和最小极值S1计算出待匹配像素点A0的像素视差空间的第三分界值X3，X3＝(S1+X2)/2；
然后，判断次最小极值S2是否小于第三分界值X3，若S2<X3，则认为待匹配像素点A0位于重复纹理场景或无纹理场景，容易导致目标视差确认出错，故滤除待匹配像素点A0；若S2≥X3，则判断次次最小极值S3是否小于第二分界值X2的二分之一，若S3<X2/2，则也认为待匹配像素点A0位于重复纹理场景或无纹理场景，容易导致目标视差确认出错，故滤除待匹配像素点A0；经过两次滤除之后，若S3≥X2/2，则认为通过待匹配像素点A0确认目标视差的 准确率较高，选择待匹配像素点A0作为基准像素点。
同理，通过上述方法，依次对待匹配像素点A1至A19进行两次滤除的筛选，确定待匹配像素点A1至A19中哪些待匹配像素点能够作为基准像素点。
双目摄像装置40筛选出基准像素点后，分别根据每一个基准像素点在每个视差下的代价聚合值，确定满足预设聚合条件的代价聚合值对应的视差作为目标视差。
即对每一个基准像素点都确定一个目标视差，双目摄像装置40筛选出多少个基准像素点，就能够确定多少个目标视差，每个基准像素点对应一个目标视差。
比如，当确定待匹配像素点A0和待匹配像素点A1为基准像素点时，先根据待匹配像素点A0在每个视差下的代价聚合值，确定满足预设聚合条件的代价聚合值对应的视差作为待匹配像素点A0的目标视差后，再根据待匹配像素点A1在每个视差下的代价聚合值，确定满足预设聚合条件的代价聚合值对应的视差作为待匹配像素点A1的目标视差。
其中，分别根据每一个基准像素点在每个视差下的代价聚合值，确定满足预设聚合条件的代价聚合值对应的视差作为目标视差，具体包括：在每一个基准像素点对应的代价聚合值中，查找出最小代价聚合值后，选择最小代价聚合值对应的视差作为目标视差。
比如，先在待匹配像素点A0对应的代价聚合值中，查找出最小代价聚合值，选择最小代价聚合值对应的视差作为待匹配像素点A0的目标视差后，再在待匹配像素点A1对应的代价聚合值中，查找出最小代价聚合值，选择最小代价聚合值对应的视差作为待匹配像素点A1的目标视差。其中，确定待匹配像素点A0对应的代价聚合值包括视差为0时的代价聚合值S1、视差为1时的代价聚合值S2、视差为2时的代价聚合值S3以及视差为3时的代价聚合值S4，且S1最小，则在待匹配像素点A0对应的代价聚合值中能够查找出S1为最小代价聚合值，此时，选择S1对应的视差0作为待匹配像素点A0的目标视差，即待匹配像素点A0的目标视差为0；确定待匹配像素点A1对应的代价聚合值包括视差为0时的代价聚合值S5、视差为1时的代价聚合值S6、视差为2时的代价聚合值S7以及视差为3时的代价聚合值S8，且S7最小，则在待匹配像素点A1对应的代价聚合值中能够查找出S7为最小代价聚合值，此时，选择S7对应的视差2作为待匹配像素点A1的目标视差，即待匹配像素点A1的目标视差为2。
进一步地，为了提高目标视差的精度，在一些实施例中，双目摄像装置40确定目标视差后，还会对目标视差进行校正。
举例而言，能够通过抛物线插值法对目标视差进行校正，包括：确定与目标视差相邻的两个参考视差；提取每个参考视差的代价聚合值；根据两个参考视差及两个参考视差对应的代价聚合值，按照抛物线插值法构建插值函数；根据插值函数与目标视差，确定最终视差，将目标视差校正为最终视差。
其中，插值函数为：
Figure PCTCN2020113584-appb-000001
d(x,y)为目标视差，S d-1(x,y)为与目标视差相邻的前一个视差的代价聚合值，S d+1(x,y)为与目标视差相邻的后一个视差的代价聚合值。
比如，对待匹配像素点A1的目标视差2，由于视差1和视差3与目标视差2相邻，故确定视差1和视差3为参考视差；由于视差为1时的代价聚合值为S6、视差为3时的代价聚合值为S8，故提取视差1的代价聚合值S6和视差3的代价聚合值S8；根据视差1及其对应的代价聚合值S6、视差3及其对应的代价聚合值S8，按照抛物线插值法构建插值函数，根据插值函数与目标视差2，确定最终视差。
在本发明实施例中，双目摄像装置通过每一个待匹配像素点的像素视差空间，对待匹配图像中的每一个待匹配像素点进行筛选，筛选出满足预设匹配条件的待匹配像素点作为基准像素点后，才通过基准像素点确定目标视差，能够滤除可能位于重复纹理或者无人理场景中的待匹配像素点，防止目标视差确定出错，提高确定目标视差的准确性，进而使得搭载双目摄像装置的无人机能够获取准确的深度图，减少在重复纹理场景或无纹理场景中的误检测问题。
实施例二
请参阅图3，是本发明其中一实施例提供的一种双目立体视差确定方法的流程示意图，应用于无人机，该无人机为上述实施例中所述的无人机100，而本发明实施例提供的方法由上述双目摄像装置40执行，用于提高确定目标视差的准确性，进而减少无人机在重复纹理或无纹理场景中的误检测问题，该双目立体视差确定方法包括：
S100：根据待匹配图像与匹配图像，建立图像视差空间。
其中，待匹配图像和匹配图像为双目摄像装置在不同视角下拍摄得到的同一时刻同一场景的双目图像，故待匹配图像和匹配图像大小一致，均为M*N。
待匹配图像由M*N个排成行列的待匹配像素点组成，每个待匹配像素点均对应灰度值。
匹配图像则由M*N个排成行列的匹配像素点组成，每个匹配像素点均对应灰度值。
图像视差空间则包含待匹配图像中每个待匹配像素点的像素视差空间，即待匹配图像中有多少个待匹配像素点，图像视差空间就由多少个像素视差空间组成，一个待匹配像素点对应一个像素视差空间。比如，待匹配图像中有M*N个待匹配像素点时，则图像视差空间包含M*N个像素视差空间。
其中，每一个待匹配像素点的像素视差空间均包含该待匹配像素点在不同视差下的代价聚合值。
基于此，根据待匹配图像与匹配图像，建立图像视差空间时，分别建立每 一个待匹配像素点的像素视差空间。
具体地，分别建立每一个待匹配像素点的像素视差空间时，按照预设顺序在待匹配图像中提取一个待匹配像素点建立像素视差空间后，再按照预设顺序提取下一个待匹配像素点建立像素视差空间，直至按照预设顺序将待匹配图像中的待匹配像素点提取完毕。其中，预设顺序包括但不限于：横向蛇形、纵向蛇形、横向平行或者纵向平行等。比如：当预设顺序为横向蛇形时，先提取待匹配图像中第一行第一个待匹配像素点建立像素视差空间后，再提取待匹配图像中第一行第二个待匹配像素点建立像素视差空间。
在一些实施例中，对待匹配像素点建立像素视差空间时，首先，根据待匹配像素点的位置，在匹配图像中确定与待匹配像素点的位置相同的匹配像素点作为起始点。比如，请参阅图2，L为待匹配图像，R为匹配图像，当对待匹配像素点A0建立像素视差空间时，确定待匹配像素点A0的位置位于第一行第一列，因此，在匹配图像中，确定位于第一行第一列的匹配像素点B0作为起始点；同理，若对待匹配像素点A6建立像素视差空间时，确定待匹配像素点A6的位置位于第二行第二列，因此，在匹配图像中，确定位于第二行第二列的匹配像素点B6作为起始点。
其次，以起始点为起点，沿水平方向进行搜索，将预设视差搜索范围内每个视差下对应的匹配像素点确定为目标点。
其中，沿水平方向进行搜索即沿起始点所在行进行搜索。
预设视差搜索范围为待匹配像素点查找对应点的范围，亦即，待匹配像素点的对应点可能存在的范围。该预设视差搜索范围由对应点与待匹配像素点的最大视差以及对应点与待匹配像素点的最小视差进行确定，包括最大视差、最小视差以及最大视差和最小视差之间的所有视差。其中，视差为正整数。比如，预设视差搜索范围D为[0,3]，则预设视差搜索范围D内的视差分别为0,1,2,3。
该预设视差搜索范围为预先设置的经验值。
举例而言，请参阅图2，当确定匹配像素点B0作为起始点，预设视差搜索范围为[0,3]时，以匹配像素点B0为起点，沿匹配像素点B0所在行——第一行进行搜索，可以确定匹配像素点B0与待匹配像素点A0的视差为0、匹配像素点B1与待匹配像素点A0的视差为1、匹配像素点B2与待匹配像素点A0的视差为2、匹配像素点B3与待匹配像素点A0的视差为3以及匹配像素点B4与待匹配像素点A0的视差为4，此时，因为预设视差搜索范围为0,1,2,3，因此，将视差为0时对应的匹配像素点B0确定为目标点1，将视差为1时对应的匹配像素点B1确定为目标点2，将视差为2时对应的匹配像素点B2确定为目标点3，将视差为3时对应的匹配像素点B3确定为目标点4。
然后，计算每个目标点的代价聚合值。
其中，计算每个目标点的代价聚合值，即分别计算目标点的代价聚合值。比如，计算目标点1的代价聚合值、计算目标点2的代价聚合值、计算目标点3的代价聚合值以及计算目标点4的代价聚合值。
具体地，能够通过局部匹配算法计算目标点的代价聚合值。局部匹配算法包括但不限于：绝对差值和(SAD)、截断绝对差值和(STAD)以及差值平方和(SSD)等。
优选地，在本发明实施例中，通过绝对差值和计算目标点的代价聚合值。举例而言，首先，确定目标点对应的待匹配像素点；其次，确定以目标点对应的待匹配像素点为中心，预设半径为半径的第一窗口，以及确定以目标点为中心，预设半径为半径的第二窗口；然后，计算第一窗口和第二窗口对应像素点的灰度差值；最后，将灰度差值的绝对值之和确定为目标点的代价聚合值。
比如，请参阅图2，计算目标点1——匹配像素点B0的代价聚合值时，由于目标点1是待匹配像素点A0对应的目标点，故确定目标点1对应的待匹配像素点为待匹配像素点A0；当预设半径为1时，以所确定的待匹配像素点A0为中心，确定半径为1的第一窗口P1，此时，所得到的第一窗口P1大小为3*3，包括待匹配像素点A0、A1、A5以及A6，然后，以目标点1——匹配像素点B0为中心，确定半径为1的第二窗口P2，此时，所得到的第二窗口P2大小为3*3，包括匹配像素点B0、B1、B5以及B6；当第一窗口P1和第二窗口P2重合时，待匹配像素点A0与匹配像素点B0对应，待匹配像素点A1与匹配像素点B1对应，待匹配像素点A5与匹配像素点B5对应，待匹配像素点A6与匹配像素点B6对应，故计算待匹配像素点A0与匹配像素点B0的灰度差值、待匹配像素点A1与匹配像素点B1的灰度差值、待匹配像素点A5与匹配像素点B5的灰度差值以及待匹配像素点A6与匹配像素点B6的灰度差值，然后，求取所计算出的待匹配像素点A0与匹配像素点B0的灰度差值的绝对值a1，求取所计算出的待匹配像素点A1与匹配像素点B1的灰度差值的绝对值a2，求取所计算出的待匹配像素点A5与匹配像素点B5的灰度差值的绝对值a3以及求取所计算出的待匹配像素点A6与匹配像素点B6的灰度差值的绝对值a4，并计算a1、a2、a3以及a4之和，此时，确定a1、a2、a3以及a4之和为目标点1的代价聚合值，即目标点1的代价聚合值S1＝a1+a2+a3+a4。
同理，通过上述方法分别计算目标点2——匹配像素点B1的代价聚合值S2、计算目标点3——匹配像素点B2的代价聚合值S3以及计算目标点4——匹配像素点B3的代价聚合值S4。
最后，根据目标点的视差存储对应的代价聚合值，以形成像素视差空间。
比如，分别算出目标点1的代价聚合值S1、目标点2的代价聚合值S2、目标点3的代价聚合值S3以及目标点4的代价聚合值S4后，由于目标点1视差为0，故将S1存储为待匹配像素点A0在视差为0时的代价聚合值，由于目标点2视差为1，故将S2存储为待匹配像素点A0在视差为1时的代价聚合值，由于目标点3视差为2，故将S3存储为待匹配像素点A0在视差为3时的代价聚合值，由于目标点4视差为3，故将S4存储为待匹配像素点A0在视差为3时的代价聚合值，形成待匹配像素点A0的像素视差空间。
同理，通过上述方法分别建立待匹配像素点A1至A19的像素视差空间， 由所建立的待匹配像素点A0至A19的像素视差空间形成图像视差空间。
可以理解的是，图像视差空间的大小与待匹配图像大小M*N以及预设视差搜索范围D内的视差个数N1相关，图像视差空间的大小为M*N*N1。比如，请参阅图2，由于待匹配图像L的大小为4*5，预设视差搜索范围D内的视差分别为0、1、2、3，个数为4，故形成的图像视差空间大小为4*5*4。
S200：分别根据每一个所述待匹配像素点的像素视差空间，筛选出满足预设匹配条件的待匹配像素点作为基准像素点。
其中，预设匹配条件为预先设置的用于指导双目摄像装置滤除位于重复纹理场景或无纹理场景的待匹配像素点的条件。
分别根据每一个待匹配像素点的像素视差空间，筛选出满足预设匹配条件的待匹配像素点作为基准像素点，亦即，根据图像视差空间中每一个待匹配像素点的像素视差空间，对每一个待匹配像素点进行筛选，确定待匹配像素点是否满足预设匹配条件，若待匹配像素点满足预设匹配条件，则筛选出该待匹配像素点作为基准像素点。比如，在根据待匹配图像L和匹配图像R建立的图像视差空间中，包括待匹配像素点A0至A19的像素视差空间，根据待匹配像素点A0的像素视差空间，对待匹配像素点A0进行筛选，确定待匹配像素点A0是否满足预设匹配条件，若待匹配像素点A0满足预设匹配条件，则筛选出待匹配像素点A0作为基准像素点后，再根据待匹配像素点A1的像素视差空间，对待匹配像素点A1进行筛选，确定待匹配像素点A1是否满足预设匹配条件，若待匹配像素点A1满足预设匹配条件，则筛选出待匹配像素点A1作为基准像素点；然后，再根据待匹配像素点A2的像素视差空间，对待匹配像素点A2进行筛选，确定待匹配像素点A2是否满足预设匹配条件，若待匹配像素点A2满足预设匹配条件，则筛选出待匹配像素点A2作为基准像素点，直至将待匹配像素点A0至A19全部筛选完毕。
具体地，在一些实施例中，对待匹配像素点进行筛选时，首先，在待匹配像素点的像素视差空间中，查找出代价聚合极值；然后，根据代价聚合极值，计算出像素视差空间的分界值；最后，根据分界值及代价聚合极值，筛选出基准像素点。
其中，代价聚合极值包括最小极值、次最小极值、次次最小极值和最大极值；分界值包括第一分界值、第二分界值和第三分界值。
当在待匹配像素点的像素视差空间中，查找出最小极值、次最小极值、次次最小极值和最大极值后，根据最小极值和最大极值计算出像素视差空间的第一分界值，并根据第一分界值和最小极值，计算出像素视差空间的第二分界值，然后根据第二分界值和最小极值，计算出像素视差空间的第三分界值。
其中，第一分界值为最小极值和最大极值之和的二分之一；第二分界值为第一分界值和最小极值之和的二分之一；第三分界值为第二分界值和最小极值之和的二分之一。
当计算出像素视差空间的第一分界值、第二分界值和第三分界值后，判断 次最小极值是否小于第三分界值，若次最小极值小于第三分界值，则滤除待匹配像素点；若次最小极值不小于第三分界值，则判断次次最小极值是否小于第二分界值的二分之一，若次次最小极值小于第二分界值的二分之一，则滤除待匹配像素点；若次次最小极值不小于第二分界值的二分之一，则选择待匹配像素点作为基准像素点。
举例而言，当对待匹配像素点A0进行筛选时，确定待匹配像素点A0的像素视差空间中包括视差为0时的代价聚合值S1、视差为1时的代价聚合值S2、视差为2时的代价聚合值S3以及视差为3时的代价聚合值S4，假设代价聚合值S1为最小极值、S2为次最小极值、S3为次次最小极值、S4为最大极值，则在待匹配像素点A0的像素视差空间中，能够查找出最小极值、次最小极值、次次最小极值和最大极值，并确定最小极值为S1、次最小极值为S2、次次最小极值为S3和最大极值为S4；
然后，根据最小极值S1和最大极值S4计算出待匹配像素点A0的像素视差空间的第一分界值X1，X1＝(S1+S4)/2；根据第一分界值X1和最小极值S1计算出待匹配像素点A0的像素视差空间的第二分界值X2，X2＝(S1+X1)/2；根据第二分界值X2和最小极值S1计算出待匹配像素点A0的像素视差空间的第三分界值X3，X3＝(S1+X2)/2；
然后，判断次最小极值S2是否小于第三分界值X3，若S2<X3，则认为待匹配像素点A0位于重复纹理场景或无纹理场景，容易导致目标视差确认出错，故滤除待匹配像素点A0；若S2≥X3，则判断次次最小极值S3是否小于第二分界值X2的二分之一，若S3<X2/2，则也认为待匹配像素点A0位于重复纹理场景或无纹理场景，容易导致目标视差确认出错，故滤除待匹配像素点A0；经过两次滤除之后，若S3≥X2/2，则认为通过待匹配像素点A0确认目标视差的准确率较高，选择待匹配像素点A0作为基准像素点。
同理，通过上述方法，依次对待匹配像素点A1至A19进行两次滤除的筛选，确定待匹配像素点A1至A19中哪些待匹配像素点能够作为基准像素点。
S300：在所述基准像素点中，分别根据每一个所述基准像素点在每个所述视差下的代价聚合值，确定满足预设聚合条件的代价聚合值对应的所述视差作为目标视差。
即对每一个基准像素点都确定一个目标视差，双目摄像装置筛选出多少个基准像素点，就能够确定多少个目标视差，每个基准像素点对应一个目标视差。
比如，当确定待匹配像素点A0和待匹配像素点A1为基准像素点时，先根据待匹配像素点A0在每个视差下的代价聚合值，确定满足预设聚合条件的代价聚合值对应的视差作为待匹配像素点A0的目标视差后，再根据待匹配像素点A1在每个视差下的代价聚合值，确定满足预设聚合条件的代价聚合值对应的视差作为待匹配像素点A1的目标视差。
在一些实施例中，分别根据每一个基准像素点在每个视差下的代价聚合值，确定满足预设聚合条件的代价聚合值对应的视差作为目标视差，具体包括： 在每一个基准像素点对应的代价聚合值中，查找出最小代价聚合值后，选择最小代价聚合值对应的视差作为目标视差。
比如，先在待匹配像素点A0对应的代价聚合值中，查找出最小代价聚合值，选择最小代价聚合值对应的视差作为待匹配像素点A0的目标视差后，再在待匹配像素点A1对应的代价聚合值中，查找出最小代价聚合值，选择最小代价聚合值对应的视差作为待匹配像素点A1的目标视差。其中，确定待匹配像素点A0对应的代价聚合值包括视差为0时的代价聚合值S1、视差为1时的代价聚合值S2、视差为2时的代价聚合值S3以及视差为3时的代价聚合值S4，且S1最小，则在待匹配像素点A0对应的代价聚合值中能够查找出S1为最小代价聚合值，此时，选择S1对应的视差0作为待匹配像素点A0的目标视差，即待匹配像素点A0的目标视差为0；确定待匹配像素点A1对应的代价聚合值包括视差为0时的代价聚合值S5、视差为1时的代价聚合值S6、视差为2时的代价聚合值S7以及视差为3时的代价聚合值S8，且S7最小，则在待匹配像素点A1对应的代价聚合值中能够查找出S7为最小代价聚合值，此时，选择S7对应的视差2作为待匹配像素点A1的目标视差，即待匹配像素点A1的目标视差为2。
进一步地，请参阅图4，在一些实施例中，为了提高目标视差的精度，步骤S300确定目标视差后，该方法还包括：
S400：校正所述目标视差。
其中，能够通过抛物线插值法对目标视差进行校正，包括：确定与目标视差相邻的两个参考视差；提取每个参考视差的代价聚合值；根据两个参考视差及两个参考视差对应的代价聚合值，按照抛物线插值法构建插值函数；根据插值函数与目标视差，确定最终视差，将目标视差校正为最终视差。
其中，插值函数为：
Figure PCTCN2020113584-appb-000002
d(x,y)为目标视差，S d-1(x,y)为与目标视差相邻的前一个视差的代价聚合值，S d+1(x,y)为与目标视差相邻的后一个视差的代价聚合值。
比如，对待匹配像素点A1的目标视差2，由于视差1和视差3与目标视差2相邻，故确定视差1和视差3为参考视差；由于视差为1时的代价聚合值为S6、视差为3时的代价聚合值为S8，故提取视差1的代价聚合值S6和视差3的代价聚合值S8；根据视差1及其对应的代价聚合值S6、视差3及其对应的代价聚合值S8，按照抛物线插值法构建插值函数，根据插值函数与目标视差2，确定最终视差。
在本发明实施例中，通过每一个待匹配像素点的像素视差空间，对待匹配图像中的每一个待匹配像素点进行筛选，筛选出满足预设匹配条件的待匹配像素点作为基准像素点后，才通过基准像素点确定目标视差，能够滤除可能位于 重复纹理或者无人理场景中的待匹配像素点，防止目标视差确定出错，提高确定目标视差的准确性，进而使得无人机能够获取准确的深度图，减少在重复纹理场景或无纹理场景中的误检测问题。
实施例三
以下所使用的术语“模块”为可以实现预定功能的软件和/或硬件的组合。尽管以下实施例所描述的装置可以以软件来实现，但是硬件，或者软件和硬件的组合的实现也是可能被构想的。
请参阅图5，是本发明其中一实施例提供的一种双目立体视差确定装置，该装置应用于无人机，该无人机为上述实施例中所述的无人机100，而本发明实施例提供的装置各个模块的功能由上述双目摄像装置40执行，用于提高确定目标视差的准确性，进而减少无人机在重复纹理或无纹理场景中的误检测问题，该双目立体视差确定装置包括：
建立模块200，用于根据待匹配图像与匹配图像，建立图像视差空间，所述图像视差空间包含所述待匹配图像中每个待匹配像素点的像素视差空间，其中，每一个所述待匹配像素点的像素视差空间均包含所述待匹配像素点在不同视差下的代价聚合值；
筛选模块300，用于分别根据每一个所述待匹配像素点的像素视差空间，筛选出满足预设匹配条件的待匹配像素点作为基准像素点；
确定模块400，用于在所述基准像素点中，分别根据每一个所述基准像素点在每个所述视差下的代价聚合值，确定满足预设聚合条件的代价聚合值对应的所述视差作为目标视差。
在一些实施例中，所述筛选模块300具体用于：
在每一个所述待匹配像素点的像素视差空间中，查找出代价聚合极值；
根据所述代价聚合极值，计算出所述像素视差空间的分界值；
根据所述分界值及所述代价聚合极值，筛选出基准像素点。
在一些实施例中，所述代价聚合极值包括最小极值、次最小极值、次次最小极值以及最大极值；所述分界值包括第一分界值、第二分界值以及第三分界值；则，
所述筛选模块300具体用于：
根据所述最小极值和所述最大极值，得到所述第一分界值；
根据所述第一分界值和所述最小极值，得到所述第二分界值；
根据所述第二分界值和所述最小极值，得到所述第三分界值。
在一些实施例中，所述第一分界值为所述最小极值和所述最大极值之和的二分之一；
所述第二分界值为所述第一分界值和所述最小极值之和的二分之一；
所述第三分界值为所述第二分界值和所述最小极值之和的二分之一。
在一些实施例中，所述筛选模块300具体用于：
判断所述次最小极值是否小于所述第三分界值；
若所述次最小极值小于所述第三分界值，则滤除所述待匹配像素点；
若所述次最小极值不小于所述第三分界值，则判断所述次次最小极值是否小于所述第二分界值的二分之一；
若所述次次最小极值小于所述第二分界值的二分之一，则滤除所述待匹配像素点；
若所述次次最小极值不小于所述第二分界值的二分之一，则选择所述待匹配像素点作为基准像素点。
在一些实施例中，所述确定模块400具体用于：
在每一个所述基准像素点对应的代价聚合值中，查找出最小代价聚合值；
选择所述最小代价聚合值对应的所述视差作为目标视差。
请参阅图6，为了提高目标视差的精度，在一些实施例中，双目立体视差确定装置还包括：
校正模块500，用于校正所述目标视差。
在一些实施例中，所述校正模块500具体用于：
通过抛物线插值法校正所述目标视差。
在一些实施例中，所述建立模块200还用于：
通过SAD算法计算所述待匹配图像中每个待匹配像素点在不同视差下的代价聚合值。
当然，在其他一些可替代实施例中，上述建立模块200、筛选模块300、确定模块400以及校正模块500可以为双目摄像装置400的图像处理芯片。
由于装置实施例和方法实施例是基于同一构思，在内容不互相冲突的前提下，装置实施例的内容可以引用方法实施例的，在此不再一一赘述。
在本发明实施例中，通过每一个待匹配像素点的像素视差空间，对待匹配图像中的每一个待匹配像素点进行筛选，筛选出满足预设匹配条件的待匹配像素点作为基准像素点后，才通过基准像素点确定目标视差，能够滤除可能位于重复纹理或者无人理场景中的待匹配像素点，防止目标视差确定出错，提高确定目标视差的准确性，进而使得无人机能够获取准确的深度图，减少在重复纹理场景或无纹理场景中的误检测问题。
实施例四
请参阅图7，是本发明其中一实施例提供的一种无人机的硬件结构示意图，本发明实施例提供的硬件模块能够集成于上述实施例所述的双目摄像装置40内，使得双目摄像装置40能够执行以上实施例所述的一种双目立体视差确定方法，还能实现以上实施例所述的一种双目立体视差确定装置的各个模块的功能。该无人机100包括：
一个或多个处理器110以及存储器120。其中，图7中以一个处理器110为例。
处理器110和存储器120可以通过总线或者其他方式连接，图7中以通过总线连接为例。
存储器120作为一种非易失性计算机可读存储介质，可用于存储非易失性软件程序、非易失性计算机可执行程序以及模块，如本发明上述实施例中的一种双目立体视差确定方法对应的程序指令以及一种双目立体视差确定装置对应的模块(例如，建立模块200、筛选模块300和确定模块400等)。处理器110通过运行存储在存储器120中的非易失性软件程序、指令以及模块，从而执行一种双目立体视差确定方法的各种功能应用以及数据处理，即实现上述方法实施例中的一种双目立体视差确定方法以及上述装置实施例的各个模块的功能。
存储器120可以包括存储程序区和存储数据区，其中，存储程序区可存储操作系统、至少一个功能所需要的应用程序；存储数据区可存储根据一种双目立体视差确定装置的使用所创建的数据等。
所述存储数据区还存储有预设的数据，包括预设顺序、预设半径、预设视差搜索范围、预设匹配条件以及预设聚合条件等。
此外，存储器120可以包括高速随机存取存储器，还可以包括非易失性存储器，例如至少一个磁盘存储器件、闪存器件、或其他非易失性固态存储器件。在一些实施例中，存储器120可选包括相对于处理器110远程设置的存储器，这些远程存储器可以通过网络连接至处理器110。上述网络的实例包括但不限于互联网、企业内部网、局域网、移动通信网及其组合。
所述程序指令以及一个或多个模块存储在所述存储器120中，当被所述一个或者多个处理器110执行时，执行上述任意方法实施例中的一种双目立体视差确定方法的各个步骤，或者，实现上述任意装置实施例中的一种双目立体视差确定装置的各个模块的功能。
上述产品可执行本发明上述实施例所提供的方法，具备执行方法相应的功能模块和有益效果。未在本实施例中详尽描述的技术细节，可参见本发明上述实施例所提供的方法。
本发明实施例还提供了一种非易失性计算机可读存储介质，所述计算机可读存储介质存储有计算机可执行指令，该计算机可执行指令被一个或多个处理器执行，例如图7中的一个处理器110，可使得计算机执行上述任意方法实施例中的一种双目立体视差确定方法的各个步骤，或者，实现上述任意装置实施例中的一种双目立体视差确定装置的各个模块的功能。
本发明实施例还提供了一种计算机程序产品，所述计算机程序产品包括存储在非易失性计算机可读存储介质上的计算机程序，所述计算机程序包括程序指令，当所述程序指令被一个或多个处理器执行，例如图7中的一个处理器110，可使得计算机执行上述任意方法实施例中的一种双目立体视差确定方法的各个步骤，或者，实现上述任意装置实施例中的一种双目立体视差确定装置的各个模块的功能。
以上所描述的装置实施例仅仅是示意性的，其中所述作为分离部件说明的模块可以是或者也可以不是物理上分开的，作为模块显示的部件可以是或者也 可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部模块来实现本实施例方案的目的。
通过以上的实施例的描述，本领域普通技术人员可以清楚地了解到各实施例可借助软件加通用硬件平台的方式来实现，当然也可以通过硬件。本领域普通技术人员可以理解实现上述实施例方法中的全部或部分流程是可以通过计算机程序指令相关的硬件来完成，所述的程序可存储于一计算机可读取存储介质中，该程序在执行时，可包括如上述各方法的实施方法的流程。其中，所述存储介质可为磁碟、光盘、只读存储记忆体(Read-Only Memory,ROM)或随机存储记忆体(RandomAccessMemory,RAM)等。
以上所述仅为本发明的实施例，并非因此限制本发明的专利范围，凡是利用本发明说明书及附图内容所作的等效结构或等效流程变换，或直接或间接运用在其他相关的技术领域，均同理包括在本发明的专利保护范围内。
最后应说明的是：以上实施例仅用以说明本发明的技术方案，而非对其限制；在本发明的思路下，以上实施例或者不同实施例中的技术特征之间也可以进行组合，步骤可以以任意顺序实现，并存在如上所述的本发明的不同方面的许多其它变化，为了简明，它们没有在细节中提供；尽管参照前述实施例对本发明进行了详细的说明，本领域的普通技术人员应当理解：其依然可以对前述各实施例所记载的技术方案进行修改，或者对其中部分技术特征进行等同替换；而这些修改或者替换，并不使相应技术方案的本质脱离本申请各实施例技术方案的范围。

Claims (20)

  1. 一种双目立体视差确定方法，应用于无人机的双目摄像装置，其特征在于，所述方法包括：
    根据待匹配图像与匹配图像，建立图像视差空间，所述图像视差空间包含所述待匹配图像中每个待匹配像素点的像素视差空间，其中，每一个所述待匹配像素点的像素视差空间均包含所述待匹配像素点在不同视差下的代价聚合值；
    分别根据每一个所述待匹配像素点的像素视差空间，筛选出满足预设匹配条件的待匹配像素点作为基准像素点；
    在所述基准像素点中，分别根据每一个所述基准像素点在每个所述视差下的代价聚合值，确定满足预设聚合条件的代价聚合值对应的所述视差作为目标视差。
  2. 根据权利要求1所述的方法，其特征在于，所述分别根据每一个所述待匹配像素点的像素视差空间，筛选出满足预设匹配条件的待匹配像素点作为基准像素点，包括：
    在每一个所述待匹配像素点的像素视差空间中，查找出代价聚合极值；
    根据所述代价聚合极值，计算出所述像素视差空间的分界值；
    根据所述分界值及所述代价聚合极值，筛选出基准像素点。
  3. 根据权利要求2所述的方法，其特征在于，所述代价聚合极值包括最小极值、次最小极值、次次最小极值以及最大极值；所述分界值包括第一分界值、第二分界值以及第三分界值；则，
    所述根据所述代价聚合极值，计算出所述像素视差空间的分界值，包括：
    根据所述最小极值和所述最大极值，得到所述第一分界值；
    根据所述第一分界值和所述最小极值，得到所述第二分界值；
    根据所述第二分界值和所述最小极值，得到所述第三分界值。
  4. 根据权利要求3所述的方法，其特征在于，
    所述第一分界值为所述最小极值和所述最大极值之和的二分之一；
    所述第二分界值为所述第一分界值和所述最小极值之和的二分之一；
    所述第三分界值为所述第二分界值和所述最小极值之和的二分之一。
  5. 根据权利要求3或4所述的方法，其特征在于，所述根据所述分界值及所述代价聚合极值，筛选出基准像素点，包括：
    判断所述次最小极值是否小于所述第三分界值；
    若所述次最小极值小于所述第三分界值，则滤除所述待匹配像素点；
    若所述次最小极值不小于所述第三分界值，则判断所述次次最小极值是否小于所述第二分界值的二分之一；
    若所述次次最小极值小于所述第二分界值的二分之一，则滤除所述待匹配像素点；
    若所述次次最小极值不小于所述第二分界值的二分之一，则选择所述待匹配像素点作为基准像素点。
  6. 根据权利要求1至5中任一项所述的方法，其特征在于，所述分别根据每一个所述基准像素点在每个所述视差下的代价聚合值，确定满足预设聚合条件的代价聚合值对应的所述视差作为目标视差，包括：
    在每一个所述基准像素点对应的代价聚合值中，查找出最小代价聚合值；
    选择所述最小代价聚合值对应的所述视差作为目标视差。
  7. 根据权利要求1至6中任一项所述的方法，其特征在于，所述方法还包括：
    校正所述目标视差。
  8. 根据权利要求7所述的方法，其特征在于，所述校正所述目标视差，包括：
    通过抛物线插值法校正所述目标视差。
  9. 根据权利要求1至8中任一项所述的方法，其特征在于，所述方法还包括：
    通过SAD算法计算所述待匹配图像中每个待匹配像素点在不同视差下的代价聚合值。
  10. 一种双目立体视差确定装置，应用于无人机的双目摄像装置，其特征在于，所述装置包括：
    建立模块，用于根据待匹配图像与匹配图像，建立图像视差空间，所述图像视差空间包含所述待匹配图像中每个待匹配像素点的像素视差空间，其中，每一个所述待匹配像素点的像素视差空间均包含所述待匹配像素点在不同视差下的代价聚合值；
    筛选模块，用于分别根据每一个所述待匹配像素点的像素视差空间，筛选出满足预设匹配条件的待匹配像素点作为基准像素点；
    确定模块，用于在所述基准像素点中，分别根据每一个所述基准像素点在每个所述视差下的代价聚合值，确定满足预设聚合条件的代价聚合值对应的所述视差作为目标视差。
  11. 根据权利要求10所述的装置，其特征在于，所述筛选模块具体用于：
    在每一个所述待匹配像素点的像素视差空间中，查找出代价聚合极值；
    根据所述代价聚合极值，计算出所述像素视差空间的分界值；
    根据所述分界值及所述代价聚合极值，筛选出基准像素点。
  12. 根据权利要求11所述的装置，其特征在于，所述代价聚合极值包括最小极值、次最小极值、次次最小极值以及最大极值；所述分界值包括第一分界值、第二分界值以及第三分界值；则，
    所述筛选模块具体用于：
    根据所述最小极值和所述最大极值，得到所述第一分界值；
    根据所述第一分界值和所述最小极值，得到所述第二分界值；
    根据所述第二分界值和所述最小极值，得到所述第三分界值。
  13. 根据权利要求12所述的装置，其特征在于，
    所述第一分界值为所述最小极值和所述最大极值之和的二分之一；
    所述第二分界值为所述第一分界值和所述最小极值之和的二分之一；
    所述第三分界值为所述第二分界值和所述最小极值之和的二分之一。
  14. 根据权利要求12或13所述的装置，其特征在于，所述筛选模块具体用于：
    判断所述次最小极值是否小于所述第三分界值；
    若所述次最小极值小于所述第三分界值，则滤除所述待匹配像素点；
    若所述次最小极值不小于所述第三分界值，则判断所述次次最小极值是否小于所述第二分界值的二分之一；
    若所述次次最小极值小于所述第二分界值的二分之一，则滤除所述待匹配像素点；
    若所述次次最小极值不小于所述第二分界值的二分之一，则选择所述待匹配像素点作为基准像素点。
  15. 根据权利要求10至14中任一项所述的装置，其特征在于，所述确定模块具体用于：
    在每一个所述基准像素点对应的代价聚合值中，查找出最小代价聚合值；
    选择所述最小代价聚合值对应的所述视差作为目标视差。
  16. 根据权利要求10至15中任一项所述的装置，其特征在于，所述装置还包括：
    校正模块，用于校正所述目标视差。
  17. 根据权利要求16所述的装置，其特征在于，所述校正模块具体用于：
    通过抛物线插值法校正所述目标视差。
  18. 根据权利要求10至17中任一项所述的装置，其特征在于，所述建立模块还用于：
    通过SAD算法计算所述待匹配图像中每个待匹配像素点在不同视差下的代价聚合值。
  19. 一种无人机，其特征在于，包括：
    机身；
    机臂，与所述机身相连；
    动力装置，设于所述机臂；以及
    双目摄像装置，与所述机身相连；
    其中，所述双目摄像装置包括：
    至少一个处理器；以及
    与所述至少一个处理器通信连接的存储器；其中，所述存储器存储有可被所述至少一个处理器执行的指令，所述指令被所述至少一个处理器执行，以使所述至少一个处理器能够用于执行如权利要求1至9中任一项所述的双目立体视差确定方法。
  20. 一种非易失性计算机可读存储介质，其特征在于，所述非易失性计算机可读存储介质存储有计算机可执行指令，所述计算机可执行指令用于使无人机的双目摄像装置执行如权利要求1至9中任一项所述的双目立体视差确定方法。
PCT/CN2020/113584 2019-09-04 2020-09-04 一种双目立体视差确定方法、装置及无人机 WO2021043285A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201910833064.2 2019-09-04
CN201910833064.2A CN110533703B (zh) 2019-09-04 2019-09-04 一种双目立体视差确定方法、装置及无人机

Publications (1)

Publication Number Publication Date
WO2021043285A1 true WO2021043285A1 (zh) 2021-03-11

Family

ID=68666834

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/113584 WO2021043285A1 (zh) 2019-09-04 2020-09-04 一种双目立体视差确定方法、装置及无人机

Country Status (2)

Country Link
CN (1) CN110533703B (zh)
WO (1) WO2021043285A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114677261A (zh) * 2022-05-27 2022-06-28 绍兴埃瓦科技有限公司 一种视差处理电路和视差处理系统及其方法
CN115297249A (zh) * 2022-09-28 2022-11-04 深圳慧源创新科技有限公司 一种双目摄像头及双目避障方法

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110533703B (zh) * 2019-09-04 2022-05-03 深圳市道通智能航空技术股份有限公司 一种双目立体视差确定方法、装置及无人机
CN111311667B (zh) 2020-02-14 2022-05-17 苏州浪潮智能科技有限公司 一种内容自适应双目匹配方法和装置

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106780590A (zh) * 2017-01-03 2017-05-31 成都通甲优博科技有限责任公司 一种深度图的获取方法及系统
CN107316326A (zh) * 2017-06-29 2017-11-03 海信集团有限公司 应用于双目立体视觉的基于边的视差图计算方法和装置
WO2018101603A1 (ko) * 2016-12-02 2018-06-07 (주)비전에스티 스테레오 카메라를 이용한 도로객체 인식방법 및 장치
CN110148181A (zh) * 2019-04-25 2019-08-20 青岛康特网络科技有限公司 一种通用双目立体匹配方法
CN110533703A (zh) * 2019-09-04 2019-12-03 深圳市道通智能航空技术有限公司 一种双目立体视差确定方法、装置及无人机

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101887589B (zh) * 2010-06-13 2012-05-02 东南大学 一种基于立体视觉的实拍低纹理图像重建方法
CN103226821B (zh) * 2013-04-27 2015-07-01 山西大学 基于视差图像素分类校正优化的立体匹配方法
CN104853184B (zh) * 2015-05-27 2017-02-01 天津大学 一种影响立体图像舒适度的视差范围的检测方法
CN106815594A (zh) * 2015-11-30 2017-06-09 展讯通信（上海）有限公司 立体匹配方法及装置
CN109961417B (zh) * 2017-12-26 2021-04-06 广州极飞科技有限公司 图像处理方法、装置及移动装置控制方法
CN108257165B (zh) * 2018-01-03 2020-03-24 上海兴芯微电子科技有限公司 图像立体匹配方法、双目视觉设备
CN108846858A (zh) * 2018-06-01 2018-11-20 南京邮电大学 一种计算机视觉的立体匹配算法

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2018101603A1 (ko) * 2016-12-02 2018-06-07 (주)비전에스티 스테레오 카메라를 이용한 도로객체 인식방법 및 장치
CN106780590A (zh) * 2017-01-03 2017-05-31 成都通甲优博科技有限责任公司 一种深度图的获取方法及系统
CN107316326A (zh) * 2017-06-29 2017-11-03 海信集团有限公司 应用于双目立体视觉的基于边的视差图计算方法和装置
CN110148181A (zh) * 2019-04-25 2019-08-20 青岛康特网络科技有限公司 一种通用双目立体匹配方法
CN110533703A (zh) * 2019-09-04 2019-12-03 深圳市道通智能航空技术有限公司 一种双目立体视差确定方法、装置及无人机

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114677261A (zh) * 2022-05-27 2022-06-28 绍兴埃瓦科技有限公司 一种视差处理电路和视差处理系统及其方法
CN114677261B (zh) * 2022-05-27 2022-08-26 绍兴埃瓦科技有限公司 一种视差处理电路和视差处理系统及其方法
CN115297249A (zh) * 2022-09-28 2022-11-04 深圳慧源创新科技有限公司 一种双目摄像头及双目避障方法
CN115297249B (zh) * 2022-09-28 2023-01-06 深圳慧源创新科技有限公司 一种双目摄像头及双目避障方法

Also Published As

Publication number Publication date
CN110533703B (zh) 2022-05-03
CN110533703A (zh) 2019-12-03

Similar Documents

Publication Publication Date Title
WO2021043285A1 (zh) 一种双目立体视差确定方法、装置及无人机
Han et al. Deepvio: Self-supervised deep learning of monocular visual inertial odometry using 3d geometric constraints
CN108323190B (zh) 一种避障方法、装置和无人机
WO2020102927A1 (zh) 拍摄方法和无人机
WO2020239093A1 (zh) 一种相机标定方法、装置和电子设备
CN111083381B (zh) 一种图像融合的方法、装置及双光相机、无人机
WO2020215188A1 (zh) 飞行航线的生成方法、控制装置及无人机系统
US20200221062A1 (en) Image processing method and device
CN110086995B (zh) 一种图像亮度调节方法、装置及无人机
WO2022042184A1 (zh) 一种跟踪目标的位置估计方法、装置及无人飞行器
US10873702B2 (en) Adaptive motion filtering in an unmanned autonomous vehicle
WO2021035731A1 (zh) 无人飞行器的控制方法、装置及计算机可读存储介质
WO2018121247A1 (zh) 控制无人机飞行的方法、装置以及无人机
EP3531375A1 (en) Method and apparatus for measuring distance, and unmanned aerial vehicle
US10917561B2 (en) Image processing in an unmanned autonomous vehicle
WO2021081707A1 (zh) 数据处理方法、装置、可移动平台及计算机可读存储介质
JP2017015704A (ja) ドローンに搭載されて土地をマッピングするように適合されたカメラユニット、およびカメラユニットによる画像撮像管理方法
WO2022048541A1 (zh) 基于双目视觉的环境感知方法、装置及无人飞行器
WO2022016348A1 (zh) 设备控制方法、装置及计算机可读存储介质
WO2021081774A1 (zh) 一种参数优化方法、装置及控制设备、飞行器
WO2020228780A1 (zh) 一种图像色彩调节方法、装置及无人机
CN108985193A (zh) 一种基于图像检测的无人机航拍人像对准方法
WO2020207411A1 (zh) 一种图像数据处理方法、装置、图像处理芯片及飞行器
WO2020114433A1 (zh) 一种深度感知方法，装置和深度感知设备
WO2021097848A1 (zh) 图像处理方法、图像采集装置、可移动平台及存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20861396

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20861396

Country of ref document: EP

Kind code of ref document: A1