WO2018210078A1 - Distance measurement method for unmanned aerial vehicle, and unmanned aerial vehicle - Google Patents

Abstract

Disclosed are a distance measurement method for an unmanned aerial vehicle, and an unmanned aerial vehicle. The distance measurement method for an unmanned aerial vehicle comprises the following steps: acquiring a first image captured by a first imaging device on an unmanned aerial vehicle at one moment, and a second image captured by a second imaging device on the unmanned aerial vehicle at the moment; determining a first pixel block in the first image, and a second pixel block, matched with the first pixel block, in the second image, any of the first pixel block and the second pixel block comprising at least 2 pixel points; and determining the distance between the unmanned aerial vehicle and a target object according to the parallax value between the first pixel block and the second pixel block. The application can improve the distance measurement efficiency.

Description

UAV distance measurement method and drone

This application claims the priority of the Chinese Patent Application entitled "A UAV Height Measurement Method and UAV" submitted to the Chinese Patent Office on May 19, 2017, application number: 201710356535.6, the entire contents of which are incorporated by reference. Combined in this application.

Technical field

The present application relates to the field of unmanned aerial vehicles, and in particular to a distance measuring method for a drone and a drone using the same.

Background technique

With the development of wireless communication technology, wireless local area network, image processing technology and battery technology, the endurance flight function and image processing of drones are becoming more and more powerful, and more and more users are interested in drone shooting and exploration. When the drone flies out of the user's field of view, the drone needs to feed back its flight data to the ground control station at a certain frequency to realize the ground control station to control the drone's obstacle avoidance flight, or the drone realizes the avoidance according to the acquired flight data. Barrier flight, these functions are necessary to protect the drone's safe flight.

Currently, drones can use the vision system to measure the distance to the target and use that distance as flight data. How to improve the measurement efficiency of the vision system in the drone has become an active research topic by those skilled in the art.

Summary of the invention

In view of this, the embodiments of the present invention provide a distance measurement method for a drone and a drone, which can improve the efficiency of distance measurement of the vision system.

In a first aspect, an embodiment of the present invention provides a method for measuring a height of a drone, including the following steps:

Obtaining a first image captured by the first imaging device on the drone at one time, and a second image captured by the second imaging device on the drone at the time;

Determining a first pixel block in the first image and a second pixel block in the second image that matches the first pixel block; wherein, in the first pixel block and the second pixel block Either of which includes at least 2 pixels;

Determining a distance between the drone and the target according to a disparity value of the first pixel block and the second pixel block.

Optionally, the determining the first pixel block in the first image, and the second pixel block in the second image that matches the first pixel block, includes:

Determining first piece of feature information of the first pixel block in the first image;

Determining block feature information of each of the at least one pixel block in the second image; matching the first block feature information with block feature information of each of the pixel blocks;

A pixel block that matches the block feature information in the at least one pixel block with the first block feature information is used as the second pixel block.

Optionally, the determining, by the first pixel block in the first image, and the second pixel block in the second image that matches the first pixel block,

Determining a first pixel block in the first image;

Determining at least one pixel block in the second image;

Pixel matching the first pixel block with the at least one pixel block;

Determining a second pixel block in the at least one pixel block according to the matching result of the pixel matching.

Optionally, determining, according to the matching result of the pixel matching, the second pixel block in the at least one pixel block, including:

A pixel block having the highest degree of matching with the pixel of the first pixel block in the at least one pixel block is used as the second pixel block.

Optionally, the determining, by the first pixel block in the first image, and the second pixel block in the second image that matches the first pixel block,

Determining a first pixel point in the first pixel block in the first image;

Determining at least one pixel block in the second image;

Matching the first pixel point with a pixel point included in each of the at least one pixel block;

A pixel block of the at least one pixel block including a pixel point matching the first pixel point is used as the second pixel block.

Optionally, the matching, by the pixel point, the pixel point included in each of the at least one pixel block comprises:

Determining point feature information of the first pixel point;

Determining point feature information of the pixel points included in each of the pixel blocks;

The point feature information of the first pixel point is matched with the point feature information of the pixel point included in each of the pixel blocks.

Optionally, the first pixel point includes a central pixel point of the first pixel block.

Optionally, the method further includes:

Determining first location information of the first pixel block and second location information of the second pixel block;

Determining a disparity value of the first pixel block and the second pixel block according to the first location information and the second location information.

Optionally, the determining the first location information of the first pixel block and the second location information of the second pixel block, including:

Determining first location information of the second pixel in the first pixel block, and second location information of the third pixel in the second pixel block, the second pixel and the third pixel Point matching.

Optionally, determining the distance between the UAV and the target according to the disparity value of the first pixel block and the second pixel block, including:

Determining a distance between the drone and the target according to installation parameters of the first imaging device and the second imaging device, and a disparity value of the first pixel block and the second pixel block .

Optionally, the installation parameters of the first imaging device and the second imaging device include at least one of the following:

The distance between the lens optical center of the first imaging device and the lens optical center of the second imaging device, the distance from the optical center of the first imaging device to the drone body, and the optical center of the second imaging device to the drone The distance of the body.

Optionally, if the first image includes at least two first pixel blocks, determining the drone and the target according to the disparity values of the first pixel block and the second pixel block. The distance between them includes:

Determine at least 2 disparity values;

Determining at least two distance values according to the at least two disparity values;

Determining a distance between the drone and the target based on the at least two distance values.

Optionally, determining, according to the at least two distance values, a distance between the UAV and the target, including:

Calculating an average of the at least two distance values and using the average value as a distance between the drone and the target.

In a second aspect, an embodiment of the present invention provides a drone, including:

a first imaging device and a second imaging device;

And a processor respectively connected to the first imaging device and the second imaging device;

Wherein the processor is configured to perform any one of the first aspects.

In a third aspect, the embodiment of the present application further provides a vision system, including:

At least 2 imaging devices;

At least one processor respectively connected to the at least two imaging devices; and

a memory communicatively coupled to the at least one processor;

Wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor after the user completes the interaction by the human interaction unit, such that the at least one processor is capable of performing the first aspect Any method.

In a fourth aspect, the embodiment of the present application further provides a non-transitory computer readable storage medium storing computer executable instructions for causing a computer to execute the first aspect Any of the methods.

In a fifth aspect, the embodiment of the present application further provides a computer program product, the computer program product comprising a computer program stored on a non-transitory computer readable storage medium, the computer program comprising program instructions, when the program instruction is The computer, when executed, causes the computer to perform any of the methods of the first aspect.

The beneficial effects of the embodiments of the present invention are: the UAV height measuring method provided in the embodiment and the UAV do not need to additionally set a special height measuring device, by presetting the positional combination between the first imaging device and the second imaging device The real-time image analysis of the drone realizes the height measurement from the ground. The height measurement method is simplified, the image matching is stable and reliable, and the height measurement response is fast. The measurement accuracy is five to ten times higher than the ultrasonic height measurement in the set height range.

DRAWINGS

1 is a schematic diagram of a drone according to an embodiment of the present invention;

2 is a schematic flow chart of a method for measuring a distance of a drone according to an embodiment of the present invention;

3 is a schematic diagram of a method for measuring a height of a drone according to an embodiment of the present invention;

4 is a schematic diagram of a module of a drone according to an embodiment of the present invention;

FIG. 5 is a schematic diagram showing the hardware structure of a vision system in a drone according to an embodiment of the present invention.

detailed description

The technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the drawings in the embodiments of the present invention. It is a partial embodiment of the invention, and not all of the embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative efforts are within the scope of the present invention.

The distance measuring method and the drone of the drone provided by the embodiment of the invention can improve the efficiency and accuracy of the distance measurement of the visual system of the drone.

The following describes an application environment provided by an embodiment of the present application.

The drone may include a flight control system (referred to as a flight control system), a vision system; alternatively, the drone may further include a plurality of systems such as an ultrasound system, an infrared system, a power system, a power system, and a data transmission system.

The flight control system is the core control device of the drone, and the flight control system may include: a main control module, a signal conditioning and interface module, a data acquisition module, and a steering gear drive module.

The flight control system collects the flight state data measured by each sensor in real time, receives the control commands and data transmitted by the ground controller via the radio uplink channel, and outputs control commands to the executing mechanism to realize various flights to the drone. Modal control and management and control of mission equipment; at the same time, the state data of the drone (ie flight data) and the operating state parameters of the engine, onboard power system and mission equipment are transmitted to the data transmission system in real time, and downlinked by radio The channel is sent back to the ground controller.

The flight control system is further configured to: perform high-precision acquisition of multi-channel analog signals, including gyro signals, heading signals, rudder angle signals, engine speed, cylinder temperature signals, dynamic and static pressure sensor signals, power supply voltage signals, and the like. The output switch signal, analog signal and PWM pulse signal can be adapted to the control of different actuators such as rudder, aileron servo, elevator, air passage and damper. Communication with onboard data terminals, GPS signals, digital sensors, and related mission devices is accomplished using a plurality of communication channels.

The software design of the flight control system is divided into two parts, namely the programming of the logic circuit chip and the application design of the flight control system. The logic circuit chip is used to form a digital logic control circuit to complete decoding and isolation as well as A/D, D/A and so on. The application is used to implement the above functions of the flight control system.

The vision system includes one or more imaging devices, for example, the vision system can include two imaging devices, in which case the two imaging devices can also be binocular imaging devices. Wherein, the imaging device includes an optical component and an image acquisition sensor. Optical components are used to image objects, and image acquisition sensors are used to acquire image data using optical components. Further, the vision system can also include a processor coupled to one or more imaging devices, respectively, which processor can include one or more. The image data collected by the image acquisition sensor may be transmitted to the processor, and the processor processes the image data collected by the image acquisition sensor. For example, the processor can calculate the distance between the target and the drone in the image according to the images captured by the plurality of imaging devices simultaneously. The processor can further transmit the calculated distance to the flight control system, so that the flight control system controls the flight of the drone according to the distance to implement functions such as obstacle avoidance, tracking, hovering and the like.

The vision system can be installed at one or more of the front end, the rear end, the upper end, and the lower end of the drone. The embodiment of the present application does not limit the installation position of the vision system on the drone. When the lower end of the drone is equipped with a vision system, the vision system can be called a lower vision system, and the lower vision system can measure the height of the drone. At this time, it can be understood as between the drone and the ground. The distance, the ground, is the target.

For other systems in the above-mentioned drone, refer to the implementation in the current drone, which will not be described here.

In the embodiment of the present application, the vision system includes two imaging devices as an example for description. Currently, the processor in the vision system can acquire images acquired by the two imaging devices at the same time, that is, the processor can acquire two images, and can perform pixel matching on the two images to obtain matching pixels. To be, the processor can calculate the distance between the target corresponding to the pixel and the drone based on the disparity value between the matched pairs of pixel points.

In this way, the vision system calculates the distance and the accuracy is not high; at the same time, the range of the detected object is small, for example, only the distance of the target within 10 meters can be calculated.

The distance measuring method and the drone of the unmanned aerial vehicle provided in the embodiments of the present application have high measurement efficiency, accurate accuracy and wide measurement range.

It should be noted that, when the distance measurement method provided by the embodiment of the present application is applied to a lower vision system for measuring a height, the flight height of the drone can be realized by directly calculating the most simplified forward view orientation. The measurement can also be applied to the side view direction height measurement occasion of the unmanned aerial vehicle under the premise that the installation angles of the first imaging device and the second imaging device are predicted.

Example 1

Referring to FIG. 1 and FIG. 4, the unmanned aerial vehicle of the present embodiment includes a fuselage 20, a pan/tilt head 50, propellers 30, 32, and a first imaging device 52 and a second imaging device 54 disposed on the pan/tilt head. It should be noted that the vision system including the first imaging device 52 and the second imaging device 54 shown in FIG. 1 is installed at the position of the main camera. Of course, the vision system can also be installed elsewhere in the drone. The position of the vision system and the orientation of the imaging device in FIG. 1 are merely exemplary, which is defined by the embodiments of the present application.

The drone further includes a flight control processor and a wireless communication module connected to the flight control processor, the wireless communication module establishing a wireless connection with the ground remote control (ie, ground control station) 80, and controlling the flight control processor The UAV flight state parameters (i.e., the above flight parameters) and image data are transmitted to the ground remote controller 80 via the wireless communication module. The wireless communication module receives an operation command sent by the ground remote controller 80, and the flight control processor completes flight control of the drone based on the operation instruction. The flight control processor can be a processor in a flight control system.

A processor may also be included in the vision system, and the processor may be connected to the first imaging device 52 and the second imaging device 54 using a communication interface, a communication bus, or the like. The embodiment of the present application does not limit the connection manner between the processor and the imaging device in the vision system.

From the hardware level:

The drone of this embodiment is provided with one or more processors. The one or more processors may be included in the flight control system to control the system hardware module to complete functions such as flight, map transmission, height measurement, and attitude adjustment of the drone.

In this embodiment, when the first imaging device 52 and the second imaging device 54 are in the down-view orientation, a more accurate UAV flight height can be measured, when the first imaging device 52 and the second imaging device 54 are not present. The processor is also used to adjust the drone attitude such that the first imaging device 52 and the second imaging device 54 are in the same altimetry orientation when the orientation is being viewed.

In another embodiment, the processor is further configured to acquire an installation angle of the first imaging device 52 and the second imaging device 54 with respect to a carrier, such as a drone, respectively; adjusting the first imaging device according to the installation angle and The optical axis direction of the second imaging device until the first imaging device 52 and the second imaging device 54 are in the same altimetry orientation.

From the software level:

In addition to the installation of battery, processor, memory, wireless communication module and other flight control hardware, the UAV body also needs to be equipped with the flight control system 22 and related software of the vision system.

The flight control system 22 is coupled to the altimetry unit 30; the altimeter unit 30 can be understood as a processor in the vision system described above. Wherein, the altimetry unit 30 in the lower vision system can be used to measure the height of the drone, and the altimeter unit 30 in the vision system at other locations can be used to measure the distance between the drone and the target. The altimetry unit 30 can acquire the first imaging device 52 and the second imaging device 54 on the drone while simultaneously capturing the first image and the second image in which the scenes are partially overlapped. Wherein, the partial overlap of the scene in the first image and the second image can be understood as an overlapping portion of the scene on the image for characterizing the same object when the first image and the second image are superimposed. That is, since the images captured by the two imaging devices have parallax, the positions of the same object in the image are not completely the same, and when they are superimposed together, they do not completely overlap, but partially overlap.

The altimetry unit 30 can control the first imaging device to simultaneously capture with the second imaging device, or the first imaging device and the second imaging device can achieve simultaneous shooting according to the respective configured crystal oscillators, and the altimetry unit 30 can acquire two simultaneous images. Images.

The altimetry unit 30 can include a matching module 32, an acquisition module 37, and a height estimation module 36.

The matching module 32 is configured to perform pixel point matching or pixel block matching to obtain a matched pixel point pair or a pixel block pair. The acquiring module 37 is configured to acquire the first image and the second image, and is further configured to acquire the installation parameters of the first imaging device 52 and the second imaging device 54. In this embodiment, the installation parameter is a baseline length, and the imaging is acquired. The imaging focal length f of the device. The measuring unit calculates an accurate height or a current flight of the drone according to the installation parameter, that is, the baseline length, the imaging focal length, and one or more parameters of the parallax value between the pixel pair or the pixel block pair. The distance between the targets.

The drone further includes an adjustment unit for adjusting the first imaging device and the second imaging device to the altimetry or ranging orientation, ie, correcting the imaging device.

Scene matching refers to an image analysis and processing technique that determines a known image area from another corresponding scene area taken by other sensors or finds a correspondence between scene areas. This technology also has important application value in military fields such as navigation and guidance. The scene matching referred to in the present application refers to an image analysis and image processing technique for identifying a reference area of an image from a target area between two images by an image comparison matching algorithm. In binocular vision, the left and right images at the same time are matched, that is, the left image and the right image are the reference image and the real-time image. Calculate the height from the ground according to the parallax.

In order to achieve scene matching, the matching module 32 includes a point module 33 of the same name. The same-named point refers to two image points formed by a certain point in space on the left and right images. These two image points are the same-named points, which can also be understood as the matching pixel point pairs. The matching module 32 can also include a pixel block module for matching pixel blocks in the two images to obtain matching pixel block pairs. A matched pair of pixel blocks can include at least one matched pair of pixel points. Alternatively, the pair of pixel blocks can be derived based on other matching methods.

In an implementation manner, the matching module 33 determines an overlapping area of the first image and the second image, wherein the same-named point module 33 uses a set area in an overlapping area of the first image as a reference area, according to the reference area, Pixel matching is performed in the overlapping area of the second image to obtain a response area having the largest response value, and the center point of the response area and the center point of the reference area are points of the same name.

The matching module 33 also includes a parallax module 34. The disparity module 34 determines a coordinate value of the point of the same name in the first image and determines a coordinate value of the point of the same name in the second image, the coordinate value of the same name point in the first image and the point of the same name in the second image The difference between the coordinate values is the disparity value. Alternatively, disparity module 34 may determine a disparity value between pairs of matched pixel blocks.

In general, an implementation manner of matching the binocular altimetry based on the first image and the second image is: first, the first imaging device 52 and the second imaging device 54 simultaneously capture a group of images, and the first The image and the second image are matched by the scene, and the overlapping regions of the two images are obtained. According to the information of the overlapping region, the position height of the current camera, that is, the flying height of the drone, can be measured.

When calculating the coordinate value of the point of the same name, since the result of the scene matching output is the image coordinate value of the point of the same name, the coordinates of the same name point can be directly performed by the two images formed by the first imaging device 52 and the second imaging device 54. Scene matching is obtained.

The adjustment unit of the drone adjusts the first imaging device and the second imaging device to an altimetry orientation. The principle of the vision system for height measurement will be described below with reference to the accompanying drawings.

Referring to FIG. 3, in the embodiment, the height-adjusting orientation is a positive-downward position, and when the first imaging device 52 and the second imaging device 54 are respectively mounted at an angle of 90 degrees with respect to a horizontal plane of the unmanned vehicle body. The adjusting unit adjusts the first imaging device and the second imaging device to the front view position such that the optical axis of the first imaging device and the optical axis of the second imaging device are both perpendicular to the ground. That is, both the first imaging device 52 and the second imaging device 54 take an image directly below. That is, the correction of the imaging device can be achieved first. The first imaging device 52 and the second imaging device 54 are directly facing the lower side, the mirror surface of the camera is parallel to the ground, and the line connecting the optical center and the center of the image is parallel to the vertical line.

In the case of a front view, the height can be directly solved based on the parallax. As shown in FIG. 3, the corresponding image points of the ground point P3 in the first image and the second image are O1P3 and O2P3, respectively, and the x-direction coordinates are respectively recorded as x ₁ and x _r , and the parallax Δ _x = x ₁ -x _r . Corresponding to the same-named point P3, the corresponding image points in the first image and the second image are O1P3 and O2P3, respectively, and the x-direction coordinates are denoted by x ₁ and x _r , respectively, and the parallax Δ _x = x _{1 -} x _r .

At this time, the camera is measured at the height of the drone, that is, the flying height of the drone has the following geometric relationship:

Where f is the equivalent focal length, B is the baseline length between the two cameras, and Δ _x is the parallax, that is, the displacement of the point of the same name obtained by the scene matching. The equivalent focal length f is the ratio of the actual physical focal length of the camera to the physical size of each pixel, and is an attribute parameter of the camera, which can be estimated by the image attributes of the first image and the second image. It should be noted that the center point mentioned here is only the center point of the reference area where the scene matching is obtained, that is, the point of the same name. The displacement referred to here refers to the difference between the x-axis coordinate values of two points of the same name.

The base length of the installation parameters of the present embodiment refers to the distance between the optical centers of the two imaging devices, which can be acquired by the acquisition module 37 from the flight control system.

In implementation, the adjustment unit of the drone adjusts the first imaging device 52 and the second imaging device 54 to the altimetry orientation to capture an image for altimetry, and the lens optical center and the second imaging device of the first imaging device The spacing between the optical centers of the lenses is a preset installation parameter, that is, a baseline length, which can be acquired by the acquisition module 37 from the flight control system. It can be understood that the installation parameter further includes the distance from the optical center of the first imaging device to the UAV body and the distance from the optical center of the second imaging device to the UAV body.

In order to further improve the measurement accuracy, the first imaging device 52 and the second imaging device 54 employ a sub-pixel image sensor that improves the coordinate accuracy of the same-named point. It can be understood that the disparity value is determined by the difference between the pixel coordinate values, and the standard image accuracy can only reach 1 pixel (pixel). In the matching algorithm using sub-pixel image precision processing, the coordinates of the same-named point can be increased to sub-pixels. Subpixels can achieve an accuracy of 0.1 to 0.3 pixels, which is at least 3 times better than standard image pixel matching. Therefore, the coordinate precision of the point of the same name can be increased accordingly, thereby directly improving the parallax accuracy; according to formula (1), it can be known that the parallax Δ _x has a direct influence on the accuracy of the height H. Therefore, sub-pixel matching can improve the accuracy of parallax solution, and thus improve the accuracy of high resolution.

In another embodiment, the altimetry orientation is a side down direction of the first imaging device 52 and the second imaging device 54, the processor adjusting the drone attitude to cause the first imaging device and the second The imaging devices are at the same height. When the mounting angles of the first imaging device and the second imaging device with respect to the UAV body are not 90 degrees, respectively, the adjusting unit adjusts the posture of the UAV so that the first imaging device and the second imaging device respectively The image is taken when the optical axis is perpendicular to the ground.

The processor acquires a mounting angle of the first imaging device and the second imaging device with respect to the drone, respectively, when the height measuring direction is a side down direction; adjusting the first imaging device and the second imaging device according to the mounting angle The direction of the optical axis. When the altimetry direction is the side down view direction, a preset mounting angle is between the first imaging device and the second imaging device. In the case of non-positive view, if the installation angle is known, it can be solved similarly. This angle parameter needs to be obtained by other means after installation, for example by calibration in a laboratory environment. The present invention defaults to this mounting angle.

The scene matching drone height measurement of the embodiment of the present application has many technical effects, and only needs to use the two onboard cameras and the onboard processing chip of the drone itself, and does not need to specifically add other drone height measuring devices. . Compared with the ultrasonic height measurement, there is no need to add ultrasonic equipment; and the calculation amount is small, simple and fast, and the absolute height of the drone is measured by the binocular scene matching (the height measurement of the drone from the ground), and the measurement accuracy is higher than the ultrasonic wave. ~10 times.

It can be known from the visual principle that, when the baseline distance is fixed, when the distance between the observed object and the observer (the first imaging device and the second imaging device) increases, the parallax also decreases accordingly. Therefore, the height measurement application height of the embodiment of the present application is in the range of about 30 meters. When the drone exceeds 30 altitudes, the flight control system will automatically switch to the altitude measurement using the air pressure drone height measurement combined with the GPS drone height to determine the relative height and the ground height.

Example 2

Referring to FIG. 2 and FIG. 4 together, the present application also relates to a distance measuring method for a drone, which is implemented based on a computer program running by the altimetry unit 30, which can be implemented as shown in FIG. The height calculation module 36, the matching module 32, the point-of-name module 33, and the computer program of the parallax module 34 function.

The distance measuring method of the drone includes the following steps:

The distance measurement by the drone through the vision system can be triggered when the vision system is turned on, or can be triggered according to a request command sent by the ground remote controller 80. Wherein, the position correction of the first imaging device and the second imaging device may be first performed. Taking the following vision system as an example, obtaining the mounting angles of the first imaging device and the second imaging device with respect to the carrier, respectively, when the mounting angles of the first imaging device and the second imaging device with respect to the carrier are respectively 90 degrees, Adjusting a photographing position of the first imaging device and the second imaging device such that an optical axis of the first imaging device and an optical axis of the second imaging device are both perpendicular to the ground;

Adjusting the first imaging device and the second imaging device to the altimetry orientation; after the altimetry unit 30 receives the activation command, the altimetry unit 30 controls the first imaging device and the second imaging device to adjust to the altimetry on the pan/tilt Orientation, such as the first imaging device and the second imaging device are oriented directly downward;

Step 101: Acquire a first image captured by a first imaging device on the drone at one time, and a second image captured by the second imaging device on the drone at the time;

Illustratively, the first imaging device and the second imaging device can be simultaneously photographed by the altimetry unit 30, so that the altimetry unit 30 can acquire the first image captured by the first imaging device on the drone, and acquire the second imaging device simultaneously. The second image taken.

Alternatively, the first imaging device and the second imaging device may perform simultaneous engraving according to the same clock crystal or clock control unit, or according to a separately configured synchronized time unit or clock crystal to obtain a group of images simultaneously photographed. That is, the first image captured by the first imaging device and the second image captured by the second imaging device.

The altimetry unit 30 can acquire the set of images taken simultaneously for further processing.

Step 102: Determine a first pixel block in the first image, and a second pixel block in the second image that matches the first pixel block; wherein the first pixel block and the second pixel Any one of the pixel blocks includes at least 2 pixel points;

Illustratively, the first block of pixels includes at least 2 pixels and the second block of pixels also includes at least 2 pixels. The number of pixels included in the first pixel block may be the same as or different from the number of pixels included in the second pixel block. The first pixel block and the second pixel block can be understood as the matched pixel block pairs described above. Alternatively, the first pixel block and the second pixel block may be understood to partially overlap the scenes in the two images described in the above embodiments.

The first pixel block in the first image can be matched with the second pixel block in the second image by any of the following methods:

In a first manner, determining a first pixel block in the first image;

For example, it may be determined that pixels in the set area in the first image constitute a first pixel block, or the first image and the second image are superimposed and aligned, and a scene overlapping area of the first image and the second image is determined, wherein the scene The overlapping area may include an image of the object. The pixels in a certain area are determined to constitute a first pixel block in the overlapping area of the scene. The region of the first pixel block in the first image may be used as a reference region to determine an overlap region in the second image associated with the reference region.

At least one pixel block in the second image is determined.

For example, the second image may be divided into a plurality of pixel blocks, and the pixel blocks may include the same pixel, that is, the area where the pixel block is overlapped, or each pixel block is independent, that is, does not include the same pixel, the pixel block. There is no overlap in the area. Or determining, according to the reference area where the first pixel block is located, an area related to the reference area in the second image, for example, at least one overlapping area, each overlapping area includes one pixel block, and determining that one of the overlapping areas is included The pixel block is a second pixel block.

Determining first piece feature information of the first pixel block and block feature information of each of the at least one pixel block in the second image.

Here, the block feature information refers to a feature of the entire block of the pixel, which is distinguished from the point feature information, and the point feature information is used to indicate the feature of one pixel. The block feature information can characterize the overall state, grayscale, size, target feature, and the like of the block of pixels. The block feature information may be represented by a block feature vector, and the block feature information may include a multi-dimensional block feature vector, which is not limited herein.

Compared with the matching of the point feature information, the block feature information can more accurately describe the features of the target, thereby improving the accuracy of the distance calculation.

Specifically, the first piece of feature information of the first pixel block is matched with the block feature information of each of the at least one pixel block. Thereby, a second pixel block that successfully matches the first pixel block is obtained from the at least one pixel block, that is, the pixel block whose block feature information matches the first block feature information is used as the second pixel block. For example, the pixel block having the highest similarity between each block feature vector and the first block feature vector is determined from the at least one pixel block as the second pixel block.

Method 2, determining a first pixel block in the first image;

Determining at least one pixel block in the second image;

For the implementation of the above steps, refer to the description in the first method.

Pixel matching the first pixel block with the at least one pixel block;

Determining a second pixel block in the at least one pixel block according to the matching result of the pixel matching.

Specifically, the above pixel matching means that the first pixel block and the pixels in one pixel block of the second image perform line-by-line matching or column-by-column matching. The relative positions of each set of pixels in the pixel block are the same. For example, if a certain pixel is located in the x-row y column in the first pixel block, the pixel point matched in the second image is also located in the x-row y column in the second pixel block. The set of matched pixel points can be further matched according to the point feature information. The point feature information of each pixel point can be represented by a point feature vector. The point feature vector may include a multi-dimensional feature vector, such as a 128-dimensional feature vector or the like.

After each group of pixels is matched, a matching result may be obtained according to the matching degree of the point feature vector, for example, the group of pixel points is successfully matched, or the group of pixel points fails to match.

When the first pixel block is matched with all the group of pixel points in the matching pixel, the matching degree may be determined according to the ratio of the matching success to the matching structure, that is, the higher the proportion, the higher the matching degree.

According to the above manner, the first pixel block is sequentially pixel-matched with each of the at least one pixel block determined in the second image. Thereby, the second pixel block matching the first pixel block can be determined according to the matching result. For example, the pixel block with the highest degree of matching is used as the second pixel block.

Manner 3: determining a first pixel point in the first pixel block in the first image;

Determining at least one pixel block in the second image;

Matching the first pixel point with a pixel point included in each of the at least one pixel block;

A pixel block of the at least one pixel block including a pixel point matching the first pixel point is used as the second pixel block.

Specifically, the first pixel block in the first pixel block is determined, wherein the first pixel point may be one or more. That is to say, in this way, it is not necessary to match all the pixel points in the pixel block, which further improves the matching efficiency.

The first pixel point may be a center point of the first pixel block, that is, a point of the same name in the above embodiment. Alternatively, the first pixel may further include other pixels in the first pixel block, which is not limited herein.

Thereby, the pixel points of the first pixel point and the pixel block in the second image can be matched based on the point feature information. The position of the pixel point matched with the first pixel is the same as the position of the first pixel with respect to the first image.

Further, a second pixel block matching the first pixel block may be determined according to the matching result.

If the first pixel includes one pixel, if there is a pixel that successfully matches the first pixel, the pixel block including the pixel is the second pixel.

When the first pixel includes a plurality of pixels, the degree of matching between each pixel block and the first pixel block is determined, thereby determining that the pixel block with the highest degree of matching is the second pixel block.

It should be noted that the order of execution of the above steps may be changed.

Step 103: Determine a distance between the UAV and the target according to the disparity value of the first pixel block and the second pixel block.

In an implementation manner, a disparity value of the first pixel block and the second pixel block may be determined by determining first position information of the first pixel block and second position information of the second pixel block. Specifically, position information of a certain pixel point in the first pixel block and position information of the pixel point in the second pixel block matching the pixel point may be determined to determine a parallax of the first pixel block and the second pixel block. value. For example, position information of a central pixel point of the first pixel block and position information of a central pixel point of the second pixel block may be determined.

Of course, other parameters can also be obtained, such as the mounting angle of each imaging device, the distance between the optical centers of the lens, the distance from the optical center of the first imaging device to the drone, and the optical center of the second imaging device to the drone. Distance, etc., to calculate the distance between the drone and the target, which is not limited herein.

Further, if at least two first pixel blocks are included in the first image, the second pixel block respectively matching the at least two first pixel blocks may be obtained according to the foregoing manner, and at least two disparity values may be calculated. Then, at least two distance values may be determined according to the two disparity values, so that the distance between the drone and the target is determined according to the at least two distance values.

In an implementation manner, the determined at least two distance values may be calculated as an average value, and the average value is used as a distance value between the drone and the target, that is, the distance data calculated by the visual system.

The following example illustrates that the drone uses the above method to measure the height.

In one embodiment, the exact altitude of the current flight of the drone is measured.

When determining the point of the same name, the matching the first image and the second image to determine the same name includes:

Determining an overlapping area of the first image and the second image, using a set area in the overlapping area in the first image as a reference area;

According to the reference region, pixel matching is performed in the overlapping region of the second image, and a response region having the largest response value is obtained, and the center point of the response region and the center point of the reference region are points of the same name.

When there are a plurality of setting areas, a plurality of height values are calculated based on a plurality of points of the same name, and an average value is taken as the height of the drone.

When determining the disparity value, determining a coordinate value of the same-named point in the first image and determining a coordinate value of the same-named point in the second image, the coordinate value of the same-named point in the first image and the same-named point in the The difference between the coordinate values in the two images is the disparity value.

And pre-storing the installation parameter, that is, the baseline length, before acquiring the installation parameters of the first imaging device and the second imaging device, further comprising installing the first imaging device and the second imaging device, such that the first imaging device The distance between the lens optical center and the lens optical center of the second imaging device is a preset installation parameter, and the installation parameters are saved.

In order to further improve the measurement accuracy, the first imaging device and the second imaging device employ a sub-pixel image sensor that improves the coordinate accuracy of the point of the same name.

In another embodiment, the drone attitude is adjusted such that the first imaging device 52 and the second imaging device 54 are in the same altimetry orientation.

The altimetry orientation is a side down view direction of the first imaging device and the second imaging device. Acquiring the mounting angle of the first imaging device and the second imaging device with respect to the carrier, respectively, adjusting the posture of the carrier when the mounting angle of the first imaging device and the second imaging device with respect to the carrier is not 90 degrees So that the first imaging device and the second imaging device respectively perform image capturing when the optical axis is perpendicular to the ground.

Alternatively, an installation angle of the first imaging device and the second imaging device with respect to the carrier is respectively obtained; and an optical axis direction of the first imaging device and the second imaging device is adjusted according to the mounting angle. When the altimetry is in the side down direction, the mounting angle is preset between the first imaging device and the second imaging device.

It is known from the visual principle that when the distance between the observed object and the observer (the first imaging device and the second imaging device) increases, the displacement caused by the parallax also decreases proportionally. Therefore, the height measurement application height of the embodiment of the present application is in the range of about 30 meters. When the drone exceeds 30 altitudes, the flight control system will automatically switch to the altitude measurement using the air pressure drone height measurement combined with the GPS drone height to determine the relative height and the ground height. Since the height required for the technology such as the industrial drone terrain following is the height from the ground, and the GPS and barometer measure the absolute height, it is necessary to subtract the corresponding ground height before the ground of the drone can be obtained. high.

The drone height measurement method of the embodiment adopts a binocular scene matching image processing technology to measure the height measurement of the drone from the ground, and the measurement accuracy is 5 to 10 times higher than that of the ultrasonic wave.

The UAV and the UAV height measurement method of the embodiment do not need to add other UAV height measurement hardware to the UAV; only the existing onboard dual camera and the onboard processing chip are used, and no special addition is required. device. Compared with ultrasonic height measurement, there is no need to increase the ultrasonic equipment; the calculation of the height measurement of the drone is small, simple and fast, and the flying height of the drone can be fed back in real time, and the height of the drone can be measured quickly and accurately. The image matching of the UAV and the UAV height measuring method of the embodiment is the same as the point name confirmation and the disparity value calculation, which is stable and reliable, and has low requirements on the scene. Moreover, the UAV and the UAV height measurement method of the embodiment can adopt a sub-pixel image processing scheme, and the measurement accuracy can be further improved by three times; the UAV and the UAV height measurement method of the embodiment have a relatively wide working distance. , for a height range of 30 meters.

Example 3

FIG. 5 is a schematic structural diagram of a vision system according to an embodiment of the present application. As shown in FIG. 5, the vision system 600 includes:

At least two imaging devices, two in Figure 5 as an example, respectively a first imaging device 610, a second imaging device 620;

a processor 630 respectively connected to at least two imaging devices;

Of course, it is also possible to include the memory 640 and the like.

The processor 630 may be implemented by at least one visual processing unit (VPU); or implemented by other processing units, which is not limited herein.

The processor 630 and the memory 640 may be connected by a bus or other means, and the bus connection is taken as an example in FIG. Alternatively, the memory 640 is integrated in the processor 630.

The memory 640 is used as a non-volatile computer readable storage medium, and can be used for storing a non-volatile software program, a non-volatile computer executable program, and a module, as in the UAV distance measurement method in the embodiment of the present application. Program instructions/modules (for example, height estimation module 36, matching module 32, point-name module 33, parallax module 34, etc. shown in FIG. 4). The processor 630 executes various functional applications and data processing of the vision system by executing non-volatile software programs, instructions, and modules stored in the memory 640, that is, the distance measurement method of the drone in the above method embodiment is implemented. .

The memory 640 may include a storage program area and a storage data area, wherein the storage program area may store any of the following: an operating system, an application required for at least one function; the storage data area may store distance data, image data, and the like. Moreover, memory 640 can include high speed random access memory, and can also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid state storage device. In some embodiments, memory 640 can optionally include memory remotely located relative to processor 630, which can be connected to the drone via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

The one or more modules are stored in the memory 640, and when executed by the one or more processors 630, perform a distance measurement method of the drone in any of the above method embodiments, for example, performing the above description The method steps 101 to 103 in FIG. 2, and the functions of the height estimation module 36, the matching module 32, the point-of-same point module 33, and the parallax module 34 in FIG. 4 are implemented.

The above products can perform the methods provided by the embodiments of the present application, and have the corresponding functional modules and beneficial effects of the execution method. For technical details that are not described in detail in this embodiment, reference may be made to the method provided by the embodiments of the present application.

The embodiment of the present application provides a non-transitory computer readable storage medium storing computer-executable instructions that are executed by one or more processors, such as in FIG. The processor 630 is configured to enable the one or more processors to perform the distance measurement method of the drone in any of the foregoing method embodiments, for example, to perform the method steps 101 to 103 in FIG. 2 described above, to implement The functions of the height estimation module 36, the matching module 32, the point-of-same point module 33, and the parallax module 34 shown in FIG. 4 in FIG.

The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, ie may be located A place, or it can be distributed to multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the embodiment.

Through the description of the above embodiments, those skilled in the art can clearly understand that the various embodiments can be implemented by means of software plus a general hardware platform, and of course, by hardware. A person skilled in the art can understand that all or part of the process of implementing the above embodiments can be completed by a computer program to instruct related hardware, and the program can be stored in a computer readable storage medium. When executed, the flow of an embodiment of the methods as described above may be included. The storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, and are not limited thereto; in the idea of the present application, the technical features in the above embodiments or different embodiments may also be combined. The steps may be carried out in any order, and there are many other variations of the various aspects of the present application as described above, which are not provided in the details for the sake of brevity; although the present application has been described in detail with reference to the foregoing embodiments, The skilled person should understand that the technical solutions described in the foregoing embodiments may be modified, or some of the technical features may be equivalently replaced; and the modifications or substitutions do not deviate from the embodiments of the present application. The scope of the technical solution.

Claims (28)

1. A method for measuring distance of a drone, characterized in that it comprises:

   Obtaining a first image captured by the first imaging device on the drone at one time, and a second image captured by the second imaging device on the drone at the time;

   Determining a first pixel block in the first image and a second pixel block in the second image that matches the first pixel block; wherein, in the first pixel block and the second pixel block Either of which includes at least 2 pixels;

   Determining a distance between the drone and the target according to a disparity value of the first pixel block and the second pixel block.
2. The method of claim 1, wherein the determining a first pixel block in the first image and a second pixel block in the second image that matches the first pixel block comprises :

   Determining first piece of feature information of the first pixel block in the first image;

   Determining block feature information of each of the at least one pixel block in the second image; matching the first block feature information with block feature information of each of the pixel blocks;

   A pixel block that matches the block feature information in the at least one pixel block with the first block feature information is used as the second pixel block.
3. The method of claim 1, wherein the determining a first pixel block in the first image and a second pixel block in the second image that matches the first pixel block comprises :

   Determining a first pixel block in the first image;

   Determining at least one pixel block in the second image;

   Pixel matching the first pixel block with the at least one pixel block;

   Determining a second pixel block in the at least one pixel block according to the matching result of the pixel matching.
4. The method according to claim 3, wherein the determining the second pixel block in the at least one pixel block according to the matching result of the pixel matching comprises:

   A pixel block having the highest degree of matching with the pixel of the first pixel block in the at least one pixel block is used as the second pixel block.
5. The method of claim 1, wherein the determining a first pixel block in the first image and a second pixel block in the second image that matches the first pixel block comprises :

   Determining a first pixel point in the first pixel block in the first image;

   Determining at least one pixel block in the second image;

   Matching the first pixel point with a pixel point included in each of the at least one pixel block;

   A pixel block of the at least one pixel block including a pixel point matching the first pixel point is used as the second pixel block.
6. The method according to claim 5, wherein the matching the first pixel point with a pixel point included in each of the at least one pixel block comprises:

   Determining point feature information of the first pixel point;

   Determining point feature information of the pixel points included in each of the pixel blocks;

   The point feature information of the first pixel point is matched with the point feature information of the pixel point included in each of the pixel blocks.
7. The method of claim 5 or claim 6, wherein the first pixel point comprises a central pixel point of the first block of pixels.
8. The method according to any one of claims 1 to 7, wherein the method further comprises:

   Determining first location information of the first pixel block and second location information of the second pixel block;

   Determining a disparity value of the first pixel block and the second pixel block according to the first location information and the second location information.
9. The method according to claim 8, wherein the determining the first location information of the first pixel block and the second location information of the second pixel block comprises:

   Determining first location information of the second pixel in the first pixel block, and second location information of the third pixel in the second pixel block, the second pixel and the third pixel Point matching.
10. The method according to any one of claims 1 to 9, wherein the determining between the drone and the target is based on a disparity value of the first pixel block and the second pixel block Distance, including:

    Determining a distance between the drone and the target according to installation parameters of the first imaging device and the second imaging device, and a disparity value of the first pixel block and the second pixel block .
11. The method according to claim 10, wherein the installation parameters of the first imaging device and the second imaging device comprise at least one of the following:

    The distance between the lens optical center of the first imaging device and the lens optical center of the second imaging device, the distance from the optical center of the first imaging device to the drone body, and the optical center of the second imaging device to the drone The distance of the body.
12. The method according to any one of claims 1 to 11, wherein if the first image includes at least two first pixel blocks, the first pixel block and the second pixel block The parallax value determines the distance between the drone and the target, including:

    Determine at least 2 disparity values;

    Determining at least two distance values according to the at least two disparity values;

    Determining a distance between the drone and the target based on the at least two distance values.
13. The method according to claim 12, wherein the determining the distance between the drone and the target according to the at least two distance values comprises:

    Calculating an average of the at least two distance values and using the average value as a distance between the drone and the target.
14. A drone that includes:

    a first imaging device and a second imaging device;

    And a processor respectively connected to the first imaging device and the second imaging device;

    Wherein the processor is used to:

    Obtaining a first image captured by the first imaging device on the drone at one time, and a second image captured by the second imaging device on the drone at the time;

    Determining a first pixel block in the first image and a second pixel block in the second image that matches the first pixel block; wherein, in the first pixel block and the second pixel block Either of which includes at least 2 pixels;

    Determining a distance between the drone and the target according to a disparity value of the first pixel block and the second pixel block.
15. The drone according to claim 14, wherein said determining a first pixel block in said first image and a second pixel in said second image matching said first pixel block In terms of blocks, the processor is specifically configured to:

    Determining first piece of feature information of the first pixel block in the first image;

    Determining block feature information of each of the at least one pixel block in the second image; matching the first block feature information with block feature information of each of the pixel blocks;

    A pixel block that matches the block feature information in the at least one pixel block with the first block feature information is used as the second pixel block.
16. The method according to claim 14, wherein said determining a first pixel block in said first image and a second pixel block in said second image matching said first pixel block The processor is specifically configured to:

    Determining a first pixel block in the first image;

    Determining at least one pixel block in the second image;

    Pixel matching the first pixel block with the at least one pixel block;

    Determining a second pixel block in the at least one pixel block according to the matching result of the pixel matching.
17. The UAV according to claim 16, wherein in the determining the second pixel block in the at least one pixel block according to the matching result of the pixel matching, the processor is specifically configured to:

    A pixel block having the highest degree of matching with the pixel of the first pixel block in the at least one pixel block is used as the second pixel block.
18. The drone according to claim 14, wherein said determining a first pixel block in said first image and a second pixel in said second image matching said first pixel block In terms of blocks, the processor is specifically configured to:

    Determining a first pixel point in the first pixel block in the first image;

    Determining at least one pixel block in the second image;

    Matching the first pixel point with a pixel point included in each of the at least one pixel block;

    A pixel block of the at least one pixel block including a pixel point matching the first pixel point is used as the second pixel block.
19. The drone according to claim 18, wherein said processor is specific in said matching of said first pixel point with a pixel point included in each of said at least one pixel block Used for:

    Determining point feature information of the first pixel point;

    Determining point feature information of the pixel points included in each of the pixel blocks;

    The point feature information of the first pixel point is matched with the point feature information of the pixel point included in each of the pixel blocks.
20. The drone according to claim 18 or 19, wherein said first pixel point comprises a central pixel point of said first pixel block.
21. The drone according to any one of claims 14 to 20, wherein the processor is further configured to:

    Determining first location information of the first pixel block and second location information of the second pixel block;

    Determining a disparity value of the first pixel block and the second pixel block according to the first location information and the second location information.
22. The drone according to claim 21, wherein said processor is specific to said first position information of said first pixel block and said second position information of said second pixel block Used for:

    Determining first location information of the second pixel in the first pixel block, and second location information of the third pixel in the second pixel block, the second pixel and the third pixel Point matching.
23. The method according to any one of claims 14 to 22, wherein the drone and the target are determined according to the disparity value of the first pixel block and the second pixel block In terms of distance, the processor is specifically used to:

    Determining a distance between the drone and the target according to installation parameters of the first imaging device and the second imaging device, and a disparity value of the first pixel block and the second pixel block .
24. The drone according to claim 23, wherein the installation parameters of said first imaging device and said second imaging device comprise at least one of the following:

    The distance between the lens optical center of the first imaging device and the lens optical center of the second imaging device, the distance from the optical center of the first imaging device to the drone body, and the optical center of the second imaging device to the drone The distance of the body.
25. The drone according to any one of claims 14 to 24, wherein if said first image includes at least two first pixel blocks, said first pixel block and said first The disparity value of the two-pixel block is used to determine the distance between the UAV and the target, and the processor is specifically configured to:

    Determine at least 2 disparity values;

    Determining at least two distance values according to the at least two disparity values;

    Determining a distance between the drone and the target based on the at least two distance values.
26. The method according to claim 25, wherein in the determining the distance between the drone and the target according to the at least two distance values, the processor is specifically configured to:

    Calculating an average of the at least two distance values and using the average value as a distance between the drone and the target.
27. A non-transitory computer readable storage medium, wherein the computer readable storage medium stores computer executable instructions for causing a drone to perform any of claims 1-13 The method described.
28. A computer program product, comprising: a computer program stored on a non-transitory computer readable storage medium, the computer program comprising program instructions, when the program instructions are executed by a drone, A method as claimed in any one of claims 1 to 13.

Images