WO2021195886A1

WO2021195886A1 - Distance determination method, mobile platform, and computer-readable storage medium

Info

Publication number: WO2021195886A1
Application number: PCT/CN2020/082199
Authority: WO
Inventors: 刘洁; 周游; 覃政科
Original assignee: 深圳市大疆创新科技有限公司
Priority date: 2020-03-30
Filing date: 2020-03-30
Publication date: 2021-10-07
Also published as: CN112771575A

Abstract

A distance determination method, a control terminal, an unmanned aerial vehicle, and a storage medium. The method comprises: obtaining a first distance acquired by a TOF ranging device (S101); obtaining a first image and a second image output by a vision sensor (S102); determining multiple second distances according to the first image and the second image (S103); and determining a target distance according to the multiple second distances and the first distance (S104). The method enables accurate measurement of the distance between a mobile platform and an object.

Description

Distance determining method, movable platform and computer readable storage medium

Technical field

This application relates to the technical field of distance measurement, in particular to a distance determination method, a movable platform, and a computer-readable storage medium.

Background technique

At present, the mobile platform is equipped with a Time Of Flight (TOF) ranging device. The TOF ranging device can emit light pulses from the transmitter to the object, and the receiver can calculate the light pulse from the transmitter to the object, and then The pixel format returns to the runtime of the receiver to determine the distance between the movable platform and the target object. In most scenarios, the distance measured by the TOF distance measuring device is more accurate, but in scenes with highly reflective objects, such as road signs, the TOF distance measuring device will experience periodic aliasing, resulting in the incorrect distance measured by the TOF distance measuring device. Accurate, easy to bring some influence to the movable platform, for example, the distance measured by the TOF distance measuring device for the movable platform to avoid obstacles. If the distance measured by the TOF distance measuring device is not accurate, the movable platform will not be able to Accurate obstacle avoidance cannot guarantee the safety of the movable platform. Therefore, how to accurately measure the distance between the movable platform and the object is a problem to be solved urgently.

Summary of the invention

Based on this, the present application provides a method for determining a distance, a movable platform, and a computer-readable storage medium, aiming to accurately measure the distance between the movable platform and an object.

In the first aspect, the present application provides a method for determining distance, which is applied to a movable platform, wherein the movable platform includes a vision sensor and a TOF ranging device, and the TOF ranging device includes a transmitter for emitting light signals. A device and a receiving device for receiving a light signal reflected by a target object, the method includes:

Acquiring the first distance between the movable platform and the target object collected by the TOF distance measuring device;

Acquiring a first image and a second image of the target object output by the visual sensor;

Determine the second distance between the movable platform and the multiple spatial points on the target object according to the first image and the second image;

The target distance between the movable platform and the target object is determined according to the plurality of second distances and the first distance.

In a second aspect, the present application also provides a movable platform including a vision sensor, a TOF distance measuring device, a memory, and a processor, and the processor is connected to the vision sensor and the TOF distance measuring device;

The TOF distance measuring device includes a transmitting device for transmitting light signals and a receiving device for receiving light signals reflected by a target object;

The memory is used to store a computer program;

The processor is configured to execute the computer program and, when executing the computer program, implement the following steps:

In the third aspect, the present application also provides a computer-readable storage medium, the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor realizes what is provided in the specification of this application. Steps of any distance determination method.

The embodiment of the present application provides a method for determining a distance, a movable platform, and a computer-readable storage medium. The first distance between the movable platform and a target object is determined by the TOF distance measuring device, and the distance of the target object is output by the visual sensor. The first image and the second image, determine the second distance between the movable platform and the multiple spatial points on the target object, and then determine the target between the movable platform and the target object according to the multiple second distances and the first distance distance. Since the distance measurement result between the TOF distance measuring device and the vision sensor is comprehensively considered, the distance between the movable platform and the object can be accurately measured.

It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the application.

Description of the drawings

In order to explain the technical solutions of the embodiments of the present application more clearly, the following will briefly introduce the drawings used in the description of the embodiments. Obviously, the drawings in the following description are some embodiments of the present application. Ordinary technicians can obtain other drawings based on these drawings without creative work.

FIG. 1 is a schematic structural diagram of a movable platform provided by an embodiment of the present application;

2 is a schematic flowchart of steps of a method for determining a distance provided by an embodiment of the present application;

FIG. 3 is a schematic diagram of a scene in which the TOF distance measuring device and the vision sensor in an embodiment of the present application measure the distance between the target object and the movable platform;

4 is a schematic flowchart of sub-steps of the distance determination method in FIG. 1;

Fig. 5 is a schematic flowchart of sub-steps of the distance determination method in Fig. 1;

Fig. 6 is a schematic block diagram of the structure of an unmanned aerial vehicle provided by an embodiment of the present application.

Detailed ways

The technical solutions in the embodiments of the present application will be described clearly and completely in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of them. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative work shall fall within the protection scope of this application.

The flowchart shown in the drawings is only an example, and does not necessarily include all contents and operations/steps, nor does it have to be executed in the described order. For example, some operations/steps can also be decomposed, combined or partially combined, so the actual execution order may be changed according to actual conditions.

Hereinafter, some embodiments of the present application will be described in detail with reference to the accompanying drawings. In the case of no conflict, the following embodiments and features in the embodiments can be combined with each other.

This application provides a distance determination method, a movable platform, and a computer-readable storage medium. The distance determination method is applied to a movable platform. As shown in FIG. 1, the movable platform 100 includes a vision sensor 110 and a time of flight (Time Of Flight). , TOF) distance measuring device 120, TOF distance measuring device 120 includes a transmitting device for emitting light signals and a receiving device for receiving light signals reflected by the target object. The vision sensor 110 may be a monocular vision device or Binocular vision device, the target object is the spot area formed by the light signal emitted by the transmitter of the TOF distance measuring device irradiating the object object, the movable platform and the target object (object object) can be measured by the TOF distance measuring device 120 The first distance between the two images of the target object output by the vision sensor 110 can determine the second distance between the movable platform and the multiple spatial points on the target object, and the second distance between the first distance and the multiple second The distance can accurately determine the distance between the movable platform and the target object (object object).

Among them, movable platforms include drones, mobile robots, and pan-tilt vehicles. Unmanned aerial vehicles include rotary-wing drones, such as quad-rotor drones, hexa-rotor drones, and octo-rotor drones. It is a fixed-wing UAV, or a combination of a rotary-wing type and a fixed-wing UAV, which is not limited here.

Please refer to FIG. 2, which is a schematic flowchart of steps of a method for determining a distance according to an embodiment of the present application. Specifically, as shown in FIG. 2, the distance determination method includes step S101 to step S104.

Step S101: Obtain a first distance between the movable platform and the target object collected by the TOF distance measuring device.

The TOF distance measuring device includes a transmitting device for transmitting light signals and a receiving device for receiving the light signals reflected by the target object. The light source of the transmitting device is an infrared light source, and the transmitting device of the TOF distance measuring device transmits the light signal and records the emission At the moment, when the light signal emitted by the transmitter meets the object, it will form a light spot on the surface of the object, thereby obtaining the target object on the object. At the same time, the light signal is emitted through the surface of the object, and the light reflected by the target object is obtained. Optical signal, the receiving device of the TOF distance measuring device can receive the light signal reflected by the target object, and record the receiving time point. The transmission time point and the receiving time point can be calculated to obtain the optical signal between the movable platform and the target object. The flight time, and then the first distance between the movable platform and the target object can be calculated according to the flight time and the speed of light. Wherein, the target object is the light spot area formed by the light signal emitted by the transmitter of the TOF distance measuring device irradiating the object.

Step S102: Acquire a first image and a second image of the target object output by the vision sensor.

The first image includes the first target image area, the first target image area is the area where the target object is projected on the first image, the second image includes the second target image area, and the second target image area is the target The area where the object is projected on the second image, the position of the center pixel of the first target image area in the first image and the position of the center pixel of the second target image area in the second image are based on the vision sensor and TOF The installation position relationship between the distance measuring devices is determined, and the position of the first target image area in the first image and the position of the second target image area in the second image are based on the installation position between the vision sensor and the TOF distance measuring device The relationship and the first distance are determined. According to the installation position relationship between the vision sensor and the TOF distance measuring device, the rotation matrix and the displacement matrix between the vision sensor and the TOF distance measuring device can be determined.

In some embodiments, the vision sensor includes any one of a monocular vision device and a binocular vision device. If the vision sensor is a monocular vision device, the first image and the second image are separated by a preset time; if the vision sensor If it is a binocular vision device, the first image is the image output by the first camera in the binocular vision device, and the second image is the image output by the second camera in the binocular vision device. The following explanation will be given by taking the vision sensor as a binocular vision device as an example.

Exemplarily, acquiring the internal parameter matrix of the first camera in the binocular vision device and the rotation matrix and displacement matrix between the first camera and the TOF distance measuring device; according to the internal parameter matrix of the first camera and the first camera The rotation matrix and displacement matrix between the TOF distance measuring device and the TOF distance measuring device determine the position point of the center space point of the target object under the angle of view of the first camera; mark the position point in the preset image of the first camera, And determine the three-dimensional position coordinates of the multiple spatial points on the target object according to the first distance; according to the three-dimensional position coordinates of the multiple spatial points, project the multiple spatial points on the target object into the preset image marked with the location point , To obtain the first image containing the first target image area.

Similarly, obtain the internal parameter matrix of the second camera in the binocular vision device and the rotation matrix and displacement matrix between the second camera and the TOF distance measuring device; according to the internal parameter matrix of the second camera and the second camera and The rotation matrix and displacement matrix between the TOF distance measuring device determine the position point of the center space point of the target object under the angle of view of the second camera; mark the position point in the preset image of the second camera, and Determine the three-dimensional position coordinates of the multiple spatial points on the target object according to the first distance; according to the three-dimensional position coordinates of the multiple spatial points, project the multiple spatial points on the target object into a preset image marked with the location point, Obtain a second image containing the second target image area.

It is understandable that, according to the internal parameter matrix of the first camera and the rotation matrix and displacement matrix between the first camera and the TOF distance measuring device, the central spatial point of the target object can be determined in advance according to the angle of view of the first camera. And mark the position point in the preset image of the first camera to obtain the first marked image. Similarly, according to the internal parameter matrix of the second camera and the relationship between the second camera and the TOF distance measuring device in advance Between the rotation matrix and the displacement matrix, determine the position of the center space point of the target object under the angle of view of the second camera, and mark the position in the preset image of the second camera to obtain the second marked image , And then store the first mark image and the second mark image in the memory of the movable platform. When subsequently measuring the distance between the target object and the movable platform, the first mark image and the second mark image can be directly obtained. Real-time determination is needed to reduce the amount of calculation on the movable platform.

In some embodiments, the method of determining the three-dimensional position coordinates of the multiple spatial points on the target object according to the first distance is specifically: obtaining the effective field of view of the light source of the TOF distance measuring device, and according to the effective field of view of the light source and The first distance is to determine the three-dimensional position coordinates of the multiple spatial points on the target object relative to the TOF distance measuring device. Among them, the effective field of view of the light source of the TOF distance measuring device is determined according to the aperture of the light source emission port of the TOF distance measuring device. The light signal emitted by the light source is emitted to the outside through the light source emission port. The larger the aperture of the light source emission port, The larger the effective field angle of the light source and the smaller the aperture of the light source emission port, the smaller the effective field angle of the light source. For example, the effective field angle of the light source is 10°.

As shown in Figures 1 and 3, the light signal emitted by the transmitting device 121 of the TOF distance measuring device 120 irradiates the object Q to form a circular spot area, that is, the target object P, and the receiving device 122 of the TOF distance measuring device 120 receives According to the light signal reflected by the target object, according to the installation position relationship between the first camera 111 and the TOF distance measuring device, the internal parameter matrix and the first distance, the theoretical area P _{1 of the} target object projected in the first image 10 can be determined, According to the installation position relationship between the second camera 112 and the TOF distance measuring device, the internal parameter matrix, and the first distance, the theoretical area P _{2 of the} target object projected in the second image 20 can be determined. It is understandable that the shape of the light spot area may be a circle, an ellipse or a rectangle, which is not specifically limited in this application.

Step S103: Determine a second distance between the movable platform and the multiple spatial points on the target object according to the first image and the second image.

After acquiring the first image and the second image of the target object output by the vision sensor, the depth value between the movable platform and the multiple spatial points on the target object can be determined according to the first image and the second image, that is, The second distance between the movable platform and the multiple spatial points on the target object.

In an embodiment, as shown in FIG. 4, step S103 specifically includes: sub-steps S1031 to S1032.

S1031. Determine, from the first image and the second image, feature point matching pairs corresponding to multiple spatial points on the target object, respectively.

Specifically, the first feature point corresponding to the multiple spatial points on the target object is extracted from the first image based on the preset feature point extraction algorithm; the second image is determined to match the first feature point based on the preset feature point tracking algorithm The second feature point of the target object is obtained, and the matching pairs of feature points corresponding to the multiple spatial points on the target object are obtained; or, based on a preset feature point extraction algorithm, the corresponding multiple spatial points on the target object can be extracted from the second image A first feature point; a second feature point that matches the first feature point is determined from the first image based on a preset feature point tracking algorithm, and a feature point matching pair corresponding to a plurality of spatial points on the target object is obtained.

Among them, the preset feature point extraction algorithm includes at least one of the following: corner detection algorithm (Harris Corner Detection), scale-invariant feature transform (SIFT) algorithm, scale and rotation invariant feature transform (Speeded- Up Robust Features, SURF) algorithm, FAST (Features From Accelerated Segment Test) feature point detection algorithm, preset feature point tracking algorithm includes but not limited to KLT (Kanade-Lucas-Tomasi feature tracker) algorithm.

In some embodiments, the first target image area in the first image and the second target image area in the second image of the target object are determined; the target object is determined from the first target image area and the second target image area. The multiple spatial points correspond to the matching pairs of feature points. By first determining the first target image area in the first image and the second target image area in the second image of the target object, and then determining multiple targets on the target object from the first target image area and the second target image area. The matching pairs of feature points corresponding to the spatial points can reduce the amount of calculation and increase the processing speed.

Wherein, the position of the first target image area in the first image and the position of the second target image area in the second image are determined according to the installation position relationship between the vision sensor and the TOF distance measuring device and the first distance, and according to the vision sensor The installation position relationship with the TOF distance measuring device can determine the rotation matrix and the displacement matrix between the vision sensor and the TOF distance measuring device. The first target image area in the first image and the second target image area in the second image of the target object can be accurately determined through the installation position relationship between the vision sensor and the TOF distance measuring device and the first distance.

In some embodiments, the method for determining the first target image area in the first image and the second target image area in the second image of the target object is specifically: determining the three-dimensional position coordinates of the target object according to the first distance, that is, Obtain the effective field of view of the light source of the TOF distance measuring device, and determine the three-dimensional position coordinates of multiple spatial points on the target object relative to the TOF distance measuring device according to the effective field of view of the light source and the first distance; according to the three-dimensional position coordinates , Project the target object into the first image and the second image to determine the first target image area and the second target image area. The first distance determined by the TOF distance measuring device and the effective field of view angle of the light source can determine the three-dimensional position coordinates of multiple spatial points on the target object, and then based on the three-dimensional position coordinates, the target object can be projected onto the first image and the second image. In the image, the first target image area and the second target image area can be accurately identified.

In some embodiments, if the vision sensor is a binocular vision device, the method of projecting the target object into the first image to determine the first target image area according to the three-dimensional position coordinates of the target object is specifically: acquiring the binocular vision device The internal parameter matrix of the first camera in and the rotation matrix and displacement matrix between the first camera and the TOF distance measuring device; according to the internal parameter matrix of the first camera, the rotation between the first camera and the TOF distance measuring device Matrix and displacement matrix, and the three-dimensional position coordinates of multiple spatial points on the target object, determine the two-dimensional position coordinates of multiple spatial points on the target object in the first camera coordinate system; according to the multiple spatial points on the target object The two-dimensional position coordinates in the first camera coordinate system, the pixel points corresponding to multiple spatial points are marked in the first image, and the area where the circumscribed circle formed by the pixel points corresponding to the multiple spatial points is the first target image area.

Similarly, if the vision sensor is a binocular vision device, the method of projecting the target object into the second image to determine the second target image area according to the three-dimensional position coordinates of the target object is specifically: acquiring the second target image area in the binocular vision device The internal parameter matrix of the second camera and the rotation matrix and displacement matrix between the second camera and the TOF distance measuring device; according to the internal parameter matrix of the second camera, the rotation matrix and the displacement between the second camera and the TOF distance measuring device Matrix and the three-dimensional position coordinates of multiple spatial points on the target object, determine the two-dimensional position coordinates of multiple spatial points on the target object in the second camera coordinate system; according to the multiple spatial points on the target object, the second camera In the two-dimensional position coordinates in the coordinate system, the pixel points corresponding to multiple spatial points are marked in the second image, and the area where the circumscribed circle formed by the pixel points corresponding to the multiple marked spatial points is located is the second target image area.

S1032. Determine a second distance between the movable platform and the plurality of spatial points according to a plurality of matching pairs of the characteristic points.

Through the feature point matching pairs corresponding to the multiple spatial points on the target object, the second distance between the movable platform and the multiple spatial points on the target object can be determined. The following uses a visual sensor as a binocular vision device as an example to explain the process of determining the second distance between the movable platform and the multiple spatial points based on multiple feature point matching pairs.

Specifically, according to the pixels of the two feature points in each feature point matching pair, determine the corresponding pixel difference of each feature point matching pair; obtain the preset focal length and preset binocular distance of the binocular vision device; The focal length, the preset binocular distance, and the pixel difference corresponding to each feature point matching pair are set to determine the second distance between the movable platform and the multiple spatial points. The preset focal length is determined by calibrating the focal length of the binocular vision device, and the preset binocular distance is determined according to the installation positions of the first camera and the second camera in the binocular vision device.

Step S104: Determine a target distance between the movable platform and the target object according to a plurality of the second distances and the first distance.

After the first distance and the multiple second distances are determined, the target distance between the movable platform and the target object may be determined through the first distance and the multiple second distances. Since the distance measurement result between the TOF distance measuring device and the vision sensor is comprehensively considered, the distance between the movable platform and the object can be accurately measured.

In one embodiment, as shown in FIG. 5, step S104 specifically includes: sub-steps S1041 to S1042.

S1041. Determine a credibility index of the first distance according to a plurality of the second distances and the first distance.

After the first distance and the plurality of second distances are determined, the credibility index of the first distance may be determined by the first distance and the plurality of second distances. Among them, the credibility index is used to characterize the accuracy of the first distance between the movable platform and the target object collected by the TOF distance measuring device. The greater the credibility index, the greater the accuracy of the measured first distance. High, and the smaller the credibility index, the lower the accuracy of the measured first distance.

In some embodiments, the target space point is determined from the plurality of space points according to the plurality of second distances and the first distance, wherein the difference between the second distance corresponding to the target space point and the first distance is less than or equal to The preset threshold; the credibility index of the first distance is determined according to the target space point. The preset threshold may be set based on actual conditions, which is not specifically limited in this application, for example, the preset threshold is 0.5 meters. By obtaining a space point whose difference between the second distance and the first distance is less than or equal to a preset threshold from a plurality of space points as the target space point, and based on the target space point, the feasibility of the first distance can be determined accurately and quickly. Letter index.

In some implementations, the method of determining the credibility index of the first distance according to the target space point is specifically: determining the credibility index of the first distance according to the number of target space points and the number of space points, that is, counting the number of space points The space point whose difference between the second distance and the first distance is less than or equal to the preset threshold, that is, the first number of target space points, and the second number of space points are counted, and then the first number accounts for the second number Percentage, and the percentage of the first quantity to the second quantity is used as the credibility index of the first distance. For example, assuming that the number of target space points is 75 and the number of space points is 100, the percentage of the first number to the second number is 75%, so the credibility index of the first distance is 75%.

In some embodiments, the method of determining the credibility index of the first distance according to the target space point is specifically: determining the first weight value according to the pixel coordinates and pixel value of the corresponding feature point of the target space point in the first image; The pixel coordinates and pixel values of the corresponding feature points in the first image or the second image are determined to determine the second weight value; the credibility index of the first distance is determined according to the first weight value and the second weight value, that is, the first distance is calculated. The weight value accounts for the percentage of the second weight value, and the first weight value accounts for the percentage of the second weight value as the credibility index of the first distance. For example, if the first weight value is 0.5 and the second weight value is 0.7, the first weight value accounts for 71.4% of the second weight value. Therefore, the credibility index of the first distance is 71.4%.

It is understandable that if there are multiple target space points, the first weight value is determined in a specific manner: according to the pixel coordinates and pixel values of the corresponding feature points of each target space point in the first image, determine each target space Point weight value, and accumulate the weight value of each target space point to obtain the first weight value; similarly, the method for determining the second weight value is specifically as follows: according to each spatial point in the first image or the second image Corresponding to the pixel coordinates and pixel values of the feature points, the weight value of each spatial point is determined, and the weight value of each spatial point is accumulated to obtain the second weight value.

In some embodiments, according to the pixel coordinates and pixel values of the corresponding feature points of the target space point in the first image or the second image, the method of determining the first weight value is specifically: determining the first weight value of the target object in the first image. A target image area or a second target image area in the second image; the first weight value is determined according to the pixel coordinates and pixel values of the corresponding feature points of the target space point in the first target image area or the second target image area . Similarly, determine the first target image area in the first image or the second target image area in the second image of the target object; according to the corresponding feature points of the spatial point in the first target image area or the second target image area The pixel coordinates and pixel values of, determine the second weight value. Using the feature points in the first target image area or the second target image area to participate in the calculation of the first weight value or the second weight value can reduce the amount of calculation and increase the processing speed.

In some embodiments, the method for determining the weight value of the target space point or the space point is specifically: substituting the pixel coordinates and pixel values of the corresponding feature point of the target space point or the space point in the first image or the second image into the weight value Calculate the formula to get the weight value of the target space point or space point.

Among them, the weight value calculation formula is

I is the pixel value of the target spatial point or the corresponding feature point in the first image or the second image, (u _x , v _x ) is the corresponding target spatial point or the spatial point in the first image or the second image The pixel coordinates of the feature point, (u ₀ , v ₀ ) are the pixel coordinates of the corresponding feature point of the center space point of the target object in the first image or the second image, e is a natural number, σ is a preset coefficient, and σ is based on experience Sure.

Exemplarily, the set containing the target space point is denoted as P, and the set containing the space point is denoted as P′, then the credibility index of the first distance can be calculated according to the following formula:

Among them, x _i is the target space point in the _{set P, x j} is the space point in the set P′,

Is the pixel coordinate of the corresponding feature point of the i-th target space point in the set P in the first image or the second image, I _i is the i-th target space point in the set P in the first image or the second image The pixel value of the corresponding feature point, m is the number of target space points in the set P,

Is the pixel coordinate of the corresponding feature point of the j-th spatial point in the set P′ in the first image or the second image, and I _j is the j-th spatial point in the set P′ in the first image or the second image The pixel value of the corresponding feature point, n is the number of spatial points in the set P′, and n is greater than m, (u ₀ , v ₀ ) is the correspondence of the central spatial point of the target object in the first image or the second image The pixel coordinates of the feature point, e is a natural number, σ is a preset coefficient, and σ is determined based on experience.

Understandably, through

The first weight value can be calculated by

The second weight value can be calculated.

S1042. Determine a target distance between the movable platform and the target object according to the credibility index, a plurality of the second distances, and the first distance.

After the credibility index of the first distance is determined, it is determined whether the credibility index is greater than the preset credibility index; if the credibility index is greater than the preset credibility index, the multiple second distances and the first distance are fused To determine the target distance between the movable platform and the target object. Among them, the preset credibility index can be set according to actual conditions, which is not specifically limited in this application, for example, the preset credibility index is 75%. By performing fusion processing on multiple second distances and first distances to determine the target distance between the movable platform and the target object, the accuracy of the distance between the movable platform and the object can be greatly improved.

In some embodiments, the method of performing fusion processing on the multiple second distances and the first distance is specifically: determining the third distance according to the multiple second distances, and obtaining the first weight coefficient and the third distance of the first distance The second weight coefficient of the; calculate the product of the first distance and the first weight coefficient and calculate the product of the third distance and the second weight coefficient, and the product of the first distance and the first weight coefficient and the third distance and the second weight The product of the coefficients is accumulated to obtain the target distance between the movable platform and the target object. Wherein, the sum of the first weight coefficient and the second weight coefficient is 1, and the first weight coefficient and the second weight coefficient can be set according to actual conditions, for example, the first weight coefficient is 0.5 and the second weight coefficient is 0.5.

In some embodiments, the method for determining the third distance according to the plurality of second distances is specifically: selecting any second distance from the plurality of second distances as the third distance, or calculating the first distance based on the plurality of second distances. The average value of the second distance, and the average value of the second distance as the third distance, or select the smallest second distance and the largest second distance from multiple second distances, and determine the smallest second distance and the largest second distance The average value of the second distance, the average value of the smallest second distance and the largest second distance is taken as the third distance.

In some embodiments, if the credibility index is less than or equal to the preset credibility index, the target distance between the movable platform and the target object is determined according to a plurality of second distances, that is, any one of the plurality of second distances is selected The second distance is used as the target distance between the movable platform and the target object, or the average value of the second distance is calculated based on multiple second distances, and the average value of the second distance is used as the distance between the movable platform and the target object. The target distance, or select the smallest second distance and the largest second distance from multiple second distances, and determine the average value of the smallest second distance and the largest second distance, and combine the smallest second distance and the largest second distance The average value of the second distance is used as the target distance between the movable platform and the target object.

In the distance determination method provided in the specification of this application, the first distance between the movable platform and the target object is determined by the TOF distance measuring device, and the first image and the second image of the target object output by the vision sensor are used to determine the distance between the movable platform and the target object. The second distance between the multiple spatial points on the target object, and then the target distance between the movable platform and the target object is determined according to the multiple second distances and the first distance. Since the distance measurement result between the TOF distance measuring device and the vision sensor is comprehensively considered, the distance between the movable platform and the object can be accurately measured.

Please refer to FIG. 6, which is a schematic block diagram of a movable platform provided by an embodiment of the present application. The mobile platform 200 includes a processor 201, a memory 202, a vision sensor 203, and a TOF distance measuring device 204. The processor 201, the memory 202, the vision sensor 203 and the TOF distance measuring device 204 are connected by a bus 205. The bus 205 is, for example, I2C. (Inter-integrated Circuit) bus. Among them, the mobile platform 200 includes drones, mobile robots, pan-tilt vehicles, etc. The drones can be rotary-wing drones, such as quad-rotor drones, hexa-rotor drones, and octo-rotor drones. It can also be a fixed-wing UAV, or a combination of a rotary-wing type and a fixed-wing UAV, which is not limited here.

Specifically, the processor 201 may be a micro-controller unit (MCU), a central processing unit (CPU), a digital signal processor (Digital Signal Processor, DSP), or the like.

Specifically, the memory 202 may be a Flash chip, a read-only memory (ROM, Read-Only Memory) disk, an optical disk, a U disk, or a mobile hard disk.

Specifically, the TOF distance measuring device 204 includes a transmitting device for transmitting light signals and a receiving device for receiving light signals reflected by the target object.

Wherein, the processor 201 is configured to run a computer program stored in the memory 202, and implement the following steps when the computer program is executed:

Optionally, when the processor 201 realizes that the target distance between the movable platform and the target object is determined according to a plurality of the second distances and the first distance, it is used to realize:

Determining a credibility index of the first distance according to a plurality of the second distances and the first distance;

The target distance between the movable platform and the target object is determined according to the credibility index, a plurality of the second distances, and the first distance.

Optionally, when the processor 201 implements the determination of the credibility index of the first distance according to a plurality of the second distances and the first distance, it is used to implement:

According to the plurality of second distances and the first distance, a target space point is determined from the plurality of space points, wherein the second distance corresponding to the target space point is between the first distance The difference of is less than or equal to the preset threshold;

The credibility index of the first distance is determined according to the target space point.

Optionally, when the processor 201 implements the determination of the credibility index of the first distance according to the target spatial point, it is used to implement:

The credibility index of the first distance is determined according to the number of the target space points and the number of the space points.

Determine the first weight value according to the pixel coordinates and pixel values of the corresponding feature points of the target space point in the first image or the second image;

Determine the second weight value according to the pixel coordinates and pixel values of the corresponding feature points of the spatial point in the first image or the second image;

The credibility index of the first distance is determined according to the first weight value and the second weight value.

Optionally, when the processor 201 determines the first weight value according to the pixel coordinates and pixel values of the corresponding feature points of the target space point in the first image or the second image, it is used to achieve:

Determining a first target image area in the first image or a second target image area in the second image of the target object;

The first weight value is determined according to the pixel coordinates and pixel values of the corresponding feature points of the target space point in the first target image area or the second target image area.

Optionally, when the processor 201 is configured to determine the target distance between the movable platform and the target object according to the credibility index, a plurality of the second distances, and the first distance, accomplish:

Determine whether the credibility index is greater than a preset credibility index;

If the credibility index is greater than the preset credibility index, fusion processing is performed on a plurality of the second distances and the first distances to determine the target distance between the movable platform and the target object.

Optionally, after the processor 201 determines whether the credibility index is greater than a preset credibility index, it is further used to implement:

If the credibility index is less than or equal to the preset credibility index, the target distance between the movable platform and the target object is determined according to a plurality of the second distances.

Optionally, when the processor 201 determines the second distance between the movable platform and the multiple spatial points on the target object according to the first image and the second image, it is used to achieve:

Determining, from the first image and the second image, feature point matching pairs corresponding to multiple spatial points on the target object;

According to a plurality of matching pairs of the characteristic points, a second distance between the movable platform and the plurality of spatial points is determined.

Optionally, when the processor 201 implements the determination of the feature point matching pairs corresponding to the multiple spatial points on the target object from the first image and the second image, it is used to implement:

Determining a first target image area in the first image and a second target image area in the second image of the target object;

From the first target image area and the second target image area, the feature point matching pairs corresponding to the multiple spatial points on the target object are determined.

Optionally, the position of the first target image area in the first image and the position of the second target image area in the second image are based on the difference between the vision sensor and the TOF distance measuring device. The installation position relationship between the two and the first distance is determined.

Optionally, when the processor 201 implements the determination of the first target image area in the first image and the second target image area in the second image of the target object, it is used to implement:

Determining the three-dimensional position coordinates of the target object according to the first distance;

According to the three-dimensional position coordinates, the target object is projected into the first image and the second image to determine the first target image area and the second target image area.

Optionally, the vision sensor 203 includes any one of a monocular vision device and a binocular vision device.

Optionally, if the vision sensor 203 is a monocular vision device, the first image and the second image are separated by a preset time;

If the vision sensor is a binocular vision device, the first image is the image output by the first camera in the binocular vision device, and the second image is the second image in the binocular vision device. The image output by the camera.

It should be noted that those skilled in the art can clearly understand that for the convenience and brevity of the description, the specific working process of the movable platform described above can refer to the corresponding process in the foregoing distance determination method embodiment. Repeat it again.

The embodiments of the present application also provide a computer-readable storage medium, the computer-readable storage medium stores a computer program, the computer program includes program instructions, and the processor executes the program instructions to implement the foregoing implementation The steps of the distance determination method provided in the example.

Wherein, the computer-readable storage medium may be the internal storage unit of the removable platform described in any of the foregoing embodiments, such as the hard disk or memory of the removable platform. The computer-readable storage medium may also be an external storage device of the removable platform, such as a plug-in hard disk equipped on the removable platform, a smart memory card (Smart Media Card, SMC), and Secure Digital (Secure Digital). , SD) card, flash card (Flash Card), etc.

It should be understood that the terms used in the specification of this application are only for the purpose of describing specific embodiments and are not intended to limit the application. As used in the specification of this application and the appended claims, unless the context clearly indicates other circumstances, the singular forms "a", "an" and "the" are intended to include plural forms.

It should also be understood that the term "and/or" used in the specification and appended claims of this application refers to any combination of one or more of the associated listed items and all possible combinations, and includes these combinations.

The above are only specific implementations of this application, but the protection scope of this application is not limited to this. Anyone familiar with the technical field can easily think of various equivalents within the technical scope disclosed in this application. Modifications or replacements, these modifications or replacements shall be covered within the protection scope of this application. Therefore, the protection scope of this application shall be subject to the protection scope of the claims.

Claims

A method for determining distance is applied to a movable platform, wherein the movable platform includes a vision sensor and a TOF ranging device, and the TOF ranging device includes a transmitting device for emitting light signals and a device for receiving light signals from a target object. The device for receiving reflected optical signals is characterized in that the method includes:

Acquiring the first distance between the movable platform and the target object collected by the TOF distance measuring device;

Acquiring a first image and a second image of the target object output by the visual sensor;

Determine the second distance between the movable platform and the multiple spatial points on the target object according to the first image and the second image;

The target distance between the movable platform and the target object is determined according to the plurality of second distances and the first distance.
The distance determining method according to claim 1, wherein the determining the target distance between the movable platform and the target object according to a plurality of the second distances and the first distance comprises:

Determining a credibility index of the first distance according to a plurality of the second distances and the first distance;

The target distance between the movable platform and the target object is determined according to the credibility index, a plurality of the second distances, and the first distance.
The distance determining method according to claim 2, wherein the determining the credibility index of the first distance according to a plurality of the second distances and the first distance comprises:

According to the plurality of second distances and the first distance, a target space point is determined from the plurality of space points, wherein the second distance corresponding to the target space point is between the first distance The difference of is less than or equal to the preset threshold;

The credibility index of the first distance is determined according to the target space point.
The distance determining method according to claim 3, wherein the determining the credibility index of the first distance according to the target space point comprises:

The credibility index of the first distance is determined according to the number of the target space points and the number of the space points.
The distance determining method according to claim 3, wherein the determining the credibility index of the first distance according to the target space point comprises:

Determine the first weight value according to the pixel coordinates and pixel values of the corresponding feature points of the target space point in the first image or the second image;

Determine the second weight value according to the pixel coordinates and pixel values of the corresponding feature points of the spatial point in the first image or the second image;

The credibility index of the first distance is determined according to the first weight value and the second weight value.
The distance determination method according to claim 5, wherein the first weight value is determined according to the pixel coordinates and pixel values of the corresponding feature points of the target space point in the first image or the second image ,include:

Determining a first target image area in the first image or a second target image area in the second image of the target object;

The first weight value is determined according to the pixel coordinates and pixel values of the corresponding feature points of the target space point in the first target image area or the second target image area.
The method for determining the distance according to any one of claims 2 to 6, wherein the determination of the movable platform and the distance is based on the credibility index, a plurality of the second distances, and the first distance. The target distance between the target objects includes:

Determine whether the credibility index is greater than a preset credibility index;

If the credibility index is greater than the preset credibility index, fusion processing is performed on a plurality of the second distances and the first distances to determine the target distance between the movable platform and the target object.
The distance determining method according to claim 7, wherein after determining whether the credibility index is greater than a preset credibility index, the method further comprises:

If the credibility index is less than or equal to the preset credibility index, the target distance between the movable platform and the target object is determined according to a plurality of the second distances.
The method for determining the distance according to any one of claims 1 to 8, wherein the plurality of spaces on the movable platform and the target object are determined according to the first image and the second image. The second distance between points includes:

Determining, from the first image and the second image, feature point matching pairs corresponding to multiple spatial points on the target object;

According to a plurality of matching pairs of the characteristic points, a second distance between the movable platform and the plurality of spatial points is determined.
The distance determination method according to claim 9, wherein the determining from the first image and the second image the feature point matching pairs corresponding to the multiple spatial points on the target object respectively comprises:

Determining a first target image area in the first image and a second target image area in the second image of the target object;

From the first target image area and the second target image area, the feature point matching pairs corresponding to the multiple spatial points on the target object are determined.
The distance determination method according to claim 6 or 10, wherein the position of the first target image area in the first image and the position of the second target image area in the second image It is determined according to the installation position relationship between the vision sensor and the TOF distance measuring device and the first distance.
The distance determining method according to claim 10, wherein said determining the first target image area in the first image and the second target image area in the second image of the target object, include:

Determining the three-dimensional position coordinates of the target object according to the first distance;

According to the three-dimensional position coordinates, the target object is projected into the first image and the second image to determine the first target image area and the second target image area.
The distance determination method according to any one of claims 1 to 12, wherein the visual sensor comprises any one of a monocular vision device and a binocular vision device.
The distance determination method according to claim 13, wherein if the vision sensor is a monocular vision device, the first image and the second image are separated by a preset time;

If the vision sensor is a binocular vision device, the first image is the image output by the first camera in the binocular vision device, and the second image is the second image in the binocular vision device. The image output by the camera.
A movable platform, characterized in that the movable platform comprises a vision sensor, a TOF distance measuring device, a memory and a processor, and the processor is connected to the vision sensor and the TOF distance measuring device;

The TOF distance measuring device includes a transmitting device for transmitting light signals and a receiving device for receiving light signals reflected by a target object;

The memory is used to store a computer program;

The processor is configured to execute the computer program and, when executing the computer program, implement the following steps:

Acquiring the first distance between the movable platform and the target object collected by the TOF distance measuring device;

Acquiring a first image and a second image of the target object output by the visual sensor;

Determine the second distance between the movable platform and the multiple spatial points on the target object according to the first image and the second image;

The target distance between the movable platform and the target object is determined according to the plurality of second distances and the first distance.
The movable platform according to claim 15, wherein the processor realizes the determination of the target distance between the movable platform and the target object according to a plurality of the second distances and the first distance When used to achieve:

Determining a credibility index of the first distance according to a plurality of the second distances and the first distance;

The target distance between the movable platform and the target object is determined according to the credibility index, a plurality of the second distances, and the first distance.
The mobile platform according to claim 16, wherein when the processor realizes the determination of the credibility index of the first distance according to a plurality of the second distances and the first distance, it is used to realize:

According to the plurality of second distances and the first distance, a target space point is determined from the plurality of space points, wherein the second distance corresponding to the target space point is between the first distance The difference of is less than or equal to the preset threshold;

The credibility index of the first distance is determined according to the target space point.
The mobile platform according to claim 17, wherein when the processor realizes the determination of the credibility index of the first distance according to the target spatial point, it is used to realize:

The credibility index of the first distance is determined according to the number of the target space points and the number of the space points.
The mobile platform according to claim 17, wherein when the processor realizes the determination of the credibility index of the first distance according to the target spatial point, it is used to realize:

Determine the first weight value according to the pixel coordinates and pixel values of the corresponding feature points of the target space point in the first image or the second image;

Determine the second weight value according to the pixel coordinates and pixel values of the corresponding feature points of the spatial point in the first image or the second image;

The credibility index of the first distance is determined according to the first weight value and the second weight value.
The movable platform according to claim 19, wherein the processor realizes the determination of the first image according to the pixel coordinates and pixel values of the corresponding feature points of the target space point in the first image or the second image. When a weight value is used, it is used to achieve:

Determining a first target image area in the first image or a second target image area in the second image of the target object;

The first weight value is determined according to the pixel coordinates and pixel values of the corresponding feature points of the target space point in the first target image area or the second target image area.
The movable platform according to any one of claims 16 to 20, wherein the processor realizes the determination of the available distance based on the credibility index, a plurality of the second distances, and the first distance. The target distance between the mobile platform and the target object is used to achieve:

Determine whether the credibility index is greater than a preset credibility index;

If the credibility index is greater than the preset credibility index, fusion processing is performed on a plurality of the second distances and the first distances to determine the target distance between the movable platform and the target object.
The mobile platform according to claim 21, wherein after the processor realizes the determination of whether the credibility index is greater than a preset credibility index, it is further configured to realize:

If the credibility index is less than or equal to the preset credibility index, the target distance between the movable platform and the target object is determined according to a plurality of the second distances.
The movable platform according to any one of claims 15 to 22, wherein the processor is configured to determine the relationship between the movable platform and the target object according to the first image and the second image. The second distance between multiple spatial points is used to achieve:

Determining, from the first image and the second image, feature point matching pairs corresponding to multiple spatial points on the target object;

According to a plurality of matching pairs of the characteristic points, a second distance between the movable platform and the plurality of spatial points is determined.
The movable platform according to claim 23, wherein the processor implements to determine from the first image and the second image that the characteristic points corresponding to the multiple spatial points on the target object are matched and timed. To achieve:

Determining a first target image area in the first image and a second target image area in the second image of the target object;

From the first target image area and the second target image area, the feature point matching pairs corresponding to the multiple spatial points on the target object are determined.
The movable platform according to claim 20 or 24, wherein the position of the first target image area in the first image and the position of the second target image area in the second image It is determined according to the installation position relationship between the vision sensor and the TOF distance measuring device and the first distance.
The movable platform according to claim 24, wherein the processor implements the determination of a first target image area of the target object in the first image and a second target in the second image When the image area is used to achieve:

Determining the three-dimensional position coordinates of the target object according to the first distance;

According to the three-dimensional position coordinates, the target object is projected into the first image and the second image to determine the first target image area and the second target image area.
The movable platform according to any one of claims 15 to 26, wherein the vision sensor comprises any one of a monocular vision device and a binocular vision device.
28. The movable platform of claim 27, wherein if the vision sensor is a monocular vision device, the first image and the second image are separated by a preset time;

If the vision sensor is a binocular vision device, the first image is the image output by the first camera in the binocular vision device, and the second image is the second image in the binocular vision device. The image output by the camera.
A computer-readable storage medium, characterized in that, the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor realizes as described in any one of claims 1 to 14 The distance determination method described.