CN113093176B - Linear obstacle detection method, linear obstacle detection device, electronic apparatus, and storage medium - Google Patents
Linear obstacle detection method, linear obstacle detection device, electronic apparatus, and storage medium Download PDFInfo
- Publication number
- CN113093176B CN113093176B CN201911339038.0A CN201911339038A CN113093176B CN 113093176 B CN113093176 B CN 113093176B CN 201911339038 A CN201911339038 A CN 201911339038A CN 113093176 B CN113093176 B CN 113093176B
- Authority
- CN
- China
- Prior art keywords
- detection
- aerial vehicle
- unmanned aerial
- depth
- point
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S13/00—Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
- G01S13/86—Combinations of radar systems with non-radar systems, e.g. sonar, direction finder
- G01S13/867—Combination of radar systems with cameras
Landscapes
- Engineering & Computer Science (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Computer Networks & Wireless Communication (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Radar Systems Or Details Thereof (AREA)
- Traffic Control Systems (AREA)
Abstract
The application discloses a linear obstacle detection method and device, electronic equipment and a storage medium. The method is applied to the unmanned aerial vehicle, and comprises the following steps: acquiring a detection point and a depth map of a depth detection camera which are detected by a millimeter wave radar under a first posture of the unmanned aerial vehicle; judging whether the detection point and the depth map meet a first condition, if so, controlling the unmanned aerial vehicle to perform attitude adjustment, and acquiring the detection point of the millimeter wave radar and the depth map of the depth detection camera under a second attitude of the unmanned aerial vehicle; determining the number of the matched point pairs according to the detection points and the depth map under the second posture; and judging whether the number of the matching point pairs meets a second condition, and if so, judging that the linear obstacle is detected. This scheme is through fusing two kinds of sensor work with camera and radar, realizes the accurate prediction to the position between linear barriers such as electric wire and the unmanned aerial vehicle, thereby control unmanned aerial vehicle detour avoid with the collision of linear barrier, improved unmanned aerial vehicle flight efficiency and security.
Description
Technical Field
The application relates to the field of unmanned aerial vehicle distribution, in particular to a linear obstacle detection method and device, electronic equipment and a storage medium.
Background
Unmanned aerial vehicle distribution is a future development trend, the line of unmanned aerial vehicle distribution is usually at low altitude, foreseeable barriers are linear, such as power transmission lines and telegraph poles, and the overhead electric wires of China are erected all over the country. The power line detection in the unmanned aerial vehicle image is a special image detection problem, the realization of power line obstacle avoidance through a vision method has great significance, and the problem that how to realize the detection of the power line and avoid false detection is currently required to be solved.
Content of application
In view of the above, the present application is proposed to provide a linear obstacle detection method, apparatus, electronic device and storage medium that overcome or at least partially solve the above problems.
According to one aspect of the present application, there is provided a linear obstacle detection method, which is applied to a drone, which is equipped with a millimeter wave radar and a depth detection camera,
the linear obstacle detection method includes:
acquiring a detection point of a millimeter wave radar and a depth map of a depth detection camera of the unmanned aerial vehicle in a first posture;
judging whether the detection point and the depth map meet a first condition, if so, controlling the unmanned aerial vehicle to perform attitude adjustment, and acquiring the detection point of the millimeter wave radar and the depth map of the depth detection camera under a second attitude of the unmanned aerial vehicle;
determining the number of the matched point pairs according to the detection points and the depth map under the second posture;
and judging whether the number of the matching point pairs meets a second condition, and if so, judging that the linear obstacle is detected.
Optionally, before acquiring the detection point of the millimeter wave radar and the depth map of the depth detection camera in the first posture of the unmanned aerial vehicle, the method further includes:
calibrating the coordinate systems of the depth detection camera and the millimeter wave radar, and then obtaining the projections of the detection points of the millimeter wave radar and the depth map of the depth detection camera in the world coordinate system;
the determining whether the probe point and the depth map satisfy a first condition includes:
if the millimeter wave radar detects a detection point and the depth detection camera does not detect a corresponding matching point at the detection point, the first condition is satisfied.
Optionally, if satisfy then control unmanned aerial vehicle carries out the attitude adjustment, obtains unmanned aerial vehicle at the probe point of millimeter wave radar and the depth map of degree of depth detection camera under the second gesture and includes:
and controlling the unmanned aerial vehicle to decelerate and parallel wind the unmanned aerial vehicle rotates by a preset angle along the coordinate axis of the machine head direction to obtain the detection point of the millimeter wave radar and the depth map of the depth detection camera under the second posture.
Optionally, the determining whether the number of the matching point pairs meets a second condition, and if yes, determining that the linear obstacle is detected includes:
determining the number of matching point pairs with the absolute value of the difference between the depth coordinate value of the detection point and the depth value of the corresponding matching point in the depth map, which is parallel to the detection point coordinate, being smaller than a first threshold value within a preset interval, and if the number of the matching point pairs is smaller than a second threshold value, judging that the linear obstacle is detected;
the first threshold is a distance threshold, the second threshold is a matching point logarithm threshold, the matching point logarithm is determined according to the number of the detection points, and the matching points corresponding to the detection points exist in the depth map.
Optionally, control unmanned aerial vehicle slows down, and the duplex winding unmanned aerial vehicle rotates preset angle along the fuselage to the coordinate axis of aircraft nose direction, acquires unmanned aerial vehicle is in the probe point of millimeter wave radar and the depth map of degree of depth detection camera still include under the second gesture:
and controlling the unmanned aerial vehicle to recover to a first state, then winding the unmanned aerial vehicle to rotate a preset angle in the opposite direction along the coordinate axis of the machine head direction, and acquiring a detection point of the millimeter wave radar and a depth map of the depth detection camera under the preset angle in the opposite direction of the unmanned aerial vehicle.
Optionally, the method further includes:
and if the linear obstacle is judged to be detected, controlling the unmanned aerial vehicle to fly according to the determined pitch angle so as to bypass the linear obstacle.
Optionally, the controlling the drone to fly according to the determined pitch angle so as to bypass the linear obstacle includes:
and controlling the unmanned aerial vehicle to recover to a first state, determining an initial value of the pitch angle according to the mean value of the height values of the detection points and the longitudinal coordinate value of the millimeter wave radar, determining the pitch angle according to the product of the initial value and the safety factor, and controlling the unmanned aerial vehicle to fly according to the pitch angle so as to bypass the linear barrier.
According to another aspect of this application, a linear obstacle detection device is provided, the device is applied to an unmanned aerial vehicle, the unmanned aerial vehicle is equipped with millimeter wave radar and a depth detection camera, the linear obstacle detection device includes:
the acquisition unit is used for acquiring a detection point obtained by the millimeter wave radar detection of the unmanned aerial vehicle in the first posture;
the judging unit is used for judging whether the detection point and the depth map meet a first condition or not, and if so, controlling the unmanned aerial vehicle to perform attitude adjustment to obtain the detection point of the millimeter wave radar and the depth map of the depth detection camera under a second attitude of the unmanned aerial vehicle;
the determining unit is used for determining the number of the matched point pairs according to the detection points and the depth map in the second posture;
and a determination unit which determines whether the number of the matching point pairs satisfies a second condition, and if so, determines that the linear obstacle is detected.
In accordance with yet another aspect of the present application, there is provided an electronic device including: a processor; and a memory arranged to store computer executable instructions that, when executed, cause the processor to perform a method as any one of the above.
According to a further aspect of the application, there is provided a computer readable storage medium, wherein the computer readable storage medium stores one or more programs which, when executed by a processor, implement a method as in any above.
To sum up, according to the linear obstacle detection scheme of unmanned aerial vehicle that this application disclosed, can obtain following technological effect:
the scheme realizes the fusion detection of the depth detection camera and the millimeter wave radar and obtains the maximized matching point; the existence of the obstacle is repeatedly verified by adjusting the unmanned aerial vehicle to switch between different postures, so that the position between the linear obstacle and the unmanned aerial vehicle can be accurately predicted; the unmanned aerial vehicle is controlled to detour in advance, so that collision with linear obstacles such as electric wires is avoided, and the flying efficiency and safety of the unmanned aerial vehicle are improved.
The foregoing description is only an overview of the technical solutions of the present application, and the present application can be implemented according to the content of the description in order to make the technical means of the present application more clearly understood, and the following detailed description of the present application is given in order to make the above and other objects, features, and advantages of the present application more clearly understandable.
Drawings
Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the application. Also, like reference numerals are used to refer to like parts throughout the drawings. In the drawings:
FIG. 1 shows a schematic flow diagram of a linear obstruction detection method according to one embodiment of the present application;
fig. 2 shows a schematic configuration diagram of a linear obstacle detection device according to an embodiment of the present application;
FIG. 3 shows a schematic structural diagram of an electronic device according to an embodiment of the present application;
FIG. 4 shows a schematic structural diagram of a computer-readable storage medium according to an embodiment of the present application;
FIG. 5 shows a schematic view of points of an obstacle in a camera coordinate system according to an embodiment of the present application;
fig. 6 shows a schematic view of a coordinate system and rotation angle of a drone body according to an embodiment of the present application;
fig. 7 shows a schematic diagram of a linear obstacle detection flow according to an embodiment of the present application.
Detailed Description
Exemplary embodiments of the present application will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present application are shown in the drawings, it should be understood that the present application may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
FIG. 1 shows a schematic flow diagram of a linear obstruction detection method according to one embodiment of the present application; the method is applied to an unmanned aerial vehicle, the unmanned aerial vehicle is provided with a millimeter wave radar and a depth detection camera, wherein the depth detection camera is a camera capable of detecting the distance between the camera and a detected object and comprises a binocular camera, a multi-view camera or a depth camera and the like, and the linear obstacle detection method comprises the following steps:
and step S110, acquiring a detection point of the millimeter wave radar and a depth map of the depth detection camera under the first posture of the unmanned aerial vehicle.
In this embodiment, taking an unmanned aerial vehicle equipped with a binocular camera as an example of a depth detection camera, in the case of normal flight of the unmanned aerial vehicle, the unmanned aerial vehicle or an electronic device on the ground is used to acquire point cloud data of a millimeter wave radar in a normal flight attitude (a first attitude) and image data of the depth detection camera, and a detection point and a depth map obtained by respective detection are acquired.
The millimeter wave radar works in a millimeter wave band, generally, the millimeter wave refers to a frequency band of 30 GHz-300 GHz (the wavelength is 1 mm-10 mm), and the wavelength of the millimeter wave is between centimeter wave and light wave, so that the millimeter wave has the advantages of microwave guidance and photoelectric guidance. Compared with the centimeter wave seeker, the millimeter wave seeker has the advantages of being small in size, light in weight and high in spatial resolution. Compared with optical seeker such as infrared, laser, etc., the millimeter wave seeker has strong ability of penetrating fog, smoke and dust, and has all-weather all-day characteristics. In addition, the anti-interference and anti-stealth capabilities of the millimeter wave seeker are also superior to those of other microwave seekers.
The binocular camera is similar to the eyes of human beings, and is different from a depth camera based on TOF and structured light principles, and the depth is calculated by completely depending on two pictures (color RGB or gray scale images) taken without an external active projection light source, and the binocular camera is also called a passive binocular depth camera. The binocular camera acquiring the depth map comprises the following steps: firstly, calibrating a binocular camera to obtain internal and external parameters and a homography matrix of the two cameras; then correcting the original image according to the calibration result, wherein the two corrected images are positioned on the same plane and are parallel to each other; secondly, matching pixel points of the two corrected images, and finally calculating the depth of each pixel according to a matching result so as to obtain a depth map.
And step S120, judging whether the detection point and the depth map meet a first condition, if so, controlling the unmanned aerial vehicle to perform attitude adjustment, and acquiring the detection point of the millimeter wave radar and the depth map of the depth detection camera under a second attitude of the unmanned aerial vehicle.
After the detection point and the depth map are obtained, whether the detection point and the depth map meet a first condition or not is judged, and the fact that a suspected linear obstacle exists in front of the unmanned aerial vehicle can be preliminarily judged according to the first condition.
And in order to confirm whether the suspected linear obstacle exists in the front, controlling the unmanned aerial vehicle to perform attitude adjustment, and acquiring a detection point of the millimeter wave radar for the suspected linear obstacle in the front and a depth map of the binocular camera in the second attitude after adjustment.
And step S130, determining the number of the matched point pairs according to the detection points and the depth map in the second posture.
And comparing and analyzing the detection points and the depth map under the second posture, acquiring the number of matching point pairs which can be mutually corresponding and verified by the detection points and the depth map, and determining the distribution state, the number and other specific conditions of the matching point pairs.
Step S140, determining whether the number of the matching point pairs satisfies a second condition, and if so, determining that a linear obstacle is detected.
And if the number of the matched point pairs determined by the detection points and the depth map can meet a second condition, for example, the number of the matched point pairs is lower than that of the point pairs of a certain threshold value, judging that a linear obstacle parallel to the unmanned aerial vehicle body exists in the advancing direction of the unmanned aerial vehicle. Because the fuselage shape of unmanned aerial vehicle is various, like the shape or the circular of aircraft in fig. 6, consider that the binocular camera generally sets up in the direction of advance just to unmanned aerial vehicle, consequently, can judge whether there is the linear barrier parallel with unmanned aerial vehicle according to whether there is the linear barrier parallel with the binocular camera.
The main application scene of this embodiment is the problem of detecting a linear obstacle parallel to the binocular camera ahead by the unmanned aerial vehicle in flight, and especially, when the unmanned aerial vehicle is detected by the camera alone, the unmanned aerial vehicle is easy to miss detection and collide with the linear obstacle, such as a wire, a height limiting rod and other linear objects. Through the scheme disclosed by the embodiment of the application, the detection of the linear obstacle can be well realized, so that conditions are provided for further treatment such as detour.
According to the embodiment, the accuracy of the unmanned plane for detecting the linear obstacle is obviously improved by adding a detection mode of a millimeter wave radar on the basis of the depth detection camera; the existence of obstacles is repeatedly verified by adjusting the conversion of the unmanned aerial vehicle between different postures, so that the matching point is maximized, and the false detection rate is greatly reduced.
In one embodiment, the step S110 further includes: and calibrating the coordinate systems of the depth detection camera and the millimeter wave radar, and then acquiring the projection of the detection point of the millimeter wave radar and the depth map of the depth detection camera in the world coordinate system.
For convenience of analysis and calculation, the millimeter wave radar may be calibrated to a coordinate system of the depth detection camera, and a rotation matrix of the millimeter wave radar reaching the coordinate system of the depth detection camera is denoted as Trc, specifically referring to fig. 5, a detection point in fig. 5 is a detected point on an obstacle, a z-axis of the camera coordinate system points to a direction of the detected obstacle, and a distance Zw is a depth value from the point to the camera. After the millimeter wave radar is calibrated to the coordinate system of the binocular camera, the binocular camera needs to be further mapped to the world coordinate system, and calculation and judgment are facilitated.
In order to implement the above calibration, firstly, the coordinates of the millimeter wave radar need to be converted into the coordinate system of the binocular camera, and the calibration steps may be as follows: adjusting the millimeter wave radar and the binocular camera, turning to a proper position, and taking a picture and capturing a point cloud at the same position; after the capturing is finished, at least three corresponding point pairs need to be found in the depth map and the point cloud map, the corresponding point pairs respectively correspond to the detection points or the matching points in the point cloud map and the depth map, then PNP solution is carried out by using the three point pairs, and the transformation relation between the coordinate system of the binocular camera and the coordinate system of the millimeter wave radar is calculated. Of course, other calibration methods may be used.
The world coordinate system is used to determine the position of the camera, and in binocular cameras the world coordinate system origin is typically located at the midpoint of the left camera, the right camera, or both in the X-axis direction. From the world coordinate system to the camera coordinate system, rotation and translation of the object are involved. And rotating different angles around different coordinate axes to obtain corresponding rotation matrixes. The method for converting the camera coordinate system to the world coordinate system comprises the following steps: setting a reference coordinate system; setting a rotation matrix R of a camera coordinate system and a world coordinate system; the conversion of the camera coordinate system to the world coordinate system is performed according to the following relationship: [ Xc, Yc, Zc ] × R ═ Xw, Yw, Zw ], where (Xc, Yc, Zc) are coordinates of the camera coordinate system and (Xw, Yw, Zw) are coordinates of the world coordinate system.
The step S120 of determining whether the detection point and the depth map satisfy a first condition includes: if the millimeter wave radar detects a detection point and the depth detection camera does not detect a corresponding matching point at the detection point, the first condition is satisfied.
According to experimental verification, under the condition that the binocular camera is parallel to the linear barrier, the binocular camera cannot detect the linear barrier and cannot effectively acquire a detection point on the linear barrier, and at the moment, only the detection point acquired by the millimeter wave radar when the unmanned aerial vehicle is at the initial position can be acquired.
Therefore, in the flying or driving process of the actual unmanned aerial vehicle, when the millimeter wave radar detects the detection point and the binocular camera does not have a detection matching point or a corresponding image at the corresponding position, the first condition can be determined to be met, and the suspected linear obstacle can be determined to exist at the position. Of course, other conditions may also be adopted to determine whether there is a suspected linear obstacle in front, for example, if the detection point of the millimeter wave radar is preliminarily detected to be a straight line, the first condition is satisfied.
In one embodiment, the step S120 includes: and controlling the unmanned aerial vehicle to decelerate and parallel wind the unmanned aerial vehicle rotates by a preset angle along the coordinate axis of the machine head direction to obtain the detection point of the millimeter wave radar and the depth map of the depth detection camera under the second posture.
In this embodiment, in order to fully utilize the fusion information of the millimeter wave radar and the binocular camera and avoid the situation that only the millimeter wave radar can detect a suspected obstacle, the flying unmanned aerial vehicle is rotated by a preset angle according to the roll angle of the body, so that the depth maps of the millimeter wave radar detection point and the binocular camera in the second posture are obtained, and the suspected linear obstacle is further identified according to the detection point and the depth map in the second posture.
Referring to fig. 6, the coordinate system of the drone is as follows: the fuselage barycenter is used as the original point of the fuselage, the barycenter is used as the Z axis to the coordinate of the nose direction, the barycenter is used as the Y axis to the coordinate of the fuselage below direction, and then the X axis is perpendicular to the unmanned aerial vehicle symmetry plane and points to the right side of the fuselage. In this embodiment, a preset Roll angle a may be rotated about the Z axis so that the depth detection camera such as the binocular camera may also detect the linear obstacle.
In one embodiment, the step S130 includes: and determining the number of matching point pairs with the absolute value of the difference between the depth coordinate value of the detection point and the depth value of the corresponding matching point in the depth map, which is parallel to the detection point coordinate, being smaller than a first threshold value within a preset interval, and if the number of the matching point pairs is smaller than a second threshold value, judging that the linear obstacle is detected.
The first threshold is a distance threshold, the second threshold is a matching point logarithm threshold, the matching point logarithm is determined according to the number of the detection points, and the matching points corresponding to the detection points exist in the depth map.
In this embodiment, by subscribing to depth maps of the millimeter wave radar detection point and the binocular camera, in an (xr-Tx, xr + Tx) interval parallel to the coordinate axis of the suspected linear obstacle, the coordinate relationship thereof is shown in fig. 5, and | Pr-Pc | < Td point number N is counted; and then restoring the Z-axis rotation angle of the unmanned aerial vehicle to be the initial position. Wherein xr is a horizontal coordinate of the radar tracking point; tx is a preset search threshold; pr is a depth coordinate of the millimeter wave radar detection point on a millimeter wave radar coordinate system; pc is the depth value of the matching point corresponding to the detection point obtained according to the depth map of the binocular camera; td is the first threshold value, which is a set distance threshold value, and for example, the first threshold value may be a value within 0.5m, and a preferable value of the first threshold value is 0.2 m.
If N < Tn, it indicates that the suspected linear obstruction is parallel to the binocular module. Wherein Tn is the set point number, i.e. the second threshold, and the second threshold is different according to the difference of the search threshold, preferably, the value range of the search threshold is 1-3 m, and the value range of the second threshold is 40-100 m. This is because the linear obstacle has a linear shape, and has fewer detection points or matching point pairs than other planar obstacles can obtain within a certain distance or area range.
At this moment, steerable unmanned aerial vehicle slows down, and the Z axle of duplex winding unmanned aerial vehicle is rotatory preset roll angle a. To avoid significant wobble of the drone, preferably a may take a value less than 5 degrees.
In this embodiment, the threshold values are set to a value that not only covers the situation under various lighting conditions, but also enables accurate detection or identification of most linear obstacles.
In one embodiment, the step S130 further includes: and controlling the unmanned aerial vehicle to recover to a first state, then winding the unmanned aerial vehicle to rotate a preset angle in the opposite direction along the coordinate axis of the machine head direction, and acquiring a detection point of the millimeter wave radar and a depth map of the depth detection camera under the opposite preset angle.
In order to ensure the accuracy of the detection of the linear obstacle, a verification step is provided in the embodiment, after the unmanned aerial vehicle is rotated and restored to the first state for the first time, the unmanned aerial vehicle is rotated by a preset angle a in the opposite direction around the machine body to the machine head coordinate axis, then a detection point detected by the millimeter wave radar and a depth map detected by the depth detection camera at the opposite preset angle are obtained again, and the same judgment is carried out, so that whether the detection is accurate or not is verified.
Of course, in order to further improve the detection accuracy, the steps can be repeated to realize the third rotation and the second verification, and at this time, the rotation angle of the unmanned aerial vehicle around the Z axis of the camera can be controlled to be (-1)nA, wherein n is the number of times the drone is in the second state.
In one embodiment, the method further comprises: and if the linear obstacle is judged to be detected, controlling the pitching angle of the unmanned aerial vehicle to bypass the linear obstacle.
This embodiment gives a procedure how to cope when a linear obstacle is detected, that is, to control the drone to bypass the linear obstacle at an appropriate pitch angle so as to avoid a collision with the linear obstacle.
In one embodiment, the controlling the pitch angle of the drone to bypass the linear obstruction includes: and controlling the unmanned aerial vehicle to recover to a first state, determining an initial value of the pitch angle according to the mean value of the height values of the detection points and the longitudinal coordinate value of the millimeter wave radar, then obtaining the pitch angle according to the product of the initial value and the safety factor, and then bypassing the linear barrier according to the pitch angle.
The embodiment discloses a method for obtaining a reasonable pitch angle, which comprises the following specific processes:
firstly, the unmanned aerial vehicle is controlled to rotate and return to an initial first state, and at the moment, the unmanned aerial vehicle and the linear barrier are parallel to each other.
Secondly, calculating a theoretical value of a pitch angle that the unmanned aerial vehicle should climb, wherein the calculation method comprises the following steps: according to the interval of the detection points (xr-Tx, xr + Tx), determining that the abscissa of the detection point is xi, obtaining the positions of all the detection points meeting the | Pr-Pc | < Td, setting the ordinate of the detection point as yi, calculating Hg ∑ yi/N, and obtaining the ordinate value Pr of the radar detection point; and calculating a theoretical value arctan (Hg/Pr). F of the pitching angle of the unmanned aerial vehicle according to the Hg and Pr values.
And thirdly, acquiring an actual pitching angle value arctan (Hg/Pr) × F which is actually required to climb according to the theoretical value, wherein F is a set safety factor, and for example, can be 1.5, so that the unmanned aerial vehicle is prevented from colliding with the linear obstacle.
And finally, controlling the unmanned aerial vehicle to climb according to the actual value of the pitch angle so as to avoid colliding with the linear barrier.
Through the scheme, the matching points of the linear obstacles can be maximized by adjusting the postures of the unmanned aerial vehicle under the condition of few matching points, the false detection rate is reduced, and the linear obstacles are avoided by adjusting the pitch angle.
Fig. 2 shows a schematic configuration diagram of a linear obstacle detection device according to an embodiment of the present application; the device is applied to among the unmanned aerial vehicle, and unmanned aerial vehicle is equipped with millimeter wave radar and depth detection camera, and wherein depth detection camera refers to the camera that can detect this camera and detected object distance, including binocular camera, many meshes camera or depth camera etc. linear obstacle detection device 200 includes:
the obtaining unit 210 is adapted to obtain a detection point of the millimeter wave radar and a depth map of the depth detection camera in the first posture of the unmanned aerial vehicle.
Taking an unmanned aerial vehicle loaded with a binocular camera as an example of a depth detection camera, under the condition that the unmanned aerial vehicle flies normally, the unmanned aerial vehicle or an electronic device on the ground is used for acquiring point cloud data of a millimeter wave radar and image data of the depth detection camera under a normal flying attitude (a first attitude), and a detection point and a depth map which are respectively obtained through detection are acquired.
The judging unit 220 is adapted to judge whether the detection point and the depth map satisfy a first condition, and if so, the unmanned aerial vehicle is controlled to perform attitude adjustment to obtain the detection point of the millimeter wave radar and the depth map of the depth detection camera in the second attitude of the unmanned aerial vehicle.
After the detection points and the depth map are acquired, whether the detection points and the depth map meet a first condition or not is judged, and a suspected linear obstacle in front of the unmanned aerial vehicle can be preliminarily judged according to the first condition.
And in order to confirm whether the suspected linear obstacle exists in the front, controlling the unmanned aerial vehicle to perform attitude adjustment, and acquiring a detection point of the millimeter wave radar for the suspected linear obstacle in the front and a depth map of the binocular camera in the second attitude after adjustment.
A determining unit 230 adapted to determine the number of pairs of matching points from the detected points and the depth map in the second pose.
And comparing and analyzing the detection points and the depth map under the second posture, acquiring the number of matching point pairs which can be mutually corresponding and verified by the detection points and the depth map, and determining the distribution state, the number and other specific conditions of the matching point pairs.
The determining unit 240 is adapted to determine whether the number of the matching point pairs satisfies a second condition, and if so, determine that the linear obstacle is detected.
If the number and the distribution of the matching point pairs determined by the detection points and the depth map can meet a second condition, for example, the number of the matching point pairs is less than a certain threshold, it is determined that a linear obstacle parallel to the unmanned aerial vehicle does exist in front of the unmanned aerial vehicle.
The detection device disclosed by the embodiment has the advantages that the millimeter wave radar is added for detection on the basis of the depth detection camera, so that the accuracy of the unmanned plane for detecting the linear obstacle is obviously improved; the existence of obstacles is repeatedly verified by adjusting the conversion of the unmanned aerial vehicle between different postures, so that the matching point is maximized, and the false detection rate is greatly reduced.
In one embodiment, the obtaining unit 210 is further adapted to: and calibrating the coordinate systems of the depth detection camera and the millimeter wave radar, and then acquiring the projection of the detection point of the millimeter wave radar and the depth map of the depth detection camera in the world coordinate system.
For convenience of analysis and calculation, the millimeter wave radar may be calibrated to a coordinate system of the depth detection camera, and a rotation matrix of the millimeter wave radar reaching the coordinate system of the depth detection camera is denoted as Trc, specifically, referring to fig. 5, a detection point is a detected point on an obstacle, an origin of coordinates of the camera coordinate system is Fc, coordinate axes are (Xc, Yc, Zc), an axis Zc points to a direction of the detected obstacle, and a detection point coordinate Zw is a depth value from the point to the camera.
After the millimeter wave radar is calibrated to the coordinate system of the binocular camera, the binocular camera needs to be further mapped to the world coordinate system, and calculation and judgment are facilitated.
The obtaining unit 210 is further adapted to: if the millimeter wave radar detects a detection point and the depth detection camera does not detect a corresponding matching point at the detection point, the first condition is satisfied.
According to experimental verification, under the condition that the binocular camera is parallel to the linear barrier, the binocular camera cannot detect the linear barrier and cannot effectively acquire a detection point on the linear barrier, and at the moment, only the detection point acquired by the millimeter wave radar when the unmanned aerial vehicle is at the initial position can be acquired.
In one embodiment, the determining unit 220 is adapted to: and controlling the unmanned aerial vehicle to decelerate and parallel wind the unmanned aerial vehicle rotates by a preset angle along the coordinate axis of the machine head direction to obtain the detection point of the millimeter wave radar and the depth map of the depth detection camera under the second posture.
Referring to fig. 6, the coordinate system of the drone is as follows: use the fuselage barycenter to be the initial point of fuselage, use the coordinate of barycenter to the aircraft nose direction to be the z axle, use the coordinate of barycenter to the direction of fuselage below to be the y axle, then the x axle is right-hand for perpendicular to unmanned aerial vehicle plane of symmetry and directional fuselage. In this embodiment, a preset Roll angle a may be rotated about the z-axis so that the binocular equal depth detection camera can detect the linear obstacle as well.
In one embodiment, the decision unit 230 is adapted to: and counting the logarithm of matching points of which the absolute value of the difference between the depth coordinate value of the detection point of the millimeter wave radar and the depth value of the corresponding matching point of the depth detection camera and the detection point is smaller than a first threshold value within a preset interval parallel to the coordinates of the detection point, and if the logarithm of the matching points is smaller than a second threshold value, judging that the linear obstacle is detected.
In this embodiment, by subscribing to depth maps of the millimeter wave radar detection point and the binocular camera, in an (xr-Tx, xr + Tx) interval parallel to the coordinate axis of the suspected linear obstacle, the coordinate relationship thereof is shown in fig. 5, and | Pr-Pc | < Td point number N is counted; and then restoring the Z-axis rotation angle of the unmanned aerial vehicle to be the initial position. Wherein xr is a horizontal coordinate of the radar tracking point; tx is a preset search threshold; pr is a depth coordinate of the millimeter wave radar detection point on a millimeter wave radar coordinate system; pc is the depth value of the matched detection point obtained according to the depth map of the binocular camera; td is the set distance threshold.
And if N is less than Tn, controlling the unmanned aerial vehicle to decelerate and rotate a preset roll angle a around the Z axis of the unmanned aerial vehicle. Wherein Tn is the point number of settlement, and in order to avoid appearing the obvious rocking of unmanned aerial vehicle, preferably, a can take a value less than 5 degrees. When N < Tn, the suspected linear obstacle is parallel to the binocular module.
In one embodiment, the decision unit 230 is further adapted to: and controlling the unmanned aerial vehicle to recover to a first state, then winding the unmanned aerial vehicle to rotate a preset angle in the opposite direction along the coordinate axis of the machine head direction, and acquiring a detection point of the millimeter wave radar and a depth map of the depth detection camera under the opposite preset angle.
In order to ensure the accuracy of the detection of the linear obstacle, a verification step is provided in this embodiment, after the unmanned aerial vehicle is rotated and restored to the first state for the first time, the unmanned aerial vehicle is rotated around the preset angle a in the opposite direction to the handpiece coordinate axis Zc along the fuselage, then the detection point obtained by the detection of the millimeter wave radar and the depth map obtained by the detection of the depth detection camera at the opposite preset angle are obtained again, and the same judgment is performed, so as to verify whether the detection is accurate.
In one embodiment, the apparatus further comprises a collision avoidance unit adapted to: and if the linear obstacle is judged to be detected, controlling the pitching angle of the unmanned aerial vehicle to bypass the linear obstacle.
This embodiment presents a unit how to cope when a linear obstacle is detected, i.e., to control the drone to bypass the linear obstacle at an appropriate pitch angle so as to avoid a collision with the linear obstacle.
In one embodiment, the collision avoidance unit is adapted to: and controlling the unmanned aerial vehicle to recover to a first state, determining an initial value of the pitch angle according to the mean value of the height values of the detection points and the longitudinal coordinate value of the millimeter wave radar, then obtaining the pitch angle according to the product of the initial value and the safety factor, and then bypassing the linear barrier according to the pitch angle.
Through foretell anticollision unit, can be though under the few condition of the matching point number of linear barrier, through adjusting the unmanned aerial vehicle gesture for the maximize of matching point number has reduced the false retrieval rate, and avoids linear barrier through the pitch angle adjustment.
In summary, referring to the overall flow diagram of the linear obstacle detection shown in fig. 7, firstly, the projections of the detection points of the millimeter wave radar and the depth maps of the binocular camera in the world coordinate system are calculated, then, whether the number of the detection points in the first state meets a preset first condition is judged, if yes, the unmanned aerial vehicle is controlled to perform attitude adjustment, and the detection points of the millimeter wave radar and the depth maps of the depth detection cameras in the second attitude after the unmanned aerial vehicle rotates along the roll angle are obtained; and judging whether the detection points and the depth map meet a second preset condition, if so, judging that the linear obstacle is detected, and then controlling the pitch angle of the unmanned aerial vehicle to rise to bypass the linear obstacle. This scheme is through fusing two kinds of sensor work with camera and radar, realizes the accurate prediction to the position between linear barriers such as electric wire and the unmanned aerial vehicle, thereby control unmanned aerial vehicle detour avoid with the collision of linear barrier, improved unmanned aerial vehicle flight efficiency and security.
It should be noted that:
the algorithms and displays presented herein are not inherently related to any particular computer, virtual machine, or other apparatus. Various general purpose devices may be used with the teachings herein. The required structure for constructing such a device will be apparent from the description above. In addition, this application is not directed to any particular programming language. It will be appreciated that a variety of programming languages may be used to implement the teachings of the present application as described herein, and any descriptions of specific languages are provided above to disclose the best modes of the present application.
In the description provided herein, numerous specific details are set forth. However, it is understood that embodiments of the application may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description.
Similarly, it should be appreciated that in the foregoing description of exemplary embodiments of the application, various features of the application are sometimes grouped together in a single embodiment, figure, or description thereof for the purpose of streamlining the application and aiding in the understanding of one or more of the various application aspects. However, the disclosed method should not be interpreted as reflecting an intention that: this application is intended to cover such departures from the present disclosure as come within known or customary practice in the art to which this invention pertains. Rather, as the following claims reflect, application is directed to less than all features of a single foregoing disclosed embodiment. Thus, the claims following the detailed description are hereby expressly incorporated into this detailed description, with each claim standing on its own as a separate embodiment of this application.
Those skilled in the art will appreciate that the modules in the devices in an embodiment may be adaptively changed and arranged in one or more devices different from the embodiment. The modules or units or components of the embodiments may be combined into one module or unit or component, and furthermore they may be divided into a plurality of sub-modules or sub-units or sub-components. All of the features disclosed in this specification (including any accompanying claims, abstract and drawings), and all of the processes or elements of any method or apparatus so disclosed, may be combined in any combination, except combinations where at least some of such features and/or processes or elements are mutually exclusive. Each feature disclosed in this specification (including any accompanying claims, abstract and drawings) may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise.
Furthermore, those skilled in the art will appreciate that while some embodiments described herein include some features included in other embodiments, rather than other features, combinations of features of different embodiments are meant to be within the scope of the application and form different embodiments. For example, in the following claims, any of the claimed embodiments may be used in any combination.
The various component embodiments of the present application may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. It will be appreciated by those skilled in the art that a microprocessor or Digital Signal Processor (DSP) may be used in practice to implement some or all of the functions of some or all of the components of the linear obstruction detection device according to embodiments of the present application. The present application may also be embodied as apparatus or device programs (e.g., computer programs and computer program products) for performing a portion or all of the methods described herein. Such programs implementing the present application may be stored on a computer readable medium or may be in the form of one or more signals. Such a signal may be downloaded from an internet website or provided on a carrier signal or in any other form.
For example, fig. 3 shows a schematic structural diagram of an electronic device according to an embodiment of the present application. The electronic device 300 comprises a processor 310 and a memory 320 arranged to store computer executable instructions (computer readable program code). The memory 320 may be an electronic memory such as a flash memory, an EEPROM (electrically erasable programmable read only memory), an EPROM, a hard disk, or a ROM. The memory 320 has a storage space 330 storing computer readable program code 331 for performing any of the method steps described above. For example, the storage space 330 for storing the computer readable program code may comprise respective computer readable program codes 331 for respectively implementing various steps in the above method. The computer readable program code 331 may be read from or written to one or more computer program products. These computer program products comprise a program code carrier such as a hard disk, a Compact Disc (CD), a memory card or a floppy disk. Such a computer program product is typically a computer readable storage medium such as described in fig. 4. FIG. 4 shows a schematic structural diagram of a computer-readable storage medium according to an embodiment of the present application. The computer readable storage medium 400 has stored thereon a computer readable program code 331 for performing the steps of the method according to the application, readable by a processor 310 of an electronic device 300, which computer readable program code 331, when executed by the electronic device 300, causes the electronic device 300 to perform the steps of the method described above, in particular the computer readable program code 331 stored on the computer readable storage medium may perform the method shown in any of the embodiments described above. The computer readable program code 331 may be compressed in a suitable form.
It should be noted that the above-mentioned embodiments illustrate rather than limit the application, and that those skilled in the art will be able to design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The application may be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by one and the same item of hardware. The usage of the words first, second and third, etcetera do not indicate any ordering. These words may be interpreted as names.
Claims (9)
1. The linear obstacle detection method is applied to an unmanned aerial vehicle, the unmanned aerial vehicle is provided with a millimeter wave radar and a depth detection camera, and the linear obstacle detection method comprises the following steps:
acquiring a detection point of a millimeter wave radar and a depth map of a depth detection camera of the unmanned aerial vehicle in a first posture;
judging whether the detection point and the depth map meet a first condition, if so, controlling the unmanned aerial vehicle to perform attitude adjustment, and acquiring the detection point of the millimeter wave radar and the depth map of the depth detection camera under a second attitude of the unmanned aerial vehicle;
determining the number of the matched point pairs according to the detection points and the depth map under the second posture;
judging whether the number of the matching point pairs meets a second condition or not, and if so, judging that a linear obstacle is detected; the determining whether the number of the matching point pairs meets a second condition, and if so, determining that the linear obstacle is detected includes:
determining the number of matching point pairs with the absolute value of the difference between the depth coordinate value of the detection point and the depth value of the corresponding matching point of the detection point in the depth map being smaller than a first threshold value within a preset interval parallel to the coordinates of the detection point, and if the number of the matching point pairs is smaller than a second threshold value, judging that the linear obstacle is detected;
the first threshold is a distance threshold, the second threshold is a matching point logarithm threshold, the matching point logarithm is determined according to the number of the detection points, and the matching points corresponding to the detection points exist in the depth map.
2. The method of claim 1, wherein the obtaining the depth map of the probe point and the depth detection camera of the millimeter wave radar for the drone at the first pose further comprises:
calibrating the coordinate systems of the depth detection camera and the millimeter wave radar, and acquiring the projection of the detection point of the millimeter wave radar and the depth map of the depth detection camera in the world coordinate system;
the determining whether the probe point and the depth map satisfy a first condition includes:
if the millimeter wave radar detects a detection point and the depth detection camera does not detect a corresponding matching point at the detection point, the first condition is satisfied.
3. The method of claim 1, wherein if the first attitude is satisfied, controlling the drone to perform attitude adjustment, and obtaining the depth map of the detection point and the depth detection camera of the millimeter wave radar of the drone in the second attitude comprises:
and controlling the unmanned aerial vehicle to decelerate and parallel wind the unmanned aerial vehicle rotates by a preset angle along the coordinate axis of the machine body in the direction of the machine head to obtain the detection point of the millimeter wave radar and the depth map of the depth detection camera under the second posture of the unmanned aerial vehicle.
4. The method of claim 3, wherein the controlling the drone to decelerate and rotate about a coordinate axis of the drone in a fuselage-to-nose direction by a predetermined angle to obtain the depth map of the detection point and the depth detection camera of the millimeter wave radar at the second pose of the drone further comprises:
and controlling the unmanned aerial vehicle to recover to a first state, then winding the unmanned aerial vehicle to rotate a preset angle in the opposite direction along the coordinate axis of the machine head direction, and acquiring a detection point of the millimeter wave radar and a depth map of the depth detection camera under the preset angle in the opposite direction of the unmanned aerial vehicle.
5. The method of any one of claims 1-4, further comprising:
and if the linear obstacle is judged to be detected, controlling the unmanned aerial vehicle to fly according to the determined pitch angle so as to bypass the linear obstacle.
6. The method of claim 5, wherein the controlling the drone to fly at the determined pitch angle to bypass the line-like obstacle comprises:
and controlling the unmanned aerial vehicle to recover to a first state, determining an initial value of the pitch angle according to the mean value of the height values of the detection points and the longitudinal coordinate value of the millimeter wave radar, determining the pitch angle according to the product of the initial value and the safety factor, and controlling the unmanned aerial vehicle to fly according to the pitch angle so as to bypass the linear barrier.
7. The utility model provides a linear obstacle detection device, its characterized in that is applied to in the unmanned aerial vehicle, unmanned aerial vehicle is equipped with millimeter wave radar and degree of depth detection camera, linear obstacle detection device includes:
the acquisition unit is used for acquiring a detection point obtained by the millimeter wave radar detection of the unmanned aerial vehicle in the first posture;
the judging unit is used for judging whether the detection point meets a first condition or not, and if so, controlling the unmanned aerial vehicle to perform attitude adjustment to obtain the detection point of the millimeter wave radar and the depth map of the depth detection camera under a second attitude of the unmanned aerial vehicle;
the determining unit is used for determining the number of the matched point pairs according to the detection points and the depth map in the second posture;
a determination unit configured to determine whether the number of matching point pairs satisfies a second condition, and if so, determine that a linear obstacle is detected, the determination unit being adapted to:
determining the number of matching point pairs with the absolute value of the difference between the depth coordinate value of the detection point and the depth value of the corresponding matching point of the detection point in the depth map being smaller than a first threshold value within a preset interval parallel to the coordinates of the detection point, and if the number of the matching point pairs is smaller than a second threshold value, judging that the linear obstacle is detected;
the first threshold is a distance threshold, the second threshold is a matching point logarithm threshold, the matching point logarithm is determined according to the number of the detection points, and the matching points corresponding to the detection points exist in the depth map.
8. An electronic device, wherein the electronic device comprises: a processor; and a memory arranged to store computer-executable instructions that, when executed, cause the processor to perform the method of any one of claims 1-6.
9. A computer readable storage medium, wherein the computer readable storage medium stores one or more programs which, when executed by a processor, implement the method of any of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911339038.0A CN113093176B (en) | 2019-12-23 | 2019-12-23 | Linear obstacle detection method, linear obstacle detection device, electronic apparatus, and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911339038.0A CN113093176B (en) | 2019-12-23 | 2019-12-23 | Linear obstacle detection method, linear obstacle detection device, electronic apparatus, and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113093176A CN113093176A (en) | 2021-07-09 |
| CN113093176B true CN113093176B (en) | 2022-05-17 |
Family
ID=76663934
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911339038.0A Active CN113093176B (en) | 2019-12-23 | 2019-12-23 | Linear obstacle detection method, linear obstacle detection device, electronic apparatus, and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113093176B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114047787B (en) * | 2021-12-23 | 2024-04-05 | 普宙科技有限公司 | Unmanned aerial vehicle obstacle avoidance system, unmanned aerial vehicle obstacle avoidance method, unmanned aerial vehicle obstacle avoidance equipment and storage medium |
Family Cites Families (18)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP4193765B2 (en) * | 2004-01-28 | 2008-12-10 | トヨタ自動車株式会社 | Vehicle travel support device |
| CN102696060B (en) * | 2009-12-08 | 2015-01-07 | 丰田自动车株式会社 | Object detection apparatus and object detection method |
| JP6190758B2 (en) * | 2014-05-21 | 2017-08-30 | 本田技研工業株式会社 | Object recognition device and vehicle |
| CN105447459B (en) * | 2015-11-18 | 2019-03-22 | 上海海事大学 | A kind of unmanned plane detects target and tracking automatically |
| JP6473685B2 (en) * | 2015-11-19 | 2019-02-20 | 日立建機株式会社 | Vehicle control device and work machine |
| CN105866790B (en) * | 2016-04-07 | 2018-08-10 | 重庆大学 | A kind of laser radar obstacle recognition method and system considering lasing intensity |
| CN105892489B (en) * | 2016-05-24 | 2019-09-10 | 国网山东省电力公司电力科学研究院 | A kind of automatic obstacle avoiding UAV system and control method based on Multi-sensor Fusion |
| CN106595631B (en) * | 2016-10-25 | 2019-08-23 | 纳恩博（北京）科技有限公司 | A kind of method and electronic equipment of avoiding barrier |
| CN108052111B (en) * | 2017-10-24 | 2021-02-09 | 南京奇蛙智能科技有限公司 | Unmanned aerial vehicle real-time obstacle avoidance method based on binocular vision technology |
| CN108227738B (en) * | 2017-12-28 | 2019-07-19 | 湖北电鹰科技有限公司 | A kind of unmanned plane barrier-avoiding method and system |
| CN108482427A (en) * | 2018-02-22 | 2018-09-04 | 中车长春轨道客车股份有限公司 | A kind of contactless rail vehicle obstacle detection system and method for controlling security |
| CN108594849B (en) * | 2018-04-10 | 2022-05-13 | 深圳市易飞行科技有限公司 | Unmanned aerial vehicle obstacle avoidance method based on millimeter wave radar |
| CN108845321A (en) * | 2018-04-19 | 2018-11-20 | 北京三快在线科技有限公司 | Recognition methods, device and the unmanned smart machine of target object |
| CN108896994A (en) * | 2018-05-11 | 2018-11-27 | 武汉环宇智行科技有限公司 | A kind of automatic driving vehicle localization method and equipment |
| CN108802739B (en) * | 2018-05-31 | 2021-04-16 | 深圳臻迪信息技术有限公司 | Underwater obstacle detection method and detection device |
| CN108646761B (en) * | 2018-07-12 | 2020-07-31 | 郑州大学 | ROS-based robot indoor environment exploration, obstacle avoidance and target tracking method |
| CN109271944B (en) * | 2018-09-27 | 2021-03-12 | 百度在线网络技术（北京）有限公司 | Obstacle detection method, obstacle detection device, electronic apparatus, vehicle, and storage medium |
| CN110349221A (en) * | 2019-07-16 | 2019-10-18 | 北京航空航天大学 | A kind of three-dimensional laser radar merges scaling method with binocular visible light sensor |
-
2019
- 2019-12-23 CN CN201911339038.0A patent/CN113093176B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN113093176A (en) | 2021-07-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20210279444A1 (en) | Systems and methods for depth map sampling | |
| CN112907676B (en) | Calibration method, device and system of sensor, vehicle, equipment and storage medium | |
| CN107329490B (en) | Unmanned aerial vehicle obstacle avoidance method and unmanned aerial vehicle | |
| WO2019119350A1 (en) | Obstacle recognition method and apparatus for unmanned vehicle, and terminal device | |
| CN106444837A (en) | Obstacle avoiding method and obstacle avoiding system for unmanned aerial vehicle | |
| CN111338382B (en) | Unmanned aerial vehicle path planning method guided by safety situation | |
| JP6783950B2 (en) | Obstacle avoidance control method for unmanned aerial vehicles and unmanned aerial vehicles | |
| KR101948569B1 (en) | Flying object identification system using lidar sensors and pan/tilt zoom cameras and method for controlling the same | |
| CN104298248A (en) | Accurate visual positioning and orienting method for rotor wing unmanned aerial vehicle | |
| CN111257882B (en) | Data fusion method and device, unmanned equipment and readable storage medium | |
| CN112052788B (en) | Binocular vision-based environment sensing method and device and unmanned aerial vehicle | |
| CN109213138B (en) | Obstacle avoidance method, device and system | |
| WO2021016854A1 (en) | Calibration method and device, movable platform, and storage medium | |
| CN113013781B (en) | Laser emission and dynamic calibration device, method, equipment and medium based on image processing | |
| TW201734687A (en) | Method and apparatus for controlling aircraft | |
| CN111399535A (en) | Unmanned aerial vehicle obstacle avoidance method and device, unmanned aerial vehicle and storage medium | |
| CN110750153A (en) | Dynamic virtualization device of unmanned vehicle | |
| CN113093176B (en) | Linear obstacle detection method, linear obstacle detection device, electronic apparatus, and storage medium | |
| CN117115784A (en) | Vehicle detection method and device for target data fusion | |
| WO2022179207A1 (en) | Window occlusion detection method and apparatus | |
| CN107886544A (en) | IMAQ control method and device for vehicle calibration | |
| WO2021237535A1 (en) | Collision processing method and device, and medium | |
| Duan et al. | Image digital zoom based single target apriltag recognition algorithm in large scale changes on the distance | |
| CN111736622B (en) | Unmanned aerial vehicle obstacle avoidance method and system based on combination of binocular vision and IMU | |
| CN111192290A (en) | Blocking processing method for pedestrian image detection |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |