JP2022067672A - Landing control device of flying object - Google Patents

Abstract

To provide a novel landing control device of a flying object which can safely land a drone without using a landmark group.SOLUTION: A plurality of feature point detection sensors (long-distance narrow-angle space sensor 13, medium-distance wide-angle space sensor 14, short-distance wide-angle space sensor 15) are installed downward in a flying object. Relative positions of feature points, which exist on a landing surface, measured by the feature point detection sensors are measured in response to a change in altitude, reliability of the feature point detection sensors is set depending on altitude, and whether or not the feature points on the landing surface have changed is determined.SELECTED DRAWING: Figure 2

Description

The present invention relates to a landing control device for an air vehicle, and more particularly to a landing control device for an air vehicle that autonomously lands the air vehicle.

Recently, a system has been proposed in which a system for transporting luggage to a destination using an unmanned aerial vehicle called a drone that takes off and landing in a direction perpendicular to the landing surface. As this drone, a multi-rotor system having a plurality of blade rotors is generally used.

This drone transport system inputs data that represents the flight plan path of the drone's horizontal plane, obtains a height reference value that represents the elevation of the surface below each of the multiple positions on the flight plan path, and that position. By using the value obtained by adding the flight altitude corresponding to the above to the height reference value as the altitude data of the flight plan route, it is possible to fly on the flight plan route without colliding with an obstacle.

In such a drone transport system, it is important that the drone can land safely when it arrives at the destination and lands. Therefore, as a landing control device for a drone, for example, the method described in Japanese Patent Application Laid-Open No. 2019-51741 (Patent Document 1) has been proposed. In Patent Document 1, the drone is guided above the ground station, the position on the three-dimensional space is estimated from the image of the landmarks arranged on the landing surface by the on-board camera, and the landing surface is estimated based on this. It guides with high accuracy.

That is, when the drone is guided above the ground station and the landmark group is imaged in the captured image, the area occupied by the entire screen of the captured image is calculated, and when the calculated area is equal to or larger than the predetermined area, the landmark group is calculated. Is sequentially switched, the position of the unmanned aircraft in the three-dimensional space with respect to the ground station is estimated from the captured images of the switched landmarks, and then the landing motion range of the unmanned aircraft is calculated, and the landing motion range is calculated. Based on this, the drone is guided and controlled to the landing surface.

Japanese Unexamined Patent Publication No. 2019-51741

By the way, in the landing control device described in Patent Document 1, it is necessary to prepare a plurality of landmark groups as reference positions in the ground station, and the cost of system construction includes the installation cost and the maintenance cost thereof. Has the problem of being expensive. Furthermore, since landing can only be performed at a specific landing point called a ground station, there is a problem that the system is not easy to use. Therefore, in order to land the ground station without arranging the landmarks, it is conceivable to recognize the place where the landing is possible autonomously from the sensor camera mounted on the drone and land.

However, at least, (1) when the altitude of the drone changes to a lower altitude and the landing range deviates from the sensing range of the sensor camera, or (2) a dangerous object moves within the landing range, or (3) the position. -When it becomes difficult to track the feature points required for attitude control, a phenomenon occurs in which the drone does not reach the intended landing position. For this reason, there is a problem that it becomes difficult to land the drone safely.

An object of the present invention is to provide a novel landing control device for an air vehicle capable of safely landing the air vehicle without using a group of landmarks.

Although this air vehicle for carrying luggage is unmanned, it is being developed in the future so that it can be deployed in manned air vehicles (so-called flying vehicles). Therefore, the present invention proposes a landing control device that can be applied not only to unmanned aircraft but also to manned aircraft. In addition, the flying object is not limited to the multi-rotor system, but other flying objects that fly autonomously are also targeted.

The features of the present invention are a plurality of feature point detecting means for detecting peripheral feature points including a landing point, a feature point recognizing means for recognizing feature points detected by the feature point detecting means, and an altitude of an air vehicle. An aircraft landing control device equipped with an altitude measuring means for measuring an altitude and a landing control means for automatically landing the aircraft at a landing point.
The landing control means measures the relative positions of a plurality of the feature points according to the altitude, and selects the feature point detecting means for measuring each of the feature points according to the altitude, and changes the relative position of the feature points. It is in the landing control device of the aircraft that determines whether or not there is a change in the shape of the landing point and moves the aircraft to a new landing point if there is a change in the shape.

According to the present invention, the flying object can be safely landed without using a group of landmarks.

It is explanatory drawing explaining the structure of an air vehicle and the measurement range of the space sensor provided in this structure. It is a block diagram which shows the structure of the flying object control apparatus which becomes embodiment of this invention. It is a flowchart which shows the control flow for demonstrating the operation mode of the flying object control apparatus shown in FIG. It is a flowchart which shows the control flow for demonstrating the operation of the landing control device in the flying object control device shown in FIG. It is explanatory drawing for demonstrating the state which measures the ground surface when an air vehicle reaches a landing point. It is a flowchart explaining the 3D shape measurement flow which generates 3D shape in step S32 of FIG. It is explanatory drawing for demonstrating the information of the point cloud which forms 3D shape. It is a flowchart explaining the reliability setting flow which sets the reliability in step S52 of FIG. It is explanatory drawing explaining the method of obtaining the reliability in FIG. It is a flowchart explaining the plane / obstacle recognition flow which recognizes a plane / obstacle (unevenness) in step S32 of FIG. It is explanatory drawing explaining the form of the grouped obstacle shown in FIG. It is explanatory drawing explaining the information of the obstacle shown in FIG. It is a flowchart explaining the feature point detection flow which detects the feature point in step S32 of FIG. It is explanatory drawing for demonstrating the information of the feature point shown in FIG. It is a flowchart explaining the feature point relative position detection flow which detects the feature point relative position in step S32 of FIG. It is explanatory drawing explaining the information of the relative position group of the feature point shown in FIG. It is a flowchart explaining the danger judgment flow which performs the danger judgment whether or not there is an obstacle at the landing point in step S41 of FIG. It is a flowchart explaining the operation state management flow which corrects the landing point in step S42 of FIG.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings, but the present invention is not limited to the following embodiments, and various modifications and applications are included in the technical concept of the present invention. Is also included in that range.

FIG. 1 shows a schematic configuration of the flying object 10 used in the present invention. The flying object 10 is provided with four blade rotors 12 at positions symmetrical to each other in a rectangular housing body 11, and each blade rotor 12 is driven by an electric motor (not shown). The flying object of the present embodiment is not limited to this, and in short, any flying object that can take off and land in the vertical direction may be used.

A long-distance narrow-angle space sensor 13 is provided near the center of the housing body 11, and a medium-distance wide-angle space sensor 14 is provided on one of the left and right ends of the housing body 11, and the left and right sides of the housing body 11 are provided. A short-distance wide-angle space sensor 15 is provided on the other of both ends. These sensors 13 to 15 are characterized by various stationary and moving objects existing on the ground surface by using a sensor camera that images the ground surface and a lidar (LIDAR) that groupes the shapes of the ground surface into points. Features that detect points Functions as a point detection sensor.

In this embodiment, a three-dimensional rider is used, and the description will be described below as a “spatial sensor”. Further, although not shown, a well-known GPS sensor is mounted on the housing body 11 in order to detect the position of the flying object 10.

Further, the housing body 11 is provided with an altitude sensor 16 to detect the altitude above ground level. As the altitude sensor 16, a laser altimeter or a radar altimeter can be used, and the distance between the flying object 10 and the ground surface is measured by the reflection of laser light or microwave, and the altitude above ground level of the flying object 10 is measured. be able to. It is also possible to calculate the altitude above ground level from the altitude output by the barometer or GPS sensor.

The long-distance narrow-angle space sensor 13 is a space sensor having a narrow angle of view as shown in the measurement range A, and the medium-range wide-angle space sensor 14 is a long-range narrow-angle space sensor 13 having a narrow angle of view as shown in the measurement range B. It is a wider space sensor, and the short-range wide-angle space sensor 15 is a space sensor having a wider angle of view than the medium-range wide-angle space sensor 14 as shown in the measurement range C.

As will be described later, the point cloud information obtained by each of the spatial sensors 13 to 15 is given a high degree of reliability. For example, as the altitude of the flying object 10 decreases, the reliability of the point cloud information from the long-distance narrow-angle space sensor 13 changes from "large" to "small", and conversely, the medium- and short-distance wide-angle space sensors 14 The reliability of the point cloud information from 15 is changed from "small" to "large". It should be noted that the magnitude of the reliability includes a case where it is continuously adjusted and a case where it is adjusted like a switch. In other words, the change in reliability can be regarded as the spatial sensor being switched depending on the altitude.

Although three spatial sensors are used in the above description, two spatial sensors may be used if necessary. Further, the arrangement position of each space sensor 13 to 15 is arbitrary, and each space sensor 13 to 15 may be arranged at a distant position as shown in FIG. 1, or may be arranged close to each other as shown in FIG. 5 described later. It may be arranged.

Returning to FIG. 1, the housing body 11 of the flying object 10 is provided with a flying object control device (not shown), and the flying object control device is a route representing a flight planning path in the horizontal plane of the flying object. The flight plan is the value obtained by adding the flight altitude corresponding to the flight position to the height reference value from the information and the height reference value representing the altitude of the ground surface under each of the multiple positions on the flight plan route. By using it as altitude information of the route, it is possible to fly the flight plan route without colliding with obstacles.

On the other hand, the landing control device targeted by the present invention is also mounted on the flight object control device. This landing control device can be provided functionally integrally with the flight object control device, and can be provided separately from the flight object control device. Therefore, an appropriate configuration may be selected according to the specifications of the flight object control device.

Next, the landing control device according to the present invention will be described. FIG. 2 shows the configuration of the flight object control device 20 in which the functions of the landing control device are integrated.

The point cloud information from the long-distance narrow-angle space sensor 13, the medium-distance wide-angle space sensor 14, and the short-distance wide-angle space sensor 15 and the altitude information from the altitude sensor 16 described above are input to the environment recognition unit 21. That is, the point cloud information from each of the spatial sensors 13 to 15 is input to the 3D shape measuring unit 22, and the altitude information from the altitude sensor 16 is input to the sensor reliability setting unit 23. The sensor reliability setting unit 23 sets the reliability of the point cloud information input to the 3D shape measurement unit 22. Here, "3D" means "three-dimensional".

As described above, the sensor reliability setting unit 23 has a function of setting the reliability corresponding to the altitude to the point cloud information from each of the spatial sensors 13 to 15 corresponding to the altitude. As described above, the point cloud information obtained by each of the spatial sensors 13 to 15 is given a high degree of reliability. For example, as the altitude of the flying object 10 decreases, the reliability of the point cloud information from the long-distance narrow-angle space sensor 13 changes from "large" to "small", and conversely, the medium- and short-distance wide-angle space sensors 14 The reliability of the point cloud information from 15 is changed from "small" to "large".

Further, the 3D shape measurement unit 22 is input with the 3D shape information of the ground surface from the 3D shape storage unit 24, and conversely, the 3D shape information measured by the 3D shape measurement unit 22 is input to the 3D shape storage unit 24. It has been newly entered (updated).

The environment recognition unit 21 is further provided with a flat surface / unevenness recognition unit 25, a feature point detection unit 26, a feature point storage unit 27, a feature point relative position calculation unit 28, and a relative value group storage unit 29.

The plane / unevenness recognition unit 25 has a function of recognizing the plane information and unevenness information of the ground surface based on the point cloud information input to the 3D shape measuring unit 22.

Further, the feature point detection unit 26 is based on the point cloud information input to the 3D shape measurement unit 22 and the plane information and unevenness information of the plane / unevenness recognition unit 25, and the plane information and unevenness from the target point cloud information. It has a function to detect feature points of information. The feature points detected by the feature point detection unit 26 are temporarily stored in the feature point storage unit 27.

The feature point relative position calculation unit 28 has a function of calculating a relative position (three-dimensional position) between a plurality of feature points stored in the feature point storage unit 27. Further, the relative position group storage unit 29 has a function of grouping and storing feature points (for example, the shape of the uneven portion) related to each other from the feature points calculated by the feature point relative position calculation unit 28. The relative position group information of the relative position group storage unit 29 is fed back to the 3D shape measuring unit 22 and the sensor reliability setting unit 23.

The plane information and unevenness information from the plane / unevenness recognition unit 25, the feature point information from the feature point detection unit 26, and the position / attitude information from the position / attitude recognition unit 31 are input to the danger determination unit 30. Here, the position / attitude information represents the position and inclination of the flying object 10 in the global coordinate system.

The position / attitude recognition unit 31 is input with the three-dimensional shape information from the 3D shape storage unit 24 and the position / attitude information from the position / attitude sensor 32. The position / attitude sensor 32 detects the position of the flying object 10 and parameters around the rotation axis of the flying object 10, such as “yaw”, “roll”, and “pitch” of the flying object 10.

The danger determination unit 30 determines whether the landing point of the flying object 10 is flat or has an obstacle (unevenness), and if there is an obstacle (unevenness), the landing can be safely landed. It has a function to calculate the point.

Danger information and landing point information from the danger determination unit 30, and position / attitude information from the position / attitude recognition unit 31 are input to the operation state management unit 33. Based on this information, the operating state management unit 33 manages the operating state of the flying object 10. Here, the operating state (mode) manages five operating states of "landing mode", "takeoff mode", "cruising mode", "approaching mode", and "standby mode".

In the "landing mode" and "takeoff mode", the flight object 10 is vertically moved and turned, in the "cruising mode", the flight is performed along the flight plan route according to the map database 34, and in the "approach mode", a predetermined position is executed. It has a function to manage the operation state of executing a move to a specified point (for example, a landing point) and executing hovering without moving in the "standby mode".

In the position / posture control unit 35, the operation state management information from the operation state management unit 33, the map information from the map database 34, and the position / posture recognition information from the position / posture recognition unit 31 are input. Further, the position / posture control unit 35 generates target position / posture information by the target position / posture generation unit 36, and the target position / posture information is input to the follow-up control unit 37. The follow-up control unit 37 flies corresponding to any one of "landing mode", "takeoff mode", "cruising mode", "approach mode", and "standby mode" selected based on the target position / attitude information. It has a function to operate the body 10 autonomously.

The above are the main functional elements of the flying object control device 20, and these functional elements are basically operated by executing the control program of the microcomputer. As is well known, microcomputers input a central processing unit (CPU) that executes operations, a non-volatile memory (ROM) that stores control programs, a volatile memory (RAM) that stores calculation results, and sensor signals. It is composed of an input / output circuit that outputs a drive signal and the like. Hereinafter, the control flow of the main functional elements of the flight object control device 20 will be described.

[Operation mode setting control flow]
First, FIG. 3 shows the control flow of the position / attitude control unit 35, thereby setting the operation mode of the flying object 10.

<< Step S10 >>
In step S10, the current position / posture information is acquired. The acquisition of the position / posture information can be obtained from the position / posture sensor 32 and the position / posture recognition unit 31. The position / attitude recognition unit 31 can obtain the current position / attitude by using, for example, the SLAM (Simultaneous Localization and Mapping) method, and the flying object 10 is flying in a certain state in this state. Can be recognized. When the current position / posture information is acquired, the process proceeds to step S11.

≪Step S11≫
In step S11, in order to determine which operating state (the above-mentioned five operating modes) the current flight state of the flying object 10 is, the operating state set by the operating state management unit 33 at the present time is acquired. do. When the current operating state is acquired, the process proceeds to step S12.

≪Step S12≫
In step S12, it is determined whether or not the operating state acquired from the operating state management unit 33 is the "cruising mode". In this determination step, the determination is made based on the information on the operation state sequentially sent from the operation state management unit 33. If it is determined in this determination step that the state is in the "cruising mode", the process proceeds to step S13, and if it is determined that the state is not in the "cruising mode", the process proceeds to step S14.

≪Step S13≫
In step S13, the current position / posture acquired in step S10 is matched with the map information from the map database 34, and the target position / posture transfer information is set. In this state, if it is determined to be "no" in the following control steps S14, S18, and S20, the process proceeds to step S22 and the flying object 10 is controlled according to the target position / attitude information. As a result, the flying object 10 heads for the destination in the "cruising mode".

<< Step S14 >>
When the control steps of steps S12 and S13 are executed, in this step S14, it is determined whether or not the operation state acquired from the operation state management unit 33 is the "approach mode". If it is determined in this determination step that the state is in the "approach mode", the process proceeds to step S15, and if it is determined that the state is not in the "approach mode", the process proceeds to step S18.

≪Step S15≫
Since the "approach mode" is determined in step S14, it is determined in step S15 whether or not there is a landing request for the aircraft 10. The landing request can be determined by whether or not there is an output of the landing point from the operation state management unit 33. In step S15, if it is determined that there is an output at the landing point, the process proceeds to step S16, and if it is determined that there is no output at the landing point, the process proceeds to step S17.

≪Step S16≫
In step S16, since it is determined that there is a landing request in step S15, the target position / attitude information set in step S13 is reset as landing point information above the new landing point. In this state, if it is determined to be "no" in the following control steps S18 and S20, the process proceeds to step S22, and the flight body 10 is controlled to fly over the landing point according to the landing point information. As a result, the aircraft 10 will head toward the landing point in the "approach mode".

≪Step S17≫
On the other hand, since it is determined in step S15 that there is no output of the landing point, the target position / attitude information is newly set from the map database. In this state, if it is determined to be "no" in the following control steps S18 and S20, the process proceeds to step S22 and the flying object 10 is controlled according to the target position / attitude information.

≪Step S18≫
When the landing point information or the target position / attitude information is set in steps S16 and S17, in step S18, it is determined whether or not the operation state acquired from the operation state management unit 33 is the "standby mode". If it is determined in this determination step that the state is in the "standby mode", the process proceeds to step S19, and if it is determined that the state is not in the "standby mode", the process proceeds to step S20.

≪Step S19≫
In step S19, since the aircraft 10 arrives at the landing point and is set to the "standby mode", the current position / attitude information is reset to the new target position / attitude information. In this state, the flying object 10 will hover and stand by in the sky above the landing point. Next, in this state, the process proceeds to step S20.

≪Step S20≫
When the control steps of steps S18 and S19 are executed, in this step S20, it is determined whether or not the operating state is in the "landing mode" or the "takeoff mode". If it is determined that it is not in the "landing mode" or "takeoff mode", the process proceeds to step S22, and if it is determined that it is in the "landing mode" or "takeoff mode", the process proceeds to step S21.

≪Step S21≫
In step S21, the current position / posture information is maintained as it is, and the point moved in the vertical direction is set as the target position / posture. As a result, the flying object 10 is moved from the hovering state in the ascending direction or the descending direction. When the target position / posture moved in the vertical direction is set, the process proceeds to step S22.

≪Step S22≫
In step S22, the target point of the flying object 10 is controlled by controlling the operating state of the flying object 10 based on the target position / attitude information set in any one of steps S13, S16, S17, S19, and S21. Can be flown to.

The above is a rough control flow of the position / attitude control unit 35 of the flight object control device 20. Next, a specific control flow of the landing control device targeted by the present invention will be described. In the following, the overall control flow of the landing control device will be described, and then the main individual control flows executed in the overall control flow will be described.

[Overall landing control flow]
The overall landing control flow is as shown in FIG. 4, and the control steps S20 to S22 shown in FIG. 3 are shown in more detail. Hereinafter, the overall landing control flow will be described with reference to FIG.

<< Step S30 >>
In step S30, the flight object 10 is moved directly above the landing point from the map database 34. In this case, the position of the flying object 10 can be determined by using the altitude information from the altitude sensor 16, the position information of the GPS sensor, and the like. When the aircraft 10 moves directly above the landing point, the process proceeds to step S31.

Note that FIG. 5 schematically shows the state when the flying object 10 lands. In the flying object 10 of FIG. 5, the spatial sensors 13 to 15 are arranged in close proximity to each other. When the aircraft 10 arrives around the landing point, the space sensors 13 to 15 acquire information on three point clouds around the landing point. At this time, the landing points L and obstacles B1 and B2 exist around the landing points, and the characteristic points of the landing points L and obstacles B1 and B2 are extracted from the acquired point cloud information.

At the landing point L (indicated by the alphabet H), the feature points on the plane indicated by "○" are detected, and at the obstacles B1 and B2, the feature points of the uneven portion of the obstacle indicated by "●". Is detected. In the explanation of other drawings, "○" indicates a feature point on a plane, and "●" indicates a feature point of an uneven portion of an obstacle.

≪Step S31≫
In step S31, since the flying object 10 exists at a high altitude position, the point cloud information acquired by the long-distance narrow-angle space sensor 13 is used to generate 3D shape information around the landing point. This 3D shape information can be obtained by various methods, but in the present embodiment, it can be obtained from the "3D shape generation flow" shown in FIGS. 6, 7, 8 and 9. The details of the control step of the "3D shape generation flow" will be described later. When the 3D shape information around the landing point is generated, the process proceeds to step S32.

<< Step S32 >>
In step S32, the plane around the landing point and the obstacle (unevenness) are recognized from the 3D shape information obtained in step S31. Recognition of this plane and obstacles (unevenness) can also be obtained by various methods, but in this embodiment, from the "plane, obstacle (unevenness) recognition flow" shown in FIGS. 10, 11, and 12. You can ask. The details of the control step of "plane, obstacle (unevenness) recognition flow" will be described later.

Further, when the plane and the obstacle (unevenness) are recognized, the feature points of the plane and the obstacle (unevenness) are detected. The detection of feature points can also be obtained by various methods, but in the present embodiment, it can be obtained from the "feature point detection flow" shown in FIGS. 13 and 14. Details of the control steps of the "feature point detection flow" will be described later.

When a feature point is detected, the relative position between the feature points is measured and a specific identification ID (feature point ID) is given to each feature point. The assignment of the identification ID (feature point ID) can also be obtained by various methods, but in the present embodiment, it can be obtained from the "feature point relative position detection flow" shown in FIGS. 15 and 16. The details of the control step of the "feature point relative position detection flow" will be described later.

When the process of assigning a specific identification ID (feature point ID) to the feature point is completed, the process proceeds to step S33.

≪Step S33≫
In step S33, it is determined whether or not there is a plane sufficiently wide for landing from the relative positions of the feature points on the plane described above. The size of the plane can be determined based on the projected area of the flying object 10 on the landing surface. Since the physique of the flying object 10 is predetermined, it is determined whether or not landing is possible by comparing it with the width of the plane obtained from the relative positions of the feature points on the plane.

If it is determined that landing is not possible, the process proceeds to step S34, and if it is determined that landing is possible, the process proceeds to step S35.

<< Step S34 >>
In step S34, since it is determined that the aircraft 10 cannot land, a process of changing the landing location is performed, the process returns to step S30 again, and the same process is repeated.

≪Step S35≫
On the other hand, since it is determined that the landing can be performed in step S33, the landing operation of the flying object 10 is started in step S35. Since the point cloud information around the landing point at this time is the point cloud information when the altitude is high, it is the point cloud information within the measurement range by the long-distance narrow-angle space sensor 13. For this reason, the measurement range becomes narrower as the altitude becomes lower, so the feature points disappear and cannot be tracked. If a dangerous object (for example, a vehicle) moves within the landing range, it cannot be detected. -There is a problem that it becomes difficult to track the feature points required for attitude control.

Therefore, in the present embodiment, the control steps after step S36 are executed to eliminate the above-mentioned problems.

≪Step S36≫
In step S36, the altitude change amount is acquired from the altitude sensor 16. This makes it possible to determine the current altitude above ground level of the aircraft 10. This altitude above ground level is used to determine the reliability of the point cloud information by the spatial sensors 13 to 15. When the altitude above ground level is required, the process proceeds to step S37.

≪Step S37≫
In step S37, the reliability is set for each of the spatial sensors 13 to 15 that can detect the feature points. The reliability is set by obtaining the required angle of view including the feature points from the altitude above ground level and the relative position of the feature points, and from the required angle of view and the angles of view of the spatial sensors 13 to 15. In this embodiment, the reliability can be obtained from the "reliability setting flow" shown in FIG. The details of the control steps of the "reliability setting flow" will be described later. When the reliability setting process is completed, the process proceeds to step S38.

≪Step S38≫
In step S38, 3D shape information is newly generated from the measurement data (three point cloud information) of the spatial sensors 13 to 15 and the reliability obtained in step S37. The method of generating the 3D shape information can be performed by the same method as in step S31. However, the 3D shape information in this case becomes the 3D shape information after a certain time has elapsed with respect to the 3D shape information generated in step S31. Therefore, it is assumed that a new obstacle has occurred. When the generation of new 3D shape information is completed, the process proceeds to step S39.

≪Step S39≫
In step S39, the plane around the landing point and the obstacle (unevenness) are recognized again, and a new feature point is detected. This control step can be performed in the same manner as in step S32. As described in step S38, it is assumed that a new obstacle has occurred. Therefore, when a new feature point is detected, the relative position between the feature points is measured and a specific identification ID (feature point ID) is given to each feature point.

When the process of assigning a specific identification ID (feature point ID) to the new feature point is completed, the process proceeds to step S40.

<< Step S40 >>
In step S40, the position / attitude recognition unit 31 specifies the current position and attitude of the flying object 10. The current position / posture can be obtained by using the SLAM method as in step S10. The position / attitude at this time is the position / attitude in the landing mode, and the aircraft 10 is in a state of descending toward the landing point. When the current position and attitude of the flying object 10 are specified, the process proceeds to step S41.

≪Step S41≫
In step S41, it is determined whether or not a new obstacle exists at the landing point. Whether or not a new obstacle exists can be determined from the "danger determination flow" shown in FIG. The details of the control steps of the "danger judgment flow" will be described later.

At the time of step S35, there was no obstacle at the time of executing the landing operation, but as time passed, for example, a vehicle approached and a new obstacle was found at the landing point. May occur.

Therefore, from the new feature points obtained in step S39, if it is determined that a new obstacle has occurred at the landing point, the process proceeds to step S42, and if it is determined that no new feature point has occurred, the step is taken. Move to S43.

≪Step S42≫
Since it is determined in step S41 that there is a new obstacle around the landing point, in step S42, the landing place is corrected and a new landing place is sought. The new landing location can be obtained from the "operating state management flow" shown in FIG. The details of the control steps of the "operation state management flow" will be described later. When a new landing place is requested, the process proceeds to step S43.

≪Step S43≫
In step S43, the movement control of the aircraft 10 is executed toward the landing place set or the landing place modified in step S42. When the movement control of the flight object 10 is executed, the process proceeds to step S44.

<< Step S44 >>
In step S44, it is determined whether or not the aircraft 10 has landed at the landing site. The landing determination can be made by providing the flying object 10 with a landing switch that is opened and closed by the weight of the flying object 10. If it is determined that the landing is not completed, the process returns to step S36, and the control steps after step S36 are executed again. On the other hand, when it is determined that the landing is completed, it exits to the end and completes this control flow.

The above is the overall control flow of the landing control device. Next, the main individual control flows will be described.

[3D shape generation flow]
The 3D shape generation flow in step S31 will be described with reference to FIGS. 6, 7, 8 and 9. Note that FIG. 6 shows a control flow for newly measuring and updating the 3D shape around the landing point, FIG. 7 shows detailed information of the point cloud at that time, and FIG. 8 shows a control flow for obtaining the reliability in FIG. Shown, FIG. 9 shows a method of calculating the required angle of view for obtaining reliability.

Hereinafter, the 3D shape generation flow will be described based on the control flow shown in FIG. 6, but in the following control flow, the ICP (Iterative Closest Point) method is used.

<< Step S50 >>
In step S50, the point cloud information around the landing point is measured from each of the spatial sensors 13 to 15. The point cloud information can be measured by the same method as before. When the point cloud information around the landing point is measured, the process proceeds to step S51.

≪Step S51≫
In step S51, the 3D shape information near the nearest landing point is read from the 3D shape storage unit 24. Since the shape data of the 3D shape storage unit 24 is sequentially updated, it is regarded as the latest shape data. When the 3D shape information near the landing point is read, the process proceeds to step S52.

<< Step S52 >>
In step S52, the reliability is set for each of the three point cloud information obtained from each of the spatial sensors 13 to 15. This reliability is determined based on the altitude above ground level. A specific setting method will be described with reference to FIGS. 8 and 9. When the reliability is set, the process proceeds to step S53.

≪Step S53≫
In step S53, the feature points that can be observed from the point cloud information of each space sensor 13 to 15 are listed. The feature points listed are feature points within the range of the circumscribed circle set by the required angle of view shown in FIG. 9 to be described later. In addition, the feature points listed are tracked even when the altitude above ground level is low. When the feature points are listed, the process proceeds to step S54.

<< Step S54 >>
In step S54, the feature point relative positions of the spatial sensors 13 to 15 are obtained from the feature points obtained in step S53, and the point cloud conversion matrix is calculated from the feature point relative positions. The feature point relative position can be obtained from the "feature point relative position detection flow" shown in FIGS. 15 and 16.

Here, the point cloud has the information shown in FIG. 7. The point cloud information is composed of three-dimensional data having information such as basic position information and colors of X, Y, and Z in the Cartesian coordinate system. Further, the point cloud includes feature points. For example, in FIG. 7, for the position (P1), the color (C1), the luminance (K1), the reflection intensity (I1), and the like are defined, and the position (P1) is defined. For P2), color (C2), luminance (K2), reflection intensity (I2) ... Are defined. A 3D space can be expressed based on these enormous amounts of points. When the calculation of the point cloud transformation matrix is completed, the process proceeds to step S55.

≪Step S55≫
In step S55, weights are set for the feature points derived from the spatial sensors 13 to 15 obtained in step S53 according to the reliability of the three point cloud information of each spatial sensor 13 to 15 set in step S52. For example, if the reliability of a certain point cloud information is "1", the information is as it is, and if it is "0", the feature point is ignored. When the setting of the weight according to the reliability of the feature point is completed, the process proceeds to step S56.

≪Step S56≫
In step S56, the three point cloud information of each space sensor 13 to 15 is converted into the coordinates of the 3D shape information by the point cloud conversion matrix, and the three 3D shape information of each space sensor 13 to 15 and the read 3D shape information are converted. And integrate with. When the integration of the 3D shape information is completed, the process proceeds to step S57.

≪Step S57≫
In step S57, the 3D shape information integrated in step S56 is transferred to the 3D shape storage unit 24, and the 3D shape information of the 3D shape storage unit 24 is updated. When the update is completed, it exits to the end and ends the process.

[Reliability setting flow]
Next, the method of setting the reliability in step S52 will be described with reference to FIGS. 8 and 9.

<< Step S60 >>
In step S60, the operation state management unit 33 issues an execution command for the "landing mode", and determines whether or not the "landing mode" is being executed. If it is determined that it is not in "landing mode", it exits to the end and ends the process. On the other hand, if it is determined that the current operation mode is the "landing mode", the process proceeds to step S61.

≪Step S61≫
In step S61, the altitude sensor 16 acquires the altitude change of the flying object 10. That is, the altitude above ground level is detected from the change in altitude, and this altitude above ground level is used as a parameter for setting the reliability. When the altitude above ground level is detected, the process proceeds to step S62.

≪Step S62≫
In step S62, the feature point relative position is acquired from the relative position group storage unit 29. The feature point relative position can be obtained from the "feature point relative position detection flow" shown in FIGS. 15 and 16. This feature point relative position is also used as a parameter to set the reliability. When the relative position of the feature point is obtained, the process proceeds to step S63.

≪Step S63≫
In step S63, the required angle of view (θ) is calculated from the altitude above ground level and the relative position of the feature points. As shown in FIG. 9, the feature points on the plane (indicated by ○) and the feature points of the uneven portion of the obstacle (indicated by ●), which are related to each other, are relative distances (a, b, c). Have. Therefore, the required angle of view is determined by using the inverse trigonometric function from the diameter (D) and altitude (H) of the circumscribed circle including all the feature points of the identification ID (feature point ID) constituting the required plane or obstacle. You can ask. When the required angle of view (θ) including all the feature points of the identification ID (feature point ID) is obtained, the process proceeds to step S64.

≪Step S64≫
In step S64, the obtained required angle of view (θ) is compared with the angle of view of each of the spatial sensors 13 to 15, and the reliability of the image of each of the spatial sensors 13 to 15 is set. For example, if the angle of view of the long-distance narrow-angle space sensor 13 is smaller than the required angle of view (θ), it is determined that the reliability is low, and the angle of view of the long-distance narrow-angle space sensor 13 is larger than the required angle of view (θ). If it is large, it is judged that the reliability is high. The same determination is performed in the medium-range wide-angle space sensor 14 and the short-range wide-angle space sensor 15. The reliability is set for each of the spatial sensors 13 to 15 according to this determination result.

As described above, the required angle of view (θ) increases as the altitude of the flying object 10 decreases, so that the reliability of the medium-range wide-angle space sensor 14 and the short-range wide-angle space sensor 15 increases. Here, the reliability is set in the range of "1 to 0" and can be set as appropriate. Further, the reliability can be set to "1" or "0". When the reliability is set, it exits to the end and ends the process.

[Plane / Obstacle recognition flow]
Next, a plane / obstacle recognition method for recognizing a plane or an uneven portion of an obstacle in step S32 will be described with reference to FIGS. 10, 11, and 12.

≪Step S70≫
In step S70, 3D shape information around the latest landing point is acquired from the 3D shape storage unit 24. Since there are planes and obstacles (uneven parts) around the landing point, it is necessary to avoid obstacles when landing. Therefore, obstacles are detected from the 3D shape information around the landing point. When the 3D shape information around the landing point is acquired, the process proceeds to step S71.

≪Step S71≫
In step S71, the plane is estimated from the 3D shape information around the landing point. The reason for estimating this plane is to use it as a reference for detecting a convex portion protruding upward from the plane in the direction of gravity, a concave portion recessed downward in the direction of gravity from the plane, and the like. Of course, it can also be used to set the landing surface of the aircraft 10.

Here, there are various methods for estimating the plane, but in the present embodiment, the plane can be estimated by using a well-known RANSAC (Random Sample Consensus) method. When the plane is estimated, the process proceeds to step S72.

≪Step S72≫
In step S72, a point (feature point) equal to or higher than a predetermined threshold value in the direction of gravity is detected with reference to the plane estimated in step S71, and is transmitted to the feature point detection unit. The operation of the feature point detection unit will be described later. When a point equal to or higher than a predetermined threshold value is detected, the process proceeds to step S73. Since the feature points are a part of the point cloud, the point cloud information shown in FIG. 7 is also associated with them.

≪Step S73≫
In step S73, points equal to or larger than a predetermined threshold are grouped, and a vector representing an identification ID, an origin, and a size is stored. This grouping identifies "obstacles" by treating points related to each other as a group (point cloud).

FIG. 11 shows the concept of grouping, where “◯” indicates a feature point on a plane and “●” indicates a feature point of an obstacle. The feature points of the obstacles can be grouped because they are related to each other with a relative distance, and can be further represented by a size vector (xyz) with respect to a predetermined origin position. When the storage of the size vector (xyz) is completed, the process proceeds to step S74.

≪Step S74≫
In step S74, an unevenness flag is set for each of the grouped point clouds. This unevenness flag is a point cloud (obstacle) that protrudes upward with respect to the reference plane, or a point cloud (obstacle) that is depressed downward with respect to the reference plane. It is a flag indicating whether or not. In the present embodiment, "+" (convex flag) is set for the point cloud protruding upward, and "-" (concave flag) is set for the point cloud depressed downward. When the setting of the unevenness flag is completed, the process proceeds to step S75.

≪Step S75≫
In step S75, a group identification ID for identifying each is set in the grouped point cloud (unevenness group). An example of the group identification ID is shown in FIG.

In FIG. 12, the grouped point cloud information is composed of data having information such as the origin position of the orthogonal coordinate system, the size vector, and the unevenness flag. For example, the origin position (P1), the size vector (S1), and the concave flag (-1) are defined for the group identification ID (G1), and the origin position (P2), the size vector for the group identification ID (G2). (S2), the convex flag (+1) is defined. When the setting of the group identification ID is completed, the process proceeds to step S76.

≪Step S76≫
In step S76, the group identification ID (including the above-mentioned origin position, size vector, unevenness flag, etc.) obtained in step S75 is transmitted to the danger determination unit 30. As a result, the danger determination unit 30 determines whether or not there is an obstacle, the size, and the like, and determines whether or not the landing of the flight object 10 is hindered. The control operation of the danger determination unit 30 will be described later. When the transmission of the group identification ID is completed, the process ends with the end.

[Feature point detection flow]
Next, a feature point detection method for recognizing a plane around the landing point and an obstacle (unevenness) and detecting the feature points in step S32 will be described with reference to FIGS. 13 and 14.

≪Step S80≫
In step S80, the latest 3D shape information around the landing point is acquired from the 3D shape storage unit 24. When the 3D shape information around the landing point is acquired, the process proceeds to step S81.

≪Step S81≫
In step S81, changes in shape, color, and reflection intensity (difference, gradient, etc.) are calculated in the 3D shape information around the landing point. This is to detect that a new feature point has occurred. The new feature point is assumed to be, for example, the case where a vehicle enters the vicinity of the landing point. When changes in shape, color, and reflection intensity (difference, gradient, etc.) are calculated, the process proceeds to step S82.

≪Step S82≫
In step S82, points (feature points) where the amount of change in shape, color, and reflection intensity obtained in step S81 exceeds a predetermined threshold value are extracted. When a new feature point is detected, the process proceeds to step S83.

≪Step S83≫
In step S83, all the feature points in the area to be compared are acquired from the feature point storage unit 27. When all the feature points are acquired, the process proceeds to step S84.

≪Step S84≫
In step S84, the feature points acquired in step S84 and the feature points for which the amount of change obtained in step S82 exceeds the threshold value are compared (matched). This makes it possible to grasp the occurrence of new feature points. When the comparison of each feature point is completed, the process proceeds to step S85.

≪Step S85≫
In step S85, it is determined whether or not a new feature point (point that does not match) exists by using the comparison result of step S84. If it is determined that there is no new feature point, it exits to the end and ends the process. On the other hand, if it is determined that a new feature point exists, the process proceeds to step S86.

≪Step S86≫
In step S86, an identification ID is added to the new feature point, and the feature point storage unit 27 is updated and stored together with the position information. An example of memory of feature points is shown in FIG. In FIG. 14, the information of the identification ID is composed of data having information such as the position (xyz) of the orthogonal coordinate system and the unevenness flag. For example, for the identification ID (D1), the position (P1 = x1, y1, z1) and the concave flag (0 / plane) are defined, and for the identification ID (D2), the position (P2 = x2, y2, z2). , Concave flag (-1 / concave) is defined, and for the identification ID (D3), the position (P3 = x3, y3, z3) and concave flag (+ 1 / convex) are defined. When a new feature point is stored in the feature point storage unit 27, the process ends with the end.

[Characteristic point relative position measurement flow]
Next, the feature point relative position measuring method for measuring the feature point relative position in step S32 will be described with reference to FIGS. 15 and 16.

≪Step S90≫
In step S90, it is determined whether or not the new feature point stored in step S86 of FIG. 13 exists in the feature point storage unit 27. If it is determined that there is no new feature point, it exits to the end and ends the process. On the other hand, if it is determined that a new feature point exists, the process proceeds to step S91.

≪Step S91≫
In step S91, a new feature point is acquired from the feature point storage unit 27. When a new feature point is detected, the process proceeds to step S92.

≪Step S92≫
In step S92, all combinations are obtained by combining related feature points including new feature points, relative position groups are generated for each of the obtained combinations, and a group ID is added to these. This forms a group of related feature points. Obstacles and the like can be identified by generating this relative position group. When the relative position group is generated, the process proceeds to step S93.

≪Step S93≫
In step S93, the relative distance of each feature point of the relative position group is calculated. Returning to FIG. 9, in the group consisting of the combination of the feature points "○" and the feature points "●", the relative distance of each feature point can be obtained from the coordinate position of the coordinate system including three dimensions, and each "a". , "B", "c". The unit is "meter (m)", which allows the size of obstacles to be determined.

If the size (physical shape) of the obstacle is large, it can be recognized as a dangerous object that hinders the landing of the flying object 10. For example, in the danger judgment flow described later, the danger judgment can be executed using the information of the phase-relative position group. When the calculation of the relative distance of the feature points of the relative position group is completed, the process proceeds to step S94.

≪Step S94≫
In step S94, the identification ID and the relative distance of the relative position group obtained in steps S92 and S93 are stored and updated in the relative position group storage unit 29. FIG. 16 shows an example of the information of the relative position group. In FIG. 16, the identification ID information of the relative position group is composed of data having information such as feature points and relative distances of each of the relative position groups.

For example, for the relative position group identification ID (RG1), as shown in FIG. 9, the relative distances (a, b, c) between the identification IDs (1, 3, 6) of the related feature points and the respective feature points. ) Is defined, and for the relative position group identification ID (RG2), the relative distance between the identification IDs (1, 4, 2) of the related feature points and the respective feature points is defined. When a new relative position group ID or relative distance is stored in the relative position group storage unit 29, the process ends with the end.

[Danger judgment flow]
Next, a danger determination method for executing determination of whether or not a new obstacle exists at the landing point in step S41 of FIG. 4 will be described with reference to FIG.

≪Step S100≫
In step S100, the current position / attitude of the flying object 10, obstacles (unevenness group) near the landing point, and feature points are acquired. When these information are acquired, the process proceeds to step S101.

<< Step S101 >>
In step S101, the landing point perpendicular to the direction of gravity and its surroundings are calculated from the current position and attitude of the flying object 10. This makes it possible to identify the ground surface on which the aircraft 10 is about to land. When the area around the landing point is specified, the process proceeds to step S102.

<< Step S102 >>
In step S102, it is determined whether or not there is an obstacle or a new feature point (obstacle) other than the plane around the landing point. When the landing mode is executed, an obstacle or a feature point on a plane is already detected and a landing point is set, and the aircraft 10 is descending toward this point. Therefore, if a vehicle or the like enters the landing point, it may become a new obstacle. In order to avoid this new obstacle, in this control step, it is determined whether or not a new feature point is generated.

In this control step, the process is terminated by exiting to the end where it is determined that no new feature point exists. On the other hand, if it is determined that a new feature point exists, the process proceeds to step S103.

<< Step S103 >>
In step S103, if the flying object 10 continues the landing operation as it is, it may come into contact with a new obstacle. Therefore, the descent stop signal of the flying object 10 is output to the operation state management unit 33. When the output of the descent stop signal is transmitted to the operation state management unit 33, the process proceeds to step S104.

<< Step S104 >>
In step S104, the landing point closest to the landing point is newly calculated, in which there are no obstacles (unevenness group) around the landing point and new feature points other than the plane. Of course, even in this case, it goes without saying that a plane having an area on which the flying object 10 can land is calculated. When a new landing point is calculated, the process proceeds to step S105.

<< Step S105 >>
In step S105, the new landing point obtained in step S104 is output to the operation state management unit 33. When a new landing point is output, the exit process ends at the end. The descent stop signal from step S103 and the new landing point in step S104 are used in the operation state management state flow described below.

[Operation status management flow]
Next, a landing point correction method for correcting the landing point in step S42 of FIG. 4 will be described with reference to FIG.

<< Step S110 >>
In step S110, the current position / attitude of the flying object 10 is acquired from the position / attitude recognition unit 31. When the position / attitude of the flying object 10 is acquired, the process proceeds to step S111.

<< Step S111 >>
In step S111, the current position / attitude is matched with the map information of the map DB, and the current state of the flying object 10 is determined. When the current state of the flying object 10 is grasped, the process proceeds to step S112.

<< Step S112 >>
In step S112, it is determined from the danger determination unit 30 whether or not there is an output of the descent stop signal (output from step S103 of FIG. 17). If it is determined that there is no descent stop signal, the process proceeds to step S114, and if it is determined that there is a descent stop signal, the process proceeds to step S113.

<< Step S113 >>
In step S113, if the flying object 10 continues the landing operation as it is, there is a possibility that it may come into contact with a new obstacle. Therefore, the operation state management unit 33 changes the operation mode to the “standby mode”. By changing to the "standby mode", the flying object 10 will hover and will be stopped at the current position. When the mode is changed to the "standby mode", the process proceeds to step S114.

<< Step S114 >>
In step S114, it is determined from the danger determination unit 30 whether or not there is an output of a new landing point (output from step S105 in FIG. 17). If it is determined that there is no output at the new landing point, the process proceeds to step S117, and if it is determined that there is an output at the new landing point, the process proceeds to step S115.

≪Step S115≫
In step S115, the new landing point obtained in step S114 is output to the position / attitude control unit 35. As a result, a new landing point for the aircraft 10 is set. When a new landing point is output to the position / attitude control unit 35, the process proceeds to step S116.

<< Step S116 >>
In step S116, in order to move the aircraft 10 to the landing point, the operation state management unit 33 changes the operation mode to the “approach mode”. By changing to the "approach mode", the aircraft 10 starts moving toward the new landing point. When the mode is changed to the "approach mode", the process proceeds to step S117.

≪Step S117≫
In step S117, it is determined whether or not the operation mode is "approach mode" and a new landing point has been reached. If the landing point has not been reached in the "approach mode", the process proceeds to step S119, and if the landing point has been reached in the "approach mode", the process proceeds to step S118.

≪Step S118≫
In step S118, since the aircraft 10 has reached the landing point in the "approach mode", the operation mode is changed to the "landing mode" so that the aircraft 10 is lowered to the new landing point. Control. When the mode is changed to the "landing mode", the process proceeds to step S119.

≪Step S119≫
In step S119, any one of the operation mode of the "standby mode" of step S113, the "approach mode" of step S116, and the "landing mode" of step S118 is output to the environment recognition unit 21. After that, it exits to the end and ends the process.

By executing the landing control described above, at least (1) when the altitude of the drone changes to the lower side and the landing range deviates from the sensing range of the sensor camera, or (2) it is dangerous within the landing range. The drone can be landed at the intended landing position either when the object moves or (3) it becomes difficult to track the feature points required for position / attitude control.

The present invention is not limited to the above-described embodiment, and includes various modifications. For example, the above-described embodiment has been described in detail in order to explain the present invention in an easy-to-understand manner, and is not necessarily limited to the one including all the described configurations. Further, it is possible to replace a part of the configuration of one embodiment with the configuration of another embodiment, and it is also possible to add the configuration of another embodiment to the configuration of one embodiment. Further, it is possible to add / delete / replace a part of the configuration of each embodiment with another configuration.

10 ... flying object, 11 ... housing body, 12 ... blade rotor, 13 ... long-distance narrow-angle space sensor, 14 ... medium-distance wide-angle space sensor, 15 ... short-distance wide-angle space sensor, 16 ... altitude sensor.

Claims (8)

A plurality of feature point detecting means for detecting peripheral feature points including a landing point, a feature point recognizing means for recognizing the feature points detected by the feature point detecting means, and an altitude measuring means for measuring the altitude of an air vehicle. And a landing control device for an air vehicle provided with a landing control means for automatically landing the air vehicle at the landing point.
The landing control means measures the relative positions of a plurality of the feature points corresponding to the altitude, and selects the feature point detecting means for measuring each of the feature points according to the altitude, and the relative of the feature points. A landing control device for an air vehicle, which determines whether or not there is a change in the shape of the landing point from a change in position, and moves the air vehicle to a new landing point when the shape changes. In the landing control device for the aircraft according to claim 1,
The landing control means imparts reliability to the feature points measured by the plurality of feature point detecting means according to the altitude measured by the altitude measuring means, and uses the measured feature points as the reliability. An air vehicle landing control device characterized by tracking accordingly. In the landing control device for the aircraft according to claim 1,
When the landing control means detects a new obstacle that hinders landing while the aircraft is landing, the landing control means stops the descent of the aircraft and holds the aircraft in its current position. A characteristic aircraft landing control device. In the landing control device for the aircraft according to claim 1,
When the landing control means detects a new obstacle that hinders landing while the aircraft is landing, the landing control means obtains a new landing point and moves the aircraft to the new landing point. The landing control device for the aircraft, which is characterized by that. In the landing control device for the flying object according to claim 3 or 4.
The landing control means is a landing control device for an air vehicle, which is determined by the size of unevenness with respect to a plane of the landing point as the obstacle that hinders landing. In the landing control device for the flying object according to claim 5.
The landing control means forms the obstacle that is an obstacle to landing from a plurality of grouped feature points related to each other, and determines the size of the obstacle from the relative positions of the grouped feature points. An air vehicle landing control device characterized by In the landing control device for the aircraft according to claim 1,
The feature point detecting means is a landing control device for an air vehicle, characterized by being a spatial sensor (LIDAR / light detection and ranging) that detects the shape of the ground surface as point cloud information. In the landing control device for the flying object according to claim 7.
The landing control of an air vehicle is provided with at least two spatial sensors, one of which is a long-distance narrow-angle space sensor having a narrow angle of view and the other of which is a short-distance wide-angle space sensor having a wide angle of view. Device.

Images