JP7455044B2 - Aircraft landing control system - Google Patents

Description

The present invention relates to a landing control system for an aircraft, and more particularly to a landing control system for an aircraft that autonomously lands the aircraft.

Recently, a system has been proposed in which cargo is transported to a destination using an unmanned flying vehicle called a drone that takes off and lands perpendicularly to a landing surface. This drone generally uses a multi-rotor system having multiple blade rotors.

This drone transportation system inputs data representing the drone's flight plan path in a horizontal plane, obtains a height reference value representing the elevation of the surface below each of multiple positions on the flight plan path, and obtains a height reference value representing the elevation of the surface below each of multiple positions on the flight plan path. By using the value obtained by adding the flight altitude corresponding to the height reference value as the altitude data for the flight plan route, it is possible to fly the flight plan route without colliding with obstacles.

In such a transportation system using a drone, when the drone arrives at the destination and lands, it is important that the drone can land safely. For this reason, as a landing control device for a drone, a method described in, for example, Japanese Patent Application Publication No. 2019-51741 (Patent Document 1) has been proposed. In Patent Document 1, a drone is guided above a ground station, and its position in three-dimensional space is estimated from an image of a group of landmarks arranged on the landing surface using an onboard camera, and the position on the landing surface is estimated based on this. This guides the user with high precision.

In other words, when the drone is guided above the ground station and a group of landmarks is captured in the photographed image, the area occupied by the entire screen of the photographed image is calculated, and when the calculated area is larger than a predetermined area, the landmark group The position of the unmanned aircraft in three-dimensional space with respect to the ground station is estimated from the captured images of the switched landmark group, and then the landing movement range of the unmanned aircraft is calculated, and the landing movement range is adjusted to the calculated landing movement range. Based on this, the drone is guided and controlled to the landing surface.

JP2019-51741A

By the way, the landing control system described in Patent Document 1 requires the preparation of a plurality of landmark groups that serve as reference positions at the ground station, and the cost of constructing the system is high when including the installation cost and the cost of its maintenance and management. The problem is that it is expensive. Furthermore, since it is possible to land only at a specified landing site called a ground station, there is a problem that the system is not easy to use. Therefore, in order to land without setting up a landmark group at the ground station, it is conceivable to use a sensor camera mounted on the drone to autonomously recognize a possible landing place and land the drone.

However, at least (1) if the altitude of the drone changes to a lower side and the landing range deviates from the sensing range of the sensor camera, or (2) if a dangerous object moves within the landing range, or (3) if the position・When it becomes difficult to track the feature points necessary for attitude control, a phenomenon occurs in which the drone does not land at the intended landing position. This poses a problem in that it becomes difficult to land the drone safely.

An object of the present invention is to provide a novel landing control system for an aircraft that can safely land an aircraft without using a group of landmarks.

Although this cargo transport vehicle is unmanned, development is underway to develop it into a manned vehicle (so-called flying car) in the future. Therefore, the present invention proposes a landing control system that can be applied not only to unmanned flying vehicles but also to manned flying vehicles. Further, the flying object is not limited to the multi-rotor type, but also covers other autonomous flying objects.

The features of the present invention include a plurality of feature point detection means for detecting feature points in the vicinity including the landing site, a feature point recognition means for recognizing the feature points detected by the feature point detection means, and a feature point recognition means for measuring the altitude of the aircraft. A landing control device for an aircraft, comprising: an altitude measuring means for automatically landing the aircraft at a landing site;
The landing control means measures the relative positions of the plurality of feature points in accordance with the altitude, selects a feature point detection means for measuring each of the feature points in accordance with the altitude, and detects changes in the relative positions of the feature points. The aircraft's landing control system determines whether or not there has been a change in the shape of the landing site, and if there is a change in shape, moves the aircraft to a new landing site.

According to the present invention, a flying object can be safely landed without using a landmark group.

FIG. 2 is an explanatory diagram illustrating the configuration of a flying object and the measurement range of a spatial sensor provided therein. 1 is a block diagram showing the configuration of an aircraft control device according to an embodiment of the present invention. 3 is a flowchart showing a control flow for explaining an operation mode of the aircraft control device shown in FIG. 2. FIG. 3 is a flowchart showing a control flow for explaining the operation of a landing control device in the aircraft control device shown in FIG. 2. FIG. FIG. 3 is an explanatory diagram for explaining a state in which the ground surface is measured when the flying object reaches the landing point. 5 is a flowchart illustrating a 3D shape measurement flow for generating a 3D shape in step S32 of FIG. 4. FIG. FIG. 3 is an explanatory diagram for explaining information on a point group forming a 3D shape. 7 is a flowchart illustrating a reliability setting flow for setting reliability in step S52 of FIG. 6. FIG. 9 is an explanatory diagram illustrating a method for determining reliability in FIG. 8. FIG. 5 is a flowchart illustrating a plane/obstacle recognition flow for recognizing a plane/obstacle (uneven object) in step S32 of FIG. 4. FIG. 11 is an explanatory diagram illustrating the form of grouped obstacles shown in FIG. 10. FIG. 12 is an explanatory diagram illustrating information on obstacles shown in FIG. 11. FIG. 5 is a flowchart illustrating a feature point detection flow for detecting feature points in step S32 of FIG. 4. FIG. 14 is an explanatory diagram for explaining information on feature points shown in FIG. 13. FIG. 5 is a flowchart illustrating a feature point relative position detection flow for detecting feature point relative positions in step S32 of FIG. 4. FIG. 16 is an explanatory diagram illustrating information on relative position groups of feature points shown in FIG. 15. FIG. 5 is a flowchart illustrating a risk determination flow for determining whether there is an obstacle at the landing site in step S41 of FIG. 4; 5 is a flowchart illustrating an operation state management flow for correcting the landing site in step S42 of FIG. 4. FIG.

Hereinafter, embodiments of the present invention will be described in detail using drawings, but the present invention is not limited to the following embodiments, and various modifications and applications can be made within the technical concept of the present invention. is also included within the scope.

FIG. 1 shows a schematic configuration of a flying vehicle 10 used in the present invention. The flying object 10 has four blade rotors 12 provided in symmetrical positions on a rectangular housing body 11, and each blade rotor 12 is driven by an electric motor (not shown). Note that the flying object of this embodiment is not limited to this, and any flying object that can take off and land in the vertical direction may be used.

A long-range narrow-angle space sensor 13 is provided near the center of the casing body 11, and a medium-range wide-angle space sensor 14 is provided at one of the left and right ends of the casing body 11. A short-range wide-angle spatial sensor 15 is provided at the other end. These sensors 13 to 15 detect the characteristics of various stationary objects and moving objects on the ground surface by using sensor cameras that image the ground surface and lidar (LIDAR) that converts the shape of the ground surface into point clouds. It functions as a feature point detection sensor that detects points.

In this embodiment, a three-dimensional lidar is used, and will be described below as a "spatial sensor". Further, although not shown, a well-known GPS sensor is mounted on the housing body 11 in order to detect the position of the flying object 10.

Furthermore, an altitude sensor 16 is provided in the housing body 11 to detect the altitude above the ground. As the altitude sensor 16, a laser altimeter or a radar altimeter can be used, and the distance between the aircraft 10 and the ground surface is measured by reflection of laser light or microwaves, and the altitude of the aircraft 10 above the ground is measured. be able to. It is also possible to calculate the altitude above the ground from the altitude output from a barometer or GPS sensor.

The long-range narrow-angle space sensor 13 is a space sensor with a narrow angle of view as shown in measurement range A, and the middle-range wide-angle space sensor 14 is a long-range narrow-angle space sensor 13 with a narrow angle of view as shown in measurement range B. The short-range wide-angle space sensor 15 is a wider space sensor, and has a wider angle of view than the middle-range wide-angle space sensor 14, as shown in measurement range C.

As will be described later, reliability is given to the point cloud information obtained by each of the spatial sensors 13 to 15 in accordance with the altitude. For example, as the altitude of the flying object 10 becomes lower, the reliability of point cloud information from the long-range narrow-angle space sensor 13 changes from "high" to "low"; The reliability of the point cloud information from No. 15 is changed from "low" to "high". Note that the degree of reliability includes cases in which it is adjusted continuously and cases in which it is adjusted in a switch-like manner. In other words, the change in reliability can be understood as the spatial sensor being switched depending on the altitude.

Although three spatial sensors are used in the above description, two spatial sensors may be used if necessary. Further, the placement positions of the spatial sensors 13 to 15 are arbitrary, and the spatial sensors 13 to 15 may be placed at separate positions as shown in FIG. 1, or close to each other as shown in FIG. 5, which will be described later. You can also place it.

Returning to FIG. 1, the casing body 11 of the aircraft 10 is provided with an aircraft control device (not shown), and this aircraft control device is used to determine the flight plan path of the aircraft in the horizontal plane. Based on the information and the height reference value representing the elevation of the ground surface below each of the multiple positions on the flight plan route, the flight plan calculates the value obtained by adding the flight altitude corresponding to that flight position to the height reference value. By using this as altitude information for the route, it is possible to fly along the flight plan route without colliding with obstacles.

On the other hand, the aircraft control system is also equipped with a landing control system, which is the object of the present invention. This landing control device can be provided functionally integrally with the flight vehicle control device, or can be provided separately from the flight vehicle control device. Therefore, an appropriate configuration may be selected depending on the specifications of the aircraft control device.

Next, a landing control system according to the present invention will be explained. FIG. 2 shows the configuration of an aircraft control system 20 in which the functions of a landing control system are integrated.

Point cloud information from the above-mentioned long-range narrow-angle space sensor 13 , medium-range wide-angle space sensor 14 , and short-range wide-angle space sensor 15 and altitude information from the altitude sensor 16 are input to the environment recognition unit 21 . That is, point cloud information from each of the spatial sensors 13 to 15 is input to the 3D shape measuring section 22, and altitude information from the altitude sensor 16 is input to the sensor reliability setting section 23. The sensor reliability setting unit 23 sets the reliability of the point cloud information input to the 3D shape measurement unit 22. Here, "3D" means "three dimensions."

In this way, the sensor reliability setting unit 23 has a function of setting reliability corresponding to the altitude to the point cloud information from each of the spatial sensors 13 to 15 in accordance with the altitude. As described above, the reliability of the point cloud information obtained by each of the spatial sensors 13 to 15 is given in accordance with the altitude. For example, as the altitude of the flying object 10 becomes lower, the reliability of point cloud information from the long-range narrow-angle space sensor 13 changes from "high" to "low"; The reliability of the point cloud information from No. 15 is changed from "low" to "high".

Furthermore, the three-dimensional shape information of the ground surface from the 3D shape memory section 24 is input to the 3D shape measurement section 22, and conversely, the three-dimensional shape information measured by the 3D shape measurement section 22 is input to the 3D shape memory section 24. Newly entered (updated).

The environment recognition section 21 further includes a plane/unevenness recognition section 25 , a feature point detection section 26 , a feature point storage section 27 , a feature point relative position calculation section 28 , and a relative value group storage section 29 .

The plane/unevenness recognition unit 25 has a function of recognizing plane information and unevenness information of the ground surface based on the point cloud information input to the 3D shape measurement unit 22.

Further, the feature point detection unit 26 extracts plane information and unevenness from the target point cloud information based on the point cloud information input to the 3D shape measurement unit 22 and the plane information and unevenness information of the plane/unevenness recognition unit 25. It has a function to detect feature points of information. The feature points detected by the feature point detection section 26 are temporarily stored in the feature point storage section 27.

The feature point relative position calculation unit 28 has a function of calculating relative positions (three-dimensional positions) between the plurality of feature points stored in the feature point storage unit 27. Further, the relative position group storage unit 29 has a function of grouping and storing mutually related feature points (for example, the shape of an uneven portion) from the feature points calculated by the feature point relative position calculation unit 28. The relative position group information in the relative position group storage section 29 is fed back to the 3D shape measurement section 22 and the sensor reliability setting section 23.

Plane information and unevenness information from the plane/unevenness recognition section 25 , feature point information from the feature point detection section 26 , and position/posture information from the position/posture recognition section 31 are input to the danger judgment section 30 . Here, the position/attitude information represents the position and inclination of the flying object 10 in the global coordinate system.

Three-dimensional shape information from the 3D shape memory section 24 and position/orientation information from the position/orientation sensor 32 are input to the position/orientation recognition section 31 . The position/attitude sensor 32 detects the position of the aircraft 10 and parameters around the rotation axis of the aircraft 10, such as "yaw", "roll", and "pitch" of the aircraft 10.

The danger determination unit 30 determines whether the landing site of the aircraft 10 is a flat surface or whether there is an obstacle (unevenness), and if there is an obstacle (unevenness), it determines whether the landing site of the aircraft 10 is a flat surface or whether there is an obstacle (unevenness), a landing point that allows a safe landing. It has a function to calculate the location.

The danger information and landing point information from the danger determination section 30 and the position/attitude information from the position/attitude recognition section 31 are input to the operating state management section 33. Based on this information, the operating state management unit 33 manages the operating state of the aircraft 10. Here, five operating states (modes) are managed: "landing mode", "takeoff mode", "cruise mode", "approach mode", and "standby mode".

In the "landing mode" and "takeoff mode", the aircraft 10 moves vertically and turns, in the "cruise mode" it moves along the flight plan route according to the map database 34, and in the "approach mode" It has a function to manage the operating state of the robot, such as moving to a designated point (for example, a landing site) and hovering without moving in "standby mode."

In the position/orientation control section 35, operating state management information from the operating state management section 33, map information from the map database 34, and position/orientation recognition information from the position/orientation recognition section 31 are input. Furthermore, the position/attitude control unit 35 generates target position/attitude information by the target position/attitude generation unit 36, and this target position/attitude information is input to the follow-up control unit 37. The tracking control unit 37 performs flight according to one of "landing mode", "takeoff mode", "cruise mode", "approach mode", and "standby mode" selected based on the target position/attitude information. It has a function of autonomously operating the body 10.

The above are the main functional elements of the aircraft control device 20, and these functional elements are basically operated by executing a control program of a microcomputer. As is well known, a microcomputer consists of a central processing unit (CPU) that executes calculations, a non-volatile memory (ROM) that stores control programs, a volatile memory (RAM) that stores calculation results, and input sensor signals. It consists of input/output circuits that output drive signals. The control flow of the main functional elements of the aircraft control device 20 will be described below.

[Operation mode setting control flow]
First, FIG. 3 shows a control flow of the position/attitude control section 35, which sets the operation mode of the aircraft 10.

<<Step S10>>
In step S10, current position/orientation information is acquired. This position/orientation information can be obtained from the position/orientation sensor 32 and the position/orientation recognition unit 31. The position/attitude recognition unit 31 can obtain the current position/attitude using, for example, SLAM (Simultaneous Localization and Mapping) method, and can determine that the aircraft 10 is flying in a certain state in this state. Recognizable. Once the current position/orientation information is acquired, the process moves to step S11.

<<Step S11>>
In step S11, in order to determine which operating state (the above-mentioned five operating modes) the flight state of the aircraft 10 is currently in, the operating state currently set by the operating state management unit 33 is acquired. do. Once the current operating state is acquired, the process moves to step S12.

<<Step S12>>
In step S12, it is determined whether the operating state acquired from the operating state management unit 33 is "cruise mode". In this determination step, the determination is made based on the operational state information sequentially sent from the operational state management section 33. If it is determined that the vehicle is in the "cruise mode" in this determination step, the process proceeds to step S13, and if it is determined that the vehicle is not in the "cruise mode", the process proceeds to step S14.

<<Step S13>>
In step S13, the current position/orientation acquired in step S10 is matched with map information from the map database 34, and target position/orientation concession information is set. In this state, if "no" is determined in the following control steps S14, S18, and S20, the process moves to step S22 and the flying object 10 is controlled according to the target position/attitude information. As a result, the aircraft 10 heads to the destination in "cruise mode".

<<Step S14>>
After the control steps in steps S12 and S13 are executed, in step S14, it is determined whether the operating state acquired from the operating state management unit 33 is the "approach mode". If it is determined that the state is in the "approach mode" in this determination step, the process moves to step S15, and if it is determined that the state is not the "approach mode", the process moves to step S18.

<<Step S15>>
Since the "approach mode" is determined in step S14, it is determined in step S15 whether or not there is a landing request for the aircraft 10. A landing request can be determined based on whether or not there is an output of the landing site from the operating state management unit 33. In step S15, if it is determined that there is an output from the landing site, the process moves to step S16, and if it is determined that there is no output from the landing site, the process moves to step S17.

<<Step S16>>
In step S16, since it is determined that there is a landing request in step S15, the target position/attitude information set in step S13 is reset as landing point information above the new landing point. In this state, if "no" is determined in the following control steps S18 and S20, the process moves to step S22, and control is executed for the flying object 10 to move toward the sky above the landing site according to the landing site information. This causes the aircraft 10 to head toward the landing site in "approach mode."

<<Step S17>>
On the other hand, since it is determined in step S15 that there is no landing point output, target position/attitude information is newly set from the map database. In this state, if "no" is determined in the following control steps S18 and S20, the process moves to step S22 and the flying object 10 is controlled according to the target position/attitude information.

<<Step S18>>
When the landing site information or target position/attitude information is set in steps S16 and S17, it is determined in step S18 whether the operating state acquired from the operating state management unit 33 is "standby mode". If it is determined that the state is in "standby mode" in this determination step, the process proceeds to step S19, and if it is determined that the state is not in "standby mode", the process proceeds to step S20.

<<Step S19>>
In step S19, since the flying object 10 has arrived at the landing site and is set to the "standby mode", the current position/attitude information is reset to new target position/attitude information. In this state, the flying object 10 hovers and waits above the landing site. Next, in this state, the process moves to step S20.

≪Step S20≫
After the control steps in steps S18 and S19 are executed, in step S20, it is determined whether the operating state is in "landing mode" or "takeoff mode". If it is determined that the vehicle is not in the "landing mode" or "takeoff mode", the process proceeds to step S22, and if it is determined that the vehicle is in the "landing mode" or "takeoff mode", the process proceeds to step S21.

<<Step S21>>
In step S21, the current position/orientation information is maintained as is, and the point moved in the vertical direction is set as the target position/orientation. As a result, the flying object 10 is moved from a hovering state to an ascending direction or a descending direction. Once the target position/attitude moved in the vertical direction is set, the process moves to step S22.

<<Step S22>>
In step S22, the operating state of the aircraft 10 is controlled based on the target position/attitude information set in any one of steps S13, S16, S17, S19, and S21, so that the aircraft 10 is directed to the target point. can be flown.

The above is the general control flow of the position/attitude control section 35 of the aircraft control device 20. Next, a specific control flow of the landing control system to which the present invention is directed will be explained. In addition, the overall control flow of the landing control system will be explained below, and then the main individual control flows executed in the overall control flow will be explained.

[Overall landing control flow]
The overall landing control flow is as shown in FIG. 4, which shows the control steps S20 to S22 shown in FIG. 3 in more detail. The overall landing control flow will be described below with reference to FIG.

≪Step S30≫
In step S30, the flying object 10 is moved from the map database 34 to a position directly above the landing site. In this case, the position of the aircraft 10 can be determined using altitude information from the altitude sensor 16, position information from the GPS sensor, and the like. When the flying object 10 moves directly above the landing site, the process moves to step S31.

Incidentally, FIG. 5 schematically shows the state when the flying object 10 lands. In the flying object 10 of FIG. 5, the spatial sensors 13 to 15 are arranged close to each other. When the flying object 10 arrives near the landing site, spatial sensors 13 to 15 acquire three point cloud information around the landing site. At this time, the landing site L and obstacles B1 and B2 are present around the landing site, and feature points of the landing site L and obstacles B1 and B2 are extracted from the acquired point cloud information.

At landing point L (indicated by the alphabet H), feature points on the plane indicated by "○" are detected, and at obstacles B1 and B2, feature points on the uneven parts of the obstacles indicated by "●" are detected. is detected. In the explanations of other drawings as well, "O" indicates a feature point on a plane, and "●" indicates a feature point on an uneven part of an obstacle.

≪Step S31≫
In step S31, since the flying object 10 is located at a high altitude, point cloud information acquired by the long-range narrow-angle space sensor 13 is used to generate 3D shape information around the landing site. This 3D shape information can be obtained by various methods, but in this embodiment, it can be obtained from the "3D shape generation flow" shown in FIGS. 6, 7, 8, and 9. Further, details of the control steps of the "3D shape generation flow" will be described later. Once the 3D shape information around the landing site is generated, the process moves to step S32.

<<Step S32>>
In step S32, the plane around the landing site and obstacles (uneven objects) are recognized from the 3D shape information obtained in step S31. Recognition of this plane and obstacles (uneven objects) can be obtained by various methods, but in this embodiment, from the "plane and obstacle (uneven object) recognition flow" shown in FIGS. 10, 11, and 12. You can ask for it. Details of the control steps of the "plane and obstacle (uneven object) recognition flow" will be described later.

Furthermore, when a plane and an obstacle (uneven object) are recognized, feature points of the plane and the obstacle (uneven object) are detected. Although feature points can be detected using various methods, in this embodiment, feature points can be detected using the "feature point detection flow" shown in FIGS. 13 and 14. Details of the control steps of the "feature point detection flow" will be described later.

When the feature points are detected, the relative positions between the feature points are measured and a specific identification ID (feature point ID) is assigned to each feature point. Although the identification ID (feature point ID) can be determined by various methods, in this embodiment, it can be determined from the "feature point relative position detection flow" shown in FIGS. 15 and 16. Details of the control steps of the "feature point relative position detection flow" will be described later.

When the process of assigning a specific identification ID (feature point ID) to the feature point is completed, the process moves to step S33.

<<Step S33>>
In step S33, it is determined from the relative positions of the feature points on the plane described above whether a plane large enough for landing exists. The width of the plane can be determined based on the projected area of the aircraft 10 onto the landing surface. Since the physique of the flying object 10 is predetermined, it is determined whether landing is possible by comparing it with the width of the plane determined from the relative positions of the feature points on the plane.

If it is determined that landing is not possible, the process moves to step S34, and if it is determined that landing is possible, the process moves to step S35.

<Step S34>
In step S34, since it is determined that the aircraft 10 cannot land, a process to change the landing site is performed, and the process returns to step S30 to repeat the same process.

<<Step S35>>
On the other hand, since it is determined in step S33 that landing is possible, the landing operation of the aircraft 10 is started in step S35. Note that the point cloud information around the landing site at this time is point cloud information at a high altitude, so it is point cloud information within the measurement range by the long-distance narrow-angle space sensor 13. For this reason, as the altitude decreases, the measurement range becomes narrower, so feature points disappear and cannot be tracked. If a dangerous object (such as a vehicle) moves within the landing range, it cannot be detected. - This causes problems such as difficulty in tracking feature points necessary for attitude control.

Therefore, in this embodiment, the control steps after step S36 are executed to eliminate the above-mentioned problems.

<<Step S36>>
In step S36, the altitude change amount is acquired from the altitude sensor 16. Thereby, the current altitude above the ground of the aircraft 10 can be determined. This altitude above the ground is used to determine the reliability of point cloud information from the spatial sensors 13-15. Once the altitude above the ground is determined, the process moves to step S37.

<<Step S37>>
In step S37, reliability is set for each of the spatial sensors 13 to 15 that can detect feature points. The reliability is set by determining the required angle of view that includes the feature point from the altitude above the ground and the relative position of the feature point, and from this required angle of view and the angle of view of each of the spatial sensors 13 to 15. In this embodiment, the reliability can be determined from the "reliability setting flow" shown in FIG. 8. Details of the control steps of the "reliability setting flow" will be described later. When the reliability setting process is completed, the process moves to step S38.

<<Step S38>>
In step S38, 3D shape information is newly generated from the measurement data (three point group information) of the spatial sensors 13 to 15 and the reliability obtained in step S37. This 3D shape information can be generated using the same method as step S31. However, the 3D shape information in this case becomes 3D shape information after a certain period of time has passed with respect to the 3D shape information generated in step S31. For this reason, it is assumed that new obstacles are occurring. When the generation of new 3D shape information is completed, the process moves to step S39.

<<Step S39>>
In step S39, the plane and obstacles (uneven objects) around the landing site are recognized again, and new feature points are detected. This control step can be performed in the same way as step S32. Note that, as explained in step S38, a situation in which a new obstacle has occurred is assumed. Therefore, when a new feature point is detected, the relative positions between the feature points are measured and a specific identification ID (feature point ID) is assigned to each feature point.

When the process of assigning a specific identification ID (feature point ID) to a new feature point is completed, the process moves to step S40.

≪Step S40≫
In step S40, the position/attitude recognition unit 31 identifies the current position and attitude of the flying object 10. The current position/orientation can be obtained using the SLAM method similarly to step S10. The position and attitude at this time are those in the landing mode, and the flying object 10 is descending toward the landing site. Once the current position and attitude of the flying object 10 are specified, the process moves to step S41.

<<Step S41>>
In step S41, it is determined whether a new obstacle exists at the landing site. Whether or not a new obstacle exists can be determined from the "danger determination flow" shown in FIG. 17. Details of the control steps of the "danger judgment flow" will be described later.

At the time of step S35, there were no obstacles when the landing operation was executed, but as time passes, for example, a vehicle approaches, and a new obstacle appears at the landing point. may occur.

Therefore, if it is determined from the new feature point obtained in step S39 that a new obstacle has occurred at the landing site, the process moves to step S42, and if it is determined that no new feature point has occurred, the process proceeds to step S42. The process moves to S43.

<<Step S42>>
Since it is determined in step S41 that there is a new obstacle around the landing site, in step S42 the landing site is corrected and a new landing site is found. A new landing location can be obtained from the "operating state management flow" shown in FIG. Details of the control steps of the "operating state management flow" will be described later. When a new landing location is determined, the process moves to step S43.

<<Step S43>>
In step S43, movement control of the flying object 10 is performed toward the set landing site or the landing site corrected in step S42. Once the movement control of the flying object 10 is executed, the process moves to step S44.

<<Step S44>>
In step S44, it is determined whether the flying object 10 has landed at the landing site. Landing can be determined by providing the aircraft 10 with a landing switch that is opened and closed by the weight of the aircraft 10. If it is determined that the landing is not completed, the process returns to step S36, and the control steps after step S36 are executed again. On the other hand, when it is determined that the landing has been completed, the control flow exits to the end and completes this control flow.

The above is the overall control flow of the landing control system. Next, the main individual control flows will be explained.

[3D shape generation flow]
The 3D shape generation flow in step S31 will be explained using FIGS. 6, 7, 8, and 9. Furthermore, Fig. 6 shows the control flow for newly measuring and updating the 3D shape around the landing site, Fig. 7 shows the detailed information of the point cloud at that time, and Fig. 8 shows the control flow for calculating the reliability in Fig. 6. FIG. 9 shows a method for calculating the required angle of view for determining reliability.

The 3D shape generation flow will be described below based on the control flow shown in FIG. 6, and the ICP (Iterative Closest Point) method is used in the control flow below.

≪Step S50≫
In step S50, point cloud information around the landing site from each of the spatial sensors 13 to 15 is measured. This point cloud information can be measured using a conventional method. After measuring the point cloud information around the landing site, the process moves to step S51.

<<Step S51>>
In step S51, 3D shape information in the vicinity of the most recent landing point is read from the 3D shape memory unit 24. Since the shape data in the 3D shape memory section 24 is updated sequentially, it is considered to be the latest shape data. After reading the 3D shape information near the landing site, the process moves to step S52.

<<Step S52>>
In step S52, reliability is set for each of the three point cloud information obtained from each of the spatial sensors 13 to 15. This reliability is determined based on altitude above ground. A specific setting method will be explained with reference to FIGS. 8 and 9. Once the reliability is set, the process moves to step S53.

<<Step S53>>
In step S53, observable feature points are listed from the point cloud information of each spatial sensor 13 to 15. The feature points listed are feature points within the range of the circumscribed circle set by the required angle of view shown in FIG. 9, which will be described later. Furthermore, the listed feature points are tracked even if the altitude above the ground becomes low. Once the feature points are listed, the process moves to step S54.

<<Step S54>>
In step S54, the relative positions of the feature points of each of the spatial sensors 13 to 15 are determined from the feature points determined in step S53, and a point group transformation matrix is calculated from the relative positions of the feature points. The feature point relative position can be determined from the "feature point relative position detection flow" shown in FIGS. 15 and 16.

Here, the point group has information shown in FIG. The point group information is composed of three-dimensional data having information such as basic positional information of X, Y, and Z in an orthogonal coordinate system and information such as color. In addition, the point cloud includes feature points. For example, in the point cloud in FIG. 7, for the position (P1), color (C1), brightness (K1), reflection intensity (I1)... Regarding P2), color (C2), brightness (K2), reflection intensity (I2), etc. are defined. 3D space can be expressed based on these huge amounts of points. When the calculation of the point group transformation matrix is completed, the process moves to step S55.

<<Step S55>>
In step S55, weights are set for the feature points derived from the spatial sensors 13 to 15 obtained in step S53 according to the reliability of the three point cloud information of each spatial sensor 13 to 15 set in step S52. For example, if the reliability of certain point group information is "1", the information will remain as is, and if it is "0", the feature point will be ignored. When the setting of weights according to the reliability of the feature points is completed, the process moves to step S56.

<<Step S56>>
In step S56, the three point cloud information of each spatial sensor 13 to 15 is converted into coordinates of 3D shape information using a point cloud transformation matrix, and the three 3D shape information of each spatial sensor 13 to 15 and the read 3D shape information are converted. integrate with. When the integration of 3D shape information is completed, the process moves to step S57.

<<Step S57>>
In step S57, the 3D shape information integrated in step S56 is transferred to the 3D shape memory section 24, and the 3D shape information in the 3D shape memory section 24 is updated. When the update is completed, the process exits to END and ends the process.

[Reliability setting flow]
Next, a method for setting reliability in step S52 will be explained using FIGS. 8 and 9.

<<Step S60>>
In step S60, a command to execute the "landing mode" is issued from the operating state management unit 33, and it is determined whether the "landing mode" is being executed. If it is determined that the mode is not "landing mode", the process exits to the end and ends the process. On the other hand, if it is determined that the current operation mode is "landing mode", the process moves to step S61.

<<Step S61>>
In step S61, the altitude sensor 16 acquires the altitude change of the flying object 10. That is, the altitude above the ground is detected from the change in altitude, and this altitude above the ground is used as a parameter for setting reliability. When the altitude above the ground is detected, the process moves to step S62.

<<Step S62>>
In step S62, the feature point relative positions are acquired from the relative position group storage section 29. This feature point relative position can be determined from the "feature point relative position detection flow" shown in FIGS. 15 and 16. This feature point relative position is also used as a parameter for setting reliability. Once the relative positions of the feature points are determined, the process moves to step S63.

<<Step S63>>
In step S63, the required angle of view (θ) is calculated from the altitude above the ground and the relative position of the feature point. As shown in Figure 9, the relative distances (a, b, c) of the feature points on the plane (indicated by ○) and the feature points on the uneven parts of obstacles (indicated by ●) that are related to each other are have. Therefore, from the diameter (D) and height (H) of the circumscribed circle that includes all the feature points of the identification ID (feature point ID) that constitute the required plane and obstacle, use inverse trigonometric functions to calculate the required angle of view. You can ask for it. Once the necessary angle of view (θ) including all the feature points of the identification ID (feature point ID) is determined, the process moves to step S64.

<<Step S64>>
In step S64, the obtained required angle of view (θ) is compared with the angle of view of each of the spatial sensors 13 to 15, and the reliability of the image of each of the spatial sensors 13 to 15 is set. For example, if the angle of view of the long-distance narrow-angle space sensor 13 is smaller than the required angle of view (θ), the reliability is determined to be low; If it is large, it is judged that the reliability is high. Similar determinations are made for the medium-range wide-angle spatial sensor 14 and the short-range wide-angle spatial sensor 15. Reliability is set for each of the spatial sensors 13 to 15 in accordance with this determination result.

In this way, as the altitude of the flying object 10 decreases, the required angle of view (θ) increases, so the reliability of the medium-range wide-angle spatial sensor 14 and the short-range wide-angle spatial sensor 15 increases. Here, the reliability is set in a range of "1 to 0" and can be set as appropriate. Further, the reliability can also be set to "1" or "0". Once the reliability is set, the process exits to END and ends the process.

[Plane/obstacle recognition flow]
Next, a plane/obstacle recognition method for recognizing uneven parts of a plane and obstacles in step S32 will be explained using FIGS. 10, 11, and 12.

<<Step S70>>
In step S70, the latest 3D shape information around the landing site is acquired from the 3D shape memory section 24. There are flat surfaces and obstacles (uneven areas) around the landing site, so it is necessary to land while avoiding obstacles. Therefore, obstacles are detected from 3D shape information around the landing site. Once the 3D shape information around the landing site is acquired, the process moves to step S71.

<<Step S71>>
In step S71, a plane is estimated from 3D shape information around the landing site. The purpose of estimating this plane is to use it as a reference for detecting convex portions that protrude upward from the plane in the direction of gravity, recesses that sink downward from the plane in the direction of gravity, and the like. Of course, it can also be used to set the landing surface of the aircraft 10.

Here, there are various methods for estimating the plane, but in this embodiment, the well-known RANSAC (Random Sample Consensus) method can be used to estimate the plane. Once the plane is estimated, the process moves to step S72.

<<Step S72>>
In step S72, points (feature points) that are equal to or greater than a predetermined threshold in the direction of gravity are detected based on the plane estimated in step S71, and transmitted to the feature point detection section. The operation of the feature point detection section will be described later. If a point equal to or greater than a predetermined threshold is detected, the process moves to step S73. Note that since the feature points are part of the point cloud, the point cloud information shown in FIG. 7 is also linked.

≪Step S73≫
In step S73, points equal to or greater than a predetermined threshold are grouped, and vectors representing the identification ID, origin, and size are saved. This grouping identifies "obstacles" by capturing mutually related points as a group (point cloud).

FIG. 11 shows the concept of grouping, where "O" indicates a feature point on a plane, and "●" indicates a feature point of an obstacle. Since the feature points of the obstacle are related to each other by a relative distance, they can be grouped, and furthermore, they can be represented by a size vector (xyz) based on a predetermined origin position. When the storage of the size vector (xyz) is completed, the process moves to step S74.

<<Step S74>>
In step S74, an unevenness flag is set for each of the grouped points. This unevenness flag is a point group (obstacle) that protrudes upward from the reference plane, or a point group (obstacle) that is depressed downward from the reference plane. This is a flag indicating whether In this embodiment, a "+" (convex flag) is set for a point group that protrudes upward, and a "-" (concave flag) is set for a point group that is depressed downward. When the setting of the unevenness flag is completed, the process moves to step S75.

<<Step S75>>
In step S75, a group identification ID for identifying each grouped point group (unevenness group) is set. An example of group identification ID is shown in FIG. 12.

In FIG. 12, the information on the grouped point group is composed of data having information such as the origin position of the orthogonal coordinate system, the size vector, and the unevenness flag. For example, for group identification ID (G1), origin position (P1), size vector (S1), and concave flag (-1) are defined, and for group identification ID (G2), origin position (P2), size vector (S2), a convex flag (+1) is defined. When the setting of the group identification ID is completed, the process moves to step S76.

<<Step S76>>
In step S76, the group identification ID obtained in step S75 (including the above-mentioned origin position, size vector, unevenness flag, etc.) is transmitted to the risk determination section 30. As a result, the danger determining unit 30 determines the presence and size of obstacles, and determines whether or not there is an obstacle to the landing of the aircraft 10. The control operation of the danger determining section 30 will be described later. When the transmission of the group identification ID is completed, the process exits to the end and ends the process.

[Feature point detection flow]
Next, a feature point detection method for detecting feature points by recognizing the plane and obstacles (uneven objects) around the landing site in step S32 will be explained using FIGS. 13 and 14.

<<Step S80>>
In step S80, the latest 3D shape information around the landing site is acquired from the 3D shape memory section 24. Once the 3D shape information around the landing site is acquired, the process moves to step S81.

<<Step S81>>
In step S81, changes in shape, color, and reflection intensity (difference, slope, etc.) are calculated in the 3D shape information around the landing site. This is to detect the occurrence of a new feature point. The new feature point is assumed to be, for example, a case where a vehicle approaches the vicinity of the landing site. Once changes in shape, color, and reflection intensity (difference, slope, etc.) are calculated, the process moves to step S82.

<<Step S82>>
In step S82, points (feature points) for which the amount of change in shape, color, and reflection intensity determined in step S81 exceed a predetermined threshold are extracted. When a new feature point is detected, the process moves to step S83.

<<Step S83>>
In step S83, all feature points of the area to be compared are acquired from the feature point storage unit 27. Once all feature points have been acquired, the process moves to step S84.

<<Step S84>>
In step S84, the feature point acquired in step S84 is compared (matched) with the feature point for which the amount of change found in step S82 exceeds a threshold value. This makes it possible to grasp the occurrence of new feature points. When the comparison of each feature point is completed, the process moves to step S85.

<<Step S85>>
In step S85, it is determined whether a new feature point (unmatched point) exists using the comparison result in step S84. If it is determined that no new feature points exist, the process exits to END and ends the process. On the other hand, if it is determined that a new feature point exists, the process moves to step S86.

<<Step S86>>
In step S86, an identification ID is added to the new feature point, and the new feature point is updated and stored in the feature point storage unit 27 together with the position information. FIG. 14 shows an example of storing feature points. In FIG. 14, the identification ID information is composed of data having information such as the position (xyz) of the orthogonal coordinate system and the unevenness flag. For example, for identification ID (D1), position (P1 = x1, y1, z1) and concave flag (0/plane) are defined, and for identification ID (D2), position (P2 = x2, y2, z2) is defined. , a concave flag (-1/concave part) are defined, and for the identification ID (D3), a position (P3=x3, y3, z3) and a concave flag (+1/convex part) are defined. When a new feature point is stored in the feature point storage section 27, the process exits to END and ends the process.

[Feature point relative position measurement flow]
Next, a feature point relative position measuring method for measuring the relative position of the feature point in step S32 will be described using FIGS. 15 and 16.

≪Step S90≫
In step S90, it is determined whether the new feature point stored in step S86 of FIG. 13 exists in the feature point storage unit 27. If it is determined that no new feature points exist, the process exits to END and ends the process. On the other hand, if it is determined that a new feature point exists, the process moves to step S91.

<<Step S91>>
In step S91, new feature points are acquired from the feature point storage section 27. When a new feature point is detected, the process moves to step S92.

<<Step S92>>
In step S92, all combinations of related feature points including new feature points are determined, relative position groups are generated for all of the determined combinations, and group IDs are added to these groups. This forms groups of related feature points. Obstacles and the like can be identified by generating this relative position group. Once the relative position group is generated, the process moves to step S93.

<Step S93>
In step S93, the relative distance of each feature point in the relative position group is calculated. Returning to Fig. 9, in a group consisting of a combination of feature points "◯" and "●", the relative distance of each feature point can be found from the coordinate position of a coordinate system including three dimensions, and can be found as "a", "b", and "c", respectively. The unit is "meter (m)", and the size of the obstacle can be determined from this.

If the size (physique) of the obstacle is large, it can be recognized as a dangerous object that will hinder the landing of the flying object 10. For example, in the risk judgment flow described later, the risk judgment can be executed using information on the relative position group. When the calculation of the relative distance between the feature points of the relative position group is completed, the process moves to step S94.

<<Step S94>>
In step S94, the identification ID and relative distance of the relative position group obtained in steps S92 and S93 are stored in the relative position group storage section 29 and updated. An example of relative position group information is shown in FIG. 16. In FIG. 16, the identification ID information of a relative position group is composed of data having information such as the feature points and relative distances of each of the relative position groups.

For example, for the relative position group identification ID (RG1), as shown in FIG. ) is defined, and for the relative position group identification ID (RG2), the identification IDs (1, 4, 2) of related feature points and the relative distances between the respective feature points are defined. When the new relative position group ID and relative distance are stored in the relative position group storage section 29, the process exits to END and ends the process.

[Risk judgment flow]
Next, a risk determination method for determining whether a new obstacle exists at the landing site in step S41 of FIG. 4 will be described using FIG. 17.

<<Step S100>>
In step S100, the current position and attitude of the flying object 10, obstacles (unevenness groups) near the landing site, and feature points are acquired. Once these pieces of information are acquired, the process moves to step S101.

<<Step S101>>
In step S101, a landing point perpendicular to the direction of gravity and its surroundings are calculated from the current position and attitude of the flying object 10. This makes it possible to specify the ground surface on which the flying object 10 is about to land. Once the area around the landing site is specified, the process moves to step S102.

<<Step S102>>
In step S102, it is determined whether an obstacle or a new feature point (obstacle) other than a plane exists around the landing site. When executing the landing mode, obstacles and feature points on the plane have already been detected, a landing point has been set, and the flying object 10 is descending toward this point. Therefore, if a vehicle or the like approaches the landing site, it may become a new obstacle. In order to avoid this new obstacle, in this control step it is determined whether or not a new feature point has occurred.

In this control step, the process exits to the end where it is determined that no new feature point exists, and the process ends. On the other hand, if it is determined that a new feature point exists, the process moves to step S103.

<<Step S103>>
In step S103, if the flying object 10 continues the landing operation as it is, there is a risk that it will come into contact with a new obstacle, so a signal to stop the descent of the flying object 10 is output to the operation state management section 33. After transmitting the output of the descent stop signal to the operating state management unit 33, the process moves to step S104.

<<Step S104>>
In step S104, the closest landing point to the landing point is newly calculated, where there are no obstacles (unevenness groups) around the landing point or new feature points other than flat surfaces. Of course, even in this case, it goes without saying that a plane having an area on which the flying object 10 can land is calculated. Once the new landing site is calculated, the process moves to step S105.

<<Step S105>>
In step S105, the new landing site determined in step S104 is output to the operating state management unit 33. When the new landing site is output, the exit process ends at the end. Note that the descent stop signal from step S103 and the new landing site from step S104 are used in the operating state management state flow described below.

[Operating state management flow]
Next, a landing point correction method for correcting the landing point in step S42 in FIG. 4 will be described using FIG. 18.

<<Step S110>>
In step S110, the current position and attitude of the flying object 10 is acquired from the position and attitude recognition unit 31. Once the position and attitude of the flying object 10 are acquired, the process moves to step S111.

<<Step S111>>
In step S111, the current position/attitude is matched with the map information in the map DB, and the current state of the aircraft 10 is determined. Once the current state of the aircraft 10 is grasped, the process moves to step S112.

<<Step S112>>
In step S112, it is determined whether or not a descent stop signal is output from the danger determining section 30 (output from step S103 in FIG. 17). If it is determined that there is no descent stop signal, the process proceeds to step S114, and if it is determined that there is a descent stop signal, the process proceeds to step S113.

<<Step S113>>
In step S113, if the flying object 10 continues the landing operation as it is, there is a risk that it will come into contact with a new obstacle, so the operating state management unit 33 changes the operating mode to "standby mode". By changing to the "standby mode", the flying object 10 hovers and comes to a halt at its current position. When the mode is changed to "standby mode", the process moves to step S114.

<<Step S114>>
In step S114, it is determined whether there is an output of a new landing site from the danger determination unit 30 (output from step S105 in FIG. 17). If it is determined that there is no output from a new landing site, the process moves to step S117, and if it is determined that there is an output from a new landing site, the process moves to step S115.

<<Step S115>>
In step S115, the new landing site determined in step S114 is output to the position/attitude control unit 35. As a result, a new landing site for the aircraft 10 is set. When the new landing site is output to the position/attitude control unit 35, the process moves to step S116.

<<Step S116>>
In step S116, the operating state management unit 33 changes the operating mode to "approach mode" in order to move the flying object 10 to the landing site. By changing to the "approach mode", the flying object 10 starts moving toward a new landing site. When the mode is changed to "approach mode", the process moves to step S117.

<<Step S117>>
In step S117, it is determined whether the operation mode is "approach mode" and whether or not the aircraft has arrived at a new landing point. If the landing site has not been reached in the "approach mode" state, the process moves to step S119, and if the landing site has been reached in the "approach mode" state, the process moves to step S118.

<<Step S118>>
In step S118, since the aircraft 10 has arrived at the landing site in the "approach mode", the operation mode is changed to "landing mode" and the aircraft 10 is lowered to the new landing site. Control. When the mode is changed to "landing mode", the process moves to step S119.

<<Step S119>>
In step S119, any one of the operating modes of "standby mode" in step S113, "approach mode" in step S116, and "landing mode" in step S118 is output to the environment recognition unit 21. After that, the process exits to the end and ends the process.

By executing the landing control described above, at least (1) the altitude of the drone changes to a lower direction and the landing range deviates from the sensing range of the sensor camera, or (2) there is a danger within the landing range. If the object moves, or (3) if it becomes difficult to track the feature points necessary for position/attitude control, the drone will be able to land at the intended landing position.

Note that the present invention is not limited to the above-described embodiments, and includes various modifications. For example, the embodiments described above are described in detail to explain the present invention in an easy-to-understand manner, and the present invention is not necessarily limited to having all the configurations described. Furthermore, it is possible to replace a part of the configuration of one embodiment with the configuration of another embodiment, and it is also possible to add the configuration of another embodiment to the configuration of one embodiment. Further, it is possible to add, delete, or replace a part of the configuration of each embodiment with other configurations.

DESCRIPTION OF SYMBOLS 10... Aircraft, 11... Housing body, 12... Blade rotor, 13... Long range narrow angle space sensor, 14... Middle range wide angle space sensor, 15... Short range wide angle space sensor, 16... Altitude sensor.

Claims (8)

a plurality of feature point detection means for detecting feature points around the landing site; feature point recognition means for recognizing the feature points detected by the feature point detection means; and altitude measurement means for measuring the altitude of the aircraft. A landing control device for an aircraft, comprising: and a landing control means for automatically landing the aircraft at the landing site,
The landing control means measures the relative positions of the plurality of feature points in accordance with the altitude, selects the feature point detection means for measuring each of the feature points in accordance with the altitude, and measures the relative positions of the plurality of feature points in accordance with the altitude. A landing control system for an aircraft, characterized in that it determines whether there is a change in the shape of the landing site based on a change in position, and if there is a change in shape, moves the aircraft to a new landing site. The landing control device for a flying object according to claim 1,
The landing control means assigns reliability to the feature points measured by the plurality of feature point detection means according to the altitude measured by the altitude measurement means, and assigns reliability to the measured feature points. A landing control device for a flying object, which is characterized by tracking according to the following. The landing control device for a flying object according to claim 1,
The landing control means stops the descent of the flying object and maintains the flying object at the current position when a new obstacle that becomes an obstacle to landing is detected while the flying object is landing. A distinctive feature of the aircraft's landing control system. The landing control device for a flying object according to claim 1,
When the landing control means detects a new obstacle that obstructs landing while the flying object is landing, the landing control means determines a new landing point and moves the flying object to the new landing point. A landing control device for an aircraft, characterized by: The landing control device for a flying object according to claim 3 or 4,
A landing control system for a flying object, wherein the landing control means determines the obstacle that becomes an obstacle to landing based on the size of unevenness with respect to a plane of the landing site. The landing control device for a flying object according to claim 5,
The landing control means forms the obstacle that becomes an obstacle to landing from a plurality of mutually related feature points grouped together, and determines the size of the obstacle from the relative position of the plurality of grouped feature points. A landing control device for an aircraft, characterized by: The landing control device for a flying object according to claim 1,
A landing control system for an aircraft, wherein the feature point detection means is a space sensor (LIDAR/light detection and ranging) that detects the shape of the ground surface as point group information. The landing control device for a flying object according to claim 7,
Landing control of an aircraft, characterized in that at least two space sensors are provided, one of which is a long-range, narrow-angle space sensor with a narrow field of view, and the other is a short-range, wide-angle space sensor with a wide field of view. Device.

Images