JP7539688B2 - Unmanned aerial vehicle descent system - Google Patents

Description

The present invention relates to a descent system for an unmanned aerial vehicle.

The use of small unmanned aerial vehicles (also called "drones") has been proposed. A technology has been proposed in which such drones descend to aim at targets (also called "markers") placed on the ground when landing (see, for example, Patent Document 1).

Patent Application No. 2019-501868

In the above-mentioned technology, the drone photographs the target with a camera and descends towards the target. The camera has a range that it can photograph (defined, for example, by the "angle of view"; hereafter referred to as the "photography range"), and the lower the flying altitude of the drone, the smaller the area of the ground within the photography range becomes. For this reason, when the drone lowers its altitude, eventually some or all of the markers will fall outside the photography range, and the drone may not be able to land accurately at the designated landing position.

The present invention attempts to solve this problem, and aims to provide a descent system for an unmanned aerial vehicle that can descend to a precise target location.

The first invention is a descent system for landing an unmanned aerial vehicle at a target position, the unmanned aerial vehicle flying by controlling the rotation of propellers connected to the rotating shafts of multiple motors, the system having a target marker arranged in an area including the target position, the target marker being composed of multiple levels of hierarchical markers having different external sizes in the horizontal direction, the hierarchical markers having relatively small external sizes being arranged inside the hierarchical markers having relatively large external sizes, the hierarchical markers of the multiple levels including the target position on the inside in the horizontal direction, the unmanned aerial vehicle detecting the relative positional relationship of each part constituting the hierarchical marker. The descent system has relative information, which is information indicating the location of the object, and an image acquisition means capable of capturing images below, and when the relatively large-sized floor sign can be recognized within the capture range of the image acquisition means, the unmanned aerial vehicle is controlled to descend so that the relatively large-sized floor sign is located within the capture range, and when the relatively large-sized floor sign cannot be recognized within the capture range as a result of the descent of the unmanned aerial vehicle, the unmanned aerial vehicle is controlled to descend so that the relatively small-sized floor sign is located within the capture range.

First, the technical concept of the present invention will be described with reference to Figures 23 to 25. As shown in Figure 23, if a hollow tube-shaped frame member W1 is placed in the air between the position of the unmanned aerial vehicle 1 located in the air and the target position CP0, the unmanned aerial vehicle 1 can land accurately at the target position CP0 by descending while controlling the position so that it passes through the frame member W1. Also, if a frame member W2 whose inner shape gradually becomes smaller from top to bottom is placed instead of the frame member W1 as shown in Figure 24, the unmanned aerial vehicle 1 descends while controlling the position so that it passes through the frame member W2, and as it descends, the horizontal position approaches the vertical direction of the target position CP0. Furthermore, as shown in Figure 25, the same result can be obtained by placing multiple frames W2a to W2d in the air instead of one frame member W2, and by reducing the size of the frame from the upper frame W2a to the lower frame W2d, the unmanned aerial vehicle 1 approaches the target position CP0 as it descends. However, it is not realistic to place the frame members W1, W2, or W2a to W2d in any air. In this respect, the present invention is configured to achieve an effect that is technically equivalent to passing through multiple frames W2a, etc., by descending while recognizing multiple levels of floor signs. That is, according to the configuration of the first invention, when the unmanned aerial vehicle 1 has a relatively high flight altitude and the outer floor sign can be recognized within the shooting range, it descends while maintaining the outer floor sign within the shooting range, and when the outer floor sign cannot be recognized within the shooting range as a result of the unmanned aerial vehicle descending, it can descend while maintaining the inner floor sign within the shooting range. The unmanned aerial vehicle 1 controls the position of the unmanned aerial vehicle 1 so that the floor sign is within the shooting range (hereinafter referred to as "shooting position control"). The technology of descending while performing shooting position control is technically equivalent to descending while passing through multiple frames W2a, etc. Furthermore, by descending based on the outer hierarchical marker at relatively high altitudes and descending based on the inner hierarchical marker at relatively low altitudes, the horizontal position of the unmanned aerial vehicle approaches a vertical line passing through the target position as it descends (hereinafter referred to as the "descent approach effect"). This allows the unmanned aerial vehicle to descend to an accurate target position by referring to the target marker. Also, because the unmanned aerial vehicle has relative information, it can descend while adjusting its nose direction in a specified direction.

The second invention is a descent system in which, in the configuration of the first invention, the unmanned aerial vehicle, when descending while controlling the unmanned aerial vehicle so that the relatively large-sized floor level sign is positioned within the shooting range, also recognizes the relatively small-sized floor level sign, and switches to control for descending while controlling the relatively small-sized floor level sign to be positioned within the shooting range before the relatively large-sized floor level sign becomes unrecognizable within the shooting range as a result of the descent of the unmanned aerial vehicle.

According to the configuration of the second invention, when the unmanned aerial vehicle descends while controlling the outer floor sign so that it is positioned within the shooting range, it already recognizes the inner floor sign, and before the outer floor sign becomes unrecognizable, the control is switched to descending while controlling the inner floor sign so that it is positioned within the shooting range, so that shooting position control can be carried out without interruption.

The third invention is a descent system in which, in the configuration of either the first or second invention, the innermost level sign is sized to have an external shape that can be recognized within the shooting range of the unmanned aerial vehicle even when the unmanned aerial vehicle has landed.

The configuration of the third invention allows for continuous control of the shooting position until the moment of landing, ensuring that the drone lands at the target position.

The fourth invention is a descent system in which, in the configuration of any one of the first to third inventions, the multi-stage hierarchical markers are composed of multiple element markers, the relative information is information indicating the direction and distance of the target position based on each of the element markers, and the unmanned aerial vehicle controls the nose direction of the unmanned aerial vehicle itself in a predetermined direction based on the direction and distance of the target position indicated by the element markers.

According to the fourth aspect of the invention, the unmanned aerial vehicle can recognize the element marker and control the nose direction to a specified direction while descending toward the target position.

The fifth invention is a descent system in which, in the configuration of the fourth invention, the unmanned aerial vehicle has an estimation means for estimating the entirety of the hierarchical marker based on the element marker that is a part of the hierarchical marker, and descends while controlling the estimated hierarchical marker so that it is positioned within the shooting range.

According to the configuration of the fifth invention, even if part of a floor sign cannot be recognized due to a shadow or the like, the entire floor sign can be estimated and the shooting position control can be performed while descending.

The sixth invention is a descent system in the configuration of the fourth or fifth invention, in which a charging device for charging a rechargeable power source of the unmanned aerial vehicle is arranged in an area including the target position, the unmanned aerial vehicle lands at the target position with the nose facing the specified direction, and the charging device is configured to charge the power source of the unmanned aerial vehicle.

According to the configuration of the sixth invention, the unmanned aerial vehicle can land while controlling the nose direction in a specified direction, so that, for example, when the charging terminal of the charging device is set to a specific position, the charging terminal on the unmanned aerial vehicle can be connected to the charging terminal on the charging device.

The seventh invention is a descent system having a configuration of any one of the first to sixth inventions, the descent system having a plurality of the unmanned aerial vehicles and a plurality of the target markers, each of the unmanned aerial vehicles being assigned a different target marker, and each of the unmanned aerial vehicles being configured to descend toward the target position indicated by the assigned target marker.

According to the seventh aspect of the invention, multiple unmanned aerial vehicles can descend to their respective target positions with high accuracy.

The eighth invention is a descent system according to any one of claims 1 to 7, in which the unmanned aerial vehicle is configured to recognize the target sign based on the degree of correlation between data indicating the characteristics of the target sign that it has in advance and characteristic data extracted from an image in the image data acquired by the image acquisition means, and the data indicating the characteristics of the target sign includes data generated by deep learning.

Even the same target sign can have a variety of appearances depending on factors such as the intensity of light in the environment. In this regard, according to the configuration of the eighth invention, various appearances of the same target sign can be learned through deep learning, and the target sign can be recognized using the data that indicates the characteristics of the target sign generated as a result. Therefore, the target sign can be recognized with high accuracy even when influenced by changes in the environment.

The present invention allows for accurate landing at the target location.

A schematic diagram showing an example of operation of the descent system related to the first embodiment of the present invention. 1 is a schematic perspective view showing an unmanned aerial vehicle. 1 is a schematic diagram showing the functional blocks of an unmanned aerial vehicle. FIG. 2 is a schematic plan view showing an example of a target marker. FIG. 13 is a schematic plan view showing the outer level markers. FIG. 13 is a schematic plan view showing inner level indicators. A conceptual diagram showing the structure of data related to target markers stored by an unmanned aerial vehicle. 10 is a schematic plan view showing the relationship between floor signs and the imaging range of an image acquisition means. FIG. 13 is a schematic plan view showing the relationship between the outer floor signs and the shooting range of the image acquisition means. FIG. 13 is a schematic plan view showing the relationship between the inner floor signs and the shooting range of the image capture means. FIG. 1 is a schematic diagram showing the relationship between the altitude of an unmanned aerial vehicle and the shooting range. 1 is a schematic diagram showing the relationship between the altitude of an unmanned aerial vehicle and the shooting range. A schematic diagram showing an example of a descending trajectory of an unmanned aerial vehicle. 1 is a schematic flowchart for explaining the operation of an unmanned aerial vehicle. 11 is a schematic flowchart for explaining the operation of an unmanned aerial vehicle according to a second embodiment of the present invention. 11 is a schematic flowchart for explaining the operation of an unmanned aerial vehicle according to a third embodiment of the present invention. FIG. 11 is a schematic diagram showing an example of operation of a descent system according to a fourth embodiment of the present invention. FIG. 1 is a schematic diagram showing an example of operation of the descent system. FIG. 2 is a schematic diagram illustrating an example of image estimation. FIG. 2 is a schematic diagram illustrating an example of image estimation. A schematic diagram showing an example of operation of a descent system according to a fifth embodiment of the present invention. A conceptual diagram showing the structure of data regarding target markers stored in an unmanned aerial vehicle in accordance with the sixth embodiment of the present invention. FIG. 1 is a conceptual diagram for explaining the technical idea of the present invention. FIG. 1 is a conceptual diagram for explaining the technical idea of the present invention. FIG. 1 is a conceptual diagram for explaining the technical idea of the present invention.

The following provides a detailed description of the form for implementing the present invention (hereinafter, "embodiment"). In the following description, the same components are given the same reference numerals, and their description will be omitted or simplified. Descriptions of components that can be implemented appropriately by a person skilled in the art will be omitted, and only the basic components of the present invention will be described.

The drone 1 shown in FIG. 1 is configured to obtain thrust from the rotation of a propeller and fly autonomously along a predetermined route. The drone 1 is an example of an unmanned flying vehicle.

The drone 1 flies autonomously along route R1 in the specified area 300. The drone 1 can fly autonomously along route R1 while receiving positioning radio waves from navigation satellites such as GPS satellites (Global Positioning System) and determining the position of the drone 1 itself.

The specified area 300 includes structures such as mountains 200A to 200F, a golf course 202, a zoo 204, a field 206, a pond 208, and a road 210. The drone 1 flies while photographing the specified area 300. The drone 1 takes off from the take-off and landing area 90 and returns to the take-off and landing area 90. The target position CP0 for the drone 1 to land is located within the take-off and landing area 90.

As shown in FIG. 2, the drone 1 has a housing 2. The housing 2 contains a computer that controls each part of the drone 1, an autonomous flight device, a wireless communication device, a positioning device that uses positioning radio waves from a navigation satellite system such as GPS (Global Positioning System), a battery, etc. In addition, a camera 14 is disposed in the housing 2 via a fixing device 12. The camera 14 is an example of an image acquisition means.

Uranium 1 acquires images of the external area (area below) using camera 14. Uranium 1 can also capture images of the direction directly below it using camera 14. Camera 14 is a visible light camera, but may alternatively be a hybrid camera that can switch between a visible light camera and a near-infrared camera. However, drone 1 uses the visible light camera when descending toward target position CP0. Hereinafter, the image captured by camera 14 will be referred to as the "camera image."

The fixing device 12 is a three-axis fixing device (a so-called gimbal) that minimizes camera image blur and can control the optical axis of the camera 14 in any direction.

A round-rod-shaped arm 4 is connected to the housing 2. A motor 6 is connected to each arm 4, and a propeller 8 is connected to each motor 6. Each motor 6 is a direct current motor (brushless DC motor). Each motor 6 is independently controlled, and the drone 1 can be freely moved up and down and horizontally, stopped in the air (hovering), and changed in nose direction. When the drone 1 moves forward, backward, left, or right, the number of rotations of the two motors 6 on the rear side of the traveling direction is made greater than the number of rotations of the two motors 6 on the traveling direction side. When the drone 1 changes the nose direction while maintaining the horizontal position, the number of rotations of the two diagonal motors 6 is made greater than the number of rotations of another set of two diagonal motors 6. The drone 1 is configured to rotate the adjacent motors 6 in opposite directions to cancel out the reaction force. When the drone 1 changes the nose direction while maintaining the horizontal position, it uses the reaction force without canceling it out. The drone 1 can descend vertically by gradually reducing the rotation speed of all motors 6.

A protective frame 10 is connected to the arm 4 to prevent the propeller 8 from coming into direct contact with external objects. The arm 4 and protective frame 10 are made of, for example, carbon fiber reinforced plastic, and are constructed to be lightweight while maintaining strength.

Figure 3 is a diagram showing the functional configuration of the drone 1. As shown in Figure 3, the drone 1 has a CPU (Central Processing Unit) 50, a memory unit 52, a wireless communication unit 54, a satellite positioning unit 56, an inertial sensor unit 58, a drive control unit 60, an image processing unit 62, and a power supply unit 64.

UAV 1 is capable of communicating with a control device (not shown) that wirelessly transmits instructions such as launch to UAV 1 via wireless communication unit 54. The control device is a computer operated by an administrator of UAV 1, and is capable of wireless communication.

UAV 1 can measure its own position by satellite positioning unit 56. Satellite positioning unit 56 basically receives positioning radio waves from four or more navigation satellites to measure the position of drone 1. As described above, drone 1 autonomously flies route R1 based on the measured position. Inertial sensor unit 58 outputs the movement of drone 1 by, for example, an acceleration sensor and a gyro sensor. Information from inertial sensor unit 58 is output to an autonomous flight device and used for attitude control of drone 1. Attitude control means correcting the drone 1 when it deviates from the planned inclination. For example, if the height of the rotation plane of the four propellers 8 of drone 1 is the same, the horizontal state, then the inclination is the deviation from the horizontal state. For example, when drone 1 is stopping in the air (hovering) or descending vertically, the planned inclination is 0, and drone 1 maintains the horizontal state.

The drive control unit 60 controls the rotation of the propellers 8 (see Figure 2) connected to each motor 6 (see Figure 2) of the drone 1, thereby controlling vertical and horizontal movement, airborne stopping, and nose direction.

The image processing unit 62 enables the drone 1 to operate the camera 14 (see FIG. 2) to acquire an external image (camera image). When the drone 1 descends toward the target position CP0, the optical axis of the camera 14 is fixed vertically downward.

The power supply unit 64 is, for example, a replaceable rechargeable battery, and is configured to supply power to each part of the drone 1.

The memory unit 52 stores various data and programs necessary for autonomous movement, such as data showing a movement plan for autonomously moving from a starting point to a destination position, as well as data related to the target marker 100 and a landing program. The CPU 50 and the landing program are an example of a landing control means.

The landing program is a program for controlling the drone 1 to lower its flight altitude and descend toward the target position CP0. When the drone 1 determines, based on its current position determined by the satellite positioning unit 56, that it has reached the sky above the target position CP0, the landing program controls the rotation of the motor 6 via the drive control unit 60, gradually lowering its altitude and descending toward the target position CP0.

Now, with reference to Figures 4 to 8, the target sign 100 placed in the landing area 90 will be described. The target sign 100 shown in Figure 4 is placed in the landing area 90. The target sign 100 is an example of a target sign. The landing area 90 is an example of an area that includes the target position CP0. The drone 1 and the target sign 100 form a descent system.

As shown in FIG. 4, the target sign 100 is composed of multiple levels of hierarchical signs 102 and 104 with different external sizes in the horizontal direction. The inner hierarchical sign 104 has a smaller external size in the horizontal direction than the outer hierarchical sign 102. The inner hierarchical sign 104 is arranged inside the outer hierarchical sign 102. The outer hierarchical sign 102 has a square external shape, and the length L1 of one side is, for example, 130 cm (centimeters). The inner hierarchical sign 104 has a square external shape, and the length L2 of one side is, for example, 20 cm (centimeters). The inner hierarchical sign 104 is regulated to a size of an external shape that is entirely within the shooting range of the camera 14 and recognizable by the unmanned aircraft 1 even when the unmanned aircraft 1 lands. Specifically, the inner hierarchical sign 104 is regulated to a size of an external shape that is entirely within the shooting range of the camera 14 when the unmanned aircraft 1 lands. In this embodiment, the length of the diagonal of the square outline 104out of the inner floor sign 104 (see FIG. 6) is equal to the diameter of the circle that is the shooting range of the camera 14 when the drone 1 lands.

The target position CP0 for the drone 1 to land is located inside the hierarchy signs 102 and 104. That is, the hierarchy signs 102 and 104 each include a target position CP0 inside. The target position CP0 is a coordinate indicated by a specific latitude and longitude. In this embodiment, the target position CP0 is the center position of the outer shape 102out of the hierarchy sign 102 (see FIG. 5). The target position CP0 is also the center position of the outer shape 104out of the hierarchy sign 104 (see FIG. 6).

The hierarchical markers 102 and 104 are each composed of a number of element markers. As shown in Figures 4 and 5, the outer hierarchical marker 102 is composed of element markers 102a to 102h and 102in. The element marker 102in is configured as a square outline, and the element markers 102a to 102h are formed adjacent to the outer periphery of the element marker 102in. As shown in Figure 6, the inner hierarchical marker 104 is composed of part markers 104a to 104i.

Figure 7 is a conceptual diagram for explaining an example of data related to the target marker 100. In Figure 7, the target marker is designated as target marker X1, which is composed of an outer hierarchical marker A and an inner hierarchical marker B, with hierarchical marker A composed of element markers Aa and Ab, and hierarchical marker B composed of element markers Ba and Bb.

The data related to the target marker X1 includes position data indicating the coordinates of the target marker X1, identification data of the target marker X1, feature data indicating the features of the target marker X1, and data indicating the relationship between the target marker X1 and the target position. The position indicated in the position data of the target marker X1 is the same as the position of the target position. The feature data is data indicating features such as the feature points of the object, the contour, and the direction of each component. The feature data is data used in tracking methods based on feature-point-based images such as the KLT (Kanada-Lucas-Tomasi) method and SIFT (Scale Invariant Feature Transform). The feature data of the target marker X1 is data indicating features such as the feature points of the target marker X1 as a whole, the contour, and the direction of each component.

The data relating to hierarchy marker A includes position data of hierarchy marker A, identification data of hierarchy marker A, feature data of hierarchy marker A, and data indicating the relationship between hierarchy marker A and the target position. The content of the feature data is the same as the feature data of target marker X1 described above. The position indicated in the position data of hierarchy marker A is the same as the position of the target position. The structure of the data relating to hierarchy marker B is the same as the data structure of hierarchy marker A described above.

The data related to the element sign Aa includes position data of the element sign Aa, identification data of the element sign Aa, characteristic data of the element sign Aa, and data indicating the relationship between the element sign Aa and the target position. The position data of the element sign Aa is data indicating the reference position of the element sign Aa itself, and is different from the target position. The reference position of the element sign Aa is, for example, the center position of the element sign Aa in the horizontal direction. The data indicating the relationship between the position data of the element sign Aa and the target position is data indicating the direction and distance of the target position based on the reference position of the element sign Aa. The configurations of the other element signs Ab, Ba, and Bb are similar to those described above. That is, the drone 1 stores information indicating the direction and distance of the target position based on each element sign in the memory unit 52. The information indicating the direction and distance of the target position based on each element sign is an example of relative information.

The landing program includes a recognition target determination program and a target sign recognition program. The recognition target determination program and CPU 50 are an example of a recognition target determination means, and the target sign recognition program and CPU 50 are an example of a target sign recognition means. The recognition target determination program is a program for the drone 1 to determine the recognition target to be recognized in the camera image. In this embodiment, the drone 1 uses the recognition target determination program to determine either the hierarchical signs 102 or 104 as the recognition target. The target sign recognition program is a program for the drone 1 to recognize the recognition target.

When the drone 1 determines that it has reached the sky above the target position CP0, it uses the camera 14 to capture an image of the vertical downward direction. The drone 1 then uses a recognition target determination program to set the outer hierarchical sign 102 as the recognition target. As described below, if the drone 1 descends and the outer hierarchical sign 102 falls out of the camera image and cannot be recognized, it sets the inner hierarchical sign 104 as the recognition target.

The drone 1 uses a target sign recognition program to extract features in the camera image, and recognizes the hierarchical sign 102 or 104 set as the recognition target in the camera image based on the feature data of the hierarchical signs 102 and 104 previously stored in the memory unit 52. Specifically, the drone 1 recognizes the hierarchical sign 102 or 104 by comparing the features in the camera image with the feature data stored in the memory unit 52 and determining the correlation (degree of correlation). The drone 1 also uses the target sign recognition program to recognize the element signs that make up the hierarchical sign 102 or 104 set as the recognition target.

The drone 1 performs a feature-based tracking method using a target sign recognition program. For example, the drone 1 extracts a large number of features from the camera image, such as the contour and the direction of each component, and recognizes the features of the object projected on the camera image (hereinafter referred to as the "camera image features"). The drone 1 compares the features of the camera image with the feature data stored in the memory unit 52 to determine the correlation (degree of correlation). The higher the degree of correlation, the more likely it is that the object in the camera image is the hierarchical signs 102 and 104, or a specific element sign. For example, when the degree of correlation is 0, the possibility that the area in the camera image is a specific hierarchical sign or element sign (hereinafter referred to as the "category common probability") is 0%, and when the degree of correlation is at its maximum, the category common probability is defined as 100%. The drone 1 determines that the object in the camera image is a specific hierarchical sign or element sign when the category common probability is a predetermined reference value, for example, 95% or more.

Figure 8 shows the relationship between the shooting range of the camera 14 of the drone 1 and the hierarchy markers 102 and 104. When the drone 1 descends with the shooting direction of the camera 14 facing vertically downward, the area of the ground surface within the shooting range of the camera 14 becomes narrower as the altitude of the drone 1 decreases. The angle of view of the camera 14 is constant. The optical axis of the camera 14 is fixed vertically downward. For example, as the altitude of the drone 1 decreases, the area of the ground surface within the shooting range of the camera 14 becomes narrower, such as within the range of circle S1, the range of circle S2, the range of circle S3, the range of circle S4, and the range of circle S5. And as the altitude of the drone 1 decreases, the size of the hierarchy markers 102 and 104 in the camera image increases. From another perspective, this means that as the altitude of the drone 1 decreases, the proportion of the hierarchy markers 102 and 104 in the pixels in the camera image increases.

As the altitude of the drone 1 decreases, the state changes from one in which both the outer hierarchical sign 102 and the inner hierarchical sign 104 are within the camera 14's shooting range to one in which the outer hierarchical sign 102 falls outside the shooting range and only the inner hierarchical sign 104 is within the shooting range.

When the landing program causes the outer hierarchical sign 102 to enter the shooting range and the entirety of the hierarchical sign 102 can be recognized, the drone 1 descends while controlling the position of the drone 1 so that the hierarchical sign 102 is located within the camera image. In other words, the drone 1 descends while performing shooting position control based on the hierarchical sign 102. Then, when the drone 1 descends and the hierarchical sign 102 does not fit within the camera image and the entirety of the hierarchical sign 102 cannot be recognized, the drone 1 is configured to descend while controlling the position of the drone 1 so that the inner hierarchical sign 104 is located within the camera image. That is, the drone 1 first descends while performing shooting position control based on the hierarchical sign 102, and when the hierarchical sign 102 falls outside the shooting range, it switches to shooting position control based on the hierarchical sign 104. However, since the attitude and position of the drone 1 fluctuate due to the influence of wind, etc., the target position CP0, which is the center position of the hierarchical signs 102 and 104, is not always located at the center of the camera image, as shown in FIG. 8.

The control of the drone 1's descent will be specifically described with reference to Figures 9 and 10. As shown in Figure 9, the drone 1 descends while controlling its position so that the outer hierarchical sign 102 is within the camera image. At a relatively high altitude, for example, as shown in Figure 9, in the shooting range S11 of the camera image, the center position of the hierarchical sign 102 is shifted to the left, but as the drone 1 lowers in altitude and reaches shooting ranges S12, S13, S14, and S15, the center position of the hierarchical sign 102 approaches and moves away from the center of the camera image, but overall gradually approaches the center, and finally reaches a state close to the center. In this way, a descent approach effect is achieved by descending while performing shooting position control.

The same is true for the inner hierarchical sign 104. For example, as shown in FIG. 10, even if the center position of the hierarchical sign 104 is shifted to the right in the shooting range S21 of the camera image, as the drone 1 descends in altitude and moves to shooting ranges S22, S23, S24, and S25, the center position of the hierarchical sign 104 in the camera image moves closer to and further away from the center of the camera image, but overall it gradually moves closer to the center, and finally the center position of the hierarchical sign 104 becomes closer to the center of the camera image. In other words, a descending approach effect is achieved.

If the hierarchical sign 102 or 104 to be recognized is included in the camera image, the drone 1 continues to descend and controls the horizontal position of the drone 1 so that the center position of the hierarchical sign 102 is located at the center of the camera image. In contrast, if the hierarchical sign 102 or 104 to be recognized is not included in the camera image, the drone 1 stops descending or reduces the descent speed to adjust the horizontal position of the drone 1 so that the hierarchical sign 102 or 104 to be recognized is included in the camera image, and resumes descent while controlling the shooting position at the point when the hierarchical sign 102 or 104 to be recognized is included in the camera image.

As shown in FIG. 11, when the drone 1 reaches the sky above the landing area 90, the camera 14 captures an image of the vertical direction downward, which is the direction indicated by the arrow Z1. The imaging range of the camera 14 is indicated, for example, by the angle of view θ1. In the state of FIG. 11, the imaging range of the angle of view θ1 is within the circle S31, and both the hierarchical sign 102 and the hierarchical sign 104 are recognizable. However, the drone 1 is configured to recognize only the outer hierarchical sign 102 by the recognition target determination program. The drone 1 recognizes the hierarchical sign 102 and the element signs 102a to 102h and 102in.

The drone 1 controls the nose direction to a specified direction by referring to the position data of the hierarchy sign 102 stored in the memory unit 52 and the data showing the relationship of the element signs 102a to 102h to the target position CP0. The drone 1 then descends while controlling the shooting position so that the entire outer shape 102out of the hierarchy sign 102 is included in the camera image.

As shown in FIG. 12, when the drone 1 descends and the hierarchical sign 102 falls outside the camera image, and the entire hierarchical sign 102 cannot be recognized within the shooting range of the camera 14, the recognition target determination program switches the recognition target to the inner hierarchical sign 104. The drone 1 then recognizes the hierarchical sign 104 and the element signs 104a to 104i using the target sign recognition program, and controls the nose direction to a specified direction by referring to the position data of the hierarchical sign 104 stored in the memory unit 52 and the data showing the relationship between the element signs 104a to 104i and the target positions. The drone 1 then descends while controlling the shooting position so that the outline 104out of the hierarchical sign 104 is included in the camera image.

By descending while performing shooting position control so that the outer shapes 102out and 104out of the hierarchy signs 102 and 104 are within the shooting range of the camera image, as shown in FIG. 13, the drone 1 can descend while gradually approaching the horizontal position of the target position CP0. In FIG. 13, the frame 102air and the frame 104air are shown as imaginary frames in the air on the vertical line of the target position. The frame 102air corresponds to the outer hierarchy sign 102, and the frame 104air corresponds to the inner hierarchy sign 104. Descending while performing shooting position control based on the hierarchy signs 102 and 104 is technically equivalent to the drone 1 descending with the aim of passing through the frame 102air and the frame 104air. Therefore, as the drone 1 descends while controlling the shooting position based on the multi-stage hierarchical markers 102 and 104, a descent approach effect is achieved in which the horizontal position of the drone 1 approaches a vertical line including the target position CP0 as the drone 1 descends. This allows the drone 1 to descend to the exact target position CP0 by referring to the target marker 100. Furthermore, because the drone 1 has relative information, it can descend while maintaining the nose direction in a specified direction.

Below, an outline of the operation of the drone 1 will be described with reference to the flowchart in Fig. 14. When the drone 1 reaches the sky above the target position CP0 (step ST1 in Fig. 14), descent control by the landing program is started. The drone 1 photographs the vertical downward direction, and sets the outer layer sign 102 as the recognition target by the recognition target determination program (step ST2). Next, when the drone 1 recognizes the outer layer sign 102 by the target sign recognition program (step ST3), it descends while controlling the horizontal position of the drone 1 so as to project the entire layer sign 102 onto the camera image (step ST4). When the altitude of the drone 1 decreases and the hierarchical sign 102 goes out of the shooting range of the camera 14 (step ST5), the recognition target determination program changes the recognition target to the inner hierarchical sign 104 (step ST6), the target sign recognition program recognizes the hierarchical sign 104, and the drone 1 descends while controlling its horizontal position so as to project the entire hierarchical sign 104 onto the camera image (step ST7), and when it is determined that it has landed (step ST8), it stops. The landing may be determined, for example, by determining that the downward movement has stopped from the output of the inertial sensor unit 58 (see FIG. 3).

In this embodiment, the hierarchical markers are two levels, hierarchical markers 102 and 104. Since the present invention is characterized by using hierarchical markers of multiple levels, unlike this embodiment, the hierarchical markers are not limited to two levels, and may be, for example, three or more levels. Also, unlike this embodiment, a priority order may be given to the attitude control and the shooting position control. For example, if the hierarchical marker 102 or 104 does not approach the center of the camera image due to the attitude control, the shooting position control may be performed. Alternatively, unlike this embodiment, the shooting position control may be performed at a time interval longer than the time interval of the attitude control. For example, if the attitude control is performed every 1/1000th of a second, the shooting position control is performed every 1/10th of a second. Also, unlike this embodiment, instead of descending while controlling the horizontal position of the drone 1 so that the entire hierarchical markers 102 and 104 are projected onto the camera image, the drone 1 may descend while controlling the horizontal position so that the center positions of the hierarchical markers 102 and 104 are located at the center position of the camera image. In this case, the center positions of the hierarchical signs 102 and 104 are recognized by recognizing the entirety of the hierarchical signs 102 and 104. Furthermore, the technique for controlling the descent direction by projecting the hierarchical signs 102 and 104 onto the camera image is not limited to the above. For example, when the hierarchical sign 102 is used to control the horizontal position of the drone 1, the number of pixels of the hierarchical sign 102 in the camera image may be controlled to increase, and when the hierarchical sign 104 is used, the number of pixels of the hierarchical sign 104 in the camera image may be controlled to increase.

Second Embodiment
Next, a second embodiment will be described with reference to Fig. 15. Explanations of matters common to the first embodiment will be omitted, and explanations will be focused on matters different from the first embodiment. In the second embodiment, as shown in steps ST4A and ST7A in Fig. 15, the drone 1 descends while controlling the horizontal position of the drone 1 so that the center positions of the hierarchical signs 102 and 104 are projected onto the center position of the camera image.

Third Embodiment
Next, the third embodiment will be described with reference to FIG. 16. Explanation of matters common to the first embodiment will be omitted, and explanation will be focused on matters different from the first embodiment. In the third embodiment, when the drone 1 reaches the sky above the target position CP0, it recognizes the outer hierarchy sign 102, and descends while controlling the horizontal position so that the entire outer shape 102out of the hierarchy sign 102 is projected onto the camera image, but at this time, it also recognizes the inner hierarchy sign 104. Then, just before the entire outer shape 102out of the outer hierarchy sign 102 can no longer be projected onto the camera image, it switches to control to descend while projecting the entire outer shape 104out of the inner hierarchy sign 104 onto the camera image. Just before the entire outer shape 102out of the outer hierarchy sign 102 can no longer be projected onto the camera image, it is, for example, when a part of the outer shape 102out reaches the outermost part of the camera image, but is not limited thereto.

As shown in step ST4B of FIG. 16, when the drone descends while controlling its horizontal position so as to project the entire hierarchical sign 102 onto the camera image, it simultaneously recognizes the inner hierarchical sign 104. However, the inner hierarchical sign 104 is not used for shooting position control. When the drone 1 determines that a part of the outline 102out of the outer hierarchical sign 102 is about to go out of the camera image (step ST5B), it switches to control to descend while controlling the horizontal position of the drone 1 so as to project the outline 104out of the inner hierarchical sign 104 onto the camera image (step ST6). This control allows the drone 1 to descend while performing shooting position control without interruption.

<Fourth embodiment>
Next, a fourth embodiment will be described with reference to Figs. 17 to 20. Explanations of matters common to the first embodiment will be omitted, and differences from the first embodiment will be mainly described. In the fourth embodiment, as shown in Figs. 17 and 18, the target marker 100 is disposed in a charging base 400. The charging base 400 is composed of a base 402 and left and right movable housings 404A and 404B.

The left and right movable housings 404A and 404B can move back and forth horizontally in the directions of arrows X1 and X2 as shown in Figures 17 and 18, forming the open state shown in Figure 17 and the closed state shown in Figure 18.

The target marker 100 is located in the horizontal center of the base 402, and is configured to be exposed to the outside in the open state shown in FIG. 17, and to be covered by the movable housings 404A and 404B in the closed state shown in FIG. 18.

With the charging base 400 open, the drone 1 descends toward the target position indicated by the target marker 100. When the drone 1 lands on the base 402, the charging base 400 closes and charging of the drone 1 is carried out.

A shape estimation program is stored in the memory unit 52 of the drone 1. The shape estimation program allows the drone 1 to estimate the overall shape of the hierarchical sign 102 when only a portion of the hierarchical sign 102 can be recognized. The drone 1 then descends while maintaining the estimated hierarchical sign 102 within the shooting range of the camera image using a landing program, and controlling the shooting position so that the entire estimated hierarchical sign 102 is within the camera image. The above control is also similar for the hierarchical sign 104. The shape estimation program and CPU 50 are an example of a shape estimation means.

For example, as shown in Figures 19 and 20, parts of the hierarchical signs 102 and 104 may be difficult to recognize due to shadows SD1 and SD2.

UAV 1 can recognize the element signs of hierarchical signs 102 and 104 using a target sign recognition program, and estimate the overall shapes of hierarchical signs 102 and 104 based on the recognized element signs.

For example, as shown in FIG. 19, if element markers 102a, 102b, 102g, and 102h of the element markers of the outer hierarchical marker 102 are recognizable, while other element markers such as 102c are not recognizable, the overall shape of the outer layer marker 102 is estimated from the recognizable element markers 102a, 102b, 102g, and 102h. This is also true in the case where some element markers of the inner hierarchical marker 104 are recognizable, as shown in FIG. 20.

Then, the drone 1 descends while performing shooting position control by the landing program so that the estimated hierarchical sign 102 or hierarchical sign 104 is located within the shooting range of the camera image.

In this embodiment, the use of artificial intelligence is not essential, but the drone 1 may be configured to recognize the hierarchical signs 102 and 104 from parts of the hierarchical signs 102 and 104 using the results of machine learning using image data captured under various environmental conditions. By learning images of the target sign 100 in various appearances, it is possible to extract features of a specific part and generate learning result data for identifying the specific part regardless of how it appears. The features shown in the feature data include both features that are constant under various environmental conditions and features for each of the various environmental conditions. In this case, the drone 1 stores the learning result data in the memory unit 52.

Machine learning does not have to be performed by the drone 1 itself; it may be performed by an external computer, with the drone 1 using the learning results. For example, deep learning is used as the machine learning method. Deep learning is machine learning that uses a multi-layered neural network, and image recognition is one of the most promising fields of application. The neural network is, for example, a convolutional neural network (CNN). The convolutional neural network is composed of an input layer, a convolution layer, a pooling layer, a fully connected layer, and an output layer, and the convolutional layer and the pooling layer are repeated multiple times to form a deep layer, followed by multiple fully connected layers. By using the learning result data generated by deep learning, the drone 1 can identify the hierarchical sign 102 or 104 no matter how it appears.

Fifth embodiment
Next, the fifth embodiment will be described with reference to Fig. 21. Explanations of matters common to the first and second embodiments will be omitted, and differences will be mainly described. In the fifth embodiment, as shown in Fig. 21, a plurality of drones 1A, 1B, and 1C are operated, and target markers 100A, 100B, and 100C are assigned to each drone 1A, etc., respectively.

Each drone 1A, etc. is configured to descend toward a target position indicated by a target marker 100A, etc., assigned to it, by a landing program.

Sixth Embodiment
Next, a sixth embodiment will be described with reference to Fig. 22. Explanations of matters common to the fifth embodiment will be omitted, and differences will be mainly described. In the sixth embodiment, specific target markers are assigned to the multiple drones 1A, 1B, and 1C, but the target markers can be changed by the administrator of the drone 1A, etc.

For example, as shown in FIG. 22, the memory unit of drone 1A stores data on target markers X1 and X2 as multiple target markers. The memory unit of drone 1B also stores data on target markers X1 and X2 as multiple target markers, similar to drone 1A. Assume that the target marker initially assigned to drone 1A is target marker X1, and the target marker initially assigned to drone 1B is target marker X2. In this embodiment, the administrator can change the target marker initially assigned to drone 1A, etc., to another target marker. This makes it possible to operate drone 1B in the event of a breakdown in drone 1A, for example, by changing the target marker of drone 1B to target marker X1 and having drone 1A carry out its mission.

The present invention is not limited to the above-mentioned embodiments, and any modifications or improvements that can achieve the object of the present invention are included in the present invention. In addition, the above-mentioned embodiments can be combined as appropriate.

1, 1A, 1B, 1C drone 2 housing 6 motor 14 camera 52 memory unit 56 satellite positioning unit 90 take-off and landing area 100 target sign 102, 104 hierarchical sign 102a, 102b, 102c, 102d, 102e, 102f, 102g, 102h, 102in, 104a, 104b, 104c, 104d, 104e, 104f, 104g, 104h, 104i element sign

Claims (8)

A descent system for landing an unmanned aerial vehicle at a target position by controlling the rotation of propellers connected to the rotation shafts of a plurality of motors,
a target marker disposed in an area including the target position;
The target sign is composed of a plurality of hierarchical signs having different external dimensions in the horizontal direction,
The hierarchy sign having a relatively small outer shape is disposed inside the hierarchy sign having a relatively large outer shape,
The hierarchical markers of the plurality of stages include the target position on the inside in the horizontal direction,
The unmanned aerial vehicle is
Relative information indicating a relative positional relationship between each part constituting the hierarchy sign;
An image acquisition means capable of photographing an area below;
having
When the floor sign having a relatively large outer shape can be recognized within the photographing range of the image acquisition means, the unmanned aerial vehicle is controlled to descend so that the floor sign having a relatively large outer shape is positioned within the photographing range,
When the floor sign having a relatively large outline becomes unrecognizable within the shooting range as a result of the descent of the unmanned aerial vehicle, the unmanned aerial vehicle is controlled to descend so that the floor sign having a relatively small outline is positioned within the shooting range ;
When the unmanned aerial vehicle descends while controlling the unmanned aerial vehicle so that the relatively large-sized floor level sign is positioned within the shooting range, the unmanned aerial vehicle also recognizes the relatively small-sized floor level sign, and before the unmanned aerial vehicle descends and the relatively large-sized floor level sign becomes unrecognizable within the shooting range, the control is switched to descending while controlling the relatively small-sized floor level sign to be positioned within the shooting range.
Descent system. immediately before the entire outer shape of the outer floor sign can no longer be projected onto the camera image acquired by the image acquisition means, a control is switched to descending while the entire outer shape of the inner floor sign is projected onto the camera image.
The descent system of claim 1 . The innermost level sign is defined to have an outer size that allows the unmanned aerial vehicle to be recognized within the shooting range even when the unmanned aerial vehicle lands,
A descent system according to claim 1 or claim 2. The hierarchical indicator of the plurality of stages is composed of a plurality of element indicators,
the relative information is information indicating a direction and a distance of the target position based on each of the element markers;
The unmanned aerial vehicle controls the nose direction of the unmanned aerial vehicle itself in a predetermined direction based on the direction and distance of the target position indicated by the element marker.
A descent system according to claim 1 or claim 2 . The unmanned aerial vehicle is
an estimation means for estimating the entirety of the hierarchical label based on the element labels of a part of the hierarchical label;
and descending while controlling the vehicle so that the estimated floor sign is positioned within the shooting range.
5. A descent system as claimed in claim 4. A charging device for charging a rechargeable power source of the unmanned aerial vehicle is disposed in an area including the target position;
The unmanned aerial vehicle is configured to land at the target position with the nose facing the predetermined direction, and to charge the power source of the unmanned aerial vehicle by the charging device.
6. A descent system as claimed in claim 5 . the descent system includes a plurality of the unmanned air vehicles and a plurality of the target markers;
Each of the unmanned air vehicles is assigned a different target signature;
Each of the unmanned air vehicles is configured to descend toward the target location indicated by the assigned target marker.
A descent system according to claim 1 or claim 2 . The unmanned aerial vehicle is
The target marker is recognized based on a degree of correlation between data indicating the characteristics of the target marker that is stored in advance and characteristic data extracted from an image in the image data acquired by the image acquisition means,
The descent system of claim 1 or claim 2 , wherein the data indicative of the characteristics of the target marker includes data generated by deep learning.