JP2022036109A - Management apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a technique for managing an obstacle avoiding method.

SOLUTION: A management apparatus comprises a management information acquisition section 231 by which, in a case where a flying device flies along a second flight route which is generated on the basis of a first picked-up image including an obstacle picked up by the flying device flying along a first flight route and flight location information indicating a location where the first picked-up image is picked up, for avoiding the obstacle, a second picked-up image including an obstacle picked up by the flying device in a flight location on the second flight route is acquired from the flying device. The management information acquisition section 231 may also acquire the first picked-up image from the flying device. The management information acquisition section 231 may further acquire information indicating a direction in which the first picked-up image or the second picked-up image is picked up, from the flight location in which the first picked-up image or the second picked-up image is picked up.

SELECTED DRAWING: Figure 3

COPYRIGHT: (C)2022,JPO&INPIT

Description

The present invention relates to a management device.

In a flight device such as a drone, it is common for a user to set a flight route by designating a way point on a map. Among such techniques, there is one in which an obstacle that is not reflected on the map is imaged by a drone and the user specifies a flight route using the captured image (for example, Patent Document 1).

Japanese Patent No. 6138326

However, in the method described in Patent Document 1, the information on obstacles and the method for avoiding obstacles are not managed in association with each other, and flight management is insufficient.

Therefore, the present invention has been made in view of the above problems, and an object of the present invention is to provide a technique for managing an obstacle avoidance method.

The management device of the first aspect of the present invention is based on the first captured image including an obstacle captured by the flight device flying on the first flight route and the flight position information in which the first captured image is captured. When the flight device flies on the second flight route for avoiding the generated obstacle, the second image pickup including the obstacle imaged by the flight device at the flight position on the second flight route. It is provided with a management information acquisition unit that acquires an image from the flight device. The management information acquisition unit may acquire the first captured image from the flight device. The management information acquisition unit further acquires information indicating the direction of imaging of the first captured image or the second captured image from the flight position where the first captured image or the second captured image is captured. May be good.

The management information acquisition unit may acquire the flight position information of the flight device at the time of capturing the first captured image or the second captured image from the flight device. The management information acquisition unit may acquire unique device identification information assigned to the flight device from the flight device.

The management information acquisition unit may acquire three-dimensional information indicating the three-dimensional position of the obstacle acquired by the flight apparatus during the flight of the second flight route. The management information acquisition unit includes distance information indicating the distance from the flight position on the second flight route to an obstacle, imaging date / time information indicating the date and time when the second captured image was acquired, and flight on the second flight route. Any one of the direction information indicating the direction of capturing the second captured image from the position may be acquired from the flight device.

The management information acquisition unit includes distance information indicating the distance from the flight position on the second flight route to an obstacle, imaging date / time information indicating the date and time when the second captured image was acquired, and flight on the second flight route. Any one of the direction information indicating the direction of capturing the second captured image from the position may be stored in association with the second flight route. The management information acquisition unit may store the acquired second captured image in association with the second flight route.

The management information acquisition unit may acquire information indicating the second flight route generated by the flight device from the flight device. The management device may further include a specific unit that identifies the size of the obstacle based on the second captured image and the distance from the flight position of the flight device at the time of capturing the second captured image. good.

The management device may further include a specific unit that identifies the type of obstacle based on the second captured image. Even if it is further provided with a storage unit that stores map information and a map display unit that adds information on the obstacle specified based on the second captured image acquired by the management information acquisition unit to the map information. good.

The map display unit may add information indicating the position of the obstacle specified based on the second captured image acquired by the management information acquisition unit to the map information.

The map display unit may add information indicating the type of the obstacle specified based on the second captured image acquired by the management information acquisition unit to the map information.

The management information acquisition unit further acquires the imaging date and time of the second captured image, stores the map information, the flight position information at the time of capturing the second captured image, and the second captured image. Based on the direction of imaging and the distance from the flight position to the obstacle, a specific part that specifies the position of the obstacle imaged in the second captured image and the obstacle specified by the specific part. A map display unit that displays the obstacle icon at the position on the map of the map information corresponding to the position and associates the second captured image of the obstacle and the imaging date and time with the icon of the obstacle. You may also prepare further.

The specific unit may specify the size of the obstacle, and the map display unit may associate the size of the obstacle specified by the specific unit with the icon of the obstacle.

The management device acquires a flight schedule of another flight device different from the flight device, and an area within a predetermined distance from the position of the obstacle specified based on the second captured image acquired by the management information acquisition unit. It may further include a schedule acquisition unit that identifies the other flight equipment that is scheduled to fly.

The management device may further include a notification unit that notifies the device that manages the other flight device specified by the schedule acquisition unit of information about the obstacle or the second flight route.

According to the present invention, there is an effect that the obstacle avoidance method can be managed.

It is a figure which shows the structure of the map management system which concerns on embodiment. It is a figure which shows the structure of a flight apparatus. It is a figure which shows the structure of a management device. An example of a display image of a 3D map displayed on a display device by a map display unit is shown. It is a flowchart which showed an example of the operation of a flight apparatus. It is a flowchart which showed an example of the operation of a management device.

[Configuration of map management system S]
FIG. 1 is a diagram showing a configuration of a map management system S according to the present embodiment. The map management system S manages a 3D map showing information on obstacles in the airspace in which the flight device 100 flies. This 3D map contains information indicating altitude.

The map management system S includes a management device 200. The flight device 100 can acquire a captured image during flight, and is, for example, a drone. The flight device 100 receives from the management device 200 the first route information indicating the first flight route for the flight device 100 to fly from the flight start position to the flight end position by communicating with the management device 200 by wireless communication. .. In FIG. 1, the first flight route is shown by a broken line. The flight device 100 moves along the first flight route received from the management device 200.

Further, when the flight device 100 detects an obstacle A on the first flight route, it generates a second flight route for avoiding the obstacle A and moves along the generated second flight route. .. In FIG. 1, the second flight route is shown by a solid line. The second flight route is a route that branches from the first flight route, avoids the obstacle A, and then joins the original first flight route. Obstacle A is, for example, a lightning rod or a building under construction, and in the example of FIG. 1, it is a lightning rod installed on the roof of the building. When the flight device 100 moves along the second flight route, the flight device 100 acquires a captured image of the obstacle A and transmits the acquired captured image to the management device 200.

The management device 200 stores map information indicating a 3D map. The management device 200 refers to the map information and generates a first flight route so that the flight device 100 in flight does not come into contact with each obstacle recorded in the map information. Further, the management device 200 receives the captured image of the obstacle A from the flight device 100 when the flight device 100 moves on the second flight route.

The management device 200 generates a first flight route so that the flight device 100 in flight does not come into contact with the obstacle recorded in the map information. However, in the example of this embodiment, the first flight route is generated based on the map information that does not include the information about the obstacle A due to the circumstances such as the obstacle A being installed after the map information is created. It is assumed that there is. The management device 200 stores information on the obstacle A detected by the flight device 100.

[Structure of flight device 100]
FIG. 2 is a diagram showing the configuration of the flight device 100. The flight device 100 includes a GPS receiving unit 11, a detecting unit 12, a distance measuring unit 13, an imaging unit 14, a communication unit 15, a storage unit 16, and a control unit 17. The GPS receiving unit 11 receives signals from a plurality of GPS satellites.

The detection unit 12 detects an obstacle A in a predetermined area including the first flight route. The predetermined area is a permissible value in consideration of the measurement error of the positioning by the GPS receiving unit 11, and is, for example, a range within 20 meters from the first flight route. The obstacle A is an object on the first flight route or within a predetermined range from the first flight route. Further, the obstacle A is an object that is expected to come into contact with the flight device 100 when the flight device 100 flies on the first flight route, or a communication failure with the flight device 100 due to being close to the flight device 100. It may be an object that is expected to cause an obstacle.

The detection unit 12 is, for example, a stereo camera. The stereo camera is activated continuously or intermittently during the flight of the first flight route, and acquires and acquires a plurality of images obtained by simultaneously capturing a subject on the first flight route from a plurality of different directions. Obstacle A is detected by matching the images of. The closer the object is to the flight device 100, the larger the parallax when images are taken from a plurality of different directions. On the other hand, parallax does not occur when the object does not exist at a predetermined position to be imaged. Therefore, the stereo camera detects an object having a parallax of a certain value or more when images taken from a plurality of different directions in the traveling direction of the flight device 100 as an obstacle A. The constant value is, for example, the minimum value of parallax that can be detected by a stereo camera. Further, the detection unit 12 may be an ultrasonic sensor or an infrared distance measuring sensor. An ultrasonic sensor detects an obstacle A by transmitting an ultrasonic wave toward a target and receiving the reflected wave by a receiver. The infrared distance measuring sensor detects an obstacle A by irradiating an object with infrared rays and receiving reflected light.

Further, the detection unit 12 acquires detection information for specifying the position and size of the obstacle A in the horizontal direction, the vertical direction, and the depth direction. For example, when the detection unit 12 is a stereo camera, the detection unit 12 acquires a plurality of images obtained by simultaneously capturing a subject from a plurality of different directions, and divides the acquired images into a large number of small sections. The detection unit 12 obtains parallax by matching a plurality of images for each small section. Further, the detection unit 12 specifies the position and size of the obstacle A in the horizontal direction, the vertical direction, and the depth direction by calculating the distance to the obstacle A for each small section based on the obtained parallax. The detection unit 12 notifies the recognition unit 174 of the presence / absence of detection of the obstacle A and the detection information.

The distance measuring unit 13 measures the distance from the flight position of the flight device 100 to the obstacle A. The distance measuring unit 13 is, for example, an ultrasonic sensor that transmits ultrasonic waves toward an obstacle A and receives the reflected waves to measure the distance from the flight position to the obstacle A. Further, the distance measuring unit 13 may be a stereo camera or an infrared distance measuring sensor. Further, the detection unit 12 and the distance measurement unit 13 may be the same sensor.

The image pickup unit 14 is a camera for acquiring a captured image by an optical element. The communication unit 15 is a communication interface for wireless communication. The storage unit 16 is a storage medium such as a ROM (Read Only Memory) and a RAM (Random Access Memory). The storage unit 16 may be a volatile memory or a non-volatile memory. The storage unit 16 stores a program executed by the control unit 17.

The control unit 17 is, for example, a CPU (Central Processing Unit). The control unit 17 functions as a position information acquisition unit 171, a flight control unit 172, a generation unit 173, a recognition unit 174, an image capture image acquisition unit 175, and a communication control unit 176 by executing a program stored in the storage unit 16. do.

The captured image acquisition unit 175 acquires an captured image by controlling the image pickup unit 14. When the detection unit 12 detects the obstacle A, the captured image acquisition unit 175 acquires the captured image of the obstacle A. Further, the captured image acquisition unit 175 acquires the imaging date and time and the imaging direction of the captured image of the obstacle A.

The position information acquisition unit 171 acquires flight position information indicating the flight position of the flight device 100 by the signal from the GPS satellite received by the GPS reception unit 11. Further, the position information acquisition unit 171 acquires flight altitude information indicating the flight altitude of the flight device 100 by using a stereo camera, an ultrasonic sensor, a barometer, or the like installed below the flight device 100. For example, the position information acquisition unit 171 acquires flight position information and flight altitude information at the time of acquisition of the captured image by the captured image acquisition unit 175.

The recognition unit 174 analyzes the detection information acquired by the detection unit 12, the distance information from the flight position acquired by the distance measurement unit 13 to the obstacle A, and the captured image acquired by the captured image acquisition unit 175. , Recognize the position and size of the obstacle A in the horizontal, vertical and depth directions. For example, the recognition unit 174 recognizes the position and size of the obstacle A in the depth direction based on the detection information acquired by the detection unit 12. Further, the recognition unit 174 recognizes the horizontal position and size of the obstacle A and the vertical position and size of the obstacle A based on the captured image acquired by the captured image acquisition unit 175.

The generation unit 173 generates a second flight route for avoiding the obstacle A. When the detection unit 12 detects the obstacle A, the generation unit 173 refers to the position and size of the obstacle A recognized by the recognition unit 174 in the horizontal direction, the vertical direction, and the depth direction, so that the second flight can be performed. Generate a route.

For example, the generation unit 173 generates a second flight route that turns around the obstacle A so that the flight device 100 keeps a certain distance or more from the obstacle A. A constant distance is, for example, 2 meters. The generation unit 173 refers to the first flight route and the current flight position, and after branching from the first flight route and avoiding the obstacle A, the generation unit 173 sets the second flight route until it rejoins the first flight route. Generate. The generation unit 173 may generate a second flight route that avoids the obstacle A so that the flight distance is the shortest while maintaining a certain distance or more from the obstacle A.

The flight control unit 172 controls the flight. For example, the flight control unit 172 specifies the moving direction and the moving speed of the flight device 100 by designating the rotation speeds of a plurality of rotors (not shown). The flight control unit 172 acquires the first route information indicating the first flight route from the management device 200 by the communication unit 15. The flight control unit 172 controls the flight so as to move along the first flight route indicated by the acquired first route information.

The flight control unit 172 specifies the movement direction corresponding to the first flight route by comparing the acquired first route information with the flight position information acquired by the position information acquisition unit 171. Further, when the detection unit 12 detects an obstacle A on the first flight route, the flight control unit 172 hovered, and after the generation unit 173 generated the second flight route, the flight control unit 172 along the second flight route. Control the flight to move. The flight control unit 172 performs flight control to move along the first flight route again after the second flight route joins the first flight route.

The communication control unit 176 communicates with the management device 200 via the communication unit 15. When the flight device 100 flies on the second flight route, the communication control unit 176 (1) identifies the device ID of the flight device 100, and (2) the flight of the flight device 100 measured by the distance measurement unit 13. Distance information indicating the distance from the position to the obstacle A, (3) the captured image of the obstacle A acquired by the imaging unit 14, and the imaging date and time information indicating the imaging date and time of the captured image, and (4) the position information acquisition unit. Flight position information and flight altitude information acquired at the time of acquisition of the captured image in 171, (5) direction information indicating the direction of imaging of the captured image, and (6) horizontal and vertical obstacle A recognized by the recognition unit 174. Three-dimensional information indicating the position and size in the direction and the depth direction is stored in the storage unit 16 and transmitted to the management device 200. Further, the communication control unit 176 transmits (7) the second route information indicating the second flight route to the management device 200 after merging from the second flight route to the first flight route.

[Configuration of management device 200]
FIG. 3 is a diagram showing the configuration of the management device 200. The management device 200 includes a communication unit 21, a storage unit 22, and a control unit 23. The communication unit 21 is a communication interface for transmitting and receiving various information via a network. The storage unit 22 is a storage medium such as a ROM and a RAM. The storage unit 22 may be a volatile memory or a non-volatile memory. The storage unit 22 stores, for example, map information indicating a 3D map. Further, the storage unit 22 stores a program executed by the control unit 23.

The control unit 23 is, for example, a CPU. The control unit 23 functions as a management information acquisition unit 231, a specific unit 232, a map display unit 233, a schedule acquisition unit 234, and a notification unit 235 by executing the program stored in the storage unit 22. When the management information acquisition unit 231 acquires (1) the device ID, (2) distance information, (3) the captured image and the captured date and time information of the obstacle A, and (4) the position information acquisition unit 171 obtains the captured image. The communication unit 15 acquires the acquired flight position information and flight altitude information, (5) direction information, and (6) three-dimensional information. Further, the management information acquisition unit 231 acquires (7) the second route information by the communication unit 21. The management information acquisition unit 231 stores the acquired information (1) to (7) in the storage unit 22.

The identification unit 232 identifies the position of the obstacle A captured in the captured image acquired by the management information acquisition unit 231. The identification unit 232 specifies, for example, the position of the obstacle A by longitude and latitude. For example, the identification unit 232 identifies the position of the obstacle A as the position moved by the distance measured by the distance measuring unit 13 from the flight position at the time of imaging of the captured image in the direction of imaging. The specifying unit 232 identifies the vicinity of the center of the obstacle A as the position of the obstacle A based on the positions and sizes of the obstacle A recognized by the recognition unit 174 in the horizontal direction, the vertical direction, and the depth direction. May be good.

Further, the specifying unit 232 specifies the type of the obstacle A based on the captured image of the obstacle A. The type of obstacle A is, for example, a lightning rod, a crane, a skyscraper under construction, or the like. The specific unit 232 extracts the feature information of the obstacle A from the captured image of the obstacle A, and compares the extracted feature information with the feature information stored by the storage unit 22 for each type of obstacle. Specify the type of object A.

The map display unit 233 transmits the map information stored in the storage unit 22 to the display device (not shown) by the communication unit 21, and displays the 3D map indicated by the map information on the display of the display device. Further, the map display unit 233 adds the information of the obstacle A acquired by the management information acquisition unit 231 to the map information of the storage unit 22. The map display unit 233 displays the icon of the obstacle A at the position on the 3D map corresponding to the position of the obstacle A specified by the specific unit 232. Further, the map display unit 233 displays the captured image of the obstacle A and the captured date and time in association with the icon of the obstacle A. For example, the map display unit 233 displays the captured image and the captured date and time of the obstacle A when the display device accepts the operation of selecting the icon of the obstacle A.

[Example of 3D map display image]
The operation of the map display unit 233 will be described with reference to FIG. FIG. 4 shows an example of a display image of a 3D map displayed on the display device by the map display unit 233. As shown in FIG. 4, the map display unit 233 displays an icon corresponding to an obstacle such as a building on a 3D map. The plurality of rectangular parallelepiped icons in FIG. 4 each indicate a building. Further, the map display unit 233 displays the icon of the obstacle A acquired by the management information acquisition unit 231 at a position corresponding to the position specified by the specific unit 232. The icon of obstacle A is shown with hatching.

The map display unit 233 displays an icon based on the size of the obstacle A acquired by the management information acquisition unit 231 in the horizontal direction, the vertical direction, and the depth direction. For example, the map display unit 233 displays an icon obtained by reducing the horizontal direction, the vertical direction, and the depth direction of the obstacle A acquired by the management information acquisition unit 231 at a predetermined reduction rate. The predetermined reduction ratio is, for example, 1/1000. Further, when the map display unit 233 accepts the operation of selecting the icon of the obstacle A in the display device, the map display unit 233 displays the captured image of the obstacle A and the captured date and time as shown in the lower right area of FIG. ..

Returning to the description of FIG. The schedule acquisition unit 234 acquires the flight schedule of another flight device by the communication unit 21 from one or a plurality of devices (not shown) that manage the other flight devices. The schedule acquisition unit 234 identifies a flight device that is scheduled to move in an area within a predetermined distance from the obstacle A by referring to the flight schedules of the other flight devices that have been acquired. The predetermined distance is a distance at which it is assumed that another flight device may come into contact with the obstacle A due to an error in positioning the signal from the GPS satellite or the influence of the wind. For example, 20 meters from the obstacle A. Within.

Information about the obstacle A is useful for other flight devices that are planning to move around the obstacle A from the viewpoint of suppressing contact with the obstacle A. The information about the obstacle A is, for example, a captured image, type, position or altitude of the obstacle. Therefore, the notification unit 235 acquires the management information acquisition unit 231 for one or a plurality of devices that manage the flight device specified by the schedule acquisition unit 234 that the flight device is scheduled to move within a predetermined distance from the obstacle A. The communication unit 21 notifies the information about the obstacle A or the new flight route for avoiding the obstacle A.

[Processing procedure of flight device 100]
FIG. 5 is a flowchart showing an example of the operation of the flight device 100. The processing procedure of FIG. 5 starts when the detection unit 12 detects an obstacle A in a predetermined area including the first flight route.

First, the flight control unit 172 starts hovering (step S101), and the distance measuring unit 13 measures the distance from the flight position of the flight device 100 to the obstacle A (step S102). Further, the captured image acquisition unit 175 acquires the captured image of the obstacle A (step S103). In addition, the captured image acquisition unit 175 acquires the acquisition date and time of the captured image and the direction of imaging (step S104). The position information acquisition unit 171 acquires flight position information and flight altitude information at the time of acquisition of the captured image by the captured image acquisition unit 175 (step S105).

The generation unit 173 generates a second flight route for avoiding the obstacle A (step S106). The communication control unit 176 acquires (1) the device ID, (2) the distance information measured by the distance measuring unit 13, (3) the captured image of the obstacle A and the captured date and time information, and (4) the captured image. The acquired flight position information and flight altitude information, (5) direction information, and (6) three-dimensional information recognized by the recognition unit 174 are transmitted to the management device 200 (step S107).

After that, the flight control unit 172 controls the flight to fly along the second flight route generated by the generation unit 173 (step S108). Further, the communication control unit 176 transmits the second route information indicating the second flight route to the management device 200 after the flight device 100 returns to the first flight route along the second flight route (step S109). The communication control unit 176 may transmit the second route information to the management device 200 before or during the flight of the flight device 100 on the second flight route.

[Processing procedure of management device 200]
FIG. 6 is a flowchart showing an example of the operation of the management device 200. The processing procedure of FIG. 6 starts when the communication control unit 176 of the flight device 100 transmits various information shown in steps (1) to (6) of step S107 of FIG. 5 to the management device 200.

First, the management information acquisition unit 231 acquires various information shown in (1) to (6) of step S107 of FIG. 5 from the flight device 100 (step S201). Further, the management information acquisition unit 231 acquires the second route information by the communication unit 15 (step S202). The identification unit 232 specifies the position of the obstacle A captured in the captured image acquired by the management information acquisition unit 231 (step S203). The map display unit 233 displays the icon of the obstacle A at the position on the 3D map corresponding to the position of the obstacle A specified by the specific unit 232 (step S204). Further, the map display unit 233 displays the captured image of the obstacle A and the captured date and time in association with the icon of the obstacle A (step S205).

Further, the schedule acquisition unit 234 acquires the flight schedule of the other flight device from the one or a plurality of devices that manage the other flight devices by the communication unit 21 (step S206). The schedule acquisition unit 234 identifies a flight device that is scheduled to move within a predetermined distance from the obstacle A by referring to the flight schedules of the other flight devices that have been acquired. Further, the notification unit 235 manages one or a plurality of devices that manage the flight device specified by the schedule acquisition unit 234 that the notification unit 235 plans to move within a predetermined distance from the obstacle A, and the obstacle acquired by the management information acquisition unit 231. Information about the object A is notified by the communication unit 21 (step S207).

[Effect of the present invention]
According to the present embodiment, the generation unit 173 determines the second flight route based on the flight position information acquired by the position information acquisition unit 171 and the image captured of the obstacle A acquired by the captured image acquisition unit 175. Generate. Therefore, when the detection unit 12 detects the obstacle A in a predetermined area including the first flight route, the flight control unit 172 can avoid the obstacle A along the second flight route. ..

Further, according to the present embodiment, the management information acquisition unit 231 acquires the shooting date and time of the captured image from the flight device 100, and the specific unit 232 captures the flight position information at the time of capturing the captured image and the imaging of the captured image. The position of the obstacle captured in the captured image is specified based on the direction of the vehicle and the distance from the flight position to the obstacle. Further, the map display unit 233 displays the icon of the obstacle A at the position on the map of the map information corresponding to the position of the obstacle A specified by the specific unit 232, and the captured image and the imaging date and time of the obstacle A. Is associated with the icon of the obstacle A. Therefore, the user can easily grasp the position of the obstacle A, the captured image, and the captured date and time on the map.

Further, according to the present embodiment, the map display unit 233 associates the size of the obstacle A specified by the specific unit 232 with the icon of the obstacle A. Therefore, the user can easily grasp the size of the obstacle A on the map.

Further, according to the present embodiment, the notification unit 235 informs the device that manages the flight device scheduled to fly in the area within a predetermined distance from the obstacle A about the obstacle A or the second flight route. Notify. Therefore, the device that manages the other flight devices can set a flight route to avoid the obstacle A in advance.

[Modification 1]
In the present embodiment, the communication control unit 176 manages the flight position information, the captured image of the obstacle A, the direction information, and the second route information when the flight device 100 flies on the second flight route. An example of sending to is explained. However, the present invention is not limited to this. For example, when the flight device 100 flies on the second flight route, the flight position information, the captured image, the direction information, and the second route information are stored in the storage unit 16 without being transmitted to the management device 200, and the flight is performed. After the flight of the device 100 is completed, these information may be read out from the storage unit 16 to the outside of the flight device 100. By adopting such a configuration, even if a communication failure occurs between the flight device 100 and the management device 200, the captured image of the obstacle A detected by the detection unit 12 can be read out and used.

[Modification 2]
Further, in the present embodiment, the configuration in which the detection unit 12 detects an obstacle A in a predetermined region including on the first flight route has been described. However, the present invention is not limited to this. For example, the detection unit 12 may detect an object outside a predetermined area.

For example, the storage unit 16 of the flight device 100 stores map information indicating the same 3D map as the storage unit 22 of the management device 200. When the detection unit 12 detects an object outside the predetermined area, the recognition unit 174 specifies the position of the object by, for example, the same method as the method in which the identification unit 232 of the management device 200 specifies the position of the obstacle A. .. The recognition unit 174 refers to the map information stored in the storage unit 22 and determines whether or not an object such as an obstacle is shown at a position on the 3D map corresponding to the specified position. When the object is not shown at the position on the 3D map corresponding to the specified position, the recognition unit 174 determines that the object detected by the detection unit 12 is a new object not shown on the 3D map. On the other hand, when the object is shown at the position on the 3D map corresponding to the specified position, the recognition unit 174 determines that the object detected by the detection unit 12 is a known object shown on the 3D map. do.

When the recognition unit 174 determines that the object detected by the detection unit 12 is a new object not shown in the 3D map, the communication control unit 176 (1) determines that the device ID for identifying the flight device 100. , (2) Distance information indicating the distance from the flight position of the flight device 100 measured by the distance measuring unit 13 to the object, and (3) imaging indicating the captured image of the object acquired by the imaging unit 14 and the imaging date and time of the captured image. Date and time information, (4) flight position information and flight altitude information acquired at the time of acquisition of the captured image by the position information acquisition unit 171, (5) direction information indicating the direction of imaging of the captured image, and (6) recognition unit 174. 3D information indicating the position and size of the object recognized by the recognition unit in the horizontal direction, the vertical direction, and the depth direction, and (7) the position information indicating the position of the object specified by the recognition unit 174 are transmitted to the management device 200.

Further, the communication control unit 176 may store the information of (1) to (7) in the storage unit 22 without transmitting the information to the management device 200. Further, the map display unit 233 displays the icon of the object at the position on the 3D map corresponding to the position of the object specified by the recognition unit 174. By adopting such a configuration, it is possible to share information about an object that is not stored in the map information stored in the storage unit 16 for the flight of another flight device.

[Modification 3]
The detection unit 12 may detect that the object indicated in the map information stored in the storage unit 16 does not exist. For example, the recognition unit 174 reads out objects existing within a certain range from the flight position of the flight device 100 in the map information, and determines whether or not the detection unit 12 has detected these objects. A certain range is, for example, a range in which an object can be detected by the detection unit 12.

When the recognition unit 174 determines that there is no object indicated to exist within a certain range from the flight position of the flight device 100 in the map information, the communication control unit 176 identifies this object in the map information. The specific information to be used and the information indicating that this object does not exist are transmitted to the management device 200. The management information acquisition unit 231 acquires the specific information and the information indicating that the object does not exist by the communication unit 21, and erases the object specified by the specific information in the map information stored in the storage unit 22. Updates the map information. Further, the communication control unit 176 may store these information in the storage unit 16 instead of transmitting the information to the management device 200.

By adopting such a configuration, when the detection unit 12 detects that the object shown in the map information does not exist, the absence of this object is shared for the flight of other flight devices. be able to.

[Modification 4]
Further, in the present embodiment, an example in which the generation unit 173 of the flight device 100 generates the second flight route has been described. However, the present invention is not limited to this. For example, in the specific unit 232 of the management device 200, the flight device 100 may generate a second flight route for avoiding the obstacle A.

When the detection unit 12 detects the obstacle A, the communication control unit 176 of the flight device 100 includes (1) the device ID, (2) the distance information measured by the distance measurement unit 13, and (3) the obstacle A. (4) Flight position information and flight altitude information acquired at the time of acquisition of the captured image, (5) direction information, and (6) horizontal direction of the obstacle A recognized by the recognition unit 174. , 3D information indicating the position and size in the vertical direction and the depth direction is transmitted to the management device 200.

The identification unit 232 specifies the size of the obstacle A based on the captured image and the distance information indicating the distance from the flight position at the time of capturing the captured image to the obstacle A. Further, the specific unit 232 generates the second flight route by the same method as the generation unit 173. The storage unit 22 stores the information related to the above-mentioned (1) to (6) and the second flight route in association with each other. In this case, the notification unit 235 notifies the flight device 100 of the second flight route generated by the specific unit 232. Further, the map display unit 233 associates the size of the obstacle A specified by the specific unit 232 with the icon of the obstacle A.

By adopting such a configuration, the second flight route can be generated by the processor in the specific unit 232 of the management device 200. Since the processor in the management device 200 has few design restrictions, it is possible to increase the processing capacity as compared with the processor in the flight device 100. Therefore, the time required to generate the second flight route can be shortened.

[Modification 5]
Further, the specific unit 232 may recognize the three-dimensional information indicating the position and size of the obstacle A in the horizontal direction, the vertical direction, and the depth direction by the same method as the recognition unit 174. In this case, the recognition unit 174 in the flight device 100 is unnecessary, and three-dimensional information indicating the horizontal, vertical, and depth directions and the size of the obstacle A acquired by the flight device 100 is transmitted to the management device. , The specific unit 232 generates the second flight route by referring to the positions and sizes of the obstacle A recognized by the specific unit 232 in the horizontal direction, the vertical direction, and the depth direction, similarly to the generation unit 173. May be good.

[Modification 6]
Further, the captured image acquisition unit 175 has described an example of acquiring an captured image of the obstacle A during hovering. However, the present invention is not limited to this. For example, the captured image acquisition unit 175 may acquire captured images of the obstacle A at a plurality of flight positions on the second flight route.

In this case, the distance measuring unit 13 measures the distance information for each flight position for acquiring the captured image, and the captured image acquisition unit 175 acquires the imaging date / time information and the direction information of each captured image, and the position information acquisition unit 171 acquires flight position information and flight altitude information at the time of acquisition of each captured image. In addition to the device ID of the flight device 100 and the three-dimensional information of the obstacle A recognized by the recognition unit 174, the communication control unit 176 provides the distance information, the imaging date / time information, the direction information, the flight position information, and the flight altitude information, respectively. It is transmitted to the management device 200.

Although the present invention has been described above using the embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments, and various modifications and changes can be made within the scope of the gist. be. For example, the specific embodiment of the distribution / integration of the device is not limited to the above embodiment, and all or a part thereof may be functionally or physically distributed / integrated in any unit. Can be done. Also included in the embodiments of the present invention are new embodiments resulting from any combination of the plurality of embodiments. The effect of the new embodiment produced by the combination has the effect of the original embodiment together.

11 GPS receiver 12 Detection unit 13 Distance measurement unit 14 Imaging unit 15 Communication unit 16 Storage unit 17 Control unit 21 Communication unit 22 Storage unit 23 Control unit 100 Flight device 171 Position information acquisition unit 172 Flight control unit 173 Generation unit 174 Recognition unit 175 Captured image acquisition unit 176 Communication control unit 200 Management device 231 Management information acquisition unit 232 Specific unit 233 Map display unit 234 Schedule acquisition unit 235 Notification unit

Claims (19)

A first image including an obstacle imaged by a flight device in flight on the first flight route and a first for avoiding the obstacle generated based on the flight position information captured by the first image. 2 A management information acquisition unit that acquires a second captured image including an obstacle captured by the flight device at a flight position on the second flight route when the flight device flies on the flight route. Equipped with
A management device characterized by that. The management information acquisition unit acquires the first captured image from the flight device.
The management device according to claim 1. The management information acquisition unit further acquires information indicating the direction of imaging of the first captured image or the second captured image from the flight position where the first captured image or the second captured image was captured.
The management device according to claim 1 or 2. The management information acquisition unit acquires the flight position information of the flight device at the time of capturing the first captured image or the second captured image from the flight device.
The management device according to any one of claims 1 to 3. The management information acquisition unit acquires unique device identification information assigned to the flight device from the flight device.
The management device according to any one of claims 1 to 4. The management information acquisition unit acquires three-dimensional information indicating the three-dimensional position of the obstacle acquired by the flight apparatus during the flight of the second flight route.
The management device according to any one of claims 1 to 5. The management information acquisition unit includes distance information indicating the distance from the flight position on the second flight route to an obstacle, imaging date / time information indicating the date and time when the second captured image was acquired, and flight on the second flight route. One of the direction information indicating the direction of capturing the second captured image from the position is acquired from the flight device.
The management device according to any one of claims 1 to 6. The management information acquisition unit includes distance information indicating the distance from the flight position on the second flight route to an obstacle, imaging date / time information indicating the date and time when the second captured image was acquired, and flight on the second flight route. One of the direction information indicating the direction of capturing the second captured image from the position is stored in association with the second flight route.
The management device according to claim 7. The management information acquisition unit stores the acquired second captured image in association with the second flight route.
The management device according to any one of claims 1 to 8. The management information acquisition unit acquires information indicating the second flight route generated by the flight device from the flight device.
The management device according to any one of claims 1 to 9. A specific portion for specifying the size of an obstacle based on the second captured image and the distance from the flight position of the flying device at the time of capturing the second captured image is further provided.
The management device according to any one of claims 1 to 10. A specific unit for specifying the type of obstacle based on the second captured image is further provided.
The management device according to any one of claims 1 to 11. A storage unit that stores map information and
A map display unit that adds information on the obstacle specified based on the second captured image acquired by the management information acquisition unit to the map information, and a map display unit.
Further prepare,
The management device according to any one of claims 1 to 12. The map display unit adds information indicating the position of the obstacle specified based on the second captured image acquired by the management information acquisition unit to the map information.
The management device according to claim 13. The map display unit adds information indicating the type of the obstacle specified based on the second captured image acquired by the management information acquisition unit to the map information.
The management device according to claim 13 or 14. The management information acquisition unit further acquires the imaging date and time of the second captured image, and further acquires the imaging date and time.
A storage unit that stores map information and
An obstacle captured in the second captured image based on the flight position information at the time of capturing the second captured image, the direction in which the second captured image is captured, and the distance from the flight position to the obstacle. A specific part that identifies the position of
The icon of the obstacle is displayed at the position on the map of the map information corresponding to the position of the obstacle specified by the specific unit, and the second captured image of the obstacle and the imaging date and time are set to the obstacle. Further equipped with a map display associated with the icon of
The management device according to any one of claims 1 to 10 and 13 to 15. The specific part identifies the size of the obstacle and
The map display unit associates the size of the obstacle specified by the specific unit with the icon of the obstacle.
The management device according to claim 16. The flight schedule of another flight device different from the flight device is acquired, and the flight schedule is to fly in an area within a predetermined distance from the position of the obstacle specified based on the second captured image acquired by the management information acquisition unit. Further comprises a schedule acquisition unit that identifies the other flight device.
The management device according to any one of claims 1 to 17. Further, the device for managing the other flight device specified by the schedule acquisition unit is further provided with a notification unit for notifying the information about the obstacle or the second flight route.
The management device according to claim 18.

Images