JP7031698B2 - Unmanned flight device control system and unmanned flight device control method - Google Patents

Description

The present invention relates to an unmanned flight device control system for controlling an unmanned flight device, an unmanned flight device control method, and an image projection device applicable to the unmanned flight device control system.

A building condition inspection (infrastructure deterioration diagnosis) is being conducted using an unmanned projectile. For example, the deterioration state of a building can be estimated from the surface state of the building captured by a camera mounted on an unmanned projectile. Hereinafter, the unmanned aerial vehicle will be referred to as a UAV (Unmanned Aerial Vehicle). The UAV can also be referred to as an unmanned aerial vehicle.

In order to use the UAV for infrastructure deterioration diagnosis, it is necessary to control the UAV in the vicinity of the building to be diagnosed.

In general technology, the UAV control unit uses a GPS (Global Positioning System) signal to determine the position of the UAV in order to guide the UAV to the target position. Patent Document 1 describes an aerial photography device including a GPS device having a GPS antenna and a camera including an image pickup unit.

Japanese Unexamined Patent Publication No. 2014-62789

When performing an infrastructure deterioration diagnosis using a UAV that grasps a position by a GPS signal, the following problems occur. In GPS with a single antenna attached to a moving object such as a UAV, the real-time positioning accuracy includes an error of several meters. As a result, when the UAV approaches the building that is the target of the infrastructure deterioration diagnosis, the distance between the building and the UAV may be too far or too close due to the large GPS measurement position error of the UAV. Further, depending on the positional relationship between the UAV and the GPS satellite, the UAV may pass through a place where the strength of the GPS signal is weak. When the UAV passes through a place where the strength of the GPS signal is weak, the UAV cannot accurately grasp the position of the UAV, and the guidance accuracy to the target position deteriorates.

In order to solve this problem with a simple configuration, an image for controlling the operation of the UAV is projected on an object such as a building, and the projected image is captured by a camera mounted on the UAV. It is conceivable to induce the UAV near the target position near the object.

In UAV guidance control based on such an image, the image captured by the camera mounted on the UAV may be depending on the environment in which the image is projected, such as the color of the object on which the image is projected and the ambient brightness at the time of image projection. May become unclear and image-based UAV control may not be performed correctly.

It should be noted that such a problem may occur not only in UAV control in infrastructure deterioration diagnosis but also in UAV control for guiding to a target position near an object such as patrol monitoring of a building.

Therefore, the present invention considers an environment in which an image is projected when an image for controlling the operation of the unmanned flight device is projected onto the object and the unmanned flight device is guided to the vicinity of a target position near the object. It is an object of the present invention to provide an unmanned flight device control system capable of controlling an unmanned flight device, an unmanned flight device control method, and an image projection device applicable to the unmanned flight device control system.

The unmanned flight device control system according to the present invention is an image for controlling the operation of the unmanned flight device equipped with the image pickup device, and relates to information indicating control contents regarding the flight or speed of the unmanned flight device and the orientation of the image pickup device. The operation of the unmanned flight device is controlled based on the image projection unit that projects an image including the information representing the control content and the information representing the control content regarding the attitude of the unmanned flight device onto the object, and the image captured by the image pickup device. It is equipped with a control unit.

Further, the unmanned flight device control method according to the present invention is an image for controlling the operation of the unmanned flight device equipped with the image pickup device, and includes information indicating the control content regarding the flight or speed of the unmanned flight device and the image pickup device. An image including information indicating the control content regarding the orientation and information indicating the control content regarding the attitude of the unmanned flight device is projected onto the object, and the operation of the unmanned flight device is controlled based on the image captured by the image pickup device.

According to the present invention, when an image for controlling the operation of an unmanned flight device is projected onto an object to guide the unmanned flight device near a target position near the object, an environment for projecting the image is considered. Can control unmanned flight equipment.

It is a schematic diagram which shows the unmanned flight apparatus control system of this invention, and the building to be inspected. It is a schematic diagram which shows the situation which the UAV grasps the movement control content to the next target position when the UAV takes a picture at the place deviated from the target position. It is explanatory drawing which shows the example of the information image. It is explanatory drawing which shows the meaning of the symbol in the information image exemplified in FIG. It is explanatory drawing which shows the example of the display mode of a bar. It is a block diagram which shows the structural example of a UAV. It is a schematic diagram which shows the example of the field of view of a camera. It is a schematic diagram which shows the example of the camera image generated by imaging the field of view shown in FIG. 7B. It is a schematic diagram which shows the example of the state which superposed the contour in the camera image which the contour information shows on the camera image shown in FIG. It is a schematic block diagram which shows the structural example of a projection station. It is a flowchart which shows the example of the processing progress of 1st Embodiment of this invention. It is a schematic diagram which shows the example of the camera image which cannot recognize an information image. It is a schematic diagram which shows the example of the camera image which the information image and the shadow of a UAV are overlapped. It is a schematic diagram which shows the example of the camera image in which the information image and the shadow of the UAV do not overlap due to the movement of the UAV. It is a schematic diagram which shows the example of the route of UAV. It is a schematic diagram which shows the example of the situation which the projection surface of an information image and the object surface of an image | imaging for state inspection are different. It is a schematic diagram which shows the example of the pattern image which shows the horizontal straight line. It is a schematic diagram which shows the state which the pattern image is projected on the surface of a building where a step is generated, and the camera is taking an image of the projection part. It is a schematic side view of the state shown in FIG. It is a schematic diagram which shows the example of the camera image obtained as a result of taking a pattern image. It is a schematic diagram which shows the outline of the unmanned flight apparatus control system of this invention.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.

Embodiment 1.
FIG. 1 is a schematic view showing an unmanned flight device control system of the present invention and a building to be inspected. The unmanned flight device control system of the present invention includes an unmanned aerial vehicle (hereinafter referred to as UAV) 10 and a projection station 20. As mentioned above, the UAV can also be referred to as an unmanned aerial vehicle.

The UAV 10 is equipped with a camera. The camera mounted on the UAV 10 captures the surface of the building 30 to be inspected and generates an image of the surface of the building. This image can be used to inspect the condition of a building (infrastructure deterioration diagnosis).

The projection station 20 includes a projector 21, and projects an image (hereinafter referred to as an information image) including information indicating the control content of the UAV 10 onto the surface of the building 30 from the projector 21. The projection station 20 is mounted on the vehicle 29 and can move or continue to stop at the same location.

The projection station 20 stores information on the appearance and the three-dimensional shape of the building 30 to be inspected in advance. Further, the projection station 20 recognizes the positional relationship between the projection station 20 and the building 30. The positional relationship between the projection station 20 and the building 30 is determined by using various surveying devices (not shown), and not only the position in the three-dimensional space but also the orientation is measured with high accuracy. The method of obtaining the three-dimensional shape of the building 30 may be a method using drawings at the time of design or 3D-CAD (3Dimension Computer-Aided Design) data, or using 3D-LiDAR (3Dimension Light Detection and Ranging) or the like. It may be a method of measuring the actual product in advance. Further, the information on the three-dimensional shape of the building 30 is not limited to the information on the entire building 30, but may be information on the portion of the building 30 to be observed using the present invention. Further, as for the method of accurately knowing the positional relationship between the building 30 and the projection station 20, since there are many known methods such as triangulation using a total station or the like, the description thereof will be omitted. Basically, the positional relationship between the projection station 20 and the building 30 is measured in advance. However, if the building 30 may be deformed during work due to the influence of wind or the like, or if the projection station 20 is in an unstable position such as on a ship, the positional relationship is measured during the work according to the present invention. You may.

The projection station 20 determines a plurality of projection points of the information image on the surface of the building 30 based on the positional relationship and the stored information, and derives the path 35 of the UAV 10. Further, the projection station 20 generates an information image to be projected on each projection point. Further, the projection station 20 determines a position (hereinafter referred to as a target position) for the UAV 10 to capture an information image, a posture (tilt or direction) at the target position, and a direction of the camera. This posture and the orientation of the camera are the posture and the orientation of the camera when capturing an information image. The projection station 20 generates, for example, an information image for instructing the movement control content to the next target position, the posture at the time of movement, and the direction of the camera, based on the positional relationship between the target positions. The information image also includes information instructing the UAV 10 to take an image for a state inspection. The projection station 20 sequentially projects an information image onto the surface of the building 30. As will be described later, in reality, the UAV 10 generally captures an information image at a position deviated from the target position, and the posture when capturing the information image and the orientation of the camera are also defined postures. It doesn't have to be.

The UAV 10 captures the surface of the building 30 on which the information image is projected by a camera. Hereinafter, the image obtained by the image taken by the camera of UAV10 will be referred to as a camera image. The UAV 10 operates according to the control content indicated by the information image in the camera image (information image shown in the camera image). For example, the UAV 10 takes an image for a state inspection or moves in the direction of the next target position based on the information image in the camera image.

By operating based on the information image in the camera image, the UAV 10 moves in the vicinity of the path 35 and images the surface of the building 30 for state inspection.

The UAV 10 does not always capture an information image at a target position on the path 35. Since the UAV 10 is affected by various factors such as wind, it is generally difficult for the UAV 10 to reach the same location as a predetermined target position. Therefore, when the UAV 10 reaches the vicinity of the target position, the UAV 10 images the surface of the building 30. The UAV 10 calculates the deviation between the position and the target position based on the camera image obtained as a result. Then, the UAV 10 derives the control content for movement from the current position to the next target position based on the movement control content indicated by the information image in the camera image and the calculated deviation.

FIG. 2 is a schematic diagram showing a situation in which the UAV 10 grasps the movement control content to the next target position when the UAV 10 takes a picture at a position deviated from the target position. In the example shown in FIG. 2, the areas A1 and A2 on the surface of the building 30 are projection points of the information image. Further, it is assumed that the target position P1 is defined as a position for capturing the information image projected on the region A1. Similarly, it is assumed that the target position P2 is defined as a position for capturing the information image projected on the region A2. Further, the posture of the UAV 10 and the orientation of the camera at each of the target positions P1 and P2 are also determined. As described above, since the UAV 10 is affected by various influences such as wind, it is difficult for the UAV 10 to capture an information image projected on the region A1 at a position that completely coincides with the target position P1. Therefore, the UAV 10 captures an information image projected on the region A1 at a position P1'near the target position P1. At this time, the posture of the UAV 10 and the orientation of the camera generally do not match the posture and the orientation of the camera at the predetermined target position P1.

The UAV 10 has the outline of the information image in the camera image when it is assumed that the information image projected on the region A1 is captured at the target position P1 in a predetermined posture and the direction of the camera, and actually, the position P1'. The deviation M (see FIG. 2) between the target position P1 and the position P1'is calculated based on the contour of the information image in the camera image obtained by capturing the information image in. Further, the information image projected on the area A1 shows the movement control content for moving from the target position P1 to the target position P2. The UAV 10 derives the control content for moving from the position P1'to the target position P2 based on the movement control content and the calculated deviation M, and is directed to the next target position P2 based on the control content. And move. The UAV 10 also captures the information image projected on the region A2 in the vicinity of the target position P2, and similarly derives the control content for moving to the next target position.

It should be noted that the information on the contour of the information image in the camera image when it is assumed that the information image is taken in the predetermined posture and the direction of the camera at the predetermined target position is given to the UAV 10 in advance.

Further, the UAV 10 generally cannot capture an information image at a front position of the information image due to various influences such as wind. Therefore, in the camera image, the information image is generally distorted. The UAV 10 recognizes the control content indicated by the information image captured in the camera image after performing a process of removing the distortion of the information image.

FIG. 3 is an explanatory diagram showing an example of an information image projected by the projection station 20. FIG. 4 is an explanatory diagram showing the meaning of the symbols in the information image illustrated in FIG.

The information image 40 includes, for example, a movement control symbol 41, a camera orientation control symbol 43, and a posture control symbol 44.

The movement control symbol 41 is a symbol indicating the control content regarding the movement of the UAV 10. The movement control symbol 41 can be used to indicate the direction of travel, the flight, and the speed. The movement control symbol 41 illustrated in FIG. 3 has the shape of an arrow, and the direction indicated by the arrow is the traveling method instructed. In the example shown in FIG. 3, the downward movement is instructed. The traveling direction indicated by the movement control symbol 41 is not limited to the downward direction.

Further, the movement control symbol 41 can instruct the UAV 10 to stay in the air by the added bar 42. Bar 42 means an instruction to stay in the air. When the bar 42 is provided at the end opposite to the arrowhead of the movement control symbol 41, it means that the bar 42 moves in the direction of the arrow (downward in this example) after staying in the air. Further, by displaying the bar 42 at the end of the movement control symbol 41 on the arrow head side, it is possible to instruct to move and then stay in the air. In the following description, in order to simplify the explanation, the UAV 10 recognizes the control content indicated by the information image 40, then stays in the air, takes an image for a state inspection within the flight time, and then is instructed. It will be described as moving in a direction. In this case, the bar 42 is displayed at the end opposite to the arrow head. Further, the bar 42 will be described as having a meaning of instructing an image pickup for a state inspection while in the air.

Further, the movement control symbol 41 indicates the movement speed when the UAV 10 moves, depending on how the shade of color inside the movement control symbol 41 (arrow-shaped symbol) changes. In FIGS. 3 and 4, the change in the color shade of the arrow-shaped symbol is represented by the change in the pattern for convenience. The shade of color in the actual arrow shape may change gradually, for example.

The direction of the bar 42 is determined to be perpendicular to the direction of the arrow, for example. However, the method of determining the orientation of the bar 42 is not limited to this example, and may be always horizontal, for example. FIG. 5 is an explanatory diagram showing an example of a display mode of the bar 42. In FIG. 5, the display of the shade of color inside the movement control symbol 41 is omitted. FIG. 5A shows an example in which the direction of the bar 42 is set perpendicular to the direction of the arrow. FIG. 5B shows an example in which the direction of the bar 42 is always set horizontally. When the arrow points to the right as illustrated in FIG. 5, in the example shown in FIG. 5 (a), the bar 42 is defined as a vertically oriented bar. On the other hand, in the example shown in FIG. 5B, the bar 42 is horizontal and is shown inside the arrow.

The camera orientation control symbol 43 is a symbol representing the control content regarding the orientation of the camera mounted on the UAV 10. The camera orientation control symbol 43 controls the elevation / depression angle and the horizontal angle of the camera. FIG. 3 illustrates a case where the camera orientation control symbol 43 is a U-shaped rotation arrow. In this example, it is assumed that the U-shaped rotation arrow means that the elevation / depression angle of the camera faces upward and the horizontal angle is fixed. The correspondence between the control content related to the camera orientation and the shape of the camera orientation control symbol 43 may be determined in advance.

If the orientation of the camera moving to the next target position and the orientation of the camera at the time of shooting corresponding to the next target position are separately specified, the projection station 20 is set to each orientation. The corresponding camera orientation control symbol may be included in the information image 40.

The attitude control symbol 44 is a symbol representing the control content related to the attitude of the UAV 10. FIG. 3 illustrates a case where the attitude control symbol 44 is a semicircular figure. In this example, the semicircular figure is assumed to mean that the posture of the UAV 10 is controlled horizontally. The relationship between the posture control content of the UAV 10 and the shape of the posture control symbol 44 may be determined in advance.

If the posture during movement to the next target position and the posture at the time of shooting corresponding to the next target position are separately defined, the projection station 20 controls the posture corresponding to each posture. The symbol may be included in the information image 40.

Further, although not shown in FIG. 3, the information in the camera image is assumed to be taken at a certain target position in a posture and a camera orientation determined corresponding to the target position. Information representing the outline of the image may be included in the information image 40. Hereinafter, information representing the contour of the information image in the camera image when it is assumed that the information image is captured at a certain target position in the posture and the direction of the camera determined corresponding to the target position is referred to as contour information.

The movement control symbol 41, the bar 42, the camera orientation control symbol 43, and the attitude control symbol 44 are identification information for identifying the control content of the UAV 10, and the information image 40 is an image including such identification information. Can be done. Further, FIG. 3 illustrates a case where the identification information (movement control symbol 41, bar 42, camera orientation control symbol 43, and attitude control symbol 44) for identifying the control content is represented by a graphic symbol. As illustrated in FIG. 3, the identification information that identifies the control content is represented by a symbol of a figure or the like, so that the inspection worker can easily see it even from a distance.

Further, the identification information that identifies the control content of the UAV 10 may be a symbol such as a QR code (registered trademark) or a barcode.

Further, the identification information that identifies the control content of the UAV 10 may be a symbol such as a Chinese character, a number, or a hieroglyph.

Further, the identification information that identifies the control content of the UAV 10 may be, for example, a sentence such as "flying 1 meter below after staying for 30 seconds".

Further, from the viewpoint of visibility of the inspection worker, the projection station 20 may project the information image 40 by blinking the information (identification information for identifying the control content) in the information image 40, for example. good. By projecting the information image 40 in this way, the content of the information image 40 can be made conspicuous, and the inspection worker can easily recognize the content.

Further, the projection station 20 may include other information in the information image 40 in addition to the identification information for identifying the control content. For example, the projection station 20 may include state information (for example, the current altitude of the UAV 10 or a step number indicating the progress of work) in the information image 40.

The above-mentioned contour information is used when obtaining the deviation M between the target position P1 and the actual position P1'. The information required to obtain the deviation M between the target position P1 and the actual position P1'does not have to be the contour information described above. The projection station 20 may include information necessary for obtaining the deviation M between the target position P1 and the actual position P1'in the information image 40, and the UAV 10 may store the information in advance. For example, the projection station 20 projects an information image 40 including special icons at four corners instead of contour information, and the UAV 10 may obtain a deviation M based on the positional relationship of those icons captured in the camera image. good. As described above, by using a shape that is easy to recognize without using a contour, it is possible to reduce the possibility that the UAV 10 misrecognizes the deviation M.

Further, the projection station 20 may fixedly include the information necessary for obtaining the deviation M between the target position P1 and the actual position P1'in the information image 40. For example, it is assumed that the information image 40 includes an inspection area number (not shown) indicating an inspection area. In the projection station 20, the first half of the inspection area number may be fixedly determined by, for example, an alphabet, and the second half may be arbitrarily determined. The UAV 10 may obtain the deviation M by the deformation of the fixed portion captured in the camera image. By fixing a part of the information image 40 in this way and obtaining the deviation M based on the image of the part in the camera image, the element in the information image 40 for obtaining the deviation M is an inspection worker. Can be out of the way.

Next, a configuration example of the UAV 10 will be described. FIG. 6 is a block diagram showing a configuration example of the UAV 10. In this example, a case where the UAV 10 has four propellers (not shown in FIG. 6) and has four motors corresponding to one-to-one with the four propellers will be described as an example. However, the number of propellers and motors is not limited to the above example.

The UAV 10 includes a camera 11, a control unit 13, a camera angle control unit 15, a motor driver 16, four motors 17a to 17d, and a communication unit 18.

The camera 11 takes an image of the surface of the building 30 (see FIG. 1) to be inspected for the infrastructure deterioration diagnosis, and generates a camera image showing the surface of the building. When the information image is projected in the field of view of the camera, the information image is also reflected in the camera image. The camera 11 inputs the generated camera image to the control unit 13. The orientation of the camera 11 is controllable.

Here, the field of view of the camera on the surface of the building 30 (see FIG. 1) and the camera image when the field of view is photographed will be described.

FIG. 7 is a schematic diagram showing an example of the field of view of the camera on the surface of the building 30 (see FIG. 1). FIG. 7A is a schematic top view showing an example of a situation in which the camera 11 of the UAV 10 (not shown in FIG. 7) captures the surface 30a of the building 30. Further, it is assumed that the information image 40 is projected on the surface 30a of the building 30. Due to the influence of wind and the like, it is difficult for the camera 11 to completely face the surface 30a of the building 30 and take an image of the surface 30a. Therefore, as shown in FIG. 7A, it is common for the camera 11 to image the surface 30a of the building 30 from an oblique direction. FIG. 7B is a schematic view showing the field of view of the camera 11 on the surface 30a of the building 30 in the state shown in FIG. 7A. The field of view 49 of the camera 11 on the surface 30a is a quadrangle. In the example shown in FIG. 7B, the case where the field of view 49 is trapezoidal is illustrated by the state in which the camera 11 images the surface 30a of the building 30 from an oblique direction. The field of view 49 is not always trapezoidal. However, the field of view 49 becomes rectangular when the camera 11 faces the surface 30a of the building 30, and as shown in FIG. 7A, the camera 11 obliquely faces the surface 30a of the building 30. The field of view 49 does not become rectangular when the image is taken. Further, the projection station 20 projects the information image 40 so that the information image 40 becomes a rectangle on the surface 30a. Therefore, the information image 40 projected on the surface 30a is rectangular. Let the vertices of the field of view 49 be A, B, C, and D. Further, the vertices of the information image 40 projected on the surface 30a of the building 30 are designated as a, b, c, and d. In FIG. 7B, the illustration of various symbols in the information image 40 is omitted. This point is the same in FIG. 8 and the like.

The camera 11 captures the field of view 49 and generates a camera image. FIG. 8 is a schematic diagram showing an example of a camera image generated by imaging the field of view 49 shown in FIG. 7 (b). The camera image 50 is a rectangle. The individual vertices of the camera image 50 correspond to the individual vertices of the field of view 49 shown in FIG. 7 (b). The apex of the camera image 50 corresponding to the apex A of the field of view 49 is also represented by the reference numeral A. The same applies to the vertices of other camera images. Further, the individual vertices of the information image 40 (see FIG. 8) captured in the camera image 50 correspond to the individual vertices of the information image 40 (see FIG. 7B) projected on the surface 30a of the building 30. ing. The vertices of the information image 40 (see FIG. 8) in the camera image 50 corresponding to the vertices a of the information image 40 shown in FIG. 7B are also represented by the reference numeral a. The same applies to the other vertices of the information image 40 in the camera image 50. Hereinafter, the contour of the information image 40 in the camera image 50 is referred to as a contour abcd.

The camera 11 captures a non-rectangular field of view 49 and produces a rectangular camera image 50. Therefore, the rectangular information image 40 projected in the field of view 49 of the camera 11 is distorted in the camera image 50 as shown in FIG.

When the camera image 50 is input from the camera 11, the control unit 13 detects the contour abcd of the information image 40 in the camera image 50.

Further, contour information is given to the control unit 13 in advance. As described above, the contour information is the contour of the information image in the camera image when it is assumed that the information image is taken at a certain target position in the posture and the direction of the camera determined corresponding to the target position. Information to represent. FIG. 9 is a schematic diagram showing an example of a state in which the contour in the camera image indicated by the contour information is superimposed on the camera image shown in FIG. The control unit 13 calculates the deviation M between the current position and the target position of the UAV 10 based on the contour 51 in the camera image 50 indicated by the contour information and the contour abcd of the information image 40 in the camera image 50. It should be noted that the deviation M includes not only the position but also the posture information. Since the same applies to the following, the posture information will be omitted unless otherwise specified.

When the information image cannot be recognized in the camera image, the control unit 13 moves the UAV 10 by controlling the motors 17a to 17d via the motor driver 16.

Further, when the information image can be recognized in the camera image, the control unit 13 calculates the deviation M between the current position of the UAV 10 and the target position. Further, the control unit 13 performs a process of removing the distortion of the information image 40 in the camera image 50 (in other words, a process of making the information image 40 in the camera image 50 rectangular), and the information image 40 after the process performs the process. The control content to be shown is grasped, and the UAV 10 is controlled according to the control content. For example, the control unit 13 performs an image pickup for inspecting the state of the surface of the building 30 according to the control content indicated by the information image 40. Further, the control unit 13 includes the calculated deviation M (deviation between the current position of the UAV 10 and the target position), the movement control content to the next target position shown by the information image 40, and the movement from the current position to the next target position. The UAV 10 is moved by deriving the control contents for the above and controlling the motors 17a to 17d via the motor driver 16 according to the control contents. Further, the control unit 13 controls the posture of the UAV 10 or controls the orientation of the camera 11 via the camera angle control unit 15 according to the control content indicated by the information image 40.

The camera angle control unit 15 controls the orientation of the camera 11 according to the control unit 13. Specifically, the camera angle control unit 15 controls the elevation / depression angle and the horizontal angle of the camera 11.

The motor driver 16 drives each of the motors 17a to 17d according to the control unit 13. The motor driver 16 can drive each of the motors 17a to 17d separately. Therefore, the control unit 13 can realize control for moving the UAV 10 back and forth, control for moving the UAV 10 left and right, control for moving the UAV 10 up and down, pitch angle control, roll angle control, and yaw angle control.

The communication unit 18 is a communication interface used for communication with the projection station 20.

The control unit 13 and the camera angle control unit 15 may be realized by, for example, a CPU of a computer that operates according to a program. In this case, the CPU may read the program and operate as the control unit 13 and the camera angle control unit 15 according to the program. Further, the control unit 13 and the camera angle control unit 15 may be realized by different hardware.

Next, a configuration example of the projection station 20 will be described. FIG. 10 is a schematic block diagram showing a configuration example of the projection station 20. The projection station 20 includes a projector 21, a projector control unit 22, a data storage unit 23, a route derivation unit 24, an information image generation unit 25, and a communication unit 27.

The data storage unit 23 is a storage device that stores information on the appearance of the building and the three-dimensional shape of the building.

The route deriving unit 24 recognizes the positional relationship between the projection station 20 and the building. At this time, the route deriving unit 24 may be given the current position of the projection station 20, for example, from the outside. The route derivation unit 24 determines a plurality of projection points of the information image on the surface of the building 30 based on the information of the positional relationship and the information on the appearance of the building and the three-dimensional shape of the building. Further, the route deriving unit 24 derives the route of the UAV 10.

The route derivation unit 24 defines a plurality of inspection areas by, for example, dividing the surface of the building in a grid pattern based on the information stored in the data storage unit 23, and determines a plurality of inspection areas, and a predetermined location (in each inspection area). For example, the center of the inspection area) may be defined as the projection point of the information image. However, the method for determining the projection location of the information image is not limited to the above example, and the projection location may be determined by another method. Further, the route deriving unit 24 may determine the route of the UAV 10 so that the information image projected on each projection point can be captured. At this time, it is preferable that the route deriving unit 24 determines the route that minimizes the power consumption of the UAV 10. However, the route deriving unit 24 may determine the route based on a standard other than power consumption.

At this time, the route deriving unit 24 determines the target position for the UAV 10 to capture the information image projected on the projection location in correspondence with the projection location. Further, the route deriving unit 24 determines the posture of the UAV 10 and the orientation of the camera at the target position. This posture and the orientation of the camera are the posture and the orientation of the camera when capturing an information image. However, due to the influence of wind and the like, it is difficult for the UAV 10 that actually moves to capture an information image at a predetermined target position with a specified posture and camera orientation. Therefore, the UAV 10 may capture an information image in the vicinity of the target position. Further, the posture of the UAV 10 and the orientation of the camera at that time do not have to match the posture and the orientation of the camera determined together with the target position.

The information image generation unit 25 generates an information image for each projection location. The information image generation unit 25 has a target position (#Pi) corresponding to the projection point ( _#i ) and a target position (#i + 1) corresponding to the next projection point (#i + 1). It suffices to generate an information image to be projected on the projection point #i based on the positional relationship with P _{i + 1} ). Further, the information image generation unit 25 generates contour information for each target position.

The projector 21 projects an information image on the surface of a building. The orientation of the projector 21 is variable.

The projector control unit 22 controls the orientation of the projector 21. By changing the direction of the projector 21 by the projector control unit 22, an information image can be projected on a predetermined projection location.

Further, when the projector 21 is directed to the projection point of the information image, the projector control unit 22 is on the surface of the building based on the information of the orientation of the projector 21, the appearance of the building, and the three-dimensional shape of the building. Correct the information image so that the information image is not distorted. In other words, the projector control unit 22 corrects the information image so that the rectangular information image is projected on the surface of the building. The projector 21 projects an information image corrected by the projector control unit 22. In the following description, the description of the information image correction operation by the projector control unit 22 may be omitted.

The communication unit 27 is a communication interface used for communication with the UAV 10.

The route derivation unit 24, the information image generation unit 25, and the projector control unit 22 may be realized by, for example, a CPU of a computer that operates according to a program. In this case, the CPU may read the program and operate as the route derivation unit 24, the information image generation unit 25, and the projector control unit 22 according to the program. Further, the route derivation unit 24, the information image generation unit 25, and the projector control unit 22 may be realized by separate hardware.

Further, the control unit 13 (see FIG. 6) may be provided in the projection station 20 or another information processing device instead of the UAV 10. Then, the operation of the control unit 13 is performed by the projection station 20 or another information processing device, and the movement control, the attitude control, the orientation control of the camera 11 and the like of the UAV 10 are performed by communication with the outside (for example, the projection station 20). It may be realized.

Next, the processing process of the first embodiment of the present invention will be described. FIG. 11 is a flowchart showing an example of the processing progress of the first embodiment of the present invention. The flowchart shown in FIG. 11 is an example, and the processing process of the present invention is not limited to the flowchart shown in FIG.

Further, the path deriving unit 24 of the projection station 20 determines in advance a plurality of projection points of the information image on the surface of the building, derives the path of the UAV 10, and sets each target position for the UAV 10 to capture the information image, and , It is assumed that the posture of the UAV 10 and the orientation of the camera at each target position are determined. Further, in the first embodiment, in order to simplify the explanation, it is assumed that the route is not changed during the inspection. Further, it is assumed that there is no influence on the UAV 10 due to a large momentary disturbance (for example, the UAV 10 is greatly deviated from the path due to a momentary gust). However, deviation from the target position due to normal wind or the like (see FIG. 2) may occur.

The projection station 20 projects an information image from the projector 21 onto a projection point of the image on the surface of the building (step T1). In the first step T1, the projection station 20 projects the first information image onto the projection point of the image.

Further, the control unit 13 moves the UAV 10 toward the target position for capturing the target image projected in step T1 (step B1). When the first information image is projected, the inspection worker remotely controls the UAV 10. The inspection worker moves the UAV 10 so that the first information image is within the field of view of the camera 11. The control unit 13 moves the UAV 10 so that the information image projected on the surface of the building is within the field of view of the camera 11 according to the remote control by the inspection worker.

Further, the UAV 10 captures the surface of the building by the camera 11 while moving, and obtains a camera image (step B2). The camera 11 inputs the generated camera image to the control unit 13.

When the camera image is input, the control unit 13 searches for an information image in the camera image (step B3). Successful search means that the information image could be recognized in the camera image. In addition, the fact that the search failed means that the information image could not be recognized in the camera image. When the information image can be recognized in the camera image (Yes in step B3), the control unit 13 calculates the deviation between the current position of the UAV 10 and the target position (step B4). If the information image cannot be recognized in the camera image in step B3, such as when the information image is not in the camera image or when the information image is interrupted, the UAV 10 continues to move.

FIG. 12 is a schematic diagram showing an example of a camera image in which an information image cannot be recognized.

FIG. 12A shows a case where the information image is not shown in the camera image 50. In this case, since the information image does not exist in the camera image, the control unit 13 cannot recognize the information image in the camera image.

FIG. 12B shows a case where only a part of the information image 40 is captured in the camera image 50. In this case, since the information image 40 is interrupted, the control unit 13 cannot recognize the information image in the camera image.

FIG. 12C shows a case where the information image 40 is shown in the camera image 50, but the area of the information image 40 in the camera image 50 is very small (smaller than the threshold value). In this case, the area of the information image 40 is too small, so that the control unit 13 cannot recognize the information image in the camera image.

When the information image cannot be recognized in the camera image (No in step B3), the UAV 10 repeats the operations after step B1. If the control unit 13 is moving in the target position direction corresponding to the first projection point, the control unit 13 may not be able to determine in which direction the movement should be based on the most recently obtained camera image. Subsequently, the UAV 10 is moved according to the remote control by the inspection worker. For example, if no information image is shown in the most recently obtained camera image, the control unit 13 continues to move the UAV 10 according to a remote control by the inspection worker. Further, if the control unit 13 can determine in which direction to move based on the latest camera image, the control unit 13 starts an autonomous operation regardless of the remote control by the inspection worker. For example, as illustrated in FIG. 12B, when a part of the information image 40 is shown in the camera image 50 obtained most recently, the control unit 13 puts the entire information image 40 in the field of view of the camera 11. It is possible to determine in which direction the UAV 10 should be moved in order to put it in. In this case, the control unit 13 autonomously moves the UAV 10 so that the entire information image 40 is within the field of view of the camera 11.

If the information image cannot be recognized in the camera image (No in step B3), steps B1 to B3 are repeated until the information image can be recognized in the camera image.

When the information image can be recognized in the camera image, as illustrated in FIG. 9, the information image 40 is shown in the camera image 50 in an appropriate size (in other words, the area is larger than the threshold value). ), The outline abcd of the information image 40 can be detected. In this case, the control unit 13 determines the current state of the UAV 10 based on the contour 51 (see FIG. 9) in the camera image 50 indicated by the contour information and the contour abcd (see FIG. 9) of the information image 40 in the camera image 50. The deviation between the position and the target position is calculated (step B4).

The fact that the information image can be recognized in the camera image can be said to mean that the UAV 10 exists in the vicinity of the target position. In reality, the UAV 10 repeats the processes of steps B1 to B3 while moving. Therefore, at the moment when all the information images are included in the camera image, it is assumed that the information image is completely recognized in step B3, and the process proceeds to step B4. In other words, even if the UAV 10 is relatively far from the target position, the control unit 13 executes step B4 at the moment when the information image enters toward the edge of the camera image. In order to prevent this, although omitted in FIG. 11, "stop the transition to step B4 until the information image falls within a certain range of the camera image" or "information within a certain range of the camera image". "Determine the moving direction for inserting the image and repeat step B1 and subsequent steps", or "Stop the transition to step B4 until the information image is recognized in the camera image for a certain period of time (or a certain number of frames)", etc. May be processed. Alternatively, since the deviation M can be calculated in step B4, it is more preferable to calculate the movement amount corresponding to this deviation M and repeat steps B1 and subsequent steps. In addition to this, it is even better to repeat step B1 while controlling the orientation of the camera so that the deviation M has a predetermined shape. This is because the UAV 10 can be moved to a more accurate position or the UAV 10 can be moved to a more accurate posture by moving the UAV 10 based on the deviation M or controlling the camera 11.

After calculating the deviation between the current position and the target position of the UAV 10, the control unit 13 removes the distortion of the information image 40 in the camera image 50 (step B5). As described above, the information image 40 projected on the surface of the building is rectangular, and the field of view 49 of the camera is not rectangular (see FIG. 7B). On the other hand, the camera image 50 obtained by imaging the field of view 49 is rectangular. Therefore, the information image 40 captured in the camera image 50 is not a rectangle but a distorted state (see FIG. 8). In step B5, the control unit 13 removes this distortion.

Subsequently, the control unit 13 recognizes the command (control content) indicated by the information image 40 after the distortion is removed, and executes the command (step B6).

In this example, a case where the control unit 13 recognizes a command (control content) shown by the information image 40 exemplified in FIG. 3 and executes the command will be described as an example.

Specifically, the control unit 13 recognizes that it stays in the air for a predetermined time on the spot and takes an image for inspecting the state of the surface of the building during the flight. Then, the control unit 13 stays in the air for a predetermined time on the spot according to the command, and takes an image of the surface of the building in the air with the camera 11. At this time, the information image 40 may remain projected in the field of view of the camera 11.

Further, the control unit 13 recognizes that the camera 11 descends after staying in the air for a predetermined time, that the camera 11 is turned upward while the horizontal angle of the camera 11 is fixed when descending, and that the attitude of the UAV 10 is controlled horizontally. .. At this time, the control unit 13 determines the control content for moving to the next target position based on the deviation between the current position and the target position of the UAV 10 calculated in step B4 and the movement control content for descending. The UAV 10 is moved by deriving and controlling each of the motors 17a to 17d via the motor driver 16 according to the control content. That is, the control unit 13 corrects the control content related to the movement indicated by the information image 40 by the deviation calculated in step B4, and moves the UAV 10 based on the corrected control content.

Further, after recognizing the command indicated by the information image 40, the control unit 13 notifies the projection station 20 that the command recognition is completed (step B7).

When the control unit 13 starts moving the UAV 10 based on the above-mentioned corrected control content, the control unit 13 repeats the operations after step B1. In this step B1, unlike the case of moving toward the target position corresponding to the first projection point, the control unit 13 autonomously moves the UAV 10 without depending on the remote control by the inspection worker.

When the projector control unit 22 receives the notification from the UAV 10 that the command recognition is completed, the projector control unit 22 stops the projection of the information image previously projected on the projector 21. Further, when the information image generation unit 25 receives the notification from the UAV 10 that the command recognition is completed, the information image generation unit 25 identifies the projection location of the next information image (here, the projection location of the second information image), and the projection location thereof. Generate a second information image to be projected onto (step T2). In this example, the information image generation unit 25 is a command to instruct imaging for a state inspection in the vicinity of the second target position and a command to move the UAV 10 from the second target position to the third target position. And so on to generate an information image.

The UAV 10 takes an image of the surface of the building by the camera 11 while moving (step B2), and the control unit 13 executes step B3. Since the second information image is not captured in the camera image until the second information image is projected, the UAV 10 repeats steps B1 to B3.

When the information image generation unit 25 generates an information image (here, the second information image) to be projected on the next projection point in step T2, the projector control unit 22 projects the information image on the projection point. Controls the projector 21. As a result, the projector 21 projects the second information image on the projection portion of the second information image (step T1).

When the UAV 10 moves to a position where the newly projected information image enters the field of view of the camera 11 and the information image can be recognized in the camera image (Yes in step B3), the UAV 10 repeats the processes after step B4. Just do it. Further, when the projection station 20 receives the notification in step B7, the projection station 20 may stop the projection of the information image projected up to that point and execute steps T2 and T1.

By repeating the above process, the UAV 10 can image the surface of the building for the condition inspection while moving in the vicinity of the defined path.

Further, when the projection station 20 projects the last information image and the control unit 13 executes the command shown in the last information image, the unmanned flight device control system does not give the notification of step B7. The process may be terminated. In this case, the projection station 20 should include a predetermined symbol indicating that it is the last information image in the last information image so that the control unit 13 can recognize that it is the last information image. Just do it.

According to the present embodiment, the control unit 13 repeats steps B1 to B3 until the information image can be recognized in the camera image, and when the information image can be recognized in the camera image, the deviation M is calculated. It removes the distortion of the information image in the camera image, recognizes the command indicated by the information image, and operates according to the command. By repeating this operation, the control unit 13 can take an image of the surface of the building for the state inspection while moving in the vicinity of the defined path. Therefore, the UAV 10 can be guided to the vicinity of the target position in the vicinity of the building with a simple configuration.

Further, in the present invention, since the UAV 10 is guided by the information image, the UAV 10 does not need to receive the GPS signal. Therefore, in the present invention, it is not necessary to provide a large number of facilities as substitutes for GPS satellites in order to obtain position information by GPS signals at any place. In this respect as well, the present invention can simplify the system configuration.

Further, in the present invention, the UAV 10 is induced by an information image. Since the UAV 10 approaches the information image and obtains a deviation, it is possible to accurately obtain the position and orientation of the information image and the UAV 10. That is, since the UAV 10 which is a measuring device is close to the information image to be measured, the error factor is reduced, and as a result, the position measurement is more precise than that of GPS or other methods, and the measurement result is based on the measurement result. Guidance is possible.

Further, the UAV 10 derives the deviation between the current position of the UAV 10 and the target position based on the information image. By using this deviation as a difference, the UAV 10 can be accurately moved to the target position.

Further, the UAV 10 derives the movement control content to the next target position based on the deviation between the current position and the target position of the UAV 10 and the movement control content indicated by the information image. As a result, the UAV 10 can move to the vicinity of the next target position. That is, the movement of the UAV 10 can be controlled with good accuracy. Further, as described above, in the present invention, the UAV 10 does not need to receive a GPS signal. Therefore, the UAV 10 can be controlled accurately even in a place where it is difficult to receive GPS signals.

Further, the projection station 20 projects an information image onto a building with a certain size, and the UAV 10 monitors this information image. Further, since the positional relationship between this information image and the UAV 10 can be calculated inside the UAV 10, the UAV 10 can continuously acquire the positional relationship in substantially real time. Therefore, for example, even if the position of the UAV 10 is significantly changed or the posture is lost due to a disturbance such as a gust, the UAV 10 can correct the position and the posture before coming into contact with the building.

Next, a modification of the first embodiment will be described.

In the above embodiment, in step B5, the control unit 13 has described the case where the distortion of the information image 40 in the camera image 50 is removed. If the control unit 13 can recognize the command indicated by the information image 40 without removing the distortion of the information image 40, step B5 may be omitted.

Further, in the above embodiment, when the surface of the building is imaged for the state inspection of the building in step B6, the surface of the building remains as the information image 40 is projected in the field of view of the camera 11. The case of imaging is shown. When the surface of the building is imaged for the condition inspection of the building, the control unit 13 may instruct the projection station 20 to stop the projection of the information image. In this case, the information image generation unit 25 of the projection station 20 generates an information image instructing an image to be imaged for a state inspection and an information image instructing to move to the next target position for one projection point. do it. The information image generation unit 25 gives an instruction to notify the projection station 20 of the cancellation of projection of the information image in the information image instructing the imaging for the state inspection, and after the notification (in other words, the projection station 20 It includes a command to image the building for condition inspection (after stopping the projection in response to the notification) and a command to notify the projection station 20 of the completion of the imaging after the imaging is completed. The projection station 20 first projects the information image onto the projection location. When the control unit 13 recognizes these commands and notifies the projection station 20 that the projection is stopped, the projection station 20 stops the projection of the information image. The UAV 10 images the surface of the building in that state. Further, when the UAV 10 notifies the projection station 20 of the completion of imaging, the information image instructing the movement to the next target position is projected on the same projection point. At this time, the UAV 10 may move to some extent due to the influence of wind or the like during the imaging for the state inspection. Therefore, it is preferable that the UAV 10 repeats the operations after step B2 even after the information image instructing the movement to the next target position is projected. This is to recalculate the deviation between the current position of the UAV 10 and the target position. Then, in step B6, the control unit 13 may correct the control content related to the movement indicated by the information image by the deviation.

When the surface of a building is imaged with the information image projected, the part other than the information image may be relatively dark in the camera image. As described above, when the UAV 10 performs imaging for state inspection, the projection station 20 stops the projection of the information image, so that such a situation can be prevented.

Further, when the projection station 20 stops the projection of the information image when the UAV 10 performs the image pickup for the state inspection, the projector control unit 22 stops the projection of the information image on the projector 21 and then causes the projector 21 to be white. It may be irradiated with light or light of an appropriate color. At this time, the projector 21 irradiates the surface of the building to be inspected with light according to the projector control unit 22. As a result, the UAV 10 takes an image of the surface of the building for inspection of the state of the building while the surface of the building is irradiated with light.

By irradiating the surface of the building with light by the projector 21, the camera 11 of the UAV 10 compensates for the lack of light when the surface of the building is imaged. For example, when the camera 11 takes an image of a deep part of a building, or when the unmanned flight device control system is operated at night, the amount of light is insufficient and a camera image clearly showing the surface of the building can be obtained. do not have. When the camera 11 of the UAV 10 images the surface of the building for the condition inspection of the building, the projector 21 irradiates the surface of the building with light, so that the lack of light can be compensated for as described above, and the camera. 11 can generate a camera image in which the surface of the building is clearly captured.

The projection station 20 may be provided with a floodlight for irradiating light, in addition to the projector 21.

Further, the shape of the contour of the information image projected from the floodlight station 20 does not always have to be the same. The contour information needs to be known prior to step B3 of recognizing the information image. Therefore, it may be included in the previous information image, the projection station 20 may notify the UAV 10 by communication, or the contour information for each information image may be stored in the UAV 10 in advance. The control unit 13 may apply contour information corresponding to the information image each time the deviation M is calculated. Further, by deforming the shape of the contour of the information image, it is possible to prevent the information image from becoming an obstacle when inspecting the state of the building.

In each of the following embodiments, the same matters as those already described will be omitted.

Embodiment 2.
In the second embodiment of the present invention, the operation when the posture and position of the UAV 10 are changed due to a large disturbance of the UAV 10 and the control unit 13 loses sight of the information image captured in the camera image will be described. do. Even if an information image is shown in the camera image before the UAV 10 is disturbed and the control unit 13 recognizes the information image, it is projected onto the building due to the change in the posture and position of the UAV 10 due to the disturbance. If the information image is out of the field of view of the camera 11, the information image will not be captured in the camera image. That is, the control unit 13 loses sight of the information image in the camera image.

In this case, the control unit 13 changes the posture of the UAV 10 in each direction and takes an image with the camera 11. Alternatively, the control unit 13 changes the direction of the camera 11 via the camera angle control unit 15 and takes an image with the camera 11. At this time, the control unit 13 may change the posture of the UAV 10 and the orientation of the camera 11. As a result, if the control unit 13 acquires the camera image in which the information image is captured, the control unit 13 may perform the operations after step B4 (see FIG. 11).

Further, at this time, the control unit 13 changes the position and posture of the UAV 10 from the position and posture when the control unit 13 recognizes the information image in the camera image based on the camera images of the latest plurality of frames. You may detect how much it has changed.

Alternatively, the UAV 10 may be equipped with an inertial navigation system. Then, the inertial navigation system may detect how much the position and attitude of the UAV 10 have changed from the position and attitude when the control unit 13 recognized the information image in the camera image.

Then, the control unit 13 determines in which direction the posture of the UAV 10 and the direction of the camera 11 are directed in order to direct the field of view of the camera 11 toward the information image, based on the amount of change in the position and the posture of the UAV 10. You may. The control unit 13 may change the posture of the UAV 10 and the orientation of the camera 11 according to the determination result. In this case, the control unit 13 can quickly turn the field of view of the camera 11 toward the information image, and can quickly capture the projected information image.

Further, the control unit 13 may notify the projection station 20 of the amount of change in the position and posture of the UAV 10 from the time when the control unit 13 recognized the information image in the camera image. Similar to the above example, the control unit 13 may detect this amount of change based on the camera images of the latest plurality of frames. Alternatively, the inertial navigation system may detect the amount of change in the position and attitude of the UAV 10 from the time when the UAV 10 is provided with the inertial navigation system and the control unit 13 recognizes the information image in the camera image.

Upon being notified of the amount of change in the position and posture of the UAV 10, the information image generation unit 25 (see FIG. 10) of the projection station 20 calculates the field of view of the camera 11 based on the amount of change, and the field of view is within the field of view. Generate an information image to project. This information image is an information image for guiding the UAV 10 to a position where the control unit 13 recognized the information image in the camera image. The projector control unit 22 directs the projector 21 in the direction of the calculated field of view of the camera, and causes the projector 21 to project the information image. Then, the camera 11 may take an image (step B2), and then the UAV 10 may perform the operations after step B3. As a result, the UAV 10 can return to the position where the control unit 13 recognized the information image in the camera image according to the information image. Further, the projector control unit 22 may reproject the original information image onto the original projection location on the projector after receiving the notification in step 7. Also in this case, the camera 11 may take an image (step B2), and then the UAV 10 may perform the operations after step B3.

Further, the projection station 20 detects a change in the position and posture of the UAV 10 based on a tracking camera (not shown) that captures the UAV 10 while tracking the UAV 10 and an image of the UAV 10 obtained by the tracking camera. It may be configured to include a change amount detection unit (not shown). In this case, when the state change amount detection unit detects that the UAV 10 has suddenly moved due to a disturbance based on the image obtained by the tracking camera, the change in the position and posture of the UAV 10 from the time immediately before the movement. Detect the amount. Then, as in the above case, the information image generation unit 25 (see FIG. 10) calculates the visual field range of the camera based on the amount of change, and generates an information image to be projected on the visual field range. This information image is an information image for guiding the UAV 10 to a position where the control unit 13 recognized the information image in the camera image. The projector control unit 22 directs the projector 21 in the direction of the calculated field of view of the camera, and causes the projector 21 to project the information image. Subsequent operations are the same as above, and description thereof will be omitted.

According to the second embodiment, even if the UAV 10 has the field of view of the camera 11 oriented in a direction different from the information image due to the disturbance, the UAV 10 captures the information image again and returns to the normal control. can do.

Embodiment 3.
In the third embodiment of the present invention, the unmanned flight device control system avoids a state in which the information image and the shadow of the UAV 10 overlap and appear in the camera image.

The path derivation unit 24 of the projection station 20 determines a projection location that does not overlap with the shadow of the UAV 10 as a projection location of the information image.

For example, the route deriving unit 24 defines a plurality of inspection areas by dividing the surface of the building in a grid pattern based on the information stored in the data storage unit 23, and determines a predetermined location (in each inspection area). For example, the center of the inspection area) is defined as the projection point of the information image. Further, the route deriving unit 24 also determines the target position of the UAV 10 and the posture of the UAV 10 and the orientation of the camera at the target position for each projection location. Then, the route derivation unit 24 determines whether or not the projection location and the shadow of the UAV 10 overlap each other at each projection location. This determination can be made based on the positional relationship between the projection station 20, the target position, and the projection location. The route deriving unit 24 changes the projection location that overlaps the shadow of the UAV 10 to a position that does not overlap the shadow of the UAV 10 in the inspection area, and in accordance with the change, the posture of the UAV 10 corresponding to the projection location and the orientation of the camera are also changed. change. At this time, the route deriving unit 24 may also change the target position of the UAV 10. If the position of the projection point in the inspection area overlaps with the shadow of the UAV10 even if the position of the projection point is changed, it is determined that the projection point is not determined for the inspection area. Then, the route deriving unit 24 determines a route for capturing the information image projected on each projection portion of the projection portion that does not overlap with the shadow of the UAV 10.

By defining the projection location of the information image in this way, it becomes easy to prevent the information image projected on the projection location and the shadow of the UAV 10 from being reflected in the camera image in an overlapping state.

Further, as described above, even when the path deriving unit 24 determines the projection location that does not overlap with the shadow of the UAV 10, the UAV 10 is affected by wind and the like, so that the information image is displayed at the target position corresponding to the projection location. Imaging is generally rare. As a result, even when the projection location is determined as described above, the information image projected on the projection location and the shadow of the UAV 10 may overlap.

In this case, the control unit 13 may move the UAV 10. FIG. 13 is a schematic diagram showing an example of a camera image in which the information image and the shadow of the UAV 10 are superimposed. When the control unit 13 determines that a part of the outline of the information image 40 is interrupted by the presence of a black pixel group, as illustrated in FIG. 13, a part of the outer peripheral portion of the information image 40 and the shadow 55 are formed. Judge that they overlap. When the control unit 13 determines in this way, for example, the control unit 13 may move in an arbitrary direction and execute the operations after step B1 (see FIG. 11). Further, at this time, the control unit 13 may determine the direction in which the UAV 10 is moved based on the portion of the contour of the information image 40 that is interrupted by the shadow 55, and may move the UAV 10 in that direction. FIG. 14 is a schematic diagram showing an example of a camera image in which the information image 40 and the shadow 55 of the UAV 10 do not overlap due to the movement of the UAV 10. In the camera image illustrated in FIG. 14, the position of the entire contour of the information image 40 can be detected. After moving the UAV 10, the control unit 13 can calculate the deviation of the current position from the target position by using the camera image illustrated in FIG. After that, the control unit 13 may execute the operations after step B5 (see FIG. 11).

When the overlap between the information image 40 and the shadow of the UAV 10 is detected in this way, the control unit 13 can avoid the overlap between the information image 40 and the shadow of the UAV 10 by moving the UAV 10. This operation can be applied even when the path deriving unit 24 does not determine the projection location so that the shadow of the UAV 10 and the projection location do not overlap.

In the above description, the operation of avoiding the overlap between the information image 40 and the shadow of the UAV 10 by moving the UAV 10 has been described. The control unit 13 may avoid the overlap between the information image 40 and the shadow of the UAV 10 by causing the projection station 20 to change the projection position of the information image. For example, it is assumed that the control unit 13 determines that a part of the outer peripheral portion of the information image 40 and the shadow 55 overlap each other, as in the above case. In this case, the control unit 13 has the projection station 20 projecting the information image in order to avoid the overlap between the information image 40 and the shadow 55 based on the portion of the contour of the information image 40 that is interrupted by the shadow 55. It is determined in which direction and how much the image should be moved, and the movement direction and the amount of movement of the projection point are notified to the projection station 20. It can be said that this notification is an instruction to change the projection location. Upon receiving this notification, the information image generation unit 25 of the projection station 20 changes the content of the information image 40 according to the notified movement direction and movement amount. This is because the movement control content to the next projection location changes as the projection location of the information image changes. Further, the route deriving unit 24 also determines the target position when the projection point is moved according to the notified movement direction and the amount of movement, and the posture and the direction of the camera at the target position. The projector control unit 22 directs the projector 21 toward the projection location according to the notified movement direction and movement amount, and causes the projector 21 to project the information image 40. As a result, the UAV 10 obtains, for example, the camera image illustrated in FIG. After that, the control unit 13 may execute the operations after step B4 (see FIG. 11).

In this way, even if the UAV 10 causes the projection station 20 to change the projection position of the information image 40, it is possible to avoid the overlap between the information image 40 and the shadow of the UAV 10. This operation can be applied even when the path deriving unit 24 does not determine the projection location so that the shadow of the UAV 10 and the projection location do not overlap.

Embodiment 4.
In the fourth embodiment of the present invention, the UAV 10 changes the route in the middle of the route when the power consumption is large due to the influence of the weather or the like and the imaging for the state inspection cannot be performed in all the inspection areas.

As described in the first embodiment, the route deriving unit 24 defines the route of the UAV 10 so that the information image projected on each projection point can be captured, for example. That is, the path of the UAV 10 is defined so as to pass near each projection point.

At this time, it is preferable that the route deriving unit 24 determines the route that minimizes the power consumption of the UAV 10. Here, the UAV 10 consumes a large amount of electric power when the flight speed vector changes. Therefore, the route deriving unit 24 may determine, for example, a route that minimizes the change in the velocity vector of the UAV 10.

It is assumed that the route deriving unit 24 defines the route illustrated in FIG. FIG. 15 illustrates each inspection area on the surface of the building and the route 61 of the UAV 10. Further, the inspection area Ra is the first inspection area, and the inspection area Rz is the last inspection area.

The route derivation unit 24 determines on the route 61 a position where the UAV 10 can be switched to a shortcut route (hereinafter referred to as a switchable position). The switchable position may be one or a plurality. The switchable position 71 shown in FIG. 15 will be described as an example.

After defining the route 61, the route deriving unit 24 determines a plurality of route candidates that are shortcuts from the switchable position 71 to the end point 79. The route derivation unit 24 determines route candidates so as not to pass through the inspection area that has already passed. Further, since the route candidate is a shortcut from the switchable position 71 to the end point 79, it is not necessary to pass through all the unpassed inspection areas. Further, the number of inspection areas through which the plurality of route candidates defined for the switchable position 71 pass may be different for each route candidate. In other words, the lengths of the route candidates from the switchable position 71 to the end point 79 may be different. In addition, in this example, in order to simplify the explanation, in each candidate of the route, the movement route for one section from the switchable position 71 to the next inspection area is assumed to be the same as the predetermined route.

The route derivation unit 24 determines a plurality of route candidates that are shortcuts to the end point 79 for each switchable position for each switchable position other than the switchable position 71.

Hereinafter, for the sake of simplicity, the target position of the UAV 10 corresponding to the inspection area is defined as the switchable position.

When the control unit 13 of the UAV 10 notifies the projection station 20 that the command recognition is completed (step B7), the control unit 13 determines the energy consumption after the start of the flight, the remaining battery level at that time, and the weather (here, the wind speed of the crosswind). (For example) is measured, and those values are also notified to the projection station 20. The UAV 10 may be equipped with a sensor for measuring these values. However, the control unit 13 consumes energy, the remaining battery power, and the weather (for example, the condition that the remaining battery power is equal to or higher than the threshold value) that a sufficient flight distance can be secured (for example, the condition that the remaining battery power remains above the threshold value) is satisfied. It is not necessary to notify the projection station 20 of the crosswind wind speed).

Further, the route derivation unit 24 holds a prediction formula for the flightable distance in advance, in which the energy consumption, the remaining battery level, and the weather (wind speed of the crosswind) are used as explanatory variables, and the flightable distance of the UAV 10 is used as the explained variable. .. This prediction formula may be predetermined by machine learning such as regression analysis, for example. When the UAV 10 notifies the energy consumption, the remaining battery level, and the weather, the route deriving unit 24 calculates the flight distance of the UAV 10 by substituting those values into the explanatory variables of the prediction formula.

Further, the route deriving unit 24 subtracts the flightable distance calculated by the prediction formula from the length of the route candidate for each route candidate predetermined for the switchable position where the UAV 10 exists. The route derivation unit 24 determines that the candidate for the route having the smallest subtraction result is the optimum shortcut, and thereafter, the candidate for the route is used as the route of UAV10.

The power consumption of the UAV 10 is affected by the weather such as wind. For example, when a strong wind is blowing, a thrust facing the strong wind is required, so that the power consumption is higher than when there is no wind. As a result, when the UAV 10 tries to continue moving along the path 61 initially determined, the remaining battery level may become 0 in the middle of the path. According to the present embodiment, since it is possible to switch to a route that is a shortcut to the end point 79 at the switchable position, the flight of the UAV 10 is maintained up to the end point 79 in exchange for reducing the number of inspection sections to pass through. Can be done.

In the above example, energy consumption, battery level, and weather are exemplified as explanatory variables used in the formula for predicting the flight distance, but other explanatory variables may be used.

Embodiment 5.
In the fifth embodiment of the present invention, when the position of the UAV 10 changes due to a large disturbance of the UAV 10, the projection station 20 again derives the path from the changed position to the end point of the path. do.

When the UAV 10 moves due to a disturbance and the control unit 13 of the UAV 10 loses sight of the information image in the camera image, the control unit 13 recognizes the information image in the camera image as in the case illustrated in the second embodiment. Notify the projection station 20 of the amount of change in the position and posture of the UAV 10 from the time when the UAV 10 was used.

Upon receiving this notification, the route deriving unit 24 of the projection station 20 calculates the current position of the UAV 10 based on this notification. Then, the route deriving unit 24 derives the route from that position again. However, the route derivation unit 24 may newly derive a route so as not to pass through the inspection area where the UAV 10 has already passed. Also, the new route does not have to pass through all unpassed inspection areas. Further, at this time, it is preferable that the route deriving unit 24 selects, for example, the route that minimizes the power consumption. For example, as described above, the route derivation unit 24 may determine a route that minimizes the change in the velocity vector of the UAV 10.

According to the present embodiment, even if the UAV 10 moves due to a disturbance such as a gust, the route deriving unit 24 redefines the route from that position. Therefore, the increase in the power consumption of the UAV 10 can be suppressed, and the operational efficiency of the UAV 10 can be improved.

Embodiment 6.
In the sixth embodiment of the present invention, the projection station 20 generates (corrects) an information image according to the reflectance of the projection surface (hereinafter referred to as “projection target surface”) of an object such as a building. Project the generated information image.

In the following description, the reflectance means each reflection amount of light at each wavelength peculiar to an object. Therefore, even if the reflectance of a certain wavelength is known for the same object, the reflectance at another wavelength is not always the same. The reflectance in a certain wavelength region indicates an integral value of the reflectance in the wavelength region. Further, in the following description, the luminance means the energy of light per unit area at each wavelength. Further, in the following description, the spectrum of light indicates the luminance distribution at each wavelength in a certain wavelength range.

The reflectance is calculated based on, for example, an image captured by the projection target surface captured by the camera 11 mounted on the UAV 10. Specifically, for example, the control unit 13 of the UAV 10 predicts the spectrum of the light illuminating the projection target surface from the captured image of the projection target surface captured by the camera 11, and determines the wavelength of each wavelength from the prediction result. Calculate the reflectance. That is, the control unit 13 calculates the reflectance based on the color information of the captured image of the projection target surface. Further, the control unit 13 may calculate the reflectance of each wavelength on the assumption that sunlight is reflected. Since various methods are known for estimating the spectrum of illumination light, the method is omitted.

In addition, for example, the reflectance may be calculated by projecting a predetermined reference image (hereinafter referred to as “reference image”) onto the projection target surface by the projection station 20. For example, when an image having a white frame shape is projected onto a projection target surface as a reference image, the projection target surface changes its spectrum according to the reflectance of each wavelength and outputs it as reflected light. The camera 11 of the UAV 10 takes an image of the surface of the building including the reflected light (white frame shape) and generates a camera image. Then, the control unit 13 calculates the reflectance by recognizing the spectrum of the frame-shaped portion in the camera image (what color the image reflecting the white light of the frame shape is reflected in the camera). do. That is, the control unit 13 calculates the reflectance based on the color information of the reference image projected on the projection target surface and the color information of the reference captured image. The color of the reference image is not limited to white, and may be light having a constant spectral shape in a certain wavelength range. As described above, when the reference image is projected, it is not necessary to estimate the spectrum of the light illuminating the projection target surface, so that more accurate reflectance is required.

In the above description, the case where the control unit 13 of the UAV 10 calculates the reflectance is described as an example, but the present invention is not limited to the control unit 13, but the projector control unit 22 of the projection station 20, the information image generation unit 25, and others. The device may calculate the reflectance. These configurations for calculating the reflectance function as a reflectance calculation unit. Further, in the above description, a case where the camera 11 mounted on the UAV 10 captures a reference image projected on the projection target surface or the projection target surface has been described as an example. However, the camera that captures these images is not limited to the camera 11, and may be a camera mounted on the projection station 20 or another camera. Also in the following description, a case where the camera 11 mounted on the UAV 10 captures a reference image or the like and the control unit 13 of the UAV 10 calculates the reflectance will be described as an example.

The control unit 13 notifies the projector control unit 22 of the projection station 20 of the calculated reflectance. When the projector control unit 22 of the projection station 20 receives the reflectance information of each wavelength, the projector control unit 22 corrects the spectrum of the information image generated by the information image generation unit 25 according to the reflectance.

When the reflectance is uniform, the projected image is reproduced with the same color. On the other hand, when the reflectance is different for each wavelength, for example, a frame-shaped image projected by white light is recognized as a different color in the camera image. For example, when the surface of a building is painted green, the reflectance is close to 100% in the wavelength range of, for example, 550 nm ± 30 nm, and the other wavelengths are close to 0%. That is, for example, even if an information image with white light of 400 nm to 650 nm is projected, only the green light component of 550 nm ± 30 nm is observed by the camera, and the light of other wavelength components is not observed. In this case, the information drawn at a wavelength other than 550 nm ± 30 nm is lost.

Therefore, when the projector control unit 22 receives the reflectance, the projector control unit 22 corrects the spectrum of the information image according to the wavelength so that the wavelength falls within the range of, for example, 530 nm to 570 nm. As a result, the camera 11 can capture the information image projected on the surface of the building without excess or deficiency. At the same time, the projector control unit 22 may correct the brightness of the information image in addition to the color. The projector control unit 22 of the projection station 20 corrects the brightness of the information image based on the reflectance. For example, when the reflectance of the building is low, the projector control unit 22 corrects so as to increase the brightness of the information image.

By correcting the spectrum of the information image based on the reflectance in this way, it is possible to prevent a part or all of the information image projected on the surface of the building from being unrecognizable by the control unit 13 of the UAV 10 and lacking information. .. The information image generation unit 25 may newly generate an information image of the spectrum corresponding to the reflectance received from the control unit 13. Such correction or generation of an information image by the information image generation unit 25 may be used to generate an image according to the reflectance.

Further, the information image may be composed of only a specific wavelength. That is, the projector 21 may be, for example, a projector (for example, a laser projector) having a light source that irradiates only light having a predetermined wavelength for each of the red, green, and blue colors. The projector control unit 22 may correct the color of the information image by switching the light source of the projector 21 according to the reflectance.

Further, the camera 11 of the UAV 10 may be provided with a filter (light transmitting portion) that is a filter that transmits only light having a predetermined wavelength and that can switch the wavelength of the transmitted light. For example, the wavelength of the light transmitted by the filter may be determined according to the individual light sources of the projector 21. For example, the filter of the camera 11 transmits only the light having the wavelength corresponding to the red light source of the projector 21, the state transmitting only the light having the wavelength corresponding to the green light source of the projector 21, and the blue color of the projector 21. It is a filter that can be switched to any of the states that transmit only the light of the wavelength corresponding to the light source of. For example, when the control unit 13 notifies the projection station 20 of the information of red color, the control unit 13 switches the filter of the camera 11 to a state in which only the light having the wavelength corresponding to the red light source of the projector 21 is transmitted. Similarly, when the control unit 13 notifies the projection station 20 of the information of green, the control unit 13 switches the filter of the camera 11 to a state in which only the light having the wavelength corresponding to the green light source of the projector 21 is transmitted. Similarly, when the control unit 13 notifies the projection station 20 of the information of blue, the control unit 13 switches the filter of the camera 11 to a state in which only the light having the wavelength corresponding to the blue light source of the projector 21 is transmitted. The mode of the filter included in the camera 11 is not particularly limited. For example, in a general color camera, an R / G / B filter having a Bayer arrangement is arranged in each pixel. Only the light of the corresponding light source may be used by this filter. That is, if only a green light source is used, only green pixels may be used. Similarly, if a light source of another color is used, only the pixels corresponding to the light source may be used. Further, a combination of a monochrome camera and an externally attached interchangeable color filter may be used. Further, the camera 11 may not be provided with the above filter.

In particular, when the ambient light is strong with respect to the brightness of the projector, such as in direct sunlight, a light source with a narrow wavelength width such as a laser is used for the projector 21 of the projection station 20, and a narrow band filter that transmits only light of this wavelength is used in the UAV10. It is even better to attach it to. More precisely, in the light spectrum of the image captured by the camera 11 of the UAV 10, both the light projected by the projector 21 and the ambient light (such as sunlight) are reflected according to the reflectance of the surface of the building. It is a light. Further, the luminance acquired by the camera 11 is the integrated luminance. Therefore, for example, when the integrated luminance of the ambient light in a certain wavelength width is 100 and the integrated luminance of the light projected by the projector 21 is 10, the reflected light also has a component of the ambient light 10 times larger. That is, even if the camera 11 captures the information image at the same time as the ambient light, the integrated luminance difference from the region other than the information image is small, and the control unit 13 may not be able to extract the information image. To prevent this, the wavelength range used by the narrowband light source and filter is limited, and only light of a specific wavelength is used. That is, if the integrated range of the light captured by the camera 11 is narrowed, the integrated brightness of the projector 21 remains unchanged at 10, but the integrated brightness of the peripheral light decreases from 100 to, for example, 10, so that the brightness difference of the reflected light is the same. The information image can be detected.

Further, when the reflection from the projector light is weaker than the reflection from the ambient light, the projection station 20 may compare the images with and without the projector light and extract the reflection component from the projector light. That is, the UAV 10 and the projection station 20 are synchronized by a timer or a timing signal so that the control unit 13 of the UAV 10 can determine the timing at which the projector 21 of the projection station 20 switches the presence or absence of projection. According to this timing, the UAV 10 captures an image when the projector light is included and when the projector light is not included. Since both images contain peripheral light components, if the control unit 13 generates a difference image between the former and the latter, it is possible to extract only the reflected component from the projector light. That is, the difference between the captured image of the projection surface when the projection station 20 is projecting an image on the projection surface of the object and the captured image of the projection surface when the image is not projected on the projection surface of the object. Based on, an image corresponding to the reflectance of the projection surface is generated.

Further, when the projection station 20 is provided with a camera, the camera of the projection station 20 captures an image of the surface of the building in advance, and the route deriving unit 24 has a weak peripheral light component on the surface of the building based on the image. A portion (a region that is not exposed to direct sunlight and has a shadow) may be extracted. Then, the path deriving unit 24 may determine the projection point of the information image in the region where the peripheral light component is weak.

If it is necessary to project an information image on the boundary area between the strong and weak peripheral light components or the area where the pattern is drawn on the projection surface, the intensity distribution of the peripheral light and the reflectance of the pattern in that area. The information image generation unit 25 may generate an information image in consideration of the above. That is, for example, the brightness level of the background portion of the information image in the portion exposed to sunlight is lowered, and the brightness level is increased in the background portion of the information image in the portion not exposed to sunlight and shaded. It is possible to prevent the information image from being erroneously recognized by the control unit 13 of the UAV 10 depending on the presence or absence. Also, regarding the pattern, the spectrum of the light of each part is changed so that it looks uniform when it becomes reflected light, and an image that avoids erroneous recognition is projected.

According to the present embodiment, the camera 11 can generate a camera image in which an information image is captured even when the surface of the building reflects only light having a specific wavelength. In addition, it is possible to prevent the information image projected on the surface of the building from being buried in the ambient light. Therefore, it is possible to easily detect the information image in the camera image.

Embodiment 7.
In the seventh embodiment of the present invention, the operation when the projection surface of the information image and the target surface for imaging for the state inspection are different will be described.

FIG. 16 is a schematic diagram showing an example of a situation in which the projection surface of the information image and the target surface for imaging for state inspection are different. It is assumed that the region to be imaged for the state inspection is the region 82. However, the region 82 is horizontal. Further, in the region 82, there is a region that becomes a blind spot when viewed from the projector 21 due to the structure 39 of the building 30. Further, the information image cannot be projected on the area that becomes a blind spot when viewed from the projector 21. Even if the projector 21 projects an information image on a region of the region 82 that does not become a blind spot when viewed from the projector 21, the UAV 10 may not be able to capture the information image projected on the horizontal region 82.

In such a situation, the projection surface of the information image and the target surface of the image pickup for the state inspection may be different surfaces.

The area 81 is an area including a projection point of the information image. The projector 21 projects an information image (hereinafter, this information image is referred to as an information image Q) on the area 81. When the information image generation unit 25 (see FIG. 10) of the projector 21 generates the information image Q, the control content of the orientation of the camera in the air is also converted into the information image Q in addition to the control content of the orientation of the moving camera. Include it. In this example, the information image generation unit 25 includes in the information image Q a control content instructing the camera to face straight up while in the air.

The UAV 10 proceeds in the direction of the target position P3 based on the information image immediately before the information image Q, and performs steps B1 to B3 (see FIG. 11). Then, if the information image can be recognized in the camera image, the UAV 10 performs the operations after step B4.

In step B6 of the control unit 13 of the UAV 10 (see FIG. 6), the UAV 10 is suspended on the spot according to the information image Q. Further, the control unit 13 directs the camera directly upward while in the air according to the above-mentioned control contents included in the information image Q. The control unit 13 performs an image pickup for a state inspection while staying in the air in this state. At this time, since the camera 11 faces directly upward, the region 82 to be inspected can be imaged and the camera image of the region 82 can be generated.

According to the present embodiment, even if the information image cannot be projected on the target surface for imaging for the state inspection, the information image Q projected on the other surface can be used to reach the vicinity of the target position. .. Then, according to the control content included in the information image Q, the UAV 10 can change the camera direction in the air, image the target surface for imaging for the state inspection, and generate a camera image of that surface.

Embodiment 8.
Due to aging of the building, defects such as cracks and floats may occur on the surface of the building. The unmanned flight device control system of the eighth embodiment of the present invention detects such defects. In addition, the unmanned flight device control system corrects the information image so that the projected information image is not distorted according to the defect.

In the eighth embodiment of the present invention, the projector 21 also projects an image representing a predetermined pattern (hereinafter referred to as a pattern image) on the surface of the building. The control unit 13 of the UAV 10 detects the state of defects on the surface of the building according to the shape of the pattern shown in the camera image. The projection station 20 corrects the information image according to the state of the defect.

Hereinafter, a more specific description will be given. The projector 21 projects a pattern image onto the surface of a building. The pattern image may be, for example, an image representing a mesh pattern. In this embodiment, for the sake of simplicity, a horizontal straight line will be described as an example as a predetermined pattern. FIG. 17 is a schematic diagram showing an example of a pattern image representing a horizontal straight line. Although FIG. 17 shows a pattern image representing one straight line, a plurality of straight lines may be represented in the pattern image.

FIG. 18 is a schematic diagram showing a state in which a pattern image is projected on the surface of a building where a step is generated due to aged deterioration, and the camera 11 of the UAV 10 is capturing the projected portion. FIG. 19 is a schematic side view of the state shown in FIG. In the example shown in FIGS. 18 and 19, the surface 91 of the building is in a state of being more floating than the surface 92, and a step is generated.

The camera 11 captures the surface 91 and the surface 92 on which the pattern image (horizontal straight line) is projected, and generates a camera image. FIG. 20 is a schematic diagram showing an example of a camera image obtained as a result of capturing a pattern image. The camera image shown in FIG. 20 shows the surfaces 91 and 92 and the line 93. The projector 21 projects a horizontal straight line on the surfaces 91 and 92, but the step of the surfaces 91 and 92 causes the straight line to appear as a curved line 93 in the camera image.

Further, the projection station 20 transmits the angle formed by the projection direction of the pattern image and the wall surface to the UAV 10. The projection station 20 may transmit this angle to the UAV 10 by communication. Alternatively, the projection station 20 may project an information image including this angle information, and the control unit 13 of the UAV 10 may recognize the angle information from the information image in the camera image.

The control unit 13 of the UAV 10 recognizes the shape of the pattern (line 93 in this example) captured in the camera image. The control unit 13 determines the state of the surface of the building based on the shape of the pattern captured in the camera image. In this example, the control unit 13 determines that there is a step between the surface 91 and the surface 92 from the shape of the line 93 captured in the camera image. That is, the control unit 13 detects the size of the step between the surface 91 and the surface 92 and the position where the step is generated. For example, the control unit 13 may store in advance the correspondence between the projection direction of the pattern image and the angle formed by the wall surface and the shape deformation of the pattern in the camera image. Then, the control unit 13 may determine the state of the surface of the building by specifying the state of the surface of the building corresponding to the shape deformation of the pattern in the camera image.

The control unit 13 transmits information on the state of the surface of the building (in this example, the size of the step between the surface 91 and the surface 92 and the position where the step is generated) to the projection station 20.

The projector control unit 22 of the projection station 20 receives the information on the surface condition of the building transmitted from the control unit 13. Then, the projector control unit 22 receives information on the appearance of the building and the three-dimensional shape of the building stored in the data storage unit 23, and information on the surface state of the building received from the UAV 10 (control unit 13). The information image is corrected so that the projected information image is not distorted based on.

According to the present embodiment, the control unit 13 can detect a defect in a building caused by deterioration over time and the magnitude of the defect. Further, since the control unit 13 transmits information on the state of the surface of the building to the projection station 20, the projection station 20 (projector control unit 22) corrects the information image so that the projected information image is not distorted. can do. Therefore, the projection station 20 can project the information image on the surface of the building in good condition, and the guidance accuracy of the UAV 10 can be further improved.

Each embodiment of the present invention can be applied not only to the diagnosis of infrastructure deterioration but also to the case of controlling the UAV for patrol monitoring of a building.

Next, the outline of the present invention will be described. FIG. 21 is a schematic diagram showing an outline of the unmanned flight device control system of the present invention. The unmanned flight device control system 100 includes an image projection unit 101 and a control unit 103.

The image projection unit 101 (for example, the projection station 20) is an image (for example, an information image) that controls the operation of an unmanned flight device (for example, UAV10) equipped with an image pickup device (for example, a camera 11), and is an object. An image generated according to the reflectance of the projection surface of (for example, a building to be inspected for a state) is projected onto the object.

The control unit 103 (for example, the control unit 13) controls the operation of the unmanned flight device based on the image captured by the image pickup device.

With such a configuration, when an image for controlling the operation of the unmanned flight device is projected on the object and the unmanned flight device is guided to the vicinity of the target position near the object, the environment for projecting the image is taken into consideration. Can control unmanned flight equipment.

The present invention is suitably applied to an unmanned flight device control system for controlling an unmanned flight device.

10 UAV (Unmanned Aerial Vehicle)
11 Camera 13 Control unit 15 Camera angle control unit 16 Motor driver 17a to 17d Motor 18 Communication unit 20 Projection station 21 Projector 22 Projector control unit 23 Data storage unit 24 Route derivation unit 25 Information image generation unit 27 Communication unit

Claims (6)

An image for controlling the operation of an unmanned flight device equipped with an image pickup device, which is information indicating control contents regarding the flight or speed of the unmanned flight device, information indicating control contents regarding the orientation of the image pickup device, and the above-mentioned. An image projection unit that projects an image containing information indicating the control content regarding the attitude of an unmanned flight device onto an object, and an image projection unit.
An unmanned flight device control system including a control unit that controls the operation of the unmanned flight device based on the image captured by the image pickup device. The unmanned flight device control system according to claim 1, wherein the information representing the control content regarding the flight of the unmanned flight device includes an instruction for the unmanned flight device to take an image while in the air. The unmanned flight device control system according to claim 1 or 2, wherein the image projected on the object includes state information of the unmanned flight device. The unmanned flight device control system according to any one of claims 1 to 3, wherein the image projected on the object is generated according to the reflectance of the projection surface of the object. The unmanned flight device control system according to any one of claims 1 to 4, wherein the image pickup device is controlled for capturing an image used for infrastructure deterioration diagnosis and patrol monitoring of a building. An image for controlling the operation of an unmanned flight device equipped with an image pickup device, which is information indicating control contents regarding the flight or speed of the unmanned flight device, information indicating control contents regarding the orientation of the image pickup device, and the above-mentioned. An image containing information showing the control content regarding the attitude of the unmanned flight device is projected onto the object.
An unmanned flight device control method for controlling the operation of the unmanned flight device based on the image captured by the image pickup device.

Images