JP2017173966A - Pilotless flight device control system, pilotless flight device controlling method, and pilotless flight device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To continue to control a pilotless flight device if images taken by the pilotless flight device have gone out of sight, when the pilotless flight device is controlled based on images taken by the pilotless flight device to be guided to around a target position near a target.SOLUTION: An image projector 101 projects an image for controlling the operation of a pilotless flight device equipped with an imaging device, to a target. A controller 103 controls the operation of the pilotless flight device based on images taken by the imaging device, and controls the operation of the pilotless flight device based on the amount of change in the flying condition if the pilotless flight device flies in the changed condition and thus has become incapable of taking images.SELECTED DRAWING: Figure 21

Description

  The present invention relates to an unmanned flight apparatus control system for controlling an unmanned flight apparatus, an unmanned flight apparatus control method, and an unmanned flight apparatus applicable to the unmanned flight apparatus control system.

  A state inspection (infrastructure deterioration diagnosis) of a building using an unmanned flying object is performed. For example, the deterioration state of the building can be estimated from the surface state of the building imaged by a camera mounted on the unmanned flying object. Hereinafter, the unmanned flying vehicle is referred to as UAV (Unmanned Aerial Vehicle). UAV can also be referred to as an unmanned flight device.

  In order to use the UAV for infrastructure deterioration diagnosis, it is necessary to control the UAV in the vicinity of the building to be diagnosed.

  In a general technique, the control unit of the UAV grasps the position of the UAV using a GPS (Global Positioning System) signal in order to guide the UAV to the target position. Patent Document 1 describes an aerial photographic apparatus including a camera having a GPS device having a GPS antenna and an imaging unit.

JP 2014-62789 A

  When infrastructure deterioration diagnosis is performed using UAVs whose positions are determined by GPS signals, the following problems occur. In GPS in which a single antenna is attached to a moving body such as a UAV, the real-time positioning accuracy includes an error of several meters. As a result, when a UAV approaches a building to be subjected to infrastructure degradation diagnosis, the UAV GPS measurement position error is large, and therefore the distance between the building and the UAV may be too far or too close. Also, depending on the positional relationship between the UAV and the GPS satellite, the UAV may pass through a place where the GPS signal strength is weak. When the UAV passes through a place where the intensity of the GPS signal is weak, the UAV cannot accurately grasp the position of the UAV, and the accuracy of guidance to the target position is deteriorated.

  In order to solve this problem with a simple configuration, an image for controlling the operation of the UAV is projected on an object such as a building, and the projected image is captured by a camera mounted on the UAV. It is conceivable to guide the UAV near the target position near the object.

  In the UAV guidance control based on such an image, when the UAV is subjected to disturbance such as wind, the posture and the position may change, and the image projected on the object may be lost. As a result, UAV control based on images cannot be performed.

  Such a problem can occur not only in UAV control in infrastructure deterioration diagnosis, but also in UAV control for guiding to a target position near an object, such as patrol monitoring of a building.

  Therefore, the present invention controls an unmanned aerial vehicle based on an image captured by the unmanned aerial vehicle and guides the unmanned aerial vehicle near the target position near the object. An object is to provide an unmanned flying device control system, an unmanned flying device control method, and an unmanned flying device applicable to the unmanned flying device control system that can control the unmanned flying device even if it is lost. And

  An unmanned aerial vehicle control system according to the present invention includes an image projecting unit that projects an image for controlling an operation of an unmanned aerial vehicle equipped with an imaging device onto an object, and an unmanned aerial vehicle based on an image captured by the imaging device. The control unit controls the operation of the unmanned aerial vehicle based on the amount of change in the flight state when an image cannot be captured due to a change in the flight state of the unmanned aerial vehicle.

  The unmanned flying device control method according to the present invention projects an image for controlling the operation of the unmanned flying device equipped with the imaging device onto an object, and the operation of the unmanned flying device based on the image captured by the imaging device. And the operation of the unmanned aerial vehicle is controlled based on the amount of change in the flight state.

  An unmanned aerial vehicle according to the present invention is an image for controlling the operation of an unmanned aerial vehicle, and is an unmanned flight based on an imaging device that captures an image projected on an object and an image captured by the imaging device. A control unit that controls the operation of the device, and the control unit controls the operation of the unmanned flight device based on the amount of change in the flight state when an image cannot be captured due to a change in the flight state of the unmanned flight device. .

  According to the present invention, when controlling the unmanned aerial vehicle based on the image captured by the unmanned aerial vehicle to guide the unmanned aerial vehicle near the target position near the object, the image captured by the unmanned aerial vehicle is displayed. Even if it is lost, the unmanned flight device can be controlled.

It is a schematic diagram which shows the building which becomes the unmanned flight apparatus control system and inspection object of this invention. It is a schematic diagram which shows the situation where UAV grasps | ascertains the content of the movement control to the next target position when UAV image | photographs in the location shifted | deviated from the target position. It is explanatory drawing which shows the example of an information image. It is explanatory drawing which shows the meaning of the symbol in the information image illustrated in FIG. It is explanatory drawing which shows the example of the display mode of a bar. It is a block diagram which shows the structural example of UAV. It is a schematic diagram which shows the example of the visual field of a camera. It is a schematic diagram which shows the example of the camera image produced | generated by imaging the visual field shown in FIG.7 (b). It is a schematic diagram which shows the example of the state which overlap | superposed the contour in the camera image which contour information shows on the camera image shown in FIG. It is a typical block diagram which shows the structural example of a projection station. It is a flowchart which shows the example of the process progress of the 1st Embodiment of this invention. It is a schematic diagram which shows the example of the camera image which cannot recognize an information image. It is a schematic diagram which shows the example of the camera image which the information image and the shadow of UAV reflected and were taken. It is a schematic diagram which shows the example of the camera image used as the state which the information image and the shadow of UAV did not overlap by moving UAV. It is a schematic diagram which shows the example of the path | route of UAV. It is a schematic diagram which shows the example of the condition from which the projection surface of an information image differs from the object surface of the imaging for a state test | inspection. It is a schematic diagram which shows the example of the pattern image showing a horizontal straight line. It is a schematic diagram which shows the state which projects a pattern image on the surface of the building in which the level | step difference produced, and the camera is imaging the projection location. It is a typical side view of the state shown in FIG. It is a schematic diagram which shows the example of the camera image obtained as a result of imaging a pattern image. It is a schematic diagram which shows the outline | summary of the unmanned flight apparatus control system of this invention. It is a schematic diagram which shows the building which becomes an unmanned flight apparatus control system and inspection object including a projection station provided with the tracking imaging device of this invention. It is a schematic diagram which shows the example in which a tracking imaging device calculates | requires the variation | change_quantity of the position and attitude | position of UAV.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

Embodiment 1. FIG.
FIG. 1 is a schematic view showing an unmanned flying device control system of the present invention and a building to be inspected. The unmanned aerial vehicle control system of the present invention includes an unmanned aerial vehicle (hereinafter referred to as UAV) 10 and a projection station 20. As mentioned above, the UAV can also be referred to as an unmanned aerial vehicle.

  The UAV 10 is equipped with a camera. The camera mounted on the UAV 10 images the surface of the building 30 to be inspected and generates an image of the surface of the building. This image can be used for building condition inspection (infrastructure deterioration diagnosis).

  The projection station 20 includes a projector 21, and projects an image including information indicating the control content of the UAV 10 (hereinafter referred to as an information image) from the projector 21 onto the surface of the building 30. The projection station 20 is mounted on the vehicle 29 and can move or stop at the same place.

  The projection station 20 stores in advance information on the appearance and three-dimensional shape of the building 30 to be inspected. Further, the projection station 20 recognizes the positional relationship between the projection station 20 and the building 30. The positional relationship between the projection station 20 and the building 30 is measured using various surveying devices (not shown), and not only the position in the three-dimensional space but also the orientation is measured with high accuracy. The method for obtaining the three-dimensional shape of the building 30 may be a method using drawing at the time of design or 3D-CAD (3Dimension Computer-Aided Design) data, or 3D-LiDAR (3Dimension Light Detection and Ranging). Alternatively, a method of measuring the actual product in advance may be used. The information on the three-dimensional shape of the building 30 is not limited to information on the entire building 30 but may be information on a portion of the building 30 to be observed using the present invention. Further, since there are a number of known methods such as triangulation using a total station or the like, a method for accurately knowing the positional relationship between the building 30 and the projection station 20 will not be described. The positional relationship between the projection station 20 and the building 30 is basically measured in advance. However, when there is a possibility that the building 30 may be deformed during work due to the influence of wind or the like, or when the projection station 20 is in an unstable position on the ship or the like, the positional relationship is measured during the work according to the present invention. May be.

  The projection station 20 determines a plurality of information image projection locations on the surface of the building 30 based on the positional relationship and stored information, and derives a path 35 of the UAV 10. Further, the projection station 20 generates an information image to be projected onto each projection location. In addition, the projection station 20 determines a position (hereinafter referred to as a target position) for the UAV 10 to capture an information image, a posture (tilt or orientation) at the target position, and a camera direction. The posture and the camera direction are the posture and the camera direction when an information image is captured. The projection station 20 generates, for example, information on movement control up to the next target position, an information image for instructing the posture during movement and the direction of the camera, based on the positional relationship between the target positions. The information image also includes information for instructing the UAV 10 to perform imaging for state inspection. The projection station 20 sequentially projects information images on the surface of the building 30. As will be described later, in practice, the UAV 10 generally captures an information image at a position deviated from the target position, and the posture at which the information image is captured and the orientation of the camera are also determined. Not necessary.

  The UAV 10 images the surface of the building 30 on which the information image is projected with a camera. Hereinafter, an image obtained by imaging by the UAV 10 camera is referred to as a camera image. The UAV 10 operates in accordance with the control content indicated by the information image in the camera image (information image shown in the camera image). For example, the UAV 10 performs imaging for the state inspection based on the information image in the camera image or moves in the direction of the next target position.

  The UAV 10 operates based on the information image in the camera image, and images the surface of the building 30 for the state inspection while moving in the vicinity of the path 35.

  The UAV 10 does not always capture an information image at a target position on the path 35. Since the UAV 10 is affected by various effects such as wind, it is generally difficult for the UAV 10 to reach the same location as a predetermined target position. Therefore, the UAV 10 images the surface of the building 30 when it reaches the vicinity of the target position. The UAV 10 calculates the difference between the position and the target position based on the camera image obtained as a result. Then, the UAV 10 derives control contents for movement from the current position to the next target position based on the movement control contents indicated by the information image in the camera image and the calculated deviation.

  FIG. 2 is a schematic diagram illustrating a situation in which the UAV 10 grasps the content of movement control up to the next target position when the UAV 10 performs shooting at a location deviated from the target position. In the example illustrated in FIG. 2, the areas A1 and A2 on the surface of the building 30 are the projected portions of the information image. Further, it is assumed that the target position P1 is determined as a position for capturing the information image projected on the area A1. Similarly, it is assumed that a target position P2 is determined as a position for capturing an information image projected on the area A2. Further, the posture of the UAV 10 and the direction of the camera at the target positions P1 and P2 are also determined. As described above, since the UAV 10 is affected by various effects such as wind, it is difficult for the UAV 10 to capture an information image projected on the area A1 at a position that completely coincides with the target position P1. Therefore, the UAV 10 captures the information image projected on the area A1 at the position P1 ′ near the target position P1. At this time, the posture of the UAV 10 and the orientation of the camera are generally not consistent with the posture and the orientation of the camera at the predetermined target position P1.

  The UAV 10 assumes that the information image projected on the area A1 is captured at the target position P1 with the determined posture and the camera orientation, and the contour of the information image in the camera image and actually the position P1 ′. The deviation M (see FIG. 2) between the target position P1 and the position P1 ′ is calculated based on the contour of the information image in the camera image obtained by imaging the information image. Further, the information image projected on the area A1 shows the content of movement control for moving from the target position P1 to the target position P2. The UAV 10 derives the control content for moving from the position P1 ′ to the target position P2 based on the movement control content and the calculated deviation M, and toward the next target position P2 based on the control content. Move. The UAV 10 also captures an information image in the vicinity of the target position P2 even when capturing an information image projected on the area A2, and similarly derives control details for moving to the next target position.

  Note that the information on the contour of the information image in the camera image when it is assumed that the information image is taken at a predetermined target position with a predetermined posture and camera orientation is given to the UAV 10 in advance.

  Also, the UAV 10 generally cannot capture an information image at the front position of the information image due to various influences such as wind. Therefore, in general, information images are distorted in camera images. The UAV 10 recognizes the control content indicated by the information image shown in the camera image after performing processing for removing the distortion of the information image.

  FIG. 3 is an explanatory diagram illustrating an example of an information image projected by the projection station 20. FIG. 4 is an explanatory diagram showing the meaning of symbols in the information image illustrated in FIG.

  The information image 40 includes, for example, a movement control symbol 41, a camera orientation control symbol 43, and an attitude control symbol 44.

  The movement control symbol 41 is a symbol indicating control content related to movement of the UAV 10. With the movement control symbol 41, it is possible to give a direction of travel direction, a stray direction, and a speed direction. The movement control symbol 41 illustrated in FIG. 3 has an arrow shape, and the direction indicated by the arrow is the instructing traveling method. In the example shown in FIG. 3, a downward movement is instructed. The traveling direction indicated by the movement control symbol 41 is not limited to the downward direction.

  In addition, the movement control symbol 41 can instruct the UAV 10 to stay behind by the added bar 42. The bar 42 indicates a stray indication. When the bar 42 is provided at the end opposite to the arrowhead of the movement control symbol 41, it means that the bar 42 moves and moves in the direction of the arrow (downward in this example). In addition, by displaying a bar 42 at the end of the movement control symbol 41 on the arrow head side, it is possible to instruct to stay after moving. In the following description, in order to simplify the explanation, the UAV 10 stays in the air after recognizing the control content indicated by the information image 40, performs imaging for the state inspection within the time of the air hover, and is instructed thereafter. A description will be given on the assumption that it moves in the direction. In this case, the bar 42 is displayed at the end opposite to the arrow head. Further, the bar 42 will be described as having a meaning of instructing imaging for a state inspection in the air.

  Further, the movement control symbol 41 indicates the movement speed when the UAV 10 moves depending on how the color of the color inside the movement control symbol 41 (arrow-shaped symbol) changes. In FIGS. 3 and 4, the change in shade of the color of the arrow-shaped symbol is represented by a change in pattern for convenience. The shade of the color in the actual arrow shape may change gradually, for example.

  The direction of the bar 42 is determined to be perpendicular to the direction of the arrow, for example. However, the method of determining the orientation of the bar 42 is not limited to this example, and may be always horizontal, for example. FIG. 5 is an explanatory diagram illustrating an example of a display mode of the bar 42. In FIG. 5, the display of color shading inside the movement control symbol 41 is omitted. FIG. 5A shows an example in which the direction of the bar 42 is determined to be perpendicular to the direction of the arrow. FIG. 5B shows an example in which the direction of the bar 42 is always set to be horizontal. As illustrated in FIG. 5, when the arrow points to the right, in the example illustrated in FIG. 5A, the bar 42 is defined as a vertically oriented bar. On the other hand, in the example shown in FIG. 5B, the bar 42 is horizontal and is shown inside the arrow.

  The camera orientation control symbol 43 is a symbol representing the control content related to the orientation of the camera mounted on the UAV 10. The camera elevation control symbol 43 controls the elevation angle and horizontal angle of the camera. FIG. 3 illustrates a case where the camera orientation control symbol 43 is a U-shaped rotation arrow. In this example, the U-shaped rotation arrow means that the elevation angle of the camera is upward and the horizontal angle is fixed. The correspondence between the control content related to the camera direction and the shape of the camera direction control symbol 43 may be determined in advance.

  In addition, when the direction of the camera that is moving to the next target position and the direction of the camera at the time of shooting corresponding to the next target position are separately defined, the projection station 20 is in each direction. A corresponding camera orientation control symbol may be included in the information image 40.

  The attitude control symbol 44 is a symbol representing the control content related to the attitude of the UAV 10. FIG. 3 illustrates the case where the attitude control symbol 44 is a semicircular figure. In this example, the semicircle figure means that the posture of the UAV 10 is controlled horizontally. The relationship between the control content related to the posture of the UAV 10 and the shape of the posture control symbol 44 may be determined in advance.

  If the posture during movement to the next target position and the posture at the time of shooting corresponding to the next target position are separately defined, the projection station 20 controls the posture control corresponding to each posture. A symbol may be included in the information image 40.

  Although not shown in FIG. 3, information in the camera image when it is assumed that an information image is captured at a certain target position with a posture and a camera orientation corresponding to the target position. Information representing the contour of the image may be included in the information image 40. Hereinafter, information representing the contour of the information image in the camera image when it is assumed that the information image is captured at a certain target position with the posture and the orientation of the camera determined in correspondence with the target position is referred to as contour information.

  The movement control symbol 41, the bar 42, the camera orientation control symbol 43, and the attitude control symbol 44 are identification information for identifying the control contents of the UAV 10, and the information image 40 is an image including such identification information. Can do. FIG. 3 shows an example in which identification information (movement control symbol 41, bar 42, camera orientation control symbol 43, and attitude control symbol 44) for identifying control contents is represented by a graphic symbol. As illustrated in FIG. 3, the identification information for identifying the control content is represented by a graphic symbol or the like, so that the inspection operator can easily recognize the information even from a distance.

  Further, the identification information for identifying the control content of the UAV 10 may be a symbol such as a QR code (registered trademark) or a barcode.

  The identification information for identifying the control content of the UAV 10 may be a symbol such as a Chinese character, a number, or a hieroglyph.

  Further, the identification information for identifying the control content of the UAV 10 may be a sentence such as “1 meter flight after staying for 30 seconds”, for example.

  Further, from the viewpoint of visibility of the inspection worker, the projection station 20 may project the information image 40 by blinking the information in the information image 40 (identification information for identifying the control content), for example. Good. By projecting the information image 40 in this manner, the contents of the information image 40 can be made conspicuous, and the inspection operator can easily recognize the contents.

  Further, the projection station 20 may include other information in the information image 40 in addition to the identification information for identifying the control content. For example, the projection station 20 may include state information (for example, the current altitude of the UAV 10 or a step number indicating the progress of work) in the information image 40.

  The above-described contour information is used when obtaining the deviation M between the target position P1 and the actual position P1 '. Information necessary for obtaining the deviation M between the target position P1 and the actual position P1 'may not be the above-described contour information. The projection station 20 may include information necessary for obtaining the deviation M between the target position P1 and the actual position P1 'in the information image 40, and the UAV 10 may store the information in advance. For example, the projection station 20 projects the information image 40 including special icons at the four corners instead of the contour information, and the UAV 10 may calculate the deviation M according to the positional relationship of these icons captured in the camera image. Good. Thus, by using a shape that is easy to recognize without using the contour, the possibility that the UAV 10 misrecognizes the shift M can be reduced.

  In addition, the projection station 20 may include information necessary for obtaining the deviation M between the target position P1 and the actual position P1 'in the information image 40 in a fixed manner. For example, it is assumed that the inspection area number (not shown) indicating the inspection area is included in the information image 40. The projection station 20 may fixedly determine the first half of the inspection area number using, for example, the alphabet, and arbitrarily determine the second half. The UAV 10 may obtain the deviation M by deformation of the fixed portion shown in the camera image. In this way, a part of the information image 40 is fixedly determined, and the deviation M is obtained based on the image of the part in the camera image. You can avoid getting in the way.

  Next, a configuration example of the UAV 10 will be described. FIG. 6 is a block diagram illustrating a configuration example of the UAV 10. In this example, the case where the UAV 10 has four propellers (not shown in FIG. 6) and has four motors corresponding to the four propellers on a one-to-one basis will be described as an example. However, the number of propellers and motors is not limited to the above example.

  The UAV 10 includes a camera 11, a control unit 13, a camera angle control unit 15, a motor driver 16, four motors 17 a to 17 d, and a communication unit 18.

  The camera 11 captures an image of the surface of a building 30 (see FIG. 1) to be inspected for infrastructure deterioration diagnosis, and generates a camera image in which the surface of the building is copied. When an information image is projected within the field of view of the camera, the information image is also reflected in the camera image. The camera 11 inputs the generated camera image to the control unit 13. The orientation of the camera 11 can be controlled.

  Here, the field of view of the camera on the surface of the building 30 (see FIG. 1) and the camera image when the field of view is photographed will be described.

  FIG. 7 is a schematic diagram showing an example of the field of view of the camera on the surface of the building 30 (see FIG. 1). FIG. 7A is a schematic top view showing an example of a situation in which the camera 11 of the UAV 10 (not shown in FIG. 7) images the surface 30a of the building 30. FIG. Further, it is assumed that the information image 40 is projected on the surface 30a of the building 30. Due to the influence of wind or the like, it is difficult for the camera 11 to image the surface 30a completely facing the surface 30a of the building 30. Therefore, as shown to Fig.7 (a), it is common for the camera 11 to image the surface 30a of the building 30 from the diagonal direction. FIG.7 (b) is a schematic diagram which shows the visual field of the camera 11 on the surface 30a of the building 30 in the state shown to Fig.7 (a). The field of view 49 of the camera 11 on the surface 30a is rectangular. In the example illustrated in FIG. 7B, the case where the field of view 49 is a trapezoid is illustrated by the camera 11 being in a state of imaging the surface 30 a of the building 30 from an oblique direction. The field of view 49 is not necessarily trapezoidal. However, the field of view 49 is rectangular when the camera 11 faces the surface 30a of the building 30. As shown in FIG. 7A, the camera 11 faces the surface 30a of the building 30 from an oblique direction. The field of view 49 is never rectangular. The projection station 20 projects the information image 40 so that the information image 40 is rectangular on the surface 30a. Therefore, the information image 40 projected on the surface 30a is rectangular. The vertices of the visual field 49 are A, B, C, and D. Moreover, let the vertex of the information image 40 projected on the surface 30a of the building 30 be a, b, c, d. In FIG. 7B, illustration of various symbols in the information image 40 is omitted. This is the same in FIG.

  The camera 11 images the visual field 49 and generates a camera image. FIG. 8 is a schematic diagram illustrating an example of a camera image generated by imaging the visual field 49 illustrated in FIG. The camera image 50 is a rectangle. Each vertex of the camera image 50 corresponds to each vertex of the field of view 49 shown in FIG. The vertex of the camera image 50 corresponding to the vertex A of the visual field 49 is also represented by the symbol A. The same applies to the vertices of other camera images. Each vertex of the information image 40 (see FIG. 8) shown in the camera image 50 corresponds to each vertex of the information image 40 (see FIG. 7B) projected on the surface 30a of the building 30. ing. The vertex of the information image 40 (see FIG. 8) in the camera image 50 corresponding to the vertex a of the information image 40 shown in FIG. The same applies to the other vertices of the information image 40 in the camera image 50. Hereinafter, the contour of the information image 40 in the camera image 50 is referred to as a contour abcd.

  The camera 11 captures a non-rectangular field of view 49 and generates a rectangular camera image 50. Therefore, the rectangular information image 40 projected in the visual field 49 of the camera 11 is distorted in the camera image 50 as shown in FIG.

  When the camera image 50 is input from the camera 11, the control unit 13 detects the outline abcd of the information image 40 in the camera image 50.

  Further, contour information is given to the control unit 13 in advance. As already described, the contour information is the contour of the information image in the camera image when it is assumed that the information image is captured at a certain target position with the posture and the camera orientation corresponding to the target position. It is information to represent. FIG. 9 is a schematic diagram illustrating an example of a state in which the contour in the camera image indicated by the contour information is superimposed on the camera image illustrated in FIG. 8. The control unit 13 calculates a deviation M between the current position and the target position of the UAV 10 based on the contour 51 in the camera image 50 indicated by the contour information and the contour abcd of the information image 40 in the camera image 50. The deviation M includes not only the position but also posture information. Since it is the same thereafter, the posture information is omitted unless otherwise specified.

  When the information image cannot be recognized in the camera image, the control unit 13 moves the UAV 10 by controlling the motors 17 a to 17 d via the motor driver 16.

  In addition, when the information image can be recognized in the camera image, the control unit 13 calculates a deviation M between the current position of the UAV 10 and the target position. Further, the control unit 13 performs a process of removing the distortion of the information image 40 in the camera image 50 (in other words, a process of making the information image 40 in the camera image 50 a rectangle). The control content shown is grasped, and the UAV 10 is controlled according to the control content. For example, the control unit 13 performs imaging for the state inspection of the surface of the building 30 according to the control content indicated by the information image 40. Further, the control unit 13 calculates the calculated deviation M (the deviation between the current position of the UAV 10 and the target position), the content of the movement control to the next target position indicated by the information image 40, and the movement from the current position to the next target position. The UAV 10 is moved by deriving the control contents for, and controlling the motors 17a to 17d via the motor driver 16 according to the control contents. Further, the control unit 13 controls the posture of the UAV 10 according to the control content indicated by the information image 40, and controls the orientation of the camera 11 via the camera angle control unit 15.

  The camera angle control unit 15 controls the orientation of the camera 11 according to the control unit 13. Specifically, the camera angle control unit 15 controls the elevation angle and horizontal angle of the camera 11.

  The motor driver 16 drives the motors 17 a to 17 d according to the control unit 13. The motor driver 16 can drive each motor 17a-17d separately. Therefore, the control unit 13 can realize control for moving the UAV 10 back and forth, control for moving the UAV 10 left and right, control for moving the UAV 10 up and down, pitch angle control, roll angle control, and yaw angle control.

  The communication unit 18 is a communication interface used for communication with the projection station 20.

  The control unit 13 and the camera angle control unit 15 may be realized by a CPU of a computer that operates according to a program, for example. In this case, the CPU may read the program and operate as the control unit 13 and the camera angle control unit 15 according to the program. Further, the control unit 13 and the camera angle control unit 15 may be realized by separate hardware.

  Next, a configuration example of the projection station 20 will be described. FIG. 10 is a schematic block diagram illustrating a configuration example of the projection station 20. The projection station 20 includes a projector 21, a projector control unit 22, a data storage unit 23, a route derivation unit 24, an information image generation unit 25, and a communication unit 27.

  The data storage unit 23 is a storage device that stores information on the appearance of the building and the three-dimensional shape of the building.

  The route deriving unit 24 recognizes the positional relationship between the projection station 20 and the building. At this time, the route deriving unit 24 may be provided with the current position of the projection station 20 from the outside, for example. The route deriving unit 24 determines a plurality of information image projection locations on the surface of the building 30 based on the positional relationship and information on the appearance of the building and the information on the three-dimensional shape of the building. Further, the route deriving unit 24 derives the route of the UAV 10.

  For example, the route deriving unit 24 determines a plurality of inspection areas by dividing the surface of the building into a lattice shape based on information stored in the data storage unit 23, and determines a predetermined location ( For example, the center of the examination area) may be determined as the projection location of the information image. However, the method for determining the projection location of the information image is not limited to the above example, and the projection location may be determined by another method. In addition, the route deriving unit 24 may determine the route of the UAV 10 so that the information image projected on each projection location can be captured. At this time, it is preferable that the route deriving unit 24 determines a route that minimizes the power consumption of the UAV 10. However, the route deriving unit 24 may determine the route based on criteria other than power consumption.

  At this time, the route deriving unit 24 determines a target position for the UAV 10 to capture the information image projected on the projection location in association with the projection location. Furthermore, the route deriving unit 24 determines the posture of the UAV 10 and the direction of the camera at the target position. The posture and the camera direction are the posture and the camera direction when an information image is captured. However, it is difficult for the UAV 10 that actually moves due to the influence of wind or the like to capture an information image at a predetermined target position with a predetermined posture and camera orientation. Therefore, UAV10 should just image an information image in the vicinity of a target position. Further, the posture of the UAV 10 and the orientation of the camera at that time do not have to coincide with the orientation and the orientation of the camera determined together with the target position.

The information image generation unit 25 generates an information image for each projection location. Information image generating unit 25 includes a target position corresponding to the projection portion (the #i.) (A #P _i.), Target position corresponding to the next projection point (. To # i + 1) (# An information image to be projected onto the projection location #i may be generated based on the positional relationship with P _{i + 1} . In addition, the information image generation unit 25 generates contour information for each target position.

  The projector 21 projects an information image on the surface of a building. The direction of the projector 21 is variable.

  The projector control unit 22 controls the orientation of the projector 21. When the projector control unit 22 changes the orientation of the projector 21, the information image can be projected onto a predetermined projection location.

  In addition, when the projector control unit 22 points the projector 21 toward the projection location of the information image, the projector control unit 22 determines the position of the projector 21 on the surface of the building based on the orientation of the projector 21, the appearance of the building, and the three-dimensional shape of the building. The information image is corrected so that the information image is not distorted. In other words, the projector control unit 22 corrects the information image so that a rectangular information image is projected on the surface of the building. The projector 21 projects the information image corrected by the projector control unit 22. In the following description, the description of the information image correction operation by the projector control unit 22 may be omitted.

  The communication unit 27 is a communication interface used for communication with the UAV 10.

  The route deriving unit 24, the information image generating unit 25, and the projector control unit 22 may be realized by a CPU of a computer that operates according to a program, for example. In this case, the CPU may read the program and operate as the route deriving unit 24, the information image generating unit 25, and the projector control unit 22 according to the program. Further, the route deriving unit 24, the information image generating unit 25, and the projector control unit 22 may be realized by separate hardware.

  Further, the control unit 13 (see FIG. 6) may be provided in the projection station 20 or another information processing apparatus instead of the UAV 10. The operation of the control unit 13 is performed by the projection station 20 or another information processing apparatus, and the movement control of the UAV 10, the posture control, the control of the orientation of the camera 11 and the like are performed by communication with the outside (for example, the projection station 20). It may be realized.

  Next, the process progress of the first embodiment of the present invention will be described. FIG. 11 is a flowchart illustrating an example of processing progress according to the first embodiment of this invention. The flowchart shown in FIG. 11 is an exemplification, and the processing progress of the present invention is not limited to the flowchart shown in FIG.

  Further, the route deriving unit 24 of the projection station 20 determines in advance a plurality of projection positions of the information image on the surface of the building, derives the route of the UAV 10, and each target position for the UAV 10 to capture the information image, and Assume that the posture of the UAV 10 and the orientation of the camera are determined at each target position. In the first embodiment, for the sake of simplicity of explanation, it is assumed that there is no change in the route during the inspection. In addition, it is assumed that there is no influence on the UAV 10 due to a large disturbance that occurs instantaneously (for example, the UAV 10 greatly deviates from the path due to an instantaneous gust of wind). However, a deviation (see FIG. 2) from the target position due to a normal wind or the like may occur.

  The projection station 20 projects the information image from the projector 21 onto the projected portion of the image on the surface of the building (step T1). In step T1 for the first time, the projection station 20 projects the first information image onto the projected portion of the image.

  Further, the control unit 13 moves the UAV 10 toward the target position for capturing the target image projected in step T1 (step B1). When the first information image is projected, the inspection operator remotely operates the UAV 10. The inspection worker moves the UAV 10 so that the first information image falls within the field of view of the camera 11. The control unit 13 moves the UAV 10 so that the information image projected on the surface of the building falls within the field of view of the camera 11 according to the remote operation by the inspection worker.

  Further, while moving, the UAV 10 captures the surface of the building with the camera 11 to obtain a camera image (step B2). The camera 11 inputs the generated camera image to the control unit 13.

  When the camera image is input, the control unit 13 searches for an information image in the camera image (step B3). Successful search means that the information image can be recognized in the camera image. Further, the failure of the search means that the information image cannot be recognized in the camera image. When the information image can be recognized in the camera image (Yes in Step B3), the control unit 13 calculates the deviation between the current position of the UAV 10 and the target position (Step B4). When the information image is not recognized in the camera image in step B3, such as when the information image is not in the camera image or when the information image is interrupted, the UAV 10 continues to move.

  FIG. 12 is a schematic diagram illustrating an example of a camera image in which an information image cannot be recognized.

  FIG. 12A shows a case where no information image is shown in the camera image 50. In this case, since there is no information image in the camera image, the control unit 13 cannot recognize the information image in the camera image.

  FIG. 12B shows a case where only a part of the information image 40 is shown in the camera image 50. In this case, since the information image 40 is interrupted, the control unit 13 cannot recognize the information image in the camera image.

  FIG. 12C shows the case where the information image 40 is shown in the camera image 50, but the area of the information image 40 in the camera image 50 is very small (smaller than the threshold value). In this case, since the area of the information image 40 is too small, the control unit 13 cannot recognize the information image in the camera image.

  When the information image cannot be recognized in the camera image (No in Step B3), the UAV 10 repeats the operations after Step B1. In addition, when moving to the target position direction corresponding to the first projection location, when the control unit 13 cannot determine which direction to move based on the most recently obtained camera image, Subsequently, the UAV 10 is moved according to a remote operation by the inspection worker. For example, when no information image is captured in the most recently obtained camera image, the control unit 13 continues to move the UAV 10 according to a remote operation by the inspection operator. In addition, when it is possible to determine in which direction the control unit 13 should move based on the most recently obtained camera image, the control unit 13 starts an autonomous operation regardless of the remote operation by the inspection worker. For example, as illustrated in FIG. 12B, when a part of the information image 40 is captured in the camera image 50 obtained most recently, the control unit 13 places the entire information image 40 within the field of view of the camera 11. In order to enter, it can be determined in which direction the UAV 10 should be moved. In this case, the control unit 13 autonomously moves the UAV 10 so that the entire information image 40 is within the field of view of the camera 11.

  When the information image cannot be recognized in the camera image (No in Step B3), Steps B1 to B3 are repeated until the information image can be recognized in the camera image.

  When the information image can be recognized in the camera image, as illustrated in FIG. 9, the information image 40 is captured in an appropriate size in the camera image 50 (in other words, the information image 40 is captured in an area larger than the threshold value). ), When the outline abcd of the information image 40 can be detected. In this case, the control unit 13 determines the current UAV 10 based on the contour 51 (see FIG. 9) in the camera image 50 indicated by the contour information and the contour abcd (see FIG. 9) of the information image 40 in the camera image 50. The deviation between the position and the target position is calculated (step B4).

  If the information image can be recognized in the camera image, it can be said that the UAV 10 exists in the vicinity of the target position. Actually, the UAV 10 repeats the processing of steps B1 to B3 while moving. Therefore, at the moment when all the information images are included in the camera image, the information image is completely recognized in step B3, and the process proceeds to step B4. In other words, even when the UAV 10 is at a position relatively far from the target position, the control unit 13 executes Step B4 at the moment when the information image enters toward the end of the camera image. In order to prevent this, although omitted in FIG. 11, “stopping the transition to step B4 until the information image enters a certain range of the camera image” or “information in a certain range of the camera image” Determine the moving direction for entering the image and repeat step B1 and subsequent steps "or" Stop the transition to step B4 until the information image is recognized in the camera image for a certain period of time (or a certain number of frames) " There may be processing. Alternatively, since the deviation M can be calculated in step B4, it is better to calculate the amount of movement corresponding to this deviation M and repeat step B1 and subsequent steps. In addition to this, it is better to repeat step B1 while controlling the orientation of the camera so that the deviation M has a prescribed shape. This is because the UAV 10 can be moved to a more accurate position or the UAV 10 can be moved to a more accurate posture by the movement of the UAV 10 based on the deviation M and the control of the camera 11.

  After calculating the deviation between the current position of the UAV 10 and the target position, the control unit 13 removes the distortion of the information image 40 in the camera image 50 (step B5). As already described, the information image 40 projected on the surface of the building is rectangular, and the field of view 49 of the camera is not rectangular (see FIG. 7B). On the other hand, the camera image 50 obtained by imaging the visual field 49 is rectangular. Therefore, the information image 40 shown in the camera image 50 is not rectangular but distorted (see FIG. 8). In step B5, the control unit 13 removes this distortion.

  Subsequently, the control unit 13 recognizes a command (control content) indicated by the information image 40 after the distortion is removed, and executes the command (step B6).

  In this example, a case where the control unit 13 recognizes a command (control content) indicated by the information image 40 illustrated in FIG. 3 and executes the command will be described as an example.

  Specifically, the control unit 13 stays on the spot for a predetermined time, and recognizes that imaging is performed for checking the state of the surface of the building in the air. Then, the control unit 13 stays on the spot for a predetermined time in accordance with the command, and images the surface of the building with the camera 11 in the stay. At this time, the information image 40 may be projected in the field of view of the camera 11.

  In addition, the control unit 13 recognizes that the camera 11 is lowered after a predetermined time, that the camera 11 is turned upward while the horizontal angle of the camera 11 is fixed, and that the posture of the UAV 10 is controlled horizontally. . At this time, based on the deviation between the current position of the UAV 10 calculated in step B4 and the target position and the movement control content indicating that the control unit 13 moves down, the control unit 13 determines the control content for movement to the next target position. The UAV 10 is moved by controlling each of the motors 17a to 17d via the motor driver 16 according to the control contents. That is, the control unit 13 corrects the control content related to the movement indicated by the information image 40 by the deviation calculated in step B4, and moves the UAV 10 based on the corrected control content.

  Further, after recognizing the command indicated by the information image 40, the control unit 13 notifies the projection station 20 that the command recognition has been completed (step B7).

  When the control unit 13 starts moving the UAV 10 based on the control content after the correction, the control unit 13 repeats the operations after Step B1. In this step B1, unlike the case of moving toward the target position corresponding to the first projection location, the control unit 13 autonomously moves the UAV 10 without depending on the remote operation by the inspection worker.

  When the projector control unit 22 receives notification from the UAV 10 that the command recognition is completed, the projector control unit 22 stops the projection of the information image that has been projected to the projector 21. In addition, when the information image generation unit 25 receives a notification that the instruction recognition is completed from the UAV 10, the information image generation unit 25 specifies the projection position of the next information image (here, the projection position of the second information image), and the projection position A second information image to be projected onto is generated (step T2). In this example, the information image generation unit 25 is a command for instructing imaging for state inspection near the second target position, or a command for moving the UAV 10 from the second target position to the third target position. An information image including such as is generated.

  The UAV 10 images the surface of the building with the camera 11 while moving (step B2), and the control unit 13 executes step B3. Until the second information image is projected, since the second information image is not captured in the camera image, the UAV 10 repeats steps B1 to B3.

  When the information image generation unit 25 generates an information image to be projected onto the next projection location (here, the second information image) at step T2, the projector control unit 22 projects the information image onto the projection location. The projector 21 is controlled. As a result, the projector 21 projects the second information image on the projection location of the second information image (step T1).

  When the UAV 10 moves to a position where the newly projected information image falls within the field of view of the camera 11 and the information image can be recognized in the camera image (Yes in step B3), the UAV 10 can repeat the processing after step B4. That's fine. When the projection station 20 receives the notification in step B7, the projection station 20 may stop projecting the information image that has been projected so far and execute steps T2 and T1.

  By repeating the above processing, the UAV 10 can image the surface of the building for the state inspection while moving in the vicinity of the determined route.

  In addition, when the projection station 20 projects the last information image and the control unit 13 executes the command shown in the last information image, the unmanned flight apparatus control system does not perform the notification in step B7. Processing may be terminated. In this case, the projection station 20 includes a predetermined symbol indicating that it is the last information image in the last information image so that the control unit 13 can recognize that it is the last information image. That's fine.

  According to the present embodiment, the control unit 13 repeats steps B1 to B3 until the information image can be recognized in the camera image. If the information image can be recognized in the camera image, the control unit 13 calculates the deviation M, The distortion of the information image in the camera image is removed, the command indicated by the information image is recognized, and the operation is performed according to the command. By repeating this operation, the control unit 13 can image the surface of the building for the state inspection while moving in the vicinity of the determined route. Therefore, the UAV 10 can be guided near the target position near the building with a simple configuration.

  In the present invention, since the UAV 10 is guided by the information image, the UAV 10 does not need to receive a GPS signal. Therefore, in the present invention, it is not necessary to provide a large number of facilities in place of GPS satellites in order to obtain position information by GPS signals at any location. In this respect as well, the present invention can simplify the system configuration.

  In the present invention, the UAV 10 is guided by the information image. Since the UAV 10 approaches the information image and obtains a shift, the information image and the position and posture of the UAV 10 can be accurately obtained. In other words, the UAV 10 that is a measuring device is close to the information image that is the measurement target, so that there are fewer error factors. As a result, the position measurement is based on precise position measurement and measurement results compared to GPS and other methods. Guidance is possible.

  Further, the UAV 10 derives the deviation between the current position of the UAV 10 and the target position based on the information image. By using this shift as a difference, the UAV 10 can also accurately move to the target position.

  Further, the UAV 10 derives the movement control content up to the next target position based on the shift between the current position of the UAV 10 and the target position and the movement control content indicated by the information image. As a result, the UAV 10 can move to the vicinity of the next target position. That is, the movement of the UAV 10 can be controlled with good accuracy. Further, as described above, in the present invention, the UAV 10 does not need to receive a GPS signal. Therefore, the UAV 10 can be accurately controlled even in a place where it is difficult to receive GPS signals.

  The projection station 20 projects an information image on a building with a certain size, and the UAV 10 monitors this information image. Further, since the positional relationship between the information image and the UAV 10 can be calculated inside the UAV 10, the UAV 10 can acquire the positional relationship continuously in substantially real time. Therefore, for example, even if the position of the UAV 10 is greatly changed or the posture is broken due to a disturbance such as a gust, the UAV 10 can correct the position and the posture before contacting the building.

  Next, a modification of the first embodiment will be described.

  In the above embodiment, the case where the control unit 13 removes the distortion of the information image 40 in the camera image 50 in step B5 has been described. If the control unit 13 can recognize the command indicated by the information image 40 without removing the distortion of the information image 40, step B5 may be omitted.

  Moreover, in said embodiment, when imaging the surface of a building for the state test | inspection of a building in step B6, the surface of a building remains in the state where the information image 40 was projected in the visual field of the camera 11. The case of imaging was shown. When the surface of the building is imaged for the state inspection of the building, the control unit 13 may instruct the projection station 20 to stop projecting the information image. In this case, the information image generation unit 25 of the projection station 20 generates an information image instructing imaging for state inspection and an information image instructing movement to the next target position for one projection location. do it. The information image generation unit 25 includes a command for notifying the projection station 20 of the stop of projection of the information image in the information image instructing imaging for state inspection, and in other words after the notification (in other words, the projection station 20 A command to image the building for the state inspection (after stopping the projection in response to the notification) and a command to notify the projection station 20 of the completion of the imaging after the completion of the imaging are included. The projection station 20 projects the information image on the projection location first. When the control unit 13 recognizes those commands and notifies the projection station 20 of the projection stop, the projection station 20 stops the projection of the information image. In this state, the UAV 10 images the surface of the building. When the UAV 10 notifies the projection station 20 of the completion of imaging, the information image instructing movement to the next target position is projected onto the same projection location. At this time, the UAV 10 may move somewhat due to the influence of wind or the like while imaging for state inspection is performed. Therefore, it is preferable that the UAV 10 performs the operation after Step B2 again after the information image instructing the movement to the next target position is projected. This is because the deviation between the current position of the UAV 10 and the target position is calculated again. In step B <b> 6, the control unit 13 may correct the control content related to movement indicated by the information image based on the deviation.

  If the surface of a building is imaged in a state where an information image is projected, a portion other than the information image may be relatively dark in the camera image. As described above, when the UAV 10 performs imaging for state inspection, such a situation can be prevented by the projection station 20 stopping the projection of the information image.

  Further, when the projection station 20 stops projecting the information image when the UAV 10 performs imaging for the state inspection, the projector control unit 22 causes the projector 21 to stop projecting the information image, and then causes the projector 21 to Light or light of an appropriate color may be irradiated. At this time, the projector 21 irradiates light on the surface of the building to be inspected according to the projector control unit 22. As a result, the UAV 10 images the surface of the building for the state inspection of the building in a state where the surface of the building is irradiated with light.

  When the projector 21 irradiates light on the surface of the building, the shortage of light when the camera 11 of the UAV 10 images the surface of the building can be compensated. For example, when the camera 11 takes an image of a place where a building is buried or when an unmanned flight control system is operated at night, a camera image can be obtained in which the surface of the building is clearly shown due to insufficient light quantity. Absent. When the camera 11 of the UAV 10 images the surface of the building for the state inspection of the building, the projector 21 can compensate for the shortage of the light amount as described above by irradiating the surface of the building with light. 11 can generate a camera image in which the surface of the building is clearly shown.

  The projection station 20 may include a projector for irradiating light separately from the projector 21.

  Further, the contour shape of the information image projected from the light projecting station 20 may not always be the same. The contour information needs to be known before step B3 for recognizing the information image. Therefore, it may be included in the previous information image, the projection station 20 communicates to the UAV 10 by communication, or the contour information for each information image may be stored in the UAV 10 in advance. The control unit 13 may apply contour information corresponding to the information image every time the deviation M is calculated. Further, by deforming the shape of the contour of the information image, the information image can be prevented from getting in the way during the state inspection of the building.

  In each embodiment described below, the description of the same matters as those already described is omitted.

Embodiment 2. FIG.
In the second embodiment of the present invention, when the UAV 10 receives a large disturbance, the flight state such as the posture and position of the UAV 10 changes, and the control unit 13 loses sight of the information image shown in the camera image ( The operation when the information image cannot be captured will be described. Even if the information image is captured in the camera image before the UAV 10 receives the disturbance, and the control unit 13 recognizes the information image, the UAV 10 is projected onto the building because the posture and position of the UAV 10 are changed due to the disturbance. If the information image that has been displayed goes out of the field of view of the camera 11, the information image will not appear in the camera image. That is, the control unit 13 loses sight of the information image in the camera image.

  In this case, the control unit 13 changes the posture of the UAV 10 in each direction and performs imaging with the camera 11. Alternatively, the control unit 13 changes the orientation of the camera 11 and the range of the visual field 49 via the camera angle control unit 15 and performs imaging with the camera 11. At this time, the control unit 13 may change the orientation of the UAV 10 and also change the orientation of the camera 11 and the range of the visual field 49. As a result, if the control unit 13 acquires a camera image in which the information image is captured, the control unit 13 may perform the operations after Step B4 (see FIG. 11). Note that when the range of the field of view 49 (imaging range) is increased, the possibility that the information image 40 enters the inside of the camera image 50 increases, but the resolution decreases and the information image 40 in the camera image 50 becomes smaller. Therefore, after discovering the information image 40 by expanding the range of the visual field 49, the UAV 10 controls the posture and position of the camera 11 and the UAV 10 so that the information image 40 is near the center of the visual field 49, and the visual field 49 is restored to the original. Perform processing to return to the range. When the range of the visual field 49 changes, the calculation parameter in step B4 is also changed accordingly. Note that the process of returning the visual field 49 to the original range is not limited to the original range, and may be a process of reducing the information image 40 to an imaging range that provides a recognizable resolution.

  At this time, the control unit 13 determines the position and orientation of the UAV 10 based on the camera images of a plurality of frames from the position and orientation when the control unit 13 recognizes the information image in the camera image. You may detect whether it changed. In other words, images of a plurality of frames before and after the time when the information image is lost are arranged in the order of photographing, and the amount of change in pixels considered to be identical is obtained as a vector. In other words, the optical flow is obtained. Since the change vector of the pixel and the movement vector of the UAV 10 substantially coincide with each other, this change vector may be regarded as a change in the position and orientation of the UAV 10. Note that the movement vector of the UAV 10 is not constant and changes with time. However, since this change is considered to be gentle compared to the frame rate, not only the change vector before losing sight of the information image but also the change vector after losing sight. Even if is used, there is not much difference in the results. Also, instead of a vector including the amount of movement, only the direction may be detected and this may be used as the amount of change. Furthermore, it is better to obtain each of the position change and the attitude change of the UAV 10 from the difference between the change vector of the object near the camera and the change vector of the far position. Various methods, such as PTAM (Parallel Tracking and Mapping), are known for estimating the amount of change in the position and orientation of a camera using the fact that the movement of pixels in the screen varies depending on the depth when the camera is moving. It is omitted here.

  Alternatively, the UAV 10 may include an inertial navigation device. Then, the inertial navigation apparatus may detect how much the position and orientation of the UAV 10 have changed from the position and orientation when the control unit 13 recognizes the information image in the camera image. The inertial navigation device is a device in which one or a plurality of accelerometers, gyroscopes, and geomagnetic azimuth detectors are combined with a calculation unit. When the inertial navigation device moves, acceleration, angular velocity, and direction are detected, and these are integrated by a calculation unit to determine the amount of movement and the amount of rotation. That is, when the UAV 10 receives a disturbance, the inertial navigation device can directly determine the amount of change with the inertial navigation value. As in the case of the camera layer, it may be obtained from the measurement result before the time when the information image is lost or may be obtained later.

Alternatively, the projection station 20 may include a tracking imaging device 26 that detects the position of the UAV 10. FIG. 22 is a schematic diagram showing an unmanned flying device control system including a projection station 20 including a tracking imaging device 26 and a building to be inspected. The tracking imaging device 26 includes, for example, a camera that is installed substantially coaxially with the image projection unit 101, images the UAV 10 as illustrated in FIG. 22, and obtains its position. Then, the tracking imaging device 26 detects how much the position and orientation of the UAV 10 have changed from the position and orientation when the control unit 13 recognized the information image in the camera image. FIG. 23 is a schematic diagram illustrating an example in which the tracking imaging device 26 obtains the change amount of the position and posture of the UAV 10. The tracking imaging device 26 can obtain the positions of the UAV 10 and the information image 40 viewed from the tracking imaging device 26 when the UAV 10 recognizes the information image 40 (indicated by a dotted line in FIG. 23). Further, when the UAV moves due to a disturbance, as shown in FIG. 23, the relative position in the camera image 51 of the tracking imaging device 26 also changes. Therefore, the tracking imaging device 26 obtains a change in the relative value of the UAV 10. The tracking imaging device 26 may be a device including a 3D camera, such as a stereo camera or a TOF (Time of Flight) camera, in addition to a device including a camera installed substantially coaxially with the image projecting unit 101 described above. In the case of a 3D camera, the coordinates of the UAV 10 in the three-dimensional space are always recognized, and the coordinate change of the UAV 10 before and after the moment when the information image is lost is obtained.

  Then, the control unit 13 sets the position and orientation of the UAV 10 and the orientation and field of view of the camera 11 in order to direct the field of view of the camera 11 in the information image direction based on the amount of change in the position and orientation of the UAV 10. You may determine whether to change. The control unit 13 may change the position and orientation of the UAV 10, the orientation of the camera 11, and the visual field range according to the determination result. In this case, the control unit 13 can quickly point the field of view of the camera 11 in the direction of the information image, and can capture the projected information image early. When the range of the visual field 49 is expanded based on the amount of change, the UAV 10 moves so that the information image 50 is near the center of the visual field 49 and returns the visual field 49 to the original range, as in the previous example. I do.

  Further, the control unit 13 may notify the projection station 20 of the change amount of the position and orientation of the UAV 10 since the control unit 13 recognized the information image in the camera image.

  When notified of the change amount of the position and orientation of the UAV 10, the information image generation unit 25 (see FIG. 10) of the projection station 20 calculates the visual field range of the camera 11 based on the change amount, A new information image to be projected is generated. This new information image is an information image including correction. For example, referring to FIG. 2, it is assumed that the UAV 10 first recognizes the information image A1 at the position P1, and moves to the position P1 'due to disturbance. The new information image A1 ′ (not shown) is projected at a position that can be recognized even at the position P1 ′. This new information image A1 ′ shows the contents of movement control to the position P1 before the disturbance movement. Alternatively, the information image A1 may be used. That is, the new information image A1 'may be content of movement control from the position P1' to the next position P2. In any case, the projector control unit 22 directs the projector 21 in the direction of the calculated visual field range of the camera, and causes the projector 21 to project the information image. Then, the camera 11 captures an image (step B2), and then the UAV 10 may perform the operations after step B3. As a result, the UAV 10 can continue processing according to the information image.

  In particular, when it is considered that the UAV 10 is continuously subjected to random disturbance, the information processing generation unit 25 of the projection station 20 has the movement prediction range and the field of view 49 of the UAV 10 based on the change amount. Assuming that the range is wide, the information image 40 may be generated for the entire range where the visual field 49 may exist. That is, when the position of the UAV 10 is continuously unstable due to turbulence, the position of the visual field 49 changes every moment. Therefore, even if the information image 40 of the normal size is projected at one place, the control unit 13 displays the information image 40. It may not be captured continuously. In order to avoid this, the control unit 13 projects the position correction information image 40 larger than the normal size, and the UAV 10 is directed toward the center of the information image based on the result of the camera 11 capturing a part of the information image. It is preferable to control or project a plurality of information images 40 at the same time so that at least one information image 40 can be captured even when the UAV 10 moves randomly. Note that the position and range of the new information image 40 may be changed simultaneously.

  According to the second embodiment, even if the UAV 10 causes the visual field of the camera 11 to face in a direction different from the information image due to disturbance, the UAV 10 captures the information image again and returns to normal control. can do. That is, according to the second embodiment, in the case where the unmanned flight apparatus is controlled based on the image captured by the unmanned flight apparatus and the unmanned flight apparatus is guided near the target position near the object, the unmanned flight apparatus captures the image. Even if the user loses sight of the image, the unmanned flying device can be controlled.

Embodiment 3. FIG.
In the third embodiment of the present invention, the unmanned aerial vehicle control system avoids a situation in which the information image and the shadow of the UAV 10 overlap and appear in the camera image.

  The route deriving unit 24 of the projection station 20 determines a projection location that does not overlap with the shadow of the UAV 10 as the projection location of the information image.

  For example, the route deriving unit 24 defines a plurality of inspection areas by dividing the surface of the building into a lattice shape based on the information stored in the data storage unit 23, and determines a predetermined location ( For example, suppose that the center of the examination area is defined as the projection location of the information image. The path deriving unit 24 also determines the target position of the UAV 10, the posture of the UAV 10 at the target position, and the direction of the camera for each projection location. Then, the route deriving unit 24 determines, for each projection location, whether the projection location and the shadow of the UAV 10 overlap each other. This determination can be made based on the positional relationship among the projection station 20, the target position, and the projection location. The route deriving unit 24 changes the projection location that overlaps the shadow of the UAV 10 to a position that does not overlap with the shadow of the UAV 10 within the examination area, and in accordance with the change, the posture of the UAV 10 corresponding to the projection location and the orientation of the camera are also changed. change. At this time, the route deriving unit 24 may also change the target position of the UAV 10. Even if the position of the projection location is changed within the examination area, if it overlaps with the shadow of the UAV 10, it is determined that no projection location is determined for the examination area. Then, the route deriving unit 24 determines a route for capturing an information image projected on each projection location, targeting a projection location that does not overlap with the shadow of the UAV 10.

  By determining the projection location of the information image in this way, it is easy to prevent the information image projected on the projection location and the shadow of the UAV 10 from appearing in the camera image.

  In addition, as described above, even when the route deriving unit 24 determines a projection location that does not overlap with the shadow of the UAV 10, the UAV 10 is affected by wind or the like, so that the information image is displayed at the target position corresponding to the projection location. It is generally rare to take an image. As a result, even when the projection location is determined as described above, the information image projected on the projection location may overlap with the shadow of the UAV 10.

  In this case, the control unit 13 may move the UAV 10. FIG. 13 is a schematic diagram illustrating an example of a camera image in which an information image and a shadow of the UAV 10 are overlapped. When the control unit 13 determines that a part of the outline of the information image 40 is interrupted due to the presence of the black pixel group, a part of the outer peripheral part of the information image 40 and the shadow 55 are generated as illustrated in FIG. Judged as overlapping. When determining in this way, the control unit 13 may move in an arbitrary direction and execute the operations after Step B1 (see FIG. 11), for example. At this time, the control unit 13 may determine the direction in which the UAV 10 is moved based on the portion of the contour of the information image 40 that is interrupted by the shadow 55, and may move the UAV 10 in that direction. FIG. 14 is a schematic diagram illustrating an example of a camera image in which the information image 40 and the shadow 55 of the UAV 10 do not overlap with each other as the UAV 10 moves. In the camera image illustrated in FIG. 14, the position of the entire contour of the information image 40 can be detected. After moving the UAV 10, the control unit 13 can calculate the deviation of the current position from the target position using the camera image illustrated in FIG. 14. Then, the control part 13 should just perform operation | movement after step B5 (refer FIG. 11).

  As described above, when the overlapping of the information image 40 and the shadow of the UAV 10 is detected, the control unit 13 can avoid the overlapping of the information image 40 and the shadow of the UAV 10 by moving the UAV 10. This operation is applicable even when the route deriving unit 24 has not determined a statistical location so that the shadow of the UAV 10 and the projected location do not overlap.

  In the above description, the operation for avoiding the overlapping of the information image 40 and the shadow of the UAV 10 due to the movement of the UAV 10 has been described. The control unit 13 may avoid the overlapping of the information image 40 and the shadow of the UAV 10 by causing the projection station 20 to change the projection position of the information image. For example, it is assumed that the control unit 13 determines that a part of the outer peripheral portion of the information image 40 and the shadow 55 overlap each other as in the above case. In this case, the control unit 13 uses the projection station 20 to project the information image in order to avoid the overlap between the information image 40 and the shadow 55 based on the portion of the contour of the information image 40 that is interrupted by the shadow 55. It is determined how much to move in which direction, and the projection station 20 is notified of the movement direction and amount of projection. This notification can be said to be an instruction to change the projection location. Upon receiving this notification, the information image generating unit 25 of the projection station 20 changes the content of the information image 40 in accordance with the notified movement direction and movement amount. This is because the content of movement control to the next projection location changes as the projection location of the information image changes. The route deriving unit 24 also determines a target position when the projection location is moved according to the notified movement direction and movement amount, and the posture and camera direction at the target position. The projector control unit 22 directs the projector 21 in the direction of the projection location according to the notified movement direction and movement amount, and causes the projector 21 to project the information image 40. As a result, the UAV 10 obtains, for example, a camera image illustrated in FIG. Then, the control part 13 should just perform operation | movement after step B4 (refer FIG. 11).

  In this way, even when the UAV 10 causes the projection station 20 to change the projection location of the information image 40, it is possible to avoid the overlap between the information image 40 and the shadow of the UAV 10. This operation is applicable even when the route deriving unit 24 has not determined a statistical location so that the shadow of the UAV 10 and the projected location do not overlap.

Embodiment 4 FIG.
In the fourth embodiment of the present invention, the UAV 10 changes the route in the middle of the route when the power consumption is large due to the influence of the weather or the like and imaging for the state inspection cannot be performed in all the inspection areas.

  As described in the first embodiment, the route deriving unit 24 determines the route of the UAV 10 so that, for example, an information image projected on each projection location can be captured. That is, the route of the UAV 10 is determined so as to pass near each projection location.

  At this time, it is preferable that the route deriving unit 24 determines a route that minimizes the power consumption of the UAV 10. Here, the UAV 10 consumes a lot of power when the flight speed vector changes. Therefore, the route deriving unit 24 may determine, for example, a route that minimizes the change in the velocity vector of the UAV 10.

  It is assumed that the route deriving unit 24 determines the route illustrated in FIG. In FIG. 15, each inspection area on the surface of a building and the path 61 of the UAV 10 are illustrated. The inspection area Ra is the first inspection area, and the inspection area Rz is the last inspection area.

  The route deriving unit 24 determines a position on the route 61 where the UAV 10 can be switched to a shortcut route (hereinafter referred to as a switchable position). There may be one or more switchable positions. A description will be given by taking the switchable position 71 shown in FIG. 15 as an example.

  After determining the route 61, the route deriving unit 24 determines a plurality of route candidates that are shortcuts from the switchable position 71 to the end point 79. The route deriving unit 24 determines route candidates so as not to pass through the inspection area that has already passed. Further, since the route candidate is a shortcut from the switchable position 71 to the end point 79, it is not necessary to pass through all the inspection areas that have not passed. Further, the number of examination areas through which a plurality of route candidates determined with respect to the switchable position 71 passes may be different for each route candidate. In other words, the lengths of the route candidates from the switchable position 71 to the end point 79 may be different from each other. In this example, in order to simplify the description, it is assumed that the movement route for one section from the switchable position 71 to the next examination area is not different from a predetermined route in each route candidate.

  The route deriving unit 24 also determines a plurality of route candidates that are shortcuts to the end point 79 for each switchable position other than the switchable position 71.

  Hereinafter, in order to simplify the description, it is assumed that the target position of the UAV 10 corresponding to the examination area is determined as a switchable position.

  When the control unit 13 of the UAV 10 notifies the projection station 20 that the command recognition has been completed (step B7), the energy consumption after the start of the flight, the remaining battery level at that time, and the weather (here, the wind speed of the cross wind) In this example, the projection station 20 is also notified. In addition, what is necessary is just to mount the sensor for measuring these values in UAV10. However, the control unit 13, for example, if a condition that can ensure a sufficient flight distance (for example, a condition that the remaining battery level is equal to or greater than a threshold) is satisfied, the energy consumption, the remaining battery level, and the weather ( It is not necessary to notify the projection station 20 of the wind speed of the cross wind.

  In addition, the route deriving unit 24 holds in advance a prediction formula for a flightable distance using energy consumption, remaining battery capacity and weather (wind speed of cross wind) as explanatory variables, and the flightable distance of the UAV 10 as an explanatory variable. . This prediction formula may be determined in advance by machine learning such as regression analysis. When the route derivation unit 24 is notified of the energy consumption, the remaining battery level, and the weather from the UAV 10, the route deriving unit 24 calculates the flight distance of the UAV 10 by substituting those values into the explanatory variables of the prediction formula.

  Further, the route deriving unit 24 subtracts the flightable distance calculated by the prediction formula from the length of the route candidate for each route candidate predetermined for the switchable position where the UAV 10 exists. The route deriving unit 24 determines the route candidate having the smallest subtraction result as the optimum shortcut, and thereafter designates the route candidate as the route of the UAV 10.

  The power consumption of the UAV 10 is affected by weather such as wind. For example, when a strong wind is blowing, a thrust force against the strong wind is required, so that power consumption is greater than when there is no wind. As a result, when the UAV 10 continues to move along the route 61 that is initially determined, the remaining battery level may become zero in the middle of the route. According to the present embodiment, since it is possible to switch to a route that is a shortcut to the end point 79 at the switchable position, it is possible to maintain the flight of the UAV 10 to the end point 79 in exchange for reducing the number of inspection sections that pass through. Can do.

  In the above example, the energy consumption, the remaining battery level and the weather are exemplified as the explanatory variables used in the prediction formula of the flightable distance, but other explanatory variables may be used.

Embodiment 5. FIG.
In the fifth embodiment of the present invention, when the position of the UAV 10 changes due to a large disturbance to the UAV 10, the projection station 20 derives the route from the changed position to the end point of the route again. To do.

  When the UAV 10 moves due to disturbance and the control unit 13 of the UAV 10 loses sight of the information image in the camera image, the control unit 13 recognizes the information image in the camera image as in the case of the second embodiment. The change amount of the position and posture of the UAV 10 from the time when it has been informed is notified to the projection station 20.

  Upon receiving this notification, the route deriving unit 24 of the projection station 20 calculates the current position of the UAV 10 based on this notification. Then, the route deriving unit 24 derives the route from the position again. However, the route deriving unit 24 may derive a new route so as not to pass through the examination area that the UAV 10 has already passed. Further, the new route may not pass through all the inspection areas that have not passed. At this time, the route deriving unit 24 preferably selects, for example, a route that minimizes power consumption. For example, as described above, the route deriving unit 24 may determine a route that minimizes the change in the velocity vector of the UAV 10.

  According to this embodiment, even if the UAV 10 moves due to a disturbance such as a gust, the route deriving unit 24 again determines a route from that position. Therefore, an increase in power consumption of the UAV 10 can be suppressed, and the operational efficiency of the UAV 10 can be increased.

Embodiment 6. FIG.
In the sixth embodiment of the present invention, the projector control unit 22 not only corrects the generated information image so as not to be distorted during projection, but also corrects the color and brightness of the information image.

  In the sixth embodiment, the projector 21 also projects an image serving as a predetermined reference (in this embodiment, a white frame-shaped image) on the surface of the building. The projector control unit 22 corrects the color of the information image in accordance with the color of the frame shape portion that appears in the camera image. The projector 21 may always project a white frame-shaped image. Alternatively, the projector 21 projects the white frame-shaped image first when the projection position of the information image changes, and then corrects the information according to the color of the frame-shaped portion that is reflected in the camera image. An image may be projected.

  More specific description will be given below. When the camera 11 of the UAV 10 captures the surface of a building on which a white frame-shaped image is projected and generates a camera image, the control unit 13 determines the color (white frame) of the frame-shaped portion in the camera image. The color of the shape image in the camera). Then, the control unit 13 notifies the projection station 20 of the color information of the frame shape portion in the camera image.

  When the projector control unit 22 of the projection station 20 receives the color information of the frame shape portion in the camera image, the projector control unit 22 corrects the color of the information image generated by the information image generation unit 25 according to the color information.

  The fact that the frame-shaped image projected with white light has some color in the camera image means that the surface of the building reflects only that color. For example, it is assumed that the surface of a building is green and reflects only a wavelength of 550 nm ± 30 nm. In this case, the color of the frame-shaped portion in the camera image is a color having a wavelength within the range of 550 nm ± 30 nm. When the projector control unit 22 receives this information from the UAV 10, the projector control unit 22 corrects the color of the information image so that the wavelength of the color falls within a range of 530 nm to 570 nm, for example.

  As a result, the camera 11 can capture an information image projected on the surface of the building. In the above example, even if an information image having a wavelength of less than 520 nm or a wavelength of more than 580 nm is projected on the reflection surface, the light is not reflected and the information image is not reflected in the camera image. In this embodiment, the projector control unit 22 corrects the color of the information image according to the color of the image projected with white light in the camera image, so that the surface of the building is specified. The camera 11 can generate a camera image in which an information image is reflected even when only light having a wavelength of is reflected.

  In addition to the color, the projector control unit 22 may correct the luminance of the information image. In this case, the control unit 13 recognizes the brightness of the surface of the building on which the information image is projected based on the image of the frame shape portion in the camera image. The control unit 13 notifies the projection station 20 of the brightness.

  When the brightness of the information image projected on the surface of the building is notified, the projector control unit 22 of the projection station 20 corrects the brightness of the information image according to the brightness. For example, when the surface of the building looks dark, the projector control unit 22 corrects the brightness of the information image to be increased.

  By correcting the luminance in this way, it is possible to prevent the information image projected on the surface of the building from being buried in ambient light.

  The control unit 13 may recognize the color of the frame shape portion in the camera image and the brightness of the surface of the building, and the projector control unit 22 may correct both the color and luminance of the information image.

  In the above example, the frame-shaped image projected on the surface of the building is captured by the camera 11 of the UAV 10, and the control unit 13 of the UAV 10 controls the color of the frame-shaped portion in the camera image and the surface of the building. The case of recognizing the brightness of was shown. The projection station 20 may also include a camera, and the camera may capture a frame-shaped image projected on the surface of the building. And the projection station 20 may be provided with the image recognition part (not shown) which recognizes the color of the frame shape part in a camera image, and the brightness of the surface of a building. That is, the color of the frame-shaped portion in the camera image and the brightness of the surface of the building may be recognized on the projection station 20 side. However, even in this case, the UAV 10 includes the camera 11 and the control unit 13 as shown in FIG.

  Further, as described above, when the projection station 20 also includes a camera, the camera of the projection station 20 may capture an image of the surface of the building in advance and generate a camera image. Based on the camera image, the route deriving unit 24 has a region or crack in the surface of the building that is not exposed to direct sunlight and has a shadow (not a shadow of the UAV 10). An area may be specified. Then, the route deriving unit 24 may determine the projection location of the information image in the area where the shadow is generated and in the area where the crack is not generated.

  According to this embodiment, even if the surface of the building reflects only light of a specific wavelength, the camera 11 can generate a camera image in which an information image is reflected. Moreover, it is possible to prevent the information image projected on the surface of the building from being buried in the ambient light. Therefore, it is possible to easily detect the information image in the camera image.

  The projector 21 may be a projector (for example, a laser projector) having a light source that emits only light of a predetermined wavelength for each of red, green, and blue colors. The projector control unit 22 may correct the color of the information image by switching the light source of the projector 21 in accordance with the color information notified from the control unit 13 of the UAV 10. According to such a configuration, since the wavelength of light emitted from the selected light source is limited, the camera 11 of the UAV 10 clearly shows an information image even in an environment where sunlight exists. A camera image can be obtained.

  Furthermore, the camera 11 of the UAV 10 may be provided with a filter that transmits only light of a predetermined wavelength and that can switch the wavelength of the transmitted light. For example, the wavelength of the light transmitted by the filter may be determined according to the individual light sources included in the projector 21. For example, the filter of the camera 11 transmits only light having a wavelength corresponding to the red light source of the projector 21, transmits only light having a wavelength corresponding to the green light source of the projector 21, and blue of the projector 21. It is a filter that can be switched to either of the states that transmit only the light of the wavelength corresponding to the light source. For example, when the control unit 13 notifies the projection station 20 of information of red, the control unit 13 switches the filter of the camera 11 to a state in which only light having a wavelength corresponding to the red light source of the projector 21 is transmitted. Similarly, when the information about green is notified to the projection station 20, the control unit 13 switches the filter of the camera 11 to a state in which only light having a wavelength corresponding to the green light source of the projector 21 is transmitted. Similarly, when notifying the projection station 20 of the blue information, the control unit 13 switches the filter of the camera 11 to a state in which only light having a wavelength corresponding to the blue light source of the projector 21 is transmitted. Since the camera 11 includes the above-described filter, it is possible to further improve the effect that the camera 11 can obtain a camera image in which an information image is clearly captured even in an environment where sunlight is present. . In addition, the aspect of the filter with which the camera 11 is provided is not specifically limited. For example, in a general color camera, an R / G / B filter having a Bayer arrangement is arranged for each pixel. Only the light of the corresponding light source may be used by this filter. That is, if only a green light source is used, only a green pixel may be used. Similarly, if light sources of other colors are used, only pixels corresponding to the light sources may be used. A combination of a monochrome camera and a replaceable color filter attached to the outside may be used. Further, the camera 11 may not be provided with the above filter.

  In addition, when the color of the surface of the building is known and is one color, the projector 21 may be configured to include one light source that emits only light having a predetermined wavelength corresponding to the color. In this case, the camera 11 may be provided with a filter that transmits only light having a wavelength corresponding to the light source. That is, when the color of the surface of the building is known and is one color, the light source of the projector 21 and the filter of the camera 11 may not be switchable. As already described, the camera 11 may not be provided with a filter.

  Further, particularly when the ambient light is strong with respect to the brightness of the projector, such as in direct sunlight, the projector 21 of the projection station 20 uses a light source with a narrow wavelength width, such as a laser, and transmits only light of this wavelength. It is even better to attach to the UAV10. Sunlight irradiates light over a wide wavelength range, and a general camera outputs a value obtained by integrating light in a wide wavelength range as brightness. However, when a narrow band filter is attached, the sunlight component is only in the transmission range of this filter, so the detection value of the sunlight component is lowered. On the other hand, the laser beam is not affected by the narrow band filter. Therefore, by using a laser light source and a narrow band filter corresponding to the laser light source, it is possible to make it less susceptible to disturbance light.

Embodiment 7. FIG.
In the seventh embodiment of the present invention, the operation when the projection plane of the information image is different from the imaging target plane for the state inspection will be described.

  FIG. 16 is a schematic diagram illustrating an example of a situation where the projection plane of the information image is different from the imaging target plane for state inspection. It is assumed that an area to be imaged for state inspection is an area 82. However, region 82 is horizontal. In the area 82, there is an area that becomes a blind spot as viewed from the projector 21 due to the structure 39 of the building 30. In addition, an information image cannot be projected onto an area that is a blind spot as viewed from the projector 21. Even if the projector 21 projects an information image in a region that does not become a blind spot when viewed from the projector 21 in the region 82, the UAV 10 may not be able to capture the information image projected on the horizontal region 82.

  In such a situation, the projection plane of the information image and the imaging target plane for state inspection may be different planes.

  A region 81 is a region including a projection location of the information image. The projector 21 projects an information image (hereinafter, this information image is referred to as an information image Q) in the area 81. When the information image generation unit 25 (see FIG. 10) of the projector 21 generates the information image Q, the control content of the direction of the moving camera is also included in the information image Q in addition to the control content of the direction of the moving camera. Include it. In this example, the information image generation unit 25 includes, in the information image Q, the control content instructing to turn the camera in the upward direction in the air.

  The UAV 10 proceeds in the direction of the target position P3 based on the information image immediately before the information image Q, and performs steps B1 to B3 (see FIG. 11). If the information image can be recognized in the camera image, the UAV 10 performs the operations after step B4.

  In the control unit 13 of the UAV 10 (see FIG. 6), in step B6, the UAV 10 is suspended on the spot according to the information image Q. Furthermore, the control unit 13 directs the camera in the upward direction in the air according to the control content included in the information image Q. The control unit 13 performs imaging for state inspection while the vehicle is stagnating in this state. At this time, since the camera 11 faces directly upward, the area 82 to be inspected can be imaged, and a camera image of the area 82 can be generated.

  According to the present embodiment, even if an information image cannot be projected onto a target surface for imaging for state inspection, the information image Q projected onto another surface can be used to reach the vicinity of the target position. . Then, according to the control content included in the information image Q, the UAV 10 can change the direction of the camera in the air, capture an image of an imaging target surface for state inspection, and generate a camera image of that surface.

Embodiment 8. FIG.
Due to aging of the building, defects such as cracks and floating may occur on the surface of the building. The unmanned aerial vehicle control system according to the eighth embodiment of the present invention detects such a defect. Further, the unmanned aerial vehicle control system corrects the information image according to the defect so that the projected information image is not distorted.

  In the eighth embodiment of the present invention, the projector 21 also projects an image representing a predetermined pattern (hereinafter referred to as a pattern image) onto the surface of the building. The control unit 13 of the UAV 10 detects the state of a defect on the surface of the building according to the shape of the pattern shown in the camera image. The projection station 20 corrects the information image according to the state of the defect.

  More specific description will be given below. The projector 21 projects the pattern image on the surface of the building. The pattern image may be an image representing a mesh pattern, for example. In the present embodiment, for the sake of simplicity of explanation, a horizontal straight line will be described as an example of the predetermined pattern. FIG. 17 is a schematic diagram illustrating an example of a pattern image representing a horizontal straight line. Although FIG. 17 shows a pattern image representing a single straight line, a plurality of straight lines may be represented in the pattern image.

  FIG. 18 is a schematic diagram illustrating a state in which a pattern image is projected on the surface of a building where a step is generated due to deterioration over time, and the camera 11 of the UAV 10 captures the projected portion. FIG. 19 is a schematic side view of the state shown in FIG. In the example shown in FIG. 18 and FIG. 19, the surface 91 of the building is in a state of being floated more than the surface 92, and a step is generated.

  The camera 11 captures the surface 91 and the surface 92 on which the pattern image (horizontal straight line) is projected, and generates a camera image. FIG. 20 is a schematic diagram illustrating an example of a camera image obtained as a result of imaging a pattern image. In the camera image shown in FIG. 20, surfaces 91 and 92 and a line 93 are shown. The projector 21 projects a horizontal straight line onto the surfaces 91 and 92, but due to the level difference between the surfaces 91 and 92, the straight line appears as a bent line 93 in the camera image.

  In addition, the projection station 20 transmits the angle formed by the projection direction of the pattern image and the wall surface to the UAV 10. Projection station 20 may communicate this angle to UAV 10 by communication. Alternatively, the projection station 20 may project an information image including this angle information, and the control unit 13 of the UAV 10 may recognize the angle information from the information image in the camera image.

  The control unit 13 of the UAV 10 recognizes the shape of a pattern (in this example, a line 93) shown in the camera image. The control part 13 determines the state of the surface of a building based on the shape of the pattern reflected in the camera image. In this example, the control unit 13 determines that there is a step between the surface 91 and the surface 92 based on the shape of the line 93 captured in the camera image. That is, the control unit 13 detects the size of the step between the surface 91 and the surface 92 and the position where the step is generated. For example, the control unit 13 may store in advance the correspondence between the angle formed by the projection direction of the pattern image and the wall surface and the shape deformation of the pattern in the camera image. And the control part 13 should just determine the state of the surface of a building by specifying the state of the surface of the building corresponding to the shape deformation of the pattern in a camera image.

  The control unit 13 transmits information on the state of the surface of the building (in this example, the size of the step between the surface 91 and the surface 92 and the position where the step is generated) to the projection station 20.

  The projector control unit 22 of the projection station 20 receives the information on the state of the building surface transmitted from the control unit 13. Then, the projector control unit 22 receives information on the exterior of the building and the three-dimensional shape of the building stored in the data storage unit 23, and information on the surface state of the building received from the UAV 10 (control unit 13). Based on the above, the information image is corrected so that the projected information image is not distorted.

  According to this embodiment, the control part 13 can detect the defect of the building and the magnitude | size of the defect which arose by aged deterioration etc. Furthermore, since the control unit 13 transmits information on the state of the surface of the building to the projection station 20, the projection station 20 (projector control unit 22) corrects the information image so that the projected information image is not distorted. can do. Therefore, the projection station 20 can project the information image on the surface of the building in a favorable state, and the guidance accuracy of the UAV 10 can be further increased.

  Each embodiment of the present invention is applicable not only to infrastructure deterioration diagnosis but also to the case of controlling a UAV to perform patrol monitoring of buildings.

  Next, the outline of the present invention will be described. FIG. 21 is a schematic diagram showing an outline of the unmanned flight control system of the present invention. The unmanned flying device control system 100 includes an image projection unit 101 and a control unit 103.

  The image projection unit 101 (for example, the projection station 20) receives an image (for example, an information image) for controlling an unmanned flight apparatus (for example, the UAV 10) on which an imaging device (for example, the camera 11) is mounted, as an object (for example, Project to the building subject to condition inspection.

  The control unit 103 (for example, the control unit 13) controls the operation of the unmanned flight apparatus based on the image captured by the imaging apparatus. The control unit 103 controls the operation of the unmanned aerial vehicle based on the amount of change in the flight state when an image cannot be captured due to a change in the unmanned aerial vehicle flight state (for example, the posture or position of the UAV 10). .

  With such a configuration, when controlling the unmanned aerial vehicle based on the image captured by the unmanned aerial vehicle and guiding the unmanned aerial vehicle near the target position near the object, the image captured by the unmanned aerial vehicle is displayed. Even if it is lost, the unmanned flight device can be controlled.

  The present invention is preferably applied to an unmanned flight apparatus control system for controlling an unmanned flight apparatus.

10 UAV (Unmanned Flying Object)
DESCRIPTION OF SYMBOLS 11 Camera 13 Control part 15 Camera angle control part 16 Motor driver 17a-17d Motor 18 Communication part 20 Projection station 21 Projector 22 Projector control part 23 Data storage part 24 Path | route derivation part 25 Information image generation part 27 Communication part

Claims (10)

An image projecting unit that projects an image for controlling the operation of the unmanned flight apparatus equipped with the imaging device onto an object;
A control unit that controls the operation of the unmanned flight device based on the image captured by the imaging device;
The said control part is an unmanned flight apparatus control system which controls operation | movement of the said unmanned flight apparatus based on the variation | change_quantity of the said flight state, when it becomes impossible to image the said image by the change of the flight state of the said unmanned flight apparatus. The unmanned flight apparatus control system according to claim 1, wherein the control unit calculates a change amount of the flight state based on a change amount of a plurality of images captured by the imaging device. The unmanned flight device includes an inertial navigation device,
The unmanned flight apparatus control system according to claim 1, wherein the control unit calculates an amount of change in the flight state using the inertial navigation apparatus. The image projection unit
A tracking imaging device that tracks and images the unmanned flight device,
2. The unmanned flight according to claim 1, wherein when the image cannot be captured due to a change in a flight state of the unmanned flight device, an amount of change in the flight state is calculated based on an image captured by the tracking imaging device. Device control system. The control unit changes one or both of an attitude of the unmanned flight apparatus and an orientation of the imaging apparatus based on a change amount of the flight state. An unmanned aerial vehicle control system as described. The said image projection part produces | generates the image for controlling operation | movement of the said unmanned flight apparatus according to the variation | change_quantity of the said flight state, and projects it on the said target object. The unmanned aerial vehicle control system according to item. The image for controlling the operation of the unmanned flight apparatus includes identification information for identifying control contents,
The controller is
Based on the change amount of the flight state, expand the imaging range of the imaging device,
Controlling the operation of the unmanned aerial vehicle based on the position of an image for controlling the operation of the unmanned aerial vehicle imaged in the enlarged imaging range;
The unmanned flight according to any one of claims 1 to 4, wherein the enlarged imaging range is reduced to an imaging range in which the identification information can be recognized in accordance with control of the operation of the unmanned flying device. Device control system. The image projection unit includes a route deriving unit that determines a projection position of an image on the object and derives a route of the unmanned flight apparatus,
The route deriving unit calculates a position of the unmanned flying device based on the change amount of the flight state, and derives a route from the calculated position of the unmanned flying device. The unmanned aerial vehicle control system according to item. Project an image for controlling the operation of the unmanned aerial vehicle equipped with the imaging device onto the object,
Based on the image captured by the imaging device, control the operation of the unmanned flight device,
An unmanned aerial vehicle control method for controlling an operation of the unmanned aerial vehicle based on a change amount of the flying state when the image cannot be captured due to a change in a flying state of the unmanned aerial vehicle. An imaging device for controlling the operation of the unmanned flying device, the imaging device capturing an image projected on the object;
A control unit that controls the operation of the unmanned flight device based on the image captured by the imaging device;
The said control part is an unmanned flight apparatus which controls operation | movement of the said unmanned flight apparatus based on the variation | change_quantity of the said flight state, when it becomes impossible to image the said image by the change of the flight state of the said unmanned flight apparatus.

Images