JP2017138162A - Inspection system and inspection method of structure - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an inspection system and an inspection method capable of photographing an image for inspecting a structure safely and easily.MEANS FOR SOLVING THE PROBLEM: An inspection system includes: an automated flight unit 100; photographing means 110 mounted on the automated flight unit 100; three-dimensional shape model generation means 245 for generating a three-dimensional shape model P of a structure, at least based on an image of the structure photographed at a position relatively far away from the structure by the photographing means 110; flight enable surface creation means 246 for creating a flight enable surface Q of the automated flight unit 100 at a position closer to each surface for composing the three-dimensional shape model 245 than the long distance above; and flight enable surface correction means 247 for moving at least one portion of the flight enable surface 246 relative to a surface of the corresponding three-dimensional shape model and for correcting the surface. The automated flight unit 100 autonomously flies according to a flight route scheduled within the flight enable surface Q, and the photographing means 100 photographs an image for inspecting a structure during autonomous flight of the automated flight unit 100.SELECTED DRAWING: Figure 1

Description

  The present invention has an unmanned flight unit and a photographing means such as a camera mounted thereon, and a structure inspection system for inspecting deterioration such as paint peeling or corrosion of a roof or an outer wall in a structure such as a house or a factory. And the inspection method.

  Among the structures, especially the roof is rapidly deteriorated in places exposed to harsh natural environments such as rain and wind and strong sunlight in the summer, and if left unaffected, it can cause the structure to age quickly. . In order to maintain the good state of the structure over a long period of time, it is important to conduct periodic inspections and repair / repair or replace the roof as necessary.

  Traditionally, such inspection work has been done by workers actually climbing the roof and walking, but it is difficult to access the roof. Large factories with a length of several hundred meters in the direction of the beam When inspecting roofs, etc., it takes a lot of time for inspection, and it is necessary to take safety measures to prevent falling, so the cost of inspection is high, and owners who hesitate to regularly inspect The current situation is.

  By the way, in order to inspect at a position away from a place where inspection is necessary, it is conceivable to use an unmanned flight unit equipped with a camera. As such an unmanned flight unit, for example, a drone ( Among the unmanned aerial vehicles, a multicopter is known that flies, flies, and stops (hovering) in the air with a plurality of rotors. In general, a multicopter is equipped with a microcomputer, GPS (Global Positioning System), 3-axis gyro, 3-axis accelerometer, etc., and is capable of self-sustained flight. Remote operation with a control device such as a smartphone is possible. Moreover, it is equipped with a flight camera that captures the field of view, such as the front and the bottom, and can send images of the field of view from the multicopter to the pilot's control device. ing.

  Patent Document 1 below proposes a technique for inspecting a dam using such a multi-copter. Specifically, this technique is applied to a radio-operated rotary wing aircraft with a predetermined flight path. An image of the structure is taken with a mounted camera, and the taken image is subjected to image analysis to determine whether or not the structure is defective.

Japanese Patent Laying-Open No. 2015-194069

  However, remote inspection using a multi-copter as described above enables inspection of places that are difficult for people to approach, and prevents accidents at high places, etc., compared to performing inspection work manually. On the other hand, there is a risk that the safety of work can be enhanced, but there is the following danger. In other words, in remote inspection using a multicopter, there is a risk of colliding with surrounding obstacles during flight and becoming impossible to maneuver, causing death or injury or damage to objects when crashed. . Recently, multi-copters equipped with sensors for avoiding collisions with surrounding obstacles have been developed, but in such sensors, antennas installed in houses and factories, wires for fixing them, etc. In some cases, it is difficult to detect small obstacles, and the risk of collision is still high.

  In order to avoid collision with surrounding obstacles during flight, it is considered effective to set the flight path of the multicopter in advance so as to avoid obstacles. There is no study on a specific setting method, and obstacles cannot be avoided.

  Therefore, the present invention solves the problems of the prior art described above, and is safe and simple even when a thin obstacle such as an antenna or a wire for fixing the antenna is provided on the structure. It is an object of the present invention to provide an inspection system and an inspection method capable of taking an image for checking a structure.

  The structure inspection system of the present invention for solving the above-mentioned problems is an inspection system for inspecting a structure by photographing the structure to be inspected from the outside, and an unmanned flight unit capable of independent flight and A third-order model for generating a three-dimensional shape model of the structure based on an image of the imaging unit mounted on the unmanned flight unit and at least an image of the structure imaged at a relatively far distance from the structure by the imaging unit Original shape model generation means, flightable surface creation means for creating a flightable surface of the unmanned flight unit at a position closer to the far distance than the far distance from each surface constituting the three-dimensional shape model, and the unmanned In order to avoid the flight unit from colliding with obstacles attached to the structure during self-sustained flight, at least a part of the flightable surface is moved relative to the surface of the corresponding 3D shape model. And the unmanned flight unit makes a self-sustained flight in accordance with a flight path planned in the flightable surface, and the photographing unit performs the self-sustained flight during the unmanned flight of the unmanned flight unit. An image for inspecting the structure is taken. Here, the “far distance position” is a position sufficiently away from the structure that does not cause a collision with an obstacle such as an antenna or a wire attached to the structure when the unmanned flight unit is made to fly. Yes, specifically, 3 m or more is preferable. However, if the image is taken from an excessively distant position, the accuracy of the three-dimensional shape model generated based on the taken image may be lowered.

  In the structure inspection system according to the present invention, the obstacles when the flightable surface is corrected may be confirmed by checking the image of the structure taken at the position relatively far from the structure. This is preferably done by reference.

  Further, in the structure inspection system of the present invention, when photographing the structure from the position relatively far from the structure, the automatic take-off that takes off the unmanned flight unit and vertically raises it to a predetermined altitude. It is preferable to provide command means.

  Furthermore, the structure inspection system according to the present invention further comprises an automatic landing command means for vertically landing the unmanned flight unit after photographing the structure from the position relatively far from the structure. Is preferred.

  Furthermore, in the structure inspection system of the present invention, it is preferable that flight path creation means for creating a flight path of the unmanned flight unit is provided in the flightable plane.

  Moreover, in the structure inspection system according to the present invention, an arbitrary range in the image of the structure photographed at a relatively far position from the structure or an arbitrary surface of the three-dimensional shape model is reduced. It is preferable to provide image image creation means for creating a renovated image image by replacing a part of the image with a sample image for reform prepared in advance.

  Further, the structure inspection method of the present invention for solving the above-mentioned problem is an inspection method for inspecting a structure by photographing the structure to be inspected from the outside, and is an unmanned flight capable of independent flight A three-dimensional shape model of the structure is generated based on the unit, the photographing means mounted on the unmanned flight unit, and at least the image of the structure photographed at a relatively far position from the structure by the photographing means. Three-dimensional shape model generating means, and flightable surface creation means for creating a flightable surface of the unmanned flight unit at a position closer to the far distance than each of the surfaces constituting the three-dimensional shape model, In order to avoid the unmanned flight unit from colliding with an obstacle attached to the structure, at least a part of the flightable surface is moved and corrected with respect to the surface of the corresponding three-dimensional shape model. The unmanned flight unit is allowed to fly autonomously according to a planned flight path within the flightable surface, and the photographing means is used to perform a self-sustained flight of the structure during the autonomous flight of the unmanned flight unit. An image for inspection is taken.

  In the structure inspection method according to the present invention, the obstacle image when the flightable surface is corrected is checked with the image of the structure photographed at the position relatively far from the structure. This is preferably done by reference.

  Further, in the structure inspection method of the present invention, when photographing the structure from the position relatively far from the structure, the unmanned flight unit is automatically taken off and automatically vertical to a predetermined altitude. It is preferable to raise.

  Further, in the structure inspection method of the present invention, it is preferable that the unmanned flight unit is automatically lowered and landed after the structure is photographed from the position relatively far from the structure.

  Moreover, in the structure inspection method of the present invention, an arbitrary range in the image of the structure photographed at a relatively far position from the structure or an arbitrary surface of the three-dimensional shape model is reduced. It is also preferable to create a post-reform image by replacing part of the sample with a sample image for renovation prepared in advance.

  In the structure inspection system and the inspection method of the present invention, a configuration for generating a three-dimensional shape model of a structure based on an image photographed from a relatively long distance by photographing means mounted on an unmanned flight unit; Therefore, it is possible to easily grasp the outer shape of the structure and to prevent the unmanned flight unit from colliding with an obstacle attached to the structure at the time of photographing. Furthermore, since the flightable surface of the unmanned flight unit is created at a short distance from the generated 3D shape model, and the flightable surface is modified as necessary, it is modeled as a 3D shape. When there is a thin obstacle such as an antenna or wire that has not been present, it is possible to prevent the unmanned flight unit from colliding with the obstacle by correcting the flightable surface in consideration of such an obstacle. . And since the unmanned flight unit flies independently according to the planned flight path in the flightable plane set in this way, it is possible to safely and easily check the structure even if it is not a skilled pilot of the unmanned flight unit. An image can be taken at a relatively short distance from the structure.

  Therefore, according to the inspection system and inspection method for a structure of the present invention, even when a thin obstacle such as an antenna or a wire for fixing the antenna is provided on the structure, the structure is safe and simple. An image for inspection can be taken.

It is the schematic which shows the system configuration | structure of the inspection system of one Embodiment according to this invention. It is a flowchart which shows one implementation inspection method of the inspection method of the structure of this invention. (A), (b) is a schematic diagram showing how the unmanned flight unit is automatically taken off and raised to a predetermined altitude using the above inspection system, and (c) performs fine adjustment after ascending. It is a schematic diagram which shows a mode. It is a schematic diagram which shows a mode that a structure is image | photographed from the position of comparatively long distance with respect to a structure using the said inspection system. It is a schematic diagram which shows a mode that a structure is image | photographed while remotely controlling an unmanned flight unit using the said inspection system. It is a schematic diagram which shows a mode that a three-dimensional shape model is produced | generated from a pair of stereo image (photograph) of a structure using the said inspection system. It is the model which shows a mode that the flightable surface was created and displayed with respect to the three-dimensional shape model using the said inspection system, (a) displays the three-dimensional shape model and the flightable surface from diagonally, (B) displays the three-dimensional shape model and the flightable surface from the front side. It is the schematic diagram which showed a mode that the flightable surface was corrected using the said inspection system. It is a schematic diagram which shows a mode that a flight path | route is created by designating the takeoff and landing point and the passing point on the flightable surface using the inspection system. It is a schematic diagram which shows a mode that a flight path | route is created by designating the takeoff and landing point and the passing point on the flightable surface using the inspection system. It is a schematic diagram which shows a mode that a flight path | route is produced using the said inspection system, specifying the takeoff / landing point and the flightable surface which wants to actually fly. It is a schematic diagram which shows a mode that remodeling simulation is performed using the said inspection system. It is a schematic diagram which shows a mode that the flightable surface is partially corrected using the said inspection system.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. FIG. 1 is a schematic diagram showing a system configuration of an inspection system according to an embodiment of the present invention.

  The inspection system of this embodiment shown in FIG. 1 is mainly composed of an unmanned flight unit 100 and an information terminal device 200 on the ground side. In this embodiment, the unmanned flight unit 100 can fly independently according to the flight path created by the method described in detail below. However, the unmanned flight unit 100 can be remotely operated manually using the remote control unit 243 incorporated in the information terminal device 200. Steering is also possible. It is also possible to remotely control the unmanned flight unit 100 by switching from the remote control unit 243 to the digital proportional remote control device 300 (FIG. 5).

  The unmanned flight unit 100 has a plurality of (for example, four) rotors that are rotationally driven by the power of the battery mounted on the unmanned flight unit 100, and balances the lifts generated by the rotors with each other so as to float and stop in the air (hovering). By making the lift of the rotor on the rear side larger than the lift of the rotor on the front side with respect to the flight direction, the attitude of the aircraft can be tilted in the flight direction and the aircraft can fly in any direction. By rotating half of the rotors in opposite directions, it is possible to counteract the rotational reaction of the rotors and prevent the airframe from rotating.

  As shown in FIG. 1, the unmanned flight unit 100 includes an imaging device 110 as an imaging means, a wireless communication unit 120, a 3-axis acceleration sensor 130, a 3-axis gyro sensor 140, a GPS (global positioning system) 150, an altimeter 160, A control unit 180 and a storage unit 190 are included.

  The imaging device 110 is preferably capable of capturing still images and moving images. A commercially available digital camera can be used as the imaging device 110. The photographing apparatus 110 preferably has a mechanism for changing the photographing direction (up and down, left and right).

  The wireless communication unit 120 is a communication interface for performing wireless communication with the information terminal device 200, and transmits position information of the unmanned flight unit 100, image data captured by the image capturing device 110, and the like to the information terminal device 200. On the contrary, a command signal, flight path data, and the like for remotely maneuvering the unmanned flight unit 100 from the information terminal device 200 are received.

  The control unit 180 controls the entire unmanned flight unit 100, and includes an attitude control unit 182, a flight control unit 184, an imaging control unit 186, and an image data processing unit 188. Actually, programs corresponding to these functional units are stored in a ROM (read only memory) or a non-volatile memory (not shown), and these programs are read and executed by a CPU (central processing unit), respectively. The processing corresponding to is executed.

  The attitude control unit 184 stabilizes the attitude of the unmanned flight unit 100, acquires the attitude information of the unmanned flight unit 100 from the 3-axis acceleration sensor 130, the 3-axis gyro sensor 140, and the like, and converts the obtained attitude information into the obtained attitude information. Based on this, the rotation speed of each rotor of the unmanned flight unit 100 is controlled.

  The flight control unit 184 controls the flight of the unmanned flight unit 100. For example, the current position information (latitude, longitude, and altitude) of the aircraft obtained from the GPS 150 and the altimeter 160, and the storage unit 190 Based on the stored flight path, when the unmanned flight unit 100 controls the rotation speed of each rotor of the unmanned flight unit 100 to fly along the flight path, or when the unmanned flight unit 100 is remotely controlled, Based on the signal related to flight control transmitted from the information terminal device 200, the rotation speed of each rotor of the unmanned flight unit 100 is controlled.

  The shooting control unit 186 performs control related to shooting with a still image or a moving image, control related to changing the direction of the lens (up and down, left and right) of the shooting device 110 based on a command from the information terminal device 200.

  The image data processing unit 188 acquires the still image data when the imaging device 110 captures a still image, adds the position information of the unmanned flight unit 100 at the time of shooting, etc. to the still image data, and stores it. Save to 190. Further, the image data processing unit 188 acquires moving image data and stores it in the storage unit 190 when the photographing apparatus 110 captures a moving image. At this time, the image data processing unit 188 acquires still image data from the moving image data every predetermined time, adds position information of the unmanned flight unit 100 at the time of shooting, etc. to the still image data and stores it in the storage unit 190. May be saved.

  The information terminal device 200 is composed of, for example, at least one of a personal computer, a tablet terminal, and a smartphone, and can be held by an inspection worker located around the structure.

  As illustrated in FIG. 1, the information terminal device 200 includes a display unit 210, an input unit 220, a wireless communication unit 230, a control unit 240, and a storage unit 250.

  The display unit 210 displays image data of a structure photographed by the photographing apparatus 110, a three-dimensional shape model P, a flightable surface Q, and the like described later, and a monitor, a touch panel, or the like can be used.

  The input unit 220 performs various inputs to the information terminal device 200, for example, input related to shooting by the imaging device 110, input related to setting of a flight path described later, input related to correction of a flightable surface Q described later, and remote control of the unmanned flight unit 100. Input related to maneuvering, and a keyboard, mouse, touch panel, or the like can be used. When a touch panel is used, the function of the display unit 210 can be shared.

  The wireless communication unit 230 is a communication interface for performing wireless communication with the unmanned flight unit 100, and transmits to the unmanned flight unit 100 a command signal, flight path data, and the like for remotely maneuvering the unmanned flight unit 100. On the other hand, the position information of the unmanned flight unit 100, the image data photographed by the photographing device 110, and the like are received from the unmanned flight unit 100.

  The control unit 240 controls the entire information terminal device 200, and includes an automatic take-off command unit 241 as an automatic take-off command unit, an automatic landing command unit 242 as an automatic landing command unit, a remote control unit 243, and a photographing command unit. 244, a three-dimensional shape model generation unit 245 as a three-dimensional shape model generation unit, a flightable surface generation unit 246 as a flightable surface generation unit, a flightable surface correction unit 247 as a flightable surface correction unit, and a flight path generation unit As a flight path creation unit 248 and an image image creation unit 249 as an image creation unit. Actually, programs corresponding to these functional units are stored in a ROM (read only memory) or a non-volatile memory (not shown), and these programs are read and executed by a CPU (central processing unit), respectively. The processing corresponding to is executed.

  The automatic take-off command unit 241 instructs the inspection operator to specify the altitude after take-off, inputs an “ascending” instruction, and based on the “ascending” instruction, takes off the unmanned flight unit 100 and A command for raising the unmanned flight unit 100 vertically on the field is issued to the flight control unit 184 of the unmanned flight unit 100.

  The automatic landing command unit 242 inputs a “landing” instruction to the inspection operator, and on the basis of the “landing” instruction, issues a command for descending and landing the unmanned flying unit 100 vertically on the spot. It outputs to the control part 184.

  The remote control unit 243 is for manually performing the remote control of the unmanned flight unit 100, and issues a command related to the remote control input from the inspection operator through the input unit 220 to the flight control unit 184 of the unmanned flight unit 100.

  The shooting command unit 244 sends a command related to shooting with a still image or a moving image or a command related to changing the lens direction (vertical or horizontal direction) of the shooting device 110 based on the shooting command input through the input unit 220. The image is sent to the 100 shooting control unit 186.

  The three-dimensional shape model generation unit 245 generates a three-dimensional shape model P (FIG. 6) of the structure from a pair of images (stereo images) obtained by photographing the structure at different angles according to a well-known photogrammetry method, and a storage unit 250. Specifically, the three-dimensional shape model generation unit 245 first acquires the pair of images from the image group stored in the storage unit 250, and then automatically acquires corresponding points between the pair of images. After that, the three-dimensional information of the structure is calculated by locating these images, and the three-dimensional shape model P of the structure is generated.

  The flightable surface creation unit 246 calculates a flightable surface Q that is spaced a predetermined distance outward from each surface that constitutes the three-dimensional shape model P of the structure generated by the three-dimensional shape model generation unit 245 to obtain a three-dimensional shape. It is displayed on the display unit 210 together with the model P (FIG. 7). Here, the flightable surface Q is preferably created at a position 3 to 10 m away from each surface constituting the three-dimensional shape model P. In addition, the flightable surface creation unit 246 may allow an inspection operator to input whether or not the flightable surface Q needs to be corrected. It is possible to cause the inspection operator to input a “no correction necessary” instruction for the flightable surface Q via the input unit 220, and to determine the created flightable surface Q and store it in the storage unit 250.

  The flightable surface correction unit 247 specifies the correction range and the correction distance for the flightable surface Q that needs to be corrected, and the specified correction range is specified based on the specified correction range and correction distance. The distance is moved relative to the surface of the corresponding 3D shape model, and the corrected flightable surface Q is output to the display unit 210 together with the 3D shape model P (FIG. 8). At this time, the inspection operator refers to, for example, a thin obstacle such as an antenna or a wire by referring to an image of a structure photographed for generating a three-dimensional shape model (FIG. 4B) or by visual observation at an actual site. The flightable surface Q can be corrected in consideration of the obstacle by checking the presence or absence of the obstacle and the protruding length of the obstacle from the outer surface of the structure. The flightable surface correction unit 247 determines the corrected flightable surface Q and stores it in the storage unit 250.

  The flight path creation unit 248 reads the determined flightable plane Q from the storage unit 250 and causes the display unit 210 to display it, and also allows the inspection operator to pass the take-off and landing points a and c and the unmanned flight unit 100. Points b1, b2, b3... Bn (FIGS. 9 and 10) and at least one of the flightable planes Q (FIG. 11) on which the unmanned flight unit 100 is desired to fly, and within specified conditions The flight path is calculated, and the created flight path is stored in the storage unit 250. At this time, the flight path creation unit 248, for example, based on the imaging range of the imaging device 110 considering the distance from the imaging target portion of the structure to the imaging device 110 mounted on the unmanned flight unit 100, The flight path is calculated so that the part is photographed without omission. The flight route may be manually set on the flightable surface Q by the inspection operator.

  The image image creation unit 249 reads the three-dimensional shape model P of the structure from the storage unit 250 and gives the user (not only the inspection worker, but also a remodeling person or a customer) a three-dimensional shape model. A reform target surface or range is designated from the surfaces constituting P, and one reform sample image (for example, a plurality of reform sample images m1, m2, m3 (FIG. 12) stored in the storage unit 250 in advance) (for example, m1) is selected, and the selected reform sample image m1 is pasted on the reform target surface or range specified above to create an image image after the reform. Alternatively, the image image creation unit 249 reads out, for example, an image of a structure used when generating the three-dimensional shape model P (FIG. 4B), allows the user to specify the remodeling target range in the image, and One reform sample image (for example, m1) is selected from the plurality of reform sample images m1, m2, and m3 stored in the storage unit 250, and the selected reform sample image m1 is set in the reform target range specified above. It is also possible to create an image after reforming by pasting.

  Next, an embodiment of the structure inspection method of the present invention using the above-described embodiment of the structure inspection system of the present invention will be described. Here, FIG. 2 is a flowchart showing an implementation inspection method of the structure inspection method of the present invention.

  The structure inspection method of this embodiment can be roughly divided into three steps. First, it is a group of steps (steps S1, S2) for flying the unmanned flight unit 100 and photographing the structure from a relatively long distance. Second, based on the image of the structure photographed from a long distance, Steps (steps S3 to S6) for obtaining a flight path through which the unmanned flight unit 100 can fly safely. Third, the structure is operated while the unmanned flight unit 100 is flying independently according to the obtained flight path. This is a group of steps (steps S7 and S8) in which images are taken from a relatively short distance, an image for inspection is acquired, and inspection is performed. In this example, a tablet terminal having a touch panel screen having the functions of the display unit 210 and the input unit 220 is used as the information terminal device 200. However, the display unit 210 and the input unit 220 are different personal computers. Alternatively, some steps may be performed by a tablet terminal and the remaining steps may be performed by a personal computer. Further, in this example, in addition to the above three step groups, there is a step of performing remodeling simulation using the generated three-dimensional shape model P or an image of a structure photographed from a long distance. . Hereinafter, each step will be described in order.

  In step S1, as shown in FIG. 3A, the inspection operator places the unmanned flight unit 100 in a place where it can safely take off and land near the structure. In the information terminal device 200, the automatic take-off command unit 241 and the imaging command unit 244 are activated. The automatic takeoff command unit 241 displays an altitude setting field and an “up” button on the touch panel screens 210 and 220, and the inspection operator considers the height of the structure from the altitude setting field (for example, 3 to 100 m). Then, a desired altitude (“20 m” in the illustrated example) is selected, and the “rise” button is pressed. As a result, the unmanned flight unit 100 ascends vertically to the specified altitude as shown in FIG. At this time, real-time video of the photographing apparatus 110 is displayed on the touch panel screens 210 and 220. If the entire structure is not contained in the real-time video, the inspection operator can finely adjust the altitude and direction of the unmanned flight unit 100 as shown in FIG.

  In step S2, as shown in FIG. 4A, the imaging command unit 244 displays a “measurement” button on the touch panel screen. The inspector confirms that the entire structure is contained in the real-time video and presses the “Measure” button. When the “Measure” button is pressed, the imaging device 110 rotates the lens stepwise as shown in FIG. 4B, and images the structure multiple times (at least twice) from different directions. At this time, if it is necessary to take an image of the blind spot of the structure, the unmanned flight unit 100 may be switched to the remote control mode, and the unmanned flight unit 100 may be photographed while being remotely operated. When the operator is not a skilled pilot of the unmanned flight unit 100, it is preferable that the unmanned flight unit 100 is automatically landed once and the unmanned flight unit 100 is carried to a predetermined position and then the operation of step S1 is performed again. The captured image data is stored in the storage unit of the unmanned flight unit 100 together with position information (latitude, longitude, altitude) and the like of the unmanned flight unit 100 at the time of shooting. The stored image data and related information may be transmitted to the information terminal device 200 via the wireless communication unit, or after landing the unmanned flight unit 100, a portable storage medium such as a USB flash memory or the like You may move to the information terminal device 200 using a transmission cable.

  If the inspection operator is a skilled pilot of the unmanned flight unit 100, the remote control of the information terminal device 200 is performed as shown in FIG. 5A as steps S1 ′ and 2 ′ in place of steps S1 and S2. The unit 243 is activated or the unmanned flight unit 100 is manually taken off by using the remote control device 300, and while the unmanned flight unit 100 is visually confirmed from the ground, stop (hovering), imaging and movement are repeated, and the structure May be taken multiple times from different positions. When the unmanned flight unit 100 cannot be confirmed visually from the ground, it is operated and photographed while viewing the real-time video of the photographing device 110 and the GPS 150 position information displayed on the display unit 210 (touch panel screen in this operation example). Alternatively, the inspection operator may perform operation and photographing while moving to a position where the unmanned flight unit 100 can be visually confirmed. In the case of shooting while remotely maneuvering the unmanned flying unit 100, as shown in FIG. 5B, the surroundings of the structure are spirally formed with the unmanned flying unit 100 sufficiently separated from the structure. You may fly. At this time, as shown in FIG.5 (c), the eaves of a roof can also be image | photographed and it can be reflected in a three-dimensional shape model.

  In step S <b> 3, the three-dimensional shape model generation unit 245 is activated in the information terminal device 200. The three-dimensional shape model generation unit 245 is photographed from a relatively long distance in step S2 or step S2 ′, and a plurality of images stored in the storage unit 250 of the information terminal device 200 are as shown in FIG. The structure is obtained by acquiring a pair of images (stereo images) taken from different angles of the structure, then automatically acquiring corresponding points between the images, and then locating these images. The three-dimensional information of the object is calculated, and a three-dimensional shape model (for example, a surface model) of the structure is generated. In this example of operation, the texture derived from the said image is affixed on each surface which comprises a three-dimensional shape model. When a thin obstacle such as an antenna is attached to the structure, such an obstacle is also represented in the three-dimensional shape model as a two-dimensional image, and the presence and position of the thin obstacle can be confirmed. it can.

  In step S <b> 4, the flightable surface creation unit 246 is activated in the information terminal device 200. As shown in FIG. 7, the flightable surface creation unit 246 calculates a flightable surface Q that is spaced apart from each surface constituting the three-dimensional shape model P of the structure by a predetermined distance (for example, 2 to 5 m). It is displayed on the touch panel screens 210 and 220 together with the original shape model P. The flightable surface creation unit 246 causes the inspection operator to input whether or not the flightable surface Q needs to be corrected on the touch panel screens 210 and 220, and when the correction is not necessary, that is, the inspection operator selects “ When the “no correction required” instruction is input, the flightable surface creation unit 246 stores the created flightable surface Q in the storage unit 250 and proceeds to step S6. When the “correction required” instruction is input, the flightable surface correction unit 247 is activated and the process proceeds to step S5.

  In step S5, as shown in FIG. 8, the flightable surface correction unit 247 causes the inspection operator to specify a correction range and a correction distance for the flightable surface Q that needs to be corrected on the touch panel screens 210 and 220. . The inspection operator inputs the correction range and the correction distance of the flightable surface Q. The flightable surface correction unit 247 moves the specified correction range of the flightable surface Q by the specified correction distance with respect to the surface of the corresponding three-dimensional shape model, and the corrected flightable surface Q is three-dimensionally moved. It is displayed on the touch panel screens 210 and 220 together with the shape model P. At this time, for example, the inspection operator refers to the image of the structure photographed from a long distance for generating the three-dimensional shape model (FIG. 4B) or visually at the actual site to check the antenna and the wire. It is possible to check the length of protrusion of a thin obstacle such as the outer surface of the structure and correct the flightable surface Q in consideration of the obstacle. The flightable surface correction unit 247 determines the corrected flightable surface Q and stores it in the storage unit 250.

  In step S <b> 6, the flight route creation unit 248 is activated in the information terminal device 200. The flight path creation unit 248 reads the confirmed flightable plane Q from the storage unit 250 and displays it on the touch panel screens 210 and 220 as shown in the upper diagram of FIG. The take-off and landing points a and c of the unit 100 and the passing points b1, b2,. The flight path creation unit 248 calculates the flight path within the specified conditions (the lower diagram in FIG. 9), and stores the created flight path in the storage unit 250. For example, the flight path creation unit 248 leaks the imaging target portion of the structure based on the imaging range of the imaging device 110 considering the distance from the imaging target portion of the structure to the imaging device 110 mounted on the unmanned flight unit 100. The flight path can be calculated so that it can be photographed without any problems. In the case of a large structure such as a factory, a large number of passing points b1, b2,... Bn can be designated as shown in FIG. In addition, as shown in FIG. 11, the flight path creation unit 248 may designate the take-off and landing points a and c and the flightable surface Q (indicated by a cross mark) that is desired to fly among the plurality of flightable surfaces Q. it can.

  In step S7, the flight path created in step S6 is stored in the storage unit 190 of the unmanned flight unit 100, and the flight control unit 184 obtains the current position information (latitude, longitude) of the aircraft obtained from the GPS 150, the altimeter 160, and the like. And the altitude) and the flight path stored in the storage unit 190, the number of rotations of the motor of each rotor of the unmanned flight unit 100 is controlled so that the unmanned flight unit 100 flies independently along the flight path. In addition, during this self-sustained flight, the imaging device 110 captures an image of the structure from a relatively short distance (for example, 3 to 5 m). The image to be taken may be a moving image or a still image, but a moving image is preferred. The captured image is preferably stored in the storage unit 190 together with the position information of the unmanned flight unit 100 at the time of shooting. According to this, when the structure is inspected by looking at the captured image later. In addition, the position of the deteriorated portion can be easily grasped. The captured image data may be sequentially transmitted to the information terminal device 200 via the wireless communication unit 230, or after landing, to the information terminal device 200 via a portable storage medium such as a USB flash memory or a transmission cable. May be moved.

  In step S8, the image photographed in step 7 is displayed or reproduced on the touch panel screens 210 and 220, and the presence or absence or degree of deterioration of the roof or outer wall of the structure is checked. When partial or complete renovation (repair) of the roof or outer wall of the structure is required, remodeling can be simulated in the next step S9.

  In step S9, the image creation unit 249 is activated. As shown in FIG. 12, the image image creation unit 249 reads the three-dimensional shape model P of the structure from the storage unit 250 and displays it on the touch panel screens 210 and 220 to configure the user with the three-dimensional shape model P. The reform target surface or range is designated among the surfaces, and one reform sample image (for example, m1) is selected from the plurality of reform sample images m1, m2, and m3 stored in advance in the storage unit 250 and selected. The reform sample image m1 is pasted on the designated reform target surface or range to create an image image after reform. Alternatively, the image image creation unit 249 reads an image of the structure photographed from a long distance (FIG. 4B) from the storage unit 250 and displays it on the touch panel screens 210 and 220, and allows the user to display the image within the image. A reform target range is designated, and one reform sample image (for example, m1) is selected from a plurality of reform sample images m1, m2, and m3 stored in advance in the storage unit 250, and the selected reform sample image m1 is selected. Affixed to the above remodeling target area to create a remodeled image. For example, the customer can easily determine the type and color of the building material by confirming the image after remodeling obtained in this way.

  According to the structure inspection system and the inspection method of the embodiment described above, the three-dimensional shape model P of the structure is generated based on the image photographed from a relatively long distance by the photographing device 110 mounted on the unmanned flight unit 100. Thus, the outer shape of the structure can be easily grasped, and the unmanned flight unit 100 can be prevented from colliding with an obstacle attached to the structure at the time of photographing. Further, since the flightable surface Q of the unmanned flight unit 100 is created at a short distance from the generated three-dimensional shape model P, and the flightable surface Q is corrected as necessary, the three-dimensional shape When there is a thin obstacle such as an antenna or a wire that has not been modeled as an unmanned flight unit 100, the unmanned flight unit 100 collides with the obstacle by correcting the flightable plane Q in consideration of such an obstacle. Can be prevented. And since the unmanned flight unit 100 flies independently according to the planned flight path in the flightable plane Q set in this way, the structure can be safely and easily even if it is not a skilled pilot of the unmanned flight unit 100. An image for inspection can be taken at a short distance from the structure.

  Furthermore, according to the structure inspection system and the inspection method of the embodiment described above, the obstacle in correcting the flightable plane Q with reference to the image of the structure photographed from a position far away from the structure. Obstacles can be easily confirmed by checking the objects.

  Furthermore, according to the structure inspection system and the inspection method of the embodiment described above, when photographing the structure from a position far away from the structure, the unmanned flight unit 100 is automatically taken off and automatically up to a predetermined altitude. Therefore, even if it is not a skilled pilot of the unmanned flight unit 100, the photographing can be performed easily and safely.

  Furthermore, according to the structure inspection system and inspection method of the embodiment described above, the unmanned flight unit 100 is automatically lowered vertically and landed after the structure is photographed from a position far away from the structure. Therefore, the unmanned flight unit 100 can be landed easily and safely even if it is not a skilled pilot of the unmanned flight unit 100.

  Furthermore, according to the structure inspection system and the inspection method of the embodiment described above, an arbitrary range in an image of the structure photographed from a position far away from the structure or an arbitrary surface of the three-dimensional shape model can be obtained. Since at least a part is replaced with sample images m1, m2, and m3 for remodeling prepared in advance, an image image after remodeling is created, so that the simulation after remodeling can be performed easily.

  In the above-described embodiment, the three-dimensional shape model P of the structure is generated based on a pair of stereo images obtained by photographing the structure from different angles. However, the present invention is not limited to this, and a sensor such as a laser distance sensor or an infrared distance sensor. You may acquire the three-dimensional shape of a structure using. Specifically, a sensor such as a laser distance sensor is mounted on the unmanned flying unit 100 together with a photographing device 110 (digital camera), and the structure is photographed while changing the orientation of the photographing device 110 and the sensor with respect to the structure together. And do a scan. A three-dimensional shape model P on which a thin obstacle is drawn can be generated by superimposing an image photographed by the photographing device 110 on the three-dimensional shape obtained by the sensor.

  In the illustrated example, the flightable surface Q is entirely corrected. However, the flightable surface Q may be partially corrected. FIG. 13 shows an example in which the flightable plane Q is partially modified so as to avoid the antenna attached to the roof of the structure.

  Thus, according to the inspection system and inspection method for a structure of the present invention, the structure is safe and simple even when the structure is provided with a thin obstacle such as an antenna or a wire for fixing the antenna. An image for inspection can be taken.

DESCRIPTION OF SYMBOLS 100 Unmanned flight unit 110 Image pick-up device 120 Wireless communication part 130 3-axis acceleration sensor 140 3-axis gyro sensor 150 GPS
160 altimeter 180 control unit 182 attitude control unit 184 flight control unit 186 imaging control unit 188 image data processing unit 190 storage unit 200 information terminal device 210 display unit 220 input unit 230 wireless communication unit 240 control unit 241 automatic takeoff command unit 242 automatic landing Command unit 243 Remote control unit 244 Imaging command unit 245 Three-dimensional shape model generation unit 246 Flightable surface creation unit 247 Flightable surface modification unit 248 Flight path creation unit 249 Image image creation unit 250 Storage unit a Takeoff point b1, b2,.・ ・ Bn Passing point c Landing point m1, m2, m3 Reform sample image P 3D shape model Q Flightable surface

Claims (11)

An inspection system for inspecting a structure by photographing the structure to be inspected from the outside,
An unmanned flying unit capable of independent flight,
Photographing means mounted on the unmanned flight unit;
At least a three-dimensional shape model generating means for generating a three-dimensional shape model of the structure based on an image of the structure photographed at a relatively far position from the structure by the photographing means;
A flightable surface creation means for creating a flightable surface of the unmanned flight unit at a position closer to the distance than the long distance from each surface constituting the three-dimensional shape model;
In order to avoid the unmanned flight unit from colliding with obstacles attached to the structure during self-sustained flight, at least a part of the flightable surface is moved and corrected with respect to the surface of the corresponding three-dimensional shape model A flightable surface correcting means,
The unmanned flight unit fly autonomously according to a planned flight path in the flightable plane;
The structure inspection system is characterized in that the imaging unit captures an image for inspection of the structure during the self-sustained flight of the unmanned flight unit.   2. The structure according to claim 1, wherein confirmation of an obstacle when correcting the flightable surface is performed by referring to the image of the structure taken at the position relatively far from the structure. Inspection system.   3. The structure according to claim 1, further comprising an automatic take-off command means for taking off the unmanned flight unit and vertically raising it to a predetermined altitude when photographing the structure from the position relatively far from the structure. 4. Inspection system.   The structure according to any one of claims 1 to 3, further comprising an automatic landing command means for vertically landing the unmanned flight unit after photographing the structure from the position relatively far from the structure. Inspection system.   The structure inspection system according to any one of claims 1 to 4, further comprising flight path creation means for creating a flight path of the unmanned flight unit in the flightable plane.   An arbitrary range in the image of the structure photographed at a position relatively far from the structure or at least a part of an arbitrary surface of the three-dimensional shape model is a sample image for renovation prepared in advance. The structure inspection system according to any one of claims 1 to 5, further comprising an image image creating unit that replaces and creates an image image after reforming. An inspection method for inspecting a structure by photographing the structure to be inspected from the outside,
An unmanned flying unit capable of independent flight,
Photographing means mounted on the unmanned flight unit;
At least a three-dimensional shape model generating means for generating a three-dimensional shape model of the structure based on an image of the structure photographed at a relatively far position from the structure by the photographing means;
A flightable surface creation means for creating a flightable surface of the unmanned flight unit at a position closer to the distance than the long distance from each surface constituting the three-dimensional shape model;
A flightable surface modification in which at least a portion of the flightable surface is moved and modified relative to the surface of the corresponding three-dimensional shape model to avoid the unmanned flight unit colliding with an obstacle attached to the structure. Means, and
Allowing the unmanned flight unit to fly autonomously according to a planned flight path in the flightable plane;
A method for inspecting a structure, wherein an image for inspecting the structure is photographed during the self-sustained flight of the unmanned flight unit using the photographing means.   The obstacle according to claim 7, wherein the confirmation of the obstacle in correcting the flightable surface is performed by referring to the image of the structure taken at the position relatively far from the structure. Inspection method.   9. The inspection of a structure according to claim 7 or 8, wherein when the structure is photographed from the position relatively far from the structure, the unmanned flight unit is automatically taken off and vertically raised to a predetermined altitude. Method.   The method for inspecting a structure according to any one of claims 7 to 9, wherein the unmanned flight unit is automatically vertically lowered and landed after the structure is photographed from the position relatively far from the structure.   An arbitrary range in the image of the structure photographed at a position relatively far from the structure or at least a part of an arbitrary surface of the three-dimensional shape model is a sample image for renovation prepared in advance. The method for inspecting a structure according to any one of claims 7 to 10, wherein the image after remodeling is created by replacement.

Images