JP6675537B1 - Flight path generation device, flight path generation method and program, and structure inspection method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a flight path generation device capable of automatically flying an unmanned aerial vehicle equipped with a camera to generate a flight path for photographing a structure. SOLUTION: A three-dimensional image 9 of a structure SR, a shooting target area on the structure SR, and a shooting direction of a camera 2 of an unmanned aerial vehicle toward the shooting target area are displayed on a display 30 and input to an input device 20. The shooting target area and the shooting direction are changed according to the instruction. When an instruction to determine the image capturing target area and the image capturing direction is input to the input device 20, the image capturing position on the flight path of the unmanned aerial vehicle and the attitude of the camera 2 capturing the structure SR at the image capturing position are determined. It is calculated based on the shooting target area and the shooting direction. The photographing position is calculated so as to be at a predetermined distance from a virtual center line Ay that penetrates the structure SR in the vertical direction. [Selection diagram] Fig. 1

Description

  The present disclosure relates to a flight path generation device that generates a flight path of an unmanned aerial vehicle such as a drone, a flight path generation method and a program thereof, and a structure inspection method that performs a structure inspection using an unmanned aerial vehicle.

  Generally, in aeronautical surveying, an image of the ground is taken by a camera or a line sensor mounted on an aircraft, and a map is created from the image (for example, see Patent Document 1 below). In recent years, aerial surveying using an unmanned aerial vehicle such as a drone (hereinafter sometimes abbreviated as UAV (unmanned aerial vehicle)) has been put to practical use.

JP-A-10-153426

  On the other hand, in recent years, services for inspecting structures such as roofs, outer walls of buildings, bridges, and steel towers using a drone equipped with a camera have been put to practical use. Based on the video taken by the drone camera, it is possible to diagnose the state of damage or deterioration of the structure. In particular, by using a drone for inspection work at a high place where it is difficult to secure a scaffold, there is an advantage that the safety of the operator can be easily ensured and the time required for the inspection can be reduced.

  However, it is not easy to safely operate the drone and accurately photograph a target inspection point, and a skill of the drone operator is required. If an inexperienced operator controls the drone, mistakes such as accidentally colliding the drone with the structure or the ground, or forgetting to photograph the inspection point to be photographed are more likely to occur, which in turn lowers work efficiency. Problem. To reduce such mistakes depending on the skill of the operator, it is conceivable to automate the flight of the drone. However, in order to realize automatic flight of a drone, it is necessary to determine a flight route in advance and program the information into the drone.

  Therefore, the present disclosure provides a flight path generation device, a flight path generation method, and a program therefor, which can generate a flight path for photographing a structure by automatically flying a UAV such as a drone equipped with a camera, and An object of the present invention is to provide a structure inspection method for photographing a structure by automatically flying an unmanned aerial vehicle.

A first aspect of the present disclosure is a flight path generation device that generates a flight path for an unmanned aerial vehicle equipped with a camera to capture an image of a structure, an input device that inputs a user instruction, a display, and a flight. A processing unit that performs processing for generating a route. The processing unit displays, on a display, a three-dimensional image of the structure, a shooting target area on the structure, and a shooting direction of the camera toward the shooting target area, and performs shooting in response to an instruction input to the input device. When an instruction to change the target area and the shooting direction and determine the shooting target area and the shooting direction is input to the input device, the shooting position is located at a predetermined distance from a virtual center line penetrating the structure in the vertical direction. Then, the photographing position on the flight path and the attitude of the camera that photographs the structure at the photographing position are calculated based on the decided photographing target area and photographing direction. A virtual cylinder that is rotationally symmetric with respect to the center line and has a radius equal to a predetermined distance when viewed from a direction parallel to the center line is referred to as a first cylinder, and viewed from a direction parallel to the center line. The angle between two straight lines connecting the two positions on the first cylinder and the center line is called a center angle. The processor, when calculating a plurality of shooting positions, sets a passing order in which the unmanned aerial vehicle passes through the plurality of shooting positions, and, when the passing order is such that the central angle is larger than a predetermined angle at two adjacent shooting positions. One or more relay positions on the first cylinder, which are one or more relay positions through which the unmanned aerial vehicle passes in the flight path between the two photographing positions, are calculated. The one or more relay positions equally divide a flight path between the two shooting positions into two or more sections, and each of the two or more sections has a central angle smaller than a predetermined angle.
According to such a configuration, it is possible to generate a flight path for photographing a structure by automatically flying an unmanned aerial vehicle.

A second aspect of the present disclosure is a flight path generation method in which a computer generates a flight path for an unmanned aerial vehicle equipped with a camera to capture an image of a structure, the method comprising: displaying a three-dimensional image of the structure on a display; The photographing target area on the object and the photographing direction of the camera toward the photographing target area are displayed, and the photographing target area and the photographing direction are changed according to an instruction input by the user to the input device, and the photographing target area and the photographing direction are changed. When an instruction to determine the direction is input to the input device, the shooting position at a predetermined distance from a virtual center line penetrating the structure in the vertical direction, the shooting position on the flight path, and the structure at the shooting position The posture of the camera for photographing the object is calculated based on the decided photographing target area and photographing direction. A virtual cylinder that is rotationally symmetric with respect to the center line and has a radius equal to a predetermined distance when viewed from a direction parallel to the center line is referred to as a first cylinder, and viewed from a direction parallel to the center line. The angle between two straight lines connecting the two positions on the first cylinder and the center line is called a center angle. When a plurality of shooting positions are calculated, a passing order in which the unmanned aerial vehicle passes through the plurality of shooting positions is set. If the center angle is larger than a predetermined angle at two adjacent shooting positions, the two shooting positions are set. One or more relay positions on the first cylinder, which are one or more relay positions through which the unmanned aerial vehicle passes in the flight path between the two, are calculated. The one or more relay positions equally divide a flight path between the two shooting positions into two or more sections, and each of the two or more sections has a central angle smaller than a predetermined angle.

  A third aspect of the present disclosure is a program that causes a computer to execute the above-described flight path generation method.

  According to a fourth aspect of the present disclosure, a flight path is generated by the above-described flight path generation method, an unmanned aerial vehicle is caused to fly automatically around the structure based on the generated flight path, and each imaging target region of the structure is imaged. This is the structure inspection method.

  According to the present disclosure, it is possible to provide a flight path generation device, a flight path generation method, and a program for generating a flight path for photographing a structure by automatically flying a UAV such as a drone equipped with a camera. Further, according to the present disclosure, it is possible to provide a structure inspection method for photographing a structure by automatically flying an unmanned aerial vehicle.

FIG. 1 is a diagram illustrating an example in which an unmanned aerial vehicle equipped with a camera flies around a structure and photographs the structure. FIG. 2 is a flowchart for explaining an example of the structure inspection method according to the present embodiment. FIG. 3 is a diagram illustrating an example of a configuration of the flight path generation device according to the present embodiment. FIG. 4 is a flowchart illustrating an example of an operation of generating a flight path in the flight path generation device according to the present embodiment. FIG. 5 is a first diagram illustrating an example of a three-dimensional image of a structure displayed on a display. FIG. 6 is a second diagram illustrating an example of a three-dimensional image of the structure displayed on the display. FIG. 7 is a third diagram illustrating an example of a three-dimensional image of the structure displayed on the display. FIG. 8 is a diagram for explaining an example in which the shooting target area and the shooting direction are changed in accordance with an input from the input device, and is a diagram viewed from a direction parallel to the center line. 9A and 9B are diagrams for explaining an example in which the shooting target area and the shooting direction are changed in accordance with the input of the input device, and are diagrams viewed from a direction perpendicular to the center line. 10A to 10D are diagrams for explaining an example in which a relay position is added between two photographing positions.

  FIG. 1 is a diagram illustrating an example in which an unmanned aerial vehicle (UAV8) including the camera 2 flies around the structure SR and photographs the structure SR. FIG. 2 is a flowchart for explaining an example of the structure inspection method according to the present embodiment. The structure SR to be inspected in the present embodiment is, for example, a structure built on the ground, such as a steel tower, a transmission line, and an exhaust tower. A UAV 8 such as a drone equipped with the camera 2 automatically flies around the structure SR based on the information on the flight path generated by the flight path generation device 1 described later, and photographs each part of the structure SR with the camera 2. I do. The UAV 8 includes a GNSS (global navigation satellite system) receiver, an acceleration sensor, an angular velocity sensor, and the like, and can fly autonomously while positioning its own geographical position. The UAV 8 sequentially moves to a plurality of photographing positions indicated by the given flight path information, and directs the camera 2 to a photographing direction set for each photographing position to photograph a photographing target region of the structure SR. The attitude of the camera 2 mounted on the UAV 8 can be controlled, and the shooting direction can be set to a desired direction by controlling the attitude of the camera 2. By examining the image of each part of the structure SR captured by the camera 2 of the UAV 8, it is possible to diagnose the state of damage or deterioration of the structure SR.

  In the structure inspection method according to the present embodiment, first, a step of acquiring three-dimensional shape data of the structure SR to be inspected is performed (ST100). For example, while the UAV 8 is flying at a relatively distant location around the structure SR, a plurality of images of the entire structure SR are captured. This shooting may be performed, for example, by manually manipulating the UAV 8, or may be performed by automatically flying the UAV 8 on a preset flight path. When images of a plurality of structures SR having different photographing positions and photographing directions are photographed, a three-dimensional model is formed from the photographed plural two-dimensional images using a known technique such as SfM (structure from motion). Generated.

  Note that the model of the three-dimensional shape of the structure SR can be generated using a three-dimensional measuring device such as a laser scanner.

  When the three-dimensional shape data of the structure SR is acquired, a step of generating a flight path of the UAV 8 by the later-described flight path generation device 1 using the three-dimensional shape data is performed (ST105). The flight path generation device 1 calculates a photographing position at which the UAV 8 should perform photographing and a posture of the camera 2 to be set at the photographing position based on the photographing target area and the photographing direction on the structure SR designated by the user. . When calculating a plurality of shooting positions based on a user's instruction, the flight path generating device 1 generates a flight path of the UAV 8 so as to pass through the plurality of shooting positions. The information on the flight path generated by the flight path generation device 1 includes information on the attitude of the camera 2 at each shooting position.

  When the flight path of the UAV 8 is generated in the flight path generation device 1, the UAV 8 is automatically flown around the structure SR based on the information of the flight path, and as shown in FIG. Is carried out (ST110). The photographing of the structure SR by the UAV 8 may be performed automatically or partially manually. For example, the UAV 8 that is automatically flying may be temporarily stopped at each shooting position, and the shooting may be manually performed after appropriately adjusting the zoom ratio and the like of the camera 2 while visually checking the image of the shooting target area.

  FIG. 3 is a diagram illustrating an example of a configuration of the flight path generation device 1 according to the present embodiment. The flight route generation device 1 illustrated in FIG. 3 includes a communication device 10, an input device 20, a display 30, a storage device 40, and a processing unit 50.

  The communication device 10 is a device that communicates with an external device according to a predetermined communication protocol, and communicates with, for example, a server connected to a network (LAN, the Internet, or the like). By using the communication device 10, for example, information (such as three-dimensional shape data of the structure SR) necessary for generating a flight route is obtained from a server or the like on a network, or the flight route generated by the processing unit 50 is obtained. It is possible to provide information to the UAV 8 and its control device.

  The input device 20 is a device for inputting information to the processing unit 50, and includes a device (a keyboard, a mouse, a touch pad, a touch panel, or the like) for inputting a user's instruction by operating the user. The input device 20 may include a reading device for a recording medium such as an optical disk, and an input / output interface device such as a USB.

  The display 30 is a device that displays an image under the control of the processing unit 50, and includes, for example, a liquid crystal display and an organic EL display.

  The storage device 40 stores a program 41 including an instruction code to be executed by the computer of the processing unit 50, data (such as three-dimensional shape data) used for the processing of the processing unit 50, It stores temporarily stored data and the like. The storage device 40 includes, for example, a hard disk device, a flash memory, a DRAM, an SRAM, an optical disk, and the like. Note that the program 41 may be recorded on a non-transitory recording medium such as a USB memory or an optical disk, input by an interface device or a reading device of the input device 20, and stored in the storage device 40.

  The processing unit 50 is a device that executes a process related to generation of a flight path, and includes, for example, one or more computers (microprocessors and the like) that execute a process according to an instruction code of a program 41 stored in the storage device 40. The processing unit 50 may execute all of the processing related to the generation of the flight path according to the instruction code of the program 41, or may execute at least a part of the processing using dedicated hardware (FPGA, ASIC).

  The processing unit 50 generates the three-dimensional image 9 based on the three-dimensional shape data of the structure SR (ST100: FIG. 2), and displays the generated three-dimensional image 9 on the display 30. The three-dimensional image 9 displayed on the display 30 may be, for example, a point group represented by three-dimensional shape data, or a captured image (a structure SR captured to acquire three-dimensional shape data) for a polygon obtained from the point group. May be an image on which texture mapping based on the image is performed.

  The processing unit 50 displays the above-described three-dimensional image 9 on the display 30 and displays the shooting target area on the structure SR and the shooting direction of the camera 2 toward the shooting target area so that the user can visually recognize the shooting target area. . In addition, the processing unit 50 changes a shooting target area and a shooting direction displayed on the display 30 according to a user's instruction input by operating a mouse or the like of the input device 20. Thus, the user can freely change and adjust the shooting target area and the shooting direction displayed on the display 30 by operating the mouse or the like.

  When an instruction to determine the shooting target area and the shooting direction displayed on the display 30 is input to the input device 20, the processing unit 50 determines the shooting position of the UAV 8 and the camera 2 based on the determined shooting target area and shooting direction. Is calculated.

  The processing unit 50 calculates a shooting position on the flight path of the UAV 8 so as to be separated from the virtual center line Ay that penetrates the structure SR in the vertical direction by a predetermined distance R1. That is, the processing unit 50 calculates the photographing position of the UAV 8 and the attitude of the camera 2 based on the decided photographing target area and photographing direction so as to satisfy the condition of being separated from the center line Ay by the distance R1. By appropriately increasing the distance R1 from the center line Ay, it is possible to prevent the photographing position from being set near the structure SR.

  FIG. 4 is a flowchart for explaining an example of an operation of generating a flight route of the UAV 8 in the flight route generation device 1 having the above-described configuration.

  The processing unit 50 displays the three-dimensional image 9 of the structure SR on the display 30 based on the three-dimensional shape data (ST100: FIG. 2) of the structure SR acquired in advance and is input to the input device 20. The display on the display 30 is updated according to the user's instruction (ST200).

  5 to 7 are diagrams illustrating an example of the three-dimensional image 9 of the structure SR displayed on the display 30. The processing unit 50 includes, in addition to the three-dimensional image 9 of the structure SR, a center line Ay penetrating the structure SR in the vertical direction, and an X axis Ax and a Z axis perpendicular to the center line Ay (also referred to as the Y axis). Az is displayed on the display 30. The X-axis Ax and the Z-axis Az are perpendicular to each other, and extend in directions parallel to the ground (for example, east-west and north-south). The origin at which the center line Ay (Y axis), the X axis Ax, and the Z axis Az intersect is set, for example, near the ground surface where the structure SR is installed. The processing unit 50 calculates a shooting position (latitude, longitude, and altitude) on the flight path of the UAV 8 based on the position of the origin (latitude, longitude, and altitude).

  For example, the position of the center line Ay may be manually set by the user with reference to the three-dimensional image 9 of the structure SR, or the processing unit 50 may automatically set the position based on the three-dimensional shape data of the structure SR. It may be calculated. Alternatively, after the processing unit 50 automatically calculates the position of the center line Ay, the user may be able to manually change the position.

  In addition, the processing unit 50 displays the first cylinder C1 and the second cylinder C2, which are virtual cylinders rotationally symmetric with respect to the center line Ay, on the display 30, as shown in FIGS.

  In the first cylinder C1, the radius of the circle having a cross section perpendicular to the center line Ay is “R1”, and is equal to the distance from the center line Ay to the photographing position. That is, the first cylinder C1 represents an area where the shooting position exists.

  On the other hand, the second cylinder C2 represents a range occupied by the structure SR in a horizontal direction (a direction perpendicular to the center line Ay). In the second cylinder C2, the radius of a circle having a cross section perpendicular to the center line Ay is “R2”, and is smaller than the radius “R1” of the first cylinder C1, as shown in FIG.

  The radius “R1” of the first cylinder C1 and the radius “R2” of the second cylinder C2 may be manually set by the user with reference to the three-dimensional image 9 of the structure SR, or may be set manually. The processing unit 50 may automatically calculate based on the three-dimensional shape data. Alternatively, after the processing unit 50 automatically calculates “R1” and “R2”, the user may be able to manually change those values.

  When the user manually sets “R1” and “R2”, the processing unit 50 determines that the difference “R1−R2” between “R1” and “R2” does not become smaller than a predetermined minimum value (for example, 5 m). Thus, the settable range of “R1” and “R2” may be limited. That is, when changing at least one of “R1” and “R2” in accordance with the user's instruction input to the input device 20, the processing unit 50 ensures that “R1−R2” does not become less than a predetermined minimum value. , “R1” and “R2” may be restricted. Thereby, it is possible to effectively prevent generation of a flight path in which the UAV 8 is too close to the horizontal range (the second cylinder C2) of the structure SR.

  Further, as shown in FIG. 5, the processing unit 50 displays the upper limit line L1 indicating the upper limit of the flight altitude of the UAV 8 and the lower limit line L2 indicating the lower limit of the flight altitude of the UAV 8 on the display 30. The processing unit 50 limits the height of the calculated shooting position so that the height of the UAV 8 is included between the height H1 of the upper limit line L1 and the height H2 of the lower limit line L2. Thus, it is possible to effectively prevent the UAV 8 from colliding with an obstacle located at a position higher than the altitude H1 on the upper limit line L1 or from falling too close to the ground surface.

  The height H1 of the upper limit line L1 and the height H2 of the lower limit line L2 may be manually set by a user based on, for example, an image of the structure SR and its surroundings, or the processing unit 50 may set the altitude H2 based on those images. May be calculated automatically. Alternatively, after the processing unit 50 automatically calculates the altitude H1 and the altitude H2, the user may manually change the values.

  In addition, the processing unit 50 displays the second cylinder C2 on the display 30 as shown in FIGS. 5 to 7, and marks the target area mark Md on the second cylinder C2 at a location overlapping with the imaging range of the camera 2. Place. When the processing unit 50 changes the shooting target area and the shooting direction in accordance with an instruction input by operating the mouse or the like of the input device 20, the position and the structure of the mark Md of the shooting target area on the screen of the display 30 The three-dimensional image 9 is changed such that at least one of the viewpoint and the line of sight with respect to the structure SR is changed while maintaining the relative relationship between the direction of the line of sight with respect to the structure SR and the imaging direction of the camera 2 with respect to the structure SR.

  In the examples of FIGS. 5 to 7, the positions of the marks Md of the target regions on the screen of the display 30 are kept the same, although the target regions on the structure SR represented by the marks Md are different from each other. That is, the processing unit 50 displays the three-dimensional image 9 of the structure SR in which the viewpoint and the direction of the line of sight have been changed while maintaining the position of the mark Md on the screen.

  In the examples of FIGS. 5 to 7, the shooting directions of the camera 2 with respect to the structure SR are different from each other, but the direction of the line of sight with respect to the structure SR and the shooting direction of the camera 2 are all kept the same. That is, the processing unit 50 displays, on the display 30, the three-dimensional image 9 of the structure SR in which the viewpoint and the direction of the line of sight have been changed while matching the direction of the line of sight with respect to the structure SR with the shooting direction of the camera 2.

  Note that maintaining the relative relationship between the direction of the line of sight to the structure SR and the shooting direction of the camera 2 with respect to the structure SR is performed by changing the direction of the line of sight and the shooting direction as shown in the examples of FIGS. It is not limited to matching. For example, the processing unit 50 may maintain a relative relationship between the direction of the line of sight and the shooting direction by maintaining the same angle.

  When the processing unit 50 changes the shooting target area and the shooting direction as shown in FIGS. 5 to 7 according to the instruction input to the input device 20, the processing target area and the shooting direction are, for example, by the following two methods. To change.

(First method for changing the shooting target area and the shooting direction)
In the first method, the processing unit 50 changes the azimuth of the shooting position as viewed from the center line Ay while directing the shooting direction to the same position on the center line Ay. FIG. 8 is a view for explaining an example in which the shooting target area and the shooting direction are changed by the first method, and is a view seen from a direction parallel to the center line Ay. Specifically, the azimuth of the shooting position is the direction of the shooting position with respect to the center line Ay when viewed from a direction parallel to the center line Ay as shown in FIG. In the example of FIG. 8, three shooting positions having different directions with respect to the center line Ay are shown. For example, the processing unit 50 moves the photographing position of the UAV 8 to a different position at the same altitude on the first cylinder C1 and directs the photographing direction 3 of the camera 2 to the same position on the center line Ay. When the photographing target area and the photographing direction are changed by the first method, as shown in FIG. 8, the position of the viewpoint 5 rotates around the center line Ay together with the photographing position (the position of the camera 2).

(Second method for changing shooting target area and shooting direction)
In the second method, the processing unit 50 turns the shooting direction to a different position on the center line Ay while maintaining the orientation of the shooting position viewed from the center line Ay. FIGS. 9A and 9B are diagrams for explaining an example in which the imaging target area and the imaging direction are changed by the second method, and are diagrams viewed from a direction perpendicular to the center line Ay.

  The second method includes two more methods. In the example of FIG. 9A, the processing unit 50 directs the photographing direction 3 to a different position on the center line Ay while fixing the photographing position at the same position. In this case, the processing unit 50 changes the angle of the photographing direction 3 with respect to the center line Ay. In the example of FIG. 9B, the processing unit 50 moves the photographing position in the vertical direction while keeping the same angle of the photographing direction 3 with respect to the center line Ay (moves the photographing position in parallel with the center line Ay).

  The user can freely set the photographing target area and the photographing direction displayed on the display 30 by appropriately using the first method and the second method described above.

Referring back to FIG.
Since the shooting target area and the shooting direction are set as intended by the user by the above-described method, the user's instruction to determine the shooting target area and the shooting direction (for example, an instruction by pressing a determination button on the screen) is input device 20. (YES in ST210), the processing unit 50 calculates the photographing position of the UAV 8, the posture of the camera 2, and the posture of the UAV 8 based on the decided photographing target area and photographing direction (ST215). .

  The attitude of the UAV8 is, for example, an attitude related to the horizontal orientation of the UAV8, and the processing unit 50 calculates the attitude of the UAV8 so that the reference horizontal direction of the UAV8 body always faces the center line Ay. In this case, by matching the reference horizontal direction with the horizontal direction of the camera 2, the attitude of the camera 2 in the above-described first method and second method can be determined only by the tilt angle of the camera 2 with respect to the vertical direction. You only have to adjust it. Therefore, even if the camera 2 mounted on the UAV 8 can adjust only the tilt angle with respect to the vertical direction, the shooting target area and the shooting direction can be arbitrarily set by the above-described first and second methods. It is possible.

  When calculating the shooting position of the UAV 8 based on the determined shooting target area and shooting direction, the processing unit 50 sets the order in which the UAV 8 passes through the shooting position. For example, the processing unit 50 sets the passing order of the UAV 8 at each shooting position according to the order in which the shooting positions of the UAV 8 are calculated based on the determined shooting target area and the shooting direction. That is, the processing unit 50 sets the passage order of the Nth (N is a natural number) calculated photographing position to the Nth.

  When calculating the shooting position of the UAV 8, the processing unit 50 arranges the mark Ms of the calculated shooting position on the first cylinder C1, as shown in FIGS. 5 to 7 (ST220). “Ms (1)”, “Ms (2)”, and “Ms (3)” in FIGS. 5 to 7 are marks indicating the first, second, and third shooting positions, respectively. The mark Ms (2) of the second shooting position shown in FIG. 6 indicates a shooting position calculated when the shooting target area and the shooting direction are determined in the state shown in FIG. The mark Ms (3) of the third shooting position shown in FIG. 7 indicates a shooting position calculated when the shooting target area and the shooting direction are determined in the state shown in FIG.

  When the shooting position calculated in step ST215 is the second or later shooting position (Yes in ST225), the processing unit 50 determines whether the center angle θ is larger than the predetermined angle TH at the two shooting positions adjacent to each other in the passing order. (ST230). Here, the center angle θ is an angle formed by two straight lines connecting the two cylindrical positions of the first cylinder C1 and the center line Ay as viewed in a direction parallel to the center line Ay. That is, the processing unit 50 determines the angle formed by the two straight lines connecting the two photographing positions adjacent to each other and the center line Ay, and the angle formed by the two straight lines viewed from a direction parallel to the center line Ay. It is determined whether (center angle θ) is greater than a predetermined angle TH.

  When the center angle θ is greater than the predetermined angle TH at two adjacent photographing positions in the passing order (Yes in ST230), the processing unit 50 determines that one or more relay positions through which the UAV 8 passes in the flight path between the two photographing positions. Is calculated (ST235). Here, the relay position is a passage position of the UAV 8 that is added to form a flight path that bypasses the structure SR between the two shooting positions. The processing unit 50 calculates a position separated from the center line Ay by the distance R1, that is, one or more relay positions on the first cylinder C1. The one or more relay positions calculated by the processing unit 50 equally divide the flight path between the two shooting positions into two or more equal-length sections. In each of two or more sections equally divided by one or more relay positions, the center angle θ is smaller than a predetermined angle TH.

  When calculating one or more relay positions on the flight path between the two photographing positions, processing unit 50 arranges marks Mr of the calculated one or more relay positions on first cylinder C1 (ST240).

  FIGS. 10A to 10D are diagrams for explaining an example in which a relay position is added between two photographing positions in which the passing order of the UAV 8 is adjacent, and a diagram viewed from a direction parallel to the center line Ay. Show. 10A to 10D, the processing unit 50 calculates the relay position (Mr) when the central angle θ at the two shooting positions (Ms) is larger than 30 degrees.

When the two photographing positions (Ms) in which the central angle θ is greater than 30 degrees are calculated as illustrated in FIG. 10A, the processing unit 50 calculates five relay positions (Mr) as illustrated in FIG. The two photographing positions (Ms) are equally divided into six sections by the relay position (Mr). In each of the six equally divided sections, the central angle θ is smaller than 30 degrees.
10C, the processing unit 50 also calculates five relay positions (Mr) as shown in FIG. 10D. The five relay positions (Mr) equally divide the two shooting positions (Ms) into six sections. In each of the six equally divided sections, the central angle θ is smaller than 30 degrees.

  As can be seen by comparing FIG. 10B and FIG. 10D, the processing unit 50 determines that the center angle θ between the two shooting positions is within an angle range smaller than 180 degrees (in the example of FIG. One or more relay positions are calculated within the left angle range in the example of 10D.

  When the imaging position forming the flight path is calculated as intended by the user, and a user's instruction to end the processing of generating the flight path is input on the input device 20 (Yes in ST205), the processing unit 50 The information of the flight path (the photographing position, the attitude of the camera 2 and the attitude of the UAV 8) generated so far is converted into data of a predetermined format such as a KML file and stored in the storage device 40 (ST250).

  As described above, according to the present embodiment, the display 30 displays the three-dimensional image 9 of the structure SR, the shooting target region of the structure SR, and the shooting direction of the camera 2 toward the shooting target region. . In addition, in accordance with an instruction input to the input device 20, the shooting target area and the shooting direction displayed on the display 30 are changed. Then, when an instruction for determining the shooting target area and the shooting direction is input to the input device 20, the shooting position at a predetermined distance R1 from the virtual center line Ay that penetrates the structure SR in the vertical direction, and the shooting position The posture of the camera 2 for photographing the structure SR is calculated based on the decided photographing target region and photographing direction. Therefore, a flight path for automatically flying the UAV 8 and photographing the structure SR can be easily generated by a user input operation.

  Further, according to the present embodiment, when the passing order of the UAV 8 is such that the center angle θ is larger than the predetermined angle TH at two adjacent photographing positions, one or more relays through which the UAV 8 passes in the flight path between the two photographing positions. The position is calculated. The one or more relay positions are on the first cylinder C1 separated by a predetermined distance R1 from the center line Ay, and equally divide the flight path between the two photographing positions into two or more equal-length sections. In each of the two or more sections, the center angle θ is smaller than the predetermined angle TH. The larger the central angle θ between the two photographing positions, the easier the flight path connecting the two photographing positions to approach the structure SR. By calculating one or more relay positions in this way, the distance between the two photographing positions can be increased. The flight path can be kept away from the structure SR. Therefore, it is possible to effectively prevent the UAV 8 from colliding with the structure SR.

  Furthermore, according to the present embodiment, when the shooting target area and the shooting direction are changed in accordance with an instruction input to the input device 20, the position of the mark Md of the shooting target area on the screen of the display 30 and the structure The three-dimensional image 9 of the display 30 is changed such that at least one of the viewpoint and the line of sight with respect to the structure SR is changed while maintaining the relative relationship between the direction of the line of sight with respect to the SR and the shooting direction of the camera 2 with respect to the structure SR. Is done. This makes it easier for the user to intuitively grasp the photographing target area and the photographing direction displayed on the display 30, so that the flight path can be more easily generated.

  Although the present embodiment has been described above, the technology of the present disclosure is not limited to the above-described embodiment, and includes various variations.

  For example, in the above-described embodiment, an example is given in which the passing order of the UAVs 8 at a plurality of shooting positions is set in accordance with the order in which the shooting positions are calculated, but the passing order may be set by another method. For example, after calculating a plurality of photographing positions, the passing order may be set so that the center angle θ with respect to the center line Ay becomes smaller. This makes it difficult to calculate the relay position, so that the flight path of the UAV 8 can be shortened.

  In the shooting target area and the shooting direction changing direction (second method) illustrated in FIG. 9A, the shooting direction 3 is directed to a different position on the center line Ay while the shooting position is fixed at the same position. The technology of the present disclosure is not limited to this example. That is, the processing unit 50 may direct the photographing direction 3 not only in the direction toward the center line Ay but also in other directions while the photographing position is fixed at the same place. This makes it easier to photograph the photographing target area of the structure 9 from various directions, and thus makes it easier to photograph an appropriate image for inspection of the structure 9.

  Hereinafter, additional notes related to the technology of the present disclosure will be described.

[Appendix 1]
A flight path generation device that generates a flight path for an unmanned aerial vehicle equipped with a camera to photograph a structure,
An input device for inputting a user instruction;
Display and
A processing unit that performs a process for generating the flight path,
The processing unit includes:
On the display, a three-dimensional image of the structure, a shooting target area on the structure, and a shooting direction of the camera toward the shooting target area are displayed,
According to the instruction input to the input device, change the shooting target area and the shooting direction,
When an instruction for determining the imaging target area and the imaging direction is input to the input device, the imaging position is a predetermined distance from a virtual center line vertically penetrating the structure, and the flight path The above photographing position and the posture of the camera that photographs the structure at the photographing position are calculated based on the decided photographing target area and the photographing direction,
Flight path generator.

[Appendix 2]
A virtual cylinder that is rotationally symmetric with respect to the center line, wherein the virtual cylinder whose radius as viewed from a direction parallel to the center line is equal to the predetermined distance is referred to as a first cylinder,
An angle formed by two straight lines connecting the two positions on the first cylinder and the center line as viewed from a direction parallel to the center line is referred to as a center angle,
The processing unit includes:
When calculating the plurality of shooting positions, set a passing order in which the unmanned aerial vehicle passes through the plurality of shooting positions,
When the passing angle is larger than a predetermined angle at the two adjacent shooting positions, the passing order is one or more relay positions through which the unmanned aerial vehicle passes on the flight path between the two shooting positions, Calculating the one or more relay positions on the first cylinder;
The one or more relay positions equally divide the flight path between the two shooting positions into two or more sections;
In each of the two or more sections, the central angle is smaller than the predetermined angle;
The flight path generation device according to claim 1.

[Appendix 3]
The processing unit calculates the one or more relay positions within an angle range in which the central angle of the two shooting positions is smaller than 180 degrees.
The flight path generation device according to attachment 2.

[Appendix 4]
The processing unit sets the passing order according to the order in which the shooting positions are calculated,
The flight path generation device according to attachment 2 or 3.

[Appendix 5]
The processing unit includes:
Displaying the first cylinder on the display;
When calculating one or more shooting positions, a mark of the calculated one or more shooting positions is arranged on the first cylinder,
When calculating one or more relay positions, a mark of the calculated one or more relay positions is arranged on the first cylinder;
The flight path generation device according to any one of supplementary notes 2 to 4.

[Appendix 6]
A virtual cylinder that is rotationally symmetric with respect to the center line and that represents a range occupied by the structure in the horizontal direction is referred to as a second cylinder;
The processing unit includes:
Displaying the second cylinder on the display, and placing a mark of the imaging target area at a location on the second cylinder that overlaps the imaging range of the camera,
When changing the shooting target area and the shooting direction in accordance with an instruction input to the input device, the position of the mark of the shooting target area on the screen of the display, and the direction of the line of sight to the structure and the Changing the three-dimensional image so that at least one of the viewpoint and the direction of the line of sight to the structure changes while maintaining a relative relationship between the camera and the shooting direction with respect to the structure.
The flight path generation device according to any one of supplementary notes 1 to 4.

[Appendix 7]
The processing unit, when changing the shooting target area and the shooting direction according to an instruction input to the input device, the shooting as viewed from the center line while directing the shooting direction to the same position on the center line. Change the orientation of the position, or direct the shooting direction to a different position on the center line while maintaining the orientation of the shooting position,
The flight path generation device according to attachment 6.

[Appendix 8]
A flight path generation method in which a computer generates a flight path for an unmanned aerial vehicle equipped with a camera to photograph a structure,
On a display, a three-dimensional image of the structure, a shooting target area on the structure, and a shooting direction of the camera toward the shooting target area are displayed,
According to the instruction input to the input device by the user, change the shooting target area and the shooting direction,
When an instruction for determining the imaging target area and the imaging direction is input to the input device, the imaging position is a predetermined distance from a virtual center line vertically penetrating the structure, and the flight path The above photographing position and the posture of the camera that photographs the structure at the photographing position are calculated based on the decided photographing target area and the photographing direction,
Flight path generation method.

[Appendix 9]
A virtual cylinder that is rotationally symmetric with respect to the center line, wherein the virtual cylinder whose radius as viewed from a direction parallel to the center line is equal to the predetermined distance is referred to as a first cylinder,
An angle formed by two straight lines connecting the two positions on the first cylinder and the center line as viewed from a direction parallel to the center line is referred to as a center angle,
When calculating the plurality of shooting positions, set a passing order in which the unmanned aerial vehicle passes through the plurality of shooting positions,
When the passing angle is larger than a predetermined angle at the two adjacent shooting positions, the passing order is one or more relay positions through which the unmanned aerial vehicle passes on the flight path between the two shooting positions, Calculating the one or more relay positions on the first cylinder;
The one or more relay positions equally divide the flight path between the two shooting positions into two or more sections;
In each of the two or more sections, the central angle is smaller than the predetermined angle;
A flight route generation method according to attachment 8.

[Appendix 10]
When calculating the one or more relay positions, the one or more relay positions are calculated within an angle range in which the central angle of the two shooting positions is smaller than 180 degrees.
A flight route generation method according to attachment 9.

[Appendix 11]
When setting the passing order, the passing order is set according to the order in which the shooting positions are calculated,
The flight route generation method according to attachment 9 or 10.

[Supplementary Note 12]
Displaying the first cylinder on the display;
When calculating one or more shooting positions, a mark of the calculated one or more shooting positions is arranged on the first cylinder,
When calculating one or more relay positions, a mark of the calculated one or more relay positions is arranged on the first cylinder;
The flight route generation method according to any one of supplementary notes 9 to 11.

[Appendix 13]
A virtual cylinder that is rotationally symmetric with respect to the center line and that represents a range occupied by the structure in the horizontal direction is referred to as a second cylinder;
Displaying the second cylinder on the display, and placing a mark of the imaging target area at a location on the second cylinder that overlaps the imaging range of the camera,
When changing the shooting target area and the shooting direction in accordance with an instruction input to the input device, the position of the mark of the shooting target area on the screen of the display, and the direction of the line of sight to the structure and the Changing the three-dimensional image so that at least one of the viewpoint and the direction of the line of sight to the structure changes while maintaining a relative relationship between the camera and the shooting direction with respect to the structure.
13. The flight path generation method according to any one of supplementary notes 8 to 12.

[Appendix 14]
When changing the shooting target area and the shooting direction according to an instruction input to the input device, change the orientation of the shooting position viewed from the center line while directing the shooting direction to the same position on the center line. Or, or direct the shooting direction to a different position on the center line while maintaining the orientation of the shooting position,
A flight route generation method according to attachment 13.

[Appendix 15]
A program for causing a computer to execute the flight path generation method according to any one of Supplementary Notes 8 to 14.

[Appendix 16]
The flight route is generated by the flight route generation method according to any one of supplementary notes 8 to 14,
Automatically fly the unmanned aerial vehicle around the structure based on the generated flight path, and photograph each of the imaging target areas of the structure.
Structure inspection method.

DESCRIPTION OF SYMBOLS 1 ... Flight path generation apparatus, 2 ... Camera, 3 ... Photographing direction, 5 ... Viewpoint, 8 ... Unmanned aerial vehicle (UAV), 9 ... 3D image, 10 ... Communication device, 20 ... Input device, 30 ... Display, 40 ... Storage device, 41 program, 50 processing unit, SR structure, C1 first cylinder, C2 second cylinder, L1 upper limit line, L2 lower limit line, Ay center line (Y axis), Ax X-axis, Az: Z-axis, Md: mark of shooting target area, Ms: mark of shooting position, Mr: mark of relay position, θ: center angle

Claims (16)

A flight path generation device that generates a flight path for an unmanned aerial vehicle equipped with a camera to photograph a structure,
An input device for inputting a user instruction;
Display and
A processing unit that performs a process for generating the flight path,
The processing unit includes:
On the display, a three-dimensional image of the structure, a shooting target area on the structure, and a shooting direction of the camera toward the shooting target area are displayed,
According to the instruction input to the input device, change the shooting target area and the shooting direction,
When an instruction for determining the imaging target area and the imaging direction is input to the input device, the imaging position is a predetermined distance from a virtual center line vertically penetrating the structure, and the flight path The above photographing position and the posture of the camera that photographs the structure at the photographing position are calculated based on the decided photographing target area and the photographing direction ,
A virtual cylinder that is rotationally symmetric with respect to the center line, wherein the virtual cylinder whose radius as viewed from a direction parallel to the center line is equal to the predetermined distance is referred to as a first cylinder,
An angle formed by two straight lines connecting the two positions on the first cylinder and the center line as viewed from a direction parallel to the center line is referred to as a center angle,
The processing unit includes:
When calculating the plurality of shooting positions, set a passing order in which the unmanned aerial vehicle passes through the plurality of shooting positions,
When the passing angle is larger than a predetermined angle at the two adjacent shooting positions, the passing order is one or more relay positions through which the unmanned aerial vehicle passes on the flight path between the two shooting positions, Calculating the one or more relay positions on the first cylinder;
The one or more relay positions equally divide the flight path between the two shooting positions into two or more sections;
In each of the two or more sections, the central angle is smaller than the predetermined angle;
Flight path generator. The processing unit calculates the one or more relay positions within an angle range in which the central angle of the two shooting positions is smaller than 180 degrees.
The flight path generation device according to claim 1 . The processing unit sets the passing order according to the order in which the shooting positions are calculated,
The flight path generation device according to claim 1 . The processing unit includes:
Displaying the first cylinder on the display;
When calculating one or more shooting positions, a mark of the calculated one or more shooting positions is arranged on the first cylinder,
When calculating one or more relay positions, a mark of the calculated one or more relay positions is arranged on the first cylinder;
The flight path generation device according to claim 1. A virtual cylinder that is rotationally symmetric with respect to the center line and that represents a range occupied by the structure in the horizontal direction is referred to as a second cylinder;
The processing unit includes:
Displaying the second cylinder on the display, and placing a mark of the imaging target area at a location on the second cylinder that overlaps the imaging range of the camera,
When changing the shooting target area and the shooting direction in accordance with an instruction input to the input device, the position of the mark of the shooting target area on the screen of the display, and the direction of the line of sight to the structure and the Changing the three-dimensional image so that at least one of the viewpoint and the direction of the line of sight to the structure changes while maintaining a relative relationship between the camera and the shooting direction with respect to the structure.
Flight path generation device according to any one of claims 1-4. A flight path generation device that generates a flight path for an unmanned aerial vehicle equipped with a camera to photograph a structure,
An input device for inputting a user instruction;
Display and
A processing unit that performs a process for generating the flight path,
The processing unit includes:
On the display, a three-dimensional image of the structure, a shooting target area on the structure, and a shooting direction of the camera toward the shooting target area are displayed,
According to the instruction input to the input device, change the shooting target area and the shooting direction,
When an instruction for determining the imaging target area and the imaging direction is input to the input device, the imaging position is a predetermined distance from a virtual center line vertically penetrating the structure, and the flight path The above photographing position and the posture of the camera that photographs the structure at the photographing position are calculated based on the decided photographing target area and the photographing direction ,
A virtual cylinder that is rotationally symmetric with respect to the center line and that represents a range occupied by the structure in the horizontal direction is referred to as a second cylinder;
The processing unit includes:
Displaying the second cylinder on the display, and placing a mark of the imaging target area at a location on the second cylinder that overlaps the imaging range of the camera,
When changing the shooting target area and the shooting direction in accordance with an instruction input to the input device, the position of the mark of the shooting target area on the screen of the display, and the direction of the line of sight to the structure and the Changing the three-dimensional image so that at least one of the viewpoint and the direction of the line of sight to the structure changes while maintaining a relative relationship between the camera and the shooting direction with respect to the structure.
Flight path generator. The processing unit, when changing the shooting target area and the shooting direction according to an instruction input to the input device, the shooting as viewed from the center line while directing the shooting direction to the same position on the center line. Change the orientation of the position, or direct the shooting direction to a different position on the center line while maintaining the orientation of the shooting position,
The flight path generation device according to claim 5 . A flight path generation method in which a computer generates a flight path for an unmanned aerial vehicle equipped with a camera to photograph a structure,
On a display, a three-dimensional image of the structure, a shooting target area on the structure, and a shooting direction of the camera toward the shooting target area are displayed,
According to the instruction input to the input device by the user, change the shooting target area and the shooting direction,
When an instruction for determining the imaging target area and the imaging direction is input to the input device, the imaging position is at a predetermined distance from a virtual center line vertically penetrating the structure, and the flight path The above photographing position and the posture of the camera that photographs the structure at the photographing position are calculated based on the decided photographing target area and the photographing direction ,
A virtual cylinder that is rotationally symmetric with respect to the center line, wherein the virtual cylinder whose radius as viewed from a direction parallel to the center line is equal to the predetermined distance is referred to as a first cylinder,
An angle formed by two straight lines connecting the two positions on the first cylinder and the center line as viewed from a direction parallel to the center line is referred to as a center angle,
When calculating the plurality of shooting positions, set a passing order in which the unmanned aerial vehicle passes through the plurality of shooting positions,
In the case where the center angle is larger than a predetermined angle at the two adjacent photographing positions, the passing order is one or more relay positions through which the unmanned aerial vehicle passes on the flight path between the two photographing positions, Calculating the one or more relay positions on the first cylinder;
The one or more relay positions equally divide the flight path between the two shooting positions into two or more sections;
In each of the two or more sections, the central angle is smaller than the predetermined angle;
Flight path generation method. When calculating the one or more relay positions, the one or more relay positions are calculated within an angle range in which the central angle of the two shooting positions is smaller than 180 degrees.
The flight path generation method according to claim 8 . When setting the passing order, the passing order is set according to the order in which the shooting positions are calculated,
The flight path generation method according to claim 8 . Displaying the first cylinder on the display;
When calculating one or more shooting positions, a mark of the calculated one or more shooting positions is arranged on the first cylinder,
When calculating one or more relay positions, a mark of the calculated one or more relay positions is arranged on the first cylinder;
The flight path generation method according to claim 8 . A virtual cylinder that is rotationally symmetric with respect to the center line and that represents a range occupied by the structure in the horizontal direction is referred to as a second cylinder;
Displaying the second cylinder on the display, and placing a mark of the imaging target area at a location on the second cylinder that overlaps the imaging range of the camera,
When changing the shooting target area and the shooting direction in accordance with an instruction input to the input device, the position of the mark of the shooting target area on the screen of the display, and the direction of the line of sight to the structure and the Changing the three-dimensional image so that at least one of the viewpoint and the direction of the line of sight to the structure changes while maintaining a relative relationship between the camera and the shooting direction with respect to the structure.
The flight path generation method according to claim 8 . A flight path generation method in which a computer generates a flight path for an unmanned aerial vehicle equipped with a camera to photograph a structure,
On a display, a three-dimensional image of the structure, a shooting target area on the structure, and a shooting direction of the camera toward the shooting target area are displayed,
According to the instruction input to the input device by the user, change the shooting target area and the shooting direction,
When an instruction for determining the imaging target area and the imaging direction is input to the input device, the imaging position is a predetermined distance from a virtual center line vertically penetrating the structure, and the flight path The above photographing position and the posture of the camera that photographs the structure at the photographing position are calculated based on the decided photographing target area and the photographing direction ,
A virtual cylinder that is rotationally symmetric with respect to the center line and that represents a range occupied by the structure in the horizontal direction is referred to as a second cylinder;
Displaying the second cylinder on the display, and placing a mark of the imaging target area at a location on the second cylinder that overlaps the imaging range of the camera,
When changing the shooting target area and the shooting direction in accordance with an instruction input to the input device, the position of the mark of the shooting target area on the screen of the display, and the direction of the line of sight to the structure and the Changing the three-dimensional image so that at least one of the viewpoint and the direction of the line of sight to the structure changes while maintaining a relative relationship between the camera and the shooting direction with respect to the structure.
Flight path generation method. When changing the shooting target area and the shooting direction according to an instruction input to the input device, change the orientation of the shooting position viewed from the center line while directing the shooting direction to the same position on the center line. Or, or direct the shooting direction to a different position on the center line while maintaining the orientation of the shooting position,
The flight path generation method according to claim 12 .   A program for causing a computer to execute the flight path generation method according to any one of claims 8 to 14. The flight path is generated by the flight path generation method according to any one of claims 8 to 14,
Automatically fly the unmanned aerial vehicle around the structure based on the generated flight path, and photograph each of the imaging target areas of the structure.
Structure inspection method.

Images