JP2020135327A - Flight body system, flight body, position measuring method and program - Google Patents

Abstract

To provide a flight body or the like capable of measuring a correct position of a structure.SOLUTION: A flight body system 100 for imaging a structure from the upper air by a flight body 10 having an imaging unit 13, comprises: a current place acquisition part 57 for acquiring a current position where the flight body flies; a structure detection part 76 for detecting the structure from image data imaged by the imaging unit; an area information detection part 75 for detecting area information in the image data of the structure detected by the structure detection part; a position determination part 74 for determining a position of the flight body when area information on the image data of the structure detected by the area information detection part can be determined to be minimum; and a position recording part 71 for storing a position of the flight body determined by the position determination part, as the position of the structure.SELECTED DRAWING: Figure 10

Description

The present invention relates to an air vehicle system, an air vehicle, a position measurement method, and a program.

Flying objects such as drones are used for inspection of structures such as steel towers, topographical surveys of places where people are difficult to enter such as high places and cliffs, and disaster surveys. The user sets the position information of the structure etc. on the flying object, automatically flies to the structure, and makes the image pickup device mounted on the flying object image the structure etc., so that the inspection and investigation of the structure etc. can be done at low cost. And it can be done safely. The flying object has an automatic tracking function that captures images while pointing the optical axis of the imaging device toward the structure for which position information is set, and images the structure without the user having to operate the orientation of the imaging device during automatic navigation. It is possible to do.

A technique for detecting a specific object using such an air vehicle has been devised (see, for example, Patent Document 1). Patent Document 1 discloses an information identification system that divides an imaging range into areas and reads out information on objects existing in the areas to narrow down candidates for objects in the image.

JP-A-2017-58831

However, with the conventional technique, there is a problem that it is difficult to measure the accurate position of the structure. First, the flying object flies while grasping its position with GNSS (Global Navigation Satellite System), but since it is difficult to easily obtain accurate position information of the structure, the structure is moved from a fixed position. It was difficult to image. This will be described with reference to FIG.

FIG. 1 is a diagram for explaining the deviation between the position information of the map data and the actual position of the steel tower 11 at the time of imaging of the steel tower 11 which is an example of the structure. As shown in FIG. 1A, the latitude and longitude of the steel tower 11 are registered as position information in the map data. However, it is common that there is a slight discrepancy between the position information of the map data and the actual position. When the flying object flies based on the latitude and longitude of the structure described in the map data, the flying object 10 cannot reach directly above the steel tower 11 as shown in FIG. 1 (b). For example, in a use case where the tower 11 is inspected from a fixed position directly above the tower 11, or in a use case where an overhead ground wire or an electric wire is inspected from directly above the tower 11 to another tower 11, the position is fixed. If it cannot be reached, the inspection cannot be started. Therefore, the exact location of the structure is needed.

In this case, the user needs to operate the transmitter (also referred to as a radio) while visually observing the flying object 10 and the captured image data to control the position of the flying object 10 from a remote location (or the corrected position). You will need to enter the information). As shown in FIG. 1C, the flying object 10 can image the tower 11 from directly above the tower 11 by the user's control, but the work efficiency is lowered.

As shown in FIG. 1D, the exact position of the tower 11 can be measured by on-site survey using the total station 12. However, it is difficult to adopt because the work load is large and the cost increases.

In view of the above problems, an object of the present invention is to provide an air vehicle or the like capable of measuring an accurate position of a structure.

In view of the above problems, the present invention is an air vehicle system that images a structure from the sky by an air vehicle having an image pickup device, the current location acquisition unit that acquires the current position of flight, and the image pickup device. The structure detection unit that detects the structure from the image data, the area information detection unit that detects the area information in the image data of the structure detected by the structure detection unit, and the area information detection unit detect the structure. The position determining unit for determining the position of the flying object when the area information in the image data of the structure can be regarded as the minimum, and the position of the flying object determined by the positioning unit are used as the positions of the structure. It is characterized by having a position recording unit for storing.

It is possible to provide an air vehicle or the like capable of measuring the accurate position of a structure.

It is a figure explaining the deviation of the position information of the map data at the time of imaging of the steel tower which is an example of a structure, and the actual position of a steel tower. This is an example of an external view of an air vehicle. This is an example of a system configuration diagram of an air vehicle system. It is a figure which shows an example of the setting screen of the flight condition of an air vehicle. It is a figure which shows typically the flight plan set in this embodiment. This is an example of a functional block diagram showing the functions of an information processing device, a transmitter, and an air vehicle in a block shape. It is a figure explaining the method of determining the azimuth based on the relative position of an air vehicle and a waypoint. It is a figure explaining the relationship between the behavior of an air vehicle and the rotation direction of four rotors. This is an example of a functional block diagram showing the functions of the flight controller in a block shape. It is a figure which shows the image image example of the steel tower with respect to the relative position of a flying object and a steel tower. It is a figure which shows an example of the method which the position determination part determines the position which minimizes the area of a structure. This is an example of a diagram for explaining a method of determining whether or not the structure is point-symmetrical. This is an example of a flowchart showing a procedure in which an air vehicle measures an accurate position of a structure.

Hereinafter, as an example of the embodiment for carrying out the present invention, the flying object system and the position measuring method performed by the flying object system will be described.

<Terminology>
The structure mainly refers to a building, but may include a natural object. The objects to be inspected and investigated are structures. For example, in addition to steel towers, buildings, utility poles, etc. can be mentioned, but are not limited to these. The structure of the present embodiment may be smaller in size when viewed from directly above than in size when viewed from the side.

In the present embodiment, the moving image is mainly referred to as an image, and the frame of the image is referred to as image data, but since the image data is a part of the image, it is not strictly distinguished.

The area information of the structure in the image data may be information that shows not only the area itself but also the size of the structure in the image data, and includes, for example, the ratio of the structure in the image data.

<About the basic functions of the flying object>
In the present embodiment, a drone will be mainly assumed as an example of the flying object 10. A drone is an airplane, rotorcraft, glider, or airship that cannot be boarded by a person due to its structure and can be flown by remote control or autopilot. Sometimes called a multicopter or radio-controlled model. In this embodiment, it is preferable that the rotorcraft is capable of hovering.

FIG. 2 is an example of an external view of the flying object 10. The aircraft body 10 of FIG. 2 is a drone of the type called a rotorcraft and has four rotors. The number of rotors varies depending on the model and application, and some are equipped with 3, 6 or 8 rotors. The number of rotors may be any number in this embodiment.

Further, an imaging device 13 is fixed to the bottom surface of the flying object 10 via a 3-axis gimbal. The gimbal is a mechanism that keeps the orientation of the image pickup apparatus 13 constant even if the posture of the main body changes. As a result, it is possible to prevent the image from tilting even if the attitude of the flying object 10 is tilted. Further, since the gimbal can change the direction of the image pickup device 13, the gimbal can always point the optical axis at the structure and take an image even during flight.

FIG. 3 shows an example of a system configuration diagram of the air vehicle system 100. The air vehicle system 100 includes a transmitter 20, an air vehicle 10, and an information processing device 40. First, the transmitter 20 and the air vehicle 10 can communicate wirelessly, and the user can control the air vehicle 10 from a remote location. In many cases, the transmitter 20 and the air vehicle 10 are distributed integrally, but in the case of the air vehicle 10 that can be operated by the general-purpose transmitter 20, a commercially available transmitter 20 can be used for the air vehicle 10 that is distributed independently. .. The transmitter 20 is not always essential in this embodiment. There is also an air vehicle 10 in which a smartphone, a tablet terminal, or the like is used as the transmitter 20. In this case, the user installs the dedicated application software on a smartphone, tablet terminal, or the like, and operates the application software to control the flying object 10.

The levers of the transmitter 20 are assigned to the directions of the elevator (forward / backward), ellon (left / right), rudder (rotation), throttle (up / down), and image pickup device 13, and the user is in the position of the flying object 10. , Height, orientation, speed, and optical axis in the imaging direction can be controlled arbitrarily.

Further, the flying object 10 can wirelessly communicate with the information processing device 40. The information processing device 40 can receive the image captured by the image pickup device 13 mounted on the flying object 10 in real time. The information processing device 40 is, for example, a notebook PC, a tablet terminal, a smartphone, a PDA (Personal Digital Assistant), a game machine, a car navigation device, or the like, and may have a communication function and a display.

Special application software is installed in the air vehicle 10 in the information processing device 40. The application software corresponding to the application software of the information processing device 40 is also installed in the aircraft body 10. In the case of automatic navigation, the user uses the application software of the information processing device 40 to make settings (referred to as flight plans) necessary for automatic navigation. The information processing device 40 wirelessly transmits the set flight plan to the flying object 10. Further, the flying object 10 periodically transmits the current position information, the battery state, the temperature, the humidity, and the like to the information processing device 40 in addition to the image captured by the image pickup device 13. Information about these flying objects 10 is called flying object information.

The aircraft 10 maintains communication with the transmitter 20 and the information processing device 40 during automatic navigation, but if communication with either one or both is interrupted, fail-safe returns to the departure point. It has a function.

FIG. 4 is an example of the flight condition setting screen 501 of the flying object 10. The user inputs the flight plan from the setting screen 501 displayed by the application software running on the information processing device 40. The setting screen 501 of FIG. 4 has a map area 502 and a detailed setting area 503. The user displays a map of the flight location in the map area 502, and the user taps the location where the accurate position is to be measured in the map area 502. Alternatively, the longitude, latitude, altitude, etc. are set in the detailed setting area 503. The place where the flying object 10 is set to pass is called a waypoint. The detailed setting area 503 of FIG. 4 has an item for one waypoint.

In this embodiment, the user switches the detailed setting area 503 to set two waypoints, or taps the two towers 11 in the map area 502. As described in the subject of the present embodiment, the position information of the tower 11 is not always accurate.

The user can set a detailed flight plan for each waypoint in each item of the detailed setting area 503. First, the latitude and longitude are the coordinates of the waypoint, and the altitude is the altitude of the flying object 10 when flying the waypoint. Speed is the flight speed from the starting point to the first waypoint, between waypoints, or from the last waypoint to the starting point.

Heading means the model direction (front direction), and is a setting for turning the nose to the angle specified by clockwise rotation with the north as 0 degrees. A POI number specified separately on the map is specified for the POI (Point Of Interest). It may be specified by coordinates. When a POI is specified, the optical axis of the imaging device 13 is automatically directed to this POI. In this embodiment, the POI may be the same as a waypoint (eg, tower 11).

The gimbal pitch specifies a method of setting the optical axis of the imaging device 13. Either "invalid", "POI focus", or "write" can be selected. When "Disable" is selected, the pilot operates by himself during the flight. When "POI focus" is selected, the optical axis of the image pickup apparatus 13 is automatically aligned with the POI set in the POI item. If you select "Write", you can set any optical axis when setting each waypoint.

In the present embodiment, by setting a structure such as a steel tower 11 in this POI, when the structure is reached from the side surface when there is a distance to the structure, or from diagonally above the structure when approaching the structure. Can be automatically imaged from almost above the structure. Therefore, it is preferable to input a value larger than the height of the object in the height item in the detailed setting area 503 of FIG.

When a flight plan for a plurality of waypoints is set, the aircraft body 10 flies between the waypoints. Generally, route-following travel is performed using a straight line connecting waypoints as a route, but in the present embodiment, the distance to the overhead ground wire is kept constant while the overhead ground wire is within the angle of view of the imaging device 13. Aircraft 10 flies between waypoints while maintaining. However, the flight method between waypoints may be along a route, and is not particularly limited in this embodiment.

FIG. 5 schematically shows a flight plan set in this embodiment. As described with reference to FIG. 4, the user inputs a flight plan including position information and the like of the two towers 11 into the information processing device 40. The flight plan is transmitted to the aircraft body 10.
(1) When the user inputs an operation for starting automatic navigation from the information processing device 40, this operation is transmitted to the air vehicle 10, and the landing vehicle 10 rises in the air. After recording the position information of the departure place, the aircraft body 10 starts the flight to the first tower 11.
(2) The flying object 10 approaches the steel tower 11 with the optical axis of the imaging device 13 aligned with the steel tower 11 set in the POI, and in a state of being close to some extent, the exact position of the steel tower 11 described in the present embodiment. Start measuring.
(3) If the accurate position of the first tower 11 can be measured, the flying object 10 starts flying to the next tower 11 based on the flight plan. The aircraft body 10 flies to the next tower 11 while keeping the distance from the overhead ground wire constant and imaging the overhead ground wire. You may fly along a straight line connecting the two towers 11.
(4) When the flying object 10 approaches the position information of the second tower 11 set on the setting screen 501, the accurate position of the tower 11 described in the present embodiment is measured in the same manner as the first tower 11.
(5) If the accurate position of the second tower 11 can be measured, the flight plan starts the flight up to the position information of the departure point because the next tower 11 is not set in the flight plan. The optical axis of the image pickup apparatus 13 at this time may be oriented anywhere.

<Configuration example>
Subsequently, a configuration example of the flying object 10 will be described with reference to FIG. FIG. 6 is an example of a functional block diagram showing the functions of the information processing device 40, the transmitter 20, and the flying object 10 in a block shape. The information processing device 40, the transmitter 20, and the flying object 10 all have functions as a general computer having a CPU, RAM, ROM, flash memory, I / O, communication device, battery, and the like. Have. The illustrated function is a function or means realized by the CPU executing application software (an example of a program in the claims) developed from the flash memory to the RAM and controlling various hardware.

<< Information processing device >>
The information processing device 40 includes a display control unit 41, an operation reception unit 42, an information management unit 43, a flight plan information transmission unit 44, and a first wireless communication unit 45. The operation reception unit 42 receives various operations on the information processing device 40 (application software). For example, it accepts flight plan settings.

For example, the display control unit 41 displays the setting screen 501 on the display, and displays the waypoints set by the user, the flight route connecting the waypoints, and the like in the map area 502 of the setting screen 501. Further, the flight object information received from the flight object 10 is displayed on the display of the information processing device 40.

The flight plan information transmission unit 44 transmits the flight plan information received by the operation reception unit 42 to the aircraft body 10 via the first wireless communication unit 45. The information management unit 43 manages the aircraft body information received from the aircraft body 10 by the first wireless communication unit 45. For example, the position information at the same time and the video are associated and managed. The flying object 10 may associate the position information and the video at the same time. The user can play back the recorded video or the like and check the position information or the like at that time. However, the aircraft information does not have to be transmitted in real time. In this case, the flying object 10 holds the flying object information, and the user moves or copies the flying object information to the information processing device 40 after returning to the departure place.

The first wireless communication unit 45 mainly communicates with the flying object 10 wirelessly, but may communicate by wire if it is not in flight. In the case of wireless, for example, radio waves in the frequency band of 2.4 GHz are used, but the frequency and communication protocol may be appropriately used in accordance with laws and regulations. General-purpose radio waves such as wireless LAN and mobile phone networks may be used. The first wireless communication unit 45 transmits flight plan information to the transmitter 20 and receives the flight object information from the transmitter 20.

<< Transmitter >>
The transmitter 20 has a control reception unit 21, a control unit 22, a control signal transmission unit 23, and a third wireless communication unit 24. The maneuvering reception unit 21 receives maneuvers to the flying object 10 by operating a lever provided on the transmitter 20. For example, while pushing down the lever, the operation related to the operation (elevator, aileron, rudder, throttle) associated with the lever is accepted. Further, when the lever corresponding to the image pickup device 13 is operated, the operation of changing the direction of the image pickup device 13 is accepted. The control interface corresponding to each lever is also referred to as a channel.

The control unit 22 converts the operation content received by the control reception unit 21 into a control signal associated with the channel. The control signal is generally digital, but it may be converted into an analog signal. The control signal transmission unit 23 transmits a control signal to the flying object 10 via the third wireless communication unit 24.

The third wireless communication unit 24 carries information on a carrier wave in the 2.4 Ghz band, for example, and transmits the information from the antenna. The frequency and communication protocol may be appropriately used in accordance with the law.

The transmitter 20 and the information processing device 40 may communicate wirelessly or by wire. In this case, since either the transmitter 20 or the information processing device 40 may wirelessly communicate with the flying object 10, either the information processing device 40 or the transmitter 20 can eliminate the function related to wireless communication. Alternatively, the information processing device 40 and the transmitter 20 may be integrated.

<< Flying Body >>
The aircraft body 10 mainly includes a second radio communication unit 31, a fourth radio communication unit 32, a flight controller 36, a GNSS receiver 33, an acceleration sensor 63, a gyro sensor 64, an altitude sensor 65, a distance sensor 66, and a rotor. It has as many ESC34s (Electric Speed Controllers) and motors 35 as there are.

The second wireless communication unit 31 receives the flight plan information from the information processing device 40 and sends it to the flight controller 36. The fourth wireless communication unit 32 receives the control signal related to maneuvering from the transmitter 20 and sends it to the motor control unit 59 of the flight controller 36. The second wireless communication unit 31 and the fourth wireless communication unit 32 are constantly communicating with the information processing device 40, monitor the radio wave strength, and detect any disconnection of communication.

The flight controller 36 provides overall control over the flight of the aircraft 10. First, control according to a control signal by maneuvering from the transmitter 20 will be described. The motor control unit 59 controls the rotation speed of each rotor so as to rotate in response to the control signal or so that the flying object 10 does not rotate in opposition to the control signal. In addition, the rotation speed of each rotor is controlled so as to fly at a speed according to the control signal. In addition, the rotation speed of each rotor is controlled so as to rise or fall according to the control signal. During flight, the motor control unit 59 estimates the attitude with the acceleration sensor 63 and controls the rotation speed of each rotor so that the attitude is maintained horizontal. In flight, the speed based on the temporal change of the position measured by the GNSS receiver 33, the acceleration detected by the acceleration sensor 63, and the rotation speed (yaw rate) detected by the gyro sensor 64 are integrated and measured. The direction, the altitude detected by the altitude sensor 65, and the distance to the lower structure detected by the distance sensor 66 are measured.

Further, the flight controller 36 controls the image pickup device 13 in a direction corresponding to the control signal. In FIG. 6, for convenience of explanation, the flight controller 36 controls the image pickup device 13, but the control of the image pickup device 13 may be performed by an IC chip or the like other than the flight controller 36.

Next, the function of the flying object 10 for flying based on the flight plan information will be described. The flight controller 36 includes a route determination unit 51, a processing reception unit 52, an image pickup device control unit 53, a direction determination unit 54, a flight plan storage unit 55, an attitude estimation unit 56, a current location acquisition unit 57, a speed conversion unit 58, and a motor control unit. It has 59, a rotation speed detection unit 60, an altitude detection unit 61, and a distance detection unit 62.

The processing reception unit 52 receives the flight plan information and stores it in the flight plan storage unit 55. The current location acquisition unit 57 acquires the position information (latitude, longitude, altitude) detected by the GNSS receiver 33. The speed conversion unit 58 converts the position information into the speed with respect to the ground surface from the time-series position information acquired by the current location acquisition unit 57 and the time interval at which the position information is acquired. Alternatively, the speed may be calculated by integrating the acceleration detected by the acceleration sensor 63.

The route determination unit 51 acquires flight plan information from the flight plan storage unit 55, and determines the route to which the flight body 10 should fly. For example, the position information of the position information (that is, the departure place) when the start of automatic navigation is input is retained. Then, set a straight line connecting the first waypoint from the departure point, switch to the mode of flying while keeping the distance to the fictitious ground wire between adjacent waypoints constant, and connect the departure point from the last waypoint. Set straight lines and determine each as a route.

When the flying object 10 flies along a path such as a straight line, the direction determining unit 54 determines the traveling direction of the flying object 10 by using the current position and the speed. Details will be described with reference to FIG. When flying between waypoints, the flight direction is determined while maintaining a constant distance from the overhead ground wire so that the overhead ground wire is imaged in the center of the image data.

The altitude detection unit 61 detects the altitude of the flying object 10 based on the signal detected by the altitude sensor 65. If the altitude sensor 65 is a barometer, it converts the barometric pressure to altitude. When calculating the distance from the time from when the altitude sensor 65 transmits light or sound wave to when it is reflected on the ground and returned, this distance is defined as the altitude.

The rotation speed detection unit 60 detects the turning speed (rotation speed) based on the signal detected by the gyro sensor 64. In addition, the speed of roll motion and the speed of pitching motion can be detected.

The distance detection unit 62 detects the distance to the structure based on the signal detected by the distance sensor 66. For example, the distance can be detected by a laser radar (LiDARLight Detection and Ranging, Laser Imaging Detection and Ranging). When the imaging device 13 is a stereo camera, the distance information detected by the stereo camera may be used.

The attitude estimation unit 56 estimates the attitude of the flying object 10 based on the signal detected by the acceleration sensor 63. Since the acceleration sensor 63 detects the acceleration of the three axes, the attitude estimation unit 56 can estimate the yaw angle, pitch angle, and roll angle of the flying object 10. The yaw angle indicates, for example, how many directions the front faces clockwise from the north direction, the pitch angle indicates how many times the front faces upward or downward with respect to the horizontal direction, and the roll angle indicates the front. Indicates how many times it is tilted to the right or left with respect to the direction.

The image pickup device control unit 53 cancels the yaw angle, pitch angle, and roll angle of the gimbal corresponding to the yaw angle, pitch angle, and roll angle of the three-axis gimbals based on the attitude estimated by the attitude estimation unit 56. Control. For example, if the pitch angle is tilted downward by 10 degrees, the gimbal is rotated upward by 10 degrees. By doing so, the image pickup device 13 can always face the front or a fixed direction such as POI even if the attitude of the flying object 10 changes.

When "POI focus" is set at the gimbal pitch of the flight plan, the image pickup device control unit 53 takes an image in the linear direction connecting the POI (iron tower in this embodiment) closest to the waypoint and the position information of the flying object 10. Align the optical axis of the device 13. Since the three-dimensional coordinates of the POI and the flying object 10 are known, the difference between the X coordinate, the Y coordinate, and the Z coordinate is obtained, and the gimbal is controlled so as to face the direction of the vector having this difference as an element.

Further, the image pickup device control unit 53 periodically acquires the image captured by the image pickup device 13 and sends it to the second wireless communication unit 31. The second wireless communication unit 31 transmits other aircraft information including video to the information processing device 40.

The motor control unit 59 controls each motor 35 so that the altitude and speed of each waypoint set in the flight plan can be obtained, and makes each motor 35 fly in the direction determined by the direction determination unit 54. Control. The rotation speed of each motor 35 is feedback-controlled with respect to the target altitude, speed, and turning speed, for example, by PID control.

The ESC 34 converts the user's operation or the control of the motor control unit 59 into a PWM (Pulse Width Modulation) signal to control the rotation speed of the motor 35. The rotation speed of the motor 35 is controlled by changing the voltage ON time according to the control from the motor control unit 59 regarding the lever position or speed, altitude, and turning of the transmitter 20.

<Method of determining the direction>
A method of determining the direction will be described with reference to FIG. 7. FIG. 7 is a diagram illustrating a method of determining the azimuth based on the relative positions of the flying object 10 and the waypoint. In FIG. 7, the horizontal plane is shown without considering the height.

FIG. 7A shows the direction in which the aircraft 10 is currently facing the two adjacent waypoints A and B. First, the straight line connecting the waypoints A and B is set as the route 511. It is desirable that the aircraft 10 flies toward waypoint B on the route, but in FIG. 7A, the aircraft 10 flies away from the route 511 in a direction different from the route. ..

Let θ be the difference between the direction 512 parallel to the path 511 and the direction of the current flying object 10, and α be the angle formed by the direction of the current flying object 10 and the straight line connecting the flying object 10 and the target point T on the path. .. The target point T moves at a constant speed on the path 511 ahead of the aircraft body 10 (waypoint A departs before the aircraft body). When the direction of the air vehicle 10 is changed by α, the air vehicle 10 flies toward the target point T. Therefore, α is first obtained. Assuming that the coordinates of the target point T are (x0, y0) and the coordinates of the flying object 10 are (x, y), α can be obtained as follows from the relationship shown in the figure.
α = arctan {(y0-y) / (x0-x)} -θ
Since the posture may become unstable if α is suddenly set to zero, it is considered to set α to zero when the target point T is reached. In this case, the flying object 10 may fly in a circular motion. FIG. 7B is an example of a diagram for explaining how to obtain a circle. Let the straight line connecting the flying object 10 and the target point T be the string 513 of the circle, the length of the string 513 be L, the center of the circle be O, and the radius be R. The current direction of the aircraft 10 is the tangent 514 of the circle. When the perpendicular line 515 is drawn from the center O to the string, the apex angle of the triangle HOP is α. From the above, the radius R is obtained as follows.
R = L / (2sinα)
Ω = Rv for speed v, turning speed (angular velocity ω), and radius R
Therefore, the angular velocity ω of the flying object 10 can be obtained from the velocity. By repeatedly calculating this turning speed and flying at the speed v and the angular velocity ω, the direction of the flying object 10 can be controlled so as to fly on the path.

Various path tracking control methods have been devised, and the method described with reference to FIG. 7 is only an example.

<Control of speed, altitude, and turning of the aircraft>
FIG. 8 is a diagram illustrating the relationship between the behavior of the flying object 10 and the rotation directions of the four rotors. FIG. 8A shows the rotation speed of the rotor when operating the elevator or aileron. For example, in the case of an elevator in the front, the rotation of the rotor in the rear is faster than that in the front, so that a difference in lift is created between the front and the rear of the flying object 10, and the rear is lifted higher. That is, the flying object 10 is in a forward leaning attitude and moves forward.

When moving to the right with ailerons, the rotation of the left rotor is faster than that of the right, so that a difference in lift is created between the right and left of the flying object 10, and the flying object 10 tilts to the right and moves to the right.

FIG. 8B shows the rotation speed of the rotor when performing an operation called turning (rudder). The airframe 10 originally has the adjacent rotors rotating in opposite directions so that the airframe does not rotate due to the torque generated by the rotation of the rotors. Therefore, for example, when turning to the left, if the rotation speed of the clockwise rotor exceeds the rotation speed of the counterclockwise rotor, the entire aircraft turns to the left. The turning direction can be changed by changing the rotation speed of the rotor on the diagonal line.

On the other hand, if there is no difference in rotational speed between the front-rear, left-right, or diagonal rotors, in other words, the four rotors rotate at almost the same rotational speed, causing the flying object 10 to stand still in the air or throttle (up and down). ) Can be done. Hovering is when you stand still in the air.

<Structure for measuring the accurate position of the tower>
Subsequently, the configuration for the flying object 10 to measure the accurate position of the steel tower 11 will be described with reference to FIG. FIG. 9 is an example of a functional block diagram showing the functions of the flight controller 36 in a block shape. The function shown in FIG. 9 may be possessed by a component other than the flight controller 36.

The flight controller 36 includes a position recording unit 71, a height determination unit 72, a symmetry determination unit 73, a position determination unit 74, an area information detection unit 75, a structure detection unit 76, an image data acquisition unit 77, and a motor control unit 59. have. The motor control unit 59 is the same as that described with reference to FIG.

The image data acquisition unit 77 repeatedly acquires the image data constituting the image captured by the image pickup apparatus 13. As described above, from the starting point to the first tower 11, the imaging device 13 always directs the optical axis to the first tower 11.

The structure detection unit 76 detects the structure from the image. For example, it may be detected by pattern matching in which an image of a steel tower 11 prepared in advance is used as a template and compared with an image, or it may be detected by a classifier generated by machine learning.

The area information detection unit 75 detects the area occupied by the structure with respect to the image data. For example, the extrinsic rectangle of the structure is detected, and the number of pixels occupied by the extrinsic rectangle is taken as the area. Instead of detecting the area, the ratio of the structure to the image data may be detected.

The positioning unit 74 determines the position of the flying object 10 that minimizes the area of the structure. This is because when the area of the structure is minimized, it can be estimated that the aircraft is flying almost directly above the structure. The area information detection unit 75 and the motor control unit 59 may cooperate with each other to repeat feedback control or feedforward control, and the motor control unit 59 may determine the position of the flying object 10 so that the area is minimized.

The symmetry determination unit 73 determines whether or not the shape of the structure is point-symmetrical when the area of the structure is minimized. This is because just because the area of the structure is minimized does not mean that it is flying directly above the structure. However, if the area of the structure is the minimum, it is often the case that the aircraft is flying directly above the structure, and it is not necessary to judge the symmetry.

The height determination unit 72 determines the height of the structure when the symmetry determination unit 73 determines that the height is directly above the structure. The position recording unit 71 records the position and height of the structure in the memory. Further, when it is determined that the position recording unit 71 is directly above the structure, the position recording unit 71 records the image data in association with the position of the structure, along with the fact that the position recording unit 71 is directly above the structure. The information recorded by the position recording unit 71 is transmitted to the information processing device 40 as air vehicle information.

<Example of how to determine the exact position of the tower>
A method of determining the exact position of the tower 11 by the flying object 10 will be described with reference to FIG. FIG. 10 is a diagram showing an imaging example of the steel tower 11 with respect to the relative positions of the flying object 10 and the steel tower 11.

FIG. 10A shows an air vehicle 10 flying right beside the steel tower 11, and FIG. 10B shows image data captured by the image pickup device 13 of the air vehicle 10 flying right beside it. As shown in FIG. 10B, the area of the image data of the steel tower 11 imaged from the side surface becomes large.

FIG. 10 (c) shows an air vehicle 10 flying diagonally above the steel tower 11, and FIG. 10 (d) shows image data captured by the image pickup device 13 of the air vehicle 10 flying diagonally above. As shown in FIG. 10D, the area of the image data of the tower 11 imaged from the oblique upper level is smaller than that of the imaged from the side surface.

FIG. 10 (e) shows an air vehicle 10 flying directly above the steel tower 11, and FIG. 10 (f) shows image data captured by the image pickup device 13 of the air vehicle 10 flying directly above. As shown in FIG. 10 (f), the area of the image data of the tower 11 imaged from directly above is smaller than that of the imaged from diagonally above. Further, since the shape of the steel tower 11 is vertically long, the area is the smallest when the image is taken from directly above.

In the present embodiment, using this knowledge, the flying object 10 is controlled so that the area of the imaged structure is minimized, and the position directly above is searched. Further, the flying object estimates (or determines) that the position where the shape of the imaged structure is point-symmetrical is directly above. By judging whether or not it is point-symmetrical, it can be confirmed that it is directly above. In addition, when there are some positions where the area can be regarded as the minimum and there is only a difference in error, the position directly above can be determined.

<Determination of the position where the area of the structure is minimized>
FIG. 11 shows an example of a method in which the position determination unit 74 determines the position where the area of the structure is minimized. As shown in FIG. 11, when the position of the tower 11 set on the setting screen 501 is reached, it is unlikely that the flying object 10 exists directly above the tower 11. Therefore, the positioning unit 74 sets a circle 521 having a predetermined area horizontal to the ground centering on the current location of the flying object 10, and divides the circle 521 with a mesh. The radius of the circle 521 is a length including the area directly above the steel tower 11. This may be determined by how much the position of the tower 11 held by the map data is different from the actual position of the tower 11. The size of the mesh may be, for example, several tens [cm] to 1 [m] on one side. This is determined in consideration of the allowable deviation from directly above. The circle 521 may be a square, a rectangle, or an ellipse.

The positioning unit 74 causes the motor control unit 59 to control the motor 35 so as to follow the intersection point 522 of the mesh from the center of the circle. As a result, it is possible to move while passing through the intersection points 522 of the mesh and accumulate the area of the structure at each intersection point 522. The positioning unit 74 determines the coordinates of the intersection 522 of one or more meshes whose area is minimized or considered to be the minimum.

Note that state feedback control may be applied to the method of determining the position where the area of the structure is minimized. The state feedback control is a method of performing control by feeding back the state quantity of the controlled object. The user moves the flying object 10 in the direction in which the area is minimized, and establishes the equation of state of equation (1) and the output equation of equation (2). If x obtained by solving the equation of state is, for example, a position, the structure can be moved to the position where the area is minimized.

＜点対称の判断＞
図１２は構造物が点対称か否かの判断方法を説明する図の一例である。図１２（ａ）では映像のほぼ中央に鉄塔１１が写っている。対称性判断部７３は、図１２（ｂ）に示すように、構造物の外接矩形の領域を切り出す。

<Judgment of point symmetry>
FIG. 12 is an example of a diagram for explaining a method of determining whether or not the structure is point-symmetrical. In FIG. 12A, the steel tower 11 is shown in the center of the image. As shown in FIG. 12B, the symmetry determination unit 73 cuts out a region of the circumscribing rectangle of the structure.

Next, as shown in FIG. 12 (c), the cut-out area is copied to generate a duplicate. Further, as shown in FIG. 12 (d), the cut out region is rotated by 180 degrees. The symmetry determination unit 73 compares the images of FIGS. 12 (c) and 12 (d) for each pixel, and determines whether or not the image is point-symmetric. As a comparison method, there is a method of calculating the difference between two images such as SAD (Sum of Absolute Difference), SSD (Sum of Squared Difference), NCC (Normalized Cross Correlation), and ZNCC (Zero means Normalized Cross Correlation). SAD and SSD have smaller values as they are similar, but NCC and ZNCC have larger values as they are similar. In the following, a high degree of similarity is simply referred to as a good symmetry score.

In FIG. 12, it is determined whether or not it is point-symmetrical, but it may be determined whether or not it is line-symmetrical. Whether the structure seen from directly above should be point-symmetrical or line-symmetrical depends on the shape of the structural part.

The symmetry determination unit 73 processes as follows according to the number of positions of the flying object 10 that minimizes the area of the structure.
(i) If the area of the structure is minimized at one position and it can be confirmed that the structure is point-symmetrical, the position where the area of the structure is minimized is set to the position directly above the structure. decide.
(ii) If there is only one position where the area of the structure is minimized and it cannot be confirmed that the structure is point-symmetrical, the flying object 10 records that fact and uses the information processing device 40 as the flying object information. Send to. In this case, the user may visually confirm whether or not the aircraft is directly above the structure in real time, and if not directly above the structure, the position of the flying object 10 may be steered from the transmitter 20 so that the area of the structure is minimized. When it is difficult for the user to intervene in real time, for convenience, the flying object 10 determines the position where the area of the structure is minimized to the position directly above the structure, and continues the flight to the next tower 11. To do. In this case, the user determines whether or not the user is directly above the structure from the image data recorded by the position recording unit 71 and the position information of the flying object 10, and if not directly above, the recorded position information is used. Discard.
(iii) There are multiple positions where the area of the structure is minimized (when there are several positions where the area can be regarded as the minimum and there is only a difference in error), and the structure is point-symmetrical at any position. If it can be confirmed, the position where the point symmetry can be confirmed is determined to be the position directly above the structure.
(iv) There are multiple positions where the area of the structure is minimized (when there are several positions where the area can be regarded as the minimum and there is only a difference in error), and the structure is point symmetric at any position. If this cannot be confirmed, the position with the best symmetry score is determined to be directly above the structure. Alternatively, the user may intervene as in (ii).

<About the height of the tower>
Since the exact position of the tower 11 can be determined from the above, the height of the tower 11 is also determined. The altitude of the flying object 10 is detected by the altitude sensor 65, and the distance to the tower 11 is detected by the distance sensor 66. Therefore, the height determination unit 72 can calculate the height of the tower 11 by the following.
Height of tower 11 = Altitude of flying object 10-Distance to tower 11 <Processing by information processing device>
The information processing device 40 may have the function shown in FIG. When the current location of the aircraft 10 reaches the position of the steel tower 11 set on the setting screen 501, the information processing device 40 has the position recording unit 71, the height determination unit 72, the symmetry determination unit 73, and the position shown in FIG. The processing performed by the determination unit 74, the area information detection unit 75, and the structure detection unit 76 is performed.

When the position of the steel tower 11 set on the setting screen 501 is reached, the position determination unit 74 sets a horizontal mesh on the ground centering on the current location, and transmits the position of the intersection of the meshes to the flying object 10. The image at the intersection of the mesh is received from the flying object 10, the structure detecting unit 76 detects the structure from the image data, and the area information detecting unit 75 detects the area information of the structure. The position determination unit 74 determines the position of the intersection of the mesh having the smallest area, and the symmetry determination unit 73 transmits the position of the intersection of the mesh determined to have symmetry to the flying object 10. The aircraft body 10 starts imaging the overhead ground wire from this position.

<Operation procedure>
FIG. 13 is an example of a flowchart showing a procedure in which the flying object 10 measures the accurate position of the structure. In the process of FIG. 13, it is assumed that the flying object 10 has already flown to the destination of the first tower 11.

The image pickup apparatus 13 always directs the optical axis to the steel tower 11 and takes an image. The image data acquisition unit 77 acquires image data showing the iron tower 11 from the image pickup apparatus 13, and the structure detection unit 76 detects the structure from the image data.

The position determination unit 74 determines whether or not the current location of the flying object 10 has reached the steel tower 11 based on whether or not it is substantially equal to the position set on the setting screen 501 (S1). In this state, it is considered that the imaging device 13 images the steel tower 11 from diagonally above. Alternatively, it may be determined whether or not the steel tower 11 has been reached by determining whether or not the steel tower 11 is imaged from diagonally above. It is not necessary to completely reach the position set on the setting screen 501, and if the flying object 10 approaches the tower 11 and moves to some extent (for example, in a range of several tens of meters), the minimum area of the tower 11 can be obtained. If it reaches a position where it can be imaged, it may be determined that the tower 11 has been reached.

When the tower 11 is reached, the position determination unit 74 determines the intersection point 522 of the mesh for detecting the area of the tower 11 (S2). Then, the position of each intersection 522 is sent to the motor control unit 59 to be moved, and the area information detection unit 75 detects the area of the steel tower 11 at the intersection 522 of each mesh (S3).

When the area information detection unit 75 calculates the area at all the intersections 522, the position determination unit 74 determines one or more positions (mesh intersections) that minimize the area (S4). It is assumed that a plurality of positions have been determined in the process of FIG.

The symmetry determination unit 73 determines the symmetry of the tower 11 imaged at the position where the area of the tower 11 is minimized (S5).

When there are a plurality of positions where the area of the tower 11 is minimized, the position having the best symmetry score is determined directly above the tower 11 (S6).

Since the position directly above the tower 11 can be determined, the height determining unit 72 calculates the height of the tower 11 from the altitude of the flying object 10 and the distance to the tower 11 (S7).

The position recording unit 71 records the fact that it is directly above the steel tower 11 and records the position (latitude, longitude) and height directly above the steel tower 11 in association with the image data (S8).

As a result, the flying object 10 was able to measure the accurate position of the first tower 11, and then started moving to the second tower 11 according to the flight plan. Between the tower 11 and the tower 11, the flying object 10 flies while maintaining a constant distance to the overhead ground wire with the overhead ground wire within the angle of view of the image pickup device 13, so that the overhead ground wire is almost equal to the above. The image can be taken with a certain size, and the user can inspect the overhead ground wire from the image.

<Summary>
As described above, the flying object 10 of the present embodiment can measure the accurate position of the structure by determining the position so that the area of the structure is minimized.

<Other application examples>
Although the best mode for carrying out the present invention has been described above with reference to Examples, the present invention is not limited to these Examples, and various modifications are made without departing from the gist of the present invention. And substitutions can be made.

For example, after determining the position directly above the structure, the positions of the four sides of the structure as viewed from above may be determined. The position where the distance sensor moves from directly above to the front, back, left, and right and the distance suddenly increases (the position where the distance to the ground, not the structure, is measured) is the side (end) of the structure viewed from above. The position can be determined.

Also, the number of waypoints is not limited to two, and more waypoints can be set as long as the battery lasts.

In addition, not only overhead ground wire but also electric wires and other cables may be inspected between waypoints. Further, a structure different from that of the tower 11 may be set as a waypoint.

Further, the configuration examples shown in FIG. 9 and the like are divided according to the main functions in order to facilitate understanding of the processing of the flying object 10. The present invention is not limited by the method of dividing the processing unit or the name. Further, the processing of the flying object 10 can be divided into more processing units according to the processing content. Further, one processing unit can be divided so as to include more processing.

In addition, the functions of the air vehicle 10 may be realized by software, or ASIC (Application Specific Integrated Circuit), DSP (digital signal processor), FPGA (field programmable gate array), etc. designed to execute each function. It shall include the case where it is realized by a hardware module.

10 Aircraft 11 Steel Tower 13 Imaging Device 20 Transmitter 36 Flight Controller 40 Information Processing Device 100 Aircraft System

Claims (8)

An air vehicle system that images a structure from the sky by an air vehicle having an image pickup device.
The current location acquisition department that acquires the current position of flight, and
A structure detection unit that detects the structure from the image data captured by the image pickup device, and
An area information detection unit that detects area information in the image data of the structure detected by the structure detection unit, and an area information detection unit.
A position determining unit that determines the position of the flying object when the area information in the image data of the structure detected by the area information detecting unit can be regarded as the minimum.
A position recording unit that stores the position of the flying object determined by the position determining unit as the position of the structure, and
An air vehicle system characterized by having. It has a symmetry determination unit that determines the symmetry of the shape of the structure reflected in the image data captured at the position of the flying object determined by the position determination unit.
When the symmetry determination unit determines that the shape of the structure shown in the image data is point-symmetrical or line-symmetrical.
The flying object system according to claim 1, wherein the position recording unit stores the position of the flying object determined by the positioning unit as the position of the structure. The positioning unit is characterized in that the position of the flying object when the area information in the image data of the structure detected by the area information detecting unit can be regarded as the minimum is estimated to be directly above the structure. The flying object system according to claim 1 or 2. The position recording unit determines the position of the flying object when the area information in the image data of the structure detected by the area information detection unit can be regarded as the minimum.
The flying object system according to any one of claims 1 to 3, wherein the image data captured by the imaging device is recorded at the position of the flying object in association with the image data. The positioning unit sets a horizontal mesh on the ground in a state of approaching a predetermined range from the structure, and makes each intersection of the meshs fly to the flying object.
At each intersection of the mesh, the structure detection unit detects the structure from the image data captured by the imaging device.
The area information detection unit detects the area information in the image data of the structure imaged at each intersection of the mesh, and detects the area information.
The position according to any one of claims 1 to 4, wherein the position determination unit determines an intersection of meshes having the minimum area information in the image data of the structure at the position of the flying object. Aircraft system. It is a flying object that images a structure from the sky with an image pickup device.
The current location acquisition department that acquires the current position of flight, and
A structure detection unit that detects the structure from the image data captured by the image pickup device, and
An area information detection unit that detects area information in the image data of the structure detected by the structure detection unit, and an area information detection unit.
A position determining unit that determines the position of the flying object when the area information in the image data of the structure detected by the area information detecting unit can be regarded as the minimum.
A position recording unit that stores the position of the flying object determined by the position determining unit as the position of the structure, and
An air vehicle characterized by having. It is a position measurement method performed by an air vehicle system that images a structure from the sky by an air vehicle having an image pickup device.
The step that the current location acquisition department acquires the current position of flight,
A step in which the structure detection unit detects the structure from the image data captured by the imaging device.
A step in which the area information detection unit detects area information in the image data of the structure detected by the structure detection unit, and
A step of determining the position of the flying object when the area information in the image data of the structure detected by the area information detecting unit can be regarded as the minimum.
A step in which the position recording unit stores the position of the flying object determined by the position determining unit as the position of the structure.
A position measuring method characterized by having. An air vehicle that images a structure from the sky with an image pickup device,
The current location acquisition department that acquires the current position of flight, and
A structure detection unit that detects the structure from the image data captured by the image pickup device, and
An area information detection unit that detects area information in the image data of the structure detected by the structure detection unit, and an area information detection unit.
A position determining unit that determines the position of the flying object when the area information in the image data of the structure detected by the area information detecting unit can be regarded as the minimum.
A position recording unit that stores the position of the flying object determined by the position determining unit as the position of the structure.
A program to function as.

Images