JP7352908B2 - System, method, program, and storage medium storing the program for correcting the estimated position of an unmanned aircraft - Google Patents

Description

The present invention relates to a system, method, program, and storage medium storing the program for correcting the estimated position of an unmanned aircraft.

Recently, attempts have been made to inspect architectural structures using drones. Drones can estimate their own position using GPS satellite positioning information and fly autonomously along the flight plan route, but when flying under bridges or around architectural structures, accurate position information is not provided by GPS. difficult to obtain. Therefore, in such an environment, a technique has been proposed in which self-position estimation using Visual SLAM is used as a substitute for self-position estimation using GPS (see Non-Patent Document 1 below). Here, Visual SLAM is a technology that estimates a self-position and a map in parallel by tracking a plurality of feature points between a plurality of frames of continuously photographed images.

Kenzo Nonami, “All about drone industry applications”, Ohmsha, February 2018, p.73-79

However, in self-position estimation using Visual SLAM, errors accumulate as the flight distance increases, so that the actual flight route deviates from the flight plan route.

Therefore, one object of the present invention is to provide a system and method for correcting the estimated position of an unmanned aircraft when the unmanned aircraft flies autonomously by estimating its own position using Visual SLAM.

One aspect of the present invention is an unmanned aircraft that is equipped with an imaging device capable of capturing images above, extracts feature points from the image acquired by the imaging device, and flies while estimating its own position based on the feature points. Based on a flight plan path formed by interconnecting the ends of a plurality of crossing paths that cross the bridge via connection paths extending along the side edges of the bridge, the bridge is moved from below the bridge. A system for correcting the estimated position of the unmanned aircraft when flying while taking images, wherein the connecting route has a first end and a second end in the order of flight of the unmanned aircraft, In the planned route, the first end is set as a waypoint, and while the unmanned aircraft is flying along the traversal route, the estimated position of the unmanned aircraft estimated by the self-position estimation is set at the waypoint. When detecting, before reaching the unmanned aerial vehicle, that the region in which the feature points are distributed corresponds to being biased toward the opposite side of the traveling direction of the unmanned aircraft by a first predetermined degree or more in the image; The present invention provides a system for correcting the estimated position of the unmanned aircraft so that the estimated position of the unmanned aircraft at that time corresponds to the waypoint.

The fact that the region in which the feature points are distributed is biased toward the opposite side of the traveling direction of the unmanned aircraft by a first predetermined degree or more in the image means that the region in which the feature points are distributed is It may be assumed that the first representative point of the unmanned aircraft enters a first region in the image on the opposite side to the traveling direction of the unmanned aircraft.

The first representative point of the region in which the feature points are distributed may be a point closest to the direction of travel of the unmanned aircraft among the extracted feature points.

The fact that the region in which the feature points are distributed is biased toward the opposite side of the traveling direction of the unmanned aircraft by a first predetermined degree or more in the image means that the region in which the feature points are distributed is A parameter related to the area of may be less than or equal to a first threshold value.

One aspect of the present invention is an unmanned aircraft that is equipped with an imaging device capable of capturing images above, extracts feature points from the image acquired by the imaging device, and flies while estimating its own position based on the feature points. Based on a flight plan path formed by interconnecting the ends of a plurality of crossing paths that cross the bridge via connection paths extending along the side edges of the bridge, the bridge is moved from below the bridge. A system for correcting the estimated position of the unmanned aircraft when flying while taking images, wherein the connecting route has a first end and a second end in the order of flight of the unmanned aircraft, In the planned route, the first end is set as a waypoint, and while the unmanned aircraft is flying along the traversal route, the estimated position of the unmanned aircraft estimated by the self-position estimation is set at the waypoint. When it is detected that the region in which the feature points are distributed is biased toward the direction of travel of the unmanned aircraft by a second predetermined degree or more in the image, the unmanned aerial vehicle The aircraft is allowed to proceed in the direction of travel, and the region in which the feature points are distributed no longer exists in the image in a biased direction toward the direction of travel of the unmanned aircraft by a second predetermined degree or more. The present invention provides a system that, when detected, corrects the estimated position of the unmanned aircraft so that the estimated position of the unmanned aircraft at that time corresponds to the waypoint.

The fact that the region in which the feature points are distributed is biased towards the direction of travel of the unmanned aircraft by a second predetermined degree or more in the image means that It is detected that the representative point of is located in the second area on the traveling direction side of the unmanned aircraft in the image, and the area where the characteristic points are distributed is located in the second area in the image. Corresponding to the fact that the unmanned aerial vehicle no longer exists biased towards the direction of travel of the unmanned aerial vehicle to a predetermined extent or more, the second representative point of the area in which the feature points are distributed is located in the second area in the image. It can be assumed that this is due to the detection of deviation from the range.

The second representative point of the region in which the feature points are distributed may be a point closest to the traveling direction of the unmanned aircraft among the extracted feature points.

The fact that the area where the feature points are distributed is biased toward the direction of travel of the unmanned aircraft by a second predetermined degree or more in the image is related to the area of the area where the feature points are distributed. detecting that the parameter is equal to or higher than a second threshold;
This corresponds to the fact that the region in which the feature points are distributed no longer exists in the image in a biased direction toward the direction of travel of the unmanned aircraft by a second predetermined degree or more. This can be determined by detecting that the parameter related to area has become smaller than the second threshold.

One aspect of the present invention is an unmanned aircraft that is equipped with an imaging device capable of capturing images above, extracts feature points from the image acquired by the imaging device, and flies while estimating its own position based on the feature points. Based on a flight plan path formed by interconnecting the ends of a plurality of crossing paths that cross the bridge via connection paths extending along the side edges of the bridge, the bridge is moved from below the bridge. A system for correcting the estimated position of the unmanned aircraft when flying while taking images, wherein the connecting route has a first end and a second end in the order of flight of the unmanned aircraft, In the planned route, the first end is set as a waypoint, and while the unmanned aircraft is flying along the traversal route, the estimated position of the unmanned aircraft estimated by the self-position estimation is set at the waypoint. When detecting, before reaching the unmanned aerial vehicle, that the region in which the feature points are distributed corresponds to being biased toward the opposite side of the traveling direction of the unmanned aircraft by a first predetermined degree or more in the image; , the estimated position of the unmanned aircraft at that time is corrected so that the estimated position of the unmanned aircraft corresponds to the waypoint, and the estimated position of the unmanned aircraft is estimated by the self-position estimation while the unmanned aircraft is flying the crossing route. When the estimated position of the unmanned aerial vehicle reaches the waypoint, an area in which the feature points are distributed is biased toward the direction of travel of the unmanned aerial vehicle by a second predetermined degree or more in the image. When it is detected that the unmanned aircraft corresponds to the above, the unmanned aircraft is allowed to proceed in the direction of travel, and the area in which the feature points are distributed is moved toward the direction of travel of the unmanned aircraft by a second predetermined degree or more in the image. A system that corrects the estimated position of the unmanned aircraft so that the estimated position of the unmanned aircraft at that time corresponds to the waypoint when it is detected that the unmanned aircraft corresponds to the waypoint.

One aspect of the present invention provides an unmanned aircraft equipped with the above system.

One aspect of the present invention is an unmanned aircraft that is equipped with an imaging device capable of capturing images above, extracts feature points from the image acquired by the imaging device, and flies while estimating its own position based on the feature points. Based on a flight plan path formed by interconnecting the ends of a plurality of crossing paths that cross the bridge via connection paths extending along the side edges of the bridge, the bridge is moved from below the bridge. A method for correcting the estimated position of the unmanned aircraft when flying while taking images, the connecting path including a first end and a second end in the order of flight of the unmanned aircraft. In the flight plan route, the first end is set as a waypoint, and the estimated position of the unmanned aircraft is estimated by the self-position estimation while the unmanned aircraft is flying the traversal route. corresponds to the fact that before the pointer reaches the waypoint, an area in which the feature points are distributed is biased toward the opposite side of the direction of travel of the unmanned aircraft by a first predetermined degree or more in the image. When this is detected, a method is provided for correcting the estimated position of the unmanned aircraft so that the estimated position of the unmanned aircraft at that time corresponds to the waypoint.

One aspect of the present invention is an unmanned aircraft that is equipped with an imaging device capable of capturing images above, extracts feature points from the image acquired by the imaging device, and flies while estimating its own position based on the feature points. Based on a flight plan path formed by interconnecting the ends of a plurality of crossing paths that cross the bridge via connection paths extending along the side edges of the bridge, the bridge is moved from below the bridge. A method for correcting the estimated position of the unmanned aircraft when flying while taking images, the connecting path including a first end and a second end in the order of flight of the unmanned aircraft. In the flight plan route, the first end is set as a waypoint, and the estimated position of the unmanned aircraft is estimated by the self-position estimation while the unmanned aircraft is flying the traversal route. reaches the waypoint, it is detected that the area in which the feature points are distributed corresponds to being biased toward the direction of travel of the unmanned aircraft by a second predetermined degree or more in the image; when the unmanned aircraft continues to advance in the direction of travel, and the area in which the feature points are distributed no longer exists in the image biased toward the direction of travel of the unmanned aircraft by a second predetermined degree or more; The present invention provides a method for correcting the estimated position of the unmanned aircraft so that the estimated position of the unmanned aircraft at that time corresponds to the waypoint.

One aspect of the present invention is an unmanned aircraft that is equipped with an imaging device capable of capturing images above, extracts feature points from the image acquired by the imaging device, and flies while estimating its own position based on the feature points. Based on a flight plan path formed by interconnecting the ends of a plurality of crossing paths that cross the bridge via connection paths extending along the side edges of the bridge, the bridge is moved from below the bridge. A method for correcting the estimated position of the unmanned aircraft when flying while taking images, the connecting path including a first end and a second end in the order of flight of the unmanned aircraft. In the flight plan route, the first end is set as a waypoint, and the estimated position of the unmanned aircraft is estimated by the self-position estimation while the unmanned aircraft is flying the traversal route. corresponds to the fact that before the pointer reaches the waypoint, an area in which the feature points are distributed is biased toward the opposite side of the direction of travel of the unmanned aircraft by a first predetermined degree or more in the image. When this is detected, the estimated position of the unmanned aircraft is corrected so that the estimated position of the unmanned aircraft at that time corresponds to the waypoint, and while the unmanned aircraft is flying along the crossing route, When the estimated position of the unmanned aerial vehicle estimated by position estimation reaches the waypoint, the area in which the feature points are distributed is located on the advancing direction side of the unmanned aerial vehicle by a second predetermined degree or more in the image. When it is detected that the unmanned aircraft corresponds to the presence of the unmanned aircraft biased to When it is detected that the unmanned aerial vehicle is no longer present in the direction of travel, the estimated position of the unmanned aerial vehicle is corrected so that the estimated position of the unmanned aerial vehicle at that time corresponds to the waypoint. The present invention provides a method.

One aspect of the present invention provides a program for causing a computer to execute the method.

One aspect of the present invention provides a computer-readable recording medium on which the program is recorded.

According to the present invention having the above configuration, it is possible to provide a system and method for correcting the estimated position of an unmanned aircraft when the unmanned aircraft flies autonomously by estimating its own position using Visual SLAM.

1 is an external view of an unmanned aircraft according to one embodiment of the present invention. 1 is a diagram showing the overall configuration of an unmanned aircraft according to one embodiment of the present invention. 1 is a diagram showing a functional configuration of an estimated position correction system according to one embodiment of the present invention. It is a flow chart of an example of estimated position modification processing of an estimated position modification system concerning one embodiment of the present invention. It is a diagram showing an example of a flight plan route. 6 is a diagram showing an example of the relationship between the position of an unmanned aircraft and the distribution of feature points by enlarging a part of FIG. 5; FIG.

Embodiments of the present invention will be described below with reference to the drawings. However, the present invention is not limited to the specific embodiments described below, and may take various embodiments within the scope of the technical idea of the present invention. For example, the unmanned aircraft of the present invention is not limited to the multicopter shown in FIG. 1, but may be any other unmanned aircraft such as a rotary wing aircraft or a fixed wing aircraft. Furthermore, the system configuration of the unmanned aircraft 1 is not limited to that shown in the figure, but can be any configuration as long as it can perform similar operations. For example, operations performed by multiple components may be performed by a single component, such as integrating the functions of a communication circuit into the flight control section, or distributing the functions of the main processing section to multiple processing sections. , operations performed by a single component may be performed by multiple components. Further, the various data stored in the memory of the unmanned aircraft 1 may be stored in a different location, and the information recorded in the various memories may be distributed from one type of information to multiple types. Alternatively, multiple types of information may be stored together as one type.

FIG. 1 is an external view of a multicopter, which is an example of an unmanned aircraft (multicopter) 1 according to the present embodiment. In terms of appearance, the unmanned aircraft 1 includes a control unit 101, six motors 102 driven by control signals from the control unit 101, and six rotors (rotating It includes a wing) 103, six arms 104 that connect the control unit 101 and each motor 102, and landing legs 105 that support the unmanned aircraft during landing. The number of motors 102, rotors 103, and arms 104 may be three or more, such as three or four, respectively. The six motors 102 are rotated by a control signal from the control unit 101, and by controlling the rotation speed of each of the six rotors 103, the unmanned aircraft 1 can perform various operations such as ascending, descending, flying forward and backward, left and right, and turning. flight is controlled. Further, a pedestal 106 is attached to the control unit 101, and a first camera 108 for photographing an object with high resolution is attached to the pedestal via a support member 107 that rotatably supports the first camera 108. 106. In addition, a second camera 109 is installed directly upward in front of the pedestal 106 to take an image of an object for performing Visual SLAM processing, which will be described later. A processing unit 124 is attached. The unmanned aircraft 1 also has a flight position sensor 110 and an antenna 117.

FIG. 2 is a diagram showing the overall configuration of the unmanned aircraft 1 shown in FIG. 1. The unmanned aircraft 1 mainly includes a control unit 101, a motor 102 electrically connected to the control unit 101, a rotor 103 mechanically connected to the motor 102, a first camera 108, a second camera 109, and a flight position. It is composed of a sensor 110, an attitude sensor 111, a direction sensor 112, an altitude sensor 113, a distance sensor 115, and an antenna 117.

The control unit 101 is configured to process information for controlling the flight of the unmanned aircraft 1 and control electrical signals for that purpose, and typically has various electronic components arranged and wired on a board. This is a unit that includes the circuits necessary to realize such functions. The control unit 101 further includes an information processing unit 120, a communication circuit 121, a control signal generation section 122, a speed controller 123, and an interface 125.

The first camera 108 is a camera for capturing an image of a target object with high resolution. The first camera 108 is attached to the pedestal 106 via a support member 107 that rotatably supports the first camera 108, so that the imaging direction can be changed. The first camera 108 acquires image data of the photographing range of the unmanned aircraft 1 while it is in flight, and the acquired images are stored in the storage unit 221, which will be described later. The image is typically a moving image consisting of a series of still images.

The second camera 109 is a camera for photographing an image of an object for performing Visual SLAM processing, which will be described later, and is mounted on the pedestal 106 so as to face directly above. In order to reduce the load of Visual SLAM processing, a second camera 109 that captures images at low resolution is provided separately from the first camera 108. The second camera 109 acquires image data of the shooting range of the unmanned aircraft 1 while it is in flight, and the acquired images are stored in the information processing unit 120 and the storage unit 207, which will be described later. The image is typically a moving image consisting of a series of still images.

The flight position sensor 110 is a navigation sensor that senses the coordinates of the flight position of the unmanned aircraft 1, such as a GPS (Global Positioning System) sensor. Flight position sensor 110 preferably senses three-dimensional coordinates. The flight position sensor 110 is mainly used to control the position of the unmanned aircraft 1 when it flies along the flight plan route in an environment where GPS signals can be well received.

The attitude sensor 111 is a sensor, such as a 6-axis gyro (acceleration/angular velocity sensor), for detecting the movement and inclination of the unmanned aircraft 1, and is used to control the attitude of the unmanned aircraft 1 during flight. be done.

The direction sensor 112 is a sensor, such as a magnetic sensor, for controlling the flight direction.

The altitude sensor 113 is a sensor that detects the altitude of the unmanned aircraft 1, such as a barometric pressure sensor, and is used to control the flight altitude of the unmanned aerial vehicle 1.

The unmanned aircraft 1 may be equipped with a distance sensor such as an ultrasonic sensor. The distance sensor 115 is a sensor, such as an ultrasonic sensor, that measures the distance between an obstacle and the ground, avoids a collision with the obstacle, and precisely measures the height above the ground.

The antenna 117 is an antenna for receiving radio signals including information and various data for operating and controlling the unmanned aircraft 1, and for transmitting radio signals including telemetry signals from the unmanned aircraft 1.

The communication circuit 121 demodulates a control signal, a control signal, various data, etc. for the unmanned aircraft 1 from the radio signal received through the antenna 117 and inputs the demodulated data to the information processing unit 120, and also demodulates the telemetry signal output from the unmanned aircraft 1. It is an electronic circuit for generating a wireless signal that carries signals, etc., and is typically a wireless signal processing IC. Note that, for example, communication of maneuver signals and communication of control signals and various data may be performed by different communication circuits in different frequency bands. For example, it communicates with the transmitter of a controller (propo) for manual operation using a frequency of 950 MHz band, and communicates data using a frequency of 2 GHz band/1.7 GHz band/1.5 GHz band/800 MHz band. It is also possible to adopt a similar configuration.

The control signal generation unit 122 is configured to convert control command value data obtained by calculation by the information processing unit 120 into a pulse signal (PWM signal, etc.) representing voltage, and typically includes an oscillation circuit and a switching circuit. It is an IC that includes. The speed controller 123 is configured to convert a pulse signal from the control signal generator 122 into a drive voltage for driving the motor 102, and is typically a smoothing circuit and an analog amplifier. Although not shown, the unmanned aircraft 1 includes a power supply system including a battery device such as a lithium polymer battery or a lithium ion battery, and a power distribution system to each element.

The VSLAM processing unit 124 is an example of the estimated position correction system of the present invention, and includes a CPU 124a, a RAM 124b, a ROM 124c, an external memory 124d, and an interface 124g. RAM 124b, ROM 124c, external memory 124d, and interface 124e are connected to CPU 124a via system bus 124f. VSLAM processing unit 124 may include a GPU.

Each part of the estimated position correction system 2 shown in FIG. 3, which will be described later, is realized by various programs stored in the ROM 124c and external memory 124d using the CPU 124a, RAM 124b, ROM 124c, external memory 124d, interface 124e, etc. as resources. be done.

Regarding Visual SLAM, various algorithms have been developed, such as MonoSLAM and PTAM (Parallel Tracking and Mapping).The VSLAM processing unit 124 executes a program implementing such an algorithm, so that the information from the second camera 109 is Self-position estimation and map creation are performed using Visual SLAM using image signals, and the estimated self-position (relative position of the unmanned aircraft 1 with respect to elements existing around the unmanned aircraft 1) and velocity (based on the time differential of the position) are estimated. ), attitude (obtained by geometric calculation from the arrangement of a plurality of feature points in the photographed image), etc., which represent the state of the unmanned aircraft 1. Signals indicating these quantities are sent to the information processing unit. 120, and the information processing unit 120 uses the information input from the VSLAM processing unit 124.The map information estimated by the VSLAM processing unit 124 is also output to the information processing unit 120, and is recorded on the recording device. Note that a stereo camera may be used instead of a monocular camera as the second camera 109, and in this case as well, self-position etc. can be estimated by Visual SLAM using the same principle.

The interface 125 converts the form of the signal so that it can be transmitted and received between functional elements such as the information processing unit 120, flight position sensor 110, attitude sensor 111, direction sensor 112, altitude sensor 113, and distance sensor 115. This configuration electrically connects them. Note that for convenience of explanation, the interface is shown as one configuration in the drawings, but different interfaces are usually used depending on the type of functional element to be connected. Further, the interface 125 may not be necessary depending on the type of signal input/output by the functional element to be connected. Further, in FIG. 2, even if the information processing unit 120 is connected without an interface 125, an interface may be necessary depending on the type of signal input/output by the functional element to be connected.

Information processing unit 120 includes a CPU, RAM, ROM, external memory, and a system bus. The RAM, ROM, external memory, communication circuit 121, control signal generator 122, and interface 125 are connected to the CPU via a system bus.

The information processing unit 120 appropriately controls the flight of the unmanned aircraft 1 based on control signals from the operator (during non-autonomous flight), flight plan route data (during autonomous flight), and the like. Specifically, the attitude, speed, etc. of the unmanned aircraft 1 are determined based on information obtained from various sensors for flight control and the VSLAM processing unit 124, and the current flight position of the unmanned aircraft 1 is determined based on information obtained from the flight position sensor 107. The information processing unit 120 calculates a control command value for each rotor 103 by determining the control signal, flight plan route, speed limit, altitude limit, etc. by comparing it with target values such as the control signal, flight plan route, speed limit, altitude limit, etc., and controls the data indicating the control command value. It is output to the signal generation section 122. The control signal generation unit 122 converts the control command value into a pulse signal representing voltage and transmits it to each speed controller 123. Each speed controller 123 converts the pulse signal into a drive voltage and applies it to each motor 102, thereby controlling the drive of each motor 102 and controlling the rotation speed of each rotor 103, thereby controlling the speed of the unmanned aircraft 1. Flight is controlled.

Here, the flight plan route data is data representing a three-dimensional (latitude, longitude, altitude) flight plan route of the unmanned aircraft 1, and typically includes a series of multiple waypoints existing on the flight plan route. This is a set of data. The flight plan route is typically a set of straight lines connecting the plurality of waypoints in order, but it can also be a curved line with a predetermined curvature within a predetermined range of waypoints. Preferably, the target location exists near a specific waypoint on the flight plan path. Information indicating that the vehicle will head to the target position after passing through the waypoint is added to the waypoint. The waypoint can also be the end point (turnaround point) of the flight plan route. Note that a certain waypoint can also be set as the target position. In this case, information indicating that the waypoint is a target position is added to the data of that waypoint. The flight plan route data may include data defining flight speeds at multiple waypoints. Flight plan path data is typically used to define a flight plan path in autonomous flight, but can also be used for flight guidance in non-autonomous flight. Flight plan route data is typically input into and stored in the unmanned aircraft 1 before flight.

FIG. 3 is a diagram showing the functional configuration of an estimated position correction system according to one embodiment of the present invention. The corrected position correction system 2 includes a VSLAM section 201, a feature point region parameter calculation section 203, an estimated position correction section 205, and a storage section 207.

The VSLAM unit 201 acquires the image data of the image captured by the second camera 109, detects feature points regarding the bridge 3 from the acquired image data, and estimates and maps the self-position based on the detected feature points. Create.

The feature point distribution region parameter calculation unit 203 calculates parameters related to the region in which the detected feature points are distributed. The parameters related to the area where the detected feature points are distributed may be any suitable parameter, such as the point closest to the direction of travel of the unmanned aircraft among the detected feature points, the area of the area where the detected feature points are distributed, etc. It can be a parameter related to feature points.

The estimated position correction unit 205 corrects the estimated position when parameters related to the area in which the calculated feature points are distributed satisfy a predetermined condition. That is, while the unmanned aerial vehicle 1 is flying along a cross-country route (described later), before the estimated position of the unmanned aerial vehicle estimated by self-position estimation reaches the waypoint, the area where the feature points are distributed becomes the first point in the image. When it is detected that the unmanned aircraft is biased to the opposite side of the direction of travel to a predetermined degree or more, the estimated position of the unmanned aircraft at that time corresponds to the waypoint. Fix it. Further, when the unmanned aerial vehicle is flying along a crossing route and the estimated position of the unmanned aerial vehicle estimated by self-position estimation reaches a waypoint, the area in which the feature points are distributed is determined to be within a second predetermined degree in the image. As described above, when it is detected that the unmanned aircraft 1 is present biased toward the direction of travel, the unmanned aircraft is allowed to proceed in the direction of travel, and the area where the feature points are distributed is located in the second predetermined area in the image. When it is detected that the unmanned aircraft 1 is no longer present in the direction of travel, the estimated position is corrected so that the estimated position of the unmanned aircraft at that time corresponds to the waypoint. do.

The storage unit 207 stores various data and various programs.

Based on the above system configuration, an example of the estimated position correction process of the estimated position correction system according to one embodiment of the present invention will be described below with reference to FIGS. 1 to 6. In this embodiment, an example will be described in which the underside of a bridge is imaged and inspected.

FIG. 4 is a flowchart of an example of estimated position correction processing of the estimated position correction system according to one embodiment of the present invention. FIG. 5 is a diagram showing an example of a flight plan route. FIG. 6 is a diagram showing an example of the relationship between the position of the unmanned aircraft and the distribution of feature points by enlarging a part of FIG.

As a preliminary preparation before the flight of the unmanned aircraft 1, an appropriate flight plan route is created, and a set of a series of multiple waypoints existing on the flight plan route is stored in the information processing unit 120 as flight plan route data. . The flight plan route data can also include information on altitude or altitude above ground at each waypoint. When the flight plan route data of the unmanned aircraft 1 is stored in the information processing unit 120, it is transmitted to the unmanned aircraft 1 as a wireless signal and read into the information processing unit 120 via the antenna 117 and the communication circuit 121. suitable. If the unmanned aircraft 1 has another suitable interface, the information may be read into the information processing unit 120 of the unmanned aircraft 1 via that interface.

FIG. 5 is a diagram showing an example of a part of the flight plan route RT. The intersections of the cross paths T1, T2, T3, . . . extending in a direction perpendicular to the longitudinal direction of the bridge 3 and the side edges 31 of the bridge 3 are set as waypoints P1, P2, P3, . T1, T2, T3, . . . are connected via connection paths J1, J2, J3, . . . extending along the side edge 31 of the bridge 3. The interval between adjacent crossing paths is determined based on the imaging width of the first camera 108, taking into consideration the amount of overlap between the imaging widths of the adjacent crossing paths. Further, the altitude of the waypoint is set to be a predetermined distance below the bridge 3. The connection path does not necessarily need to be on the side edge of the bridge, but may just be along the side edge of the bridge.

When the preflight preparations are completed, the information processing unit 120 performs autonomous flight control along the flight plan route. That is, the information processing unit 120 reads the flight plan route data from the information processing unit 120, and controls the unmanned aircraft 1 to fly to the target position along the flight plan route determined by the read flight plan route data. Specifically, the flight direction and altitude of the unmanned aircraft 1 are controlled so that the unmanned aircraft 1 flies through the waypoints defined by the flight plan route data in that order. The flight plan path preferably includes data on a target flight speed, and the unmanned aircraft 1 is controlled to fly along the flight plan path at that flight speed. Note that the information processing unit 120 can perform non-autonomous flight by accepting manual operations from the user during flight. In this case, the flight plan route is used as a guide, and for example, the unmanned aircraft 1 can be returned to the flight plan route when the manual operation is completed.

The VSLAM unit 201 acquires image data of an image of the bridge 3 taken by the second camera 109 from below the bridge 3 during the flight, detects feature points regarding the bridge 3 from the acquired image data, and detects the detected feature points. Based on the feature points, the self-position is estimated and a map is created (S101).

The feature point distribution area parameter calculation unit 205 calculates a point Ct closest to the moving direction of the aircraft among the detected feature points, which is a parameter related to the area where the detected feature points are distributed (S103).

The estimated position correction unit 205 corrects the estimated position when the point closest to the traveling direction of the aircraft among the calculated detected feature points satisfies a predetermined condition.

FIG. 6 is a diagram showing an example of the relationship between the position of the unmanned aircraft and the distribution of feature points by enlarging a part of FIG. When the unmanned aerial vehicle 1 is flying under the bridge 3 (for example, point A in FIG. 6), the bridge 3 is imaged in the entire image in the image I acquired by the second camera 109 that can image upward. Therefore, the detected feature points C and Ct are distributed over the entire acquired image, as shown in FIG. 6(I). On the other hand, when the unmanned aerial vehicle 1 reaches near the side edge 31 of the bridge (for example, point B in FIG. 6), the image I acquired by the second camera 109 shows that the bridge 3 is imaged, and the sky is imaged on the advancing direction side of the unmanned aircraft 1, so the detected feature points C and Ct are, as shown in (II) of FIG. 6, of the acquired image I. The feature points C and Ct are distributed in a region opposite to the direction of movement of the unmanned aircraft 1, and feature points C and Ct are not distributed on the side of the direction of movement of the unmanned aircraft 1 in the acquired image I. That is, in the acquired image I, the region in which the feature points C and Ct are distributed is biased to a predetermined extent toward the opposite side of the direction of travel of the unmanned aircraft 1.

Therefore, when the detected feature points C and Ct are distributed over the entire acquired image I when flying a crossing route using this distribution of feature points, the unmanned aircraft 1 3, and if the area where the feature points C and Ct are distributed is biased to the opposite side of the direction of travel of the unmanned aircraft 1 to a predetermined extent, the unmanned aircraft 1 is flying on the side of the bridge 3. It can be determined that the aircraft is flying near the edge 31, that is, near the next waypoint.

On the other hand, in self-position estimation using Visual SLAM, errors accumulate as the flight distance increases, but when positive errors accumulate, even if the actual self-position reaches the next waypoint, the estimated self-position If a situation arises in which the user has not yet reached the waypoint and negative errors accumulate, when the estimated self-position reaches the waypoint on the flight plan path, the actual self-position will be before the waypoint. A situation arises.

Therefore, the estimated position correction unit 207 adjusts the self-position estimated by the VSLAM unit 203 to the first point in the feature point area (S105) before the self-position estimated by the VSLAM unit 203 reaches the waypoint (for example, P3) while flying on the crossing route (for example, T2). Of the detected feature points C calculated in step S103, the point Ct closest to the moving direction of the aircraft, which is a representative point of When it is detected that the aircraft 1 has entered the region R1 on the opposite side to the direction of travel ((III) in FIG. 6) (S107), the estimated position of the unmanned aircraft 1 at that time is set as the position of the waypoint (for example, P3). (S113). The estimated position may be corrected not to the position of the waypoint itself but to a position corresponding to any appropriate waypoint in the vicinity of the waypoint.

In addition, when the self-position estimated by the VSLAM unit 201 reaches a waypoint (for example, P3) while flying on a crossing route (for example, T2), the estimated position correction unit 205 adjusts the detected position calculated in step S103. Of the feature points C, the point Ct closest to the direction of movement of the aircraft is located in the second region in the acquired image, which is the region R2 on the side of the direction of movement of the unmanned aircraft 1 from the center of the image. If detected ((I), (II) in FIG. 6) (S109), an instruction is given to the control signal generation unit 122 to advance the unmanned aircraft 1 in the same direction of travel without changing the direction of travel (S111), Of the detected feature points C calculated in step S103, the point Ct closest to the aircraft traveling direction, which is the second representative point of the feature point region, is the second representative point in the acquired image I. When it is detected that the unmanned aerial vehicle 1 is out of the area R2 on the advancing direction side from the center of the image ((III) in FIG. 6) (S109), the estimated position of the unmanned aerial vehicle 1 at that time is set as a waypoint (for example, P3). ) to the position (S113).

In the above embodiment, an example has been described in which the parameter related to the area where the extracted feature points are distributed is the point closest to the aircraft's traveling direction among the extracted feature points. Parameters related to the area to be detected are not limited to these, but include the fact that the area in which the feature points are distributed is biased to a predetermined degree or more in the direction of movement of the unmanned aircraft or to the side opposite to the direction of movement in the image. can be any suitable parameter that is indicative of the .

As such a parameter, for example, a parameter related to the area of a region in which extracted feature points are distributed can be considered. In other words, referring to (I) to (III) in Figure 6, when an unmanned aircraft is flying a cross route, there is no region where feature points are distributed only on the advancing direction side of the image. For example, the smaller the parameter related to the area of the region where the feature points are distributed, such as the area of the region where the feature points are distributed, or the ratio of the area of the region where the feature points are distributed to the area of the entire area of the image, the more the feature points will be distributed. The extent to which the area is biased toward the opposite side of the direction in which the unmanned aircraft is traveling increases. Therefore, in this case, while the unmanned aircraft is flying a cross route, before the estimated position of the unmanned aircraft estimated by self-position estimation reaches the waypoint, the parameter regarding the area of the region where the feature points are distributed is When the estimated position of the unmanned aircraft is detected to be below the threshold, the estimated position of the unmanned aircraft at that time is corrected so that it corresponds to the waypoint, and the unmanned aircraft adjusts its own position while flying the crossing route. When the estimated position of the unmanned aircraft reaches the waypoint, and it is detected that the parameter related to the area of the area where the feature points are distributed is equal to or larger than the second threshold, the unmanned aircraft is moved in the direction of travel. When it is detected that the parameter related to the area of the region where the feature points are distributed has become smaller than the second threshold, the unmanned aircraft is adjusted so that the estimated position of the unmanned aircraft at that time corresponds to the waypoint. The configuration may be such that the estimated position of the aircraft is corrected.

In addition, in the above embodiment, the parameter regarding the area where the extracted feature points are distributed is the point closest to the direction of travel of the unmanned aircraft among the extracted feature points, but the parameter regarding the area where the extracted feature points are distributed is It is also possible to use a representative point of a region where any other suitable feature points are distributed, such as the center of gravity or a weighted average of a plurality of extracted feature points closest to the direction of travel of the unmanned aircraft.

Further, in the above embodiment, the first representative point and the second representative point are the same, but they may be different.

Further, in the above embodiment, the first area is the area opposite to the direction of movement of the unmanned aircraft 1 from the center of the image, and the second area is the area on the side of the movement direction of the unmanned aircraft 1 from the center of the image. Although not limited thereto, any other suitable region can be used.

Further, in the above embodiment, the cross route is a route extending in a direction perpendicular to the longitudinal direction of the bridge, but it is not limited to this, and may be in any other suitable direction. Each traversal path can be in a different direction. Further, the crossing route may not be a straight route but may be a curved route.

According to this embodiment, the error that accumulates in the self-position estimated by Visual SLAM can be corrected at a predetermined timing.

When negative errors accumulate in self-position estimation using Visual SLAM, when the estimated self-position reaches a waypoint on the flight plan route, the actual self-position will be before the waypoint. If the bridge is turned back, a portion that will not be inspected will remain on the end side of the bridge, but according to this embodiment, the unmanned aircraft 1 is advanced to the end of the bridge, so it is possible to avoid leaving any portion that will not be inspected.

Additionally, when positive errors accumulate in self-position estimation using Visual SLAM, even if the actual self-position reaches the waypoint, the estimated self-position has not yet reached the waypoint, so the aircraft will continue to move forward. Go in the direction. As a result, the aircraft protrudes outside under the bridge, and the captured image shows only the sky, making it impossible to detect feature points, causing it to lose its direction and become unable to fly autonomously. According to this embodiment, it is possible to prevent such a direction from being lost.

Although the present invention has been described above with respect to several embodiments for illustrative purposes, the present invention is not limited thereto, and may be modified in form and detail without departing from the scope and spirit of the present invention. It will be apparent to those skilled in the art that various changes and modifications can be made.

1 Unmanned aerial vehicle 101 Control unit 102 Motor 103 Rotor 104 Arm 105 Landing legs 106 Pedestal 107 Support member 108 First camera 109 Second camera 110 Flight position sensor 111 Attitude sensor 112 Direction sensor 113 Altitude sensor 115 Distance sensor 117 Antenna 120 Information Processing unit 121 Communication circuit 122 Control signal generation section 123 Speed controller 124 VSLAM unit 124a CPU
124b RAM
124c ROM
124d External memory 124e Interface 124f System bus 125 Interface 2 Estimated position correction system 201 VSLAM section 203 Feature point area parameter calculation section 205 Estimated position correction section 207 Storage section 3 Bridge 31 Side edge RT Flight plan route T1, T2, T3 Crossing route J1 , J2, J3 Connecting routes P1, P2, P3 Waypoint I Image C Feature point Ct Feature point CA closest to the aircraft's traveling direction Feature point region R1 First region R2 Second region

Claims (15)

An unmanned aircraft that is equipped with an imaging device capable of capturing an image above, extracts feature points from the image acquired by the imaging device, and flies while estimating its own position based on the feature points, traverses a plurality of crossing paths across the bridge. When flying while imaging the bridge from below the bridge based on a flight plan path formed by interconnecting the ends of the bridge via a connecting path extending along the side edges of the bridge, A system for correcting the estimated position of the unmanned aerial vehicle, the system comprising:
The connection route has a first end and a second end in the order of flight of the unmanned aircraft,
In the flight plan path, the first end is set as a waypoint,
While the unmanned aircraft is flying along the crossing route, before the estimated position of the unmanned aircraft estimated by the self-position estimation reaches the waypoint, the area where the feature points are distributed is When it is detected that the unmanned aerial vehicle is biased toward the opposite side of the traveling direction of the unmanned aerial vehicle by a predetermined degree or more, the estimated position of the unmanned aerial vehicle at that time corresponds to the waypoint. and a system for correcting the estimated position of the unmanned aerial vehicle. The fact that the region in which the feature points are distributed is biased toward the opposite side of the traveling direction of the unmanned aircraft by a first predetermined degree or more in the image means that the region in which the feature points are distributed is The first representative point has entered a first region in the image on the opposite side to the direction of travel of the unmanned aircraft, and the first representative point has entered the region in which the feature points are distributed. 2. The system according to claim 1, wherein in the image, the index corresponds to the fact that the unmanned aerial vehicle is biased toward the opposite side of the traveling direction of the unmanned aircraft by more than the first predetermined degree. 3. The system according to claim 2, wherein the first representative point of the region where the feature points are distributed is a point closest to the direction of travel of the unmanned aircraft among the extracted feature points. The fact that the region in which the feature points are distributed is biased toward the opposite side of the traveling direction of the unmanned aircraft by a first predetermined degree or more in the image means that the region in which the feature points are distributed is The system of claim 1, wherein a parameter related to the area of is less than or equal to a first threshold value. An unmanned aircraft that is equipped with an imaging device capable of capturing an image above, extracts feature points from the image acquired by the imaging device, and flies while estimating its own position based on the feature points, traverses a plurality of crossing paths across the bridge. When flying while imaging the bridge from below the bridge based on a flight plan path formed by interconnecting the ends of the bridge via a connecting path extending along the side edges of the bridge, A system for correcting the estimated position of the unmanned aerial vehicle, the system comprising:
The connection route has a first end and a second end in the order of flight of the unmanned aircraft,
In the flight plan route, the first end is set as a waypoint,
When the estimated position of the unmanned aircraft estimated by the self-position estimation reaches the waypoint while the unmanned aircraft is flying along the crossing route, the area where the feature points are distributed becomes When it is detected that the unmanned aircraft is biased toward the direction of travel of the unmanned aircraft by a predetermined degree or more, the unmanned aircraft is allowed to proceed in the direction of travel, and the area where the feature points are distributed is When it is detected in the image that the unmanned aerial vehicle is no longer present in the direction of travel by a second predetermined degree or more, the estimated position of the unmanned aerial vehicle at that time is set to the waypoint. A system for correspondingly modifying the estimated position of the unmanned aerial vehicle. The fact that the region in which the feature points are distributed is biased towards the direction of travel of the unmanned aircraft by a second predetermined degree or more in the image means that detecting that the representative point was located in a second region on the traveling direction side of the unmanned aircraft in the image;
This corresponds to the fact that the region in which the feature points are distributed no longer exists in the image in a biased direction toward the direction of travel of the unmanned aircraft by a second predetermined degree or more. detecting that the second representative point is out of the second area in the image ;
The second representative point is an indicator that corresponds to the fact that the area in which the feature points are distributed is biased towards the direction of travel of the unmanned aircraft by more than the second predetermined degree in the image. The system according to claim 5. 7. The system according to claim 6, wherein the second representative point of the region where the feature points are distributed is a point closest to the direction of travel of the unmanned aircraft among the extracted feature points. The fact that the area where the feature points are distributed is biased toward the direction of travel of the unmanned aircraft by a second predetermined degree or more in the image is related to the area of the area where the feature points are distributed. detecting that the parameter is equal to or higher than a second threshold;
This corresponds to the fact that the region in which the feature points are distributed no longer exists in the image in a biased direction toward the direction of travel of the unmanned aircraft by a second predetermined degree or more. 6. The system according to claim 5, wherein the detection is that a parameter related to area has become smaller than the second threshold. An unmanned aircraft that is equipped with an imaging device capable of capturing an image above, extracts feature points from the image acquired by the imaging device, and flies while estimating its own position based on the feature points, traverses a plurality of crossing paths across the bridge. When flying while imaging the bridge from below the bridge based on a flight plan path formed by interconnecting the ends of the bridge via a connecting path extending along the side edges of the bridge, A system for correcting the estimated position of the unmanned aerial vehicle, the system comprising:
The connection route has a first end and a second end in the order of flight of the unmanned aircraft,
In the flight plan route, the first end is set as a waypoint,
While the unmanned aircraft is flying along the crossing route, before the estimated position of the unmanned aircraft estimated by the self-position estimation reaches the waypoint, the area where the feature points are distributed is When it is detected that the unmanned aerial vehicle is biased toward the opposite side of the direction of travel of the unmanned aerial vehicle by a predetermined degree or more, the estimated position of the unmanned aerial vehicle at that time corresponds to the waypoint. correcting the estimated position of the unmanned aerial vehicle;
When the unmanned aerial vehicle is flying along the crossing route and the estimated position of the unmanned aerial vehicle estimated by the self-position estimation reaches the waypoint, the area where the feature points are distributed becomes the When it is detected that the unmanned aircraft is biased towards the direction of travel of the unmanned aircraft by a predetermined degree or more, the unmanned aircraft is allowed to proceed in the direction of travel, and the area where the feature points are distributed is When it is detected in the image that the unmanned aerial vehicle is no longer present toward the direction of travel of the unmanned aerial vehicle by a second predetermined degree or more, the estimated position of the unmanned aerial vehicle at that time is set to the waypoint. A system for correspondingly modifying the estimated position of the unmanned aerial vehicle. An unmanned aircraft equipped with the system according to any one of claims 1 to 9. An unmanned aircraft that is equipped with an imaging device capable of capturing an image above, extracts feature points from the image acquired by the imaging device, and flies while estimating its own position based on the feature points, traverses a plurality of crossing paths across the bridge. When flying while imaging the bridge from below the bridge based on a flight plan path formed by interconnecting the ends of the bridge via a connecting path extending along the side edges of the bridge, A method of correcting an estimated position of an unmanned aerial vehicle performed by a computer, the method comprising:
The connection route has a first end and a second end in the order of flight of the unmanned aircraft,
In the flight plan route, the first end is set as a waypoint,
While the unmanned aircraft is flying along the crossing route, before the estimated position of the unmanned aircraft estimated by the self-position estimation reaches the waypoint, the area where the feature points are distributed is When it is detected that the unmanned aerial vehicle is biased toward the opposite side of the direction of travel of the unmanned aerial vehicle by a predetermined degree or more, the estimated position of the unmanned aerial vehicle at that time corresponds to the waypoint. and a method for correcting the estimated position of the unmanned aerial vehicle. An unmanned aircraft that is equipped with an imaging device capable of capturing an image above, extracts feature points from the image acquired by the imaging device, and flies while estimating its own position based on the feature points, traverses a plurality of crossing paths across the bridge. When flying while imaging the bridge from below the bridge based on a flight plan path formed by interconnecting the ends of the bridge via a connecting path extending along the side edges of the bridge, A method of correcting an estimated position of an unmanned aerial vehicle performed by a computer, the method comprising:
The connection route has a first end and a second end in the order of flight of the unmanned aircraft,
In the flight plan route, the first end is set as a waypoint,
When the estimated position of the unmanned aircraft estimated by the self-position estimation reaches the waypoint while the unmanned aircraft is flying along the crossing route, the area where the feature points are distributed becomes When it is detected that the unmanned aircraft is biased toward the direction of travel of the unmanned aircraft by a predetermined degree or more, the unmanned aircraft is allowed to proceed in the direction of travel, and the area where the feature points are distributed is When it is detected in the image that the unmanned aerial vehicle is no longer present in the direction of travel by a second predetermined degree or more, the estimated position of the unmanned aerial vehicle at that time is set to the waypoint. A method for correspondingly modifying the estimated position of the unmanned aerial vehicle. An unmanned aircraft that is equipped with an imaging device capable of capturing an image above, extracts feature points from the image acquired by the imaging device, and flies while estimating its own position based on the feature points, traverses a plurality of crossing paths across the bridge. When flying while imaging the bridge from below the bridge based on a flight plan path formed by interconnecting the ends of the bridge via a connecting path extending along the side edges of the bridge, A method of correcting an estimated position of an unmanned aerial vehicle performed by a computer, the method comprising:
The connection route has a first end and a second end in the order of flight of the unmanned aircraft,
In the flight plan route, the first end is set as a waypoint,
While the unmanned aircraft is flying along the crossing route, before the estimated position of the unmanned aircraft estimated by the self-position estimation reaches the waypoint, the area where the feature points are distributed is When it is detected that the unmanned aerial vehicle is biased toward the opposite side of the direction of travel of the unmanned aerial vehicle by a predetermined degree or more, the estimated position of the unmanned aerial vehicle at that time corresponds to the waypoint. correcting the estimated position of the unmanned aerial vehicle;
When the estimated position of the unmanned aircraft estimated by the self-position estimation reaches the waypoint while the unmanned aircraft is flying along the crossing route, the area where the feature points are distributed becomes When it is detected that the unmanned aircraft is biased toward the direction of travel of the unmanned aircraft by a predetermined degree or more, the unmanned aircraft is allowed to proceed in the direction of travel, and the area where the feature points are distributed is When it is detected in the image that the unmanned aerial vehicle is no longer present in the direction of travel by a second predetermined degree or more, the estimated position of the unmanned aerial vehicle at that time is set to the waypoint. A method for correspondingly modifying the estimated position of the unmanned aerial vehicle. A program for causing a computer to execute the method according to any one of claims 11 to 13. A computer-readable recording medium recording the program according to claim 14.

Images