JP2021028188A - Drone imaging device and method - Google Patents

Abstract

To take high-quality images efficiently in filming using a drone.SOLUTION: In making a film of a wall surface using a camera fixed on a drone supported by a support member including two wheels and an axle, a distance between the camera and the wall surface 80 is calculated by resetting a yaw angular velocity included in gyro data to a reference value when both wheels contact the wall surface; a yaw angle θY is calculated based on the yaw angular velocity in which the amount of deviation of the reference value is corrected when one wheel contacts the wall surface; a distance DY to the wall surface 80 in relation to the yaw angle θY is calculated by a formula DY=R+LtanθY using the radius of wheel R, the length L of the axle, and the calculated yaw angle θY; and the distance D to the wall surface 80 is calculated by synthesizing the distance DY and a distance DP to the wall surface 80 relevant to a pitch angle θP.SELECTED DRAWING: Figure 11

Description

The disclosed technique relates to a drone photographing device and a drone photographing method.

In recent years, images of the wall surface have been used to inspect cracks and the like that have occurred on the wall surface of structures such as bridges. By using a drone equipped with a camera to capture this image, work efficiency and cost reduction are expected. For example, the wall surface of a bridge or the like is continuously photographed by a camera fixed to a drone, and these images are combined into one panoramic image.

When the image taken by the camera is used as a panoramic image for soundness diagnosis such as inspection of cracks, it is necessary to take an image in focus so that a crack of about 0.1 mm can be confirmed. In addition, the cameras mounted on the drones that may fall are often lightweight and inexpensive, and images are taken by the focus adjustment left to the autofocus of such inexpensive cameras. In this case, for example, by taking a close-up shot of a shooting surface that is close to a single gray color, such as a concrete wall surface, a blurred image that is out of focus may be obtained.

Since it is not possible to inspect cracks and the like in a blurred image, conventionally, a person visually inspects all or some of the captured images to determine whether or not the image is a blurred image, and in the case of a blurred image. The image is being retaken.

In addition, as a technique related to drones, a distance measuring device that can obtain information for safely moving a moving object has been proposed. This distance measuring device is a distance measuring device mounted on a moving body, and is attached to the moving body via a gimbal. Then, this distance measuring device detects a distance measuring system that measures the distance information to the object, and posture change information that is information on the relative posture change between the distance measuring system and the moving body due to the movement of the moving body. It is equipped with a detection system. Further, this distance measuring device includes a control calculation unit that corrects the distance information by using the attitude change information.

Further, a measuring device for measuring the moving speed of an air vehicle with high accuracy even at a low speed has been proposed. The computer of this measuring device integrates the measured values obtained from the acceleration sensor of the IMU (inertial measurement unit) when the drone is moving in the air to calculate the moving speed of the drone. In addition, the computer makes the drone rest in the air when the estimated value of the measurement error of the acceleration sensor of the IMU exceeds the threshold value while the drone is moving in the air. Further, the computer corrects the measurement error of the acceleration sensor of the IMU based on the measured value obtained from the rotation speed sensor, and then restarts the movement of the drone.

JP-A-2017-227516 JP-A-2018-179579

As described above, in shooting using a drone, it is difficult to mount a high-performance and expensive camera on the drone, considering that it is mounted on a drone that may fall.

It is also conceivable to focus with an inexpensive camera by measuring the distance to the object as in the conventional technique. However, in this case, a special ranging system is required in addition to the sensor built in the drone, and there are problems that the flight time is limited by the weight and the drone becomes expensive.

Further, when a blurred image is found by visual confirmation and re-shooting using a drone is performed, there is a problem that work efficiency such as inspection is lowered.

As one aspect, the disclosed technology aims to efficiently capture a high-quality image in shooting using a drone.

In one embodiment, the disclosed technique includes a photographing unit that is fixed to the drone and captures an image, and a support unit that supports the drone's posture with respect to the photographing surface so as to maintain a predetermined reference posture. Further, the disclosed technique includes a discriminating unit for determining whether or not the posture of the drone has changed from the reference posture. Further, the disclosed technique includes a correction unit that corrects the angular velocity acquired by the gyro sensor built in the drone to a reference value when the drone is determined to be in the reference posture by the determination unit. .. Further, the disclosed technique is calculated by a calculation unit that calculates the distance from the imaging unit to the imaging surface using a value based on the angular velocity corrected to the reference value by the correction unit, and the calculation unit. It includes an adjusting unit that adjusts the focus of the photographing unit based on the distance.

As one aspect, there is an effect that an image with good image quality can be efficiently taken in shooting using a drone.

It is a figure for demonstrating the use scene of the drone photographing apparatus which concerns on this embodiment. It is external perspective view of the drone photographing apparatus which concerns on this embodiment. It is a figure for demonstrating the reference posture of a drone. It is a figure for demonstrating the reference posture of a drone. It is a figure for demonstrating the reference posture of a drone. It is a figure for demonstrating the reference posture of a drone. It is a figure for demonstrating the change from the reference posture of a drone. It is a block diagram which shows the part about the functional structure of the drone photographing apparatus which concerns on this embodiment. It is a figure which shows an example of the fluctuation of the yaw angular velocity. It is a figure for demonstrating the correction of the deviation amount of the reference value of a yaw angular velocity. It is a figure for demonstrating the calculation of the distance to a wall surface. It is a figure for demonstrating the calculation of the distance to a wall surface. It is a figure for demonstrating the calculation of the distance to a wall surface. It is a block diagram which shows the schematic structure of the computer which functions as the control part of the drone included in the drone photographing apparatus. It is a flowchart which shows an example of a drone shooting process. It is the schematic which shows the other configuration example of the drone photographing apparatus. It is the schematic which shows the other configuration example of the drone photographing apparatus.

Hereinafter, an example of the embodiment according to the disclosed technology will be described with reference to the drawings. In the present embodiment, a drone photographing device for photographing a concrete wall surface such as a bridge by using a drone will be described.

As shown in FIG. 1, the drone photographing apparatus 10 according to the present embodiment is used at a site where a structure to be inspected such as a bridge exists. An image taken by the drone photographing apparatus 10, for example, a crack image for inspecting a crack on a concrete wall surface is stored in, for example, a storage installed in an office. In addition, the 3D model of the structure to be inspected, other inspection records, sensor data, etc. are also stored in the storage of the office. The various information stored in the storage is used by various applications that operate on an information processing device installed in the office or a tablet terminal used in the field. Applications used in the office include, for example, ortholation of crack images, bridge abnormality monitoring based on crack images, sensor data analysis, asset management, and the like. In addition, applications used in the field include applications such as inspection using a tablet terminal and automatic detection of cracked damaged parts.

FIG. 2 shows an external perspective view of the drone photographing apparatus 10 according to the present embodiment. As shown in FIG. 2, the drone photographing apparatus 10 includes a drone 20, a support portion 32, pressure sensors 34A and 34B, and a camera 40.

The drone 20 is an unmanned flying object that flies by remote control or autonomous control. In the present embodiment, the drone 20 moves in a predetermined direction (for example, upward or downward) along the wall surface 80 while applying a load in the direction in which the wheels 32A and 32B of the support portion 32 described later come into contact with the wall surface 80. It is controlled to do. Since a general configuration can be adopted for the flight mechanism of the drone 20 (propeller, communication unit, control mechanism thereof, etc.), detailed description thereof will be omitted in the present embodiment.

The support portion 32 supports the drone 20 so that the posture of the drone 20 with respect to the wall surface maintains a predetermined reference posture. Specifically, the support portion 32 includes the wheels 32A and 32B and the axle 32C, the wheels 32A and 32B are attached to both ends of the axle 32C, and the drone 20 is fixed to the central portion of the axle 32C in the length direction. Wheel. As described above, when the drone 20 is controlled to move in a predetermined direction along the wall surface 80 while applying a load in the direction in which the wheels 32A and 32B come into contact with the wall surface 80, the support portion 32 is a reference. Support the drone 20 to maintain its posture.

The reference posture of the drone 20 will be described with reference to FIGS. 3 to 6. FIG. 3 is a schematic view of the wall surface 80, the drone 20, and the wheels 32A and 32B as viewed from the side surface, and FIG. 4 is a schematic view as viewed from the upper surface. As shown in FIGS. 3 and 4, the radius of the wheels 32A and 32B is R, the length from the center of the axle 32C to the wheels 32A and 32B (hereinafter, simply referred to as “the length of the axle 32C”) is L, and the camera 40. Let H be the length from the center of the lens to the center line of the axle 32C. These values are information for specifying the positional relationship between the drone 20 and the wall surface 80, and are used for calculating the distance to the wall surface 80 at the time of one wheel, which will be described later.

As shown in FIGS. 3 and 4, the posture of the drone 20 in a state where both wheels are in contact with the wall surface 80 is used as a reference posture. In the present embodiment, the normal direction of the wall surface 80 is the x-axis, the horizontal direction parallel to the wall surface 80 is the y-axis, the vertical direction is the z-axis, the rotation angle around the x-axis is the roll angle, and the rotation angle around the y-axis is. Let the pitch angle and the rotation angle around the z-axis be the yaw angle. Therefore, the state in which both wheels are in contact with the wall surface 80 is when the yaw angle of the drone 20 is 0 °.

In the present embodiment, if both wheels are in contact with the wall surface, the pitch angle θ _{P of the drone 20 is larger than 0 ° as shown in FIG. 5, and the roll angle θ R} of the drone 20 is as shown in FIG. Even if is greater than 0 °, the drone 20 shall be in the reference position. As shown in FIG. 7, when one wheel (wheel 32B in the example of FIG. 7) is not in contact with the wall surface 80, the yaw angle θ _Y of the drone 20 becomes larger than 0 °, and the drone 20 is in the reference posture. Indicates that it has changed.

The pressure sensors 34A and 34B are installed on the axle 32C portion to which the wheels 32A and 32B are attached, detect the pressure generated between each of the wheels 32A and 32B and the wall surface 80, and output the detected sensor value.

The camera 40 is fixed to the drone 20 and adjusts the focus based on the set focal length to take an image. The distance on the optical axis from the image receiving surface of the camera 40 to the wall surface 80 when the posture of the drone 20 is _{0 ° for all of the roll angle θ R} , the pitch angle θ _P , and the yaw angle θ _{Y (hereinafter, simply “wall surface 80”).} Distance to) is set as a fixed focal length. When the posture of the drone 20 changes, the distance to the wall surface 80 is calculated (details will be described later), the calculated distance is set as the focal length, and the focus is adjusted.

In the following, for the sake of simplicity, the image receiving surface of the camera 40 is at the center of the drone 20, and the posture of the drone 20 is 0 for the roll angle θ _R , the pitch angle θ _P , and the yaw angle θ _Y. It will be described as being positioned so as to be parallel to the wall surface 80 in the state of °.

The image data showing the image taken by the camera 40 may be stored in a storage medium such as an SD card inserted in the camera 40, or may be output to the drone 20 and stored in the storage area of the drone 20. May be good. Further, it may be transmitted to an external storage device via a communication line and stored.

Next, with reference to FIG. 8, a part related to the functional configuration of the drone photographing apparatus 10 according to the present embodiment will be described.

As shown in FIG. 8, the drone 20 includes a gyro sensor unit 22 and a control unit 24.

The 22 gyro sensor unit includes, for example, a 3-axis gyro sensor, an acceleration sensor, and a magnetic sensor. The 3-axis gyro sensor detects angular velocities (roll angular velocity, pitch angular velocity, and yaw angular velocity) around each of the x-axis, y-axis, and z-axis. The roll angular velocity and pitch angular velocity detected by the 3-axis gyro sensor are corrected mainly by using the values detected by the acceleration sensor, and the yaw angular velocity is corrected mainly by using the values detected by the magnetic sensor. To. The gyro sensor unit 22 outputs the corrected angular velocity values as gyro data.

Functionally, as shown in FIG. 8, the control unit 24 includes a discrimination unit 25, a correction unit 26, a gyro data DB (Database) 27, and a calculation unit 28.

The determination unit 25 acquires the sensor values output from the pressure sensors 34A and 34B, and determines whether or not the posture of the drone 20 has changed from the reference posture based on the acquired sensor values. Specifically, the discriminating unit 25 determines whether or not the wheels 32A and 32B are in contact with the wall surface 80 under predetermined conditions.

More specifically, the discriminating unit 25 compares the sensor values acquired from the pressure sensors 34A and 34B with a predetermined threshold value, and determines whether or not each of the wheels 32A and 32B is in contact with the wall surface 80. Determine. When both wheels are in contact with the wall surface 80, it means that the yaw angle θ _Y of the drone 20 is 0 °, that is, the drone 20 is in the reference posture. On the other hand, when one of the wheels 32A and 32B is not in contact with the wall surface 80, the yaw angle θ _Y of the drone 20 is larger than 0 °, that is, the drone 20 has changed from the reference posture. Represents that.

In the present embodiment, the drone 20 is controlled so that the wheels 32A and 32B move while applying a load in the direction in which the wheels 32A and 32B are in contact with the wall surface 80. Therefore, basically, both wheels are in contact with the wall surface 80. That is, the reference posture can be maintained. However, due to a disturbance such as a gust, the contact with the wall surface 80 may become one wheel, and in this embodiment, even in such a case, it is possible to take an in-focus image without retaking the image. Is what you want.

The determination unit 25 notifies the correction unit 26 and the calculation unit 28 of the determination result of whether or not the posture of the drone 20 has changed from the reference posture.

If both wheels are not in contact with the wall surface 80, the distance to the wall surface 80, which will be described later, cannot be calculated. Therefore, the discriminating unit 25 outputs an instruction to return the position of the drone 20 to the non-contact position in order to retake the image. The instruction can be output by displaying a message to the controller for remote control of the drone 20, blinking an LED (Light Emitting Diode) lamp provided on the controller or the drone 20 itself, and the like. According to this instruction, the user steers the drone 20 and returns it to the non-contact position. Further, a GPS (Global Positioning System) is provided on the drone 20, and the determination unit 25 stores the position information of the drone 20 determined by GPS and the determination result of the contact between the wheels 32A and 32B and the wall surface 80 in association with each other. You may leave it. In this case, the discrimination unit 25 may output control information for returning the drone 20 to the position where the two wheels are non-contact when the non-contact between the two wheels is determined, and autonomously control the drone 20. ..

When the discrimination result notified from the discrimination unit 25 indicates that the posture of the drone 20 is the reference posture, the correction unit 26 acquires the gyro data output from the gyro sensor unit 22. Then, the correction unit 26 stores the yaw angular velocity included in the acquired gyro data in the gyro data DB 27, corrects the yaw angular velocity acquired from the gyro sensor unit 22 to a reference value (for example, 0), and calculates the calculation unit. Hand over to 28. That is, the correction unit 26 constantly resets while maintaining the history of the yaw angular velocity detected by the gyro sensor unit 22 while the drone 20 maintains the reference posture.

The reason for constantly resetting the yaw angular velocity will be described. In the region close to the structure (wall surface 80), the geomagnetism is disturbed due to the influence of the reinforcing bars in the structure. Therefore, when the drone 20 is approaching the structure (wall surface 80), the yaw angular velocity detected by the gyro sensor unit 22 is corrected by using the value detected by the magnetic sensor, so that the error increases. There is. Therefore, when both wheels are in contact with the wall surface 80, the value of the yaw angular velocity corrected mainly by using the value detected by the magnetic sensor is reset. Since the disturbance of the geomagnetism has little influence on the value detected by the acceleration sensor, the roll angular velocity and the pitch angular velocity, which are mainly corrected by the value detected by the acceleration sensor, are not reset.

The calculation unit 28 calculates the distance to the wall surface 80 using a value based on the yaw angular velocity corrected to the reference value by the correction unit 26. Specifically, the calculation unit 28 uses the reference value when the drone 20 is in the reference posture as a value based on the yaw angular velocity corrected to the reference value, and when the posture of the drone 20 is changed from the reference posture, Use the value corrected based on the reference value.

An example of the value corrected based on the reference value will be described. For example, based on the yaw angular velocity history stored in the gyro data DB 27, the amount of deviation of the reference value (origin, for example, “0”) is predicted while the posture of the drone 20 is changing from the reference posture. Then, the calculation unit 28 uses a value obtained by correcting the deviation amount of the predicted reference value with respect to the yaw angular velocity acquired from the gyro sensor unit 22 as a corrected value based on the reference value.

The reference value of the yaw angular velocity output from the gyro sensor unit 22 will fluctuate due to changes in temperature and atmospheric pressure unless corrected. Therefore, while the two wheels are in contact with each other, the yaw angular velocity θ _Y can be kept at 0 ° by continuously resetting the yaw angular velocity to the reference value as described above. However, it is necessary to correct the amount of deviation of the origin in consideration of the deviation of the origin when measuring the yaw angle due to one wheel. An example of correction of the amount of deviation of the origin is shown below.

FIG. 9 shows an example of the variation (t-ω coordinate system) of the yaw angular velocity ω from the time when both wheels are in contact to one wheel and the time when both wheels are in contact (end of one wheel). Since the wheels 32A and 32B are actually in contact with the wall surface 80 during the period from the contact between the two wheels to the one wheel, the yaw angular velocity ω was acquired from the gyro sensor unit 22 even though the displacement was 0. The yaw angular velocity ω tends to increase. _{Therefore, if the yaw angle θ Y} is calculated based on the detected yaw angular velocity ω from the start of one wheel without correcting the origin, an angle larger than the actual angle can be obtained.

Therefore, the calculation unit 28 determines the origin of the yaw angular velocity ω from the history of the yaw angular velocity ω at the time of contact between the two wheels, that is, the time change of the yaw angular velocity ω before the reset detected at each time, which is stored in the gyro data DB 27. Find the regression line that indicates the displacement. Specifically, the calculation unit 28 uses the yaw angular velocity ω from the time t ₁ to t _{n , where} the time at the moment of becoming one wheel is t n, the past time of a certain period is t _1, and the yaw angular velocity ω is used. The slope a of the regression line shown in Eq. (1) below is obtained by the least squares method.

(1) In the formula, n 2 variables (t, omega) number of, t _i and omega _i are individual values, t ^- (in a formula overlined above the "t") and omega ^- in (formula Overline over "ω") is the average value of each.

The calculation unit 28 predicts the value of the regression line at each time after the start of one wheel in the coordinate system whose origin is the point at the start of one wheel on the obtained regression line as the amount of deviation of the origin, and determines the value of the origin. _{The yaw angle θ Y} is calculated from the yaw angle velocity corrected for the deviation amount. For example, the calculation unit 28 integrates the difference between the yaw angular velocity ω acquired from the gyro sensor unit 22 and the obtained regression line at regular time intervals to calculate _{the yaw angular velocity θ Y at regular time intervals.}

A more specific description will be given with reference to FIG. 10, which is an enlarged portion (within the dotted line frame of FIG. 9) _{from t n} at the start of one wheel to t _{m at the end of one wheel in FIG.} FIG. 10 shows the coordinate system (t'−ω' coordinate system) having the point _{t n} at the start of one wheel on the regression line as the origin. Calculating unit 28, as shown in the following equation (2), the yaw angle theta _Y every predetermined time τ from one wheel at the start _{t n} to one wheel end _{t 's (t' s =} t m), It is calculated as the integrated value of the difference between the function of the angular velocity ω and the regression line.

As described above, by calculating a correction to the yaw angle theta _Y deviation of the origin of the yaw angular velocity ω time varying, it is possible to calculate an accurate yaw angle theta _Y, using the calculated yaw angle theta _Y Therefore, the distance to the wall surface 80 can be calculated accurately.

Calculating unit 28, when one wheel contact, as shown in FIG. 11, the wheels 32A, and the radius R of 32B, and the length L of the axle 32C, calculated using the yaw angle theta _Y, about the yaw angle theta _Y the distance _{D Y} from the wall _80, and calculates a D _Y = R + Ltanθ Y. _{Further, the calculation unit 28 calculates the pitch angle θ P} from the pitch angular velocity included in the gyro data acquired from the gyro sensor unit 22. Further, as shown in FIGS. 12 and 13, _{the value of the pitch angle θ P} is 0 ° on the positive direction of the Z axis, the rotation in the wall surface 80 direction is in the positive direction (counterclockwise in FIGS. 12 and 13), and the wall surface. The rotation in the direction away from 80 is defined as the negative direction (clockwise in FIGS. 12 and 13). Then, the calculation unit 28 uses the radius R of the wheels 32A and 32B, the length H from the center of the lens of the camera 40 to the center line of the axle 32C, and the calculated pitch angle θ _P, and the pitch angle θ _P. the distance _{D P} to the wall surface 80 about _to calculate the _{D P = R / cosθ P -Htanθ} P. The calculation unit 28 synthesizes the _{D Y} and _{D P,} and calculates the distance D from the wall 80.

Further, the calculation unit 28 calculates the distance D in the same manner as described above by using the yaw angular velocity corrected to the reference value delivered from the correction unit 26 at the time of contact between the two wheels. That is, since the yaw angle θ _Y = 0 °, _DY = R.

In this embodiment, _{the change in the roll angle θ R} does not affect the change in the distance to the wall surface 80, and therefore is not used in the calculation of the distance D. Incidentally, by using the roll angle theta _R, it may be added to processing such as performing the rotation correction of an image, such as to match the longitudinal direction of the vertical direction and the image.

The calculation unit 28 outputs the calculated distance D to the wall surface 80. In the present embodiment, the case where the above distance D is calculated assuming that the image receiving surface of the camera 40 is parallel to the wall surface 80 at the center of the drone 20 has been described. Actually, the actual distance from the image receiving surface to the wall surface 80 may be calculated based on the positional relationship between the center of the drone 20 and the image receiving surface of the camera 40.

As shown in FIG. 8, the camera 40 includes a photographing mechanism 42 and an adjusting unit 44. The photographing mechanism 42 includes an image receiving surface such as a CCD (Charged-coupled devices) or CMOS (Complementary metal-oxide-semiconductor), a lens, and a mechanism for adjusting the positions thereof.

The adjustment unit 44 adjusts the focus of the camera 40 based on the distance D output from the calculation unit 28 of the drone 20.

_{It has been explained that the yaw angle θ Y of} τ for a certain period of time is obtained by the above equation (2), but the value of this τ is set as follows in consideration of focusing all the images. Is desirable.

1 / V> τ + α + β + γ
That is, τ <1 / V- (α + β + γ)

V (f / s) is the frame rate of the camera 40, α is the calculation time of the regression line indicating the amount of deviation of the origin, β is the calculation time of the distance D using the _{yaw angle θ Y, and γ is the distance.} This is the focus adjustment time using D.

The control unit 24 of the drone 20 included in the drone photographing device 10 can be realized by, for example, the computer 50 shown in FIG. The computer 50 includes a CPU (Central Processing Unit) 51, a memory 52 as a temporary storage area, and a non-volatile storage unit 53. Further, the computer 50 includes an input / output device 54 such as an input unit and a display unit, and an R / W (Read / Write) unit 55 that controls reading and writing of data to the storage medium 59. Further, the computer 50 includes a communication I / F (Interface) 56 connected to a network such as the Internet. The CPU 51, the memory 52, the storage unit 53, the input / output device 54, the R / W unit 55, and the communication I / F 56 are connected to each other via the bus 57.

The storage unit 53 can be realized by an HDD (Hard Disk Drive), an SSD (Solid State Drive), a flash memory, or the like. The storage unit 53 as a storage medium stores a drone photographing program 60 for causing the computer 50 to function as a control unit 24 of the drone 20 included in the drone photographing device 10. The drone photographing program 60 has a discrimination process 65, a correction process 66, and a calculation process 68. In addition, the storage unit 53 has an information storage area 67 in which information constituting the gyro data DB 27 is stored.

The CPU 51 reads the drone photographing program 60 from the storage unit 53, expands it into the memory 52, and sequentially executes the processes included in the drone photographing program 60. By executing the discrimination process 65, the CPU 51 operates as the discrimination unit 25 shown in FIG. Further, the CPU 51 operates as the correction unit 26 shown in FIG. 8 by executing the correction process 66. Further, the CPU 51 operates as the calculation unit 28 shown in FIG. 8 by executing the calculation process 68. Further, the CPU 51 reads information from the information storage area 67 and expands the gyro data DB 27 into the memory 52. As a result, the computer 50 that executes the drone photographing program 60 functions as the control unit 24 of the drone 20 included in the drone photographing device 10. The CPU 51 that executes the program is hardware.

The function realized by the drone photographing program 60 can also be realized by, for example, a semiconductor integrated circuit, more specifically, an ASIC (Application Specific Integrated Circuit) or the like.

Next, the operation of the drone photographing apparatus 10 according to the present embodiment will be described. The drone 20 is steered and moved to the shooting start position, the wheels 32A and 32B are in contact with the wall surface 80, and the posture of the drone 20 is 0 ° in all of the _{roll angle θ R} , the pitch angle θ _P , and the yaw angle θ _Y. At that point, the camera 40 starts taking an image at a predetermined frame rate. Then, in a state where the drone 20 is controlled to move in a predetermined direction (for example, upward or downward) along the wall surface 80, the control unit 24 of the drone 20 executes the drone photographing process shown in FIG. Will be done. The drone photography process is an example of the drone photography method of the disclosed technology.

In step S12, the discrimination unit 25 acquires the sensor values output from the pressure sensors 34A and 34B.

Next, in step S14, the discriminating unit 25 compares the sensor values acquired from the pressure sensors 34A and 34B with the predetermined threshold values, and whether or not each of the wheels 32A and 32B is in contact with the wall surface 80. To determine. If both wheels are in contact, the process proceeds to step S16, if only one wheel is in contact, the process proceeds to step S22, and if both wheels are not in contact, the process proceeds to step S30. Move to.

In step S16, the correction unit 26 acquires the gyro data output from the gyro sensor unit 22, and stores the yaw angular velocity ω included in the acquired gyro data in the gyro data DB 27.

Next, in step S18, the correction unit 26 corrects the yaw angular velocity ω acquired from the gyro sensor unit 22 to a reference value (here, 0) and hands it over to the calculation unit 28.

_{Next, in step S20, the calculation unit 28 relates to the yaw angle θ Y} by using the radii R of the wheels 32A and 32B based on the yaw angular velocity passed from the correction unit 26 being 0, which is the reference value. the distance _{D Y} from the wall _80, and calculates a _D Y = R. Further, calculation section 28 calculates the pitch angle theta _P from the pitch angular velocity contained in the gyro data acquired from the gyro sensor unit 22, by using the pitch angle theta _P and the calculated R and L, with respect to the pitch angle theta _P The distance D _P to the wall surface 80 is calculated as _{D P} = R / cos θ _P. The calculation unit 28 synthesizes the _{D Y} and _{D P,} and calculates the distance D from the wall 80.

In step S22, the calculation unit 28 determines the yaw angular velocity ω from the history of the yaw angular velocity ω at the time of contact between the two wheels, that is, the time change of the yaw angular velocity ω before the reset detected at each time, which is stored in the gyro data DB 27. Find a regression line that indicates the displacement of the origin. Then, the calculation unit 28 predicts the value of the regression line at each time after the start of one wheel in the coordinate system whose origin is the point at the start of one wheel on the obtained regression line as the amount of deviation of the origin. _{The yaw angle θ Y} is calculated from the yaw angle velocity corrected by the amount of deviation of the origin.

If the one-wheel state continues, the above-mentioned regression line is obtained at the start of one wheel, and the regression line is executed each time the process of this step is executed at each time in the subsequent one-wheel. It is not necessary to calculate the _{yaw angle θ Y} by using the regression line obtained at the start of one wheel.

Next, in step S24, using the radii R of the wheels 32A and 32B, the length L of the axle 32C, and the yaw angle θ _Y calculated in step S26, the distance D to the wall surface 80 with respect to the yaw angle θ _Y. the _Y, is calculated as D _Y = R + Ltanθ Y. Further, calculation unit 28, similarly to the step S20, the distance _{D P} to the wall surface 80 with respect to the pitch angle theta _P, calculated as _{D P = R / cosθ P -Htanθ} P, combines the _{D Y} and _{D P} Then, the distance D to the wall surface 80 is calculated.

Next, in step S26, the calculation unit 28 outputs the distance D to the wall surface 80 calculated in step S20 or S24. As a result, the adjustment unit 44 of the camera 40 adjusts the focus of the camera 40 based on the distance D output from the calculation unit 28, and the imaging mechanism 42 captures an in-focus image.

Next, in step S28, the determination unit 25 determines whether or not the photographing is completed. This determination can be made based on whether or not the drone 20 has reached a predetermined shooting end position, whether or not a shooting end instruction has been received from the controller, and the like. If the shooting is not completed, the process returns to step S12, and if the shooting is completed, the drone shooting process ends.

Further, in step S30, the discriminating unit 25 outputs an instruction to return the position of the drone 20 to the non-contact position, and the process returns to step S12.

As described above, according to the drone imaging device according to the present embodiment, the drone imaging device determines whether or not the wheels are in contact with the wall surface based on the sensor value of the pressure sensor installed on the axle. In the case of two-wheel contact, the yaw angle velocity of the gyro data is corrected to the reference value. Then, the distance on the optical axis from the image receiving surface of the camera to the wall surface is calculated using the value based on the yaw angular velocity corrected to the reference value, and the focus of the camera is adjusted based on the calculated distance. As a result, even if the contact with the wall surface becomes one wheel due to a disturbance such as a gust, the accurate distance between the camera and the wall surface can be calculated immediately, and the focus can be adjusted quickly and precisely. Therefore, it is possible to shoot an image with good image quality in focus, and it is possible to suppress re-shooting due to the occurrence of a blurred image, and it is possible to shoot efficiently in one flight.

Further, since the distance between the camera and the wall surface is calculated using the gyro data acquired from the gyro sensor built in the drone, it is not necessary to equip the drone with a special ranging system.

In addition, when the drone maintains the reference posture, that is, when both wheels are in contact, the yaw angular velocity is corrected to the reference value, and the error of the gyro data becomes unstable when approaching the structure. Can be suppressed.

Further, the yaw angular velocity is corrected by predicting the deviation amount of the reference value based on the yaw angular velocity acquired when the drone maintains the reference posture, that is, when both wheels are in contact with each other. As a result, it is possible to suppress an error in the gyro data due to the amount of deviation of the reference value that fluctuates due to changes in temperature and atmospheric pressure.

In the above embodiment, the case where the drone photographing apparatus includes two wheels has been described, but the present invention is not limited to this, and a configuration including three or more wheels may be used. 16 and 17 schematically show a configuration including four wheels 232A, 232B, 232D, and 232E. In this case, when all four wheels are in contact with the wall surface 80, the same processing as in the case of contacting both wheels in the above embodiment may be performed. Further, when at least one wheel is in contact and the other at least one wheel is not in contact, the same processing as in the case of one-wheel contact in the above embodiment may be performed. In the examples of FIGS. 16 and 17, the drone 20 includes a propeller for flying the drone 20 so as to apply a load to the wall surface 80 side, and a propeller for flying the drone 20 in the vertical direction. Further, instead of the wheels, for example, a plate-shaped guide along the wall surface 80 may be used.

Further, in the above embodiment, the case where the regression line is obtained based on the history of the yaw angular velocity before the reset at the time of contact between the two wheels and the deviation amount of the reference value is corrected has been described, but the present invention is not limited to this example. For example, the value of the yaw angular velocity of the gyro data at the start of one wheel may be used as the amount of deviation of each time in the subsequent one wheel. In this case, the deviation amount of the reference value can be easily predicted by omitting the process of obtaining the regression line.

Further, in the above embodiment, the mode in which the drone photographing program is stored (installed) in the storage unit in advance has been described, but the present invention is not limited to this. The program according to the disclosed technology can also be provided in a form stored in a storage medium such as a CD-ROM, a DVD-ROM, or a USB memory.

The following additional notes will be further disclosed with respect to the above embodiments.

(Appendix 1)
A shooting unit that is fixed to the drone and shoots images,
A support part that supports the drone's posture with respect to the shooting surface so as to maintain a predetermined reference posture, and
A discriminating unit that determines whether or not the posture of the drone has changed from the reference posture, and
When the discriminating unit determines that the drone is in the reference posture, a correction unit that corrects the angular velocity acquired by the gyro sensor built in the drone to a reference value.
A calculation unit that calculates the distance from the imaging unit to the imaging surface using a value based on the angular velocity corrected to the reference value by the correction unit.
An adjustment unit that adjusts the focus of the photographing unit based on the distance calculated by the calculation unit, and
Drone photography equipment including.

(Appendix 2)
The calculation unit uses the reference value when the drone is in the reference posture as a value based on the angular velocity corrected to the reference value, and when the posture of the drone is changed from the reference posture, the reference is used. The drone photographing apparatus according to Appendix 1, which uses a value corrected based on the value.

(Appendix 3)
The correction unit further holds a history of angular velocities acquired by the gyro sensor while the drone is determined to be in the reference posture.
As a value corrected based on the reference value, the calculation unit corrects the deviation amount of the reference value predicted based on the history of the angular velocity with respect to the angular velocity acquired by the gyro sensor. The drone photographing apparatus according to Appendix 2 to be used.

(Appendix 4)
The calculation unit obtains a regression line from the time change of the angular velocity indicated by the history of the angular velocity, and the drone is said to be in a coordinate system whose origin is a point on the regression line at the time when the drone changes from the reference posture. The drone photographing apparatus according to Appendix 3, which predicts the value of the regression line at each time after changing from the reference posture as the deviation amount of the reference value.

(Appendix 5)
The calculation unit integrates the difference between the angular velocity acquired by the gyro sensor and the regression line at regular time intervals, calculates the angle of the drone with respect to the reference posture at regular time intervals, and calculates the angle. The drone photographing apparatus according to Appendix 4, wherein the distance is calculated using the above.

(Appendix 6)
The support portion includes a contact portion that comes into contact with the imaging surface, and when the contact portion is in contact with the imaging surface under predetermined conditions, the drone maintains the reference posture.
The discriminating unit determines whether or not the posture of the drone has changed from the reference posture based on whether or not the contacting portion is in contact with the photographing surface under the predetermined conditions. The drone photographing apparatus according to any one of the above items.

(Appendix 7)
The support portion includes a plurality of contact portions, and when a part of each of the plurality of contact portions is in contact with the photographing surface, the drone maintains the reference posture.
The discriminant unit is in the posture of the drone when at least one contact portion of the plurality of contact portions is in contact with the photographing surface and the other at least one contact portion is not in contact with the photographing surface. The drone photographing apparatus according to Appendix 6 for determining that is changed from the reference posture.

(Appendix 8)
The support includes a plurality of wheels for allowing the drone to move along the imaging surface while maintaining the reference posture.
Whether or not each of the plurality of wheels and the photographing surface are in contact with each other based on the sensor value of the pressure sensor that detects the pressure between each of the plurality of wheels and the photographing surface. The drone photographing apparatus according to Appendix 7.

(Appendix 9)
The drone photographing apparatus according to any one of Supplementary note 1 to Supplementary note 8, wherein the angular velocity is a yaw angular velocity around a vertical axis.

(Appendix 10)
The shooting unit fixed to the drone shoots the image,
The support unit supports the drone so that the posture of the drone with respect to the shooting surface maintains a predetermined reference posture.
The discriminating unit determines whether or not the posture of the drone has changed from the reference posture.
When the correction unit determines that the drone is in the reference posture by the discrimination unit, the correction unit corrects the angular velocity acquired by the gyro sensor built in the drone to the reference value.
The calculation unit calculates the distance from the imaging unit to the imaging surface using a value based on the angular velocity corrected to the reference value by the correction unit.
A drone photographing method in which the adjusting unit adjusts the focus of the photographing unit based on the distance calculated by the calculating unit.

(Appendix 11)
The calculation unit uses the reference value when the drone is in the reference posture as a value based on the angular velocity corrected to the reference value, and when the posture of the drone is changed from the reference posture, the reference is used. The drone photographing method according to Appendix 10, which uses a value corrected based on the value.

(Appendix 12)
The correction unit further holds a history of angular velocities acquired by the gyro sensor while the drone is determined to be in the reference posture.
As a value corrected based on the reference value, the calculation unit corrects the deviation amount of the reference value predicted based on the history of the angular velocity with respect to the angular velocity acquired by the gyro sensor. The drone photographing method according to Appendix 11 to be used.

(Appendix 13)
The calculation unit obtains a regression line from the time change of the angular velocity indicated by the history of the angular velocity, and the drone is said to be in a coordinate system whose origin is a point on the regression line at the time when the drone changes from the reference posture. The drone photographing method according to Appendix 12, wherein the value of the regression line at each time after the change from the reference posture is predicted as the deviation amount of the reference value.

(Appendix 14)
The calculation unit integrates the difference between the angular velocity acquired by the gyro sensor and the regression line at regular time intervals, calculates the angle of the drone with respect to the reference posture at regular time intervals, and calculates the angle. The drone photographing method according to Appendix 13, wherein the distance is calculated using the above.

(Appendix 15)
The support portion includes a contact portion that comes into contact with the imaging surface, and when the contact portion is in contact with the imaging surface under predetermined conditions, the drone maintains the reference posture.
The discriminating unit determines whether or not the posture of the drone has changed from the reference posture based on whether or not the contacting portion is in contact with the photographing surface under the predetermined conditions. The drone shooting method according to any one of the above.

(Appendix 16)
The support portion includes a plurality of contact portions, and when a part of each of the plurality of contact portions is in contact with the photographing surface, the drone maintains the reference posture.
The discriminant unit is in the posture of the drone when at least one contact portion of the plurality of contact portions is in contact with the photographing surface and the other at least one contact portion is not in contact with the photographing surface. The drone photographing method according to Appendix 15, which determines that the subject has changed from the reference posture.

(Appendix 17)
The support includes a plurality of wheels for allowing the drone to move along the imaging surface while maintaining the reference posture.
Whether or not each of the plurality of wheels and the photographing surface are in contact with each other based on the sensor value of the pressure sensor that detects the pressure between each of the plurality of wheels and the photographing surface. The drone photographing method according to Appendix 16.

(Appendix 18)
The drone photographing method according to any one of Supplementary note 10 to Supplementary note 17, wherein the angular velocity is a yaw angular velocity around a vertical axis.

(Appendix 19)
It is determined whether or not the posture of the drone supported by the support portion has changed from the reference posture so that the shooting portion for capturing the image is fixed and the posture with respect to the shooting surface maintains the predetermined reference posture.
When it is determined that the drone is in the reference posture, the angular velocity acquired by the gyro sensor built in the drone is corrected to the reference value.
Using a value based on the angular velocity corrected to the reference value, the distance from the imaging unit to the imaging surface is calculated.
A drone photographing program for causing a computer to execute a process including outputting the calculated distance to an adjusting unit that adjusts the focus of the photographing unit based on the distance.

(Appendix 20)
It is determined whether or not the posture of the drone supported by the support portion has changed from the reference posture so that the shooting portion for capturing the image is fixed and the posture with respect to the shooting surface maintains the predetermined reference posture.
When it is determined that the drone is in the reference posture, the angular velocity acquired by the gyro sensor built in the drone is corrected to the reference value.
Using a value based on the angular velocity corrected to the reference value, the distance from the imaging unit to the imaging surface is calculated.
A storage medium that stores a drone photographing program for causing a computer to execute a process including outputting the calculated distance to an adjusting unit that adjusts the focus of the photographing unit based on the distance.

10 Drone shooting device 20 Drone 22 Gyro sensor unit 24 Control unit 25 Discrimination unit 26 Correction unit 27 Gyro data DB
28 Calculation unit 32 Support unit 32A, 32B Wheel 32C Axle 34A, 34B Pressure sensor 40 Camera 42 Imaging mechanism 44 Adjustment unit 50 Computer 51 CPU
52 Memory 53 Storage unit 59 Storage medium 60 Drone shooting program 80 Wall surface

Claims (10)

A shooting unit that is fixed to the drone and shoots images,
A support part that supports the drone's posture with respect to the shooting surface so as to maintain a predetermined reference posture, and
A discriminating unit that determines whether or not the posture of the drone has changed from the reference posture, and
When the discriminating unit determines that the drone is in the reference posture, a correction unit that corrects the angular velocity acquired by the gyro sensor built in the drone to a reference value.
A calculation unit that calculates the distance from the imaging unit to the imaging surface using a value based on the angular velocity corrected to the reference value by the correction unit.
An adjustment unit that adjusts the focus of the photographing unit based on the distance calculated by the calculation unit, and
Drone photography equipment including. The calculation unit uses the reference value when the drone is in the reference posture as a value based on the angular velocity corrected to the reference value, and when the posture of the drone is changed from the reference posture, the reference is used. The drone photographing apparatus according to claim 1, wherein a value corrected based on the value is used. The correction unit further holds a history of angular velocities acquired by the gyro sensor while the drone is determined to be in the reference posture.
As a value corrected based on the reference value, the calculation unit corrects the deviation amount of the reference value predicted based on the history of the angular velocity with respect to the angular velocity acquired by the gyro sensor. The drone photographing apparatus according to claim 2 to be used. The calculation unit obtains a regression line from the time change of the angular velocity indicated by the history of the angular velocity, and the drone is said to be in a coordinate system whose origin is a point on the regression line at the time when the drone changes from the reference posture. The drone photographing apparatus according to claim 3, wherein the value of the regression line at each time after changing from the reference posture is predicted as the amount of deviation of the reference value. The calculation unit integrates the difference between the angular velocity acquired by the gyro sensor and the regression line at regular time intervals, calculates the angle of the drone with respect to the reference posture at regular time intervals, and calculates the angle. The drone photographing apparatus according to claim 4, wherein the distance is calculated using the above. The support portion includes a contact portion that comes into contact with the imaging surface, and when the contact portion is in contact with the imaging surface under predetermined conditions, the drone maintains the reference posture.
Claims 1 to claim that the discriminating unit determines whether or not the posture of the drone has changed from the reference posture based on whether or not the contacting portion is in contact with the photographing surface under the predetermined conditions. Item 5. The drone photographing apparatus according to any one of Item 5. The support portion includes a plurality of contact portions, and when a part of each of the plurality of contact portions is in contact with the photographing surface, the drone maintains the reference posture.
The discriminant unit is in the posture of the drone when at least one contact portion of the plurality of contact portions is in contact with the photographing surface and the other at least one contact portion is not in contact with the photographing surface. The drone photographing apparatus according to claim 6, wherein it is determined that is changed from the reference posture. The support includes a plurality of wheels for allowing the drone to move along the imaging surface while maintaining the reference posture.
The discriminating unit determines whether or not each of the plurality of wheels and the photographing surface are in contact with each other based on the sensor value of the pressure sensor that detects the pressure between each of the plurality of wheels and the photographing surface. The drone photographing apparatus according to claim 7. The drone photographing apparatus according to any one of claims 1 to 8, wherein the angular velocity is a yaw angular velocity around a vertical axis. The shooting unit fixed to the drone shoots the image,
The support unit supports the drone so that the posture of the drone with respect to the shooting surface maintains a predetermined reference posture.
The discriminating unit determines whether or not the posture of the drone has changed from the reference posture.
When the correction unit determines that the drone is in the reference posture by the discrimination unit, the correction unit corrects the angular velocity acquired by the gyro sensor built in the drone to the reference value.
The calculation unit calculates the distance from the imaging unit to the imaging surface using a value based on the angular velocity corrected to the reference value by the correction unit.
A drone photographing method in which the adjusting unit adjusts the focus of the photographing unit based on the distance calculated by the calculating unit.