JP2023072597A - Appearance checking device, method and program - Google Patents

Abstract

To provide an appearance checking technique capable of observing damage that may be overlooked in imaging depending on sunlight.SOLUTION: An appearance checking device comprises: a first transmission unit which transmits a first control signal for controlling a first drone 27a; a second transmission unit which transmits a second control signal for controlling a second drone 27b; and a reception unit which receives a photographed image captured by making reflection light 39 obtained by surface reflection of irradiation light 38 of a light source 28 incident on a camera 29.SELECTED DRAWING: Figure 2

Description

An embodiment of the present invention relates to a technique for inspecting the appearance of a structure by drone photography.

Wind power is one of the renewable energies that exists everywhere, does not emit carbon dioxide, and can be used permanently without depletion. In order to promote power generation using such wind power, it is important to establish technology for maintaining sound operation in order to improve the stability and sustainability of power generation. Blade breakage is the most common type of wind turbine accident reported, and there is a need to establish specific countermeasures for inspections and repairs.

By the way, in the case of a wind turbine installed on the sea, it is difficult for workers to access the wind turbine depending on the state of the sea. Therefore, it is being considered to fly a drone equipped with an optical camera near the blade surface, photograph the damage, and monitor the progress of the damage over time from multiple photographs.

Japanese Unexamined Patent Application Publication No. 2020-118141

However, in an image captured using natural sunlight, there is a possibility of overlooking small damages and scratches that do not accompany changes in color development. In addition, the irradiation angle, intensity, spectral distribution, etc. of sunlight depend on the inspection time period, season, and weather conditions, and the photographing conditions are not constant.

The embodiments of the present invention have been made in consideration of such circumstances, and it is an object of the present invention to provide an appearance inspection technique that allows observation of damage that may be overlooked in photographing that relies on sunlight.

In the appearance inspection device according to the embodiment, a first transmitter that transmits a first steering signal for steering a first drone equipped with a light source, and a second steering signal that transmits a second steering signal for steering a second drone equipped with a camera. and a receiving unit for receiving a photographed image captured by causing reflected light obtained by surface-reflecting the irradiation light from the light source to enter the camera.

Embodiments of the present invention provide visual inspection techniques that allow the observation of damage that may be overlooked in daylight-dependent photography.

1 is a front view of a drone employed in an appearance inspection apparatus according to an embodiment of the present invention and a windmill to be inspected; FIG. 1 is a top view of a drone employed in an appearance inspection apparatus according to an embodiment of the present invention and a windmill to be inspected; FIG. FIG. 2 is a block diagram of a data processing unit employed in the appearance inspection device according to the embodiment; FIG. 4 is an explanatory diagram of reflected light incident on the camera among the light emitted from the light source and reflected from the surface. (A), (B), and (C) are explanatory diagrams of images captured by a drone in the embodiment. 4 is a flowchart for explaining steps of an appearance inspection method and an algorithm of an appearance inspection program according to the embodiment;

(First embodiment)
An embodiment of the present invention will be described below with reference to the accompanying drawings. FIG. 1 is a front view of a drone 27 (27a, 27b) and a windmill 20 to be inspected, which are employed in an appearance inspection apparatus according to an embodiment of the present invention. FIG. 2 is a top view of the same. Although the wind turbine 20 shown in FIG. 1 is of a floating offshore installation system, the wind turbine to which the embodiment is applied may be of a fixed offshore installation system or a land installation system. As described above, the wind turbine 20 includes blades 25 , tower 26 , nacelle 30 , hub 21 and floating body 22 .

The blades 25 are connected to a hub 21 at the tip of a rotor shaft (not shown) and arranged radially. The pitch angle of these blades 25 is adjusted with respect to the inflow direction of the wind so that the flow energy of the wind can be efficiently converted into rotational energy. Drive mechanisms (not shown) such as a motor for adjusting the pitch angle, a brake, and an emergency power source are provided in the hub 21 . The tower 26 is connected to the floating body 22 which is connected to the foundation 24 built on the seabed via the mooring cable 23 so as to be exposed from the sea surface.

The drones 27 (27a, 27b) are remotely controlled unmanned aerial vehicles. The drone 27 may start from land and fly to the offshore windmill 20, or may wait at a port with power supply equipment installed on the sea (not shown) and take off and land there.

As shown in FIG. 2, the first drone 27a is equipped with a light source 28 that emits irradiation light 38. As shown in FIG. The second drone 27b is equipped with a camera 29 that captures an image of the surface of the blade 25 with reflected light 39 obtained by surface-reflecting the irradiation light 38 . In addition to the camera 29, a separate light source (not shown) may be mounted on the second drone 27b.

Also, the first drone 27a and the second drone 27b are each equipped with a light source of the illumination light 38 and the camera 29, and the irradiation of the illumination light 38 of the first drone 27a and the second drone 27b is alternately switched to It is also conceivable that the drones 27a and 27b take pictures with the cameras 29 and at the same time carry out an external inspection over a wider area.

The drone 27 (27a, 27b) autonomously flies around the exterior of the windmill 20, and the flight route 15 is defined in advance. Information on the actual position and attitude of the drone 27 required for autonomous flight is acquired by a known method. Specifically, a method of photographing the drone 27 with a fixed camera (not shown) (for example, attaching an AR marker to the side of the drone 27 and recognizing it with a fixed camera), a method of fixing the wind turbine 20 with a camera mounted on the drone 27, and A method of photographing a block (not shown), a method of transmitting/receiving positioning radio waves emitted from the wind turbine 20 to/from the drone 27, and a self-position estimation method such as VisualSLAM or LiDAR-SLAM can be considered.

FIG. 3 is a block diagram of the data processing section 10 employed in the appearance inspection device according to the embodiment. In this way, the data processing unit 10 includes the first transmission unit 11 that transmits the first steering signal 31 for steering the first drone 27a and the second transmission unit that transmits the second steering signal 32 for steering the second drone 27b. 12, and a receiving unit 36 for receiving a photographed image 35 captured by causing a reflected light 39 obtained by surface-reflecting the irradiation light 38 of the light source 28 to enter the camera 29 .

In this way, the data processing unit 10 that transmits the control signals 31 and 32 of the drones 27 (27a, 27b) and receives the captured image 35 of the camera 29 is installed on land or installed in the offshore windmill 20. or When the data processing unit 10 is installed in the offshore windmill 20, a communication means (not shown) is further provided for communicating the flight plan 15 and the captured image 35 with a database on land.

The first transmitter 11 transmits a first maneuver signal 31 according to the flight plan 15 to a controller (not shown) of the first drone 27a. Similarly, the second transmitter 12 also transmits a second maneuver signal 32 according to the flight plan 15 to the controller (not shown) of the second drone 27b. In this manner, the first drone 27a and the second drone 27b are cooperatively controlled with each other.

There are various examples of deployment of the flight plan 15, but the following are examples. While the second drone 27b flies statically, the first drone 27a moves and flies so that the optical axis of the camera 29 is irradiated with the irradiation light 38 from a plurality of different directions. According to the first development example of this flight plan 15, the type and size of damage (collision marks, cracks, deposits, etc.) are estimated by utilizing the fact that the shape of the shadow 33 changes depending on the irradiation direction. can.

A second development of the flight plan 15 causes the light source 28 to emit illuminating light 38 of different wavelengths. According to the second development example, the diffraction phenomenon that wraps around the surface steps differs depending on the wavelength size of the irradiation light 38, so the shape of the shadow 33 appearing in the captured image 35 changes. According to the first development example of this flight plan 15, the type and size of damage (collision marks, cracks, deposits, etc.) can be determined by utilizing the fact that the shape of the shadow 33 changes depending on the wavelength size. I can guess.

A third development example of the flight plan 15 acquires the rotation period of the blade 25 and causes the camera 29 to perform still photography based on this rotation period. That is, if the number of blades 25 is three, the shutter timing of the camera 29 is taken at intervals of one-third of the rotation cycle. Furthermore, the passage timing of the blade 25 and the shutter timing are synchronized.

Thereby, the photographed images 35 of all the blades 25 can be obtained while the first drone 27a and the second drone 27b are in stationary flight. Furthermore, by moving the first drone 27a and the second drone 27b linearly from the tip position of the blade 25 to the base end position connected to the hub 21, the photographed image 35 of the entire one-sided side of all the blades 25 can be obtained. Obtainable.

The data processing unit 10 includes an operation unit 16 for setting the photographing position and photographing attitude of the camera 29 and arbitrarily defining and correcting the flight plan 15 . As a result, even if a satisfactory shot image 35 cannot be obtained with the flight plan 15 defined in advance, the flight plan 15 with the corrected position and attitude of the camera 29 can be taken again.

In the above, an example of deployment of the flight plan 15 has been described by exemplifying the blade 25 of the wind turbine 20 as an object to be inspected. However, other structural surfaces of the wind turbine 20, such as the tower 26, the nacelle 30, the hub 21, etc., can also be inspected in the flight plan 15. FIG. Further, the object to be inspected is not limited to the surface of the wind turbine 20, and the flight plan 15 can be formulated with the surface of other structures as the object to be inspected.

FIG. 4 is an explanatory diagram of a phenomenon in which the irradiation light 38 of the light source 28 is surface-reflected and the reflected light 39 is incident on the camera 29. As shown in FIG. FIG. 5 is an explanatory diagram of an image 35 captured by the camera 29 mounted on the second drone 27b in the embodiment, and FIG. (B) and (C) show how the shadow 33 of the deposit 34 on the surface of the structure changes depending on the direction of the irradiation light 38 .

As shown in FIG. 4, the second drone 27b is stationary so that the optical axis of the camera 29 is perpendicular to the surface of the inspection target (blade 25). When irradiation light 38 having a constant incident angle with respect to the optical axis of the camera 29 is emitted from the light source 28, a shadow 33 is generated at a stepped portion such as a crack or an adhering substance. On the other hand, the irradiation light 38 reaching the surface of the object to be inspected (blade 25 ) is diffusely reflected by this surface and part of it enters the camera 29 as reflected light 39 .

Then, the reflected light 39 incident on the camera 29 is imaged by a lens (not shown) and becomes a photographed image 35 of the inspection target (blade 25) surface. In this photographed image 35, the presence of minute steps on the surface of the inspection target (blade 25) is distinguished from the surroundings by the contrast of the shadow 33 and identified. As a result, it is possible to increase the detection sensitivity of damage or the like occurring on the surface of the structure, and at the same time, the shape thereof can be accurately grasped.

Returning to FIG. 3, the description continues. The captured image 35 is stored in the storage unit 17 of the data processing unit 10 from the camera 29 via the reception unit 36 and is displayed and observed on the display unit 37 . Then, the diagnosis unit 18 discovers surface damage in the structure or analyzes its progress based on the captured images 35 received and accumulated in time series. This makes it possible to diagnose the soundness of the structure.

The steps of the appearance inspection method and the algorithm of the appearance inspection program according to the embodiment will be described based on the flowchart of FIG. 6 (see FIG. 3 as appropriate). First, the first control signal 31 is transmitted from the first drone 27a on which the light source 28 is mounted (S11). Then, the second control signal 32 of the second drone 27b on which the camera 29 is mounted is transmitted (S12).

The first control signal 31 and the second control signal 32 are transmitted so that the drone 27 (27a, 27b) autonomously flies according to the flight plan 15 defined in advance. Alternatively, the first steering signal 31 and the second steering signal 32 are transmitted based on manual operation by the operator. Next, irradiation light 38 is emitted from the light source 28 (S13), and reflected light 39 reflected on the surface of the structure is made incident on the camera 29 to take an image (S14).

In the process of receiving the captured image 35, while the second drone 27b mounted with the camera 29 flies stationary, the first drone 27a moves and flies so that the irradiation direction of the light source 28 changes. Alternatively, the light source 28 emits irradiation light 38 having different wavelengths. Furthermore, when the inspection target is the blades 25 of the wind turbine 20, the camera 29 is caused to take still pictures based on the rotation period thereof.

Next, the captured image 35 output by the camera 29 is received (S15), and the data is accumulated when the flight plan 15 is completed (S16 No, Yes, S17). Then, the photographed images 35 that have been received and accumulated in the past are called (S18), and the detection of surface damage or its progress is analyzed in time series (S19), thereby diagnosing the soundness of the structure (S20, END). .

According to the appearance inspection device of at least one embodiment described above, by introducing a drone equipped with a light source separately from a drone equipped with a camera, there is a possibility of overlooking in shooting that relies on sunlight. It becomes possible to observe the damage.

While several embodiments of the invention have been described, these embodiments have been presented by way of example and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, replacements, changes, and combinations can be made without departing from the scope of the invention. These embodiments and their modifications are included in the scope and spirit of the invention, as well as the scope of the invention described in the claims and equivalents thereof.

The wind turbine visual inspection device described above includes a control device with a highly integrated processor such as a dedicated chip, FPGA (Field Programmable Gate Array), GPU (Graphics Processing Unit), or CPU (Central Processing Unit), and ROM Storage devices such as (Read Only Memory) and RAM (Random Access Memory), external storage devices such as HDD (Hard Disk Drive) and SSD (Solid State Drive), display devices such as displays, and mouse and keyboard It has an input device and a communication I/F, and can be realized with a hardware configuration using a normal computer. Therefore, the constituent elements of the wind turbine visual inspection apparatus can be realized by a computer processor, and can be operated by a wind turbine visual inspection program.

Further, the external appearance inspection program for the wind turbine is preinstalled in a ROM or the like and provided. Alternatively, this program is stored in a computer-readable storage medium such as a CD-ROM, CD-R, memory card, DVD, flexible disk (FD) as an installable or executable file. You may make it

Further, the wind turbine visual inspection program according to the present embodiment may be stored on a computer connected to a network such as the Internet, and may be downloaded via the network and provided. The wind turbine visual inspection device can also be configured by connecting and combining separate modules that independently exhibit the respective functions of the constituent elements through a network or a dedicated line.

DESCRIPTION OF SYMBOLS 10... Data processing part 11... 1st transmission part 12... 2nd transmission part 15... Flight plan 16... Operation part 17... Storage part 18... Diagnosis part 20... Windmill 21... Hub 22... Floating body, 23... mooring cable, 24... foundation, 25... blade, 26... tower, 27 (27a, 27b)... drone, 27a (27)... first drone, 27b (27)... second drone, 28... light source, 29... Camera, 30... Nacelle, 31... First control signal, 32... Second control signal, 33... Shadow, 34... Attached matter, 35... Captured image, 36... Receiving unit, 37... Display unit, 38... Irradiation light , 39... Reflected light.

Claims (9)

a first transmitter that transmits a first steering signal for steering a first drone equipped with a light source;
a second transmitter that transmits a second control signal for controlling a second drone equipped with a camera;
and a receiving unit configured to receive a photographed image captured by causing reflected light obtained by surface-reflecting the irradiation light of the light source to enter the camera. In the appearance inspection device according to claim 1,
The appearance inspection device, wherein the first control signal and the second control signal are transmitted so that the first drone and the second drone autonomously fly according to a flight plan defined in advance. In the appearance inspection device according to claim 1 or claim 2,
the second maneuver signal is transmitted to cause the second drone to fly stationary;
The appearance inspection device, wherein the first control signal for moving and flying the first drone is transmitted so that the irradiation light hits the optical axis of the camera from a plurality of different directions. In the appearance inspection device according to any one of claims 1 to 3,
An appearance inspection apparatus comprising a diagnostic unit that diagnoses soundness by detecting surface damage or analyzing its progress based on the captured images received in time series. In the appearance inspection device according to any one of claims 1 to 4,
The light source is an appearance inspection device that emits the irradiation light having different wavelengths. In the appearance inspection device according to any one of claims 1 to 5,
An external inspection device in which a separate light source is also mounted on the second drone. In the appearance inspection device according to any one of claims 1 to 6,
An appearance inspection device that allows the camera to take still images of the blade based on the rotation period of the blade that converts wind flow energy into rotational energy. transmitting a first steering signal to steer a first drone equipped with a light source;
transmitting a second steering signal to steer a second drone equipped with a camera;
and receiving a photographed image captured by causing reflected light obtained by surface-reflecting the irradiation light of the light source to enter the camera. to the computer,
transmitting a first steering signal to steer a first drone equipped with a light source;
transmitting a second steering signal to steer a second drone equipped with a camera;
An appearance inspection program for executing a step of receiving a photographed image captured by causing reflected light obtained by surface-reflecting the irradiation light of the light source to enter the camera.

Images