JP2022019737A - Patrol inspection system and patrol inspection method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a patrol inspection system which can detect an abnormal spot generated in a power transmission facility by a simple method with image processing.

SOLUTION: A patrol inspection system for patrol-inspecting an inspection object based on image data captured by a camera mounted on an aircraft comprises: a recording unit which records a plurality of pieces of first image data continuously captured for the inspection object; a specification unit which specifies the inspection object from the plurality of pieces of first image data; a determination unit which determines the quality of the imaging state of the inspection object of each of the plurality of pieces of first image data; a setting unit which records the image data whose imaging state is defective as defective image data in the recording unit on the basis of a determination result of the determination unit among the plurality of pieces of first image data; and an abnormality determination unit which determines abnormality of the inspection object on the basis of the image data recorded in the recording unit.

SELECTED DRAWING: Figure 8

COPYRIGHT: (C)2022,JPO&INPIT

Description

The present disclosure relates to a patrol inspection system and a patrol inspection method using an air vehicle or the like.

Conventionally, for periodic inspections of power transmission equipment such as overhead power transmission lines (hereinafter referred to as power transmission lines) and patrols in the event of an accident, a patrolman rides on a helicopter and flies along the power transmission line to see the power transmission line with binoculars. However, I was checking for abnormal parts and checking the power transmission equipment while moving on foot.

However, the inspection / patrol by the patrolman with the binoculars is a burden on the patrolman, and there is a risk of overlooking the abnormal part.

In addition to the cost of using a helicopter, inspection / patrol of power transmission equipment using a helicopter will increase the cost of inspection / patrol because multiple patrol officers will be on board in addition to the helicopter operator. In addition, if an inspection / patrol is suddenly required, it cannot be handled simply and quickly. In addition, an image of a transmission line is taken from a helicopter, and a patrolman looks at the taken image to check for a broken wire of the transmission line and the presence or absence of an arc mark (lightning mark).

In addition, the inspection work of the transmission line by taking an image also has a problem that a quick inspection cannot be performed because the patrolman detects an abnormal part while looking at all the taken images.

In this respect, a method of detecting wire breakage and arc marks by using an image processing technique has been proposed (Patent Document 1).

Japanese Unexamined Patent Publication No. 2018-74757

The present disclosure is to provide a patrol inspection system and a patrol inspection method capable of detecting an abnormal part generated in a power transmission facility by a simple method by image processing.

It is a patrol inspection system for patrol inspection of an inspected object based on image data taken by a camera mounted on the aircraft of the present disclosure, and is a plurality of first images continuously taken of the inspected object. A recording unit that records image data, a specific unit that identifies an inspected object from a plurality of first image data, and a shooting state of the inspected object of the first image data of each of the plurality of first image data. A judgment unit for determining quality, a setting unit for recording image data in a poor shooting state as defective image data based on the determination result of the determination unit among a plurality of first image data, and a recording unit for recording. It is provided with an abnormality determination unit that determines an abnormality of the object to be inspected based on the image data obtained.

Preferably, the determination unit calculates the width of the inspected object of the first image data of each of the plurality of first image data, and the width of the inspected object among the plurality of first image data is equal to or less than a predetermined value. The first image data determined to be the above is determined to be in a bad state of photography of the object to be inspected.

Preferably, the determination unit determines whether or not there is an outer peripheral region of the inspected object of the first image data of each of the plurality of first image data, and the inspected object among the plurality of first image data. The first image data in which it is determined that there is no outer peripheral region of the inspected object is determined to be in a poor state of photography.

Preferably, the recording unit records a plurality of second image data continuously re-photographed with respect to the object to be inspected, the specific unit identifies the object to be inspected from the plurality of second image data, and the determination unit is a determination unit. , The quality of the shooting state of the inspected object of the second image data of each of the plurality of second image data is determined, and among the plurality of second image data, the defect recorded in the recording unit for the inspected object. A plurality of first images by changing the re-photographed image data for which the re-photographed image data at the position corresponding to the image data is determined to be in good condition to the re-photographed image data specified by the specific unit for the defective image data. It has an editorial department for editing as data.

This is a patrol inspection method for patrol inspection of the inspected object based on the image data taken by the camera mounted on the aircraft of the present disclosure, and continuously photographs the inspected object recorded in the recording unit. A plurality of steps of identifying the inspected object from the plurality of first image data and a step of determining the quality of the photographed state of the inspected object of the first image data of each of the plurality of first image data. Of the first image data of the above, the step of recording the image data whose shooting state is defective based on the determination result in the recording unit as defective image data, and the abnormality of the inspected object based on the image data recorded in the recording unit. A determination step is provided.

The patrol inspection system and the patrol inspection method of the present disclosure can detect an abnormal part generated in a power transmission facility by a simple method by image processing.

It is a figure explaining the patrol inspection according to Embodiment 1. FIG. It is a figure explaining the structure of the patrol inspection system 1 according to Embodiment 1. FIG. FIG. 3 is a block diagram showing a functional configuration realized by executing a patrol inspection control program by a processor 2 in a patrol inspection system 1 according to the first embodiment. It is a figure explaining the learning model according to Embodiment 1. FIG. It is a figure explaining the abnormality determination processing of the abnormality candidate image data in the abnormality determination unit 26 according to Embodiment 1. FIG. It is a figure explaining the flow of the abnormality determination processing in the patrol inspection system 1 according to Embodiment 1. FIG. It is a figure explaining the subroutine flow of the abnormality determination processing of the abnormality determination part 26 according to Embodiment 1. FIG. FIG. 3 is a block diagram showing a functional configuration realized by executing a patrol inspection control program by a processor 2 in a patrol inspection system 1 according to the second embodiment. It is a figure explaining the structure of the data verification unit 29 according to Embodiment 2. It is a figure explaining the quality determination process of the image data in the quality determination unit 32 according to Embodiment 2. It is a figure explaining the flow of the data verification processing in the patrol inspection system 1 according to Embodiment 2. FIG. It is a figure explaining the subroutine flow of the good / bad judgment process of the good / bad judgment part 32 according to Embodiment 2. FIG. It is a flow diagram explaining the editing process of the editing unit 36 according to the second embodiment. It is a figure explaining the editing process of the editing unit 36 according to the second embodiment.

The embodiments will be described in detail with reference to the drawings. The same or corresponding parts in the drawings are designated by the same reference numerals, and the description thereof will not be repeated.

(Embodiment 1)
FIG. 1 is a diagram illustrating a patrol inspection according to the first embodiment.

As shown in FIG. 1, the patrol inspection system 1 uses an aircraft, for example, a helicopter 100, for example, for a power transmission facility (transmission line, steel tower) 200 for transmitting power from a power plant or a substation during normal times or when an accident occurs. Then, it is a system for patrol inspection.

In this example, the case of using the helicopter 100 will be described, but it is also possible to use an unmanned aerial vehicle (UAV), not limited to a manned vehicle.

The patrol inspection system 1 is generated in a plurality of different objects to be inspected (for example, overhead ground wire, transmission line, etc.) included in the power transmission facility 200 based on the image data taken by the camera mounted on the helicopter 100. Detect an abnormal part.

The patrol inspection system 1 according to the first embodiment is configured by using, for example, an information processing apparatus. It is also possible to realize it as a plurality of devices via a network. For example, it may be realized by cloud computing, and may be realized by one server connected via a network (Internet) or a plurality of servers operating in cooperation with each other.

FIG. 2 is a diagram illustrating a configuration of a patrol inspection system 1 according to the first embodiment.
With reference to FIG. 2, the patrol inspection system 1 includes a processor 2, a memory 4, a storage device 6, an input / output I / F 8, a display device 10, an input device 12, and a communication device 14.

Each part is connected to each other by an internal bus, and data can be exchanged with each other.

The processor 2 executes a basic program (OS) or an application program stored in the memory 4 to realize various functions. For example, the processor 2 detects an abnormal part generated in an inspected object of a power transmission facility based on an image taken by a camera mounted on the helicopter 100 by executing a patrol inspection control program, and the detection result thereof. The process of outputting the patrol inspection result is executed based on.

The memory 4 stores a program executed by the processor 2, temporary data, and the like.

The storage device 6 stores various programs and various data. The data stored in the storage device 6 includes acquired data (image data, position data) received from the helicopter 100, and learning model data used for image processing for detecting an abnormal portion generated in the inspected object from the image data. , Includes image data of abnormal parts, etc.

The input / output I / F8 is an interface for transmitting / receiving data to / from an external device. The input / output I / F8 can input / output data via, for example, a portable memory medium.

The display device 10 is an LCD (Liquid Crystal Display) or the like, and displays a screen corresponding to the processing of the processor 2.

The input device 12 is a keyboard, a pointing device, or the like, and is operated by an operator or the like.

The communication device 14 controls wireless communication or wired communication, and controls wireless communication or wired communication with the helicopter 100 or with a base station housed in the wireless public network. The communication device 14 controls communication with an external device, for example, another information processing device or the like through the network. The network includes a wireless or wired WAN (Wide Area Network), a LAN (Local Area Network), and the like.

The patrol inspection system 1 executes detection of an abnormal portion generated in the inspected object based on image data taken by a camera mounted on the helicopter 100, confirmation processing of an image of the abnormal portion, and the like.

The helicopter 100 is a patrol inspection target while storing position data and the like generated by using GNSS (Global Navigation Satellite System), for example, position data and the like generated from GPS signals received from GPS (Global Positioning System) satellites. The power transmission facility 200 is photographed and image data is stored.

The patrol inspection system 1 can also receive image data taken by the camera mounted on the helicopter 100 in real time using the communication device 14 to detect an abnormal portion generated in the inspected object. However, the image data taken by the camera may be once recorded on an information storage medium such as a disk, and then the data may be stored in the storage device 6 to detect an abnormal portion generated in the inspected object.

In this example, the patrol inspection system 1 describes a configuration provided outside the helicopter 100, but may be mounted inside the helicopter 100.

FIG. 3 is a block diagram showing a functional configuration realized by executing a patrol inspection control program by the processor 2 in the patrol inspection system 1 according to the first embodiment.

As shown in FIG. 3, the processor 2 is based on a patrol inspection control program, for example, a learning model storage unit 20, an abnormality location detection unit 22, an acquisition data storage unit 24, an abnormality determination unit 26, and confirmation data. The function with the storage unit 28 is realized.

The learning model storage unit 20 stores the learning model used in the patrol inspection system 1. The learning model is for detecting the electric wire region and the abnormal portion from the image data acquired by the camera mounted on the helicopter 100, and represents the characteristics of the abnormal state occurring in the inspected object. As an abnormality that has occurred in the object to be inspected, for example, in the case of a transmission line, there is an arc mark.

FIG. 4 is a diagram illustrating a learning model according to the first embodiment.
As shown in FIG. 4, the image data of the abnormal portion actually generated and the label image data to which the information about the image data is added are combined and stored as a trained model.

Specifically, label image data classified into a background image (label "0"), an electric wire image (label "1"), and an abnormal image (label "2") is stored. In this example, the case of storing the label image data classified into three will be described, but the label image data classified into the electric wire image (label "1") and the abnormal image (label "2"). You may try to remember.

The patrol inspection system 1 can reliably detect the abnormal part generated in the inspected object by searching for the abnormal part from the image data using the learning model stored in the learning model storage unit 20. ..

In addition to the image data of the abnormal part that actually occurred, deep learning (deep learning) technology that utilizes a neural network to generate a pseudo image (similar image) that shows the abnormal / deteriorated state that occurs in the object to be inspected. It may be used to generate a trained model. Using deep learning technology, you can create similar images by changing the combination of the image of the abnormal part (abnormal image) and the normal image around the abnormal part, and the ratio of combining the normal image and the abnormal image. By generating it, it is also possible to train (generate) a learning model based on a large number of similar images.

The acquired data storage unit 24 stores the acquired data acquired from the helicopter 100. The acquired data includes, for example, moving image data (which may be still image data) of an object to be inspected, position data (latitude, longitude, altitude) indicating the position of the helicopter 100 at the time of image shooting, and the like.

The abnormality location detection unit 22 is subjected to image processing based on the learning model (characteristics of the abnormality portion of the inspected object) stored in the learning model storage unit 20 from the acquired data stored in the acquisition data storage unit 24. Detect areas and abnormal areas. Specifically, the abnormality location detection unit 22 uses the learning model for the acquired image data to generate the label image data described with reference to FIG. 4 corresponding to the image data. For example, among the acquired image data, the pixel determined as the background is set to the label "0". Further, among the acquired image data, the pixel determined as the electric wire is set to the label "1". Of the acquired image data, the pixel determined to be abnormal is set to the label "2". When the label "2" is included in the label image data, the acquired image data is determined to be an abnormality candidate image. The abnormality location detection unit 22 outputs the generated label image data as abnormality candidate image data to the abnormality determination unit 26 together with the image data stored in the acquisition data storage unit 24.

The abnormality determination unit 26 determines whether or not there is an abnormality in the abnormality candidate image (also referred to as an abnormality candidate image) detected by the abnormality location detection unit 22.

The confirmation data storage unit 28 stores each data indicating the abnormality image data, the detection position, the detection date and time, etc., for which the abnormality candidate image is confirmed to be abnormal by the abnormality determination unit 26, as confirmation data. This makes it possible to easily confirm the abnormal part.

FIG. 5 is a diagram illustrating an abnormality determination process of abnormality candidate image data in the abnormality determination unit 26 according to the first embodiment.

In this example, the abnormality determination unit 26 determines whether or not there is an abnormality based on the label image data included in the abnormality candidate image data.

As shown in FIG. 5A, regarding the abnormality candidate image data (label image data), when the number of candidate pixels (label “2”) at the abnormality location is equal to or more than the first predetermined number, the abnormality is concerned. The candidate image data is determined to be abnormal. If the image has a large number of pixels that are judged to be abnormal due to scratches or deterioration over time, it is judged that there is an abnormality.

As shown in FIG. 5B, even when the number of candidate pixels (label “2”) at the abnormal portion is not equal to or more than the first predetermined number with respect to the abnormality candidate image data (label image data). When the candidate pixels (label "2") of the abnormal portion of the set group are equal to or larger than the second predetermined number, the abnormal candidate image data is determined to be abnormal. In the case of an image having pixels that are locally determined to be abnormal in the set group, such as arc marks, it is determined that there is an abnormality.

As shown in FIG. 5C, with respect to the abnormality candidate image data (label image data), the number of candidate pixels (label “2”) at the abnormality location is not equal to or more than the first predetermined number, and the abnormality of the set group Even if the number of candidate pixels (label "2") at the location is not equal to or greater than the second predetermined number, if there are candidate pixels (label "2") at the abnormal location adjacent to the outer peripheral region of the electric wire, there is a candidate pixel (label "2") at the location. Judged as abnormal. In the case of an image having pixels judged to be abnormal in the outer peripheral region such as a broken wire, it is judged that there is an abnormality.

As shown in FIG. 5D, regarding the abnormality candidate image data (label image data), even if there are a plurality of candidate pixels of the abnormality portion, the number is less than the first predetermined number and the abnormality portion of the set group is abnormal. When the number of candidate pixels is less than the second predetermined number, it is determined that the abnormality candidate image data is not abnormal. Even if there are pixels that are determined to be abnormal, if the number is small, it is determined that there is no abnormality.

As shown in FIG. 5 (E), regarding the abnormality candidate image data (label image data), when the candidate image of the abnormality portion is outside the outer peripheral region of the electric wire, the abnormality candidate image data is not abnormal. judge. Since it is outside the outer peripheral region of the electric wire, there is a high possibility of noise, so even if there is a pixel that is determined to be abnormal, it is determined that there is no abnormality.

FIG. 6 is a diagram illustrating a flow of abnormality determination processing in the patrol inspection system 1 according to the first embodiment.

With reference to FIG. 6, the patrol inspection system 1 acquires image data (step S2).
Specifically, the image data captured by the camera mounted on the helicopter 100 is stored in the storage device 6. The acquired data storage unit 24 stores the acquired data acquired from the helicopter 100. The abnormality location detection unit 22 acquires the image data stored in the acquisition data storage unit 24.

Next, the patrol inspection system 1 detects the electric wire region and the abnormal portion (step S4). The abnormality location detection unit 22 detects an abnormality location based on the image data stored in the acquired data storage unit 24 and the learning model stored in the learning model storage unit 20. The abnormality location detection unit 22 generates label image data using the learning model, and when it is determined that there is an abnormality portion in the image data stored in the acquisition data storage unit 24, the abnormality determination unit 22 is used as abnormality candidate image data. Output to 26.

Next, the patrol inspection system 1 determines an abnormality (step S6). The abnormality determination unit 26 determines whether or not the abnormality candidate image is actually abnormal. The details of the abnormality determination process will be described later.

Then, the process ends (end).
FIG. 7 is a diagram illustrating a subroutine flow of abnormality determination processing of the abnormality determination unit 26 according to the first embodiment.

As shown in FIG. 7, the patrol inspection system 1 determines whether or not the number of abnormal pixels is equal to or greater than the first predetermined number (step S10). The abnormality determination unit 26 determines whether or not the number of abnormal pixels determined to be abnormal parts included in the acquired abnormality candidate image data is equal to or greater than the first predetermined number. In this example, the abnormality determination unit 26 counts the number of labels "2" included in the label image data, and determines whether or not the counted number is equal to or greater than the first predetermined number. The first predetermined number can be set to any number.

In step S10, when the patrol inspection system 1 determines that the number of abnormal pixels is equal to or greater than the first predetermined number (YES in step S10), it determines that there is an abnormality (step S18). When the abnormality determination unit 26 determines that the number of labels "2" included in the label image data is equal to or greater than the first predetermined number, the abnormality determination unit 26 causes an abnormality according to scratches and aging deterioration as described in FIG. 5 (A). Image data with a high possibility of being abnormal is judged to be abnormal.

Then, the process ends (return).
Next, when the patrol inspection system 1 determines that the number of abnormal pixels is not equal to or greater than the first predetermined number (NO in step S10), whether or not the number of abnormal pixels in the set group is equal to or greater than the second predetermined number. Is determined (step S12). The abnormality determination unit 26 has a second predetermined number or more of abnormal pixels in the set group when the number of abnormal pixels determined to be abnormal parts included in the acquired abnormality candidate image data is not equal to or more than the first predetermined number. Judge whether or not. In this example, the abnormality determination unit 26 counts the number of adjacent labels “2” included in the label image data, and determines whether or not the counted number is equal to or greater than the second predetermined number. .. The second predetermined number can be set to any number.

In step S12, when the patrol inspection system 1 determines that the number of abnormal pixels in the set group is equal to or greater than the second predetermined number (YES in step S12), it determines that there is an abnormality (step S18). When the abnormality determination unit 26 determines that the number of labels "2" adjacent to each other contained in the label image data is equal to or greater than the second predetermined number, the abnormality determination unit 26 has an arc mark as described with reference to FIG. 5 (B). Image data with a high possibility of such anomalies is determined to have anomalies.

Then, the process ends (return).
Next, when the patrol inspection system 1 determines that the number of abnormal pixels in the set group is not equal to or greater than the second predetermined number (NO in step S12), whether or not there are abnormal pixels adjacent to the outer peripheral region of the electric wire. Is determined (step S14). When the abnormality determination unit 26 does not have more than the first predetermined number of abnormal pixels determined to be abnormal parts included in the acquired abnormality candidate image data and the number of abnormal pixels in the set group is not equal to or more than the second predetermined number. , It is determined whether or not there is an abnormal pixel adjacent to the outer peripheral region of the electric wire. In this example, the abnormality determination unit 26 specifies the outer peripheral region of the electric wire based on the label "1" included in the label image data. The abnormality determination unit 26 determines whether or not there is an abnormality pixel of the label “2” adjacent to the outer peripheral region of the specified electric wire.

In step S14, when the patrol inspection system 1 determines that there is an abnormal pixel adjacent to the outer peripheral region of the electric wire (YES in step S14), it determines that there is an abnormality (step S18). When the abnormality determination unit 26 has an abnormality pixel of the label “2” adjacent to the outer peripheral region of the specified electric wire, there is a possibility of an abnormality such as a wire break as described with reference to FIG. 5 (C). High image data is judged to have an abnormality.

Then, the process ends (return).
On the other hand, in step S14, when the patrol inspection system 1 determines that there is no abnormal pixel adjacent to the outer peripheral region of the electric wire (NO in step S14), it determines that there is no abnormality (step S16). The abnormality determination unit 26 determines that there is no abnormality in the image data having a low possibility of abnormality such as noise as described with reference to FIG. 5 (D) or FIG. 5 (E).

Then, the process ends (return).
Although the case of executing the determination process of steps S10, S12, and S14 has been described, the present invention is not limited to this, and any one of them may be used, or two determination processes may be executed in any combination.

The patrol inspection system 1 according to the first embodiment can detect an abnormal portion generated in an object to be inspected (for example, an overhead ground wire, a transmission line, etc.) by a simple method by image processing according to the above. Further, the patrol inspection system 1 according to the first embodiment is not only for abnormalities due to scratches and aged deterioration of the object to be inspected (for example, overhead ground wire, transmission line, etc.), but also for abnormalities such as arc marks and broken wires. Highly accurate abnormality detection is possible.

(Embodiment 2)
In the first embodiment described above, a method for determining an abnormality in image data has been described. On the other hand, there is a possibility that the object to be photographed (for example, an overhead ground wire, a power transmission line, etc.) is not normally photographed with respect to the image data acquired by the camera mounted on the helicopter 100.

FIG. 8 is a block diagram showing a functional configuration realized by executing a patrol inspection control program by the processor 2 in the patrol inspection system 1 according to the second embodiment.

As shown in FIG. 8, the point that the data verification unit 29 is further provided is different from the configuration described with reference to FIG. Since the other configurations are the same, the detailed explanation will not be repeated.

The data verification unit 29 verifies whether or not the acquired image data stored in the acquired data storage unit 24 is normal image data. That is, it is verified whether or not the inspected object is normally photographed. In this example, if it is determined that the inspected object has not been photographed normally, the inspected object is photographed again. In this example, a case where an electric wire is photographed as an example of an object to be inspected will be described.

FIG. 9 is a diagram illustrating the configuration of the data verification unit 29 according to the second embodiment.
As shown in FIG. 9, the data verification unit 29 includes a specific unit 30, a pass / fail determination unit 32, a setting unit 34, and an editing unit 36.

The identification unit 30 identifies the electric wire to be inspected from the image data.
The quality determination unit 32 determines the quality of the imaged state of the electric wire, which is the object to be inspected in the image data.

The setting unit 34 records the image data in a good shooting state as good image data and the image data in a bad shooting state as bad image data in the storage device 6 based on the determination result of the quality determination unit 32.

The editorial unit 36 changes the defective image data recorded in the storage device 6 into good re-photographed image data.

FIG. 10 is a diagram illustrating a quality determination process of image data in the quality determination unit 32 according to the second embodiment.

The quality determination unit 32 determines whether or not the shooting state is defective based on the image data.
As shown in FIG. 10A, in the case of image data in which a part of the electric wire, which is the object to be inspected, is missing, the photographing state is defective, and it is determined that the image data is defective. Specifically, when there is a place where the wire width of the inspected object included in the image data is equal to or less than a predetermined value, it is determined as defective image data.

As shown in FIG. 10B, even if the image is an image in which a part of the electric wire to be inspected is not missing, if the size of the inspected object is small, the shooting condition is poor and the defective image. Judge as data. Specifically, when there is a place where the wire width of the inspected object included in the image data is equal to or less than a predetermined value, it is determined as defective image data.

As shown in FIG. 10C, in the case of image data in which the outer peripheral region is missing even when the wire width, which is the object to be inspected, included in the image data exceeds a predetermined value, the shooting state. Is defective and is determined to be defective image data. Specifically, it is determined whether or not the outer peripheral region of the electric wire of the object to be inspected included in the image data exists, and if it does not exist, it is determined as defective image data. For example, it may be determined whether or not the outer peripheral region exists depending on whether or not pixels other than the electric wire are present in the vertical direction of the pixels constituting the electric wire of the image data.

FIG. 11 is a diagram illustrating a flow of data verification processing in the patrol inspection system 1 according to the second embodiment.

With reference to FIG. 11, the patrol inspection system 1 acquires image data (step S20).

Specifically, the image data captured by the camera mounted on the helicopter 100 is stored in the storage device 6. The acquired data storage unit 24 stores the acquired data acquired from the helicopter 100. The specific unit 30 acquires the image data stored in the acquisition data storage unit 24.

Next, the patrol inspection system 1 specifies the electric wire region (step S22). The identification unit 30 identifies the electric wire region to be inspected from the image data stored in the acquired data storage unit 24.

Specifically, as described above, the electric wire region is specified by image processing based on the learning model stored in the learning model storage unit 20. Alternatively, the specifying unit 30 may specify the electric wire region by the shape, or may specify the electric wire region based on the color data included in the image data.

Next, the patrol inspection system 1 determines whether the shooting condition is good or bad (step S24). The quality determination unit 32 determines the quality of the photographing state based on the specified electric wire region of the image data. The details of the quality determination process of the shooting state will be described later.

FIG. 12 is a diagram illustrating a subroutine flow of the pass / fail determination process of the pass / fail determination unit 32 according to the second embodiment.

As shown in FIG. 12, the patrol inspection system 1 calculates the wire width (step S30). The pass / fail determination unit 32 calculates the wire width of the specified wire region. For example, the wire width may be calculated from the number of pixels (the number of pixels (label “1”)) in the specified wire region of the image data. The wire width is a distance in a direction orthogonal to the length direction of the wire region.

Next, the patrol inspection system 1 determines whether or not the wire width is equal to or less than a predetermined value (step S32). The quality determination unit 32 determines whether or not the wire width is equal to or less than a predetermined value. Specifically, the quality determination unit 32 may determine whether or not the number of pixels in the direction orthogonal to the length direction of the specified electric wire region of the image data is equal to or less than a predetermined number.

In step S32, when the patrol inspection system 1 determines that the wire width is equal to or less than a predetermined value (YES in step S32), the patrol inspection system 1 determines that the photographing state is defective (step S38). When the quality determination unit 32 determines that the wire width is equal to or less than a predetermined value, the imaging state is defective as described with reference to FIGS. 10A and 10B, and the quality determination unit 32 determines that the image data is defective.

Then, the process ends (return).
On the other hand, when the patrol inspection system 1 determines that the wire width is not equal to or less than a predetermined value (NO in step S32), it determines whether or not there is an outer peripheral region of the specified wire region of the image data (step). S34). The quality determination unit 32 determines whether or not there is an outer peripheral region of the electric wire region included in the acquired image data. For example, it may be determined whether or not the outer peripheral region exists depending on whether or not the pixels other than the electric wire region exist in the vertical direction of the pixels constituting the electric wire region included in the acquired image data. If at least one pixel is present, it may be determined that the outer peripheral region of the electric wire region exists, and if a plurality of pixels are provided with a margin, it is determined that the outer peripheral region of the electric wire region exists. You may do so.

In step S34, when the patrol inspection system 1 determines that there is no outer peripheral region of the specified electric wire region (NO in step S34), the patrol inspection system 1 determines that the photographing state is defective (step S38). When the quality determination unit 32 determines that there is no outer peripheral region of the specified electric wire region, as described with reference to FIG. 10C, the imaging state is defective, and the quality determination unit 32 determines that the image data is defective.

Then, the process ends (return).
On the other hand, in step S34, when the patrol inspection system 1 determines that there is an outer peripheral region of the specified electric wire region (YES in step S34), it determines that the photographing state is good (step S36).

Then, the process ends (return).
With reference to FIG. 11 again, the patrol inspection system 1 sets the determination result in the captured image data (step S26).

Specifically, the setting unit 34 sets the result of the quality determination of the quality determination unit 32 in the image data. For example, the setting unit 34 sets the determination results in association with each of the plurality of image data stored in the storage device 6 as good or bad image data.

Then, the process ends (end).
In this example, if it is determined that the inspected object has not been photographed normally, the inspected object is photographed again. In this case, the process described with reference to FIG. 11 is also executed for the re-photographed image data. The setting unit 34 sets the result of the quality determination of the quality determination unit 32 in the re-photographed image data. The setting unit 34 sets the determination results in association with each of the plurality of re-photographed image data stored in the storage device 6 as good or bad image data.

FIG. 13 is a flow chart illustrating an editing process of the editing unit 36 according to the second embodiment.
As shown in FIG. 13, the patrol inspection system 1 acquires defective image data stored in the storage device 6 (step S40). The editing unit 36 acquires defective image data set as defective stored in the acquired data storage unit 24.

Next, the patrol inspection system 1 identifies the corresponding re-photographed image data (step S42). Specifically, the editorial unit 36 identifies the re-photographed image data corresponding to the same position information based on the position information associated with the defective image data. The position information may be the position information regarding the specified electric wire area, or the position information of the helicopter 100 provided with the camera may be used.

Next, the patrol inspection system 1 edits the image data stored in the storage device 6 (step S44). The editorial unit 36 acquires the position data of the image data whose shooting state is set to be defective, identifies and replaces the re-shooting image data corresponding to the position data.

Next, the patrol inspection system 1 determines whether or not all the image data set for defects have been checked (step S46).

If it is determined in step S46 that the patrol inspection system 1 has not checked all the image data set for defects (NO in step S46), the process returns to step S40 and the image set for other defects is set. The data is acquired and the above process is repeated.

On the other hand, if it is determined in step S46 that the patrol inspection system 1 has checked all the image data set to be defective (YES in step S46), the process ends (end).

FIG. 14 is a diagram illustrating an editing process of the editing unit 36 according to the second embodiment.
As shown in FIG. 14, for example, a case where a part of the moving image data is set as defective image data because the shooting state is defective is shown.

The editorial unit 36 acquires the position data of the defective image data whose shooting state is set to be defective, and identifies the re-photographed image data corresponding to the position data. In this example, the case where the retaken image data in the position data PE corresponds to the defective image data is shown.

Then, the editing unit 36 replaces the re-photographed image data with the defective image data and edits the data.
This makes it possible to replace the defective image data with the re-photographed image data and acquire appropriate image data. It is assumed that the retaken image data is set as good image data. Regarding the replacement of the re-photographed image data, it is determined that not only the re-photographed image data corresponding to the same position as the position data PE determined to be defective image data but also the photographing state corresponding to the positional relationship before and after the re-photographed image data is good. It is also possible to replace a plurality of re-photographed image data.

The patrol inspection system 1 according to the second embodiment determines whether or not the image data obtained by photographing the inspected object is normal before the detection process of the abnormal portion generated in the inspected object.

Then, in the case of defective image data, it is possible to use the re-photographed image data acquired by re-imaging and replace it with the defective image data to create an appropriate set of image data.

For example, when photographing an object to be inspected over a long distance (for example, an overhead ground wire, a power transmission line, etc.), there is a possibility that some photographs may not be appropriate due to environmental factors. By this method, it is possible to replace the defective image data determined to be defective with good re-photographed image data, and it is possible to perform a highly accurate patrol inspection.

The method described in each of the above embodiments is a storage of a magnetic disk (hard disk, etc.), an optical disk (CDROM, DVD, etc.), a magneto-optical disk (MO), a semiconductor memory, etc., as a program that can be executed by a computer. It can also be stored in a medium and distributed. Further, the storage medium may be in any form as long as it is a storage medium that can store a program and can be read by a computer.

Further, in order to realize the above embodiment, an OS (operating system) running on the computer based on instructions of a program installed on the computer from a storage medium, MW (middleware) such as database management software and network software, and the like. You may execute a part of each process of.

Further, the storage medium in each embodiment is not limited to a medium independent of the computer, but also includes a storage medium in which a program transmitted by a LAN, the Internet, or the like is downloaded and stored or temporarily stored.

Further, the storage medium is not limited to one, and the case where the processing in each of the above embodiments is executed from a plurality of media is also included in the storage medium in the present invention, and the medium configuration may be any configuration.

The computer in each embodiment executes each process in each of the above embodiments based on the program stored in the storage medium, and is composed of one device such as a personal computer and a network of a plurality of devices. Any configuration such as a connected system may be used.

It should be considered that the embodiments disclosed this time are exemplary in all respects and not restrictive. The scope of the present disclosure is shown by the scope of claims, not the description described above, and is intended to include all modifications within the meaning and scope equivalent to the scope of claims.

1 patrol inspection system, 2 processors, 4 memories, 6 storage devices, 10 display devices, 12 input devices, 14 communication devices, 20 learning model storage units, 22 abnormality location detection units, 24 acquisition data storage units, 26 abnormality determination units, 28 Confirmation data storage unit, 29 Data verification unit, 30 Specific unit, 32 Good / bad judgment unit, 34 Setting unit, 36 Editorial department, 100 Helicopter, 200 Transmission equipment.

Claims (5)

It is a patrol inspection system for patrol inspection of the object to be inspected based on the image data taken by the camera mounted on the aircraft.
A recording unit that records a plurality of first image data continuously captured for the object to be inspected, and a recording unit.
A specific part that identifies the object to be inspected from the plurality of first image data, and
A determination unit for determining whether or not the imaged state of the inspected object of the first image data of each of the plurality of first image data is good or bad.
Among the plurality of first image data, a setting unit for recording image data in a poor shooting state as defective image data in the recording unit based on the determination result of the determination unit, and a setting unit.
A patrol inspection system including an abnormality determination unit for determining an abnormality of the inspected object based on image data recorded in the recording unit. The determination unit
The width of the inspected object of the first image data of each of the plurality of first image data is calculated.
The first image data in which the width of the inspected object is determined to be equal to or less than a predetermined value among the plurality of first image data is determined to be defective in the photographing state of the inspected object, according to claim 1. Patrol inspection system. The determination unit
It is determined whether or not there is an outer peripheral region of the inspected object of the first image data of each of the plurality of first image data.
The patrol inspection according to claim 2, wherein the first image data in which it is determined that there is no outer peripheral region of the inspected object among the plurality of first image data is determined to be in a defective photographing state of the inspected object. system. The recording unit records a plurality of second image data that are continuously re-photographed for the object to be inspected.
The specific unit identifies the object to be inspected from the plurality of second image data, and determines the object to be inspected.
The determination unit determines whether or not the imaged state of the inspected object of the second image data of each of the plurality of second image data is good or bad.
Among the plurality of second image data, a specific unit for specifying the re-photographed image data for which the imaging state at the position corresponding to the defective image data recorded in the recording unit for the object to be inspected is determined to be good. ,
The patrol inspection system according to claim 1, further comprising an editorial unit that changes the defective image data into rephotographed image data specified by the specific unit and edits the defective image data as the plurality of first image data. It is a patrol inspection method for patrol inspection of the object to be inspected based on the image data taken by the camera mounted on the aircraft.
A step of identifying the inspected object from a plurality of first image data continuously photographed about the inspected object recorded in the recording unit.
A step of determining whether or not the imaged state of the inspected object of the first image data of each of the plurality of first image data is good or bad.
A step of recording image data in a poor shooting state as defective image data in the recording unit based on a determination result among the plurality of first image data.
A patrol inspection method comprising a step of determining an abnormality of the object to be inspected based on image data recorded in the recording unit.

Images