JP2021057957A - Patrol inspection system and patrol inspection method - Google Patents

Abstract

To provide a patrol inspection system that can detect abnormalities in power transmission facilities in a simple way by image processing.SOLUTION: A patrol inspection system for performing a patrol inspection of an object to be inspected based on image data captured by a camera mounted on an aircraft according to a certain phase includes an abnormality location detection unit that identifies an abnormality location in which an abnormality may have occurred in the object to be inspected from the image data and an abnormality determination unit that determines whether the number of abnormality pixels of an abnormality location identified by the abnormality location detection unit in the image data is more than a first predetermined number, and determines that the abnormality is an abnormal state if the number of abnormality pixels is more than the first predetermined number.SELECTED DRAWING: Figure 3

Description

The present disclosure relates to a patrol inspection system and a patrol inspection method using an air vehicle or the like.

Conventionally, for periodic inspections of power transmission equipment such as overhead power transmission lines (hereinafter referred to as power transmission lines) and patrols in the event of an accident, a patrolman rides on a helicopter and flies along the power transmission line to see the power transmission line with binoculars. However, he was checking for abnormalities and checking the power transmission equipment while moving on foot.

However, inspection / patrol by the patrolman with binoculars is a burden on the patrolman, and there is a risk of overlooking the abnormal part.

In addition to the cost of using a helicopter, inspection / patrol of power transmission equipment using a helicopter requires a plurality of patrols in addition to the helicopter operator, so the cost for inspection / patrol is high. In addition, if an inspection / patrol is suddenly required, it cannot be handled simply and quickly. In addition, an image of a transmission line is taken from a helicopter, and a patrolman looks at the taken image to check for a broken wire of the transmission line and the presence or absence of an arc mark (lightning mark).

In addition, the inspection work of the transmission line by taking an image also has a problem that a quick inspection cannot be performed because an abnormal part is detected while the patrolman is looking at all the taken images.

In this respect, a method of detecting wire breakage and arc marks by using an image processing technique has been proposed (Patent Document 1).

JP-A-2018-74757

The present disclosure is to provide a patrol inspection system and a patrol inspection method capable of detecting an abnormal portion generated in a power transmission facility by a simple method by image processing.

It is a patrol inspection system for patrol inspection of the object to be inspected based on the image data taken by the camera mounted on the aircraft according to a certain situation, and there is a possibility that an abnormality has occurred in the object to be inspected from the image data. It is determined whether or not the number of abnormal pixels of the abnormal part identified by the abnormal part detecting unit for identifying the abnormal part and the abnormal part detecting unit in the image data is equal to or more than the first predetermined number, and the abnormal part is the first. When the number is equal to or more than a predetermined number, an abnormality determining unit for determining an abnormal state is provided.

Preferably, when the number of abnormal pixels in the abnormal portion in the image data is less than the first predetermined number, the abnormality determining unit determines whether or not there are abnormal pixels in the second predetermined number or more of the set group, and determines whether or not there are abnormal pixels in the second predetermined number or more. When the number of abnormal pixels in the abnormal portion in the data is less than the first predetermined number, and when there are abnormal pixels in the set group of the second predetermined number or more, it is determined to be an abnormal state.

Preferably, the abnormality determining unit is the outer periphery of the object to be inspected when the number of abnormal pixels in the abnormal portion in the image data is less than the first predetermined number and when there is no abnormal pixel in the set group of the second predetermined number or more. It is determined whether or not there are abnormal pixels adjacent to the area, and when the number of abnormal pixels in the abnormal part in the image data is less than the first predetermined number, the abnormal pixels of the second predetermined number or more of the set group are If there are no abnormal pixels adjacent to the outer peripheral area of the object to be inspected, it is judged to be in an abnormal state.

A patrol inspection system for patrol inspection of an object to be inspected based on image data taken by a camera mounted on an aircraft according to another aspect, and a plurality of first shots of the object to be inspected continuously. A recording unit that records the image data of the above, a specific unit that identifies the inspected object from the plurality of first image data, and a shooting state of the inspected object of the first image data of each of the plurality of first image data. It is provided with a determination unit for determining the quality of the image, and a setting unit for recording image data in a poor shooting state as defective image data in the recording unit based on the determination result of the determination unit among the plurality of first image data.

Preferably, the determination unit calculates the width of the object to be inspected of the first image data of each of the plurality of first image data, and the width of the object to be inspected among the plurality of first image data is equal to or less than a predetermined value. For the first image data determined to be, it is determined that the photographing state of the object to be inspected is defective.

Preferably, the determination unit determines whether or not there is an outer peripheral region of the object to be inspected in each of the first image data of the plurality of first image data, and the object to be inspected among the plurality of first image data. The first image data, which is determined to have no outer peripheral region, is determined to be in a poor imaging state of the object to be inspected.

Preferably, the recording unit records a plurality of second image data continuously re-photographed about the object to be inspected, the specific unit identifies the object to be inspected from the plurality of second image data, and the determination unit , The quality of the shooting state of the inspected object of each of the second image data of the plurality of second image data is determined, and among the plurality of second image data, the defect recorded in the recording unit for the inspected object. A plurality of first images by changing a specific unit that identifies the re-photographed image data determined to be in a good shooting state at a position corresponding to the image data and a re-photographed image data that specifies the defective image data by the specific unit. It has an editorial department for editing as data.

It is a patrol inspection method for patrol inspection of the inspected object based on the image data taken by the camera mounted on the aircraft according to a certain situation, and there is a possibility that an abnormality has occurred in the inspected object from the image data. A step of identifying an abnormal part, a step of determining whether or not the number of abnormal pixels of the specified abnormal part in the image data is equal to or more than the first predetermined number, and a case where the abnormal part is equal to or more than the first predetermined number. Is provided with a step of determining an abnormal state.

It is a patrol inspection method for patrol inspection of the inspected object based on the image data taken by the camera mounted on the aircraft according to another aspect, and the inspected object recorded in the recording unit is continuously inspected. A step of identifying an inspected object from a plurality of captured first image data, a step of determining the quality of the photographed state of the inspected object of the first image data of each of the plurality of first image data, and a step of determining the quality of the inspected object. The present invention includes a step of recording image data in a poor shooting state as defective image data in a recording unit based on a determination result among the plurality of first image data.

The patrol inspection system and the patrol inspection method of the present disclosure can detect an abnormal part generated in a power transmission facility by a simple method by image processing.

It is a figure explaining the patrol inspection according to Embodiment 1. FIG. It is a figure explaining the structure of the patrol inspection system 1 according to Embodiment 1. FIG. FIG. 5 is a block diagram showing a functional configuration realized by executing a patrol inspection control program by a processor 2 in a patrol inspection system 1 according to the first embodiment. It is a figure explaining the learning model according to Embodiment 1. FIG. It is a figure explaining the abnormality determination processing of the abnormality candidate image data in the abnormality determination unit 26 according to Embodiment 1. FIG. It is a figure explaining the flow of the abnormality determination processing in the patrol inspection system 1 according to Embodiment 1. FIG. It is a figure explaining the subroutine flow of the abnormality determination processing of the abnormality determination part 26 according to Embodiment 1. FIG. FIG. 5 is a block diagram showing a functional configuration realized by executing a patrol inspection control program by a processor 2 in a patrol inspection system 1 according to the second embodiment. It is a figure explaining the structure of the data verification unit 29 according to Embodiment 2. It is a figure explaining the quality determination process of image data in the quality determination unit 32 according to Embodiment 2. FIG. It is a figure explaining the flow of the data verification processing in the patrol inspection system 1 according to Embodiment 2. It is a figure explaining the subroutine flow of the quality determination process of the quality determination unit 32 according to Embodiment 2. It is a flow diagram explaining the editing process of the editorial unit 36 according to the second embodiment. It is a figure explaining the editing process of the editorial unit 36 according to Embodiment 2.

The embodiment will be described in detail with reference to the drawings. The same or corresponding parts in the drawings are designated by the same reference numerals, and the description thereof will not be repeated.

(Embodiment 1)
FIG. 1 is a diagram illustrating a patrol inspection according to the first embodiment.

As shown in FIG. 1, the patrol inspection system 1 uses an aircraft, for example, a helicopter 100, for example, for a power transmission facility (transmission line, steel tower) 200 for transmitting power from a power plant or a substation during normal times or when an accident occurs. Then, it is a system for patrol inspection.

In this example, the case where the helicopter 100 is used will be described, but it is also possible to use an unmanned aerial vehicle (UAV), not limited to a manned vehicle.

The patrol inspection system 1 is generated in a plurality of different objects to be inspected (for example, overhead ground wire, transmission line, etc.) included in the power transmission facility 200 based on the image data taken by the camera mounted on the helicopter 100. Detect an abnormal part.

The patrol inspection system 1 according to the first embodiment is configured by using, for example, an information processing device. It is also possible to realize it as a plurality of devices via a network. For example, it may be realized by cloud computing, and may be realized by one server connected via a network (Internet) or a plurality of servers operating in cooperation with each other.

FIG. 2 is a diagram illustrating a configuration of a patrol inspection system 1 according to the first embodiment.
With reference to FIG. 2, the patrol inspection system 1 includes a processor 2, a memory 4, a storage device 6, an input / output I / F 8, a display device 10, an input device 12, and a communication device 14.

Each part is connected to each other by an internal bus, and data can be exchanged with each other.

The processor 2 executes a basic program (OS) or an application program stored in the memory 4 to realize various functions. For example, the processor 2 detects an abnormal part generated in an object to be inspected in a power transmission facility based on an image taken by a camera mounted on the helicopter 100 by executing a patrol inspection control program, and the detection result is obtained. The process of outputting the patrol inspection result is executed based on.

The memory 4 stores a program executed by the processor 2, temporary data, and the like.

The storage device 6 stores various programs and various data. The data stored in the storage device 6 includes acquired data (image data, position data) received from the helicopter 100, and learning model data used for image processing for detecting an abnormal portion generated in the inspected object from the image data. , Includes image data of abnormal parts, etc.

The input / output I / F 8 is an interface for transmitting / receiving data to / from an external device. The input / output I / F8 can input / output data via, for example, a portable memory medium.

The display device 10 is an LCD (Liquid Crystal Display) or the like, and displays a screen corresponding to the processing of the processor 2.

The input device 12 is a keyboard, a pointing device, or the like, and is operated by an operator or the like.

The communication device 14 controls wireless communication or wired communication, and controls wireless communication or wired communication with the helicopter 100 or with a base station housed in the wireless public network. The communication device 14 controls communication with an external device, for example, another information processing device or the like through the network. The network includes a wireless or wired WAN (Wide Area Network), a LAN (Local Area Network), and the like.

The patrol inspection system 1 executes detection of an abnormal portion generated in the object to be inspected based on image data taken by a camera mounted on the helicopter 100, confirmation processing of an image of the abnormal portion, and the like.

The helicopter 100 is a patrol inspection target while storing position data generated by using GNSS (Global Navigation Satellite System), for example, position data generated from GPS signals received from GPS (Global Positioning System) satellites. The power transmission facility 200 is photographed and image data is stored.

The patrol inspection system 1 can also receive image data taken by a camera mounted on the helicopter 100 in real time using the communication device 14 to detect an abnormal portion generated in the object to be inspected. However, the image data taken by the camera may be once recorded on an information storage medium such as a disk, and then the abnormal portion generated in the inspected object may be detected after the data is stored in the storage device 6.

In this example, the patrol inspection system 1 will be described as being provided outside the helicopter 100, but may be mounted inside the helicopter 100.

FIG. 3 is a block diagram showing a functional configuration realized by executing a patrol inspection control program by the processor 2 in the patrol inspection system 1 according to the first embodiment.

As shown in FIG. 3, the processor 2 is based on the patrol inspection control program, for example, a learning model storage unit 20, an abnormality location detection unit 22, an acquisition data storage unit 24, an abnormality determination unit 26, and confirmation data. The function with the storage unit 28 is realized.

The learning model storage unit 20 stores the learning model used in the patrol inspection system 1. The learning model is for detecting the electric wire region and the abnormal portion from the image data acquired by the camera mounted on the helicopter 100, and represents the characteristics of the abnormal state occurring in the object to be inspected. As an abnormality that has occurred in the object to be inspected, for example, in the case of a power transmission line, there is an arc mark.

FIG. 4 is a diagram illustrating a learning model according to the first embodiment.
As shown in FIG. 4, the image data of the abnormal portion actually generated and the label image data to which the information about the image data is added are combined and stored as a trained model.

Specifically, label image data classified into a background image (label "0"), an electric wire image (label "1"), and an abnormal image (label "2") is stored. In this example, the case of storing the label image data classified into three categories will be described, but the label image data classified into the electric wire image (label "1") and the abnormal image (label "2"). You may try to remember.

The patrol inspection system 1 can reliably detect the abnormal portion generated in the inspected object by searching for the abnormal portion from the image data using the learning model stored in the learning model storage unit 20. ..

In addition to the image data of the abnormal part that actually occurred, a deep learning (deep learning) technology that utilizes a neural network to generate a pseudo image (similar image) that shows the abnormal / deteriorated state that occurs in the object to be inspected. It may be used to generate a trained model. Using deep learning technology, you can create similar images by changing the combination of the image of the abnormal part (abnormal image) and the normal image around the abnormal part, and the ratio of combining the normal image and the abnormal image. By generating it, it is also possible to train (generate) a learning model based on a large number of similar images.

The acquired data storage unit 24 stores the acquired data acquired from the helicopter 100. The acquired data includes, for example, moving image data (which may be still image data) of an object to be inspected, position data (latitude, longitude, altitude) indicating the position of the helicopter 100 at the time of image shooting, and the like.

The abnormality location detection unit 22 performs image processing based on the learning model (characteristics of the abnormality portion of the object to be inspected) stored in the learning model storage unit 20 from the acquired data stored in the acquisition data storage unit 24. Detect areas and abnormal areas. Specifically, the abnormality location detection unit 22 uses the learning model for the acquired image data to generate the label image data described in FIG. 4 corresponding to the image data. For example, among the acquired image data, the pixels determined as the background are set to the label "0". Further, among the acquired image data, the pixel determined as the electric wire is set to the label "1". Of the acquired image data, the pixels determined to be abnormal are set to the label "2". When the label image data includes the label "2", the acquired image data is determined to be an abnormality candidate image. The abnormality location detection unit 22 outputs the generated label image data as abnormality candidate image data to the abnormality determination unit 26 together with the image data stored in the acquisition data storage unit 24.

The abnormality determination unit 26 determines whether or not there is an abnormality in the abnormality candidate image (also referred to as an abnormality candidate image) detected by the abnormality location detection unit 22.

The confirmation data storage unit 28 stores each data indicating the abnormality image data, the detection position, the detection date and time, etc., for which the abnormality candidate image is confirmed to be abnormal by the abnormality determination unit 26, as confirmation data. This makes it possible to easily confirm the abnormal part.

FIG. 5 is a diagram illustrating an abnormality determination process of abnormality candidate image data in the abnormality determination unit 26 according to the first embodiment.

In this example, the abnormality determination unit 26 determines whether or not there is an abnormality based on the label image data included in the abnormality candidate image data.

As shown in FIG. 5A, with respect to the abnormality candidate image data (label image data), when the number of candidate pixels (label “2”) at the abnormality location is equal to or greater than the first predetermined number, the abnormality is concerned. The candidate image data is determined to be abnormal. If the image has a large number of pixels that are judged to be abnormal due to scratches or deterioration over time, it is judged that there is an abnormality.

As shown in FIG. 5B, with respect to the abnormality candidate image data (label image data), even when the number of candidate pixels (label “2”) at the abnormality portion is not equal to or more than the first predetermined number. When the candidate pixels (label "2") of the abnormal portion of the set group are equal to or larger than the second predetermined number, the abnormal candidate image data is determined to be abnormal. In the case of an image having pixels that are locally determined to be abnormal in the set group, such as arc marks, it is determined that there is an abnormality.

As shown in FIG. 5C, with respect to the abnormality candidate image data (label image data), the number of candidate pixels (label “2”) at the abnormality portion is not equal to or more than the first predetermined number, and the abnormality of the set group is abnormal. Even if the number of candidate pixels (label "2") at the location is not equal to or greater than the second predetermined number, if there are candidate pixels (label "2") at the abnormal location adjacent to the outer peripheral region of the electric wire, there is a candidate pixel (label "2") at the location. Judge as abnormal. If the image has pixels that are determined to be abnormal in the outer peripheral region, such as broken wires, it is determined that there is an abnormality.

As shown in FIG. 5D, with respect to the abnormality candidate image data (label image data), even if there are a plurality of candidate pixels for the abnormality portion, the number is less than the first predetermined number and the abnormality portion of the set group is abnormal. If the number of candidate pixels is less than the second predetermined number, it is determined that the abnormality candidate image data is not abnormal. Even if there are pixels that are determined to be abnormal, if the number is small, it is determined that there is no abnormality.

As shown in FIG. 5 (E), regarding the abnormality candidate image data (label image data), when the candidate image of the abnormality portion is outside the outer peripheral region of the electric wire, the abnormality candidate image data is not abnormal. judge. Since it is outside the outer peripheral region of the electric wire, there is a high possibility of noise, so even if there are pixels that are judged to be abnormal, it is judged that there is no abnormality.

FIG. 6 is a diagram illustrating a flow of abnormality determination processing in the patrol inspection system 1 according to the first embodiment.

With reference to FIG. 6, the patrol inspection system 1 acquires image data (step S2).
Specifically, the image data captured by the camera mounted on the helicopter 100 is stored in the storage device 6. The acquired data storage unit 24 stores the acquired data acquired from the helicopter 100. The abnormality location detection unit 22 acquires the image data stored in the acquisition data storage unit 24.

Next, the patrol inspection system 1 detects the electric wire region and the abnormal portion (step S4). The abnormality location detection unit 22 detects the abnormality location based on the image data stored in the acquired data storage unit 24 and the learning model stored in the learning model storage unit 20. The abnormality location detection unit 22 generates label image data using the learning model, and when it is determined that there is an abnormality portion in the image data stored in the acquisition data storage unit 24, the abnormality determination unit 22 serves as an abnormality candidate image data. Output to 26.

Next, the patrol inspection system 1 determines the abnormality (step S6). The abnormality determination unit 26 determines whether or not the abnormality candidate image is actually abnormal. The details of the abnormality determination process will be described later.

Then, the process ends (end).
FIG. 7 is a diagram illustrating a subroutine flow of abnormality determination processing of the abnormality determination unit 26 according to the first embodiment.

As shown in FIG. 7, the patrol inspection system 1 determines whether or not the number of abnormal pixels is equal to or greater than the first predetermined number (step S10). The abnormality determination unit 26 determines whether or not the number of abnormal pixels determined to be abnormal locations included in the acquired abnormality candidate image data is equal to or greater than the first predetermined number. In this example, the abnormality determination unit 26 counts the number of labels “2” included in the label image data, and determines whether or not the counted number is equal to or greater than the first predetermined number. The first predetermined number can be set to any number.

In step S10, when the patrol inspection system 1 determines that the number of abnormal pixels is equal to or greater than the first predetermined number (YES in step S10), it determines that there is an abnormality (step S18). When the abnormality determination unit 26 determines that the number of labels "2" included in the label image data is equal to or greater than the first predetermined number, the abnormality determination unit 26 is abnormal according to scratches and aging deterioration as described in FIG. 5 (A). Image data with a high possibility of being abnormal is judged to be abnormal.

Then, the process ends (return).
Next, when the patrol inspection system 1 determines that the number of abnormal pixels is not equal to or greater than the first predetermined number (NO in step S10), whether or not the number of abnormal pixels in the set group is equal to or greater than the second predetermined number. Is determined (step S12). When the number of abnormal pixels determined to be abnormal parts included in the acquired abnormality candidate image data is not equal to or greater than the first predetermined number, the abnormality determination unit 26 has the number of abnormal pixels in the set group to be equal to or greater than the second predetermined number. Judge whether or not. In this example, the abnormality determination unit 26 counts the number of adjacent labels “2” included in the label image data, and determines whether or not the counted number is equal to or greater than the second predetermined number. .. The second predetermined number can be set to any number.

In step S12, when the patrol inspection system 1 determines that the number of abnormal pixels in the set group is equal to or greater than the second predetermined number (YES in step S12), it determines that there is an abnormality (step S18). When the abnormality determination unit 26 determines that the number of adjacent labels “2” included in the label image data is equal to or greater than the second predetermined number, the arc marks of the arc marks are described as described in FIG. 5 (B). Image data with a high possibility of such anomalies is judged to have anomalies.

Then, the process ends (return).
Next, when the patrol inspection system 1 determines that the number of abnormal pixels in the set group is not equal to or greater than the second predetermined number (NO in step S12), whether or not there are abnormal pixels adjacent to the outer peripheral region of the electric wire. Is determined (step S14). The abnormality determination unit 26 determines that the number of abnormal pixels determined to be abnormal parts included in the acquired abnormality candidate image data is not equal to or greater than the first predetermined number, and the number of abnormal pixels in the set group is not equal to or greater than the second predetermined number. , Determine if there are abnormal pixels adjacent to the outer peripheral region of the wire. In this example, the abnormality determination unit 26 identifies the outer peripheral region of the electric wire based on the label “1” included in the label image data. The abnormality determination unit 26 determines whether or not there is an abnormality pixel of the label “2” adjacent to the outer peripheral region of the specified electric wire.

In step S14, when the patrol inspection system 1 determines that there are abnormal pixels adjacent to the outer peripheral region of the electric wire (YES in step S14), it determines that there is an abnormality (step S18). When the abnormality determination unit 26 has an abnormality pixel of the label “2” adjacent to the outer peripheral region of the specified electric wire, there is a possibility of an abnormality such as a broken wire as described in FIG. 5 (C). High image data is judged to have an abnormality.

Then, the process ends (return).
On the other hand, in step S14, when the patrol inspection system 1 determines that there are no abnormal pixels adjacent to the outer peripheral region of the electric wire (NO in step S14), it determines that there is no abnormality (step S16). The abnormality determination unit 26 determines that there is no abnormality in the image data having a low possibility of abnormality such as noise as described with reference to FIG. 5 (D) or FIG. 5 (E).

Then, the process ends (return).
Although the case of executing the determination processing of steps S10, S12, and S14 has been described, the present invention is not limited to this, and any one of them may be used, or two determination processes may be executed in any combination.

The patrol inspection system 1 according to the first embodiment can detect an abnormal portion generated in an object to be inspected (for example, an overhead ground wire, a power transmission line, etc.) by a simple method by image processing according to the above. Further, the patrol inspection system 1 according to the first embodiment is not only for abnormalities due to scratches and aging deterioration of the object to be inspected (for example, overhead ground wire, power transmission line, etc.), but also for abnormalities such as arc marks and broken wires. Highly accurate abnormality detection is possible.

(Embodiment 2)
In the first embodiment described above, a method for determining an abnormality in image data has been described. On the other hand, there is a possibility that the object to be inspected (for example, an overhead ground wire, a power transmission line, etc.) is not normally photographed with respect to the image data acquired by the camera mounted on the helicopter 100.

FIG. 8 is a block diagram showing a functional configuration realized by executing a patrol inspection control program by the processor 2 in the patrol inspection system 1 according to the second embodiment.

As shown in FIG. 8, the point that the data verification unit 29 is further provided is different from the configuration described in FIG. Since the other configurations are the same, the detailed description thereof will not be repeated.

The data verification unit 29 verifies whether or not the acquired image data stored in the acquired data storage unit 24 is normal image data. That is, it is verified whether or not the object to be inspected is photographed normally. In this example, if it is determined that the object to be inspected is not photographed normally, the object to be inspected is photographed again. In this example, a case where an electric wire is photographed as an example of an object to be inspected will be described.

FIG. 9 is a diagram illustrating the configuration of the data verification unit 29 according to the second embodiment.
As shown in FIG. 9, the data verification unit 29 includes a specific unit 30, a pass / fail determination unit 32, a setting unit 34, and an editorial unit 36.

The identification unit 30 identifies the electric wire to be inspected from the image data.
The quality determination unit 32 determines the quality of the photographing state of the electric wire, which is the object to be inspected in the image data.

The setting unit 34 records the image data in a good shooting state as good image data and the image data in a bad shooting state as bad image data in the storage device 6 based on the determination result of the quality determination unit 32.

The editorial unit 36 changes the defective image data recorded in the storage device 6 into good re-photographed image data.

FIG. 10 is a diagram illustrating a quality determination process of image data in the quality determination unit 32 according to the second embodiment.

The quality determination unit 26 determines whether or not the shooting state is defective based on the image data.
As shown in FIG. 10A, in the case of image data in which a part of the electric wire, which is the object to be inspected, is missing, the photographing state is defective, and it is determined that the image data is defective. Specifically, if the wire width of the object to be inspected included in the image data is equal to or less than a predetermined value, it is determined to be defective image data.

As shown in FIG. 10B, even if an image in which a part of the electric wire to be inspected is not missing, if the size of the inspected object is small, the photographing state is poor and the defective image. Judge as data. Specifically, if the wire width of the object to be inspected included in the image data is equal to or less than a predetermined value, it is determined to be defective image data.

As shown in FIG. 10C, even if the width of the electric wire, which is the object to be inspected, included in the image data exceeds a predetermined value, the image data is in a state of being photographed in the case where the outer peripheral region is missing. Is defective and is determined to be defective image data. Specifically, it is determined whether or not the outer peripheral region of the electric wire of the object to be inspected included in the image data exists, and if it does not exist, it is determined as defective image data. For example, it may be determined whether or not the outer peripheral region exists depending on whether or not pixels other than the electric wire are present in the vertical direction of the pixels constituting the electric wire of the image data.

FIG. 11 is a diagram illustrating a flow of data verification processing in the patrol inspection system 1 according to the second embodiment.

With reference to FIG. 11, the patrol inspection system 1 acquires image data (step S20).

Specifically, the image data captured by the camera mounted on the helicopter 100 is stored in the storage device 6. The acquired data storage unit 24 stores the acquired data acquired from the helicopter 100. The specific unit 30 acquires the image data stored in the acquired data storage unit 24.

Next, the patrol inspection system 1 identifies the electric wire region (step S22). The identification unit 30 identifies the electric wire region to be inspected from the image data stored in the acquired data storage unit 24.

Specifically, as described above, the electric wire region is specified by image processing based on the learning model stored in the learning model storage unit 20. Alternatively, the identification unit 30 may specify the electric wire region by the shape, or may specify the electric wire region based on the color data included in the image data.

Next, the patrol inspection system 1 determines whether the shooting condition is good or bad (step S24). The quality determination unit 32 determines the quality of the photographing state based on the specified electric wire region of the image data. The details of the quality determination process of the shooting state will be described later.

FIG. 12 is a diagram illustrating a subroutine flow of the pass / fail determination process of the pass / fail determination unit 32 according to the second embodiment.

As shown in FIG. 12, the patrol inspection system 1 calculates the wire width (step S30). The quality determination unit 32 calculates the wire width of the specified wire region. For example, the wire width may be calculated from the number of pixels (the number of pixels (label "1")) in the specified wire region of the image data. The wire width is the distance in the direction orthogonal to the length direction of the wire region.

Next, the patrol inspection system 1 determines whether or not the wire width is equal to or less than a predetermined value (step S32). The quality determination unit 32 determines whether or not the wire width is equal to or less than a predetermined value. Specifically, the quality determination unit 32 may determine whether or not the number of pixels in the direction orthogonal to the length direction of the specified electric wire region of the image data is equal to or less than a predetermined number.

In step S32, when the patrol inspection system 1 determines that the wire width is equal to or less than a predetermined value (YES in step S32), the patrol inspection system 1 determines that the photographing state is defective (step S38). When the quality determination unit 32 determines that the wire width is equal to or less than a predetermined value, the imaging state is defective as described with reference to FIGS. 10A and 10B, and the quality determination unit 32 determines that the image data is defective.

Then, the process ends (return).
On the other hand, when the patrol inspection system 1 determines that the wire width is not equal to or less than a predetermined value (NO in step S32), it determines whether or not there is an outer peripheral region of the specified wire region of the image data (step). S34). The quality determination unit 32 determines whether or not there is an outer peripheral region of the electric wire region included in the acquired image data. For example, it may be determined whether or not the outer peripheral region exists depending on whether or not the pixels other than the electric wire region exist in the vertical direction of the pixels constituting the electric wire region included in the acquired image data. If at least one pixel exists, it may be determined that the outer peripheral region of the electric wire region exists, and if a margin is provided and a plurality of pixels exist, it is determined that the outer peripheral region of the electric wire region exists. You may do so.

In step S34, when the patrol inspection system 1 determines that there is no outer peripheral region of the specified electric wire region (NO in step S34), it determines that the photographing state is defective (step S38). When the quality determination unit 32 determines that there is no outer peripheral region of the specified electric wire region, as described with reference to FIG. 10C, the imaging state is defective, and the quality determination unit 32 determines that the image data is defective.

Then, the process ends (return).
On the other hand, in step S34, when the patrol inspection system 1 determines that there is an outer peripheral region of the specified electric wire region (YES in step S34), it determines that the photographing state is good (step S36).

Then, the process ends (return).
With reference to FIG. 11 again, the patrol inspection system 1 sets the determination result in the captured image data (step S26).

Specifically, the setting unit 34 sets the result of the quality determination of the quality determination unit 32 in the image data. For example, the setting unit 34 sets the determination results in association with each of the plurality of image data stored in the storage device 6 as good or bad image data.

Then, the process ends (end).
In this example, if it is determined that the object to be inspected is not photographed normally, the object to be inspected is photographed again. In this case, the process described with reference to FIG. 11 is also executed for the retaken image data. The setting unit 34 sets the result of the quality determination of the quality determination unit 32 in the re-photographed image data. The setting unit 34 sets the determination results in association with each of the plurality of re-photographed image data stored in the storage device 6 as good or bad image data.

FIG. 13 is a flow chart illustrating the editing process of the editing unit 36 according to the second embodiment.
As shown in FIG. 13, the patrol inspection system 1 acquires defective image data stored in the storage device 6 (step S40). The editorial unit 36 acquires defective image data set as defective stored in the acquired data storage unit 24.

Next, the patrol inspection system 1 identifies the corresponding re-photographed image data (step S42). Specifically, the editorial unit 36 identifies the re-photographed image data corresponding to the same position information based on the position information associated with the defective image data. The position information may be the position information regarding the specified electric wire area, or the position information of the helicopter 100 provided with the camera may be used.

Next, the patrol inspection system 1 edits the image data stored in the storage device 6 (step S44). The editorial unit 36 acquires the position data of the image data in which the shooting state is set to be defective, identifies and replaces the re-shooting image data corresponding to the position data.

Next, the patrol inspection system 1 determines whether or not all the image data set to be defective have been checked (step S46).

If it is determined in step S46 that the patrol inspection system 1 has not checked all the image data set for defects (NO in step S46), the process returns to step S40 and the images set for other defects are set. The data is acquired and the above process is repeated.

On the other hand, in step S46, when it is determined that the patrol inspection system 1 has checked all the image data set to be defective (YES in step S46), the process ends (end).

FIG. 14 is a diagram illustrating an editing process of the editing unit 36 according to the second embodiment.
As shown in FIG. 14, for example, a case where a part of the moving image data is set as defective image data because the shooting state is defective is shown.

The editorial unit 36 acquires the position data of the defective image data whose shooting state is set to be defective, and identifies the re-photographed image data corresponding to the position data. In this example, the case where the retaken image data in the position data PE corresponds to the defective image data is shown.

Then, the editorial unit 36 replaces the re-photographed image data with the defective image data and edits the data.
Thereby, it is possible to replace the defective image data with the re-photographed image data and acquire appropriate image data. It is assumed that the re-photographed image data is set as good image data. Regarding the replacement of the re-photographed image data, it is determined that not only the re-photographed image data corresponding to the same position as the position data PE determined to be defective image data but also the photographing state corresponding to the positional relationship before and after the re-photographed image data is good. A plurality of retaken image data may be exchanged.

The patrol inspection system 1 according to the second embodiment determines whether or not the image data obtained by photographing the inspected object is normal before the detection process of the abnormal portion generated in the inspected object.

Then, in the case of defective image data, it is possible to use the re-photographed image data acquired by re-imaging and replace it with the defective image data to create an appropriate set of image data.

For example, when photographing a long-distance object to be inspected (for example, an overhead ground wire, a power transmission line, etc.), there is a possibility that some photographs may not be appropriate due to environmental factors. According to this method, defective image data determined to be defective can be replaced with good re-photographed image data, and highly accurate patrol inspection is possible.

The method described in each of the above embodiments stores a magnetic disk (hard disk, etc.), an optical disk (CDROM, DVD, etc.), a magneto-optical disk (MO), a semiconductor memory, etc. as a program that can be executed by a computer. It can also be stored on a medium and distributed. Further, the storage medium may be in any form as long as it is a storage medium capable of storing a program and readable by a computer.

Further, in order for the OS (operating system) running on the computer, the database management software, the MW (middleware) such as the network software, etc. to realize the above embodiment based on the instruction of the program installed on the computer from the storage medium. You may execute a part of each process of.

Further, the storage medium in each embodiment is not limited to a medium independent of the computer, but also includes a storage medium in which a program transmitted by a LAN, the Internet, or the like is downloaded and stored or temporarily stored.

Further, the storage medium is not limited to one, and the case where the processing in each of the above embodiments is executed from a plurality of media is also included in the storage medium in the present invention, and the medium configuration may be any configuration.

The computer in each embodiment executes each process in each of the above embodiments based on the program stored in the storage medium, and is composed of one device such as a personal computer and a plurality of devices in a network. Any configuration such as a connected system may be used.

It should be considered that the embodiments disclosed this time are exemplary in all respects and not restrictive. The scope of the present disclosure is indicated by the scope of claims, not the above description, and is intended to include all modifications within the meaning and scope equivalent to the scope of claims.

1 Patrol inspection system, 2 processors, 4 memories, 6 storage devices, 10 display devices, 12 input devices, 14 communication devices, 20 learning model storage units, 22 abnormality location detection units, 24 acquisition data storage units, 26 abnormality determination units, 28 Confirmation data storage unit, 29 Data verification unit, 30 Specific unit, 32 Good / bad judgment unit, 34 Setting unit, 36 Editorial department, 100 Helicopter, 200 Transmission equipment.

Claims (9)

It is a patrol inspection system for patrol inspection of the object to be inspected based on the image data taken by the camera mounted on the aircraft.
An abnormality location detection unit that identifies an abnormality location where an abnormality may have occurred in the object to be inspected from the image data,
When it is determined whether or not the number of abnormal pixels of the abnormal portion specified by the abnormal portion detection unit in the image data is equal to or greater than the first predetermined number, and the abnormal portion is equal to or greater than the first predetermined number. Is a patrol inspection system equipped with an abnormality discriminating unit that discriminates from an abnormal state. The abnormality determination unit
When the number of abnormal pixels in the abnormal portion in the image data is less than the first predetermined number, it is determined whether or not there are abnormal pixels in the set group of the second predetermined number or more.
Claim 1 in which, when the number of abnormal pixels in the abnormal portion in the image data is less than the first predetermined number and there are abnormal pixels in the set group of the second predetermined number or more, it is determined to be an abnormal state. Described patrol inspection system. The abnormality determination unit
When the number of abnormal pixels in the abnormal portion in the image data is less than the first predetermined number, and when there is no abnormal pixel in the set group of the second predetermined number or more, the image data is adjacent to the outer peripheral region of the object to be inspected. Judge whether there is an abnormal pixel that is
When the number of abnormal pixels in the abnormal portion in the image data is less than the first predetermined number and there is no abnormal pixel in the set group of the second predetermined number or more, the image data is adjacent to the outer peripheral region of the object to be inspected. The patrol inspection system according to claim 2, wherein if there is an abnormal pixel, it is determined to be in an abnormal state. It is a patrol inspection system for patrol inspection of the object to be inspected based on the image data taken by the camera mounted on the aircraft.
A recording unit that records a plurality of first image data continuously captured for the object to be inspected, and a recording unit.
A specific part that identifies the object to be inspected from the plurality of first image data, and
A determination unit for determining the quality of the shooting state of the object to be inspected of the first image data of each of the plurality of first image data, and a determination unit.
A patrol inspection system including a setting unit that records image data in a poor shooting state as defective image data in the recording unit based on a determination result of the determination unit among the plurality of first image data. The determination unit
The width of the object to be inspected of the first image data of each of the plurality of first image data is calculated.
The fourth aspect of claim 4, wherein the first image data in which the width of the inspected object is determined to be equal to or less than a predetermined value among the plurality of first image data is determined to be in a defective photographing state of the inspected object. Patrol inspection system. The determination unit
It is determined whether or not there is an outer peripheral region of the object to be inspected in each of the first image data of the plurality of first image data.
The patrol inspection according to claim 5, wherein among the plurality of first image data, the first image data for which it is determined that there is no outer peripheral region of the object to be inspected is determined to have a defective imaging state of the object to be inspected. system. The recording unit records a plurality of second image data that are continuously re-photographed with respect to the object to be inspected.
The specific unit identifies the object to be inspected from the plurality of second image data, and then
The determination unit determines whether or not the imaged state of the inspected object of the second image data of each of the plurality of second image data is good or bad.
Among the plurality of second image data, a specific unit for specifying the re-photographed image data for which the imaging state at the position corresponding to the defective image data recorded in the recording unit for the object to be inspected is determined to be good. ,
The patrol inspection system according to claim 4, further comprising an editorial unit that changes the defective image data into rephotographed image data specified by the specific unit and edits the defective image data as the plurality of first image data. It is a patrol inspection method for patrol inspection of the object to be inspected based on the image data taken by the camera mounted on the aircraft.
A step of identifying an abnormal part where an abnormality may have occurred in the object to be inspected from the image data, and
A step of determining whether or not the number of abnormal pixels of the specified abnormal portion in the image data is equal to or greater than the first predetermined number, and
A patrol inspection method comprising a step of determining an abnormal state when the number of abnormal points is equal to or greater than the first predetermined number. It is a patrol inspection method for patrol inspection of the object to be inspected based on the image data taken by the camera mounted on the aircraft.
A step of identifying the inspected object from a plurality of first image data continuously photographed about the inspected object recorded in the recording unit, and
A step of determining the quality of the photographing state of the inspected object of the first image data of each of the plurality of first image data, and
A patrol inspection method comprising a step of recording image data in a poor shooting state as defective image data in the recording unit based on a determination result among the plurality of first image data.

Images