JP2021148589A - Trolley wire inspection device and trolley wire inspection method - Google Patents

Abstract

To provide a trolley wire inspection device and a trolley wire inspection method, with which it is possible to execute inspection of trolley wires easily with good accuracy.SOLUTION: Provided is a trolley wire inspection device 20 comprising: a line sensor image generation unit 30 for arranging input images having been imaged by a line sensor 22 in order of time series and generating a line sensor image; a learning model 40 having been trained by deep learning using, as training data, the line sensor image and a correct answer value, for each pixel in the line sensor image, of whether or not the pixel corresponds to a worn region of trolley wire; a worn region extraction unit 31 for inputting the line sensor image to the learning model 40, inferring, for each pixel in the line sensor image, whether or not the pixel corresponds to the worn region, and generating a worn region extraction image in which the worn region is extracted; and a trolley wire inspection unit 32 for executing, on the basis of the worn region extraction image, one of calculation of the remaining diameter of the trolley wire, calculation of deviation, and determination of excessive wear, or a combination of some of these.SELECTED DRAWING: Figure 2

Description

The present invention relates to a trolley line inspection device and a trolley line inspection method.

When traveling on a railroad track, an electric railroad vehicle obtains power by supplying power from a trolley wire provided above the railroad track via a current collector such as a pantograph. Each time an electric railroad vehicle passes, the current collector slides against the underside of the trolley wire. Therefore, if the electric railway vehicle is continuously operated, the trolley wire gradually wears and eventually breaks. Therefore, a wear limit is set for the trolley wire, and when the remaining diameter of the trolley wire falls below the control value, the trolley wire is replaced with a new one.
On the other hand, since the contact part of the current collector also wears, the trolley wire is installed in a zigzag manner in the direction perpendicular to the rail so that the contact part is not concentrated in one place, and this deviation amount is also controlled. ing.
It takes a lot of time and effort for the worker to manually perform the inspection and measurement of the trolley wire. Therefore, automatic trolley wire inspection devices and systems are in operation.

In a device as described above, an illumination that irradiates a wear area located below the trolley line and a line sensor that images the wear area illuminated by the illumination may be provided above the electric railway vehicle. .. The line sensor is provided so that the scanning line crosses the trolley line upward, and the images captured by the line sensor are arranged in chronological order to generate the line sensor image. After applying image processing to this line sensor image, the deviation and residual diameter of the trolley wire are calculated by using known data on the specifications of the trolley wire such as the height and thickness of the trolley wire.
For example, Patent Document 1 discloses the above-mentioned device.

Japanese Patent No. 5287177

In the above device, since the line sensor is provided facing upward, the sky is basically reflected in the background in the line sensor image, but in some cases, surrounding buildings, trees, etc. Objects other than the sky may be reflected.
In addition, the brightness value of the background fluctuates greatly depending on the time zone in which the trolley line is imaged. For example, in the daytime when it is sunny, the brightness value becomes high, but when it is cloudy, the brightness value becomes low, and in the nighttime or in a tunnel, the brightness value further decreases.

As described above, the brightness value of the background of the line sensor image and the degree of reflection of an object other than the sky greatly differ depending on the track section on which the device is executed and the weather.
In order to accurately extract the wear area, it is basically desirable to appropriately select an image processing algorithm and parameters suitable for the image to be processed in consideration of the influence of such a background.
However, it takes a lot of man-hours for the worker to check the image each time and select an image processing algorithm or parameter suitable for the image.
It is desired to inspect the trolley wire easily and accurately by reducing the influence of the imaging environment.

The problem to be solved by the present invention is that when the trolley line is inspected based on the image captured by the line sensor, the inspection can be easily performed and the influence of the imaging environment can be reduced and the trolley can be accurately executed. It is to provide a line inspection device and a trolley line inspection method.

The present invention employs the following means in order to solve the above problems. That is, the present invention is a trolley line inspection device comprising a line sensor that captures a trolley line and inspecting the trolley line based on an image captured by the line sensor as an input image. For each pixel in the line sensor image generator, the line sensor image, and the line sensor image, which generates a line sensor image by arranging the input images in chronological order, the pixel corresponds to the wear region of the trolley line. The line sensor image is input to the training model deeply trained by semantic segmentation and the training model using the correct answer value of whether or not to be training data, and the line sensor image is input to each pixel in the line sensor image. The residual diameter of the trolley wire is based on the wear region extraction unit that infers whether or not the pixel corresponds to the wear region and generates a wear region extract image that extracts the wear region, and the wear region extract image. Provided is a trolley line inspection device including a trolley line inspection unit that executes any or a combination of calculation, deviation calculation, and determination of excess wear.

Further, the present invention is a trolley line inspection method in which an image captured by a line sensor that captures a trolley line is used as an input image and the trolley line is inspected based on the input image, and the input image is time-series. A line sensor image is generated by arranging them side by side, and the line sensor image and the correct answer value for each pixel in the line sensor image as to whether or not the pixel corresponds to the wear region of the trolley wire are used as learning data and semantically. The line sensor image is input to the learning model deeply trained by segmentation, and for each pixel in the line sensor image, it is inferred whether or not the pixel corresponds to the wear region, and the wear region. A wear region extract image is generated, and based on the wear region extract image, any one or a combination of calculation of the residual diameter of the trolley wire, calculation of deviation, and determination of excess wear is executed. A trolley line inspection method is provided.

According to the present invention, when inspecting a trolley line based on an image captured by a line sensor, the trolley line inspection device can easily perform the inspection and can accurately execute the inspection by reducing the influence of the imaging environment. And a trolley line inspection method can be provided.

It is explanatory drawing which shows the traveling environment of the electric railroad vehicle equipped with the tram line inspection apparatus in embodiment of this invention. It is a block diagram of the said trolley line inspection apparatus. This is an example of a line sensor image generated based on an image captured by the line sensor of the trolley line inspection device. It is explanatory drawing of the teacher image used at the time of learning of the learning model of the said tram line inspection apparatus. It is a block diagram of the said learning model. It is explanatory drawing of the trolley line deviation calculation part of the said trolley line inspection apparatus. It is explanatory drawing of the trolley line residual diameter calculation part of the said trolley line inspection apparatus. It is a flowchart of the trolley line inspection method operated by using the said trolley line inspection apparatus. It is a block diagram of the trolley line inspection apparatus which concerns on the 1st modification of the said embodiment. It is explanatory drawing of the image after the binarization processing in the trolley line inspection apparatus which concerns on the said 1st modification. It is explanatory drawing of the state which the area was selected in the image after the binarization processing in the tram line inspection apparatus which concerns on the 1st modification. It is explanatory drawing of the state which applied the selected area to the line sensor image in the trolley line inspection apparatus which concerns on the 1st modification. It is explanatory drawing of the state which cut out the line sensor image in the selected area in the trolley line inspection apparatus which concerns on the 1st modification. It is a flowchart of the trolley line inspection method operated by using the trolley line inspection apparatus which concerns on the 1st modification. It is a block diagram of the trolley line inspection apparatus which concerns on the 2nd modification of the said embodiment. It is explanatory drawing of the image output by the learning model in the trolley line inspection apparatus which concerns on the said 2nd modification. It is a flowchart of the trolley line inspection method operated by using the trolley line inspection apparatus which concerns on the 2nd modification.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
FIG. 1 is an explanatory diagram showing a running environment of an inspection vehicle 1 that inspects a trolley line and is equipped with the trolley line inspection device 20 according to the present embodiment.
The inspection vehicle 1 travels on a pair of railroad tracks 2 laid on the ground. On the side of the track 2, a plurality of utility poles 3 are erected at intervals along the track 2. A beam 5 and a bending metal fitting 6 are provided on the utility pole 3. Above the beam 5, a power line 4 for supplying electric power to the inspection vehicle 1 extends along the extending direction R of the line 2 and is supported by the beam 5. On the lower side of the beam 5 and the bending metal fitting 6, the hanging wire 7 for suspending the trolley wire 9 described below extends along the extending direction R of the track 2 as well as the beam 5 and the bending wire 7. It is provided by being supported by the pull metal fitting 6. A plurality of hangers 8 are suspended from the suspension line 7 at a substantially constant interval in the extending direction R of the line 2.
The trolley wire 9 is suspended from the plurality of hangers 8 so as to have a constant tension, and is provided below the suspension wire 7 so as to be substantially parallel to the suspension wire 7. Electric power is supplied to the trolley wire 9 from the feeder wire 4 by a feeder branch line (not shown).
A current collector 1b such as a pantograph is provided on the upper surface 1a of the inspection vehicle 1. When the current collector 1b comes into contact with the trolley wire 9 while the inspection vehicle 1 is traveling, the trolley wire 9 to the inspection vehicle 1 Is supplied with power.

If the trolley wire 9 is provided parallel to the line 2 when viewed in a plan view, the portion of the current collector 1b in contact with the trolley wire 9 is limited, so that the limited portion of the current collector 1b is biased. It may be worn and damaged. In order to prevent this, the trolley wire 9 is provided so as to gently meander in a zigzag along the extending direction R of the track 2. In the trolley wire 9, the deviation caused by this meandering, which is the left-right deviation of the trolley wire 9 with respect to the center between the pair of lines 2, is strictly controlled.

Every time an electric railway vehicle such as the inspection vehicle 1 passes, the current collector 1b slides with respect to the lower surface of the trolley wire 9. Therefore, when the electric railway vehicle is continuously operated, the trolley wire 9 gradually wears and eventually breaks. Therefore, a wear limit is set for the trolley wire 9, and when the remaining diameter of the trolley wire 9 falls below the control value, the trolley wire 9 is replaced with a new one.
The trolley wire inspection device 20 in the present embodiment inspects the trolley wire 9 by measuring the residual diameter and deviation of the trolley wire 9.
FIG. 2 is a block diagram of the trolley line inspection device 20. As shown in FIGS. 1 and 2, the trolley line inspection device 20 includes an illumination 21, a line sensor 22, and a control device 23.

The illumination 21 is provided so as to irradiate the upper surface 1a of the inspection vehicle 1 with light upward. When the trolley wire 9 is imaged by the line sensor 22 described below, the wear region formed in a flat shape due to the wear of the trolley wire 9 due to the sliding of the current collector 1b under the trolley wire 9 is formed. By reflecting the light of the illumination 21, the image is taken so as to be displayed as a bright region.

The line sensor 22 is provided on the upper surface 1a of the inspection vehicle 1 like the lighting 21. The line sensor 22 is a camera that captures an image composed of one pixel in the vertical direction and a plurality of pixels in the horizontal direction, for example, 8192 pixels. In the present embodiment, since the line sensor 22 captures a grayscale image, each pixel has one luminance value.
The line sensor 22 is provided so as to be upward in the vertical direction and so that the direction 22b of the scanning line 22a is orthogonal to the extending direction R of the line 2 and is the same as the extending direction of the sleepers (not shown). As a result, the scanning line 22a of the line sensor 22 crosses the trolley line 9.
The line sensor 22 continuously images the trolley line 9 at short time intervals while the inspection vehicle 1 is traveling. The image captured by the line sensor 22 is transmitted to the control device 23 as an input image.

The control device 23 is an information processing device such as a personal computer. The control device 23 is mounted on the inspection vehicle 1.
The inspection vehicle 1 includes a line sensor image generation unit 30, a wear area extraction unit 31, and a trolley line inspection unit 32. The wear region extraction unit 31 includes a learning unit 35 and an interpolation processing unit 37. The learning unit 35 includes a learning model 40, which will be described later, and a learning data storage unit 36. The trolley line inspection unit 32 includes a trolley line deviation calculation unit 38 and a trolley line residual diameter calculation unit 39.
Of the components of the control device 23, other than the learning data storage unit 36, for example, software or a program executed by the CPU in each of the above-mentioned information processing devices may be used. Further, the learning data storage unit 36 may be realized by a storage device such as a semiconductor memory or a magnetic disk provided inside or outside each of the information processing devices.

The line sensor image generation unit 30 receives an input image from the line sensor 22, generates an aligned image in which 1000 lines are arranged in a time series in the vertical direction, and uses this as a line sensor image.
FIG. 3 is an explanatory diagram of the line sensor image 50. In the line sensor image 50, the horizontal direction X is the direction 22b of the scanning line 22a of the line sensor 22, and the vertical direction Y is the time series direction, that is, the direction in which the input images are arranged.

In the line sensor image 50 shown in FIG. 3, the trolley line 9 is imaged in the foreground. The trolley wire 9 has a substantially circular cross section in a new state, and the lower side thereof is scraped by the sliding of the current collector 1b to form a flat wear region 9a. Since the wear region 9a is irradiated with light by the illumination 21, the image is brightly captured. On both sides of the wear region 9a of the trolley wire 9, the surface of the trolley wire 9 that continues upward while curving outward from the widthwise end of the wear region 9a is imaged as a side surface region 9b. That is, the side surface region 9b is a non-sliding surface of the trolley wire 9 where the current collector 1b does not slide or slide. In the line sensor image 50 of FIG. 3, the portion where the trolley wire 9 is bent by the bending metal fitting 6 is imaged.
Behind the trolley wire 9, the suspension wire 7 is imaged. Further behind the suspension wire 7, a wire wire 4, a beam 5 supporting the electric wire 4, and a bending metal fitting 6 are imaged. The beam 5 and the bending metal fitting 6 are imaged thinner than they actually are due to the influence of the imaging interval of the line sensor 22.
The sky 10 is imaged as the background of the trolley wire 9, the suspension wire 7, the electric wire 4, the beam 5, and the bending metal fitting 6.

The line sensor image generation unit 30 generates a line sensor image 50 which is such an aligned image and transmits it to the wear region extraction unit 31.

The learning unit 35 of the wear region extraction unit 31 receives the line sensor image 50, infers whether or not the pixel corresponds to the wear region 9a for each pixel in the line sensor image 50, and determines whether or not the pixel corresponds to the wear region 9a. Is extracted. In order to effectively perform this inference, the learning unit 35 is provided with a machine learning device, and the machine learning device is deeply trained to generate a learning model 40. When the learning unit 35 actually performs the process of extracting the wear region 9a, the learning model 40 using the learning model 40 for which the learning is completed is used for each pixel in the input line sensor image 50. It is inferred whether or not corresponds to the wear region 9a.
That is, the machine learning device generates a learning model 40 in which appropriate learning parameters are learned, which is used as a program module that is a part of artificial intelligence software.
Hereinafter, for the sake of simplicity, both the machine learning device included in the learning unit 35 and the learning model generated by learning the machine learning device are referred to as the learning model 40 with the same reference numerals.

At the time before learning of the learning model 40, many line sensor images 50 are stored as learning data in the learning data storage unit 36.
Each line sensor image 50 stored in the learning data storage unit 36 is used as an input image in the deep learning of the learning model 40. In order for the learning model 40 to perform deep learning on the input line sensor image 50, teacher data corresponding to the line sensor image 50, which is the target of deep learning, is required.

The teacher data corresponding to each line sensor image 50 is a correct answer value for each pixel in the line sensor image 50 whether or not the pixel corresponds to the wear region 9a. More specifically, the correct answer value corresponds to either of the classification categories in which the object displayed corresponding to each pixel in the learning image 45 is roughly classified as the wear region 9a or other than that. It indicates whether to do it.
In particular, in the present embodiment, for each pixel, the wear area value which is a value indicating whether or not the pixel corresponds to the wear area 9a, and whether or not the pixel corresponds to something other than the wear area 9a. The combination of the two values of the background value, which is the value indicating, is the correct answer value. The wear area value and the background value are set to, for example, 1 when the pixel corresponds to each element in the classification category, and 0, for example, when the pixel does not correspond to each element. Therefore, for each pixel, a combination of a wear region value and a background value in which each element is 0 or 1 (wear region value, background value) is the correct answer value.
For example, in FIG. 3, the correct answer value of the pixel corresponding to the wear region 9a can be (1, 0). The correct answer value of the pixel corresponding to the area other than the wear region 9a can be (0, 1).

An image in which each pixel is colored with the color of the classification category to which the corresponding correct answer value corresponds when the unique color is associated with each classification category is hereinafter referred to as a teacher image. That is, the teacher image has the same resolution as the line sensor image 50, and each pixel has a color corresponding to the classification category in which the object displayed in the pixel corresponding to the pixel in the line sensor image 50 corresponds to the corresponding classification. doing.
FIG. 4 is an explanatory diagram of the teacher image 51. In the line sensor image 50, the region corresponding to the wear region 9a is white, and the background region 11 other than the wear region 9a is black, and each is colored.
The teacher image 51 is associated with the line sensor image 50 and is stored in the learning data storage unit 36 as learning data together with the line sensor image 50.

The learning model 40 of the learning unit 35 is deep-learned by the learning data as described above. FIG. 5 is a schematic block diagram of the learning model 40. In the present embodiment, the learning model 43 is realized by a full-layer convolutional network, and is trained by semantic segmentation to infer whether or not each pixel in the input image corresponds abnormally. ..
The learning model 40 includes a convolution processing unit 41 having a plurality of convolution layers 41a, 41b, 41c, and a deconvolution processing unit 42 having a plurality of deconvolution layers 42c, 42b, 42a.

The wear area extraction unit 31 acquires the line sensor image 50 from the learning data storage unit 36 and inputs it to the convolution layer 41a of the first stage.
The convolution layer 41a includes a predetermined number of filters (not shown). The learning model 40 executes a convolution filter process using these filters, and generates a feature map according to the number of filters.
The feature map generated in the convolution layer 41a becomes an input image of the convolution layer 41b in the next stage.
Similarly, the convolution filter processing is executed in the convolution layers 41b and 41c, and the feature map is generated. The feature map generated in the convolution layer 41c serves as an input to the deconvolution layer 42c of the deconvolution processing unit 42.

The deconvolution processing unit 42 has a structure symmetrical to that of the convolution processing unit 41. That is, the line sensor image 50 is compressed to a low dimension by the convolution processing unit 41, but the deconvolution processing unit 42 operates so as to expand and restore the compressed state from the low dimension. More specifically, the deconvolution layer 42c that executes the deconvolution process corresponding to the convolution layer 41c, the deconvolution layer 42b that executes the deconvolution process corresponding to the convolution layer 41b, and the deconvolution process corresponding to the convolution layer 41a are performed. Output data is generated by sequentially passing through the deconvolution layer 42a to be executed.

The learning model 40 outputs, as output data, the probability that each pixel in the line sensor image 50 can correspond to each of the classification categories in the same manner as the teacher data. For example, when the learning model 40 infers that the probability that the pixel corresponds to the wear region 9a is 0.8 and the probability that the pixel corresponds to other than the wear region 9a is 0.2 for a certain pixel, the pixel is The combination of values (0.8, 0.2) is output.
An image expressed in the same manner as the teacher image 51 shown in FIG. 4 based on the output data is hereinafter referred to as an inference result image 52. That is, the inference result image 52 is an image in which information about each of the inference results is reflected as a color at the position of the corresponding pixel, and as shown in FIG. 4, each pixel is the pixel. Among the plurality of probability values included in the output value corresponding to, the color corresponding to the classification category corresponding to the probability of the highest value is obtained.

As described above, the feature maps generated in each of the convolution layers 41a, 41b, and 41c are not easily affected by the displacement of the position of the object in the image and the difference in size. In other words, much of the information about the position of the object is lost in the feature map generated by the convolution processing unit 41.
Here, in each of the deconvolution layers 42c, 42b, 42a, the processing corresponding to the corresponding convolution layers 41c, 41b, 41a is executed as described above, and the processing data in the respective convolution layers 41c, 41b, 41a is used. By doing so, the position information lost in each of the convolution layers 41a, 41b, and 41c is complemented. As a result, the inference result image 52 output from the deconvolution processing unit 42 has the position information restored in pixel units.

In the learning model 40, machine learning is performed so that the inference result for each pixel in the inference result image 52 is close to the correct answer value of the teacher data corresponding to the input line sensor image 50. For this purpose, the learning unit 35 acquires the teacher image 51 corresponding to the input line sensor image 50 from the learning data storage unit 36, and obtains the correct answer value in the teacher image 51 and the value of the inference result image 52 in pixel units. By comparison, for example, the squared error of these differences between each pixel is calculated as a cost function.
Then, the learning model 40 is machine-learned by adjusting the weight value of each filter by the error back propagation method or the like so as to reduce the cost function.
As a result, for example, in FIG. 3, the correct answer value of the pixel corresponding to the wear region 9a on the teacher image 51 is (1, 0), so that it is the first output value of the output data corresponding to the pixel. The probability value corresponding to the wear region 9a of is closer to 1 than the others, and the other probability values can be a combination of values closer to 0 than the first probability value.
In this way, when some image is input as an input image, the learning model 40 that has completed learning outputs the probability that the pixel can correspond to each of the classification categories for each pixel in the input image.

When actually inspecting the trolley wire 9, the wear region extraction unit 31 inputs the line sensor image 50 generated by the line sensor image generation unit 30 to the learning model 40 machine-learned as described above. Then, the inference result image 52 is generated.
The interpolation processing unit 37 performs a closing process on the inference result image 52 by expanding and contracting the image in the vertical direction Y to connect the wear regions 9a represented by being divided close to the vertical direction Y. Run.
In this way, the wear region extraction unit 31 generates a wear region extraction image from which the wear region 9a is extracted based on the line sensor image 50, and transmits the wear region extraction unit 31 to the trolley line inspection unit 32.

The trolley line deviation calculation unit 38 of the trolley line inspection unit 32 receives the wear region extraction image, and calculates the deviation of the trolley line 9 based on the wear region extraction image. FIG. 6 is an explanatory diagram of the deviation calculation principle.
The trolley line deviation calculation unit 38 calculates the coordinates d (unit: pixel) of the center of gravity of the wear region 9a. The trolley line deviation calculation unit 38 has the coordinates of the center of gravity, the distance hs (mm) from the line 2 to the sensor surface 22c of the line sensor 22, and the focal distance f of the lens of the line sensor 22, which are preset parameters. (Mm), the length s (mm) of the sensor surface 22c of the line sensor 22, the number of pixels p (pixels) of the line sensor 22, and calculated by a measuring device other than the trolley line inspection device 20. Using the distance h (mm) from the line 2 to the trolley line 9, the deviation D (unit: mm) is calculated by the following equation.

The trolley line deviation calculation unit 38 calculates the deviation of the trolley line 9 as described above, and the trolley line 9 is located within the range of the deviation allowed in the actual place corresponding to the wear region extraction image. Determine if it is.
If the trolley wire 9 is not located within the allowable deviation range, it is judged to be abnormal and recorded.

The trolley wire residual diameter calculation unit 39 receives the wear region extraction image, and calculates the residual diameter of the trolley wire 9 based on the image. FIG. 7 is an explanatory diagram of the calculation principle of the remaining diameter.
In the present embodiment, the residual diameter of the trolley wire 9, that is, the height of the trolley wire 9 after wear is calculated as the residual diameter.
The trolley wire residual diameter calculation unit 39 calculates the wear surface width w (pixels) of the wear region 9a. The trolley wire residual diameter calculation unit 39 calculates the actual wear surface width W (mm) of the trolley wire 9 based on the center of gravity coordinates d (pixels) and the wear surface width w (pixels) of the wear region 9a. The residual diameter Dr (mm) of the trolley wire 9 is expressed by the following equation using the wear surface width W (mm) and the radius of the trolley wire 9.

The trolley wire residual diameter calculation unit 39 calculates the residual diameter of the trolley wire 9 as described above, and determines whether the residual diameter is less than the control value.
If the remaining diameter is less than the control value, it is judged to be abnormal and a record to that effect is recorded.

Next, a trolley line inspection method using the above-mentioned trolley line inspection device 20 will be described with reference to FIGS. 1 to 7 and FIG. FIG. 8 is a flowchart of the trolley line inspection method according to the present embodiment.

First, the line sensor image 50 and the correct answer value for each pixel in the line sensor image 50 as to whether or not the pixel corresponds to the wear region 9a of the trolley wire 9 are used as training data, and a learning model is performed by semantic segmentation. 40 is deep learning.
When actually inspecting the trolley wire 9, when the process is started (step S1), the line sensor image generation unit 30 receives an input image from the line sensor 22, and based on this, the line sensor image 50 Is generated and transmitted to the wear region extraction unit 31.
The wear region extraction unit 31 receives the line sensor image 50, inputs the line sensor image 50 to the learning model 40 for which learning has been completed, and for each pixel in the line sensor image 50, the pixel is in the wear region 9a. It is inferred whether or not it corresponds, and the inference result image 52 is generated (step S9).
The interpolation processing unit 37 executes a closing process on the inference result image 52 (step S11).
In this way, the wear region extraction unit 31 generates a wear region extraction image from which the wear region 9a is extracted based on the line sensor image 50, and transmits the wear region extraction unit 31 to the trolley line inspection unit 32.

The trolley line deviation calculation unit 38 of the trolley line inspection unit 32 receives the wear region extraction image, and calculates the deviation of the trolley line 9 based on the wear region extraction image (step S15).
Further, the trolley wire residual diameter calculation unit 39 of the trolley wire inspection unit 32 calculates the residual diameter of the trolley wire 9 based on the wear region extraction image (step S17).

Next, the effects of the trolley line inspection device 20 and the trolley line inspection method described above will be described.

The trolley wire inspection device 20 in the present embodiment includes a line sensor 22 that captures the trolley wire 9, and uses the image captured by the line sensor 22 as an input image to inspect the trolley wire 9 based on the line sensor 22. The inspection device 20, the line sensor image generation unit 30 that generates the line sensor image 50 by arranging the input images in chronological order, the line sensor image 50, and each pixel in the line sensor image 50. The line sensor image 50 is input to the learning model 40 deeply trained by semantic segmentation and the learning model 40 using the correct answer value of whether or not the pixel corresponds to the wear region 9a of the trolley wire 9 as training data. For each pixel in the line sensor image 50, a wear region extraction unit 31 that infers whether or not the pixel corresponds to the wear region 9a and generates a wear region extraction image that extracts the wear region 9a, and wear. A trolley line inspection unit 32 that calculates the remaining diameter of the trolley line 9 and the deviation is calculated based on the region extracted image.
Further, the trolley line inspection method in the present embodiment is a trolley line inspection method in which an image captured by the line sensor 22 that captures the trolley line 9 is used as an input image and the trolley line 9 is inspected based on the input image. Then, the input images are arranged in chronological order to generate the line sensor image 50, and for the line sensor image 50 and each pixel in the line sensor image 50, whether or not the pixel corresponds to the wear region 9a of the trolley wire 9. The line sensor image 50 is input to the learning model 40 deeply trained by semantic segmentation using the correct answer value as training data, and the pixels are set in the wear region 9a for each pixel in the line sensor image 50. It is inferred whether or not it corresponds, a wear region extraction image from which the wear region 9a is extracted is generated, and the calculation of the residual diameter of the trolley wire 9 and the calculation of the deviation are executed based on the wear region extraction image.
According to the above configuration and method, the learning model 40 determines whether or not the pixel corresponds to the wear region 9a of the trolley wire 9 for the line sensor image 50 and each pixel in the line sensor image 50. Deep learning is performed by semantic segmentation using the correct answer value as training data. Using this learning model 40, it is inferred for each pixel in the line sensor image 50 whether or not the pixel corresponds to the wear region 9a.
That is, since it is inferred for each pixel of the line sensor image 50 whether or not it corresponds to the wear region 9a on a pixel-by-pixel basis, the wear region extraction unit 31 can output accurate information on the wear region 9a. ..
Further, since the learning model 40 is trained to grasp the characteristics of the wear region 9a by semantic segmentation, the influence of the imaging environment such as the weather and time imaged by the line sensor 22 is affected when the wear region 9a is extracted. Hard to receive.
Therefore, the inspection of the trolley wire 9 and the calculation of the residual diameter and the deviation in the present embodiment can be performed accurately while reducing the influence of the imaging environment.
Further, once the learning model 40 is properly learned, it can be used without the need for re-learning or setting special parameters. In particular, as described above, the extraction of the wear region 9a by the learning model 40 is not easily affected by the imaging environment, so it is not necessary to change the processing content according to the imaging environment. Therefore, the operation of the trolley wire inspection device 20 is easy, and the trolley wire 9 can be easily inspected.

[First Modified Example of Embodiment]
Next, a first modification of the trolley line inspection device 20 and the trolley line inspection method shown as the above-described embodiment will be described with reference to FIGS. 9 to 14. FIG. 9 is a block diagram of a trolley line inspection device for this modified example. FIG. 10 is an explanatory diagram of an image after binarization processing in the trolley line inspection device for this modified example. FIG. 11 is an explanatory diagram of a state in which a region is selected in the image after the binarization process in the trolley line inspection device according to the present modification. FIG. 12 is an explanatory diagram of a state in which the selected region is applied to the line sensor image in the trolley line inspection device according to the present modification. FIG. 13 is an explanatory diagram of a state in which a line sensor image is cut out in a selected region in the trolley line inspection device according to the present modification. FIG. 14 is a flowchart of a trolley line inspection method operated by using the trolley line inspection device according to the present modification. The trolley line inspection device 20A in this modification is different from the trolley line inspection device 20 of the above embodiment in that the line sensor image generation unit 30A has a line sensor image extraction unit 60, a binarization unit 61, and an image cutout unit. The difference is that it has 62.
In the line sensor image generation unit 30A of this modification, the wear region extraction unit 31 uses an aligned image in which the input images from the line sensor 22 are arranged in chronological order, which is shown with reference to FIG. 3 in the above embodiment, as a line sensor image. Do not send to. The line sensor image generation unit 30A generates a line sensor image by applying further processing to the aligned image by the streak extraction unit 60, the binarization unit 61, and the image cropping unit 62.

The linear object extraction unit 60 applies a GSTH (grayscale top hat) process to the aligned image shown in FIG. 3, for example, by contracting and expanding the image to open it, and then subtracting the image that is the source of the expansion and contraction. By doing so, the streaks extending in the lateral direction X are extracted.
After that, the binarization unit 61 determines the binarization threshold according to the image so that the variance ratio of the intraclass variance and the interclass variance for each of the linear object and the other region is maximized. The binarization process is performed by the discriminant analysis binarization process. FIG. 10 shows the binarized image 65, which is the image after the binarization. In this modification, the binarized image 65 is generated so that the streaks are white and the areas other than the streaks are black.

The image cutout unit 62 sets the binarized image 65 so that the region R1 surrounding the white region has a sufficient distance between the white region and the outer edge of the region R1 in the lateral direction X in particular. do. Then, as shown in FIG. 12, the image cutout unit 62 provides the aligned image with a region R1 set for the binarized image 65 in the same portion as the binarized image 65, and provides the aligned image. By cutting out the portion in the region R1 of the above, the line sensor image 66 as shown in FIG. 13 is generated.
In this modification, the image in which the portion corresponding to the wear region 9a is cut out and the resolution in the lateral direction X is reduced is used as the line sensor image 66 for learning the learning model 40 and inspecting the trolley line 9. The process is executed.

In the trolley line inspection method operated by using the trolley line inspection device 20A of this modification, as shown in FIG. 14, a process of extracting streaks (step S3) and a binarization process (step S3). Step S5) and the process of cutting out the line sensor image 66 from the aligned image (step S7) are added to the trolley line inspection method of the above embodiment shown in FIG.

In the trolley line inspection device 20A of this modification, the line sensor image generation unit 30A extends from the aligned image in which the input images are arranged in time series to the direction Y in which the input images are arranged in time series. A strip extraction unit 60 for extracting stripes, and an image cropping section 62 for cutting out an image from the aligned image so as to include the extracted stripes and outputting the image as a line sensor image 66. , Equipped with.
In the configuration of the above embodiment, the width of the wear region 9a tends to be smaller than the width of the line sensor image 50 in the lateral direction X. In particular, in the learning model 40, the convolution process is performed, but such a small width portion may be crushed and disappear by the convolution process. Therefore, it may be difficult to improve the learning accuracy of the learning model 40 and the inference accuracy of the wear region 9a.
On the other hand, in this modification, by cutting out the vicinity of the trolley wire 9, the ratio of the width of the wear region 9a to the width in the lateral direction X becomes large. By using such a cut-out image for learning of the learning model 40 and inference by the learning model 40, the learning accuracy of the learning model 40 and the inference accuracy of the wear region 9a are improved.
Needless to say, this modification has other effects similar to those of the embodiments already described.

[Second variant of the embodiment]
Next, a second modification of the trolley line inspection device 20 and the trolley line inspection method shown as the above-described embodiment will be described with reference to FIGS. 15 to 17. FIG. 15 is a block diagram of a trolley line inspection device for this modified example. FIG. 16 is an explanatory diagram of an image output by the learning model in the trolley line inspection device for this modified example. FIG. 17 is a flowchart of a trolley line inspection method operated by using the trolley line inspection device according to the present modification. The trolley line inspection device 20B in this modification is a further modification of the trolley line inspection device 20A of the first modification. The difference is that in addition to the calculation of the residual diameter and deviation of the line 9, the determination of excess wear is performed.

In this modification, in the trolley wire inspection device 20B, the wear of the trolley wire 9 progresses, and the height of the wear region 9a is shown as H1 at a position above the center of the trolley wire 9, for example, FIG. Excessive wear and excessive wear are determined until the position is reached.
When the height of the wear region 9a of the trolley wire 9 does not exceed the center of the trolley wire 9, the line sensor image is along the wear region 9a on both sides of the wear region 9a, for example, as shown in FIG. The surface of the trolley wire 9 that continues upward while curving outward from the widthwise end of the wear region 9a is imaged as the side surface region 9b. On the other hand, if the height of the wear region 9a of the trolley wire 9 exceeds the center of the trolley wire 9 and is excessively worn, the line sensor 22 images the trolley wire 9 from below, so that the line sensor image is worn. The side area 9b is not imaged on both sides of the area 9a.
The trolley wire inspection device 20B determines the presence or absence of the side surface region 9b, and determines whether or not the trolley wire 9 is excessively worn.

In order to correspond to this, in the learning model 40B in this modification, the correct answer value applied at the time of learning is that the pixels of the line sensor image corresponding to the correct answer value are the wear region 9a, the side surface region 9b, and the other overhead wire region 12. , And which of the classification categories including the background area 11 is applicable. As a result, the learning model 40B learns to output the inference result image 52B inferring which of the wear region 9a, the side surface region 9b, the other overhead wire region 12, and the background region 11 each pixel corresponds to. Has been done.

The interpolation processing unit 37 executes a closing process on the inference result image 52B to generate a wear region extraction image.
The trolley wire inspection unit 32B of this modified example includes a trolley wire wear progress determination unit 70. The trolley wire wear progress determination unit 70 receives the wear region extraction image, determines whether or not there is a side surface region 9b around the wear region 9a, and if not, the trolley wire 9 is excessively worn. It is determined that it is.

In the trolley wire inspection method operated by using the trolley wire inspection device 20B of the present modification, as shown in FIG. 17, the process of determining the excessive wear of the trolley wire 9 (step S13) is shown in FIG. It is added to the trolley line inspection method of the first modification shown in 14.

In the trolley wire inspection device 20B of this modification, the correct answer values are the wear region 9a, the side surface region 9b of the trolley wire 9 adjacent to the wear region 9a, and the wear of the pixels of the line sensor image corresponding to the correct answer value. It indicates which of the classification categories including the region 9a and the other regions 11 and 12 other than the side region 9b corresponds to, and the wear region extraction unit 31B inputs a line sensor image to the learning model 40B and inputs the line sensor image. For each pixel in the line sensor image, it is inferred whether or not the pixel corresponds to the wear region 9a, the side surface region 9b, and the other regions 11 and 12, and the trolley line inspection unit 32B determines whether or not the pixel corresponds to the wear region 9a, the side surface region 9b, and the other regions 11 and 12. Whether or not the trolley wire 9 is excessively worn is determined based on the presence or absence of the side surface region 9b.
The width of the side surface region 9b tends to be very small, and it is difficult to accurately determine the presence or absence of the side surface region 9b in normal image processing.
On the other hand, in this modification, the learning model 40B infers the region of the side region 9b by semantic segmentation, and the trolley line inspection unit 32B determines the excessive wear of the trolley wire 9 based on this result. The side region 9b can be well extracted. Therefore, it is possible to accurately determine excess wear.
Needless to say, this modification has other effects similar to those of the embodiments already described.

The trolley line inspection device and the trolley line inspection method of the present invention are not limited to the above-described embodiment and each modification described with reference to the drawings, and various other trolley line inspection devices are provided within the technical scope thereof. A modified example is conceivable.

For example, in the above embodiment and the first modification, the trolley wire inspection device includes both the trolley wire deviation calculation unit 38 and the trolley wire residual diameter measurement unit 39, and both the residual diameter and the deviation of the trolley wire 9 are provided. Was configured to calculate. Further, in the second modification, the trolley wire inspection device includes a trolley wire deviation calculation unit 38, a trolley wire residual diameter measurement unit 39, and a trolley wire wear progress determination unit 70, and the trolley wire 9 has a trolley wire wear progress determination unit 70. The calculation of the residual diameter, the calculation of the deviation, and the determination of excess wear were performed.
The configuration of the trolley line inspection device is not limited to this. For example, the trolley wire inspection device includes only one of the trolley wire deviation calculation unit 38, the trolley wire residual diameter calculation unit 39, and the trolley wire wear progress determination unit 70, and has the residual diameter of the trolley wire 9. Only one of the calculation, the deviation calculation, and the determination of excess wear may be performed. Alternatively, the trolley wire inspection device may include a trolley wire deviation calculation unit 38 and a trolley wire wear progress determination unit 70, and may be configured to calculate the deviation and determine excess wear. Alternatively, the trolley wire inspection device may include a trolley wire residual diameter calculation unit 39 and a trolley wire wear progress determination unit 70, and may be configured to calculate the residual diameter and determine excess wear.

Further, in the above embodiment and the first modification, the classification division is provided corresponding to the wear region 9a and the background region 11, and in the wear region extraction unit 31, each pixel of the line sensor image is classified into these two classification divisions. The wear region 9a was extracted by inferring which of the above was applicable by semantic segmentation.
Similarly, in the second modification, the classification division is provided corresponding to the wear region 9a, the side surface region 9b, and the other regions 11 and 12, and the wear region extraction unit 31B has each pixel of the line sensor image. The wear region 9a was extracted by inferring which of these plurality of classifications it corresponds to by semantic segmentation.
The setting of the classification category is not limited to the above. For example, the classification classification may be provided according to the type of the object to be imaged in the input image.
For example, the classification category corresponds to each of the types of objects shown in FIG. 3, such as wear area 9a, side area 9b, wire 4, beam 5, curved metal fitting 6, overhead wire 7, and other areas. It may be attached. Alternatively, in addition to these, the classification classification is assigned so as to correspond to each of objects that may be imaged as a background, such as building structures, tunnels, steel towers, trees, and the sky, which are not shown in FIG. It may be further subdivided.
In the above embodiment and each modification, it has been confirmed that the wear region 9a is extracted with sufficient accuracy. However, as described above, by further subdividing the classification classification, each of the objects other than the wear region 9a is classified into the corresponding classification classification, so that the other objects are erroneously classified into the wear region 9a. As a result, the extraction accuracy of the wear region 9a is further improved.
Alternatively, even if the classification classification is not subdivided in this way, the classification classification may be provided only for an object that is likely to be mistaken for, for example, the wear region 9a. For example, when the wear region 9a, the electric wire 4 and the suspension wire 7 have the same characteristic that they are strips extending in the vertical direction Y of the line sensor image, so that the wire 4 and the suspension wire 7 are erroneously worn. It may be extracted as region 9a. In such a case, a plurality of classification categories may be provided corresponding to each of the wear region 9a, the wire 4, the suspension wire 7, and other objects.
As described above, in effectively extracting the wear region 9a, a plurality of classification categories may be provided in any manner depending on the situation.
In the above-described form, the correct answer value corresponds to which of the classification categories provided according to the type of the object imaged in the input image, the pixels of the line sensor image corresponding to the correct answer value. The classification classification includes the wear region 9a, and the wear region extraction unit inputs a line sensor image to the learning model, and for each pixel in the line sensor image, which classification the pixel is. Infer whether it corresponds to the classification.
According to such a configuration, the extraction accuracy of the wear region 9a can be improved.

In addition to this, as long as the gist of the present invention is not deviated, the configurations given in the above-described embodiment and each modification can be selected or changed to other configurations as appropriate.

1 Inspection vehicle 9 Tram wire 9a Wear area 9b Side area 11 Background area (other area)
12 Other overhead line areas (other areas)
20, 20A, 20B Trolley line inspection device 21 Lighting 22 Line sensor 23 Control device 30, 30A Line sensor Image generation unit 31, 31B Wear area extraction unit 32, 32B Trolley line inspection unit 35 Learning unit 36 Learning data storage unit 37 Interpolation processing unit 38 Trolley line deviation calculation unit 39 Trolley line residual diameter calculation unit 40, 40B Learning model 50, 66 Line sensor image 60 Line strip extraction unit 61 Binarization unit 62 Image cutout unit 70 Trolley wire wear progress determination Department

Claims (5)

It is a trolley line inspection device provided with a line sensor that captures a trolley line and inspects the trolley line based on an image captured by the line sensor as an input image.
A line sensor image generator that generates a line sensor image by arranging the input images in chronological order,
A learning model deeply trained by semantic segmentation using the line sensor image and the correct answer value of whether or not the pixel corresponds to the wear region of the trolley wire for each pixel in the line sensor image as training data. ,
The line sensor image is input to the learning model, and for each pixel in the line sensor image, it is inferred whether or not the pixel corresponds to the wear region, and the wear region is extracted from the wear region. A wear area extractor that generates an extracted image,
A trolley wire inspection unit that executes any or a combination of calculation of the remaining diameter of the trolley wire, calculation of deviation, and determination of excess wear based on the wear region extraction image.
A trolley line inspection device equipped with. The line sensor image generation unit
A streak extraction unit that extracts streaks extending in the direction in which the input images are arranged in time series from an aligned image in which the input images are arranged in time series.
An image cutout portion that cuts out an image from the aligned image so as to include the extracted streaks and outputs the image as the line sensor image.
The trolley line inspection device according to claim 1. The correct answer value is such that the pixels of the line sensor image corresponding to the correct answer value cover the wear region, the side region of the trolley wire adjacent to the wear region, and the wear region and other regions other than the side region. It indicates which of the included classifications it corresponds to.
The wear region extraction unit inputs the line sensor image to the learning model, and for each pixel in the line sensor image, the pixel is the wear region, the side surface region, and the other region. Infer which one it corresponds to,
The trolley wire inspection device according to claim 1 or 2, wherein the trolley wire inspection unit determines whether or not the trolley wire is excessively worn based on the presence or absence of the side surface region. The correct answer value indicates which of the classification categories provided according to the type of the object imaged in the input image corresponds to the pixel of the line sensor image corresponding to the correct answer value. The classification category includes the wear area.
The claim that the wear area extraction unit inputs the line sensor image to the learning model, and infers which of the classification categories the pixel corresponds to for each pixel in the line sensor image. The trolley line inspection device according to 1 or 2. This is a trolley line inspection method in which an image captured by a line sensor that captures a trolley line is used as an input image and the trolley line is inspected based on the input image.
A line sensor image is generated by arranging the input images in chronological order.
The line sensor image and the correct answer value of whether or not the pixel corresponds to the wear region of the trolley wire for each pixel in the line sensor image are used as training data in a learning model deeply trained by semantic segmentation. On the other hand, the line sensor image is input, and for each pixel in the line sensor image, it is inferred whether or not the pixel corresponds to the wear region, and a wear region extraction image obtained by extracting the wear region is generated. death,
A trolley line inspection method for executing any or a combination of calculation of the remaining diameter of the trolley wire, calculation of deviation, and determination of excess wear based on the wear region extracted image.

Images