JP7210142B2 - DATA PROCESSING DEVICE, IMAGE ANALYSIS METHOD, AND PROGRAM - Google Patents

Description

The present invention relates to a data processing device, an image analysis method, and a program, and more particularly to a data processing device, an image analysis method, and a program for diagnosing the state of deterioration inside steel pipes of a steel pipe tower.

As a method for diagnosing the state of deterioration inside steel pipes in a steel pipe tower, an industrial endoscope is inserted into the steel pipe to take an image of the inside of the steel pipe, and the photographed image is analyzed to determine the presence or absence of corrosion inside the steel pipe. It is known how to

Conventionally, when diagnosing the state of deterioration inside a steel pipe, the presence or absence of corrosion inside the steel pipe was determined by visually checking the images taken with an industrial endoscope. A corrosion analysis apparatus has been developed that determines the presence or absence of corrosion inside a steel pipe by analyzing an image taken with a scope by image processing of a computer (see, for example, Patent Document 1).

JP 2012-13675 A

In general, an image taken when diagnosing the state of deterioration inside a steel pipe often includes not only the inside of the steel pipe but also the outside of the steel pipe. For example, when inspecting a desired steel pipe of a steel tower, first, in order to identify which part of the steel pipe the steel pipe to be inspected belongs to, the identification number given to the steel pipe to be inspected is given to the steel pipe to be inspected. Shoot in a mirror. For example, a whiteboard on which the identification number of the steel pipe to be inspected is written, the outer peripheral surface of the steel pipe on which the identification number is written, or the like is photographed. After that, the industrial endoscope is inserted into the steel pipe and moved inside the steel pipe while the imaging is continued. Once the interior of the steel pipe has been photographed, the industrial endoscope is removed from the interior of the steel pipe to complete the photographing. When the industrial endoscope is taken out from the inside of the steel pipe, the scenery outside the steel pipe such as the sky, the ground, and the appearance of the steel tower is photographed.

A series of moving images taken by such a procedure includes images other than the inside of the steel pipe as described above. If such a video is input as is to a conventional corrosion analysis device, analysis processing will be performed to determine the presence or absence of corrosion on images other than the inside of the steel pipe, and it will be erroneously determined that there is corrosion in places other than the inside of the steel pipe. is likely to occur.

Therefore, when diagnosing the state of deterioration inside a steel pipe using a corrosion analysis device, an operator visually checks a series of videos taken with an industrial endoscope in advance, and then photographs the inside of the steel pipe in the video. After specifying the starting point and the shooting end point inside the steel pipe, it is necessary to cause the corrosion analysis device to perform the analysis processing. In general, there are dozens to hundreds of steel pipes to be inspected in a steel pipe tower, so if you visually check all the steel pipes to be inspected, it would take a lot of time. Become.

As described above, even when the diagnosis work is performed using the corrosion analysis device, the operator needs to check the moving image, so there is a problem that sufficient improvement in work efficiency cannot be expected.

The present invention has been made in view of the problems described above, and an object of the present invention is to improve work efficiency when diagnosing the state of deterioration inside a steel pipe.

A data processing apparatus according to a representative embodiment of the present invention includes an image data acquisition unit that acquires image data including a plurality of time-series still images; and a determination unit that determines whether the still image is an image of the interior of a steel pipe, based on the variation in brightness of the still image extracted by the image extraction unit. It is characterized by

According to the data processing device of the present invention, it is possible to improve the work efficiency when diagnosing the state of deterioration inside the steel pipe.

1 is a diagram showing a functional block configuration of a data processing device according to one embodiment of the present invention; FIG. It is a figure which shows the hardware constitutions of a data processor. FIG. 4 is a diagram for explaining a method of extracting a still image from image data; It is a figure which shows the functional block structure of a determination part. It is a figure which shows an example of image division by a block division part. 1 is a flow chart showing the overall flow of an image analysis method for diagnosing the state of deterioration inside a steel pipe according to one embodiment of the present invention. 4 is a flow chart showing the flow of steel pipe interior determination processing (step S2). FIG. 4 is a diagram showing temporal changes in luminance dispersion of a moving image of the inside of a steel pipe. FIG. 9 is a diagram showing an image of one frame at time t1 of the moving image shown in FIG. 8; FIG. 9 is a diagram showing an image of one frame at time t3 of the moving image shown in FIG. 8; FIG. 9 is a diagram showing an image of one frame at time t4 of the moving image shown in FIG. 8; FIG. 9 is a diagram showing an image of one frame at time t6 of the moving image shown in FIG. 8;

1. Outline of Embodiment First, an outline of a representative embodiment of the invention disclosed in the present application will be described. In the following description, as an example, reference numerals on the drawings corresponding to constituent elements of the invention are described with parentheses.

[1] A data processing device (100) according to a representative embodiment of the present invention includes an image data acquisition unit (1) that acquires image data (40) including a plurality of time-series still images (400); an image extraction unit (2) for extracting the still image from the image data acquired by the image data acquisition unit; and a determination unit (3) for determining whether or not the image is a captured image.

[2] In the above data processing device, the determination unit includes a block dividing unit (31) for dividing the region of the still image extracted by the image extracting unit into a plurality of blocks (401), and a luminance value for each block. A variance calculation unit (32) for calculating variance, and based on the brightness variance for each block calculated by the variance calculation unit, whether the still image extracted by the image extraction unit is an image of the inside of the steel pipe. and an image determination unit (33) for determining whether or not.

[3] In the above data processing device, the variance calculator includes a first brightness variance calculator (321) for calculating a first brightness variance value (43) representing the variance of the brightness of the block; a second brightness variance calculator (322) for calculating a second brightness variance value (44) representing the variance of one brightness variance value, wherein the image determination unit calculates the first brightness variance value of at least one of the blocks; is greater than the first threshold value (41), the still image extracted by the image extraction unit is determined to be an image of an area other than the inside of the steel pipe, and the first luminance variance value of all the blocks is the a first steel pipe interior determination unit (331) that determines that the still image extracted by the image extraction unit is an image of the inside of the steel pipe taken when the image is smaller than the first threshold, and the first steel pipe interior determination unit If the second brightness variance value of at least one of the blocks in the still image determined to be the image of the interior of the steel pipe is greater than a second threshold value (42), the still image is determined to be an image other than the interior of the steel pipe. , if the second luminance variance value of all the blocks in the still image determined to be an image inside the steel pipe by the first steel pipe interior determining unit is smaller than the second threshold, the still image is captured inside the steel pipe; and a second steel pipe interior determination unit (332) that determines that the image is an image that has been drawn.

[4] In the above data processing device, the determination unit determines a shooting start point inside the steel pipe in the image data based on the determination result of the image determination unit for the plurality of continuous still images extracted by the image extraction unit. and an imaging range detection unit (34) for detecting an imaging end point inside the steel pipe.

[5] In the above data processing device, the photographing range detection unit detects at least One image is the image of the shooting start point inside the steel pipe, and if each of the plurality of continuous still images is determined to be an image outside the steel pipe after the determination of the shooting start point, the plurality of continuous still images The still image captured before the still image may be used as the image at the end point of capturing inside the steel pipe.

[6] A program (1021) according to a representative embodiment of the present invention includes an image data acquisition step (S1) for acquiring image data including a plurality of time-series still images, and and an image extraction step (S2, S21) for extracting the still image from the image data obtained by the image extraction, and based on the luminance variation of the still image extracted by the image extraction step, the still image is an image of the inside of the steel pipe. and a determination step (S2, S22 to S30) for determining whether or not.

[7] In the above image analysis method, the determination step includes a block division step (S23) of dividing the region of the still image extracted by the image extraction step into a plurality of blocks (401); The still image extracted by the image extraction step is an image of the inside of the steel pipe, based on the variance calculation step (S24, S28) for calculating the variance, and the brightness variance for each block calculated by the variance calculation step. and an image determination step (S25, S29) for determining whether or not there is.

[8] In the above image analysis method, the variance calculation step includes a first brightness variance calculation step (S24) of calculating a first brightness variance value (43) representing the variance of brightness of the block; and a second brightness variance calculation step (S28) of calculating a second brightness variance value (44) representing the variance of one brightness variance value, wherein the image determination step is performed by calculating the first brightness variance of at least one of the blocks. If the value is greater than the first threshold value (41), it is determined that the still image extracted by the image extraction step is an image of an area other than the inside of the steel pipe, and the first luminance variance value of all the blocks is a first steel pipe interior determination step (S25, S27, S26) for determining that the still image extracted by the image extraction step is an image of the interior of the steel pipe when the image is smaller than the first threshold; When the second brightness variance value of at least one of the blocks in the still image determined to be the image inside the steel pipe by the steel pipe interior determination step is larger than a second threshold value (42), the still image is an image other than the inside of the steel pipe. and when the second brightness variance values of all the blocks in the still image determined to be an image inside the steel pipe by the first steel pipe interior determination step are smaller than the second threshold, the still image and a second steel pipe interior determination step (S29, S30, S26) for determining that the is an image of the interior of the steel pipe.

[9] In the above image analysis method, the determination step determines the imaging start point inside the steel pipe in the image data based on the determination result of the image determination step for the plurality of continuous still images extracted by the image extraction step. and an imaging range detection step (S31 to S36) for detecting an imaging end point inside the steel pipe.

[10] In the above image analysis method, the imaging range detection step includes, when each of the plurality of continuous still images is determined to be an image inside a steel pipe, at least one of the plurality of continuous still images After the step (S31, S32) of setting one image as the image of the imaging start point inside the steel pipe and determining the imaging start point, it is determined that each of the plurality of continuous still images is an image of the exterior of the steel pipe. and setting the still image taken before the plurality of continuous still images as the image of the shooting end point inside the steel pipe (S34, S35, S36).

[11] A program (1021) according to a representative embodiment of the present invention is characterized by causing a computer to execute each step (S1 to S3, S21 to S36) in the above image analysis method.

2. Specific Examples of Embodiments Specific examples of embodiments of the present invention will be described below with reference to the drawings. In the following description, constituent elements common to each embodiment are denoted by the same reference numerals, and repeated descriptions are omitted.

<Configuration of data processor>
FIG. 1 is a diagram showing a functional block configuration of a data processing device according to one embodiment of the present invention.
A data processing device 100 shown in the figure is a diagnostic device for diagnosing the state of deterioration inside each steel pipe that constitutes, for example, a steel pipe tower. Specifically, the data processing device 100 determines whether or not there is corrosion inside the steel pipe by analyzing an image of the inside of the steel pipe captured by an industrial endoscope.

The data processing device 100, for example, before executing image analysis processing for determining the presence or absence of corrosion inside the steel pipe, determines whether or not there is an image inside the steel pipe from the image data to be diagnosed, and determines whether there is an image of the inside of the steel pipe. and the imaging end point inside the steel pipe are automatically detected.

As shown in FIG. 1, the data processing device 100 includes an image data acquiring unit 1, an image extracting unit 2, a judging unit 3, an image data acquiring unit 1, an image extracting unit 2, a determining unit 3, A storage unit 4 and a corrosion determination processing unit 5 are provided.

The data processing device 100 is, for example, an information processing device such as a personal computer (PC), a server, or a tablet terminal. The image data acquisition unit 1, the image extraction unit 2, the determination unit 3, the storage unit 4, and the corrosion determination processing unit 5 are configured by a processor such as a CPU in the information processing apparatus and a storage device such as a RAM and a ROM in the information processing apparatus. It is realized by executing various operations according to a program stored in the memory and controlling peripheral circuits.

The above program is a program for realizing the image analysis method for diagnosing the state of deterioration inside the steel pipe according to the present embodiment, and is installed in advance in the storage device within the data processing device 100, for example.

The program may be distributed via a network, or written on a computer-readable storage medium such as a CD-ROM (non-transitory computer readable medium) and distributed. .

FIG. 2 is a diagram showing the hardware configuration of the data processing device 100. As shown in FIG.
As shown in FIG. 2, the data processing device 100 includes an arithmetic device 101, a storage device 102, an input device 103, an I/F (Interface) device 104, an output device 105, and a bus 106 as main hardware components. ing.

The arithmetic unit 101 is configured by a processor such as a CPU (Central Processing Unit) or a DSP (Digital Signal Processor). The storage device 102 has a storage area for storing a program 1021 for causing the arithmetic device 101 to execute various data processing, and data 1022 such as parameters and calculation results used in the data processing by the arithmetic device 101. For example, it includes ROM (Read Only Memory), RAM (Random Access Memory), HDD, and flash memory.

A program for realizing the image analysis method for diagnosing the state of deterioration inside the steel pipe according to the present embodiment is stored in storage device 102 as program 1021, for example.

The input device 103 is a functional unit that detects input of information from the outside, and includes, for example, a keyboard, a mouse, a pointing device, buttons, a touch panel, or the like. The I/F device 104 is a functional unit that transmits and receives information to and from the outside, and includes a communication control circuit, an input/output port, an antenna, and the like for performing wired or wireless communication.

The output device 105 is a functional unit that outputs information and the like obtained by data processing by the arithmetic device 101 . Examples of the output device 105 include external storage devices such as SSD and HDD, and display devices such as LCD (Liquid Crystal Display) and organic EL (Electro Luminescence). Bus 106 is a functional unit that interconnects arithmetic device 101, storage device 102, input device 103, I/F device 104, and output device 105 and enables data transfer between these devices.

The data processing device 100 executes calculations according to the program 1021 stored in the storage device 102 by the arithmetic device 101, and uses the storage device 102, the input device 103, the I/F (Interface) device 104, the output device 105, and the bus 106. By controlling, various functional units shown in FIG. 1, that is, the image data acquisition unit 1, the image extraction unit 2, the determination unit 3, the storage unit 4, and the corrosion determination processing unit 5 are realized.

Note that the data processing apparatus 100 may be implemented by a single computer as shown in FIG. 2, or may be implemented by a plurality of computers that are communicatively connected to each other by wire or wirelessly, such as a client-server system. The data processing device 100 is not limited to the hardware configuration shown in FIG.

Each functional unit constituting the data processing apparatus 100 will be described below.

The image data acquisition unit 1 is a functional unit that acquires image data 40 . For example, the image data acquisition unit 1 acquires the image data 40 of the steel pipe to be diagnosed for deterioration from the storage unit 4, which will be described later.

Here, the image data 40 is data including a plurality of time-series still image data, such as moving image data.

The image data acquisition unit 1 is, for example, a flash memory such as a USB memory, an external storage medium such as a CD-ROM, or an external information processing device connected to the data processing device 100 via a wireless or wired communication network. The image data 40 may be acquired from (for example, a server, a PC, etc.).

The image extraction unit 2 is a functional unit that extracts still images from the image data 40 acquired by the image data acquisition unit 1 . The image extraction unit 2 sequentially extracts still images of one frame from a moving image as image data 40, for example, at predetermined time intervals.

FIG. 3 is a diagram for explaining a method of extracting a still image from the image data 40. FIG.
The image extraction unit 2 extracts one frame of still images 400_1 to 400_k every j frames from the image data 40 having a predetermined frame rate. Here, j is an integer of 2 or more, for example, j is an integer in the range of 10-50. Also, k is an integer of 1 or more.

For example, the image extraction unit 2 sequentially extracts still images 400_1 to 400_k of one frame every 30 (j=30) frames from a moving image (image data 40) of 30 fps (frames per second).

The determination unit 3 is a functional unit that determines an image of the inside of the steel pipe and an image of the outside of the steel pipe (outside the steel pipe) in the image data 40 . Specifically, the determining unit 3 determines whether or not the image extracted by the image extracting unit 2 is an image of the inside of the steel pipe, based on the luminance variation of the image. Further, the determination unit 3 detects the imaging start point inside the steel pipe and the imaging end point inside the steel pipe from the image data 40 . Details of the determination unit 3 will be described later.

The storage unit 4 is a functional unit for storing various programs, parameters, etc. for diagnosing the state of deterioration inside the steel pipe. For example, the storage unit 4 stores image data 40 acquired from a storage medium such as a U flash memory or a CD-ROM, or from another information processing device. The storage unit 4 also stores a first threshold value 41 and a second threshold value 42 as parameters necessary for determination processing by the determination unit 3, which will be described later, and a first brightness variance value 43 and a second brightness variance value 44, which will be described later. Calculation results and the like by the unit 3 are stored.

The corrosion determination processing unit 5 is a functional unit that performs corrosion determination processing for determining the presence or absence of corrosion inside the steel pipe based on the image data 40 acquired by the image data acquisition unit 1 . For example, the corrosion determination processing unit 5 detects corrosion inside the steel pipe based on the image from the imaging start point inside the steel pipe to the imaging end point inside the steel pipe detected by the determination unit 3 in the image data 40 (moving image data). Determine the presence or absence of As a method for determining the presence or absence of corrosion, for example, a conventional technique as disclosed in Patent Document 1 can be adopted.

<Configuration of determination unit 3>
Next, the determination section 3 will be described in detail.
FIG. 4 is a diagram showing the functional block configuration of the determination section 3. As shown in FIG.
As shown in the figure, the determination unit 3 has an image conversion unit 30, a block division unit 31, a variance calculation unit 32, an image determination unit 33, and a shooting range detection unit 34 as functional blocks.

The image conversion unit 30 is a functional unit that converts the still image 400 extracted by the image extraction unit 2 into grayscale. The image conversion unit 30 converts the color still image 400 into grayscale using, for example, a known luminance preservation conversion technique or the like.

The block dividing unit 31 is a functional unit that divides the region of the still image extracted by the image extracting unit 2 into a plurality of regions. The block division unit 31 divides the one-frame still image 400 extracted by the image extraction unit 2 and converted into grayscale by the image conversion unit 30 into (n×m) small regions (hereinafter also referred to as “blocks”). .) 401 respectively. Here, n and m are integers of 2 or more.

FIG. 5 is a diagram showing an example of image division by the block division unit 31. As shown in FIG.
As shown in the figure, the block division unit 31 divides the still image 400 extracted by the image extraction unit 2 into (n×m) blocks 401 . In the figure, a case where a still image 400_1 is divided into 25 (=5×5) blocks 401_1 to 401_25 is shown as an example.

The variance calculator 32 is a functional unit that calculates the brightness variance for each block 401 of the still image 400 (grayscale) extracted by the image extractor 2 . As shown in FIG. 4 , the variance calculator 32 includes a first brightness variance calculator 321 and a second brightness variance calculator 322 .

The first brightness variance calculator 321 calculates a first brightness variance value 43 representing the variance of the block 401 . For example, in the case of the example shown in FIG. 5, the first luminance variance calculator 321 first calculates the luminance of each block 401_1 to 401_25 of the grayscale still image 400_1, and calculates the average luminance value of the blocks 401_1 to 401_25. calculate. Next, the first luminance variance calculator 321 calculates the luminance variance for each of the blocks 401_1 to 401_25 based on the luminance of each of the blocks 401_1 to 401_25 and the average luminance of the blocks 401_1 to 401_25. It is stored in the storage unit 4 as the brightness variance value 43 .

The second luminance variance calculator 322 is a functional unit that calculates the second luminance variance value 44 representing the variance of the first luminance variance value 43 . For example, in the case of the example shown in FIG. Calculate the value. Next, the second brightness variance calculation unit 322 calculates each of the blocks 401_1 to 401_25 based on the first brightness variance values 43 of the blocks 401_1 to 401_25 and the average value of the first brightness variance values 43 of the blocks 401_1 to 401_25. is calculated and stored in the storage unit 4 as a second luminance variance value 44 .

The image determination unit 33 is a functional unit that determines whether or not the still image 400 extracted by the image extraction unit 2 is an image of the inside of the steel pipe based on the luminance variance for each block 401 calculated by the variance calculation unit 32. be. As shown in FIG. 4 , the image determination section 33 includes a first steel pipe interior determination section 331 and a second steel pipe interior determination section 332 .

The first steel pipe internal determination unit 331 compares the first brightness variance value 43 of each block 401 with the first threshold value 41 .

Here, the first threshold value 41 is a variance value that serves as a reference for determining whether the still image 400 to be determined is an image of the interior of the steel pipe. The first threshold 41 is a value that can be arbitrarily set by the user of the data processing device 100 . In this embodiment, as an example, the first threshold value 41 is set to "20".

For example, when a still image 400 shows a subject with a large overall gray level difference, such as the ground and trees (first case), the brightness of each block 401 of the still image 400 varies. The variance (first luminance variance value 43) tends to be small. Similarly, when the still image 400 shows the sky or the inside of the steel pipe without deterioration (second case), the luminance variance of each block 401 of the still image 400 tends to be small. On the other hand, when half of the still image 400 shows trees and the other half of the still image 400 shows the sky (third case), the luminance variance of each block 401 of the still image 400 tends to increase. There is

Therefore, the first steel pipe interior determination unit 331 determines the above-described first to third cases by comparing the first brightness variance value 43 and the first threshold value 41 of each block 401 .
Specifically, when the first luminance variance value 43 of at least one block 401 is greater than the first threshold value 41 (the third case above), the first steel pipe interior determination unit 331 determines the still image extracted by the image extraction unit 2 . It is determined that the image 400 is an image other than the inside of the steel pipe (outside the steel pipe).

On the other hand, when the first brightness variance values 43 of all the blocks 401 are smaller than the first threshold value 41 (the above first case and the above second case), the first steel pipe interior determination unit 331 determines that the image extraction unit 2 It is determined that the extracted still image 400 is an image of the inside of the steel pipe.

The first steel pipe interior determination unit 331 can distinguish between the still image 400 of the third case described above and the still images 400 of the first and second cases described above.
On the other hand, the first steel pipe interior determination unit 331 cannot distinguish between the still image 400 of the first case and the still image 400 of the second case. Therefore, when the first luminance variance values 43 of all the blocks 401 are smaller than the first threshold value 41 (the above first case and the above second case), the first steel pipe interior determination unit 331 determines that the still image to be determined It is "provisionally" determined that 400 is the image of the inside of the steel pipe.

The second steel pipe interior determination unit 332 compares the second brightness variance value 44 of each block 401 with the second threshold value 42 .

Here, the second threshold 42, like the first threshold 41 described above, is a variance value that serves as a reference for determining whether the still image 400 to be determined is an image of the interior of the steel pipe. A value that can be set arbitrarily. In this embodiment, as an example, the second threshold 42 is set to "20". Note that the first threshold value 41 and the second threshold value 42 may be different values.

For example, when a still image 400 shows a subject with a large overall gray level difference, such as the ground and trees (first case), the brightness of each block 401 of the still image 400 varies. The variance of variance values (second luminance variance value 44) tends to increase. Further, when the still image 400 shows the sky or the interior of the steel pipe without deterioration (second case), the variance of the brightness variance values of the blocks 401 of the still image 400 tends to be small. Also, when trees are shown in half of the still image 400 and the sky is shown in the other half of the still image 400 (third case), the variance of the luminance variance value of each block 401 of the still image 400 is It tends to be a value between the variance in the first case and the variance in the second case.

Therefore, the second steel pipe interior determination unit 332 compares the second brightness variance value 44 of each block 401 with the second threshold value 42 to distinguish between the above-described first case and second case.

Specifically, when the second brightness variance value 44 of at least one block 401 of the still image 400 (temporarily) determined to be inside the steel pipe by the first steel pipe interior determination unit 331 is greater than the second threshold value 42 (first case), the second steel pipe interior determination unit 332 determines that the still image 400 is an image other than the interior of the steel pipe.

On the other hand, when the second luminance variance values 44 of all the blocks 401 of the still image 400 determined to be inside the steel pipe by the first steel pipe interior determination unit 331 are smaller than the second threshold value 42 (second case), the second steel pipe The interior determination unit 332 determines that the still image 400 is an image of the interior of the steel pipe.

The second steel pipe interior determination unit 332 can distinguish between the still image 400 of the first case described above and the still image 400 of the second case described above.

Based on the determination result of the image determination unit 33 for the plurality of continuous still images 400 (frames) extracted by the image extraction unit 2, the imaging range detection unit 34 determines the imaging start point ST inside the steel pipe and the imaging end point ST inside the steel pipe. Detect ED.

Specifically, when it is determined that h (h is an integer equal to or greater than 2) consecutive still images 400 (frames) extracted by the image extraction unit 2 are images of the interior of the steel pipe, the imaging range detection unit 34 , at least one of the h consecutive still images 400 is set as the image of the imaging start point ST inside the steel pipe, and the information of the imaging start point ST is stored in the storage unit 4 .

In addition, after the imaging start point ST is determined, the imaging range detection unit 34 determines that g (g is an integer equal to or greater than 2) continuous still images 400 (frames) extracted by the image extraction unit 2 are images of the outside of the steel pipe. If it is determined, the still image 400 photographed before the g consecutive still images 400 is set as the image of the photographing end point ED inside the steel pipe, and the information of the photographing end point ED is stored in the storage unit 4. .

<Image analysis method by data processing device 100>
Next, the overall flow of the image analysis method for diagnosing the state of deterioration inside the steel pipe by the data processing apparatus 100 according to the present embodiment will be described.

FIG. 6 is a flow chart showing the overall flow of the image analysis method for diagnosing the state of deterioration inside the steel pipe according to one embodiment of the present invention.

First, the data processing device 100 acquires the image data 40 of the steel pipe to be diagnosed (step S1). For example, the image data acquisition unit 1 acquires the image data 40 (for example, moving image data) of the steel pipe to be diagnosed for deterioration from the storage unit 4, an external storage medium such as a flash memory, or an external information processing device.

Next, the data processing device 100 executes steel pipe interior determination processing for specifying the range in which the steel pipe interior is photographed from the image data 40 acquired in step S1 (step S2). Specifically, the data processing device 100 identifies the still image 400 of the inside of the steel pipe from among the plurality of frames in the image data 40, Identify the end point ED. Details of step S2 will be described later.

Next, the data processing device 100 performs corrosion determination processing based on the image data 40 acquired in step S1 (step S3). Specifically, in the image data 40 acquired in step S1, the corrosion determination processing unit 5 performs , the presence or absence of corrosion of the steel pipe is determined by the method described above.

The deterioration diagnosis of the steel pipe is performed by the data processing device 100 executing the data processing according to the above procedure.

<Steel pipe internal judgment processing>
Next, the steel pipe interior determination process (step S2) described above will be described in detail.
FIG. 7 is a flow chart showing the flow of the steel pipe interior determination process (step S2).

First, the data processing device 100 extracts one frame of the still image 400 from the image data 40 acquired in step S1 (step S21). Specifically, as described above, the image extraction unit 2 extracts one still image 400 from the image data 40 acquired in step S1 at predetermined time intervals. For example, as shown in FIG. 3, the image extraction unit 2 sequentially extracts still images 400_1 to 400_k from the image data 40 for each j frame.

Next, the data processing device 100 converts the color still image 400 extracted in step S21 into grayscale (step S22). Specifically, in the case of FIG. 3 described above, the image conversion unit 30 converts the still images 400_1 to 400_k extracted in step S21 into grayscale.

Next, the data processing device 100 selects one still image 400 as a determination target from the still images 400_1 to 400_k extracted in step S21, and divides the area of the selected still image 400 into a plurality of blocks 401 (step S23). ). Specifically, the block division unit 31 divides the still image 400 converted to grayscale in step S22 into (n×m) blocks 401 . For example, as shown in FIG. 5, the block division unit 31 divides the still image 400_1 into 25 blocks 401_1 to 401_25.

Next, the data processing device 100 calculates the luminance variance of the plurality of blocks 401 divided in step S23 (step S24). Specifically, the first brightness variance calculation unit 321 calculates the first brightness variance value 43 for each block 401 and stores it in the storage unit 4 by the method described above. In the case of the example of FIG. 5 described above, the first brightness variance calculator 321 calculates the first brightness variance values 43 of the blocks 401_1 to 401_25 of the still image 400_1.

Next, the data processing device 100 compares the first luminance variance value 43 calculated in step S24 with the first threshold value 41 as the first steel pipe interior determination process (step S25). Specifically, the first steel pipe internal determination unit 331 determines whether or not the first brightness variance values 43 of all the blocks 401 are smaller than the first threshold value 41 (for example, variance value: 20).

In step S25, if the first luminance variance values 43 of all blocks 401 are not less than the first threshold value 41, that is, if the first luminance variance value 43 of at least one block 401 is greater than the first threshold value 41, The first steel pipe interior determination unit 331 determines that the still image 400 to be determined is an image other than the interior of the steel pipe (step S26).
For example, in FIG. 5 described above, among the blocks 401_1 to 401_25 of the still image 400_1, the first luminance variance value 43 of the block 401_1 is greater than the first threshold value 41, and the first luminance variance values 43 of the blocks 401_2 to 401_25 are the If it is smaller than the 1 threshold 41, the first steel pipe interior determination unit 331 determines that the still image 400_1 is an image of the steel pipe exterior.

On the other hand, in step S25, when the first luminance variance values 43 of all the blocks 401 are smaller than the first threshold value 41, the first steel pipe interior determination unit 331 determines that the still image 400 to be determined is the image inside the steel pipe. Provisional determination is made (step S27).
For example, in FIG. 5 described above, when the first luminance variance values 43 of all the blocks 401_1 to 401_25 of the still image 400_1 are smaller than the first threshold value 41, the first steel pipe interior determination unit 331 determines that the still image 400_1 is inside the steel pipe. is provisionally determined to be the image of

If it is tentatively determined in step S27 that the still image 400 to be determined is an image of the inside of the steel pipe, the variance of the brightness variance values of the plurality of blocks 401 divided in step S23 is calculated (step S28). Specifically, the second brightness variance calculation unit 322 calculates the second brightness variance value 44 for each block 401 and stores it in the storage unit 4 by the method described above. In the case of the example of FIG. 5 described above, the second brightness variance calculator 322 calculates the second brightness variance value 44 of each of the blocks 401_1 to 401_25 of the still image 400_1.

After step S28, the data processing device 100 compares the second brightness variance value 44 calculated in step S28 with the second threshold value 42 as second steel pipe interior determination processing (step S29). Specifically, the second steel pipe internal determination unit 332 determines whether or not the second brightness variance values 44 of all the blocks 401 are smaller than the second threshold value 42 (for example, variance value: 20).

In step S29, if the second luminance variance values 44 of all blocks 401 are not less than the second threshold value 42, that is, if the second luminance variance value 44 of at least one block 401 is greater than the second threshold value 42, The second steel pipe interior determination unit 332 determines that the determination target still image 400 is an image of the steel pipe exterior (step S26).
For example, in FIG. 5 described above, among the blocks 401_1 to 401_25 of the still image 400_1, the second luminance variance value 44 of the block 401_1 is greater than the second threshold value 42 (for example, 20), and the second luminance of the blocks 401_2 to 401_25 When the variance value 44 is smaller than the second threshold 42, the second steel pipe interior determination unit 332 determines that the still image 400_1 is an image of the steel pipe exterior.

On the other hand, in step S29, if the second luminance variance values 44 of all the blocks 401 are smaller than the second threshold value 42, the second steel pipe interior determination unit 332 determines that the still image 400 to be determined is the image inside the steel pipe. Determine (step S30). For example, in FIG. 5 described above, when the second luminance variance values 44 of all the blocks 401_1 to 401_25 of the still image 400_1 are smaller than the second threshold value 42, the second steel pipe interior determination unit 332 determines that the still image 400_2 is inside the steel pipe. is determined to be the image of

After step S30, the data processing device 100 determines whether or not the same determination result as in step S30 has continued h times (step S31). Specifically, the photographing range detection unit 34 determines whether or not all the h continuous still images 400 extracted in step S21 have been determined to be images inside the steel pipe.

If it is determined in step S31 that all the h consecutive still images 400 extracted in step S21 are images of the interior of the steel pipe, the photographing range detection unit 34 detects at least one of the h consecutive frames. The first image is taken as the image of the imaging start point ST inside the steel pipe (step S32).
For example, in FIG. 3 described above, when each of the three (h=3) still images 400_2 to 400_4 extracted by the image extraction unit 2 is determined to be an image inside the steel pipe, the imaging range detection unit 34 Among the still images 400_2 to 400_4, the still image 400_2 taken at the earliest time is taken as the image of the shooting start point ST inside the steel pipe.

After step S32, or when it is determined in step S31 that at least one of the h consecutive still images 400 extracted in step S21 is not an image of the inside of the steel pipe, the data processing device 100 performs step S23. , and the above-described processing is executed for the next determination target still image 400 .

On the other hand, if it is determined in step S26 that the still image 400 to be determined is an image of the outside of the steel pipe, the data processing device 100 determines whether or not the imaging start point ST inside the steel pipe has been determined ( step S34).

In step S34, if the imaging start point ST inside the steel pipe has already been determined, the data processing device 100 determines whether or not the same determination result as in step S26 has continued g times (step S35). Specifically, the imaging range detection unit 34 determines whether or not all of the g consecutive still images 400 extracted in step S21 have been determined to be images of the outside of the steel pipe.

If it is determined in step S35 that at least one of the g consecutive still images 400 extracted in step S21 is an image of the interior of the steel pipe, or in step S34, the imaging start point ST of the interior of the steel pipe is determined. If not, the data processing device 100 returns to step S23 and performs the above-described processing on the next determination target still image 400 .

On the other hand, when it is determined in step S35 that all the g consecutive still images 400 extracted in step S21 are images of the outside of the steel pipe, the photographing range detection unit 34 detects the g consecutive still images. The still image 400 (frame) before 400 (frame) is set as the image of the imaging end point ED inside the steel pipe (step S36).
For example, in FIG. 3 described above, after the still image 400_1 is detected as the image at the shooting start point ST inside the steel pipe, all of the three (g=3) consecutive still images 400_9 to 400_11 are images outside the steel pipe. If it is determined that there is, the imaging range detection unit 34 sets the still image 400_8 that was captured immediately before the still image 400_9 as the image of the imaging end point ED inside the steel pipe.

By the above procedure, the imaging start point ST inside the steel pipe and the imaging end point ED inside the steel pipe are detected, and the imaging range inside the steel pipe in the image data 40 acquired in step S1 can be specified.

Next, a specific example of the steel pipe interior determination process (step S2) will be shown.
FIG. 8 is a diagram showing temporal changes in luminance dispersion of a moving image of the inside of a steel pipe.
In the figure, first, the outer peripheral surface of the steel pipe (outside the steel pipe) on which the identification number of the steel pipe to be diagnosed is written is photographed with an industrial endoscope, and then the industrial endoscope is inserted inside the steel pipe. Then, the change in brightness dispersion over time is shown in a series of moving images from the time the industrial endoscope is moved while photographing the inside of the steel pipe to the time the industrial endoscope is removed from the inside of the steel pipe. In the figure, the vertical axis represents luminance variance (eg, second luminance variance value 44), and the horizontal axis represents time.

9A to 9D are diagrams showing one frame image at a predetermined time of the moving image shown in FIG. 9A shows an image at time t1 in FIG. 8 (image taken of the outer peripheral surface of the steel pipe), FIG. 9B shows an image at time t3 in FIG. 8 (image taken inside the steel pipe), and FIG. 9C. represents an image at time t4 in FIG. 8 (image taken inside the steel pipe), and FIG. 9D represents an image at time t6 in FIG. 8 (image taken outside the steel pipe).

As shown in FIG. 8, imaging with an industrial endoscope is started at time t0. In the period from time t0 to time t1, since the outer peripheral surface (outside of the steel pipe) of the steel pipe on which the identification number of the steel pipe is written is photographed, the luminance dispersion tends to increase. After that, at time t2, when the industrial endoscope is inserted into the steel pipe to start photographing the inside of the steel pipe, the luminance dispersion tends to be smaller than when the outside of the steel pipe is photographed. After that, at time t5, when the industrial endoscope is taken out from the inside of the steel pipe, the luminance dispersion tends to increase again.

In the case of the moving image shown in FIG. 8, the luminance dispersion starts to stabilize at a low value immediately after time t2 when the photography of the inside of the steel pipe is started. Therefore, the data processing device 100 determines that the still image 400 around the time t2 is the image of the imaging start point ST inside the steel pipe by the process of step S2 described above.

After that, at time t5, the luminance variance rises and starts to stabilize at a high value. Therefore, the data processing device 100 determines the still image 400 around the time t5 as the image of the imaging end point ED inside the steel pipe by the process of step S2 described above.

Thus, according to the data processing device 100, it is possible to specify the range in which the inside of the steel pipe is captured in the image data (moving image).

<Effects of data processing apparatus 100 according to the present embodiment>
As described above, the data processing apparatus 100 according to the present embodiment can extract the still images 400 from the image data 40 including a plurality of time-series still images based on the variation in luminance of the still images 400, and the still images 400 are images of the interior of the steel pipe. Therefore, it is possible to automatically specify the range in which the inside of the steel pipe is captured in the image data 40 (moving image) of the steel pipe to be diagnosed without relying on the operator's eyes. Become. As a result, it is possible to identify only the part where the inside of the steel pipe is filmed from a series of videos that film the inside and outside of the steel pipe, and execute image analysis processing to determine the presence or absence of corrosion in the steel pipe. Therefore, it is possible to improve the efficiency of the diagnostic work for diagnosing the state of deterioration of the steel pipe.

In addition, the data processing device 100 divides the area of the still image 400 extracted from the image data 40 into a plurality of blocks 401, and determines whether the extracted image is an image inside the steel pipe based on the luminance distribution of each block 401. determine whether or not

According to this, since it is possible to appropriately grasp the variation in luminance of the entire still image 400, it is possible to determine with high accuracy whether or not the extracted image is the image of the inside of the steel pipe.

Further, the data processing device 100 performs the first steel pipe interior determination process, when the first brightness variance value 43 representing the brightness variance of at least one block 401 is greater than the first threshold value 41, the extracted image is determined as the inside of the steel pipe. If the first brightness variance value 43 of all the blocks 401 is smaller than the first threshold value 41, it is provisionally determined that the extracted image is the image inside the steel pipe. This makes it possible to distinguish between the still image 400 in the third case described above and the still images 400 in the first and second cases described above.

Further, as the second steel pipe interior determination process, the data processing device 100 performs a second luminance distribution representing the dispersion of the first luminance variance value 43 of at least one block 401 of the image determined to be inside the steel pipe by the first steel pipe interior determination process. When the variance value 44 is larger than the second threshold value 42, the image is determined to be an image other than the inside of the steel pipe, and the second When the brightness variance value 44 is smaller than the second threshold value 42, it is determined that the image is an image inside the steel pipe. This makes it possible to distinguish between the still image 400 in the first case and the still image 400 in the second case from the image data 40 .

In this way, the data processing device 100 performs two-stage determination of the first steel pipe interior determination process and the second steel pipe interior determination process, so that whether or not the image is an image of the interior of the steel pipe can be determined with higher accuracy. becomes possible.

Further, the data processing device 100 detects the imaging start point ST and the imaging end point ED inside the steel pipe based on the determination results for the plurality of consecutive extracted frames. According to this, since it is possible to grasp the temporal change in luminance distribution of each still image 400 extracted from the image data 40, the imaging start point inside the steel pipe and the imaging end point inside the steel pipe can be determined with higher accuracy. detection becomes possible.

Further, when it is determined that a plurality of continuous frames are images of the inside of the steel pipe, the data processing device 100 sets at least one of the plurality of continuous frames as an image of the shooting start point inside the steel pipe, and starts shooting. After determining the points, if it is determined that a plurality of consecutive frames are images of the outside of the steel pipe, the frames preceding the plurality of consecutive frames are taken as images of the imaging end points inside the steel pipe. According to this, it becomes possible to detect the imaging start point and the imaging end point inside the steel pipe without performing complicated data processing.

<<Expansion of Embodiment>>
Although the invention made by the inventors of the present invention has been specifically described above based on the embodiments, it goes without saying that the invention is not limited thereto, and that various modifications can be made without departing from the gist of the invention. stomach.

For example, in the above-described embodiment, a case where the image data 40 is a moving image was exemplified, but the present invention is not limited to this. For example, the image data 40 may be data including a plurality of time-series still images. In this case, the image extraction unit 2 sequentially extracts one still image from among the plurality of time-series still images at predetermined time intervals.

Moreover, the above-mentioned flowchart shows an example for explaining the operation, and is not limited to this. That is, the steps shown in each figure of the flowchart are specific examples, and the flow is not limited to this flow. Specifically, the order of some processes may be changed, other processes may be inserted between each process, and some processes may be performed in parallel.

For example, in the flowchart shown in FIG. 7, the process of converting the image into grayscale (step S22) may be performed on the entire image data 40 immediately after the image data 40 is acquired in step S1.

Further, after extracting a plurality of still images 400_1 to 400_k in steps S21 and S22 and converting them to grayscale, one still image 400 to be determined is selected, and steps S23 to S36 are executed. Although exemplified, it is not limited to this. For example, the processes from steps S21 to S36 may be continuously executed for each still image 400. FIG.

Further, when the image data 40 is grayscale data, the process of step S22 may be omitted.

DESCRIPTION OF SYMBOLS 1... Image data acquisition part 2... Image extraction part 3... Determination part 4... Storage part 5... Corrosion determination process part 30... Image conversion part 31... Block division part 32... Variance calculation part 33... Image determination unit 34 Shooting range detection unit 40 Image data 41 First threshold value 42 Second threshold value 43 First brightness variance value 44 Second brightness variance value 100 Data processing device 321 First brightness variance calculator 322 Second brightness variance calculator 331 First steel pipe interior determination unit 332 Second steel pipe interior determination unit 400, 400_1 to 400k Still images 401, 401_1 to 401_25 ... block, 1021 ... program, ED ... photography end point, ST ... photography start point.

Claims (7)

an image data acquisition unit that acquires image data including a plurality of time-series still images;
an image extraction unit for extracting the still image from the image data acquired by the image data acquisition unit;
a determination unit that determines whether the still image is an image of the interior of the steel pipe, based on the luminance variation of the still image extracted by the image extraction unit;
The determination unit is
a block division unit that divides the region of the still image extracted by the image extraction unit into a plurality of blocks;
a variance calculation unit that calculates a brightness variance for each block;
an image determination unit that determines whether the still image extracted by the image extraction unit is an image of the interior of the steel pipe, based on the luminance variance for each block calculated by the variance calculation unit. death,
The variance calculator,
a first brightness variance calculator that calculates a first brightness variance value representing the variance of the brightness of the block ;
a second brightness variance calculator that calculates a second brightness variance value representing a variance of the first brightness variance values of the plurality of blocks ;
The image determination unit
If the first luminance variance value of at least one of the blocks is greater than a first threshold, the still image extracted by the image extracting unit is determined to be an image of an area other than the inside of the steel pipe, and all the blocks a first steel pipe interior determination unit that determines that the still image extracted by the image extraction unit is an image of the interior of a steel pipe when the first luminance variance value of is smaller than the first threshold ;
determining whether or not the second brightness variance value of all the blocks in the still image determined to be an image inside the steel pipe by the first steel pipe interior determination unit is smaller than a second threshold; When the second luminance variance value is not smaller than the second threshold, the still image is determined to be an image other than the inside of the steel pipe, and the still image determined to be the image inside the steel pipe by the first steel pipe interior determination unit a second steel pipe interior determination unit that determines that the still image is an image of the interior of the steel pipe when the second luminance variance values of all the blocks are smaller than the second threshold value. data processing equipment. The data processing device according to claim 1,
The determination unit
An imaging range detection unit for detecting an imaging start point and an imaging end point inside the steel pipe in the image data based on the determination result of the image determination unit for the plurality of continuous still images extracted by the image extraction unit. A data processing device, further comprising: In the data processing device according to claim 2,
When it is determined that each of the plurality of continuous still images is an image of the interior of the steel pipe, the imaging range detection unit detects at least one of the plurality of continuous still images as the imaging start point of the interior of the steel pipe. and when it is determined that each of the plurality of continuous still images is an image of the outside of the steel pipe after the determination of the shooting start point, the image is captured before the plurality of continuous still images. A data processing device, wherein the still image is an image of an imaging end point inside the steel pipe. an image data acquisition step in which the information processing device acquires image data including a plurality of time-series still images;
an image extraction step in which the information processing device extracts the still image from the image data acquired in the image data acquisition step;
a determining step in which the information processing device determines whether or not the still image is an image of the interior of the steel pipe, based on variations in brightness of the still image extracted in the image extracting step;
The determination step includes
a block dividing step in which the information processing device divides the region of the still image extracted by the image extracting step into a plurality of blocks;
a variance calculation step in which the information processing device calculates a brightness variance for each of the blocks;
An image for determining whether the still image extracted by the image extracting step is an image of the inside of the steel pipe, based on the luminance variance for each block calculated by the variance calculating step, by the information processing device. a determining step;
The variance calculation step includes:
a first brightness variance calculation step in which the information processing device calculates a first brightness variance value representing the variance of the brightness of the block ;
a second luminance variance calculation step in which the information processing device calculates a second luminance variance value representing a variance of the first luminance variance values of the plurality of blocks ;
The image determination step includes:
The information processing device determines that the still image extracted by the image extraction step is an image of an area other than the interior of the steel pipe when the first luminance variance value of at least one of the blocks is greater than a first threshold. and determining that the still image extracted by the image extracting step is an image of the inside of the steel pipe when the first luminance variance values of all the blocks are smaller than the first threshold value. a determination step ;
The information processing device determines whether or not the second brightness variance value of all the blocks in the still image determined to be the image inside the steel pipe in the first steel pipe interior determination step is smaller than a second threshold, If the second luminance variance values of all the blocks are not smaller than the second threshold value, the still image is determined to be an image other than the inside of the steel pipe, and the step of determining the inside of the first steel pipe determines the image of the inside of the steel pipe. a second steel pipe interior determination step of determining that the still image is an image of the interior of a steel pipe when the second luminance variance values of all the blocks in the still image determined as are smaller than the second threshold value and an image analysis method comprising: In the image analysis method according to claim 4,
The determination step includes
The information processing device determines a shooting start point and a shooting end point inside the steel pipe in the image data based on the determination result of the image determination step for the plurality of continuous still images extracted in the image extraction step. An image analysis method, further comprising a photographing range detection step for detection. In the image analysis method according to claim 5,
In the imaging range detection step, when the information processing device determines that each of the plurality of continuous still images is an image of the interior of the steel pipe, at least one of the plurality of continuous still images is detected. When it is determined that each of the plurality of continuous still images is an image of the outside of the steel pipe after the determination of the imaging start point, the plurality of continuous still images An image analysis method, characterized in that the still image photographed before is used as an image of the photographing end point inside the steel pipe. A program that causes the information processing device to execute each step in the image analysis method according to any one of claims 4 to 6.

Images