JP5764238B2 - Steel pipe internal corrosion analysis apparatus and steel pipe internal corrosion analysis method - Google Patents

Description

  The present invention relates to a steel pipe internal corrosion analysis apparatus and a steel pipe internal corrosion analysis method, and more specifically, a steel pipe internal corrosion analysis apparatus that extracts and evaluates deterioration by analyzing an image of the inside of a steel pipe tower. And a steel pipe internal corrosion analysis method.

Steel pipes of power transmission line facilities may have corrosion inside due to aging. Therefore, it is necessary to periodically inspect the inside of the steel pipe. However, the steel pipe internal corrosion of the steel pipe tower cannot be confirmed from the appearance, so there is a risk of overlooking deterioration and thinning over time. For this reason, conventionally, an industrial endoscope or the like is inserted into a steel pipe, the image is analyzed, and a qualitative determination of the degree of deterioration has been performed. However, not only a great deal of manpower and experience is required to investigate steel pipes that are several meters to several tens of meters long, but it also qualitatively determines the degree of deterioration by identifying the deterioration from the collected images. Therefore, there is a problem in that the determination results vary when the determiners are different and a long time is required for the determination.
In order to solve this problem, Patent Document 1 discloses a rusting state determination apparatus capable of quantitatively determining the rusting state of a structure made of steel. According to this, the pixel occupancy rate for each degree of rusting in the rust color area is calculated, and the result is output to determine the rusting status.

JP 2005-291984 A

However, the conventional technique disclosed in Patent Document 1 calculates the pixel occupancy ratio of the rusting region from the color of the photographed image, so that rust and other identification are not necessarily accurately performed. There's a problem.
The present invention has been made in view of such problems, and when performing analysis by image processing based on an actual video image, the shooting environment is unified to some extent by preprocessing, and after the conditions are unified, the deterioration degree portion It is an object of the present invention to provide a steel pipe internal corrosion analysis apparatus and a steel pipe internal analysis method capable of accurately and efficiently extracting a deteriorated portion and evaluating the deterioration degree by extracting and classifying the steel pipe.

For the present invention to solve the above problems, claim 1 is a steel tube corrosion analysis system you analyze the image in the steel pipe, based on the center portion of the steel pipe estimated by the O-flop optical flow, estimating a center position of the steel pipe, laminated on the basis of the movement positions of the camera and the developed image forming means, the pre-Symbol deployment image to create an expanded image obtained by planar development of the image in each frame on the basis of the center position, and an image-clad combined means the creating a developed image of the whole of the steel pipe, characterized by the Turkey to extract the degradation region to the entire of the developed image prior Symbol steel.

According to a second aspect of the present invention, the center position is estimated based on the opening of the steel pipe estimated from the luminance of the image and the center portion .

Claim 3, and an image creating device and an image-clad combined unit deployment, a steel internal corrosion analysis method by the steel tube corrosion analysis system you analyze the images in the steel tube, said expanded image creating device, Oh Estimating a center position of the steel pipe based on a central portion of the steel pipe estimated by optical flow, creating a developed image in which the image is planarly developed for each frame based on the center position; and a step combined unit, bonded based pre Symbol expand image to the moving position of the camera, to create the developed image of the whole of the steel pipe,
Characterized in that it have a, a step that to extract the degradation region to the entire of the developed image prior Symbol steel.

Claim 4, the opening of the steel pipe which is estimated from the luminance of the image, based on the central portion, characterized in that it have the step of estimating the center position.

  According to the present invention, a steel pipe internal corrosion analysis apparatus that extracts and evaluates deterioration of a deteriorated portion in the steel pipe by analyzing an image obtained by an endoscopic inspection of the steel pipe. The image is estimated based on the camera position estimation result information using the brightness of the aperture and the optical flow, and a developed image creating means for creating an image in which the image for each frame is developed using the estimated result, and the development for each frame Image pasting means for pasting images based on the movement position of the camera, creating an image in which the entire steel pipe is flattened by the developed image creating means, and extracting a degradation region for the created image. It is possible to grasp the area of the deteriorated portion of the steel sheet and the deterioration state of the entire steel pipe.

It is a block diagram which shows an example of the hardware constitutions of the degradation prediction mapping apparatus of this invention. It is a figure which shows the example of a captured image. It is a figure which shows the correction | amendment (1) result of a color nonuniformity. It is an enlarged view of the part in a frame of FIG. It is a figure which shows a bihexagonal-pyramid model. It is a figure which shows the process sequence of a color correction (bihexagonal pyramid model). It is a figure which shows the example of a result of a color correction (bihexagonal pyramid model). It is a figure which shows the process sequence of illumination nonuniformity correction (method 1). It is a figure which shows the process sequence of illumination nonuniformity correction (method 2). It is a figure which shows the example of a result of illumination nonuniformity correction | amendment. It is a figure which shows the example of a result of Retinex. It is a figure which shows a lens distortion image. It is a figure which shows the lens distortion correction result to a steel pipe image. It is a figure which shows the example of an embossing process. It is a figure which shows the filter used in FIG. It is a figure which shows an emphasis display result. It is a figure which shows 7 steps | paragraph classification by RGB average value. It is a figure which shows the RGB average value for every deterioration degree. It is a figure which shows the area | region extraction result using RGB. It is a figure which shows the example of an image mask for the feature-value collection of a degradation area | region. It is a figure which shows the deterioration degree distribution by a UCS coordinate system. It is a figure which shows the example of YIQ conversion from RGB. It is a figure which shows the example of an image converted into YIQ. It is FIG. (1) which shows the example of red rust extraction by YIQ. It is a figure (the 2) which shows the example of red rust extraction by YIQ. It is a figure which shows the list of the image file extracted from the sample. It is a figure which shows the degradation degree distribution (I-lightness) of the extracted image. It is a figure which shows a brightness normalization process. It is a figure which shows a brightness correction | amendment comparison. It is a figure which shows the variation of a correction object image. It is a figure which shows the parameter by subjective classification. It is a figure which shows the coincidence degree of the classification | category by subjective classification. It is a figure which shows the image of a support vector machine. It is a figure which shows the utilization structure of a support vector machine. It is a figure which shows the coincidence degree of the classification | category by a support vector machine. It is a figure which shows the coincidence degree of the classification | category by AdaBoost. It is a figure which shows the matching degree comparison by a classification system. It is a figure which shows the structure of a prototype. It is a figure which shows No2-32 degradation degree classification result (by subjective classification). It is a figure which shows No3-04 degradation classification result (by subjective classification). It is a figure which shows No2-32 degradation degree classification result (by AdaBoost). It is a figure which shows No3-04 degradation classification result (by AdaBoost). It is a figure which shows No2-32 degradation classification result (by support vector machine). It is a figure which shows No3-04 degradation classification result (by a support vector machine). It is a figure which shows a degradation degree classification result. It is a figure which shows the example of plane expansion | deployment. It is a figure which shows the presumed image of the steel pipe center position. It is a figure which shows a moving image reproduction program structure. It is a figure which shows the example of a moving image reproduction. It is a figure which shows the deterioration degree determination result with respect to a steel pipe expansion | deployment image. It is a figure which shows the example of a foreign material area | region extraction result. (A) is a figure which shows the comparison result of a deterioration degree determination result, (B) is a figure which shows the deterioration degree determination result with respect to a steel pipe expansion | deployment image. (A) is a figure which shows the unevenness | corrugation estimation result (contour line display) of the whole steel pipe, (b) is a figure which shows the unevenness | corrugation estimation result (3D display) of the whole steel pipe.

  Hereinafter, the present invention will be described in detail with reference to embodiments shown in the drawings. However, the components, types, combinations, shapes, relative arrangements, and the like described in this embodiment are merely illustrative examples and not intended to limit the scope of the present invention only unless otherwise specified. .

  FIG. 1 is a block diagram showing an example of a hardware configuration of a steel pipe internal corrosion analysis apparatus of the present invention. The steel pipe internal corrosion analysis apparatus 100 according to the present invention has a CPU (central processing unit) that sequentially performs processing according to a program and a ROM that stores programs or universal data (similar to a general computer hardware configuration). A read-only memory), a RAM (temporary access memory) that temporarily stores data and reads the data and supplies it to the CPU, a database (DB) 4 that stores a large amount of data, a keyboard and a mouse, etc. An input unit 5 for inputting data by an input device, an output unit 6 for displaying mapping data, and a bus 7 for connecting the respective units are provided. Note that the CPU, ROM, and RAM serve as the control unit 8. Further, the present invention is not limited only to the above-described embodiments. Each function constituting the steel pipe internal corrosion analysis apparatus 100 of the above-described embodiment is programmed, written in advance on a recording medium such as a CD-ROM, and this is stored in a medium driving apparatus such as a CD-ROM drive mounted on a computer. It goes without saying that the object of the present invention is achieved by mounting a CD-ROM or the like, storing these programs in a memory or storage device of a computer, and executing them. In this case, the program itself read from the recording medium realizes the functions of the above-described embodiment, and the program and the recording medium recording the program also constitute the present invention.

This steel pipe internal corrosion analysis apparatus 100 is a steel pipe internal corrosion analysis apparatus 100 that extracts and evaluates deterioration of a deteriorated portion in the steel pipe by analyzing an image obtained by a steel pipe endoscopic survey. A control unit 8 having a CPU that performs sequential processing, a ROM that stores programs and data, and a RAM that temporarily stores data, reads the stored data, and supplies the RAM to the CPU, under various conditions. The preprocessing means 10 for removing the difference in the photographing conditions from the obtained image, the deteriorated part preliminary extracting means 11 for preliminarily extracting the deteriorated part, and the preprocessing means 10 unify the photographing conditions, and the deteriorated part preparatory part. Based on the extraction result obtained by the extraction unit 11, a degraded part extraction / classification unit 12 that extracts and classifies an area for each degree of degradation from an image, and a prototype is actually created. The degradation region verification evaluation unit 14 that verifies and evaluates the extraction result of the degradation region and the display processing unit 15 that displays the degradation region verified and evaluated by the degradation region verification evaluation unit 14 are configured. .
The captured images used in the present invention are a moving image captured by an industrial endoscope and a still image obtained by extracting a deteriorated portion from the moving image. A part of it is shown in FIG. 2A to 2D correspond to the degradation degrees II to V. FIG.
Many of the colors are particularly bluish (especially openings) (part A), but some are reddish (part B). In addition, there are some CCD noises where one of the RGB values appears bright in dark areas due to insufficient light quantity.
The brightness (brightness) of the entire image is considerably biased depending on the distance between the illumination and the object at the time of shooting. In extreme cases, there is a part that is completely white when the lighting approaches the side of the steel pipe.

Next, color correction for reducing color unevenness that is often seen when the amount of light is insufficient will be described.
First, the RGB image was once converted into a HSV (hue, saturation, brightness) color space, and H (hue) and S (saturation) were subjected to a blurring effect (low-pass filter). This is a technique often used in photographic processing and the like.
The processing procedure is shown below.
1) Convert RGB to HSV.
RGB → HSV conversion formula:
When MAX is the maximum value of RGB (r, g, b) and MIN is the minimum value of RGB (r, g, b),
When MAX is r: H = 60 * (g−b) / (MAX−MIN) +0
When MAX is g: H = 60 * (b−r) / (MAX−MIN) +120
When MAX is b: H = 60 * (r−g) / (MAX−MIN) +240
S = (MAX-MIN) / MAX
V = MAX
2) For the H and S images, the average of the pixel values in the vicinity of each pixel 5 × 5 is obtained and set as a new pixel value (low-pass filter).
3) Convert HSV to RGB.
HSV → RGB conversion formula:
Hi = (60 / H)% 6 (% means the remainder divided)
f = H / 60-Hi
p = V (1-S)
q = V (1-fS)
t = V (1- (1-f) S)
When Hi is 0: r = V, g = t, b = p
When Hi is 1: r = q, g = V, b = p
When Hi is 2: r = p, g = V, b = t
When Hi is 3: r = p, g = q, b = V
When Hi is 4: r = t, g = p, b = V
When Hi is 5: r = V, g = p, b = q

FIG. 3 shows an application example of the above method. In (a) to (d) of FIG. 3, the left is an image before correction and the right is an image after correction. At first glance, no significant change is seen, but it can be confirmed that the CCD noise can be reduced as shown in FIG. 4 by enlarging the A part and the B part in FIG.
The previous color correction (1) corrects local color unevenness, and cannot correct the overall color shift (a strong blue image) that seems to be caused by the color balance characteristics of the camera. . Here, the color of the entire image was corrected using a bihexagonal spindle model. FIG. 5 is a diagram showing a bihexagonal spindle model. The bihexagonal weight model is a color space in which hexagonal weights are stacked one above the other. The hexagon corresponding to the base of the hexagonal weight represents H (hue), and the tip of the hexagonal weight represents lightness.

The color correction procedure is shown below using FIG.
1) Of the RGB values having a value in the range of 0 to 255, obtain a pixel in which all of RGB falls within 120-200 (values are determined from the results for each of the plurality).
2) Calculate the average value (avrR, avrG, avrB) of the pixels in 1).
3) The result of 2) is subjected to HSV conversion to obtain brightness (avrR, avrG, avrB)-> lightness avrV.
4) Converts RGB (nR, nG, nB) to RGB (nR, nG, nB) having the same lightness (avrV), no hue (H = arbitrary), and saturation (S = 0), so all converted RGB values are the same Value).
5) Conversion ratio rateR = nR / avrRateG = nG / avrG for converting the RGB average value (avrR, avrG, avrB) of the original image into this RGB (nR, nG, nB).
6) Correction is performed by multiplying all pixels with the conversion ratio (however, the result is trimmed so that it falls within the range of 0-255).
The basic idea of this method is to obtain the hue (hue) bias of the entire image and correct it to the center of the hue to remove the overall hue bias.

FIG. 7 shows an example of the result of applying this method. (A) is before correction, (b) is a method according to the present invention, and (c) is a result of automatic correction by Adobe Photoshop (registered trademark), which is a typical image editing software, for comparison.
The overall color (for example, strong bluishness) is improved both in the present method of (b) and the automatic correction of PhotoShop in (c). However, in the automatic correction of PhotoShop, the brightness is also automatically corrected at the same time.
This method is effective for correcting the color tone of the entire image, while the noise removal method using a low-pass filter is effective for local CCD noise.

As a method for removing the illumination unevenness, two methods of method 1 in FIG. 8 and method 2 in FIG. 9 are applied.
Method 1: The image is divided into blocks and corrected so that the brightness of each block is equal.
1) Convert the image from RGB to HSV.
2) The image converted into HSV is divided into 15 × 15 blocks, and the average (avrVi, j) of each lightness (V) is obtained (in order to remove the prominent values, the standard deviation 35 to 65 is obtained when obtaining the average) Use a range value).
3) Obtain the average brightness (avrVall) of the entire image.
4) The brightness correction ratio rateVi, j = avrVall / avrVi, j is obtained for each block, and the brightness of all the pixels in the image is updated (at this time, the adjacent blocks avrVi ± 1, j ± 1 are used to update each pixel). Interpolate the correction ratio).
5) Convert to HSV image RGB image.

Method 2: Correction is performed so that the brightness of the block centered on the pixel of interest is equal to the brightness of the entire image.
1) Convert the image from RGB to HSV.
2) Obtain the average brightness avrVall of the entire V image.
3) For the V image, obtain the average brightness avrVi, j of the 15 × 15 surrounding pixels of each pixel Vi, j.
4) Lightness after correction V′i. Let j be Vi, j- (avrVi, j-avrVall).
5) Convert the HSV image to an RGB image.

FIG. 10 shows the result of applying Method 1 and Method 2. 10A shows the original image, FIG. 10B shows method 1, and FIG. 10C shows method 2.
Method 1 can relatively remove illumination unevenness. In Method 2, illumination unevenness can be removed, but the characteristic shades of other deteriorated portions are uniformed.
For the purpose of tone correction, Retinex theory is a method that has been attracting attention in recent years. It is assumed that the input image is represented by the product of the illumination light and the reflectance of the object, and the reflectance image is obtained as a corrected image by separating the illumination light from the input image.

FIG. 11 shows an example of performing Retinex processing provided in the image editing software GIMP2. (A) is the original image, (b) is the Retinex parameter 1, (c) is the Retinex parameter 2, and (d) is the result of method 1 of illumination unevenness correction (lightness uniformity in the image) for comparison.
GIMP2 is one of image processing tools such as Adobe PhotoShop.
Retinex parameter 1: Scale = 240, Scale division = 3, Dynamic = 1.2 (default value)
Retinex parameter 2: Scale = 120, Scale division = 3, Dynamic = 0.8
In the upper image in FIG. 11, the Retinex parameter 2 is almost the best state, but in the lower image, the default Retinex parameter 1 is almost the best state than the Retinex parameter 2.
Trends include the following:
・ The shaded area is made uniform, but the overall color is lost.
・ Camera noise is emphasized when parameters are changed to produce color.

FIG. 12 shows a lens distortion image.
Usually, a photographed image using a camera or the like is distorted so that the periphery of the video is expanded. This becomes more conspicuous as the lens becomes compatible with a wide angle (a fisheye lens is an example). In general image processing, particularly when measuring the position of an object, it is indispensable to correct this lens distortion. We will check whether correction of lens distortion is necessary even for shooting inside the steel pipe.
Although there is a method of strict correction using a calibration plate or the like as a method for correcting lens distortion (calibration), a simple method is adopted because work in an actual site is laborious and impractical.
The principle of this method assumes that the lens 1 is spherical. The image (light) of the object to be imaged is refracted when passing through this lens surface, reaches the CCD surface 3 and is recorded as an image. Therefore, the lens distortion can be corrected by performing a process of moving the captured image (pixels on the blue line) 3 to the actual picture (pixels on the green line) 2 as shown in FIG.

FIG. 13 shows the result of lens distortion correction processing performed on an image obtained by photographing the actual inside of a steel pipe. (A) shows an image before correction, and (b) shows an image after correction. Since the inside of the steel pipe, which is originally cylindrical, is taken, there is no apparent distortion around the image. Rather, image degradation (lower resolution) occurs because the peripheral portion is extended for correction. Considering that the same image deterioration occurs in the development process for a plan view described later, it is often possible to obtain a better result by performing only the plan view development process.
Although it does not necessarily contribute directly to the automatic extraction of the deteriorated part, it is expected to improve the overall diagnosis efficiency including confirmation of the automatic extraction result by making the inspection image easy for humans to understand. Here, an attempt is made to highlight a captured image using a pseudo-stereoscopic display technique. FIG. 14 is a diagram showing an example of embossing processing, and FIG. 15 is a diagram showing an embossing filter.
There is a so-called embossing process as a technique used for pseudo-stereoscopic display. This focuses on the edge portion of the image area and brightens the image in the upper left direction of the edge and darkens the lower right direction to make the area rise three-dimensionally.
There are several methods for embossing, but this time the embossing filter processing of FIG. 15 was used. As shown in FIG. 14, each pixel of an image and its peripheral pixels are multiplied by the above parameters and added to obtain a new value for the pixel. As a result, a portion with a change in luminance (for example, the boundary of the corrosion region) or the like appears to be three-dimensionally emphasized.

FIG. 16 shows an example adapted to an actual captured image. Obviously, compared with the image before processing, the deteriorated portion is emphasized and easily recognized, and the effect is recognized.
In “Study on Degradation Evaluation of Hot-Dip Galvanized Steel Surface by Image Analysis”, a seven-stage deterioration degree sample was created as shown in FIG. 17 instead of the four-stage deterioration degree sample by NTT, and this was adopted as an evaluation standard. A continuous small area is extracted from the parallel part of the specimen that is the basis of the deterioration degree sample, and an average value of each RGB is obtained and plotted on the three-dimensional RGB.

FIG. 18 shows the average RGB values for each degree of deterioration used in the above paper (values are not necessarily accurate because they are collected from RGB three-dimensional plots). It can be seen that the deterioration levels I to V correspond to white rust, and the deterioration levels VI and VII correspond to red rust.
FIG. 19 shows the result of region extraction performed on the sample image using this RGB average value (the degradation level display on the left side of the image in FIG. 19 is a diagnosis result of five steps on the sample image, Note that this is different from the 7-level classification using RGB average values (see “Extraction Degradation Partial Legend” at the bottom of FIG. 19).
The extraction method using RGB average values is as follows.
1) The RGB value of each pixel of the target image is obtained.
2) It is determined to which RGB degree the RGB value matches the RGB value shown in FIG.
At this time, the allowable error of the RGB value is ± 20, and when it is included in the plurality of average RGB values ± 20, the one closer to the RGB average value is adopted.
For example, if RGB of a certain pixel is (130, 130, 140), it is determined that the pixel corresponds to the degree of deterioration III.

From the results, the white rust portion having a low degree of deterioration can be extracted as it is, but the red rust portion (VI, VII in terms of the deterioration degree in the five-step evaluation) cannot be extracted well. The reason for this is that comparing the deteriorated portion of the image used in the extraction with FIG. 17, the deterioration degrees VI and VII colors according to the RGB classification are brighter than the corroded portion of the actual image, so the corrosion progresses and becomes almost black. This is because the portion does not match.
Since the RGB average value is bright overall as compared with the deteriorated portion of the sample image obtained by photographing the inside of the steel pipe, it can be inferred that the paper is not on the inside of the steel pipe but on the surface of the steel tower exposed to the outside.
The steel surface degradation assessment system using self-organizing feature maps combines the UCS coordinate system (y, u, v) color space with patterns quantified using self-organizing maps. ing.
UCS (Uniform Chromaticity Scale) is a display method that corrects the non-linearity of the eye with respect to color and uses the maximum color space that can be displayed by the CRT used. Humans can detect even a slight chromaticity difference in the vicinity of blue, but have non-linearity that cannot detect a color difference unless there is a large chromaticity difference in the vicinity of green. A color space in which this chromaticity difference is proportional to the difference in sensory color properties is called a uniform perception space, and a display system using this space is the UCS color system.

A self-organizing map is a kind of neural network that can map multi-dimensional data in two dimensions and is used for visualization of information in a high-dimensional space.
Here, the effectiveness of the UCS color system was examined. First, as a deteriorated image sample, feature values (RGB) for each deteriorated region of a sample image (91 sheets) shown in FIG.
The feature amount collection for each deteriorated area from the sample images (91 sheets) was performed as follows.
1) A mask image is manually created for an area of a degraded part of the image (FIGS. 20A to 20D).
2) The RGB value of each pixel in the area 1) is obtained, and the average value of the entire area is obtained. → Characteristic quantity for each degraded area 3) The above process is performed for each of the 91 images.
Next, the collected feature quantity (RGB) for each deteriorated region is converted into UCS by the following conversion formula from RGB (r, g, b) to UCS (u, v, y).
u = 4x / (-2x + 12y + 3)
v = 9y / (-2x + 12y + 3)
x = (0.49000r + 0.31000g + 0.2000b) / m
y = (0.1769r + 0.81240 g + 0.01063b) / m
z = (0.00000r + 0.01000g + 0.99000b) / m
m = 0.66697r + 1.13240g + 1.20063b
In FIG. 21, the UCS average values collected from 91 images are mapped.
As a result, the degradation degrees III, IV, and V are mixed, and it is difficult to say that the classification is clear only by the UCS coordinate system.

The YIQ color space is a color system used as a standard in NTSC, which is one of the standards for television images. The feature is that the skin color system (orange to cyan) that is easy to distinguish for human eyes is taken on the I axis, and the cold color system (green to magenta) that is difficult to distinguish is taken on the Q axis (Y is luminance) Showing).
The conversion formula from the RGB color space (r, g, b) to the YIQ color space (y, i, q) is shown below.
y = 0.299r + 0.587g + 0.114b
i = 0.596r−0.274g−0.322b
q = 0.211r-0.522g + 0.311b

FIG. 22 shows an example of converting an RGB image having all hues into YIQ as an example of YIQ conversion. FIG. 23 is a diagram showing an example of converting a captured image into YIQ.
As a result of region extraction while changing the value of I for several images, it was found that a relatively red rust region can be extracted with a threshold value of I = 0.05 or more. 24 and 25 show examples of extraction. (A) is an original image, (b) is an image obtained by normalizing I with 0 to the maximum value, and (c) is a diagram showing a region where I is 0.05 or more. The value of I is displayed by normalizing 0 to the maximum value of 0 to 255 for RGB for display.
Although it cannot cope with a slightly thin portion of redness and white rust, the other conspicuous red rust regions (deterioration degree of about III or more) can be almost extracted.

FIG. 27 shows a graph in which the deterioration degree parts (II to V) extracted from the sample image (91 images in total) are extracted, and YIQ I (average) in the deterioration region is plotted on the horizontal axis and the brightness (average) is plotted on the vertical axis. . The straight line on the graph is a linear approximation of the value for each degree of deterioration. Degradation degree II can be classified relatively clearly like RGB. Although there are some deterioration degrees III and IV that have a slight deterioration degree IV higher than the deterioration degree III, it is considered that a certain degree of area division is possible. In parameter determination (subjective classification) used for deterioration degree extraction, region extraction parameters are subjectively determined based on this result.
The feature amount of the sample image (91 sheets) obtained in the previous section was compared with the deterioration sample A (general environment). Eliminating variations in brightness due to differences in shooting conditions is an essential process for stable deterioration level extraction. FIG. 28 shows the lightness normalization processing procedure.
1) Image normalization in steel pipe A as a reference a) Color correction using a bihexagonal pyramid model is performed on each image.
b) Creation of correction reference image for steel pipe A-The brightness of pixels at the same coordinates of all images in steel pipe A is taken out and rearranged in order of brightness, and the brightness of the pixel located at the center is used as a representative value.
The image created by this processing is used as the correction reference image for the steel pipe A.
c) The brightness of each image of the steel pipe A is corrected based on the reference image for correction.
The average brightness of each image and the average brightness of the reference image for correction b are obtained.
The brightness of all the images in the steel pipe A is made uniform by multiplying the ratio of (average brightness of the correction reference image) / (average brightness of the processing target image) to all the pixels.
2) Image normalization of other steel pipes B The processing of a and b is the same as 1) c) The reference image for correction of steel pipe B is adjusted so that the average brightness of the reference images for correction of steel pipe A and steel pipe B is the same. Correct the brightness. The correction method uses the same method as in 1) -c.
d) Using the correction reference image B ′ obtained in c, the brightness of each image of the steel pipe B is corrected in the same manner as in 1) -c.

FIG. 29 shows an example of the brightness correction result.
Looking at the entire steel pipe, it can be seen that the difference in brightness average before correction was 0.94 for steel pipe A and steel pipe B, but it was improved to 0.96 after brightness correction. This means that the difference in brightness average between the steel pipe A and the steel pipe B is slightly reduced.
However, in this method, all images in one steel pipe are targeted, so that the center of the steel pipe, which is a general state, is photographed, or the deteriorated part is photographed in detail close to the wall, etc. The images taken in different directions are processed in the same row. In addition, in the case of an image obtained by photographing the center of a steel pipe, when the opening is visible, the portion is very bright, but there is a mixture of images with different illumination conditions such as darkening when the opening is separated. The image focusing on the degraded part of the wall is an image in which the wall side is strongly illuminated, but the illuminated part also varies depending on which steel pipe has the degraded part in which direction. FIG. 30 shows variations of the correction target image.

Such images with different illumination conditions are desirably classified and corrected for each similar photographing state. The following are suggestions for classification.
1) Detection of the opening portion • The region of the portion RGB (255, 255, 255) close to the center position of the image where the video is overexposed is detected, and the region is excluded from the processing target.
2) Estimating the camera shooting direction-The optical flow method used in plane development described later is used. Thereby, the steel pipe center position can be estimated regardless of the presence / absence of the opening.
-The brightness distribution of the entire image is obtained, and if the bright part is extremely at the edge of the image, it can be estimated that the image is taken close to the wall.
When using YIQ I and lightness for region extraction for each degree of deterioration, it is necessary to determine which degree of deterioration corresponds to which I and lightness.
First, as shown in FIG. 31, the extraction parameters were determined subjectively from the distribution map of I and lightness. A region surrounded by a square frame for each deterioration degree is set as an extraction parameter.

FIG. 32 shows how much the above classification parameters are matched by applying data (I, brightness) for each degree of degradation of N = 91. The coincidence rate of the degree of degradation IV that varies as a whole is low.
FIG. 33 is a diagram showing an image of a support vector machine. A support vector machine is a pattern identification method using a neural network, and employs a learning model. The support vector machine uses, for example, the concept of margin maximization to obtain a separation plane that maximizes the distance to each data point in order to separate data into two types. FIG. 33 shows an example of the boundary line (black thick line) that maximizes the margin in order to separate the two data groups of red and blue. This time, the free tool “libsvm” was used.
Here, as shown in FIG. 34, the deterioration degree and feature quantity (YIQ I and brightness) of 91 images selected from the sample images are used as learning patterns, and only I and brightness of this learning data are given again. Thus, the degree of coincidence of the obtained deterioration was confirmed.

FIG. 35 shows the degree of coincidence for each degree of deterioration. Degradation levels II and IV show a high coincidence rate, but degradation levels III and V are very low values. This is because the support vector machine attempts to divide the area for each class (degradation degree), but the learning data is not necessarily well organized for each deterioration degree.
AdaBoost is also a clustering method using a learning model. As with the support vector machine, learning is performed using the given supervised data.
AdaBoost is the most basic method called the boosting method. Instead of directly determining one discriminant rule (called discriminator), a single discriminant rule is combined as a whole by combining many simple discriminators. To build. Learning is performed using the given supervised data, and by sequentially adjusting the weight based on the learning result (addition of a discriminator), a plurality of learning results are obtained, and the results are combined to improve accuracy.

Here again, learning was performed using the same data (YIQ I and lightness and deterioration) as in the previous section, and the coincidence ratio of deterioration obtained by giving the same data (YIQ I and lightness) again was confirmed. FIG. 36 is a diagram showing the degree of coincidence of classification by AdaBoost.
The result is better than subjective classification, support vector machine. Compared to support vector machines, AdaBoost is due to the flexibility that one class does not necessarily have to be in one region.
FIG. 37 shows a table comparing the three methods. As a comparison result of the three classification methods, AdaBoost shows the highest matching rate.

Region extraction for each degree of deterioration is performed using the region extraction method discussed above. FIG. 38 shows the configuration of the prototype.
In this prototype, color correction and lightness correction are performed as pre-processing, and a part for extracting and classifying a deteriorated part by subjective classification and AdaBoost is implemented. The processing procedure is shown below.
Extraction procedure for each degree of deterioration:
1) Normalization of brightness as pre-processing 2) Extraction and classification of degradation-specific parts using subjective classification method or Adaboost Also, it is possible to perform emphasis display processing as an option.

39 and 40 show the classification results based on the parameters determined by the subjective classification. The degree of deterioration on the left side of the image corresponds to the degree of deterioration in the image in which the color of the legend on the right is classified by the parameter determined by the subjective classification as a result of human judgment.
41 and 42 show the classification results based on the parameters learned using Adaboost. The degree of deterioration on the left of the image corresponds to the degree of deterioration in the image in which the color (number) of the legend on the right is classified by AdaBoost as a result of human judgment.
FIG. 43 and FIG. 44 show the classification results based on the parameters learned in “Determining the parameters used for extracting the deterioration level (support vector machine)”. Although the prototype implementation was not performed for the comparison, the result of the support vector machine classified by the degradation degree using the tool is shown. As a result of human judgment, the color (number) of the legend on the right corresponds to the degree of deterioration in the image classified by the support vector machine.

The results of “region extraction / classification result (subjective classification)” and “region extraction / classification result (AdaBoost)” will be considered.
For example, looking at the images (image numbers E and F) that the human being has judged as the degree of degradation IV in FIGS. 39 and 41, the degradation region can be almost extracted in both cases. Similarly, in an image that has been determined by a human as a degree of deterioration IV, focusing on the deterioration degree classification of the extracted region, the classification result based on the subjective classification is a deterioration degree V (a part painted in red) and a degree of deterioration III (a part filled in yellow) ) And a degree of deterioration IV (a portion painted in orange) is hardly seen. On the other hand, in the classification result by AdaBoost, the portion determined as the deterioration level V in the subjective classification is determined as the deterioration level IV, and is closer to the human determination result. However, when looking at an image that is determined to be a degradation level V by a human, almost no portion determined as the degradation level V is found in the classification result in AdaBoost, and most of them are determined as the degradation level IV. As one of the causes, it is conceivable that the data used for learning has a small number of samples of the deterioration degree V (six samples out of 91 samples are applicable).
On the other hand, the result of the support vector machine is concentrated on the degree of deterioration II and the degree of deterioration IV as in the case of the verification using the learning data.

45 (a) to 45 (c) show graphs of degradation classification results for each image of subjective classification, AdaBoost, and support vector machine. The horizontal axis of the graph is (degradation level determined by humans) − (image number), and the vertical axis is the area ratio of each degradation level with respect to the imaged region of the image (thing excluding the surrounding black frame).
At first glance, it seems that there is not much correlation between the degree of degradation determined by humans and the degree of degradation by image processing. For example, in the AdaBoost result, the image I of the degree of degradation II and the image G of the degree of degradation V are similar in the tendency of the graph, but in the image of G, the area determined by the AdaBoost to be the degree of degradation IV is relatively actual degradation. In contrast to the concentration in the portion, the I image misrecognizes the CCD noise around the image as a deteriorated portion.
Even if "color correction (low-pass filter)" is applied, complete CCD noise cannot be removed, so a certain degree of improvement can be expected by applying processing that removes isolated areas that are scattered after extraction / classification processing. However, it is necessary to consider in order not to remove the small spot-like originally deteriorated portion.

When the area of the deteriorated part is obtained accurately after extracting the region in the steel pipe, the result can be obtained more accurately by developing on the plane than by “lens distortion correction”. FIG. 46 shows a planar development example. Here, the image inside the steel pipe is expanded in a plane by designating the center position and effective range. The center position and effective range can be specified using the mouse.
When this method is applied to a moving image, since the direction of the central axis of the camera is not constant, it is not realistic to input manually one frame at a time, and a method for automatically detecting the steel pipe center position is required. Although omitted in the present invention because of priority, FIG. 47 shows a proposed method for detecting the center portion of the steel pipe. Here, an optical flow technique used in image processing is used.

Detection plan for steel pipe center:
1) For an image at a certain time (t), a plurality of parts serving as feature points are detected on the basis of brightness and edge strength (in the yellow-green circle of the left image).
2) A movement vector for each feature point of 1) is obtained for an image after a certain time (Δt) (pink arrow).
3) The movement vector is extended (green arrow), and the most intersecting part becomes the estimated steel pipe center part.

The inside of the steel pipe is basically recorded as a video image, and the region extraction processing so far has been aimed at still images, but we have created a program that adapts to continuous images of video and displays the results. FIG. 48 is a diagram showing the configuration of a moving image playback program.
The video image is converted to a bitmap format with sequential numbers. A commercially available tool or the like was used for this conversion. The converted serial number bitmap is used as a source image, and a series of processing results of preprocessing, lightness leveling, and region extraction are stored again as serial number bitmaps in the respective folders.
In the display program, a processing result folder is selected (up to four folders), and images in the folder are sequentially reproduced to display a moving image. FIG. 49 shows an example of a moving image reproduction example.
As a result of examining the pre-processing using the sample photographed image as a clue, it was found that several pre-processing are effective. In particular, the correction of illumination unevenness is considered to be useful in terms of preventing a reduction in extraction accuracy such as white rust due to non-uniformity of camera illumination.

As a result, in the preliminary examination of the deteriorated portion extraction, it was found that red rust extraction can be performed relatively stably with respect to the shooting conditions by focusing on the YIQ color space with respect to the conventional RGB or UCS color space. Further, by unifying the brightness of images taken under different shooting conditions, the possibility of extracting a deteriorated part was obtained by combining the I value of YIQ and the brightness.
In addition, I value and brightness data of YIQ for each degree of degradation are collected, and this is a method of determining extraction parameters by subjective classification and a learning algorithm based on AdaBoost. Classification became possible.

FIG. 50 is a diagram showing the degradation degree determination result for the steel pipe development image. In order to grasp the actual area of the degradation part and the overall degradation status in the steel pipe by creating a flat image of the entire steel pipe and extracting the degradation area corresponding to this, the following examination was conducted.
1) Estimate the steel pipe center position based on the camera position estimation result information using the brightness of the opening and the optical flow, and use this to create an image in which the image for each frame is developed. That is, the feature point by the optical flow is extracted, the center position is estimated using the flow vector, the opening focused on the brightness is estimated, the center position is estimated using the flow vector and the brightness, and FIG. Such a steel pipe whole development image is created and a degradation area is extracted.
2) The developed images for each frame are pasted based on the movement position of the camera.

FIG. 51 is a diagram illustrating an example of a foreign substance region extraction result. In FIGS. 51A and 51B, the left side is an original image, and the right side is a diagram illustrating a region extraction result. The following studies were conducted as a method for extracting areas not subject to degradation processing (foreign substances such as nesting).
1) Extraction of frequency components using Fourier transform and examination of trends 2) Extraction of areas focusing on edge shapes and examination of trends 3) Extraction of areas using texture features and trial using prototype Results of examination 3) Texture By estimating the feature quantity as a learning algorithm, it was possible to extract the non-degradation target area relatively well.

  FIG. 52A is a diagram showing a comparison result of deterioration degree determination results, and FIG. 52B is a diagram showing a deterioration degree determination result for a steel pipe development image. From FIG. 52 (A), (a) is an original image, (b) is a diagram showing a result of I-lightness, and (c) is a diagram showing a result of I-lightness and edge information.

FIG. 53A is a diagram showing the unevenness estimation result (contour line display) of the entire steel pipe, and FIG. 53B is a diagram showing the unevenness estimation result (3D display) of the entire steel pipe. The method of estimating the unevenness of the deteriorated part in the steel pipe was examined, and the following procedure was used for estimation.
1) Simultaneously track a plurality of feature points and calculate the amount of movement (flow vector group) of feature points between images with different shooting positions.
2) Estimate the camera position in the steel pipe from the flow vector group.
3) The three-dimensional coordinates of the feature point are obtained by estimating the position of the feature point in the steel pipe again from the estimated camera position.

  As a result of the examination from FIG. 50 to FIG. 53, it is possible to develop an image plane of the entire steel pipe, and it is possible to grasp at a glance the overall situation inside the steel pipe that could only be confirmed with moving image data that has been captured in the past. Is assumed to be easy. In addition, a relatively good result was obtained for the degradation target area. In addition, in the improvement of the degradation degree determination accuracy, it was possible to compensate for the poor portion of the degradation area extraction using the color and brightness information by using the texture analysis information. Furthermore, the estimation of unevenness of the steel pipe degradation degree part was possible to some extent from limited information, but to obtain higher accuracy including image plane development, improvement of the imaging equipment was included. Preferably additional information is provided.

4 DB, 5 input section, 6 output section, 7 bus, 8 control section, 10 preprocessing means, 11 deteriorated partial preliminary extraction means, 12 deteriorated partial extraction classification means, 14 deteriorated area verification evaluation means, 15 display processing means, 100 Steel pipe internal corrosion analyzer

Claims (4)

A steel tube corrosion analysis system you analyze the image in the steel tube,
Based on the central portion of the steel pipe estimated by the O-flop optical flow to estimate the center position of the steel pipe, the developed image forming means for forming a developed image obtained by planar development of the image in each frame on the basis of the center position ,
Bonded based pre Symbol expand image to the moving position of the camera, and an image-clad combined means for creating the developed image of the whole of the steel pipe,
Before Symbol steel internal corrosion analysis apparatus characterized by and Turkey to extract the degradation region to the entire of the developed image of the steel pipe. In the steel pipe internal corrosion analysis device according to claim 1,
The steel pipe internal corrosion analysis apparatus , wherein the center position is estimated based on the opening of the steel pipe estimated from the brightness of the image and the central portion . Expand and an image forming means and the image-clad combined unit, a steel internal corrosion analysis method by the steel tube corrosion analysis system you analyze the image in the steel tube,
The developed image creation means, based on the center portion of the steel pipe estimated by the O-flop optical flow to estimate the center position of the steel pipe, the expanded image obtained by planar development of the image in each frame on the basis of the center position A step to create,
A step wherein the image-clad combined unit, bonded based pre Symbol expand image to the moving position of the camera, to create the developed image of the whole of the steel pipe,
A step that to extract the degradation region to the entire of the developed image prior Symbol steel,
Steel tube corrosion analysis how to characterized in that have a. In the steel pipe internal corrosion analysis method according to claim 3,
The opening of the steel pipe which is estimated from the luminance of the image, on the basis of the central portion, the steel pipe corrosion analyzing how to characterized in that have a step of estimating the center position.

Images