JP2022110266A - Image inspection device, inspection image display device, image inspection method, and image inspection program - Google Patents

Abstract

To provide an image inspection device capable of effectively improving the image quality of an inspection image.SOLUTION: An image inspection device 100 includes: an image quality abnormality type registration unit 131 for registering types of image quality abnormality that can occur in an inspection image obtained by photographing an inspection object; an inspection image acquisition unit 110 for acquiring the inspection image; an image quality abnormality type estimation unit 120 for estimating a type of image quality abnormality having occurred in the inspection image among the registered image quality abnormality types based on the acquired inspection image; and an image quality correction unit 140 for generating a corrected inspection image by subjecting the acquired inspection image to image quality correction processing according to the type of estimated image quality abnormality.SELECTED DRAWING: Figure 3

Description

The present invention relates to a technology for improving image quality of an inspection image obtained by imaging an inspection object.

2. Description of the Related Art A technique of using an inspection image obtained by photographing an inspection object when inspecting the inspection object is known. For example, Patent Literature 1 discloses a technique of inspecting damage based on an inspection image of a civil engineering structure as an inspection object.

JP 2016-21725 A

When inspecting large structures such as boilers, coke ovens, construction machinery, other plants, bridges, dams, buildings, etc., the camera that captures the inspection image is a robot, drone, etc. that can move along the inspection surface. Attached to other moving bodies. Alternatively, a person takes an image of an inspection object while moving with a camera. In such a case, there is a problem that the image quality of the inspection image captured by the moving camera is not stable. In addition, large structures are often placed in harsh environments with many changes, and there is also the problem that disturbances caused by the environment deteriorate the image quality of inspection images. As described above, the image quality of the inspection image can deteriorate due to various factors.

SUMMARY OF THE INVENTION The present invention has been made in view of such circumstances, and an object of the present invention is to provide an image inspection apparatus and an inspection image display apparatus capable of effectively improving the image quality of an inspection image.

In order to solve the above-described problems, an image inspection apparatus according to one aspect of the present invention includes an image quality abnormality type registration unit that registers types of image quality abnormality that can occur in an inspection image obtained by photographing an inspection object; Based on this, an image quality abnormality type estimating unit for estimating the type of image quality abnormality occurring in the inspection image among the registered image quality abnormality types, an image quality correction unit that generates a corrected inspection image that has been subjected to image quality correction processing.

According to this aspect, a corrected inspection image with improved image quality can be generated by the image quality correction processing according to the estimation result of the type of image quality abnormality occurring in the photographed inspection image.

Another aspect of the present invention is an inspection image display device. This apparatus includes an image quality abnormality type registration section for registering types of image quality abnormality that can occur in an inspection image obtained by photographing an inspection object, a weight display section for displaying the weight of each registered type of image quality abnormality, and a Individual corrected images are generated by subjecting inspection images to individual correction processing for individually correcting image quality abnormalities of each registered type, and a composite image obtained by weighting them with the weight for each type is used as a corrected inspection image. and a corrected inspection image display unit for displaying.

According to this aspect, the corrected inspection image is a combined image obtained by weighting the individual corrected images obtained by individually correcting each type of image quality abnormality that can occur in the inspection image with the weight displayed on the weight display section. Therefore, even when a plurality of types of image quality abnormalities occur simultaneously in the inspection image, the image quality can be effectively improved.

Yet another aspect of the present invention is an image inspection method. This method includes an image quality abnormality type registration step of registering a type of image quality abnormality that can occur in an inspection image obtained by photographing an object to be inspected; An image quality abnormality type estimation step for estimating the type of image quality abnormality that has occurred; and an image quality correction for generating a corrected inspection image by subjecting the photographed inspection image to image quality correction processing according to the estimated image quality abnormality type. and a step.

Any combination of the above constituent elements, and any conversion of expressions of the present invention into methods, devices, systems, recording media, computer programs, etc. are also effective as embodiments of the present invention.

According to the present invention, it is possible to effectively improve the image quality of inspection images.

Fig. 2 is a side view schematically showing an in-furnace observation device for a coke oven; It is a figure which shows the inside of a coke oven typically. 1 is a functional block diagram of an image inspection apparatus according to an embodiment; FIG. FIG. 5 is a diagram showing a display example on a display screen by a display control unit;

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments for carrying out the present invention will be described in detail with reference to the drawings. In the description and drawings, the same or equivalent components, members, and processes are denoted by the same reference numerals, and overlapping descriptions are omitted as appropriate. The scales and shapes of the illustrated parts are set for convenience in order to facilitate explanation, and should not be construed as limiting unless otherwise specified. The embodiments are illustrative and do not limit the scope of the invention in any way. Not all features or combinations thereof described in the embodiments are essential to the invention.

The image inspection apparatus and inspection image display apparatus of the present invention can be used for inspection of arbitrary inspection objects. Therefore, the inspection object is not particularly limited, but in this embodiment, an example in which the oven wall of the coke oven is the inspection object will be described first. Other examples of inspection objects will be described later.

FIG. 1 is a side view schematically showing an in-furnace observation apparatus in which an image inspection apparatus and an inspection image display apparatus of this embodiment are used. This furnace interior observation device is used to observe the inside of a coking chamber of a coke oven (hereinafter also simply referred to as coke oven). FIG. 2 is a diagram schematically showing the inside of the coke oven.

A coke oven 10 is a constricted oven in which a pair of oven walls 11 made of bricks are provided facing each other. Each oven wall 11 extends from an oven inlet 12 on one side of the coke oven 10 to an oven outlet 13 on the other side, and has a total length of, for example, ten and several meters. The interval between the facing furnace walls 11 is, for example, several tens of centimeters. The height from the bottom 15 of the coke oven 10 to the ceiling 14 is, for example, several meters.

The extrusion device 20 shown in FIG. 1 reciprocates within the coke oven 10 . In the outward path, the extrusion device 20 is inserted into the coke oven 10 from the oven entrance 12 and pushes out the coke C produced by carbonization in the coke oven 10 to the oven exit 13 . On the return trip, the extruder 20 moves back through the coke oven 10 from the oven outlet 13 to the oven inlet 12 . The extrusion device 20 comprises a push plate 21 and a beam 22, and the beam 22 connects the push plate 21 to a driving device (not shown). This driving device allows the push plate 21 to move freely between the furnace inlet 12 and the furnace outlet 13 of the coke oven 10 . Since the pushing plate 21 has the same shape as the cross section of the coke oven 10, the coke C can be pushed out by moving the pushing plate 21.例文帳に追加

A camera 30 as a photographing device for photographing an inspection image is attached to an extrusion device 20 as a moving body for moving the camera 30, and moves together with the extrusion device 20 in the coke oven 10 between the furnace inlet 12 and the furnace outlet 13. The furnace wall 11 is photographed continuously while The camera 30 may be a still camera that continuously captures still images, or a video camera that captures moving images. The camera 30 is attached to the back surface of the push plate 21 or to a support base installed behind the push plate 21 .

The cameras 30 include two cameras attached to the left and right furnace walls 11, respectively, and can photograph front view images of the furnace walls 11 on both the left and right sides. In order to photograph the left and right furnace walls 11 as a whole, the two cameras may photograph while slightly changing the angle in the vertical direction. In addition, the camera 30 may be mounted so as to face the direction opposite to the direction of extrusion of the coke C by the extrusion device 20 (rightward in FIG. 1) so that the view is not obstructed by the push plate 21 or the coke C. In this case, the camera 30 is installed facing the furnace entrance 12 and can photograph the furnace walls 11 on both the left and right sides as perspective images.

In order to protect the camera 30 from the high-temperature environment (for example, 1000° C. or higher) inside the coke oven 10, heat countermeasures are taken, such as housing it in a heat-resistant housing or a cooling box.

In the in-furnace observation apparatus configured as described above, the inspection image captured by the camera 30 may have various image quality abnormalities as described below.

Abnormal image quality caused by the photographing state of the camera 30 Focus or defocus (so-called blurring): Caused by variation in the distance between the moving camera 30 and the surface to be inspected Posture of the inspection object between multiple still images or within a moving image variation: due to variation in the posture of the moving camera 30 Abnormal viewpoint (non-front view image): placement of the camera 30 with respect to the surface to be inspected (example of the oblique view image above) and change in the posture of the moving camera 30 caused by fluctuations

Abnormal image quality caused by the environment in which the camera 30 is placed ・Noise: Caused by the reflection of carbon chips scattered in the coke oven 10, malfunction of the camera 30 due to high temperature, etc. ・Haze and cloudiness: Fire in the coke oven 10 , due to reflection of smoke, steam, etc. ・Light abnormality (diffused reflection, backlight, etc.): due to light fluctuation in the coke oven 10 in a red-hot environment

FIG. 3 is a functional block diagram of the image inspection apparatus 100 according to the embodiment. The image inspection apparatus 100 includes an inspection image acquisition unit 110, an image quality abnormality type estimation unit 120, an image quality abnormality estimation setting unit 130, an image quality correction unit 140, an abnormality detection unit 150, an abnormality detection setting unit 160, and an inspection result. An image generator 170 and a display controller 180 are provided.

The camera 30 and the image memory 31 constitute an image input unit for inputting an inspection image obtained by photographing the furnace wall 11 as the surface to be inspected of the inspection object to the image inspection apparatus 100 . The display screen 40 displays the processing contents of the image inspection apparatus 100 . The operation unit 50 includes a touch panel integrated with the display screen 40 and input devices such as a keyboard and a mouse that are separate from the display screen 40, and generates various control information for the image inspection apparatus 100 based on user operations. A part or all of the operations by the operation unit 50 may be programmed so that the computer autonomously performs them.

The image memory 31 stores inspection images of the furnace wall 11 continuously photographed by the camera 30 . The image memory 31 may be a built-in memory of the camera 30 or a general-purpose removable medium such as a memory card. Alternatively, storage outside the coke oven 10 that can communicate with the camera 30 by wire or wirelessly may be used. The image inspection apparatus 100 executes each process described later on the inspection image group stored in the image memory 31 . Here, if the camera 30 and the image inspection apparatus 100 can communicate by wire or wirelessly, and if the image inspection apparatus 100 can acquire and process the inspection images captured by the camera 30 in real time, all the inspection images are necessarily stored in the image memory. 31, and the image memory 31 can be unnecessary or have a small capacity.

The inspection image acquisition unit 110 acquires an inspection image obtained by photographing an inspection object from the camera 30 or the image memory 31 .

Based on the inspection image acquired by the inspection image acquisition unit 110, the image quality abnormality type estimation unit 120 estimates the type of image quality abnormality occurring in the inspection image. The image quality abnormality estimation setting unit 130 includes an image quality abnormality type registration unit 131 and a training data providing unit 132, and under the operation of the operation unit 50, the image quality abnormality type estimation unit 120 can estimate the image quality abnormality type based on the inspection image. set as The image quality abnormality type registration unit 131 registers types of image quality abnormality that can occur in the inspection image. Examples of types of image quality abnormality that can occur when the oven wall 11 of the coke oven 10 is the inspection object have been described above. In the following, among these types, four types of image quality abnormality, namely, "bokeh", "slanted (perspective image)", "noise", and "haze", which are registered by the image quality abnormality type registration unit 131, are assumed to occur in the inspection image. It is assumed that there is

The image quality abnormality type estimating unit 120 detects the possibility that each of the “bokeh”, “oblique”, “noise”, and “haze” registered by the image quality abnormality type registration unit 131 has occurred in the inspection image acquired by the inspection image acquisition unit 110. output an estimate of the sex. Each estimated value is represented by W1, W2, W3, and W4 below. W5 represents an estimated value representing the possibility that the inspection image is of the “normal” type in which none of the above image quality abnormalities occurs. V denotes an estimated vector having these estimated values W1 to W5 as components. That is, V= ^t (W1, W2, W3, W4, W5). It is also assumed that the estimated values W1 to W5 have been normalized so that their sum is 1.0. Therefore, W1+W2+W3+W4+W5=1.0, and each value of W1-W5 represents the possibility or probability that each type occurs in the inspection image. For example, when V = ^t (0.5, 0.2, 0.1, 0.0, 0.2), the possibility of "bokeh" occurring is 50% (W1 = 0.5), and " Possibility of occurrence of "oblique" is 20% (W2=0.2), possibility of occurrence of "noise" is 10% (W3=0.1), possibility of occurrence of "haze" is 0 % (W4=0.0), and the probability of occurrence of "normal" is 20% (W5=0.2). The inspection image acquired by the inspection image acquisition unit 110 may be divided into a plurality of arbitrary partial images, and the image quality abnormality type estimation unit 120 may estimate the image quality abnormality type for each partial image. At this time, an estimated vector V representing the possibility of occurrence of "bokeh", "oblique", "noise", "haze", and "normal" is generated for each partial image.

The image quality abnormality type estimation unit 120 performs machine learning for generating the estimated vector V based on the inspection image acquired by the inspection image acquisition unit 110 . Although any machine learning method can be used, for example, deep learning using a convolutional neural network with convolutional layers and fully connected layers can be used. The training data providing unit 132 in the image quality abnormality estimation setting unit 130 provides training data for machine learning of the image quality abnormality type estimation unit 120 . The training data covers all types of image quality abnormalities registered by the image quality abnormality type registration unit 131 and a large number of inspection image examples in which each image quality abnormality actually occurs in various places and in various modes. The image quality abnormality type estimating unit 120 machine-learns these comprehensive training data during the initial setup and maintenance of the image inspection apparatus 100 to generate an estimated vector based on the inspection image acquired by the inspection image acquiring unit 110. V can be generated.

The image quality correction unit 140 generates a corrected inspection image by subjecting the inspection image acquired by the inspection image acquisition unit 110 to image quality correction processing according to the type of image quality abnormality estimated by the image quality abnormality type estimation unit 120. . Specifically, the image quality correction unit 140 performs individual correction processing for individually correcting each type of image quality abnormality estimated by the image quality abnormality type estimation unit 120 on the inspection image acquired by the inspection image acquisition unit 110. The individual corrected images are generated, and a combined image is generated as a corrected inspection image by weighting them with the estimated values W1 to W5 for each type. The image quality correcting section 140 includes an individual corrected image generating section 141, a weighting section 142, and an image synthesizing section 143 in order to execute a series of these processes.

The individual corrected image generation unit 141 calculates the image quality of each type of “bokeh”, “oblique”, “noise”, and “haze” estimated by the image quality abnormality type estimation unit 120 for the inspection image acquired by the inspection image acquisition unit 110. An individual corrected image is generated by performing individual correction processing for individually correcting anomalies. Individual correction processing is performed using known image correction techniques. "Bokeh" can be corrected by a deblurring technique called deblurring. The “oblique” oblique image can be corrected to a front view image by viewpoint conversion technology using position information and posture information of the camera 30 . The "noise" can be corrected by a noise reduction technique according to the characteristics of the noise. "Haze" can be corrected by a haze/fog removal processing technique called dehazing. It should be noted that other types of image quality anomalies that are not registered in the example of the present embodiment can be corrected as follows. Regarding "variation in the posture of the inspection object between multiple still images or within a video", by adjusting the position and angle of each image that constitutes multiple still images or videos, the inspection object can be positioned at a fixed position within the screen. It can be displayed in a fixed posture. "Abnormalities in light" can be corrected using a known image correction technique according to the mode of abnormalities such as diffused reflection and backlight.

The estimated values generated by the image quality abnormality type estimation unit 120 for "bokeh", "oblique", "noise", "haze", and "normal" are W1 to W5, respectively. , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , . Here, since a special individual correction process is not required for "normal", the image I5 is the inspection image itself acquired by the inspection image acquiring unit 110. FIG. Note that the notations I1 to I5 do not necessarily represent only the entire image (all pixels), but may represent one or more pixels forming part of the image as long as the context permits. For example, when the inspection image is divided into a plurality of arbitrary partial images as described above, and the estimated vector V= ^t (W1, W2, W3, W4, W5) is generated for each partial image, I1 to I5 are also Represents an individual corrected image for a partial image.

The weighting unit 142 weights the individual corrected images I1 to I5 generated by the individual corrected image generating unit 141 by the estimated values W1 to W5 generated by the image quality abnormality type estimating unit 120 . That is, the product W1×I1 for “bokeh”, the product W2×I2 for “oblique”, the product W3×I3 for “noise”, the product W4×I4 for “haze”, and the product W4×I4 for “normal” is calculated as the product W5×I5.

The image synthesizing unit 143 synthesizes the individual corrected images I1 to I5 weighted by the estimated values W1 to W5 in the weighting unit 142 to generate a corrected inspection image. Specifically, the image synthesizing unit 143 synthesizes the corrected inspection image by adding the five products calculated by the weighting unit 142 . That is, assuming that the corrected inspection image is I, I=W1*I1+W2*I2+W3*I3+W4*I4+W5*I5. Here, by weighting the individual corrected images I1 to I5 with the probabilities W1 to W5 of occurrence of each type of image quality abnormality, the image quality can be effectively balanced in a well-balanced manner even when a plurality of image quality abnormalities occur simultaneously. It can be improved.

Note that the image synthesizing unit 143 may generate the corrected inspection image I by selecting a part of the five products calculated by the weighting unit 142 . At this time, it is assumed that renormalization is performed so that the sum of the estimated values for the selected products becomes 1.0. For example, the image synthesizing unit 143 may select one product with the largest estimated value. In the above example, the estimated value W1 of "bokeh" is the largest, so it is selected. Therefore, the corrected inspection image I becomes W1'×I1. W1' is the estimated value after renormalization, in this case W1'=1.0. Alternatively, the image synthesizing unit 143 may select, for example, three products in descending order of estimated values. In the above example, the three estimated values W1, W2, and W5 of "bokeh", "oblique", and "normal" are large. Therefore, the corrected inspection image I becomes W1'.times.I1+W2'.times.I2+W5'.times.I5. W1', W2', W5' are estimates after renormalization, where W1'+W2'+W3'=1.0. Specifically, W1'=0.5/0.9, W2'=0.2/0.9, and W3'=0.2/0.9. where 0.9 is the sum of pre-renormalized estimates W1, W2, and W5.

Based on the corrected inspection image I generated by the image quality correcting unit 140 , the abnormality detection unit 150 detects an abnormality of the inspection object appearing in the corrected inspection image I. The anomaly detection setting unit 160 includes an anomaly type registration unit 161 and a training data providing unit 162, and allows the anomaly detection unit 150 to detect an anomaly of the inspection object shown in the corrected inspection image I under the operation of the operation unit 50. set. The abnormality type registration unit 161 registers the type of abnormality that can occur in the inspection object. Types of abnormalities that can occur when the furnace wall 11 of the coke oven 10 is the object to be inspected include spalling in which the furnace wall 11 is destroyed due to distortion due to rapid heating or rapid cooling, and carbon originating from coal, which is the raw material of coke. Adhesion to the wall 11 and deterioration of joints of bricks forming the furnace wall 11 are exemplified.

The abnormality detection unit 150 performs machine learning for detecting an abnormality of the inspection object of the type registered by the abnormality type registration unit 161 based on the corrected inspection image I generated by the image quality correction unit 140 . Although any machine learning method can be used, for example, deep learning using a convolutional neural network with convolutional layers and fully connected layers can be used. A training data providing unit 162 in the anomaly detection setting unit 160 provides training data for machine learning of the anomaly detection unit 150 . The training data includes all types of abnormalities of the inspection object registered by the abnormality type registration unit 161, and a large number of inspection image examples in which each abnormality actually occurs at various locations of the inspection object in various modes. cover. During the initial setup and maintenance of the image inspection apparatus 100, the abnormality detection unit 150 machine-learns these comprehensive training data to determine the abnormality type based on the corrected inspection image I generated by the image quality correction unit 140. Abnormalities of inspection objects of types registered in the registration unit 161 can be detected. By using the corrected inspection image I whose image quality has been improved by the image quality correction unit 140, the abnormality detection unit 150 can detect an abnormality of the inspection object with high accuracy.

Based on the corrected inspection image I, the inspection result image generation unit 170 generates an inspection result image that highlights the abnormality of the inspection object detected by the abnormality detection unit 150 . The mode of highlighting is arbitrary, but for example, the brightness of the detected abnormal portion in the corrected inspection image I may be increased, the color may be changed, the abnormal portion may be surrounded by a line, or a note or note may be displayed.

The display control unit 180 constitutes the main part of the inspection image display apparatus of the present invention, and causes the display screen 40 to display the processing contents of the image inspection apparatus 100 . As will be described later, the user can operate the content displayed on the display screen 40 by the display control unit 180 using the operation unit 50 .

FIG. 4 shows an example of display on the display screen 40 by the display control unit 180. As shown in FIG. On the left side of the display screen 40, an inspection image display section 41, a corrected inspection image display section 42, and an inspection result image display section 43 are provided in this order from the top. The inspection image display unit 41 displays the inspection image before image quality correction by the image quality correcting unit 140, that is, the inspection image acquired by the inspection image acquisition unit 110 as it is. The corrected inspection image display unit 42 displays the corrected inspection image generated by the image quality correcting unit 140 performing image quality correction processing on the inspection image. The inspection result image display unit 43 displays an inspection result image emphasizing abnormalities of the inspection object appearing in the corrected inspection image.

In the illustrated example, the inspection image displayed on the inspection image display unit 41 is selected from the types of image quality abnormality registered in the image quality abnormality type registration unit 131, namely, "bokeh", "oblique", "noise", and "haze". The image is unclear due to the strong influence of The corrected inspection image displayed on the corrected inspection image display unit 42 is a sharp image in which "bokeh" and other image quality abnormalities are corrected by the image quality abnormality type estimation unit 120 and the image quality correction unit 140 . The inspection result image displayed on the inspection result image display unit 43 is based on the corrected inspection image, and is an image in which various abnormalities 431, 432, and 433 detected by the abnormality detection unit 150 are highlighted.

These three types of images are obtained or generated autonomously by the image inspection apparatus 100. By displaying them side by side on the display screen 40, it is possible to effectively assist the user's visual inspection, thereby increasing the inspection accuracy. can be improved. For example, when the user visually compares the uncorrected inspection image displayed on the inspection image display unit 41 and the corrected inspection image displayed on the corrected inspection image display unit 42, it is found that the image quality of the corrected inspection image is sufficiently improved. When it is determined that there is no image quality improvement processing performed by the image quality abnormality type estimation unit 120, the image quality abnormality estimation setting unit 130, and the image quality correction unit 140 can be changed by operating the operation unit 50, as will be described later. Similarly, when the user visually compares the corrected inspection image displayed on the corrected inspection image display unit 42 and the inspection result image displayed on the inspection result image display unit 43, abnormalities 431, 432, and 433 are found in the inspection result images. If it is determined that the highlighting is not correctly performed, the abnormality highlighting process executed by the abnormality detection unit 150, the abnormality detection setting unit 160, and the inspection result image generation unit 170 is performed by operating the operation unit 50, as will be described later. can be changed.

A weight display section 44 is provided in the upper center of the display screen 40 . The weight display unit 44 displays weights of the types of image quality abnormality registered in the image quality abnormality type registration unit 131, namely, "bokeh", "oblique", "noise", "haze", and "normal". These weights are, for example, the estimated values W1 to W5 output by the image quality abnormality type estimation unit 120, and the weighting unit 142 when the image quality correction unit 140 generates the corrected inspection image displayed on the corrected inspection image display unit 42. used for weighting operations in

The weights displayed on the weight display unit 44 are not limited to the estimated values W1 to W5 output by the image quality abnormality type estimation unit 120, and may be arbitrary numerical values (however, the sum of W1 to W5 may be set to 1.0 with the help of a computer). shall be automatically normalized so that ). For example, it may be a default value preset in the image inspection apparatus 100, or an arbitrary value input by the user via the operation unit 50. FIG. Each weight displayed in the weight display section 44 can be changed by the user via the operation section 50 (weight change section). Therefore, when the user determines that the image quality of the corrected inspection image displayed in the corrected inspection image display section 42 is not sufficiently improved, the weight displayed in the weight display section 44 is appropriately changed by itself, It is also possible to improve the image quality while confirming the corrected inspection image after change on the corrected inspection image display unit 42 in real time.

When the user selects a text or a weight area representing each type of "bokeh", "oblique", "noise", "haze", and "normal" displayed in the weight display section 44 via the operation section 50, each type can be displayed. It is also possible to switch to another screen displaying the individual corrected images I1 to I5. At this time, the text and weight areas representing each type constitute an individual corrected image display section for displaying the individual corrected image. It is preferable to display all the individual corrected images I1 to I5 side by side on the display screen of the individual corrected images I1 to I5. By visually comparing and examining the individual corrected images I1 to I5, the user can effectively examine the weights to be set in the weight display section 44 while confirming the effect of each individual correction process.

An attribute information display section 45 provided below the weight display section 44 displays various attribute information of the inspection image. For example, as shown in the figure, the name of the object to be inspected (fourth furnace wall), the position coordinates (X, Y, Z) of the photographing location, and the file name of the inspection image (img.jpg) are displayed. be done.

Below the attribute information display section 45, an image quality improvement model selection section 46 and an abnormality detection model selection section 47 are provided.

The image quality improvement model selection unit 46 selects a model for image quality improvement processing based on the operation of the operation unit 50 by the user. A model of the image quality improvement process is configured by combining the processes of the image quality abnormality type estimation unit 120 , the image quality abnormality estimation setting unit 130 , and the image quality correction unit 140 . A plurality of different image quality improvement models are registered in advance in the image inspection apparatus 100 by changing the processing parameters and algorithms of each unit.

Regarding the processing of the image quality abnormality type estimation unit 120, the target for generating the estimated vector V (the entire inspection image or each partial image), the algorithm for generating the estimated vector V, and the like can be changed. Regarding the processing of the image quality abnormality estimation setting unit 130 , the image quality abnormality type registered by the image quality abnormality type registration unit 131 and displayed on the display unit 44 can be changed. The processing of the image quality correction unit 140 includes an algorithm for generating an individual corrected image by the individual corrected image generation unit 141, a selection criterion for the estimated values W1 to W5 used by the weighting unit 142 for weighting calculation, and an image composition unit 143 for image composition. Algorithms etc. can be changed. Also, the image quality improvement model selectable by the image quality improvement model selection section 46 may specify a typical combination ^t (W1, W2, W3, W4, W5) of weights that can overwrite the weight display section 44 .

The result of switching the image quality improvement model by the image quality improvement model selection unit 46 is immediately reflected in the corrected inspection image displayed on the corrected inspection image display unit 42 by pressing the update button 48 . Accordingly, the inspection result image displayed on the inspection result image display section 43 is also updated. Further, when the weight is changed due to the change of the image quality improvement model, the weight of the weight display section 44 is also changed at the same time. Thus, the image quality improvement model selection unit 46 constitutes a weight change unit that changes the weight of each type of image quality abnormality.

The anomaly detection model selection unit 47 selects an anomaly detection processing model based on the operation of the operation unit 50 by the user. A model of the abnormality detection process is configured by combining the processes of the abnormality detection unit 150 , the abnormality detection setting unit 160 , and the inspection result image generation unit 170 . A plurality of different abnormality detection models are registered in advance in the image inspection apparatus 100 by changing the processing parameters and algorithms of each unit.

As for the processing of the abnormality detection unit 150, an abnormality detection algorithm or the like for each type can be changed. In the processing of the abnormality detection setting unit 160, the type of abnormality of the inspection object registered by the abnormality type registration unit 161 can be changed. As for the processing of the inspection result image generation unit 170, it is possible to change the mode of highlighting the detected abnormality. The result of switching the abnormality detection model by the abnormality detection model selection unit 47 is immediately reflected in the inspection result image displayed on the inspection result image display unit 43 by pressing the update button 48 .

When the user presses the image quality improvement model selection unit 46 or the abnormality detection model selection unit 47 via the operation unit 50, a plurality of selection candidate models and corrected inspection images or inspection images obtained when these models are applied are displayed. It may be possible to transition to a selection screen displaying the result images side by side. At this time, the image quality improvement model selection unit 46 or the abnormality detection model selection unit 47 constitutes a model selection screen display unit that displays a selection screen for the image quality improvement model or the abnormality detection model. On such a model selection screen, the user can effectively examine the model to be selected while confirming the effect of each model.

When the save button 491 is pressed, the processing contents displayed on the display screen 40 are saved in the memory of the image inspection apparatus 100 . When a pair of left and right inspection image selection buttons 492 is pressed, the inspection image displayed on the inspection image display section 41 is switched, and other portions of the display screen 40 are updated accordingly.

The present invention has been described above based on the embodiments. It should be understood by those skilled in the art that the embodiments are examples, and that various modifications can be made to combinations of each component and each treatment process, and such modifications are also within the scope of the present invention.

In the embodiment, the coke oven 10 has been described as an example of the inspection object of the image inspection apparatus 100, but the inspection object is not limited to this. For example, the object to be inspected may be boilers, various industrial machines such as construction machines (including coke ovens), social infrastructures such as bridges, various industrial structures such as environmental plants and water treatment facilities, or other industrial equipment. The inspected surface may be an internal or external inspected surface of such industrial equipment. In addition, the camera for photographing the group of images of the surface to be inspected can be attached to a movable body of any configuration that can move along the surface to be inspected. For example, the camera may be attached to a flying object such as a so-called drone.

In an inspection object other than the coke oven 10, an inspection image captured by the camera 30 may have the following image quality abnormalities. First, image quality abnormalities caused by the imaging state of the camera 30, such as focus or defocus (bokeh), fluctuations in the posture of the inspection object between a plurality of still images or within a moving image, and abnormal viewpoints (non-front view images) are , arise for the same reasons as in the embodiment for coke oven 10 . Next, the image quality abnormality caused by the environment in which the camera 30 is placed varies depending on the environment in which the inspection object is placed. For example, when the inspection object is installed outdoors, noise is generated in the inspection image due to weather such as rain and snow. Similarly, dust, electromagnetic waves, etc. during strong winds also cause noise. When shooting in dark conditions such as at night, white spot noise is also generated due to the dark current of the image sensor of the camera. In addition, fog, haze, photochemical smog, and the like cause blurring and dulling of inspection images. Changes in outdoor sunlight conditions cause light abnormalities such as diffused reflection and backlight in inspection images.

Note that the functional configuration of each device described in the embodiments can be realized by hardware resources, software resources, or cooperation between hardware resources and software resources. Processors, ROMs, RAMs, and other LSIs can be used as hardware resources. Programs such as operating systems and applications can be used as software resources.

10 coke oven, 11 oven wall, 20 extrusion device, 30 camera, 40 display screen, 41 inspection image display unit, 42 corrected inspection image display unit, 43 inspection result image display unit, 44 weight display unit, 46 image quality improvement model selection unit , 50 operation unit 100 image inspection device 110 inspection image acquisition unit 120 image quality abnormality type estimation unit 130 image quality abnormality estimation setting unit 131 image quality abnormality type registration unit 140 image quality correction unit 141 individual corrected image generation unit 142 Weighting unit 143 Image synthesis unit 150 Abnormality detection unit 160 Abnormality detection setting unit 170 Inspection result image generation unit 180 Display control unit.

Claims (11)

an image quality abnormality type registration unit that registers types of image quality abnormality that can occur in an inspection image obtained by photographing an inspection object;
an image quality abnormality type estimating unit for estimating, based on the photographed inspection image, a type of image quality abnormality occurring in the inspection image among the registered image quality abnormality types;
An image inspection apparatus comprising: an image quality correcting unit configured to generate a corrected inspection image obtained by subjecting the photographed inspection image to image quality correction processing corresponding to the estimated type of image quality abnormality. The type of image quality abnormality registered by the image quality abnormality type registration unit includes an image quality abnormality caused by an imaging state of an imaging device that captures the inspection image and an image quality abnormality caused by an environment in which the imaging device is placed. 2. The image inspection apparatus according to 1. The image quality abnormality type estimating unit outputs an estimated value representing the possibility that each of the registered types of image quality abnormality occurs in the photographed inspection image,
The image quality correcting unit generates an individual corrected image by performing individual correction processing for individually correcting the estimated image quality abnormality of each type on the photographed inspection image, and correcting the image quality abnormality of each type. 3. The image inspection apparatus according to claim 1, wherein a synthesized image weighted by said estimated value is generated as said corrected inspection image. The image inspection apparatus according to any one of claims 1 to 3, further comprising an abnormality detection unit that detects an abnormality of the inspection object appearing in the corrected inspection image based on the corrected inspection image. 5. The image inspection apparatus according to claim 4, further comprising an inspection result image generation unit that generates an inspection result image highlighting the detected abnormality based on the corrected inspection image. an image quality abnormality type registration unit that registers types of image quality abnormality that can occur in an inspection image obtained by photographing an inspection object;
a weight display unit for displaying the weight of each registered type of image quality abnormality;
Individually corrected images are generated by subjecting the photographed inspection images to individual correction processing for individually correcting the image quality abnormality of each of the registered types, and combining them by weighting them with the weights for the respective types. An inspection image display device comprising: a corrected inspection image display unit that displays an image as a corrected inspection image. 7. The inspection image display device according to claim 6, further comprising a weight changing unit that changes the weight of each type of image quality abnormality. 8. The inspection image display device according to claim 6, further comprising an inspection image display unit that displays the inspection image before correction. The inspection image display device according to any one of claims 6 to 8, further comprising an inspection result image display unit that displays an inspection result image highlighting an abnormality of the inspection object appearing in the corrected inspection image. an image quality abnormality type registration step of registering the type of image quality abnormality that can occur in an inspection image obtained by photographing an inspection object;
an image quality abnormality type estimation step of estimating, based on the photographed inspection image, a type of image quality abnormality occurring in the inspection image among the registered image quality abnormality types;
and an image quality correction step of generating a corrected inspection image by subjecting the photographed inspection image to image quality correction processing corresponding to the estimated type of image quality abnormality. an image quality abnormality type registration step of registering the type of image quality abnormality that can occur in an inspection image obtained by photographing an inspection object;
an image quality abnormality type estimation step of estimating, based on the photographed inspection image, a type of image quality abnormality occurring in the inspection image among the registered image quality abnormality types;
and an image quality correction step of generating a corrected inspection image by subjecting the photographed inspection image to image quality correction processing corresponding to the estimated type of image quality abnormality.

Images