JP5956935B2 - Image processing method and image processing apparatus - Google Patents

Description

  The present invention relates to an image processing method and an image processing apparatus for observing an object including a red hot part of approximately 1000K or more.

  Refractory bricks and irregular refractories used in various steelmaking furnaces and molten steel pans are gradually damaged with operation. For this reason, it is necessary to observe the damaged state of the refractory brick and the irregular refractory (hereinafter collectively referred to as “refractory”) and repair the damaged portion.

  In order to observe the damaged state of the refractory, it is necessary to remove intense heat radiation from the high temperature part. As a technique for removing heat radiation light, a technique is known in which a diffused laser beam having a wavelength of 532 nm, which has a relatively low black body radiation density, is irradiated on the inner wall surface of the furnace, and reflected light from the inner wall surface of the furnace is photographed with a CCD camera. (For example, refer to Patent Document 1).

  In addition, the image is visually recognized by irradiating the subject with radiation, detecting the radiation transmitted through the subject with an image intensifier, and subjecting the detection signal to RGB separation processing and subtraction of the background reference image. A technique for improving the property is known (see, for example, Patent Document 2).

  However, the technique of Patent Document 1 requires a laser light source. For this reason, in order to implement | achieve an apparatus structure like patent document 1, cost and an effort are required.

  Moreover, the technique of patent document 2 is a technique of grasping | ascertaining the breakage | damage of a substance, etc. using a X ((gamma)) ray, Comprising: A measuring object is a human body etc. For this reason, the thing containing the red hot part is not made into the measuring object. Further, the background removal technique described in Patent Document 2 is a process of subtracting a reference image taken in advance or after the background noise removal, and a process of removing a background signal obtained from the taken image. is not.

  Furthermore, the RGB separation process described in Patent Document 2 improves the dynamic range when used in combination with a color image intensifier that outputs RGB signals in different states at the same time according to the ratio of X-ray transmission. The effect is obtained, and the same effect is not exhibited except in the device configuration.

JP 2008-32396 A JP 2004-294287 A

  The problem to be solved by the present invention is to provide an image processing technique capable of easily and reliably observing an object including a red hot part.

According to an aspect of the present invention, a photographing process for color photographing an object including a red hot part, and a separation for generating a separated image by separating a color photographed image photographed by the photographing process into RGB three components In each of the image generation step and the separated image separated by the separated image generation step, the average luminance value of the pixel group within a predetermined range including the pixel is calculated for each pixel in the separated image, and the calculated average luminance value only contains the background signal removal processing steps subtracted from the luminance value of the pixel, and a combined image generation step of generating a synthesized and the synthesized image separation image processed by the background signal removal processing steps, the synthetic The image generation step calculates an average luminance value before separation, which is an average luminance value of a pixel group within a predetermined range including the pixel, for each pixel in the color photographed image. In addition, the maximum value and the minimum value of the average luminance value before separation of each pixel are calculated. In each pixel, as the average luminance value before separation of the pixel is closer to the maximum value, the composition ratio of the B component separated image becomes R There is provided an image processing method for generating the synthesized image by synthesizing it so as to be larger than the synthesis ratio of the separated images of components .

According to another aspect of the present invention, a photographing unit that performs color photographing of an object including a red hot part and a color photographing image photographed by the photographing unit are separated into RGB three components to generate a separated image. In each of the separated image generating unit and the separated image separated by the separated image generating unit, the average luminance value of the pixel group within the predetermined range including the pixel is calculated for each pixel in the separated image, and the calculated average luminance is calculated. It includes a background signal removal processing means for subtracting a value from the luminance value of the pixel, and a synthesized image generating means for generating a composite image by combining the separate image processed by the background signal elimination processing unit, the composite image The generation means calculates an average luminance value before separation, which is an average luminance value of a pixel group within a predetermined range including the pixel, for each pixel in the color photographed image. The maximum value and the minimum value of the calculated average luminance value before separation of each pixel are calculated. In each pixel, as the average luminance value before separation of the pixel is closer to the maximum value, the composition ratio of the B component separated image is higher. There is provided an image processing apparatus that generates the composite image by combining the R component separation images so as to be larger than the composite ratio .

  According to the present invention, after a color image is separated into three RGB components and a separated image is generated, a background signal based on the average luminance value is removed from the separated image, so that an object including a red hot part Even so, the surface state can be observed easily and reliably. For example, by referring to an image obtained by removing the background signal from the separated image of the B component, it is possible to clearly confirm the crack peeling at the high temperature portion. Further, for example, by referring to an image obtained by removing the background signal from the separated image of the R component, it is possible to clearly confirm the crack peeling at the low temperature portion. Furthermore, since no special equipment is required for the image processing of the present invention, excessive cost and labor are not required.

It is a block diagram which shows the structural example of the image processing apparatus which enforces the image processing method of this invention. It is a flowchart which shows each process of the image processing method by the image processing apparatus of FIG. Multiple color images It is a figure which shows the relationship between the wavelength in each temperature calculated | required by calculation, and the intensity | strength (spectral radiation divergence) of emitted light. An example of the composition ratio of the B component and the R component when the composite image generation process is executed in a luminance value dependent manner is shown. An example of a color photographed image (after grayscale conversion) obtained in the photographing process is shown. FIG. 6 shows an arrangement of digital cameras that photograph the color photographed image of FIG. 5. FIG. 5 shows a separated image obtained by separating the color photographed image (original image) of FIG. 5 into three RGB components in the separated image generation process, where (a) is an R component separated image (R image) and (b) is a G component separated. Images (G image) and (c) are B component separated images (B images). 7 shows a separated image after the background signal removing process is performed on each separated image of FIG. 7 by the background signal removing process step, where (a) is an R component separated image (R image), and (b) is a G component. The separated images (G image) and (c) show the separated image (B image) of the B component. FIG. 9 shows a composite image obtained by combining the separated images shown in FIG. 9 shows a color image (after gray scale conversion) obtained by combining the original image of FIG. 5 with the composite image of FIG. 9 in the color image generation process.

  FIG. 1 is a block diagram illustrating a configuration example of an image processing apparatus that implements the image processing method of the present invention.

  The image processing apparatus shown in FIG. 1 includes an imaging unit that performs color imaging of an object, and an image processing unit that processes a color image captured by the imaging unit.

  A digital camera can be used as the photographing means, and one or more filters that transmit a predetermined wavelength range can be selectively added to the digital camera.

  The image processing unit includes a memory unit and an image processing unit including a separated image generation unit, a background signal removal processing unit, a composite image generation unit, and a color image generation unit. The memory means stores color photographed image data photographed by the photographing means, provides the color photographed image data to the image processing means, and has a function of saving data after image processing by the image processing means. . Further, the separated image generating means, the background signal removing processing means, the synthesized image generating means, and the color image generating means, which are image processing means, are respectively a separated image generating process, a background signal removing process process, a synthesized image generating process, and A color image generation process is executed. These image processing means can be realized by a CPU, and the image processing unit can be realized by a single computer together with the memory means described above.

  FIG. 2A is a flowchart showing each step of the image processing method by the image processing apparatus of FIG. Hereinafter, each process is demonstrated with reference to FIG.1 and FIG.2A.

(A) Shooting process ((A1) in FIG. 2A)
The photographing means (digital camera) takes a color photograph of an object including a hot red part, and the data of the color photographed image is stored in the memory means. If necessary, an optical filter that transmits a predetermined wavelength range can be added to the imaging means.

  As the optical filter, for example, an optical filter that cuts off a wavelength range of 500 nm or more where emitted light is strongly generated at the time of red heat can be used. By using a color photographed image photographed by the photographing means having such an optical filter, an effect of facilitating observation of the red hot part can be obtained as compared with the case where the optical filter is not used.

(B) Separation image generation process ((B1) in FIG. 2A)
The separated image generating means generates a separated image by separating the color photographed image photographed in the above photographing process into RGB three components.

  The technical significance of separating a color photographed image into three RGB components in the present invention is as follows.

  The intensity of synchrotron radiation emitted from a red-hot object can be calculated by the Planck equation. FIG. 3 shows the relationship between the wavelength at each temperature obtained by calculation and the intensity of the emitted light (spectral radiation divergence). It can be confirmed that the intensity of the emitted light varies depending on the wavelength even at the same temperature.

  In a color digital camera, incident light is separated into red (R), green (G), and blue (B) by a color filter, a prism, or the like provided in the image sensor, and emitted from a red-hot object. When the radiated light is separated into RGB, it can be confirmed from FIG. 3 that the intensity increases in the order of red, green, and blue in the temperature range of 700K to 3000K. For this reason, in a color photographed image, the red image is processed with a signal that emits more intense light than the blue image.

  When it is difficult to observe a red-hot area in a color image that includes a hot red-hot part (red-hot area), the luminance value in the red-hot area is close to the maximum value of the luminance range in the red image. In many cases, there is almost no change in luminance. In the same color photographed image, since the intensity of blue radiated light is weaker than that of red in a blue image, a luminance change due to a difference in the surface state of the object is often obtained in the red heat region. In general, an intermediate characteristic between a red image and a blue image can be obtained with a green image, because the intensity of the emitted green light is located between red and blue.

  On the contrary, the low temperature region in the color photographed image including the red heat region is often easy to grasp the surface state when observed with the red image. The reason why it is difficult to observe the low temperature region is that the luminance in the low temperature region is generally insufficient because the luminance in the red region is high. The light from the object is radiated light and reflected light, and the intensity of the combined light is sensed by the sensor and recorded as an image. The red image has higher brightness in the low temperature region. For this reason, it is easy to grasp the surface state when the low temperature region is observed with a red image.

  In this way, by separating the color photographed image into three RGB components, the surface state of the red-hot region can be clearly confirmed by referring to the B component separated image, and the R component separated image is referred to. Thus, the surface state of the low temperature region can be clearly confirmed.

(C) Background signal removal step ((C1) and (C2) in FIG. 2A)
Memory means for calculating an average luminance value of a pixel group within a predetermined range including the pixel for each pixel in the separated image in each of the separated images separated by the above-described separated image generation step by the background signal removing means (C1 in FIG. 2A), and the calculated average luminance value is subtracted from the luminance value (initial luminance value) of the pixel ((C2) in FIG. 2A. That is, for all the pixels in each separated image, A background signal removal process for subtracting the average luminance value from the initial luminance value is performed, and the “predetermined range” for calculating the average luminance value is set in advance as a radius (number of pixels) centered on the pixel, for example. Keep it.

  When background signal removal processing is performed using an 8-bit or 16-bit unsigned expression used in many images as an output image, the value may be negative and information may be lost. In this case, the same offset amount is added to all the pixels simultaneously with the background signal removal process so that no information is lost. The offset amount may be arbitrarily determined by judging from the state of the image, but in many cases, it is about half of the maximum brightness that the image to be processed can be displayed. In that case, it may be about 32768.

  The effect of this background signal removal process is as follows. In a color photographed image (original image), a wide luminance width is used to express a difference between a region with a high average luminance and a region with a low average luminance. This difference in average luminance reflects the temperature difference of the object, and is an unnecessary signal for grasping the surface state in each region of high temperature and low temperature. By performing background signal removal processing, signals unnecessary for grasping the above-described surface state are removed, and as a result, it is possible to expand and express the difference in brightness in each region of high temperature and low temperature, The contrast in each region can be greatly improved.

  In the flow of FIG. 2A, a process for adjusting the contrast is performed on the separated image after the background signal removal step in order to make it easy to confirm the difference in luminance between the high temperature region and the low temperature region. The contrast (amplification ratio) is set in advance. However, the process of adjusting the contrast can be omitted.

(D) Composite image generation step ((D1) or (D2) in FIG. 2A)
The synthesized image generation unit generates a synthesized image by synthesizing the separated images processed in the background signal removal process described above. In the generation of the composite image, the determination method of the composite ratio (blend ratio (RGB ratio)) of each of the RGB separated images is determined according to a fixed type ((D1) in FIG. 2A) and a luminance value dependent type ((D2 in FIG. 2A). )).

  In the fixed type, a combined image is generated by combining RGB separated images with a predetermined (fixed) RGB ratio.

  On the other hand, in the luminance value dependent type, the RGB ratio is changed for each pixel according to the luminance value of each pixel. The specific procedure is as follows.

  First, for each pixel (x, y) in the color-captured image before RGB separation stored in the memory means, the average luminance value before separation, which is an average luminance value of a pixel group within a predetermined range including the pixel (x, y) The value (va (x, y)) is calculated, and the maximum value (vamax) and the minimum value (vamin) of the calculated average luminance value (va (x, y)) before separation of each pixel are calculated (here (X, y) are x, y coordinate values in the color photographed image that specify the position of the pixel.) Then, in each pixel (x, y), as the average luminance value (va (x, y)) before separation of the pixel is closer to the maximum value (vamax), the B component separated image composition ratio (cB (x, y, The composite image is generated by combining so that y)) is larger than the composite ratio (cR (x, y)) of the R component separated image. In other words, in each pixel (x, y), the composite ratio (cR (x) of the R component separated image becomes larger as the pre-separation average luminance value (va (x, y)) of the pixel is closer to the minimum value (vamin). , y)) are combined so as to be larger than the combined ratio (cB (x, y)) of the B component separated images to generate a combined image. The composition ratio (cG (x, y)) of the G component separated images is set to a predetermined constant value.

More specifically, the composition ratio (cB (x, y)) of the B component separated image and the composition ratio (cR (x, y)) of the R component separated image are determined by the following relational expression, for example.
cB (x, y) = (va (x, y) -vamin) / (vamax-vamin)
cR (x, y) = 1-cB

  The combination ratio of the B component and the R component based on this relational expression is as shown in FIG. The horizontal axis of FIG. 4 shows the average luminance value before separation after normalizing the average luminance value (va (x, y)) of each pixel as the maximum value (vamax) as 100, and the vertical axis shows the B component and The synthesis ratio of R component is shown.

  In the above relational expression, as shown in FIG. 4, the combination ratio of the B component and the R component is linearly changed, but the average luminance value (va (x, y)) before the separation of the pixel is the maximum value ( The condition that the composition ratio (cB (x, y)) of the B component separated image is larger than the composition ratio (cR (x, y)) of the R component separated image is closer to vamax). If it satisfies, it is not necessary to change linearly, for example, it may be changed exponentially. However, in terms of simplification of arithmetic processing, it is preferable to change linearly as shown in FIG.

  As described above, the synthesized image is generated by synthesizing the separated images processed by the background signal removal process described above by the synthesized image generating process, thereby generating the surface of the high temperature part (red hot region) by the separated image of the B component. The state and the surface state of the low temperature region can be easily confirmed with a single image. In particular, when combining by the above-described luminance value dependent type, it becomes possible to combine the R component and the B component with a balance suitable for each region (pixel), and the surface state is much more than when the combining ratio is constant. Can be obtained.

(E) Color image generation process ((E1) in FIG. 2A)
The color image generation unit generates a color image by combining the composite image generated by the above-described composite image generation step with the color captured image (original image) captured by the capturing unit at a predetermined incorporation ratio. The incorporation ratio of the original image is set in advance.

  Thus, by referring to the color image obtained by combining the original image with the composite image, the user can easily grasp the configuration and positional relationship of the target object. That is, since the synthesized image does not include color information of the original image and a relative luminance change between the high temperature portion and the low temperature portion, it is difficult for the user to grasp the configuration and positional relationship of the target object. By combining the original image with the composite image at a predetermined incorporation ratio, the color information of the original image and the relative luminance change between the high temperature part and the low temperature part are incorporated, so the user can easily configure the configuration and positional relationship of the target object. I can grasp it.

  In the flow of FIG. 2A described above, a single color photographed image obtained in the photographing process is processed. However, in the present invention, a plurality of color photographed images are obtained by changing photographing conditions in the photographing process. It is also possible to process color photographed images. The flow at that time is shown in FIG. 2B.

  First, in the photographing process, a plurality of color photographed images are stored in the memory means by the first loop. Specifically, using the same shooting means (digital camera), obtain multiple color images taken with the same field of view and changing the shooting conditions such as shutter speed, aperture, sensitivity, and optical filter conditions. Save to memory means for targeting. The change in the “optical filter condition” means a change in the optical filter condition due to the use of the optical filter and the change in the type of the optical filter.

  Next, for each of a plurality of color photographed images obtained in the photographing process by the second loop, the “(B) separated image generating step”, “(C) background signal removing step” and “ (D) A composite image generation step ”is executed to generate a plurality of composite images. Then, a plurality of generated composite images are combined at a preset ratio (blend ratio) to generate a second composite image (second composite image generation step). This blend ratio can be arbitrarily set by the user. However, when a color photographed image is visually confirmed, it is easy to obtain a good result by increasing the ratio for an image having a large contrast in a region to be observed such as a high temperature region.

  Finally, by the “(E) color image generation step” described in FIG. 2A, a color image is obtained by combining the above-described second composite image with the color photographed image (original image) photographed in the photographing step at a predetermined incorporation ratio. Generate. At this time, since there are a plurality of color photographed images, an image obtained by combining any one or a plurality of color photographed images from these color photographed images at an arbitrary ratio can be used as the original image.

  In this way, by making it possible to use a plurality of color photographed images whose photographing conditions are changed, for example, a color photographed image in which a high-temperature region can be easily observed can be obtained by adjusting the average luminance to be low by changing the photographing conditions. Conversely, if the average brightness is adjusted to be low, a color photographed image that allows easy observation of the low-temperature region is obtained. By processing multiple images with different average brightness as a color photographed image, it is suitable for observation over a wide temperature range. Can be processed.

  The furnace wall of the coke oven was observed as an object according to the flow of FIG. 2A using the image processing apparatus of FIG.

  FIG. 5 shows an example of a color photographed image photographed in the photographing process. However, FIG. 5 shows an actual color photograph image converted to gray scale. This color photographed image is color photographed by a digital camera arranged in the positional relationship shown in FIG. 6, and corresponding portions are shown in FIGS. 5 and 6, respectively. In this photographing process, an infrared cut filter built in the digital camera is used.

  FIG. 7 shows a separated image obtained by separating the color photographed image (original image) of FIG. 5 into three RGB components in the separated image generation step, where (a) is an R component separated image (R image), and (b) is a separated image. G component separated images (G images) and (c) show B component separated images (B images).

  FIG. 8 shows a separated image after the background signal removal process is performed on each separated image of FIG. 7 by the background signal removal processing step, and (a) shows an R component separated image (R image), (b) ) Shows a separated image (G image) of the G component, and (c) shows a separated image (B image) of the B component. In this background signal removal processing step, the “predetermined range” when calculating the average luminance value of each pixel in each separated image is a circle range with a radius of 30 pixels centering on the pixel.

  Referring to the B image in FIG. 8C, a damaged portion that cannot be confirmed in the original image in FIG. 5 can be confirmed. Although it may be difficult to determine from the B image alone that it is a damaged part, it can be determined more reliably by periodically photographing the same part and observing changes over time. In any case, referring to the B image in FIG. 8C, the surface state of the furnace wall high temperature portion (red hot region) that cannot be confirmed in the original image in FIG. 5 can be confirmed. Similarly, referring to the R image of FIG. 8A, the surface state of the furnace wall low temperature portion that cannot be confirmed in the original image of FIG. 5 can be confirmed.

  FIG. 9 shows a combined image obtained by combining the separated images of FIG. 8 in the combined image generation process. In this composite image generation step, each separated image is synthesized by the above-described fixed type in which the synthesis ratio (RGB ratio) of each separated image is R image: 0.3, G image: 0.1, and B image: 0.6. did. Referring to FIG. 9, the surface state of the furnace wall high temperature portion based on the B image in FIG. 8C and the surface state of the furnace wall low temperature portion based on the R image in FIG. be able to.

  FIG. 10 shows a color image obtained by combining the original image of FIG. 5 with the composite image of FIG. 9 in the color image generation process. However, FIG. 10 shows an actual color image converted to a gray scale. In this color image generation step, the incorporation ratio of the original image was set to 0.05 (5%). Referring to FIG. 10, the configuration and positional relationship of the object (coke oven) can be easily grasped as compared with FIG. 9.

Claims (6)

A shooting process for color shooting of an object including a high-temperature part that is red hot,
A separated image generating step of generating a separated image by separating the color photographed image photographed in the photographing step into three components of RGB;
In each of the separated images separated by the separated image generation step, an average luminance value of a pixel group within a predetermined range including the pixel is calculated for each pixel in the separated image, and the calculated average luminance value is calculated as the luminance of the pixel. A background signal removal process to be subtracted from the value;
A combined image generating step of generating a combined image by combining the separated images processed by the background signal removal processing step;
Only including,
The composite image generation step includes
For each pixel in the color photographed image, an average luminance value before separation, which is an average luminance value of a pixel group within a predetermined range including the pixel, is calculated, and the maximum and minimum average luminance values before separation of the calculated pixels are calculated. A value is calculated and combined in each pixel so that the composite ratio of the B component separated image is larger than the composite ratio of the R component separated image as the average luminance value before separation of the pixel is closer to the maximum value. An image processing method for generating the composite image . The photographing process includes
A plurality of images taken by changing at least one of the shutter speed, aperture, sensitivity, and optical filter conditions of the photographing means are the color photographed images,
The composite image generation step includes
Generating a composite image for each of a plurality of color photographed images photographed in the photographing step;
The image processing method according to claim 1 , further comprising a second composite image generation step of generating a second composite image by combining the plurality of composite images generated by the composite image generation step at a preset ratio. The synthesis claim comprising a color image generating step of said combined color photographic image to generate a color image in the second synthesized image which the image generated by the generation process the composite image and the second composite image generating step is generated by one or 3. The image processing method according to 2. Photographing means for color-taking an object including a red hot part,
A separated image generating means for generating a separated image by separating a color photographed image photographed by the photographing means into three components of RGB;
In each of the separated images separated by the separated image generating means, an average luminance value of a pixel group within a predetermined range including the pixel is calculated for each pixel in the separated image, and the calculated average luminance value is calculated as the luminance of the pixel. Background signal removal processing means to subtract from the value;
Combined image generation means for generating a combined image by combining the separated images processed by the background signal removal processing means;
Equipped with a,
The composite image generation means includes
For each pixel in the color photographed image, an average luminance value before separation, which is an average luminance value of a pixel group within a predetermined range including the pixel, is calculated, and the maximum and minimum average luminance values before separation of the calculated pixels are calculated. A value is calculated and combined in each pixel so that the composite ratio of the B component separated image is larger than the composite ratio of the R component separated image as the average luminance value before separation of the pixel is closer to the maximum value. An image processing device that generates the composite image . The photographing means includes
A plurality of images captured by changing at least one of shutter speed, aperture, sensitivity, and optical filter conditions are referred to as the color image,
The composite image generation means includes
Generating a composite image for each of a plurality of color photographed images photographed by the photographing means;
5. The image processing apparatus according to claim 4 , further comprising: a second composite image generation unit configured to generate a second composite image by combining a plurality of composite images generated by the composite image generation unit at a preset ratio. The composite image generating unit synthesized image generated by or the second composite image generating unit according to claim 4 or comprising a color image generating means for generating a color image by combining the color photographic image on the second synthesized image generated by 5. The image processing apparatus according to 5 .