JP2005303705A - Image processing device and image processing method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To eliminate or reduce luminance unevenness at a high speed and with simple algorithms, without generating noise, such as pseudo-edges, regardless of the existence/absence of periodicity of luminance unevenness in an image processing device and image processing method. <P>SOLUTION: A luminance information calculating means 102 calculates average luminance and luminance deviation representative value of an input image, and a luminance replaced image preparing means 103 replaces singular luminance data, the deviation from the average luminance of which is larger than a prescribed value with replacement luminance data to prepare a replaced image. A luminance smoothing means 108 smoothes the replaced image, and a luminance unevenness image preparing means 104 prepares a luminance unevenness image. Then, a luminance unevenness elimination image preparing means 106 calculates the input image, average luminance and luminance unevenness image to be able to eliminate the luminance unevenness. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to an image processing apparatus and method. For example, the present invention relates to an image processing apparatus and method for removing or reducing luminance unevenness generated in a captured image due to illumination unevenness and shading characteristics of an imaging optical system.

Conventionally, in an imaging apparatus that captures an image of an imaging target using an imaging device and obtains image data by photoelectric conversion, luminance unevenness may occur due to uneven illumination or shading characteristics of the imaging optical system, and image quality may deteriorate. there were. Therefore, such luminance unevenness that causes image quality degradation is removed or reduced by image processing.
As an example of such an image processing apparatus and method, for example, in Patent Document 1, the original image is thinned and subjected to Fourier transform, and after removing low-frequency components by filtering, inverse Fourier transform is performed. An image processing method is described in which a processed image is enlarged to an image size before thinning, and luminance unevenness is removed by differential processing with the original image.
In Patent Document 2, the periodic pattern information is removed by performing two-dimensional Fourier transform on the captured image, removing pattern cells and frequency components corresponding to noise from the power spectrum, and performing two-dimensional inverse Fourier transform. In addition, there is described a macro inspection method for performing defect inspection by acquiring uneven gray image data and performing threshold processing.
Japanese Patent Laid-Open No. 06-189134 (page 3-5, FIG. 2) Japanese Patent Laid-Open No. 11-339040 (page 3-5, FIG. 1)

However, the conventional image processing apparatus and method as described above have the following problems.
In the techniques described in Patent Documents 1 and 2, luminance unevenness is reduced by Fourier transforming an image, separating luminance unevenness and a normal image by frequency analysis, and cutting a frequency component of the luminance unevenness.
Therefore, there is a problem that sufficient luminance unevenness cannot be removed unless the frequency component of luminance unevenness and the frequency component of a normal image are clearly separated. That is, there is a problem that it is difficult to remove luminance unevenness in an image that does not include a periodic pattern in a normal image.
Even if the normal image includes a periodic pattern, if it is close to the frequency component of uneven brightness, the frequency component of the periodic pattern in the normal image or the vicinity of the edge region must be set unless the frequency to be cut is set appropriately. As a result of cutting the frequency up to the harmonics, there is a problem that noise such as a mosquito noise-like pseudo edge region is generated.
In addition, the Fourier transform and inverse Fourier transform have a problem that the size of image data is practically limited because the calculation takes time when the number of data increases. Therefore, there is a problem that it cannot be used for an inspection apparatus that requires a high-speed inspection tact on a production line or the like.

  The present invention has been made in view of the above problems, and even with an input image having a periodic pattern, for example, without generating noise such as a mosquito noise-like pseudo edge, a high-speed and simple algorithm is used. An object of the present invention is to provide an image processing apparatus and method that can remove or reduce luminance unevenness, and can similarly remove or reduce luminance unevenness even in an input image that does not include a periodic pattern.

In order to solve the above-described problem, according to the first aspect of the present invention, there is provided an image processing apparatus for removing luminance unevenness of an input image, wherein the average luminance and luminance deviation representative values of the input image in a predetermined area Brightness information calculating means for calculating the brightness information of the input image in the predetermined area, and the specific brightness data having a deviation from the average brightness calculated by the brightness information calculating means greater than a predetermined value from the average brightness A replacement image creating means for creating a replacement image replaced with replacement brightness data having a smaller deviation; a brightness smoothing means for smoothing the brightness of the target pixel by a brightness value in the vicinity of the target pixel in the predetermined area; Luminance unevenness image creating means for creating a brightness unevenness image by performing brightness smoothing by the brightness smoothing means on the replacement image created by the replacement image creating means, and the input Brightness unevenness removing means for calculating an average brightness calculated by the brightness information calculating means and a brightness unevenness image created by the brightness unevenness image creating means for an image to create an output image from which the brightness unevenness has been removed It is set as the structure provided with.
According to this invention, the luminance information calculating means calculates the average luminance and the luminance deviation representative value of the input image in the predetermined area, calculates the deviation from the average luminance of the luminance data of the input image, In comparison, if it is larger than the predetermined value, it is determined as specific luminance data.
Then, the replacement image creating means creates a replacement image replaced with replacement luminance data that reduces the deviation from the average luminance.
Further, the luminance unevenness image creating means smoothes the luminance of each pixel of interest in the replacement image by the luminance smoothing means, and creates a luminance unevenness image from which noise other than the luminance unevenness has been removed.
Then, the luminance unevenness removing means calculates the input image, the average luminance, and the luminance unevenness image to create an output image from which the luminance unevenness is removed. The luminance unevenness is removed by subtracting the average luminance from the luminance unevenness image, extracting the luminance unevenness component, and subtracting the luminance unevenness component from the input image, thereby obtaining an output image in which the luminance unevenness is removed or reduced.

Here, the luminance deviation representative value is a representative value of deviation from the average luminance, and for example, a standard deviation or a value obtained by multiplying the standard deviation by an appropriate coefficient can be adopted.
As the smoothing process by the brightness smoothing means, a process of setting the block average brightness near the target pixel as the brightness of the target pixel, or an appropriate smoothing filter process such as weighting the brightness around the vicinity can be adopted.

According to a second aspect of the present invention, in the image processing apparatus according to the first aspect of the present invention, the replacement image creating unit uses the singular luminance data as a pixel of interest to obtain the luminance smoothed by the luminance smoothing unit. The replacement luminance data for luminance data is used.
According to the present invention, the pixel determined to be the specific luminance data is used as the target pixel, and the luminance smoothing unit performs the luminance smoothing to obtain the replacement luminance data. Deviation can be reduced.

According to a third aspect of the present invention, in the image processing device according to the first or second aspect, the replacement image creating means includes, as a predetermined value for determining the specific luminance data, only the luminance deviation representative value in the average luminance. Using the adjusted boundary luminance value, the replacement image creating means uses the boundary luminance value in the vicinity of the specific luminance data as the replacement luminance data for the specific luminance data.
According to the present invention, since the replacement is performed with the boundary luminance value instead of the specific luminance data, it is possible to create a replacement image in which the deviation from the average luminance is reliably reduced by an extremely simple process.

According to a fourth aspect of the present invention, in the image processing apparatus according to any one of the first to third aspects, the luminance unevenness image creating unit is configured to perform the luminance smoothing unit with respect to target pixels separated from each other in the replacement image. The luminance unevenness image is created by interpolating the luminance of the pixel of interest between the pixels of interest separated from each other by interpolating the luminance of the pixel of interest after luminance smoothing.
According to the present invention, the luminance smoothing means is used for the target pixels separated from each other in the replacement image, and the luminance of the target pixel after the luminance smoothing is obtained for the pixels other than the target image by interpolation. The number of operations by the brightness smoothing means can be reduced. As a result, it is possible to speed up image processing.

According to a fifth aspect of the present invention, in the image processing apparatus according to any one of the first to fourth aspects, the luminance smoothing means performs the luminance smoothing when a repetitive pattern having a predetermined pitch exists in the input image. The size of the block in the vicinity of the area to be converted is set to an integral multiple of the predetermined pitch.
According to the present invention, when a repetitive pattern with a predetermined pitch exists in the input image, the size of the neighboring peripheral block subjected to luminance smoothing by the luminance smoothing means is set to an integral multiple of the predetermined pitch. Luminance smoothing is performed in the block corresponding to the repetition period. Therefore, the influence of the repeated pattern can be eliminated from the luminance unevenness. As a result, the luminance unevenness removal accuracy can be improved.

According to a sixth aspect of the present invention, in the image processing apparatus according to any one of the first to fifth aspects, an image reduction unit that thins out an input image to create a reduced input image having a predetermined image size; Image size enlarging means for enlarging the image to the image size of the input image, and using the reduced input image as the input image in the predetermined area, the luminance information calculating means, the replacement image creating means, and the luminance smoothing means The brightness unevenness image creating means creates the brightness unevenness image, the image size of the brightness unevenness image is enlarged by the image size enlargement means, and the average brightness and the brightness of the reduced input image are enlarged. From the unevenness image and the input image, an output image from which the unevenness in brightness is removed by the unevenness in brightness removal unit can be created.
According to this invention, the calculation of the luminance information calculation unit, the replacement image generation unit, the luminance smoothing unit, and the luminance unevenness generation unit is performed on the reduced input image obtained by thinning the input image by the image reduction unit and reducing the image size. The luminance unevenness image is enlarged to the same image size as the input image by the image size enlargement means, and the luminance unevenness is removed. Therefore, the amount of calculation of the luminance information calculation unit, the replacement image generation unit, the luminance smoothing unit, and the luminance unevenness generation unit can be significantly reduced as compared with the case where the reduced image is not used, so that high-speed image processing can be performed. The brightness unevenness can be quickly removed even with a large image size.
And

The invention according to claim 7 is an image processing method for removing luminance unevenness of an input image, a luminance information calculation step of calculating an average luminance and a luminance deviation representative value of the input image in a predetermined region; Among the luminance data of the input image in the predetermined area, the specific luminance data whose deviation from the average luminance calculated in the luminance information calculation step is larger than a predetermined value is replaced with the replacement luminance data whose deviation from the average luminance is smaller. A replacement image creation step of creating a replacement image by replacement, a luminance smoothing step of smoothing the luminance of the target pixel by a luminance value in the vicinity of the target pixel in the predetermined region, and a replacement image generation step For the replacement image, the luminance unevenness image creation step for obtaining the block average luminance by the luminance smoothing step to create the luminance unevenness image, and for the input image, the luminance The average luminance calculated by the distribution calculating step, and the calculates the luminance unevenness image created by uneven brightness image generation step, a method and a luminance unevenness removing step of creating an output image obtained by removing the luminance unevenness.
According to this invention, the replacement image in which the average luminance and the luminance deviation representative value of the input image are calculated in the luminance information calculation step and the specific luminance data is replaced with the replacement luminance data in which the deviation from the average luminance is small in the replacement image creation step. The brightness unevenness image creation process smooths the replacement image to create a brightness unevenness image, and the brightness unevenness removal process creates the output image with the brightness unevenness removed by calculating the input image, average brightness, and brightness unevenness image. can do.
Further, the method is suitable for using the image processing apparatus according to claim 1, and has the same effects as the invention according to claim 1.

  According to the image processing apparatus and method of the present invention, the substitution image is formed by replacing the singular luminance data in which the deviation from the average luminance is larger than the predetermined value among the input images with the luminance substitution data in which the deviation is smaller, By smoothing it, a brightness unevenness image can be created with high accuracy, and by calculating the input information, average brightness, and brightness unevenness image, an output image with reduced or reduced brightness unevenness can be obtained. It is possible to remove or reduce luminance unevenness with a fast and simple algorithm without generating pseudo edges due to computations, and to eliminate or reduce luminance unevenness in an input image that does not contain a periodic pattern as well. Play.

Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. In all the drawings, even if the embodiments are different, the same or corresponding members are denoted by the same reference numerals, and redundant description is omitted.
[First Embodiment]
An image processing apparatus according to a first embodiment of the present invention will be described.
FIG. 1 is a functional block diagram for explaining a schematic configuration of a substrate inspection system using the image processing apparatus according to the first embodiment of the present invention. FIG. 2 is a functional block diagram for explaining the configuration of the image unevenness removal unit of the image processing apparatus according to the first embodiment of the present invention.

  The image processing apparatus of the present embodiment can be used alone or as a part of an image processing system in order to remove luminance unevenness of an image signal. In the following, the image unevenness removing unit 11 incorporated in the substrate inspection system 1 for performing the defect inspection of the semiconductor wafer and the liquid crystal substrate shown in FIG. 1 will be described as an example.

The board inspection system 1 will be briefly described.
The substrate inspection system 1 irradiates a subject fixed to a stage (not shown) with illumination light, images reflected light, scattered light, diffracted light, and the like emitted from the subject with the CCD camera 4, and so on. The acquired image is subjected to image processing by the main computer 12 to perform defect inspection.
The main computer 12 includes an image capturing circuit 2 that captures an image captured by the CCD camera 4 and forms image data, an image storage unit 8 and 240, a drive control unit 13 that controls driving of the CCD camera 4 and the stage, and image data. Defect extraction algorithm units 40 and 41 for extracting defects by performing image processing, and a defect determination unit 32 for determining the type of defect extracted with reference to the defect dictionary 31. The defect extraction algorithm units 40 and 41 have the same configuration, and can perform two image processes in parallel.
Connected to the main computer 12 are an operation input unit 43 for external control and a display unit 29 for displaying input images and inspection results.

The defect extraction algorithm unit 40 (41) includes an image unevenness removing unit 11 (image processing apparatus) that removes unevenness in brightness of an image, and a feature portion extracting unit that detects a position detection pattern in the image data after removing unevenness in brightness. 17. Image position correction unit 20 that performs image position correction based on the position information of the detected position detection pattern, defect extraction unit 24 that extracts defects from the image after position correction, and coordinates extraction of the extracted defects. And a defect coordinate extraction unit 28.
The image unevenness removal unit 11 is connected to an image unevenness storage unit 16 that stores the image unevenness extracted for removal as necessary.
A feature storage unit 21 that stores a comparison pattern for extracting features is connected to the feature portion extraction unit 17.
The defect extraction unit 24 is connected to an image storage unit 240 for storing an image of the extracted defect.

  As shown in FIG. 2, the schematic configuration of the image unevenness removing unit 11 includes an image input unit 100, an image reduction unit 101, a luminance information calculation unit 102, a luminance replacement image generation unit 103 (substitution image generation unit), and a luminance unevenness image generation. The image forming apparatus includes means 104, luminance unevenness image enlargement means 105 (image size enlargement means), luminance unevenness removal image creation means 106, image output means 107, and luminance smoothing means 108.

The image input unit 100 obtains an input image (image size (M × N), where M and N are integers) corresponding to each pixel of the CCD camera 4 from the image storage unit 8, and sets the pixel position (i, j). This is means for storing luminance data in a two-dimensional array I _in (i, j) (where i, j is an integer of 1 ≦ i ≦ M and 1 ≦ j ≦ N). Hereinafter, I _in (i, j) is simply referred to as an input image.

The image size is the image size input by the image input means 100, and may be the entire image range captured by the CCD camera 4 or a part thereof.
When selecting a part of the image range, it goes without saying that the entire image range can be processed by dividing the image processing into a plurality of parts.

The image reduction means 101 thins out the luminance data stored _in the input image I _in (i, j), reduces it to the image size (m × n), and reduces the reduced image I _r (i, j) (i, j Is a means for forming 1 ≦ i ≦ m and an integer of 1 ≦ j ≦ n.
The thinning interval is determined as appropriate depending on the size of the image to be reduced. In the present embodiment, the thinning interval is thinned at equal intervals, and the reduction ratio is an integer.

The luminance information calculation unit 102 calculates the average luminance I _A and the luminance deviation representative value s of the reduced image I _r (i, j), and determines the pixel region Q in which specific luminance data having a large deviation from the average luminance I _A exists. It is a means for seeking.
Luminance deviation representative value may be employed by a numerical value representing the deviation from the average luminance I _A of the luminance data, a variety of statistical representative value and appropriately calculating the values it. For example, k can be an appropriate constant, σ can be a standard deviation, and the following equation can be used as a luminance deviation representative value.
s = k · σ (1)
Here, the value of k is preferably about 2-3.
The extraction of the singular luminance data and the pixel region Q can determine whether the reduced image I _r (i, j) satisfies the following equation.
| I _r (i, j) −I _A |> s (2)
A set of all pixel positions (i, j) satisfying Expression (2) is the pixel region Q. Hereinafter, when emphasizing that (i, j) belongs to the pixel region Q, it is expressed as (I, J).
Here, the luminance value (I _A ± s) obtained by the luminance information calculation unit 102 is a boundary luminance value for determining singular luminance data.

The luminance smoothing unit 108 is a unit for designating the target pixel at an arbitrary position and performing luminance smoothing processing on blocks in the vicinity thereof. And it is connected to the brightness information calculation means 102 and the brightness unevenness image creation means 104 described later.
FIG. 3 is a conceptual diagram for explaining the smoothing process.
The block area B to be averaged is not limited to a rectangular area, and may be, for example, a circular area, but for the sake of simplicity, the size is {(2 · Δi + 1) × (2 · Δj + 1)} (Δi, Δj Will be described as a rectangular area (see FIG. 3A).
If the luminance of the pixel at the pixel position (i, j) is represented by P (i, j), the block average luminance p ₁ (i ₁ , j ₁ ) at the pixel position (i ₁ , j ₁ ) of the pixel of interest. Is expressed by the following equation.
p ₁ (i ₁ , j ₁ ) = {ΣP (i _B , j _B )} / K (3)
Here, the summation symbol Σ is assumed to be the sum for all pixel positions (i _B , j _B ) in the block region B. Further, K represents the number of pixels in the block area B. That is,
i ₁ −Δi ≦ i _B ≦ i ₁ + Δi (4a)
j ₁ −Δj ≦ j _B ≦ j ₁ + Δj (4b)
K = (2 · Δi + 1) · (2 · Δj + 1) (4c)
The shape and size of such a block can be appropriately set according to the image data and the purpose of smoothing.

Here, if the pixel position (i ₁ , j ₁ ) is sequentially moved, the block region B is also moved in parallel, and the block average luminance at an arbitrary pixel position is obtained. For example, the value of i ₁ can be updated by 1 to obtain the block average luminance p ₂ (i ₁ +1, j ₁ ) adjacent to p ₁ (i ₁ , j ₁ ) (see FIG. 3B). ). Further, by updating the value of i ₁ by (2 · Δi + 1), the block average luminance p ₃ (i ₁ + 2 · Δi + 1, j ₁ ) at the position where the block region B is adjacent can be obtained (FIG. 3 ( c)).

The brightness replacement image creating means 103 replaces the specific brightness data determined by the brightness information calculation means 102 with brightness replacement data having a smaller brightness deviation, thereby forming a replacement image with reduced brightness deviation. It is.
That is, the replacement image I _t (i, j) for the reduced image I _r (i, j) is
In the range of the pixel region Q where i and j satisfy Expression (2),
_{I t (I, J) =} I R (I, J) ··· (5a)
For other i and j,
I _t (i, j) = I _r (i, j) (5b)
It is said.

Here, the luminance replacement data I _R (I, J) may be any data as long as the luminance deviation can be reduced. For example, the luminance smoothing means 108 calculates the pixel (I, J) to be subjected to luminance replacement. The block average luminance around the neighborhood can be adopted. That is, instead of i ₁ and j ₁ in equation (3), I and J are used, and instead of P (i _B and j _B ), I _r (i _B and j _B ) is used.
I _R (I, J) = {ΣI _r (i _B , j _B )} / K (6)
Here, the meaning of the summation symbol Σ is the same as that in Equation (3).
In addition, the block region B which is the vicinity peripheral region of the singular luminance data may be obtained by replacing i ₁ and j ₁ with I and J in equations (4a) and (4b), respectively. In order to perform the replacement, it is preferable that the pixel area Q is obtained only from the pixel area expanded by the morphological expansion process (i _B and J _B do not include Q). Further, the image area Q may be substituted by the average luminance I _A for simplicity.

The luminance unevenness image creating means 104 captures high-frequency component noise, for example, the light reception sensitivity and response characteristics of each pixel of the image sensor from the replacement images I _t (i, j) shown in equations (5a) and (5b). This is a means for removing luminance noise and extracting luminance unevenness over a relatively wide area caused by illumination unevenness and shading characteristics of the imaging optical system to create a luminance unevenness image.
Specifically, the replacement image I _t (i, j) is smoothed by the brightness smoothing means 108. That is, the luminance unevenness image I _tave (i, j) is replaced with i ₁ , j ₁ in equation (3), i, j, and P (i _B , j _B ) instead of I _t (i _B , J _B ).
I _tave (i, j) = {Σ (I _t (i _B , j _B )} / K (7)
Here, the meaning of the summation symbol Σ is the same as that in Equation (3). The block area B is obtained by replacing i ₁ and j ₁ with i and j in equations (4a) and (4b), respectively. However, the magnitudes of Δi and Δj can be different values in the equations (6) and (7) as necessary.

The luminance unevenness image enlargement unit 105 interpolates the luminance data corresponding to the pixels obtained by thinning out the data by the image reduction unit 101 with respect to the luminance unevenness image I _tave (i, j), thereby setting the pixel size (m × n). A means for enlarging the pixel size (M × N) to create an enlarged luminance unevenness image I _tex (i, j) (i, j is an integer of 1 ≦ i ≦ M, 1 ≦ j ≦ N). is there.
Any interpolation algorithm may be used for the luminance data interpolation. For example, linear interpolation may be employed.

The luminance unevenness removal image creating means 106 calculates the average luminance I _A and the enlarged luminance unevenness image I _tex (i, j) for the input image I _in (i, j), and removes the luminance unevenness from the input image. The output image I _out (i, j) (i, j is an integer satisfying 1 ≦ i ≦ M and 1 ≦ j ≦ N).
The output image I _out (i, j) is calculated from the following equation.
I _out (i, j) = I _in (i, j) + I _A −I _tex (i, j) (8)
The image output unit 107 is a unit for outputting the output image I _out (i, j) to the feature portion extraction unit 17.

Next, the operation of the image unevenness removal unit 11 of this embodiment will be described.
FIG. 4A is a schematic diagram illustrating an example of an input image. FIG. 4B is a graph showing an example of a one-dimensional luminance distribution of the input image (reduced image). FIG. 4C is a graph illustrating an example of a replacement image. 4B and 4C, the horizontal axis indicates the pixel position, and the vertical axis indicates the luminance of each pixel. FIG. 5A is a flowchart for explaining the image processing method according to the first embodiment of the present invention.

For example, as shown in FIG. 4A, for example, an image pattern of a semiconductor wafer, a liquid crystal substrate, or the like is distributed in a block shape, and high-intensity portions 51 containing a lot of image information appear repeatedly at a predetermined pitch. In many cases, the pattern has a relatively high contrast periodic pattern in which a low-luminance portion 52 having a significantly lower luminance and a smaller area ratio than the luminance portion 51 appears. Moreover, even if there is no periodicity, for example, there are areas where the luminance changes suddenly, such as defect areas in image inspection, boundary areas of various components, and wiring lines.
In addition to such a steep brightness change, the input image 50 includes brightness unevenness due to illumination unevenness and shading characteristics of the imaging optical system of the CCD camera 4. This luminance unevenness is lower in spatial frequency than noise due to sensitivity variation and instability of the image sensor of the CCD camera 4. In addition, the variation width of such luminance unevenness is often smaller than the difference in luminance between the high luminance portion 51 and the low luminance portion 52.

Hereinafter, each step will be described with reference to FIG.
In the image input step (step 150), the input image 50 is stored _{in the} two-dimensional array I _in (i, j) by the image input means 100 and can be referred to. The storage location of I _in (i, j) is provided in the image storage unit 8.
For example, to display the luminance distribution of the pixel column of j = j _k of FIG. 4 (a), as an example, the curve 301 showing the luminance distribution as shown in FIG. 4 (b) is obtained.
A curved line 301 alternates between a wide high-brightness part 51 and a low-brightness part 52 that is narrow and whose luminance level is almost the minimum level depending on the pixel position. In addition, uneven luminance unevenly occurs downward with respect to all pixels.
In the high luminance part 51, variation due to luminance unevenness in each block is further superimposed. Since the luminance unevenness occurs in proportion to the luminance, the luminance level of the low luminance portion 52 is hardly affected.

Next, in the image reduction process (step 151), the image reduction means 101 thins out the data of I _in (i, j) at equal intervals, and reduces the image size (M × N) to the image size (m × n). Then, the reduced image I _r (i, j) is created. At this time, FIG. 4B is compressed at a magnification (m / M) in the horizontal axis direction, but in the following description, FIG. 4B is a reduced image I _r (unless otherwise misunderstood. It is assumed that i, j _k ) is also expressed.

Next, a luminance information calculation step (step 152) for calculating luminance information from the reduced image I _r (i, j) will be described.
FIG. 5B is a flowchart for explaining details of the luminance information calculation step.
In step 201, the luminance information calculation means 102 obtains the average luminance I _A and the luminance deviation representative value s (see equation (1)). In FIG. 4B, the average luminance I _A and the boundary luminance value I _A ± s are indicated by broken lines.
At this time, since the reduced image is used, the number of data is reduced, but the processing target area is an area corresponding to the image size (M × N).

In step 202, it is determined whether each data of the reduced image I _r (i, j) satisfies the equation (2), and the pixel region Q is extracted.
For example, the areas 51A and 51B and the low luminance part 52 shown in FIG. 4B belong to the pixel area Q.

In step 203, the pixel that belongs to the pixel region Q is used as a pixel of interest, the luminance smoothing means 108 calculates the block average luminance of the neighboring peripheral region, and the luminance replacement data I _R (I, J) is calculated by equation (6) Calculate (brightness smoothing step, step 153 in FIG. 5A).
In the luminance replacement data I _R (I, J), the average luminance I _A is determined from the specific luminance data by appropriately setting the size of the block area B to be averaged and averaging the luminance data other than the specific luminance data. The deviation from can be reduced. Such a block area B is made possible by setting the size of the block area B to an area extending outside the pixel area Q. For example, by performing a morphological expansion process on the pixel region Q, such a neighboring peripheral region can be easily obtained.
Thus, the luminance information calculation process ends.

Next, in the brightness replacement image creation step (step 154 in FIG. 5A), the brightness replacement image creation means 103 performs brightness replacement on the specific brightness data extracted in the brightness information calculation step, and the formula ( A replacement image I _t (i, j) represented by 5a) and (5b) is created.
A curve 302 shown in FIG. 4C shows an example of the replacement image I _t (i, j _k ). Brightness (I A _-s) In the above (I A _{+ s)} the range, the luminance substituted is not performed, the brightness deviation is reduced in the range.
In addition, since FIG.4 (c) is a schematic diagram, a dimensional ratio is not necessarily exact. For example, although the luminance replacement data is illustrated as being equal to the boundary luminance value, in the present embodiment, the luminance replacement data is always equal to the boundary luminance value because the luminance value changes depending on the size of the block area B. Is not limited.

Subsequently, in the luminance unevenness image creation step (step 155), the luminance smoothing means 108 smoothes the replacement image I _t (i, j), and the luminance unevenness image I _tave (i) expressed by the equation (7). , J).
That is, the size of the block area B is set as appropriate, and all pixels in the predetermined area are sequentially smoothed (luminance smoothing step, step 156).
In the replacement image, the singular luminance data having a large luminance change is replaced, and therefore, blurring that cannot be smoothed occurs due to the influence of the singular luminance data in the vicinity region of the luminance change, and the extraction accuracy of luminance unevenness decreases. Can be prevented.

  At this time, when there is a periodic pattern unrelated to luminance unevenness in the input image 50, the size of the block region B is set to the size of the direction in which the periodic pattern exists, and the spatial period of the periodic pattern is set. Set to an integer multiple. By doing so, since smoothing is performed in units of the spatial period of the periodic pattern, the luminance fluctuation caused by the spatial period of the pattern is smoothed, and it is possible to prevent noise from luminance unevenness. There is.

  The periodic pattern of the input image 50 can be set to a known value, for example, the chip interval of the semiconductor wafer. However, if necessary, it is automatically discriminated by performing image processing for detecting the periodic pattern. You may make it do.

A curve 303A shown in FIG. 4C shows an example of the luminance unevenness image I _tave (i, j _k ) obtained in this way. The curve 302 is substantially included in a range from (I _A −s) to (I _A + s) and is a smooth downward convex curve.
Since the singular luminance data is replaced, the change width of the portion showing a steep change from the high luminance portion 51 to the low luminance portion 52 (see FIG. 4B) is remarkably reduced, and the influence is not so much. There are no smooth curves. Therefore, the curve 303 </ b> A has a shape that substantially follows the luminance change in the high luminance part 51.
Such a luminance unevenness image does not have a clear periodicity, and therefore cannot be accurately extracted by Fourier transform. However, in this embodiment, since image processing is performed in the spatial domain, a pseudo edge or the like is not generated. There is an advantage that luminance unevenness can be extracted with high accuracy.

Next, in the luminance unevenness image enlargement step (step 157), the luminance unevenness image enlargement unit 105 interpolates the luminance unevenness image I _tave (i, j) to obtain the enlarged luminance unevenness image I _tex (i, j) (i, j is an integer of 1 ≦ i ≦ M and 1 ≦ j ≦ N). Thus, in the input image I _in (i, j), brightness unevenness image in which only substantially uneven luminance component remains is the pixel position of the input image I _in (i, j), are determined to correspond to the pixel size.
FIG. 6A is a graph in which a curve 301 representing the input image I _in (i, j _k ) before reduction and a curve 303B representing the enlarged luminance unevenness image I _tex (i, j _k ) are superimposed. It is.

Subsequently, in the luminance unevenness removal step (step 158), the luminance unevenness removed image creating means 106 creates an output image I _out (i, j) from which the luminance unevenness has been removed based on the equation (8).
FIG. 6B is a graph showing the output image I _out (i, j _k ).
The tendency of uneven brightness unevenness of the high brightness portion 51 (see FIG. 4B) is obtained by subtracting the enlarged brightness unevenness image I _tex (i, j _k ) from the input image I _in (i, j _k ). There is corrected, it can be seen that is formed flat high-luminance portion 51a substantially along the average luminance I _a.
On the other hand, in the low luminance portion 52 (see FIG. 4B), the low luminance portion 52a having a slightly high luminance value in the range of the region 502 is formed by the above correction.
As in this example, the average luminance I _A if the image pattern that is biased to the high brightness side, it is often uneven brightness does not become generally safe to convert the low-luminance portion.

In the image output step (step 159), the image output means 107 outputs the output image I _out (i, j) to the feature portion extraction unit 17 (see FIG. 1).
Thus, the image processing process by the image unevenness removal unit 11 is completed.

As described above, according to the image unevenness removing unit 11, the amount of luminance unevenness can be directly reduced in the spatial region by relatively simple arithmetic processing such as addition / subtraction operation, averaging operation, data comparison, and data replacement. As in the case of image processing in the frequency domain by Fourier transform or the like, luminance unevenness can be easily removed or reduced without generating noise such as pseudo edges.
Further, there is an advantage that luminance unevenness can be removed with or without a periodic pattern.
Further, since an algorithm such as Fourier transform is not used, there is an advantage that image processing can be performed quickly because the calculation time is relatively short even when the amount of data increases.
In addition, since the input image is reduced and then the luminance replacement is performed to create a luminance unevenness image, the number of data to be subjected to these operations can be reduced, and relatively high-speed image processing can be performed compared to the case where the input image is not reduced. It can be performed. In recent years, for example, in image processing such as substrate inspection, the size of the subject has increased and the pattern to be inspected has become higher definition, so the image size tends to increase. Unevenness can be removed or reduced.

Next, a first modification of the present embodiment will be described.
In the present modification, in the above-described embodiment, in the output image I _out (i, j), the luminance value is slightly increased due to the influence of the luminance unevenness correction, thereby preventing the occurrence of the low luminance portion 52a (see FIG. 6B). Therefore, the luminance unevenness-removed image creating means 106 can create the output image I _out (i, j) by the following equation instead of the equation (8).
I _out (i, j) = I _in (i, j) + {I _A −I _tex (i, j)} · α (9)
However, α is α = 1 for the range of (i, j) that satisfies the following conditional expression (10), and α = α ₀ for the other (i, j). = 0 to 1.
I _in (i, j) −I _tex (i, j) + I _o ≧ 0 (10)
Here, _Io is an offset luminance having a constant value set as appropriate.
As the values of the coefficient α _o and the offset luminance I _o , for example, values such as α _o = 0.5, I _o = S can be adopted.

The operation of this modification will be described with reference to FIG.
FIG. 7 is a graph showing an output image I _out (i, j _k ) according to the first modification.
For example, when the offset luminance is I _o = S, the high luminance portion 51 (see FIG. 6A) of the input image I _in (i, j) satisfies the equation (10). ), Α = 1, which is the same as the equation (8). Therefore, luminance unevenness removal is performed as in the above embodiment.
On the other hand, in the low luminance portion 52, since the equation (10) is not satisfied, the formula (9), the luminance unevenness removal is performed as α = α _o. For this reason, the influence of the luminance unevenness correction of the low luminance portion 52 is reduced.
Therefore, as shown in a region 502 in FIG. 7, the low luminance portion 52 (see FIG. 6A) of the input image is a low luminance portion 52b having substantially the same luminance as that before correction in the output image. .
As a result, even if luminance unevenness removal is performed, it is possible to prevent the correction from affecting the low luminance portion.

Next, a second modification of the present embodiment will be described.
In this variation, in the luminance replacement image creation unit 103 of the above embodiment, as the luminance replacement data I _R (I, J), a boundary luminance value close to singular luminance data as in the following equation instead of equation (6): Is substituted. That is,
For the pixel position (I, J) of the pixel region Q,
When I _r (I, J)> I _A + s,
I _R (I, J) = I _A + s (11a)
I _r (I, J) _{<when I} A -s,
I _R (I, J) = I _A −s (11b)
In this way, since I _R (I, J) can be obtained only by addition / subtraction, there is an advantage that the calculation time is remarkably reduced as compared with the case of calculating equation (6).

Note that, according to the present modification, the replacement image I _t (i, j) is in the range of the following expression for all pixel positions (i, j).
I _A −s ≦ I _t (i, j) ≦ I _A + s (12)
Therefore, when a range of luminance unevenness is predicted in advance, the luminance unevenness can be corrected with high accuracy by matching the boundary luminance value with the range of the luminance unevenness.

[Second Embodiment]
An image processing apparatus according to the second embodiment of the present invention will be described.
The image processing apparatus according to the present embodiment has a configuration in which the image reduction unit 101 and the luminance unevenness image enlargement unit 105 are omitted from the image unevenness removal unit 11 according to the first embodiment. Accordingly, the luminance information calculation unit 102, the luminance replacement image creation unit 103, and the luminance unevenness image creation unit 104 use the input image I _in (i, j) instead of the reduced image I _r (i, j) to remove the luminance unevenness. In the image creating means 106, the uneven luminance image I _tave (i, j) is used instead of the enlarged uneven luminance image I _tex (i, j).
The image processing method of this embodiment is a method in which the image reduction process (step 151) and the luminance unevenness image enlargement process (step 157) of the first embodiment are omitted using such an image unevenness removal unit 11. (See FIG. 8). This is applicable when the number of pixels is small and the calculation time can be sufficiently shortened.
Description of these details will be omitted because it will be easily understood if the description and formulas of the first embodiment are sequentially read based on the above changes.

According to the present embodiment, since it is the same as the first embodiment in that an algorithm such as Fourier transform is not used, luminance unevenness can be easily removed or reduced without generating noise such as a pseudo edge, and the period There is an advantage that luminance unevenness can be removed with or without a pattern.
In addition, even if the amount of data increases, there is an advantage that image processing can be performed quickly because the calculation time resulting from the algorithm is relatively short.
The difference from the first embodiment is that the image reduction means and the luminance unevenness image enlargement means are omitted, so that a simple device can be configured. Further, since the luminance unevenness is removed using the entire luminance data of the input image, there is an advantage that the luminance unevenness can be removed with high accuracy.

  In the above description, an example of an input image having a high luminance part with a large area ratio and a low luminance part with a small area ratio has been described as an example. However, the present invention can easily eliminate luminance unevenness included in the vicinity of the average luminance. The luminance distribution is not limited to such an example because there is no relationship between whether the average luminance is high luminance or low luminance.

  In the above description, the image reduction unit 101 has been described as an example in which data is thinned out at equal intervals and reduced to an integer, but other reduction ratios may be employed.

  In the above description, the case where image reduction / enlargement is performed as the first embodiment, and the case where image reduction / enlargement is not performed as the second embodiment have been described. For example, FIG. A pixel number determination step is provided immediately before the image reduction step (step 151) and immediately before the luminance unevenness image enlargement step (step 158). When the number of pixels of the image is larger than the predetermined pixel number X, steps 151 and 158 are performed. When the number of pixels is less than or equal to the predetermined number of pixels X, steps 152 and 159 may be executed without executing steps 151 and 158.

  In the above description, an example in which the image processing apparatus is used as a part of a substrate inspection system has been described. However, any image processing may be used as long as it is an image process for removing luminance unevenness. A single image processing apparatus may be used.

It is a functional block diagram for demonstrating schematic structure of the board | substrate inspection system using the image processing apparatus which concerns on the 1st Embodiment of this invention. It is a functional block diagram for demonstrating the structure of the image nonuniformity removal part of the image processing apparatus which concerns on the 1st Embodiment of this invention. It is a conceptual diagram for demonstrating a brightness | luminance smoothing process. It is a schematic diagram which shows an example of an input image, a graph which shows the luminance distribution, and a graph which shows the luminance distribution of a replacement image. It is a flowchart for demonstrating the image processing method and luminance information calculation process which concern on the 1st Embodiment of this invention. 5 is a graph schematically showing an example of an input image, an enlarged luminance unevenness image, and an output image. Output image I _out (i, _{j k)} according to a first modification is a graph showing a. It is a flowchart for demonstrating the image processing method which concerns on the 2nd Embodiment of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Board | substrate inspection system 4 CCD camera 8, 240 Image memory | storage part 11 Image nonuniformity removal part (image processing apparatus)
40, 41 Defect extraction algorithm unit 50 Input image 100 Image input means 101 Image reduction means 102 Luminance information calculation means 103 Luminance replacement image creation means (replacement image creation means)
104 brightness unevenness image creation means 105 brightness unevenness image enlargement means (image enlargement means)
106 Brightness unevenness removal image creation means 107 Image output means 108 Brightness smoothing means

Claims (7)

An image processing apparatus for removing luminance unevenness of an input image,
Luminance information calculating means for calculating an average luminance and a luminance deviation representative value of the input image in a predetermined area;
Among the luminance data of the input image in the predetermined area, the specific luminance data whose deviation from the average luminance calculated by the luminance information calculation means is larger than a predetermined value is replaced with the replacement luminance data whose deviation from the average luminance is smaller. A replacement image creating means for creating a replaced image;
Brightness smoothing means for smoothing the brightness of the target pixel by the brightness value near the target pixel in the predetermined area;
A brightness unevenness image creating means for creating a brightness unevenness image by performing brightness smoothing by the brightness smoothing means on the replacement image created by the replacement image creating means;
Brightness unevenness for calculating an average brightness calculated by the brightness information calculating means and a brightness unevenness image created by the brightness unevenness image creating means for the input image to create an output image from which the brightness unevenness has been removed An image processing apparatus comprising: a removing unit.   2. The replacement image creation unit according to claim 1, wherein the luminance smoothed by the luminance smoothing unit using the specific luminance data as a pixel of interest is the replacement luminance data for the specific luminance data. Image processing device. The replacement image creating means uses, as a predetermined value for determining the specific brightness data, a boundary brightness value obtained by adding or subtracting the brightness deviation representative value to the average brightness,
The image processing apparatus according to claim 1, wherein the replacement image creating unit uses the boundary luminance value in the vicinity of the specific luminance data as the replacement luminance data for the specific luminance data. The brightness unevenness image creating means performs brightness smoothing by the brightness smoothing means on the target pixels separated from each other in the replacement image,
The luminance unevenness image is generated by interpolating the luminance of the pixel of interest after luminance smoothing by interpolating the luminance of the pixel between the pixels of interest separated from each other. An image processing apparatus according to 1. The luminance smoothing means;
5. The block according to claim 1, wherein when a repetitive pattern having a predetermined pitch exists in the input image, the size of a block in the vicinity of the luminance smoothing is set to an integral multiple of the predetermined pitch. An image processing apparatus according to 1. Image reduction means for creating a reduced input image of a predetermined image size by thinning out the input image;
Image size enlarging means for enlarging the image of the predetermined image size to the image size of the input image,
Using the reduced input image as an input image in the predetermined area, the luminance information calculation unit, the replacement image generation unit, the luminance smoothing unit, and the luminance unevenness image generation unit, to create the luminance unevenness image,
The image size of the uneven brightness image is enlarged by the image size enlargement means,
2. An output image from which brightness unevenness is removed by the brightness unevenness removing means can be created from the brightness unevenness image in which the average brightness and the image size of the reduced input image are enlarged and the input image. The image processing device according to any one of? An image processing method for removing luminance unevenness of an input image,
A luminance information calculation step of calculating an average luminance and a luminance deviation representative value of the input image in a predetermined region;
Among the luminance data of the input image in the predetermined area, the specific luminance data whose deviation from the average luminance calculated in the luminance information calculation step is larger than a predetermined value is replaced with the replacement luminance data whose deviation from the average luminance is smaller. A replacement image creation step of creating a replacement image by replacing,
A luminance smoothing step of smoothing the luminance of the target pixel by a luminance value in the vicinity of the target pixel in the predetermined region;
For the replacement image created by the replacement image creation step, a luminance unevenness image creation step of creating a brightness unevenness image by obtaining a block average brightness by the brightness smoothing step;
Brightness unevenness that creates an output image from which the brightness unevenness is removed by calculating the average brightness calculated by the brightness information calculating step and the brightness unevenness image created by the brightness unevenness image creating step for the input image An image processing method comprising: a removing step.

Images