JPH09265537A - Image processing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To detect a nonrepetition part such as defect developed on an object in repetitive structure and a foreign matter, and emphasizes and display it. SOLUTION: An image of the object in repetitive structure is fetched by a video camera 2 and converted by an A/D conversion part 3 into digital data, which is stored in a frame memory 4. A repetitive pattern detection part 15 provided for an image processing part 5 fetches the image data stored in the frame memory 4 and calculates the repetition pitch, and a block division part 52 divides the data into areas consisting of a given number of pixels according to the pitch. Then an averaging process part 52 calculates the mean image data of the blocks, an image arithmetic part 54 calculates the difference image between the input image data and mean image data, and the nonrepetition part is emphasized and displayed at a display part 6.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing method.

[0002]

2. Description of the Related Art As a method for inspecting a fine pattern having a repetitive structure formed on a semiconductor wafer, the difference has been observed so far by observing the surface with an optical microscope or comparing it with an adjacent pattern or an adjacent chip. I was judging a certain part.

[0003]

However, the image processing technique using the optical microscope as described above has some problems as described below.

That is, in the method in which a person observes the surface by using an optical microscope, it is difficult to judge a minute defect (foreign matter) or the like generated on a pattern having a repeating structure.
That is, since the entire pattern has a repeating structure, it takes a long time to determine a defect.

In addition, since information on adjacent patterns and adjacent chips can be used for inspection of semiconductor wafers, etc., a defect image and a normal image are captured, and an image of the difference between these two images (hereinafter referred to as difference image) is displayed. The method has been adopted.
However, the method of simultaneously capturing the images of the adjacent chips has a problem that the cost of the apparatus is increased and the processing time is increased. There is also a method of moving the chip or the optical system to capture two images in order, but there is a problem in that high accuracy is required for alignment when moving and processing time increases.

An object of the present invention is to provide a method and apparatus for detecting a pattern such as a defect or a foreign substance by a simple image processing by using an image of a defective portion generated on a pattern having a repetitive structure and a surrounding normal portion. To provide.

[0007]

The outline of typical ones of the present invention will be briefly described as follows.

That is, the image processing method of the present invention includes a step of capturing input image data, a step of calculating a pitch of a repeating pattern from the captured image data, and a plurality of pieces of image data captured based on the pattern pitch. Divided into blocks, a step of calculating average image data of the plurality of blocks, a step of calculating the input image data and the average image data, and a step of displaying the image data of the calculation result. is there.

Further, the second method of image processing of the present invention is to detect a foreign portion which is not a repetitive pattern included in the input image data and remove the influence of this portion. Specifically, a step of estimating an area in which the difference between the input image data and the average image data is outside the set value range, and a second step of calculating the average image data of the repeated pattern area that does not include this area inside And a step of calculating the input image data and the second average image data, and a step of displaying the image data of the calculation result.

Next, the process of calculating the pitch (Px, Py) of the repetitive pattern, which is important in these image processing methods, will be described. Where the horizontal repeat pitch is P
Let x be the repeating pitch in the vertical direction and Py be the vertical pitch.

One method of calculating the pitch (Px, Py) of the repetitive pattern is to use one-dimensional or two-dimensional frequency conversion. The frequency conversion methods include (fast) Fourier transform, discrete (fast) Fourier transform, discrete cosine (sine) transform, Hadamard transform, Haar transform,
There are Walsh transform and wavelet transform. By such a frequency conversion method, the input image including the repeating pattern is frequency-converted. The maximum value of the frequency component may be calculated from the converted data to determine the pitch (Px, Py) of the repeating pattern.

Next, a second method for calculating the pitch of the repeating pattern will be described. In this case, the method is applicable when the entire screen is not rotated, that is, when the repetitive pattern is not tilted in the horizontal and vertical directions with respect to the screen. In this method, image data is integrated (projected) in the horizontal and vertical directions, and the pitch of a repeating pattern is determined from the maximum value or the minimum value of the integrated data. FIG.
FIG. 4 shows an example in which the input image data is integrated in the horizontal direction and the vertical direction with respect to the input image including the repeated pattern twice each in the horizontal direction and the vertical direction. From this result, it can be seen that the maximum (minimum) value occurs twice in both the horizontal and vertical directions, and it can be determined that the horizontal and vertical pitches are Px and Py, respectively.

Further, as a third method for calculating the pitch of the repeating pattern, there is a method of calculating the correlation of the image data. For example, as shown in FIG. 7, an input image (A) including a repeating pattern twice each in the horizontal direction and the vertical direction.
think of. A partial area (arbitrary area) of this image is set as a reference image (B) as shown in FIG. The correlation coefficient between the reference image (B) and the input image (A) is calculated while changing the position of the reference image (B). As the correlation coefficient, the sum of the squared errors of the two image data or the sum of the absolute values of the differences may be calculated. The result is shown in the correlation processing of FIG. The outline of the correlation coefficient of "H" in the horizontal direction is shown on the lower side, and the outline of the correlation coefficient of "V" in the vertical direction is shown on the right side. The repeating pitches Px and Py can be determined from the pitch at which the correlation coefficient is maximum and the position.

As a method of calculating the pitches Px and Py of the repeating pattern, it is possible to more accurately determine the repeating pitches Px and Py by applying three kinds of methods in combination. For example, it is possible to make a more precise determination by applying a method of integrating image data in the horizontal direction and the vertical direction and then applying a method of utilizing frequency conversion or a method of utilizing correlation processing.

Using the pitches Px and Py of the repeating pattern calculated by the above method, the image data is divided into a plurality of blocks with the repeating pitch as a unit. If the average value of the image data of the divided blocks is calculated and the input image data and this average image data are calculated (difference), only the foreign matter (irregular portion) on the repeated pattern can be obtained.

According to the image processing technique of the present invention, image data is taken in by a photographing means such as a TV camera, divided into a plurality of regions based on the repeating pattern pitch of the image data, and an average image of the corresponding portions. Calculate the data. Furthermore, by calculating the input image data and this average image data, the foreign matter (irregular part) on the repeating pattern is calculated, and by displaying only this part with emphasis, the defective part is shown to humans. It is possible to display in a form that is easy to understand.

Furthermore, if an area in which the difference between the input image data and the average image data is equal to or more than a set value can be estimated, the average image level can be recalculated for the area excluding only this area. Since this is image data that is not affected by foreign matter, it is possible to more accurately estimate the position and pattern of the defect.

Further, by displaying the brightness or the color in accordance with the size of the foreign matter and the like, it is possible to emphasize and display the feature of the defect.

[0019]

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, description will be given with reference to specific drawings of the present invention.

(Embodiment 1) FIG. 1 is a block diagram of an image processing apparatus according to Embodiment 1 of the present invention, and FIG. 2 is Embodiment 1 of the present invention.
3 is a detailed block diagram of the image processing unit according to FIG. 3, FIGS. 3 to 5 are explanatory diagrams showing an example of the repetitive pattern detection unit, and FIGS. It is a figure.

In the first embodiment, the output signal of the video camera 2 connected to the semiconductor inspection device or the like is connected to the A / D converter 3 for converting an analog signal into a digital signal. A
A memory 4 is provided after the / D converter 3 and
The image data output from the / D converter 3 is stored in the memory 4
Stored in. A line memory or a frame memory may be used as the memory 4. The image data at an arbitrary position stored in the memory 4 is read out by the image processing unit 5 and subjected to image processing as necessary. The image processing unit 5 is, for example, a display unit (display means) such as a monitor.
The image data processed by the image processing unit 5 is displayed.

The details of the image processing section 5 are shown in FIG. The image processing unit 5 includes a repeated pattern detection unit 51, a block division unit 52, an averaging processing unit 53, and an image calculation unit 54. The memory 4 will be described below on the assumption that it is a two-dimensional frame memory, but even in the case of a one-dimensional line memory, image processing can be realized by substantially the same processing. In addition, the target image includes a repeating pattern that is repeated a plurality of times in the horizontal and vertical directions in the image size.

The repetitive pattern detecting section 51 shown in FIG.
Then, in the image data stored in the memory 4, X
It is calculated whether or not a constant repeating pattern exists in the (horizontal) direction and the Y (vertical) direction. An example of this repeated pattern detection unit 51 is shown in FIGS.

FIG. 3 shows an example in which the method of detecting a repetitive pattern by one-dimensional or two-dimensional frequency conversion is applied.
The image data read from the memory 4 is used as the frequency conversion unit 7
Enter 1 In the frequency converter 71, for example (high speed)
Fourier transform is performed. In addition to this, frequency conversion methods include discrete (fast) Fourier transform, discrete cosine transform,
Hadamard transformation, Haar transformation, Walsh transformation and the like are known. By such a frequency conversion method, the input image including the repeating pattern is frequency-converted. The frequency-converted data is output to the pitch detector 72. The pitch detector 72 determines the pitches Px and Py of the repeating pattern based on information such as the maximum value of the frequency component. The result is output to the block division unit 52.

FIG. 4 shows an example of realizing a repeating pattern detecting method by integration (projection). The image data read from the memory 4 is input to the horizontal direction integration unit 73 and the vertical direction integration unit 74. In these circuits, the horizontal integration unit 7
In 3, the image data is integrated in the horizontal direction, and in the vertical integration unit 74, the image data is integrated in the vertical direction. The integration section
For example, it can be realized by a line memory and an addition circuit. The integrated data is output to the pitch detection unit 72, and the pitch of the repeating pattern is determined from the maximum value or the minimum value of the integrated data. FIG. 6 shows an example of an image of a repeating pattern which is repeated twice in each of the horizontal direction and the vertical direction. With respect to this input image, image data is integrated in the horizontal and vertical directions. These integrated data are output to the pitch detection unit 72,
The pitch Px, Py of the repetitive pattern is determined from the maximum (or minimum) value in both the horizontal and vertical directions. Further, by applying the frequency conversion (for example, Fourier transform) as shown in FIG. 3 to the integrated value in the horizontal and vertical directions, the pitch of the repetitive pattern with high accuracy can be determined.

FIG. 5 shows an example in which a method of detecting a repetitive pattern by correlation calculation of image data is applied. The image data read from the memory 4 is input to the correlation calculation unit 75 and the reference image setting unit 76. The reference image setting unit 76 selects the set arbitrary region and outputs it to the correlation calculation unit 75. The correlation calculator 75 calculates a correlation coefficient with the input image data while moving the reference image in the horizontal and vertical directions. For the correlation coefficient, the sum of the squared errors of the corresponding two pieces of image data or the sum of the absolute values of the differences may be calculated. The result is output to the pitch detection unit 72, and the pitches Px and Py of the repetitive pattern are started from the position where the correlation coefficient has the maximum value.
Should be determined. FIG. 7 shows an example of the input image (A) including a repeating pattern twice in each of the horizontal direction and the vertical direction. A part of this image is set as the reference image of (B). The correlation coefficient between the reference image of (B) and the input image of (A) is calculated while changing the position of the reference image of (B). This result is shown as the correlation processing of (C). Here, the two-dimensional display is not shown, but the outline of the correlation coefficient of the horizontal “H” line is shown on the lower side, and the outline of the correlation coefficient of the vertical “V” line is shown on the right side. As shown in this figure, the correlation calculation becomes maximum at two positions in both the horizontal and vertical directions, so that the repeating pitches Px and Py can be determined from this position. The correlation coefficient need not be calculated over the entire screen, and may be calculated for each set pixel or line.

Returning to FIG. 2, the block division unit 52 divides the image data into a plurality of blocks based on the pitches Px and Py based on the repetition pitches Px and Py calculated by the above method. 8A to 8D schematically show the relationship between the input image data and the processed data according to the first embodiment of the present invention. In the input image of FIG. 8A, the repeating pattern is M (= 2) times in the X direction and N (= 2) times in the Y direction. Therefore, the area shown by the dotted line is the basic repeating pattern (basic block). In addition, the case where there is a foreign matter (defect) in the lower right is shown.

Returning to FIG. 2, the averaging unit 53 calculates the average value of the image data of the divided blocks, and the average value is used as the image data of each block as in the average image of FIG. 8B. . In this case, the influence of foreign matter (defect) may appear. If the size of the foreign matter is smaller than one block, and the number of repetitions is M times in the horizontal direction and N times in the vertical direction, a total of MN times, the influence of the foreign matter is 1 / (MN-1). Therefore, when the MN is large, this is not a big problem. A countermeasure for removing this effect will be described later in the second embodiment of the present invention.

Next, the processing result of the averaging processor 53 and the image data read from the memory 4 are input to the image calculator 54. The image calculation unit 54 outputs the result of calculation (difference) between the average image data and the input image data to the display unit 6. At this time, as shown in the difference image of FIG. 8C, if processing such as adding a level of 1/2 of the maximum value of the brightness level is performed, a human can judge when the processed image is displayed on the display unit. Easier to do. As a result, the areas other than the foreign matter are flat (gray), and only the foreign matter is clearly displayed.

Even when the pattern color or line width slightly changes depending on the screen location instead of the clear foreign substance as shown in FIG. 8, only the difference from the average value of the repeated pattern can be displayed. It becomes possible, and is easy for humans to judge.

Further, in the image calculation section 54, (C) of FIG.
By encircling a portion in which the absolute value of the difference image data is out of the set level range with a circle or a rectangle, only the foreign matter is emphasized and displayed as shown in the emphasized image of FIG. be able to. At this time, it is convenient for a human to observe by calculating the center of gravity of the area where the image data is equal to or more than the set value and displaying a circle or a rectangle centering on the center of gravity in the image. In addition, even if a block including an area equal to or larger than the set value is changed in color or brightness level and displayed on the image, the display is easy for a human to judge.

(Embodiment 2) FIG. 9 is a block diagram of an image processing unit 5 according to Embodiment 2 of the present invention.
(E) is an explanatory view showing the relationship between image data and processed data.

In the second embodiment, the processing of the image processing unit 5 is 2
Processed in stages. That is, the image processing unit 5 includes the repetitive pattern detection unit 51, the block division unit 52, and the averaging processing unit 5 surrounded by the dotted line similar to those shown in the first embodiment.
3, an image calculation unit 54, and a foreign substance region estimation unit 55, a reblock division unit 56, a re-averaging processing unit 57, and an image recalculation unit 58. Since the operation of the part surrounded by the broken line is the same as that of the first embodiment, it is omitted. The output image data of the image calculation unit 54 is shown in the difference image in FIG. 10C, but almost only the foreign matter is displayed. Therefore, the foreign matter region estimation unit 55 determines whether or not this data is within the set range. For example, in the example shown in FIG. 10, it is determined that there is a foreign substance in the lower right block. Based on this result, the re-block division unit 56
Then, as shown in the re-averaging image of FIG. 10D, the re-averaging processing unit 57 for the image data other than the lower right block.
The averaging process is performed with. When the size of the foreign matter is 1 block or less, the average value of (MN-1) blocks is obtained for the image repeated M times in the horizontal direction and N times in the vertical direction.

Further, the image re-calculation unit 58 calculates a difference image between the output data of the re-averaging processing unit 57 and the data in the lower right portion of the output signal (input image) of the memory 4 to calculate the difference image as shown in FIG.
It is output to the display unit 6 as the emphasized image of (E). At this time, similarly to the first embodiment, it is convenient for a human to observe by calculating the center of gravity of the area where the image data is equal to or larger than the set value and displaying a circle or a rectangle centered on the center of gravity in the image. is there. In addition, even if a block including an area equal to or larger than the set value is changed in color or brightness level and displayed on the image, the display is easy for a human to judge. An example of the emphasized image is shown in FIG.
Although only one block is shown as in (E), it goes without saying that four blocks may be displayed as in the input image.

Next, when the foreign substance is spread over two or more blocks, the blocks are shifted and subdivided under the condition that the repeating pitch is the same. This situation is shown in Figure 11.
It shows in (A)-(E). In the example shown here, there is a foreign substance in the center of the right side, so it will span the first two blocks. Therefore, as shown in the difference image in (C), the block delimiters are shifted and redivided. Based on this,
As shown in the (D) re-averaged image, the re-averaging processing unit 57 may perform the averaging process on the image signals other than the right center block. The other processing is the same as that shown in FIG.

The functions of the first and second embodiments can be processed by separating them according to color components or wavelengths.

Next, an example in which the present invention is realized by software will be described with reference to the flowcharts of FIGS.

FIG. 12 shows the procedure of image processing according to the first embodiment of the present invention. Image capturing 81, A / D conversion
Up to 82 and memory writing 83 are realized by hardware.
The subsequent image processing enclosed by the dotted line is realized by software.

The data is read from the memory and the repeating pattern is calculated 84. Details of this calculation method are shown in FIG.
This will be described with reference to FIG. The image data is divided into blocks 85 using the determined repeating pitch. This block-unit data is read from the memory and the average value of the block image is calculated 86. Next, the difference calculation 87 between the input image and the average image is performed, and the data is output 88 to the display unit.

FIG. 13 shows a flow of detecting a repetitive pattern by two-dimensional frequency conversion. Image data is read out from the memory 101, and two-dimensional frequency conversion 102 is performed. next,
The repetition pitch may be determined 104 from the data after the two-dimensional frequency conversion.

FIG. 14 shows a flow of detecting a repeating pattern by integration (projection). The image data is read out from the memory 111, the image data in the horizontal direction is integrated 112, and the horizontal pitch is calculated 113 from the maximum (or minimum) value of this data. Next, the image data in the vertical direction is integrated 114, and the vertical pitch is calculated 115 from the maximum (or minimum) value of this data.

FIG. 15 shows a flow of detecting a repetitive pattern by the correlation calculation of image data. The image data is read 121 from the memory and the reference image data is determined 122. Next, the comparison image data is read 123 and the correlation calculation 124 is performed. From this result, search for the peak value in the horizontal direction.
Then, the horizontal pitch is determined 126. Further, the peak value in the vertical direction may be searched 125 and the vertical pitch may be determined 126.

FIG. 12 shows the procedure of image processing according to the second embodiment of the present invention, in which image capturing 81 and A / D conversion are performed.
Up to 82 and memory writing 83 are realized by hardware.
The subsequent image processing is realized by software.

The data is read from the memory and the repeating pattern is calculated 84. Details of this calculation method are shown in FIG.
This will be described with reference to FIG. The image data is divided into blocks 85 using the determined repeating pitch. This block-unit data is read from the memory and the average value of the block image is calculated 86. Next, the difference calculation 87 between the input image and the average image is performed, and the data is output 88 to the display unit.

In the second embodiment, the processing of the image processing section is performed in two stages. That is, the repetitive pattern calculation 84, the block division 85, the block image averaging 86, and the difference calculation 87 between the input image and the average image, which are surrounded by the dotted lines similar to those shown in the first embodiment, are performed. A foreign substance region outside the set range is estimated 88 in this processed image, and the image data is re-block divided 89 based on this result. The re-averaging process 90 is performed on the image data that does not include the foreign matter region. Further, the difference calculation 91 between the input image and the average image is performed, and the data is output 92 to the display unit.

[0046]

According to the present invention, [1] it is possible to detect and display a non-repeating portion such as a defect or a foreign substance generated on an object having a repeating structure with a simple circuit configuration and system configuration.

[2] Non-repeating portions such as defects and foreign matters generated on an object having a repetitive structure are detected, and only the repetitive portions are reprocessed, so that non-repeating portions such as defects and foreign matters are reduced with a small error. The part can be detected and displayed.

[3] The image information of the non-repeated portion to be displayed can be displayed while being distinguished by brightness modulation, color modulation, etc., and the shape of the DUT can be more easily recognized.

[4] Due to [1] and [2], the cost of the apparatus using the image processing technique such as the foreign substance inspection apparatus can be reduced, and the working time for human judgment can be greatly shortened.

[Brief description of drawings]

FIG. 1 is a block diagram of an image processing apparatus according to a first embodiment of the present invention.

FIG. 2 is a block diagram of an image processing unit in the image processing apparatus according to the first embodiment of the present invention.

FIG. 3 is a block diagram of an image processing unit in the image processing apparatus of the present invention.
The block diagram by the frequency conversion system of a repeated pattern detection part.

FIG. 4 shows an image processing unit of the image processing apparatus according to the present invention.
FIG. 3 is a block diagram of a repeating pattern detection unit according to an integration method.

FIG. 5 shows an image processing unit of the image processing apparatus according to the present invention.
FIG. 4 is a block diagram of a correlation calculation method of a repetitive pattern detection unit.

FIG. 6 shows an image processing unit of the image processing apparatus according to the present invention,
Explanatory drawing which shows the principle of the integration system of a repeated pattern detection part.

FIG. 7 shows an image processing unit of the image processing apparatus of the present invention.
Explanatory drawing which shows the principle of the correlation calculation system of a repeated pattern detection part.

FIG. 8 is an explanatory diagram of image data and processed data according to the first embodiment of the present invention.

FIG. 9 is a block diagram of an image processing unit in the image processing apparatus according to the second embodiment of the present invention.

FIG. 10 is an explanatory diagram of image data and processed data according to the second embodiment of the present invention.

FIG. 11 is an explanatory diagram showing another processing status of image data and processing data according to the second embodiment of the present invention.

FIG. 12 shows image data processing according to the first embodiment of the present invention.
Flow chart to realize with software.

FIG. 13 is a flowchart for calculating a repeating pattern of image processing according to the present invention by a frequency conversion method.

FIG. 14 is a flowchart for calculating a repeating pattern of image processing of the present invention by an integration method.

FIG. 15 is a flowchart for calculating a repeating pattern of image processing according to the present invention by a correlation calculation method.

FIG. 16 shows image data processing according to the second embodiment of the present invention.
Flow chart to realize with software.

[Explanation of symbols]

2 ... Video camera, 3 ... A / D conversion unit, 4 ... Frame memory, 5 ... Image processing unit, 6 ... Display unit, 51 ... Repeat pattern detection unit, 52 ... Block division unit, 53 ... Averaging processing unit, 54 ... Image calculation unit.

 ─────────────────────────────────────────────────── ─── Continuation of front page (72) Inventor Takuo Okahashi 5-22-1, Josuihoncho, Kodaira-shi, Tokyo Inside Hitachi Microcomputer System Co., Ltd. No. 20-1 Hitachi, Ltd. Semiconductor Division

Claims (7)

[Claims] 1. A step of capturing image data, a step of calculating a pitch of a repeating pattern from the captured image data, and a step of dividing the captured input image data into a plurality of blocks based on the pitch of the pattern. When,
An image processing method comprising: a step of calculating average image data of the plurality of blocks; a step of calculating the input image data and the average image data; and a step of displaying image data of the calculation result. 2. A step of fetching image data, a step of calculating a pitch of a repeating pattern from the fetched image data, and a step of dividing the fetched image data into a plurality of blocks based on the pitch of the pattern. When,
A first step of calculating average image data of the plurality of blocks; a step of calculating difference image data between the image data and the average image data; and an area in which an absolute value of the difference image data is outside a setting range. And a second step of calculating average image data of an area that does not include a block outside the set range inside, the captured image data and the second
An image processing method, comprising: a step of calculating the average image data of 1 .; and a step of displaying the image data of the calculation result. 3. The step of calculating the pitch of the repetitive pattern according to claim 1 or 2, wherein the step of performing one-dimensional or two-dimensional frequency conversion, and the repetitive pattern from the maximum value of the frequency component after the frequency conversion. Image processing method, which is the step of calculating the pitch of the. 4. The step of calculating the pitch of the repetitive pattern according to claim 1, wherein the step of integrating data in the horizontal direction and / or the vertical direction and the maximum value or the minimum value of the data after the integration are performed. An image processing method that is a step of calculating the pitch of a repeating pattern. 5. The step of calculating the pitch of the repeating pattern according to claim 1 or 2 comprises the step of calculating the autocorrelation of horizontal and / or vertical data, and the step of calculating the pitch of the repeating pattern from the autocorrelation. An image processing method that is a calculation step. 6. The step of calculating the pitch of the repeating pattern according to claim 1 or 2, wherein:
An image processing method, which is a step of combining any two or three of the above. 7. The above steps according to claim 1 or 2,
An image processing method in which processing is performed according to color components or wavelengths.