JPWO2006006525A1 - Image processing apparatus and method - Google Patents

Abstract

The image processing apparatus of the present invention pastes partial images of a target object with a predetermined overlapping area, generates a target image of the entire target object of a predetermined size, or a part of the target object. First imaging means for obtaining first image information by photographing at a first magnification, and second for obtaining a partial image by photographing the object at a second magnification that is higher than the first magnification. An image model generation unit that generates a model of a target image generated by pasting partial images from the size of the target image and an overlap rate that is a ratio of an overlapping area in the partial image, and a partial image An imaging position calculation unit that searches for a position of the target image to be generated in the first image information by a model, and a high-definition image generation unit that generates the target image by pasting the partial images according to the layout position. Have

Description

The present invention relates to an image processing apparatus and method for capturing an image of a subject divided into a plurality of partial images and combining the captured partial images to form an entire image of the subject.
This application claims priority based on Japanese Patent Application No. 2004-203108 for which it applied on July 9, 2004, and uses the content here.

In industrial microscopes and inspection apparatuses for inspecting FPD (flat panel display) substrates, PDP (plasma display) substrates, semiconductor wafers, etc., there is a method using image information as a method for inspecting defects that cause functional problems of the substrate. Generally done.
In the above inspection, when inspecting a fine defect that affects pattern formation or the like with high accuracy, it is necessary to detect the defect by comparing the pattern to be inspected with a normal reference pattern. In many cases, “high definition (high resolution)” images that can cover the entire subject at a higher magnification are required in addition to the entire image.

However, in a high-definition image, the entire subject or a necessary range cannot be acquired at a time depending on the size of the subject.
For this reason, as one of the methods for obtaining this high-definition image, the entire subject is divided into a plurality of regions, each of these regions is imaged, and the partial images obtained by the imaging are bonded to each other to obtain a subject. A method for obtaining an entire high-definition image is often used.
In the above-described method for obtaining a high-definition image, a method of capturing a partial image with a high magnification based on an entire image with a low magnification and performing a process of combining the captured partial images is limited to industrial applications. It is used for various applications.

As a prior art, there is a method of obtaining a high-definition image by capturing a whole image and a partial image in the enlarged whole image, estimating where the partial image corresponds to the whole image, and performing a combination (for example, Patent Document 1).
There is also a method of obtaining a high-definition image by designating a part of a region from a low-magnification microscopic image, capturing the designated region with a plurality of high-magnification microscopic images, and performing a pasting process (for example, , See Patent Document 2).
JP 2000-59606 A Japanese Patent Laid-Open No. 11-271645

  However, in the high-definition image creating apparatus shown in Patent Document 1, unlike an image obtained by capturing a landscape, when an inspection apparatus for industrial use is intended for an image of a periodic pattern such as an FPD board or a PDP board, Finding where the partial images correspond to the entire image, that is, because the pattern shape is periodically the same, it becomes difficult to specify the position in the alignment when the partial images are pasted together. The case where the correspondence of the part cannot be taken occurs.

  In addition, the microscope image display device disclosed in Patent Literature 2 is an inspection device for industrial use that targets an image with a sparse pattern density. In some cases, it may occur that an attempt is made to overlap the areas where there is no overlap, and because there is no overlapping area pattern in the bonding, the bonding process cannot be performed or the bonding area is extremely shifted. There arises a defect having a defect such as generation of an image.

  The present invention has been made in view of such circumstances, and also for an image including a periodic and / or sparse part of a pattern (circuit pattern and wiring pattern) such as an FPD board or a PDP board. An image processing apparatus and method for generating a high-definition (high-resolution) image by pasting partial images are provided.

The image processing apparatus according to the present invention combines partial images of an object photographed at a predetermined resolution with a predetermined overlapping area, and generates an entire or partial target image of the predetermined size. A first imaging unit that captures first object information by photographing the object at a first magnification, and a second magnification that is higher than the first magnification. The partial image is pasted from the second photographing means for obtaining the second image information as the partial image and the size of the target image and the degree of the overlapping region in the partial image from the overlapping region information. Image model generating means for generating a model of the target image generated together, and a shooting position for searching for the position of the target image generated by pasting the partial images in the first image information using the model Calculation means (for example, implementation A pattern density evaluation value calculation unit 17 and a photographing position calculation unit 18) and a high-definition image generation unit that generates the target image by combining the partial images based on the arrangement position. .
In the image processing method of the present invention, partial images of an object photographed at a predetermined resolution are pasted together with a predetermined overlapping area to generate a target image of the entire object or a part of the object of a predetermined size. A first imaging process in which an object is photographed at a first magnification to obtain first image information, and a second magnification in which the object is magnified higher than the first magnification. The partial image is pasted from the second photographing process for obtaining the second image information as the partial image by photographing at a magnification of, and the overlapping area information indicating the size of the target image and the degree of the overlapping area in the partial image. An image model generation process for generating a model of a target image to be generated together, and a shooting position for searching for the arrangement position in the first image information of the target image generated by pasting the partial images using the model Calculation process and the placement position Based on, and having a high resolution image generation step of generating the target image by bonding the partial images.
With the configuration described above, the image processing apparatus according to the present invention forms in advance a model of a target image to be combined and combined with partial images based on the first image information with low resolution (low magnification) when the partial images are combined. In order to adjust the shooting position of the partial image for generating the high-resolution target image including the overlapping area within the predetermined area of the first image information in a wide range using this model, the high-resolution image was taken in advance. Compared with the conventional method of pasting partial images, it is possible to calculate the appropriate partial image shooting position by calculation in a wider field of view and easily generate high-definition images with the desired high resolution (high magnification). It becomes possible to do.

In the image processing apparatus of the present invention, the shooting position calculation unit searches for an arrangement position of the target image by detecting an optimum arrangement position of the overlapping region in the model combining in the first image information. It is characterized by that.
With the above-described configuration, the image processing apparatus of the present invention positively uses the overlapped portion that is overlapped and combined in searching for the photographing position of the partial image when combining (that is, the overlapping region at the time of combining). The image pattern part that makes it easy to combine images can be extracted), so when generating the target image, the position of combining the partial images, that is, the accuracy of pasting overlapping parts is improved, making it easier than before It is possible to generate a desired high-resolution high-definition image with high accuracy.

In the image processing apparatus of the present invention, the shooting position calculation means searches for the arrangement position of the overlapping area while moving the model by a predetermined moving distance in the search area set in advance of the first image information. It is characterized by that.
With the configuration described above, the image processing apparatus of the present invention sets a search area of a predetermined size in advance, particularly when generating a high-definition image of an object composed of a repetitive pattern, In the above, the model is moved in a defined direction from a predetermined position by a predetermined moving distance (for example, in units of a plurality of pixels) while searching for the arrangement position of the overlapping region, so that the search process can be speeded up. It becomes possible.

The image processing apparatus according to the present invention is characterized in that the shooting position calculation means searches for an arrangement position of the overlapping area in the search area based on the pattern information of the overlapping area.
With the configuration described above, the image processing apparatus according to the present invention sets the arrangement position of the overlapping area based on the pattern information of the overlapping area (for example, the pattern density evaluation value indicating the density of the pattern). Since it is possible to detect a dense position, when pasting partial images, it is possible to select a position where the pasting can be easily performed as an overlapping area, and a desired high resolution with high accuracy. A fine image can be generated.

The image processing apparatus of the present invention is characterized in that the shooting position calculation means searches the arrangement position by changing the overlapping area information in the model in the search area based on the pattern information of the overlapping area. .
With the above-described configuration, the image processing apparatus according to the present invention can perform overlapping area information, for example, overlapping of overlapping areas, necessary for partial image combining processing according to image pattern information (for example, pattern density information). Since the rate is changed, it is possible to change the pattern information to a numerical value suitable for matching as necessary regardless of whether the pattern of the substrate is rough or dense, and the position of the optimum partial image, That is, the generation position of the target image can be calculated, and a high-definition image can be easily generated.

The image processing apparatus of the present invention has moving means for moving the object relative to the first photographing means and the second photographing means in a predetermined distance unit in the XY directions, The shooting position calculation means sets the shooting position of the target image on the object based on the arrangement position of the target image detected by the model.
With the above-described configuration, the image processing apparatus of the present invention has a relative moving unit, and therefore, when the shooting position is detected, it is possible to perform processing to move to that position and take a picture. This processing can be performed in real time, and the generation speed of a high-resolution high-definition image can be improved.

The image processing apparatus according to the present invention is characterized in that a photographing position of a partial image used for pasting is calculated based on the photographing position and an arrangement position of a target image detected by the model.
With the configuration described above, the image processing apparatus of the present invention sets the position of the overlapping area by the model, and therefore detects the position where the pattern of the overlapping area is dense, so that the shooting position of the target image is calculated. Thus, the shooting position of the partial image used for pasting can be easily calculated, and a high-definition image with a desired high resolution can be generated with high accuracy.

The image processing apparatus according to the present invention is characterized in that each of the first image information and the second image information obtained by the first and second photographing means has been subjected to distortion correction or / and shading correction. To do.
With the above-described configuration, the image processing apparatus of the present invention can generate a high-definition image that is not affected by distortion or shading.

  As described above, according to the present invention, when the partial images are combined, a model of the target image to be combined and combined with the partial images is formed in advance based on the low-resolution first image information. In order to adjust the shooting position of the partial image for generating the high-resolution target image including the overlapping area within the predetermined area of the first image information over a wide range, Thus, it is possible to obtain an appropriate partial image capturing position by calculation, and it is possible to easily generate a desired high-resolution high-definition image.

FIG. 1 is a conceptual diagram illustrating a configuration example of a microscope apparatus according to an embodiment of the present invention. FIG. 2 is a block diagram illustrating a configuration example of the image processing unit 5 of FIG. FIG. 3 is a conceptual diagram for explaining a model generated by the image model generation unit 16 of FIG. FIG. 4 is a conceptual diagram for explaining the Sobel filter. FIG. 5 is a conceptual diagram for explaining the pattern density evaluation value. FIG. 6 is a flowchart illustrating an operation example of the microscope apparatus including the image processing unit 5 according to the first embodiment. FIG. 7 is a conceptual diagram illustrating the operation of the image processing unit 5 according to the first embodiment. FIG. 8 is a conceptual diagram illustrating the operation of the image processing unit 5 according to the first embodiment. FIG. 9 is a conceptual diagram illustrating the operation of the image processing unit 5 according to the first embodiment. FIG. 10 is a conceptual diagram illustrating the maximum value detection process within the pattern density evaluation value search region. FIG. 11 is a flowchart illustrating an operation example of the microscope apparatus including the image processing unit 5 according to the second embodiment. FIG. 12 is a conceptual diagram illustrating the operation of the image processing unit 5 according to the second embodiment. FIG. 13 is a conceptual diagram illustrating the operation of the image processing unit 5 according to the second embodiment. FIG. 14 is a conceptual diagram illustrating the operation of the image processing unit 5 according to the second embodiment. FIG. 15 is a conceptual diagram illustrating the operation of the image processing unit 5 according to the second embodiment. FIG. 16 is a conceptual diagram illustrating the operation of the image processing unit 5 according to the second embodiment. FIG. 17 is a conceptual diagram illustrating the operation of the image processing unit 5 according to the second embodiment. FIG. 18 is a conceptual diagram for explaining the maximum value and the minimum value of the overlapping rate of partial image frames. FIG. 19 is a conceptual diagram illustrating an inspection apparatus according to the third embodiment. FIG. 20 is a conceptual diagram illustrating an inspection apparatus according to the fourth embodiment.

Explanation of symbols

1 Tube
2 Objective lens
3 Imaging camera
4 stages
5 Image processing section
6 Stage movement controller
7 System controller
8 Microscope Z-axis Movement Control Unit 11 Imaging Control Unit 12 Shading / Distortion Correction Processing Unit 13 Captured Image Data Storage Buffer Unit 14 First Captured Image Reading Unit
15 Second captured image reading unit 16 Image model generation unit
17 pattern density evaluation value calculation unit 18 photographing position calculation unit
19 Image generation unit 20 Image storage unit
F1, F2, F3, F4 Partial image frame

<First embodiment>
An image processing apparatus according to a first embodiment of the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram showing a configuration example of the embodiment.
In this figure, the first embodiment is provided with the image processing function of the present invention with respect to a microscope, and the microscope has a lens barrel 1 to which an objective lens 2 is attached in the Z-axis direction (up and down as viewed from the figure). A vertical drive mechanism that can be driven is provided.
The microscope Z-axis movement control unit 8 controls the vertical drive mechanism to move the lens barrel 1 up and down to adjust the focus on the object placed on the stage 4.

The stage 4 is provided in the lower part of the microscope, and has a mechanism (biaxial movement drive mechanism) that drives in the X direction and the Y direction (left and right direction and depth direction as seen from the figure), and is observed in the upper part. The object, which is a sample for use, is placed.
The stage movement control unit 6 performs movement control on the two axes of the stage 4 and adjusts the relative position between the objective lens 2 and the object.
Further, an imaging camera 3 is provided on the upper part of the lens barrel 1, and a video signal (image signal) output from the imaging camera 3 is transferred to the image processing unit 5 to perform various image processing.
The imaging camera 3 is a CCD camera and outputs, for example, gradation (luminance) data for each RGB-compatible pixel as image information.
The image processing unit 5, the stage movement control unit 6, and the microscope Z-axis movement control unit 8 are controlled by the system control unit 7 as necessary.

Next, the image processing unit 5 according to the first embodiment of the present invention will be described with reference to the drawings. FIG. 5 is a block diagram illustrating a configuration example of the image processing unit 5 of the embodiment.
The portion surrounded by the wavy line is the image processing unit 5, and the imaging control unit 11, the shading / distortion correction processing unit 12, the captured image data storage buffer unit 13, the first captured image reading unit 14, and the second captured image reading unit 15. , An image model generation unit 16, a pattern density evaluation value calculation unit 17, a shooting position calculation unit 18, an image generation unit 19, and an image storage unit 20.

The imaging control unit 11 controls the magnification of the objective lens 2 by changing the lens and adjusts the focus by the microscope Z-axis movement control unit 8 under the control of the system control unit 7, and the low-magnification image information captured by the imaging camera 3. (First image information, that is, an entire image obtained by photographing the entire object) or high-magnification image information (second image information, that is, a partial image) is input and output to the shading / distortion correction processing unit 12. .
The shading / distortion correction processing unit 12 performs shading correction and distortion correction for shading and distortion generated from the imaging system including the objective lens 2 on each of the first image information and the second image information, and then the captured image data. In the storage buffer unit 13, information on the magnification is added and stored.
This magnification information is added to the first image information and the second image information as lens information of the objective lens 2 in the imaging control unit 11 via the system control unit 7.

The first captured image reading unit 14 reads the first image information indicating that the added magnification information indicates a low magnification from the captured image data storage buffer unit 13, and temporarily stores the first image information.
The second captured image reading unit 15 reads second image information (hereinafter referred to as a partial image) whose added magnification information is a high magnification from the captured image data storage buffer unit 13, and temporarily stores the partial image. To do.

The image model generation unit 16 generates a model of the target image that is finally generated by pasting the partial images. This model includes an overlapping area that becomes an overlapping portion when the partial images are pasted together.
In other words, the image model generation unit 16 generates a first magnification as a low magnification set in advance by the user, a second magnification as a high magnification, and a partial image, which are input from the system control unit 7. The model is generated from the image size to be overlapped and the size of the overlapping area to be overlapped when pasting.

The pattern density evaluation value calculation unit 17 reads a model from the image model generation unit 16, reads first image information from the first captured image reading unit 14, and has a search area for searching for a part that generates a target image. It is set in the first image information by the system control unit 7 (set while the user confirms the screen).
Further, as shown in FIG. 3, the pattern density evaluation value calculation unit 17 has a predetermined movement distance, for example, a plurality of pixel units, in the search area, with the model as a predetermined position, for example, the upper left of the search area. The pattern density evaluation value (pattern information) in the overlapping area is calculated while moving in the X-axis direction and the Y-axis direction, and these are sequentially stored in correspondence with the calculated positions.

Here, the movement distance in the search area may be performed in units of one pixel (pixel), but depending on the target pattern, there is no change before and after the movement, and the obtained pattern density evaluation value is almost the same. Therefore, in order to reduce useless calculation time and improve the search efficiency of the overlapping area, the unit is set to a predetermined number of pixels.
As the moving distance, if the object is a periodic pattern as in this embodiment, it is 1/5, 1/10, 1/50, 1/100,... Of the number of pixels forming one period. Thus, it is set according to the number of pixels of the pattern period.

Further, if the minimum size of the target pattern included in the overlapping region (for example, the width of the signal line through which the current flows) is known, the number of pixels of the minimum pattern width is 1, 2 or 3 times. As described above, the movement distance may be set according to the size of the pattern.
The moving distance according to the size of the pattern takes into consideration that the pattern density evaluation value changes due to the entire pattern appearing or disappearing from the overlapping area before and after moving.

The pattern density evaluation value is calculated for each block (horizontal direction and vertical direction) of the size of the partial image, with four overlapping calculation units (vertical direction and horizontal direction described later) for each overlapping position, that is, for each calculation position. Direction edge strength).
Here, the pattern density evaluation value is calculated in the pattern density evaluation value calculation unit 17 according to the flow shown below.
In the present embodiment, the pattern density evaluation value is obtained by paying attention to the edge strength for each direction (the magnitude of the luminance change in the pattern).

The edge strength for each direction represents the edge strength with respect to the vertical (up and down the screen) direction and the horizontal (left and right of the screen) direction.
A Sobel filter is used as a method for calculating the edge strength. This Sobel filter multiplies nine pixel values in the vicinity, that is, adjacent top, bottom, left, and right, centering on a certain target pixel, by multiplying each by a coefficient (center is the target pixel) mask as shown in FIG. Then, this processing is performed using two coefficient matrices in the vertical direction and the horizontal direction.

  That is, the pixel (X, Y) (X is a coordinate value on the screen in the horizontal direction, where the right direction is positive, the left direction is negative, and Y is the vertical direction on the screen. The edge intensity in each direction with respect to the origin, with the downward direction being positive and the upward direction being negative), the luminance value of the pixel (X, Y) is expressed as I (X, Y), and The absolute value of the numerical value R is obtained as Abs (R) as the horizontal strength EH (X, Y) and the vertical strength EV (X, Y) by the following equations.

Intensity EH (X, Y) = Abs {I (X + 1, Y-1) + 2 × I (X + 1, Y) + I (X + 1, Y + 1)
-I (X-1, Y-1) -2 * I (X-1, Y) -I (X-1, Y + 1)}
Intensity EV (X, Y) = Abs {I (X-1, Y + 1) + 2 × I (X, Y + 1) + I (X + 1, Y + 1)
−I (X−1, Y−1) −2 × I (X, Y−1) −I (X + 1, Y−1)}

Using the above formula, calculation is performed for each pixel in the target region (overlapping region) (at this time, the edge portion of the image cannot be calculated because the edge intensity cannot be calculated).
Here, the pattern density evaluation value calculation unit 17 adds the calculated edge strength for each pixel in the direction of the target unit in the direction unit, the horizontal edge strength total value AEH, and the vertical edge strength total value. Find AEV.
However, the edge strength in either direction may be extremely low. For example, as shown in FIG. 5, if the inside of the overlapping region is formed only with the horizontal line pattern, the pattern in the vertical direction Since there is an edge, a certain numerical value of the edge strength total value AEV is obtained, but the horizontal edge strength total value AHE is almost “0” because there is no change in luminance in the horizontal direction.
In this case, if an attempt is made to perform the pasting process using only a one-way pattern, the matching portion cannot be limited, and thus an optimal matching process cannot be performed.

For this reason, the pattern density evaluation value calculation unit 17 obtains a predetermined threshold value Thres for the edge intensity total value AEH and the edge intensity total value AEV, and this threshold value is obtained only when each edge intensity total value is equal to or greater than the threshold value Thres. A value equal to or greater than Thres is output as a pattern density evaluation value.
This threshold value Thres is expressed by the following equation, taking into consideration the influence of noise, for example, where Q is the minimum luminance difference detected as the edge of the Sobel filter for one pixel.
Thres = 4 · Q × (number of pixels subject to Sobel filter operation in overlapping region)
A value obtained by multiplying the threshold Thres by a predetermined coefficient is used as the actual threshold. As a coefficient to be multiplied, a small value such as 1 or 2 may be used in order to suppress the influence of noise.
Further, when an area having a high edge strength, that is, a pattern having a clear feature is extracted, a large value corresponding to the value of Q (the luminance gradation is 256, and if Q is 10, 10 to 15 Value between).

Further, the pattern density evaluation value calculation unit 17 calculates the pattern density evaluation value PDEV using the above formula as follows.
If AEH <Thres or AEV <Thres, set PDEV to 0,
When AEH ≧ Thres and AEV ≧ Thres, PDEV = AEH + AEV.
As a result, the calculated pattern density evaluation value is likely to fail in pattern matching when pasting partial images when either one of the edge strengths is lower than a threshold value (extremely small). Only the numerical values of the arrangement positions at which accurate pattern matching can be performed remain in the pasting process after being excluded from the target of later evaluation.
Here, the higher the edge strength is, the larger the luminance difference is, and the accuracy of matching the partial images in the overlapping area in the pasting is improved.
In other words, even in the case of an entire image having a relatively sparse pattern, it is possible to search for the position of the model where the overlapping area pattern is dense.

Next, returning to FIG. 2, the imaging position calculation unit 18 captures the target image at the position with the largest pattern density evaluation value selected by the pattern density evaluation value calculation unit 17, that is, the first image information (entire image). Based on the position, the shooting location of each partial image is obtained, and shooting location information of this shooting location is output to the system control unit 7.
In the image generation unit 19, the system control unit 7 controls the microscope Z-axis movement control unit 8, the stage movement control unit 6, the imaging control unit 11, and the imaging camera 3 based on the imaging position output by the imaging position calculation unit 18. Control and paste together partial images taken for multiple blocks of the model.
The image storage unit 20 stores a target image (high-definition image) generated by pasting partial images in the image generation unit 19.
The system control unit 7 reads out the target image from the image storage unit 20 and displays it on a display device (not shown) by access from the user.

  As the pattern information, in the first embodiment described above, the edge strength indicating the change in the luminance value of the image is used. However, the “spatial characteristic” of the direction based on the luminance value of the image is used. Rather than an evaluation value consisting solely of luminance values, for example, a luminance average value, a difference between the minimum and maximum values (dynamic range), a mode value, and a median value for a histogram formed from the luminance values of the image The variance (standard deviation) can be used as pattern information.

Here, the pattern frequency evaluation value PDEV is formed from the frequency FR and the standard deviation SD using the frequency FR of the histogram relating to the luminance value of the mode value regarding the overlapping region and the standard deviation SD of the histogram as the pattern information. Express the power of as 2 ^ x.
PDEV = FR + (2 ^ x) × SD
The pattern density evaluation value PDEV represents 0 to 2 ^ x-1 with a frequency FR and 2 ^ x or more with a standard deviation SD. However, in the above formula, “FR <2 ^ x”.
Thus, first, the pattern density evaluation value calculation unit 17 pays attention only to the frequency FR, determines whether or not it is a predetermined threshold value or more, and if it is equal to or higher than this threshold value, also evaluates the standard deviation.
Since the maximum value of FR is (2 ^ x), the above-described search method is possible.
Further, the above-described operation is realized by bit operation (logical operation), and pattern information having different characteristics can be used as one pattern density evaluation value.

Next, the operation of the above-described image processing apparatus will be described with reference to FIGS. FIG. 6 is a flowchart specifically showing an operation example of the image processing apparatus according to the first embodiment of FIG.
Here, the FPD board shown in FIG. 7 will be described as an example of the object. In the FPD substrate, pixel portions and transistors for driving the pixels are periodically arranged.
The user uses an input device (not shown) as a processing parameter to the system control unit 7 as a processing parameter, the overall image magnification (first magnification), the partial image magnification (second magnification), and the combined image (target image). And the overlap rate of each partial image are set (step S1).

Next, when the process of acquiring the target image is started, the system control unit 7 drives the stage 4 by the stage movement control unit 6, adjusts the relative position between the objective lens 2 and the target object, and The objective lens 2 is switched so as to achieve the first magnification.
Then, the system control unit 7 adjusts the focus by moving the lens barrel 1 up and down via the microscope Z-axis movement control unit 8 to capture the entire image shown in FIG. 7 of the object, and the imaging control unit 11. The whole image is transferred to the shading / distortion correction processing unit 12 via
Further, with respect to the entire image shown in FIG. 7, a field frame as a partial image (shootable range when captured at a second magnification: partial image frame) is an area within a broken line shown in FIG.

Then, the shading / distortion correction processing unit 12 performs distortion correction and shading correction on the entire input image, and temporarily stores them in the captured image data storage buffer unit 13 (step S2).
Next, the image model generation unit 16 calculates the figure based on the size of the target image (the number of vertical pixels × the number of horizontal pixels) and the overlapping rate of the partial images when the target image is generated. A model of a target image having an overlapping area as shown in FIG. 3 is generated (step S3).

Then, the image model generation unit 16 calculates the number of partial images and the size of the overlapping region so that the overlapping region has the above-described overlapping rate with respect to the partial image (step S4).
For example, in FIG. 9, the frame of the partial image from the partial image frame shown in FIG. 8 is 4, and the overlapping area is a hatched portion where any two or more of the four partial image frames overlap each other (the shape of + in the center of the screen). Defined as).
As a result, in this embodiment, the target image is formed by four partial images, and the model of the size of the target image is configured by four partial image frames.
Next, in the entire image displayed on the display device, the user sets a search area for searching for the shooting position of the target image from the model (step S5).
This search area may be an entire part of the entire image as long as it is larger than the size of the model.

  Then, the pattern density evaluation value calculation unit 17 calculates the pattern density evaluation value at each movement position while moving (shifting) the X density in the X axis direction and the Y axis direction at a predetermined movement distance, and the entire search range. The above processing is repeated until a search is performed, and the calculated pattern density evaluation value is sequentially associated with the coordinate value (calculated position) in the entire image, stored in the internal storage unit, and the pattern density in the entire search region When the evaluation value calculation is completed, the process proceeds to step S7 (step S6).

Next, the pattern density evaluation value calculation unit 17 searches for the largest value among the pattern density evaluation values stored in the internal storage unit, and pastes the coordinate values corresponding to the searched pattern density evaluation value. Is output as the optimum position of the target image (step S7).
At this time, as shown in FIG. 10, the pattern density evaluation value calculation unit 17 is a three-dimensional graph showing the size of the pattern density evaluation value in the Z-axis direction for each evaluated coordinate value (on the XY plane). The pattern density evaluation values at the respective coordinate values are sequentially compared to search for the maximum pattern density evaluation value.
For example, in FIG. 10, since the pattern density evaluation value on the left side is the maximum, the position of this model, that is, the coordinate value is output as the generation position of the optimum target image.

Next, the shooting position calculation unit 18 calculates the shooting position of the partial image from the generation position of the target image output by the pattern density evaluation value calculation unit 17 (step S8).
At this time, the shooting position calculation unit 18 uses the arrangement position of the partial image frame of the model at the generation position as the shooting position of the partial image to be shot at the second magnification (high magnification). The coordinate value of the partial image frame corresponding to each image is output as the partial image position.
In the present embodiment, since the target image is composed of four partial images, the photographing position calculation unit 18 outputs the coordinate values of the partial image frames corresponding to the four partial images to the system control unit 7. To do.

Next, the system control unit 7 changes the objective lens 2 to a lens corresponding to the second magnification via the microscope Z-axis movement control unit 8 and corresponds to the partial image position input from the imaging position calculation unit 18. Then, the stage 4 is moved to the coordinate position where the imaging camera 3 takes an image via the stage movement control unit 6, the focus is adjusted by the microscope Z-axis movement control unit 8, and each partial image is taken by the imaging camera 3. I do.
Here, the system control unit 7 captures all of the plurality of partial images constituting the target image by the processing described above.
The imaging control unit 11 outputs a partial image each input from the imaging camera 3 to Shedin Gu distortion correction processing unit 12.
Accordingly, the shading / distortion correction processing unit 12 performs distortion correction and shading correction processing on the sequentially input partial images, and stores them in the captured image data storage buffer unit 13 (step S9).

Next, the image processing unit 5 reads a partial image constituting the target image from the captured image data buffer unit 13 and temporarily stores it in the second captured image reading unit 15.
Then, the image generation unit 19 sequentially reads out the partial images from the second captured image reading unit 15, and based on the model shown in FIG. 9, that is, for each partial image frame of the model, at the partial image position of this partial image frame. Correspondingly photographed partial images are arranged, the partial images are pasted together to generate a target image, and the generated high-definition target image is stored in the image storage unit 20 (step S10).

At this time, the image generation unit 19 superimposes the patterns arranged in the overlapping region, performs pattern matching, and aligns the positions for pasting. For this reason, it is necessary to use an area where the pattern density evaluation value exceeds a predetermined density, that is, an area exceeding a predetermined threshold, as the overlapping area.
Therefore, in the present embodiment, since the arrangement position of the overlapping region to be bonded is determined by the pattern information such as the pattern density evaluation value, it is formed on the substrate like the FPD substrate pattern shown in FIG. A bonded image can be generated with high accuracy even for an object having many periodic patterns and having a periodicity that is not suitable for the bonding process.
The system control unit 7 reads the target image from the image storage unit 20 as necessary, and displays the target image on the display unit.

<Second embodiment>
The second embodiment has the same configuration as that of the first embodiment, and only differences from the first embodiment will be described below.
FIG. 11 is a flowchart specifically showing an operation example in the second embodiment.
The difference is that step S8 of the first embodiment is changed to step S15, and this step will be described.
In the first embodiment, the partial image position for acquiring a high-definition partial image, that is, the photographing position is determined based on the position of the partial image frame in the model from the optimal position of the model to which the partial images are pasted. The search in the search area is performed with the overlapping area between the partial image frames in the model fixed.
On the other hand, in the second embodiment, when the optimum position of the stitching model is determined, the position of the partial image frame constituting the stitching model is determined using the low-magnification whole image (first image information). is doing.

In step S6, a pattern density evaluation value at each position is calculated while moving the search area by a predetermined movement distance using a fixed overlapping area model.
At this time, the minimum pattern density threshold PDEV_Min is set as an expression shown below.
PDEV_Min = AEH_Min + AEV_Min
= 4 · Q × PixNum + 4 · Q × PixNum
= 8 ・ Q × PixNum
Here, PixNum = “the number of pixels subject to Sobel filter calculation in the overlapping region”
Then, the pattern density evaluation value calculation unit 17 calculates the pattern density evaluation value at each moving position, repeats the above processing until the entire search range is searched, and sequentially calculates those exceeding the threshold PDEV_Min. The obtained pattern density evaluation value is associated with the coordinate value (calculated position) in the entire image, stored in the internal storage unit, and when the calculation of the pattern density evaluation value in the entire search area is completed, the process proceeds to step S7.

In step S7, the pattern density evaluation value calculation unit 17 selects and outputs the largest pattern density evaluation value from the internal storage unit, as in the first embodiment.
Next, in step S15, the pattern density evaluation value of the overlapping region in the model of coordinate values corresponding to the selected pattern density evaluation value is recalculated.
At this time, the overlapping regions of the partial image frames in the model are the region A by the partial image frames F1 and F2 in FIG. 12, the region B by the partial image frames F3 and F4 in FIG. 13, and the region by the partial image frames F1 and F3 in FIG. C, a region D is formed by the partial image frames F2 and F4 in FIG.
About these, the pattern density evaluation value of each area | region A-D is calculated from the image of the position corresponding to the whole image of a low magnification with respect to each partial image frame.

Here, the pattern density evaluation value calculation unit 17 determines whether each of the areas A to D exceeds a predetermined threshold value.
This threshold value is obtained as a pattern density evaluation value in the horizontal direction and the vertical direction in the first embodiment, but in the second embodiment, the threshold values of two partial image frames adjacent in the horizontal direction and the vertical direction are obtained. Since this is an overlapping area unit, the value is defined by the following formula.
Thres2 = 2 · Q × (the number of pixels subject to Sobel filter calculation in the overlapping region)
Then, when the pattern density evaluation value calculation unit 17 detects that the pattern density of all the regions A to D exceeds the threshold value, the process proceeds to step S9, and thereafter the same process as in the first embodiment is performed. Do.
A value obtained by multiplying the threshold Thres2 by a predetermined coefficient is used as the actual threshold. As a coefficient to be multiplied, a small value such as 1 or 2 may be used in order to suppress the influence of noise.
Further, when an area having a high edge strength, that is, a pattern having a clear feature is extracted, a large value corresponding to the value of Q (the luminance gradation is 256, and if Q is 10, 10 to 15 Value between).

On the other hand, when the pattern density evaluation value calculation unit 17 detects that the pattern density evaluation value of the area A indicated by the oblique lines in FIG. 16 does not exceed the threshold value Thres2 with respect to the area that does not exceed the threshold value Thres2, for example, FIG. As shown in FIG. 5, the partial image frame F1 is moved rightward by a predetermined movement distance, and the area of the region A that is an overlapping region of the partial image frame F1 and the partial image frame F2 is expanded.
Then, the pattern density evaluation value calculation unit 17 calculates the pattern density evaluation value of the region A again, detects whether or not the pattern density evaluation value exceeds the threshold value Thres2, and detects that it exceeds the threshold value Thres2. Then, the process proceeds to step S9, and if not exceeded, the partial image frame F1 is moved rightward again to determine the pattern density evaluation value of the area A.

Here, the limitation on the overlapping rate of the overlapping area when the partial image frame is moved will be described with reference to the drawings. FIG. 18 is a conceptual diagram for explaining the overlapping rate of overlapping regions.
As the maximum value of the overlapping rate, 50% is appropriate if the same pattern is included in the two partial images.
That is, if the overlapping rate is 50% or more, the same pattern is included in the three partial images.
Then, when the overlapping rate of the overlapping area exceeds the maximum value, the pattern density evaluation value calculating unit 17 performs the above-described process again on the coordinates of the model having the second largest numerical value of the pattern density evaluation value of the overlapping area.

On the other hand, as the minimum value of the overlapping rate, a prescribed value of the number of pixels that is a real number multiple of 1 or more is provided for the number of pixels of the minimum pattern in the pattern formed on the substrate, and the ratio of the prescribed value in the entire partial image Ask.
For example, if the size of the partial image is 640 (horizontal direction) × 480 (vertical direction) pixels and the minimum pattern is 4 (horizontal direction) × 4 (vertical direction) pixels, the number of pixels is twice that of the minimum pattern. Is the specified value.
Accordingly, the minimum overlap ratio in the horizontal direction is (4 × 2/640) = 1.25%, and the minimum overlap ratio in the vertical direction is (4 × 2/480) = 1.67%. .

  As described above, in the second embodiment, when determining the optimum position of the model for pasting, when all the pattern density evaluation values of the regions A to D in the model do not exceed the threshold in the entire search region, Initially, the position where the evaluation value is the maximum as a whole is determined in the entire image, and the position of the partial image frame is adjusted so that the pattern density evaluation value exceeds the threshold value for the overlapping portion of each partial image frame at this coordinate value Thus, the overlapping position of the overlapping area is changed, and the shooting position of the partial image to be shot is determined.

Further, in order to adjust the pattern density evaluation value in a timely manner by changing the overlapping rate of the overlapping area, the degree of freedom of search is increased compared to the first embodiment, and even when an appropriately set shooting position is started (the entire search area) Without searching for the image), it is possible to automatically determine the optimum photographing position for the pasting using the pattern density evaluation value.
Therefore, in the present embodiment, since the arrangement position of the overlapping region to be bonded is determined by the pattern information such as the pattern density evaluation value, it is formed on the substrate like the FPD substrate pattern shown in FIG. Even with objects that have many patterns that are sparse and have periodicity that is not suitable for the bonding process, or when the bonding process fails at an appropriately set overlapping part, the pattern is pasted with high accuracy. A combined image can be generated.

<Third embodiment>
The third embodiment shown in FIG. 19 is a large substrate inspection apparatus equipped with a microscope. The substrate inspection apparatus shown in FIG. 19 is the same as the first and second embodiments in the configuration of an observation system such as a microscope, objective lens 2, and imaging camera 3.
The difference is a drive mechanism for moving the FPD substrate of the object relative to the objective lens 2, and the stage movement control unit 6 moves the stage 4 on which the object is disposed in one axial direction (in FIG. Lower left direction: Drive only in arrow O).
On the other hand, the system control unit 7 drives the microscope T itself in one axis direction in a direction perpendicular to the stage 4 (upper left-lower right direction: arrow P in FIG. 19).
Thereby, the relative position of the objective lens 2 and the object can be moved in the XY direction.

<Fourth embodiment>
The target image, which is a high-definition image generated in the first to third embodiments, is a reference image (from a normal substrate for comparison) that is compared with the image of the substrate being inspected when detecting a defect in the substrate. The generated image) is used in the inspection apparatus.
For example, in the inspection of the FPD substrate, the inspection apparatus shown in FIG. 20 is provided with a line sensor as an imaging unit. After the imaging unit is adjusted with a calibration sample, the stage is moved in the direction of arrow G by the holding and moving unit. The reflected light of the light radiated from the illumination means is detected by the line sensor at every predetermined moving distance.
Then, the integrated control means compares the detected intensity of the reflected light with the detected value of the reflected light sampled immediately before, and if the difference exceeds a predetermined range, it is detected as a defective candidate, The coordinate value at is stored.

Then, the FPD board is placed on the stage 4 of the image processing apparatus in the first to third embodiments, and the coordinate value of the defect candidate is input to the system control unit 7.
As a result, the system control unit 7 moves the stage 4 via the stage movement control unit 6, and the position of the defect candidate moves to the position of the objective lens 2, that is, the position where the imaging candidate can be imaged by the imaging camera 3. To do.
At this time, the system control unit 7 has generated the target image as a high-definition image, that is, in a state including the position of the defect candidate by the second magnification, that is, in the first to third embodiments, that is, optimal Move to the location corresponding to the position of the correct model.

Then, the system control unit 7 compares the image information including the captured defect candidates with the target images generated in the first and second embodiments by pattern matching, and determines the pattern shape of the defect candidates and the reference image. Is compared with the pattern shape of the corresponding part of the target image, and it is determined whether or not they are different.
At this time, if the system control unit 7 detects that the difference is not different, the system determines that the defect candidate is a non-defective product. On the other hand, if the system control unit 7 detects a difference, the system control unit 7 determines that the defect candidate is defective. Display on the display device.
By the above-described inspection method of the present invention, what is determined as a defective candidate in the high-speed inspection by the line sensor is accurately determined by comparing it with an actual normal substrate pattern, thereby improving the inspection speed and inspecting. It is possible to improve the accuracy.

  A program for realizing the functions of the image processing unit in FIGS. 1 and 2 is recorded on a computer-readable recording medium, and the program recorded on the recording medium is read into a computer system and executed. Image processing may be performed. Here, the “computer system” includes an OS and hardware such as peripheral devices. The “computer system” includes a WWW system having a homepage providing environment (or display environment). The “computer-readable recording medium” refers to a portable medium such as a flexible disk, a magneto-optical disk, a ROM, and a CD-ROM, and a storage device such as a hard disk built in the computer system. Further, the “computer-readable recording medium” refers to a volatile memory (RAM) in a computer system that becomes a server or a client when a program is transmitted via a network such as the Internet or a communication line such as a telephone line. In addition, those holding programs for a certain period of time are also included.

  The program may be transmitted from a computer system storing the program in a storage device or the like to another computer system via a transmission medium or by a transmission wave in the transmission medium. Here, the “transmission medium” for transmitting the program refers to a medium having a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line. The program may be for realizing a part of the functions described above. Furthermore, what can implement | achieve the function mentioned above in combination with the program already recorded on the computer system, and what is called a difference file (difference program) may be sufficient.

  According to the image processing apparatus and the image processing method of the present invention, when combining partial images, a model of a target image to be combined and combined with the partial images is formed in advance based on the low-resolution first image information. A wide field of view based on the first image information is used to adjust the shooting position of the partial image for generating the high-resolution target image including the overlapping region within the predetermined region of the wide range of the first image information using the model. In the region, an appropriate photographing position of the partial image can be obtained by calculation, and a desired high-resolution high-definition image can be easily generated.

Claims (11)

An image processing apparatus that generates a target image of a whole or a part of the target with a predetermined size by pasting partial images of the target captured at a predetermined resolution with a predetermined overlap area.
First imaging means for imaging a subject at a first magnification to obtain first image information;
Second imaging means for imaging the object at a second magnification that is higher than the first magnification, and obtaining second image information as the partial image;
Image model generation means for generating a model of the target image generated by pasting the partial images from the size of the target image and the image region information indicating the degree of the overlapping region in the partial image;
Shooting position calculation means for searching for an arrangement position in the first image information of the target image generated by pasting the partial images using the model;
An image processing apparatus comprising: a high-definition image generation unit that generates the target image by combining the partial images based on the arrangement position.   The image processing apparatus according to claim 1, wherein the photographing position calculation unit detects an optimal arrangement position of an overlapping region in the pasting of the models in the first image information, so that the arrangement position of the target image is detected. Image processing device for searching   The image processing apparatus according to claim 2, wherein the shooting position calculation unit moves the model by a predetermined moving distance within a preset search area of the first image information, and arranges the overlapping area. Image processing device for searching   4. The image processing apparatus according to claim 2, wherein the photographing position calculating unit searches for an arrangement position of the overlapping area in the search area based on the pattern information of the overlapping area.   5. The image processing device according to claim 1, wherein the shooting position calculation unit changes the overlapping area information in the model in the search area based on the pattern information of the overlapping area, thereby determining the arrangement position. An image processing device to search for. 6. The image processing apparatus according to claim 1, wherein the object is relative to the first imaging unit and the second imaging unit in a predetermined distance unit in the XY direction. Moving means to move to
An image processing apparatus that sets a shooting position of a target image on the object based on an arrangement position of the target image detected by the shooting model by the shooting position calculation unit.   The image processing apparatus according to claim 6, wherein a photographing position of a partial image used for pasting is calculated based on the photographing position and an arrangement position of a target image detected by the model.   8. The image processing apparatus according to claim 1, wherein each of the first image information and the second image information obtained by the first and second imaging units includes distortion correction and / or shading. Corrected image processing device. An image processing method for generating a target image of a whole or a part of the target object having a predetermined size by pasting partial images of the target object photographed at a predetermined resolution with a predetermined overlap area,
A first imaging process in which an object is photographed at a first magnification to obtain first image information;
A second imaging process in which the object is imaged at a second magnification that is higher than the first magnification to obtain second image information as the partial image;
An image model generation process for generating a model of the target image generated by pasting the partial images from the size of the target image and the overlapping region information indicating the degree of the overlapping region in the partial image;
A shooting position calculation process for searching for an arrangement position in the first image information of the target image generated by pasting the partial images using the model;
And a high-definition image generation step of generating the target image by combining the partial images based on the arrangement position.   10. The image processing method according to claim 9, wherein, in the shooting position calculation process, an optimum arrangement position of an overlapping area in the pasting of the models in the first image information is detected, thereby arranging the arrangement position of the target image. Image processing method for searching.   The image processing method according to claim 10, wherein in the shooting position calculation process, the model is moved by a predetermined moving distance within a preset search area of the first image information, and an overlapping area is detected. An image processing method for searching for an arrangement position.

Images