WO2006006525A1

WO2006006525A1 - Image processing device and method

Info

Publication number: WO2006006525A1
Application number: PCT/JP2005/012661
Authority: WO
Inventors: Kazuhito Horiuchi
Original assignee: Olympus Corporation
Priority date: 2004-07-09
Filing date: 2005-07-08
Publication date: 2006-01-19
Also published as: KR20070026792A; JP4709762B2; JPWO2006006525A1; CN1981302A; TW200606753A; TWI366150B; CN1981302B; KR100888235B1

Abstract

An image processing device includes: first imaging means for bonding partial images of an object with a predetermined duplicate area so as to generate an entire or partial object image of the object of a predetermined size and imaging the object with a first magnification so as to obtain first image information; second imaging means for imaging the object with a second magnification higher than the first magnification so as to obtain a partial image; image model generation means for generating a model of the object image generated by bonding the partial images from the size of the object image and the duplicate ratio as a ratio of the duplicate area in the partial image; imaging position calculation means for searching the arrangement position of the object image generated from the partial image in the first image information by using a model; and highly accurate image generation means for generating the object image by bonding the partial images according to the arrangement position.

Description

Specification

Image processing apparatus and method

Technical field

[0001] The present invention relates to an image processing apparatus and method for capturing an image of a subject divided into a plurality of partial images and combining the captured partial images to form an entire image of the subject.

This application claims priority based on Japanese Patent Application No. 2004-203108 filed on July 9, 2004, the contents of which are incorporated herein by reference.

Background art

[0002] Image information is used as a method for inspecting defects that cause functional problems in substrates in industrial microscopes and inspection devices that inspect FPD (flat panel display) substrates, PDP (plasma display) substrates, semiconductor wafers, etc. In general, when a fine defect that affects pattern formation or the like is inspected with high accuracy, the pattern to be inspected is compared with a normal reference pattern. Therefore, it is necessary to detect defects, so that a high-definition (high-resolution) image that can cover the entire subject at a higher magnification is required than just an image that covers the entire subject at a lower magnification. Many cases are coming.

However, in a high-definition image, depending on the size of the subject, the entire subject or a necessary range cannot be obtained at a time.

For this reason, as one method for obtaining this high-definition image, the entire subject is divided into a plurality of regions, each of these regions is imaged, and the partial images obtained by the imaging are bonded to each other to obtain a subject. A method for obtaining an entire high-definition image is often used.

In the above method for obtaining a high-definition image, a method is often used in which a partial image with a high magnification is picked up based on an entire image with a low magnification and the picked-up partial images are combined and processed only for industrial use. It is used for various applications.

[0004] As a prior art, taking a whole image and a partial image in the enlarged whole image, estimating where the partial image corresponds to the whole image, and pasting them together to obtain a high-definition image There is a law (see, for example, Patent Document 1).

There is also a method for obtaining a high-definition image by designating a part of a region from a low-magnification microscopic image, capturing the designated region with a plurality of high-magnification microscopic images, and performing a pasting process (for example, , See Patent Document 2).

Patent Document 1: Japanese Patent Laid-Open No. 2000-59606

Patent Document 2: Japanese Patent Laid-Open No. 11-271645

Disclosure of the invention

Problems to be solved by the invention

[0005] However, in the high-definition image creation device shown in Patent Document 1, unlike an image obtained by capturing a landscape, an inspection device for industrial use is a periodic pattern such as an FPD substrate or a PDP substrate. When targeting images, it is necessary to find out where the partial images correspond to the entire image, that is, because the pattern shapes are periodically the same, when aligning the partial images, alignment is required. In this case, it becomes difficult to specify the position in the case, and the case where the overlapped part cannot be dealt with occurs.

[0006] In addition, the microscope image display device shown in Patent Document 2 is an inspection device for industrial use and targets an image with a sparse pattern density. In some cases, it may happen that there are no overlapping patterns where there are no turns, and there is no overlapping area pattern in the pasting, so the pasting process cannot be performed! There is a defect such as / ヽ, or a combined image in which the bonded area is extremely shifted is generated!

[0007] The present invention has been made in view of such circumstances, and an image including a periodic and Z or sparse portion of a pattern (circuit pattern and wiring pattern) such as an FPD substrate or a PDP substrate. The present invention also provides an image processing apparatus and method for generating a high-definition (high-resolution) image by pasting partial images.

Means for solving the problem

[0008] The image processing apparatus of the present invention combines partial images of an object photographed at a predetermined resolution with a predetermined overlap area, and pairs all or a part of the object with a predetermined size. An image processing device that generates an elephant image, which is a first image obtained by photographing an object at a first magnification. First imaging means for obtaining image information; and second object for obtaining second image information as the partial image by photographing the object at a second magnification that is higher than the first magnification. An image model generating unit that generates a model of the target image generated by pasting the partial images from the overlapping region information based on the size of the target image and the degree of the overlapping region in the partial image; and the partial image Shooting position calculation means for searching for the arrangement position in the first image information of the target image generated by pasting the images using the model (for example, the pattern density evaluation value calculation unit 17 and the shooting position calculation in the embodiment) Unit 18) and high-definition image generation means for generating the target image by combining the partial images based on the arrangement position.

In the image processing method of the present invention, partial images of an object photographed at a predetermined resolution are pasted together with a predetermined overlapping area, and a target image of the entire object or a part of the object of a predetermined size is attached. An image processing method to be generated, wherein a first photographing process of photographing a subject at a first magnification to obtain first image information, and the subject at a magnification higher than the first magnification. From the second photographing process for obtaining the second image information as the partial image by photographing at the second magnification, and the size of the target image and the degree of the overlapping region in the partial image are used as the overlapping region information. An image model generation process for generating a model of a target image generated by pasting images and an arrangement position of the target image generated by pasting the partial images in the first image information using the model. The shooting position calculation process to be searched and the position And a high-definition image generation process for generating the target image by pasting the partial images based on the position.

With the configuration described above, the image processing apparatus according to the present invention forms in advance a model of a target image to be combined by combining the partial images based on the first image information of low resolution (low magnification) when the partial images are combined. Thus, in order to adjust the shooting position of the partial image that generates the high-resolution target image including the overlapping area within the predetermined area of the first image information in a wide range using this model, Compared to the conventional method of pasting together the captured partial images, it is possible to obtain the appropriate partial image shooting position by calculation in a wider field of view, and to obtain a high-definition image with the desired high resolution (high magnification). Can be generated easily. [0009] In the image processing apparatus of the present invention, the photographing position calculation unit detects an optimum arrangement position of an overlapping region in the model image combining in the first image information, thereby The arrangement position is searched.

With the above-described configuration, the image processing apparatus of the present invention positively uses the overlapping portion that is overlapped and combined when searching for the shooting position of the partial image (that is, the overlapping region at the time of combining). Therefore, it is easier to compare compared with the past by improving the accuracy of pasting of partial images, that is, overlapping portions when generating target images. In addition, it is possible to generate a high-definition image having a desired high resolution with high accuracy.

[0010] In the image processing apparatus of the present invention, the shooting position calculation means arranges overlapping areas while moving the model by a predetermined moving distance within a preset search area of the first image information. It is characterized by searching for a position.

With the above-described configuration, the image processing apparatus of the present invention sets a search area of a predetermined size in advance, particularly when generating a high-definition image of an object composed of a repetitive pattern. The search process is performed at high speed in order to search for the arrangement position of the overlapping region while moving the model in a defined direction from the predetermined position at a predetermined moving distance (for example, in units of a plurality of pixels). It becomes possible to do.

The image processing apparatus according to the present invention is characterized in that the photographing position calculation means searches for an arrangement position of the overlapping area in the search area based on the pattern information of the overlapping area.

With the configuration described above, the image processing apparatus according to the present invention sets the position of the overlapping area based on the pattern information of the overlapping area (for example, the pattern density evaluation value indicating the density of the pattern). Since it is possible to detect a dense position, when pasting partial images, it is possible to select a position where the positioning of the pasting can be easily performed as the overlapping region, and to achieve a desired high accuracy and high accuracy. It is possible to generate high-resolution images with high resolution.

[0012] In the image processing apparatus of the present invention, the imaging position calculation unit changes the overlapping area information in the model in the search area based on the pattern information of the overlapping area. Thus, the arrangement position is searched.

With the above-described configuration, the image processing apparatus according to the present invention is capable of overlapping area information, for example, overlapping, required for the partial image combining processing according to image pattern information (for example, pattern density information). Since the overlap rate of the area is changed, it is possible to change the pattern information to a value suitable for matching as needed, regardless of whether the substrate pattern is rough or dense. The position of the image, that is, the generation position of the target image can be calculated, and a high-definition image can be easily generated.

[0013] The image processing apparatus of the present invention has moving means for moving the object relative to the first photographing means and the second photographing means in a predetermined distance unit in the XY directions. Shi

The shooting position calculation means sets the shooting position of the target image on the target based on the arrangement position of the target image detected by the model.

With the configuration described above, since the image processing apparatus of the present invention has a relative moving unit, when the shooting position is detected, it is possible to perform processing for moving to that position and shooting. It is possible to improve the generation speed of high-resolution and high-definition images by performing shooting processing in real time.

[0014] The image processing apparatus of the present invention is characterized in that, based on the photographing position and the arrangement position of the target image detected by the model, the photographing position of the partial image used for the pasting is calculated.

With the configuration described above, the image processing apparatus of the present invention sets the position of the overlapping area by the model, and therefore detects the position where the pattern of the overlapping area is dense. Thus, it is possible to easily calculate the shooting position of the partial image used for pasting, and it is possible to generate a high-definition image having a desired high resolution with high accuracy.

[0015] In the image processing apparatus of the present invention, the first image information and the second image information obtained by the first and second imaging units are each subjected to distortion correction or Z and shading correction. It is characterized by that.

With the configuration described above, the image processing apparatus of the present invention can generate V and high-definition images that are not affected by distortion or shading. The invention's effect

[0016] As described above, according to the present invention, when the partial images are combined, a model of the target image to be combined and combined with the partial images is formed in advance based on the low-resolution first image information, Using this model, in order to adjust the shooting position of the partial image that generates the high-resolution target image including the overlapped area within the predetermined area of the wide range of the first image information, In the field of view, it is possible to obtain an appropriate photographing position of the partial image by calculation, and it is possible to easily generate a high-definition image having a desired high resolution.

Brief Description of Drawings

FIG. 1 is a conceptual diagram showing a configuration example of a microscope apparatus according to an embodiment of the present invention.

FIG. 2 is a block diagram showing a configuration example of the image processing unit 5 in FIG.

FIG. 3 is a conceptual diagram for explaining a model generated by the image model generation unit 16 of FIG.

FIG. 4 is a conceptual diagram for explaining a Sobel filter.

FIG. 5 is a conceptual diagram for explaining a pattern density evaluation value.

FIG. 6 is a flowchart showing an operation example of the microscope apparatus including the image processing unit 5 according to the first embodiment.

FIG. 7 is a conceptual diagram for explaining the operation of the image processing unit 5 according to the first embodiment.

FIG. 8 is a conceptual diagram illustrating the operation of the image processing unit 5 according to the first embodiment.

FIG. 9 is a conceptual diagram illustrating the operation of the image processing unit 5 according to the first embodiment.

FIG. 10 is a conceptual diagram illustrating maximum value detection processing within a search area for pattern density evaluation values.

FIG. 11 is a flowchart showing an operation example of the microscope apparatus including the image processing unit 5 according to the second embodiment.

FIG. 12 is a conceptual diagram for explaining the operation of the image processing unit 5 according to the second embodiment.

FIG. 13 is a conceptual diagram for explaining the operation of the image processing unit 5 according to the second embodiment.

FIG. 14 is a conceptual diagram for explaining the operation of the image processing unit 5 according to the second embodiment.

FIG. 15 is a conceptual diagram for explaining the operation of the image processing unit 5 according to the second embodiment. FIG. 16 is a conceptual diagram for explaining the operation of the image processing unit 5 according to the second embodiment.

FIG. 17 is a conceptual diagram illustrating the operation of the image processing unit 5 according to the second embodiment.

FIG. 18 is a conceptual diagram for explaining the maximum value and the minimum value of the overlapping rate of partial image frames.

FIG. 19 is a conceptual diagram for explaining an inspection apparatus according to a third embodiment.

FIG. 20 is a conceptual diagram for explaining an inspection apparatus according to a fourth embodiment.

Explanation of symbols

[0018] 1 lens barrel

2 Objective lens

3 Imaging camera

4 stages

5 Image processing section

6 Stage movement controller

7 System controller

8 Microscope Z-axis movement controller

11 Imaging control unit

12 Shading 'Distortion correction processing section

13 Captured image data storage buffer

14 First shot image reading section

15 Second shot image reading section

16 Image model generator

17 Pattern density evaluation value calculator

18 Shooting position calculator

19 Image generator

20 Image storage

Fl, F2, F3, F4 Partial image frame

BEST MODE FOR CARRYING OUT THE INVENTION

[0019] <First embodiment> An image processing apparatus according to a first embodiment of the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram showing a configuration example of the embodiment.

In this figure, the first embodiment is a microscope equipped with the image processing function of the present invention. The microscope is provided with a lens barrel 1 for attaching an objective lens 2 in the Z-axis direction (up and down as viewed in the figure). A vertical drive mechanism capable of being driven is provided.

The microscope Z-axis movement control unit 8 controls the vertical drive mechanism to move the lens barrel 1 up and down to adjust the focus on the object placed on the stage 4.

[0020] The stage 4 is provided in the lower part of the microscope, and has a mechanism (two-axis movement drive mechanism) for driving in the X direction and the Y direction (left and right direction and depth direction as seen from the figure). The target object, which is a sample for observation, is placed on the top.

The stage movement control unit 6 performs movement control of the stage 4 in two axes, and adjusts the relative position between the objective lens 2 and the object.

In addition, an imaging camera 3 is provided on the upper part of the lens barrel 1, and a video signal (image signal) output from the imaging camera 3 is transferred to the image processing unit 5 for various image processing. .

The imaging camera 3 is a CCD camera and outputs, for example, gradation (luminance) data for each RGB-compatible pixel as image information.

The image processing unit 5, the stage movement control unit 6, and the microscope Z-axis movement control unit 8 are controlled by the system control unit 7 as necessary.

Next, the image processing unit 5 according to the first embodiment of the present invention will be described with reference to the drawings. FIG. 5 is a block diagram illustrating a configuration example of the image processing unit 5 of the embodiment.

The portion surrounded by the wavy line is the image processing unit 5, the imaging control unit 11, the shading 'distortion correction processing unit 12, the captured image data storage buffer unit 13, the first captured image reading unit 14, and the second captured image reading. The image forming apparatus includes a unit 15, an image model generation unit 16, a pattern density evaluation value calculation unit 17, a shooting position calculation unit 18, an image generation unit 19, and an image storage unit 20.

The imaging control unit 11 is controlled by the system control unit 7, the magnification is changed by exchanging the objective lens 2, and the focus is adjusted by the microscope Z-axis movement control unit 8, and the image is captured by the imaging camera 3. Low magnification image information (first image information, i.e. The whole image) or high-magnification image information (second image information, that is, a partial image) is input and output to the shading / distortion correction processing unit 12.

The shading / distortion correction processing unit 12 performs shading correction and distortion correction on shading and distortion caused by the imaging system force including the objective lens 2 for each of the first image information and the second image information, and then performs imaging. In the image data storage buffer unit 13, information of magnification is added and stored.

This magnification information is added to the first image information and the second image information in the imaging control unit 11 via the system control unit 7 as lens information of the objective lens 2.

[0023] The first captured image reading unit 14 reads first image information in which the added magnification information indicates a low magnification from the captured image data storage buffer unit 13, and temporarily stores the first image information. To store.

The second captured image reading unit 15 reads the second image information (hereinafter referred to as a partial image) with the added magnification information from the captured image data storage buffer unit 13 and temporarily stores the partial image. Store.

The image model generation unit 16 generates a model of the target image that is finally generated by pasting the partial images. This model includes an overlapping area that will be overlapped when the partial images are pasted together!

That is, the image model generation unit 16 generates a first magnification as a low magnification preset by the user, a second magnification as a high magnification, and a partial image, which are input from the system control unit 7 and pasted together. The above model is generated from the image size to be overlapped and the size of the overlapping area to be overlapped when pasting.

[0025] The no-turn density evaluation value calculation unit 17 reads a model from the image model generation unit 16 and reads the first image information from the first captured image reading unit 14 to generate a target image. The search area to be searched is set in the first image information by the system control unit 7 (set while the user confirms the screen).

In addition, as shown in FIG. 3, the pattern density evaluation value calculation unit 17 has a predetermined movement distance, for example, a plurality of movements within the search area, with the model as a predetermined position, for example, the upper left of the search area. While moving in the X-axis direction and Y-axis direction in pixel units, The pattern density evaluation value (pattern information) in the overlapping area is calculated, and these are sequentially stored in correspondence with the calculated positions.

Here, the movement distance in the search area may be performed in units of one pixel (pixel), but depending on the target pattern, there is no change before and after the movement, and the obtained pattern density evaluation value is also Since the values are almost the same, in the present invention, a unit of a predetermined number of pixels is used in order to reduce useless calculation time and improve the search efficiency of the overlapping area. As the moving distance, if the object is a periodic pattern as in this embodiment, the number of pixels forming one period is 1Z5, 1/10, 1/50, 1/100,. It is set according to the number of pixels in the pattern period.

[0027] Further, if the minimum size of the target pattern included in the overlapping area (for example, the width of the signal line through which the current flows) is known, the minimum pattern width is 1 or 2 times the number of pixels. , 3 times, • You can set the movement distance according to the size of the pattern, such as-.

The movement distance according to the size of the no-turn takes into account that the pattern density evaluation value changes depending on whether the entire pattern appears or disappears from the overlap area before and after moving.

[0028] The pattern density evaluation value has four calculated values (vertical units to be described later) for each block of the size of the partial image (horizontal direction and vertical direction). Directional and horizontal edge strength).

Here, the pattern density evaluation value is calculated by the pattern density evaluation value calculation unit 17 according to the following flow.

In this embodiment, the pattern density evaluation value is obtained by paying attention to the edge strength for each direction (the magnitude of the luminance change in the pattern).

The edge strength for each direction represents the edge strength in each of the vertical (up and down the screen) direction and the horizontal (left and right of the screen) direction.

A Sobel filter is used as a method for calculating the edge strength. This Sobel filter multiplies each of the nine pixel values in the vicinity, that is, adjacent to the top, bottom, left, and right, centering on a certain pixel of interest, by multiplying the coefficient (center is the pixel of interest) mask as shown in Fig. 4 to obtain the result. Totally, this process is performed using two coefficient matrices in the vertical and horizontal directions. [0030] That is, the pixel at the center of the mask (X, Y) (X is the coordinate value on the screen in the horizontal direction, the right direction is positive, the left direction is negative, and Y is in the vertical direction. The coordinate values on the screen, the downward direction with respect to the origin is positive and the upward direction is negative), the edge intensity in each direction, and the luminance value of the pixel (X, Y) as I (X, Y). In addition, the absolute value of the numerical value R is defined as Abs (R), and the horizontal strength EH (X, Y) and the vertical strength EV (X, Y) are obtained by the following formula.

[0031] Intensity EH (X, Y) = Abs {I (X + 1, Y- 1) + 2 X I (X + 1, Y) + I (X + 1, Y + 1)

Ι (Χ-Ι, Υ-Ι) 2 X Ι (Χ-Ι, Υ) Ι (Χ-1, Υ + 1)}

Strength EV (X, Y) = Abs (I (X-l, Y + l) + 2 X I (X, Y + l) + I (X + l, Y + l)

Ι (Χ-Ι, Υ-Ι) 2 X Ι (Χ, Υ-Ι) Ι (Χ + 1, Υ-1)}

[0032] Using the above-described equation, the calculation is performed for each pixel in the target region (overlapping region) (at this time, the edge portion of the image cannot be calculated because the edge intensity cannot be calculated).

Here, the pattern density evaluation value calculation unit 17 adds the calculated edge strength for each pixel to the direction unit in the target region, and adds the horizontal edge strength total value ΑΕΗ and the vertical edge strength. Calculate the total value AEV.

In some cases, however, the edge strength in either direction is extremely low.For example, as shown in Fig. 5, if the inside of the overlap area is formed with only a horizontal line pattern,ヽ Since there is an edge of the pattern, the edge strength total value AEV of a certain value can be obtained, but the horizontal edge strength total value ΑΕΗ is because the change in luminance does not exist in the horizontal direction. It becomes almost “0”.

In this case, if the pasting process is performed only with a one-way pattern, the matching point cannot be limited, so the optimum matching process cannot be performed.

For this reason, the pattern density evaluation value calculation unit 17 obtains a predetermined threshold value Thres for the edge intensity total value ΑΕΗ and the edge intensity total value AEV, and each edge intensity total value is equal to or greater than the threshold value Thres. Only this threshold Thres or more is output as the pattern density evaluation value.

This threshold value Thres is expressed by the following formula, taking into consideration the influence of noise, for example, where Q is the minimum luminance difference detected as the edge of the Sobel filter for one pixel. Thres = 4′QX (Number of Sobel filter calculation target pixels in the overlapping region) The threshold value Thres is multiplied by a predetermined coefficient and used as the actual threshold value. The coefficient to be multiplied may be a small value such as 1 or 2 to suppress the effect of noise. Also, when extracting an area where the edge strength is high, that is, the pattern features are clear, a large value according to the value of Q (if the luminance gradation is 256 and Q is 10,

A value between ~ 15).

Further, the pattern density evaluation value calculation unit 17 uses the above formula to calculate the pattern density evaluation value PD.

EV is calculated as follows.

If AEH <Thres or AEV <Thres, set PDEV to 0,

If AEH ≥ Thres and AEV ≥ Thres, PDE V = AEH + AE V. As a result, when the calculated pattern density evaluation value is lower than one of the edge strengths (extremely small), there is a high possibility that pattern matching will fail when partial images are pasted together. Therefore, it will be excluded from the object of later evaluation, and only the numerical value of the arrangement position where accurate pattern matching can be performed remains in the pasting process. Here, the larger the edge strength, the larger the luminance difference, and the accuracy of matching the partial images in the overlapping region in the pasting is improved.

In other words, even in the case of an entire image having a relatively sparse pattern, it is possible to search for the position of the model where the pattern of the overlapping region is dense.

Next, returning to FIG. 2, the photographing position calculation unit 18 selects the position with the largest pattern density evaluation value selected by the pattern density evaluation value calculation unit 17, that is, the first image information (entire image). Based on the shooting position of the target image, the shooting location of each partial image is obtained, and the shooting location information of this shooting location is output to the system control unit 7.

Based on the shooting position output from the shooting position calculation unit 18, the image generation unit 19 includes a system control unit 7 that includes a microscope Z-axis movement control unit 8, a stage movement control unit 6, an imaging control unit 11, and a shooting camera. Control 3 to paste the partial images of multiple blocks of the model.

The image storage unit 20 stores the target image (high-definition image) generated by pasting the partial images in the image generation unit 19. The system control unit 7 reads out the target image from the image storage unit 20 and displays it on a display device (not shown) by the access by the user.

As the pattern information, the edge intensity indicating the change in the luminance value of the image is used in the above-described first embodiment. However, the “spatial characteristic” of the direction based on the luminance value of the image is used. An evaluation value that consists of purely luminance values, such as the luminance average value for the histogram formed from the image luminance value, the difference between the minimum and maximum values (dynamic range). ), Mode, median, and variance (standard deviation) can be used as pattern information.

[0037] Here, the pattern frequency evaluation value PDEV is obtained from the frequency FR and the standard deviation SD using the histogram frequency FR relating to the luminance value of the mode value relating to the overlapping region and the standard deviation SD of this histogram as the pattern information. And the power of 2 is expressed as 2 to x.

PDEV = FR + (2) X SD

This pattern density evaluation value PDEV represents 0 to X-1 with the frequency FR and 2 or more with the standard deviation SD. However, in the above formula, “FR 2”.

Thereby, first, the pattern density evaluation value calculation unit 17 pays attention only to the frequency FR, determines whether or not the force is equal to or greater than a predetermined threshold value, and if it is equal to or greater than this threshold value, also evaluates the standard deviation.

Since the maximum value of FR is (2), the search method described above becomes possible. The above-described operation is realized by bit operation (logical operation), and pattern information having different characteristics can be used as one pattern density evaluation value.

Next, the operation of the above-described image processing apparatus will be described with reference to FIGS. 1, 2, and 6. FIG. 6 is a flowchart specifically showing an operation example of the image processing apparatus according to the first embodiment of FIG.

Here, the FPD board shown in FIG. 7 will be described as an example of the object. In the FPD substrate, pixel portions and transistors for driving the pixels are periodically arranged.

The user uses an input device (not shown) as a processing parameter to the system control unit 7 as the overall image magnification (first magnification), partial image magnification (second magnification), and the combined image (target image). And the overlap rate of each partial image are set (step S1). [0039] Next, when the process of acquiring the target image is started, the system control unit 7 drives the stage 4 by the stage movement control unit 6, and adjusts the relative position between the objective lens 2 and the target object. And switch the objective lens 2 to achieve the first magnification.

Then, the system control unit 7 adjusts the focus by moving the lens barrel 1 up and down via the microscope Z-axis movement control unit 8 to capture the entire image shown in FIG. The whole image is transferred to the shading / distortion correction processing unit 12 via the.

Further, with respect to the entire image shown in FIG. 7, a field frame as a partial image (shootable range when captured at a second magnification: partial image frame) is an area within a broken line shown in FIG.

Then, the shading / distortion correction processing unit 12 performs distortion correction and shading correction on the entire input image, and stores it in the captured image data storage buffer unit 13 (step S 2). ).

Next, the image model generation unit 16 calculates the figure based on the size of the target image (the number of vertical pixels X the number of horizontal pixels) and the overlapping ratio of the partial images when the target image is generated. A model of the target image having an overlapping area as shown in Fig. 3 is generated (step S3).

[0041] Then, the image model generation unit 16 calculates the number of partial images and the size of the overlapping region so that the overlapping region has the above-described overlapping rate with respect to the partial image (step S4). For example, in Fig. 9, the partial image frame force shown in Fig. 8 also has a partial image frame of 4, and the overlap area is a shaded area where two or more of the four partial image frames overlap each other (with a + Shown in shape! / Declared part).

As a result, in this embodiment, the target image is formed by four partial images, and the model of the size of the target image is configured by four partial image frames.

Next, the user sets a search area for searching for the shooting position of the model force target image in the entire image displayed on the display device (step S5).

This search area can be an arbitrary part of the entire image as long as the entire image is larger than the size of the model.

[0042] Then, the pattern density evaluation value calculation unit 17 calculates the pattern density evaluation value at each movement position while moving (shifting) it in the X axis direction and the Y axis direction at a predetermined movement distance. The above process is repeated until the entire search range is searched, and the calculated pattern is sequentially calculated. The density evaluation value is associated with the coordinate value (calculated position) in the entire image and stored in the internal storage unit. When the calculation of the pattern density evaluation value in the entire search area is completed, the process proceeds to step S7. (Step S6).

Next, the pattern density evaluation value calculation unit 17 searches for the largest value among the pattern density evaluation values stored in the internal storage unit, and coordinates corresponding to the searched pattern density evaluation value The value is output as the optimal position of the target image at the time of pasting (Step S7

) o

At this time, as shown in FIG. 10, the pattern density evaluation value calculation unit 17 displays a three-dimensional graph indicating the size of the pattern density evaluation value in the Z-axis direction for each evaluated coordinate value (on the XY plane). Then, the pattern density evaluation value at each coordinate value is sequentially compared to search for the maximum pattern density evaluation value.

For example, in FIG. 10, since the pattern density evaluation value on the left side is the maximum, the position of this model, that is, the coordinate value is output as the optimum target image generation position.

Next, the shooting position calculation unit 18 calculates the shooting position of the partial image from the generation position of the target image output from the pattern density evaluation value calculation unit 17 (step S8).

At this time, the shooting position calculation unit 18 uses the arrangement position of the partial image frame of the model at the generation position as the shooting position of the partial image shot at the second magnification (high magnification). The coordinate value of the partial image frame corresponding to each partial image is output as the partial image position.

In the present embodiment, the shooting position calculation unit 18 controls the coordinate values of the partial image frames corresponding to the four partial images because the target image is composed of four partial images! / Output to part 7.

Next, the system control unit 7 changes the objective lens 2 to a lens corresponding to the second magnification via the microscope Z-axis movement control unit 8, and the partial image input from the imaging position calculation unit 18. Corresponding to the position, the stage 4 is photographed by the imaging camera 3 via the stage movement control unit 6, moved to the coordinate position, focused by the microscope Z-axis movement control unit 8, and each image is captured by the imaging camera 3. Take a partial image.

Here, the system control unit 7 has described all of the plurality of partial images constituting the target image. Take a picture by processing.

Then, the imaging control unit 11 outputs each partial image input from the imaging camera 3 to the shader / distortion correction processing unit 12.

As a result, the shading / distortion correction processing unit 12 performs distortion correction and shading correction on the sequentially input partial images, and stores them in the captured image data storage buffer unit 13 (step S9).

Next, the image processing unit 5 reads out the partial images constituting the target image from the captured image data buffer unit 13 and temporarily stores them in the second captured image reading unit 15.

Then, the image generation unit 19 sequentially reads out the partial images from the second captured image reading unit 15, and based on the model shown in FIG. 9, that is, for each partial image frame of the model, at the partial image position of this partial image frame. Corresponding captured partial images are arranged, and the partial images are combined to generate a target image. The generated high-definition target image is stored in the image storage unit 20 (step S10).

[0047] At this time, the image generation unit 19 performs pattern matching by superimposing patterns arranged in the overlapping area, and performs alignment of the bonding. For this reason, it is necessary to use an area where the pattern density evaluation value exceeds a predetermined density, that is, an area exceeding a predetermined threshold, as the overlapping area.

Therefore, in this embodiment, since the arrangement position of the overlapping region to be bonded is determined by pattern information such as the pattern density evaluation value, it is formed on the substrate like the FPD substrate pattern shown in FIG. A bonded image can be generated with high accuracy even for an object having periodicity, which is suitable for a bonding process in which many patterns are sparse.

The system control unit 7 reads the target image from the image storage unit 20 as necessary, and displays the target image on the display unit.

[0048] <Second embodiment>

The second embodiment has the same configuration as that of the first embodiment, and only differences from the first embodiment will be described below.

FIG. 11 is a flowchart specifically showing an operation example in the second embodiment. The difference is that step S8 of the first embodiment is changed to step S15, and this step will be described.

According to the first embodiment, the optimal position force of the model for pasting the partial images is based on the position of the partial image frame in the model! The shooting position is determined, and the search is performed in the search area with the overlapping area between the partial image frames in the model fixed.

On the other hand, in the second embodiment, when the optimum position of the stitching model is determined, the position of the partial image frames constituting the stitching model is determined using the low-magnification whole image (first image information). Decide and speak.

In step S 6, the pattern density evaluation value at each position is calculated while moving the search area by a predetermined movement distance using the fixed overlapping area model.

At this time, the minimum pattern density threshold PDEV-Min is set as an expression shown below.

PDE V_Min = AEH_Min + AEV— Min

= 4-Q X PixNum + 4-Q X PixNum

= 8 -Q X PixNum

Here, PixNum = “the number of pixels subject to the Sobel filter operation in the overlapping region” Then, the pattern density evaluation value calculation unit 17 calculates the pattern density evaluation value at each moving position, and searches the entire search range. The above process is repeated until the above pattern PDEV —Min is exceeded, and the calculated pattern density evaluation value is sequentially associated with the coordinate value (calculated position) in the entire image, stored in the internal storage unit, and the search area When the calculation of the overall pattern density evaluation value is completed, the process proceeds to step S7.

In step S7, the pattern density evaluation value calculation unit 17 selects and outputs the largest pattern density evaluation value from the internal storage unit, as in the first embodiment.

Next, in step S15, the pattern density evaluation value of the overlapping region in the model of the coordinate value corresponding to the selected pattern density evaluation value is recalculated.

At this time, the overlapping area of the partial image frames in the model is the area A by the partial image frames F1 and F2 in FIG. 12, the area B by the partial image frames F3 and F4 in FIG. 13, and the partial image frames F1 and F3 in FIG. This area C is the area D formed by the partial image frames F2 and F4 in FIG. For these, the pattern density evaluation value for each of the regions A to D is calculated from the image at the corresponding position of the low magnification overall image for each partial image frame.

Here, the pattern density evaluation value calculation unit 17 determines whether or not each region A to D exceeds a predetermined threshold value.

This threshold value was obtained as the pattern density evaluation value in the horizontal and vertical directions in the first embodiment, but in the second embodiment, two partial images adjacent in the horizontal and vertical directions are obtained. Since this is a unit of overlapping area of the frame, the value is defined by the following formula.

Thres2 = 2'QX (the number of pixels subject to Sobel filter operation in the overlapping region) Then, the pattern density evaluation value calculation unit 17 detects that the pattern density of all the regions A to D exceeds the threshold value, and performs processing. Proceed to step S9, and thereafter perform the same processing as in the first embodiment.

A value obtained by multiplying the threshold Thres2 by a predetermined coefficient is used as the actual threshold. The coefficient to be multiplied may be a small value such as 1 or 2 to suppress the effect of noise. Also, when extracting an area where the edge strength is high, that is, the pattern features are clear, a large value according to the value of Q (if the luminance gradation is 256 and Q is 10, 10 to 15 Between the values).

On the other hand, when the pattern density evaluation value calculation unit 17 detects that the pattern density evaluation value of the region A indicated by the oblique lines in FIG. 16 does not exceed the threshold value Thres2, for the region that does not exceed the threshold value Thres2, for example. As shown in FIG. 17, the partial image frame F1 is moved rightward by a predetermined moving distance, and the area of the area A that is an overlapping area with the partial image frame F1 and the partial image frame F2 is expanded.

Then, the pattern density evaluation value calculation unit 17 calculates the pattern density evaluation value of the region A again, detects whether or not the pattern density evaluation value exceeds the threshold value Thres2, and If detected, the process proceeds to step S9. If not, the partial image frame F1 is moved rightward again to determine the pattern density evaluation value of the area A.

Here, the limitation on the overlapping rate of the overlapping region when the partial image frame is moved will be described with reference to the drawings. FIG. 18 is a conceptual diagram for explaining the overlapping rate of overlapping regions. The maximum overlap rate is 50% if the same pattern is included in two partial images.

In other words, if the overlap rate is 50% or more, the same pattern is included in the three partial images.

Then, when the overlap rate of the overlapping area exceeds the maximum value, the pattern density evaluation value calculating unit 17 again sets the pattern density evaluation value of the entire overlapping area to the coordinate of the model with the second largest numerical value! Perform the process.

[0054] On the other hand, as the minimum value of the overlapping rate, a prescribed value of the number of pixels that is a real number multiple of 1 or more than the number of pixels of the minimum pattern is provided in the pattern formed on the substrate. Obtain from the ratio of the entire partial image.

For example, if the partial image size is 640 (horizontal direction) X 480 (vertical direction) pixels and the minimum pattern is 4 (horizontal direction) X 4 (vertical direction) pixels, the pixel is twice the minimum pattern. The number is the specified value.

As a result, the minimum overlap ratio in the horizontal direction is (4 X 2Z640) = 1.25%, and the minimum overlap ratio in the vertical direction is (4 X 2Z480) = 1.67%.

[0055] As described above, in the second embodiment, when determining the optimum position of the model for pasting, all pattern density evaluation values of the regions A to D in the model in the entire search region have the threshold value. If not, first, the position where the evaluation value is the maximum as a whole is determined on the entire image, and the pattern density evaluation value exceeds the threshold value for the overlapping portion of each partial image frame at this coordinate value. Adjust the position of the partial image frame to change the overlap ratio of the overlapping area, and determine the shooting position of the partial image to be shot.

[0056] Further, in order to adjust the pattern density evaluation value in a timely manner by changing the overlapping rate of the overlapping area, the degree of freedom of search is increased with respect to the first embodiment, and an appropriately set photographing position is used as a start. Even if the entire search area is not searched, it is possible to automatically determine the optimum photographing position for pasting using the pattern density evaluation value.

Therefore, in this embodiment, since the arrangement position of the overlapping region to be bonded is determined by pattern information such as the pattern density evaluation value, the pattern of the FPD board shown in FIG. In the case of objects with periodicity that are not suitable for the bonding process, where the pattern formed on the substrate is sparse, as in the case of a screen, the bonding process fails for the overlapped part set appropriately. Even in such a case, a combined image can be generated with high accuracy.

[0057] <Third embodiment>

The third embodiment shown in FIG. 19 is a large substrate inspection apparatus equipped with a microscope. The substrate inspection apparatus shown in FIG. 19 is the same as the first and second embodiments in the configuration of an observation system such as a microscope, objective lens 2, and imaging camera 3.

The difference is the drive mechanism that moves the FPD substrate of the object relative to the objective lens 2, and the stage movement controller 6 moves the stage 4 on which the object is placed in one axis direction (upper right in FIG. 19). Lower left direction: Drive only in arrow O).

On the other hand, the system control unit 7 drives the microscope T itself in one axial direction perpendicular to the stage 4 (upper left and lower right in FIG. 19: arrow P).

As a result, the relative position between the objective lens 2 and the object can be moved in the XY directions.

[0058] <Fourth embodiment>

The target image, which is a high-definition image generated in the first to third embodiments, is a reference image that is compared with the image of the substrate being inspected when detecting a substrate defect (a normal substrate for comparison). The image is also used in the inspection device as a force generated image).

For example, in the inspection of an FPD substrate, the inspection apparatus shown in FIG. 20 is provided with a line sensor as an image pickup means. After the image pickup means is adjusted by a calibration sample, the stage is moved in the direction of arrow G by the holding movement means. The reflected light of the light emitted by the illumination means is detected by the line sensor at every predetermined moving distance.

Then, the integrated control means compares the intensity of the reflected light detected with the detection value of the reflected light sampled immediately before, and if it differs beyond a predetermined range, it is detected as a defective candidate and The coordinate value at is stored.

Then, the FPD board is mounted on the stage 4 of the image processing apparatus in the first to third embodiments, and the coordinate value of the defect candidate is input to the system control unit 7. As a result, the system control unit 7 moves the stage 4 via the stage movement control unit 6, and the position of the defect candidate is set to the position of the objective lens 2, that is, the position where the substrate portion of the defect candidate can be imaged by the imaging camera 3. Moving.

At this time, the system control unit 7 generates the target image as a high-definition image, that is, in a state where the position of the defective candidate is included by the second magnification, and in the first to third embodiments, that is, Move to the location corresponding to the optimal model position.

[0060] Then, the system control unit 7 compares the image information including the captured defect candidate with the target image generated in the first and second embodiments by pattern matching, and determines the defect candidate pattern. And the pattern shape of the corresponding part of the target image, which is the reference image, are compared to determine whether they are different!

At this time, if the system control unit 7 detects that the difference is not different, it determines that the defect candidate is a non-defective product, while if it detects a difference, determines that the defect candidate is defective. The determination result is displayed on the display device.

By using the inspection method of the present invention described above to determine a defect candidate in a high-speed inspection by a line sensor accurately by comparing it with an actual normal substrate pattern, the inspection speed is improved, In addition, the accuracy of inspection can be improved.

[0061] It should be noted that a program for realizing the functions of the image processing unit in FIGS. 1 and 2 is recorded on a computer-readable recording medium, and the program recorded on the recording medium is read into a computer system. Image processing may be performed by executing. The “computer system” here includes the OS and hardware such as peripheral devices. “Computer system” includes a WWW system equipped with a homepage provision environment (or display environment). The “computer-readable recording medium” refers to a storage device such as a flexible disk, a magneto-optical disk, a portable medium such as a ROM and a CD-ROM, and a hard disk incorporated in a computer system. Furthermore, the “computer-readable recording medium” means a volatile memory (RAM) inside a computer system that becomes a server or a client when a program is transmitted via a communication line such as a network such as the Internet or a telephone line. In this way, the program is held for a certain period of time. [0062] Further, the program may be transmitted from a computer system storing the program in a storage device or the like to another computer system via a transmission medium or by a transmission wave in the transmission medium. Here, the “transmission medium” for transmitting a program refers to a medium having a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line. Further, the program may be for realizing a part of the functions described above. Furthermore, what can realize the above-mentioned functions in combination with a program already recorded in the computer system, that is, a so-called differential file (differential program) may be used.

Industrial applicability

[0063] According to the image processing apparatus and the image processing method of the present invention, when partial images are combined, a model of a target image to be combined and combined with the partial images is formed in advance based on the low-resolution first image information. Then, using this model, the first image information is used to adjust the shooting position of the partial image that generates the high-resolution target image including the overlapping region within the predetermined region of the first image information over a wide range. Thus, in the wide field of view, it is possible to obtain an appropriate partial image shooting position by calculation, and it is possible to easily generate a high-definition image with a desired high resolution.

Claims

The scope of the claims

[1] An image processing device that combines partial images of an object photographed at a predetermined resolution with a predetermined overlapping area, and generates a target image of the entire object or a part of the object of a predetermined size And

A first photographing means for photographing the object at a first magnification to obtain first image information; and photographing the object at a second magnification which is higher than the first magnification! Second imaging means for obtaining second image information as the partial image;

Image model generation means for generating a model of the target image generated by pasting the partial images from the size of the target image and the image area information indicating the degree of overlapping area in the partial image;

Photographing position calculation means for searching for the position of the target image generated by pasting the partial images in the first image information using the model;

High-definition image generating means for generating the target image by pasting the partial images based on the arrangement position;

An image processing apparatus.

[2] The image processing device according to claim 1, wherein the photographing position calculation unit detects an optimal arrangement position of an overlapping area in the pasting of the models in the first image information. Image processing apparatus for searching for an arrangement position of.

[3] The image processing device according to claim 2, wherein the shooting position calculation means overlaps the model while moving the model by a predetermined movement distance within a preset search region of the first image information. An image processing apparatus that searches for an arrangement position of a region.

[4] The image processing device according to claim 2 or 3, wherein the photographing position calculation means searches for an arrangement position of the overlapping area in the search area based on the pattern information of the overlapping area. apparatus.

[5] In the image processing device according to any one of claims 1 to 4, the imaging position calculation unit may change the overlapping area information in the model in the search area based on the pattern information of the overlapping area. An image processing apparatus for searching for an arrangement position.

[6] The image processing device according to any one of claims 1 to 5, wherein the first imaging is performed. Moving means for moving the object relative to each other in a predetermined distance unit in the XY directions with respect to the shadowing means and the second photographing means;

An image processing apparatus that sets a shooting position of a target image on the target based on a position of the target image detected by the model by the shooting position calculation unit.

7. The image processing device according to claim 6, wherein the imaging position of the partial image used for pasting is calculated based on the imaging position and the arrangement position of the target image detected by the model.

[8] In the image processing device according to any one of claims 1 to 7, each of the first image information and the second image information obtained by the first and second imaging units is subjected to distortion correction or Is an image processing device with Z and shading correction.

[9] An image processing method in which partial images of an object photographed at a predetermined resolution are pasted together with a predetermined overlapping area, and an entire or a part of the target image of a predetermined size is generated. And

A first imaging process in which an object is photographed at a first magnification to obtain first image information; and the object is photographed at a second magnification that is higher than the first magnification! A second imaging process for obtaining second image information as the partial image,

An image model generation process for generating a model of the target image generated by pasting the partial images from the size of the target image and the overlapping region information indicating the degree of the overlapping region in the partial image;

A shooting position calculation process of searching for an arrangement position in the first image information of the target image generated by pasting the partial images using the model;

A high-definition image generation process for generating the target image by combining the partial images based on the arrangement position.

An image processing method.

[10] In the image processing method according to claim 9, in the imaging position calculation process, the optimal arrangement of overlapping regions in the pasting of the models in the first image information is performed. An image processing method for searching for an arrangement position of a target image by detecting a position.

[11] The image processing method according to claim 10, wherein in the shooting position calculation step, An image processing method for searching for an arrangement position of an overlapping area while moving the model by a predetermined moving distance within a preset search area of the first image information.