TWI489101B - Apparatus and method for combining 3d and 2d measurement - Google Patents

Description

Method and device for measuring three-dimensional and two-dimensional shapes

The disclosure relates to a measuring method and device, and in particular to a measuring method and device for combining three-dimensional and two-dimensional shapes.

Traditionally, mechanical inspection of integrated circuit (IC) packages includes two-dimensional inspection and three-dimensional topography measurement, in which the front of the IC is mainly measured by two-dimensional inspection and two-dimensional shape/size measurement; Measure the coplanarity of the 3D IC pin, the 2D scale of the pin and the defect detection.

However, current measurement techniques typically require multiple scans of the object through two (or more) different light sources to obtain 2D and 3D measurements of the object, respectively. This not only consumes the measurement time, but also increases the cost of the machine, and leads to excessive data transmission bandwidth being occupied.

Therefore, how to provide a measuring method and device capable of effectively combining three-dimensional and two-dimensional topography is one of the current topics in the industry.

The disclosure relates to a measuring method and device for combining three-dimensional and two-dimensional shapes, which can establish three-dimensional and two-dimensional shapes of objects by single-time multi-line scanning. Appearance to achieve the effect of rapid measurement.

According to an embodiment of the present disclosure, a measurement method is proposed for measuring a three-dimensional and two-dimensional shape of an object, and the surface of the object has a plurality of object points. This measurement method includes the following steps. First, the stripe pattern is projected onto the surface of the object via the projection device. Then, the multi-line image is received and stored by the multi-line photoelectric image capturing system. Thereafter, the object and the phase shifting device system are relatively moved to capture a plurality of phase light intensity images through the multi-line photoelectric image capturing system, and the phase shifting device system includes a projection device and a multi-line photoelectric image capturing system. Then, the phase value of the object point is determined according to the phase light intensity image, and the phase value is converted into the height corresponding to the object point through the triangular geometric relationship to generate the three-dimensional shape of the object. And, according to the phase light intensity image, the object image without phase change is reconstructed, and the object image without phase change presents the two-dimensional shape size of the object.

According to an embodiment of the present disclosure, a measuring device is provided for measuring a three-dimensional and two-dimensional shape of an object, the surface of the object having a plurality of object points. The measuring device comprises a phase shifting device system and an image processing unit. The phase shifting device system includes a projection device and a multi-line photoelectric image capturing system. The projection device is used to project the stripe pattern onto the surface of the object. A multi-line photo-electric imaging system is used to receive and store multi-line images. The object and the phase shifting device system are relatively moved to capture multiple phase light intensity images through the multi-line photoelectric image capturing system. The image processing unit is configured to determine the phase value of the object point according to the phase light intensity image, and convert the phase value into a height corresponding to the object point through a triangular geometric relationship to generate a three-dimensional shape of the object. The image processing unit reconstructs an object image without phase change according to the phase light intensity image, and the object image system without phase change Presents the two-dimensional shape of the object.

In order to better understand the above and other aspects of the present disclosure, the following specific embodiments, together with the accompanying drawings, are described in detail below:

10‧‧‧ objects

100‧‧‧Measurement device

102‧‧‧ Phase shifting device system

104‧‧‧Image Processing Unit

106‧‧‧Projecting device

108‧‧‧Multi-line photoelectric imaging system

110‧‧‧ scan carrier

200‧‧‧Measurement method

Steps 202, 204, 206, 208, 210‧‧

L1‧‧‧first image line

L2‧‧‧second image line

L3‧‧‧ third line

O1~O5‧‧‧ points

PL‧‧‧ striped light

RP‧‧‧ reference plane

A, A’, B, C, C’, D, O‧‧ points

θ ₀ , θ _n ‧‧‧ angle

d ₀ ‧‧‧effective wavelength is

502, 504, 506, 508 ‧ ‧ images

FIG. 1 is a block diagram of a measuring device according to an embodiment of the present disclosure.

FIG. 2 is a flow chart of a measurement method according to an embodiment of the present disclosure.

3A to 3C are schematic diagrams showing the scanning mechanism of the phase shifting device system.

FIG. 4 is a schematic diagram showing the geometric relationship between the streak light projected by the projection device and the height of the object.

Figure 5 is a schematic diagram showing the reconstruction of an image of an object without phase change.

The following is a detailed description of the embodiments, which are intended to be illustrative only and not to limit the scope of the disclosure. In addition, the drawings in the embodiments omit unnecessary elements to clearly show the technical features of the disclosure.

Please refer to both Figure 1 and Figure 2. FIG. 1 is a block diagram of a measuring device 100 in accordance with an embodiment of the present disclosure. FIG. 2 is a flow chart of a metrology method 200 in accordance with an embodiment of the present disclosure. The measuring device 100 is used to measure the three-dimensional and two-dimensional topography of the object 10, which includes a phase shifting device system 102 and an image processing unit 104. The phase shifting device system 102 includes a projection device 106 and a multi-line optical imaging system 108. Projection device 106 is, for example, a raster projector or other structured light source. The multi-line photoelectric imaging system 108 is, for example, a three-wire photosensitive coupling. Charge Coupled Device (CCD) camera or other multi-line CCD camera. The image processing unit 104 is, for example, an integrated circuit or other processor with computing power.

First, in step 202, the projection device 106 projects a stripe pattern onto the surface of the object 10, on which a plurality of object points are defined.

Next, in step 204, the multi-line optical imaging system 108 receives and stores the multi-line image.

Thereafter, in step 206, the object and the phase shifting device system are moved relative to each other to capture a plurality of phase intensity images through the multi-line optical imaging system 108.

Then, in step 208, the image processing unit 104 determines the phase value of the object point according to the phase light intensity image, and converts the phase value into the height corresponding to the object point through the triangular geometric relationship to generate the three-dimensional shape of the object 10.

In step 210, the image processing unit 104 reconstructs the object image without phase change according to the phase light intensity image. This object image without phase change presents the two-dimensional topographical dimensions of the object 10. It can be understood that although steps 208 and 210 in FIG. 2 are executed sequentially, in other examples, the order of execution of steps S208 and S210 may be interchanged or executed simultaneously.

After obtaining the image of the object without phase change, the image processing unit 104 may further compare some or all of the image of the object without phase change with a preset object pattern to detect the surface topography, size or defect of the object 10. The preset object pattern is, for example, an engineering pattern in which part or all of the object 10 is free from defects. For example, suppose that after comparison, it is found that the image of the object without phase change is different. The portion of the preset object pattern, the image processing unit 104 can determine the surface defects (such as nicks, depressions, broken patterns, etc.) of the object 10 according to the characteristics of the different portions.

Similarly, after detecting the three-dimensional shape of the object 10, the image processing unit 104 may further compare some or all of the three-dimensional topography with a preset object topography to detect the object 10. Shape, size or defect. For example, it is assumed that the three-dimensional shape of the scanned object has a black block, but the black block does not exist in the corresponding preset object topography, and the image processing unit 104 can determine the surface of the object 10 at the time. There are defects at the black block (such as surface depressions).

According to the above, the measuring method and device of the embodiment can generate the three-dimensional shape of the object 10 through the plurality of phase light intensity images captured, and can also establish the two-dimensional shape of the object 10 through the phase light intensity images. Dimensions, so measurements can be made to analyze the shape, size or defect of the object.

In one embodiment, the projection device 106 can project a structured light onto the surface of the object 10 to form a stripe pattern whose light intensity varies sinusoidal intensity as a function of position. The stripe pattern is, for example, a pattern in which a stripe sinusoidal intensity changes, a pattern in which a sinusoidal intensity changes, or a pattern formed by a moiré. Since the stripe pattern projected onto the surface of the object 10 is deformed due to the undulation of its surface, the height of the object 10 can be obtained from the offset of the stripe pattern. In addition, moving the object 10 and the phase shifting device system 102 in a relatively straight line for scanning can capture a plurality of different phase intensity images. As shown in FIG. 1, the object to be tested 10 is placed on the scanning carrier 110. Scan carrier 110 can be aligned with phase shifting device system 102 with phase shifting device system 102 being fixed. Movement (e.g., moving in the X-axis direction in Figure 1) causes at least one object point of object 10 to be scanned by phase shifting device system 102. However, the disclosure is not limited thereto, and the object 10 can also be placed on the fixed scanning carrier 110, and the phase shifting device system 102 can be moved in a straight line relative to the scanning carrier 110. To clearly illustrate the scanning mechanism of the phase shifting device system 102, the description will be made with reference to Figures 3A through 3C.

As shown in FIG. 3A, the multi-line photoelectric imaging system 108 transmits the object point O3, the object point O2, and the object on the object 10 through the first image taking line L1, the second image taking line L2, and the third image taking line L3, respectively. Point O1 for image acquisition. Next, as shown in FIG. 3B, in the next time point, the object 10 is moved by 1 unit in the scanning direction, so that the first image taking line L1, the second image taking line L2, and the third image taking line L3 are respectively paired with the object. The object point O4, the object point O3, and the object point O2 on the 10 are taken as images. Thereafter, as shown in FIG. 3C, at the next time point, the object 10 is further moved by 1 unit in the scanning direction, so that the first image taking line L1, the second image taking line L2, and the third image taking line L3 are respectively The object point O5, the object point O4, and the object point O3 on the object 10 are taken as images.

The above process is performed until the scanning of all the object points of the object 10 is completed, and at least one object point is imaged three times. Therefore, when the phase shifting device system 102 performs a single scan on the object 10, the multi-line optical image capturing system 108 can sequentially capture three phase intensity images I ₁ , I _{2 ,} and I ₃ having equal phase shift amounts. . In other words, when the phase shifting device system 102 and the object 10 make a single linear relative movement, the multi-line photoelectric image capturing system 108 can sequentially capture the multiple phases corresponding to the object 10 in response to the single linear relative motion. Light intensity image. However, the present disclosure is not limited thereto, and the phase shifting device system 102 can also move relative to the object 10 multiple times, and capture multiple phase light intensity images through the multi-line photoelectric image capturing system 108. In this example, the light intensity values corresponding to the respective object points in the phase light intensity images I ₁ , I _{2 ,} and I ₃ can be expressed as

In the above formula, (x, y) represents the position of the object point, I ₀ (x, y) represents the background light intensity of the image, A (x, y) represents the structural light in the image, and φ (x, y) represents the object point. Phase value. After finishing the above expression, you can get

However, the present disclosure is not limited to the above examples, and the multi-line photoelectric imaging system 108 can also capture more phase light intensity images having equal phase shift amounts. These phase shifts can be expressed as

Where n represents the number of phase light intensity images captured by the multi-line optical imaging system 108.

Taking four phase intensity images I ₁ , I ₂ , I _{3 ,} and I ₄ as an example, the light intensity values of the corresponding phase points in the phase light intensity images I ₁ , I ₂ , I _{3 ,} and I ₄ may be Expressed as

After finishing the above expression, you can get

Moreover, taking five phase intensity images I ₁ , I ₂ , I ₃ , I _{4 ,} and I ₅ as examples, the corresponding phase light intensity images I ₁ , I ₂ , I ₃ , I _{4 ,} and I ₅ correspond to The light intensity value of each object point can be expressed as

After finishing the above expression, you can get

After obtaining the phase values φ(x, y) of the object points, the image processing unit 104 can convert the phase values φ(x, y) corresponding to the object points into corresponding points according to the triangular geometric relationship. The height, in turn, establishes the three-dimensional shape of the object 10. On the other hand, the image processing unit 104 can also reconstruct the image of the object without phase change according to the phase light intensity image captured by the multi-line photoelectric imaging system 108. For the sake of clarity, the description is made for (1) establishing a three-dimensional topography and (2) reconstructing an image of an object without phase change.

(1) Establish a three-dimensional shape

Please refer to FIG. 4 , which is a schematic diagram showing the geometric relationship between the stripe ray PL projected by the projection device 106 and the height of the object 10 . The effective wavelength of the stripe ray PL is d ₀ . As shown in Fig. 4, points A, B, C, and O are points on the reference plane RP, and the straight line distance from point B to point D corresponds to the height of the object at point D at point D. It is assumed that the multi-line photo-electric imaging system 108 captures light from point A on the reference plane RP, and its corresponding imaging position is point A'. However, due to the change in height of the object 10, the multi-line photo-electric imaging system 108 actually captures light from point D, with its corresponding imaging position being point C'. Thus, the stripe offset of point A' to point C' may correspond to the height of object 10 at point D (ie, the linear distance from point B to point D). Through the transformation of the triangular geometric relationship, the linear distance from point B to point D can be obtained (ie )as follows:

among them Representing the stripe offset, θ ₀ represents the projection angle of the stripe ray PL, θ _n represents the image capture angle of the multi-line photo-electric imaging system 108, φ _A represents the stripe phase of the point A, and φ _B represents the stripe of the point B. Phase. Therefore, the measuring method and device of the embodiment of the present disclosure can effectively calculate the three-dimensional shape of the object by phase value information and triangular geometric relationship conversion.

(2) Reconstructing an image of an object without phase change

Please refer to FIG. 5, which is a schematic diagram of reconstructing an image of an object without phase change. In FIG. 5, the images 502, 504, and 506 respectively represent three phase intensity images captured by the multi-line photoelectric imaging system 108 in one scan (for convenience of explanation, the images 502, 504, and 506 are only 4 pixels), image 508 stands for The image of the object without phase change established by the maximum light intensity method according to the images 502, 504, and 506. As shown in FIG. 5, the light intensity values of the four pixels of the phase light intensity image 502 are 20, 55, 28, and 78, respectively; the light intensity values of the four pixels of the phase light intensity image 504 are 80, 102, and 95, respectively. 65; the intensity values of the four pixels of the phase light intensity image 506 are 110, 90, 100, and 67, respectively. To create an image of the object without phase change (ie, image 508), image processing unit 104 compares the intensity values of the pixels having the same image position in images 502, 504, and 506, and selects the maximum light intensity value as the phaseless change. The intensity value of the pixels in the object image at the same image position. For example, since the light intensity values of the upper left pixels of the images 502, 504, and 506 are 20, 80, and 110, respectively, and the maximum light intensity value is 110, the image processing unit 104 sets the light intensity of the upper left pixel of the image 508. The value is 110. Similarly, since the light intensity values of the upper right pixels of the images 502, 504, and 506 are 55, 102, and 90, respectively, and the maximum light intensity value is 102, the image processing unit 104 sets the light intensity value of the upper left pixel of the image 508. Is 102. And so on. In this way, multiple different phase intensity images can be reconstructed into an object image without phase change. This image of the object without phase change presents the two-dimensional shape of the object and can be used as an analysis of the shape, size or defect of the object.

In another example, an image of an object without phase changes can be created by image averaging. Further, the image processing unit 104 may average the light intensity values of the pixels having the same image position in the phase light intensity image to generate an average light intensity value, and use the average light intensity value as the object image without phase change. The intensity value of the pixel of this same image position. Taking Figure 5 as an example, due to images 502, 504, 506 The light intensity values of the upper left pixels are 20, 80, and 110, respectively, and the average value is 70. Therefore, the image processing unit 104 sets the light intensity value of the upper left pixel of the object image without phase change to 70. Similarly, since the light intensity values of the upper right pixels of the images 502, 504, and 506 are 55, 102, and 90, respectively, and the average value thereof is about 82, the image processing unit 104 sets the upper left pixel of the object image without phase change. The light intensity value is 102. And so on.

In another example, an object image without phase change can be created using a High Dynamic Range (HDR) display technique. Furthermore, since the plurality of phase light intensity images captured by the multi-line photoelectric image capturing system 108 can be regarded as images representing different light intensities (corresponding to exposure time) of the respective pixels, different pixels of different light intensities can be used. The image is restored to a high dynamic range image as an image of the object without phase changes. In this example, the pixel value of the pixel of the object image without phase change can be expressed as Z _ij = f ( E _i Δ t _j )

The subscript i represents the pixel position index value, and the subscript j represents the exposure time index value.

Then, inversely transforming the pixel value Z _ij to obtain f ^-1 ( Z _ij )= E _i Δ t _j

Then, taking the natural logarithm of the above formula, we can get ln f ^-1 ( Z _ij )=ln E _i +ln △ t _j

In this way, the pixel values obtained by reducing the image into a high dynamic range image are as follows: g ( Z _ij )=ln E _i +ln Δ t _j

Therefore, the image processing unit 104 can convert the phase light intensity images into a high dynamic range two-dimensional image as the object image without phase change according to the light intensity value corresponding to the pixels of each phase light intensity image.

In summary, the measurement method and apparatus of the present disclosure can establish a three-dimensional shape of an object and reconstruct a two-dimensional shape of the object through the acquired plurality of phase light intensity images, so that the measurement can be performed to perform the object. Morphology, size or defect analysis.

The present disclosure has been disclosed in the above embodiments, but it is not intended to limit the present invention. A person skilled in the art can make various changes and modifications without departing from the spirit and scope of the invention. Therefore, the scope of the invention is defined by the scope of the appended claims.

200‧‧‧Measurement method

Steps 202, 204, 206, 208, 210‧‧

Claims (20)

A measuring method for measuring a three-dimensional and two-dimensional shape of an object, the surface of the object having a plurality of object points, the measuring method comprising: projecting a stripe pattern to a projection device The surface of the object; receiving and storing a multi-line image by a multi-line photoelectric image capturing system; moving the object relative to a phase shifting device system to extract multiple phase lights through the multi-line photoelectric image capturing system a strong image, the phase shifting device system includes the projection device and the multi-line photoelectric image capturing system; determining phase values of the object points according to the phase light intensity images, and converting the phase values into the triangular geometric relationship The height corresponding to the object points to generate a three-dimensional shape of the object; and reconstructing an image of the object without phase change according to the phase light intensity images, the image of the object without phase change exhibiting a two-dimensional appearance of the object size. The measuring method of claim 1, further comprising: comparing part or all of the image of the object without phase change with a preset object pattern to detect the surface topography, size or defect. The measuring method of claim 1, wherein the step of projecting the stripe pattern onto the surface of the object comprises: projecting a structured light onto a surface of the object to form the stripe pattern, The light intensity of the structured light changes sinusoidal intensity depending on the position. The measuring method according to claim 3, wherein the stripe pattern It is a pattern of a stripe-shaped sinusoidal intensity change, a pattern of sinusoidal intensity changes, or a pattern formed by a moiré. The measuring method according to claim 1, wherein the multi-line photoelectric image capturing system is a multi-line Photocoupled Device (CCD) camera. The measuring method of claim 1, wherein the step of capturing the phase light intensity image comprises: the phase shifting device system moves a single linear relative motion with the object; and responding to the single time The straight lines are relatively moved, and the phase light intensity images corresponding to the objects are sequentially captured by the multi-line photoelectric image capturing system. The measuring method according to claim 6, wherein the phase light intensity images sequentially acquired during the single linear relative movement have an equal phase shift amount. The measuring method of claim 1, wherein the phase light intensity images respectively comprise a plurality of pixels, wherein the pixels respectively correspond to a light intensity value, and the step of reconstructing the object image without phase change comprises: The phase light intensity images are converted into a high dynamic range 2D image as the object image without phase change according to the light intensity values corresponding to the pixels of the phase light intensity image. The measurement method of claim 1, wherein the phase light intensity images respectively comprise a plurality of pixels, wherein the pixels respectively correspond to a light intensity value and an image position, and the image of the object without phase change is reconstructed The steps include: Comparing the light intensity values of the pixels having the same image position in the phase light intensity images, and selecting the maximum light intensity value as the light intensity value of the pixels in the same image position in the object image without phase change. The measurement method of claim 1, wherein the phase light intensity images respectively comprise a plurality of pixels, wherein the pixels respectively correspond to a light intensity value and an image position, and the image of the object without phase change is reconstructed The step of averaging the light intensity values of the pixels having the same image position in the phase light intensity images to generate an average light intensity value, and using the average light intensity value as the object image with no phase change The intensity value of the pixel at the same image position. A measuring device for measuring a three-dimensional and two-dimensional shape of an object, the surface of the object having a plurality of object points, the measuring device comprising: a phase shifting device system comprising: a projection device For projecting a stripe pattern onto the surface of the object; and a multi-line photoelectric image capturing system for receiving and storing a multi-line image, wherein the object moves relative to the phase shifting device system to transmit the multi-line image The line photoelectric image capturing system captures a plurality of phase light intensity images; and an image processing unit is configured to determine phase values of the object points according to the phase light intensity images, and convert the phase values into The height corresponding to the object points to generate a three-dimensional shape of the object, and reconstructing an object image without phase change according to the phase light intensity images, wherein the object image with no phase change presents a two-dimensional shape of the object Appearance size. The measuring device of claim 11, wherein the image processing unit compares part or all of the image of the object without phase change with a preset object pattern to detect the surface topography of the object, Size or defect. The measuring device of claim 11, wherein the projection device projects a structured light onto a surface of the object to form the stripe pattern, and the light intensity of the structured light varies with position. Make a change in sinusoidal intensity. The measuring device of claim 13, wherein the stripe pattern is a stripe-shaped sinusoidal intensity-changing pattern, a sinusoidal intensity-changing pattern, or a pattern formed by a moiré. The measuring device according to claim 11, wherein the multi-line photoelectric image capturing system is a multi-line Photocoupled Device (CCD) camera. The measuring device of claim 11, wherein the phase shifting device system moves relative to the object in a single straight line, and the multi-line photoelectric image capturing system responds to the single linear relative movement in sequence. Taking the phase light intensity images corresponding to the object. The measuring device of claim 16, wherein the phase light intensity images sequentially acquired during the single linear relative movement have equal phase shift amounts. The measuring device of claim 11, wherein the phase light intensity images respectively comprise a plurality of pixels, wherein the pixels respectively correspond to a light intensity value, and the image processing unit is configured according to the phase light intensity image The light intensity corresponding to some pixels The values are converted into a high dynamic range 2D image as the object image without phase change. The measuring device of claim 11, wherein the phase light intensity images respectively comprise a plurality of pixels, wherein the pixels respectively correspond to a light intensity value and an image position, and the image processing unit compares the phase lights The intensity value of the pixel having the same image position in the strong image, and selecting the maximum light intensity value as the light intensity value of the pixel in the same image position in the object image without phase change. The measuring device of claim 11, wherein the phase light intensity images respectively comprise a plurality of pixels, the pixels respectively corresponding to a light intensity value and an image position, and the image processing unit is configured to the phase light The light intensity values of the pixels having the same image position in the strong image are averaged to generate an average light intensity value, and the average light intensity value is used as the light intensity value of the pixel in the same image position in the object image without phase change. .