KR102282722B1 - Apparatus and Method For Detecting Defects - Google Patents

Abstract

The present invention discloses a method and apparatus for detecting binding of a measurement object.
A method for detecting a defect in an object according to the present invention comprises the steps of calculating a compensated hologram of an object to be measured using a digital holographic microscope; extracting 3D phase information from the hologram of the object; and determining whether the object is defective by applying the phase information to a phase image defect detection convolutional neural network in which a convolutional filter is clustered. .

Description

Apparatus and Method For Detecting Defects

The present invention relates to a method and apparatus for detecting a defect.

More specifically, the present invention relates to a method and apparatus for detecting a defect using three-dimensional shape information and quantitative thickness information of an object obtained using the holographic restoration apparatus and method.

It is possible to solve the complex optical device structure required for one-shot digital holographic restoration using a single object holographic image of the prior art and a significant high cost problem accordingly, and defects through holographic restoration with a simple structure and low cost Detection is possible, and it has versatility that can be applied to both reflective and transmissive hologram restoration devices of the prior art. For detection, confirmation or display in various fields, including devices for detecting defects in ultra-fine structures such as TFTs and semiconductors, medical devices requiring precise three-dimensional image display, and other transparent objects such as lenses, etc. Applicable to the device.

A digital holographic microscope refers to a microscope that measures the shape of an object using digital holography technology.

While a general microscope is a device that measures the shape of an object by measuring the intensity of light reflected or transmitted from the object by illuminating it with a general light source, a digital holographic microscope can reduce the interference and diffraction of light that occurs when light is irradiated onto an object. It is a device that measures and records it digitally, and restores the shape information of an object from this information.

In other words, digital holography technology generates light of a single wavelength like a laser, splits it into two lights using a light splitter, and illuminates one light directly on the image sensor (referred to as reference light), and the other light When the light reflected from the object to be measured is irradiated onto the object and reflected on the image sensor (referred to as object light), the reference light and the object light interfere with each other in the image sensor. It is a technique of recording and restoring the shape of a measurement target object by using a computer with the recorded interference fringe information. In this case, the recorded interference fringe information is generally referred to as a hologram.

On the other hand, in the case of the conventional optical holography technology other than digital holography, the interference fringe information of the light is recorded with a special film, and the reference light is irradiated onto the special film on which the interference fringe is recorded to restore the shape of the measurement target. This is a method in which the shape of the virtual measurement object is restored to the position where the measurement object was originally located.

Compared with the conventional optical holography method, the digital holographic microscope measures the interference fringe information of light with a digital image sensor and stores it digitally, and the stored interference fringe information is numerically calculated using a computer device rather than an optical method. It is different in that it restores the shape of the object to be measured by processing it.

In a conventional digital holographic microscope, a laser light source of a single wavelength may be used first. However, when using a single laser light source, there is a problem in that the measurement resolution of the object, that is, the minimum measurement unit is limited by the wavelength of the laser light source used. In addition, when a laser light source of two or multiple wavelengths is used among existing digital holographic microscopes, the cost increases by using light sources having different wavelengths, or holographic images are sequentially acquired using light sources of different wavelengths. Therefore, there is a problem in that it is difficult to measure the three-dimensional change information of the object to be measured in real time.

In addition, in the above-described conventional digital holography technology, a computer generated hologram (CGH) is generated with a computer to restore the shape of a measurement target object, and it is displayed on a spatial light modulator (SLM) and then the reference light is irradiated. , a three-dimensional holographic image of an object is obtained by diffraction of the reference light. In this case, since the use of an expensive spatial light modulator (SLM) (more than tens of thousands of won) is required, there is considerable difficulty in practical application.

As one of the methods for solving the problems of the conventional digital holography technology described above, for example, as of September 5, 2014, by Kim Eun-soo et al., the invention of a digital holographic microscope and a digital holographic image generation method is named Republic of Korea Patent Application No. 10 It is specifically exemplified in Korean Patent Application Laid-Open No. 10-2016-0029606 (hereinafter referred to as "published prior art"), which was applied for -2014-0119395 and published on March 15, 2016.

1 is a block diagram illustrating in detail a two-wavelength digital holographic microscope apparatus according to the disclosed prior art.

Referring to FIG. 1 , the disclosed two-wavelength digital holographic microscope apparatus of the prior art includes a mixed light source unit 100 , a wavelength division unit 200 , an interference fringe acquisition unit 300 , an object unit 400 , and an image sensor unit 500 . ), an image storage unit 600 , a control unit 700 , and an object shape restoration unit 800 .

The mixed light source unit 100 includes a mixed light source light emitting unit 110 and a light source unit lens 120 . The mixed light source and light-emitting unit 110 emits mixed light having a wavelength band distributed in several non-uniform bands. The light source lens 120 optically adjusts the mixed light generated by the mixed light source emitting unit 110 , and makes it incident on the wavelength dividing unit 200 .

The wavelength dividing unit 200 includes a first light splitter 210 , a first filter plate 220 , a second filter plate 230 , and a first reflector 240 . The first light splitter 210 receives the mixed light incident from the mixed light source unit 100 and splits it into two lights. In this case, the first light splitter 210 divides the incident mixed light in different directions to proceed. The first filtering plate 220 receives one light among the lights divided by the first light splitter 210 to obtain a first light beam having a predetermined single wavelength. Here, the light input to the first filtering plate 220 is filtered while passing through the first filtering plate 220 , and a first ray having a single wavelength determined according to the characteristics of the first filtering plate 220 is obtained. The second filter plate 230 receives the other one of the lights divided by the first light splitter 210 in the same manner as the first filter plate 220 , and receives a second light having a wavelength different from the wavelength of the first light beam. get the beam And the second ray is sent to the interference fringe acquisition unit 300 . The first reflector 240 serves to receive the first ray obtained from the first filtering plate 220 and reflect it to the interference fringe acquisition unit 300 .

The interference fringe acquisition unit 300 includes a second light splitter 310 , a third light splitter 320 , a second reflector 330 , a third filter plate 340 , and a third reflector 350 . The second beam splitter 310 receives the first beam input from the wavelength splitter 200 and splits it into a first object beam and a first reference beam. In this case, the second beam splitter 210 divides the incident first beam in different directions to advance the beam. The third beam splitter 320 also receives the second beam in the same manner as the second beam splitter 310 and splits it into a second object beam and a second reference beam. The second reflector 330 receives the first reference light, and transmits the reflected first reference light to the second light splitter 310 . The third filter plate 340 may receive the first reference light divided by the second beam splitter 310 and send it to the second reflector 330 , and may receive the reflected first reflection reference light and send it to the second beam splitter. In addition, the third filter plate 340 blocks the progress of the second object light so that it does not reach the second reflector 330 when the light of the second object reaches the second light splitter 310 and is split so that some of the light travels in the direction of the second reflector 330 . . To this end, the third filter plate 340 is a filter plate having the same characteristics as the first filter plate 220 in transmitting light. The third reflector 350 receives the second reference light, and sends the reflected second reference light to the third light splitter 320, where the second reflector 330 and the third reflector 350 are the control unit 700 ) can be configured so that the angle can be adjusted according to the control of an off-axis hologram.

Meanwhile, the first and second object lights obtained as described above are converted into first and second reflecting object lights, respectively, and sent to the image sensor unit 500 through the following process. The second light splitter 310 injects the first object light divided as described above to the measurement target object mounted on the objective unit 400 , and the second object is divided and sent from the third light splitter 320 . Light is incident on the measurement target object. In this case, the reflected light reflecting the first object light incident from the measurement target object is referred to as the first reflecting object light. In addition, the reflected light reflecting the second object light incident from the measurement target object is referred to as a second reflecting object light. The second light splitter 310 receives the first and second reflector light reflected as described above and transmits them to the third light splitter 320 . The third light splitter 320 transmits the first and second reflecting object light received as described above to the image sensor unit 500 again.

In addition, the first reflection reference light and the second reflection reference light obtained as described above are sent to the image sensor unit 500 through the following process. Specifically, the second beam splitter 310 receives the first reflection reference light reflected from the second reflector 330 and transmits it to the third beam splitter 320 . As described above, the third light splitter 320 receives the first reflection reference light sent from the second light splitter 310 and the second reflection reference light reflected from the third reflector 350 and receives the image sensor unit 500 again. send to Accordingly, in the third beam splitter 320, the first reflecting object light, the first reflecting reference light, the second reflecting object light, and the second reflecting reference light are all sent in the same direction to the image sensor unit 500, and then mutually interfere with each other. An interference fringe is created.

On the other hand, the second reflector 330 and the third reflector 350 are angled according to the control of the controller 700 in order to configure an off-axis system in which light rays of different wavelengths form different interference fringes. It is characterized in that it can be adjusted in multiple directions. That is, as the angles of the second reflector 330 and the third reflector 350 are different from each other, the first reflection reference light reflected from the second reflector 330 and the second reference light reflected from the third reflector 350 are different. When the separation occurs in the direction of the light, the first reflection reference light and the second reflection reference light are combined with the first reflection object light and the second reflection object light reaching the image sensor unit 500 to form an interference fringe, each wavelength Differently deaxially descaled interference fringes are formed.

The objective unit 400 includes an object holder 410 and an objective lens 420 . The object holder 410 fixes the measurement target object to the holder to be measured, and the objective lens 420 optically adjusts the first object light and the second object light incident on the measurement target object.

The image sensor unit 500 projects the interference fringe obtained by the interference fringe acquisition unit 300 onto a digital image sensor, measures the projected interference fringe using the digital image sensor, and converts the measured value to a discrete signal. convert to In general, a recording of the interference fringe is called a hologram. As such a digital image sensor, various image sensors such as a CCD may be used.

The image storage unit 600 stores the interference fringe information converted into the discrete signal by the image sensor unit 500 in various storage media such as a memory or a disk device.

The control unit 700 implements the above-described off-axis system and controls the positions and angles of the second reflector 330 and the third reflector 350 to obtain the interference fringe, such as the interference fringe acquisition unit 300 ) and control the objective part 400 such as adjusting the objective lens 420 to adjust the first and second object lights incident on the measurement target object, and the interference fringes are measured and corresponding The image sensor unit 500 is controlled to convert the information into a discrete signal, and the image storage unit 600 is controlled to store the interference fringe information converted into the discrete signal.

The object shape restoration unit 800 includes a phase information acquisition unit 810 , a thickness information acquisition unit 820 , and a shape restoration unit 830 . The phase information acquisition unit 810 obtains phase information of the interference fringe for the first ray and phase information of the interference fringe for the second ray, respectively, using the interference fringe information, and the thickness information acquisition unit 820 , respectively. acquires thickness information of the measurement target object using the phase information, and the shape restoration unit 830 restores the real-time three-dimensional shape of the measurement target object using the thickness information. In this case, the thickness information of the object to be measured includes information on the difference between the paths traveled by the object light and the reference light, respectively. Due to the optical path difference between the object light and the reference light, the interference fringe is formed when the object light and the reference light overlap.

In the disclosed prior art described above, the effect of increasing the measurement resolution of the measurement object and measuring and recording the hologram of the measurement object that changes with time in real time to restore the three-dimensional shape information of the measurement object in real time is achieved, but the following problems still occur.

More specifically, in the disclosed prior art, since a mixed light source having a wavelength band distributed in several non-single bands is used, the wavelength dividing unit 200 performs the first wavelength division in order to obtain at least two or more single wavelengths. In order to split the light beam and the second light source, the first filter plate 220 , the second filter plate 230 , and the first reflector 240 must be used. In addition, the interference fringe acquisition unit 300 includes a third light splitter 320 for splitting the second light source, a third reflector 350 for reflecting the second light source, and a second light source for the second reflector 330 . It is necessary to additionally use the third filter plate 340 to block the incident. Accordingly, there is still a problem that the structure of the entire device becomes complicated and the overall manufacturing cost is high.

Therefore, it is possible to solve the above-mentioned problems while using a light source of a single wavelength, as well as devices for detecting defects of ultra-fine structures such as TFTs and semiconductors, medical devices requiring precise three-dimensional image display, and other lenses. There is a need for a new method that can be applied to devices for detection, confirmation, or display in various fields, including detecting a refractive index error of a transparent object.

Republic of Korea Patent Publication No. 10-2016-0029606 Republic of Korea Patent Publication No. 10-2010-0095302 Republic of Korea Patent Publication No. 10-2012-0014355 Korean Patent No. 10-1139178 Korean Patent No. 10-1441245 US Patent No. 7,649,160

The present invention is to solve the problems of the prior art described above, by acquiring only a single object hologram image, using only an object hologram image obtained without using a reference hologram image, and a digital reference light generated from the obtained object hologram. By restoring the three-dimensional shape information and quantitative thickness information of an object, it is possible to solve the complex optical device structure required for one-shot digital holography restoration using a single object hologram image of the prior art and the significant expensive cost problem. In addition, holographic restoration is possible with a simple structure and low cost, and has versatility that can be applied to both reflective and transmissive hologram restoration devices of the prior art. Quantitative 3D image restoration of the target object is possible, and the refractive index error detection of transparent objects such as devices for detecting defects in ultra-fine structures such as TFTs and semiconductors, medical devices requiring precise 3D image display, and other lenses A method and apparatus for detecting defects using three-dimensional shape information and quantitative thickness information of an object obtained using an improved holographic restoration device that can be applied to devices for detection, confirmation or display in various fields, including is to provide

According to a first aspect of the present invention, there is provided a method for detecting a defect in a measurement object, the method comprising: calculating a compensated hologram of an object to be measured using a digital holographic microscope; extracting 3D phase information from the hologram of the object; and determining whether the object is defective by applying the phase information to a phase image defect detection convolutional neural network in which a convolutional filter is clustered. .

According to a second aspect of the present invention, there is provided an apparatus for detecting a defect in a measurement object, comprising: a hologram measurement unit for measuring hologram data of an object to be measured; and calculating a compensated hologram of the object, extracting three-dimensional phase information from the hologram of the object, and applying the phase information to a phase image defect detection convolutional neural network in which a convolutional filter is clustered. Convolutional Neural Network) and a control unit for determining whether the object is defective.

The following advantages are achieved using the above-described apparatus and method for detecting a defect in a measurement object according to the present invention.

1. It is possible to solve the problem of a complex optical device structure required for one-shot digital holography restoration using a single object holographic image of the prior art and a significant high cost thereof.

2. Holographic restoration is possible with a simple structure and low cost.

3. It has versatility that can be applied to both reflective and transmissive hologram restoration devices of the prior art.

4. In particular, a reference hologram image is unnecessary when hologram restoration is performed, and a quantitative 3D image restoration of a measurement target is possible in real time.

5. Detection, confirmation, or display in various fields, including devices for detecting defects in ultra-fine structures such as TFTs and semiconductors, medical devices requiring precise three-dimensional image display, and other transparent objects such as lenses to detect refractive index errors It can be applied to the device for

6. Both lateral and axial defects (three-dimensional defects) of an object can be detected.

7. Since defect detection can be automated, it is expected that the manufacturing cost of the manufacturer will be reduced and the cost burden of consumers will be reduced through this.

Further advantages of the present invention may become apparent from the following description with reference to the accompanying drawings in which like or similar reference numerals denote like elements.

1 is a block diagram showing in detail a two-wavelength digital holographic microscope apparatus according to the disclosed prior art.
2 is a view showing a digital holographic microscope belonging to the hologram measuring unit of the present invention.
3 is a flowchart of a defect detection method according to an embodiment of the present invention.
4 is a diagram illustrating a method of determining whether a defect is defective through the implementation of a phase image defect detection convolutional neural network according to an embodiment of the present invention.
5 is a block diagram illustrating an internal structure of a defect detection apparatus according to an embodiment of the present invention.

In describing embodiments in the present specification, descriptions of technical contents that are well known in the technical field to which the present invention pertains and are not directly related to the present invention will be omitted. This is to more clearly convey the gist of the present invention by omitting unnecessary description.

For the same reason, some components are exaggerated, omitted, or schematically illustrated in the accompanying drawings. In addition, the size of each component does not fully reflect the actual size. In each figure, the same or corresponding elements are assigned the same reference numerals.

Advantages and features of the present invention, and a method for achieving them will become apparent with reference to the embodiments described below in detail in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but may be implemented in various different forms, and only these embodiments allow the disclosure of the present invention to be complete, and common knowledge in the art to which the present invention pertains. It is provided to fully inform the possessor of the scope of the invention, and the present invention is only defined by the scope of the claims. Like reference numerals refer to like elements throughout.

Additionally, each block may represent a module, segment, or portion of code that includes one or more executable instructions for executing specified logical function(s). It should also be noted that in some alternative implementations it is also possible for the functions recited in blocks to occur out of order. For example, two blocks shown one after another may in fact be performed substantially simultaneously, or it is possible that the blocks are sometimes performed in the reverse order according to the corresponding function.

In this case, the term '~ unit' used in this embodiment means a software or hardware component, and '~ unit' performs certain roles. However, '-part' is not limited to software or hardware. The '~ unit' may be configured to reside on an addressable storage medium or may be configured to refresh one or more processors. The functions provided in the components and '~ units' may be combined into a smaller number of components and '~ units' or further separated into additional components and '~ units'.

Hereinafter, the present invention will be described in detail with reference to embodiments and drawings of the present invention.

2 is a view showing a digital holographic microscope belonging to the hologram measuring unit of the present invention.

The digital holographic microscope includes a light source unit 210 for emitting single wavelength light; a collimator 220 for collimating the single wavelength light emitted from the light source unit 210; a light splitter 230 for splitting the single wavelength light passing through the collimator 220 into an object light O and a reference light R; an object light objective lens 240 for passing the object light O divided by the light splitter 230; a reference light objective lens 260 for passing the reference light R divided by the light splitter 230; an optical mirror 270 for reflecting the reference light R that has passed through the reference light objective lens 60; The object light O reflected from the surface of the measurement target 250 after passing through the object light objective lens 240 and the reference light R reflected by the optical mirror 270 are respectively transmitted to the object light objective lens ( 240) and a recording medium 280 for recording an interference fringe formed by passing through the reference light objective lens 260 and being transmitted to the light splitter 230; and a processor 290 for receiving and storing an image file generated by converting the interference fringe in the recording medium 280 .

The processor 290 extracts reference light information of the object hologram from the object hologram obtained from the image file to generate a digital reference light, calculates a compensated object hologram using the object hologram and the digital reference light, and calculates the compensation The three-dimensional information of the measurement target object 250 may be restored by extracting the phase information of the object hologram.

And the object 250 to be measured by the microscope may be measured by the microscope through the moving rail 255 . The defect determination unit 295 connected to the processor 290 may use the compensated hologram of the object to determine whether the object is defective.

The processor 290 is, for example, implemented as a device capable of arithmetic operation, such as a microprocessor or a personal computer (PC), and the recording medium 80 is, for example, a charge coupled device (CCD) or a complementary metal (CMOS). -Oxide Semiconductor) may be implemented as an image sensor.

In addition, the information of the object hologram obtained by the processor 290 includes wavelength and phase information of the object, and an aberration of the object light objective lens 240 , and noise (eg, laser photon use). may additionally include speckle noise and the like). The object hologram obtained by the processor 290 is a complex conjugated hologram, and may be expressed as Equation 1 below.

Equation 1: | U _o (x,y,0)| ² = | O (x,y)| ² +| R (x,y)| ² + O ^* (x,y) R (x,y)+ O (x,y) R ^* (x,y)

In Equation 1, x and y represent spatial coordinates, U _o (x,y,0) represents the obtained object hologram, and O (x,y) and R (x,y) represent object light (O) and the reference light (R), and O ^* (x,y) and R ^* (x, y) represent the complex conjugate of the object light (O) and the reference light (R), respectively.

Hereinafter, a specific method for generating a digital reference light and a compensated object hologram from the obtained object hologram will be described.

First, the processor 290 obtains an object hologram from the image file of the interference fringe recorded on the recording medium 280 . The obtained object hologram consists of an interference pattern of an object light O having phase information of the object and a reference light R having no phase information of the object.

Thereafter, a 2D Fourier transform is performed on the obtained object hologram in order to extract information on the reference light R without the phase information of the object from the obtained object hologram. The frequency spectrum of the object hologram obtained by the two-dimensional Fourier transform includes spectrum information including real image spot-position, spectrum information including imaginary image spot-position, and direct current (DC) information. ) is separated into spectral information including Only the separated real image coordinate information is extracted from the frequency spectrum using an automatic real image spot-position extraction algorithm. Reference light information of the obtained object hologram is extracted using the extracted real image coordinate information.

Thereafter, the extracted reference light information of the processor 290 may cause a phase break every 2π due to the wave nature of light, and is extracted using a known wavenumber algorithm to compensate for the phase break Calculate the wavenumber vector constant of the reference light information. A compensation term of the extracted reference light information is calculated using the calculated wavenumber vector constant. The compensation term of the extracted reference light information calculated from the wavenumber vector constant is a conjugate term of the obtained object hologram. The calculated compensation term of the reference light information extracted in this way is referred to as a digital reference light, and it is expressed as Equation 2 below.

Equation 2: R _c (x,y)= conj [ R (x,y)]

In Equation 2, R _c (x,y) is the digital reference light, R (x,y) is the reference light information of the obtained object hologram, and conj is a function for obtaining the conjugate of a complex number.

Thereafter, the processor 290 extracts aberration information from the object hologram to compensate for the curvature aberration of the object light objective lens 240 used to obtain the object hologram. Thereafter, the processor 290 generates a curvature aberration information compensation term using an automatic frequency curvature compensation algorithm. Here, the curvature aberration information compensation term will be referred to as digital curvature.

Thereafter, the processor 290 calculates a compensated object hologram by multiplying the obtained object hologram by the compensation term of the extracted reference light information. If this is expressed as an equation, it is as Equation 3.

Equation 3: U _C (x,y,0)= O (x,y) R ^* (x,y) R _C (x,y) R _CA (x,y)

In Equation 3, U _C (x,y,0) is the compensated object hologram, O (x,y) and R ^* (x,y) are the object light and reference light of the obtained object hologram, respectively, and R _C (x,y) is the digital reference light, and R _CA (x,y) is the digital curvature.

Thereafter, the processor 290 converts the compensated object hologram into information of a reconstruction image plane using an Angular Spectrum Propagation algorithm. Here, the reconstructed image plane means a virtual image display plane equal to the distance corresponding to the distance between the measurement target 250 and the recording medium 280 by the processor 290 , and is calculated by the processor 290 . and simulated. The processor 90 extracts phase information of the compensated object hologram through an inverse 2D Fourier transform. It should be noted that the phase information of the extracted compensated object hologram includes only the phase information of the object, since the phase information extracted in this way removes the information of light and the aberration information of the objective lens from the obtained object hologram.

Thereafter, the processor 290 calculates quantitative thickness information of the measurement target object 250 by using the extracted phase information of the compensated object hologram. In this case, since the extracted phase information of the compensated object hologram may additionally include fine noise such as speckle noise according to the use of photons of the laser, the processor 290 measures it. Before calculating the quantitative thickness information of the target object 250 , such fine noise may be removed in advance. Specifically, the processor 290 uses a two-dimensional phase unwrapping algorithm to extract the phase information of the compensated object hologram from the distorted phase information due to fine noise and phase-wrapped phase. can be compensated When the distorted phase information due to the fine noise and the phase folded (wrapped phase) phenomenon is removed, the quantitative thickness information of the measurement target object 250 can be more precisely calculated based on the phase information of the compensated object hologram. . Quantitative thickness information of the measurement target object 50 calculated in the above manner is expressed as in Equation 4 below.

Equation 4: ΔL = λΔφ (x,y)/2π Δn(x,y)

In Equation 4, ΔL is quantitative thickness information of the measurement target object 50, λ is the wavelength of the light source unit 210 used to obtain the object hologram, φ (x,y) is phase information of the compensated object hologram, Δn(x,y) denotes a refractive index difference between the background (or air) and the measurement target 250 . The processor 290 may reconstruct the three-dimensional shape of the measurement object 250 on the above-described reconstructed image plane by using the quantitative thickness information of the measurement object 250 calculated according to Equation 4 above.

The processor 290 may correspond to the compensation hologram calculator 531 and the 3D phase information extractor 532 in the controller 530 of FIG. 5 to be described later.

3 is a flowchart of a defect detection method according to an embodiment of the present invention.

First, in step S310, the defect detection apparatus may calculate a compensated hologram of the object to be measured. The device may measure the hologram before compensation of the object using the digital holographic microscope of FIG. 2 .

Specifically, the defect detection apparatus of the present invention extracts reference light information from the hologram of the object, calculates a wavenumber vector constant of the reference light information, and calculates a compensation term of the reference light information using the calculated wavenumber vector constant. A digital reference light may be generated. Then, the device extracts aberration information from the object hologram, generates a digital curvature that compensates for aberration based on the extracted aberration information, and multiplies the compensation term of the reference light information by the hologram of the object to compensate for the object hologram. can be calculated.

And in step S320, the defect detection apparatus may extract 3D phase information from the compensated hologram. Specifically, the defect detection apparatus converts the compensated object hologram into information of a reconstruction image plane using an Angular Spectrum Propagation algorithm, and an Inverse 2D Fourier transform transform) can be used to extract phase information of the compensated object hologram.

The extracted phase information includes only the phase information of the object by removing light information and aberration information of the objective lens from the hologram of the object.

Then, in step S330, the defect detection apparatus applies the extracted phase information to a phase image defect detection convolutional neural network in which convolutional filters are clustered to determine whether the object is defective. can be decided

The defect detection apparatus according to the present invention may group the convolutional filters before determining whether the defect is defective. First, the defect detection apparatus of the present invention may receive design data of an object to be measured. The design data means three-dimensional size data of the object, and may be position data in the lateral direction (xy plane) and longitudinal direction (thickness or z-axis), but is not limited to such position data, and is not limited to volume, curvature, etc. It can contain various data such as

In addition, the defect detection apparatus of the present invention may receive a plurality of actual data for which a good product or a defective product is actually determined. An actual numerical value of a good product or a defective product corresponding to the design data may be input. For example, when the design data is 3D size data, the actual data may also receive 3D size data (transverse direction and longitudinal direction data) of a good product or a defective product. In addition, the actual data may receive at least 5000 or more for each of good and defective products, but is not limited thereto. The user of the defect detection apparatus of the present invention may distinguish the good product from the defective product based on whether the function of the product produced based on the design data operates normally.

In addition, after it is determined whether the object to be measured is defective, the defect detection apparatus of the present invention may update the actual data by including the data of the object in the actual data. In this case, since the number of actual data acquired increases as the number of times of determining whether the object is defective increases, the accuracy of determining whether the object to be measured is defective may increase.

In addition, the defect measuring apparatus of the present invention may calculate a similarity by comparing each of the design data and the actual data. The defect measuring apparatus of the present invention may calculate the overall similarity between the design data and the actual data by using a correlation function. If the similarity value is the same, it is expressed as 1, and as it is not similar, the similarity value decreases and converges to 0.

In addition, the defect measuring apparatus of the present invention may check the coordinate values of the pixels having the similarity out of an arbitrary error range. The coordinate values mean characteristic determination position coordinates. Specifically, when a good product is determined even outside the error range, the characteristic determination position coordinate is the coordinate of a pixel out of the error range. If the object to be measured thereafter corresponds to the characteristic determination position coordinates, the probability of receiving a defective product judgment is high even if it is outside the error range.

Conversely, even when a defective product is determined out of the error range, the characteristic determination position coordinate is the coordinate of a pixel out of the error range. If the object to be measured later corresponds to the characteristic determination position coordinates, there is a high probability of receiving a defective product determination.

In addition, the defect measuring apparatus of the present invention may determine a convolutional filter by using the information in the characteristic determination position coordinates. Specifically, the defect measuring apparatus of the present invention can determine a convolutional filter of a certain size by categorizing the information in the characteristic determination position coordinates in the lateral direction and the longitudinal direction. The size of the convolution filter may be a 3 x 3 matrix, and the defect measuring apparatus of the present invention may determine each convolution filter by using actual data on 5000 or more good and defective products, respectively. Since the convolution filter includes characteristics of good or bad products, the value of the convolution filter may be determined by reflecting the characteristics.

Even if all of the convolution filters are applied to the 3D phase information, it is possible to know whether the object is defective. However, if the number of actual data increases in order to increase the accuracy of determination, there is a disadvantage in that the amount of calculation of the defect measuring apparatus of the present invention is rapidly increased, thereby slowing down the calculation speed. Therefore, the defect measuring apparatus of the present invention can group filters having similar characteristics to the convolution filters. The clustering process can be implemented with a general algorithm used in a convolutional neural network. By utilizing the above-described clustering, the defect measuring apparatus of the present invention can efficiently apply the convolution filter to determine whether there is a defect.

The defect measuring apparatus of the present invention may apply the phase information to a filter clustered by the similar characteristics. In addition, the defect measuring apparatus of the present invention may generate a convolution layer by applying each of the filter-applied values to the ReLu function. The convolution layer is made of values obtained by applying the phase information to the convolution filter, and the values may indicate characteristics of a good product or a defective product, respectively. The ReLu function can be expressed as Equation 5 below.

Equation 5:

In addition, the defect measuring apparatus of the present invention may determine whether the object is defective by combining the convolutional layer and the Softmax function. The Softmax function can be expressed as Equation 6 below.

Equation 6:

The inputs of the Softmax function are the values of the convolutional layer, and N is the number of outputs. Since the Softmax function can represent the frequency of input values as a probability, the defect measuring apparatus of the present invention can determine whether a defect is defective based on a characteristic corresponding to a maximum value among the result values of the function. This defect determination process will be described in detail with reference to FIG. 4 below.

In addition, the phase information is applied to the clustered convolution filter to generate a convolution layer, and a neural network implemented by applying a Softmax function to the convolution layer is used to detect a phase image defect. It is called a detection convolutional neural network.

The defect measuring apparatus of the present invention may determine whether the 3D object is defective through the neural network. In contrast to existing devices that can determine whether a defect exists only in a two-dimensional plane, the present invention uses a three-dimensional hologram, so that a three-dimensional defect of an object to be measured can be detected.

4 is a diagram illustrating a method of determining whether a defect is defective through the implementation of a phase image defect detection convolutional neural network according to an embodiment of the present invention.

In the defect detection apparatus of the present invention, the 3D phase information 410 may be input to the phase image defect detection convolutional neural network. The phase information 410 may be extracted from the hologram compensated in step S320. Since the present invention can detect a defect using a three-dimensional image instead of a conventional two-dimensional image, in particular, a defect in the longitudinal direction (height or thickness of an object) can be detected.

When the phase information 410 is input to the convolutional neural network 440, the defect measuring apparatus of the present invention may determine whether the measured object is defective. Specifically, the defect measuring apparatus of the present invention may apply a convolutional filter clustered with similar characteristics to the phase information 410 . In addition, the device may generate a convolution layer 420 by applying the ReLu function to each of the values to which the filter is applied. The device may implement the neural network 430 by combining the convolutional layer and the Softmax function to determine whether the object is defective. The neural network 430 implemented by combining the Softmax function to the convolutional layer 420 may be referred to as a convolutional neural network 440 , and this convolutional neural network 440 is a phase image defect detection convolutional neural network (Phase). Image defect detection is called a Convolutional Neural Network). The method of generating the phase image defect detection convolutional neural network may be included in step S330 of FIG. 3 .

A method of determining whether a defect is defective using the neural network is as follows. The result of the convolutional layer is all positive numbers by the ReLu function. The result value is a number expressing the characteristics of a good product or a defective product. And when all of the result values are input to the Softmax function, the probability of the result values having the characteristics of a good product and the probability of the result values having the characteristics of the defective product can be known.

The result determined by the neural network is a good product or a bad product, and each is expressed as a probability. For example, when the probability of a good product is 0.8 and the probability of a defective product is 0.2, the object measured by the defect measuring apparatus of the present invention may be determined as a good product.

5 is a block diagram illustrating an internal structure of a defect detection apparatus according to an embodiment of the present invention. As will be described in detail below, the defect detection apparatus of the present invention may include a hologram measurement unit 510 , a display unit 520 , and a control unit 530 .

The hologram measuring unit 510 of the defect measuring apparatus of the present invention may measure a hologram of an object to be measured. For the measurement of the hologram, the hologram measuring unit 510 may include a digital holographic microscope, and since the digital holographic microscope has already been described in detail above with reference to FIG. 2 , a detailed description thereof will be omitted. .

The display unit 520 of the defect measuring apparatus of the present invention may include a display capable of displaying characteristic information of the object and whether the object is defective.

The controller 530 of the defect measuring apparatus of the present invention may control the overall operation of the defect measuring apparatus. The control unit 530 includes a compensation hologram calculation unit 531 capable of calculating a compensation hologram from the hologram of the object measured by the hologram measurement unit 510, and a 3D phase capable of extracting 3D phase information from the compensation hologram. The information extraction unit 532, the convolution filter clustering unit 533 that determines a convolution filter based on the design data and the actual data for at least one good or defective product, and clusters similar characteristics among similar characteristics, and the phase image defect detection convolution and a defect determination unit 534 capable of determining whether the object is defective by using a phase image defect detection convolutional neural network.

The compensation hologram calculation unit 531 may calculate a compensation hologram from the hologram of the object measured by the hologram measurement unit 510 . The compensation hologram calculation method has been described in detail with reference to FIG. 2 described above.

The 3D phase information extraction unit 532 may extract 3D phase information from the compensation hologram. In order to extract 3D phase information from the compensation hologram, the 3D phase information extractor 532 may perform a 2D Fourier transform. The detailed extraction method has been described in detail with reference to FIG. 2 .

The convolution filter grouping unit 533 may determine a convolution filter based on design data and actual data on at least one good or defective product, and may group similar characteristics. When it is determined whether the object to be measured is defective, the convolution filter clustering unit 533 may determine a convolution filter based on the measurement data of the object, and apply the determined convolution filter to the clustered convolution filter. can be added to Accordingly, when the defect measuring apparatus of the present invention determines whether there is a defect, the clustered convolution filter may be updated.

The method of determining and clustering the convolution filter has been described in detail in step S330 of FIG. 3 above.

The defect determination unit 534 may apply the clustered convolution filter to the phase information to generate a convolution layer, and may determine whether the object is defective by implementing a phase image defect detection convolutional neural network. The defect determination unit 534 applies a ReLu function to each value to which the convolution filter is applied to the phase information to generate a convolution layer, and combines the convolution layer and the Softmax function to obtain a phase image. A defect detection convolutional neural network can be implemented.

The method of determining whether the object is defective has been described in detail with reference to step S340 of FIG. 3 and FIG. 4 .

Embodiments of the present invention disclosed in the present specification and drawings are merely provided for specific examples in order to easily explain the technical contents of the present invention and help the understanding of the present invention, and are not intended to limit the scope of the present invention. It will be apparent to those of ordinary skill in the art to which the present invention pertains that other modifications based on the technical spirit of the present invention can be implemented in addition to the embodiments disclosed herein.

Claims (15)

A method for detecting a defect in a measurement object, the method comprising:
calculating a compensated hologram of an object to be measured using a digital holographic microscope;
extracting 3D phase information from the compensated hologram of the object; and
and determining whether the object is defective by applying the three-dimensional phase information to a phase image defect detection convolutional neural network in which a convolutional filter is clustered,
Determining whether the object is defective includes generating a clustered convolutional filter,
The step of generating the clustered convolution filter comprises:
receiving design data as a reference for the object;
receiving actual data of at least one object that has been determined as good or defective in advance based on whether the function of the object produced based on the design data operates normally;
calculating a degree of similarity by comparing the design data with the actual data;
confirming characteristic determination position coordinates whose similarity is outside a predetermined error range;
determining a convolutional filter for each of the real data based on the characteristic determination position coordinates; and
clustering the convolutional filters to have similar properties
A method characterized in that. The method of claim 1, wherein calculating the compensated hologram of the object comprises:
obtaining a hologram of the object;
extracting reference light information from the hologram of the object;
generating a digital reference light by calculating a wavenumber vector constant of the reference light information and calculating a compensation term of the reference light information using the calculated wavenumber vector constant;
extracting aberration information from the hologram of the object and then generating a digital curvature compensating for the aberration; and
and calculating a compensated object hologram by multiplying the compensation term of the reference light information by the hologram of the object. The method of claim 1, wherein the extracting of the three-dimensional phase information comprises:
converting the compensated hologram of the object into information of a reconstruction image plane using an Angular Spectrum Propagation algorithm; and
Extracting the three-dimensional phase information of the compensated hologram of the object using an inverse 2D Fourier transform,
The 3D phase information includes only 3D phase information of the object by removing light information and objective lens aberration information from the hologram of the object. The method of claim 1, wherein the determining whether the object is defective comprises: after generating the clustered convolutional filter,
applying the three-dimensional phase information to the clustered convolution filter;
generating a convolution layer by applying a ReLu function to the value to which the clustered convolution filter is applied; and
and determining whether the object is defective by applying a Softmax function to the convolutional layer. delete 5. The method of claim 4,
The method further comprising the step of updating the clustered convolution filter by adding a convolution filter determined based on the measurement data of the object to the clustered convolution filter when it is determined whether the object is defective method. The method of claim 1,
The method further comprising the step of displaying the characteristic information of the object and whether the object is defective on a display (display). An apparatus for detecting a defect in a measurement object, the apparatus comprising:
a hologram measurement unit for measuring hologram data of an object to be measured; and
The phase image defect detection convolutional neural network (Phase) calculates the compensated hologram of the object, extracts 3D phase information from the compensated hologram of the object, and collects the 3D phase information with a convolutional filter. A control unit for determining whether the object is defective by applying it to an image defect detection Convolutional Neural Network,
the control unit
A defect determination unit that determines whether the object is defective by generating a clustered convolution filter, and
Receive design data as a reference for the object to be measured, and receive actual data of at least one object that has been judged as good or defective in advance based on whether the function of the object produced based on the design data operates normally, The similarity is calculated by comparing the design data with the actual data, the characteristic determination position coordinates where the similarity degree is out of a predetermined error range are identified, and a convolutional filter for each of the actual data is performed based on the characteristic determination position coordinates. filter) and a convolutional filter clustering unit configured to cluster the convolutional filters to have similar properties. The method of claim 8, wherein the hologram measuring unit,
Device according to claim 1, further comprising a digital holographic microscope for measuring the hologram of the object. According to claim 8, wherein the control unit,
Digital reference light by obtaining a hologram of the object, extracting reference light information from the hologram of the object, calculating a wavenumber vector constant of the reference light information, and calculating a compensation term of the reference light information using the calculated wavenumber vector constant A compensation hologram calculation unit for generating a hologram of the object, extracting aberration information from the hologram of the object, generating a digital curvature compensating for aberration, and multiplying the compensation term of the reference light information by the hologram of the object to calculate the compensated object hologram Device further comprising. According to claim 8, wherein the control unit,
The compensated hologram of the object is converted into information of a reconstruction image plane using an Angular Spectrum Propagation algorithm, and the object using an Inverse 2D Fourier transform Further comprising a three-dimensional phase information extraction unit for extracting the three-dimensional phase information of the compensated hologram of,
The 3D phase information includes only 3D phase information of the object by removing light information and objective lens aberration information from the hologram of the object. The method of claim 8, wherein the defect determination unit,
The three-dimensional phase information is applied to the clustered convolution filter, a ReLu function is applied to the value to which the clustered convolution filter is applied to generate a convolution layer, and a Softmax function is applied to the convolution layer. Apparatus, characterized in that it determines whether the object is defective. delete The method of claim 12, wherein the convolution filter clustering unit,
When it is determined whether the object is defective, the clustered convolution filter is updated by adding a convolution filter determined based on the measurement data of the object to the clustered convolution filter. 9. The method of claim 8,
The device further comprising a display unit (or display unit) for displaying the characteristic information of the object and whether the object is defective.

Images