KR102194642B1 - A method to judge process defects using reconsructed hologram - Google Patents

Abstract

The three-dimensional shape of the object to be measured from an image including intensity information of the object hologram generated by interference between the reference light reflected from the optical mirror and the object light affected by the object to be measured according to an embodiment of the present invention. A method of generating information includes: identifying at least one frequency component included in the image; Extracting real image components corresponding to a real image from among the at least one frequency component; Generating a real hologram including real image information of the object to be measured and correction light having a conjugate relationship with the reference light based on the real image components; Generating an intermediate hologram from which information of the reference light is removed from the real hologram based on the correction light; Generating curvature aberration correction information from the intermediate hologram; Generating a correction hologram from which an error due to curvature aberration is removed from the intermediate hologram based on the curvature aberration correction information; And generating the 3D shape information of the object to be measured from the correction hologram.

Description

A method of determining defects in the process using the restored hologram {A METHOD TO JUDGE PROCESS DEFECTS USING RECONSRUCTED HOLOGRAM}

An embodiment of the present invention relates to a method of determining a defect in a process using a restored hologram.

A digital holographic microscope refers to a microscope that acquires the shape of an object using digital holography technology.

If a general microscope is a device that acquires the shape of an object by acquiring the reflected light reflected from the object, the digital holographic microscope is a device that acquires the interference and/or diffracted light generated by the object, and obtains the shape of the object from it. to be.

A digital holographic microscope uses a laser that generates light of a single wavelength as a light source, and divides the light generated by the laser into two pieces of light using an optical splitter. At this time, one light (hereinafter referred to as reference light) is directed toward the image sensor, and the other light (hereinafter referred to as object light) is reflected from the target object and directed toward the above-described image sensor so that interference between the reference light and the object light occurs. do.

The image sensor may record the interference fringe according to the interference phenomenon as a digital image, and restore the three-dimensional shape of the object to be measured from the recorded interference fringe. At this time, the interference pattern recorded by the image sensor is usually referred to as a hologram.

Conventional optical holographic microscopes record the interference pattern according to the interference phenomenon between reference light and object light with a special film. At this time, when the reference light is irradiated on the special film on which the interference pattern is recorded, the shape of the virtual object to be measured is restored at the place where the object to be measured was located.

Compared with conventional optical holographic microscopes, digital holographic microscopes digitize (or digitize) the interference pattern of light through an image sensor, and restore the shape of the object to be measured through electronic calculations rather than optical methods. There is a difference in

Meanwhile, a conventional digital holographic microscope using a single wavelength laser light source has a problem that the minimum unit length of measurement of an object is limited to the wavelength of the laser. In order to compensate for this, another conventional digital holographic microscope using a laser light source of two or more wavelengths has a problem in that the manufacturing cost of the microscope is high, and a three-dimensional shape of an object cannot be obtained in real time.

In addition, the above-described conventional digital holographic microscopes generate CGH (Computer Generated Hologram) with a computer in order to restore the shape of the object to be measured, then display it on a spatial light modulator (SLM), and The 3D hologram image of the object was acquired by illuminating the reference light. However, this method not only required the use of an expensive spatial light modulator (SLM), but was merely digitizing a special film in the above-described optical holographic microscope, and technical limitations were clear.

In order to solve the problems of such conventional digital holographic microscopes, for example, Korean Patent Laid-Open Publication No. 10-2016-0029606 (hereinafter referred to as "public prior art") proposes a digital holographic microscope and a digital holographic image generation method. Hereinafter, a brief look at the disclosed prior art.

1 is a block diagram showing in detail a two-wavelength digital holographic microscope apparatus according to the disclosed prior art.

Referring to FIG. 1, the disclosed conventional two-wavelength digital holographic microscope apparatus includes a mixed light source unit 100, a wavelength division unit 200, an interference fringe acquisition unit 300, an object unit 400, and an image sensor unit 500. ), an image storage unit 600, a control unit 700, and an object shape restoration unit 800.

The mixed light source unit 100 includes a mixed light source light emitting unit 110 and a light source unit lens 120. The mixed light source light emitting unit 110 emits mixed light having wavelength bands distributed in multiple bands that are not single. The light source unit lens 120 optically adjusts the mixed light generated by the mixed light source light emitting unit 110 and makes it incident on the wavelength dividing unit 200.

The wavelength splitter 200 includes a first light splitter 210, a first filter plate 220, a second filter plate 230, and a first reflector 240. The first light splitter 210 receives the mixed light incident from the mixed light source unit 100 and divides it into two lights. At this time, the first light splitter 210 serves to divide the incident mixed light in different directions to proceed. The first filter plate 220 receives one of the lights divided by the first light splitter 210 and acquires a first ray having a predetermined single wavelength. Here, light input to the first filter plate 220 is filtered while passing through the first filter plate 220, and a first ray of light having a single wavelength determined according to the characteristics of the first filter plate 220 is obtained. The second filter plate 230 receives the remaining one of the lights divided by the first light splitter 210 in the same manner as the first filter plate 220, and receives the second light having a wavelength different from that of the first light ray. Acquire a ray. And the second ray is sent to the interference pattern acquisition unit 300. The first reflector 240 serves to receive the first light ray acquired from the first filter plate 220 and reflect it to the interference fringe acquisition unit 300.

The interference fringe acquisition unit 300 includes a second light splitter 310, a third light splitter 320, a second reflector 330, a third filter plate 340, and a third reflector 350. The second light splitter 310 receives the first light input from the wavelength splitter 200 and divides it into a first object light and a first reference light. At this time, the second light splitter 310 serves to divide the incident first light rays in different directions to proceed. The third light splitter 320 also receives the second light beam in the same manner as the second light splitter 310 and splits it into a second object light and a second reference light. The second reflector 330 receives the first reference light and transmits the reflected first reference light to the second light splitter 310. The third filter plate 340 may receive the first reference light divided by the second light splitter 310 and send it to the second reflector 330, and receive the reflected first reflected reference light and send it to the second light splitter. In addition, the third filter plate 340 prevents the progress of the second object light from reaching the second reflector 330 when the second object light reaches the second light splitter 310 and is split into light, so that a part of the light proceeds toward the second reflector 330. . To this end, the third filtering plate 340 is a filtering plate having the same characteristics as the first filtering plate 220 in transmitting light. The third reflector 350 receives the second reference light and transmits the reflected second reference light to the third optical splitter 320, where the second reflector 330 and the third reflector 350 are controlled by the controller 700. ), it is possible to implement an off-axis hologram by configuring the angle to be adjustable according to the control.

Meanwhile, the first object light and the second object light obtained as described above are converted into first and second reflective object lights through the following process, and are sent to the image sensor unit 500. The second optical splitter 310 causes the light of the first object divided as described above to be incident on the object to be measured mounted on the object part 400, and is divided and sent from the third optical splitter 320. Light is incident on the object to be measured. In this case, the reflected light obtained by reflecting the first object light incident on the object to be measured is referred to as first reflective light. In addition, the reflected light that reflects the second object light incident from the object to be measured is referred to as second reflective object light. The second optical splitter 310 receives the first and second reflective light reflected as described above and sends them to the third optical splitter 320. The third optical splitter 320 sends the first and second reflective object light input as described above to the image sensor unit 500 again.

In addition, the first reflection reference light and the second reflection reference light obtained as described above are transmitted to the image sensor unit 500 through the following process. Specifically, the second light splitter 310 receives the first reflected reference light reflected from the second reflector 330 and sends it to the third light splitter 320. The third light splitter 320 receives the first reflection reference light sent from the second light splitter 310 and the second reflection reference light reflected from the third reflection body 350 as described above, and the image sensor unit 500 Send to Accordingly, from the third optical splitter 320, the first reflective object light, the first reflective reference light, the second reflective object light, and the second reflective reference light are all sent to the image sensor unit 500 in the same direction, and then interfere with each other. An interference fringe is created.

On the other hand, the second reflector 330 and the third reflector 350 are angled according to the control of the controller 700 in order to configure an off-axis system in which light rays of different wavelengths form different interference patterns. It is characterized in that it can be adjusted in multiple directions. That is, as the angles of the second reflector 330 and the third reflector 350 become different from each other, the first reflective reference light reflected from the second reflector 330 and the second reference reflected from the third reflector 350 When a separation occurs in the direction of the light, when the first reflection reference light and the second reflection reference light are combined with the first reflection object light and the second reflection object light reaching the image sensor unit 500 to form an interference pattern, each wavelength Differently, a deaxial interference pattern is formed.

The objective part 400 includes an object holder 410 and an objective lens 420. The object cradle 410 fixes the object to be measured to the cradle to be measured, and the objective lens 420 optically adjusts the first object light and the second object light incident on the object to be measured.

The image sensor unit 500 projects the interference pattern obtained by the interference pattern acquisition unit 300 onto a digital image sensor, measures the projected interference pattern using the digital image sensor, and measures the measured value as a discrete signal. Convert to Usually, the recording of the interference pattern is called a hologram. Various image sensors such as CCD may be used as such a digital image sensor.

The image storage unit 600 stores interference pattern information converted into discrete signals by the image sensor unit 500 in various storage media such as a memory or a disk device.

The control unit 700 implements the above-described off-axis system and adjusts the positions and angles of the second reflector 330 and the third reflector 350 in order to obtain the interference fringe. ) And controlling the objective part 400, such as adjusting the objective lens 420 to adjust the first and second object light incident on the object to be measured, and the interference pattern is measured The image sensor unit 500 is controlled to convert the information into a discrete signal, and the image storage unit 600 is controlled to store the interference fringe information converted into a discrete signal.

The object shape restoration unit 800 includes a phase information acquisition unit 810, a thickness information acquisition unit 820 and a shape restoration unit 830. The phase information acquisition unit 810 obtains phase information of the interference fringe for the first ray and phase information of the interference fringe for the second ray using the interference fringe information, respectively, and the thickness information acquisition unit 820 Acquires thickness information of the object to be measured using the phase information, and the shape restoration unit 830 restores a real-time 3D shape of the object to be measured using the thickness information. In this case, the thickness information of the object to be measured includes information on the difference between the paths of the object light and the reference light, respectively. The interference fringes are formed when the object light and the reference light overlap because of the difference in the optical path between the object light and the reference light.

According to the disclosed prior art including the above, it is possible to improve measurement resolution and secure real-time image acquisition. However, the following problems still arise.

In the prior art disclosed first, since a mixed light source having a wavelength band distributed in several bands is used, the wavelength dividing unit 200 divides the first and second light sources having different wavelengths to obtain at least two single wavelengths. To do this, the first filter plate 220, the second filter plate 230, and the first reflector 240 must be used.

In addition, the interference fringe acquisition unit 300 is a third light splitter 320 for dividing a second light source, a third reflector 350 for reflecting a second light source, and a second light source 330 A third filtering plate 340 to block incident on the vehicle should be additionally used.

Accordingly, the structure of the microscope is complicated, and this entails various problems such as an increase in manufacturing cost and an increase in design complexity. Therefore, a new method for solving the above-described problem is required while using a light source of a single wavelength.

Republic of Korea Patent Publication No. 10-2016-0029606 Republic of Korea Patent Publication No. 10-2010-0095302 Republic of Korea Patent Publication No. 10-2012-0014355 Korean Patent No. 10-1139178 Korean Patent No. 10-1441245 U.S. Patent No. 7,649,160

An embodiment of the present invention provides a method of determining a defect in a process using a restored hologram.

The method of determining a defect in a process using a restored hologram according to embodiments of the present invention is the intensity of an object hologram generated by interference between a reference light reflected from an optical mirror and an object light affected by the object to be measured. Identifying at least one frequency component included in an image including information; Extracting real image components corresponding to a real image from among the at least one frequency component; Generating a real hologram including real image information of the object to be measured and correction light having a conjugate relationship with the reference light based on the real image components; Generating an intermediate hologram from which information of the reference light is removed from the real hologram based on the correction light; Generating curvature aberration correction information from the intermediate hologram; Generating a correction hologram from which an error due to curvature aberration is removed from the intermediate hologram based on the curvature aberration correction information; Generating the 3D shape information of the object to be measured from the correction hologram; And determining a location of a defect included in the object to be measured and a process in which the defect occurs, using at least one of the real component, the real hologram, the intermediate hologram, the correction hologram, and the 3D shape information. It may include.

The step of determining the process in which the defect is generated may be present in the measurement target object using at least one of the real component, the real hologram, the intermediate hologram, the correction hologram, and the 3D shape information for the measurement target object. To determine the presence or absence of a defect to be measured, compare the 3D shape information of the measurement target object with pre-registered reference shape information to detect the location of the defect or the defect area of the measurement target object, and the defect of the measurement target object It may be characterized by inferring a defect in the measurement target object and a process in which the defect occurs by using information on the location or defect area of and a previously registered process.

The step of determining the process in which the defect is generated may be present in the measurement target object using at least one of the real component, the real hologram, the intermediate hologram, the correction hologram, and the 3D shape information for the measurement target object. To determine the presence or absence of a defect to be measured, compare the 3D shape information of the measurement target object with pre-registered reference shape information to detect the location of the defect or the defect area of the measurement target object, and the defect of the measurement target object Inferring the upper process in which the defect and the defect occurred in the object to be measured using information on the location or defect area of and the previously registered process, and the real component, the real hologram, and the intermediate hologram for the measurement target object , The correction hologram, and the 3D shape information may be used to infer a sub-process in which a defect has occurred.

According to the present invention, defects in these structures can be detected with a high probability by accurately obtaining a three-dimensional shape of an ultrafine structure such as a TFT and a semiconductor.

Further advantages of the present invention can be clearly understood from the following description with reference to the accompanying drawings in which the same or similar reference numerals indicate the same elements.

1 is a block diagram showing in detail a two-wavelength digital holographic microscope apparatus according to the disclosed prior art.
2A is a block diagram showing a schematic configuration of a holographic restoration apparatus according to a first embodiment of the present invention.
2B is a block diagram showing a schematic configuration of a holographic restoration apparatus according to a second embodiment of the present invention.
3A and 3B are diagrams for explaining an external shape of an exemplary measurement target object.
4 is an example of an image of a part of an object to be measured.
FIG. 5 is a diagram illustrating a frequency component of an image of a portion of the object to be measured shown in FIG. 4.
6 is a diagram for describing a method of extracting frequency components corresponding to an actual image from the frequency components shown in FIG. 5.
7A is a diagram showing the intensity of digital reference light.
7B is a diagram showing a phase of a reference light.
7C is a diagram showing the intensity of correction light.
7D is a diagram showing the phase of the correction light.
8 is a diagram showing an exemplary real hologram.
9 and 10 are diagrams for explaining a method of determining, by a processor, a curvature aberration correction term from an intermediate hologram according to an embodiment of the present invention.
11 is a diagram illustrating an example of a three-dimensional shape of an object to be measured generated from a hologram.
12 is a flowchart illustrating a method of generating 3D shape information of an object to be measured performed by a holographic restoration apparatus according to an embodiment of the present invention.
13 and 14 are block diagrams of a processor according to embodiments of the present invention.
15A and 15B are diagrams illustrating a design example of a hologram restoration device.
16, 17, and 18 are flowcharts of a defect detection method according to embodiments of the present invention.
19 is a diagram illustrating an embodiment of a machine learning unit and an image processing unit.
20 is a block diagram of a machine learning unit according to an embodiment.
21 is a diagram illustrating a connection relationship between a defect detection algorithm modeled through a machine learning unit.

Since the present invention can apply various transformations and have various embodiments, specific embodiments are illustrated in the drawings and will be described in detail in the detailed description. Effects and features of the present invention, and a method of achieving them will be apparent with reference to the embodiments described later in detail together with the drawings. However, the present invention is not limited to the embodiments disclosed below and may be implemented in various forms.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings, and when describing with reference to the drawings, the same or corresponding constituent elements are assigned the same reference numerals, and redundant descriptions thereof will be omitted. .

In the following embodiments, terms such as first and second are not used in a limiting meaning, but are used for the purpose of distinguishing one component from another component. In the following examples, the singular expression includes the plural expression unless the context clearly indicates otherwise. In the following embodiments, terms such as include or have means that the features or elements described in the specification are present, and do not preclude the possibility of adding one or more other features or elements in advance. In the drawings, components may be exaggerated or reduced in size for convenience of description. For example, the size and shape of each component shown in the drawings are arbitrarily shown for convenience of description, and thus the present invention is not necessarily limited to what is shown.

2A is a block diagram showing a schematic configuration of a holographic restoration apparatus 1A according to the first embodiment of the present invention.

In the present invention, the'holography restoration device' may mean a device that acquires a hologram of an object to be measured (hereinafter, it will be described as a'object hologram'), and analyzes and/or displays the acquired object hologram. .

For example, the holography restoration apparatus 1A may be a device disposed on a semiconductor manufacturing line to obtain an object hologram of a semiconductor to be produced, and determine whether the semiconductor is integrity from the obtained object hologram. However, this is merely an example and the spirit of the present invention is not limited thereto.

Meanwhile, in the present invention, the'object hologram' is a hologram that can be generated from an image obtained by the holographic restoration apparatus 1A, and may mean a hologram before various processes by the holographic restoration apparatus 1A are performed. have. A detailed description of this will be described later.

Referring to FIG. 2A, the holographic restoration apparatus 1A according to the first embodiment of the present invention includes a light source unit 10 for emitting single wavelength light, and a collimator 20 for collimating single wavelength light emitted from the light source unit 10. ), a light splitter 30 that divides the single wavelength light that has passed through the collimator 20 into object light (O) and reference light (R), and an object that passes object light (O) divided by the light splitter 30 The optical objective lens 40, the reference light objective lens 60 for passing the reference light R divided by the optical splitter 30, the optical mirror 70 for reflecting the reference light R passing through the reference light objective lens 60 ), the object light (O) reflected from the surface of the object to be measured 50 passing through the object light objective lens 40 and the reference light (R) reflected by the optical mirror 70, respectively, the object light objective lens 40 ) And an image sensor 80 for recording an image formed by passing through the reference light objective lens 60 and being transmitted to the optical splitter 30, and a processor 90 for processing the image obtained by the image sensor 80. I can.

In this case, the processor 90 may generate 3D information of the object to be measured 50 from the image acquired by the image sensor 80. A detailed description of the operation of the processor 90 will be described later.

2B is a block diagram showing a schematic configuration of a holographic restoration apparatus 1B according to a second embodiment of the present invention.

Referring to FIG. 2B, the holographic restoration apparatus 1B according to the second embodiment of the present invention includes a light source unit 10 for emitting single wavelength light, and a collimator 20 for collimating single wavelength light emitted from the light source unit 10. ), a light splitter 30 that divides the single wavelength light that has passed through the collimator 20 into object light (O) and reference light (R), and the object light (O) divided by the light splitter 30 is the object to be measured After passing through (50), the object light that passes through the object transmitted light (T) including the information of the object to be measured 50 is reflected, the object light objective lens 40, and the object transmitted light (T) that has passed through the object light objective lens 40 The second optical mirror 72 to allow, the reference light objective lens 60 for passing the reference light R divided by the light splitter 30, the first for reflecting the reference light R passing through the reference light objective lens 60 The optical mirror 70, the second optical splitter 32 to which the reference light R reflected by the first optical mirror 70 and the transmitted object light T reflected by the second optical mirror 72 are transmitted, respectively, An image sensor 80 that records an image formed by the reference light R and object light transmitted light T transmitted to the second optical splitter 32 and a processor 90 that processes the image acquired by the image sensor 80 ) Can be included.

Of course, even in this second embodiment, the processor 90 may generate 3D information of the object to be measured 50 from the image acquired by the image sensor 80. A detailed description of the operation of the processor 90 will be described later.

The object light O is measured in the holographic restoration apparatus 1A according to the first exemplary embodiment of the present invention and the holographic restoration apparatus 1B according to the second exemplary embodiment shown in FIGS. 2A and 2B, respectively. The fact that the object 50 reflects (the embodiment of Fig. 2A) or the object light O transmits the measurement object 50 (the embodiment of Fig. 2B) and some components accordingly (for example, The second optical mirror 72 and the second optical splitter 32 of the embodiment of FIG. 2B have substantially the same configuration except for the additional use and consequent arrangement of some components).

In particular, it should be noted that the image is acquired by the image sensor 80 and has the same characteristic in that the processor 90 generates the reference light R from the acquired image.

Hereinafter, the holographic restoration apparatuses 1A and 1B according to the first and second embodiments of the present invention will be collectively referred to as the holographic restoration apparatus 1.

Meanwhile, the processor 90 of the holographic restoration apparatus 1 according to an embodiment of the present invention may include all kinds of devices capable of processing data. For example, the processor 90 may mean a data processing device embedded in hardware having a circuit physically structured to perform a function represented by a code or instruction included in a program.

As an example of such a data processing device built into the hardware, a microprocessor, a central processing unit (CPU), a processor core, a multiprocessor, and an application-specific integrated (ASIC) Circuit) and processing devices such as a Field Programmable Gate Array (FPGA) may be covered, but the scope of the present invention is not limited thereto.

In addition, the image sensor 80 according to an embodiment of the present invention may be implemented with at least one image sensor such as a charge coupled device (CCD) and a Complimentary Metal-Oxide Semiconductor (CMOS).

3A and 3B are diagrams for explaining an external shape of an exemplary measurement target object 50.

As shown in FIGS. 3A and 3B, the object to be measured 50 may include structures 51A to 51I in a rectangular parallelepiped shape disposed on one surface at a predetermined interval. In other words, the measurement target object 50 may include rectangular parallelepiped-shaped structures 51A to 51I protruding in the Z direction on a plane parallel to the X-Y plane.

Hereinafter, the holographic restoration apparatus 1 irradiates the object light O in a direction perpendicular to the surface on which the rectangular parallelepiped structures 51A to 51I of the measurement target 50 are disposed to obtain an image of the measurement target object 50. It will be described on the premise of obtaining.

First, the image sensor 80 according to an embodiment of the present invention may acquire an image of the object 50 to be measured.

In the present invention, the'Image' of the object to be measured 50 is information on the intensity at each position of the hologram (U0(x,y,0)) of the object to be measured 50 (ie | (U0(x,y,0)| ² ) may be included, and may be expressed as Equation 1 below.

[Equation 1]

Here, the object hologram Uo(x,y,0) represents the phase information at each x,y point of the object to be measured, and x, y are coordinates in the space where the object to be measured is placed, perpendicular to the object light (O). Represents the coordinates defining the plane, O(x,y) and R(x,y) represent object light (O) and reference light (R), respectively, and O*(x,y) and R*(x, y) denotes the complex conjugate of the object light O and the reference light R, respectively.

For example, the image sensor 80 may acquire an image as shown in FIG. 4 for a portion of the object to be measured 50 shown in FIGS. 3A and 3B (eg, a portion including 51A and 51B). I can.

Since the image acquired by the image sensor 80 includes intensity information at each position of the object hologram U0(x,y,0) as described above, the image sensor 80 It may be different from the image of the object to be measured 50 (that is, taken only with the object light O).

Referring to Equation 1, the object hologram (U0(x,y,0)) does not include the object light (0) including the phase information of the object to be measured 50 at each point and the phase information of the object to be measured. It may be generated by interference of the non-referred reference light R.

In addition, the object hologram (U0(x,y,0)) is in addition to the phase information (that is, height information of the object) at each point (that is, each x,y point) of the object to be measured 50, the object light objective lens 40 ) And noise (eg, speckle noise due to the use of photons of the laser), etc. may be further included.

Accordingly, the processor 90 according to an embodiment of the present invention may perform various calculation processes as described below in order to remove the above-described error and noise from the image acquired by the image sensor 80.

The processor 90 according to an embodiment of the present invention may check the frequency components of the image acquired by the image sensor 80. For example, the processor 90 may perform 2D Fourier Transform on the image to check the frequency components of the image.

In other words, the processor 90 stores the frequency components included in the image including the intensity information for each location of the object hologram (U0(x,y,0)) (that is, |(U0(x,y,0)| ² )). In this case, the image may include a frequency component corresponding to a real image, a frequency component and a DC component corresponding to an immaginary image.

Of course, the image may further include various components in addition to the above-described three components (a frequency component corresponding to an actual image, a frequency component corresponding to a virtual image, and a DC component). For example, the image may further include frequency components due to noise. However, this is merely an example and the spirit of the present invention is not limited thereto.

The processor 90 according to an embodiment of the present invention may extract only components corresponding to the actual among the identified frequency components. In this case, the processor 90 may extract components corresponding to the actual image in various ways.

For example, the processor 90 extracts components having a peak value (hereinafter, peak components) from among frequency components included in the image, and includes a peak component corresponding to the actual image among the extracted peak components. Components within the frequency difference of firing can be extracted as components corresponding to the actual reality.

In this case, the processor 90 may determine the components corresponding to the actual image in various ways based on the peak component corresponding to the actual image. For example, the processor 90 may determine frequency components in the cross region centered on the peak component corresponding to the actual image as components corresponding to the actual image. However, this is merely an example and the spirit of the present invention is not limited thereto.

In an optional embodiment, the processor 90 may extract only components corresponding to a real image from among frequency components included in the hologram using an automatic real image spot-position extraction algorithm.

In the present invention,'extracting' a specific frequency component may mean extracting the frequency of the corresponding frequency component and the magnitude (or intensity) of the corresponding frequency component.

FIG. 5 is a diagram illustrating a frequency component of an image for a portion of the object to be measured 50 illustrated in FIG. 4.

As described above, the processor 90 can check the frequency components of the image acquired by the image sensor 80, and accordingly, the processor 90 is a frequency component 911 corresponding to a real image and a frequency component corresponding to a virtual image. Various frequency components including 912 and DC component 913 can be identified.

In addition, the processor 90 may extract only the frequency component 911 corresponding to the actual image among the identified components. In this case, the processor 90 may determine, for example, frequency components 911B in the cross region centered on the peak component 911A corresponding to the actual image as components corresponding to the actual image as shown in FIG. 6.

The processor 90 according to an embodiment of the present invention may generate digital reference light from frequency components corresponding to the actual image extracted by the above-described process. Looking at this in more detail, the processor 90 may calculate the propagation direction and wave number of the digital reference light based on frequency components corresponding to the actual image. In other words, the processor 90 can calculate the wave number vector of the digital reference light.

In addition, the processor 90 generates a digital reference light based on the propagation direction and wave number (or wave number vector) of the digital reference light, and calculates the conjugate term of the digital reference light R(x,y) generated as in Equation 2 below. By obtaining, correction light Rc(x,y) can be generated.

[Equation 2]

Rc(x,y)= conj[R(x,y)]

In this case, R(x,y) denotes digital reference light generated based on frequency components corresponding to the actual image, and Rc(x,y) denotes a correction light.

Since the digital reference light (R(x,y)) and the correction light (Rc(x,y)) are in a conjugate relationship, the intensity is the same as shown in FIGS. 7A and 7C, and as shown in FIGS. 7B and 7D. Likewise, the phase can be reversed. Here, FIG. 7A is a diagram showing the intensity of the digital reference light R(x,y), FIG. 7B is a diagram showing the phase of the reference light, and FIG. 7C shows the intensity of the correction light Rc(x,y). Fig. 7D is a diagram showing the phase of the correction light.

The generated correction light Rc(x,y) may be used for correction of a real hologram Um(x,y,0) to be described later.

Meanwhile, the'digital reference light' is light having the same properties as the reference light R generated by the above-described light splitter 30 from light of a single wavelength, and the processor 90 is restored from the image acquired by the image sensor 80 It could be a virtual light.

The processor 90 according to an embodiment of the present invention may generate a real hologram based on frequency components corresponding to the real image extracted by the above-described process. For example, the processor 90 may generate a real hologram as shown in FIG. 8 by performing an inverse 2D Fourier transform on frequency components corresponding to the real image.

In this case, the hologram can be represented by Equation 3 below.

[Equation 3]

Um(x,y,0)=O(x,y)R*(x,y)

Here, Um(x,y,0) represents a real hologram, O(x,y) represents an object light (O), and R*(x,y) represents a complex conjugate of the reference light (R).

Meanwhile, the real hologram Um(x,y,0) includes information on the reference light R and errors due to aberration of the object light objective lens 40 in addition to information on the height of the object to be measured 50. Can include.

Accordingly, the processor 90 according to an embodiment of the present invention takes into account the effect of the reference light R and the error due to the aberration of the object light objective lens 40 from the actual hologram Um(x,y,0). It is possible to generate a correction hologram Uc(x,y,0).

For example, as shown in Equation 4 below, the processor 90 has a term for corrected light (Rc(x,y)) and a term for curvature aberration correction (Rca(x)) in a real hologram (Um(x,y,0)). The correction hologram Uc(x,y,0)) can be generated by multiplying by ,y)).

[Equation 4]

Uc(x,y,0)=Um(x,y,0)Rc(x,y)Rca(x,y)

Here, Uc(x,y,0) denotes a correction hologram from which information on the reference light R and aberration information of the object light objective lens 40 are removed, and Um(x,y,0) denotes a real hologram, Rc(x,y) denotes a term for correction light, and Rca(x,y) denotes a term for curvature aberration correction.

Meanwhile, the processor 90 according to an embodiment of the present invention may generate a term (Rca(x,y)) for the above-described curvature aberration correction in various ways.

For example, the processor 90 is actually the hologram (Um(x,y,0)) multiplied by only the term (Rc(x,y)) for the corrected light (hereinafter, referred to as the intermediate hologram). A dimensional shape may be generated, and a term for curvature aberration correction (Rca(x,y)) may be generated from the generated 3D shape.

Looking at this in more detail, the processor 90 may determine at least one parameter for determining the curvature aberration correction term from the three-dimensional shape of the measurement target object 50 generated from the intermediate hologram. In this case, the parameter may include, for example, a coordinate and a radius of a center point defining a hemispherical curved surface.

9 and 10 are diagrams for explaining a method of determining, by the processor 90, a curvature aberration correction term from an intermediate hologram according to an embodiment of the present invention.

For convenience of explanation, it is assumed that the image sensor 80 acquires an image of the rectangular parallelepiped-shaped structure 51D of FIG. 3B, and that the processor 90 generates an intermediate hologram of the structure 51D according to the above-described process. do. In addition, it is assumed that the three-dimensional shape 920 of the structure 51D generated from the intermediate hologram of the structure 51D is as shown in FIG. 9.

Under the above assumption, the processor 90 according to an embodiment of the present invention may determine at least one parameter for determining the curvature aberration correction term from the 3D shape 920. For example, the processor 90 may determine the coordinates (Cx, Cy) of the center point of the hemispherical curved surface and the radius (r) of the curved surface from the curve on the II section of the three-dimensional shape 920 as shown in FIG. 10 as parameters. . At this time, the processor 90 according to an embodiment of the present invention may determine the position and/or direction of the cut surface such that the cut surface such as the I-I cross-section includes the center point of the 3D shape 920 (ie, the center point of the hemispherical shape). In addition, the processor 90 may determine that a cut surface such as an I-I cross-section is parallel to the traveling direction of the object light 0.

The processor 90 according to an embodiment of the present invention may generate (or determine) a curvature aberration correction term based on at least one parameter determined by the above-described process. For example, the processor 90 generates a curved surface in 3D space by referring to the coordinates (Cx, Cy) of the center point of the curved surface and the radius (r) of the curved surface, and reflected in the phase correction of each x,y point from the generated curved surface. The curvature aberration correction term can be generated (or determined) by generating information.

In an alternative embodiment, the processor 90 may determine the correction term from an intermediate hologram of an object to be measured (eg, an object having the same z value in all x,y coordinates) whose shape is known in advance.

In the case of a measurement object whose shape is known in advance, since the z value at each x and y point is known in advance, the processor 90 determines the three-dimensional shape of the measurement object generated from the intermediate hologram and the known shape of the measurement object The correction term can also be determined by checking the difference between the z values at each x and y point. However, this is merely an example and the spirit of the present invention is not limited thereto.

The processor 90 according to an embodiment of the present invention may generate a three-dimensional shape of the measurement target object 50 based on the correction hologram Uc(x,y,0). In other words, the processor 90 may calculate the height of the object at each x and y point in the z direction.

For example, the processor 90 may convert the correction hologram Uc(x,y,0) into information on the reconstructed image plane. In this case, the reconstructed image plane means a virtual image display plane corresponding to a distance corresponding to the distance between the object to be measured and the image sensor by the processor, and may be a virtual plane calculated and simulated by the processor 90.

The processor 90 may calculate the height of the object at the x and y points in the z direction from the reconstructed information in consideration of the reconstructed image plane. In FIG. 11, the three-dimensional shape of the two rectangular parallelepiped-shaped structures 51A and 51B disposed on the object 50 to be measured is illustrated as an example.

12 is a flowchart illustrating a method of generating 3D shape information of an object to be measured 50 performed by the holographic restoration apparatus 1 according to an embodiment of the present invention. Hereinafter, descriptions of contents overlapping with those described in FIGS. 1 to 11 will be omitted, but will be described with reference to FIGS. 1 to 11.

The holographic restoration apparatus 1 according to an embodiment of the present invention may acquire an image of the object to be measured 50 (S1201).

In the present invention, the'Image' of the object to be measured 50 is information on the intensity at each position of the hologram (U0(x,y,0)) of the object to be measured 50 (ie | (U0(x,y,0)| ² ) may be included, and may be expressed as in Equation 1 above.

For example, the holographic restoration apparatus 1 acquires an image as shown in FIG. 4 for a part of the object to be measured 50 shown in FIGS. 3A and 3B (eg, a part including 51A and 51B). can do.

Since the image acquired by the holographic restoration apparatus 1 includes intensity information at each position of the object hologram U0(x,y,0) as described above, the holographic restoration apparatus 1 is acquired. It may be different from the image of the object 50 to be measured in one general (that is, taken only with the object light O).

Referring to Equation 1, the object hologram (U0(x,y,0)) does not include the object light (0) including the phase information of the object to be measured 50 at each point and the phase information of the object to be measured. It may be generated by interference of the non-referred reference light R.

In addition, the object hologram (U0(x,y,0)) is in addition to the phase information (that is, height information of the object) at each point (that is, each x,y point) of the object to be measured 50, the object light objective lens 40 ) And noise (eg, speckle noise due to the use of photons of the laser), etc. may be further included.

Accordingly, the holographic restoration apparatus 1 according to an embodiment of the present invention may perform the calculation process of steps S1202 to S1207 to remove the above-described error and noise from the image obtained by the holographic restoration apparatus 1. .

The holography restoration apparatus 1 according to an embodiment of the present invention may check the frequency components of the image acquired by the holography restoration apparatus 1 (S1202). For example, the holography restoration apparatus 1 By performing a Fourier Transform (2D Fourier Transform), it is possible to check the frequency components of the image.

In other words, the holographic restoration apparatus 1 includes the intensity information for each location of the object hologram (U0(x,y,0)) (ie |(U0(x,y,0)| ² )). In this case, the image may include a frequency component corresponding to a real image, a frequency component corresponding to an immaginary image, and a DC component.

Of course, the image may further include various components in addition to the above-described three components (a frequency component corresponding to an actual image, a frequency component corresponding to a virtual image, and a DC component). For example, the image may further include frequency components due to noise. However, this is merely an example and the spirit of the present invention is not limited thereto.

The holographic restoration apparatus 1 according to an embodiment of the present invention may extract only the components corresponding to the real image among the identified frequency components. (S1203) At this time, the holographic restoration apparatus 1 Components can be extracted.

For example, the holographic restoration apparatus 1 extracts components (hereinafter referred to as peak components) having a peak value among frequency components included in an image, and peaks corresponding to the actual image among the extracted peak components. Components within the frequency difference between the component and the firing can be extracted as components corresponding to the actual reality.

At this time, the holographic restoration apparatus 1 may determine components corresponding to the actual image in various ways based on the peak component corresponding to the actual image. For example, the holographic restoration apparatus 1 may determine frequency components in a cross region centered on a peak component corresponding to the actual image as components corresponding to the actual image. However, this is merely an example and the spirit of the present invention is not limited thereto.

In an optional embodiment, the holographic reconstructing apparatus 1 may extract only components corresponding to a real image from among frequency components included in a hologram using an automatic real image spot-position extraction algorithm.

In the present invention,'extracting' a specific frequency component may mean extracting the frequency of the corresponding frequency component and the magnitude (or intensity) of the corresponding frequency component.

Referring back to FIG. 5, the holography restoration apparatus 1 can check the frequency components of the image acquired by the holography restoration apparatus 1, and accordingly, the holography restoration apparatus 1 ), various frequency components including a frequency component 912 and a DC component 913 corresponding to the virtual image can be identified.

Also, the holographic restoration apparatus 1 may extract only the frequency component 911 corresponding to the actual image among the identified components. In this case, the holographic restoration apparatus 1 may determine frequency components 911B in the cross region centered on the peak component 911A corresponding to the actual image as components corresponding to the actual image, as illustrated in FIG. 6.

The holographic restoration apparatus 1 according to an embodiment of the present invention may generate a digital reference light from frequency components corresponding to the actual image extracted by the above-described process (S1204). [0048] Looking in more detail, the holographic restoration apparatus ( 1) can calculate the propagation direction and wave number of the digital reference light based on frequency components corresponding to the actual image. In other words, the holographic restoration apparatus 1 can calculate the wave number vector of the digital reference light.

Further, the holographic restoration apparatus 1 generates a digital reference light based on the propagation direction and wave number (or wave number vector) of the digital reference light, and the digital reference light R(x,y) generated as in Equation 2 Correction light (Rc(x,y)) can be generated by finding the conjugate term.

Since the digital reference light (R(x,y)) and the correction light (Rc(x,y)) are in a conjugate relationship, the intensity is the same as shown in FIGS. 7A and 7C, and as shown in FIGS. 7B and 7D. Likewise, the phase can be reversed. Here, FIG. 7A is a diagram showing the intensity of the digital reference light R(x,y), FIG. 7B is a diagram showing the phase of the reference light, and FIG. 7C shows the intensity of the correction light Rc(x,y). Fig. 7D is a diagram showing the phase of the correction light.

The generated correction light Rc(x,y) may be used for correction of a real hologram Um(x,y,0) to be described later.

On the other hand,'digital reference light' is light having the same properties as the reference light R generated by the above-described light splitter 30 from light of a single wavelength, and the holography restoration apparatus 1 is obtained by the holography restoration apparatus 1. It may be a virtual light restored from an image.

The holographic restoration apparatus 1 according to an embodiment of the present invention may also generate a real hologram based on frequency components corresponding to the real image extracted by the above-described process (S1204). For example, the holographic restoration apparatus 1 8 may generate a real hologram as shown in FIG. 8 by performing an inverse 2D Fourier transform on frequency components corresponding to the real image. In this case, the hologram can be represented as in Equation 3 above.

The holographic restoration apparatus 1 according to an embodiment of the present invention may generate an intermediate hologram to generate a term (Rca(x,y)) for curvature aberration correction (S1205). For example, the holographic restoration apparatus 1 ) Can generate an intermediate hologram by multiplying the actual hologram Um(x,y,0) by the term Rc(x,y) for the correction light. The generated intermediate hologram may be used to generate curvature aberration correction information in step S1206.

The holographic restoration apparatus 1 according to an embodiment of the present invention generates a three-dimensional shape of the object to be measured 50 from the intermediate hologram generated in step S1205, and a term for curvature aberration correction from the generated three-dimensional shape ( Rca(x,y)) can be generated. (S1206) Looking more closely at this, the holographic restoration apparatus 1 determines a curvature aberration correction term from the three-dimensional shape of the object 50 generated from the intermediate hologram. It is possible to determine at least one parameter. In this case, the parameter may include, for example, a coordinate and a radius of a center point defining a hemispherical curved surface.

Referring again to FIGS. 9 and 10, a method of determining a curvature aberration correction term from an intermediate hologram by the holographic restoration apparatus 1 according to an embodiment of the present invention will be described. For convenience of explanation, the holographic restoration apparatus 1 obtained an image of the rectangular parallelepiped structure 51D of FIG. 3B, and the holographic restoration apparatus 1 generates an intermediate hologram of the structure 51D according to the above-described process. Is assumed to have been created. In addition, it is assumed that the three-dimensional shape 920 of the structure 51D generated from the intermediate hologram of the structure 51D is as shown in FIG. 9.

Under the above-described assumption, the holographic restoration apparatus 1 according to an exemplary embodiment of the present invention may determine at least one parameter for determining a curvature aberration correction term from the three-dimensional shape 920. For example, the holographic restoration apparatus 1 determines the coordinates (Cx, Cy) of the center point of the hemispherical curved surface and the radius (r) of the curved surface from the curve on the II section of the three-dimensional shape 920 as shown in FIG. 10 as parameters. I can. At this time, the holographic restoration apparatus 1 according to an embodiment of the present invention may determine the position and/or direction of the cut surface such that the cut surface such as the II cross-section includes the center point of the three-dimensional shape 920 (ie, the center point of the hemispherical shape). . Further, the holographic restoration apparatus 1 may determine such that a cut surface such as an I-I cross-section is parallel to the traveling direction of the object light 0.

The holographic restoration apparatus 1 according to an embodiment of the present invention may generate (or determine) a curvature aberration correction term based on at least one parameter determined by the above-described process. For example, the holographic restoration apparatus 1 creates a curved surface in a three-dimensional space by referring to the coordinates (Cx, Cy) of the center point of the curved surface and the radius (r) of the curved surface, and performs phase correction of each x,y point from the generated curved surface. A curvature aberration correction term may be generated (or determined) in a manner of generating information to be reflected.

In an alternative embodiment, the holographic restoration apparatus 1 may determine a correction term from an intermediate hologram of an object to be measured (for example, an object having the same z value in all x and y coordinates) whose shape is known in advance.

In the case of an object to be measured whose shape is known in advance, since the z value at each x and y point is known in advance, the holographic restoration device (1) is the three-dimensional shape of the object to be measured and the object to be measured. The correction term can also be determined by checking the difference between the z values at each x and y point of the shape. However, this is merely an example and the spirit of the present invention is not limited thereto.

The holographic restoration apparatus 1 according to an embodiment of the present invention is a real hologram (Um(x,y,0)) in consideration of the effect of the reference light R and the error due to the aberration of the object light objective lens 40. The correction hologram Uc(x,y,0) can be generated from (S1207). For example, the holographic restoration apparatus 1 is actually a hologram Um(x,y,0) as in Equation 4 above. The correction hologram Uc(x,y,0) can be generated by multiplying the term for correction light (Rc(x,y)) and the term for curvature aberration correction (Rca(x,y)). At this time, the term (Rc(x,y)) for the correction light may be generated in step S1204, and the term for curvature aberration correction (Rca(x,y)) may be generated in step S1206.

The holographic restoration apparatus 1 according to an embodiment of the present invention may generate 3D shape information of the object to be measured 50 based on the correction hologram Uc(x,y,0) (S1208). In other words, the holographic restoration apparatus 1 can calculate the height of the object at each x and y point in the z direction.

For example, the holographic restoration apparatus 1 may convert the correction hologram Uc(x,y,0) into information on the reconstructed image plane. In this case, the reconstructed image plane refers to a virtual image display plane corresponding to a distance corresponding to the distance between the object to be measured and the image sensor by the processor, and may be a virtual plane calculated and simulated by the holographic restoration apparatus 1. .

The holographic restoration apparatus 1 may calculate the height in the z direction of the object at x and y points as shown in FIG. 11 from the restored information in consideration of the reconstructed image plane. In FIG. 11, the three-dimensional shape of the two rectangular parallelepiped-shaped structures 51A and 51B disposed on the object 50 to be measured is illustrated as an example.

13 is a block diagram of a processor 90 according to embodiments of the present invention. 13 and 14 show the structure of the processor 90 performing a function of detecting a defect using a hologram. 13, the processor 90 of the hologram restoration apparatus 1 may further include a defect detection unit 97 for detecting defects.

The processor 90 includes an image acquisition unit 91, a real information extraction unit 92, a real hologram generation unit 93, an intermediate hologram generation unit 94, a correction hologram generation unit 95, and a 3D shape information generation unit. (96), it may include a defect detection unit (97).

The defect detection unit 97 uses at least one of an image acquired through light irradiated onto the target object, real image information obtained from the image, real image hologram, intermediate hologram, correction hologram, and 3D shape information. Can judge presence or absence.

The defect detection unit 97 detects the presence or absence of a defect included in the 3D shape information of the target object, the location of the defect, and a defect area by comparing the 3D shape information of the target object with the reference shape information. The reference shape information is set for each target object, and may be obtained through light irradiated onto the target object in a defect-free state. The reference shape information may be a 3D image restored through irradiated light, a 2D image itself, or a set of parameters from the image.

The defect detection unit 97 first determines whether or not a defect exists by comparing the 3D shape information of the target object with the reference shape information, and based on the 3D shape information of the defective target object, the defect location and defect area It is possible to detect detailed defect information such as.

The defect detection unit 97 may infer a defect in the measurement object and a process in which the defect occurs by using information on the location or defect region of the measurement object and information on a previously registered process. The information on the process may include a function for each process of manufacturing a target object, and location information generated or changed by each process. Here, the process refers to a general circuit board manufacturing process, and may include diffusion, photo, etching, deposition, ion implantation, polishing, rear polishing, wafer cutting, chip bonding, mold, printing, plating, solder ball attachment, and testing. .

Specifically, the defect detection unit 97 determines that a defect has occurred in at least one of mask fabrication, wafer processing, and chip assembly in consideration of the location of the defect, and additional input data such as real information, real hologram, Determining the process in which the defect occurs among detailed processes such as diffusion, photo, etching, deposition, ion implantation, and polishing, which are detailed processes included in wafer processing, by further considering at least one of intermediate hologram, correction hologram, and 3D shape information. I can. As described above, the defect detection unit 97 may determine a process in an upper range in which a defect occurs, and determine a process in a range in which a lower defect occurs by further considering input data. The defect detection unit 97 may determine a process in which a defect occurs in an upper range process by considering only a part of the input data, and may specifically determine a detailed process in which a defect occurs by considering all of the input data.

14 is a diagram illustrating a structure and operation of a processor 90 that detects a defect using the machine learning modeling apparatus 200 according to embodiments of the present invention.

As shown in FIG. 14, the hologram restoration apparatus 1 for detecting a defect may further include a machine learning modeling apparatus 200.

The processor 90 for detecting defects may transmit at least one of real image information, real hologram, intermediate hologram, correction hologram, 3D shape information, and defect information acquired from an image to the modeling unit 200.

The modeling unit 200 may include a machine learning unit 210 that performs machine learning and an image processing unit 220 that processes images for machine learning.

The machine learning unit 210 may generate an algorithm between the input data by repeatedly inputting the input data. In addition, the machine learning unit 210 may generate an algorithm between real information and defect information, real hologram and defect information, intermediate hologram and defect information, correction hologram and defect information, 3D shape information and defect information, respectively. For example, using an algorithm, the type of input data having a high correlation with the presence or absence of a defect can be specified as one of an image, real information, real hologram, intermediate hologram, correction hologram, and 3D shape information.

The image processing unit 220 may analyze an image using a preset filter. For example, one or more pixels included in an image may be divided into n feature maps using a 3X3 filter. The machine learning unit 210 may generate an algorithm for defect information based on images analyzed by the image processing unit 220.

The defect detection algorithm may be updated through the modeling unit 200 including the machine learning unit 210 and the image processing unit 220. The modeling unit 200 may transmit the updated defect detection algorithm to the processor 90 so that the defect is detected more accurately and quickly.

15A and 15B are diagrams for explaining a design example of the hologram restoration device 1.

As shown in FIG. 15A, one hologram restoration device 1 may be installed to face one surface of a target object including a defect. The hologram restoration apparatus 1 may detect a defect in the corresponding part by irradiating light on a part of one surface of the target object. The hologram restoration apparatus 1 can detect defects in the entire target object by dividing the area of the target object into a plurality of parts (N) and repeating the process of irradiating light onto the divided area N times. have.

As shown in FIG. 15B, K hologram restoration devices 1 may be installed to face one surface of the target object. The K hologram restoration devices 1 are arranged in a row to irradiate light to the row of the target object at a time to detect the defect of the target object. The K hologram restoration apparatuses 1 may detect defects present in the entire target object by irradiating light to L columns of the target object.

16 is a flowchart of a defect detection method according to embodiments of the present invention.

As shown in FIG. 16, in S1610, the hologram restoration apparatus 1 irradiates light onto a region of the target substrate. The K hologram restoration devices 1 may irradiate light to K areas. In S1620, the hologram restoration apparatus 1 senses an image including intensity information of an object hologram generated by interference between the reference light reflected from the optical mirror and the object light affected by the target substrate. In S1630, the hologram restoration apparatus 1 analyzes the image to obtain real information, real hologram, intermediate hologram, correction hologram, and 3D shape information. In S1640, the hologram restoration apparatus 1 detects defect information including the type of defect included in the target substrate, the location of the defect, and a process related to the defect based on the 3D shape information. The hologram restoration apparatus 1 may detect a defect location on a target substrate based on 3D shape information, and determine a type of defect and a process related to the defect using the defect location. The hologram restoration apparatus 1 may store and manage the LUT further including detailed defect information for each defect location.

Through this, the hologram restoration apparatus 1 may detect a defect present in the target object by using the hologram of the target object manufactured through one or more automated processes. The hologram restoration apparatus 1 may detect defect information existing in the target object by comparing the 3D shape information of the completed target object with the previously registered 3D shape information.

As shown in FIG. 17, after step S1640, the hologram restoration apparatus 1 transmits real image information, real hologram, intermediate hologram, correction hologram, 3D shape information and defect information obtained from the image to the machine learning unit It is possible to learn the detection algorithm (S1650).

18, in S1810, the hologram restoration apparatus 1 may receive an updated defect detection algorithm.

In S1820, the hologram restoration device 1 irradiates light onto an area of the target substrate.

In S1830, the hologram restoration apparatus 1 may sense an image including intensity information of the object hologram generated by interference between the reference light reflected from the optical mirror and the object light affected by the target substrate.

In S1840, the hologram restoration apparatus 1 may analyze an image and selectively acquire input data selected as meaningful data by a defect detection algorithm.

In S1850, the hologram restoration apparatus 1 may detect defect information including the type of defect included in the target substrate, the location of the defect, and a process related to the defect based on the input data.

19 is a diagram illustrating an embodiment of the machine learning unit 210 and the image processing unit 220.

In S1910, the machine learning modeling apparatus 200 may acquire input data. The machine learning modeling apparatus 200 may perform a convolution operation after clustering the input data. Through clustering, the processing time of input data can be shortened. In addition to clustering, processing time can be shortened through compression of input data. In S1920, the machine learning modeling apparatus 200 may determine a filter for performing a convolution operation on input data in one of the plurality of convolution layers. In S1930, the machine learning modeling apparatus 200 may determine a plurality of sub-filters corresponding to different filtering regions in the filter. In S1940, the machine learning modeling apparatus 200 may generate a plurality of feature maps based on a plurality of sub-filters. In S1950, the machine learning modeling apparatus 200 may acquire output data based on a plurality of feature maps. The machine learning modeling apparatus 200 may cluster a plurality of feature maps or output data. Through clustering, the processing time of the feature map or output data can be shortened. Through clustering, the processing time of the feature map or output data can be shortened. In addition to clustering, processing time can be shortened by compressing feature maps or output data.

For example, K-Means Clustering, Mean-Shift Clustering, Density-Based Spatial Clustering of Applications with Noise (DBSCAN), Expectation-Maximization (EM) Clustering using Gaussian Mixture Models (GMM), Agglomerative Hierarchical Clustering, etc. There is this. In addition to clustering, performance can be improved through various algorithms.

20 is a block diagram of a machine learning unit 210 according to an embodiment.

Referring to FIG. 20, a data learning unit 210 according to an embodiment includes a data acquisition unit 211, a preprocessor 212, a training data selection unit 213, a model learning unit 214, and a model evaluation unit ( 215) may be included.

The data acquisition unit 211 may acquire data necessary to acquire a recognition result. The data acquisition unit 211 may acquire data necessary for learning to acquire a recognition result.

For example, the data acquisition unit 211 included in the data learning unit 210 for learning an input image or input hologram and output data (defect information) determined according to the input image or input hologram is a hologram restoration device 1 An input image or an input hologram may be input from.

The preprocessor 212 may preprocess the acquired data so that the acquired data can be used for learning to determine a defect. The preprocessor 212 may process the acquired data into a preset format so that the model learning unit 214, which will be described later, can use the acquired data for learning to determine a defect.

The learning data selection unit 213 may select data necessary for learning from among the preprocessed data. The selected data may be provided to the model learning unit 214. The learning data selection unit 213 may select data necessary for learning from among pre-processed data according to a preset criterion for determining a defect. In addition, the learning data selection unit 213 may select data according to a preset criterion by learning by the model learning unit 214 to be described later.

The model learning unit 214 may learn a criterion for how to determine a defect based on the training data. In addition, the model learning unit 214 may learn a criterion for which training data to be used to determine a defect.

In addition, the model learning unit 214 may train a data recognition model used to determine a defect using the training data. In this case, the data recognition model may be a pre-built model. For example, the data recognition model may be a model built in advance by receiving basic training data (eg, a sample image, a sample hologram, etc.).

The data recognition model may be constructed in consideration of the application field of the recognition model, the purpose of learning, or the computer performance of the device. The data recognition model may be, for example, a model based on a neural network. For example, a model such as a deep neural network (DNN), a recurrent neural network (RNN), a bidirectional recurrent deep neural network (BRDNN), and a convolutional neural network (CNN) may be used as a data recognition model, but is not limited thereto.

According to various embodiments, when there are a plurality of pre-built data recognition models, the model learning unit 214 may determine a data recognition model having a high correlation between input training data and basic training data as a data recognition model to be trained. have. In this case, the basic training data may be pre-classified by data type, and the data recognition model may be pre-built for each data type.

In addition, the model learning unit 214 may train the data recognition model using, for example, a learning algorithm including error back-propagation or gradient descent.

In addition, the model learning unit 214 may train the data recognition model through supervised learning using, for example, training data as an input value. In addition, the model learning unit 214, for example, by self-learning the type of data necessary for determining a defect without any special guidance, through unsupervised learning to discover a criterion for determining a defect, data recognition You can train the model. In addition, the model learning unit 214 may train the data recognition model through reinforcement learning using feedback on whether a result of determining a defect according to training is correct.

In addition, when the data recognition model is trained, the model learning unit 214 may store the learned data recognition model. In this case, the model learning unit 214 may store the learned data recognition model in the memory of the hologram reconstruction apparatus 1. Alternatively, the model learning unit 214 may store the learned data recognition model in a memory of the hologram restoration apparatus 1 to be described later. Alternatively, the model learning unit 214 may store the learned data recognition model in a memory of a server connected to the electronic device through a wired or wireless network.

In this case, the memory in which the learned data recognition model is stored may also store commands or data related to at least one other component of the electronic device together. In addition, the memory may store software and/or programs. The program may include, for example, a kernel, middleware, an application programming interface (API), and/or an application program (or “application”).

The model evaluation unit 215 may input evaluation data to the data recognition model, and when a recognition result output from the evaluation data does not satisfy a predetermined criterion, the model learning unit 214 may retrain. In this case, the evaluation data may be preset data for evaluating the data recognition model.

For example, the model evaluation unit 215 does not satisfy a predetermined criterion when the number or ratio of evaluation data whose recognition result is not accurate among the recognition results of the learned data recognition model for the evaluation data exceeds a preset threshold. It can be evaluated as not. For example, when a predetermined criterion is defined as a ratio of 2%, when the learned data recognition model outputs an incorrect recognition result for more than 20 evaluation data out of a total of 1000 evaluation data, the model evaluation unit 215 learns It can be evaluated that the data recognition model is not suitable.

On the other hand, when there are a plurality of learned data recognition models, the model evaluation unit 215 evaluates whether each of the learned data recognition models satisfies a predetermined criterion, and determines a model that satisfies the predetermined criterion as a final data recognition model. You can decide. In this case, when there are a plurality of models that satisfy a predetermined criterion, the model evaluation unit 215 may determine any one or a predetermined number of models previously set in the order of the highest evaluation scores as the final data recognition model.

Meanwhile, at least one of the data acquisition unit 211, the preprocessor 212, the training data selection unit 213, the model learning unit 214 and the model evaluation unit 215 in the machine learning unit 210 is at least one It can be manufactured in the form of a hardware chip and mounted on an electronic device. For example, at least one of the data acquisition unit 211, the preprocessing unit 212, the training data selection unit 213, the model learning unit 214, and the model evaluation unit 215 is artificial intelligence (AI). It may be manufactured in the form of a dedicated hardware chip, or may be manufactured as a part of an existing general-purpose processor (eg, a CPU or application processor) or a graphics dedicated processor (eg, a GPU) and mounted on the aforementioned various electronic devices.

In addition, the data acquisition unit 211, the preprocessing unit 212, the training data selection unit 213, the model learning unit 214 and the model evaluation unit 215 may be mounted on one electronic device, or separate Each of the electronic devices may be mounted. For example, some of the data acquisition unit 211, the preprocessing unit 212, the training data selection unit 213, the model learning unit 214, and the model evaluation unit 215 are included in the electronic device, and some Can be included in the server.

In addition, at least one of the data acquisition unit 211, the preprocessor 212, the training data selection unit 213, the model learning unit 214, and the model evaluation unit 215 may be implemented as a software module. At least one of the data acquisition unit 211, the preprocessor 212, the training data selection unit 213, the model training unit 214, and the model evaluation unit 215 includes a software module (or instruction) When implemented as a program module), the software module may be stored in a computer-readable non-transitory computer readable media. In addition, in this case, at least one software module may be provided by an operating system (OS) or a predetermined application. Alternatively, some of the at least one software module may be provided by an operating system (OS), and the remaining part may be provided by a predetermined application.

21 is a diagram illustrating a connection relationship between a defect detection algorithm modeled through the machine learning unit 210.

The location information D2 of the defect may be determined using the attribute information D1 obtained from the image and holograms as input data, and the type D3 of the defect may be determined based on the location information of the defect.

The embodiment according to the present invention described above may be implemented in the form of a computer program that can be executed through various components on a computer, and such a computer program may be recorded in a computer-readable medium. In this case, the medium may store a program executable by a computer. Examples of media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical image sensors such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, And ROM, RAM, flash memory, and the like may be configured to store program instructions.

Meanwhile, the computer program may be specially designed and configured for the present invention, or may be known and usable to those skilled in the computer software field. Examples of the computer program may include not only machine language codes such as those produced by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like.

The specific implementations described in the present invention are examples and do not limit the scope of the present invention in any way. For brevity of the specification, descriptions of conventional electronic configurations, control systems, software, and other functional aspects of the systems may be omitted. In addition, the connection or connection members of the lines between the components shown in the drawings exemplarily represent functional connections and/or physical or circuit connections, and in an actual device, various functional connections that can be replaced or additionally It may be referred to as a connection, or circuit connections. In addition, if there is no specific mention such as "essential", "important", etc., it may not be an essential component for the application of the present invention.

Accordingly, the spirit of the present invention is limited to the above-described embodiments and should not be defined, and all ranges equivalent to or equivalently changed from the claims to be described later as well as the claims to be described later are the scope of the spirit of the present invention. It will be said to belong to.

1, 1A, 1B: holographic restoration device
10: light source
20: collimator
30,32: optical splitter
40: object light objective lens
50: object to be measured
60: reference light objective lens
70,72: optical mirror
80: image sensor
90: processor

Claims (3)

Checking, by a holographic restoration apparatus, at least one frequency component included in an image including intensity information of an object hologram generated by interference between a reference light reflected from an optical mirror and an object light affected by an object to be measured;
Extracting, by the holographic restoration apparatus, real image components corresponding to a real image from among the at least one frequency components;
Generating, by the holographic restoration apparatus, a real hologram including real image information of the object to be measured and correction light having a conjugate relationship with the reference light based on the real image components;
Generating, by the holographic restoration apparatus, an intermediate hologram from which information of the reference light is removed from the real hologram based on the correction light;
Generating, by the holographic restoration apparatus, curvature aberration correction information from the intermediate hologram;
Generating, by the holographic restoration apparatus, a correction hologram in which an error due to curvature aberration is removed from the intermediate hologram based on the curvature aberration correction information; And
Generating, by the holographic restoration apparatus, 3D shape information of the object to be measured from the correction hologram; And
The step of judging is,
Determine the upper range process in which the defect occurred among the plurality of upper range processes in consideration of the location of the defect, and use at least one of the real component, the real hologram, the intermediate hologram, the correction hologram, and the 3D shape information. Including, determining a lower range process included in the upper range process,
The holographic restoration device uses an algorithm between real information and defect information, real hologram and defect information, intermediate hologram and defect information, correction hologram and defect information, 3D shape information and defect information by a separate machine learning modeling device. Thus, the type of input data having a high correlation with the presence or absence of a specified defect is received from one of an image, real information, real hologram, intermediate hologram, correction hologram, and 3D shape information,
Determining the location of the defect included in the object to be measured and the process in which the defect occurred using one of the image, real information, real hologram, intermediate hologram, correction hologram, and three-dimensional shape, specified by the machine learning modeling device A method for determining a defect in a process using the restored hologram, including. The method of claim 1
The step of determining the process in which the defect occurred
Determine the presence or absence of a defect in the object to be measured using one of the image, real information, real hologram, intermediate hologram, correction hologram, and three-dimensional shape, specified by the machine learning modeling device,
By comparing the three-dimensional shape information of the measurement target object and the previously registered reference shape information to detect the location of the defect or the defect area of the measurement target object,
Process using a restored hologram, characterized by inferring a defect in the measurement object and a process in which the defect occurred by using information on the location or defect area of the measurement target object and the previously registered process How to judge the defect of the image. The method of claim 1
The step of determining the process in which the defect occurred
Determine the presence or absence of a defect in the object to be measured using one of the image, real information, real hologram, intermediate hologram, correction hologram, and three-dimensional shape, specified by the machine learning modeling device,
By comparing the three-dimensional shape information of the measurement target object and the previously registered reference shape information to detect the location of the defect or the defect area of the measurement target object,
Inferring a defect in the measurement object and an upper process in which the defect occurred using information on the location or defect area of the measurement target object and the previously registered process,
Using the real component, the real hologram, the intermediate hologram, the correction hologram, and the three-dimensional shape information for the object to be measured How to determine defects in the process using.

Images