KR20200015804A - Generation of high resolution images from low resolution images for semiconductor applications - Google Patents

Abstract

A method and system are provided for generating a high resolution image of a sample from a low resolution image of the sample. One system includes one or more computer subsystems configured to acquire low resolution images of a specimen. The system also includes one or more components executed by one or more computer subsystems. One or more components include a deep convolutional neural network that includes one or more first layers configured to produce a representation of a low resolution image. The deep convolutional neural network also includes one or more second layers configured to generate a high resolution image of the sample from the representation of the low resolution image. The second layer (s) includes a final layer configured to output a high resolution image and configured as a subpixel convolutional layer.

Description

Generation of high resolution images from low resolution images for semiconductor applications

The present invention generally relates to a method and system for generating high resolution images from low resolution images for semiconductor applications.

The following description and examples are included within this section and are not admitted to be prior art.

Manufacturing semiconductor devices such as logic and memory devices generally involves processing a substrate, such as a semiconductor wafer, using a vast number of semiconductor fabrication processes to form various features and multiple levels of semiconductor devices. Lithography, for example, is a semiconductor manufacturing process that involves transferring a pattern from a reticle to a resist arranged on a semiconductor wafer. Additional examples of semiconductor fabrication processes include, but are not limited to, chemical-mechanical polishing (CMP), etching, deposition, and ion implantation. Multiple semiconductor devices can be fabricated in an array on a single semiconductor wafer and then separated into individual semiconductor devices.

Inspection processes are used at various stages to detect defects on specimens during the semiconductor fabrication process to seek higher yield and thus higher profits in the fabrication process. Inspection has always been an important part of manufacturing semiconductor devices. However, as the dimensions of semiconductor devices are reduced, inspection becomes even more important in the successful manufacture of acceptable semiconductor devices, as smaller defects can cause them to fail.

Defect review generally involves redetecting the defects detected by the inspection process and generating additional information about the defects at higher resolutions using a high magnification optical system or scanning electron microscope (SEM). Thus, defect review is performed at discrete locations on the samples where defects were detected by inspection. High resolution data for defects generated by defect review is more appropriate for determining the nature of defects, such as profile, roughness, more accurate size information, and the like.

Metrology processes are also used in various steps to monitor and control the process during the semiconductor manufacturing process. Unlike inspection processes in which defects are detected on specimens, metrology processes differ from inspection processes in that they are used to measure one or more characteristics of a specimen that cannot be determined using currently used inspection tools. For example, metrology processes measure one or more properties of a sample, such as the dimensions (eg, line width, thickness, etc.) of features formed on the samples during the process so that the performance of the process can be determined from one or more properties. It is used to In addition, if one or more characteristics of the specimens are unacceptable (eg, outside of a predetermined range for the characteristic (s)), one of the specimens may be used so that additional specimens made by the process have acceptable characteristic (s). Measurements of the above characteristics can be used to change one or more parameters of the process.

Unlike defect review processes, in which defects detected by inspection are re-examined in defect review, metrology processes are also different from defect review processes in that metrology processes can be performed at locations where no defects were detected. In other words, unlike defect review, the locations where the metrology process is performed on the samples may be independent of the results of the inspection process performed on the samples. In particular, the location at which the metrology process is performed can be selected irrespective of the test results.

Thus, as described above, due to the limited resolution at which inspections (optical inspections and sometimes electron beam inspections) are performed, specimens generally have defect review of defects detected on the specimen (verification of detected defects, classification of detected defects, And determining the nature of the defects). In addition, higher resolution images are generally needed to determine information about patterned features formed on a specimen, such as in metrology, whether or not defects were detected in the patterned features. Thus, defect review and metrology can be time-consuming processes that require the use of additional tools (other than inspectors) required to generate the physical specimen itself and high resolution images.

However, defect review and measurement are not simply processes that can be eliminated to save time and money. For example, due to the resolution at which inspection processes are performed, inspection processes generally generate image signals or data that can be used to classify defects and / or determine information about detected defects sufficient to determine the root cause of the defects. Do not create Also, due to the resolution at which inspection processes are performed, inspection processes generally do not generate image signals or data that can be used to determine with sufficient accuracy information about patterned features formed on a specimen.

Therefore, it would be desirable to develop a system and method for generating high resolution images of a specimen that do not have one or more of the above-mentioned disadvantages.

The description of the various embodiments below should not be construed as in any way limiting the invention of the appended claims.

One embodiment relates to a system configured to generate a high resolution image of a specimen from a low resolution image of the specimen. The system includes one or more computer subsystems configured to acquire a low resolution image of the sample. The system also includes one or more components executed by one or more computer subsystems. One or more components include a deep convolutional neural network that includes one or more first layers configured to generate a representation of a low resolution image. The deep convolutional neural network also includes one or more second layers configured to generate a high resolution image of the sample from the representation of the low resolution image. One or more second layers include a final layer configured to output a high resolution image. The final layer is configured as a subpixel convolutional layer. The system may be further configured as described herein.

Further embodiments relate to another system configured to generate a high resolution image of a sample from a low resolution image of the sample. This system is configured as described above. The system also includes an imaging subsystem configured to generate a low resolution image of the specimen. In this embodiment, the computer subsystem (s) is configured to obtain a low resolution image from the imaging subsystem. This embodiment of the present system may be further configured as described herein.

Another embodiment is directed to a computer-implemented method for generating a high resolution image of a specimen from a low resolution image of the specimen. The method includes obtaining a low resolution image of the sample. The method also includes generating a representation of the low resolution image by inputting the low resolution image into one or more first layers of the deep convolutional neural network. The method also includes generating a high resolution image of the specimen based on the representation. Generation of the high resolution image is performed by one or more second layers of the deep convolutional neural network. One or more second layers include a final layer configured to output a high resolution image. The final layer is configured as a subpixel convolutional layer. Acquiring, generating a representation, and generating a high resolution image are performed by one or more computer systems. One or more components are executed by one or more computer systems, and the one or more components include deep convolutional neural networks.

Each of the steps of the foregoing method may be additionally performed as further described herein. In addition, embodiments of the foregoing methods may include any other step (s) of any other method (s) described herein. In addition, the method described above may be performed by any of the systems described herein.

Another embodiment relates to a computer readable non-transitory medium storing program instructions executable on one or more computer systems to perform a computer implemented method for generating a high resolution image of a sample from a low resolution image of the sample. The computer-implemented method includes the steps of the method described above. The computer readable medium may be further configured as described herein. The steps of the computer-implemented method may be performed as further described herein. In addition, the computer-implemented method in which the program instructions are executable may include any other step (s) of any other method (s) described herein.

Further advantages of the present invention will become apparent to those skilled in the art through the following detailed description of the preferred embodiments, with reference to the accompanying drawings.
1 and 1A are schematic diagrams showing side views of embodiments of a system configured as described herein.
2 is a block diagram illustrating an embodiment of a deep convolution neural network that may be included in the embodiments described herein.
3 is a schematic diagram illustrating one embodiment of a deep convolutional neural network that may be included in the embodiments described herein.
4 and 5 are block diagrams illustrating embodiments of one or more components that may be included in the embodiments described herein.
FIG. 6 is a block diagram illustrating one embodiment of a pre-trained VGG network that may be included in a context aware loss module embodiment.
7 includes corresponding high resolution and low resolution images generated by the imaging system, high resolution images generated from low resolution images by the embodiments described herein, and line profiles generated for the respective images.
8 shows overlay measurements along the x and y axes between a high resolution image generated by the imaging system and a high resolution image generated by the embodiments described herein and between a low resolution image and the high resolution image generated by the imaging system. Examples of correlation in results are included.
9 is a block diagram illustrating one embodiment of a computer readable non-transitory medium having stored thereon program instructions for causing one or more computer systems to perform the computer-implemented method described herein.

While the invention is susceptible to various modifications and alternative forms, the drawings show specific embodiments of the invention by way of example, which are described in more detail herein. The drawings may not be drawn to scale. The drawings and detailed description, however, are not intended to limit the invention to the particular forms disclosed, and on the contrary, the invention is intended to cover all modifications, equivalents, and modifications falling within the spirit and scope of the invention as defined by the claims. Cover alternative configurations.

Referring now to the drawings, it is noted that the drawings are not drawn to scale. In particular, the scale of some of the elements of the figures has been exaggerated to emphasize their characteristics. It is also noted that the drawings are not drawn to the same scale. Similarly configurable elements, shown in more than one figure, have been indicated using the same reference numerals. Unless stated otherwise herein, any elements described and illustrated herein may include any suitable element available commercially.

One embodiment relates to a system configured to generate a high resolution image of a specimen from a low resolution image of the specimen. As further described herein, embodiments provide a platform independent, data driving method and system for generating stable and robust metrology quality images. Embodiments can also be used to create super high resolution, super-resolved images that are relatively high quality. Embodiments may further be used to increase imaging throughput. Embodiments may also be used to generate review images from relatively low frame, relatively low electron per pixel (e / p) inspection scans. ("Low frame" means a small number of image captures in the same location, for example, several frames are captured and then combined to improve image quality to obtain better imaging and increase the signal-to-noise ratio). "E / p" is basically electrons per pixel, the higher the e / p, the higher the quality but the lower throughput. Higher e / p is achieved using beam conditions.)

Embodiments described herein are electron beam (ebeam), broadband band plasma for producing relatively high quality images with much higher throughput from images generated by any of the following platforms: BBP), laser scattering, limited resolution, and metrology platforms. In other words, the images can be generated by the imaging system at a relatively high throughput and thus at a relatively low resolution, and then converted into relatively high resolution images by the embodiments described herein. This means that high resolution images can be effectively generated with a relatively high throughput. Embodiments described herein advantageously provide for trained conversion, noise reduction, and transfer of quality from high quality scans to low quality scans between relatively low and relatively high resolution imaging manifolds. do. Imaging "manifolds" can generally be defined as the theoretical probability space of all possible images.

As used herein, the term "low resolution image" of a specimen is generally defined as an image in which all patterned features formed in the region of the specimen from which the image was generated are not resolved in the image. For example, some of the patterned features in the area of the specimen from which the low resolution image was generated may be resolved in the low resolution image if the size is large enough to be resolvable. However, low resolution images are not created at a resolution that makes it possible to interpret all the patterned features in the image. In this manner, the term “low resolution image” as used herein refers to patterned features on a specimen that are sufficient to allow a low resolution image to be used in applications such as defect review, which may include defect classification and / or verification, and metrology. Contains no information on. In addition, the term “low resolution image” as used herein generally refers to images generated by an inspection system having a relatively low resolution (eg, lower than a defect review and / or metrology system) to have a relatively fast throughput. Refers to. In this way, a "low resolution image" may also be commonly referred to as a high throughput or high throughput (HT) image. For example, to produce images with higher throughput, the number of e / p and frames can be lowered, resulting in a lower quality scanning electron microscope (SEM) image.

The "low resolution image" may also be "low resolution" in that the resolution is lower than the "high resolution image" described herein. As used herein, the term “high resolution image” can generally be defined as an image in which all patterned features of a sample are resolved with relatively high accuracy. In this way, all patterned features in the region of the specimen from which the high resolution image is generated are resolved in the high resolution image regardless of the size of the features. As such, the term “high resolution image” as used herein refers to patterned features on a specimen that are sufficient for high resolution images to be used in applications such as defect review, which may include defect classification and / or verification, and metrology. Contains information about Also, as used herein, the term “high resolution image” generally refers to images that cannot be generated by inspection systems during routine operations configured to sacrifice resolution capability for increased throughput. In this way, "high resolution image" may also be referred to herein as "high sensitivity image", which is another term for "high definition image" in this specification and in the related art. For example, to produce high quality images, e / p, frames, etc. can be increased, which produces good quality SEM images but significantly reduces throughput. These images are then "high sensitivity" images in that these images can be used for high sensitivity defect detection.

In contrast to the embodiments described further herein, most older methods use heuristic and cherry-picked parameters to produce relatively noiseless images. These methods are generally designed with the statistical properties of the images they drive in mind, so they cannot be ported to other platforms without integrating the heuristics of that platform. Some of the well known methods used to reduce noise in an image are anisotropic diffusion, bilateral filter, Weiner filter, non-local means, and the like. Bi-directional and Weiner filters remove noise at the pixel level using filters designed from neighboring pixels. Anisotropic diffusion applies the diffusion law to the image to soften the texture / strength in the image according to the diffusion equation. A threshold function is used to prevent diffusion at the edges, thus preserving the edges in the image to a large extent.

The disadvantages of older methods such as wineries and bidirectional filtering are that they are parametric approaches that need to be fine tuned at the image level to get the best results. This approach is not data centric, limiting the performance that can be achieved for challenging imaging types. Another limitation is that most of the processing of this approach is done inline, which, due to throughput limitations, limits the use cases that can be used.

One embodiment of a system configured to generate a high resolution image of a specimen from a low resolution image of the specimen is shown in FIG. 1. The system includes one or more computer subsystems (eg, computer subsystem 36 and computer subsystems 102) and one or more components 100 executed by one or more computer subsystems. In some embodiments, the system includes an imaging system (or subsystem) 10 configured to generate a low resolution image of a specimen. In the embodiment of FIG. 1, the imaging system is configured to scan the physical version of the specimen with light or to detect light from the specimen while directing light thereto, thereby generating images for the specimen. The imaging system is also configured to perform scanning (or directing) and detection in multiple modes.

In one embodiment, the specimen is a wafer. The wafer can include any wafer known in the art. In another embodiment, the sample is a reticle. The reticle can include any reticle known in the art.

In one embodiment, the imaging system is an optical based imaging system. In one such example, in the embodiment of the system shown in FIG. 1, the optical based imaging system 10 includes an illumination subsystem configured to direct light to the specimen 14. The lighting subsystem includes at least one light source. For example, as shown in FIG. 1, the illumination subsystem includes a light source 16. In one embodiment, the illumination subsystem is configured to direct light to the specimen at one or more angles of incidence that may include one or more tilt angles and / or one or more vertical angles. For example, as shown in FIG. 1, light from light source 16 is directed to specimen 14 at oblique incidence angle through optical element 18 and through lens 20. The oblique incidence angle may comprise any suitable oblique incidence angle, which may vary depending on, for example, the nature of the specimen.

The imaging system may be configured to direct light to the specimen at different incidence angles at different times. For example, the imaging system can be configured to change one or more properties of one or more elements of the illumination subsystem so that light can be directed at the specimen at an angle of incidence different from that shown in FIG. 1. In one such example, the imaging system may be configured to move the light source 16, the optical element 18, and the lens 20 such that the light is directed at the specimen at different oblique or normal (or nearly vertical) angles of incidence. Can be.

In some cases, the imaging system may be configured to direct light onto the specimen at more than one angle of incidence at the same time. For example, the illumination subsystem may include more than one illumination channel, one of the illumination channels comprising a light source 16, an optical element 18 and a lens 20 as shown in FIG. 1. And other one of the illumination channels (not shown) may include similar elements that may be configured differently or the same, or at least one or more light sources and possibly one or more as described further herein. It may include other components. If such light is directed at the sample simultaneously with other light, one or more characteristics of the light directed at the sample at different angles of incidence (e.g., so that the light resulting from illuminating the sample at different angles of incidence can be distinguished from each other at the detector (s)). For example, wavelength, polarization, etc.) may be different.

In another example, the illumination subsystem may include only one light source (eg, light source 16 shown in FIG. 1), wherein light from the light source is one or more optical elements (not shown) of the illumination subsystem. ) Can be separated into different light paths (eg, based on wavelength, polarization, etc.). Then, the light in each of the different optical paths can be directed to the specimen. Multiple illumination channels can be configured to direct light to the specimen simultaneously or at different times (eg, when different illumination channels are used to illuminate the specimen sequentially). In another example, the same illumination channel can be configured to direct light having different properties to the specimen at different times. For example, in some instances, optical element 18 may be configured as a spectral filter, and the characteristics of the spectral filter may be such that light of different wavelengths may be directed at a sample at different times (eg, with a spectral filter). Change) in a variety of different ways. The illumination subsystem may have any other suitable configuration known in the art for directing light having different or the same characteristics to the specimen sequentially or simultaneously at different or angles of incidence.

In one embodiment, the light source 16 may comprise a broadband plasma (BBP) light source. In this way, the light generated by the light source and directed to the specimen may include broadband light. However, the light source may comprise any suitable other light source such as a laser. The laser can include any suitable laser known in the art and can be configured to produce light of any suitable wavelength or wavelengths known in the art. Also, the laser can be configured to produce light that is monochromatic or nearly monochromatic. In this way, the laser can be a narrowband laser. The light source may also include a multicolor light source that produces light of multiple discrete wavelengths or wavebands.

Light from optical element 18 may be focused on specimen 14 by lens 20. Although the lens 20 is shown in FIG. 1 as a single refractive optical element, in practice, the lens 20 may include a plurality of refractive and / or reflective optical elements that focus light from the optical element onto the specimen when combined. I will understand that. The illumination subsystem shown in FIG. 1 and described herein may include any other suitable optical elements (not shown). Examples of such optical elements are, by way of non-limiting example, polarization component (s), spectral filter (s), spatial filter (s), reflective optical element (s), apodizer (s), beam splitters (S), aperture (s), and the like, and may include any such suitable optical elements known in the art. In addition, the imaging system can be configured to modify one or more elements of the illumination subsystem based on the type of illumination to be used for imaging.

The imaging system can also include a scanning subsystem configured to allow light to scan the specimen. For example, the imaging system may include a stage 22 where the specimen 14 is placed during the examination. The scanning subsystem may include any suitable mechanical and / or robotic assembly (including stage 22) that may be configured to move the specimen to enable light to scan the specimen. Additionally or alternatively, the imaging system may be configured such that one or more optical elements of the imaging system perform some scanning of light onto the specimen. The specimen can be scanned with light in any suitable manner, such as a tortuous path or a spiral path.

The imaging system further includes one or more detection channels. At least one of the one or more detection channels includes a detector configured to detect light from the sample due to illumination of the sample by the system and to generate an output responsive to the detected light. For example, the imaging system shown in FIG. 1 includes two detection channels, one of which is formed by the condenser 24, the element 26 and the detector 28, and the other detection. The channel is formed by the collector 30, the element 32 and the detector 34. As shown in FIG. 1, two detection channels are configured to collect and detect light at different collection angles. In some instances, both detection channels are configured to detect scattered light and the detection channels are configured to detect light scattered at different angles from the sample. However, one or more detection channels may be configured to detect other types of light (eg, reflected light) from the specimen.

As further shown in FIG. 1, the two detection channels are shown as located in the ground, and the illumination subsystem is also shown as located in the ground. Therefore, in this embodiment, the two detection channels are located at the entrance plane (eg, at the entrance plane center). However, one or more detection channels may be located outside the entrance plane. For example, the detection channel formed by the concentrator 30, element 32, and detector 34 may be configured to collect and detect light scattered out of the plane of incidence. Therefore, such a detection channel can be collectively referred to as a "side" channel, which side channel can be centered in a plane substantially perpendicular to the incident surface.

1 illustrates an embodiment of an imaging system that includes two detection channels, but the imaging system may include a different number of detection channels (eg, only one detection channel or two or more detection channels). have. In one such example, the detection channel formed by the concentrator 30, element 32, and detector 34 may form one side channel as described above, and the imaging system is on the opposite side of the incident surface. It may include an additional detection channel (not shown) formed as another side channel located at. Thus, the imaging system comprises a light collector 24, an element 26, and a detector 28, the center of which is located at the plane of incidence, at a scattering angle (s) that is perpendicular to or near the vertical of the specimen surface. And a detection channel configured to collect and detect light. Thus, this detection channel may be referred to collectively as the "top" channel, and the imaging system may also include two or more side channels configured as described above. As such, the imaging system may include at least three channels (ie, one top channel and two side channels), each of the at least three channels having its own condenser and each concentrator being different It is configured to collect light at a scattering angle different from each of the collectors.

As described above, each of the detection channels included in the imaging system may be configured to detect scattered light. Thus, the imaging system shown in FIG. 1 can be configured for dark field (DF) imaging of a specimen. However, the imaging system may also or alternatively include detection channel (s) configured for bright field (BF) imaging of the specimen. In other words, the imaging system may include at least one detection channel configured to detect light specularly reflected from the specimen. Thus, the imaging system described herein may be configured for DF imaging only, for BF imaging only, or for both DF and BF imaging. While each collector is shown in FIG. 1 as a single refractive optical element, it should be understood that each of the collectors may include one or more refractive optical element (s) and / or one or more reflective optical element (s).

One or more detection channels may comprise any suitable detector known in the art. For example, the detector may be a photo-multiplier tube (PMT), a charge coupled device (CCD), a time delay integration (TDI) camera, and any other known in the art. Appropriate detectors may be included. The detector may also include a non-imaging detector or an imaging detector. In this way, where the detector is a non-imaging detector, each detector may be configured to detect a constant characteristic of scattered light, such as intensity, but may not be configured to detect this characteristic as a function of position in the imaging plane. As such, the output generated by each detector included in each detection channel of the imaging system may be a signal or data, but may not be an image signal or image data. In such examples, a computer subsystem, such as computer subsystem 36, may be configured to generate images of the specimen from the non-imaging output of the detector. However, in other examples, the detector may be configured as an imaging detector configured to generate an image signal or image data. Thus, the imaging system can be configured to generate the images described herein in a plurality of ways.

1 is used herein to generally illustrate a configuration of an imaging system or subsystem that may be included in the system embodiments described herein or that may generate images used by the system embodiments described herein. Note that it is provided. Clearly, the imaging system configuration described herein may be modified to optimize the performance of the imaging system that is typically performed when designing a commercial imaging system. In addition, the systems described herein include tools of the 29xx / 39xx and Puma 9xxx series commercially available from KLA-Tencor, Milpitas, CA (eg, by adding the functionality described herein to existing systems). It can be implemented using the same existing system. For some such systems, the embodiments described herein may be provided as an optional function of the system (eg, in addition to other functions of the system). Alternatively, the imaging system described herein may be designed “new from the beginning” to provide an entirely new imaging system.

The computer subsystem 36 of the imaging system may include (eg, “wired” and / or “wireless” transmission media so that the computer subsystem can receive output generated by the detector during scanning of the specimen). Can be coupled to the detector of the imaging system in any suitable manner). Computer subsystem 36 may be configured to perform a plurality of functions further described herein using the output of the detector.

The computer subsystem shown in FIG. 1 (as well as the other computer subsystems described herein) may also be referred to herein as computer system (s). Each of the computer subsystem (s) or system (s) described herein may take a variety of forms, including a personal computer system, image computer, mainframe computer system, workstation, network appliance, internet appliance, or other device. have. In general, the term “computer system” may be broadly defined to encompass any device having one or more processors that execute instructions from a memory medium. The computer subsystem (s) or system (s) may also include any suitable processor known in the art, such as a parallel processor. In addition, the computer subsystem (s) or system (s) may include a computer platform with high speed processing and software as a standalone or networked tool.

If the system includes more than one computer subsystem, the different computer subsystems may be combined with each other so that images, data, information, instructions, and the like may be transferred between the computer subsystems as further described herein. . For example, computer subsystem 36 may be by any suitable transmission medium (as shown by dashed lines in FIG. 1), which may include any suitable wired and / or wireless transmission medium known in the art. May be coupled to the computer subsystem (s) 102. Two or more of such computer subsystems may also be effectively combined by a shared computer readable storage medium (not shown).

Although the imaging system has been described above as being an optical or light based imaging system, the imaging system may be an electron beam based imaging system. In one such embodiment, shown in FIG. 1A, the imaging system includes an electron column 122 coupled to the computer subsystem 124. As also shown in FIG. 1A, the electron column includes an electron beam source 126 configured to generate electrons focused to the specimen 128 by one or more elements 130. The electron beam source may include, for example, a cathode source or emitter tip, and the one or more elements 130 may include, for example, gun lenses, anodes, beam limiting apertures, gate valves, beam current selection. It may include an aperture, an objective lens, and a scanning subsystem, all of which may include any suitable element known in the art.

The electrons returned from the sample (eg, secondary electrons) may be focused on the detector 134 by one or more elements 132. One or more elements 132 may include a scanning subsystem, which may be, for example, the same scanning subsystem included in element (s) 130.

The electron column can include any other suitable element known in the art. Also, the electronic column is US Patent No. 8,664,594, registered on April 4, 2014 by Jiang et al., US Patent No. 8,692,204, issued on April 8, 2014 by Kojima et al., Gubens ( US Patent No. 8,698,093, filed April 15, 2014, and US Patent No. 8,716,662, filed May 6, 2014, MacDonald et al. And these patents are incorporated herein by reference as if fully set forth herein.

In FIG. 1A, the electron column is shown as being configured so that electrons are directed to the sample at oblique incidence angles and scattered from the sample at different oblique angles, but it should be understood that the electron beam may be directed to the sample at any suitable angle and scattered from the sample. do. In addition, the electron beam based imaging system may be configured to use multiple modes to generate images of the specimen as described further herein (eg, at different illumination angles, focusing angles, etc.). Multiple modes of the electron beam based imaging system may be different in any image generation parameters of the imaging system.

Computer subsystem 124 may be coupled to detector 134 as described above. The detector may detect electrons returned from the surface of the specimen to form electron beam images of the specimen. The electron beam images can include any suitable electron beam images. Computer subsystem 124 may be configured to perform one or more functions described herein with respect to the specimen using the output generated by detector 134. Computer subsystem 124 may be configured to perform any additional step (s) described herein. The system including the imaging system shown in FIG. 1A can be further configured as described herein.

Note that FIG. 1A is provided herein to generally illustrate the configuration of an electron beam based imaging system that may be included within the embodiments described herein. As with the optical based imaging system described above, the electron beam based imaging system configuration described herein may be modified to optimize the performance of the imaging system that is normally performed when designing a commercial imaging system. In addition, the systems described herein can be utilized using existing systems such as the tools of the eSxxx and eDR-xxxx series commercially available from KLA-Tencor (eg, by adding the functionality described herein to existing systems). Can be implemented. For some such systems, the embodiments described herein may be provided as an optional function of the system (eg, in addition to other functions of the system). Alternatively, the system described herein may be designed "new from the ground up" to provide an entirely new system.

Although the imaging system has been described above as being an optical based or electron beam based imaging system, the imaging system may be an ion beam based imaging system. Such an imaging system can be configured as shown in FIG. 2 except that the electron beam source can be replaced with any suitable ion beam source known in the art. Imaging systems are also included in commercially available focused ion beam (FIB) systems, helium ion microscopy (HIM) systems, and secondary ion mass spectroscopy (SIMS) systems. Or other suitable ion beam based imaging system.

As noted above, the imaging system is configured to generate an actual image for the physical version of the sample by scanning energy (eg, light or electron) against the physical version of the sample. In this way, the imaging system can be configured as a "real" system rather than a "virtual" system. For example, the storage medium (not shown) and computer subsystem (s) 102 shown in FIG. 1 may be configured as a “virtual” system. In particular, the storage medium and computer subsystem (s) are not part of the imaging system 10 and do not have any capability to process a physical version of a sample. In other words, in a system configured as a virtual system, the output of the one or more "detectors" may be output previously generated by one or more detectors of the real system and stored in the virtual system, during the "scanning" You can reproduce this stored output as if the specimen were being scanned. In this way, scanning a sample into a virtual system may appear the same as if the physical sample is being scanned into a real system, but in practice "scanning" simply reproduces the output of the sample in the same way as when scanning the sample. It only includes things. Systems and methods configured as “virtual” inspection systems are commonly assigned U. S. Patent Nos. 8,126, 255 to Bashaskar et al. (Registered on February 28, 2012) and US Pat. No. 9,222,895 to Duffy et al. (2015). Registered December 29, 2008, both of which are incorporated herein by reference as if fully set forth herein. Embodiments described herein may be further configured as described in these patents. For example, one or more computer subsystems described herein may be further configured as described in these patents. In addition, configuring one or more virtual systems as a Central Computing and Storage (CCS) system may be performed as described in the above-mentioned Duffy patent. The persistent storage mechanism described herein may have distributed computing and storage, such as the CCS architecture, but the embodiments described herein are not limited to such architecture.

As mentioned above, the imaging system may be configured to generate images of the specimen in multiple modes. In general, the "mode" may be defined by the parameters of the imaging system used to generate the images of the specimen or the values of the output used to generate the images of the specimen. Thus, different modes may differ in values for at least one of the imaging parameters of the imaging system. For example, in one embodiment of an optical based imaging system, at least one of the multiple modes is at least one different from at least one illumination light wavelength used for at least one other of the multiple modes. Use the illumination light wavelength. The modes may differ in the illumination wavelength (eg, by using different light sources, different spectral filters, etc.) as described further herein for the different modes. In another embodiment, at least one of the multiple modes uses an illumination channel of the imaging system that is different from the illumination channel of the imaging system used for at least one other of the multiple modes. For example, as described above, the imaging system may include more than one illumination channel. As such, different illumination channels can be used for different modes.

In one embodiment, the imaging system is an inspection system. For example, the optical and electron beam imaging systems described herein can be configured as inspection systems. In another embodiment, the imaging system is a defect review system. For example, the optical and electron beam imaging systems described herein can be configured as defect review systems. In further embodiments, the imaging system is a metrology system. For example, the optical and electron beam imaging systems described herein can be configured as metrology systems. In particular, embodiments of the imaging system described herein and shown in FIGS. 1 and 1A may be modified with one or more parameters to provide different imaging capabilities depending on the application in which they are to be used. In one such example, the imaging system shown in FIG. 1 may be configured to have higher resolution when used for defect review or metrology than inspection. In other words, the embodiments of the imaging system shown in FIGS. 1 and 1A may be customized in various ways that will be apparent to those skilled in the art to produce imaging systems having different imaging capabilities that are more or less appropriate for different applications. Some general and various configurations for imaging systems are described.

One or more computer subsystems are configured to acquire a low resolution image of the sample. Acquiring a low resolution image may be performed using one of the imaging systems described herein (e.g., by directing a light or electron beam to a specimen and detecting the light or electron beam, respectively, from the specimen, respectively). Can be. In this way, acquisition of low resolution images can be performed using the physical specimen itself and some kind of imaging hardware. However, acquiring low resolution images does not necessarily include imaging the specimen using imaging hardware. For example, other systems and / or methods may generate low resolution images and store the generated low resolution images in one or more storage media, such as the virtual inspection system described herein or other storage media described herein. have. Thus, acquiring the low resolution image may include acquiring the low resolution image from a storage medium in which the low resolution image is stored.

In some embodiments, the low resolution image is generated by the inspection system. For example, as described herein, a low resolution image can be generated by an inspection system configured to have a low resolution and increase its throughput. The inspection system may be an optical inspection system or an electron beam inspection system. The inspection system can have any configuration that is further described herein.

In one embodiment, the low resolution image is generated by an electron beam based imaging system. In another embodiment, the low resolution image is generated by an optical based imaging system. For example, the low resolution image can be generated by any of the electron beam based or optical based imaging systems described herein.

In one embodiment, the low resolution image is generated with a single mode imaging system. In another embodiment, one or more low resolution images are generated for a specimen with a multimode imaging system. For example, the low resolution image (s) input to the deep convolutional neural network (deep CNN) described further herein may comprise a single low resolution image generated only via a single mode imaging system. Alternatively, the low resolution image input to the deep CNN described further herein may include a plurality of low resolution images (eg, a first image generated in a first mode, a second image generated by a multimode imaging system). And a second image generated as such. Single mode and multiple modes may include any of the mode (s) described further herein.

Component (s) executed by computer subsystem (s), eg, computer subsystem 36 and / or computer subsystem (s) 102, eg, component (s) 100 shown in FIG. 1. Includes a deep CNN 104. The deep CNN includes one or more first layers configured to generate a representation of the low resolution image and one or more second layers configured to generate a high resolution image for the sample from the representation of the low resolution image. In this manner, embodiments described herein may use one of the deep CNNs described herein (eg, one or more machine learning techniques) to convert a low resolution image of a sample into a high resolution image of the sample. have. For example, as shown in FIG. 2, the deep CNN is shown as an image conversion network 200. During production and / or runtime (ie, after the image conversion network has been set up and / or trained, this can be done as described further herein), the input to the image conversion network is a low resolution (high throughput) image. 202, and the output of the image conversion network may be output as a high resolution (high sensitivity) image 204.

One or more second layers include a final layer configured to output a high resolution layer, the final layer being configured as a subpixel convolutional layer. 3 illustrates one embodiment of an image conversion network architecture that may be suitable for use in the embodiments described herein. In this embodiment, the image conversion network is a deep CNN with a sub pixel layer as the final layer. In this architecture, the input may be a low resolution image 300, which is simply shown as a grid of pixels in FIG. 3 and does not represent any particular low resolution image that may be generated by the embodiments described herein. The low resolution image may be input to one or more first layers 302, 304, which may be configured as convolutional layers configured for feature map extraction. These first layers may form the hidden layers of the image conversion network architecture.

Thus, the representation of the low resolution image generated by the one or more first layers may be one or more features and / or feature maps. The features may have any suitable feature types known in the art, which may be inferred from the input and used to generate the output described further herein. For example, the features may include a vector of intensity values per pixel. The features may also include any other type of features described herein, such as a vector of scalar values, a vector of independent distributions, a joint distribution, or any other suitable feature type known in the art. Can be. As described further herein, the features are learned by the network during training, and may or may not be correlated with any actual feature known in the art.

One or more second layers include a final layer 306, which is configured as a subpixel convolutional layer that aggregates feature maps from low resolution space and builds a high resolution image 308 in a single step. The subpixel convolutional layer learns an array of upscaling filters to upscale the final low resolution feature maps into a high resolution output image. In this way, the image conversion network takes a noisy, poorly resolved high throughput input image, computes feature maps across many convolutional layers, and then Can be used to transform feature maps into relatively quiet, super-resolved images. The subpixel convolutional layer advantageously provides relatively complex upscaling filters that are specifically trained for each feature map while also reducing the computational complexity of the overall task. The deep CNN used in the embodiments described herein is described in Shi et al., "Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network" (September 2016, arXiv: 1609.05158 It can be further configured as described by v2), which is incorporated by reference as if fully set forth herein.

Deep CNNs described herein may be generally classified as deep learning models. Generally speaking, "deep learning" (also known as deep structured learning, hierarchical learning, or deep machine learning) is a branch of machine learning based on a series of algorithms that attempt to model high level abstractions in data. In a simple case, there can be two sets of neurons: one neuron receives the input signal and the other neuron sends the output signal. When the input layer receives the input, the input layer delivers the modified version of the input to the next layer. In deep networks, there are many layers between the input and the output (the layers are not composed of neurons, but they can help you think so), allowing the algorithm to use multiple processing layers consisting of several linear and nonlinear transformations. do.

In-depth learning is part of a broad family of machine learning methods based on learning of representations of data. An observation (eg, an image) may be represented in many ways, such as in a vector of intensity values per pixel, or in a more abstract manner as a set of edges, in areas of a particular shape, and so forth. Some expressions are superior to others by simplifying learning tasks (eg, facial recognition or facial expression recognition). One of the promises of deep learning is to replace manual features with efficient algorithms for unsupervised or semi-supervised feature learning and hierarchical feature extraction.

Research in this area attempts to form better representations and create models for learning these representations from large, unlabeled data. Some expressions have been inspired by advances in neuroscience and are roughly responsible for the interpretation of information processing and communication patterns in the nervous system, such as neural coding, which attempts to define the relationship between various stimuli in the brain and their associated neural responses. Based.

Deep CNNs described herein may also be classified as deep learning models. Machine learning can generally be defined as a type of artificial intelligence (AI) that provides a computer with the ability to learn without explicitly programming. Machine learning focuses on the development of computer programs that can teach themselves to grow and change when exposed to new data. In other words, machine learning can be defined as a subfield of computer science that "grants a computer the ability to learn without explicitly programming". Machine learning explores the research and construction of algorithms that can learn from data and make predictions about the data—these algorithms build models from sample inputs, thereby generating strictly static program instructions by data-driven prediction or determination. Overcome the following.

Machine learning described herein is described in Sugiyama, "Introduction to Statistical Machine Learning" (Morgan Kaufman, 2016, p. 534); Jebara's "Discriminative, Generative, and Imitative Learning" (MIT papers, 2002, 212); And Hand et al., "Principles of Data Mining (Adaptive Computation and Machine Learning)" (MIT Publication, 2001, page 578), which are incorporated by reference as if set forth in their entirety herein. It is used as. Embodiments described herein may be further configured as described in these references.

Deep CNNs are also a generative model. A "generating" model can generally be defined as a model having stochastic properties. In other words, the "generating" model is not a model that performs forward simulation or a rule-based approach, so there is no need for a physical model of the processes involved in producing the actual image (the simulated image for which it is generated). Instead, as further described herein, the generation model can be trained based on the appropriate training data set (in that the parameters of the generation model can be learned).

In one embodiment, the deep CNN is a deep generation model. For example, a deep CNN can be configured to have a deep learning architecture in that the model can include a plurality of layers that perform a plurality of algorithms or transformations. The number of layers on one or both sides of the deep CNN may differ from that shown in the figures described herein. For practical purposes, the appropriate range of layers on both sides is from two to tens of layers.

Deep CNNs can also be deep neural networks with a set of weights that model the world according to the data supplied to train them. Neural networks can generally be defined as computational approaches based on a collection of relatively large neural units that roughly model how the biological brain solves the problem of relatively large biological neuronal clusters connected by axons. Each neural unit is connected to many other units, and links can force or inhibit their influence on the activation state of the connected neural units. These systems are self-learning and training, rather than explicitly programmed, and are excellent in areas where solution or feature detection is difficult to represent in existing computer programs.

Neural networks generally consist of several layers, with signal paths traversing back and forth. The goal of neural networks is to solve problems in the same way as the human brain, but many neural networks are much more abstract. Modern neural network projects typically operate from thousands to millions of neural units and millions of connections. The neural network can have any suitable architecture and / or configuration known in the art.

Embodiments described herein may or may not be configured to train a deep CNN used to generate a high resolution image from a low resolution image. For example, other methods and / or systems can be configured to generate a trained deep CNN, which can be accessed and used by the embodiments described herein below. In general, training of a deep CNN may include acquiring data (eg, both low resolution and high resolution images, which may include any of the low resolution and high resolution images described herein). Then, training, test, and validity datasets can be built using the list of input tuples and expected output tuples. The input tuples may take the form of low resolution images, and the output tuples may be high resolution images corresponding to the low resolution images. The deep CNN can then be trained using the training dataset.

In one embodiment, the one or more components include a context aware loss module configured to train the deep CNN, wherein during the training of the deep CNN, the one or more computer subsystems are coupled to the high resolution image generated by the one or more second layers, and to the sample. A corresponding known high resolution image is entered into the context aware loss module, and the context aware loss module compares the corresponding known high resolution image to determine a context aware loss in the high resolution image generated by the one or more second layers. For example, as shown in FIG. 4, a deep CNN network is shown as an image conversion network 400. This figure shows a deep CNN during training or during setup. The input to the image conversion network is a low resolution (high throughput) image 402, which may be generated as described further herein. The image conversion network may then output a high resolution (high sensitivity) image 404 as described further herein. The output high resolution image and the corresponding, known high resolution image (eg, “ground truth” high sensitivity image) 406 may be input to the situational awareness loss module 408. In this way, the complete network architecture of the embodiments described herein may include two blocks, namely image conversion network and situational awareness loss. The context aware loss module 408 compares two images it receives as input (i.e., a high resolution image generated by an image conversion network and a high resolution image ground truth image generated by an imaging system, for example). One or more differences between the two input images may be determined. The context aware loss module may also be configured as described herein.

In this way, upon setup, the embodiments take noisy, poorly resolved images, and true superresolution image pairs, and then use a situational awareness loss to learn the transformation matrix between these images through the neural network. do. As used herein, the term "being noisy" can generally be defined as an image having a relatively low signal-to-noise ratio (SNR), while the term "to be true" as used herein is generally relative It can be defined as an image having a high SNR. Accordingly, these terms are used interchangeably herein. Such image pairs can be obtained from any imaging platforms commercially available from KLA-Tencor (and other companies), such as e-beams, BBP tools, limited resolution imaging tools, and the like. Once the training is complete, the network learns from noisy and poorly resolved images to true superresolution images while maintaining spatial fidelity. In this way, the embodiments described herein take a data intensive approach to take advantage of the data redundancy observed in semiconductor images by learning the transformation between noisy and poorly resolved images and true superresolution images. use. The trained network is then placed in production so that the imaging system generates noisy high throughput data, which is then converted to the corresponding low noise, super resolution data using the trained image conversion network. do. In production, the network performs like a normal post processing algorithm.

In one such embodiment, situational awareness loss includes content loss, style loss, and total variation (TV) normalization. 5 illustrates one such embodiment. In particular, the context aware loss module 408 shown in FIG. 4 may include the content loss module 500, the style loss module 502, and the TV normalization module 504 shown in FIG. 5. For example, situational awareness loss is a common framework and is expressed through style and content loss. Deep neural networks tend to learn image features gradually, starting at the edges, contours in lower layers, and up to more complex features, such as faces or perhaps entire objects in later layers. This is also related to biological vision. Assume that lower layers of the convolutional network learn features that are considered perceptually important. Therefore, we design our situational awareness loss in addition to the activation of the learned network. Situational awareness loss mainly consists of style, content, and normalization loss.

In one such embodiment, content loss includes loss in lower level features of the corresponding known high resolution image. For example, the content of an image is defined as lower level features such as edges, contours, and the like. Minimization of content loss helps preserve these lower level features that are important in the generation of measurement quality super resolution images. More specifically, content loss is included in the loss function to preserve the edges and contours in the images because they are important for use in measurements, etc., for high resolution images. Traditional techniques such as Bicubic interpolation or training with L2 loss need not guarantee the preservation of these edges and contours.

The next major part of the loss is called the style transfer loss. In one such embodiment, style loss includes loss in one or more abstracting entities that qualitatively define the corresponding known high resolution image. For example, style is defined as an abstraction entity that qualitatively defines an image, including attributes such as sharpness, texture, and color. One reason to use the deep learning described herein is that the difference between the low resolution / high resolution images described herein is not just the resolution, but they may have different noise characteristics, charged artifacts, textures, and the like. Thus, it is not enough to superresolution a low resolution image, and the mapping from a low resolution image to a high resolution image is learned using in-depth learning. The style of the image is characterized by the higher layer activation of the trained network. The combination of style and content loss enables the image conversion network to learn the conversion between noisy and poorly resolved images and true super resolution images. Once the image conversion network is trained using situational awareness loss, it can be deployed in production to produce true super-resolution images from noisy, poorly resolved, high-throughput images while maintaining spatial fidelity. In some embodiments, the style transfer loss is defined as the loss between the terrestrial true high resolution images and the final layer features of the super resolution high resolution images (ie, the images generated by the one or more second layers), in particular It is the case that we want to classify the resolution high resolution images.

In a further such embodiment, the context aware loss module includes a pre-trained VGG network. 6 shows how activation from predefined networks is used to calculate style and content loss. For example, as shown in FIG. 6, the pre-trained VGG network 600 may be coupled to the content loss module 500 and the style loss module 502. VGG16 (also known as OxfordNet) is a convolutional neural network architecture named after Oxford's Visual Geometry Group. VGG networks can also be further configured as described by Simonyan et al. In "Very Deep Convolutional Networks for Large-Scale Image Recognition" (arXiv: 1409.1556v6, April 2015, p. 14). The document is hereby incorporated by reference as if fully set forth herein. As shown in FIG. 6, the pre-trained VGG network includes convolutional layers (eg, conv-64, conv-128, conv-256, conv-512), maxpool layers, fully connected. It may take an image input to a plurality of layers, including layers (eg, FC-4096), and a softmax layer, all of which may be of any suitable configuration known in the art. Can have.

Activation from the VGG network may be obtained by content loss module 500 and style loss module 502 to calculate style and content loss. The embodiments described herein therefore define a new lossy framework for training neural networks using pre-trained networks. This helps to optimize the neural network while preserving use case critical features in the generated images.

Therefore, the embodiments described herein introduce use case dependent loss functions using a pre-trained deep learning network. Traditional techniques include methods such as bicubic interpolation and L2 losses when training deep networks, but introduce different losses during network training. For example, bicubic interpolation reduces contrast loss on sharp edges, while L2 loss over the entire image focuses on preserving all aspects of the image, but most of these preservation are described in the embodiments described herein. It is not necessary for these use cases, and you can create a loss function depending on what features you want to preserve in the image. In some such instances, content loss can be used to ensure that edges and outlines are preserved, and style loss can be used to ensure that textures, colors, and the like are preserved.

Embodiments described herein can use outputs from a pre-trained network layer to define a use case dependent loss function for training the network. If the use case is critical dimension uniformity or metrology measurement, embodiments may weight content loss, and if images are to be “beautified,” style loss may be used to preserve texture, color, etc. have. Also, if classification is important, the final layer features can be matched between the generated high resolution image and the ground truth image, and the loss in the final layer of the features of the pre-trained network can be defined, which is used for classification. Because they are features.

In signal processing, full variation noise cancellation, also referred to as full variation normalization, is the most used process in digital image processing with applications in noise removal. This is based on the principle that signals with excessive and possibly spurious details have a high overall variation, that is, the integral of the absolute gradient of the signal is high. According to this principle, reducing the overall variation of the signal through this principle to closely match the original signal eliminates unwanted details while preserving important details such as edges. This concept was pioneered by Rudin, Osher, and Fatemi in 1992 and is known today as the ROF model.

This noise cancellation technique has advantages over simple techniques such as linear smoothing or median filtering, which reduce noise and at the same time smooth some or less edges. In contrast, total variance noise cancellation is very effective at smoothing noise in flat areas while preserving edges, even at relatively low signal-to-noise ratios.

In some such embodiments, the one or more components include a tuning module configured to determine one or more parameters of the deep CNN based on the context aware loss. For example, as shown in FIG. 4, one or more components may include a tuning module 410 configured for back propagation of an error determined by a context aware loss module and / or a network parameter change. Each layer of the above-described deep CNN may have one or more parameters such as weight (W), and bias (B), the value of which may be determined by training a model that may be performed as further described herein. have. For example, the weights and biases of the various layers included in the deep CNN can be determined during training by minimizing situational awareness loss.

In one embodiment, the deep CNN is configured such that the high resolution image generated by the one or more second layers has less noise than the low resolution image. For example, the embodiments described herein provide a generalized framework for converting noisy and unresolution images into super low resolution images using learned representations.

In another embodiment, the deep CNN is configured such that the high resolution image generated by the one or more second layers retains the structural and spatial features of the low resolution image. For example, the embodiments described herein provide a generalized framework for converting noisy and unresolution images into low noise super resolution images while retaining structural and spatial fidelity using the learned representations. do.

In some embodiments, the deep convolutional neural network outputs a high resolution image at a higher throughput than that for generating a high resolution image with a high resolution imaging system. For example, the embodiments described herein can be used for deep learning based super resolution for higher throughput for electron beam tools. Therefore, the embodiments described herein may be particularly useful when it may be advantageous to use a relatively low dose (electron beam, light, etc.) for image acquisition to prevent alterations (such as damage, contamination, etc.) on the specimen. Can be. However, using relatively low doses to avoid altering the specimen generally produces low resolution images. Thus, creating high resolution images without altering the specimen is a challenge. Embodiments described herein provide this capability. In particular, sample images may be obtained with higher throughput and lower resolution (or lower image quality), and embodiments described herein may cause such higher throughput, lower quality images to not alter the sample. Can be converted to super-resolution or higher quality images (because the sample itself is not needed to produce super-resolution or high-quality images).

Therefore, the embodiments described herein are particularly useful for review use cases in which a wafer may go through an inspection (eg, BBP inspection) and an electron beam review sequence. In addition, in some cases, a user may inspect a wafer after an inspection tool to try another inspection recipe condition (e.g., to optimize the inspection recipe conditions for defects detected in the inspection and possibly classified in the review). I want to place it back in. However, if the electron beam (or other) review damages or changes the locations that were reviewed, the location is no longer valid for sensitivity analysis (ie, inspection recipe change and / or optimization). Thus, preventing damage or alteration to the specimen by using low frame average electron beam image acquisition is one of the advantages of in-depth learning based classification of electron beam review images (eg, clean high frame average images are not needed). Thus, arguably, deep learning classification and deep learning image enhancement can be used in combination. Deep learning based defect classification can be performed by the embodiments described herein as described in commonly assigned US patent application Ser. No. 15 / 697,426, filed September 6, 2017 by He et al. Which is hereby incorporated by reference as if fully set forth herein. Embodiments described herein may be further configured as described in this patent application.

7 shows examples of results that can be generated using the embodiments described herein. The results are the horizontal profile 700 of the noisy, poorly resolved high throughput image 702, the higher quality and the better resolution the horizontal profile 704 of the low throughput image 706, and We show a comparison between the horizontal profile 708 of the true super resolution image 710, obtained by the processing of the low resolution image using the embodiments described in. As described herein, high throughput image 702 and low throughput image 706 are generated by low resolution and high resolution imaging systems, respectively. In this way, the results shown in FIG. 7 represent horizontal variation between different images along the same line profile through the images. As shown in FIG. 7, as confirmed by the correlation in the profiles 708, 704 of the super resolution image generated by the embodiments described herein and the high resolution image generated by the imaging system, Demonstrate the ability of the embodiments described herein to substantially produce noise free high resolution images from low quality images while maintaining structural and spatial fidelity in the images.

In one embodiment, the one or more computer subsystems are configured to perform one or more metrology measurements on the specimen based on the high resolution images generated by the one or more second layers. 8 demonstrates that the embodiments described herein work by closing a loop with ground truth data. To further test the embodiments described herein in real world metrology use cases, overlay measurements were performed on the three sets of images shown in FIG. 7 and the results compiled in FIG. 8. Graphs 800 and 802 in FIG. 8 show correlations in overlay measurements along the x and y axes, respectively, between the high resolution image generated by the deep CNN embodiments described herein and the image generated by the high resolution imaging system, respectively. The graphs 804 and 806 in FIG. 8 show the correlations in overlay measurements along the x and y axes, respectively, between the low resolution image and the image generated by the high resolution imaging system. The metric used to calculate the correlation is R ² (r squared). The r-squared value of 1 shows a perfect fit. The near perfect R ² value (> 0.99) between the high resolution image generated by the imaging system and the high resolution image generated by the deep CNN indicates that the image generated by the deep CNN is high resolution in metrology measurements without affecting performance. It can be used instead of the images generated by the imaging system. Assuming relatively high precision is required in metrology use cases, the R ² value of ˜0.8 in the case of images produced by low resolution and high resolution imaging systems shows that the measurement is too low for accurate measurements, thus high resolution images Measurements need to be taken, which significantly lowers the use case throughput (eg, from about 18K defects per hour to about 8K defects per hour in the experiments described herein).

In another embodiment, the deep CNN functions independently of the imaging system that produced the low resolution image. In some embodiments, the low resolution image is generated by one imaging system having a first imaging platform, and the one or more computer subsystems are different specimens by another imaging system having a second imaging platform different from the first imaging platform. One or more first layers are configured to generate a representation of another low resolution image, and the one or more second layers generate a high resolution image for another sample from the representation of another low resolution image. It is configured to. For example, an important advantage of the embodiments described herein is that the same network architecture can be used to enhance images from different platforms, such as BBP tools, tools specifically configured for low resolution imaging, and the like. In addition, the overall burden on learning and optimizing representations is shifted offline since training only occurs during recipe setup. Once the training is complete, the runtime calculation is greatly reduced. The learning process also helps to adaptively enhance the images without having to change the parameters each time, as was needed in the case of conventional methods.

In one such embodiment, the first imaging platform is an electron beam imaging platform and the second imaging platform is an optical imaging platform. For example, the embodiments described herein can transform low resolution images generated using an electron beam imaging system and an optical imaging system. Embodiments described herein can also perform transformations for other different types of imaging platforms (eg, other charged particle type imaging systems).

In this other embodiment, the first and second imaging platforms are different optical imaging platforms. In this additional embodiment, the first and second imaging platforms are different electron beam imaging platforms. For example, the first and second imaging platforms may be the same type of imaging platform, but their imaging capabilities may vary greatly. In one such example, the first and second optical imaging platforms can be a laser scattering imaging platform and a BBP imaging platform. Such imaging platforms have distinctly different capabilities and will produce substantially different low resolution images. Nevertheless, the embodiments described herein can generate high resolution images for all of these low resolution images using the learned representations generated by training the deep CNN.

Another embodiment of a system configured to generate a high resolution image of a sample from a low resolution image of the sample includes an imaging subsystem configured to generate a low resolution image of the sample. The imaging subsystem can have any configuration described further herein. The system may also comprise one or more computer subsystems, eg, computer subsystem (s) 102 shown in FIG. 1, which may be configured as further described herein, and one or more components, eg, And component (s) 100 (which may include any component (s) described herein) executed by one or more computer subsystems. The component (s) include a deep CNN, eg, deep CNN 104, which may be configured as described herein. For example, the deep CNN includes one or more first layers configured to generate a representation of the low resolution image and one or more second layers configured to generate a high resolution image for the sample from the representation of the low resolution image. One or more second layers include a final layer configured to output a high resolution image. The final layer is also configured as a subpixel convolutional layer. One or more first layers and one or more second layers may be further configured as described further herein. This system embodiment may be further configured as described herein.

The embodiments described herein have many advantages as can be seen from the description provided above. For example, the embodiments described herein provide a general platform independent data centric framework. Embodiments use the training data during setup to learn the conversion between a high quality image and a low quality image. Learning this transform allows the embodiments to use the trained transform to convert a noisy and poorly resolved input into a relatively calm super resolution output with metrology quality at runtime. Previous approaches have been parametric methods that rely only on the current input image and do not utilize any other training data. Embodiments described herein are also general and platform independent. Since the embodiments are generic and platform independent, the same framework can be used to generate metrology quality images on different platforms such as electron beam, BBP, laser scattering, low resolution imaging, and metrology platform. Embodiments also enable higher throughput by using only low quality (high throughput) images to produce images of the required image quality in production. Embodiments also achieve noise reduction in the output image compared to the input image without affecting important features such as edges and contours in the images.

Each of the embodiments of each of the systems described above may be combined together in one single embodiment.

Another embodiment is directed to a computer-implemented method for generating a high resolution image of a specimen from a low resolution image of the specimen. The method includes acquiring a low resolution image of a sample. The method also includes generating a representation of the low resolution image by inputting the low resolution image into one or more first layers of the deep CNN. The method also includes generating a high resolution image of the specimen based on the representation. Generation of the high resolution image is performed by one or more second layers of the deep CNN. One or more second layers include a final layer configured to output a high resolution image, the final layer being configured as a subpixel convolutional layer. Acquiring, generating a representation, and generating a high resolution image are performed by one or more computer systems. One or more components are executed by one or more computer systems, and one or more components include a deep CNN.

Each of the steps of the method may be performed as further described herein. The method may also include the system, computer system (s) or subsystem (s) described herein, and / or any other step (s) that may be performed by the imaging systems or subsystems. have. One or more computer systems, one or more components, and a deep CNN may be any of the embodiments described herein, eg, computer subsystem (s) 102, component (s) 100, and It may be configured according to the deep CNN 104. In addition, the method described above may be performed by any of the systems described herein.

All of the methods described herein may include storing the results of one or more steps of the method embodiments in a computer readable storage medium. The results may include any of the results described herein and may be stored by any method known in the art. The storage medium may include any storage medium described herein or any other suitable storage medium known in the art. After the results have been stored, the results can be accessed from the storage medium and used by any of the method or system embodiments described herein, formatted for display to a user, and other software modules, methods, systems or the like. Can be used by. For example, the generated high resolution images may perform metrology measurements on the specimen, classify one or more defects detected on the specimen, verify one or more defects detected on the specimen, and / or one or more of the above. Based on this, the process used to form the patterned features on the specimen can be used to determine if it should be changed in some way that changes the patterned features formed on the other specimens in the same process.

A further embodiment relates to a computer readable non-transitory medium storing program instructions executable on one or more computer systems to perform a computer implemented method for generating a high resolution image of a sample from a low resolution image of the sample. One such embodiment is shown in FIG. 9. In particular, as shown in FIG. 9, computer readable non-transitory medium 900 includes program instructions 902 executable on computer system (s) 904. The computer-implemented method may include any step (s) of any method (s) described herein.

Program instructions 902 that implement methods as described herein may be stored on computer readable medium 900. The computer readable medium can be a magnetic or optical disk, or a storage medium such as a magnetic tape, or any suitable other computer readable non-transitory medium known in the art.

Program instructions may be implemented in any of a variety of ways, including procedure based techniques, component based techniques, and / or object oriented techniques, among others. For example, program instructions may be implemented using ActiveX control, C ++ objects, JavaBeans, "Microsoft Foundation Classes" (MFC), Streaming SIMD Extensions (SSE), or other techniques or methodologies as desired.

Computer system (s) 904 may be configured in accordance with any of the embodiments described herein.

Further modifications and alternative embodiments of various aspects of the invention will be apparent to those skilled in the art upon reviewing this description. For example, a method and system are provided for generating a high resolution image of a specimen from a low resolution image of the specimen. Accordingly, the description is merely illustrative and should be construed as to teach one of ordinary skill in the art a general method for carrying out the invention. It is to be understood that the forms of the invention shown and described herein are to be regarded as presently preferred embodiments. Elements and materials may be substituted in place of those described and illustrated herein, parts and processes may be reversed, and certain features of the invention may be utilized independently, all of which are described in the description of the invention. After having the benefit, it will be apparent to those skilled in the art. Changes may be made to the elements described herein without departing from the spirit and scope of the invention as set forth in the claims below.

Claims (24)

A system configured to generate a high resolution image of a specimen from a low resolution image of a specimen,
One or more computer subsystems configured to obtain low resolution images of the specimen; And
One or more components executed by the one or more computer subsystems
Including,
The one or more components,
Deep convolutional neural network
Including,
The deep convolutional neural network,
One or more first layers configured to generate a representation of the low resolution image; And
At least one second layer configured to generate a high resolution image for the sample from the representation of the low resolution image
Including;
The at least one second layer comprises a final layer configured to output the high resolution image,
And the final layer is also configured as a subpixel convolution layer to generate a high resolution image for the sample from a low resolution image of the sample. The method of claim 1,
And the deep convolution neural network is configured to generate a high resolution image for the sample from a low resolution image of the sample, wherein the high resolution image generated by the one or more second layers is configured to have less noise than the low resolution image. The method of claim 1,
The deep convolutional neural network is configured to generate a high resolution image for the sample from the low resolution image of the sample, wherein the high resolution image generated by the one or more second layers is configured to retain the structural and spatial features of the low resolution image. . The method of claim 1,
The one or more components are context aware loss modules configured to train the deep convolutional neural network.
More,
During training of the deep convolutional neural network, the one or more computer subsystems input the high resolution image generated by the one or more second layers and a corresponding, known high resolution image for the sample into the contextual awareness loss module. ,
The context aware loss module determines a context aware loss in the high resolution image generated by the one or more second layers, compared to the corresponding known high resolution image, from the low resolution image of the sample, the high resolution image for the sample. The system is configured to generate. The method of claim 4, wherein
Wherein the situational awareness loss comprises content loss, style loss, and total variation (TV) normalization. The method of claim 5,
And wherein the content loss comprises a loss in lower level features of the corresponding, known high resolution image. The method of claim 5,
And the style loss comprises a loss in one or more abstracting entities that qualitatively define the corresponding, known high resolution image. The method of claim 4, wherein
And the situational awareness loss module is configured to generate a high resolution image for the sample from a low resolution image of the sample that includes a pre-trained VGG network. The method of claim 4, wherein
The one or more components configured to determine one or more parameters of the deep convolutional neural network based on the situational awareness loss
And a high resolution image for the sample from the low resolution image of the sample. The method of claim 1,
The one or more computer subsystems are further configured to perform one or more metrology measurements on the sample based on the high resolution image generated by the one or more second layers. The system is configured to generate. The method of claim 1,
And the deep convolution neural network is configured to generate a high resolution image for the sample from the low resolution image of the sample that functions independently of the imaging system that generated the low resolution image. The method of claim 1,
The low resolution image is generated by one imaging system having a first imaging platform,
The one or more computer subsystems are also configured to obtain another low resolution image generated for another sample by another imaging system having a second imaging platform different from the first imaging platform,
The one or more first layers are configured to generate a representation of the other low resolution image,
Wherein the at least one second layer is further configured to generate a high resolution image for the sample from the low resolution image of the sample, wherein the at least one second layer is configured to generate a high resolution image for the other sample from the representation of the other low resolution image. The method of claim 12,
The first imaging platform is an electron beam imaging platform,
And the second imaging platform is an optical imaging platform configured to generate a high resolution image for the sample from a low resolution image of the sample. The method of claim 12,
And wherein the first imaging platform and the second imaging platform are different optical imaging platforms. The method of claim 12,
Wherein the first imaging platform and the second imaging platform are different electron beam imaging platforms, the system configured to generate a high resolution image for the specimen from a low resolution image of the specimen. The method of claim 1,
And the low resolution image is generated by an electron beam based imaging system. The method of claim 1,
And the low resolution image is generated by an optical based imaging system. The method of claim 1,
And the low resolution image is generated by a inspection system from a low resolution image of the sample. The method of claim 1,
And a high resolution image for the sample from the low resolution image of the sample wherein the sample is a wafer. The method of claim 1,
The system configured to generate a high resolution image for the sample from the low resolution image of the sample wherein the sample is a reticle. The method of claim 1,
The deep convolution neural network is configured to generate a high resolution image for the sample from a low resolution image of the sample, wherein the deep convolutional neural network outputs the high resolution image at a higher throughput than the throughput for generating the high resolution image with a high resolution imaging system. A system configured to generate a high resolution image for a sample from a low resolution image of a sample,
An imaging subsystem configured to generate a low resolution image of the sample;
One or more computer subsystems configured to obtain a low resolution image of the sample; And
One or more components executed by the one or more computer subsystems
Including,
The one or more components,
Deep Convolution Neural Network
Including,
The deep convolutional neural network,
One or more first layers configured to generate a representation of the low resolution image; And
At least one second layer configured to generate a high resolution image for the sample from the representation of the low resolution image
Including;
The at least one second layer comprises a final layer configured to output the high resolution image,
And the final layer is also configured as a subpixel convolution layer to generate a high resolution image for the sample from a low resolution image of the sample. A computer-readable non-transitory medium storing program instructions executable on one or more computer systems to perform a computer implemented method for generating a high resolution image of a specimen from a low resolution image of a specimen, wherein the computer implemented method comprises:
Obtaining a low resolution image of the sample;
Generating a representation of the low resolution image by inputting the low resolution image into at least one first layer of a deep convolutional neural network; And
Generating a high resolution image of the sample based on the representation
Including,
Generating the high resolution image is performed by one or more second layers of the deep convolutional neural network,
The at least one second layer comprises a final layer configured to output the high resolution image,
The final layer is also configured as a subpixel convolutional layer,
The acquiring, generating the representation, and generating the high resolution image are performed by the one or more computer systems,
One or more components are executed by the one or more computer systems,
And the one or more components comprises the deep convolution neural network. A computer-implemented method for generating a high resolution image of a sample from a low resolution image of a sample,
Obtaining a low resolution image of the sample;
Generating a representation of the low resolution image by inputting the low resolution image into at least one first layer of a deep convolutional neural network; And
Generating a high resolution image of the sample based on the representation
Including,
Generating the high resolution image is performed by one or more second layers of the deep convolutional neural network,
The at least one second layer comprises a final layer configured to output the high resolution image,
The final layer is also configured as a subpixel convolutional layer,
The acquiring, generating the representation, and generating the high resolution image are performed by one or more computer systems,
One or more components are executed by the one or more computer systems,
And the one or more components include the deep convolution neural network.

Images