Classifications

• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T7/00—Image analysis
  • G06T7/0002—Inspection of images, e.g. flaw detection
  • G06T7/0004—Industrial image inspection
  • G06T7/001—Industrial image inspection using an image reference approach
• • G—PHYSICS
  • G01—MEASURING; TESTING
  • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
  • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
  • G01N21/84—Systems specially adapted for particular applications
  • G01N21/88—Investigating the presence of flaws or contamination
  • G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
• • G—PHYSICS
  • G01—MEASURING; TESTING
  • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
  • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
  • G01N21/84—Systems specially adapted for particular applications
  • G01N21/88—Investigating the presence of flaws or contamination
  • G01N21/95—Investigating the presence of flaws or contamination characterised by the material or shape of the object to be examined
  • G01N21/9501—Semiconductor wafers
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F18/00—Pattern recognition
  • G06F18/20—Analysing
  • G06F18/24—Classification techniques
  • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
  • G06F18/2413—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on distances to training or reference patterns
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F18/00—Pattern recognition
  • G06F18/20—Analysing
  • G06F18/24—Classification techniques
  • G06F18/243—Classification techniques relating to the number of classes
  • G06F18/2431—Multiple classes
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
  • G06N20/00—Machine learning
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
  • G06N20/00—Machine learning
  • G06N20/10—Machine learning using kernel methods, e.g. support vector machines [SVM]
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
  • G06N20/00—Machine learning
  • G06N20/20—Ensemble learning
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
  • G06N3/00—Computing arrangements based on biological models
  • G06N3/02—Neural networks
  • G06N3/04—Architecture, e.g. interconnection topology
  • G06N3/0464—Convolutional networks [CNN, ConvNet]
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
  • G06N3/00—Computing arrangements based on biological models
  • G06N3/02—Neural networks
  • G06N3/08—Learning methods
  • G06N3/09—Supervised learning
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T3/00—Geometric image transformations in the plane of the image
  • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T5/00—Image enhancement or restoration
  • G06T5/73—Deblurring; Sharpening
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T7/00—Image analysis
  • G06T7/0002—Inspection of images, e.g. flaw detection
  • G06T7/0004—Industrial image inspection
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T7/00—Image analysis
  • G06T7/60—Analysis of geometric attributes
  • G06T7/62—Analysis of geometric attributes of area, perimeter, diameter or volume
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
  • G06V10/00—Arrangements for image or video recognition or understanding
  • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
  • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
  • G06V10/776—Validation; Performance evaluation
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
  • G06V20/00—Scenes; Scene-specific elements
  • G06V20/60—Type of objects
  • G06V20/69—Microscopic objects, e.g. biological cells or cellular parts
• • G—PHYSICS
  • G01—MEASURING; TESTING
  • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
  • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
  • G01N21/84—Systems specially adapted for particular applications
  • G01N21/88—Investigating the presence of flaws or contamination
  • G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
  • G01N2021/8854—Grading and classifying of flaws
• • G—PHYSICS
  • G01—MEASURING; TESTING
  • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
  • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
  • G01N21/84—Systems specially adapted for particular applications
  • G01N21/88—Investigating the presence of flaws or contamination
  • G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
  • G01N2021/8883—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges involving the calculation of gauges, generating models
• • G—PHYSICS
  • G01—MEASURING; TESTING
  • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
  • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
  • G01N21/84—Systems specially adapted for particular applications
  • G01N21/88—Investigating the presence of flaws or contamination
  • G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
  • G01N2021/8887—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges based on image processing techniques
• • G—PHYSICS
  • G01—MEASURING; TESTING
  • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
  • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
  • G01N21/84—Systems specially adapted for particular applications
  • G01N21/88—Investigating the presence of flaws or contamination
  • G01N21/95—Investigating the presence of flaws or contamination characterised by the material or shape of the object to be examined
  • G01N21/956—Inspecting patterns on the surface of objects
  • G01N2021/95676—Masks, reticles, shadow masks
• • G—PHYSICS
  • G01—MEASURING; TESTING
  • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
  • G01N2201/00—Features of devices classified in G01N21/00
  • G01N2201/12—Circuits of general importance; Signal processing
  • G01N2201/129—Using chemometrical methods
  • G01N2201/1296—Using chemometrical methods using neural networks
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
  • G06N3/00—Computing arrangements based on biological models
  • G06N3/02—Neural networks
  • G06N3/04—Architecture, e.g. interconnection topology
  • G06N3/045—Combinations of networks
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
  • G06N5/00—Computing arrangements using knowledge-based models
  • G06N5/01—Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T2207/00—Indexing scheme for image analysis or image enhancement
  • G06T2207/10—Image acquisition modality
  • G06T2207/10024—Color image
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T2207/00—Indexing scheme for image analysis or image enhancement
  • G06T2207/10—Image acquisition modality
  • G06T2207/10056—Microscopic image
  • G06T2207/10061—Microscopic image from scanning electron microscope
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T2207/00—Indexing scheme for image analysis or image enhancement
  • G06T2207/20—Special algorithmic details
  • G06T2207/20036—Morphological image processing
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T2207/00—Indexing scheme for image analysis or image enhancement
  • G06T2207/20—Special algorithmic details
  • G06T2207/20081—Training; Learning
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T2207/00—Indexing scheme for image analysis or image enhancement
  • G06T2207/20—Special algorithmic details
  • G06T2207/20084—Artificial neural networks [ANN]
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T2207/00—Indexing scheme for image analysis or image enhancement
  • G06T2207/30—Subject of image; Context of image processing
  • G06T2207/30108—Industrial image inspection
  • G06T2207/30148—Semiconductor; IC; Wafer
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
  • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
  • G06V2201/02—Recognising information on displays, dials, clocks

Definitions

• the present invention generally relates to the fields of specimen characterization and metrology and, more particularly, to a system and method for determining the type and see of defects utilizing machine learning techniques.
• defect sizes are estimated by computing the area of pixels belonging to defects in images and multiplying it by tine pixel size. While these conventional techniques may be used to determine the size of defects between approximately 80-200 nm, these conventional techniques are unable to determine the size of defects outside of this narrow range (e.g., defects smaller than 80 nm, defects larger than 200 nm). Furthermore, conventional techniques are often unable to determine the type of defect being inspected. The inability to determine defect types further limits the ability of conventional techniques to accurately determine the size of defects within 15-20% of tiie actual defect size. [ooos] Therefore, it would be desirable to provide a system and method that cure one or more of toe shortfalls of the previous approaches identified above.
• a system for characterizing a specimen is disclosed.
• the system indudes a controller including one or more processors configured to execute a set of program instructions stored in memory, the set of program instructions configured to cause the one or more processors to: receive one or more training images of one or more defects of the spetimen; generate a machine learning dessertifier based on the one or more training images; receive one or more product images of one or more defects of a specimen; determine one or more defed type classifications of foe one or more defects with the machine learning classifier; filter the one or more product images wife one or more smoothing filters; perform one or more binarization processes to generate one or more binarized product images; perform one or mare morphological image processing operations on toe one or more binarized product images; determine one or more algorithm-estimated defect sizes of the one or more defects based on the one or more binarized product images; and determine one or more refined estimates of one or more defect sizes of the one or more defects based on toe one or more algorithm-estimated defect
• toe system includes an inspection sub-system configured to acquire one or more images of a specimen.
• the system includes a controller communicatively coupled to toe inspection sub-system, toe controller configured to: receive one or more training images of one a * more defects of the specimen from the inspection sub-system; generate a machine learning classifier based on toe one or more training images; receive one or more product images of one or more defects of a specimen from toe inspection sub-system; determine one or more defect type classifications of toe one or more defects of the product images with the machine learning classifier; perform one or more morphological image processing operations on toe one or more product images; determine one or more algorithm-estimated defect sizes of toe one or more defects based on the one or more product images; and determine one or more refined estimates of one or more defect sizes of the one or more defects based on the one or more aigorifom- estimated defect sizes and foe one or more defect type classifications.
• a method for characterizing a specimen includes: acquiring one or more training images of one or more defects of a specimen; generating a machine learning classifier based cm the one or more training images; acquiring one or more product images of one or more defects of a specimen; determining one or more defect type classifications of the one or more defects with the machine learning classifier; filtering the one or more product images with one or more smoothing filters; performing one or more binarization processes to generate one or more binarized product images; performing one or more morphological image processing operations on the one or more binarized product images; determining one or more algorithm-estimated defect sizes of the one or mare defects based on the one or more binarized product images; and determining one or more refined- estimates of one or more defect sizes of the one or more defects based on the one or more algorithm-estimated defect sizes and the one or more defect type classifications.
• FIG. 1A illustrates a system for characterizing a specimen, in accordance with one or more embodiments of the present disclosure.
• FIG. 18 illustrates a system for characterizing a specimen, in accordance with one or more embodiments of the present disclosure.
• FIG. 1C illustrates a system for characterizing a specimen, in accordance with one or more embodiments of the present disclosure.
• FIG. 2 illustrates a flowchart for characterizing a specimen, in accordance with one or more embodiments of the present disclosure.
• FIG. 3 illustrates product images of various types of defects, in accordance with one or more embodiments of the present disclosure.
• FIG, 4 illustrates a review image of a defect, in accordance with one or more embodiments of the present disclosure.
• FIG. 5 is a graph illustrating relationships between algorithm-estimated defect size and refined estimates of defect size for pin-hole (PH) defects and resist dot (RD) defects, in accordance with one or more embodiments of the present disclosure.
• FIG. 6 shows a graph illustrating the classification of defects with a random forest classifier, in accordance with one or more embodiments of foe present disclosure.
• FIG. 7 shows graphs illustrating the classification of defects with deep neural networks, in accordance with one or more embodiments of foe present disclosure.
• FIG. 8A illustrates a flowchart of a portion of a method for characterizing a specimen, in accordance with one or more embodiments of the present disclosure.
• FIG. 8B illustrate® a flowchart of a portion of a method for characterizing a specimen, in accordance with one or more embodiments of foe present disclosure.
• Embodiments of die present disclosure are directed toward a system and method for determining the type and size of defects using image processing and machine learning techniques.
• embodiments of the present disclosure are directed to a system and method capable of accurately determining the size of defects within 15-20% of the actual defect size.
• FIG. t A illustrates a system 100 for characterizing a specimen, in accordance with one or more embodiments of the present disclosure.
• FIG. 1A illustrates a system 100 for determining type and size of defects on blank reticles using machine learning techniques.
• the system 100 may include, but is not limited to, one or more inspection sub-systems 102.
• the system 100 may additionally include, but is not limited to, a controller 104 including one or more processors 106, a memory 108, and a user interface 110.
• the inspection subsystem 102 may include any inspection sub-system 102 known in the art including, but not limited to, an optical-based inspection system, a charged particle-based inspection system, and the like.
• the inspection subsystem 102 may include an optical-based dark-field inspection system.
• the inspection sub-system 102 may include a scanning electron microscopy (SEM) inspection system.
• the controller 104 is communicatively coupled to the one or more inspection sub-systems 102.
• the one or more processors 106 of the controller 104 may be configured to generate one or more control signals configured to adjust one or more characteristics of the inspection sub-system 102.
• FIG. 1B illustrates a system 100 for characterizing a specimen, in accordance with one or more embodiments of the present disclosure.
• FIG. 1B illustrates a system 100 including an optical inspection sub-system 102a.
• the optical inspection sub-system 102a may indude any optical-based inspection/characterization system known in the art induding, but not limited to, an image- based metrology tool, a review tool, and the like.
• the inspection sub-system 102a may indude an optical dark-field inspection tod.
• the optical inspection sub-system 102a may indude, but is not limited to, an illumination source 112, an illumination arm 111, a collection arm 113, and a detector assembly 126.
• optical inspection sub-system 102a is configured to inspect and/or measure the specimen 120 disposed on the stage assembly 122.
• Illumination source 112 may include any illumination source known in tire art for generating illumination 101 including, but not limited to, a broadband radiation source.
• optical inspection sub-system 102a may include an illumination arm 111 configured to direct illumination 101 to tire specimen 120.
• illumination source 112 of optical inspection sub-system 102a may be configured in any orientation known in the art including, but not limited to, a dark-field orientation, a light-field orientation, and the like.
• the one or more optical elements 114, 124 may be selectably adjusted in order to configure the inspection sub-system 102a in a dark-field orientation, a bright-fiekj orientation, and the like.
• Specimen 120 may indude any specimen known in the art induding, but not limited to, a wafer, a reticle, a photomask, and the like.
• the specimen 120 may indude a blank reticle.
• spedmen 120 is disposed on a stage assembly 122 to facilitate movement of specimen 120.
• the stage assembly 122 is an actuatabie stage.
• the stage assembly 122 may include, but is not limited to, one or more translational stages suitable for selectably translating the specimen 120 along one or more linear directions (e.g., x-direction, y-direction and/or z-direction).
• the stage assembly 122 may indude, but is not limited to, one or more rotational stages suitable for selectively rotating the spetimen 120 along a rotational direction.
• the stage assembly 122 may indude, but is not limited to, a rotational stage and a translational stage suitable for selectably translating the spedmen 120 dong a linear direction and/or rotating the specimen 120 along a rotational direction. It is noted herein that the system 100 may operate in any scanning mode known in the art.
• the illumination arm 111 may include any number and type of optical components known in die art.
• the illumination arm 111 includes one or more optical elements 114, a set of one or more optical elements 115, a beam splitter 116, and an objective lens 118.
• illumination arm 111 may be configured to focus illumination 101 from the illumination source 112 onto the surface of the specimen 120.
• the arte or more optical elements 114 may include any optical elements known in the art including, but not limited to, one or more mirrors, one or more lenses, one or more polarizers, arte or more beam splitters, wave plates, and the like.
• optical inspection sub-system 102a indudes a collection arm 113 configured to collect illumination reflected or scattered from spedmen 120.
• collection arm 113 may direct and/or focus foe reflected and scattered light to one or more sensors of a detector assembly 126 via one or more optical elements 124.
• the one or more optical elements 124 may indude any optical elements known in the art induding, but not limited to, one or more mirrors, one or more lenses, one or more polarizers, one or more beam splitters, wave plates, and foe like.
• detedor assembly 126 may include any sensor and detector assembly known in the art for detecting illumination reflected or scattered from the specimen 120.
• the detector assembly 126 of the optical inspection subsystem 102 is configured to collect metrology data of toe specimen 120 based on illumination reflected or scattered from toe specimen 120. In another embodiment, the detector assembly 126 is configured to transmit collected/acquired images and/or metrology data to the controller 104.
• die controller 104 of system 100 may include one or more processors 106 and memory 108.
• the memory 108 may include program instructions configured to cause the one or more processors 106 to carry out various steps of the present disclosure.
• the program instructions are configured to cause the one or more processors 106 to adjust one or more characteristics of the optica! inspection sub-system 102 in order to perform one or more measurements of the specimen 120.
• the inspection sub-system 102 may include a charged particle-based inspection sub-system 102.
• toe inspection sub-system 102 may include an SEM characterization sub-system, as illustrated in FIG.
• FIG. 1C illustrates a system 100 for characterizing a specimen 120, in accordance with one or more embodiments of the present disclosure.
• FIG. 1C illustrates a system 100 including an SEM inspection sub-system 102b.
• the SEM inspection sub-system 102b is configured to perform one or more measurements on the specimen 120.
• the SEM inspection sub-system 102b may be configured to acquire one or more images of the specimen 120.
• the SEM inspection sub-system 102b may include, but is not limited to, electron beam source 128, one or more electron-optical elements 130, one or more electron-optical elements 132, and an electron detector assembly 134 including one of more electron sensors 136.
• the electron beam source 128 is configured to direct one or more electron beams 129 to foe spedmen 120.
• the electron beam source 128 may form an electron-optical column.
• electron beam source 128 indudes one or more additional and/or alternative electron-optical elements 130 configured to focus and/or direct the one or more electron beams 129 to foe surface of foe specimen 120.
• SEM inspection sub-system 102b indudes one or more electron-optical elements 132 configured to collect secondary and/or backscattered electrons 131 emanated from foe surface of the spedmen 120 in response to the one or more electron beams 129. It is noted herein that foe one or more electron-optical elements 130 and the one or more electron-optical elements 132 may indude any electron-optica!
• elements configured to direct, focus, and/or collect electrons including, but not limited to, one or more deflectors, one or more electron-optical lenses, one or more condenser lenses (e.g., magnetic condenser lenses), one or more objective lenses (e.g., magnetic condenser lenses), and foe like.
• the electron optical assembly of the SEM inspection sub-system 102b is not limited to the electron-optical elements depicted in FIG. 1C, which are provided merely for illustrative purposes. It is further noted that the system 100 may indude any number and type of electron-optical elements necessary to direct/focus foe one or more electron beams 129 onto the specimen 120 and, in response, collect and image the emanated secondary and/or backscattered electrons 131 onto foe electron detector assembly 134.
• the system 100 may indude one or more electron beam scanning elements (not shown).
• the one or more electron beam scanning elements may indude, but are not limited to, one or more electromagnetic scanning coils or electrostatic deflectors suitable for controlling a position of the one or more electron beams 129 relative to foe surface of the specimen 120.
• foe one or more scanning elements may be utilized to scan the one or more electron beams 129 across the specimen 120 in a selected pattern.
• secondary and/or backscattered electrons 131 are directed to one or more sensors 136 of the electron detector assembly 134.
• the electron detector assembly 134 of the SEM inspection sub-system 102 may indude any electron detector assembly known in the art suitable for detecting backscattered and/or secondary electrons 131 emanating from the surface of tee sped men 120.
• the electron detector assembly 134 indudes an electron detector array.
• the electron detector assembly 134 may indude an array of electron-detecting portions.
• each electron-detecting portion of the detector array of the electron detector assembly 134 may be positioned so as to detect an electron signal from spedmen 120 associated with one of the intident one or more electron beams 129.
• the electron detector assembly 134 may indude any type of electron detector known in the art.
• the electron detector assembly 134 may indude a micro-channel plate (MCP), a PIN or p-n junction detector array, such as, but not limited to, a diode array or avalanche photo diodes (APDs).
• MCP micro-channel plate
• APDs avalanche photo diodes
• the electron detector assembly 134 may indude a high-speed sdnti!lator or a photomultiplier tube (PMT) detector.
• PMT photomultiplier tube
• FIG. 1C illustrates the SEM inspection sub-system 102b as including an electron detector assembly 134 comprising only a secondary electron detector assembly, this is not to be regarded as a limitation of the present disclosure.
• the electron detector assembly 134 may include, but is not limited to, a secondary electron detector, a backscattered electron detector, and/or a primary electron detector (e.g., an in-column electron detector).
• SEM inspection subsystem 102 may include a plurality of electron detector assemblies 134.
• system 100 may include a secondary electron detector assembly 134a, a backscattered electron detector assembly 134b, and an in-column electron detector assembly 134c.
• the one or more processors 106 of the controller 104 are configured to analyze the output of detector assembly 126/electron detector assembly 134.
• the set of program instructions are configured to cause the one or more processors 106 to analyze one or more characteristics of specimen 120 based on images received from the detector assembly 126/electron detector assembly 134.
• the set of program instructions are configured to cause the one or more processors 106 to modify one or more characteristics of system 100 in order to maintain focus on the specimen 120 and/or the detector assembly 126/electron detector assembly 134.
• the one or more processors 106 may be configured to adjust one or more characteristics of the illumination source 112/electron beam source 128 and/or other elements of system 100 in order to focus the illumination 101 and/or one or more electron beams 129 onto the surface of the specimen 120.
• the one or more processors 106 may be configured to adjust the one or more elements of system 100 in order to collect illumination and/or secondary electrons 131 from the surface of the specimen 120 and focus the collected illumination on the detector assembly 126/electron detector assembly 134.
• the one or more processors 106 may be configured to adjust one or more focusing voltages applied to one or more electrostatic deflectors of electron beam source 128 in order to independently adjust the position or alignment of toe one or more electron beams 129 and scan the electron beams 129 across the specimen 120.
• system 100 includes a user interface 110 communicatively coupled to the controller 104.
• the user interface 110 includes a user input device and a display.
• the user input device of the user interface 110 may be configured to receive one or more input commands from a user, the one or more input commands configured to input data into system 100 and/or adjust one or more characteristics of system 100.
• the display of the user interface 110 may be configured to display data of system 100 to a user.
• the one or more processors 106 may be communicatively coupled to merriory 108, wherein the one or more processors 106 are configured to execute a set of program instructions stored on memory 108, the set of program instructions configured to cause the one or more processors 106 to carry out various functions and steps of toe present disclosure.
• toe controller 104 may be configured to: receive one or more training images 125 of one or more defects of the specimen 120 from the inspection sub-system 102; generate a machine learning classifier based on the one or more training images 125; receive one or more product images 135 of one or more defects of a specimen 120 from the inspection sub-system 102; determine one or more defect type classifications of the one or more defects with the machine learning classifier; filter the one or more product images 135 with one or more smoothing filters; perform one or more binarization processes to generate one or more binarized product images; perform one or more morphological image processing operations on the one or more binarized product images; determine one or more algorithm-estimated defect sizes of the one or more defects based on the one or more binarized product images; and determine one or more refined- estimates of one or more defect sizes of the one or more defects based on the one or more algorithm-estimated defect sizes and the one or more defect type classifications.
• Each of these steps/functions of the controller 104 will each be described in
• FIG. 2 illustrates a flowchart 200 for characterizing a specimen 120, in accordance with one or more embodiments of the present disclosure.
• FIG. 2 illustrates a flowchart 200 for determining the type and size of defects using image processing and machine learning techniques.
• flowchart 200 may be considered as a conceptual flowchart illustrating steps performed by/within the one or more processors 106 of the controller 104.
• a machine learning classifier is generated.
• the controller 104 may generate a machine learning classifier which may be used to identify types of defects within images of a specimen 120.
• foe controller 104 may be configured to acquire one or more training images 125 of a specimen 120.
• the controller 104 may be configured to receive one or more training images 125 of one or more defects of a specimen 120 from the inspection sub-system 102.
• the term“training images” may be regarded as images of defects whose type and size are known/designed/measured and that will be used as inputs to train a machine learning classifier.
• flue controller 104 may be configured to receive one or more optical training images 125 of one or more defects of toe specimen 120 from the optical inspection sub-system 102a.
• the controller 104 may be configured to receive one ex' more SEM training images 125 of toe one or more defects of the specimen 120 from toe SEM inspection sub-system 102b.
• toe training images 135 may include an optical training image 125, an SEM training image 125, and toe like.
• toe controller 104 may be configured to receive one or more training images 125 from a source other than the one or more inspection sub-systems 102.
• the controller 104 may be configured to receive one or more training images 125 of features of a specimen 120 from an external storage device and/or memory 108.
• controller 104 may be further configured to store received training images 125 in memory 108.
• the controller 104 may be configured to generate a machine learning classifier based on the one or more received training images 125.
• the controller 104 may be configured to generate the machine learning classifier via any techniques known in the art including, but not limited to, supervised teaming, unsupervised teaming, and the like.
• training images 125 may include images of defects with known sizes and/or known defect types.
• toe controller 104 may receive one or more known defect type classifications and/or one or more known defect sizes associated with the defects depicted in toe training images 125.
• the training images 125, known defect type classifications, and known defect sizes may be used as inputs to train the machine learning classifier.
• Known defect type classifications may include classifications for any type of defect known in toe art including, but not limited to, a pin-hole defect classification, a resist-dot defect classification, a scratch defect classification, a fast-localized defect classification, and toe like.
• the controller 104 may be further configured to store known defect type classifications, known defect sizes, and toe generated machine teaming classifier in memory 108.
• the machine learning classifier generated in step 202 may indude any type of machine learning algoritom/classifier and/or deep learning technique or josifier known in toe art induding, but not limited to, a random forest stoneifier, a support vector machine (SVM) banifier, an ensemble learning rougeifier, an artifidai neural network (ANN), and the like.
• SVM support vector machine
• ANN artifidai neural network
• toe machine teaming banifier may indude a deep convolutional neural network.
• toe machine teaming classifier may indude ALEXNET and/or GOOGUENET.
• toe machine learning ensembleifier may indude any algorithm, despreading, or predictive model configured to determine types of defects within images of a specimen 120. This will be discussed in further detail herein.
• a step 204 one or more product images are acquired.
• the controller 104 may be configured to receive one or more product images 135 from the inspection sub-system 102.
• product images may be used to refer to images of defects for which the type of defect and size of defect is to be determined.
• product images may be distinguished from“training images,” which may be regarded as images of defects which will be used as inputs to train a machine learning classifier.
• the product images 135 may be received from toe optical inspection sub-system 102a and/or toe SEM inspection sub-system 102b.
• the product images 135 may indude an optical product image 135, an SEM product image 135, and toe like.
• toe controller 104 may be configured to receive one or more product images 135 from a source other than the one or more inspection sub-systems 102.
• the controller 104 may be configured to receive one or mere product images 135 of a spedmen 120 from an external storage device and/or memory 108.
• FIG. 3 illustrates product images 135 of various types of defects, in accordance with one or more embodiments of toe present disclosure.
• FIG. 3 illustrates product images 135a- 135c of various types of defects captured by a dark-field inspection tool (e.g., inspection sub-system 102).
• a dark-field inspection tool e.g., inspection sub-system 102
• product image 135a illustrates a pin-hole (PH) defect or resist- dot (RD) defect
• product image 135b illustrates a fast-localized defect (FLD)
• product image 135c illustrates a scratch defect.
• images captured by a dark-field inspection tod e g., inspection sub-system 102 may be sized 32x32 pixels.
• FIG. 3 further illustrates a scale 302 which associates brighter pixels with the respective defect.
• the one or more product images 135 used to determine the size and/or type of defects of toe specimen 120 may be acquired during inspection and/or post inspection.
• controller 104 may be further configured to store received product images 135 in memory 108.
• one or more defect types of the one or more defects of the specimen 120 are determined.
• the one or more defects of the specimen 120 may include any type of defect which may be of interest throughout a specimen 120 fabrication/characterization process including, but not limited to, a pin-hole defect, a resist-dot defect, a scratch, a fast-localized defect, and the like.
• the controller 104 is configured to determine one of more defect type classifications of the cme or more defects within a product image 135 with the generated machine learning classifier. For example, the controller 104 may receive product image 135a depicting a pin-hole defect of the specimen 120.
• the controller 104 may be configured to determine the product image 135a includes a pin-hole defect, and associate the defect with a pin-hole defect type classification.
• the controller 104 may receive product image 135c depicting a scratch defect of the specimen 120, determine the product image 135c includes a scratch defect, and associate the defect with a scratch defect type classification.
• a step 208 one or more image processing operations are performed on the one or more product images 135.
• the controller 104 is configured to perform one or more image processing operations on the one or more product images
• the one or more product images 135 may include images of defects on the specimen 120 which are grayscale and are sized 32x32 pixels (as shown in FIG. 2). This relatively small image size may lead to large variations in estimated defect sizes. Accordingly, in some embodiments, the one or more image processing operations may indude image scaling operations configured to adjust a size of the product images 135. During an image scaling operation (e.g., image processing operation), the controller 104 may be configured to adjust a size of the one or more product images 135 with an image scaling operation to generate one or more scaled product images.
• an image scaling operation e.g., image processing operation
• the controller 104 may be configured to perform an image upscaling operation (e.g., image processing operation) in Oder to upscale tine product images 135 by a factor of eight to generate scaled product images with a size of 256x256 pixels.
• image scaling operations may include upscaling and/or downscaling operations configured to upscale and/or downscale tine product images by any selected factor.
• Image upscaling may result in scaled product images which have blurred boundaries around tile defects.
• the one or more image processing operations may further include one or more image sharpening operations.
• the controller may be configured to alter the one or more scaled product images with one or more image sharpening operations.
• the product images 135 a nd/or scaled product images may be altered via image sharpening operations using any techniques known in the art.
• the controller 104 may be configured to sharpen scaled product images using image filtering operation performed using a Laplacian filter.
• image sharpening operations may include one or more image filtering operations.
• the one or more product images 135 may indude images of defects on the specimen 120 which are in color and are sized 512x512 pixels.
• FIG. 4 illustrates a review image 135d of a defect, in accordance with one or more embodiments of the present disdosure.
• FIG. 4 illustrates a product image 135d of a defect captured by an optical review tool (e.g., inspection sub-system 102), with brighter pixels representing the defect.
• the product image 135d may be in color and sized 512x512 pixels.
• the one or more image processing operations carried out by the controller 104 may indude converting the one or more product images 135 from a first color space system to a second color space system.
• Color space systems may indude any color space system known in the art induding, but not limited to, a red-green-blue (RGB) color space system, and a hue-saturation-value (HSV) color space system.
• the product image 135d may be captured in an RGB color space system, and the controller 104 may be configured to convert the product image 135d into an HSV color space system. It is noted herein that the value channel of an HSV color space system may provide an improved intensity profile, and more consistent distinction of the defect pixels as compared to an RGB color space system.
• the one or more image processing operations may indude any image processing operations known in the art.
• exemplary image processing operations are provided solely for illustrative purposes, and are not to be regarded as a limitation of the present disclosure, unless noted otherwise herein.
• the one or more product images 135 are filtered with one or more smoothing filters.
• the controller 104 may be configured to filter the one or more product images 135 with one or more smoothing filters.
• the one or more smoothing filters may indude any smoothing filters known in the art induding, but not limited to, a mean filter, a Laplacian filter, a Weiner filter, a Gaussian filter, a minimum/maximum filter, a median filter, a midpoint filter, and the like.
• the controller 104 may be configured to smooth one or more product images 135 by convolving using a Gaussian kernel.
• one or more binarization processes are performed to generate one or more binarized product images.
• the controller 104 is configured to perform one or more binarization processes on the product images 135 in order to generate one or more binarized product images.
• pixels associated with defects e.g., defect pixels
• defect pixels may be identified from the background pixels using the binarization formula given by Equation 1:
• m defines the mean value of the background pixels * graylevel
• a defines the standard deviation value of the background pixels graylevel
• w defines a user provided weight (e.g., 3)
• d defines an user provided offset (e.g., 0.1)
• / defines the graylevel of the respective product image 135 at pixel location (x,y) (e.g., product image 135 filtered using one or more smoothing filters in step 210)
• b(x,y) defines the binarized image at pixel location (x,y).
• the step of converting a processed graylevel image using Equation 1 into an image with only two values the defect pixels with graylevel 1 and remaining pixels with graylevel 0 - may be referred to as binarization.
• the controller 104 may be configured to store the one or more binarized product images in memory 108.
• a defect withiri a product image 135 may appear to be brighter in the middle of the defect with a dark outline around the middle of the defect. This may be due to the optical properties of the defect and/or spedmen 120. Ih these cases, these two regions (e.g., bright center region, darker outline) may appear to be disconnected following binarization in step 212. Subsequently selecting only one of the regions as representing the defect may underestimate the defect size. For example, by selecting only the bright center region, the defect size may be underestimated. In order to bridge the gap between such disconnected regions, morphological image processing operations may be performed.
• one or more morphological image processing operations are performed,
• the controller 104 may be configured to perform one or more morphological image processing operations on the one or more product images 135 and/or one or more binarized product images.
• the one or more morphological image processing operations may indude any morphological image processing operations known in the art induding, but not limited to, a morphological dosing operation (e.g., morphological binary image closing operation), a morphological erosion operation, a morphological dilation operation, a morphological opening operation, or a morphological dosing operation, and the like.
• morphological image processing operations may be performed in order to bridge the gaps between disconnected regions of a defect resulting from binarization.
• connected component labeling is performed. After performing binarization operations and morphological image processing operations, noise and other factors may cause «mall dusters of pixels to be incorrectly labeled as defective (e.g., part of a defect) in addition to the pixel duster corresponding to the defect
• connected component labeling may be performed in order to label and select only foe pixel cluster corresponding to the defect.
• the controller 104 may be configured to perform connected component labeling by identifying and labeling each isolated pixel duster within foe binarized product images with a unique label.
• the controller 104 may be configured to identify and label one or more pixel dusters as corresponding to, or being associated with, a single defect [0068]
• toe controller 104 may be configured to identify a plurality of pixel clusters within toe one or more binarized product images, and determine the largest pixel cluster of toe plurality of pixel clusters as being assodated with the one or more defects.
• the controller 104 may then be further configured to disregard (e.g., ignore) other pixel dusters as bang attributable to noise.
• algorithm-estimated defect sizes are determined for the one or more defects.
• the controller 104 may be configured to determine one or more algorithm-estimated defect sizes of toe one or more detects based on the one or more binarized product images. Characteristics of toe binarized product images which are used to determine algorithm-estimated defect sizes may indude, but are not limited to, identified/labeled pixel dusters, maximum defect pixel graylevel values, minimum delect pixel graylevel values, and the like.
• the controller 104 may be configured to determine an algorithm-estimated defect size of toe defect based on the identified pixel duster.
• the term“algorithm-estimated defect size” may refer to the estimated size of the defect based on toe number of pixels determined to be assodated with the defect within a product image 135 (e.g., binarized product image 135) and/or other characteristics of the product image 135.
• “algorithm-estimated defect size” may be distinguished from“refined estimates of defect size,” as will be discussed in further detail herein.
• FIG. 5 is a graph 500 illustrating relationships between algorithm-estimated defect size and actual/designed defect size for pin-hole (PH) defects and resist dot (RD) defects, in accordance with one or more embodiments of the present disclosure.
• Graph 500 illustrates the relationship between actual and/or designed defect size, and algorithm- estimated defect size.
• Curve 502 illustrates the relationship between actual/designed defect size and algorithm-estimated defect size for resist-dot (RD) defects
• curve 504 illustrates the relationship between actual/designed defect size and algorithm-estimated defect size for pin-hole (PH) defects.
• graph 500 when algorithm-estimated defect size is plotted as a function of actual/designed defect size, distinct and independent trends are revealed corresponding to various defect types (e.g., curve 502 for RD defects, curve 504 for PH defects).
• graphs plotting algorithm-estimated defect size against actual/designed defect size may be constructed during supervised training of the machine learning classifier (step 202) via training images 125.
• distinct mathematical models/functions e.g., polynomial functions
• fee controller 104 may be configured to generate mathematical models/fUnctions (e.g., polynomial functions) which correlate algorithm- estimated defect sizes to actual/designed defect sizes for various types of defects.
• the controller 104 may be configured to generate a first mathematical model (e.g., mathematical function, polynomial function) which correlates algorithm-estimated defect sizes to actual/designed defect sizes for RD defects, and a second mathematical model (e.g., mathematical function, polynomial function) which correlates algorithm- estimated defect sizes to actual/designed defect sizes for PH defects.
• Generated mathematical models may be stored in memory 108.
• controller 104 may be configured to generate mathematical models/functions modeling curves 502 and 504 during supervised teaming.
• a single algorithm-estimated defect size may be indicative of varying actual/designed defect sizes, dependent upon fee type of defect at issue.
• feat algorithm-estimated defect size is not sufficient, on its own, to accurately determine fee size of defects.
• embodiments of the present disclosure are configured to utilize both algorithm-estimated defect size and determined defect type classifications in order to more accurately determine refined estimates of defect sizes. More particularly, embodiments of the present disdosure are configured to utilize both algorithm-estimated defect size, mathematical models/functions correlating fee algorithm-estimated defect sizes to actual/designed defect sizes, and models determining defect type rougeifications (e.g., random forest rougeifier model, deep convolutional neural network model and such) in order to more accurately determine refined estimates of defect sizes.
• a step 220 refined estimates of defect sizes are determined.
• the controller 104 may be configured to determine one or more refined estimated of one or more defect sizes (e.g., estimate® of true/actual defect sizes) of the one or more defects based on the one or more algorithm-estimated defect sizes (step 218) and the one or more defect type classifications (step 206).
• one or more refined estimated of one or more defect sizes e.g., estimate® of true/actual defect sizes
• the controller 104 may be configured to determine one or more refined estimates of one or more defect sizes (e.g., estimates of true/actual detect sizes) of fee one or more defects based on fee one or more algorithm-estimated defect sizes (step 218), fee one or more defect type classifications (step 206), and one or more mathematical models correlating algorithm-estimated defect sizes to actual/designed defect sizes for various types of defects.
• one or more defect sizes e.g., estimates of true/actual detect sizes
• the controller 104 may be configured to determine one or more refined estimates of one or more defect sizes (e.g., estimates of true/actual detect sizes) of fee one or more defects based on fee one or more algorithm-estimated defect sizes (step 218), fee one or more defect type classifications (step 206), and one or more mathematical models correlating algorithm-estimated defect sizes to actual/designed defect sizes for various types of defects.
• a machine learning ensembleifier may be trained/calibrated using supervised learning techniques.
• fee controller 104 may be configured to generate mathematical models/functions (e.g., polynomial functions) which correlate algorithm-estimated defect sizes to actual/designed defect sizes for various types of defects (e.g., mathematical functions modeling curves 502, 504).
• the controller 104 may generate a first polynomial function (e.g., first mathematical model) associated with curve 502, and a second polynomial function (e.g., a second mathematical model) associated with curve 504.
• the polynomial functions e.g., first mathematical model, second mathematical model
• the polynomial functions may be stored in memory 108.
• the controller 104 may be configured to acquire a product image 135a of a defect. Using the trained machine learning classifier, the controller 104 may determine the defect pictured in product image 135a is a pin-hole defect, and may therefore associate a pin-hole defect classification with foe defect. Subsequently, after performing various steps of flowchart 200, foe controller 104 may subsequently determine an algorithm-estimated defect size of the defect using the second polynomial function (e.g., second mathematical model modeling to curve 504 for pin-hole detects). Using foe determined pin-hole defect classification, foe second polynomial function, and the algorithm-estimated defect size, the controller 104 may then be configured to determine a refined estimate of a defect size estimating the true/actua! size of the defect.
• the second polynomial function e.g., second mathematical model modeling to curve 504 for pin-hole detects
• FIG. 6 shows a graph 600 illustrating the classification of defects with a random forest classifier, in accordance with one or more embodiments of the present disclosure.
• FIG. 7 shows graphs 700, 702 illustrating the classification of defects with deep neural networks, in accordance with one or more embodiments of the present disclosure.
• the controller 104 may be configured to generate a three-dimensional (3D) feature vector based on the features that include algorithm-estimated defect size, the minimum defect pixel graylevel value, and the maximum defect pixel graylevel value within a binarized product image.
• the features are chosen to be representative of different defect types and are not exhaustive.
• three distinct product images 135 e.g., binarized product images
• a random forest classifier using nine features may provide a defect type classification with an accuracy of approximately 100% on the test data (e.g., test defects), as may be seen in graph 600.
• graph 700 and 702 illustrate the classification of defects using a deep neural network trained with original product images 135.
• graph 700 illustrates the classification of defects with ALEXNET
• graph 702 illustrates the classification of defects with GOOGLENET.
• the controller 104 may be further configured to generate control signals based on at least one of a refined estimate of a defect size or a determined defect type classification, where the one or more control signals are configured to selectively adjust one or more characteristics of one or more process tools.
• the system 100 may further include one or more fabrication tools communicatively coupled to the controller 104.
• the one or more fabrication tools may include any fabrication tool known in the art configured to fabricate a specimen 120 including, but not limited to, a lithography tool, an etching tool, a deposition tool, a polishing tool, and the like.
• the controller 104 may be configured to generate one or more control signals configured to adjust one or more characteristics of one or more fabrication tools in a feed-forward or a feed-back loop in order to correct at least one of a refined estimate of a defect size or a determined defect type classification.
• the system and method of the present disclosure may enable more accurate defect type and size determinations for a wide range of defect sizes (e.g., smaller than 80 nm, greater than 200 nm).
• the system and method of the present disclosure may enable estimation of defect size to within 15-20% of the actual defect size (e.g., refined estimates of defect sizes within 15-20% of the actual defect size).
• the one or more components of system 100 may be communicatively coupled to the various other components of system 100 in any manner known in the art.
• the one or more processors 106 may be communicatively coupled to each other and other components via a wireline (e.g., copper wire, fiber optic cable, and the like) a- wireless connection (e.g., RF coupling, IR coupling, WiMax, Bluetooth, 3G, 4G, 4G LTE, 5G, and the like).
• fee controller 104 may be communicatively coupled to one or more components of inspection subsystem 102 via any wireline or wireless connection known in fee art.
• the one or more processors 106 may include any one or more processing elements known in the art. In this sense, the one or mane processors 106 may include any microprocessor-type device configured to execute software algorithms and/or instructions. In one embodiment, the one or more processors 106 may consist of a desktop computer, mainframe computer system, workstation, image computer, parallel processor, or other computer system (e.g., networked computer) configured to execute a program configured to operate the system 100, as described throughout the present disclosure. It should be recognized that the steps described throughout the present disclosure may be carried out by a angle computer system or, alternatively, multiple computer systems.
• foe term“processor” may be broadly defined to encompass any device having one or more processing elements, which execute program instructions from memory 108.
• different subsystems of foe system 100 e.g., illumination source 112, election beam source 128, detector assembly 126, electron detector assembly 134, controller 104, user interface 110, and the like
• the memoiy 108 may indude any storage medium known in Ihe art suitable for storing program instructions executable by the associated one or more processors 106 and the data received from the inspection sub-system 102.
• the memory 108 may indude a non-transitory memory medium.
• the memory 108 may indude, but is not limited to, a read-only memory (ROM), a random-access memory (RAM), a magnetic or optical memory device (e.g., disk), a magnetic tape, a solid-state drive and the like. It is further noted that memory 108 may be housed in a common controller housing with foe one or more processors 106.
• the memory 108 may be located remotely with respect to the physical location of foe processors 106, controller 104, and the like. In another embodiment, the memory 108 maintains program instructions for causing the one or more processors 106 to carry out the various steps described through the present disclosure.
• a user interface 110 is communicatively coupled to the controller 104.
• foe user interface 110 may indude, but is not limited to, one or more desktops, tablets, smartphones, smart watches, or foe like.
• the user interface 110 indudes a display used to display data of the system 100 to a user.
• the display of the user interface 110 may indude ariy display known in foe art.
• foe display may indude, but is not limited to, a liquid crystal display (LCD), an organic light-emitting diode (OLED) based display, or a CRT display.
• LCD liquid crystal display
• OLED organic light-emitting diode
• a user may input selections and/or instructions responsive to data displayed to the user via a user input device of toe user interface 110.
• FIGS. 8A-8B illustrate a flowchart of a method 800 fey characterizing a specimen 120, in accordance with one or more embodiments of the present disclosure.
• FIGS. 8A-8B illustrate a method 800 for determining type and size of defects of a specimen 120 using machine learning techniques. It is noted herein that the steps of method 800 may be implemented all or in part by system 100. It is further recognized, however, that tine method 800 is not limited to the system 100 in that additional or alternative system-level embodiments may carry out all or part of toe steps of method 800.
• a step 802 one or more training images of one or more defects of a specimen are acquired.
• the controller 104 may be configured to receive one or rnore optical training images 125 of one or more features of the specimen 120 from the optical inspection sub-system 102a.
• toe controller 104 may be configured to receive one or more SEM training images 125 of toe one or more features of the specimen 120 from the SEM inspection sub-system 102b.
• a machine learning classifier is generated based on the one or mere training images.
• the one or more training images 125 and known defect sizes and/or known defect types may be used as inputs to bain the machine learning classifier.
• the machine learning classifier may include any type of machine learning algorithm/classifier and/or deep learning technique or classifier known in toe art including, but not limited to, a random forest classifier, a support vector machine (SVM) classifier, an ensemble learning classifier, an artificial neural network (ANN), a deep neural network or convolutional neural network (e.g., ALEXNET, GOOGLENET), and the like.
• a step 806 one or more product images of one or more defects of a specimen are acquired.
• foe controller 104 may be configured to receive one or more optical product images 135 of one or more features of foe specimen 120 from the optical inspection sub-system 102a.
• foe controller 104 may be configured to receive one or more SEM product images 135 of foe one or more features of the specimen 120 from the SEM inspection sub-system 102b.
• one or more defect type classifications of the one or more defects are determined with the machine learning classifier.
• the controller 104 may receive product image 135a depicting a pin-hole defect of the specimen 120.
• the controller 104 may be configured to determine the product image 135a includes a pin-hole defect, and associate the defect with a pin-hole defect classification.
• the controller 104 may receive product image 135c depicting a scratch defect of the specimen 120, determine the product image 135c includes a scratch defect, and associate the defect with a scratch defect classification.
• a step 810 foe one of more product images are filtered with one or more smoothing filters.
• the one or more smoothing filters may include any smoothing filters known in the art including, bid not limited to, a mean filter, a Lapiacian filter, a Weiner filter, a Gaussian filter, a minimum/maximum filter, a median filter, a midpoint filter, and the like.
• foe controller 104 may be configured to smooth one or more product images 135 by convolving using a Gaussian kernel.
• a step 812 one or more binarization processes are performed to generate one CM * more binarized product images.
• the controller 104 may be configured to perform one or more binarization processes on the product images 135 in order to generate one or more binarized product images.
• pixels associated with defects e.g., defect pixels
• a step 814 one or more morphological image processing operations are performed on the one or more binarized product images.
• the controller 104 may be configured to perform one or more morphological image processing operations on the one or more product images 135 and/or one or more binarized product images.
• the one or more morphological image processing operations may include any morphological image processing operations known in the art including, but not limited to, a morphological closing operation (e.g., morphological binary image closing operation), a morphological erosion operation, a morphological dilation operation, a morphological opening operation, or a morphological closing operation, and the like.
• morphological image processing operations may be performed in order to bridge the gaps between disconnected regions of a defect resulting from binarization.
• one or more algorithm-estimated defect sizes of the one or more defects are determined based on the one or more binarized product images.
• the controller 104 may be configured to determine one or more algorithm-estimated defect sizes of the one or more defects based on the one or more binarized product images.
• Characteristics of the binarized product images which are used to determine algorithm- estimated defect sizes may indude, but are not limited to, identified/labeled pixel dusters, maximum defect pixel graylevel values, minimum defect pixel gray!evei values, and the like.
• one or more refined estimates of one CM- more defect sizes of the one or more defects are determined based on the one or more algorithm-estimated defect sizes and the erne or more defect type classifications.
• the controller 104 may be configured to generate mathematical models (e.g., polynomial functions) which correlate algorithm-estimated defect sizes to actual/designed defect sizes for various types of defects with known characteristics (&g,, via training images 125 with defects of known/designed size).
• the controller 104 may be configured to determine a pin-hole defect classification associated with the defect, and determine an algorithm-estimated defect size.
• the controller 104 may then be configured to determine a refined estimates of a defect size of toe defect based on toe determined defect type classification, toe generated mathematical model, and toe determined algorithm-estimated defect size.
• All of the methods described herein may include storing results of one or more steps of the method embodiments in memory.
• the results may include any of the results described herein and may be stored in any manner known in the art.
• the memory may include any memory described herein or any other suitable storage medium known in foe art.
• foe results can be accessed in the memory and used by any of the method or system embodiments described herein, formatted for display to a user, used by another software module, method, or system, and foe like.
• the results may be stored“permanently,”“semi-permanently,” temporarily,” or for some period of time.
• foe memory may be random access memory (RAM), and the results may not necessarily persist indefinitely in foe memory.
• each of the embodiments of the method described above may include any other step(s) of any other method(s) described herein.
• each of the embodiments of the method described above may be performed by any of the systems described herein.
• any two components so associated can also be viewed as being “connected,” or “coupled,” to each other to achieve the desired functionality, and any two components capable of being so associated can also be viewed as being “couplable,” to each other to achieve the desired functionality.
• Specific examples of couplable indude are not limited to physically mateable and/or physically interacting components and/or wirelessly interactable and/or wirelessly interacting components and/or logically interacting and/or logically interactable components.

Landscapes

• Engineering & Computer Science (AREA)
• Theoretical Computer Science (AREA)
• Physics & Mathematics (AREA)
• General Physics & Mathematics (AREA)
• Computer Vision & Pattern Recognition (AREA)
• Software Systems (AREA)
• Data Mining & Analysis (AREA)
• Artificial Intelligence (AREA)
• Evolutionary Computation (AREA)
• Life Sciences & Earth Sciences (AREA)
• General Engineering & Computer Science (AREA)
• General Health & Medical Sciences (AREA)
• Health & Medical Sciences (AREA)
• Computing Systems (AREA)
• Mathematical Physics (AREA)
• Medical Informatics (AREA)
• Multimedia (AREA)
• Molecular Biology (AREA)
• Biomedical Technology (AREA)
• Quality & Reliability (AREA)
• Chemical & Material Sciences (AREA)
• Pathology (AREA)
• Analytical Chemistry (AREA)
• Biochemistry (AREA)
• Immunology (AREA)
• Computational Linguistics (AREA)
• Bioinformatics & Computational Biology (AREA)
• Evolutionary Biology (AREA)
• Bioinformatics & Cheminformatics (AREA)
• Biophysics (AREA)
• Databases & Information Systems (AREA)
• Geometry (AREA)
• Signal Processing (AREA)
• Investigating Materials By The Use Of Optical Means Adapted For Particular Applications (AREA)
• Image Analysis (AREA)
• Analysing Materials By The Use Of Radiation (AREA)
• Image Processing (AREA)

Priority Applications (5)

Applications Claiming Priority (4)

Publications (1)

Family

ID=70458860

Family Applications (1)

Country Status (7)

Cited By (2)

Families Citing this family (33)

Citations (5)

Family Cites Families (27)

• 2019
  • 2019-09-17 US US16/572,971 patent/US11468553B2/en active Active
  • 2019-10-09 TW TW108136515A patent/TWI805857B/zh active
  • 2019-11-01 KR KR1020217016889A patent/KR20210069736A/ko not_active Ceased
  • 2019-11-01 IL IL282249A patent/IL282249B2/en unknown
  • 2019-11-01 KR KR1020237034866A patent/KR20230147764A/ko active Pending
  • 2019-11-01 JP JP2021523881A patent/JP7355817B2/ja active Active
  • 2019-11-01 WO PCT/US2019/059291 patent/WO2020092856A1/en not_active Ceased
  • 2019-11-01 CN CN201980072399.5A patent/CN112955732B/zh active Active

Patent Citations (5)

Also Published As

Similar Documents

Legal Events