US11168976B2 - Measuring device for examining a specimen and method for determining a topographic map of a specimen - Google Patents

Measuring device for examining a specimen and method for determining a topographic map of a specimen Download PDF

Info

Publication number
US11168976B2
US11168976B2 US16/951,612 US202016951612A US11168976B2 US 11168976 B2 US11168976 B2 US 11168976B2 US 202016951612 A US202016951612 A US 202016951612A US 11168976 B2 US11168976 B2 US 11168976B2
Authority
US
United States
Prior art keywords
image
height
regions
measurement data
overview
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
US16/951,612
Other languages
English (en)
Other versions
US20210156669A1 (en
Inventor
Manuel Amthor
Daniel Haase
Dominik STEHR
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Carl Zeiss Microscopy GmbH
Original Assignee
Carl Zeiss Microscopy GmbH
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Carl Zeiss Microscopy GmbH filed Critical Carl Zeiss Microscopy GmbH
Assigned to CARL ZEISS MICROSCOPY GMBH reassignment CARL ZEISS MICROSCOPY GMBH ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: AMTHOR, Manuel, STEHR, DOMINIK, DR., HAASE, DANIEL, DR.
Publication of US20210156669A1 publication Critical patent/US20210156669A1/en
Application granted granted Critical
Publication of US11168976B2 publication Critical patent/US11168976B2/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
    • G01B11/00—Measuring arrangements characterised by the use of optical techniques
    • G01B11/24—Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/60—Analysis of geometric attributes
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
    • G01B11/00—Measuring arrangements characterised by the use of optical techniques
    • G01B11/02—Measuring arrangements characterised by the use of optical techniques for measuring length, width or thickness
    • G01B11/06—Measuring arrangements characterised by the use of optical techniques for measuring length, width or thickness for measuring thickness ; e.g. of sheet material
    • G01B11/0608—Height gauges
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
    • G01B11/00—Measuring arrangements characterised by the use of optical techniques
    • G01B11/24—Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures
    • G01B11/25—Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures by projecting a pattern, e.g. one or more lines, moiré fringes on the object
    • G06K9/3233—
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/60—Analysis of geometric attributes
    • G06T7/62—Analysis of geometric attributes of area, perimeter, diameter or volume
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/60—Type of objects
    • G06V20/69—Microscopic objects, e.g. biological cells or cellular parts
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10056—Microscopic image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30108—Industrial image inspection
    • G06T2207/30141—Printed circuit board [PCB]

Definitions

  • the present disclosure relates to a measuring device/apparatus for examining a sample or specimen, and a method for determining a height map/topographic map of a sample.
  • a generic measuring apparatus for examining a sample for example a microscope or a coordinate measuring machine, comprises an examination device for examining the sample.
  • the examination device can comprise an objective or a tactile measuring head, in particular.
  • a generic measuring apparatus comprises an overview device which is configured to record raw height measurement data of the sample and generate an overview image of the sample.
  • a computing device of the measuring apparatus is configured to calculate height measurement data from the raw height measurement data.
  • a generic method for determining a height map of a sample includes at least the following steps: receiving height measurement data of the sample and receiving an overview image of the sample.
  • a comparatively small distance between the sample and the examination device is often needed in order to precisely examine a sample.
  • the sample should often only have a distance of a few millimetres from an objective of the examination device.
  • certain knowledge about this height structure is useful to prevent a collision between a component of the examination device and the sample.
  • Height measurement data and/or an overview image are obtained in advance, in particular for this purpose.
  • the height measurement data and/or the overview image can be used to set a relative position between the sample and the examination device. Therefore, a general objective consists of being able to obtain height measurement data that are as precise as possible and/or an overview image of the sample that is as precise as possible using the overview device.
  • samples to be examined can have regions which frequently lead to erroneous height measurement data or a lack of height measurement data in the case of typical measurement techniques. This can be the case, in particular, in the case of mirroring or transparent regions, where measurement light is not reflected in the way required for the measurement principle. If a sample region is covered by other objects, height measurement data are missing or perhaps the height measurement data of the adjacent object are incorrectly captured for this sample region. There is a danger of overlooking sample regions with a small extent. Erroneous height measurement data may also arise in sample regions with little contrast or structure.
  • An object of the invention can be considered that of specifying a measuring apparatus and a method which are able to generate a height map with the highest possible quality.
  • the computing device is configured to:
  • a height map of the sample can be formed with the supplemented or altered height measurement data.
  • certain objects or object types can be identified as such in an overview image using the invention.
  • Context information which relates to the height of the identified image regions, is ascertained from knowledge of the objects or object type. Then, a height measurement map or the height measurement data for generating a height measurement map can be corrected or supplemented with the aid of this information obtained from the overview image.
  • a certain image region can be identified as a shadow or a mirroring object part, for which no height measurement data or no plausible height measurement data were obtained. Now, height measurement data for the shadow or mirroring image region are not simply supplemented by virtue of the height measurement values for all adjacent image parts being interpolated.
  • the context information can specify object boundaries, for example, such that only height measurement values of adjacent image parts which also belong to the same object are used in the case of mirroring for the purposes of supplementing height values by interpolation.
  • height measurement values of other adjacent image parts which are located outside of the object boundary with the mirroring, are not used for the interpolation of height measurement values.
  • the context information can specify where a shadow-casting object is and where there is a substrate on which said shadow is cast.
  • the context information can specify the adjacent image parts from which the height measurement values are used to fill height values of the image region of the shadow.
  • obtaining height measurement data of the sample can include receiving height measurement data from an overview device or else loading height measurement data, which were recorded at an earlier point, from a memory.
  • obtaining height measurement data can include receiving height measurement data from a computing device, which calculates the height measurement data from measured raw data. Height measurement data can also be referred to as height profile measurement data or 3D reconstruction data.
  • the method according to the invention also includes the recording of raw height measurement data using an overview device.
  • the overview device can be designed in any way for recording tactile raw measurement data or optical raw measurement data for a contactless capture of the topography of a sample.
  • the overview device can comprise one or more cameras or camera chips. Then, the height measurement data are obtained from the raw measurement data of the camera(s).
  • One or more cameras may be preferred, depending on the measurement principle.
  • a single camera suffices for a pattern projection, within the scope of which, for example, light stripes or light point patterns are guided onto the sample, for example according to a SLAM (simultaneous localization and mapping) method. If images are recorded by a plurality of cameras directed on the sample from different observation angles, height measurement values can be obtained from differences between these images without special lighting being required.
  • a detector or a camera of the overview device can also be identical to a detector/a camera of the examination device.
  • a lateral measurement region of the detector/the camera is larger when used as an overview device than a lateral measurement region of the detector/the camera when used as an examination device.
  • different objectives can be used to this end, wherein the examination device uses an objective with a greater imaging scale than the overview device.
  • the measuring apparatus can be a coordinate measuring machine or a microscope, for example an electron or ion microscope or a light microscope, wherein, in principle, any spectral range from infrared light to UV light or else to x-rays comes into consideration for illumination purposes.
  • the invention is not restricted to specific fields of use and, as a result, the sample can be any object in principle.
  • the invention can be used in measurement inspection, in manufacturing monitoring or in the examination of microscopic samples, in particular in the life sciences or in chip testing.
  • the determination of a height map according to the invention by means of the overview device allows a faster capture and hence a greatly reduced throughput time.
  • an optical triangulation line sensor or a chromatic white light sensor or a confocal (white light) sensor in which foci of different light wavelengths are generated at different heights as a result of a chromatic longitudinal aberration.
  • Laser triangulation, interferometry and a focus variation in the height direction are also possible.
  • no illumination light source is required for recording the raw height measurement data and/or the overview image.
  • Use can also be made of stereo or multi-camera systems, in which height measurement data are derivable from differences in the plurality of recorded images. The cameras can observe the sample from different angles and/or supply various raw height measurement data by way of different focal lengths. Combinations of the aforementioned options can also be used as an overview device.
  • the examination device can operate in optical or tactile fashion and, for example, be formed by a coordinate measuring machine.
  • the latter can comprise a tactile roughness sensor or a differential transformer, in which a path measurement is implemented with the aid of a linear variable differential transformer (LVDT).
  • LVDT linear variable differential transformer
  • the overview image can be formed or calculated from or with the aid of the height measurement data or raw height measurement data.
  • the overview image and the height measurement data can therefore be recorded, at least in part, by way of the same measuring device.
  • measuring devices that are separated from one another can record the raw height measurement data and the overview image.
  • the overview image can be recorded using an overview camera, which is present in addition to a camera for recording the raw height measurement data, or, alternatively, this overview camera can also serve to record raw height measurement data.
  • this overview camera can also serve to record raw height measurement data.
  • the overview image can be calculated from one or more raw images recorded by an overview camera.
  • a plurality of laterally offset raw images can be combined to form an overview image (image stitching).
  • the overview image is spatially calibrated relative to the height measurement data.
  • a field of view of an overview camera can be spatially calibrated relative to components of the overview device which record the height measurement data.
  • the height measurement data with which a pixel in the overview image is related is known, possibly with knowledge or assumption of a distance between the sample and the overview camera or remaining components of the overview device. This renders it possible to deduce an associated position in the height map from a localization of an identified image region. If height information is obtained with the aid of the context information relating to an identified image region, the height measurement data which should be replaced or altered thereby are consequently known.
  • the relative position of a height map which is calculated with the aid of the overview device, is known relative to the examination device.
  • the height map is used to set a relative position between the sample and the examination device.
  • the height information of the height map serve to set the distance between the examination device and the sample.
  • the sample can be examined by means of the examination device at the set relative position.
  • the aforementioned height map is created in inventive fashion on the basis of the height measurement data that were supplemented or altered with the aid of the overview image.
  • Context information from which height information is obtained, is ascertained for the image regions identified in the overview image.
  • This can be direct/absolute height data, by means of which height measurement data can be replaced or supplemented.
  • the height information can directly specify the height.
  • the height information obtained with the aid of the context information can specify relative height information, for example the known height of an identified assembly relative to the unknown height of a substrate.
  • relative height information can specify a height of the object of the identified image region relative to an adjacent image region, in particular the fact that an identified mirroring area has the same height as a certain adjacent image region.
  • the certain image regions identified in the overview image can comprise critical image regions, which are one or more of the following: shadows, mirroring, transparent regions and/or highly absorbing regions. Little measurement light is reflected in the case of highly absorbing regions, and so optical measurements often supply no height measurement data or erroneous height measurement data in respect of these regions.
  • a region whose pixel brightness in the overview image lies below a specified threshold can be defined to be highly absorbing.
  • a critical image region to be identified can also be present if an object below a specified minimum size is identified: In this case, there is a high probability that no reliable height measurement data are captured, and so height information for this critical image region is derived from the overview image and supplemented in the height map.
  • the identification of certain image regions in the overview image can include a localization of these image regions, for example in the form of a list of found object forms or by virtue of a certain value being assigned to each image pixel, with each found object being assigned a different value.
  • the derivation of context information in relation to localized image regions can include an ascertainment of the type of these image regions.
  • ascertaining the type can specify whether, or what type of, critical image region is present, for example a shadow, mirroring or a transparent object.
  • the ascertained type can also be referred to as semantic of the image region.
  • the next step can be to identify (by way of an image analysis) which image regions adjacent thereto correspond to a shadow-causing object and which adjacent image regions correspond to a substrate. Height information for the “shadow” type image region can now be derived from the adjacent image regions corresponding to a substrate (and not the shadow-causing object).
  • height information for the “shadow” type image region can be derived with the aid of height information that is available in respect of the shadow-causing object type.
  • a shape of the shadow-causing object may be known, from which the height of the shadowed region may be derivable, for example if a protruding electronic assembly casts a shadow and it is known that a component (e.g., a contact pin) associated with this electronic assembly is situated next to the assembly at a point that is in the shadows.
  • height information of the shadowed region is obtained neither by averaging/extrapolating height measurement data of the shadow-casting object nor by averaging/extrapolating height measurement data of the substrate. Rather, height information is derived from the object type identified.
  • the height measurement data corresponding to this image region may be included in the height map. This is based on the assumption that if height measurement data can even be obtained from a mirroring sample region, these will be falsified height measurement data.
  • context information in relation to this mirroring region or adjacent/surrounding image regions is in addition obtained from the overview image, said context information allowing height data to be supplemented.
  • the context information can specify, for example, that an object type known in advance has been identified and the mirroring area is level with other parts of the identified object or deviates in a known manner from the height of the other parts of the identified object.
  • information improving the height measurement data is derived from the overview image.
  • the presence of certain objects is identified but that no height information can be derived for these objects from the overview image. It may be helpful in this case if information about the presence of an object is output and optionally used further in automated fashion. Provision can be made for an initial assessment to be made on the basis of context information or the type of identified image region as to whether height information for this image region is derivable from the overview image.
  • an alert can be output that no height information is derivable for this image region. The alert can either be output to a user, for example by way of a representation on a screen, or be transmitted as a signal to a computing device, in particular in order to carry out further controls in automated fashion, as will be described in more detail below.
  • image analysis methods can search for distortions in the image of a structured object (e.g., a checkerboard pattern) which are characteristic for lenses. If such an image distortion is found, it is possible to deduce the presence of a transparent lens or an optically active element. Possibly, a radius of curvature of the entire object surface can be derived from the distortion.
  • a structured object e.g., a checkerboard pattern
  • the alert can accordingly specify that an object is present but that its height is unknown.
  • an alert can be output in the case of a small unidentified object, the small size of which below a specified threshold making it probable that no height measurement data will be obtained in this respect.
  • a different function of the measuring apparatus can be started or influenced on the basis of the alert.
  • this can relate to a safety margin which part of the examination device (e.g., a microscope objective) must maintain with respect to the sample.
  • the safety margin can be increased in the case of an alert (in comparison with the case of no alert), and so a greater safety margin is also chosen when there is a greater uncertainty in respect of the sample height.
  • a safety margin to be observed by the examination device is preferably increased in a sample region corresponding to the aforementioned image region.
  • the safety margin is increased in comparison with the remaining sample regions. What this ensures is that a part of the examination device (e.g., an objective) may be situated very close to the sample in sample regions whose height is known comparatively reliably while a larger safety margin is to be maintained in other sample parts, the height of which is known less reliably or not at all.
  • the height measurement data can also be assessed according to the locations at which erroneous data are likely, and so height information is subsequently derived in a targeted manner for these locations from the overview image.
  • the already described identification of certain image regions in the overview image now includes an identification of image regions spatially corresponding to the problem regions, in particular an identification of a semantic or an object type by means of image analysis.
  • the overview image or overview image parts is/are classified, in particular using a trained machine learning algorithm.
  • a classification can specify a sample class with a mirroring surface, for instance a printed circuit board or electronics with mirroring components, e.g., capacitor regions or electrical contacts.
  • the larger unit or sample type this relates to is also ascertained. Then, whether height information for a certain image region is ascertainable from other image regions (and, if so, from which image regions) is derived from the classification and the ascertainment of the type of the certain image region. Should this be the case, height information is derived and used for the height map made of the height measurement values.
  • a quality measure to be calculated for each point of the height map in order to identify sources of errors such as noises, reflections, dirtying, an incorrect calibration, etc.
  • the quality measure expresses the level of correspondence between height information derived from context information in relation to an identified object and a height measurement value recorded in this respect.
  • Checking the plausibility of height measurement data can optionally be carried out as follows: Height information for certain image regions or all image regions are estimated from the overview image.
  • the height information can specify absolute height values, relative height values relative to other sample regions, permissible height value ranges for individual sample regions or a global permissible height value range for all height measurement values.
  • the plausibility of the height measurement data is checked with the aid of the height information from the overview image. If individual height measurement data deviate too strongly from the height information or height values obtained therewith (for example, by more than a given threshold), it is possible to output a signal that these height measurement data are implausible.
  • the information can be indicated to the user, a graphical correction tool can be displayed for the user for manual correction of the height measurement values (with values classed as implausible being marked) or there can be an automatic correction, in particular by means of the height information obtained from the overview image.
  • the overview image can also be composed by a plurality of partial images or can be formed by a plurality of partial images that are separately available. It is not mandatory for the plurality of partial images to be combined to form an image for the functionality of the invention; rather, these can also be analysed, separately in each case, in the manner described.
  • the disclosed steps in respect of the overview image can be implemented with the aid of image processing algorithms.
  • the computing device can comprise corresponding image processing algorithms or can be configured to carry out the latter.
  • the computing device can be formed by a computer situated next to the overview device or, alternatively, at least in part by a remote server or computer. Described method steps can also be carried out by software which is stored in the computing device or a memory connected therewith.
  • the software can comprise machine learning algorithms, in particular deep learning methods (convolutional neural networks) for the image analysis.
  • Respective dedicated machine learning algorithms can also be used for classification, segmentation, and object detection. These can be trained by supervised or unsupervised learning methods.
  • a training data record is annotated in the case of supervised learning methods; by way of example, sample types are classified or critical surfaces are marked.
  • Critical regions can be segmented in the training data and assigned to a type, e.g., mirroring, transparent or strongly absorbing.
  • the specification in respect of the type of image regions or critical regions that should be identified in images can consequently be taught to a machine learning algorithm in a training procedure.
  • An algorithm can also be taught to detect reflection in the overview image or in the calculated height map by way of appropriate training data.
  • an algorithm can be taught using training data (measurement data or simulated data) showing expediently reflective sample regions. Subsequently, mirroring or transparent regions (not contained in the training data) supply a non-typical result, from which the taught algorithm can deduce the presence of a critical region.
  • image regions for example an ascertainment of type of identified image regions, should be understood to mean that these apply to at least some of the image regions (and optionally to all image regions). If there is an ascertainment of type for identified image regions, this can accordingly mean that the type of at least one of the image regions is ascertained while there can also be other image regions for which no type is ascertained or no type can be ascertained.
  • the measuring apparatus in particular the computing device thereof, can also be configured to carry out the described method variants.
  • FIG. 1 shows a schematic illustration of an exemplary embodiment of a measuring apparatus of the invention
  • FIG. 2 shows a schematic illustration of a further exemplary embodiment of a measuring apparatus of the invention
  • FIG. 3A shows an overview image of a sample
  • FIG. 3B shows a height map of the sample
  • FIG. 3C shows an image processing result for the overview image of FIG. 3A ;
  • FIG. 3D shows the height map of FIG. 3B after altering or supplementing height values
  • FIG. 4A shows an overview image of a sample
  • FIG. 4B shows a height map of the sample
  • FIG. 4C shows an image processing result for the overview image of FIG. 4A .
  • FIG. 5 shows a flowchart for explaining a method according to the invention.
  • FIG. 1 shows an exemplary embodiment of a measuring apparatus 100 according to the invention which is designed as a microscope in this case and which comprises an overview device 30 .
  • the overview device 30 should measure the height of a sample 10 in order to determine a height map of said sample 10 .
  • a height map should be understood to mean that a respective height value is captured for different laterally offset surface regions of the sample 10 .
  • a corresponding data record is referred to as a height map, with a graphical illustration of the height map being optional.
  • the measuring apparatus 100 comprises an examination device 20 , which comprises an objective 21 and a camera 22 and which can also comprise further optical components.
  • the objective 21 is arranged in such a way that it guides light coming from the sample 10 (detection light) to the camera 22 .
  • the overview device 30 comprises at least one camera 31 , 32 . At least one of the cameras 31 , 32 serves to record an overview image of the sample 10 and can be referred to, accordingly, as an overview camera. Fields of view of the two cameras 31 , 32 are illustrated using dashed lines.
  • the measuring apparatus 100 comprises a light source 1 in order to illuminate the sample 10 .
  • illumination light can be visible light, IR light, UV light or else other electromagnetic radiation from other spectral regions.
  • detection light emanates from the sample 10 on account of the illumination light, some of said detection light being registered by the overview device 30 .
  • the detection light can be or comprise scattered or reflected illumination light, ambient light that has been scattered or reflected at the sample 10 and/or, in principle, also fluorescent light.
  • the two cameras 31 , 32 record images of the sample 10 from different viewing directions.
  • the two cameras 31 , 32 By combining the images of the two cameras 31 , 32 by calculation, it is possible, in a manner known per se, to obtain height information (height measurement data).
  • raw height measurement data can be considered to be the recorded images or information derived therefrom, as described in more detail below.
  • the sample 10 is held by a displaceable sample stage 5 .
  • the measuring apparatus 100 comprises a stand 40 , by means of which components of the measuring apparatus 100 are held.
  • a computing device 50 of the measuring apparatus 100 serves to process the measurement data of the cameras 31 , 32 and, optionally, also to control components of the measuring apparatus 100 , in particular the light source 1 , the camera 22 , the cameras 31 , 32 , and the sample stage 5 .
  • a relative position between the sample 10 and the examination device 20 can be set with the aid of a height map of the sample 10 , which is obtained by way of the overview device 30 .
  • a height of the sample stage 5 can be set on the basis of the height map in such a way that a distance from the objective 21 , which is desired for the examination, is present and that, moreover, a collision between the objective 21 and the sample 10 is avoided.
  • the sample 10 comprises a polished sample, in which a cylindrical object is embedded in resin and protrudes from said resin.
  • the sample 10 comprises a substrate 11 and an object 12 protruding therefrom in a height direction.
  • the protruding object 12 casts a shadow 13 on part of the substrate 11 , which is why the object 12 is also referred to as a shadow-causing object 12 .
  • the shadow 13 can thus refer to part of the sample 10 on which less illumination light is incident on account of the height profile of the sample.
  • the sample 10 can also comprise mirroring regions, in particular as a result of a grinding or polishing of the sample surface.
  • a mirroring region can be understood to mean a section of the sample 10 which reflects illumination light to one of the cameras when recording an overview image or height measurement data. Therefore, different sections of the sample can represent mirroring regions within this meaning depending on the measurement situation, e.g., illumination angle or sample stage position.
  • FIG. 2 shows a further exemplary embodiment of a measuring apparatus 100 of the invention.
  • a measuring head 23 operating on tactile principles is used as an examination device 20 .
  • the overview device 30 comprises only a single camera 31 .
  • the illumination by the light source 1 is used only for recording the image with the overview device 30 .
  • a single camera 31 can suffice for the purposes of recording raw height measurement data.
  • the same camera 31 can also record an overview image of the sample 10 .
  • the overview image and the raw height measurement data can be recorded in succession (for the purposes of which the light source 1 can optionally generate different types of illumination, for example with and without a pattern).
  • the same raw data or some of the same raw data of the camera 31 can be used to form the overview image and the height measurement data.
  • a plurality of images with different stripe illumination can be combined to form an overview image.
  • a plurality of images with different focal lengths can also be recorded.
  • the image in which a sample region appears in focus then depends on the height profile of the sample 10 , and so height measurement data can be obtained in this fashion.
  • one of the images can be used as an overview image or, alternatively, the plurality of images can be added or combined by calculation in another way to form an overview image.
  • ambient light may also be sufficient so that the light source 1 for recording an image with the overview device 30 can also be dispensed with.
  • Whether a sample stage 5 is used may depend on the type of sample. By way of example, if the sample is a product in a production line, the sample stage can be dispensed with.
  • the measurement data of the camera 22 can also be used to form the overview image and/or the height measurement data or raw height measurement data.
  • the camera 22 can be part of the overview device 30 if the objective 21 is not situated in the beam path or if a different objective is situated in the beam path, the imaging scale of which is smaller than that of the objective 21 .
  • the cameras 31 and/or 32 can be dispensed with or be part of the overview device 30 in addition to the camera 22 .
  • the height measurement data and the overview image can be recorded simultaneously or in succession.
  • one or more of the cameras 22 , 31 , and 32 can be replaced by a light detector or a stripe detector. Point or stripe detectors can be suitable for recording raw height measurement data and/or can be suitable for recording an overview image, in particular if different sample points are scanned in succession by a scan.
  • the illumination light of the light source 1 can also be guided onto the sample 10 via the objective 21 or another objective used.
  • FIGS. 1 and 2 can also be mixed; by way of example, the tactile measuring head 23 can be replaced by the objective 21 with the downstream camera 22 , or vice versa.
  • FIG. 3A shows an overview image 10 A, recorded of the sample 10 , using the overview device 30 .
  • the overview image 10 A contains an image region 11 A of the substrate 11 of the sample, an image region 12 A of the shadow-causing object 12 , an image region 13 A, which shows a shadow 13 caused by the shadow-causing object 12 , and an image region 14 A, which represents mirroring by a reflecting region of the sample.
  • the image processing, according to the invention, of such an overview image 10 A is described in more detail below.
  • a height map 10 B which is generated by height measurement data of the overview device 30 , is described with reference to FIG. 3B .
  • the overview device 30 can either record the height measurement data directly or initially record raw data, from which the height measurement data are obtained.
  • the diagonal hatching denotes a height value or a region of a certain height.
  • the diagonally hatched region 11 B which corresponds to the substrate 11 of the sample 10 , was consequently captured in plane form and has a common height value.
  • the region 12 B without diagonal hatching corresponds to a different height value and represents the protruding, shadow-generating object 12 .
  • the black regions 15 denote regions for which no, or no usable, height measurement data are available.
  • regions 15 correspond to the shadow 13 on the sample 10 and a reflecting section of the sample.
  • no (usable) height measurement data can be obtained in the case of a shadow if the height measurement is based on the principle of a patterned illumination, wherein a pattern (for example stripes) is radiated onto the sample surface. Height measurement data can be obtained from the position and/or distortion of the pattern (e.g., the stripes). However, no pattern can be radiated into the region of the shadow, and so no height measurement data are obtained here.
  • detection light permitting a reliable ascertainment of height measurement data obtained from the reflecting section of the sample either in the case of the patterned illumination example.
  • vertical black stripes are superposed in the example of FIG. 3B ; no height values are available therein on account of the measurement principle. In the case of striped illumination, this can originate for example from non-illuminated sample portions.
  • the computing device 50 of FIG. 1 or 2 carries out image analysis steps in order to obtain information from the overview image of FIG. 3A which is intended to improve the height map of FIG. 3B .
  • FIG. 3C A result of these image analysis steps is shown in FIG. 3C .
  • certain regions were identified in the overview image 10 A of FIG. 3A .
  • image analysis algorithms can recognize and localize certain shapes or learned objects in the overview image 10 A.
  • the image region 13 A was identified as a shadow 13 C. This should be understood to mean that, firstly, the limits of this image region were identified and, secondly, a semantic was assigned to the image region, i.e., a specification relating to the type of image region.
  • the image region 14 A was identified as a reflecting/mirroring region.
  • An image analysis algorithm can also classify the overview image 10 A, in order to categorize the type of sample.
  • conventional sample types can be learned, for example polished samples, in which, typically, a cylindrical or at least regularly shaped object protrudes from a plane substrate.
  • Further image information from the overview image can also be used for this type of classification, for example writing on the sample (not illustrated).
  • a classification can also be carried out without image analysis or can be specified by a user.
  • the image region 11 A can be identified as a plane substrate 11 C and the image region 12 A can be identified as a region 12 C of the protruding object 12 .
  • a surface of the protruding object 12 is plane; i.e., the mirroring region 14 C forms part of this surface and has the same height as the surface of the object 12 . Consequently, context information is obtained for an identified image region 13 A, 14 A, allowing deductions about height information.
  • context information could comprise, e.g., one or more of the following information items: classifying the image region 13 C as a shadow; identifying a region 12 C adjoining the image region 13 C as a shadow-causing object; classifying that the image region 13 C of the shadow has a lower height value than a height value of the shadow-causing object; identifying an image region 11 C, which adjoins the image region 13 C and which does not represent the shadow-causing object, as a substrate, from which height values for the image region 13 C can be derived.
  • Height information for the image region 13 C of the shadow that can be derived from this context information is for example that height values for this image region 13 C should be the same as a height measurement value of the substrate or should be ascertained by interpolation/extrapolation from height measurement values relating to the substrate.
  • the context information can specify, for example, that this is part of the surface of the object 12 C and height values for the reflecting section 14 C should be derived from height measurement values for the object 12 (e.g., by averaging these height measurement values or by equating these to a height measurement value of the object 12 ).
  • a modified height map 10 D is formed from the height map of FIG. 3B with the aid of this context information; it is shown in FIG. 3D .
  • the height measurement values which were missing in the region of the shadow were filled, by means of which a height region 11 D of uniform height is formed.
  • height values for the reflecting region were supplemented or altered such that an upper side of the shadow-causing object forms a common height region 12 D in the height map 10 D.
  • FIG. 4A shows an overview image 16 A of a sample, which is a circuit board.
  • the circuit board comprises various circuit board components, e.g., plug connectors, capacitors, conductor tracks, and integrated circuits.
  • an image region of a capacitor 18 A and an image region of a plug connector 17 A have been provided with reference signs in exemplary fashion.
  • FIG. 4B shows a height map formed from recorded height measurement data. Vertical stripes once again specify regions in which height measurement data are missing. Moreover, the protruding capacitors cast shadows in which it has not been possible to ascertain height measurement data. Reflections at various metallic surfaces moreover likewise lead to missing height measurement values.
  • FIG. 4C a calculated image processing result 16 C is shown schematically in FIG. 4C .
  • the image regions of a plurality of identified capacitors 18 C are framed in this case.
  • a plurality of plug connectors were identified and the corresponding image regions 17 C were marked.
  • Context information for the image regions of the capacitors 18 C can be used, like in the preceding example for the shadow-causing object.
  • the knowledge that the plurality of identified capacitors have the same shapes can be used. As a result, missing or erroneous height measurement values on one of the capacitors can be supplemented by available height measurement values at another one of the capacitors.
  • the region adjoining to the left might be shadowed for example in the case of a first capacitor while the region adjoining to the left is not shadowed in the case of a second capacitor.
  • height measurement values can then be supplemented from the non-shadowed region at the second capacitor.
  • These supplemented height measurement values can differ significantly from a simple interpolation/averaging of adjacent height measurement values:
  • certain connectors or lines might extend next to a capacitor, the height profile of which has not been shadowed at the second capacitor and therefore could be measured—these height measurement values of the connectors or lines can now be supplemented next to the first capacitor.
  • the knowledge of a periodic structure of a plug connector e.g., the latter being made of a plurality of similar contact faces, can be used as context information. If there was mirroring at one of these contact faces, as a result of which no usable height measurement values are available, height measurement values from other contact faces can be supplemented. In this way, it is possible, in turn, to alter and correct the height map 16 B from FIG. 4B .
  • similar objects can be identified in the overview image and height measurement values in respect of one of the objects can then be supplemented in the case of another one of the similar objects.
  • step S 1 a measurement is carried out on the sample, within the scope of which raw height measurement data are recorded.
  • Height measurement data which each specify a height value for various lateral regions of the sample surface are calculated from the raw height measurement data.
  • an overview image is also calculated from the raw height measurement data or an overview image is recorded by way of a dedicated overview measurement.
  • step S 2 the height measurement data and the overview image of S 1 are obtained. Variants of the method according to the invention can also resort to previously recorded measurement data, as a result of which step S 1 need not necessarily be part of the method of the invention but represents an optional extension.
  • image regions can be understood to mean specified types of image regions, for example shadows, mirroring and/or certain objects such as biological cells or cell components, rock samples, electronic components, manufacturing components or sample vessel parts such as coverslips or microtiter plate sections.
  • step S 4 context information is derived for the identified image regions, for example in respect of the type of the respective image region and/or how a height of the sample section of one image region is linked to a height of a sample section of another image region.
  • step S 5 The height measurement data are supplemented or altered in step S 5 with the aid of this context information. These supplemented or altered height measurement data are finally output in step S 6 as a height map.
  • This can comprise a graphical representation for a user and/or an output as an image file or a 3D model, as a result of which further automated analysis or control steps can also follow.
  • Components of the measuring apparatus can be controlled in step S 10 with the aid of the height map of S 6 .
  • a distance between the sample and a part of the examination device or the overview device can be set on the basis of the height map.
  • the method is extended by an additional method step S 7 , which is parallel with, before or after step S 5 .
  • step S 7 the plausibility of the height measurement data is checked on the basis of the context information or other information derived from the overview image.
  • an upper and lower limit for height values can be derived from the overview image. If a height measurement value is outside these limits, it can, in particular, be deleted, be pushed to the limit or be marked for further processing by a user.
  • step S 8 can also be supplemented.
  • a check is carried out as to whether, although the presence of an object has been determined in the overview image, no context information usable for supplementing or altering the height measurement data could be derived.
  • this may be the case in some transparent objects, in particular in the case of lenses. Consequently, the fact that the sample comprises a transparent object is identified in the overview image and the relative position of the transparent object is also ascertained, but height information for the transparent object cannot be derived from the overview image.
  • objects are present that are smaller than a minimum size that is necessary for height measurement data to be able to be ascertained (reliably) for this object. In this example, the presence of such a small object is determined, with the height thereof remaining unknown.
  • step S 10 can optionally include a local or global safety margin being set.
  • a measuring apparatus component in particular an objective or another part of the examination device 30 , must not come closer to the sample than said safety margin so as to keep the risk of a collision low.
  • a local safety margin specifies that a different (greater) safety margin should be maintained in the region of the identified object of unknown height than in remaining sample sections, for which height measurement values could be recorded and optionally could be verified by the overview image.
  • step S 9 can also follow step S 8 , in which settings that influence the examination device are altered. Subsequently, height measurement data once again recorded in step S 1 with the altered settings.
  • the altered settings depend on whether, or with respect to which image regions or identified objects, no height information could be obtained.
  • the corresponding sample regions can be examined again with an altered illumination setting (intensity, spectral range, polarization, illumination pattern, etc.), altered detection setting (in particular, altered measurement duration, altered height measurement region, colour filter or colour selection) or altered sample positioning (sample stage position).
  • the repeated examination of a sample region with altered measuring apparatus settings can optionally also follow step S 7 .
  • the sample regions for which the plausibility check has yielded a negative result are examined again.
  • a height map of a sample can be created particularly precisely and reliably, which can be to the benefit of, for example, a movement of measuring apparatus components on the basis of this height map.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Multimedia (AREA)
  • Geometry (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Molecular Biology (AREA)
  • General Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Health & Medical Sciences (AREA)
  • Length Measuring Devices By Optical Means (AREA)
US16/951,612 2019-11-22 2020-11-18 Measuring device for examining a specimen and method for determining a topographic map of a specimen Active US11168976B2 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
DE102019131693.6 2019-11-22
DE102019131693.6A DE102019131693A1 (de) 2019-11-22 2019-11-22 Messgerät zur untersuchung einer probe und verfahren zum bestimmen einer höhenkarte einer probe

Publications (2)

Publication Number Publication Date
US20210156669A1 US20210156669A1 (en) 2021-05-27
US11168976B2 true US11168976B2 (en) 2021-11-09

Family

ID=75784769

Family Applications (1)

Application Number Title Priority Date Filing Date
US16/951,612 Active US11168976B2 (en) 2019-11-22 2020-11-18 Measuring device for examining a specimen and method for determining a topographic map of a specimen

Country Status (3)

Country Link
US (1) US11168976B2 (zh)
CN (1) CN112833812B (zh)
DE (1) DE102019131693A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20220113533A1 (en) * 2020-10-09 2022-04-14 Carl Zeiss Microscopy Gmbh Microscopy System and Method for Analyzing an Overview Image
US20240124137A1 (en) * 2022-10-13 2024-04-18 Wing Aviation Llc Obstacle avoidance for aircraft from shadow analysis

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE102022115660A1 (de) * 2022-06-23 2023-12-28 Basler Aktiengesellschaft Verfahren und Vorrichtung zur Mikroskopie

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050173633A1 (en) * 2003-12-26 2005-08-11 Maki Tanaka Method of measuring dimensions of pattern
DE102012104282A1 (de) 2012-05-16 2013-11-21 Isra Vision Ag Verfahren und Vorrichtung zur Inspektion von Oberflächen eines untersuchten Objekts
US20170166862A1 (en) * 2015-12-11 2017-06-15 City University Of Hong Kong Substrate for a three-dimensional cell culture, its preparation and use
DE102017116758A1 (de) 2016-07-25 2018-01-25 Chromasens Gmbh Verfahren und Vorrichtung zum Abtasten von Oberflächen mit einer Stereokamera
EP3441773A1 (en) 2017-08-11 2019-02-13 Anton Paar GmbH Characterizing a height profile of a sample by side view imaging
US20190307409A1 (en) * 2016-06-15 2019-10-10 Telefield Medical Imaging Limited Three-dimensional imaging method and system
US20200149872A1 (en) * 2018-11-09 2020-05-14 Keyence Corporation Displacement Measuring Apparatus

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN201476761U (zh) * 2009-08-24 2010-05-19 苏州海兹思纳米科技有限公司 用于干涉显微镜升级的白光干涉三维轮廓测量仪
CN103075960B (zh) * 2012-12-30 2015-06-03 北京工业大学 多视角大深度显微立体视觉特征融合测量方法
CN106907988B (zh) * 2017-02-27 2019-03-22 北京工业大学 基础数据矩阵显微视觉建模方法

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050173633A1 (en) * 2003-12-26 2005-08-11 Maki Tanaka Method of measuring dimensions of pattern
DE102012104282A1 (de) 2012-05-16 2013-11-21 Isra Vision Ag Verfahren und Vorrichtung zur Inspektion von Oberflächen eines untersuchten Objekts
US20150130927A1 (en) 2012-05-16 2015-05-14 Isra Vision Ag Method and a device for the inspection of surfaces of an examined object
US20170166862A1 (en) * 2015-12-11 2017-06-15 City University Of Hong Kong Substrate for a three-dimensional cell culture, its preparation and use
US20190307409A1 (en) * 2016-06-15 2019-10-10 Telefield Medical Imaging Limited Three-dimensional imaging method and system
DE102017116758A1 (de) 2016-07-25 2018-01-25 Chromasens Gmbh Verfahren und Vorrichtung zum Abtasten von Oberflächen mit einer Stereokamera
EP3441773A1 (en) 2017-08-11 2019-02-13 Anton Paar GmbH Characterizing a height profile of a sample by side view imaging
US20200149872A1 (en) * 2018-11-09 2020-05-14 Keyence Corporation Displacement Measuring Apparatus

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
Stade, Jonas, Search Report for DE102019131693.6, dated Sep. 16, 2020, 5 pages (English translation not available).
Stade, Jonas, Search Report for DE102019131693.6, dated Sep. 16, 2020, with English translation (9 pages).

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20220113533A1 (en) * 2020-10-09 2022-04-14 Carl Zeiss Microscopy Gmbh Microscopy System and Method for Analyzing an Overview Image
US20240124137A1 (en) * 2022-10-13 2024-04-18 Wing Aviation Llc Obstacle avoidance for aircraft from shadow analysis

Also Published As

Publication number Publication date
DE102019131693A1 (de) 2021-05-27
US20210156669A1 (en) 2021-05-27
CN112833812A (zh) 2021-05-25
CN112833812B (zh) 2023-07-18

Similar Documents

Publication Publication Date Title
US11168976B2 (en) Measuring device for examining a specimen and method for determining a topographic map of a specimen
WO2013061976A1 (ja) 形状検査方法およびその装置
US9646374B2 (en) Line width error obtaining method, line width error obtaining apparatus, and inspection system
TW201826222A (zh) 判斷缺陷品質之方法及裝置
JP2013186100A (ja) 形状検査方法およびその装置
SG173068A1 (en) Methods for examining a bonding structure of a substrate and bonding structure inspection devices
US20160267648A1 (en) Mask inspection apparatus and mask inspection method
CN107339939B (zh) 光学度量中的测量对象的边缘确定的方法和设备
JP2017032548A (ja) 自動工業検査における３ｄビジョンの使用
CN109767425B (zh) 机器视觉光源均匀性评估装置及方法
KR101624120B1 (ko) 3차원 형상 계측용 구조광의 국소 영역별 패턴 광원 조사 시스템 및 방법
JP4905013B2 (ja) 外観検査装置、外観検査方法及び高さ測定方法並びに回路基板製造方法
JP5913903B2 (ja) 形状検査方法およびその装置
EP2913631A1 (en) Test apparatus and method
CN101556141A (zh) 一种测量不规则缝宽的傅里叶变换方法及其装置
US11454795B2 (en) Surface sensing in optical microscopy and automated sample scanning systems
CN117830251A (zh) 缺陷分析方法、缺陷分析装置及电子设备
US10715790B2 (en) System and method for lead foot angle inspection using multiview stereo vision
TW201317587A (zh) 尺寸檢測裝置及方法
Frangione et al. Multi-step approach for automated scaling of photogrammetric micro-measurements
TW201809592A (zh) 自動化三維量測
CN111220087B (zh) 表面形貌检测方法
CN219641581U (zh) 一种凹陷缺陷检测装置
JP2006003168A (ja) 表面形状の測定方法およびその装置
JP7279882B2 (ja) 画像計測システム、画像計測方法、画像計測プログラムおよび記録媒体

Legal Events

Date Code Title Description
FEPP Fee payment procedure

Free format text: ENTITY STATUS SET TO UNDISCOUNTED (ORIGINAL EVENT CODE: BIG.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

AS Assignment

Owner name: CARL ZEISS MICROSCOPY GMBH, GERMANY

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:AMTHOR, MANUEL;HAASE, DANIEL, DR.;STEHR, DOMINIK, DR.;SIGNING DATES FROM 20201209 TO 20201217;REEL/FRAME:055236/0585

STPP Information on status: patent application and granting procedure in general

Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER

STPP Information on status: patent application and granting procedure in general

Free format text: NOTICE OF ALLOWANCE MAILED -- APPLICATION RECEIVED IN OFFICE OF PUBLICATIONS

STPP Information on status: patent application and granting procedure in general

Free format text: PUBLICATIONS -- ISSUE FEE PAYMENT VERIFIED

STCF Information on status: patent grant

Free format text: PATENTED CASE

CC Certificate of correction