DE102020209671A1 - Method for generating a reference image and its use - Google Patents

Abstract

The invention relates to a method for generating a reference image (R) for a method for imaging a workpiece surface, in which the reference image (R) is generated by a number of images (a1, a2, a3, a4, a5) of a surface of a reference object is recorded and repositioned images (A1 to A5) are generated and stored based on the recorded image data.According to the invention, a mask data set (M) is generated by defining a number of evaluation groups (I to V) and in each of the evaluation groups (I to V), one of the individual repositioned image recordings (A1 to A5) is offset in pairs with a number of the other individual repositioned image recordings (A1 to A5), and the data records of an evaluation group (I to V) are combined to form a mask data set (Mi). The mask data sets (Mi) are applied to the recorded images (a1 to a5) and the recorded images (a1mod to a5mod) modified in this way are used to generate the reference image (R).

Description

The invention relates to a method for generating a reference image and its use in a method for detecting and imaging the topography of a workpiece surface.

In connection with this description, the term topography is understood to mean the surface structure of a material, a half element or a workpiece. In the following, reference is made to a workpiece for the sake of simplicity. The workpiece can have a complex design and can include a number of elements.

The topography of a workpiece can be recorded and displayed using imaging methods. Various imaging errors can occur. One of them is the so-called field curvature. If this is not corrected, an existing image field curvature will falsify the actual topographical conditions on the surface of the object to be measured. In particular in the case of very small characteristics of the topographical differences, ie the differences between elevations, depressions and/or plateaus, an existing image field curvature greatly falsifies the differences relative to their actual size.

Using the so-called reference topography, an existing image field curvature can be reduced or completely eliminated. For this purpose, images of a surface of a reference object (reference surface, reference sample) are recorded using recording optics. Their image data are then subtracted from the image data of the image recordings that were obtained with the recording optics of a workpiece to be measured. In this way, the contributions of the image field curvature contained both in the image recording of the reference object and in the image recording of the workpiece are offset against one another. In this sense, reference images for reducing shading (reference shading image; RGBreference shading image) can also be generated. In this case, data on image noise (background noise; background image) are subtracted from an image of a (white) area (white image).

In order to possibly minimize the effects of contamination on or at the recording optics or other faulty locations, several recorded images of the reference object are recorded, with the recorded images being slightly offset laterally in each case. To create a reference image for a topography method, the image recordings obtained in this way are then calculated pixel by pixel, forming a mean value or using a median to form a resulting topography, in this case the reference image. The data is usually also filtered in the x and y directions in order to minimize any effects that may occur.

The disadvantage of this procedure is that the image contributions of existing soiling are not eliminated, but only weakened, both when calculating using an average value and when calculating using a median (see also 2 ). Weakened contamination contributions can only be eliminated by means of subsequent strong filtering. However, such strong filtering leads to filter artifacts at the edges of the image field, which a user of the images finds very disturbing and which have the effect of an additional virtual image field curvature.

The invention is based on the object of proposing a possibility for improved reduction of artifacts in reference images.

The object is solved by a method according to independent claim 1 . Advantageous configurations are the subject matter of the dependent claims.

The method serves to generate a reference image for a method for imaging a workpiece surface. The reference image is generated by capturing a number of image recordings of at least one area of a surface of a reference object (reference surface). The individual image recordings are recorded from recording positions that are laterally shifted and/or rotated relative to one another. If, for example, there is contamination in a recording optics used, this affects different locations in the image in the shifted and/or rotated different image recordings. Repositioned images are generated and stored based on the captured image data of the image recordings. The repositioned images can be converted into a binary form. The repositioned recorded images can be converted into repositioned binary recorded images by means of a calculation rule. The evaluation groups are then formed from these. The calculation rule can be a threshold value comparison, for example. The repositioned image recordings or the repositioned binary image recordings serve as the basis for generating the reference image.

A method according to the invention is characterized by the generation of a mask data set. A number of evaluation groups are defined for this purpose. In each of the evaluation groups, one of the individual repositioned recorded images is offset either with each or with a certain number of the further individual repositioned recorded images, in particular in pairs. In this case, the mutually corresponding image data of the image recordings are subtracted from one another, for example. If a specific number of the further individual repositioned recorded images is used, this can be a specific number of randomly selected recorded images or a number of recorded images selected according to predetermined criteria. The data records of an evaluation group obtained from each of the paired calculations are combined to form a group result data record that stands for a group result image. In this case, structures detected in the data records obtained as a result of the pairwise calculation are evaluated with regard to their assignment as structures to be masked. Those structures which are detected and which exceed a predetermined threshold value of a threshold value comparison are classified as structures to be masked. These areas are caused in particular by dirt or errors in the recording optics and are defined as masked areas of the mask data set.

The threshold value comparison is, for example, a comparison of the frequency of occurrence of a detected structure with a predefined frequency threshold value. Those pixels of an image recording that represent a structure to be masked are optionally not included in the further generation of the reference image.

The mask data set is generated from a number—again, either all or a selection—of the group result images/group result data sets. The mask data set with the masked areas is applied to the recorded images and the recorded images modified in this way are used to generate the reference image.

The term recording optics is understood broadly. It includes not only the optical elements such as objective, optical lenses, mirrors, etc., but also technical elements such as detectors. For example, a faulty detector element of a matrix detector (pixel) can be understood in terms of soiling or a fault in the recording optics.

The core of the invention is that existing aberrations, such as dirt reflected in the image data, are not only weakened, but are completely masked and thus eliminated. In contrast to the procedure according to the prior art, no contributions from the dirt remain in the reference image. In addition, no complex filtering is required to remove such residual contributions. The associated occurrence of filter artifacts can also be significantly reduced or completely avoided by means of the method according to the invention.

, mit i ≠ j. Die Indizes i und j sind Laufindizes der repositionierten Bildaufnahmen. Die Bezeichnung z* steht dabei für eine Matrix der so erzeugten Bilder. Eine grafische Darstellung dazu findet sich in 3.An essential part of the method according to the invention can be expressed, for example, as follows: $${\mathrm{e.g}}_{i,j}^{*}=\left|{\mathrm{e.g}}_i\left(x,y\right)-\text{mean}\left({\mathrm{e.g}}_i\left(x,y\right)\right)-{\mathrm{e.g}}_j\left(x,y\right)+\text{mean}\left({\mathrm{e.g}}_j\left(x,y\right)\right)\right|$$

, with i ≠ j. The indices i and j are running indices of the repositioned images. The designation z* stands for a matrix of the images generated in this way. A graphic representation of this can be found in 3 .

Determined image data are then selected on the basis of a condition. For example, this can be as follows: $${\mathrm{e.g}}_{i,j}^{*}\left(x,y\right)={\mathrm{e.g}}_{i,j}\left(x,y\right)mit|\begin{array}{c}1wenn{\mathrm{e.g}}_{i,j}\left(x,y\right)<90\%Q\mathrm{and}antile\left({\mathrm{e.g}}_{i,j}\left(x,y\right)\right)\\ \text{0}wenn{\mathrm{e.g}}_{i,j}\left(x,y\right)\ge 90\%Q\mathrm{and}antile\left({\mathrm{e.g}}_{i,j}\left(x,y\right)\right)\end{array}.$$

The choice of the 90% quantile is of course exemplary. Depending on the evaluation approach, other percentage values can determine the condition. As a result, the images of the matrix z* are converted into binary images.

The next step is a pixel-by-pixel summation of the image data: $${\overline{\mathrm{e.g}}}_i\left(x,y\right)={\displaystyle \sum _j{\mathrm{e.g}}_{i,j}^{*}\left(x,y\right)}$$

In order to avoid the mean value or the median being falsified by the pixels of the masked regions, the pixel values of these pixels are regarded as invalid in a possible embodiment of the method. Although this reduces the number N of pixel values in the masked areas that can be used to form an average or median, these do not contribute to a tendency toward deviation. The pixels of an image recording defined as masked by the mask data record are therefore not included in the further generation of the reference image.

The reference image generated using the method according to the invention can be used in particular in a method for detecting the topography of the surface of a workpiece. In particular, the reference image can be used to reduce an image field curvature that occurs in a correspondingly designed method.

The reference image or its image data can be optimized by optionally subjecting them to, preferably light, filtering.

The method can also be used to correct image data in so-called shading or RGB shading.

In order to carry out the method according to the invention, any recording optics which is provided with a suitable detector can be used. In particular, the method is suitable for use in microscopy and materials testing.

The recording optics can be connected to a tripod and interact with a motor-driven table for supporting the sample and/or the reference object.

Camera-based systems, laser-scanning microscopes (laser scanning microscope), confocal microscopes, wide-field microscopes, white-light interferometers and light sheet microscopes can serve as microscopes, for example.

• 1 eine schematische Darstellung der Wirkung zueinander lateral verschobener Bildaufnahmen;
• 2 eine schematische Darstellung der Erzeugung eines Referenzbildes gemäß dem Stand der Technik;
• 3 eine schematische Darstellung eines Ausgestaltungsbeispiels des erfindungsgemäßen Verfahrens am Beispiel von fünf Auswertungsgruppen;
• 4 eine schematische Darstellung der Erzeugung eines Maskendatensatzes aus einer Anzahl von Gruppenergebnisdatensätzen in einem Ausgestaltungsbeispiel des erfindungsgemäßen Verfahrens, und
• 5 eine schematische Darstellung modifizierter Bildaufnahmen.

The invention is explained below using exemplary embodiments and illustrations. Show it:

• 1 a schematic representation of the effect of mutually laterally shifted image recordings;
• 2 a schematic representation of the generation of a reference image according to the prior art;
• 3 a schematic representation of an embodiment example of the method according to the invention using the example of five evaluation groups;
• 4 a schematic representation of the generation of a mask data record from a number of group result data records in an exemplary embodiment of the method according to the invention, and
• 5 a schematic representation of modified images.

To support the following explanations should be based on the 1 the effect of image recordings a1 to a3 that are laterally shifted in relation to one another can be illustrated. To get a reference image R (see 2 ), a number of images a1 to a3 of a reference object are recorded (top line), which are recorded from recording positions that are laterally offset and/or rotated relative to one another (rotated recording positions not shown). In each image recording a1 to a5, a small cross is drawn in for the purpose of illustrating the offset movement, which marks a fixed position on the surface of the reference object. In addition, contamination of the recording optics (symbolized by a star) is recorded. Since the contamination is stationary relative to the beam path of the recording optics, it is always shown at the same place in the images a1 to a5. As can be seen from the respective position of the small cross in the image recordings a1 to a5, the reference object is imaged in different positions as a result of the changed recording positions, depending on the direction and amount by which the displacement or rotation occurred.

The image data of the recorded images a1 to a5 are then aligned (repositioned) again, in particular by computation, and the repositioned recorded images A1, A2, A3, A4 and A5 are obtained (bottom line). In the repositioned images A1 to A5, the small cross is always shown in the same position. It is thus possible to subtract the repositioned recorded images A1 to A5 from one another, for example, in order to determine any background noise that occurs (background noise; background image). In contrast, the dirt (star) is shown at different positions in the repositioned images A1 to A5. If, for example, the repositioned images A1 and A2 were subtracted from one another, the contamination would remain as a result.

This circumstance is in 2 taken up, in which the generation of a reference image R according to the prior art is shown in an exemplary and simplified manner. The repositioned images A1 to A5 show the soiling in five different places as an example. If the repositioned images A1 to A5 are summed up and a mean value or alternatively a median is formed for each pixel in order to obtain the reference image R, the contamination at each of the five locations contributes to the image content of the reference image R. Although the contribution of the contamination is reduced to about a fifth of its original intensity as a result of the averaging or median formation, it is still present in the reference image R. These "shadows" can be further reduced with subsequent filter steps, but there is a risk that filter artifacts will occur. That in the 2 until 5 The summation sign used also stands for the formation of an average or a median.

An embodiment of the method is based on the 3 be explained. After image recordings a1 to a5 have been acquired and repositioned image recordings A1 to A5 have been created as described above, repositioned binary images are generated using formula 2 (see above). These repositioned binary images are subsequently evaluated in evaluation groups. In the present example, five evaluation groups I to V are defined (shown in rows i = 1, 2,.., 5)). In each of the evaluation groups I to V, one of the individual repositioned images is offset in pairs with each of the other individual repositioned images of the relevant evaluation group I, II, II, IV or V (formula 1). The image data of the image recordings that correspond to one another are, for example, subtracted from one another. For example, in the first evaluation group I at position i = 1, j = 2 is the result of the calculation of the repositioned and binary image A1 with the repositioned binary image A2 (see 2 ) is shown, at position i = 1, j = 3 the result of calculating the repositioned binary image A1 with the repositioned binary image A3 and so on. In the second evaluation group, the repositioned binary image A2 is offset against A1 (position i=2, j=1), the repositioned binary image A2 against A3 (position i=2, j=3), and so on. The same applies to the third to fifth evaluation groups III to V.

The results shown in the matrix of the results of an evaluation group I to V obtained by means of the paired calculations are optionally averaged (formula (3)) and each represent a group data set G _i . Using a selection criterion (formula (4)) those detected structures are selected , which were detected, for example, at least three times.

wobei NaN für Pixel ohne Messwert, also für zu maskierende Pixel, steht.Formula (4) can be as follows: $${\overline{\mathrm{e.g}}}_i\left(x,y\right)|\begin{array}{c}1wenn{\overline{\mathrm{e.g}}}_i\left(x,y\right)>2\\ NaNwenn{\overline{\mathrm{e.g}}}_i\left(x,y\right)\le 2\end{array}$$

where NaN stands for pixels without a measured value, i.e. for pixels to be masked.

The result of the frequency comparisons in the group data sets Gi of the evaluation groups is shown in column M _i .

This process design is to be explained using the first evaluation group I.

As already exemplified 2 explained, the recording positions for the recorded images a1 to a5 are changed in such a way that in the repositioned recorded images generated from this, an image of dirt, for example in the first repositioned recorded image A1 in the upper left corner, in a second repositioned recorded image A2 in the upper right corner, in a third repositioned image A3 in the lower left corner, in a fourth repositioned image A4 in in the lower right corner and in the fifth repositioned image acquisition A5 in the center of the image field.

in the in 3 The scheme shown shows, for example, in the second column (j=2) of the first evaluation group I, the offsetting of the image data of the first repositioned binary image recording A1 with the second repositioned binary image recording A2. The third column (j=3) shows the calculation of the image data of the first repositioned binary image recording A1 with the third repositioned binary image recording A3, and so on.

Due to the application of the selection criterion (formula (4)), the positions of the contamination that were initially still contained but only recorded once are eliminated in the mask data sets M _i .

It is therefore documented in the mask data record M ₁ of the first evaluation group I that dirt is imaged in the upper left corner of the images a1 to a5. Their specific form and extent have also been determined. The same principle is used in the second to fifth evaluation groups II to V.

The individual mask data records M ₁ to M ₅ can be merged into a mask data record M (also: z̅ _i (x, y)) ( 4 ).

The mask data record M or the individual mask data records M ₁ to M ₅ is or are applied to the image recordings a1 to a5 of the reference object (formula (5)), with no intensity values (NaN = 0 ) are included. The modified images a1 _mod , a2 _mod ,..., a5 _mod are thus obtained from the images a1 to a5 . $${\grave{\mathrm{e.g}}}_i\left(x,y\right)={\mathrm{e.g}}_i\left(x,y\right)\ast {\overline{\mathrm{e.g}}}_i\left(x,y\right).$$

(x,y) steht für die modifizierten Bildaufnahmen a1_mod, a2_mod,..., a5_mod.The expression

(x,y) stands for the modified images a1 _mod , a2 _mod ,..., a5 _mod .

With the 5 the function of the mask data record M is illustrated by way of example. The already in 1 The image recordings a1, a2 to a5 shown in the upper line are modified by using the mask data record M1. The contamination in the upper left corner (symbolized by the star) is masked. For the sake of clarity, the contamination is also indicated (broken solid line). The images a1 _mod , a2 _mod ,..., a5 _mod modified in this way are converted back into repositioned images A1 _mod to A5 _mod and used to calculate the reference image R (correction image; containing the image field curvature but no dirt, formula (6)). . $$R={\mathrm{e.g}}_{cO\mathrm{right}}\left(\mathrm{e.g},y\right)=\frac{{\displaystyle \sum {}_i{\mathrm{e.g}}_i\left(x,y\right)\ast {\overline{\mathrm{e.g}}}_i\left(x,y\right)}}{{\displaystyle \sum {}_i{\overline{\mathrm{e.g}}}_i\left(x,y\right)}}.$$

The reference image R can then be used for correction, for example in methods for detecting the topography of the surface of a workpiece. The NaN must be taken into account accordingly. For example, when calculating an average, the sum of all valid elements (pixels) is divided by the number of valid elements.

Reference List

a1, a2, a3, a4, a5

laterally offset / rotated images

a1mod, a2mod,..., a5mod

modified imagery

A1, A2, ..., A5

repositioned (binary) image acquisition

A1mod, A2mod, ..., A5mod

modified repositioned imaging

wed

Mask data set (i = 1, ..., n)

Gi

Group record (i = 1, ..., n)

R

reference image

I, II, III, IV, V

evaluation group

Claims (6)

Method for generating a reference image (R) for a method for imaging a workpiece surface, in which the reference image (R) is generated by a number of images (a1, a2, a3, a4, a5) of a surface of a reference object is recorded, wherein the individual recorded images (a1 to a5) are recorded from recording positions that are laterally shifted and/or rotated relative to one another, repositioned recorded images (A1 to A5) are generated and stored on the basis of the recorded image data of the recorded images (a1 to a5), and on the basis of the repositioned recorded images ( A1 to A5) the reference image (R) is generated, characterized in that a mask data set (M) is generated by defining a number of evaluation groups (I to V) and in each of the evaluation groups (I to V) - one of the individual repositioned image recordings (A1 to A5) are offset in pairs with a number of further individual repositioned image recordings (A1 to A5), and - The data records of an evaluation group (I to V) obtained from each of the paired calculations are combined to form a mask data record (M _i ), with structures detected in the data records obtained as a result of the paired calculation being evaluated with regard to their assignment as structures to be masked and those detected Structures are classified as structures to be masked that exceed a predetermined threshold value of a threshold value comparison, and the mask data sets (M _i ) are applied to the recorded images (a1 to a5) and the recorded images modified in this way (a1 _mod , a2 _mod , a3 _mod , a4 _mod , a5 _mod ) can be used to generate the reference image (R). procedure after claim 1 , characterized in that the threshold value comparison is a comparison of the frequency of occurrence of a detected structure with a frequency threshold value. procedure after claim 1 or 2 , characterized in that the repositioned recorded images (A1 to A5) are converted into repositioned binary recorded images (A1 to A5) by means of a calculation rule and the evaluation groups (I to V) are formed from these. Method according to one of the preceding claims, characterized in that pixels of an image recording which represent structures to be masked are not included in the further generation of the reference image (R). Method according to one of the preceding claims, characterized by the use of the reference image (R) for detecting the topography of the surface of a workpiece. Procedure according to one of Claims 1 until 4 , characterized by the use of the reference image (R) for a reduction of an occurring image field curvature.

Images