AT11770U1 - METHOD FOR SELECTING AN OPTIMIZED EVALUATION TEMPERATURE FOR CONTROLLING A FREE-FORM SURFACE AND METHOD FOR CHECKING A FREE-FORM SURFACE - Google Patents

Abstract

A method for selecting an optimized evaluation feature subset for inspection of freeform surfaces includes a generating step of generating a plurality of combinations of evaluation features, an evaluation step of evaluating each pixel, or a subset of the pixels of the patterns of a reference set with each combination of evaluation features to obtain a weighted state , a comparing step of comparing the evaluated state with the reference state to obtain a detection rate for each combination of evaluation features, and a determining step of determining the optimized evaluation feature subset based on the detection rates. A method of inspecting a free-form surface illuminated by a structured light includes an illuminating step of illuminating a portion of the freeform surface, a recording step of recording an image of the illuminated portion, and an evaluating step of evaluating each pixel or subset of the pixels of the image Combination of evaluation features to obtain a defective or non-defective state of the surface area.

Description

Austrian Patent Office AT 11 770 U1 2011-04-15

Description [0001] Embodiments of the invention relate to a method for selecting an optimized rating feature subset for inspection of free-form surfaces illuminated by structured light and to a method of inspecting a free-form surface illuminated by a patterned light.

Embodiments of the invention relate to the fields of surface quality control by means of optical methods and pattern analysis.

In Fig. 1 is shown a diagram of a conventional pattern analysis procedure. The pattern analysis procedure has a reference set Φ of image patterns that must be separately structured into a training set and a test set. The former is used to "teach" the classification algorithm, the latter being used for classification evaluation. The core part of this procedure has the characterization of each pattern F to be classified by means of specific features.

Two different approaches can be used for classification: the Naive Bayes classifier (NB) and the nearest neighbor classifier (k-NN; nearest neighbor). There are two main reasons to use these two classifiers. First, both methods are based on two different classification models. The NB procedure is a probabilistic learner who makes the assumption that the feature distributions of each class can be described by a normal distribution. The k-NN is an instance-based prediction model, as described in "lan H. Witten and Eibe Frank, Data mining, Oxford 2005" (instance in the sense of pattern), which does not try to create "rules" but directly to the Patterns or structures themselves works.

Second, the NB classifier is often considered to be a simple but powerful approach. This classifier often exceeds the performance of more sophisticated classifiers according to "lan H. Witten and Eibe Frank, Data mining, Oxford 2005". The method k-NN is known to address the results of the NB classifier in the case of a large data set (as shown by "TM Cover and PE Hard, Nearest Neighbor Pattern Classification, A Trans. On Information Theory 13 (1967), No. 1, 21 - 27 "), as it is here when inspecting free-form surfaces. If the Bayes minimum probability of the error of pB minus 1 is low, then the following equation holds: pB - pN - 2 * pB, where pB is the minimum probability of a Bayes classifier error and pN is the minimum probability of error of the next Neighbor decision rule is.

With regard to the computational aspect of both classifiers, the k-NN classifier is more computationally comprehensive than the NB classifier. In fact, the NB model requires only a small set of parameters that are estimated during the training procedure. These parameters define the probability function for each feature vector value for each class. The classification consists of assigning each observed feature vector c 'of the test set to the most probable class according to the value of the probability function. For k-NN, the number of operations required to classify a pattern F is directly proportional to the number of elements in the training set. For each 'F of the test set, the distance to each member of the training set must be calculated.

For the description of both classifiers let C be a vector of random variables denoting the observed features and let c be the i-th observed feature vector (i = 1, ..., NF). 'c (k) is the kth value of the observed feature vector' c, k = 1, Nc. Furthermore, let Ωκ be a random variable denoting the class of a feature QK G □ {Ω ^ Ω2, Ω3}.

[0009] NB is a probabilistic learner based on the Bayes Rule. This classifier is called naive because it is based on the important, simplistic assumption that the preceding

Austrian Patent Office AT 11 770 U1 2011-04-15 characteristics are independent.

According to the Bayes Rule (see "RO Duda and PE Hart, Pattern Classification and scene analysis, Wiley, New York, 1973"), the a posteriori probability is p (QK | C = 'c) of each class Ωκ, if the i-th vector of observed values' c is given, then KUkIC = * c) = ΜΟ «) · ρ (0. = ^ Ωκ) j> (G = * c) [0011] According to the Naive Bayesian simplification, which indicates that the conditional probabilities are independent, gives the following conditional probability: p {c = * ci ^) = riKW) = * The values of the observed feature vector 'c are numeric. Assuming that within each class label Ωκ these feature values are normally distributed, such a distribution can be fully represented in terms of its mean and variance.

The technique k-NN assigns an observed feature vector 'c from the test set to the class label Ωκ of most of its k-nearest-neighbor patterns from the training set using a distance criterion. The Euclidean distance or the city block metric are typical distance criteria. The nearest neighbor approach may also be written according to the estimated a posteriori probability as follows:

where mK is the number of patterns having the class label Ωκ in the vicinity of the observed feature vector c, and m is the total number of patterns in the same neighborhood. The next neighbor decision rule is independent of the common distribution of the sample points.

Two different classification methods (learning) can be used: a stratified, 10-fold cross validation 10-F and a one-leaving comparison test classification LoO.

The 10-F method divides the reference data set Φ into 10 parts, in which each of the z. B. three classes {Ω ^ Ω2, Ω3} is represented to the same proportion as in the full data set Φ. The learning procedure is executed ten times. Each time the training is run on the remaining 9/10 of the record. Finally, the ten estimated classification rates are averaged to give the final classification rate Cp, expressed as a percentage. Cp is the rate of correctly classified patterns belonging to the three classes {Ω ^ Ω2, Ω3}. LoO is simply a NF-fold comparison check, where NF is the number of samples of the record Φ. This method consists of classifying each of the NF patterns using the NF-1 remaining patterns for the training step.

For the selection of training and test samples from the data set Φ their size and representativeness is taken into account. Therefore, each of the classification methods involved has the following advantages.

In the case of the 10-F method, the stratification process guarantees the representativeness of each class in the training and testing set. Further, this procedure attenuates any bias caused by random sampling of the data set Φ by 2/60

Austrian Patent Office AT 11 770 U1 2011-04-15 Repeated repetition of training and testing. Ten times sampling may be referred to as the correct number of executions to obtain the best estimation error. A major advantage of the LoO procedure is that the largest amount of data is used for the training step. Another advantage is that no random sampling is involved.

The learning and evaluation process of the classification process are carried out using a commented reference image database Φ. The classical procedure uses a first set of reference patterns for training purposes of the classification algorithm. The second reference set of the patterns is used to evaluate the classification algorithm by comparing the true state with the detected state. One need for a classification problem is to gather enough information about the patterns to classify, ie. H. to have a representative reference set Φ of patterns. It is, of course, preferred to have a large and representative set Φ of reference patterns, since the quality of a classifier depends on its ability to gather enough information about the patterns F to classify , Φ can be considered representative if the calculated feature vector c defines all patterns F G □ Φ.

The classification procedure then consists of assigning a class Ωκ to each pattern F according to the corresponding feature vector c.

Since each classification result of the vision system causes a certain action, each decision is coupled with specific costs. Mathematically speaking, this corresponds to defining the cost values RAK for each pair of classes Ωλ | Ωκ among the Nk classes, as follows:

The costs rkK are generated when a pattern F is wrongly classified into a class Ωλ although it belongs to class Ωκ. The cost of misclassification is logically higher than the cost of a good classification.

The classification procedure can therefore be considered as an optimization process, where the task is to classify each image pattern F by means of the corresponding feature vector c in such a way that the average costs are minimized. The optimal classifier minimizes the value uA (c) defined as follows:

Nk ux ^ = Σλ = 1, * *, Νκ pK is the probability of the occurrence of class Ωκ, p (c | DK) is the conditional probability that the vector c is calculated and the corresponding pattern F belongs to the class Ωκ.

The cost values are task-dependent and assigned according to the requirements of the inspection process. The probabilities are calculated directly from the previously collected data. If the estimated probability of occurrence of pK is relatively easy to recover, as expressed by the ratio of patterns of class Ωκ among all patterns, the estimated conditional probability p (c | DK) 3/60 Austrian Patent Office AT 11 770 U1 2011-04 -15 and also to calculate the estimated density distribution more complex.

The first possibility is that the distributions of c follow known models. Then the task consists of determining the parameters of these models from the values of c. This is called the parametric approach. As an example, one and two parameters from the observed pattern are estimated for the Poisson and Gaussian distributions. However, if no model fits the distributions of c, a non-parametric approach may be necessary in such a way that the density distributions are estimated without prior knowledge of the shape of the observed distributions. One solution is to use the Parzen algorithm or the Gaussian mixed models to estimate these densities.

Extraction and discrimination of the image features is part of a conventional classification process consisting of the three components shown in Figure 1: feature extraction, learning and classification procedure. In this way, the pattern analysis consists of the extraction of some relevant information, its treatment and its interpretation. The feature extraction process is directed to the extraction of all relevant characteristics describing an image pattern F collected in a feature vector c. The information shown in the image pattern can be of any type and wood, walls, fabric, grass, clouds, etc. are just a few examples. The feature values may be considered to correspond directly to the pixels of the image pattern such that F = c. This is a possible approach to surface classification because the image pattern contains all the necessary, relevant information. However, processing this enormous amount of data requires more computational effort and memory, so an intermediate feature extraction step is preferred.

Once a feature vector c has been calculated, the entire classification process can be optimized using combinatorial optimization procedures, which are also feature selection procedures. Its purpose is to recover the NCsUb best features of the subset csub from the Nc features of c, cSUb G □ c. The branch-and-bound (branch-and-bound) and the genetic algorithm are two possible approaches. Two types of methods are used. The first consists of minimizing or maximizing distance values of feature inter and intra divisions. The second is minimizing false guess probabilities obtained with a classifier.

The most optimal feature set recovery is performed within the context of the industrial computer vision process. Such a classification procedure consists of different steps, starting with picking up the surface to be inspected and ending with the final classification result. The following three main steps are considered: the segmentation of the defects to be interpreted; their characterization and their interpretation. It is the goal to achieve optimal classification results.

With regard to conventional visual inspection techniques relating to the visual improvement of defective surface parts positioned on free-form objects, two main approaches can be distinguished to different types of defective surfaces, e.g. B. as two-dimensional textural or texture or three-dimensional depth defects to examine. The first approach uses diffuse illumination to test for texture defects, and the second uses three-dimensional triangulation techniques to recover the depth information. The major disadvantage of the former approach is that some depth defects can not be visually enhanced, the major disadvantage of the latter being that the recovery of depth defects is based on a pre-calibration process of the entire recording device, i. H. the recording sensor and the structured lighting.

In the case of the simultaneous improvement of different, defective surface parts on free-form geometries, different techniques are proposed in the literature, whether they are used as laboratory equipment or suitable industrial systems.

A first possibility is provided by "Ingo Reindl, Optical Configurations, Methods and Algorithms for the Automatic Inspection of Metallic Surfaces, PhD Thesis, University of Leoben, Austria, 2006", the light separation technique for the inspection of planar steel block probes in one used industrial process. In this case, the intensity of the reflected light rays is used to recover the geometric surface defect such as cracks and to determine structural defects.

A technique for reliably recovering the edges and the texture of metallic objects is proposed by "F. Puente Leon and J. Beyerer, Active Vision and Sensors Fusion for Inspection of Metallic Surfaces, Intelligent Robots and Computer Vision XVI, Algorithms, Techniques, Active Vision and Materials Handling, D.P. Casasent (ed.), Proc. Of SPIE, 3,208, 394-405, 1997 ". This method is based on the fusion of different images of the same object, recorded with different directions of illumination.

However, the recording conditions of both approaches are not optimal with respect to an industrial real-time inspection process. The first method implies the use of two cameras and does not suggest a generalization for free-form surfaces, only the case of planar surfaces is considered. With the second method, the entire inspected surface must be recorded multiple times, which is difficult to apply in an online inspection process.

With the same purpose of simultaneously improving various defective surfaces, automatic inspection systems have recently been proposed. Aceris' Ace-ris 3D, FC Substrate Bump Inspection System, Product Documentation, Clark Graham 300, Baie D'Urfe, Quebec, Canada, 2005, proposes a combined 3D laser technique with a 2D color approach for the measurement of electronic Components. In the same way, a combined 3D computed tomography technique and a 2D x-ray sensor were integrated into a system by Comet Comet AG, Feinfocus Fox, High Resolution 2D / 3D, Product documentation, Herrengasse 10, 31775 Flamatt, Switzerland, 2005. The intention is also the inspection or inspection of electronic components. The company "Solvision, Precis 3D, Wafer Bump Inspection Solution, Product Documentation, Solvision 50 De Lauzon, Suite 100, Boucherville, Quebec, Canada, 2007" proposes a wafer inspection system that combines structured 3D illumination with the 2D images of the Integrated surfaces.

Major disadvantages of such methods are that they combine two different lighting techniques.

With the same aim of improving different types of defects with a system, an alternative surface inspection method has been proposed in the patent by Yannick Caulier, Klaus Spinnler, Robert Couronne, Cylindrical Object Lighting Apparatus, Surface Inspection Method and Computer Program Product. Patent DE 10 2006 008 840.9, Fraunhofer Institute for Integrated Circuits, 2006 ". It has been demonstrated that different defective surface types can be visually enhanced by means of a specific illumination based on the projection of a stripe pattern. The image patterns have a usually vertical and periodic structure whose perturbations relate to the surface type, i. H. on the broken surface that is being recorded. The case of 2D texture and geometric 3D defects on specular cylindrical surfaces was addressed. The proposed technique can be used in real-time inspection processes when the patented application is used in a 24-hour / 7-day production chain.

A major limitation of the proposed method of DE 10 2006 008 840 is that it relates to the inspection of certain surface geometries and types, i. E. H. cylindrical and reflecting objects. In fact, the lighting was designed especially for such object types, and the fringe pattern interpretation algorithms were basically developed for the characterization of vertical and periodic stripe structures. [0040] Recording Principles of Reflecting, Reflective Freeform Surfaces and Methods for Recording are Described in "Sören Kammei, Deflectometric Investigation of Reflecting Reflective Freeform Surfaces, PhD Thesis, University of Karlsruhe (TH)," Germany, 2004 ", although these methods can not be applied to rough surfaces.

While DE 10 2006 008 840 describes vertical and periodic stripe structures for characterizing a surface, other types of features are described in the texture analysis community "M. Tuceyran, Texture Analysis, World Scientific Publishing Co., 2nd Edition, 207-248, 1998 ". Depending on the selected features and the type of surface, different acquisition rates can be achieved.

The patent DE 10 2006 008 840 limited to cylindrical objects with a lighting and associated algorithms, which are particularly adapted for cylindrical objects. The methods described in this patent can not be used for similar systems that use this particular type of strip lighting (to visually enhance and detect the errors) for detecting other surface geometries, e.g. B. planar surfaces use.

In DE 10 2006 008 840, the projected light beams having the fringe pattern are adapted to the object such that a periodic pattern or structure is imaged onto the image of the camera.

Methods of describing surface information acquired directly by the fringe pattern are described in the context of interferometric methods using coherent light, e.g. By Huang Zhi and Rolf B. Johansson, Interpretation and Classification of Fringe Patterns, 11th Int. Conference on Image, Speech and Signal Analysis (IAPR1 1992), August, 105-108, 3, 1992 "or" Xide Li, Wavelet Transform for Detec- tion of Partial Fringe Patterns Induced by Defects in None Destructive Testing of Holography Interferometry and Electronic Speckle Pattern Interferometry, Journal of Optical Engineering, October, 2,821 - 2,827, 39, 2000 ". However, these methods always use a feature type, e.g. An adaptive feature or a wavelet feature or others.

In this context, there is a need for a simple method of inspecting freeform surfaces in terms of low computational complexity and simple and inexpensive lighting and recording techniques.

Embodiments of the invention provide a method for selecting an optimized rating feature subset for inspection of free-form surfaces illuminated by a structured light based on a reference template set showing surface areas of free-form objects, each pattern of the reference set being defective or non-defective Reference state of the corresponding surface area of the free-form objects is assigned. The method comprises generating a plurality of combinations of evaluation features, each combination of evaluation features comprising a first set of features and a second set of features, wherein the first feature set is selected from a first group of adaptive features, and wherein the second feature set comprises a second set of texture features, wherein the first set of adaptive features comprises strip-based features and / or fringe-based features, and wherein the second set of texture features comprises transform-based features, structural features, and / or statistical features , The method further comprises evaluating each pixel or subset of the pixels of the patterns of the reference set with each combination of evaluation features to obtain a rated defective or non-defective state of the respective surface areas. The method further comprises comparing the evaluated, defective, or non-defective state with the defective or non-defective reference state to obtain a detection rate for each combination of evaluation features. The method further includes determining the optimized score feature portions based on the detection rates. Embodiments of the invention further provide a method of inspecting a free-form surface illuminated by a patterned light. SUMMARY OF THE INVENTION The method includes illuminating a portion of the free-form surface by the patterned light. The method further includes recording an image of the illuminated area of the free-form surface. The method further comprises evaluating each pixel or subset of the pixels of the image of the free-form surface area with a combination of evaluation features to obtain a defective or non-defective state of the surface region, wherein the combination of evaluation features comprises a first set of features and a second set of features, wherein the first feature set is selected from a first group of adaptive features, and wherein the second feature set is selected from a second group of texture features, the first group of adaptive features having strip-based features and / or fringe-based features, and wherein the second set of texture features transformation-based features, structural features and / or statistical features.

The strip-based features include geometry-based features, intensity-based features, and statistic-based features. The fringe-based features have geometry-based features, intensity-based features, and statistic-based features.

The transformation based features have Fourier based features and wavelet based features.

Embodiments of the invention do not require a periodic stripe pattern, it is sufficient to project any type of structured light, e.g. For example, any type of striped pattern. A selection of methods for generating stripe patterns is not limited to any particular method or technique for illuminating the surface. A large number of different objects with different geometries and surface types can be inspected by the structured light according to embodiments of the invention. The illumination need not be adjusted or adjusted to the object to receive a periodic pattern in the image of the camera. The image of the camera that has recorded the striped pattern of the illuminated surface of the object carries information about defects in the surface of the object. Errors can be detected when variations or distortions of the pattern caused by defects in the surface of the object can be distinguished from a pattern recorded from the image with a defect-free surface.

The lighting method may use non-coherent light. In this case, conventional fringe pattern generating methods can be applied, e.g. For example, a projector for projecting stripes onto a matte surface or a projector for projecting stripes onto a screen, the stripes being reflected off a specular surface. Further, an LCD display can be used to display strips of a specular surface.

However, embodiments of the invention are not limited to the use of non-coherent light. You can also illuminate with coherent light to produce interference patterns. Embodiments of the invention are not limited to a particular recording method when recording the image, e.g. B. no special recording position of the object and / or the camera. Embodiments of the invention provide a greater variety of recording methods, e.g. Line scan cameras, matrix cameras or line scan sensors without limited positions with respect to the object and / or the illumination. The only limitation is that the sensor, the projected fringe pattern must be positioned on the sensor and the surface in such a way that a stripe structure can be observed in the images.

Embodiments of the invention relate to the inspection of a wide range of surface types, i. H. matt to specular, and different geometries (free-form surfaces) to simultaneously or approximately simultaneously improve different defective surface types, eg. B. two-dimensional texture or three-dimensional depth defects. Advantages of embodiments of the invention in comparison to conventional techniques are their simplicity and their general approach.

No more than a camera and illumination are required to characterize the surfaces. Further, no complicated calibration processes are necessary to calculate the position of the camera or the illumination, as is the case with conventional three-dimensional depth recovery techniques.

Embodiments of the invention allow simultaneous differentiation of different types of defective surfaces with only one illumination. Two-dimensional texture and geometric, three-dimensional defects can be detected.

Mistakes can be detected when the defective surfaces to be characterized induce stripe deformations that are distinguishable from the non-defective surfaces.

In contrast to DE 10 2006 008 840, where the algorithm is adapted to the illumination and recording position to produce a particular image, such a requirement is not necessary for embodiments of the invention.

An LCD display displaying stripe patterns could replace the particular developed lighting of DE 10 2006 008 840. The camera would record the projected fringe pattern on the display. However, by using the LCD display, the recorded stripe pattern would have a different structure, i. H. a nonperiodic structure. This would directly affect the algorithmic evaluation, such that the classification methods proposed in DE 10 2006 008 840 would no longer be sufficient to classify all error types. Embodiments of the invention provide an evaluation of stripe patterns even when the patterns are not periodic. Embodiments of the invention provide (nearly) simultaneous detection and discrimination of different types of errors. By a combination and a selection of features, embodiments of the invention provide better classification results than conventional classification algorithms. The advantage of such a combination and selection approach is a higher degree of generality, i. Embodiments of the invention are better suited to generalize tasks. Depending on the inspection task, the type of stripe pattern that is to be classified changes and, as a result, the relevant features change. By embodiments of the invention that use a combination and selection of features, a diverse number of stripe patterns can be classified, resulting in improved acquisition rates.

Embodiments of the invention provide a combination of different feature types that can be distinguished by two classes, an adaptive class that is optimized for fringe patterns, and a general class that is optimized for texture analysis techniques.

The surface types of the (free-form) object can be varied. The inspection of matte to specular surfaces is addressed by embodiments of the invention.

Embodiments of the invention provide surface quality control by means of an optical sensor element, e.g. A matrix or line scan camera, and structured illumination, also called "strip lighting", is projected onto the surface as a periodic light and dark pattern to be inspected.

By means of a specific, algorithmic method, the defective surface parts shown can be characterized automatically. The characterization can be performed using the time domain, i. H. the value and the division of the pixel values of the image, or the frequency range, d. H. the spectral response of the image to be characterized. Embodiments of the invention may also be used in automatic, linear surface inspection processes.

Referring to the attached Figures 1 to 10, embodiments of a method for selecting an optimized evaluation feature subset for inspection of free-form surfaces and embodiments of a method for inspecting a free-form surface, the is illuminated by structured light.

Fig. 2 Fig. 3 Fig. 3 Fig. 3 Fig. 4 Fig. 5 Fig. 5b Fig. 5b Fig. 6 [0073 ] Fig. 7 Fig. 9 Fig. 10 is a diagram showing a conventional pattern analyzing method; FIG. 10 is a diagram of a method of inspecting a free-form surface illuminated by patterned light according to one embodiment of the invention; FIG. FIG. 12 is a schematic diagram of an inspection method of free-form surfaces illuminated by structured light according to one embodiment of the invention; FIG. shows two exemplary records of a rough freeform surface and a specular freeform surface according to an embodiment of the invention; shows two embodiments of camera-light-object geometries according to embodiments of the invention; shows a plot of a planar structure with a 3D defect at the left end and a 2D defect at the right according to an embodiment of the invention; shows a recording of a plastic tube with a 3D defect and a font according to an embodiment of the invention; shows a reference pattern set comprising three considered strip image data sets Φ0, Φι and Φ2 according to an embodiment of the invention; FIG. 12 shows a reference pattern set having a matched size pattern set and two fixed size pattern sets according to an embodiment of the invention; FIG. shows a tree structure having features to be combined for applying a combination of these features to classify an image according to an embodiment of the invention; FIG. 12 is a diagram of a method for selecting an optimized rating feature subset for inspection of free-form surfaces illuminated by structured light; FIG. and FIG. 12 is a diagram showing detection (classification) rates of different combinations of evaluation features according to an embodiment of the invention.

Fig. 2 shows a diagram of a method for inspecting a free-form surface illuminated by structured light according to an embodiment of the invention.

The method 2 includes a first step 10 of "illuminating a portion of the freeform surface 20 by the patterned light 22," a second step 12 "capturing an image 24 of the illuminated area of the freeform surface 20", and a third step 14 "evaluating each Pixels or a subset of the pixels of the image 24 of the area of the free-form surface 20 with a combination of evaluation features 26 for obtaining a defective or non-defective state 28 of the surface area 20, wherein the combination of evaluation features 26 comprises a first feature set and a second feature set, wherein the first feature set is selected from a first set of fitted features, and wherein the second feature set is selected from a second set of texture features, the first set of matched features being intensity-based features, geometry-based features, and / or fringe-based (or adaptive fringing) features e) features and wherein the second group of texture features transformation-based 9/60 Austrian Patent Office AT 11 770 U1 2011-04-15

Features, structural features and / or statistical features ".

The third step 14 comprises classifying 30 the image 24 by assigning a class 32 to the image 24, wherein the class 32 indicates the defective or non-defective state 28 of the surface area 20.

The classifier 30 includes a first step 34 of "extracting information of the image 24 described by the combination of evaluation features 26 into extracted information 36" and a second step "assigning 38 the class 32 to the image 24 Using the extracted information 36 ", wherein assigning 38 of class 32 to the image 24 is based on a mapping rule 40 derived from a reference pattern set 42 showing surface areas of freeform objects, each pattern of the reference set being associated with a reference class.

The mapping 38 of class 32 to the image 24 may include a filtering algorithm or a wrapper algorithm. The filter algorithm may include a correlation-based algorithm, the wrapper algorithm may include a neighbor-neighbor algorithm or a Naive-Bayes algorithm.

Assigning 38 the class 32 to the image 24 based on a mapping rule 40 may include a learning, setting, fitting, or optimization method.

The assignment rule 40 may comprise a stratified 10-fold comparison algorithm or may include a one-omission comparison algorithm.

The reference pattern set 42 may have patterns of a fitted size or a pattern of a fixed size. The reference class may include a class of non-defective surfaces, a class of defective 2D surfaces and / or a class of defective 3D surfaces. The reference class may further comprise a class of planar shaped surfaces, a class of surfaces formed by perspective distortion, and / or a class of surfaces formed by cylindrical distortion. Examples of the different reference patterns 42 and their assignment to reference classes are given in the following description of the figures.

The free-form surface 20 may have a matte surface or a reflective surface, the structured light 22 may have stripe structures, periodic structures or vertical structures z. B. of visible, non-coherent light. The lighting 10 may be a structured lighting, the z. As coded light used. The record 12 may use a line scan camera or a matrix camera.

The "features" block and the "classification" block of FIG. 1 may correspond to the "classify" step 30 shown in FIG. The feature vector c may correspond to the "extracted information". The pattern F may correspond to the image 24 and the "features" may correspond to the combination of evaluation features 26 as shown in FIG. The class Ωκ can correspond to class 32. The reference set Φ from image patterns may correspond to the reference pattern set 42.

The patterns under consideration need not be vertical or periodic. For high acquisition rates, the defective surfaces should induce different types of stripe deformations than those caused by the geometry of the surface.

Stripe pattern characterization can be accomplished using intensity-based and geometry-based features, but also with the help of other feature types, e.g. By a texture feature analysis. It is not possible to consider all existing feature based image content description methods in the case of free form fringe pattern characterization. Some embodiments of the invention limit the characterization of the free-form strip images by considering the intensity-based and geometry-based features Fs, the Fourier-based features Ff, and their combination Fsf. These feature sets are denoted by Fs, Ff and Fsf. The primary condition for assessing the method for the characterization of free-form surfaces is the definition of an image database, i. E. H. of a set of reference free-form stripe pattern patterns 42. Such regular patterns are only a part of the stripe structures that would be shown in the case of free-form surfaces. Further pattern structures are therefore defined. The simplest and best option is to use existing patterns obtained by illuminating planar surfaces and transform or adjust them so that they can be used to characterize free-form surfaces.

The classification is the last step of the computer vision process. After defining the appropriate illumination, selecting the best record image rendering hardware, automatically segmenting the defective region in the entire image F of the surface to be inspected, and extracting the most relevant features to fill the vector c, the considered surface is classified into a predefined class.

The following terms are used. U is assumed to be the set of all different measurable physical signals. Since pattern recognition usually deals with only one signal type, the task can be defined in a more limited way. Thus, the set includes all possible patterns F that can be generated by a sensor. It is now the task of an automatic inspection system to classify each pattern F into a corresponding suitable class Ωκ among the Nk-defined classes.

^ Ωκ = Ω with fleUQ

Jt = 1 The number of classes Nk that a system can discriminate depends on the inspection task and should be less than the number of realizations of F.

All components of the stripe image classification method are part of a directional and hierarchical data flow whose primary information source is the numerical image sensor data and whose end information is the classification result Ωκ. The processing chain is shown in FIG.

3a shows a schematic diagram of an inspection method of free-form surfaces illuminated by a structured light, and FIG. 3b shows two exemplary recordings of a coarse free-form 300 and a specular free-form 302 surface. Fig. 3a schematically describes the surface inspection principle and Fig. 3b shows the recorded fringe patterns which are produced when a fringe pattern is projected onto the surface to be inspected.

The case of matte and specular freeform surfaces is addressed. In Fig. 3a, the surface inspection principle is shown: a camera C records the object surface S to be inspected, which is illuminated by a lighting L. There are no specific requirements for a relative position of a camera C and object surface S formulated. FIG. 3b shows recordings of a coarse free-form surface 300 and a specular freeform surface 302, both of which are illuminated by a patterned light pattern. Both defective 3D depth defects, as shown in FIG. 3b, can be improved visually with the help of the shown light / dark structures in the images.

The method has a characterization of free-form strip structures that are generated when a fringe pattern is projected onto the freeform surface to be inspected. No calibration procedure of the recording device, d. H. neither the position of the recording camera nor the geometry of the protected stripe pattern must be calculated.

These two images, shown in Figures 3a and 3b, are only examples of the functionality of the inspection process. In order to achieve high collection rates, the strips should be used at the same time. 11/60 Austrian Patent Office AT 11 770 U1 2011-04-15

Perturbations induced by the object geometry may be different from the stripe perturbations induced by the defective surface to be detected.

Surface defects are very diverse, as they may show different types of roughness, 2D structural defects, different texture types, 3D structural defects, or a combination of both. The choice of lighting (s) and sensor (s) is therefore essential.

An important part of non-destructive surface inspection techniques that operate in the visible light range is task-specific lighting, which allows the improvement of defective surfaces without reinforcement of non-defective surface regions. It is known from photography that at least two complementary illumination techniques are used to show different types of defects. As far as structural or texture defective surfaces are to be improved, directional illumination to improve the former defects or diffuse illumination to show the latter defects is desired.

The application of structured illumination techniques generally serves to retrieve depth information of a scene, resulting in a three-dimensional reconstruction, i. E. H. to recover the shape of an observed object. The most common method of structural illumination is to project a particular pattern of patterned light in such a way that the knowledge of the projected pattern combined with the observed deformation of the structure on the object surface allows recovery of accurate depth information of the scene. This method can be further improved by using more complex patterns, such as. As of coded light, color-coded light or a moire projection. The principle of all these methods is the combination of three-dimensional information obtained by one or more calibrated cameras with information contained in the distorted patterns of the projected light pattern. In contrast to these solutions, another method of reconstructing shapes in the scene with only a stripe pattern and a camera can be applied by calculating the surface normals. The setting elements are the structured lighting projector, the recording camera and the scene to be reconstructed. From the calibrated elements and the illuminated scene, the surface normal vectors can be calculated. Then the complete surface shape can be reconstructed.

In contrast, a diffuse illumination technique is used when object surfaces need to be inspected for their texture. The aim of this lighting is to make visible different surface types, which differ in their roughness and / or their color. The former influences the image brightness of the surfaces shown, whereas the latter influences the type and intensity of the color. The option to use gray, eg. For automatic inspection of paper or metallic surfaces, or paint, e.g. For integrity inspection of food items or wood surface inspection images, depends on the application task.

In industrial inspection and quality assurance workflows, the main task of an inspector is to visually classify object surfaces as non-defective or as defective. Since such visual inspection tasks are usually very tedious, tedious and time consuming, more and more machine vision systems are being used for automatic inspection. The two main constraints imposed by an industrial inspection process are the high quality and high throughput of the objects to be analyzed.

The selection of a lighting technique is strongly motivated by the inspection task. Proper lighting is all the more important if it is the first element of a machine vision workflow. Inspection systems of metallic industrial workpieces include a variety of lighting techniques.

There are two distinct, quantitative approaches to visualizing both AT 11 770 U1 2011-04-15

Textural and structural defects on metallic surfaces. Quantitative means in this context that the defective surfaces are detected and not measured, as is the case for qualitative applications.

The first approach uses retroreflective screens to visualize deflections from reflective surfaces. This technique has the advantage of improving both types of defective surfaces (texture and texture), but with the disadvantage that both have a similar appearance in the images, so they can not be subsequently distinguished.

[0010] The second approach requires at least two different illumination techniques. A dual sensor for recording object surfaces with a diffused and a direct light simultaneously may be used for this purpose. By a suitable technique, more than two images of the same object recorded using different lighting techniques can be merged into only one image. The main drawback of these approaches is, of course, that they require more than one lighting. The direct consequence is that their integration into the industrial process is more complex and the data processing chain becomes more comprehensive.

[00107] To address the proposed structural lighting-based inspection task with a general approach, different types of strip geometries are considered. Such strip geometries can be obtained by means of different surface shapes, structured light positions or sensor type. It is not possible to enumerate all combinations of these components and calculate the corresponding strip geometries. Thus, to demonstrate the generalization effect of embodiments of this invention based on the direct interpretation of fringe patterns, it is preferable to demonstrate the principle of the method using a predefined number of fringe deformations. Furthermore, such deformations may be observable in the case of rough and specular surfaces.

For this purpose, two examples of light / dark geometries are shown in FIG. 4. Any geometry can be obtained when a rough or specular surface is inspected. The case of a linear moving object recorded with a line scan camera is considered.

4 shows two embodiments of camera-light-object geometries according to embodiments of the invention. The figure shows two different examples of fringe deformations that arise when "a1" and "b1" rough or "a2" and "b2" specular freeform objects are recorded. These two examples show that, depending on the surface geometry, similar stripe deformations can be observed for rough or specular objects. They also represent the inspection of a surface S illuminated by a lighting L recorded by a line scan camera C during its linear movement along v.

The case of line scan cameras is addressed in Fig. 4, but this does not limit the application of the surface inspection principle according to embodiments of the invention, since the same patterns can be obtained when matrix cameras are used.

Based on these two examples shown in Fig. 4, it can be demonstrated that it is possible to obtain the same fringe patterns for specular and rough surfaces. Thus, the same reference strip patterns can be used to characterize such surface types.

Further embodiments of the invention include automatically processing the stripe images to characterize their contents. The purpose of such an automatic procedure is to automatically classify the image for the quality control of the objects to be inspected.

[00113] The appropriate recording technique of an inspection system is imposed by the requirements of the inspection task. Thus, the number and type of sensors required depends on the scene being inspected (stationary or mobile at varying speeds), the geometry of the recorded object (planar or freeform surfaces), or the task (surface inspection or depth recovery).

Line scan and matrix cameras are the most widely used sensor geometries in machine vision systems operating in the visible light range. If the former require movement of the camera and / or scene, such as In web inspection or in airborne photogrammetry, the latter are generally used for static scene recording and remain the most widely described camera technique in the literature.

Line scan sensors are adequate when recording long objects at a constant speed. Unlike matrix cameras, line scan cameras have a one-dimensional sensor so that objects are recorded piece by piece. This recording principle implies a relative movement of sensor and object. To avoid additional recording distortions, it is preferred to have a constant, relative speed, e.g. By coupling the external trigger of the camera with the speed of the objects. In the extreme case, z. For example, in the air-borne image, when the recording camera follows a random path, the relative position of the sensor can be rectified with the data of a global positioning system.

Since the depth recovery can be performed using a camera coupled with specific illumination, another alternative is to use two or more cameras that record the same scene. This stereo or multi-vision technology can be applied to both matrix and line-scan sensors. In addition, another option for 3D scene reconstruction is to use images from a moving camera, the so-called "structure-out-of-motion" principle. Some specific cameras offer the ability to calculate the height and shape of a surface from a single "smart" sensor. This matrix sensor scans the surface using the deformations of a projected laser line. For each scan, the shape of the projected laser line is used in the depth information retrieval sensor.

Once the selection of appropriate lighting and recording geometry has been made, the recording of the scene to be inspected may be started. To capture its environment, the computer vision system extracts this scene information acquired by the specific sensors and processes it. The representation of the object to be inspected is shown by one or a sequence of images. An image gives planar information about an object so that it can be represented by a two-dimensional function (in the case of a monochrome sensor) whose domain of definition is related to the size of the scene shown and whose values are proportional to the energy of the light. which is reflected by the object and received by the sensor.

The difficulty is to process the acquired information contained in the image and to translate that relevant data into a machine-understandable description. This operation of image extraction for information extraction is part of the image processing. Different image processing parts can be distinguished as threshold operation and segmentation (or statistical) pattern recognition.

Threshold operation and segmentation reveal important information while reducing noise through filtering or other signal processing techniques. These techniques also address the calculation of improved images for better processing. This includes color correction (e.g., brightness or contrast adjustments), conversion to different color spaces, or the alignment or combination of a mosaic of sub-images. Pattern recognition deals with the identification of essential shapes, regions or points from the image. Local acquisition operators, image segmentation methods, morphological operations or geometric transformations such as enlargement, reduction or rotation may be used for this purpose.

The two main image processing themes are image segmentation and feature extraction.

The segmentation step usually consists of partitioning an image into several regions so that they can subsequently be processed by successive operators. In the case of a surface inspection, segmentation algorithms are used to separate a region showing a defective surface from the rest of the image. This operation requires that the regions show different characteristics to the surrounding background. Such characteristics can be recovered from the gray or color values of the image. There are different segmentation approaches. Contour-based approaches consist of determining the transitions between image regions and region-based approaches group pixels together with similar characteristics.

One possibility of contour recovery is to scan the image with a window defining a region of interest. An edge detection operator is applied for each window position to detect significant transitions in the image. The detection criterion is most often the maximum of the first derivative or the zero crossing of the second derivative. Edge detectors and corner detectors can be applied. These detectors do not lead to a perfect segmentation of the image, and thus further operations may be necessary, such. B. Contour closure and contour thinning. The former can be performed by analyzing the local density of the edges, comparing the texture on each side of these edges, or using morphological operators. A morphological approach can also be applied to contour dilution, e.g. B. with skeletonizing filters.

Also, a connectivity-preserving approach can be applied, the z. B. refers to active contour models (which are also called Snakes). These algorithms begin with an initial boundary shape, represented in the form of spline curves, for iteratively modifying it by applying various shrinkage / expansion operations according to an energy function.

While the contour-based approaches are based on large intensity differences between adjacent pixels, region-based methods use the similarities of gray levels.

Threshold-based methods are another region-based segmentation technique. They take into account that a monochrome picture can consist of different regions. It is assumed that each region corresponds to different gray level ranges identified with one or more thresholds. The challenging task is to automatically determine these pixel thresholds to separate object regions from background regions using local or global gray level distributions. Statistical approaches based on the minimization of a criterion function can be applied. Solutions for using threshold determination techniques on color images also exist.

Other region-based methods include merging similar pixels (or region growth), splitting different regions, and calculating columns and merging. The principle of all three methods is building (merging) and / or splitting (splitting) image regions using a homogeneity criterion. The algorithms stop when, for each merged region, no pixel in the neighborhood, or for each split region, no image region, respects the criterion. A simple decision criterion is based z. On analyzing local histogram functions.

Instead of classifying the segmented regions directly into given classes, an intermediate feature extraction step ("features" as shown in FIG. 1) may be used, the one described in US Pat serves for two purposes. The first purpose is the data reduction. In fact, processing large or multiple image regions would require a large amount of memory and computational power for the classification algorithm. The second purpose is to increase discrimination information. The information shown in the segmented regions in the form of gray level images is not necessarily optimal for the classifier. They are transformed into structured information so that all elements of the same class share similar properties and all elements of different classes have different properties. This step, based on the analysis of image characteristics, is called the feature extraction process. Different techniques for extracting different types of features from an image can be distinguished. Which features are relevant and how these features are calculated often depends on the specificity of the inspection task.

Feature extraction methods consist of intensity-based and / or geometry-based approaches. Geometric features can z. For robot vision systems, audiovisual speech recognition or document processing. Other applications may use a combination of geometry-based and intensity-based features, e.g. For video surveillance, medical image analysis or face recognition.

Four different feature categories for describing image regions can be distinguished. These feature categories are based on whether only the spatial, gray level, or combined spatial and gray level distribution is evaluated. Morphometric features describe the size and shape of the regions to be characterized, densitometric features are calculated from the gray level intensities of these regions. Two other feature types can be considered: one describes the texture aspect of the image regions and one characterizes the relationship between the different, selected image regions.

While texture characteristics can form a cohesive feature set, the texture analysis field covers the processing of a particular image type. Thus, the information contained in such texture images may be much more complex and diversified than the information shown in non-texture images.

Thus, texture analysis is a more general approach to feature extraction. The spatial distribution of gray levels can be used to recover the features, and thus integrate both intensity and shape characteristics. The image can also be transformed from the scale space to another base, such. As the Fourier or the wavelet range. The former refers to the geometry of the feature, whereas the latter also integrates the scale characteristic.

Texture features are often subdivided into distinct, specific sentences according to the feature calculation methods involved. Statistical, geometric, spectral and model-based texture features can be taken into account. Texture features can also be divided into a structural approach, a statistical approach, a model-based and a transformation approach. An advantage of general texture features is that they are applicable to different types of images and content.

Each pair of textured images may be distinguished using a characteristic feature related to the spatial distribution (geometry-based) or the gray-level distribution (intensity-based) of the object and / or the background.

The average and the variance are the two first moments of the gray level distribution in a selected region or contour. The contrast is proportional to the difference between the intensity of the selected region or contour and the background intensity. These three dimensions are important, intensity-based features for differentiating image pairs. The orientation indicates the main direction of the image structure, the scale is a measure of the size of the objects shown, whereas the shape z. B. the "circularity" of the shown 16/60 Austrian Patent Office AT 11 770 U1 2011-04-15

Can quantify objects.

Of course, all of these six features are not specific to this type of image. Many applications use similar features, sometimes suggesting the combination of geometry-based and intensity-based characteristics.

Also the average and variance characteristics for the comparison of images can be used. For example, a set of intensity and geometry features may be used to describe the image. The contrast feature can z. B. can be used to reduce the image artifacts, z. For example, in digital television. Contours of connected components can, for. B. can be used to calculate the landmarks in the image. A method based on the Fourier transform of the images may be used to calculate the relative position, orientation, and scale of a pattern. A wavelet based method may e.g. For example, to determine a structural primitive in an image having a regular texture.

The extracted features can be used for the classification process. The feature vector c as shown in Fig. 1 contains all the extracted characteristics of the pattern F.

Fig. 5a shows the recording of a planar structure 500, 502 with a 3D effect 510 at the left end and a 2D defect 512 at the right. It can be clearly seen that both defective surface parts can be visually improved and distinguished.

5a shows the images of a scene illuminated using the diffuse ambient light 500 (upper part of the figure) and a structured light pattern 502 (lower part of the figure). This scene consists of a 3D defect 510 and a 2D defect 112, both of which are positioned on matte surfaces.

Fig. 5b shows the recording of a plastic tube 504 with a 3D defect, a font and the background region. All these three surfaces can be distinguished by the projection of the stripe structure.

It can be easily observed how different defective surfaces, 3D 510 and 2D 512 in the case of Fig. 5a, can be improved using a projected light pattern. This surface enhancement principle can be adapted to any surface type (eg, reflective or dull) and to any surface shape (eg, planar or cylindrical). Fig. 5b shows the surface improvement of a specular plastic tube 504 having a striped pattern.

Both in the case of specular 504 and dull 500, 502, the following principles apply: the deformation of the surface shape, characteristic of 3D defects induces a geometric deformation of the corresponding stripe structure. A change in surface roughness, synonymous with 2D defects, provokes a change in the intensity of the observed fringe pattern. A surface may therefore be characterized by its direction, given by its surface normal and its reflectance, expressed by the reflectivity coefficient p. Since p expresses the ratio of reflected to incident light flux, its values are between 0 and 1. The direction and intensity of the reflected light flux are therefore directly related to the geometry and reflectance of the illuminated surface.

Nevertheless, to observe the projected light pattern, there is a fundamental difference in the recording technique between these two surface types. With regard to dull objects, the cameras observe the light pattern on the surface to be inspected. For reflective surfaces, however, the camera records the light pattern reflected by the inspected surface. Thus, the camera focuses on the surface to be inspected when it is dull, and focuses on the light screen when the surface is highly reflective.

For both surface types, the positions of the camera C and the exposure screen L, but also the size of the screen and the aperture of the camera are selected so that the perturbations, the are induced by defective surface elements in which the recording sensor is shown.

The detection of 3D defects is based on the variations of the surface normal as far as highly reflective surfaces are concerned. Depending on the positions of the recording camera and the illumination screen, very low deviations of the surface normal can be detected. With respect to matte surfaces, the detection of defective 3D surface parts based on the projection of a stripe pattern on the positions of the surface in the scene coordinate system.

Defective 2D parts are synonymous with a change in the reflection coefficient p. Thus, these defects can be visually enhanced for both specular and matte surfaces, as the variations of the gray levels in the image are directly related to the variations in the reflected light flux on the surface.

With respect to the inspection of specular and matte objects based on the projection of a fringe pattern, the following two conditions should be satisfied: a periodic light pattern is projected onto the image, and each defective surface induces a visible disturbance of the fringe pattern shown.

Thus, the inspection of a surface Sinspekt for specular and dull objects has the detection of defective surfaces S3D and S2d, whose size is greater than a minimum size S3D: min and S2D, min · [00149] The investigation of the most suitable features, which describe the strip parts is an essential and important step. Thus, to compare different classification approaches, the same reference image patterns are used.

[00150] When the illumination technique has been optimized to show the surfaces with a simple stripe pattern in the images, artifacts arising from non-constant recording conditions (illumination variations, object positioning tolerances, etc.) may be unavoidable. Thus, to be representative, the considered reference strip image databases include images of defective 3D and 2D surfaces, images of acceptable surfaces, and images with recording artifacts (in the case of specular surfaces).

All of the visually enhanced surfaces shown in Figures 5a and 5b can be automatically characterized using the methods of embodiments of the invention.

The inspection process of free-form surfaces has the automatic detection and classification of defective surfaces of free-form objects.

For freeform objects z. For example, two types of defects can be distinguished. The first kind of defects are synonymous with a change in the shape or geometry of the objects, such surface parts are called a 3D defect. The second type of defects is called 2D defects because they are characterized by a different texture or color of the metallic surface. Fig. 5a shows a plot of a planar structure where these two types of defective surfaces are shown. The challenge of automatically detecting and distinguishing such defective 2D and 3D surfaces shown in Figure 5a is defining a machine vision technique that meets industrial inspection requirements. Thus, an adapted hardware and software solution can be defined and developed for this specific inspection task.

A structured illumination can be used for the visual enhancement and differentiation of different parts of the surface to be inspected. When the common approaches use structured illumination to show three-dimensional defects, in combination with smoothed illumination to enhance two-dimensional, defective surfaces, embodiments of the invention accomplish both tasks with only one illumination technique. A necessary condition is that the defective surfaces, which are to be distinguished, induce characteristic deformations in such a way that these surfaces can be automatically classified by the image processing method. Thus, a presetting process of the recording device may be necessary. A knowledge of the three-dimensional position of the recording elements is not necessary.

The success of such an inspection method according to embodiments of the invention, which uses the strip deformations to distinguish defective from non-defective surfaces, may mainly depend on the geometry of the stripe pattern shown in the image. The robustness of the method depends on the distinctness of the projected light pattern as far as such pattern refers to a non-defective or a defective surface. These distortions of the pattern should be mainly synonymous with a defective part. Further distortions caused by the geometry of the inspected surface or the position of the recording sensor should be avoided as much as possible.

Thus, embodiments of the invention based on the interpretation of fringe disturbances are hardly dependent on the geometry and arrangement of the framer elements (inspected surface, lighting and recording sensors). In fact, all of these elements have a direct influence on the striped pattern geometry shown. The surface to be inspected and the illumination may be positioned to show a basic light pattern in the recording sensors. The image stripe pattern being considered may consist of a periodic and vertical structure. This projected light pattern is used for the interpretation of the surfaces to be inspected. The two types of defective object parts described above: 2D and 3D surfaces are considered.

The former are synonymous with a structural change of the surface, whereas the second correspond to a different texture or roughness of the surface. With regard to the different types of surfaces that can be inspected using the method according to embodiments of the invention, highly reflective and matte surfaces are considered. In the case of specular surfaces, the angle of the light beam reflected by the surface is equal to the angle of the incident light beam with respect to the surface normal. In the matte case, each elementary surface can be assimilated to a Lambertian emitter, i. H. that the incoming light beam is reflected spherically and evenly.

Fig. 6 shows a reference pattern set comprising the three considered strip image data sets. Φ0 is the reference set where the stripe structures are periodic and vertical, Φι is the set corresponding to the curved patterns of the set Φ0 with a perspective distortion of type -1-, and Φ2 is the set corresponding to the curved patterns of the set Φ0 corresponds to a cylindrical distortion of the type -2-. All patterns were manually classified into three classes ΦΑ, Φ ^ ο, Φι ^ ϋ, which correspond to non-defective surfaces and defective surfaces with 3D and 2D defects. The reference images for the three considered sets Φο, Φι and Φ2 serve as reference patterns for the classification of free-form surfaces. The images of the set Φ0 are images recorded by a conventional planar surface lighting process, whereas the images of Φ! and Φ2 were artificially curved to simulate the required distortions. Φ0 corresponds to patterns without distortion with respect to the shape of the object. Φι corresponds to patterns with a -1- type distortion, Φ2 corresponds to patterns with a -2- type distortion, All patterns have a size of 64 x 64 pixels and have been classified into three classes ΦΑ, Φ ^ and Φκ, 20. All sentences consist of 252 images.

All 252 patterns of the three image sets Φ0, Φι and Φ2 are characterized using the three feature sets Fs, Ff and Fsf. To evaluate the characterization possibilities offered by each of the feature sets, the classification results (acquisition rates in percent) of these three image sets are listed in the following table: 19/60 AT 11 770 U1 2011-04-15 Austrian

Patent Office

Distortion type / feature set □ o no distortion □ 1 perspective □ 2 cylindrical Fä 90.4 78.1 88.4 Fh 89.2 85.7 87.7 FSh 93.6 86.5 88.4 [00160] The results show first that in general high classification rates can even be obtained if the images are distorted. This demonstrates that the proposed feature set can be used for the characterization of free-form surfaces by the methods according to embodiments of the invention.

From the above table showing the acquisition rates, it can be seen that the combination of different feature types, here in the case of feature set Fsf where intensity-based and geometry-based features are combined with Fourier-based features, result in improved acquisition rates.

Embodiments of the invention use various feature-based techniques to characterize the free-form strip structures, such as, e.g. For example, texture analysis methods, intensity and geometry based methods, and other methods may be combined with one another or with other different feature-based approaches to improve acquisition rates.

The elements of the reference strip image data set, as shown in Fig. 6, were obtained by the industrial application comprising the structured illumination, as shown in Fig. 3a. To train and evaluate a classification algorithm, a representative set of annotated reference images is necessary. The reference image data set is used for the evaluation of the different algorithms and describes the defects that are recorded by an automatic inspection system. The images can be used for the qualification of the system and have been commented by an inspector into non-defective and defective surface classes and subclasses.

The reference data set Φ as shown in Fig. 6 consists of NF = 252 patterns. Each shows only one type of defect belonging to one of the Nk predefined classes Ωκ G □ Ω. These patterns are classified into three different classes {ΩΑ, QRi3D, DR2d} G □ Ω. Patterns showing non-defective surfaces are grouped into subclass A. These patterns show the object surfaces recorded with ideal and non-ideal recording conditions, but also all non-critical 3D defects. Patterns showing defective surfaces consist of the two different subclasses QR3D, QR2d- These classes correspond to the defective surfaces with 3D and 2D defects.

Fig. 6 shows the different classes describing the stripe patterns, and also gives examples of typical stripe images to be classified. Each class represents a type of recorded surface. The eighteen images of Figure 6 are grouped into three distinct classes {ΩΑ, QRi3D, Qr2d}. The non-critical object parts are of class ΩΑ, the critical depth defects are of class QRi3D and the critical surface defects are of the class DR2d- [00166] A typical "no noise" fringe pattern representing a surface illuminated under ideal recording conditions is shown in Fig. 6 (first column). Incorrect or imperfect recording (eg, poorly positioned or misaligned object with respect to the image sensor) causes "symmetrical deviation" of the stripes (shown in Figure 7). External graininess, compressed on the surface of the object during its manufacture, results in circular streaking. Depending on the size of the graininess, the disturbances "circular small size", "circular average size" or "circular large size" may occur, as shown in Fig. 6 (second column), (third column), (fourth column). A "low contrast" of the strip, see Fig. 6 (fifth column), is mostly due to marks. However, deeper abrasions induce local "shrinkage" of the strip, see Fig. 6 (sixth column). In the case of free-form surfaces recorded with line scan cameras, the recorded object may be different from the object profile at a time t, at a different time t + 1 is recorded. Thus, the observed stripe structure in the image may not remain periodic and vertical in the image. This happens when the shape of the surface to be inspected is not symmetrical along the direction of movement of the object. Consequently, a comprehensive range of stripe structures can be considered. Enumerating all possible stripe deformations would hardly be possible. Thus, it is preferable to focus on a limited and predefined number of stripe deformations. Therefore, two different types of deformations -1- and -2- are considered. Two different types of variations -1- and -2-of the stripe structure shown are considered, which may be caused by the varying geometry of the recorded object and not by the presence of a defect on the surface.

The first type of distortion (second line of FIG. 6) induces a deformation of the stripe structure in the images such that the period of the stripes is the same along the u-axis of the image and from top to bottom along the v-axis. Axis of the image increases. Type -1 perspective stripe distortion corresponds to the recording of planar surfaces that are not parallel to the ground plane. In this case, the degree of distortion is proportional to the angle α between these surfaces and the ground plane (a non-distorted fringe pattern corresponds to an angle of 0). At least four different degrees of distortion corresponding to four increasing values of α are considered.

The second distortion (third line of Fig. 6) causes a deformation of the stripes, so that the variations of the fringe period are the same along the v-axis. The period is maximum in the middle, vertical part of the image and decreases left and right to the edges of the image. Type -2 cylindrical strip distortions are due to the recording of cylindrical shapes. In this case, the degree of distortion is proportional to the curvature κ of the surface (a curvature of 0 corresponds to undistorted stripe patterns). Again, four different degrees of distortion corresponding to four increasing values of κ are considered.

Conventional methods based on a structural exposure technique perform the surface inspection by means of the strip deformation interpretation without 3D depth recovery. Embodiments of the invention combine the inspection process with the 3D scene information and provide an enhancement to the surface inspection process.

Two different types of 3D defects can be considered. According to their depth, surfaces are classified as not defective or defective. Thus, a combined, adaptive and depth recovery approach allows classifying the image patterns in accordance with the perturbations shown. In a first pattern arrangement χι all 3D defects can be considered as defective at a first time (all lead to similar disorders). Then, in a further step, the non-defective and the defective parts can be separated according to their calculated depth. In a second pattern arrangement χ2 for the classification of the reference image pattern, each 3D defect can be regarded as defective.

With regard to the classification of free-form surfaces, it can be observed that the detection behavior decreases with an increase in the distortion in the images, whether they are of type -1 or type -2-. Second, the use of the combination of evaluation features 26 results in better classification rates for both types of distortion.

However, improvement of the classification rates when using the wrapper (envelope) -1-NN feature selection method can be particularly achieved with respect to the distortion of type -1-.

Furthermore, it should be noted that an improvement of the classification rates can be achieved if a pattern arrangement χ2 is addressed instead of a pattern arrangement Xi. The achieved classification rates CP with the aid of a distortion of the type -1- lie between 85.7% and 91.2% for a χι arrangement and between 88.1% and 21/60 Austrian Patent Office AT 11 770 U1 2011-04-15 92.1% for an X2 arrangement. With respect to type -2- distortion, the classification rates CP are between 87.6% and 91.2% for an x ^ array and between 88.5% and 93.3% for an x2 array.

Fig. 7 shows a reference pattern set according to an embodiment of the invention. The reference pattern set has seven embodiments of image sets, one of the adjusted size Φ381 and two of the fixed size Φ64 * 2 and Φ128 * 2.

Manually setting the size of each pattern so that only the disturbed streak structure is shown was a subjective process. In fact, the defect surfaces shown are usually not marked by sharp contours. Therefore, three different sets of patterns are considered, the size of which is adapted to the size of the stripe structure to be classified. Since this selection process is not objective, each sample set was selected by a different person. Two further sets of patterns with fixed sizes of 64 x 64 pixels and 128 x 128 pixels are considered. Thus, the perturbations shown in the 252 reference pictures were preselected five times, so that at least five different sets of pictures, three with adjusted sizes Φ381, Φ382, Φ383 and two fixed sizes Φ64Λ2, Φ128Λ2 can be used. For image sets Φ64Λ2 and Φ128Λ2, the fringes are not necessarily at the center of the pattern. Fig. 7 shows seven examples of image sets Φ381, Φ64 * 2 and Φ128 * 2.

The resolution in the horizontal direction is three times larger than the resolution in the vertical direction. The patterns of the image set φ "* 2 correspond to an object surface of 2 mm width and 6 mm height, whereas the patterns of the image set Φ128Λ2 correspond to an object surface of 4 mm width and 12 mm height.

It is known that the chances of achieving the optimal computer vision process increase with the score for each part of the process, from different methods based on different mathematical models or opposite methods. However, trying out all possible methods is not feasible because the number of possible combinations is doubled each time a new method is involved. Consequently, a limited number of methods are considered for the three parts of the process.

For this purpose, different sizes of segmented regions (eg, three adapted and two fixed, as shown in Figure 7), adapted feature recovery approaches with different settings, different classification algorithms, and different classification methods may be used.

FIG. 8 shows a structural tree (feature tree) having features that may be combined to apply a combination of these features 26 to classify 30 an image 24.

[0018] A first set of features 60 and a second set of features 62 are combined into a combination of features 26. The first feature set 60 has adaptive features 64. The second feature set 62 has texture features 80. Adaptive features 64 and texture features are associated with a level III feature of the feature tree. The adaptive features 64 include strip-based features 82 and fringe-based features 102. The texture features 80 include structural features 84, statistical features 86, and transformation-based features 88. The strip-based features 82, the adaptive fringing features 102, the structural features 84, the statistical features 86, and the transform-based features 88 are associated with a level II of features of the feature tree.

The strip-based features 82 and the adaptive fringe features 102 include geometry-based features 94, intensity-based features 98, and statistic-based features 99 associated with a level I of the feature tree. The geometry-based features 94, the intensity-based features 98, and the statistics-based features 99 have pixel-based features 100 and sub-pixel-based features 96 associated with a level 0 of the feature tree.

The transformation-based features 88 include Fourier-based features 90 and wavelet-based features 92 associated with the I level of the feature tree.

Structural features 84 may e.g. Based on the method of "Y. Chen, M. Nixon and D. Thomas, Statistical Geometric Features for Texture Classification, Pattern Recognition 28 (1995), No. 4, 537-552 ", whereas statistical features 86 e.g. The methods can be based on, for example, Robert M. Haralick, K. Shanmugam and Its'hak Dinstein, Texture features for image Classification, IEEE Trans, on Systems, Man, and Cybernetics (SMC) 3 (1973), No. 6 , 610-621 "and" Michael Our, Sum and Difference Histograms for Texture Classification, IEEE Trans, on Pattern Anal, and Machine Inte. (ΡΑΜΙ) 8 (1986), No. 1,118-125 ".

The texture features 80 include transformation-based features 88. The transformation-based features 88 include Fourier-based features 90. For example, a number of 33 Fourier-based features 90 may be applied to classify 30.

Three different texture classification approaches 80 can be used to inspect freeform surfaces: the statistic-based 86, the texture-based 84, and the transformation-based 88. The word "texture" has many meanings; it can be a wide range of different Describe surface types, and although it is a general term in the field of image processing, it has not been defined precisely. For our purposes, the goal of texture analysis is to describe textured images by using local or global image properties, which refers to a representation of two basic dimensions: clay primitives that make up the texture, and their spatial organization. The following four categories of texture approaches are considered. The structural approach considers textures as a set of primitives (microtexture) whose hierarchies and spatial arrangements (macrotexture) are defined by placement rules. This approach can be used for synthetic images, but turns out to be poorly defined for natural textures where the variability of macro and microtextures is often too large and thus difficult to represent.

A more complex method first binarizes the image for different thresholds. The geometric properties (irregularity and quantity) of the connected regions for each binary image are calculated. Statistical characteristics are then calculated for the entire binary images. In contrast to structural methods, statistical approaches indirectly represent a texture through properties that govern the distribution and relationship of image intensity levels. The features are obtained directly from statistical calculations of the original image or from the derived images. Local features are easily and quickly calculated: mean, variance, minimum and maximum gray level values, median and corresponding characteristics for the derived filtered image.

Different split filters like the Sobel or Kirsch operator are used. Another method uses second-order statistics and is based on the grayscale matrix, which is a two-dimensional histogram of all pixel pairs. This calculation is generally performed for four different directions so that at least 14 different statistical features are calculated from the matrices.

In order to reduce the computational complexity of this method, two one-dimensional histograms representing the difference and summation of the pixel pairs may be used. A total of 16 different statistical features can be calculated from four different directions and for both histograms.

Another method is based on a run length. The run lengths are calculated in four different directions (0 °, 45 °, 90 °, 135 °). A two-dimensional histogram representing the length of the pixel strings and the intensity level can be constructed for each direction. Five different statistical features can then be calculated for each histogram, giving a total of 20 characteristics. A reduction in the number of intensity levels may often be necessary to give high run length values.

[00191] Another approach describes textures by fractal and stochastic models. The parameters of this model approach are estimated and then used for image analysis. The fractal approach gives good results describing natural textures. It is used and describes objects with a high degree of irregularity. An example of a stochastic model is the Markov Random Field, which takes into account that each pixel in the image can be modeled by the neighboring ones.

The transformation approaches 88 use features that are calculated from the transformed image. The Fourier 90, Gabor, or wavelet transform 92 are the most popular. The Fourier approach 90 uses the local parts of the frequency spectrum. The features are obtained from the Fourier spectrum for different regions. Because Fourier features 90 suffer from a lack of spatial localization, better results are often achieved through Gabor features that integrate spatial localization.

[00193] Gabor wavelets are defined for different directions (0 °, 45 °, 90 °, 135 °) and for different lengths. The most efficient of the transformation models is the wavelet transform, which integrates frequency, spatial localization and different image scales.

The range of applications in texture analysis methods 80 is very broad, as it relates to meteorology, medicine, industry, military, etc. A method for classification of rock uses z. B. spectral features and features that are calculated from gray value matrices. Satellite terrain image classification may be based on texture features, e.g. By connecting a model-based and a transformation texture classification approach. Satellite images of clouds can be z. B. be classified using statistical, structural and transformation models. Neural classifiers with statistical features can be used for breast cancer detection with mammographic images. Morphological texture techniques may e.g. For cervical spinal cord classification. Statistical features, such as B. Mean and standard deviation of intensity levels within a window can be used to determine e.g. B. Classify military targets in infrared images.

Texture-type classification algorithms 80 can be used in a wide range of applications. Homogeneous images on one side, such as rock, cloud, breast or even infrared images, on the other hand finer details such as cell or terrain images.

This demonstrates that a priori may not be able to know which method is best suited for a classification task where specific textured images are affected. It may therefore be necessary to test and compare different texture approaches to find the most suitable one. Different texture databases can be used to evaluate texture algorithms. These include regular to stochastic, different textural or texture gray level or color images.

All of these texture analysis methods 80 are top-down approaches, meaning that they have been developed for a wide range of texture types. Each affected texture analysis method is adapted to the patterns to be analyzed, i. H. the geometry and the gray level distributions of the stripe structures shown. For each texture analysis approach, different, adjusted parameter settings are considered. Each setting is evaluated using the classification rate.

Each image pattern F is considered to be represented by a discrete function fki. fw is the result of the discretization and quantification of a two-dimensional, continuous function fuv. The u-axis, which is horizontal with increasing values from left to right, and the v-axis, which is vertical with increasing values from top to bottom, describe a Cartesian coordinate system. An upper left pixel at pixel position (0, 0) of F corresponds to the origin of the coordinate system.

The value f (u, v) of a continuous fuv at position (u, v) after quantification 24/60 > Austrian Patent Office AT 11 770 U1 2011-04-15 sets the gray value at position (kAu, ΙΔν) of the discrete function fk! F can be represented as a matrix [f (k, l)] with Mu rows and Mv columns. Au and Av are the scanning distances in the u and v directions. Both can be considered equal to 1, so that the following expression defines the mathematical representation of an image: the quantification process

Transforms all values f (u, v) G □ R into the next integer value f (k, l) G □ N, defined by the Ng gray levels of a quantification, where 0 < f (k, l) < (2Ng -1). Ng is also called the depth of the image pattern F. Image representations in the time domain are described using these notations.

For an image F with Ng intensity levels, at least Ng different, binarized images Fb, 0b can be calculated. For each binarized image Fb, 0b, the structure-based features can be calculated as follows. All monovalent pixels are grouped into a set of interconnected pixels called connected regions. The same is done for the zero-valued pixels. Two numbers can be defined, the number of connected regions of monovalent and zero value pixels for the binarization level 0b and NOC1 (0b) and NOCO (0b), respectively. Then, a parameter can be introduced that defines the irregularity (or incompleteness) IRGL of each connected region. By calculating the average of IRGL, two more numbers are obtained for all 1 and 0 connected pixel groups of image Fb, 0b and IRGL1 (0b) and IRGL0 (0b), respectively.

Each of these four numbers are functions of a binarization threshold 0b. These functions are further qualified with four statistics, the maximum value, the average value, the sample average and the sampling standard deviation for each of the four functions of 0b, NOC1 (0b), NOCO (0b), IRGL1 (0b) and IRGLO (0b) are. For each image Fb, 0b at least 16 different features can be obtained which define a feature vector co, (0b) e □ R16.

[00203] The advantages of using different binarizations are two. First, it provides the ability to view different intensity levels of image regions and thus integrate the intensity information into the feature calculation. Then it allows you to search for the best settings, d. H. using only those binary images that lead to the best classification results.

Such images have the peculiarity that the gray level distribution appears to be generally bimodal, since light and dark stripe structures are shown. Thus, the fringe images can be completely described using only a few binary images.

Statistical features 86 are e.g. For example, the Spatial Gray Level Dependence method (SGLDM). The latter is an extension of the structural approach. The assumption is that the texture context information is contained in the spatial relationship that the gray tones have with each other. This is a different approach than the structural methods, since it describes the shape and spatial organization of the basic elements of the texture.

The relationship of texture primitives for each image F is evaluated with one or more gray level matrices (or gray scale space dependence matrices). Each gray level matrix is a two-dimensional histogram of all pixel pairs in the u and v directions in the strip image F. These matrices are symmetrical, which indicates that the gray level matrices of an image F and its rotated by 180 ° are the same. Therefore, a pixel pair calculation is performed only for the four different directions 0 °, 45 °, 90 °, and 135 °. 14 different statistical features can, for. B. used to z. B. the homogeneity or the 25/60 Austrian Patent Office AT 11 770 U1 2011-04-15

Express contrast of an image. These features are retrieved from each gray level matrix and stored in a feature vector C (du, dv) e □ R14, where you and dv are the considered distances in the u and v directions.

The idea behind the selection of these feature parameters is the modeling of the human perception of texture. The sum and difference histograms can be used as an alternative to the grayscale matrices. With the sum and difference histograms, 9 out of 14 statistical features can be calculated directly, with the other 5 being able to be estimated.

For the classification of stripe images, 15 different features can be calculated from the sum and difference histograms, where 8 features are global histogram features and 7 features are the equivalent features to those calculated from the gray scale matrices. All features are stored in a feature vector C (du, dv) G □ R15, where you and dv are considered as distances in the u and v directions. The key point of SGLDM techniques is that they characterize the relationship of a pixel pair intensity. The selection of pixel pairs, d. H. their distance along the u and v axes is therefore crucial for a good classification. Thus, the most appropriate values of these two distances should be defined according to the geometry of the stripe structures that are to be characterized.

The Fourier transform F is a transformation between an input space (spatial area) into an output space (frequency area). The inverse is called the inverse Fourier transform F'1. The Fourier transform is a generalization of the theory of the Fourier series and is applicable to all non-periodic and integrable functions.

For the discrete image F, the Fourier transformation and the inverse Fourier transformation are described by the following equations:

ί · = [Λι] f = IM Σ Σ «= Mu -1; λ o, 1, .... Mv - 1 mv-i λ /, »- i vV Σ Σ A«. Λί "Λ3 '· - ** λ = 0 k = 0,1.Μ * - 1; l = 0,1, ..., Mv -1 [00211] An important remark with respect to the Fourier transform is that when the samples of the matrix [fkd] of the input time frame F are true values ([fkj] G □ R ), the samples of the matrix [f, k] of the output frequency image FA are complex values ([f ", k] 6 C), so that the information of a Fourier-transformed image is included in the size and color of the spectral representation , In many applications, only the size information is needed and the phase information is discarded. However, in spite of this general practice, in some context, phase information should not be ignored. If we For example, to construct synthetic images consisting of the size information of one image and the phase information of another, it is the image that corresponds to the phase data that we perceive, albeit slightly worsening. In our case, only the size of the Fourier transform is considered, denoted | FA |. Their representation is performed in a Cartesian coordinate system whose vr axis is horizontal with increasing values from left to right and v2 axis is vertical with increasing values from top to bottom.

[00212] Images F and | FA | have the same dimensions. | AF | can be described with a two-dimensional, continuous function fA whose quantified values fA (vd, v2) at position (vi, v2) represent the size of the spectrum at position (κΔν-ι, κΔν2), where κ e {0, 1, ..., Mu-1} 26/60

Austrian Patent Office AT 11 770 U1 2011-04-15 and {λ e {0,1 Mv - 1}. Δνι and λv2 are the sampling frequencies for the vr and v2 axes, which are considered equal to 1 for simplicity. Then the following expression defines the mathematical representation of an image in the frequency domain:

ί! =! Ι / Μ) |], «e {0, lf ··· Ae {0,1, ... 3 / ,, - 1} [00213] All discrete images | FA | are represented with Ng levels of quantification and normalized such that 0 £ ίΑ (κ, λ) < 2Ng -1).

The Fourier analysis is used here for the characterization of the stripe structures shown in a pattern F. This pattern can also be interpreted as an observation window of the strip structure to be analyzed. It is known from signal theory that restricting the representation of a signal s to be characterized induces a bias of the spectral representation of s. This bias is all the more important if the size of the observation window is limited. In the case of stripe pattern characterization, this means that the resolution in the frequency domain is directly related to the size of the pattern in the space area. This implies that the resolution in the frequency domain is equal to 1 / MU along the vr axis and 1 / MV along the v2 axis.

For a texture analysis 80 with the Fourier transform 90 different parts of the frequency spectrum can be used for classification purposes.

The characterization of the image pattern F is based on the Fourier power spectrum P, which is expressed as the square of the size of the spectrum | FA | is defined. P is a matrix of the same size as matrix FA, Ρ (κ, λ) is the value of the power spectrum at position (κ, λ). The following definition can be used:

The power spectrum may be used as an image signature for distinguishing different types of fringe images. The radial and angular spectral distributions may be taken into account, the former being sensitive to texture granularity and the latter being to texture rectification. Further, the distributions corresponding to the fundamental spectral image axes, the vr and v2 directions, can be used.

Using different parts of the power spectrum, it can be observed that some regions may be more characteristic or illustrative for certain classes of fringe images. Such images are characterized by a vertical pattern whose perturbations are synonymous with defective surfaces. Thus, isolating the regions of the power spectrum corresponding to the vertical stripe pattern for the calculation of the characteristic features leads to better classification rates.

An embodiment of the invention uses a Fourier-based classification with a total of 33 features 90, which corresponds to the total value of the power spectrum regions. There are 8 radial regions with Pr1r2, 10 directional regions with θθι, θ2, 5 horizontal regions with Pv1jV2 and 10 vertical regions with Pui, U2: 27/60 AT 11 770 U1 2011-04-15 Austrian

Patent Office

Pri, rs Σ i / («A) P Puj,« 2 - ^ l / KA) f ο <κ · <ΜΜ ;; ί.'ΐ <X <t> a = Σ IA *. *) f «1 <λ <« 2ί> <ιί <Μν [00220] The number of calculated features is independent of the size of the image pattern. The mathematical formulation of the feature vector cF □ R33 is as follows:

Cp = {{Prl.rl h {P I? L, 02} 1 {PV1. d2} &gt; {Pul, ti2}}, Cp €! 33 with € K®, {Pfll.uzl μ * and (ι · ι; ** ζ) =: (jr; (y + ljr), r = {^ 2 + ψ %% 3 = {0, .., 7} ΨΜ = = (i + 1) $) * O &amp; II <3 * 3 = {0,..., 0} (ii; ί'Ξ> = = O; {; +1) '-'). V = ψβ, 3 = {0, -J} (w 11¾) = = in Ci + *) ") &gt; u = ^ / io, J = {0, ..9} The DFT (Discrete Fourier Transform) made spectral analysis of discrete images possible by decomposing the image function into a sum of finite sinusoids. However, with the major drawback that, if it allows good spatial resolution, it is not possible at the same time to have a good resolution in the frequency domain, this phenomenon is known as the Heisenberg inequality or uncertainty principle.

An improvement in space-frequency representation was achieved with wavelet analysis 92. This signal decomposition method is based on a slightly different concept than the frequency: the scale concept. Instead of looking at a window of constant size shifted at different positions in the image, the wavelets have the same morphology (they are all similar, but differ in size). Wavelet decomposition is a multi-resolution approach and consists of writing an image signal into a superposition of scaled and translated wavelets.

The main difference to the Fourier technique 90 is that the wavelets 92 can be adapted to the size of the image characteristics sought: they are large for low frequencies (high spatial resolutions) and small for high frequencies (low spatial resolutions). This adaptability of the wavelets is used for analyzing the information content of image information and is called the multi-resolution procedure.

The orthonormal wavelet decomposition corresponds to a pyramidal decomposition using quadrature mirror filters (QMS). This definition is the basis for the Fast Wavelet Transform (FWT) algorithm. Concentrating on the decomposition of two-dimensional, discrete signals F (u, v) of the L2 (R2) vector space of square integrable, two-dimensional functions, a two-dimensional scaling function ξ (υ, ν) = ξ (υ) ξ (ν) (the parent) Wavelet) and a one-dimensional <p (u) base wavelet (the mother wavelet) associated with the scaling function ξ (υ).

Let r be the data reduction factor, 2r the scale or resolution factor, A2r the linear discrete operator approximating f (u, v) at a resolution of 2r. The difference of information between A2r + if and A2rf is given by the three detail pictures D12rf (high frequencies in the vertical direction), D22rf (high frequencies in the horizontal direction) and D32rf (high frequencies in the horizontal and vertical directions).

The discrete wavelet transform (DWT) on level r + 1 of the function f (u, v) on level r is written as follows: 28/60 Austrian Patent Office AT 11 770 U1 2011-04-15 f + ihuv) = DWT (friu, if)) - A'jr f -p -Djr / + -PD \ rf [00227] Approximation and detail images A2rf, D12rf, D22rf, D32rf are also called the wavelet and extension coefficients.

For each plane, the wavelet coefficients are obtained by convolution operations of the input image f (u, v) with two one-dimensional filters: h a low pass filter and g a high pass filter, where g (n) = (-1) 1 '&quot; h (1-n).

The wavelet classification method has a wavelet that decomposes each image. The energy measure of the four subband images A2rf, D12rf, D22rf and D32rf for each decomposition plane r is used as a signature for each image. For each plane r (r> 1), the four energy values are as follows: E (r, 4) = £ (Arjf D <r <Ma / 2r G <$ f <Äl «/ 3r Ε (τ, Σ? ) = £ (Dj, /) 2 E (r, D ') = £ (D \ rff 0 <kMu / 2t D <| i <Jtfv / 2r E [r, D ^) = Σ (^ /) 2 for 0 <y <Ml? / 2r [00230] In contrast to the Fourier transform, which uses only the sine and cosine as basis functions, each new wavelet family brings its own set of new basis functions For the purpose of evaluating the classification of fringe patterns using different types of texture features, a wavelet family can be applied, such as the orthogonal wavelet filters initially proposed for texture classification.

The behavioral measures of a feature set are evaluated mainly by the rate Cp of correctly classified patterns. This acquisition rate is calculated for different classification algorithms, which are Naive Bayes, One-Nearest-Neighbor, and Three-Nearest-Neighbor. As a classification method, a 10-fold comparison test) and a Leaving-One-Out (One-Out) method can be used. The best classification results are obtained when using an optimized Fourier texture feature set. In this case, the general detection rate CP is 87.9% in the case of a 10-fold comparison test and 88.8% in the case of one-omission procedure.

Adaptive features 64 (stripe-based features 82 and adaptive fringing features 102) characterizing the perturbations of the stripe structures shown, which are synonymous with defective and non-defective surfaces, are defined in the following paragraphs. For a visual, human system, the interpretation of these streak perturbations with respect to the type of recorded surface is a simple task. The human vision system can even detect which type of defect surface is displayed.

Based on the recorded stripe image patterns, a set of specific stripe features (stripe-based features 82) is introduced. The calculation principle of these features includes the accurate segmentation of the light and dark stripe structures and the calculation of the specific stripe features based on the geometric and gray level characteristics of these structures. A classification method is used to evaluate how far the affected stripe features are adapted to the problem of light and dark stripe characterization. Several feature sets that correspond to different parameter settings can be used. Using an optimal setting, best rates CP = 88.9% with a 10x comparison test and CP = 88.4% in the case of a one-omission procedure, can be achieved. This result is similar to the classification result using the optimized Fourier texture feature set.

[00234] The feature extraction component plays an important role in a machine or human visual processing system. It is the way of interpreting the contents of an image that represents the scene that is to be inspected. The resulting fringe patterns contain all relevant information for identifying and distinguishing 2D and 3D defects. The same reference images as shown in Fig. 6 are used to describe the entire characterization process.

The segmentation and classification of the stripe image patterns from FIG. 6 into the three different classes {ΩΑ, ÜR, 3D, Or, 2d} € Ω is a basic task for the human vision process. It automatically retrieves the relevant characteristics from the stripe images to group similar patterns and separate different ones. Thus, similar stripe disorders are instinctively segmented and grouped according to their proximity or similarity. The distinction of the "low contrast" pattern from the others of Fig. 6 is made possible by comparing the values of the bright and dark fringe pattern gray levels. Instead, the horizontal distance between the bright stripes is obviously an interesting characteristic to distinguish the "fading" defective surface of FIG.

With regard to the defective surfaces of "circular mean size" and "circular large size" of Fig. 6, the geometry of the light structure may be relevant discrimination information. However, further characteristics are certainly necessary because the bright fringe disturbances of the defective 3D surfaces are similar to the disturbances introduced by the non-optimal recording conditions or non-critical 3D defects, see "symmetrical deviation" and "circular small size" patterns Fig. 6.

The adaptive feature method 64 recovers characteristic features for the classification of non-defective surfaces and defective 3D and 2D surfaces. The peculiarity of the method is that both light and dark stripe structures are characterized using different geometry-based 94, intensity-based and / or statistic-based 99 features extracted from stripe images.

The first step of the adaptive feature characterization algorithm consists of the segmentation of the light and dark pixel-level stripe structures 100. In the field of interferometric holography, a morphological approach is used for the segmentation of interferometric patterns shown in fringes. The two-step segmentation process consists of binarizing the image and then successive morphological closing and opening operations to recover the pixel-level skeleton of the fringe structures. These algorithms are generally called dilution methods because their principle is to recover the skeleton of a pattern by successively erasing pixel layers at the boundary of that pattern.

With regard to the light structures shown, they are characterized by high discrepancies in their gray level values. Since these types of disturbances of structure are unavoidable, the stripe segmentation procedure should overcome this difficulty. The calculation of the stripe structure represents the first step of the entire characterization process of the stripe structures. Thus, the accuracy of this preliminary process is decisive for the further operations.

The adapted method according to embodiments of the invention recovers the position of the pixel level stripe structures 100. This method is calculated independently for each of the Mv lines of a picture pattern F and for the bright stripes and the dark stripes.

The light and dark stripe structures shown in an image pattern F are considered to have approximately one period of dP, px pixels. The minimum gray value difference between two consecutive positions of a bright stripe and a dark stripe along the u-axis of the image pattern is considered to be above a certain threshold 0g. With regard to the recovery of the bright stripe structure and for a particular image line, the method comprises a first selection of all the pixels whose gray scale differences with the pixels located at half the period along the u direction of the image are above a threshold 0g , The same procedure is used for the selection of the dark stripe structure except that the gray value differences should be below a threshold 0g. The results of the light and dark stripe selection process are written in a matrix FSISCsegm. Pixel-level segmentation results in FSISCsegm are gray-coded: maximum level 255 stands for the light stripes, minimum level 0 for the dark stripes, and 100 for the remaining, unelected elements. The mathematical expression of the segmentation procedure is as follows: '366 sow' t 100 if / (fc, I) - f (k, l + dp> rtβ)> $ 9 and> 6, if f [kf l) -f {kj 4-dp, pa / 2) K Bg and f (k, i) -f (k, l-dPjwβ) &lt; Ba otherwise. This selection process is followed by a thinning operation so that for each image line each light and dark stripe is characterized by a pixel element. The pixel level segmentation can also be achieved by using a morphological dilution algorithm.

If the stripe structure is shown with a high gray level deviation, as is the case for a "low contrast" image, using the same binarization threshold on the entire image pattern may result in incomplete segmentation of the light and dark stripe structure , Thus, applying dilution techniques to such imperfectly segmented images may induce segmentation errors. An adapted procedure is therefore better suited and results in an approximately perfect segmentation of the light and dark stripes in the image. The problem of high gray level deviations of the stripe structures can be overcome if a reasonable threshold is used.

The interpretation of the banding perturbations is based on an accurate determination of characteristic features. For this task, the position of each bright stripe and dark stripe is estimated with high accuracy using subpixel methods 96.

Partial pixel level segmentation algorithms 96 are a "Gaussian approximation", a "centroid" and a "parabolic estimate" that are based on the assumption that the gray levels of the bright peaks can be modeled by a mathematical function, such as , B. a Gaussian or a parabolic distribution.

Further methods are a simple "linear interpolation" of the gray levels and the calculation of the first derivative of a Gaussian operator, called the "Blais-and-Rioux".

The best behavior in the presence of simulated additive noise can be achieved by the Blaise-and-Rioux 2 and 4 estimates, which give the best detection performance. With regard to the peak detection results with a saturated sensor, i. H. if the size of the gray levels is near some limits, the Blaise-and-Rioux 4-length estimate results in minimal peak detection errors. This refers to stripe widths up to 2 pixels and with 1 to 3 pixel saturated regions. These values, which define the size of the stripes, correspond to the geometry of the considered vertical stripe structures recorded by the industrial system. Such a subpixel method may also be used for the detection of the maximum gradients that define the stripe boundaries. 31/60 Austrian Patent Office AT 11 770 U1 2011-04-15 For the subpixel features 96, the Blais-and-Rioux and the mass-center detectors can be used. Due to possible recording disturbances, the recording setting may be set so that the stripe structures are shown in the saturation mode of the sensor. The result is that the bright stripes can have a large width of 2 to 3 pixels. Thus, to cover the entire image portion defining a bright stripe, but also a dark stripe, detectors with sizes of 5 pixels and 7 pixels can be viewed.

The expressions of the mass-center and Blais-and-Rioux detectors of lengths 5 and 7, called CM5, CM7 and BR5, BR7, are given below. For clarity of mathematical expression, the function fu (v) is defined as a one-dimensional, discrete representation of a horizontal image line of length Mu, where fk (l) is the lth element of the line k.

The mass-center-of-gravity algorithm assumes that the intensity values over a bright stripe correspond to a Gaussian distribution. Thus, the subpixel position is calculated by a weighted average method. The estimated subpixel positions of a bright stripe at the image position (k, I) for operators CM5 and CM7 CM5lA * bright and CM7lA * bright are as follows: cm6¾_, m. M + 2) + Ml + 1) - A (l - 1) - Ml - 2) * &quot; · **} fk (/ -2) -1- / * (/ -1) + h (t) + / * {/ + 1J + M + 2) cm7¾ _ fm + At / + 3) + Mi + 2) + M + 1) - M -1) - Mt - 2) - Mt - a) fk (l - 3 ) + M - 2) + fkil -1) + fk {i) + Λ (i + 1) + A (i + 2) + Ml + 3) with respect to the dark stripes as well as the bright stripes may be considered that its intensity distribution along a picture line is Gaussian. Thus, the same center of gravity algorithm can be used for the determination of dark stripes at sub-pixel positions. Before applying such an algorithm, the gray values defining a dark stripe region are first transformed. This elementary transformation consists of calculating the absolute value of the difference between the maximum intensity value of 255 and the dark stripe gray levels. Mathematical expressions of the estimated sub-pixel positions of a dark stripe at the image position (k, I) for the operators CM5 and CM7, CM5lA * dark and CM7lA * dark, are as follows: ciu * | + + + ^ _ 2) + fl (l - 1) + / 1 (0 + fl (l + 1) + flil + 2) CMTji _ «.t / η I /! (/ + 3) + /! ( ? + 2) j- flji + 1) - fl {l - 1) - fl (l - 2) - fl {t - 3) M fl (l - 3) + /! (F - 2) + fl (l - 1) + flU) + fi (I + 1) + fl (l + 2) + + 3) where / 1 (1) = | 255 - (f)! The Blais-and-Rioux method is based on the convolution of image gray values with a derivative operator. The subpixel position is determined with a local, linear interpolation. The expressions of the derivative functions BR5fkder and BR7fkder using the operators BR5 and BR7 applied to the image position (k, I) are as follows: BR5 / * r (i) _ _Λ (; _ 2) _A (j _ 1) + +1) + A ((+ 2) I) = -M &lt; - 3) - Ml - 2) - M -1) + ftV +1) + Ml + 2) + Ä (! + 3) Finding the position of the maximum of a signal corresponds to finding the zero crossing of the derivative of the signal , Thus, the determination of the subpixel of a bright 32/60 Austrian Patent Office AT 11 770 U1 2011-04-15

The result of a local, linear interpolation at the zero crossing of the derivative. The equations of the sub-pixel positions of the bright stripes using the operators BR5 and BR7 at the image position (k, I), BR5l * bnght and BR7l * bright are as follows: Β *% * *,., = Λ (ί) + Βί7ί1, , «= MO + BRü for 5flBr (I) ERs jderjj ^ _BR5 for the BRT and ERD_BR7 (I + lj whore> 0 ΒΗ5 / ^ (ί +1) <0 and BR7 / tiCT (0> 0 Ββ7 / ^ (ί +1) <0 With respect to the center of gravity method, the same transformation can be applied to dark-stripe gray levels Thus, the mathematical expressions of the estimated sub-pixel positions of dark-stripe are at the image position using the operators BR5 and BR7 (k, I), BR51A * dark and BR7IA * dark, as follows: wlwre ßR5 / tlkr (i)> 0 i) <o and BK7f ^ r (l)> 0 BRT / trfir (i + 1) <0 with BR = -fl (l-2) - fl {l - 1) + fl {i + 1) + ft {l + 2) = -fl (l - 3) - /! ((- 2) - / | (i - 1) + fl (l + 1) + flU + 2) + flu + 3) and 4 (0 = 1255 - ^ (1) 1 A central idea of embodiments of the invention is d Using the deformations of the stripe structures shown to automatically classify the recorded surface as acceptable, 3D defect or 2D defect. Thus, the first step of surface quality control includes automatic segmentation of the stripe patterns. The light and dark structures of such a pattern can be automatically extracted by sub-pixel segmentation techniques. A next step is the characterization of these structures.

For the description of the stripe features 82, the following notations are used. It defines the notations for the light stripes (B), the same notations apply to the dark stripes, in which case (B; bright) should be replaced by (D; dark).

A bright stripe in a picture pattern F of size Mu x Mv is shown with a list <B) S '. <B) N is the number of bright stripes in the image pattern, <B) n 'is the number of elements of the list (B) S' whose elements (B) s'j are defined with a position along the u Axis and the v-axis of F and with a gray value g. The following spellings apply: 33/60

Austrian Patent Office AT 11 770 U1 2011-04-15 (Β) $ ί =

The information as to whether an object is defective or not may be included in the position as well as in the gray level distributions of the light and dark stripes. Thus, specific features characterizing the light and dark fringe disturbances shown in an image F can be extracted.

For the description of the feature calculation and without loss of generality, the following two simplifications can be made. First, only one possible class of fringe disturbances is indicated in each image pattern F. Second, the feature calculation is applied to the entire pattern of image F, which shows the stripe noise.

Feature cm, rn = {1, ..., NC} is the average result of an operation applied to a light or dark stripe element (B) s'j or (D) s'j. Nc is the length of the vector c and the number of extracted features. The operator Oljm is applied to a bright element (B) s'j or a dark element (D) s'j by observing a local window wF centered on (B) s'j or (D) s'j , The size of wF can be fixed or variable.

According to the equations for the calculation of the characteristic cm, (the notations are given for the bright stripes (B), the same notations apply to the dark stripes, in which case (B) should be replaced by (D)):

[00262] fixed = fixed [00263] variable = variable [00264] For the sake of simplicity, the operations OV ^ s) | fixed) and Oljm ((B) s'j | variable) also called Oljm and Oljm. In this case, the value of m indicates whether the operation considers the local window wF to be fixed or variable.

From the strip-based features 82, a set of three geometry-based features 94 and an intensity-based feature 98, specially designed and adapted for characterizing the light and dark stripe structures whose perturbations are synonymous with defective surfaces, can be selected. These eight features are extracted from the gray levels and from the positions of the light and dark stripe structures.

Thus, for each image pattern F, a set of 8 different specific values can be calculated. These 8 values refer to the direction, the maximum and minimum distance of two consecutive strips and the gray levels of the light and dark stripe structures.

[00267] The notations and detailed expressions of the 8 operators O'j0i to Oy08 calculated for a bright pixel element (B) s'j and for a dark pixel element (D) s'j are defined as follows (these operators define the features c01 to c08): 34/60

Austrian Patent Office AT 11 770 U1 2011-04-15 [00268] {B) S 'and (D) S: [00269] (B) S j and (D) sij: [00270] wF: [00271] (B) ks, (Βνθ; [00272] (Β) ^, <B) k'e: [00273] ds ^), ·, light and dark stripes are characterized by operation Oljm elements of (B) S 'and (D ) S 'to which Oljm is applied Window defining the image region where the operator Oljm is applied, the size of wF can be fixed or variable Positions along Ü of the first and last element of and (D) Si in wF distance from (B) s'j to the next light and dark stripe along the direction u [00274]

Distance from (B) s'c to the next light and dark stripe element in the opposite direction from ü [00275] (01), (02): intensity (gray level) [00276] from (Β) β) and ^ s), wF fixed oä = lB> 4 0'- '_ w02 - [00277] (03), (04): minimum distance [00278] of ^ S1 and ^ S'wpfest 0 fj _ 03 - oä = [00279 ] (05), (06): maximum distance [00280] of ^ S1 and (D, S ', wFfestoä = Oos = max [distl ^ (D) e ,,; dist2- ^ j] [00281] (07), (08): deviation [00282] from (B) S 'and (D) Sj, wF variable ((&lt; "** _" * &lt;) |! (&Lt; * "*; _ &lt; ß &gt; 4) These features 82 may be defined for the characterization of vertical, periodic stripe structures obtained by projecting a patterned pattern of light onto the surface to be inspected .. In the field of optical, nondestructive testing, image patterns associated with oä = oä = 35/60 Austrian Patent Office AT 11 770 U1 2011-04-15 interferometric methods light and dark fring patterns 102. Similar to the interpretation of the stripes, the geometry of the edges gives important information regarding the quality control of the object to be inspected.

Specific features for the interpretation of such fringe structures are the adaptive fringe features 102. The characterization of the light and dark fringes can be achieved by these fringe adaptive features 102, which may also include geometry-based 94, intensity-based 98, and / or statistic-based 99 features.

Holographic interferometry (HI) or speckle interferometry (SP) are optical, non-destructive measuring methods that use coherent light. Both techniques are based on the interpretation of interference fringe patterns. These interference patterns are generated by two interfering wave fields from the object, recorded in two different positions or in two different load states. Thus, interferometric methods can be applied to the detection and measurement of various defects, such. B. Stress field or internal and / or surface defects.

The application of interferometric techniques is therefore very diverse, such. B. 3D measurement, material deformation measurement or surface inspection.

Most of the definitions of specific features for identification and classification purposes of patterns that show light and dark structures are proposed for applications that use a coherent light technique.

Geometric features 94 characterizing the error patterns may be used for the automatic detection of edges in patterns representing defective object parts. Further, a set of geometric 94 and statistical 99 features may be used for the interpretation of holographic interferometric edges 102, e.g. For medical purposes. Other methods include wavelet or Fourier transforms for the characterization of errors.

The classification of errors using interferometric images may also be based on the direct interpretation of the structures in the image. A feature set for describing a fringe or a set of fringes shown in a local area may be used.

The local area used to calculate the features may be positioned on the skeletal image pixels 100 or may be positioned on a pre-segmented, bright sub-pixel 96-stripe element. The latter segmentation approach is better adapted to the strip images considered.

Only the bright edges may be characterized or both light and dark structures may be characterized. Some features may be specifically defined for certain types of fringe structures, such as: For example, the "eyes" or the "eye chains" -tructuars (eyes chains).

Fringing features (adaptive fringing features 102) may be used for the characterization of both light and dark structures shown in the fringe pattern. For example, four features based on geometry 94 and two statistics 99 based features may be used for the characterization of bright fringe patterns, where the characterization of the dark stripes using these fringe features 102 may not be addressed.

In one embodiment, the first feature group describes the shape, the tangent direction, the curvature and the straightness of the bright edges. The second feature group, on the other hand, characterizes the length and number of pixel elements that define the bright edges in a window wF.

The length, curvature and shape features may be implemented according to the definitions given below, these are called F1, F2 and F3. The number of elements, straightness and tangent direction refer to the features called F4, F5 36/60 Austrian Patent Office AT 11 770 U1 2011-04-15 and F6.

[00296] The notations and detailed expressions of the 12 operators O ^ og to O '^ o calculated for a bright pixel element (B) s'j and for a dark pixel element (D) s'j are defined as follows (these operators define the features c0g to c2o): [00297] (09), (10): Tangent direction [00298] of {B) S 'and {D) S' at (Β) 5) and ^ s), wF fixed 0¾ = oä = tan-1 (Wsf) [00299] (11), (12): Curvature [00300] of {B) S 'and {D) S' at {B) s \ and ^ s), wF fixed 0¾ = 0¾ = [00301] (13), (14): length tan! (Wsj) tan'p ^ j) [00302] of and ^ S1, wF variable

Ov - Ö'iJ - 1mgthw ^ S ^) [00303] (15), (16): Form [00304] of (B) S 'and (D) S', wF variable ryi - 0¾ = [00305] (17 ), (18): straightness [00306] of (B) S 'and (D) S', wF variable often = -1) · Σ ~ i »2 OS = 3 = 2 [00307] (19), (20 ): Number of elements [00308] of {B) S 'and {D) S' in wF variable II II OO {B) M. The classification of the stripe image patterns using the adaptive features 64 can be achieved by retrieving 20 different features c0i to c2o representing the light and dark stripe structures. FIG mark, which are shown in the pictures.

The evaluation of these 20 adaptive features c0i to c2o 64 may be based on four different parameter settings. Two sets include sub-pixel operators CM5 and BR5, whereas two more sets consider different values of local window sizes dWF.

Compared with classification results of the texture features 80, classification results for adaptive features 64 are similar as far as an approach 10-F or a procedure LoO is considered. Nearest Neighbor classifiers may outperform the NB approach. Depending on a window size for a feature, selecting better classification rates may be achieved. As far as the influence of the size of the image on the classification results is considered, similar classification results can be achieved. Classification rates may decrease for 128x128 pixel image patterns classified with an NB classifier.

A method of adaptive features 64 for the characterization of fringe patterns is based on the calculation of specific features for the characterization of the light and dark stripe structures shown in the pictures. Classifying 30 the image 24 using the six geometry-based 94 and two intensity-based features (01), (02), (03), (04), (05), (06), (07), (08) in combination with the twelve fringe-based features 102 (09), (10), (11), (12), (13), (14), (15), (16), (17), (18), (19 ), (20) with a LoO classification method leads to better classification rates and lowest risk Cp = 88.4% and CR = 0.11.

A method for classifying 30 the image 24 by texture features 80 using the ten optimized Fourier-based 90 features Ρθΐθ2 results in similar classification results. The best classification rate and the lowest risk using these Fourier-based features are CP = 88.8% and Cr = 0.11 when considering a LoO classification method.

The difference between the optimized transform 66 and the optimized adaptive 64 approach lies in the detection of the defective 2D surfaces and the false alarm rate. In the case of a LoO procedure, the classification of 2D surfaces is better when the included adaptive features 64 are used, Cd, 2d = 92.3%. A rate of Cd, 2d = 89.2% can be obtained using texture features 80. Nevertheless, the false alarm rate for the adaptive method 64, Cf, 3D2d = 9.4% higher than for the transform approach 66, CF, 3D2D = 7.2%.

The texture features 80 and the adaptive features 64 described above are optimally suited for the characterizations of stripe patterns. In fact, none of the other texture approaches (84, 86, 92) outperforms the adaptive method 64 described above, except for the Fourier approach 90, which results in similar classification rates.

The fact that the ten optimized Fourier features θθιθ2 give similar results to the proposed 20 adaptive features (01) - (20) shows that as far as the perturbations of the vertical and periodic patterns need to be classified, the regions of the service spectrum are very well suited for their characterization. Thus, another step is to improve the achieved classification results by combining these two optimized Fourier and adaptive feature sets.

[00317] Embodiments of the invention provide such a combination of features.

However, the best classification results using the adaptive features 64 can be obtained for fitted sizes of image patterns that are hand-segmented such that only the disturbed stripe structure that is to be characterized is shown in the pattern. Of course, hand segmenting each image region prior to its classification is not possible with respect to a fully automatic inspection process, so unattended and adapted segmentation techniques should be defined. It may be a solution to segment the image patterns to classify by means of sliding overlapping windows of constant sizes. The size of the overlapping regions may be defined according to the specifications of the inspection task. Thus, in the case of a classification method combining both feature sets (64, 66), the same image set Φ64 * 2 is considered. Patterns of 64 x 64 pixels are often viewed by the computer vision community as far as sash windows are concerned.

As far as the classification of stripe patterns is concerned, various texture feature sets 80 are tested and compared to an adaptive set of features 64, which is especially developed for this classification task. The results have shown that the 10 directional Fourier features Ρθΐθ2 and the 20 specific adaptive features (01) - (20) lead to the best classification results. In both cases, the one-nearest neighbor approach outperforms the other two affected classifiers, which are the three nearest neighbor and Naive Bayes methods. These two feature sets have been defined in the context of different feature recovery approaches, the first using the distribution of the spectral distributions that extracts second specific geometric and statistical characteristics directly from the fringe pattern. With respect to the classification results achieved, both feature sets were tested separately and yielded approximately the same acquisition rates. The adaptive method 64 was somewhat more sensitive to detection of 2D defects with the negative effect of a slightly higher false alarm rate.

The question is whether more features lead to more discrimination performance. Thus, embodiments of the invention confirm that a combined approach involving both feature sets (64, 66) results in an improvement in classification rates.

In view of the similar results achieved with these two different feature sets, it is quite conceivable that among these 30 features resulting from the combination of the texture features and the 20 adaptive features, a certain amount of irrelevant and redundant information is unavoidable. For another feature selection process to improve the overall classification process, different feature subset selection methods may be applied to the combined 10 directional Fourier features and the 20 adaptive features to improve the classification rates.

In order to improve the achieved classification rates, suitable feature selection methods are used by embodiments of the invention. It is shown that first by combining the Fourier texture features and the matched stripe features and then retrieving the most appropriate subset, best classification results are achieved. A rate of CP = 91.2% can be achieved.

The major value that expresses the behavior of a particular classification process is the percentage of correctly classified patterns CP, expressed as a percentage. The following classification rates are also addressed: the percentage of correctly classified, defective object surfaces Cd, 3D2d (true positive), the rate of misclassified, non-defective object surfaces Cf, 3D2d (false positive), or the rates of correct classification of defective 3D and 2D object surfaces, if only object surfaces of classes QRi3D and Qr 2d are considered, Cd, 2d (true positive 3D) and 2D (true positive 2D).

As far as the industrial application is concerned, the behavior of a classification process is expressed by means of the classification risk CR, the calculation of which comprises cost values. These were defined according to the specifications of the industrial inspection task.

9 shows a diagram of a method 4 for selecting an optimized evaluation feature part 50 for inspection of free-form surfaces illuminated by structured light on the base a reference pattern set 52 showing surface areas of free-form objects, each pattern of the reference set 52 being associated with a defective or non-defective reference state 54 of the respective surface area of the free-form objects.

The method 4 comprises generating 56 a plurality of combinations of evaluation features 58, wherein each combination of evaluation features 58 comprises a first set of features 60 and a second set of features 62, wherein the first set of features 60 comprises a first Group of adaptive features 64 is selected and wherein the second set of features 62 is selected from a second group of texture features 80. The first group of adaptive features 64 may include intensity-based features, geometry-based features, and / or fringe-based features. The second group of texture features 80 may include transformation-based features, features, and / or statistical features.

The method 4 further comprises evaluating 68 each pixel or a subset of the pixels of the patterns of the reference set 52 with each combination of evaluation features 58 to obtain a rated defective or non-defective state 70 of the respective surface areas.

The method 4 further comprises comparing 72 the rated defective or non-defective state 70 with the defective or non-defective reference state 54 to obtain a detection rate 74 for each combination of evaluation features 58.

The method 4 further comprises determining 76 the optimized score feature subset 50 based on the detection rates 74.

With the optimized score feature subset 50, an image 24 may be classified 30 to obtain a class 32 indicating a defective or non-defective state 28 of the image 24. The classifying 30 may be a classifying 30 as described in FIG. The optimized score feature subset 50 may be used for a method 2 of inspecting a free-form surface illuminated by a patterned light, as shown in FIG. A combination method of inspecting a free-form surface illuminated by a structured light having a method of selecting an optimized evaluation feature subset for free-form surface inspection may be used.

Such a method for optimized inspection of a free-form surface comprises the method 4 for selecting an optimized evaluation feature subset and the method 2 for inspecting a free-form surface 20 illuminated by a structured light 22, the combination of the evaluation features 26 being the optimized feature evaluation subset 50 corresponds.

The optimized score feature subset 50 may be determined by a maximum of the acquisition rates 74 of each combination of score features 58 such that the best capture rate provides the optimized score feature subset 50. The optimized score feature subset 50 may have fewer features than a combination of all of the features of the first set of features 60 and the second set of features 62.

Each pixel or subset of the pixels of the patterns of reference set 52 may be evaluated with each combination of evaluation features 58 until a stop criterion is met. The stop criterion can be fulfilled if all combinations of evaluation features have been evaluated or if a threshold has been reached. The threshold may be a minimum number of features for the optimized score feature subset 50, a maximum number of iterations for scoring 68, a minimum value for the detection rate 74, a limitation on the number of backtracking, or a condition that does not determine a higher detection rate 74 is. Selecting the most relevant feature subset prior to learning may be beneficial to the classification task because it reduces the dimensionality of the data and may allow learning algorithms to run faster and more efficiently.

[00336] By applying a feature selection procedure to the combined set of 30 features, i. H. the 10 directional Fourier features Ρθΐθ2 and the 20 specific adaptive features (01) - (20), the accuracy of the fringe pattern classification can be improved. The purpose of a feature selection process is to create a feature subset that consists of the most relevant information. This iterative procedure can be represented by four main parts.

The generation and evaluation of each new subset for each iteration. A stop criterion and a result validation. Two different approaches of feature subset generation can be considered: filter and wrapper (cladding) methods. The first method consists of filtering out the irrelevant data independently of the classification process. The second method uses the machine learning algorithms that are used for a learning procedure. Similar feature search strategies can be used in both cases. A distinction is made between a forward selection (the process starts without selected features), a backward elimination (the process starts with the complete subset) or a bidirectional search strategy that combines both methods and starts with a random subset of features.

[00338] The score of the generated subset, i. H. his goodness, can be carried out according to a certain criterion. Two groups can be distinguished: the independent criteria and the dependent evaluation criteria. The first is commonly used in algorithms of filter models, whereas the second is applied to wrapper-based feature selection models.

Independent criteria are based on the relationships between the features that eliminate highly correlated characteristics that are redundant to the prediction. Independent criteria use z. B. the distances between the features such. B. the Bhattacharyya distance or the Hausdorff distance. The information gain or dependency between features are further independent criteria used by filter-based feature selection approaches. With regard to the dependent evaluation criteria, the evaluation and optimization of the accuracy rate is performed in accordance with the behavior of the machine learning algorithm using the selected feature subset. In the case of small sets of relevancy patterns, a cross-validation procedure can often improve the assessment score.

It is difficult to predict which of the filter or wrapper approaches would be more appropriate for a specific classification task. Dependent assessment procedures can give a better result than the independent approaches because these procedures will find features that are more appropriate for the learning algorithm. Nevertheless, their main drawback is that they cause higher computational costs because the entire learning procedure is called at each selection step.

Another important parameter with regard to an iterative feature selection procedure is the stop criterion. The simplest is to end the search of the feature subset when all calculated features have been selected. The search procedure may also stop when some threshold variables are reached. These can be z. The minimum number of features, the maximum number of iterations, or the minimum value of the error rate. More complex procedures, such. For example, limiting the number of tracebacks can also determine the stop conditions. Feature subset search algorithms may also stop when subsequent addition or deletion of any feature does not produce a better subset.

Once the selection procedure of the best features is completed, the feature subset is validated. This is generally done by comparing the classifications of classification using the full feature set and using the calculated feature subset.

9 shows the diagram of the method for selecting an optimized evaluation feature subset with the three main parts of generating 56 combinations of evaluation features 58, 68 evaluating the combinations of evaluation features 58 with a reference pattern set 52 and determining 76 (stop criterion) of the optimized evaluation feature subset 50 The classify 30 (and result validation) is based on this optimized score feature subset 50.

The feature selection principle is similar to the pattern classification task, where a learning phase and a testing phase are considered. The task of selecting the most relevant features may also be divided into a training step and a testing step. Once the best feature subset (optimized score feature subset 50) has been calculated after a learning phase using a reference template set 52, the validation (classification 30) may be performed using a different set of patterns according to the classification method 30, as shown in FIG another reference pattern set 42 is used for a learning step 40. Both methods may be combined such that the classifier 30 shown in FIG. 2 uses the optimized score feature portion 50 shown in FIG. 9 for the combination of score features 26 shown in FIG.

For the learning 40 shown in FIG. 2, a wrapper-based feature selection method, e.g. A k-NN method. Wrapper-based feature selection models are dependent evaluation criteria because they use a classification algorithm to evaluate each feature subset. The feature subset generation can be carried out by using a learning set (reference pattern sentence 52) shown in FIG. 9. The calculated optimized score feature subset 50 (cSUb) is then used for result validation using a test set (reference pattern set 42) shown in FIG. 2 that was not used during the feature subset selection process.

With respect to the classification of stripe images, the nearest neighbor 1 NN can outperform the 3 NN and NB methods. The same algorithm can be used for wrapper-based selection and for the result validation process. Thus, in the case of stripe image classification using wrapper based methods, the 1NN pattern based classifier is further considered.

This classifier gives the best classification results as far as the direct Fourier-Peie2 and the adaptive features (01) - (20) are concerned. Thus, a feature selection approach based on the 1NN classifier improves the classification of the striped patterns.

Filter-based feature selection schemes use the relationship between the features and the classes to retrieve the most appropriate feature subset. Feature selection in the case of monitored classification approaches can be achieved based on a correlation between features. Different correlation measures can be used for the classification of records. It has been shown that a correlation-based method can yield comparable results to a wrapper-based approach.

The basic principle of correlation-based feature selection methods is that good feature sub-sets contain features that are highly correlated with the class but not correlated with each other. In the case of a forward selection procedure, a feature subset of Ncsub features is considered optimal if its heuristic power or value representing the degree of feature-feature and feature-class correlation for a given feature class, Ωκ e {ΩΑ, Ω ^ ο , Ω ^} is maximum.

[00350] A correlation-based feature selection process CFS (correlation-based feature selection process) may e.g. B., as presented by "Mark A. Hall, 42/60 Austrian Patent Office AT 11 770 U1 2011-04-15

Correlation-based feature selection for machine learning, Ph.D. thesis, University of Waikato, New Zealand, 1999 ".

The equation of a merit NcsubQkM of a feature subset containing Ncsub features for a given class label Ωκ is (adapted for inspection from freeform surfaces): N,

N iRj Γβ * &quot; * β 'AL .., Σ rei ", iÄ ^' bkö

jl = 0 j2 = Q ~ ^, .Nr i Σ Σ r <£ t <i vpub ^ jpisb fcsubQK and rcsubcsub are the average feature class and mean feature-time feature correlation coefficients that determine the degree to measure the dependence of one variable on another. These two coefficients may be defined to be the Pearson correlation coefficient of two vectors. Its calculation differs according to values of the vectors, i. H. whether they are both continuous or discrete.

Thus, the equations defining rcjsubQKj and rCj1subcJ2sub differ:

AV ^ Η = Σρίζ1κ''τ ^ {Μ ftscl

Nf and &gt; V, it &gt;. = -

tai Λρ m * V

1 ÄllO -Vr V1 c · * 2 '' lu.iSufc J_fe * 1_

These equations define the feature class and feature feature coefficients for a feature subset that includes Ncsub features and for a particular class label Ωκ. ρ (Ωκ) is the previous probability of the class Ωκ. The feature subset for the Ncsub features can be retrieved from the NF image patterns. dsub (k) is the kth value of the vector cjsub (k = 1, ..., NF) whose class label is Ωκ. DjK, bi is a binary vector whose values are 1 if the class of a feature is dsub Ωκ, and 0 otherwise.

Generating 56 combinations of evaluation features 58, as shown in FIG. 9, may include an iterative feature selection process. A feature forward selection procedure with an amount of traceability of e.g. B. 5 as the stopping criterion can be addressed. An iterative feature selection process may start with an empty, initial feature subset.

The feature selection (evaluate 68 of Fig. 9) and the pattern classification (classify of Fig. 9) may follow a 10-F cross-validation procedure. FIG. 10 is a diagram showing detection (classification) rates 74 of different combinations of evaluation features 58 for a method of inspecting a freeform surface formed by. FIG structured light is illuminated, according to an embodiment of the invention. Some of the combinations of the evaluation features 58 (the two right feature sets in the diagram) have been selected by a method of selecting an optimized evaluation feature subset for inspection of the free-form surfaces according to another embodiment of the invention.

The feature set cs has the 20 adaptive features (01) - (20) as described above. The feature set cF, 0 has the ten directional Fourier features Ρθΐθ2 as described above. The feature set cF, 0_s has a combination of the 10 directional Fourier features Ρθΐθ2 and the 20 adaptive features (01) - (20). The feature set cf, 9_s_cfs has an optimized evaluation feature subset selected from combinations of the 10 directional Fourier features Ρθΐθ2 and the 20 adaptive features (01) - (20) by a CFS method. The feature set cf, 0_sj-nn has an optimized evaluation feature subset selected from combinations of the 10 directional Fourier features Ρθΐθ2 and the 20 adaptive features (01) - (20) by a wrapper 1 NN method.

The 1NN classifier is used because this approach results in the best classification (acquisition) rates for both feature extraction methods. Also, a 10-F classification approach can be considered, giving similar results to a LoO procedure.

In the diagram of Fig. 10, fixed quantities of reference patterns are considered. The classification is applied to image sets Φ64Λ2. The main reason for using this feature set is that so far as the automatic classification of the stripe patterns is concerned, an automatic segmentation procedure of the image showing the disturbed stripe structures that are to be interpreted is not addressed in the results shown in FIG are shown. Thus, in view of a fully automatic surface inspection process, it may be preferable to consider fixed image sizes.

Further, it has also been shown that the best classification rate CP = 87.9% was achieved using textural approaches cF, 0 by applying the directional Fourier features to the fixed-size images of the Set Φ64Λ2 (a 10-F procedure is considered). With regard to the adaptive features Cs, the best rate CP = 88.9% was obtained using the images with adjusted magnitudes of the set Φ3ά1 (a 10-F procedure is considered). The next best fixed frame rate was CP = 85.9% with the set Φ ^ βϊηβ 10-F procedure is considered).

To simplify the notations, the adaptive feature vector cs, g2wF, BR5, resulting in a classification rate CP = 85.9%, is referred to as cs. The notation of the feature vector cF, 0, which results in a rate of CP = 87.9%, is not changed.

In the case of the classification of the image set Φ64 * 2, the directional Fourier features result in slightly better rates than the adaptive features. However, it is interesting to note that the combination of both feature sets does not significantly improve the classification results, CP = 88.4% with CF, 0s and CP = 87.9% with cF, 0.

With regard to the selection of the corresponding features, both filter-based and wrapper-based methods improve the detection rates. The 1NN method outperformed the CFS approach, CP = 91.2% and CP = 89.6%. This result validates the method that selects an optimized score feature subset 50 according to embodiments of the invention. Namely, the 1NN method is the most appropriate approach as far as the classification of vertical stripe structures is concerned.

Closer examination of which features are selected by the feature selection process has shown that generally the same feature sets are selected for each of the (10) runs of the feature selection process. A quarter of the selected features were selected more than eight times, these features are: the strip cs (01) (intensity bright), cs (06) (maximum distance dark), the fringe cs (13) (length light), cs ( 19) 44/60 Austrian Patent Office AT 11 770 U1 2011-04-15 (number of elements bright), cs (20) (number of elements dark) and the directional Fourier cF, 0 (O5) and cF, e (06) , More than a third of the total 30 features has never been selected. The number of selected features after the 10 passes Nc, 1-NN = 82 means that on average only 8.2 features are needed to achieve such results.

Therefore, few features may be required for the characterization of the stripe patterns so that classification rates of up to 91.2% can be achieved.

For the selection of the optimized evaluation feature subset 50 for the free-form surface classification 30, it has been found that a wrapper-based 1NN selection process outperforms a filter-based CFS approach. Thus, with the aid of a first combination of two corresponding feature sets and the selection of the most relevant feature subset, an improvement of the detection rate 74 from CP = 85.9% to CP = 91.2% can be achieved. In addition, the selection process allows the number of relevant features to be reduced (from 30 to less than 10 in one embodiment).

For both distortion types -1- and -2-, as shown in Fig. 6, the number of selected features increases in proportion to the distortion of the stripe structures in the patterns. Adjusted fringing features are selected the more the light stripes differ from a periodic and vertical structure. In fact, such features have been defined in the context of a complex fringe structure interpretation.

Fewer features are selected when highly distorted stripe structures are involved. Without distortion, the average number of features selected is about 10, and decreases to about 7 for stripe structures whose distortion is type -1 and type -2-. For highly distorted patterns, the specific stripe features of the adaptive features (09) - (20) and the directional Fourier features cF, e (06) may be relevant mainly for the classification process, in particular for the patterns corresponding to the distortion type -1.

It can be observed that the pattern arrangement χ2, as shown in Fig. 6, is more suitable than the pattern arrangement χ · für for the classification of free-form surfaces. In this case, the wrapper-based feature selection process produces similar or better results than using the entire feature set. Then, according to the degree of distortion of the stripe pattern, a very low number of features is necessary to regain the best classification rates.

Embodiments of the invention that provide a method for selecting an optimized score feature subset reduce the dimensionality of the data and therefore allow learning algorithms to run faster and more efficiently. This improves the inspection process for complexity reduction as far as the algorithmic procedure is concerned. Another important benefit is the reduction of processing time.

Depending on certain implementation requirements of the methods 2, 4 according to the invention, the methods 2, 4 according to the invention can be implemented in hardware or in software. The implementation can be carried out using a digital storage medium, in particular a disc, DVD or a CD with electronically readable control signals stored thereon, which cooperate with a programmable computer system so that the methods 2, 4 according to the invention are executed. In general, therefore, the invention is a computer program product having a program code stored on a machine-readable carrier, the program code being operative for carrying out the inventive methods 2, 4 when the computer program product is executed on a computer. In other words, the methods 2, 4 according to the invention are therefore a computer program with a program code for executing at least one of the methods 2, 4 according to the invention when the computer program is executed on a computer. 45/60

Claims (16)

Austrian patent office AT 11 770 U1 2011-04-15 Claims 1. A method (4) for selecting an optimized evaluation feature subset (50) for inspecting freeform surfaces illuminated by a structured light based on a reference template set (52) Showing surface areas of free-form objects, each pattern of the reference set (52) being associated with a defective or non-defective reference state (54) of the respective surface area of the free-form objects, comprising the steps of: - generating (56) a plurality of combinations of evaluation features (58); wherein each combination of evaluation features (58) comprises a first set of features (60) and a second set of features (62), wherein the first set of features (60) is selected from a first group of adaptive features (64) the second set of features (62) is selected from a second group of texture features (80), wherein i) the first group of adaptive features (64) comprises stripe-based features (82) and / or fringe-based features (102), and wherein the second set of texture features (80) include transform-based features (88), features (84), and / or statistics features (80). 86); - evaluating (68) each pixel or a subset of the pixels of the patterns of the reference set (52) with each combination of evaluation features (58) to obtain a rated defective or non-defective state (70) of the corresponding surface areas; - comparing (72) the evaluated defective or non-defective state (70) with the defective or non-defective reference state (54) to obtain a detection rate (74) for each combination of evaluation features (58); and determining (76) the optimized score feature subset (50) based on the acquisition rates (74). The method (4) of claim 1, wherein a maximum of the acquisition rates (74) of each combination of evaluation features (58) determines the optimized evaluation feature subset (50). The method (4) of claim 1 or 2, wherein each pixel or a subset of the pixels of the patterns of the reference set (52) is evaluated with each combination of evaluation features (58) until a stop criterion is met, the stop criterion being satisfied is when all combinations of valuation features (58) have been valued or when a threshold is reached. The method (4) of claim 3, wherein the threshold includes a minimum number of features for the optimized score feature subset (50), a maximum number of iterations for the score (68), a minimum value for the acquisition rate (74) Restrict the number of traceability or a condition that no higher detection rate (74) is determined. 5. The method (4) of claim 1, wherein generating (56) a plurality of combinations of evaluation features (58) comprises a filter algorithm or a wrapper algorithm, the filter algorithm having a correlation-based algorithm, wherein the wrapper algorithm comprises a nearest neighbor algorithm or a Naive-Bayes algorithm. 46/60 Austrian Patent Office AT 11 770 U1 2011-04-15 The method (4) of one of claims 1 to 5, wherein evaluating (68) each pixel or a subset of the pixels of the patterns of the reference set (52) comprises a feature distance algorithm, an information retrieval algorithm, or a feature dependency algorithm, wherein the algorithms are adjusted are to evaluate the pixels or the subset of pixels with the combinations of evaluation features (58) using an independent criterion; or wherein evaluating (68) each pixel or a subset of the pixels of the patterns of the reference set (52) comprises a machine learning algorithm, the algorithm adapted to adjust the pixels or subset of the pixels with the combinations of evaluation features (58) using a dependent To evaluate the criteria. A method (2) of inspecting a free-form surface (20) illuminated by a patterned light (22), comprising the steps of: - illuminating (10) a portion of the free-form surface (20) by the patterned light (22); - recording (12) an image (24) of the illuminated area of the free-form surface (20); and - evaluating (14) each pixel or subset of the pixels of the image (24) of the free-form surface area (20) with a combination of evaluation features (26) to obtain a defective or non-defective state (28) of the surface area (20); - wherein the combination of the evaluation features (26) comprises a first set of features (60) and a second set of features (62), wherein the first set of features (60) is selected from a first group of adaptive features (64) and wherein the second set of features (62) is selected from a second group of texture features (80), wherein the first group of adaptive features (64) comprises stripe-based features (82) and / or fringe-based features (102) and wherein the second group from texture features (80), transformation-based features (88), features (84) and / or statistical features (86). The method (2) of claim 7, wherein evaluating (14) each pixel or a subset of the pixels of the image (24) of the area of the freeform surface (20) comprises the step of: classifying (30) the image (24) by assigning (38) a class (32) to the image (24), wherein the class (32) indicates the defective or non-defective state (28) of the surface area (20). The method (2) of claim 8, wherein classifying (30) the image (24) comprises the steps of: - extracting (34) information of the image (24) as described by the combination of evaluation features (26 ), extracted information (36); Assigning (38) the class (32) to the image (24) by using the extracted information (36); and - wherein assigning (38) the class (32) to the image (24) is based on a mapping rule (40) derived from a reference pattern set (42) showing the surface areas (20) of free-form objects, each pattern of the reference set (42) is assigned to a reference class (Ω). 10. The method (2) of claim 9, wherein assigning (38) the class (32) to the image (24) comprises a filter algorithm or a wrapper algorithm, the filter algorithm having a correlation-based algorithm, wherein the wrapper Algorithm has a nearest neighbor algorithm or a Naive-Bayes algorithm, wherein the assignment rule (40) comprises a stratified 10-fold cross-validation algorithm or a one-omission cross-validation algorithm. 47/60 Austrian Patent Office AT 11 770 U1 2011-04-15 The method (2) of claim 9 or claim 10, wherein the reference pattern set (42) has patterns of a fitted size or pattern of a fixed size; - wherein the reference class (Ω) has a class of non-defective surfaces (ΩΑ), a class of two-dimensional, defective surfaces (DR, 2D) and / or a class of three-dimensional, defective surfaces (DR, 3D); - wherein the reference class (Ω) is a class of planar shaped surfaces (Φ0), a class of surfaces formed by perspective distortion (Φ1), and / or a class of surfaces shaped by cylindrical distortion (Φ2), having. The method (2) of any of claims 7 to 11, wherein the free-form surface (20) has a dull surface or a specular surface; - wherein the structured light (22) has stripe structures, periodic structures or vertical structures of visible, non-coherent or coded light; - wherein the image (24) is recorded by using a line scan camera or a matrix camera. 13. A method for optimized inspection of a freeform surface, comprising: - the method (4) of selecting an optimized evaluation feature subset (50) according to one of claims 1 to 6; and - the method (2) of inspecting a free-form surface (20) illuminated by structured light (22) according to claims 7 to 12, wherein the combination of the evaluation features (26) corresponds to the optimized feature evaluation subset. 14. A computer program for carrying out the methods (2, 4) according to one of claims 1, 7 or 13, when the computer program is executed on a computer. An apparatus for selecting an optimized score feature subset (50) for inspecting freeform surfaces illuminated by a patterned light based on a reference template set (52) showing surface areas of freeform objects, each pattern of the reference set (52) defective or non-defective reference state (54) of the respective surface area of the free-form objects, comprising: - generating means for generating (56) a plurality of combinations of evaluation features, each combination of evaluation features comprising a first set of features and a second set of features, wherein the first set of features is selected from a first group of adaptive features and wherein the second set of features is selected from a second group of texture features, the first group of adaptive features being strip based features and and / or has fringe-based features, and wherein the second set of texture features comprises transform-based features, features, and / or statistical features; an evaluator for evaluating (68) each pixel or a subset of the pixels of the patterns of the reference set with each combination of evaluation features to obtain a rated defective or non-defective state of the respective surface areas; - comparing means for comparing (72) the rated defective or non-defective state with the defective or non-defective reference state to obtain a detection rate for each combination of evaluation features; and determining means for determining (76) the optimized score feature subset based on the acquisition rates. 48/60 Austrian Patent Office AT 11 770 U1 2011-04-15 16. An apparatus for inspecting a free-form surface which is illuminated by a structured light, comprising: a lighting device for illuminating (10) a region of the free-form surface by the structured light; - a recording device for recording (12) an image of the illuminated area of the free-form surface; and - evaluation means for evaluating (14) each pixel or a subset of the pixels of the image of the free-form surface area with a combination of evaluation features to obtain a defective or non-defective state of the surface area, wherein the combination of the evaluation features comprises a first set of features and a second set of features, wherein the first set of features is selected from a first group of adaptive features and wherein the second set of features is selected from a second set of texture features, wherein the first group of adaptive features is strip-based features and / or fringe-based Features and wherein the second group of texture features transformation-based features, structural features and / or statistical features. For this 11 sheets drawings 49/60