EP1789909A1

EP1789909A1 - Method for assigning a digital image to a class of a classification system

Info

Publication number: EP1789909A1
Application number: EP05776279A
Authority: EP
Inventors: Günther Coen
Original assignee: BFI VDEH Institut fuer Angewandte Forschung GmbH
Current assignee: BFI VDEH Institut fuer Angewandte Forschung GmbH
Priority date: 2004-09-03
Filing date: 2005-09-01
Publication date: 2007-05-30
Anticipated expiration: 2025-09-01
Also published as: ATE440342T1; US20090214123A1; US7876964B2; CN101048784B; WO2006027158A1; DE102005001224A1; EP1789909B1; DE502005007957D1; CN101048784A

Abstract

Disclosed is a method for assigning the content of a digital image to a class of a classification system. Said method comprises the following steps: —a predetermined number of F numerical shape characteristics ψm are determined; —the value of each shape characteristic of the F numerical shape characteristics determined for the image is compared to the value filed in a table for the respective shape characteristic, values for the individual numerical shape characteristics being allocated to each class in the table; —the class in which the F numerical shape characteristics determined for said image correspond best to the values of the numerical shape characteristics indicated in the table for said class is output as the class into which the image that is to be recognized is classified.

Description

Betriebsforschungsinstitut VDEh-Institute for Applied Research GmbH

Sohnstrasse 65, 40042 Dusseldorf

"Method for Assigning a Digital Image to a Class of a Classification System"

The invention relates to a method for assigning a digital image to a class of a classification system.

With the increasing automation of industrial processes, the automation of error detection on the basis of optical analysis methods is becoming increasingly important. Optical error detection methods were carried out in the past by quality assurance personnel. These have considered the object to be checked or an image representation of the object to be checked and recognized possible errors. By way of example, weld seams are checked for types of defects by means of X-ray images, such as, for example, cracks, insufficient penetration, binding defects, slag, slag lines, pores, hose pores, wort I, root defects, heavy metal inclusions and edge offset. Furthermore, it is known to view radioscopic images of castings in order to detect defects in the casting, for example foreign inclusions, gas inclusions, voids, such as yarn pockets or spongy voids, cracks or core supports. Since such errors may be similar in nature, but different in their shape and form, in the case of industrial error checking, errors have been assigned to different classes, the respective class containing errors of the same type. For example, the industry standard EN 1435 describes the classification system for weld defects. According to this standard, the errors detected in welds and identified by means of X-ray images are divided into the 30 different classes, for example classes for the faults such as cracking For example, longitudinal crack or transverse crack, insufficient Durchschweißung, Bin¬ defehler, foreign inclusions, such as slag, slag lines, gas inclusions, such as pore or hose pore, or heavy metal inclusion, Einbrandkerben, wort I score, root error, and edge offset. With the automation of the processes, efforts are now being made to achieve the optical recognition of errors and assigning these errors to given classes by image analysis with digital image recording techniques of recorded and stored images. Conventional automated error detection methods based on digital images use a so-called "heuristic approach". Here reference images are stored in an image processing unit and attempted by an image comparison to assign the content of a digital image to one of these reference patterns.

In other technological fields, the assignment of image contents to classes of a classification system is used, for example, in text recognition. Here, for example, each letter forms its own class, so that, for example, there are 26 classes for the capital letter alphabet, namely for the characters (A, B, C, D, E, F, G, H, I ₁ J, K, L, M, N, O, P, Q, R, S, T, U, V, W, X, Y, Z). The OCR technologies (Optical Character Recognition) analyze the digital image of a written page generated by a scanner and assign the singularized letter symbols to the given classes. As a result, the OCR technology "recognizes" the text and can pass the classified characters to a text processing program as an editable letter order. The published European patent specifications 0 854 435 B1 and 0 649 113 B1 cover, for example, the technical field of optical character recognition (Optical Character Recognition).

Increasingly, the technology of image processing can be subdivided into areas of different subprocesses whose technologies evolve independently of each other. These areas are frequently subdivided into image preprocessing, image analysis, analysis of image sequences, image archiving and so-called imaging. Image preprocessing is the computer-aided improvement of the quality (processing: denoising, smoothing) of the respective digital image for easier visual perception of the information content of this image for the viewer.

Image analysis is the computer-aided evaluation of the information content of the respective digital image by automatic and reproducible structuring, recognition and understanding of this image.

The analysis of image sequences means the computer-aided evaluation of the information content of the respective sequence of digital images by automatic and reproducible structuring, recognition and understanding of all individual images of this sequence and by automatic and reproducible understanding of the context of the sequence of individual images Follow-up.

Image archiving is the computer-aided compression and storage of digital images together with indexing search descriptors from a controlled vocabulary.

Imaging means the computer-aided generation of synthetic graphics and digital images for the visualization and explanation of the information content of complex processes on image and symbol level for the human observer.

The technique of associating the contents of digital images with a class of a classification system is a method of image analysis. This can be subdivided into three subareas: segmentation, object recognition and image understanding.

Segmentation is understood to mean the automatic and reproducible structuring of the respective digital image by separating the objects relevant for the analysis of the image from each other and from the background of the image. Object recognition is the automatic and reproducible zierbare classification of the separated objects. Image comprehension can be understood as the automatic and reproducible interpretation of the respective digital image by context evaluation of the classified, separated objects. The technique of associating digital images with a class of a classification system is a method of object recognition.

The object recognition can be understood as a subarea of the pattern recognition and indeed as the subarea of the pattern recognition, which recognizes as pattern only planar objects in images.

The images are regularly displayed by an image composed of pixels, whereby the content of each pixel and its position in the image must be known in order to display the image. Depending on the content attribute, the images can be stored in color images, grayscale images and binary images, with binary images as the content attribute, for example, only taking the values 0 and 1 for black and white.

A method frequently used in technology for assigning a digital image to a class of a classification system that has been used successfully for decades to distinguish military aircraft (friend-enemy recognition) is known from MK Hu: "Visual Pattern Recognition by Moment Invariants". , IRE Trans. Info. Theory, vol. IT-8, 1962, pp. 179-187 and RC Gonzalez, RE Woods: "Digital Image Processing", Addison-Wesley Publishing Company, 1992, p. 514-518 known. On the basis of the so-called normalized, centralized axial moments, which are obtained from image reproduction by means of image analysis techniques, a finite sequence {φ /} can be obtained by scaling for any, separated, bounded, planar object in a binary image 7 generate dimensionless shape characteristics len. If we consider the 7 sequence terms <j> / (0 </ <lo = 7) as the coordinates of a feature vector Φ = (φ *, φ2, φa, φ <, φs, φβ, φz), the element ei¬ nes 7-dimensional, Euclidean feature space M7, then this method induces an object recognition in this 7-dimensional feature space M7. It has, compared to the object recognition by means of heuristic features, the advantages that the classification is carried out exclusively with characteristics. malsvektoren Φ = (Φ ', φ2, φ3, φ- *, φs, φβ, φ7), whose coordinates are dimensionless shape features done so that in particular size differences between the objects to be recognized and the objects used for the preparation of Ver¬ equal table meaningless. Furthermore, within the set of dimensionless shape features .phi. / By the coordinate reference to the feature vector .phi., An unambiguous order with respect to the relevance of the features for the object recognition in digital image processing is given, so that it is clear that the first feature .phi..sub.r is the most important is.

Nevertheless, this method is disadvantageous, since the number of available dimensionless shape features is limited to 7 and thus can be misclassified for complex objects, if the same values for the two dimensionless shape features result for two different classes.

Against this background, the object of the invention is to propose a method for assigning the content of a digital image to a class of a classification system, with which also characters of more complex form can be reliably recognized.

This object is achieved by the method according to claim 1. Advantageous embodiments are specified in the subclaims.

The invention is based on the idea of determining the image to be analyzed of a predetermined number of numerical shape features Ψ _m with m as count index running from 1 to F, where Ψ _{m is} a transformed expression of dimensionless, scaled, normalized, centralized pola¬ ren moment p ^m is. These mutually independent shape features ψ _m can be compared for matching the content of the digital image with stored in a table values for these shape features. If the values of all determined F shape features ψ _m coincide with the F shape features ψ _m stored for a class in the table, the image content of the analyzed image belongs to this class. Due to digital In this case, it is preferable to work with approximate values, so that a class assignment is also already output when the calculated F shape features ψ _m approximately coincide with the F stored features ψ _{m of} a class.

In contrast to the method known from the prior art, which is limited to 7 form features, the numerical shape features ψ _m proposed according to the invention for image analysis are independent of one another in such a way that a large number of shape features can be established without the dependency of the shape features on one another arises. As a result, an unambiguous assignment of the image contents to be recognized to a designated class can be achieved.

The method according to the invention is, in particular, independent of the relative position of the content to be recognized relative to the receiving device. Also, for example, rotated by 60 ° or 180 ° objects can be clearly assigned zuge¬.

The method is based on the computation of a sequence of F functionally independent, dimensionless features of the separated, bounded content in the respective image.

The image is displayed in a conventional manner by N pixels, wherein a pixel in a predetermined coordinate system at the location (Xj, y _j), and the image is on the coordinates (0,0) to (Xi _ma χ, yim _a x) and imax is the maximum number of pixels in the direction of the x-coordinate and ymax is the maximum number of pixels in the direction of the y-coordinate and each pixel is assigned a content attribute data [j, i].

The content attribute is for a binary-represented image in which the respective pixel content assumes, for example, either the value 1 or 0 for black or white, for example a single value deposited in a table, and data [j, i] representatively for the value in this table at the location associated with the pixel. For colored pictures where the For example, if the content attribute of each pixel is composed of three values for the three color representations "red, green, blue" (RGB representation), the content attribute data [j, i] is representative of a vector which contains these three values for the respective pixel contains. Data [j, i] can also be representative of other vectors, if other color representations are used, or greyscale representations, data [j, i] can also be representative of the magnitude of such a vector, if a multi-color representation is converted from a multi-color representation, for example an RGB representation, into a greyscale or even binary representation before the use of the classification method according to the invention.

In a color representation, for example an RGB representation, data [j, i] can also stand for the individual value of the red representation, or green representation, or blue representation in the pixel. The classification method is then carried out, for example, exclusively on the basis of a representation, for example the red representation, the method then being carried out in the same way as for the binary representation above. It is then also possible to use binary values 1 and 0 for data [j, i] at the image point, where 1 stands, for example, for red and 0 for empty. Likewise, the classification method can be performed in parallel for the different color representations, ie parallel for a binary red representation, a binary green representation and a binary blue representation. This increases the accuracy of the classification.

The numeric in the shape feature ψ _m transformed moment p ^m berech¬ net from

J max; Max

A = w ₀₀ = Δo * Ab * Σ £ ύfata [/, /]

/ = 1 ι = l v "υ"

R ^m =

O _n m 0,0

/ max i max υ _m = Aa * Ab * Σ Σ Σ (λ (y, 0) "ώta [/, '1

/ = 1 Σ I = I τ -. \ 2

RU, 0 => / (('- 0> 5) * Δα - x) ² + ((- 0.5) * Ab - y)

'1.0 m, 0.0

"V ₁

^ = ^w o, oy max / max

/ «, ₀ = (Aa) ² * Ab * Σ Σ (z - 0.5) * Jαtof;, /] l = \ / = 1 y max i max

W _{0 1} = (Abf * Δα * Σ Σ (y -0,5) * </ α4 /, /]

/ = 1, = l

Δa = width of the pixel in x-coordinates direction Δb = width of the pixel in y-coordinates direction data ö, i] = content attribute of the pixel at the position (y _j , X ₁ ) m = continuous number from 1 to F.

The predetermined coordinate system is particularly preferably a Cartesian coordinate system, since most digital images define the pixels via a Cartesian coordinate system. However, it is also possible to use other coordinate systems, for example polar coordinate systems.

Whereas currently digital images can generally be represented with 1 to 3 million image points (pixels), it is to be expected that the number N will increase progressively with the progress of image acquisition and processing technology, so that the abovementioned summation functions approximately transition into integral functions.

In particular, an image content is defined by the arrangement of pixels of the same content attribute. The F-determined shape features of the image content yield a feature vector in a bounded, F-dimensional subarea (unit hypercube) of the F-dimensional feature space. The content classification finally takes place by problem-specific clustering of this n-dimensional unit hypercube.

The classification system can be, for example, a given industry standard, such as EN 1435. In the case of person recognition, for example, each person can form his own class. Thus, in the comparison table, the F shape features ψ _m , which characterize the fingerprint or the iris image of the person to be recognized, are filed. For person recognition, the image of the iris taken by a recording unit, for example a camera, is analyzed by the method according to the invention, wherein the F shape features ψ _{m of} the recorded iris are calculated and compared with the shape feature values stored in the table. If there is an (approximate) agreement with all the values of the features ψ _{m of} a class, then the system recognizes the person who is characterized by this class. For the determination of the approximate match, a method of least squares, for example according to Gauss, can preferably be used.

If a digital image is recognized that is displayed in a representation other than a binary representation, the aforementioned method steps can be carried out for a plurality of groups with F numerical shape features ψ _m , for example in a group for values of a red representation, in a group for values of a green representation and in a group for values of a blue representation. Alternatively, the aforementioned method steps can also be carried out on content attributes data [j, i] which contain as vector the individual values of the individual color representations. Divisional calculation operations are then preferably carried out on the amounts of the vectors. In a preferred embodiment, the shape feature ψ _m is the transformation

determined. As a transformation from ψ _m to p ^m , however, other transformations, even an equation ψ _m = p "'are possible.

The shape characteristic to be compared with the values stored in the table is preferably the shape feature ψ _m obtained by means of the aforementioned transformation. Before the comparison with the table values or in the transformation from p ^m , the series of F shape features can be subjected to an orthogonalization method, as is carried out, for example, according to E. Schmidt. In this case, the shape characteristic to be compared can be converted in particular such that a series of F shape features ψi, ψ2, ψ3 for a circle. ψ4, ψs ••• ΨF with values of 1, 0,0,0,0 ... 0 results.

In order to determine the number F of numerical shape features ψ _m , the number F, starting from F = 1, can be increased from several, in particular more than 29, samples per class of the classification system to such an extent that the values determined for the samples of one class for the class respective shape feature zu _m differs from the numerical value of this shape feature ψ _{m of} the other class in at least one numerical value for at least one shape feature ψ _m . Particularly preferably, the number F of the shape features is increased until the values of the shape features having the highest order numbers m decrease in all classes with increasing order number. The values of the respective shape feature ψ _m determined for the at least 29 samples per class can be arithmetically averaged to determine a value to be used for this class for this feature. The following reproduced table only to illustrate freely selected numerical values shows that a number F = 1 of numerical shape features ψ _{m is} not sufficiently precise for the determination of the weld defect on belonging to the defect classes "crack", "pore", "pore pores" is that ψi assumes nearly the same values for both the crack class and the tube pore classes. Only by the addition of the second numerical feature ψ ₂ is the assignment clear. It turns out that despite the similar numerical values for ψ ₂ in the class "pore" and "hose pore", this system of only two shape features ψ ₁ , ψ _{2 is} suitable for precisely classifying the 3 errors.

However, the number F can also be determined by way of a rotational ellipse determination method. Such "cluster processes" are described, for example, in H. Niemann, Klassifikation von Muster, Springer Verlag, Berlin, 1983, page 200ff.

The inventive method for assigning the content of a digital image in a class of a classification system is preferably used in the optical inspection of components, in particular in the optical surface chenenspektion. Furthermore, the method can be used for quality assurance, texture, shape and contour analysis, photogrammetry, character and character recognition, person recognition, robotic vision or the evaluation of radiographic or radioscopic images, ultrasound images and nuclear spin Tomography are used.

It does not matter whether the images with respect to which the object recognition is carried out are "optical" images from the spectral range of visible light or radiographic or radioscopic Images or even synthetic imagery. The method can therefore also be used in the field of optical surface inspection, such as in quality assurance, texture, shape and contour analysis, photogrammetry, character and character recognition, person recognition, robotic vision or the evaluation of radiographic or radioscopic images, ultrasound images and magnetic resonance tomography.

If one addresses a specific problem of object recognition in the context of this broad spectrum of possible applications, then the degree of complexity of the problem is clear from the beginning:

It is known in how many different object classes K the objects to be recognized are to be sorted. In contrast to the classification on the basis of heuristic features, the number of degrees of freedom of the form can be determined experimentally with reference to a representative sample of test objects in the new, algorithmic method with respect to each object class. The classification is done exclusively with feature vectors Ψ = (ψ _/ , ψ _ä ψ _A ψ ₄ ψ _Ä ...., ψp). The feature vector of any separated, bounded, planar object in the image lies inside a bounded, normalized F-dimensional subarea ("unit hypercube") of an F-dimensional feature space. Pattern classification takes place by problem-specific clustering of the interior of this F-dimensional unit hypercube.

The invention will be explained in more detail below with reference to a drawing which merely represents an exemplary embodiment. Show in it

Fig. 1 shows three different representations of a first character to be recognized;

Fig. 2 shows three representations of a second character to be recognized and

3 shows three representations of a third character to be recognized. In Figures 1, 2 and 3, the letters A, B and C are respectively in three forms of representation i) normal, ii) normal but rotated by 90 °, iii) same orientation as normal but smaller. Furthermore, in addition to the centered orientations shown in the figures, one positioning on the left and one positioning on the right were examined.

The following table shows the values for ψi, where ψi after the relationship

was calculated.

p ¹ results from the following relationships

p ^x = k _ι R ^]

/ max πnax

A = m _{0 0} = Aa * Ab * J ^ ₁ Σdata [j, i]

/ = 1 / = 1

, i _ ^y . υ,

R '= ^ = υ _n m 0.0

RU, i) = V ((/ - 0.5) * Aa - xf + (U - 0.5) * Ab - y -) \ 2

^m o, oy =

^W 0.0 m _lfi

/ max i max

W _{0 1} = (Ab) ² * Aa * Σ Σ (/ "0 = 5) * <photo [/, /]

7 = 1 / = l

Δa = width of the pixel in x-coordinate direction

= Δa = 0.3175 mm

Δb = width of the pixel in y-coordinate direction

= Δb = 0.3175 mm data [j, i] = content attribute of the pixel at the location (y ,, Xi)

With the aforementioned relationship and the respective data fields for the respective representations in which content attributes are stored at the points (y _j , x,), the values shown in the following table result:

Table of numeric values for the characteristic ψi

It can be seen that the value ψi assumes values of 0.57 for the letter A, values of 0.6 for the letter B and values of 0.44 for the letter C. Regardless of the actual position and size of the letter can thus be clearly recognized by the inventive method a previously defined character.

Claims

claims:

A method for assigning the content of a digital image to a class of a classification system, wherein the image is represented by N pixels, wherein a pixel in a predetermined

Coordinate system is located at the location (Xj, y _j ) and the image is above the

Coordinates (0,0) to (ximax, yimax) and imax is the maximum number of pixels in the x-coordinate direction and jmax is the maximum number of pixels in the y-coordinate direction and each pixel has at least one numeric content attribute data [j , i] is associated with the following steps:

Determining at least one group of a predetermined number of F numerical shape features ψ _m with m as a count index, where ψ _{m is} a transformed expression of the moment p ^m and p ^m from

p ^m = KR ^m

O _n m 0.0 i] R (j, i) = yj ((i - 0.5) * Aa - xγ + ((j - 0.5) * Ab - y - \) 2

x = • ^w o, o ^w o, iy =

0.0 7 max / max m _uo = (Aa) ² * Δ6 * Σ £ (/ - 0.5) * ύforta [/, /]

7 = 1 ι = l j max / max

W ₀ ,, = (Δfc) ² * Δa * Σ £ (y - 0.5) * cVc7to [/, /]

7 = 1 ι = l

Δa = width of the pixel in x-coordinates direction Δb = width of the pixel in y-coordinates direction data [j, i] = content attribute of the pixel at the position (y _j , Xj) m = one passing through the count of the shape features

Number from 1 to F,

Comparing the value of each shape feature of the F numerical shape features determined for the image in the at least one group with the value stored in a table for the respective shape feature of that group, wherein in the table of each class values for each numerical shape feature are assigned to that group, Output of the class as an assignment class into which the image to be determined has been classified, in which the numerical shape features determined for the image best correspond to the values of the numerical features specified in the table for this class.

2. The method according to claim 1, characterized in that the shape features ψ _m by the transformation

is determined.

3. The method according to claim 1 or 2, characterized in that the

Number F of the numerical shape characteristics ψ _{m is obtained} from at least 29 samples per class of the classification system by the

Number F is increased as far as that for the samples of a class determined values for the shape features ψ _m at least in their numer¬ merischen value for at least one shape features ψ _m different from the numer¬ merischen values of this shape feature ψ _m the other classes.

4. Use of the method according to one of claims 1 to 3 for the assignment of defects of castings to defi¬ ned in an industry standard ned error classes, wherein the digital image by means of radioscopy er¬ testifies.

5. Use of the method according to one of claims 1 to 3 for the assignment of defects of welds to error classes defined in an industry standard, wherein the digital image is generated by means of radioscopy.

6. Use of the method according to any one of claims 1 to 3 for the recognition of reproduced from paper objects, such as letters or tones in a score, wherein the digital image is generated by means of a scanner.