EP1525554A1

EP1525554A1 - Method for evaluating the signals of an electronic image sensor during pattern recognition of image contents in a test piece

Info

Publication number: EP1525554A1
Application number: EP03787716A
Authority: EP
Inventors: Volker Lohweg
Original assignee: Koenig and Bauer AG
Current assignee: Koenig and Bauer AG
Priority date: 2002-07-26
Filing date: 2003-07-22
Publication date: 2005-04-27
Also published as: DE10234086B4; US7483573B2; AU2003258460A1; CN1331086C; CN1672167A; DE10234086A1; WO2004017252A1; US20060050995A1

Abstract

Disclosed is a method for evaluating the signals of an electronic image sensor during pattern recognition of image contents in a test piece. According to the inventive method, the image sensor receives an input light signal and emits an electrical output signal which correlates with the input light signal. Said method comprises the following steps: the image content of a window which has a size of n x n pixels is analyzed; the output signals indirectly or directly emitted by the image sensor are transformed into at least one translationally invariant characteristic value by means of at least one calculation specification; the characteristic value is weighted by means of at least one fuzzy matching function which functionally correlates with the value range of the characteristic value; a higher-order fuzzy matching function is generated by linking all matching functions by means of a calculation specification comprising at least one rule; a sympathy value is determined from the higher-order fuzzy matching function; said sympathy value is compared with a threshold value; a decision is made regarding association with a class.

Description

description

Method for signal evaluation of an electronic image sensor in the pattern recognition of image contents of a test specimen

The invention relates to methods for signal evaluation of an electronic image sensor in the pattern recognition of image contents of a test specimen according to the preamble of claim 1, 2 or 19.

Known methods for analyzing image contents of a test object are usually based on metrics for determining similarities, such as distance measurements for segmented objects or the calculation of global threshold distributions. These methods are based on translation-invariant output spectra. In reality, situations often occur, such as object shifts under the recording system or different backgrounds in the recording or aliasing effects, so that a direct comparison of these output spectra in many cases can not be performed.

Through the textbook Thomas TILLI "Pattern recognition with fuzzy logic: Analyze, classify, recognize and diagnose" Franzis-Verlag GmbH, Munich, 1993, pp. 183/184, 208-210, 235-257 it is known in methods for image processing Fuzzy logic to use, where a type of signal preparation can be a spectral transformation.

In the article "Pattern recognition with fuzzy logic" by Peter ARNEMANN, Electronics 22/1992, pages 88-92 is described to perform pattern recognition using fuzzy logic.

The article by Charalampidis, D .; Kasparis, T .; Georgiopoulos, M .; Rolland, J. "A fuzzy ARTMAP based classification technique of natural textures "Fuzzy Information Processing Society, 1999. NAFIPS, 18 ^th International Conference of the North American, 10-12.06.1999, pp. 507-511 describes performing pattern recognition with a training phase and window for image recognition to use with 16x16 pixels.

The invention has for its object to provide a method for signal evaluation of an electronic image sensor in the pattern recognition of image content of a specimen.

The object is achieved by the features of claim 1, 2 or 19.

An advantage of the method lies, in particular, in the fact that a sensor signal is analyzed in an image window of size n × n pixels. As a result, the sensor signal of this image window can be considered local. The image analysis method according to the invention can be subdivided into the essential steps: feature formation, fuzzyfication, interference, defuzzification and decision on class membership.

When the feature is formed, the sensor signal is converted by means of at least one calculation rule into an invariant, in particular a translation-invariant, signal in the feature space. The aim of the feature formation is to determine such quantities by which typical signal properties of the image content are characterized. The typical signal properties of the image content are represented by so-called features. The features can be represented by values in feature space or by linguistic variables. By transferring the sensor signal into the feature space, a signal is generated which consists of a feature value or of several feature values. The membership of a feature value to a feature is described by at least one fuzzy membership function. This is a soft or fuzzy assignment, whereby, depending on the value of the feature value, the feature value belongs to the feature in a standardized interval between 0 and 1. The concept of the membership function means that a feature value is no longer either completely or not at all attributable to a feature, but rather can assume a Fuzzyzugehörigkeit, which lies between the Boolean truth values 1 and 0. The step just described is called fuzzyfication. In fuzzyfication, therefore, there is essentially a conversion of a sharp feature value into one or more fuzzy affiliations.

In the case of interference, a superordinate membership function is generated by means of a calculation rule which consists of at least one rule, with all membership functions being linked to one another. As a result, a higher-level membership function is thus obtained for each window.

During the defuzzification, a numerical value also called the sympathy value is determined from the superordinate membership function formed in the interference. When deciding on the class affiliation, the sympathy score is compared to a predetermined threshold value that determines the membership of the window to a particular class.

Which type of feature values are in the feature space is of secondary importance to the principle of the invention. In the case of time signals, for example, it is possible to determine the mean value or the variance as feature values. If the request is submitted to the evaluation method that it can process the image contents error-free irrespective of the prevailing signal intensity, and if further small but permissible fluctuations of the image signal should not lead to disturbances, then it makes sense if the conversion of the sensor signal from the two-dimensional Space is performed by means of a two-dimensional spectral transformation, such as a two-dimensional Fourier, or a two-dimensional Walsh, or a two-dimensional Hadamard or a two-dimensional circular transformation. The two-dimensional spectral transformation yields invariant feature values. Another preferred embodiment is to use the amount of spectral coefficients obtained by spectral transformation as the feature value.

According to a preferred embodiment, the membership functions are unimodal potential functions and the parent membership function is a multimodal potential function.

According to a further preferred embodiment, at least one membership function is parameterized. If the membership function has positive and negative slopes, then it is advantageous if the parameters of the positive and negative slopes can be determined separately. This ensures better adaptation of the parameters to the data records to be examined.

According to a particularly preferred embodiment, the method can be divided into a learning phase and a working phase. Once the membership functions have been parameterized, the parameters of the membership function can be determined in the learning phase from measured data records. In the learning phase, the parameters of the membership functions are adapted to so-called reference images, ie in the learning phase, a membership of the feature values resulting from the reference images is derived from the corresponding features by means of the membership functions and their parameters. In the following work phase, the feature values resulting from the now measured data sets are weighted with the membership functions whose parameters were determined in the learning phase, whereby a membership of the feature values of the now measured data sets to the appropriate characteristics is produced. By subdividing the method into a learning and a working phase, therefore, the parameters of the membership functions are determined on the basis of measured reference data records, and in the subsequent work phase the measured data records to be tested are weighted and evaluated with the membership functions defined in the learning phase.

According to a further preferred embodiment, at least one rule, by means of which the membership functions are linked to one another, is a conjunctive rule in the sense of a IF ... THEN connection.

Another preferred embodiment divides the generation of the superordinate fuzzy membership function into the processing of the sub-steps: premise evaluation, activation and aggregation. In the premise evaluation, a membership value is determined for each IF part of a rule, and upon activation, a membership function is defined for each IF ... THEN rule. Subsequently, during aggregation, the superordinate membership function is generated by superimposing all the membership functions generated during activation.

According to a further preferred embodiment, the sympathy value determination is carried out in particular according to a center of gravity and / or maximum method.

Embodiments of the invention are illustrated in the drawings and will be described in more detail below.

Show it:

1 is a flowchart of the signal evaluation method, 2 shows a sympathy curve,

3a shows a difference function with the power D = 8

3b shows a difference function with the power D = 4

3c shows a difference function with the power D = 2

1 shows a flowchart of the signal evaluation method described below. In the method for signal evaluation of image contents of a test specimen, a grid of N x N windows 01 is laid over the entire image to be analyzed. Each window 01 consists of n × n pixels 02. In the image analysis, the signal of each window 01 is analyzed separately. As a result, the image content 03 of each window 01 can be considered local.

By one or more spectral transformations 04, the two-dimensional image of the spatial domain is transformed into a two-dimensional image in the frequency domain. The spectrum obtained is called frequency spectrum. Since it is a discrete spectrum in the present embodiment, the frequency spectrum is also discrete. The frequency spectrum is formed by the spectral coefficients 06, also called spectral values 06.

In the next method step, the amount formation 07 of the spectral values 06 takes place. The magnitude of the spectral values 06 is called spectral amplitude value 08. The spectral amplitude values 08 form the feature values d in the present exemplary embodiment. H. they are identical to the characteristic values.

For transformation, a circular transformation is preferably used. In the circular transformation, the invariance properties are over the 6a

Transformation coefficients adjustable. It is possible to set a translational invariance, as well as a reflection invariance or invariance with respect to various other permutation groups. Thus, it is possible to use the above transformation, for example, in the variant variant for examining characters

(think of the distinction between the numbers "9" and "6"). In the same way, the mirror-invariant variant can be used for the examination of workpieces; because here it is just not necessary to make a distinction between a mirrored part and the original. It should be noted that the magnitude spectrum of the Fourier transform is invariant to mirror.

These transformations work with real coefficient values. So you do not have to try a complex calculation like the Fourier transformation.

The circular transformation is also extremely tolerant in the subpixel range for arbitrary shifts. Comparisons have shown that this circular transformation is superior to other known transformations in terms of displacements.

The number of working coefficients (features, features) is low, since the spectral coefficients are summarized again in groups.

By combining the tolerance arises to shifts. Even if a signal partially runs out of a measuring field, the features still remain relatively stable. Investigations have shown that stability is maintained, even if image content is up to 30% out of the measuring field.

As a further method step follows the feature selection 09; The goal in the feature selection 09 is to select the features 11 which are characteristic of the image content 03 of the image to be analyzed. Characteristics 11 are both characteristic spectral amplitude values 08, which define the feature 11 by their position in the frequency space and by their amplitude, as well as linguistic variables such as "gray", "black" or "white".

In the next step of the process, the fuzzification 12, the membership of each Specified spectral amplitude value 08 to a feature 11 by a soft or fuzzy membership function 13; ie a weighting takes place.

If the membership functions 13 can be adapted in a learning phase to so-called reference data sets, it makes sense if the membership functions 13 are parameterized monomodal, i. are one-dimensional potential functions in which the parameters of the positive and negative slope can be adjusted separately to the data sets to be examined. In the work phase following the learning phase, the data sets of the image content, from which the feature values 08 of the test images result, are then weighted with the respective membership functions 13 whose parameters were determined in the previous learning phase. Ie. For each feature 11, there is a kind of DESIRED comparison between the reference data record expressed in the parameters of the membership functions 13 and the record of the test image. The membership functions 13 produce a soft or fuzzy association between the respective feature value 08 and the feature 11.

In the next method step, the interference 14, there is essentially a conjunctive connection 15 - also called aggregation 15 - of all membership functions 13 of the features 11, whereby a higher-order membership function 16 is generated.

The next process step, the defuzzification 17, obtained from the parent membership function 16 a concrete of belonging or sympathy value ^'18. This emotional value 18 is compared in the classification 19 with a previously set threshold value 21, whereby a classification statement can be made. Threshold 21 is set either manually or automatically. The setting of the threshold value 21 also takes place in the learning phase.

In the classification, not a numeric value for a particular class is replaced by a True or false statement directly assigned, but a unimodal function set that describes a membership of a true or false statement.

In doing so, we train the class membership, ie the decision curves are learned based on measured values determined in the process. The functions with which a degree of affiliation is determined are called membership functions ZGF = μ (m _x ). The calculated value of the membership function ZGF is called the sympathetic value μ. Often several membership functions ZGF are used, which are further summarized in subsequent steps in order to obtain a clear statement.

However, it is expressly not a neural network that finds use. Neural networks are known to be able to be trained.

The fuzzy disk classification is based on a concept that accomplishes a distance measure and a feature join at the same time. "Fuzzy" is the fact that the features are "rounded", but not logical, but out of focus. This leads firstly to the fact that all features are taken into account summarily. That is, small deviations of a feature are still tolerated. Secondly, if the deviation of a feature becomes too large, it immediately has a large influence on the distance measure. Accordingly, the output of the classifier itself does not provide a "good / bad" decision, but rather a continuous output value between [0 .... 1]. A threshold value is then used downstream, which then permits a "good / bad" decision ,

The initial value for the distance measure (sympathy value) is: μ = 2 ^'2 with 10

^ (\ m _x -x ₀ (m _x ^D 0

The coefficients have the following meaning: x = counting index; z = average distance measure; M = number of features; Xo = average of C _dlff ; C _x = expansion value; D = power; μ = sympathetic value; C _dPff = differential expansion _value .

The expansion value C is learned with the aid of measured values generated with the circular transformation.

The μ value describes how close the similarity of a pattern is to a reference pattern described by features. This means that the z value assumes the actual control of the μ value. If the z-value is very small, the μ-value is close to 1. The patterns are very similar (sympathetic). On the other hand, if the z value is large, the μ value will become small; the patterns are not similar. The course of the curve - as implemented - is shown in FIG.

First of all, the values Cdiff _{x are} determined in the learning phase, specifically a value for each feature m _x .

Cdiff = _x RMX (m _x) - min (w_-)

where Cd, the difference measure of the evaluation value C and m are the features.

With the help of the learned C _dlff values is worked during the inspection. The values can still be assigned an additional tolerance a. The settlement takes place at runtime: 11

C _x - (l + 2p _Ce ), a = (.l + 2j3 _Cfe ),

where C is the expansion value and Pcβ is the percentage tolerance of C _diff .

The value range of a lies between [1 ... 3], The value pc _e indicates the percentage tolerance with which C _d iff is assigned. A 50% extension of the range of C _{d> ff should be} achieved; then a = 1 + 2 * 0.5 = 2.

The x <r value indicates the mean of C _di ff; it is calculated for each feature at runtime.

The difference between the feature value and the mean feature value, which is determined from the value C _x , is calculated. This difference is normalized with the width of the expansion value C _x . The consequence is that the corresponding feature contributes little to the z-value with little deviation; with a large deviation, however, a large deviation value will result depending on the difference measure of the expansion value C _d j _ff . Name the normalized difference d _x .

The power D (2, 4, 8) adjusts the sensitivity at the edges of the normalized difference function d _x . If the value D is set to "infinite" - which is not technically possible - you will also get an infinite slope and thus a hard "good / bad decision", which is why the values are usually set to between 2 ... 20. The curves for the values 2, 4 and 8 are shown in Figures 3c, 3b and 3a.

The exponentiated functions d _x are summed and only the number M of features m is used, which are also switched on. After the summation, the calculated value is divided by the number M. The mean value of all potentiated differences d _{x is} determined. 12

The effect is this: by potentiating, small deviations will not fall into the weight; big ones, however, will be strengthened. Averaging calculates a deviation of all feature differences. This has the consequence that even with the deviation of several features of the μ-value is not drastically lowered. Only with larger deviations, this value will be very small.

Subsequently, a threshold value evaluation takes place.

[Good, if (z)> μ _s [error, if μ (z) <μ _s

This process is performed for all windows.

An evaluation of dynamic processes - such as printing processes - requires nonlinear distance measures (sympathy values).

13

LIST OF REFERENCE NUMBERS

01 window, N x N window

02 n x n pixels

03 image content

04 2-dimensional spectral transformation, calculation rule 05

06 Spectral coefficient, spectral value

07 2-dimensional amount calculation, calculation rule

08 spectral amplitude value = characteristic value

09 Feature Selection 10

11 characteristic

12 fuzzyfication

13 membership function

14 Interference, calculation rule 15

16 superordinate membership function, linkage, aggregation, calculation rule

17 Defuzzification

18 affiliation value, sympathy value

19 Classification, class membership 20

21 threshold

C expansion value

C _diff difference _measure expansion value

D potency

M number of features 14

ZGF membership function

a tolerance d _x nominated difference m characteristic p _Ce percentage tolerance of C _diff x counting index z average distance measure

μ sympathetic value, distance measure

Claims

15 Claims

1. Method for signal evaluation of an electronic image sensor in the pattern recognition of image contents of a test specimen, the image sensor receiving a light input signal and being an electrical output signal which correlates with the light input signal, with the following steps:

- Analysis of the image content (03) of a window (01) with the size n x n pixels (02) by

Conversion of the output signal output indirectly or directly by the image sensor into at least one invariant feature value (08) by means of at least one calculation rule (04, 07),

- weighting of the feature value (08) with at least one unsharp membership function (13), the membership function (13) being functionally related to the range of values of the feature value (08),

- Generation of a superordinate fuzzy membership function (16) by linking all membership functions (13) by means of a calculation rule (14, 15) consisting of at least one rule,

- determination of a sympathy value (18) from the superordinate fuzzy membership function (16),

- Comparison of the sympathy value (18) with a threshold value (21),

- Decision on class affiliation (19).

2. Method for signal evaluation of an electronic image sensor in the pattern recognition of image contents of a test specimen, the image sensor receiving a light input signal and being an electrical output signal which correlates with the light input signal, with the following steps:

- Analysis of the image content (03) of a window (01) of size n x n pixels (02),

- Two-dimensional spectra are determined from this image content (03), 16

- Spectral amplitude values are calculated from these two-dimensional spectra and linked to one another, so that only one sympathy value (18) arises per window.

3. The method according to claim 1, characterized in that only a single sympathy value (18) is calculated per window (01) of size n x n pixels.

4. A method for signal evaluation according to claim 1 or 2, characterized in that the image of the test specimen to be examined is divided into N x N grid-like windows (01) of size n x n pixels (02).

5. A method for signal evaluation according to claim 1 or 2, characterized in that the sympathy value (18) is determined in particular according to a center of gravity and / or maximum method.

6. The method according to claim 1 or 2, characterized in that the sympathy value (18) are not linear.

7. The method for signal evaluation according to claim 1 or 2, characterized in that the method is divided into a learning phase and a working phase, wherein at least one parameter and / or at least one threshold value (21) is determined and adjusted in the learning phase, and wherein in the work phase the image content (03) of a test specimen is assessed on the basis of the results from the learning phase.

8. The method according to claim 1 or 2, characterized in that the class membership is trained in a learning phase, d. H. the membership function (13; 16)

^" learned.

9. The method according to claim 1 or 2, characterized in that an initial value for the sympathy value (18) is formed from the following formula:

The initial value for the distance measure (sympathy value) is: μ = 2 ^z with 17th

where x = a counting index, M = the number of characteristics, m = characteristic, x ₀ = mean of C _d i _ff, D = power. C _diff = difference _{measure of} the expansion value C.

10. The method for signal evaluation according to claim 1 or 2, characterized in that the calculation rule (04) for converting the signal of the image sensor into an invariant feature value (08) is a two-dimensional mathematical spectral transformation method (04), in particular a two-dimensional Fourier or Walsh , or Hadamard or circular transformation.

11. The method for signal evaluation according to claim 1, characterized in that a feature value (08) is represented by the amount of a spectral coefficient (06).

12. The method for signal evaluation according to claim 1, characterized in that at least one membership function (13) is described by at least one parameter.

13. A method for signal evaluation according to claim 1, characterized in that the membership functions (13) are unimodal functions.

14. A method for signal evaluation according to claim 1, characterized in that the higher-level membership function (16) is a multimodal function.

15. The method for signal evaluation according to claim 1, characterized in that the membership functions (13) and / or the higher-level membership function (16) is (are) potential function (s). 18th

16. A method for signal evaluation according to claim 1, characterized in that at least one calculation rule (14; 15) by means of which the membership functions (13) are linked together is a conjunctive calculation rule (14; 15) in the sense of an IF ... THEN link .

17. A method for signal evaluation according to claim 1, characterized in that the generation of the higher-level fuzzy membership function (16) is carried out by processing the sub-steps of premise evaluation, activation and aggregation (15), with the premise evaluation for each IF part of a calculation rule (14 ; 15) a membership value is determined, and in the activation a membership function is determined for every IF ... THEN calculation rule, and in the aggregation (15) the superordinate membership function (16) by superimposing all membership functions generated during the activation ( 13) is generated.

18. The method according to claim 2, characterized in that the sympathy value (18) is determined by means of fuzzy logic.

19. Method for signal evaluation of an electronic image sensor in the pattern recognition of image contents of a test specimen, the image sensor receiving a light input signal and being an electrical output signal which correlates with the light input signal, with the following steps:

- spectra are generated from at least one image content (03),

- The spectral transformation is generated by a circular transformation.

20. The method according to claim 19, characterized in that an invariant spectrum is generated. 19th

21. The method according to claim 20, characterized in that the invariance property can be set via the transformation coefficients.

22. The method according to claim 19, characterized in that the circular transformation is carried out with real coefficients.

23. The method according to claim 19, characterized in that associated working coefficients are formed by grouping together spectral coefficients.