EP1118060A1

EP1118060A1 - Method and device for assigning an object to at least one class

Info

Publication number: EP1118060A1
Application number: EP99955709A
Authority: EP
Inventors: Thomas BRÜCKNER
Original assignee: Siemens AG
Current assignee: Siemens AG
Priority date: 1998-09-30
Filing date: 1999-09-14
Publication date: 2001-07-25
Also published as: WO2000019335A1; US20020007381A1

Abstract

According to the invention an object is assigned to at least one class among a group of classes by determining, for each class, a membership criterion of said object to a class. Class-dependent threshold values for assignment are determined by optimization of an evaluation criterion under defined secondary conditions. A comparison of the membership criterion with the threshold values of the classes yields a corresponding assignment of the object to at least one class.

Description

description

Method and arrangement for assigning an object to at least one class

The invention relates to a method and an arrangement for assigning an object to at least one class from a set of predetermined classes by a computer.

A method and an arrangement for classifying a text is known from [1].

In classification, an object is assigned to one or more classes (multi-classification system) by determining a membership measure for the object and comparing it with an associated threshold value for each class. If the membership measure for the respective class is greater than the threshold value of this class, the object is assigned to the class. It is disadvantageous here that the threshold values of all classes are predetermined globally and thus an inaccurate classification takes place.

A method for solving a linear system of equations from [2] is also known.

The object of the invention is to enable a classification, wherein specific threshold values are automatically determined for several classes.

This task is carried out according to the characteristics of the independent

Claims resolved. Further developments of the invention also result from the dependent claims.

To solve the problem, a method for assigning an object to at least one class from a set of predetermined classes is specified by a computer, in which for each class a membership measure of the object to the Class is determined. A threshold value is calculated for each class from the set of classes by optimizing an evaluation measure under specified constraints. The object is assigned to a class from the set of specified classes if the membership measure is above the associated threshold value of the class. In the other case, an object is not assigned to a class from the several predetermined classes if the membership measure is below the threshold value of the class.

It is an advantage that an individual threshold value can be determined for each class, which directly takes into account class-typical requirements by optimizing the evaluation measure.

A further training consists in that the evaluation measure depends on the threshold values of the classes. In this case it is advantageous that the threshold values are included directly or indirectly in the evaluation measure.

It is also a further development that the evaluation measure comprises one of the following specifications: a) number of errors; b) recognition rate (recall); c) Detection rate of a detection system (precision).

The evaluation measure can be based on certain peculiarities of the object to be classified. In particular, it is useful to take the classification error into account in the evaluation measure and to optimize it with regard to certain specifications.

Another development is that the evaluation measure is provided with a condition that requires at least a predetermined value for the evaluation measure. This takes into account that one of the above specifications with a certain value are applied and this is taken into account when optimizing the evaluation measure under the specified secondary conditions.

Another development is that an additional constraint is specified by specifying a minimum value for an additional evaluation measure for all classes. This additional constraint is included in the optimization of the evaluation measure by adding it to the specified constraints.

The secondary conditions are preferably formulated such that:

kn + kχ2 +. • - + kiM = 1 ^k Nl + ^k N2 + - • - ^{+ k} NM =!

where kj_j is a result of the class k_ that goes into the evaluation measure at threshold value Tj, N is the number of classes,

M denotes the number of threshold values Tj and kj_j can only assume the values 0 or 1.

One embodiment also consists in the fact that the evaluation measure is optimized under the secondary conditions by solving a linear system of equations (here: evaluation measure with specified auxiliary conditions and possibly additional auxiliary conditions) by means of an LP solver (see [2]).

One embodiment also consists in the fact that the text classification method is used. Especially when it comes to text classification, it is customary to use a given text in different (thematic) classes, so-called To assign domains. Naturally, a given text can be assigned to several domains. The decision as to whether the assignment is made or not is made for each class by comparing the value determined for the text by means of an evaluation measure with the predetermined threshold value of the domain (class). For the text, there is a membership measure for each class; the assignment is made if the membership measure is above the threshold value of the respective class. The threshold values for the class have been predetermined in particular using the above-mentioned method.

It is also an embodiment that an optimal threshold value is determined for each class.

As part of an additional embodiment, a "microaveraged" evaluation is carried out on the basis of the threshold values determined for the classes. This will be discussed in detail within the scope of the exemplary embodiment.

To achieve the object, an arrangement for assigning an object to at least one class from a set of predetermined classes is also specified, in which a processor unit is provided which is set up in such a way that

a) a measure of the object's belonging to the class can be determined for each class;

b) a threshold value can be calculated for each class from the set of classes by optimizing an evaluation measure under specified secondary conditions;

c) the object can be assigned to a class from the set of predetermined classes if the membership measure lies above the threshold value of the class; d) the object cannot be assigned to a class from the set of specified classes if the membership measure is below the threshold value of the class.

This arrangement is particularly suitable for carrying out the method according to the invention or one of its developments explained above.

Embodiments of the invention are illustrated and explained below with reference to the drawing.

Show it

Fig.l is a block diagram with steps of a method for assigning an object to at least one class from a set of predetermined classes;

2 shows a table with dimensions for a text classification;

3 shows a processor unit,

1 shows a block diagram with steps of a method for assigning an object to at least one class from a set of predefined classes. In a step 101, a membership measure is determined for the object for each class. This should provide information about whether the object can be assigned to the respective class. The assignment is generally made when the membership measure exceeds a predefined threshold value (for the class). In order to determine a class-dependent threshold value, that is to say to determine a threshold value for each class, an evaluation measure is optimized in step 102 under predetermined secondary conditions, the evaluation measure depending on the threshold values of the classes. The optimization results in threshold values for the classes, specifically one threshold value for each class. In one Step 103 checks whether the membership measure is greater than the respective class-specific threshold value. If this is the case, the object is assigned to the respective class in accordance with step 104, otherwise no assignment to this class is made (cf. step 105). According to the wording used here, a measure of deviation instead of the membership measure can alternatively be used, whereby the measure of deviation is understood only as the negated wording.

Fig. 2 shows a table with dimensions for one

Text classification. Text classification can be used to assign the object to at least one class. Predefined texts are assigned to different classes (domains), with each class mostly belonging to one subject area. A concrete realization consists in the assignment of newspaper texts to one or more topics, e.g. Sports, literature, politics and / or business. As mentioned above, the evaluation measure is optimized under given constraints. The evaluation measure itself can include certain specifications. Some possible specifications are explained in more detail below using the table in FIG. Fields 201 to 204 show possible classification states. The field 201 "a" contains all automatically correctly hit by the system

Classification decisions that are actually correct, i.e. judged correct by an expert. Field 202 "b" contains the number of all assignments classified as correct by the system which actually (according to

Expert opinion) are wrong. Field 203 "c" indicates the number of classifications that the system has assigned as incorrect, but which in reality would have been correct. Finally, field 204 "d" includes all incorrect assignments that the system has classified as incorrect. There are now different specifications that can be defined based on the above sizes. The

Recall rate is defined as the number of correct (recognized) assignments divided by the number of possible assignments:

Recall =! 2) a + c

A detection rate (precision) is determined by the number of correct assignments divided by the number of all automatic assignments:

a Precision a + b

A system failure (fallout) is determined by

Fallout (4!

An error rate is determined by

b + c

Errorrate = (5) a + b + c + d

The specified specifications equation (2) to equation (5) are suitable for specifying the classification quality in the form of a suitable evaluation measure Q. On the one hand, the evaluation measure can be determined directly across all classes ("icroaveraged" evaluation measure):

where N denotes the number of classes k. Alternatively, the evaluation measure is first determined individually for each class and then averaged over all classes ("macroaveraged" evaluation measure):

Q. ( ^k l) + Q (k ₂ ) + ... + θ (k _N )

N

As already mentioned, it cannot generally be assumed that one and the same threshold value ensures a sufficiently high quality of the assignment (classification) for all classes. A determination of a threshold value for each class should therefore be made so that the

Overall classification quality is as high as possible. In the "macroaveraged" evaluation measure, this is done by determining the value for each class as a threshold that optimizes the evaluation measure for this class:

Q- (l) + θ (k ₂ ) + ... + Q (k _N ) - -—- = ax <=>

^N (8).

C (kι) + Q. (k ₂ ) +. - - + θ (k _N ) = max

In the case of the "microaveraged" evaluation measure, equation (8) does not work. A lot of threshold values are searched here

{ _L , T ₂ , ..., T _N }

which takes into account an unequal distribution of the classes. If, for example, a class ki occurs twice as often as a class k ₂ , this is not included in equation (7) and is therefore not sufficiently taken into account in the overall classification quality.

The best set of threshold values (see equation (9)) is determined by formulating a linear optimization problem and solving it using linear programming (see LP solver). With a training set of objects whose Classification is known, an evaluation is carried out with M different threshold values. In the following, the result of the class k _x for the m the evaluation measure Q comes in at a threshold value T-,. The linear optimization problem to be solved is formulated as follows:

Objective function: max Q (kn, k] _ ₂ , ..., kι _M , ..., (10)

Supplementary conditions: ^k ll + 12 + - ^• - + lM = 1 21 + ^k 22 + - ^• - ⁺ 2M = 1 di; ^k Nl + N2 + - • - + ^k NM = 1

Based on the secondary conditions, the evaluation measure Q only receives one result per class (k _x - _| only takes the values 0 or 1). The set of k -, _- ,, well

{ ^k la> 2b '• •• / Nz} (12),

that maximize the objective function are the solution of the

Optimization problem. For the optimization of the "microaveraged" evaluation measure, this means that an optimized result is achieved with the threshold values T _a for the class k _j _, T ^ for the class k ₂ , ... and T _z for the class k ^.

This procedure can also be used if the goodness of the classification system has to be indicated by two evaluation measures Q _] _ and Q ₂ , eg using recall and precision (see above). The system of equations is optimized by adding an additional constraint:

Objective function: F8 max Q_ (kn, k ^ ₂ ^ • • -> ^k lM> • • •> ^k Nl ^'k N2' • • •> ^k NMJ Supplementary conditions: F9 ll + ^k 12 + - ^• - ^{+ k} lM = 1 ^k 21 + 22 + - - - ^{+ k} 2M = 1

^k Nl + ^k N2 + - ^• - + ^k NM = 1

Qi ( ^k ll ' ^k 12' • • • ' ^k lM' • • • ' ^k Nl' ^k N2 '• • •' ^k NMJ ^{≥ x} (13 ⁾ ,

5 where X denotes a predetermined minimum value for the evaluation measure Q ^.

With regard to the above statements, in particular Fig. 2, the following definitions are agreed:

10

MIC Recall = (14),

" _Τ ,". , ^a l 'a ₂ , ..., a _N

MIC Precision = η ^ 7 r (15),

(a _lr a ₂ , ..., a _N + \ b _lf b ₂ , ..., b _N j

^a l ₊ ^a 2 _{+ +} _ ^a

1, 5, "MA..C_ _D Recal _ι l _π = ^a l + ^c l ^a 2 + ^c 2 ^a N + ^C N _I (1 _e 6 ..,

N

MAC Precision = (17),

N

where MIC "microaveraged" and MAC "macroaveraged". 20 The designations a_, b_, C_ and dj_ give the respective

Values in the table from FIG. 2 are restricted to the class k _j _.

A processor unit PRZE is shown in FIG. The processor unit PRZE comprises a processor CPU, a 25 SPE memory and an input / output interface IOS, which is used in different ways via an interface IFC: an output on a monitor MON and / or on a monitor is visible via a graphic interface PRT printer output. An entry is made with a mouse or MAS KEYBOARD. The processor unit PRZE also has a data bus BUS, which ensures the connection of a memory MEM, the processor CPU and the input / output interface IOS. Furthermore, additional components can be connected to the data bus BUS, for example additional memory, data storage (hard disk) or scanner.

Bibliography :

[1] W097 / 38382, "Method for the automatic classification of text on a document after its transformation into digital data".

[2] http: //www.wior.uni-karlsruhe.de/Bibliothek/Software_for_ OR / Linear_Programming / pub / lp_solve / index .html, 28.09.1998

Claims

claims

1. Method for assigning an object to at least one class from a set of predetermined classes by a computer,

a) in which for each class a membership measure of the object to the class is determined;

b) in which a threshold value is calculated for each class from the set of classes by optimizing an evaluation measure under specified secondary conditions;

c) in which the object is assigned to a class from the set of predetermined classes, if that

Membership measure is above the threshold of the class;

d) in which the object is not assigned to a class from the set of specified classes, if that

Affiliation measure is below the threshold of the class.

2. The method of claim 1, wherein the evaluation measure depends on the threshold values of the classes.

3. The method of claim 1 or 2, wherein the evaluation measure comprises one of the following specifications: a) number of errors; b) recognition rate (recall); c) Detection rate of a detection system (precision).

4. The method of claim 3, wherein the evaluation measure with a condition that requires at least one predetermined value for the evaluation measure.

5. The method as claimed in one of the preceding claims, in which an additional constraint is specified by specifying a minimum value of an additional evaluation measure for all classes.

6. The method according to any one of the preceding claims, in which the secondary conditions are formulated such that the following applies:

^k ll + ^k 12 + - • - ^{+ k} lM = 1 ^k 21 + ^k 22 + - - + ^k 2M = 1

^k Nl ^{+ k} N2 + - • - ^{+ k} NM ⁼ 1

where k_- _{j is} a result of the class k _x in the evaluation measure at threshold value T- _j ,

N the number of classes,

M denotes the number of threshold values T-, and k_-, can only assume the values 0 or 1.

7. The method as claimed in one of the preceding claims, in which the evaluation measure is optimized under the secondary conditions by solving a linear system of equations by means of an LP solver.

8. The method according to any one of the preceding claims, in which a text classification is carried out.

9. The method of claim 8, wherein the classes in the text classification are domains.

10. The method according to any one of the preceding claims, in which a separate optimal threshold value is determined for each class.

11. The method according to any one of the preceding claims, in which a "microaveraged" evaluation is optimized on the basis of the threshold values determined for the classes.

12. Arrangement for assigning an object to at least one class from a set of predetermined classes, in which a processor unit is provided which is set up in such a way that

c) the object can be assigned to a class from the set of predetermined classes if the membership measure lies above the threshold value of the class;

the object cannot be assigned to a class from the set of specified classes if the membership measure is below the threshold value of the class.