JPH07160658A - Method for classifying data - Google Patents

Abstract

PURPOSE:To provide a learning method for which inductive machine learning and neural network learning are combined for utilizing the merits of both inductive machine learning and neural network learning and making up demerits. CONSTITUTION:In this method for classifying data, learning is performed by the inductive machine learning and when a category can not be discriminated by the learning, the learning is performed by a neural network. Thus, by combining the inductive learning and the neural network, the recognition rate of the data is improved. Even when an erroneous answer is worked out by the neural network, between which categories an error is made is limited by the inductive learning. The categories can be limited further by using a feature that the attribute of the multidimensional space of the neural network us nonlinearly discriminated even in a part where attribute values overlap.

Description

Detailed Description of the Invention

[0001]

The present invention relates to a method of classifying data into several categories (classes) when the data is given as a set of pairs of attributes and their values, and there is a distribution in the attributes. In particular, it relates to a method combining inductive machine learning and neural network learning, and particularly to a data classification method useful for pattern recognition and accident diagnosis.

[0002]

2. Description of the Related Art Conventionally, an inductive machine learning method has created an identification tree with discrete attribute values without distribution of attribute values. Japanese Patent Application No. 4-130083 proposes an inductive learning method that can handle even when attribute values overlap.

[0003]

By the way, in Japanese Patent Application No. 4-130083 previously proposed, it is necessary to present two or more categories in which situations cannot be classified when the attribute values overlap. , And presented which category the probability distribution belongs to. However, even if two or more categories are presented, it is desirable to limit learning to one category if possible.

Further, in the learning method using the neural network, an ambiguous result or an erroneous result may be produced when the distribution of attribute values is wide, or when the number of categories and the number of data are large. However, it has the feature that attributes of multidimensional space can be identified non-linearly by using neural network. Therefore, it is possible to limit the learning to one category.

On the other hand, in Japanese Patent Application No. 4-130083, learning is possible even when the distribution of attributes is wide, but there is no feature that attributes in a multidimensional space can be identified in a non-linear manner, and identification in a linear manner is possible. Only.

Therefore, the problem to be solved by the present invention is to propose a learning method combining inductive machine learning and neural network learning, which makes use of the advantages of both inductive machine learning and neural networks and compensates for their drawbacks. It is in.

[0007]

In order to solve the above problems, the first classification method of the present invention performs learning by inductive machine learning, and when a category cannot be discriminated by the learning, performs learning by a neural network. It is a thing.

The second classification method of the present invention is to learn the categories excluding the completely identifiable categories by a neural network by the recursive machine learning method.

The third classification method of the present invention uses machine learning for attributes that identify all combinations of categories in which at least one attribute value distribution is separated by inductive machine learning. Learning is performed by a neural network for a category that cannot be learned by machine learning.

In the fourth classification method of the present invention, learning by machine learning is performed at a node higher than the machine learning performed by the third classification method, and a category that cannot be learned by the machine learning is learned by a neural network. It is a thing.

The fifth classification method of the present invention is a combination of the third classification method and the fourth classification method for performing machine learning and neural network learning.

[0012]

According to the present invention, when the distribution of attribute values is wide, the number of categories and the number of data are large, the categories are first identified by the inductive learning method, and if two or more categories cannot be identified, the neural network is used. The category is further limited by using the feature of non-linearly identifying the attributes of the multidimensional space.

[0013]

The present invention will be specifically described below. First, in order to clarify the characteristics of the present invention, the following accident cause diagnosis is first shown with examples and results based on the method disclosed in Japanese Patent Application No. 4-130083, and then using the results, Explain at what stage the neural network is combined.

FIG. 1 is an electrical system diagram showing an example of a test circuit for carrying out an accident cause diagnosis method relating to data classification. In the figure, 1 is a circuit breaker, 2 is a ZPD (zero phase voltage detector), 3 is a power source side capacitor, 4 is a zero phase current transformer,
Reference numeral 5 is a load side capacitor, 6 is a switch for accident occurrence, 7 is a high voltage switch, 8 is a controller, and 9 is a transformer. The tested accident methods are insulator, cross-linked polyethylene cable, poultry, complete ground fault, resistance ground fault and gap ground fault.

The zero-phase current at the time of failure was recorded on a digital waveform recorder (sampling rate was 48 KHz). A part of the recorded waveform is cut out for 200 ms, and the FFT (Fast Fourier Transform) waveform is used to distort (%) from the 2nd to 8th order with the fundamental wave at 60 Hz and the total harmonic distortion (%) (from the 2nd order) 20th) was calculated. The configuration of the measurement system is shown in the block diagram of FIG. In addition, a representative waveform and FFT analysis results are shown in FIG.
Shown in.

A) Decision Tree Learning Method The m categories to be selected here are C ₁ , ..., C _i ,
..., and C _m, n number of attributes that these categories have individually A _1, ..., A _i, ..., and A _n. The categories to be selected in the accident cause diagnosis are C _I : insulator, C _C :
CV cable, C _N : poultry, C _K : complete ground fault, C _R : resistance ground fault, C _G : gap ground fault. About 400 above
Data is taken, about 300 data are learning data, about 10
0 data was used for testing the learning result. The attributes above categories have individually, A _2: 2-order harmonic distortion, A _3: 3-order harmonic distortion, A _4: 4-order harmonic distortion, A
₅ : 5th harmonic distortion factor, A ₆ : 6th harmonic distortion factor, A ₇ : 7
Next THD, A _8: 8-order harmonic distortion, A _T: the total harmonic distortion.

The range of each attribute value is the maximum value and the minimum value of each attribute value. Table 1 shows the range of each attribute value.

[Table 1]

B) Creation of Decision Tree by Combination of Separated Categories Considering distribution of attribute values of arbitrary two categories in order to identify all categories, there are three cases as shown in FIG. Conceivable. Regarding the relative relationship of C _j viewed from C _{i with} respect to the distribution regarding the attribute A _{K, the} distribution of the attribute value of the state (i) C _i and the distribution of the attribute value of C _j do not overlap. State (ii) The distribution of the attribute values of C _i all overlaps the distribution of the attribute values of C _j . State (iii) The distribution of the attribute values of C _i partially overlaps the distribution of the attribute values of C _j .

Here, only the state (i) can completely distinguish C _i and C _j by the attribute A _K. That is, attribute A
_{In order} for C _i and C _{j to} be completely distinguishable by _K , it is necessary that the distribution state of the attribute values of the two categories is the state (i).

A coefficient b _K as shown in equation (1) is defined by corresponding to a Boolean variable such that C _i and C _j can be completely discriminated by the attribute A _K, and 1 can be completely discriminated. . b _K = 1 state (i), identifiable by attribute A _K b _K = 0 not completely identifiable by attribute A _K (1)

Therefore, by using f (C _i , C _j ), A _K is considered as a Boolean variable, and b _K is used as an attribute that makes C _i and C _j distinguishable. become. Further, it is necessary to newly define the symbol of the Boolean variable of the attribute A _K , but since the attribute to be used for identification can be known from the calculation result, it is not newly defined. f (C _i , C _j ) = b ₁ · A ₁ + ... + b _K · _AK +… + b _n · A _n (2)

For example, if the objects belonging to the categories C ₁ and C ₂ have the attributes A ₁ , A ₂ and A ₃ and the attributes A ₁ and A ₂ are in the state (i), the attribute A ₁ or A ₂ is used. It is clear that C ₁ and C ₂ can be discriminated from each other, but this is f in the equation (2).
(C ₁ , C ₂ ) = 1 · A ₁ + 1 · A ₂ + 0 · A ₃ = A ₁ + A ₂
Becomes [Or] corresponds to the logical sum. That is, (2)
In the formula, C _i and C _j are discriminable when f (C _i , C _j ) = 1, and if at least one attribute of the term of f (C _i , C _j ) is used, C _i and C _j are Fully identifiable.

The attribute for discriminating any two categories can be obtained from the equation (2). For example, the attribute that identifies C _I and C _C can be obtained by the following equation. f (C _I , C _C ) = A ₂ + A ₃ + A ₄ + A ₈ (3)

A set of attribute sets that can identify a combination of categories in which at least one or more distributions of attribute values are completely separated is a set of attributes for all combinations for which f (C _i , C _j ) = 1. f (C _i , C _j ) (i = 1, ..., N, j =
It can be obtained by taking the logical product E of 1, ..., M, i ≠ j). E = Πf (C _i , C _j ) where iε {1, ..., n}, jε {1, ..., m}, i ≠ j (4)

The calculation result of E = Πf (C _i , C _j ) can be expressed in the product-sum form, and if one term expressed in the product form is AS _x (set of attributes), it can be expressed as follows. E = AS ₁ + ... + AS _x + ... + AS _p However, AS _x = A _a A _b A _c (5)

Therefore, each item of AS ₁ , ..., AS _x , ..., AS _p can identify a combination of categories in which the distribution of attribute values of at least one attribute A _K is completely separated. It is an attribute set.

By the equation (5), a set of attribute sets can be selected which can identify all combinations of categories in which at least one attribute value distribution is completely separated. E = A _{2 AT} + A ₃ A _{4 AT} (6) Replace it with:

AS ₁ = A _{2 AT} , AS ₂ = A ₃ A _{4 AT} (7)

The two sets of attributes obtained, AS ₁ and AS ₂
By using the attribute of, the category can be identified by the combination of separated categories. However, insulators and poultry, insulators and gaps, and CV cables cannot be completely identified.

C) Arrangement of Attribute at Each Node of Identification Tree An arbitrary set is selected from the two sets of attributes obtained by the equation (7). Here, AS ₆ selects A _{2 AT} . The arrangement of the identification tree at each node is as follows.

When the set of attributes obtained by the equation (7) has two or more attributes, an arbitrary attribute is placed in the upper node. Place _AT attributes. When there are two or more sets of attributes obtained by equation (7), depending on the state of attribute overlap,
It is divided into an area that does not overlap with the attribute distribution and an area that overlaps with the attribute distribution. When the attributes have values in these non-overlapping regions, the root node completes the classification.

Areas with overlap cannot be classified between categories, and are classified again with other attributes. In that case, a set of attributes necessary for identifying the category of the overlapping area is recursively obtained by the expressions (2) and (4). Among them, an arbitrary attribute is selected from those falling within the set of attribute sets obtained by the expression (7). Performing these processes to re-identify the node is eliminated between the category C _j in the category C _i and state (i).

D) Category identification having no separated attribute C _i and C _j cannot acquire attribute A _K whose distribution state of attribute values of two categories is state (i), and which attribute A
_{Also for K} , if the distribution of the attribute values of the two categories C _i and C _j is state (ii) or state (iii), that is, f
The case of (C _i , C _j ) = 0 can be considered. If such a category exists, C _i and C _j cannot be identified. Hereinafter, a method when the distribution of attribute values of any two categories is the state (ii) or the state (iii) will be described. Insulators and poultry that cannot be specifically identified require category classification.

(1) Division of categories S distributions of certain attribute values A _K with certain s categories C
_{When 1} , ..., C _i , ..., C _s partially overlap, that is, all combinations of s categories are shown in FIG.
Even in the state (ii) or the state (iii), it is considered that there is a range of values that allows the category to be identified. Since partial identification is possible by using these, the categories are divided by the following method.

Arbitrary category C for some attribute A _K
i and all other categories do not overlap, any category C _i overlaps any other one category, any category C _i overlaps any other two categories, ... ., A portion where any category C _i and other arbitrary s−2 categories overlap, and a portion where any category C _i and other arbitrary s−1 categories overlap. By the above division, a new divided category can be created. Also, the number of combinations of portions in which an arbitrary category C _i and other arbitrary s−n categories overlap is given by _s C _{s-n + 1} . If the divided category is an empty set for all attributes A _K , no new category is created.

Specifically, consider the case where the categories C ₁ , C ₂ and C ₃ having three child nodes cannot be distinguished in FIG. The attributes here are A ₁ and A ₂ .

A category newly created by the non-overlapping part of the category C _i and all other categories is referred to as C _{i *} . For example, the distribution of attribute values may be separated like C _{i *} created for the attribute A _{1 in} FIG. _Let C _{ij *} be a category newly created by a portion where an arbitrary category C _i and another arbitrary category C _j overlap. For example, C _{13 *} created for attribute A ₁ in FIG. The newly created C _{13 *} means category C ₁ or category C ₃ . Hereinafter, a newly created category is similarly defined from a portion where an arbitrary category C _i and other arbitrary two categories overlap. Figure 5
As for the attribute A ₁ of C _{2 *} and C _{3 *} , a new category is not created because it is an empty set. At this time, since the newly created category satisfies the state (i) in all arbitrary two combinations, the category can be divided by the above method using the attribute A _K.

Insulators and poultry that cannot be distinguished require category separation. Any category C for attribute A _K
i can be divided into categories that do not overlap with all other categories. In the case of insulators and poultry, categories of C _{I *} and C _{N *} can be created. Similarly, a category C _{IN * of a} portion where an arbitrary category C _i and another arbitrary category overlap can be created.

(2) Category identification having no separated attribute A new category is generated by dividing a category into a set of categories whose attribute value distribution is not completely separated. Which attribute is used to generate a new category is determined by the probability distribution of the attribute.

The probability distribution of attributes can be expressed as follows. _If the random variable represented by the attribute A _K is Z and the probability at the attribute value z _K is p _i , the probability distribution of the attribute A _K can be set as P (Z = z _K ) = p _i (8), The probability in the range of a ≦ Z ≦ b in the distribution of attribute values of an arbitrary attribute A _K is

[Equation 1] The probability in the range of a ≦ Z ≦ b can be obtained from the above equation. Using the probabilities obtained above, attributes effective for identifying the child node are selected.

In the distribution of attribute values of arbitrary categories C _i and C _j , it is possible to identify with high accuracy which category an attribute value having a high probability of a portion that does not overlap with another distribution belongs to. Therefore, a category C _{i with} a certain attribute A _K
The probability distribution of the attribute values of the region that does not overlap with C _j of is calculated, and the probability is defined as p (DJ _AK (C _i , C _j )). This indicates whether A _K is the effect of extent C _j with respect to the identification of C _i.

The probability p (DJ _AK (C _i , C _j )) is used to determine the following evaluation function. The child node is identified using the attribute with the highest evaluation value.

[Equation 2]

Next, it is necessary to consider the probability distribution of attributes, but here, in order to simplify the calculation, it is assumed that the probability distribution of attribute values is uniformly distributed between the maximum value and the minimum value. . Next, the evaluation value F ^* (A _K ) is calculated and the child node is identified. It is calculated using equation (10). As a result, the maximum F ^* (A _k ) can be used for identification. 6 and 6 show the learning result by the decision tree.

8 to 14 are flowcharts of the accident cause diagnosis made using the results of the discrimination tree learning using FIGS. 6 and 7.

As a result of using about 100 data as a test, 7.53% could not be diagnosed, and more specifically, it entered "ERROR" in FIGS. 8 to 14, and in some cases, it could be limited to one. 0.54% was 21.51% when it could be limited to two, and 20.43% when it could be limited to three.

Next, how to specifically combine neural networks will be described. The neural network is described in detail in "Neural Network Information Processing" by Hideki Aso, published by Sangyo Tosho. The neural network used includes back propagation, Boltzmann machine, perceptron, and the like. Here, an example using back propagation will be described. Of course, the input is the attribute value,
The number of inputs is the type of attribute, the number of outputs is the category, and the number of outputs is the number of categories to identify.

Regarding which stage to combine, there are the following three stages and their combinations. First, Japanese Patent Application No. 4-130
In method No. 083, insulator and poultry, insulator and gap and C
It has been stated that V-cables cannot be fully identified. Therefore, when learning with a neural network using all of the above data for the four categories of insulator, poultry, gap, and CV cable, all attributes may be used and any attributes may be used. ,
Alternatively, the attribute having the highest F ^* (A _k ) may be used. This corresponds to the second method of the solution of the present invention.

The results of this method are shown below. The complete ground fault and the resistance ground fault can be identified by machine learning. When we performed about 60 data, about 40 data showed the correct answer, about 2
0 data gave the wrong answer.

2) In this machine learning, two category combinations that do not overlap can be identified by using the attributes of A _T and A ₂ . Further, the overlapping portion is identified by the attribute that is probabilistically advantageous with respect to the category set having no separate attribute of (2). It is a method to identify it with a neural network.

In other words, in the flow chart, category 1) 104, 105, 106 bird meat, insulator 2) 112, 113, 114 bird meat, insulator 3) 119, 120, 121 bird meat, insulator 4) 127, 128, 129 insulator, gap 5) 134,135,136 insulator, gap 6) 138,139,140, insulator, gap, CV cable 141,142 7) 144,145,146 CV cable, insulator 8) 151,152,153 CV Cable, insulator

Learning is performed using a neural network for each of the above eight sets.

Specifically, for example, with regard to 5), the data to be learned for the two categories "insulators and gaps" using the data falling to 134, 135, 136 among the learning data and the data to be tested are also 134,135. In the case of the data falling in the flow chart of No. 136 of FIG. As a result, about 10% gave a wrong answer, but it was possible to identify the category by limiting to one. This corresponds to the third method of solution.

3) As a third method, there is a method in which machine learning is stopped at an upper node of the method of 2) and then learning is performed by a neural network.

Specifically, in the flow chart, 1) 103, 104, 105, 106, 107, 108 bird meat, insulator 2) 111, 112, 113, 114, 115, 116 bird meat, insulator 3) 118, 119,120,121,122,123 Bird meat, insulator 4) 126,127,128,129,130,131 insulator, gap 5) 133,134,135,136,137,138, insulator, CV cave 139,140 , 141, 142, 143, 144, 145, 146, 147, 148 6) 150, 151, 152, 153, 154, 155 insulators, and the above-mentioned 6 sets of CV caves are learned by using neural networks. The result was similar to that of the second method. This corresponds to the fourth method of the present invention.

4) The fourth method is a method in which the second method and the third method are combined, and for example, the third set 5) is identified by the second method. Ie 2
5), 6), 7), that is, 5) 134, 135, 136 6) 138, 139, 140, 141, 142 7) 144, 145, 146. In this method, the second and third
Better results than th. This corresponds to the fifth method of the present invention.

[0056]

As described above, the present invention has the following effects.

The combination of inductive learning and neural networks increases the data recognition rate.

Even if a neural network gives a wrong answer, it is possible to limit which category is wrong between the categories by inductive learning.

Even in the part where the attribute values overlap, the category can be further limited by using the feature of non-linearly identifying the attributes of the multidimensional space of the neural network.

When the attribute value of data has a distribution, it can be applied to various classifications such as diagnosis, pattern recognition, and image processing.

When the distribution of attribute values is obtained by a simulator or the like, even if the parameters of the simulator are changed,
By learning the classification of data by machine learning according to the change, it is possible to create the data quickly.

An algorithm that does not include human subjectivity can be automatically created.

An efficient algorithm can be created.

[Brief description of drawings]

FIG. 1 is an electrical system diagram showing an example of a test circuit for implementing an accident cause diagnosis method related to data classification.

FIG. 2 is a block diagram showing a configuration of a measurement system in an example of the present invention.

FIG. 3 is a graph showing a typical waveform of zero-phase current at the time of a failure and an FFT analysis result.

FIG. 4 is an explanatory diagram showing an attribute value distribution relationship between two categories.

FIG. 5 is an explanatory diagram showing division of categories according to the present invention.

FIG. 6 is an explanatory diagram showing a result of discrimination tree learning according to the present invention.

FIG. 7 is an explanatory diagram showing a result of discrimination tree learning according to the present invention.

FIG. 8 is a flowchart (1) of accident cause diagnosis according to the present invention.

FIG. 9 is a flowchart (2) of accident cause diagnosis according to the present invention.

FIG. 10 is a flowchart (3) of accident cause diagnosis according to the present invention.

FIG. 11 is a flowchart (4) of accident cause diagnosis according to the present invention.

FIG. 12 is a flowchart (5) of accident cause diagnosis according to the present invention.

FIG. 13 is a flowchart (6) of accident cause diagnosis according to the present invention.

FIG. 14 is a flowchart (7) of accident cause diagnosis according to the present invention.

[Explanation of symbols]

1 circuit breaker, 2 ZPD (zero phase voltage detector), 3 power supply side capacitor, 4 zero phase current transformer, 5 load side capacitor, 6 accident switch, 7 high voltage switch, 8 controller, 9 transformer

Claims (5)

[Claims] 1. A data classification method, characterized in that learning is performed by inductive machine learning, and when a category cannot be discriminated by the learning, learning is performed by a neural network. 2. A method for classifying data, characterized in that a category except a completely discriminable category is learned by a neural network by an inductive machine learning method. 3. An attribute that identifies all combinations of categories in which at least one or more attribute value distributions are separated by inductive machine learning is learned using machine learning and cannot be performed by that machine learning. A method of classifying data, which is characterized by learning with a neural network for categories. 4. The learning is performed by machine learning at a higher node of the machine learning performed by the classification method according to claim 3, and a category which cannot be learned by the machine learning is learned by a neural network. Classification method. 5. A method for classifying data, characterized in that the classification method according to claim 3 and the classification method according to claim 4 are combined to perform learning by machine learning and a neural network.