JPH076149A - Learning recognizing device - Google Patents

Abstract

PURPOSE:To provide a learning recognizing device capable of improving the recognition ratio of a pattern situated on the boundary of category groups and efficiently executing additional learning. CONSTITUTION:In the learning recognizing device consisting of a rough classification part 2 for classifying an input pattern into plural category groups and calculating the degrees of attribution of respective groups and plural detailed classification parts 1 for calculating similarity in respective category groups, the weight changing quantity of each detailed classification part 1 calculated by a learning control part 5 is weighted for every detailed classification by using the degrees of group attribution by a learning control signal loading part 6 to execute co-operative learning. Namely respective detailed classification sorting parts 1 are co-operatively learned by using the degree of group attribution to the input pattern, so that the recognition ratio is improved and additional learning can efficiently be executed.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a learning / recognition device for learning and recognizing input pattern data.

[0002]

2. Description of the Related Art Conventionally, input pattern data is classified into large groups by first selecting a category group to which the input data belongs, and then finely classifying the selected category group. As an example of a pattern recognition device for recognizing a pattern, for example, it is shown in the Institute of Electronics, Information and Communication Engineers, D-II Vol.J75-D-II No.3 pp545-553 "Large-scale neural network" CombNET-II "". ing.

FIG. 25 shows a block diagram of this conventional pattern recognition apparatus. Reference numeral 101 denotes a large classification unit, which roughly classifies an input pattern signal into each category group. Ten
2 is a sub-classification unit, which sub-classifies the input pattern signal within each category group. Reference numeral 103 denotes a group selection unit that selects a plurality of category groups from the output value of the large classification unit 101 (hereinafter referred to as the goodness of fit). A sub-classification unit input signal selection unit 104 receives the input pattern signal based on the group selection information obtained by the group selection unit 103.
The choice is 2. 105 is an identification unit, a group selection unit
The input pattern signal is identified based on the goodness of fit of the category group selected in 103 and the output value of the fine classification unit 102.

In the large classification unit 101, 106 is an input unit for inputting an input pattern signal. Reference numeral 107 denotes a multi-input / single-output signal processing unit, which calculates the fitness of each category group with respect to the input pattern signal.

In the subclassification section 102, reference numeral 108 is an input section for inputting the input pattern signal output from the subclassification section input signal selection section 104. Reference numeral 109 denotes a multi-input / single-output signal processing unit, which is connected to the lower input unit 10
8 or the output of the multi-input / single-output signal processing unit 109 and the weighting factor, which is the degree of connection thereof, are multiplied and summed to perform threshold processing and output. Here, these multiple input / output signal processing units have a layered structure and are connected to the network so that the signals propagate only to the upper layer without mutual coupling in each layer, and thus the category group for the input pattern signal is obtained. The degree of similarity to each category in is required. Reference numeral 110 denotes a maximum value selection unit, which selects the maximum value from the outputs of the plurality of multi-input / output signal processing units in the uppermost layer.

In the identification unit 105, 111 is a similarity calculation unit, and the similarity of each category is calculated from the goodness of fit of the category group selected by the group selection unit 103 and the output value of the subclassification unit 102 corresponding to the category group. It calculates the degree. A category identification unit 112 identifies the input pattern signal by obtaining the maximum similarity value of each category obtained from the similarity calculation unit 111.

The operation of the conventional pattern recognition apparatus configured as described above will be described below. Input pattern signal X consisting of n feature data of the recognition object

[0008]

_First , X = (x ₁ , x ₂ , ..., X _n ) (1) is input to the input unit 106 of the large classification unit 101. Input section
There are n pieces of data 106, which are equal in number to the feature data of the pattern data, and each feature data x _i is input to the corresponding input unit 106. Each multi-input one-output signal processing unit 107 of the large classification unit 101 has the input x _{j of the} input unit 101 connected to it.
And the weighting coefficient v _ij (1 ≦ i ≦ m _r ;
m _r is the number of category groups, 1 ≤ j ≤ n) is multiplied, and then the sum is calculated, which is then calculated as the input pattern signal X and the weighting coefficient vector V _i (number) of each multi-input one-output signal processing unit 107. 2)

[0009]

## EQU2 ## V _i = (v _i1 , v _i2 , ..., V _in ) (2) Output by dividing by the product of norms | X | and | V _i |. That is, the output value sim (X, V _i) of the multi-input first output signal processing section 107 with the weighting coefficient vector V _i is (number 3)

[0010]

## EQU3 ## sim (X, V _i ) = (X · V _i ) / (| X || V _i |) (3) where X · V _i = Σ _j (x _j · v _ij ) | X | = (Σx _j ² ) ^1/2 | V _i | = (Σv _ij ² ) ^1/2 . Here, Σ _j represents the total sum for j.

As for the weighting coefficient vector V _i ,
It is designed in advance so that the predetermined multi-input / single-output signal processing unit 107 generates the maximum output for similar input pattern signals.

According to the conventional example, these weighting coefficient vectors V _i are designed by the following method. First, in the first step, to each input of the input pattern signal X for weighting coefficient vector design, most sim (X, V _i) determined a large V _C (in this case, X is that optimally matched to V _C. ), V _C
To approach X. Further, when the number of input pattern signals optimally matching one weighting coefficient vector exceeds a certain number, the area covered by the weighting coefficient vector is divided into two. In the second step, the optimum matching V _i is calculated for all the input pattern signals for designing the weighting coefficient vector, and it is checked whether or not it changes from the previous time. Then, if there is a change, the V _i is corrected. At this time, as in the first step, the weighting coefficient vector is also divided. This is repeated until the weight coefficient vector is corrected and there is no division.

By designing the weighting coefficient vector in this manner, the input pattern signal is roughly classified into a plurality of category groups. Then, the output value of each multi-input / one-output signal processing unit 107 is output to the group selection unit 103 as the suitability of each category group for the input pattern signal X.

The group selection unit 103 selects an arbitrary number of category groups in descending order of goodness of fit obtained by the large classification unit 101, and outputs group selection information indicating which category group has been selected and the goodness of fit corresponding thereto. To do.

Based on the group selection information obtained from the group selection unit 103, the subclassification input signal selection unit 104 selects the subclassification unit 102 to which the input pattern signal is input, and selects the input pattern signal from these subclassification units 102. Output to.

In each of the subclassification units 102 (that is, the subclassification unit to which the input pattern signal is input from the subclassification input signal selection unit 104) corresponding to the category group selected by the group selection unit 103, first, the input unit The input pattern signal X is input to 108. The number of input units 108 is equal to the number of characteristic data of pattern signals, and N pieces of the characteristic data x _i are input to the corresponding input units 108. Each multi-input / output signal processing unit 109 of the subclassification unit 102 multiplies the output of the lower-layer input unit 108 or the multi-input / output signal processing unit 109 connected to it by the weighting factor which is the degree of connection thereof. After converting the sum of all things with a threshold function, the value is output to the upper layer. Here, the uppermost multi-input / single-output signal processing section 109 of each subclassification section 102 is set to the same number as the number of categories of pattern data included in each category group, and each multi-input / one output of the uppermost layer. The signal processing unit 109 corresponds to each of these categories. The maximum value selection unit 80 selects the maximum value among the output values of the multi-input one-output signal processing unit 109 of the highest layer,
The category corresponding to the multi-input single-output signal processing unit 109 and its maximum output value are output.

The weighting factor of each multi-input one-output signal processing unit 109 is such that for an input pattern signal having each category in the category group, the multi-input one-output signal processing of the uppermost layer corresponding to each category. The unit 109 is previously learned to generate the maximum output.

Specifically, such a weighting coefficient learning method is performed by a learning algorithm called an error back propagation method. The error backpropagation method is described, for example, in Nature 323 (1986), pages 533 to 536. (DE
Rumelhart, GEHintonand RJWilliams "Learning
Representations by Back-Propagating Errors ", Natu
re, vol.323. pp.533-536, Oct. 9, 1986).

The outline of the error back propagation method will be described below.

First, the pattern signal X for weighting coefficient learning is input to the input unit 108 of the subclassification unit 102. As described above, each multi-input one-output signal processing unit 109 has a lower-layer input unit 108 connected to it, or an output of the multi-input one-output signal processing unit 109 and a weighting factor that is the degree of connection thereof. After transforming the sum of those multiplied by the threshold function,
The value is output to the upper layer. Here, the error E between the output signal o _{k of} all the multi-input one-output signal processing units 109 of the uppermost layer and the desired output signal t _k (this is called a teacher signal) is obtained as in (Equation 4). .

[0021]

Equation 4] _{E = 0.5Σ p Σ k (t} k -o k) 2 (4) However, sigma _p is the total sum related to the number of patterns of the teacher signal. The purpose of learning is to determine the value of the weighting coefficient that minimizes the error E.
The change amount Δw _ij of the weighting factor of is calculated based on ( _Equation 5).

[0022]

[Expression 5] Δw _ij = −ε∂E / ∂w _ij (5) where ε is a positive constant called a learning rate. The error E can be reduced by repeating such updating of the weighting coefficient based on (Equation 5) each time a pattern signal for learning is input. When the error E becomes sufficiently small, the learning is terminated assuming that the output signal has become sufficiently close to the desired value.

By such a weighting coefficient learning method,
With respect to the input pattern signal having each category in the category group, the highest input multi-input single-output signal processing unit 109 corresponding to each category can generate the maximum output. Therefore, among the plurality of multi-input one-output signal processing units 109 of the uppermost layer, the one that produces the maximum output is selected by the maximum value selection unit 80, so that each category group, that is, each sub-classification unit. In, the category of the input pattern signal can be recognized.

In the identification unit 105, first, in the similarity calculation unit 111, from the goodness of fit of the category group selected by the group selection unit 103 and the output value of the fine classification unit 102 corresponding to the category group (Equation 6) The similarity of each category obtained by the subclassification unit 102 is calculated using the number, and these similarities are calculated by the category identification unit.
Output to 112.

[0025]

(Similarity) = (Fitness) ^a (Output value) ^b (6) where a and b are real constants.

Finally, the category identifying section 112 compares the similarities of the categories obtained from the similarity calculating section 111 and outputs the category corresponding to the maximum similarity among them as the identification result.

[0027]

However, in the conventional recognition apparatus as described above, the learning of each subclassification unit does not consider the degree to which the input pattern signal belongs to each category group, that is, the group membership degree, and the single classification Since it is performed independently in the sub-classification unit, the recognition accuracy for the input pattern located at the boundary between the category groups deteriorates.

Further, when an unlearned pattern is erroneously recognized and the pattern is learned again (additional learning), the category of the pattern belongs to the subclassification unit determined to have the highest group membership by the large classification unit. However, since it is not possible to determine which subclassification part should be learned, there is a problem that additional learning cannot be performed.

In order to solve the problems of the conventional recognition apparatus, the present invention adds a new category to the subclassification section as needed, and learns each subclassification section collaboratively by using the group membership to the input pattern. It is an object of the present invention to provide a learning and recognition device which can improve the recognition rate of a pattern located at the boundary of a category group and can efficiently perform additional learning.

[0030]

In order to achieve the above object, the present invention provides means for calculating a group membership degree which is a degree to which an input pattern signal belongs to a category group consisting of a set of similar patterns. A large classification unit provided for a plurality of category groups; an intra-group similarity calculation unit provided with means for calculating an intra-group similarity which is a degree to which an input pattern signal belongs to a category included in each category group; Comparing the weighted intra-group similarity signal output from the discrimination signal weighting part, which consists of multiple multipliers that calculate the product of the intra-similarity and group membership, and weights the intra-group similarity Within the group based on the category discriminator performing the training, the teacher signal generator that generates the teacher signal necessary for learning, the teacher signal output from the teacher signal generator, the output of the discrimination signal weighting unit, and the output of the category discriminator. Degree of similarity A learning control unit that controls the amount of weight change of the output unit, and a plurality of multipliers that calculate the product of the learning control signal and the group membership degree output from the large classification unit, and the weighting of the learning control signal output from the learning control unit A learning control signal weighting unit that performs the following, a weighting coefficient updating unit that updates the weighting factor of the intra-group similarity calculation unit based on the outputs of the learning control signal weighting unit and the intra-group similarity calculation unit, and intra-group similarity calculation It is a learning and recognition device configured by a plurality of sub-classification units including parts.

[0031]

According to the present invention, since the learning control signal reflecting the progress of learning is weighted by the group membership degree output by the large classification unit in the above-described configuration, the weight of the fine classification unit is corrected. The fine classification unit learns in cooperation, and the recognition rate of patterns located at the boundaries of category groups can be improved.

Further, since the category adding unit adds a new category to the subclassifying unit as needed, additional learning can be efficiently performed.

Furthermore, by changing the method of calculating the degree of group membership according to the transition of learning, it is possible to calculate the appropriate degree of group membership in each of the initial learning and the additional learning, thereby enabling effective learning.

[0034]

Embodiments of the present invention will be described below with reference to the drawings.

Generally, the pattern signal input to the learning and recognition apparatus includes a time-series pattern such as voice, a spatial pattern such as characters and images, etc., but any pattern signal may be used in the present invention.

FIG. 1 is a block diagram of the first embodiment of the learning and recognizing apparatus of the present invention. In FIG. 1, 1 is a sub-classification unit, which is an intra-group similarity calculation unit 1a for calculating intra-group similarity, which is a degree to which an input pattern signal is similar to each category included in the category group, and the intra-group similarity. The weighting coefficient updating unit 1b updates the weighting factor of the intra-group similarity calculating unit 1a according to the output of the degree calculating unit 1a and the output of the learning control signal weighting unit 6 described later. A large classification unit 2 calculates a group membership degree, which is the degree to which an input pattern signal belongs to each category group for a category group having similar patterns. Reference numeral 2a is a group dictionary in which a plurality of reference pattern signals representing the category group are stored. Reference numeral 2b is a group reference pattern similarity calculation unit that calculates a group membership degree using the group reference pattern signal from the group dictionary 2a, and 7 is an identification signal weighting unit, which is included in the group obtained by the subclassification unit 1. The degree of similarity is weighted by the degree of group membership calculated by the large classification unit 2. In the identification signal weighting unit 7, 7
Reference numeral a denotes a multiplier, which multiplies the group membership degree calculated by the large classification unit 2 and the in-group similarity obtained from the fine classification unit 1. A category identification unit 3 selects a plurality of output values of the multiplier 7a having a large value for each category, calculates the sum of the output values (category similarity), and then selects the maximum value from the category similarity. By determining, the category of the input pattern signal is identified. Reference numeral 4 is a teacher signal generator. Reference numeral 5 is a learning control unit, and 5b is a comparator, which compares the output of the teacher signal generation unit 4 with the output of the category identification unit 3, and 5a is a weight change amount control unit, which shows the comparison result of the comparator 5b. As the instruction signal for performing the learning based on the output, 1 is output to the subclassification section in which the two do not match and the category of the input pattern signal exists, and 0 is output in the case of match. A learning control signal weighting unit 6 weights the instruction signal output from the learning calculation unit 5 by the degree of group membership.

FIG. 2 is a block diagram showing an embodiment of the structure of the large classification unit 2. 2a is a group dictionary for storing a plurality of group reference pattern signals representing a category group consisting of a set of similar patterns, and 2b is a group reference for calculating a group membership degree using a group reference pattern signal from the group dictionary. The pattern similarity calculation unit 21 is an input unit, which inputs an input pattern signal. 22 is a distance calculator, which is a group dictionary 2
The distance between all the group reference pattern signals of a and the input pattern signal is calculated. Reference numeral 23 is a divider that calculates the reciprocal of the output of the distance calculation unit 22. Reference numeral 24 denotes an adder, which calculates the total sum of the outputs of the respective dividers 23. 25 is a multiplier, and the output of the adder 24 and the distance calculation unit
22 outputs are multiplied. 26 is a divider for calculating the reciprocal of the output of the multiplier 25.

FIG. 3 is a block diagram showing a first embodiment of the similarity calculation section 1a of the subclassification section 1. Reference numeral 31 is an input unit for inputting an input pattern signal. Reference numeral 32 denotes a category dictionary, which stores a plurality of category-reference pattern signals representing representative values of each category of the input pattern signal. A distance calculator 33 calculates the distances between all the pattern signals for category reference of the category dictionary 32 and the input pattern signals. A divider 34 calculates the reciprocal of the output of the distance calculator 33.
Reference numeral 35 is an adder, which calculates the total sum of the outputs of the respective dividers 34. 36 is a multiplier, which multiplies the output of the adder 35 and the output of the distance calculation unit 33. A divider 37 calculates the reciprocal of the output of the multiplier 36.

The operation of the learning and recognition apparatus configured as described above will be described below. Input pattern signal X (Equation 7) consisting of n feature data of the recognition target

[0040]

## EQU7 ## X = (x ₁ , x ₂ , ..., X _n ) (7) Input to the large classification unit 2. In the large classification unit 2, the input unit 21 outputs the input pattern signal X to the r distance calculation units 22. Each distance calculation unit 22 has a group reference pattern signal V _i (1 ≦ i) representing each category group stored in the group dictionary 2 a.
≤r; r is the number of pattern signals for group reference, that is, the number of category groups), and the distance d between X and V _i shown in (Equation 8)
_i is calculated and output to the corresponding divider 23 and multiplier 25, respectively. However, f is a real number that satisfies f> 1.

[0041]

D _i = ‖X−V _i ‖ ^{2 / (f-1)} (1 ≦ i ≦ r) (8) Each divider 23 calculates the reciprocal of the distance d _i and outputs the output. Output to adder 24. The adder 24 calculates the total sum of the outputs of all the dividers 23 and outputs the output to the r multipliers 25. Each of the multipliers 25 multiplies the outputs of the corresponding distance calculator 22 and the adder 24, and inputs the output to the corresponding divider 26. Each divider 26 calculates the reciprocal of the output of the corresponding multiplier 25. Finally, in the large classification unit 2, the output of each divider 26 is output as the group membership degree μ _i (1 ≦ i ≦ r) of each category group with respect to the input pattern signal X. That is, the group membership degree μ _i (1 ≦ i ≦ r) of each category group can be expressed as in (Equation 9).

[0042]

Equation 9] _{μ i = 1 / Σ k =} 1 r (d i / d k) (9) where, sigma _{k = 1} ^r represents the sum from 1 for k to r.

The group-reference pattern signal representing each category group stored in the group dictionary 2a is previously stored in a conventional clustering method, for example, “Pattern Information Processing” (Corona, edited by The Institute of Electronics, Information and Communication Engineers, Masaru Nagao). K-means algorithm and Isodata algorithm shown in "Company" and "An by Y. Linde, A. Buzo, and RMGray.
Algorithm for Vector Quantizer design, "IEEE Tran
s. Commun., COM-28,1, pp.84-95, Jan.1980.

A method of designing the group dictionary 2a using the K-means algorithm will be briefly described below. 1. From the set of pattern signals for designing the group dictionary of the recognition object, r (where r is the predetermined number of category groups) pattern signals are appropriately selected, and these r group reference pattern signals V _{i are selected.} (1 ≦ i ≦ r). 2. For each pattern signal X for designing the group dictionary, the distance d _i shown in (Equation 10) is obtained.

[0045]

Equation 10] Request d _i = ‖X-V _i ‖ V _i to the (10) to a minimum. At this time, X is assumed to belong to the category group S _i (1 ≦ i ≦ r). 3. The average value of the pattern signals X belonging to each S _i is obtained, and this is designated as V _i ′. 4. If V _i ′ = V _i holds for all i, the group reference pattern signal V _i at this time is stored in the group dictionary 2a. Otherwise, let V _i 'be a new group reference pattern signal V _i . Return to. In this way, by designing the group reference pattern signal, all pattern signals can be divided into some similar subsets (category groups) of pattern signals. The Isodata algorithm and LBG algorithm are basically the same as the K-means algorithm.

The similarity calculation unit 1a of each subclassification unit 1 is an input unit.
31 outputs the input pattern signal X to the c distance calculation units 33. Each distance calculation unit 33 has a category reference pattern signal (weighting coefficient) W _j (1 ≦ j ≦ c; c is the number of category reference pattern signals) indicating a representative value of each category stored in the category dictionary 32. ) Is read, the distance d _j between X and W _j shown in (Equation 11) is calculated, and output to the corresponding divider 34 and multiplier 36, respectively.

[0047]

D _j = ‖X−W _j ‖ ^{2 / (f-1)} (1 ≦ j ≦ c) (11) where f is a real number satisfying f> 1.

Each divider 34 calculates the reciprocal of the distance d _j and outputs the output to the adder 35. The adder 35 calculates the sum of the outputs of all the dividers 34, and outputs the output to the c multipliers 36. Each of the multipliers 36 multiplies the output of the corresponding distance calculation unit 33 and the output of the adder 35, and inputs the output to the corresponding divider 37. Each divider 37 calculates the reciprocal of the output of the corresponding multiplier 36. Finally, in the fine classification unit 1, the output of each divider 37 is the in-group similarity vector (ν ₁ , ν ₂ , ..., Of each category for the input pattern signal X).
ν _c ), and outputs it to the identification signal weighting unit 7. That is,
The in-group similarity of each category in the subclassification unit 1 is (equation 1
It is expressed as 2).

[0049]

Ν _j = 1 / Σ _{k = 1} ^c (d _j / d _k ) (12) where Σ _{k = 1} ^c represents the sum of 1 to c for k.

In the identification signal weighting section 7, the group membership degree μ of the i-th category group calculated by the multiplier 7a in the large classification section 2
_i is multiplied by the intra-group similarity vector (ν ₁ , ν ₂ , ..., ν _c ) obtained from the fine classification unit 1 corresponding to the category group, and the outputs are output to the category identification unit 3. That is, the number of multipliers 7a provided is (the number of group reference pattern signals × the total number of category reference pattern signals in each subclassification unit), and the group membership degree μ _p of the category group _p (1 ≦ p ≦
r; r is the number of category groups) and the in-group similarity ν _{pq of} a certain category q obtained from the fine classification unit 1 corresponding to the category group p (1 ≦ q ≦ c; c is the number of categories belonging to the category group) ) Input value ξ _pq of the multiplier 7a is expressed as in ( _Equation 13).

[0051]

In Equation 13] _{ξ pq = μ p · ν pq} (13) category identification unit 3, the output values of all the multipliers 7a,
Categorize each category together and select a plurality of ones having a large output value. Then, for each category, the sum of these selected output values is calculated as the category similarity r _s (1 ≦ s
≦ N _C ; N _C is the number of categories), and the category corresponding to the maximum similarity among them is output as the recognition result. As a method of selecting a plurality of output values of the multiplier 7a for each category, it is possible to select an output value of the multiplier 7a having a certain threshold value or more.

The teacher signal calculator 4 receives the input pattern signal X
In synchronization with, the teacher signal vector T = (t ₁ ,
t ₂ , ..., tN _C ). The learning control unit 5 uses, as the instruction signal E for performing learning based on the result of comparison between the output value of the identification signal weighting unit 7 and the teacher signal, the fine classification unit in which the two do not match and the category of the input pattern signal exists. Is output for 0, and 0 is output for matching.

The learning control signal weighting unit 6 multiplies the instruction signal E _p (instruction signal for the p-th subclassification unit) and the group membership degree by the multiplier 6a as shown in (Equation 14), and the corresponding subclassification unit 6 1 to the weight coefficient updating unit 1b, and the weight coefficient updating unit 1b outputs the category reference pattern signal (weight coefficient) of the group similarity calculating unit 1a to the output of the similarity calculating unit 1a and the learning control signal weighting unit. Is updated from the output of the learning vector quantization method (LVQ) using a method similar to a learning algorithm.

[0054]

E ′ _p = E _p · μ _p (14) where E ′ _p is a weight learning control signal for the p-th subclassification unit, E _p is a learning control signal for the p-th category, μ _p
Represents the group similarity to the subclassification unit p input by the weight learning control signal.

Regarding the learning vector quantization method, for example, "Learning Vector Quantization fo" by T. Kohonen.
r Pattern Recognition ", Helsinki University of Tech
nology, Report TKK-F-A601 (1986.11).

The learning vector quantization method will be briefly described below by taking the p-th subclassification unit as an example. The initial value of the pattern signal W _j for category reference is arbitrarily selected for each category from the pattern signals for designing the category dictionary, which is composed of a set of input pattern signals included in each category group, and the group dictionary 2a. It is prepared in advance by a conventional clustering method such as the K-means algorithm explained at the time of designing, and the following steps are executed for each subclassification section each time the input pattern signal X is input.

1. The category reference pattern signal W _k having the highest similarity is selected from the in-group similarities ν _j , and the category of W _k is _defined as C _k .

2.・ Category of input pattern signal X and C
If k matches, closer to X proportionally to W _k to the output E _'p of the learning control signal loading section 6. Similarly, if they do not match, W _k is moved away from X in proportion to E ′ _p .

The category reference pattern signals other than W _k are not updated. However, the above procedure can be omitted for the subclassification unit having a small degree of group membership.

In this way, the learning recognition device can recognize the input pattern by repeating the learning operation until the identification results for all the input pattern signals X match the category.

As described above, according to the present embodiment, since the weight correction of each subclassification unit is performed based on the learning control signal weighted by the group membership degree output from the large classification unit, the category of the category to which the input pattern signal belongs is corrected. It is possible to improve the recognition rate of the patterns located at the boundary of the category group by learning in cooperation with the existing subclassification units.

FIG. 4 shows a second embodiment of the learning and recognizing device of the present invention, which has a group selecting section and an input signal selecting section. In FIG. 4, a fine classification unit 1, a similarity calculation unit 1a, a weight coefficient updating unit 1b, a large classification unit 2, a group dictionary 2a, a category identification unit 3, a teacher signal generation unit 4, a learning control unit 5, a learning control number weight. The unit 6, the identification signal weighting unit 7, and the multipliers 6a and 7a are
It is similar to that of the first embodiment described above. In the present embodiment, in addition to the above, the group selection unit 8 selects a plurality of category groups based on the group membership degree output by the large classification unit 2. The input signal selection unit 9 selects a subclassification unit to which an input pattern signal is input, based on the selection result of the group selection unit 8.

The operation of the learning / recognition device configured as described above will be described below. The learning recognition operation is the same as that of the first embodiment of the learning recognition device described above, but the group selection unit 8 selects a category group having a large group membership degree and outputs the result to the input signal selection unit 9. Further, the group membership degree is output only to the selected category group to the learning control signal weighting unit 6 and the identification signal weighting unit 7.

As described above, according to this embodiment, the group selection unit 8
Since the subclassification unit is recognized and learned only for the category group selected by, which has a high group membership degree, high-speed and effective learning recognition is possible.

FIG. 5 shows a third embodiment of the learning and recognizing device of the present invention, which further comprises a limited group selection unit 8a and a group membership degree switching unit 11. In FIG. 5, a fine classification unit 1, a similarity calculation unit 1a, a weight coefficient updating unit 1b, a large classification unit 2, a group dictionary 2a, a category identification unit 3, a teacher signal generation unit 4, a learning control unit 5, a learning control number weight. The unit 6, the identification signal weighting unit 7, the multipliers 6a and 7a, the group selection unit 8, and the input signal selection unit 9 are the same as those in the second embodiment described above. In the present embodiment, in addition to the above, the category information storage unit 10 stores the information of the categories belonging to each subclassification unit 1, and the limited group selection unit 8a determines the input pattern signal based on the stored contents of the category information storage unit 10. At least one group membership degree is selected from the group having a higher group membership degree in the subclassification section in which the category exists.
Further, the group membership degree switching unit 11 switches the output of the group selection unit 6 and the output of the limited group selection unit 8a and outputs the learning control signal weighting unit 6.

The operation of the learning / recognition device configured as described above will be described below. The learning recognition operation is similar to that of the second embodiment of the learning recognition device described above, but at the time of initial learning, the group membership degree switching unit 11 outputs the group membership degree of the group selection unit 6 to the learning control signal weighting unit 6. Then, at the time of additional learning, the group membership degree of the limited group selection unit 6 a is switched to be output to the learning control signal weighting unit 6.

As described above, according to this embodiment, when learning an unlearned input pattern signal, the limited group selection unit 6a selectively outputs the group membership degree to the subclassification unit in which the category of the input pattern signal exists. Therefore, additional learning of unlearned patterns becomes possible. Further, since only the fine classification unit related to the input pattern signal is learned, the additional learning hardly affects the learning and recognition apparatus as a whole, and the additional learning which hardly affects the recognition performance for the learned pattern can be performed.

FIG. 6 shows a learning recognition apparatus according to a fourth embodiment of the present invention having a group membership degree weighting unit 12. In FIG. 6, a fine classification unit 1, a similarity calculation unit 1a, a weight coefficient updating unit 1b, a large classification unit 2, a group dictionary 2a, a category identification unit 3, a teacher signal generation unit 4, a learning control unit 5, a learning control number weight. Part 6,
The identification signal weighting unit 7, the multipliers 6a and 7a, and the category information storage unit 10 are the same as those in the third embodiment described above.
In the present embodiment, in addition to the above, the group membership degree weighting unit 12 weights the output of the large classification unit 2 based on the stored contents of the category information storage unit 10.

FIG. 7 is a block diagram showing an embodiment of the group membership degree loading unit 12.

Reference numeral 91 is a weighting section for weighting the output of the large classification section 2. A load adjusting unit 92 selects a category group in which the category of the input pattern exists based on the information of the categories belonging to each subclassifying unit stored in the category information storage unit 10 and increases the degree of group membership by selecting the category group. Load section
The load of 92 is adjusted.

The operation of the learning / recognition device configured as described above will be described below. The learning recognition operation is the same as that of the third embodiment of the learning recognition device described above, but the group membership degree weighting unit 12 uses the stored contents of the category information storage unit 10 to perform the fine classification in which the categories of the input patterns exist. The degree of group membership is weighted while adjusting the load so that the degree of group membership of the part becomes larger.

As described above, according to this embodiment, the group membership degree weighting unit 12 increases the group membership degree for the subclassification section in which the category of the input pattern signal exists, so that the subclassification section associated with the input pattern signal is obtained. Learning is emphasized, and effective additional learning of unlearned patterns becomes possible.

FIG. 8 shows a learning recognition apparatus according to a fifth embodiment of the present invention, which has a category adding unit 13. In FIG. 8, a fine classification unit 1, a similarity calculation unit 1a, a weight coefficient updating unit 1b, a large classification unit 2, a group dictionary 2a, a category identification unit 3, a teacher signal generation unit 4, a learning control unit 5, a learning control signal weight. The section 6, the identification signal weighting section 7, the multipliers 6a and 7a, and the category information storage section 10 are the same as those of the third embodiment described above. In this embodiment, in addition to the above, the category addition unit 13
However, a means for calculating the in-group similarity for a new category is added to the subclassification unit 1, and the category information storage unit 10
Stores category information belonging to each subclassification unit 1.

The operation of the learning and recognition apparatus configured as described above will be described below. Here, the subclassification unit 1
As the similarity calculation unit 1a, the configuration shown in FIG. 3 will be described as an example. The recognition operation is the same as in the first embodiment of the learning recognition device described above. The learning operation is the same as in the first embodiment of the learning recognition device described above, but the category adding unit 13 compares the category information in the category information storage unit 10 with the teacher signal output from the teacher signal generating unit 4, When the category to which the input pattern signal to be learned belongs does not belong to the subclassification unit that performs learning, means for newly calculating the in-group similarity for the category to which the input pattern signal belongs is added to the subclassification unit 1. Explaining with reference to FIG. 3, the currently input input pattern signal is added as a category reference pattern to the category dictionary 32 of the intra-group similarity calculation unit 1a included in the target sub-classification unit 1, and the added category is added. A distance calculator 33, a divider 34, an adder 35, a multiplier 36, and a divider 37 are newly provided. At the same time, new category information is added to the category information storage unit 1
Store at 0.

As described above, according to the present embodiment, since the category adding unit 13 adds the intra-group similarity calculating means 1a for a new category to the sub-classifying unit 1, the large classifying unit 2 cannot accurately classify. It becomes possible to efficiently perform additional learning for a new pattern with large fluctuations.

FIG. 9 shows a sixth embodiment of the learning and recognizing device of the present invention, which has a degree-of-attribute-dependent category addition control unit 14. In FIG. 9, a fine classification unit 1, a similarity calculation unit 1a, a weighting factor update unit 1b, a large classification unit 2, a group dictionary 2a, a category identification unit 3, a teacher signal generation unit 4, a learning control unit 5, a learning control signal weighting unit. 6, identification signal weighting unit 7, multipliers 6a, 7a,
The category information storage unit 10 and the category addition unit 13 are the same as those of the fifth embodiment described above. In the present embodiment, in addition to the above, the category addition unit 1 according to the value of the group membership degree output from the large classification unit 2 by the belonging degree dependent category addition control unit 14.
The subclassification unit 1 determines whether or not 3 performs category addition.
Switch every time.

FIG. 10 shows an embodiment of the membership degree dependent category addition control unit 14 used in the learning recognition device of the present invention. 1
4a is a maximum value detector that detects the maximum value of a plurality of input group membership signals, 14b is a storage unit that stores the value of the group membership level that serves as a reference for category addition, and 14c is the maximum value detector 14a that detects the value. A comparator for comparing the maximum group membership degree with the reference group membership degree stored in the storage unit 14b;
Reference numeral d denotes a category addition instructing section that outputs a category addition instruction signal when the maximum group membership degree exceeds the reference group membership degree based on the comparison result of the comparator.

The operation of the learning / recognition device configured as described above will be described below. Here, the similarity calculation unit 1a of the fine classification unit will be described by taking the configuration shown in FIG. 3 as an example. The recognition operation is the same as in the first embodiment of the learning recognition device described above. The learning operation is the same as that of the fifth embodiment of the learning recognition device described above, but the belonging degree dependent category addition control unit 14 belongs to the fine classification unit 1 in which the category to which the input pattern signal to be learned belongs is learning. If not, the category is not always added to all the target subclassification units 1, but is added only when the group membership degree of the category group belonging to the target subclassification unit exceeds a predetermined criterion. The category addition unit 13 is controlled so as to perform. Further, it is also possible to set only some of the subclassification units 1 from the one having a higher group membership degree as a category addition target.

As described above, according to this embodiment, the large classification unit 2
The category is added only when it largely deviates from the classification standard of No. 1, and in other cases, efficient additional learning can be performed by switching to learning by changing the weight of the in-group similarity calculation unit 1a.

FIG. 11 shows a learning recognition apparatus according to a seventh embodiment of the present invention, which has an error-dependent category addition control unit 15. In FIG. 11, the subclassification unit 1, the similarity calculation unit 1a,
Weighting factor update unit 1b, large classification unit 2, group dictionary 2a, category identification unit 3, teacher signal generation unit 4, learning control signal weighting unit 6, identification signal weighting unit 7, multipliers 6a and 7a, category information storage unit 10 The category adding unit 13 is the same as that of the fifth embodiment described above. In this embodiment, in addition to the above,
The learning controller 5 includes the error calculator 5c and the weight change amount controller 5a.
The error-dependent category addition control unit 15 switches whether or not to perform the category addition according to the value of the learning control signal output from the learning control unit 5.

FIG. 12 shows an embodiment of the error-dependent category addition control unit 15 used in the learning and recognition device of the present invention. 15
a is a storage unit that stores an error value serving as a reference for category addition, 15b is a comparator that compares the error signal with the reference error value stored in the storage unit 15b, and 15c is a comparison result of the comparator. First, the category addition instructing section outputs a category addition instruction signal when the error signal exceeds a reference error value.

FIG. 13 is a block diagram specifically showing the configuration of the second embodiment of the similarity calculation section 1a. Reference numeral 41 is an input unit for inputting an input pattern signal. Reference numeral 42 denotes a multi-input / single-output signal processing unit, which multiplies the output of the lower-layer input unit 41 or the multi-input / single-output signal processing unit 42 connected to it, and the weighting factor which is the degree of connection thereof. The sum total is subjected to threshold processing and output. Here, the plurality of multi-input / single-output signal processing units have a layered structure and are network-connected so that signals propagate only to the upper layers, so that the degree of similarity to each category in the category group for the input pattern signal is Is calculated.

FIG. 14 shows the multi-input one-output signal processing section 42.
3 is a block diagram specifically showing the configuration of FIG. In FIG. 14, reference numeral 51 designates an input section for inputting an input pattern signal. Reference numeral 52 denotes a weighting coefficient storage unit that stores a weighting coefficient for weighting a plurality of input signals from the input unit 51. Reference numeral 53 denotes a multiplier, which multiplies the weight coefficient of the weight coefficient storage unit 52 by the input signal from the input unit 51. Reference numeral 54 is an adder, which sums the output values of all the multipliers 53. A threshold processing unit 55 limits the output value of the adder 54 to a value within a certain range.

The operation of the block diagram showing the second embodiment of the similarity calculation section 1a of the subclassification section 1 configured as described above will be described below. As in the first embodiment, the subclassification unit 1 inputs the input pattern signal X to the input unit 31. There are N input units 41, which are equal in number to the feature data of the pattern data, and each feature data x _i is input to the corresponding input unit 41. Each multi-input one-output signal processing unit 42 is stored in the lower input unit 41 connected to it as shown in FIG. 4, or the output of the multi-input one-output signal processing unit 42 and the weighting coefficient storage unit 52. The weighting coefficient w _ij, which is the degree of connection, is multiplied by the multiplier 53, the sum of the outputs of the respective multipliers 53 is calculated by the adder 54, and then converted by the threshold processing unit 55, and the output thereof is obtained. Output the value to the upper layer.
That is, the output value I _i of the i-th multi-input first output signal processing unit 42 of a layer shown in Figure 4, the input value to the input unit 51 I _j,
Letting w _ij (the connection weight between the i-th multi-input one-output signal processing section and the j-th input) be the input connected to it and the weighting coefficient which is the degree of the connection, To represent.

[0085]

I _i = f (Σ _j w _ij · I _j ) (15) Here, Σ _j represents the total sum for j. The input / output characteristics of the threshold processing unit 55 are shown in FIG. For example,
The input / output characteristic of the threshold processing unit 55 that limits the output to the range of (0, 1) is as shown in (Equation 16).

[0086]

F (a) = 1 / (1 + exp (-a + θ)) (16) where a is an input of the threshold processing unit 55. The input / output characteristic of the threshold processing unit 55 may be a threshold function other than the above.

The multi-input / single-output signal processing unit 42 in the uppermost layer is
The number of pattern signals included in each category group is the same as the number of categories, and each multi-input one-output signal processing unit 42 in the uppermost layer corresponds to each of these categories.
That is, the output of each multi-input / single-output signal processing unit 42 in the uppermost layer is output to the identification signal weighting unit 7 as a degree of similarity which is a degree of similarity to each category in the category group with respect to the input pattern signal.

The operation of the learning / recognition device configured as described above will be described below. The recognition operation is the same as in the first embodiment of the learning recognition device described above. The learning operation is the same as that of the fifth embodiment of the learning recognition device described above, but the error calculator 5c calculates the error between the output value of the identification signal weighting unit 7 and the teacher signal for each category to calculate the error signal vector E. = (E ₁ , e ₂ , ..., e _Nc ) is calculated as in ( _Equation 17).

[0089]

E _i = T _i −Σξ _i (1 ≦ i ≦ Nc; Nc is the number of categories) (17) Here, E _i is the error for the i-th category, and T _i is i.
The teacher signal for the th category, Σξ _i , represents the sum of the outputs of all multipliers 7a corresponding to the i th category.
The weight change amount control unit 5a outputs the error calculated by the error calculator 5c to the learning control signal weighting unit 6 as a learning control signal.

The learning control signal weighting unit 6 multiplies the learning control signal E and the group membership degree by the multiplier 6a as in (Equation 18) and outputs the result to the corresponding weighting coefficient updating unit 1b of the subclassification unit 1. .

[0091]

[Equation 18] _E'ip = E _i · μ _p (18) Here, _E'ip is a weighted learning control signal for the p-th subclassification part of the i-th category, and E _i is a learning control signal for the i-th category. , Μ _p represent the group similarity to the subclassification part p to which the weight learning control signal is input.

When the category to which the input pattern signal to be learned belongs does not belong to the subclassification section to be learned, the error-dependent category addition control section 15 does not always add the category to the target subclassification section. , The category adding unit 13 is controlled so as to add only when the value of the learning control signal exceeds a predetermined standard.

Regarding the weight correction method, the output of the learning control signal weighting section 6 is treated as an error signal in each subclassification section, and the weight can be updated in the direction of reducing the error by using the above-mentioned back propagation method. .

As described above, according to the present embodiment, the category is added only when the discriminant criterion of the present learning recognition device is largely deviated, and in other cases, the learning is switched to the learning by changing the weight of the in-group similarity calculation unit. This enables efficient additional learning. The error-dependent category addition control unit 15
Can also be used together with the above-mentioned membership degree-dependent category addition control unit 14.

FIG. 16 shows an eighth embodiment of the learning and recognizing device of the present invention, which has a belonging degree control unit 16. In FIG. 16, a fine classification unit 1, a similarity calculation unit 1a, a weight coefficient updating unit 1b, a large classification unit 2, a group dictionary 2a, a category identification unit 3, a teacher signal generation unit 4, a learning control signal weighting unit 6, an identification signal weighting. The unit 7, the multipliers 6a and 7a, the category information storage unit 10, the category addition unit 13, and the error-dependent category addition control unit 15 are the same as those in the above-described seventh embodiment, and the learning control unit 5 shown in FIG. 2 is similar to that of the configuration shown in FIG. 2, and the similarity calculating unit of the large classification unit is similar to that of the configuration shown in FIG. In this embodiment, in addition to the above, the learning monitoring unit 16
a monitors the error change of the error calculation unit 5c and the number of times the pattern is input, and the constant control unit 16b changes the value of the constant used in the calculation of the distance calculation unit 22 of the large classification unit 2 based on the monitoring result of 16a. .

FIG. 17 shows an embodiment of the distance calculation section used in the large classification section of the present invention. 22a _{0 to} 22a _n (n is the number of characteristic data of the pattern data) are subtractors for calculating the difference between the group reference pattern signal and the input pattern signal, 22b _{0 to} 22b _n
Is a power multiplier for calculating the power of the output of the subtractor, 2
2c is an adder for calculating the sum of the outputs of the exponentiators, and 22d
Is a power multiplier for calculating the power of the output of the adder.

The operation of the learning / recognition device configured as described above will be described below. The recognition operation is the same as in the first embodiment of the learning recognition device described above. The learning operation is similar to that of the seventh embodiment of the learning recognition device described above, but the membership control unit 16 calculates the group membership so that a small number of category groups have a relatively large group membership, As the learning progresses, the method of calculating the group membership degree is changed so that more category groups have a relatively large group membership degree. The learning transition is monitored by the learning monitor 16a. The learning monitoring unit 16a indicates that the number of input patterns exceeds a predetermined standard or the error calculation unit 5
When the amount of change in the error is below a predetermined standard based on the output of c, the constant control unit 16b is instructed to change the constant. When the constant control unit 16b receives an instruction from the learning monitoring unit 16a, the exponentiator 22b ₀ of the distance calculation unit 22 of the large classification unit.
˜22b _n , 22d to the exponent of power calculation 22b ₀ to 22b _n
For 22d and 22b ₀ to 22b _n for 22d
Set to the reciprocal of. In (Equation 8), it can be realized by changing f. That is, in the initial stage of learning, by making f have a value very close to 1, only one category group has a large group membership value, and the other category groups have a relatively small value. By increasing the value of f as the learning progresses, the plurality of category groups have a relatively large degree of group membership.
Since the amount of weight change during learning is proportional to the degree of group membership, in the early stages of learning, only a small number of category groups are strongly selected, so that only some subclassifiers learn one input pattern signal. However, as learning progresses, many subclassification sections smoothly shift to cooperative learning in which learning is performed. According to the present embodiment, learning is efficiently advanced by local learning at an early stage of learning when an appropriate weight is not obtained in each subclassification unit, and cooperative learning is performed at a later stage of learning or at a stage of performing additional learning. By doing so, it is possible to prevent the balance of the weight coefficient as a whole by changing the local weight,
Effective additional learning is possible.

FIG. 18 is a block diagram showing a third embodiment of the similarity calculation section 1a. In this embodiment, two sets of pattern input signals (first feature data, second feature data) are used to calculate the degree of similarity to three categories. A plurality of route selection units n are combined to construct a branch structure. A two-layer structure is constructed in one branch structure. Lower layers,
Is composed of route selection units n11 to n12, and the upper layer is composed of learning and recognition units q31 to q33. The feature data to be judged is input to the signal input unit of each path selection unit, and the teacher signal is input to the teacher signal input unit.

FIG. 19 shows an embodiment of the route selection unit used in the third embodiment of the similarity calculation section 1a of the present invention. A signal input unit n1 inputs a signal to be recognized, which is input through the signal input terminal n1a, to the quantizer n2. The quantizer n2 quantizes the input signal and outputs the quantized result to the path weighting unit n3b. Reference numeral n3a is a route input unit which inputs the route signal input via the route input terminal n3a0 to the route load unit n3b. n3e1
Reference numeral n3e5 is a route output terminal which connects these terminals to each other when hierarchically combining the route selection unit or the learning recognition unit.

The route load section n3b connects the route input section n3a and the route output section n3e with loads n3b01 to n3b01-n.
3b05 and a load setting unit n3b0 for setting the value of the load to the loads n3b01 to 3b05 according to the quantization result.
It consists of and. In the route load part n3b, the load setting part n3
b0 is based on the quantization result of the input signal quantizer n2,
The weight value is set to a weight corresponding to each quantization level. That is, the load setting unit n3b0 distributes the value of each load based on the quantization result. Load n3b01
To n3b05 weight the route signals input from the route input unit, and the route output unit n3e outputs the weighted route signals to the route output terminals n3e1 to n3e5.
Reference numeral n4 denotes a quantization control unit, which changes the quantization range of the quantizer 2 based on the internal state of the route selection unit stored in the internal state storage unit n5, and how many loads the route setting unit n3b0 distributes the load value to. Change. That is,
All the weights included in the path weighting unit n3b are not used, and the quantization control unit n4 changes the necessary number based on the stored contents of the internal state storage unit n5 as necessary.

FIG. 20 shows an example of a learning and recognition unit used in the third example of the similarity calculating section 1a of the present invention. q1
Is a teacher signal input unit, q3a is a route input unit, q3b is a route weighting unit, and is composed of weights q3b01 to q3b010 for weighting the output value of the route input unit q3a and a learner q3b1 for changing the coupling weight. q3c is an adder that adds the outputs of the weighted weights.

The learning device q3b1 calculates the weight q based on the output of the learning recognition unit and the input value from the teacher signal input unit q1.
The values of 3b01 to 3b050 are determined.

Next, the similarity calculator 2 configured as described above.
The learning operation of b will be described. "1" is given as a route input signal to the route input terminals of the route selection units n11 to n12 in the lower layer of each branch structure. Further, the feature data of the recognition object is input to the signal input terminal to the quantizer of these route selection units (in the case of this figure, the route selection unit of the two feature data is To enter).

In each route selection unit, these characteristic data are quantized by the quantizer n2, and based on the quantized value, the load setting unit q3b0 quantizes the weight value corresponding to each data. Set based on level. In this way, a plurality of routes are selected in one route selection unit, and a value obtained by multiplying the load value is sent to the upper layer. At this time, the internal state storage unit n5 holds the statistic amount of the input feature data as an internal state, and updates the stored content each time the feature data is input. Based on the stored contents of the internal state storage unit n5, the quantization control unit n4 increases the load and the number of routes used by the route load unit n3b when the distribution of the feature data input in the past is large, and selects a more accurate route. to enable. At the same time, the quantization control unit n4 changes the quantization standard of the quantizer n2 according to the increase in the number of routes of the route weighting unit n3b.

At the teacher signal input terminal q1a of the learning recognition unit q in the upper layer, a teacher input signal indicating which of the three categories the recognition target signal belongs to,
That is, for the learning recognition unit for which it is desirable to output the largest value among the learning recognition units q31 to q33,
1 "is input, and 0 is input for other learning recognition units. As the weight changing method, the above-described error back propagation method or the like can be used.

Next, the recognition operation of the similarity calculating section shown in FIG. 18 will be described. Just like the learning operation, "1" is first given as a route signal to the route input terminals of the route selection units n11 to n12 in the lower layer of each branch structure. Further, the feature data of the recognition object is input to the signal input terminal to the quantizer of these route selection units. (In the case of this figure, two feature data are input to each route selection unit.) For each route selection unit, the feature data is quantized by the quantizer n2 and quantized. Based on the value, the load setting part n
3b0 sets the weight value corresponding to the characteristic data based on the quantization level position. Thus, one path is selected at the time of recognition, and the learning recognition unit q31 of the upper layer is selected.
The value obtained by multiplying the route signal by the value of the load is sent to the input terminals of ~ q33.

The learning recognition unit of the uppermost layer multiplies the input value from the path input unit by the connection weight, and outputs the value obtained by the adder. At this time, the category represented by the learning recognition unit that outputs the maximum value among the plurality of learning recognition units is the recognition result. Further, the weight and the threshold value are not changed at the time of recognition. In this way, the learning recognition device can perform classification and recognition judgment of the recognition target object based on the input recognition target object feature data.

FIG. 21 shows a ninth embodiment of the learning recognition device of the present invention, which has a quantization reference storage unit 17 and a quantization reference control unit 18. In FIG. 21, a fine classification unit 1, a similarity calculation unit 1a, a weight coefficient updating unit 1b, a large classification unit 2, a group dictionary 2a, a category identification unit 3, a teacher signal generation unit 4, a learning control unit 5, a learning control signal weighting unit. 6, the identification signal weighting unit 7, the multipliers 6a and 7a, the category information storage unit 10, and the category addition unit 13 are the same as those in the above-described seventh embodiment, and are similar to the similarity calculation unit of the large classification unit. The configuration is the same as that of the configuration shown in FIG. 2, and the similarity calculating unit of the fine classification unit is the same as that of the configuration shown in FIG. In the present embodiment, in addition to the above, the quantization reference storage unit 17 stores a constant used by the quantization control unit n4 of the similarity calculation unit 1a for comparison with the storage content of the internal state storage unit n5. The quantization standard control unit 18 changes the constant stored in the quantization standard storage unit.

FIG. 22 is a block diagram showing a first embodiment of the quantization reference control unit 18 used in the ninth embodiment of the learning and recognizing device of the present invention. In FIG. 22, 18a is a quantization constant control unit and 18b is an error monitoring unit. The error monitor 18b monitors the error signal output from the error calculator 5c,
Detect a decrease in error or a decrease in error change. The quantization constant control unit 18a changes the constant stored in the quantization reference storage unit 17 based on the result detected by the error monitoring unit 18b.

FIG. 23 is a block diagram showing a second embodiment of the quantization reference control unit 18 used in the ninth embodiment of the learning and recognizing device of the present invention. In FIG. 23, 18a is a quantization constant control unit, and 18c is a learning number monitoring unit. The learning number monitoring unit 18c monitors the number of times of inputting the input pattern signal. The quantization constant control unit 18a changes the constant stored in the quantization reference storage unit 17 based on the learning count detected by the learning count monitoring unit 18c.

FIG. 24 is a block diagram showing a third embodiment of the quantization reference control unit 18 used in the ninth embodiment of the learning / recognizing device of the present invention. In FIG. 24, 18a is a quantization constant control unit, and 18d is a category addition monitoring unit. The category addition monitoring unit 18d monitors the category addition operation of the category addition unit 13. The quantization constant control unit 18a changes the constant stored in the quantization reference storage unit 17 when the category addition monitoring unit 18d detects the category addition.

The operation of the learning / recognition device configured as described above will be described below. The recognition operation is the same as that in the seventh embodiment of the learning recognition device described above. The learning operation is similar to that of the seventh embodiment of the learning recognition device described above, but when the error is large, the learning number is small, or the category addition operation is performed, the route of each subclassification unit The quantization reference control unit 18 changes the comparison reference used by the quantization control unit n4 for each subclassification unit so that the weighting unit n3b can perform detailed route selection.

According to this embodiment, the error signal, the number of learnings,
By changing the accuracy of route selection in each subclassification unit according to the category addition operation, effective learning / additional learning can be performed for each subclassification unit.

Each means of the present invention can be realized by software using a computer, or can be realized by using a dedicated hardware circuit having each of these functions.

[0115]

As is apparent from the above description,
As described above, the learning recognition device of the present invention corrects the weight of the fine classification unit based on the error signal weighted by the degree of group membership output by the large classification unit. It is possible for the classifiers to learn in cooperation and improve the recognition rate of patterns located at the boundaries of category groups.

Further, for a pattern that greatly deviates from the classification standard of the large classification section, the category addition section adds a new category to the fine classification section as needed, and thus additional learning can be efficiently performed. Is.

Further, by changing the method of calculating the degree of group membership according to the transition of learning, it is possible to perform appropriate group membership degree calculation in each of the initial learning and the additional learning, and effective learning is possible.

[Brief description of drawings]

FIG. 1 is a block diagram showing a first embodiment of a learning recognition device of the present invention.

FIG. 2 is a block diagram showing an embodiment of a large classification unit also used in the learning recognition device.

FIG. 3 is a block diagram showing a first embodiment of a similarity calculation unit also used in the learning recognition device.

FIG. 4 is a block diagram showing a second embodiment of the learning recognition device in the same manner.

FIG. 5 is a block diagram showing a third embodiment of the learning recognition device in the same manner.

FIG. 6 is a block diagram showing a fourth embodiment of the learning recognition device in the same manner.

FIG. 7 is a block diagram showing an example of a group membership degree weighting unit also used in the learning recognition device.

FIG. 8 is a block diagram showing a fifth embodiment of the learning recognition device in the same manner.

FIG. 9 is a block diagram showing a sixth embodiment of the learning recognition device in the same manner.

FIG. 10 is a block diagram showing an embodiment of a membership degree-dependent category addition control unit also used in the learning recognition device.

FIG. 11 is a block diagram showing a seventh embodiment of the learning recognition device in the same manner.

FIG. 12 is a block diagram showing an embodiment of an error-dependent category addition control unit also used in the learning recognition device.

FIG. 13 is a block diagram showing a second embodiment of the similarity calculation unit also used in the learning recognition device.

FIG. 14 is a block diagram showing one embodiment of a multi-input one-output signal processing unit of the second embodiment of the similarity calculation unit which is also used in the learning recognition device.

FIG. 15 is a diagram showing an example of input / output characteristics of the threshold value processing unit of the multi-input one-output signal processing unit of the second example of the similarity calculation unit used in the learning recognition device.

FIG. 16 is a block diagram showing an eighth embodiment of the learning recognition device in the same manner.

FIG. 17 is a block diagram showing an embodiment of a distance calculation unit also used in the learning recognition device.

FIG. 18 is a block diagram showing a third embodiment of the subclassification unit similarly used in the learning recognition device.

FIG. 19 is a block diagram showing an example of the route selection unit of the third example of the subclassification unit similarly used in the learning recognition device.

FIG. 20 is a block diagram showing an example of a learning and recognition unit of the third example of the subclassification unit used in the learning and recognition device.

FIG. 21 is a block diagram showing a ninth embodiment of the learning recognition device in the same manner.

FIG. 22 is a block diagram showing a first embodiment of a quantization reference control unit which is also used in the learning recognition device.

FIG. 23 is a block diagram showing a second embodiment of the quantization reference control unit also used in the learning recognition device.

FIG. 24 is a block diagram showing a third embodiment of the quantization reference control unit also used in the learning recognition device.

FIG. 25 is a block diagram showing an embodiment of a conventional learning and recognition device.

[Explanation of Codes] 1 Fine classification unit 24 Adder 1a In-group similarity calculation unit 25 Multiplier 1b Weight coefficient updating unit 26 Divider 2 Major classification unit 31 Input unit 2a Group dictionary 32 Category dictionary 2b Group reference pattern 33 Distance calculation unit Similarity calculation unit 34 Divider 3 Category identification unit 35 Adder 4 Teacher signal generation unit 36 Multiplier 5 Learning control unit 37 Divider 5a Weight change amount control unit 41 Input unit 5b Comparator 42 Multi-input one Output signal processing unit 6 Learning control signal weighting unit 51 Input unit 6a Multiplier 52 Weighting coefficient storage unit 7 Identification signal weighting unit 53 Multiplier 7a Multiplier 54 Adder 8 Group selection unit 55 Threshold processing unit 9 Input signal selection unit 91 Load section 10 Category information storage section 92 Load adjustment section 11 Group membership degree switching section n Path selection unit 12 Group membership degree load section n1 Signal input section 13 Category addition section n11 to n12 Path selection unit 14 Membership degree dependent category n1a Signal input Additional units Control section n2 Quantizer 14a Maximum value detector n3a Path input section 14b Storage section n3b Path selection section 14c Comparator n3b0 Load setting section 14d Category addition instruction section n3b01 to n3b05 Load 15 Error dependent category n3e Path output section Additional control section n3e1 ~ N3e5 Path output terminal 15a Storage unit n4 Quantization control unit 15b Comparator n5 Internal state storage unit 15c Category addition instruction unit q Learning recognition unit 16 Membership degree control unit q11 to q12 Learning recognition unit 16a Learning monitoring unit q1 Teacher signal input unit 16b Constant control unit q3a Path input unit 17 Quantization reference control unit q3b01 to q3b010 Weight 18 Quantization reference storage unit q3c Adder 18a Quantization constant control unit 101 Large classification unit 18b Error monitoring unit 102 Fine classification unit 18c Learning frequency monitoring unit 103 Group selection unit 18d Category addition monitoring unit 104 Sub-classification unit Input signal selection unit 21 Input unit 105 Identification unit 22 Distance calculation unit 106 Input unit 22a _{0 to} 22a _n Subtractor 107 Multi-input one-output signal processing unit 22b _{0 to} 22b _n Power multiplier 108 Input section And 22c adder 109 multi-input one-output signal processing unit 22d exponentiator 110 maximum value selection unit 23 divider 111 similarity calculation unit 112 category identification unit

 ─────────────────────────────────────────────────── ─── Continuation of the front page (72) Inventor Susumu Maruno 1006 Kadoma, Kadoma City, Osaka Prefecture Matsushita Electric Industrial Co., Ltd.

Claims (24)

[Claims] 1. A large classifying section provided with means for calculating a group membership degree, which is a degree to which an input pattern signal belongs to a category group composed of a set of similar patterns, for a plurality of category groups. An intra-group similarity calculation unit provided with means for calculating the intra-group similarity, which is the degree to which the input pattern signal belongs to a category included in each category group, and a product of the intra-group similarity and the group membership An identification signal weighting unit for weighting the intra-group similarity, a category discriminating unit for comparing weighted intra-group similarity signals output from the identification signal weighting unit, and learning. A teacher signal generating section for generating a teacher signal necessary for, a teacher signal output from the teacher signal generating section, an output of the identification signal weighting section, and an output of the category identifying section based on the output of the category identifying section. The weight of A learning control unit that controls the amount of change, a learning control signal that is output from the learning control unit, and a plurality of multipliers that calculate the product of the degree of group membership output from the large classification unit, and the weighting of the learning control signal. A learning control signal weighting unit, and a weighting factor updating unit that updates the weighting factor of the in-group similarity calculation unit based on the outputs of the learning control signal weighting unit and the in-group similarity calculation unit. A learning and recognition device, comprising: a plurality of subclassification units each including the weighting factor update unit and the in-group similarity calculation unit. 2. A comparator for comparing the teacher signal output from the teacher signal generator with the discrimination result of the category discriminator,
The learning recognition unit according to claim 1, wherein the learning control unit is configured by a weight change amount control unit that determines a weight change amount of the in-group similarity calculation unit according to a comparison result of the comparator. apparatus. 3. An error calculating section for calculating a difference between a teacher signal output from a teacher signal generating section and an output of the identification signal weighting section, and an intra-group similarity calculating section according to an output of the error calculating section. The learning recognition device according to claim 1 or 2, wherein the learning control unit is configured by a weight change amount control unit that determines a weight change amount. 4. A group selection unit that selects a plurality of category groups based on a group membership degree output from a large classification unit, and a fine classification unit that inputs an input pattern signal based on the group membership degree, the input. The learning recognition device according to any one of claims 1 to 3, further comprising a signal selection unit. 5. A limitation for selecting a plurality of category groups based on a category storage unit that stores the types of categories belonging to each category group, and the stored content of the category storage unit and the degree of group membership output by the general classification unit. The learning and recognition device according to claim 4, further comprising a group selection unit and a group selection unit switching unit that switches between the group selection unit and the limited group selection unit. 6. The group membership degree loading unit is provided, and the group membership degree loading unit weights the group membership degree based on the stored contents of the category storage unit. Learning recognizer. 7. A category addition unit is provided which adds means for calculating the similarity to a new category to the intra-group similarity calculation unit of each subclassification unit based on the stored contents of the category storage unit and the teacher signal. The learning recognition device according to any one of claims 1 to 6. 8. A degree-of-attribute-dependent category addition control unit is provided for switching, for each category group, whether or not the category addition unit performs category addition based on the group membership degree output by the large classification unit. The learning and recognition device according to claim 7. 9. A maximum value detector for obtaining the maximum value of the group membership degree, a storage unit for storing a reference group membership degree, a maximum value of the group membership degree detected by the maximum value detector, and the storage unit. And a comparator for comparing the degree of group membership stored in the above, and when the maximum value of the degree of group membership exceeds the reference group membership degree based on the comparison result of the comparator, the category is added to the category adding unit. 9. The learning and recognition device according to claim 8, wherein the category addition instructing unit that issues an instruction to perform the above-mentioned belonging degree-dependent category addition control unit is configured. 10. An error-dependent category addition control unit is provided for switching, for each category group, whether or not the category addition unit performs category addition based on an error signal output by the error calculation unit. The learning recognition device according to any one of 7 to 9. 11. A storage unit that stores a reference error value, a comparator that compares the output of the error calculation unit and the error stored in the storage unit, and an error based on the comparison result of the comparator. 11. The learning according to claim 10, wherein the error-dependent category addition control unit is configured by a category addition instructing unit that instructs the category adding unit to perform category addition when exceeds the reference error. Recognition device. 12. A group dictionary in which a plurality of group reference pattern signals representative of a category group composed of a set of similar patterns are stored, and a group reference pattern for calculating a group membership degree using the group reference pattern signal. With the similarity calculation unit,
The said large classification part was comprised, The Claims 1 to 1 characterized by the above-mentioned.
1. The learning recognition device according to any one of 1. 13. The learning / recognition device according to claim 1, wherein a membership degree control unit is provided, and a calculation constant used for calculating the group membership degree of the large classification unit is changed. 14. A degree-of-attribute control unit is configured by a learning monitoring unit that measures the number of times of learning and a constant control unit that controls a constant,
The learning recognition device according to claim 13, wherein the constant control unit changes a calculation constant used for calculating the group membership degree of the large classification unit based on the output of the learning monitoring unit. 15. A learning degree monitoring section for calculating an output change of an error calculation section and a constant control section for controlling a constant constitute a belonging degree control section, and the constant control section is classified into a large class based on an output of the learning monitoring section. The learning and recognition device according to claim 13 or 14, wherein a calculation constant of a group membership degree of a part is changed. 16. A plurality of distance calculators for calculating distances between an input pattern signal and all group reference pattern signals stored in a group dictionary, and a plurality of distance calculators for calculating an inverse of an output of the distance calculator. A divider, an adder for adding the outputs of the divider, a plurality of multipliers for multiplying the output of the adder and the output of the distance calculator, and a reciprocal of the output of the multiplier The learning and recognition device according to any one of claims 1 to 15, wherein the large classification unit is configured by a plurality of dividers. 17. A distance calculation unit of the large classification unit is configured by a plurality of subtracters for calculating a difference between a plurality of input signals and a signal enhancer provided with a plurality of adders and powers, The learning recognition device according to claim 16, wherein the belonging degree control unit is configured by a constant control unit that changes a value of an exponent of a power calculation of a power of the signal enhancer. 18. A weight coefficient storage unit for holding a plurality of weight coefficients, an input unit for inputting a plurality of input signals, and a weight coefficient stored in the weight coefficient storage unit for weighting the input signal from the input unit. Multiplying means for adding, and adding means for adding a plurality of input signals weighted by the multiplying means,
A multi-input-output signal processing unit consisting of a threshold value processing unit for limiting the output of the adding unit to a value within a certain range is arranged in a multiple layer structure and connected so that signals propagate from the lower layer to the upper layer. The learning and recognition device according to any one of claims 1 to 17, wherein the fine classification unit is configured. 19. A category dictionary in which a plurality of category reference pattern signals indicating a representative value of each category of an input pattern signal are stored, and an input pattern signal and all category reference patterns stored in the category dictionary. A plurality of distance calculators for calculating the distance to the signal, a plurality of dividers for calculating the reciprocal of the output of the distance calculator, an adder for adding the outputs of the dividers, and the adder A plurality of multipliers for multiplying the output of the multiplier and the output of the distance calculation unit, and a plurality of dividers for calculating the reciprocal of the output of the multiplier, thereby configuring the subclassification unit. The learning and recognition device according to any one of claims 1 to 17, which is characterized in that. 20. A route selection unit includes a signal input unit, a quantizer for quantizing an input signal from the signal input unit, a route input unit having a single or a plurality of route input terminals, and a single or A route output unit having a plurality of route output terminals, a route weighting unit that selects a route according to the quantized output of the quantizer, an internal state storage unit that stores an internal state of a route selection unit, and the internal state. The learning recognition unit is composed of a quantization control unit that changes the number of quantizations and a quantization range of the quantizer based on the storage content of the storage unit, and a structure storage unit that stores the configuration of the route selection unit. A teacher signal input unit, a route input unit having a plurality of route input terminals, a weight for weighting an input from the route input unit, an adder for adding an output from the load, and a weight value And a learner to determine More constructed by combining the routing unit hierarchically configured multiple branching structure, by placing the learning and recognition units thereon,
The said subclassification part was comprised, The Claims 1 to 1 characterized by the above-mentioned.
7. The learning recognition device according to any one of 7. 21. A quantization reference storage unit that stores a comparison reference for the storage content of the internal state storage unit, and a quantization reference control unit that changes the storage content of the quantization reference storage unit for each of the subclassification units. The learning recognition device according to claim 20, wherein the learning recognition device is provided. 22. A quantization reference control unit is configured by an error monitoring unit that calculates an output change of the error calculation unit and a quantization constant control unit that changes the quantization reference based on the output of the error monitoring unit. 22. The learning and recognition device according to claim 21, which is characterized in that. 23. The quantization reference control unit is configured by a learning count monitoring unit that monitors the learning count and a quantization constant control unit that changes the quantization reference based on the output of the learning count monitoring unit. Claim 21 or, characterized by
22. A learning and recognition device according to item 22. 24. The quantization reference control unit is provided with a category addition monitoring unit for monitoring the category addition of the category addition unit, and the quantization constant control unit changing the quantization reference based on the output of the category addition monitoring unit. The learning recognition device according to any one of claims 21 to 23, characterized in that