JPH04153891A - Pattern recognition system using rough code expression - Google Patents

Abstract

PURPOSE:To efficiently perform pattern recognition with a little misrecognition by realizing a general inverse matrix which is high in associating ability approximately by a local and easy process and handling even deformed pattern by a threshold value process and a feedback circuit. CONSTITUTION:In a three-layered associative storage model, an input pattern is represented in output layers I, (i), and N and fixed mapping to be fed back is realized approximately. So the model is tolerant to a noise input. Further, the outputs of intermediate layers I, K, and M in a mutual suppression coupling state are in rough code expression wherein only one element showing the recognition category of the input pattern fires. The associative process of the model is approximately equivalent to the calculation of the general inverse matrix, so the associating ability is high. Consequently, the pattern recognition with a little misrecognition is efficiently realized.

Description

[Detailed description of the invention] [Industrial application field]

The present invention relates to a pattern recognition method that dynamically recognizes patterns of characters (images), sounds, etc., and modifies a recognition dictionary through learning.

[Conventional technology]

In conventional self-(mutual) recall associative memory models, the orthogonal projection type associative memory model sets the connection weight values between input and output layers using a general inverse matrix, rather than the correlation type associative memory model where the connection weight values between the input and output layers are set using a correlation matrix of input and output patterns. They are known to have higher associative abilities. Mr. Matsuoka has already proposed a method (see Figure 3) that uses a three-layer model with an intermediate layer of mutually inhibiting connections to realize the calculation of a general inverse matrix by efficient local operations (see Figure 3). University of Technology): “About various structures of orthogonal projection type associative memory circuits”
, Journal of the Institute of Electronics, Information and Communication Engineers, Vol. J73-Dll
, No, 4. Apri1'90. pp841-847)
Furthermore, Mr. Matsuoka describes a method for calculating a general inverse matrix in a self-recall associative memory model using a feedback circuit (Figure 4).
was also proposed (see the above-mentioned journal of the Institute of Electronics, Information and Communication Engineers). These models not only realize the calculation of complex general inverse matrices using a circuit with a simple network structure, but also easily realize the learning of connection weight values, which are recognition dictionaries, by adding prototype patterns, so they are suitable for LSI implementation. is also suitable.

[Problem to be solved by the invention]

There were problems in actually applying the conventional associative memory model described above to pattern recognition. That is, this associative memory model shows the memory-retrieval of a basic prototype pattern, but has the drawback that it cannot be said to be sufficient for the memorization and retrieval of realistic deformed patterns. The present invention aims to eliminate this drawback. That is, an object of the present invention is to perform pattern recognition with high recognition ability for deformed patterns.

[Means to solve the problem]

In order to achieve the above object, the present invention dynamically converts an input pattern into a recognized cover pattern using a feedback circuit, and uses an intermediate layer using a sparse code representation in an associative memory model that acquires a recognition dictionary through learning. This is a pattern recognition method that stores information to be stored in a distributed manner in connection weights, which is a recognition dictionary. Further, according to one aspect of the present invention, the pattern recognition method having the above configuration is characterized in that an auxiliary net is provided for determining the permissible range of the distribution of each recognition category.

[Effect]

In the three-layer associative memory model of the present invention, an identity mapping in which the input pattern is reproduced and fed back to the output layer is approximately realized, so it is resistant to noise input. Further, the intermediate layer output of the mutually inhibiting combination becomes a sparse code representation in which only one element representing the recognition category of the input pattern fires (see FIG. 8(C)). The associative processing of this model is approximately equivalent to the calculation of a general inverse matrix, so it has a high associative ability. Therefore, pattern recognition with less WAIIi can be efficiently realized. Furthermore, in the aspect of the present invention in which an auxiliary net is provided, the vi recognition result of the associative memory model is determined according to the result of integrating the recognition result by the three-layer associative model and the permissible range of distribution in other category directions by the auxiliary net. Ru. Therefore,
- Pattern recognition with a small number of layers W4m can be efficiently realized.

[First Embodiment] FIG. 1 is a diagram showing an orthogonal projection type feedback associative memory model of the present invention, and FIG. 2 is a diagram showing an embodiment of the system configuration of the present invention. As shown in FIG. 2, this system includes an input register 21 that holds an input pattern or feedback output, a clock oscillator 22 that controls the timing of supplying the contents of the input register 21 to the next stage, and a W2
The sum-of-products calculation section 23 calculates "x", and the dy/dt of equation 0 is calculated based on the outputs of the sum-of-products calculation sections 23 and 26, and the output y (t) of the intermediate layer is updated by the change.
The differential calculation unit 24 obtains the output y (
t) and outputs h(y(t)); a nonlinear threshold processing unit 27 that performs threshold processing on the output of the function calculation unit 25; and a nonlinear threshold processing unit 2.
It consists of an output register 28 that holds the output of 7 as a recognition output f (y), and a product-sum operation section 29 that multiplies the output of the threshold processing section 27 by X and performs the operation of the 0 formula. Each calculation unit of the second rEJ device that performs this learning e-recognition process can be realized by a program on a general-purpose computer or a dedicated analog (or digital) calculation circuit by discretizing continuous time. The states of the elements in each layer are stored in their respective registers, and the connection weight values are stored in the memory of the product-sum operation section. FIGS. 6 and 7 are flowcharts showing the processing procedure of the present invention, and show the processing procedure of recognition and learning, respectively. In the following, an example of recognition processing when the combination weight values Wl and W2, which are recognition dictionaries, have already been set will be explained with reference to FIG. 6, and then an example of a learning method for the recognition dictionary will be explained with reference to FIG. 7. explain. In the recognition process of the present invention, as shown in the flowchart of FIG.
(Step O). That is, when the sample time interval is T, an input pattern x I m + belonging to category m is applied to the input layer during 0≦t<T. Second
The input pattern x'=x (1 is held in the input register 21 in the figure. In the meantime, by t=kT, the intermediate transmission output is updated by formula (Step 1). τd y(t)/d t = Wl h(y(t))
+ W2"x'...ΦHere, W1=-X"X,
W2=X, prototype matrix: X=[xmouth+, x(
21,,,,X(Ml]dxfl+: (χ+11...
, , , , x(11N). Also,
Let τ<T. (2) Calculate the intermediate layer output h (y(t)) using the formula. This calculation is performed using the apparatus shown in FIG. The product-sum calculation unit 23 calculates W2′ x ′, and the product-sum calculation unit 26 calculates Wlh (y(t)). This intermediate layer output h (y(t) ) is subjected to threshold processing using flo by the nonlinear threshold processing unit 27 to obtain a recognition ratio cover turn f , (h(y(t)) ). It is determined whether all elements of the recognition ratio cover turn are O or 1 within the range of error ε (5tep2). If all are 0 or 1, the process ends. If there is an element that is not O or 1, update the feedback signal and set k to +1 and repeat 5 steps.
1 ((Step 3). That is, the feedback signal is calculated using the equation
+1) Provide up to Ts to the input layer. That is, the input register 21 has kT, ≦T< (k+1
) T, holds the open signal. In the meantime, perform similar processing according to formula (2). This process is repeated until all elements of the recognition output become O or 1. Note that when 0≦t<Ts, the following is true. ■As shown in the formula, during kT, ≦t<(k+1)T, the general inverse matrix is approximately calculated by the mutually inhibiting coupling W1 in the intermediate layer, and the identity mapping is approximated by the feedback signal, so noise input Strong against Further, when the prototype pattern stored in the recognition dictionary (W2) is input, the output of the intermediate layer is a sparse code expression in which only one element representing the recognition category fires, as shown in equation (2). Here, T: transpose, +
D represents the general inverse matrix. y ((k+1)T,)! "t (XTX)"X↑X'
= (X”X)”X”Xf+ (y (kT,)
)=r+<y(kT-))...
・■y (T,) L:i(X”X)”X”X'= (
X”X)”X”x'・l= (0,...,0,1゜0
, . . 0) . . . 0 Next, the learning process of the connection weight value, which is the recognition dictionary of the present invention, will be explained according to the flowchart of FIG. First, with a prototype pattern such as the average value of multiple learning patterns in each recognition category, the combined directivity Wt and W
2 is initialized (Step O). W+=-XTX W2=X Next, perform the above-mentioned recognition process for each learning pattern (S
step 1). If the presented learning pattern can be recognized correctly, the connection weight value (dictionary) is made a little closer to this learning pattern (5te
p2). W1=-a e”'x”x e(k”+ (1-a)W
l −■W2=ae(k'+ (1-B)W2
-■If the presented learning pattern is approved by WA,
f,(o)=O,fl so as to reduce the area of attraction of the stable point for the prototype pattern of the misidentified category.
(1)=1. Under the condition of f, (θl) = θ1, 01 is thickened (and the threshold value numerical aperture f10 is adjusted (Step 2)
). The learning pattern is presented again and this operation is repeated until all learning patterns are no longer misrecognized (Step 3). After learning, recognition processing with fewer misidentifications can be performed on patterns other than the learned patterns. Examples of each parameter are shown below. Number of input dimensions: N = 258, number of categories: M = 71, time constant of decay: τ = 0.1, sampling time: T, = 10° learning constant: α = 0.1, instability parameter: θ, = Fluctuation of 0.5, θ! : δ=0.01, Threshold numerical aperture: Incidentally, when the threshold numerical aperture is shown in a diagram, it is as shown in FIG. 5. FIG. 8 shows an experimental example of pattern recognition showing the effects of this embodiment, and FIG. 9 shows an experimental example of pattern recognition using a conventional example for comparison. In both figures, (a) shows the pattern rEJ given to the input layer, this pattern "E" containing 15% random noise. (b) shows the pattern obtained in the output layer, (
C) shows the state of the intermediate layer. As shown in FIG. 9(b), in the case of the conventional example, the pattern obtained in the output layer contains considerable noise, but in the case of the embodiment of the present invention shown in FIG. 8(b), noise is removed. A beautiful pattern is obtained. Regarding the intermediate layer, as shown in FIG. 9(C), according to the prior art, a relatively large intermediate value output appears in one element, and relatively small intermediate value outputs appear in three elements. ing. In contrast, in the embodiment of the present invention, only one element is a 100% firing element, as shown in FIG. 8(C). As described above, it can be seen from the experimental examples that the present invention has a much superior pattern recognition ability compared to the prior art. [Second Embodiment] FIG. 10 shows a modified embodiment (second embodiment) of the present invention using major classification and detailed classification. This uses the associative memory model of the present invention as a plurality of modules, and includes a large classification net 101 that roughly identifies patterns consisting of similar categories, and a selection unit 102 that outputs a similar category selection signal determined by its maximum output value. and a detailed classification network that identifies the final recognition category selected by the similar category selection signal) 103. ...e, 104, and an integrating unit 105 that integrates the outputs of the detailed classification net. According to this embodiment, the major classification net 101 and the selection unit 10
2, detailed classification of similar categories) 103.
・Since 104 is selected and the final recognition category is identified, the configuration is simple and the recognition rate is high. In addition, without limiting the similar categories by selection in the major classification section,
A final recognition category may be identified by integrating the recognition outputs of each detailed classification net. Furthermore, only one of the major classification section or the detailed classification section,
It may be configured using an existing identification method. [Third Embodiment] FIG. 11 is a block diagram showing a third embodiment of the present invention. This example uses the feedback associative memory model of the three-layer net structure with mutually inhibiting connections to realize the sparse intermediate layer shown in the first example as the basic net, and further develops the same configuration to By adding an auxiliary net that determines the permissible range of the distribution, pattern recognition that is resistant to deformation and recognition dictionary learning can be performed efficiently. The device of this third embodiment has a basic net (basic recognition network) 111 having the configuration shown in FIGS. 1 and 2 described in the first embodiment, and a configuration same as that of the basic net 111. Auxiliary nets 112 and 113 of categories (1) to (m), a differential pattern extraction unit 114 that calculates the difference between the input pattern x (O) and the dictionary, an allowable variation determination unit 115 in the sine direction, and a recognition result determination unit 11
It consists of 6. The recognition processing of this embodiment will be explained with reference to FIG. Each recognition element in the intermediate layer of the basic net responds sensitively to changes in the direction of each category prototype pattern, but is insensitive to changes in the turn direction of prototype patterns in other categories. Therefore, let's set the input initial pattern x (0).
p O), the recognition processing of equations (1) to (2) is executed only once from the basic net 111. At this time, each intermediate layer output y1 is the similarity C between the input initial pattern and the dictionary pattern of each category.
05e1 (Step). Next, using the feedback signal obtained from the input initial pattern X (0), the recognition output f (h(
y)) is calculated (Step 2). The difference between the input initial pattern X(0) and each dictionary pattern
Enter numbers 2 to 113. The auxiliary net for each category has the same configuration as the basic net,
Only the thresholds are different. Each auxiliary net 112 to 113 measures how much the input pattern varies from the dictionary pattern of each category.
It is determined from the projected component (COS θ,) of each category of the difference pattern in the direction of the dictionary pattern. If these values are greater than or equal to the allowable values, it indicates that a pattern outside the allowable range has been entered in a certain category.
The intermediate element corresponding to the non-permissible direction fires (turns on). Similarly, for the auxiliary nets of other categories, it is checked whether there is a firing element that indicates that a pattern outside the permissible range has been input (Step 4). In addition, for fluctuations in the sinusoidal direction, using the degree of similarity COSθ1 to each category obtained in step 1, sinθ
, = -CO5, and check whether this value is below the allowable value for each category. If the allowable value is exceeded, a firing signal similar to that of the auxiliary net is sent to the recognition result determination unit 116 so as not to accept the recognition result of the corresponding category. If there is a firing element from the auxiliary nets 112 to 113 and the permissible variation determination unit 115 in the sinusoidal direction, the recognition output for that category is set to zero (non) by the recognition result determination unit 116.
active) (Step 5), that is, even if the recognition output of the basic net 111 is fired, the recognition output of that category is denied as an integrated result by the recognition result determination unit 116. Finally, it is determined whether only the recognition output of category i of the basic net 111 is 1 (Step 6). As a result of the determination, if only the recognition output of category i is 1, category i is set as the recognition result (Step
7). 5 In the determination of Step 6, if only the recognition output of category i is not 1, it is a misrecognition and the result is rejected (Step 8). FIG. 14 shows the state of the recognition boundary in the case of two dimensions and two categories. For example, in the recognition area of category A, the lower boundary is determined by the basic net, the right boundary is determined by the element in the category B (cos θ8) direction of the auxiliary net of category A, and the upper boundary is determined by the element in the category A (cos θA) direction of the auxiliary net of category A. The boundaries are determined, and the sine direction (sinθA)
The boundary of is determined by the permissible variation in the sinusoidal direction. The same applies to the recognition boundary of category B. Next, with reference to FIG. 13, a method of adjusting the allowable range in the other category direction in the auxiliary net of this embodiment will be explained. First, it is assumed that a prototype pattern of each category is stored so as to react to its input pattern (elements of other categories may also react), and a threshold value indicating its permissible variation range is adjusted (Step O). Next, copy this basic net to create auxiliary nets for each category, and present the input patterns in order (Step
1). It is determined whether recognition outputs of categories other than the presented pattern category are accepted (Step 2). As a result of the judgment, if there is no recognition output for other categories,
Check all sample patterns for misidentification (St
ep 3) N If there are still sample patterns that are misidentified, present the next sample pattern. The process ends when there are no more misidentified sample patterns. 5. If the intermediate element of the basic net is erroneously reacting to another category as a result of the determination in Step 2, the auxiliary net is operated (Step 4). Then, it is checked whether the category misidentified by the basic net is denied by the auxiliary net (step 5). If the auxiliary net is negative, the next pattern is presented without adjusting the auxiliary net, because during recognition processing, even if the basic net incorrectly judges that category, the correct judgment can be made by referring to the auxiliary net. and continue adjusting. 5tep In the judgment in step 5, if the category misidentified by the basic net is not denied by the auxiliary net, the threshold of the intermediate element (corresponding to the category that responds incorrectly) of the auxiliary net for that input category is set by a small amount Δ. and tighten the allowable range of variation for other categories (Step 6). The input pattern for learning is presented again and this operation is repeated until there is no misrecognition. By this, after learning, recognition processing with fewer misrecognitions can be performed even for patterns other than the learning pattern. [Effects of the invention] The present invention approximately realizes a general inverse matrix with high associative ability through local and simple processing, and can also handle deformed patterns using threshold processing and a feedback circuit, allowing pattern recognition with fewer misidentifications. This has the effect of being able to be executed efficiently and learning a recognition dictionary with simple calculations.Furthermore, according to an aspect of the present invention in which an auxiliary net is provided, the permissible range of the distribution of each category is determined, and the recognition Since −
Pattern recognition with less layer misidentification can be efficiently realized.

[Brief explanation of drawings]

FIG. 1 is a diagram showing an orthogonal projection type feedforward associative memory model of the present invention. However, in the figure, only the coupling to the interlayer element of interest is shown. FIG. 2 is a diagram showing the system configuration of the first embodiment of the present invention. FIG. 3 is a diagram showing a conventional orthogonal projection type feedforward associative memory model. FIG. 4 is a diagram showing a conventional orthogonal projection type feedforward associative memory model. FIG. 5 is a diagram showing an example of a nonlinear threshold function for applying threshold processing to the intermediate layer output. FIG. 6 is a follow chart showing an outline of the recognition process of the present invention. FIG. 7 is a flowchart showing an outline of the learning process of the present invention. FIGS. 8(a) to 8(c) are diagrams showing experimental examples of pattern recognition showing the effects of this embodiment. FIGS. 9(a) to 9(C) show experimental examples of pattern recognition according to a conventional example for comparison. FIG. 10 is a diagram showing a second embodiment of the present invention using major classification and detailed classification. FIG. 11 is a diagram showing the system configuration of the third embodiment of the present invention. FIG. 12 is a flowchart showing the procedure of recognition processing in the third embodiment. FIG. 13 is a flowchart showing the procedure of learning processing in the third embodiment. FIG. 14 is a diagram showing an example of forming two categories of identification surfaces of a two-dimensional pattern in the third embodiment. 21... Input register, 22... Clock oscillator,
23.26.29... Product sum calculation section, 24... Differential calculation section, 25... Function calculation section, 27... Nonlinear threshold processing section, 28... Output register.

Claims (2)

[Claims] (1) In an associative memory model that dynamically converts an input pattern into a recognition output pattern using a feedback circuit and acquires a recognition dictionary through learning, it has an intermediate layer using sparse code representation, and is a recognition dictionary. A pattern recognition method characterized by storing information to be stored in connection weights in a distributed manner. (2) An associative memory model that dynamically converts an input pattern into a recognition output pattern using a feedback circuit and acquires a recognition dictionary through learning. 1. A pattern recognition method in which information to be stored in connection weights is stored in a distributed manner, characterized in that an auxiliary net is provided to determine the permissible range of distribution of each recognition category.