JPH02501167A - pattern recognition device - Google Patents

Abstract

PCT No. PCT/GB88/00710 Sec. 371 Date May 9, 1989 Sec. 102(e) Date May 9, 1989 PCT Filed Aug. 26, 1988 PCT Pub. No. WO89/02134 PCT Pub. Date Mar. 9, 1989.A temporal sequence of signal vectors X (e.g., generated from a speech signal) is supplied in parallel to elements in an array each of which (after training of the array) provide an output eta indicating the vector's similarity to a reference vector. Persistence is built into the elements so that signals forming a "trajectory" within the array, corresponding to an input sequence are simultaneously available for recognition (preferably by a further such array). Arrays may be cascaded to allow for longer sequences.

Description

[Detailed description of the invention] pattern recognition device The present invention relates to a pattern recognition method and apparatus, and particularly, but not exclusively, to pattern recognition methods and apparatus. It is about peach recognition.

The invention is specifically concerned with the time series of input vectors to recognize specific patterns. do.

In this specification, an "N-dimensional vector" consists of a group of N values, each of which is is associated with each dimension of the vector. The value may be an analog signal or a digital signal. It can be expressed by digital. Such a vector is, for example, the square of the value It has a magnitude that can be revealed by the square root of the sum, and the direction of N-dimensional space is also that It's good. In this specification, for the sake of brevity, the scalar 41 (excluding N) Vector quantities are represented by high type cases.

Vectors of this type can be specifically derived from analysis of human speech.

Therefore, an analog signal representing a sequence of speech sounds is sampled regularly, and each The content of the sample is e.g. the characteristics related to the amplitude of each frequency within the sample. It can be represented by a vector consisting of a set of values.

Proceedings of’ the 5ixth International nal Conf’erence Pattern Recognition “C Lustering, Taxonomy, and Topological A paper called ``Maps of Patterns'' contains statistics on empirical data. The approach for visual representation is detailed. Set of input data (vector) is connected in parallel to each of many processing units, which can be thought of as forming a two-dimensional array. each unit has a specific input vector associated with that unit. A signal output is generated that is specific to the degree of consistency between successive internal vectors. The conformity principle is input The sequence of input vectors forming an arbitrary representation of the data is a variation of said internal vector. revealed in order to cause a change. this is, (1) The reference vector is the unit most similar to the input (for example, minimum Euclidean interval) (2) identify the neighbors in said array around this unit; west, (3) Change the internal vectors of those units belonging to this neighborhood, and The directions are such that the similarity of their internal vectors is increased (for each input vector).

To continue this “self-organization” process, the size of the neighborhood is gradually reduced. The magnitude of the adjustment is also reduced. At the conclusion of this process, the array The internal vectors reveal the mapping of the input vector space on the two-dimensional space. Ru. Kohoncn (selected to exclude those containing passages) such as an array using manually selected speech samples of the last vowel phoneme that stopped. the input vectors each consist of 15 spectral values, and the input vectors each consist of 15 spectral values; It is found that the phonemes are mapped into a dimensional array space.

Therefore, the purpose of this invention is to Input for receiving time series of input signal vectors and storing multiple reference vectors storage means for receiving the same current input vector and for receiving said input vector; emitting an output signal indicating the degree of similarity between the vector and each of said reference vectors; a plurality of similar comparison elements arranged during operation to generate For each element, compare between its reference vector and at least the previous input vector output modified depending on the output signal produced by that element as a result of a means of generating; the modified vector displaying a relatively high similarity between the input vector and the reference vector; The pattern represented by the output and the reference information are combined into the node represented by the time series. recognition means to confirm the turn An object of the present invention is to provide a pattern recognition device comprising:

This ensures the continuity of the output signal generated for each reference vector. to the sequence of input vectors to be determined relatively easily by incorporating the frequency. Allow buses to pass through the associated "most similar reference vector". Therefore, what With this uniform sampling period, even if the most similar reference vector changes, The output signal of magnitude is at least one of the other reference vectors and the input vector It is also generated as a result of a comparison.

In principle, the recognition means determines identity for stimuli leading to a succession of input vectors. Conventional pattern recognition may be used to determine from said output signal generated. Even if it is possible, it is possible to compare elements like this each with a memorized reference vector. Preferred is a device further comprising at least one further array of Each of the elements has a similar output signal representation between its reference vector and the group of said outputs. Each of the modified (or further modified) outputs of the above array to produce connected to receive this group.

Preferably, the output associated with each reference vector of or one of the high arrays is The group of force signals is determined by the aforementioned The output generated by the above arrangement of arrays centered on the arrangement in the array. derived from the force signal.

In a further method of producing a multilayer, said output signal is one of the signals feeding said array. by at least one of the arrays that is fed back to modify the group. occurs. This type of feedback can be used in speech (such as words, phrases, etc.) (in cases of recognition) allows a way to respond with high glamour. example For example, said feedback signal is concatenated with said input signal to said array. Can be connected.

These are the most similar vectors whose position in the array varies with the magnitude of the input vector. in the subsequent recognition of the sequence of input vectors. It is too complex to ignore. This means that the magnitude of the vector is As intensity is related, a non-negligible problem arises in the case of speech recognition. . In contrast, when the same phoneme is made loud or quiet, this The result is different from the "most similar reference vector". This ordering method is A turn recognition system can handle a wide range of different input vectors generated for the same phoneme. must be included in order to resist severe change.

An important improvement of the known method is that the position in the array of said reference vectors is It means that the vector is most similar to the input vector that does not change with the size of the vector. By comparing the input vector and the reference vector in a state such as can be achieved. In the case of speeches, in this preferred manner , the comparison step determines a function related to the double product of the input vector and the reference vector. It consists of the process of In this case, one of the reference and input vectors (preferably is the reference vector) or both are standardized.

This is because the maximum stimulus of the array simply depends on the type of sound whose relative stimulus is not the amplitude. Therefore, the closest reference (or weight) vector in the same direction as the input vector is It has the importance of being reached from the elements of the file. However, the absolute magnitude of the stimulus The magnitude depends on the magnitude of the input vector provides the desired characteristics of the sound type encoded by the location of the maximum stimulus and loudness. supply.

In a modified form of the double product function, the comparing step includes: It consists of the process of determining Here, m is a number larger than -, ζX is the size of the input vector, and θ is the size of the input vector. It is the angle between the input and the reference vector. In this case, the reference vector is of magnitude − be.

The apparatus is conceptually detailed as including a geometric array of reference vectors. However, as a practical matter, there are some reference vectors that are not necessary in the geometric method. The physical configuration is mapped into the sabotage configuration in a conventional manner.

The term "array" refers to the connections between elements rather than their physical arrangement. It is understood that

Some examples of devices according to the invention will be described in detail with reference to the accompanying drawings.

FIG. 1 is a schematic diagram of a first embodiment of a recognition device according to the present invention; Figure 2 is a diagram showing the principle of neurons used in the device with persistence; FIG. 3 shows a typical trajectory on a single array of the device; Figures 4A and 4B are diagrams showing the forms of soft and hard windows, respectively; FIG. 5 schematically shows an embodiment of a stacked array; FIG. 6 is a diagram schematically showing the internal connections arranged between adjacent arrays of FIG. 5; Figure 7 shows an array distributed into two auxiliary beams to achieve inter-array grammar. A diagram showing an example, FIG. 8 is a diagram showing a second example of an array that achieves inter-array grammar; Figure 9 is a diagram showing the principle of internal layer grammar, and Figure 10 is a diagram showing the specific hardware of the device. Figure 11 and 12 are nerve cells used in the arrangement shown in Figure 10. A diagram showing an embodiment of hybridization of cell elements, FIG. 13 shows a pure analogue embodiment of the neuronal element.

The apparatus shown in FIG. 1 has a front-end speech processing unit 100. ing. For example, the input speech signal S is divided into frames of 10 m5 Each frame generates multiple parameters that represent the characteristics of that frame. be analyzed. One convenient quantification is the continuum within the total spectrum of said speech. It is the amount of energy in a frequency band that is uniform depending on the bank of bandpass filters. or can be easily generated by transfer techniques. Such general characteristics Feature extraction and specific spectral analysis are well-known techniques for speech processing and recognition. It is not necessary to elaborate further here.

The output of said processing unit is a group of N signals (here in analog form). ), and is written by the vector X-(Xl, be done. Typically, N may be 16 or 32. When the vector X is a vector Generated for consecutive frames of a sequence of speech forms.

Another purpose of said device is to analyze this sequence of patterns (and this phoneme, word, etc.) This is for confirmation. Even if it is detailed in the context of a speech It has been identified that recognition of patterns in other time series of vectors is also possible. be done. The device also includes elements or units 101 that limit the two-dimensional array. has. The inputs of the array are applied to all elements of the array simultaneously. Each key point The element has an analog output signal (scalar), which is an index that describes the particular element. is written by η. In general, η represents the degree of similarity between two vectors. The input vector X and the weight vector W, −( W+, , W, □, ... WtN).

Preferably, this function is a double product, i.e., where the points (,) are vector squares. A star (★) is used to indicate a multiplicative product, and a star (★) to display the product of scalar quantities. is used. This function can be realized by elements of the form shown in FIG. , where amplifier AA uses stored weights wl (for example, if a particular element of the weight resistance R3 corresponding to the reciprocal of Forms an output voltage proportional to the sum of the currents at the value. The element shown in Figure 2 The child is what is often called a nerve cell by analogy with a biochemical system, This term is used in this description.

In order for the array to consider the characteristics of the speech data, the auxiliary computer 104 provided that it has access to the neuron's output η and this This is possible in order to change the weight vector W of during training , speech is provided to an input processor and each vector provided to said array is The torque X is processed in this way.

(a) The computer uses the array to identify the neuron with maximum output. Scan the (b) From this output and the existing weight vector W1, that in the array adjusted to create identified neurons adjacent to and their weight vectors. calculate.

In particular, such training may be obtained in the following forms:

(a) First, set the weight vector W1 to a random value, (b) Revealing the neighborhood of the array around each neuron. Said neighborhood - it's a circle A cylinder, a plane, a hexagon, or any other shape may be used for one neuron and many nearby neurons. including; (C) Obtain vector X from the input. Each neuron produces an output η, (d) For example, by scanning the neuron output, one of the maximum outputs (“maximum ``Nerve cells that were also stimulated'' (e) Placed in the center of the most stimulated neuron using the following algorithm: Modify the weight vectors of all neurons in the neighborhood of Wl　’　　−Wl　”　＋＊X Here, W,0+1 is the new weight, W,″ is the previous weight, and k is the matching ratio. is the gain function to be determined, and W−′ is a simplified intermediate term in the above calculation. , (e) further repeat steps c, d and e for vector X.

Modification of this process is necessary for neurons within the surrounding neighborhood of every neuron. It is necessary to clearly identify the most stimulated neurons by providing corrections that Neuronal output that avoids sex but reveals neighborhoods - e.g. increased to high power m The amount of adjustment is changed according to a non-linear function of the output −.

Therefore, when η,′−η1′, the resultant The adjustment of neuron i to . To avoid confusion, the superscripts of symbols are used here to indicate consecutive values. (i.e. w”−” is the previous value of W”), and here we increase the power. This means the amount by which the quantity is measured, which is specifically indicated by a sign.

As Kohonen's work, we clarify the neighborhood in the rate constant. It can be a function of the spacing of the neuron in question from the neuron, and it is It is preferable to reduce the size of the neighborhood as a buffering improvement.

Standardizing the size of the weight vector for − is an important step in this process. , the magnitude of the other weight vectors increases indefinitely, or (as a practical matter) saturates. It should be noted that there is a tendency to However, standardization also serves another purpose. do. The double product W,,X can be written as follows.

IW, 1★IXI★eOsθ Here, θ is the angle between two vectors in N-dimensional space. Wi is - large When normalized to has. This is the product of the input and the term (COSθ) that indicates the similarity of the weight vectors. , and the term (IXI) represents the input amplitude, regardless of their absolute magnitude. Nerve cell When the output of the array of is explained by considering these relative values, the result is Since the output IXI is the same for all neurons, it is independent of the input signal amplitude. It becomes something.

After the array has been continuously training, the array has been trained in terms of sound quality. and the method for adjusting the input signal vector by the array is such that the input vector and Each neuron generates an output signal of stimulation intensity that varies with the similarity between the corresponding weight vectors. Result in cells. For any given input vector, the neural Only small groups of cells produce output signals of any significant magnitude, and these They are considered to be the same group within the lei. the array can be viewed from above; Moreover, if the output of neurons is seen as light with variable intensity, the application of a single human power vector becomes Part of an array formed by output signals from grouped neuronal elements A bright “vrasov” of minutes occurs, and the position of this vrasov is represented by the input vector. It is a characteristic of the type of sound being played.

Furthermore, the position of the array of "most stimulated neurons" for a given input vector The relationship between scientifically (perhaps surprisingly) there is a tendency to map (where N is the "unique dimension" of the input data). For example, the vowel seems to be better represented by a two-dimensional space, and therefore these parts of the speech data type has two intrinsic dimensions and maps well onto a two-dimensional neuronal array. It will be done.

The physical location of neurons in the array is not important; Note that the neurons are indistinguishable in the array without be done. ``Position'' becomes meaningful only by the definition of ``neighboring'' neurons.

The recognition device may include an auxiliary computer for training or retraining. , but if it can be loaded with the already memorized weight by another machine, then this May lack such a computer.

The essence of the sound of speech produced over extreme conditions does not change instantaneously. However, Therefore, continuous speech samples obtained at intervals of a few milliseconds (for example, 16 milliseconds) The vector representing the point will be very small, then the position will be determined by only a small amount. It is fed to the array 1 which tends to produce high power "Bragg" due to variations.

As a result, the application of a sequence of input vectors representing a complete utterance is taken as a characteristic of said utterance. Bragg occurs because the array moves along the trajectory of the array. like this An example of a trajectory is shown by the "Bragg" column in array 1 of FIG.

To obtain continuous trajectories, compare the rate of change in the speech parameters that these represent. The vector is generated at a rate representing oversampling. It becomes what you want.

(i.e., changes in proportion), and conventional pattern recognition techniques can be used to determine from the trajectory the essence of (for example the intensity along the trajectory Even though the aspects are related to time distortion and noise). Figure 1 shows such a recognition unit. A cut 110 is shown.

Such arrays, such as that of Kohonen, are often referred to as "firing" To date, only a distinct continuum of high-output neurons, called neurotransmitters, have been generated. However However, it is important to note that the use of firing terms does not necessarily imply a threshold process. intended.

Firing does not coexist as it is applied to many consecutive input vectors, and the trajectory The traces are not entirely clear. However, if the trajectory is If this is not the case, then a single moment in time should be clarified as a whole. Before When this system is obtained in continuous speech, the trajectory of the firing is It is what Peach has been doing, and it will eventually become infinite in length. clear , it becomes unsuitable to hold the entire arbitrarily long trajectory, and the recognition Create it as an input to the process and the trajectory will be its maximum as shown in Figure 3. Also, only recent segments are visible through the window at the time they pass.

It is important to note that this window produces issues and obvious trajectories at a single instant in time. Creating complete parts of the trace at the same time also helps build persistence in each neuron. i.e. once a neuron is stimulated by an input, it After that continue to produce an output of equal decay magnitude. Generally, the The number may be an exponential, linear, or any other function that approaches zero monotonically over time. Before If the decay is exponential, then the system will have a “soft win” as shown in (Compare with the hard window in Figure 5).

The persistence function is revealed in the following way.

e, ’ am 5, ”-1★β+η1゜Here, e and M are nlk samples iIh neuron stimulation in hours, β determines the constant effective time of decay. Ru. This is due to the presence of the R-C integrator 115 between the neuron output and the recognition unit 110. is shown in Figure 2. In one experiment, a persistence function β of 0.99 and approximately A sampling period of 100S is used, yielding a constant decay time of 1 second.

Even if the arrangement shown in Figure 1 is entirely possible, the A preferred method provides multiple arrays, as illustrated by the example in FIG. The second layer 2 of p is arranged continuously in the first layer 1. nervous system The concept behind the stack of cell arrays is to use that layer as input for the second layer. is intended to use the trajectories generated above and therefore for small groups of neurons. A trajectory on a high layer shortened by a loop or something that can just be related to it. Traces occur through the layers of the stack until a word or series of equivalent words occurs within it. Then, a more compressed trajectory is generated. The final recognition stage 110' has a maximum output to reveal one or small arranged group of neurons with Give easily. These neurons can be clearly classified with appropriate speech.

It is a window of trajectories on one array to neurons on the next array. It is necessary to devise a schematic diagram to represent the parts.

One obvious interpretation is that the output of every neuron in one layer yields a very large base. In addition to creating a vector, it also serves as an input to every neuron in the next layer. It is meant to represent.

However, due to internal layer connections, this arrangement is incorrect on two counts. It is. First, these can be any number of neurons in the Pin; The increase given to the input vector to the original layer n+1 neurons is - this requires around 1 billion internal layer connections, which is not easily physically possible. It doesn't become Noh.

The second reason to avoid this schematic is to separate the firing ring from the overall housing. It is almost certain that the pattern has two or more unique dimensions, so the two In order to map to them in an organized manner on the next layer only with dimensions This means that it becomes possible.

An interpretation that avoids these problems is shown in FIG. Here we have the coordinates (x, y) In this case, the input vector to the neurons in layer 3 is The annular extent of layer 4 located within the small circle is very small, and layer 3 After training the array in a similar way to the array with spherical input vectors organized, except for those that are localized rather than spherical.

This connection strategy overcomes the problem of multiple internal connections. Layer 3 is the layer mentioned above. containing 10,000 neurons each connected to 100 neuronal outputs of , 106 internal connections are required.

The developed neuronal array favors the collapse persistence built into each element. This is the trajectory of the stimulated element to be partially "suppressed" on the array. occurs, and at some instant in time the most recently stimulated element of said trajectory is the most stimulated. , all other elements along the trajectory are stimulated with decreasing intensity. Therefore, detailed The system applies a variable time distortion to a single pattern of elemental stimulation of a neuronal array. It is possible to transfer a sequence of pattern vectors representing the process involved. . This single pattern becomes irrelevant when stimulus intensity aspects along the array are ignored. This results in serious time distortion.

For example, the pattern of stimuli in the array can be used as a vector in many other ways. can be expressed as the most intense in the array of some fixed number 2 can be the vector of the element whose coordinate 2 is the stimulated element. Therefore one layer of God Another way of input to the transcellular element is to divide it from the output of the previous layer, as follows: It can be done.

The output of every neuron in the lower layer is the stimulated element of said lower layer that produces the output A threshold having such a value is defined only for those elements that are approximately located in the path of the trajectory of It will be done. Alternatively, another form of non-linearity acts on the outputs of all elements of the lower layer. created to do. For example, the output of each element can rise to power m where m is 1 or more.

This type of pronounced non-linearity of the element output is not located near the trajectory, but near the trajectory. has the result of emphasizing the stimulus differences between elements located in .

Now consider the required persistence of recognition in two arrays.

If the time constant is long, for example a few seconds, the first layer trajectory will be treated with a few words.

The number of permutations of uniform 2.3 words obtained from a moderately large number of terms is huge, and these become a huge number of different trajectories, each of which leads to the next layer. is required to produce strong firing of distinct neurons in the . next The large number of neurons required in the layer is produced either artificially or biochemically. It becomes something that cannot be expressed.

At the other extreme, the time constant is In the second layer only the ring was created for a very short time. Uncompression of the trajectory is based on the firing neuron coordinates of the first layer - 1:1 reconstitution. By simply recording the sequence of speech input vectors as an information sequence, It becomes possible.

A practical choice of time constant is approximately the length of the shortest word encountered at about 200 m5. It seems to be the same. The result of this selection is that the shortest word is zero length. resulting in a trajectory on the second layer close to , i.e. only one small group of neurons The purpose is to generate the trajectory by firing the tap. clear An alternative method is to build persistence during neuronal input in the second layer. It becomes something to save.

If these are three layers, it represents a longer-lasting effect on those of the first layer. This is useful for building during the output of the second layer.

In a further embodiment, a solid concept of grammar is built into the system.

One way to devise to achieve this is because some trajectories occur more than others. We recognize that these are more favorable due to cross-linking within said layers and therefore The locus is shifted toward a higher locus. At least the first part of FIG. Separate layer 1 of. The first part 5 (Figure 7) is just a standard application with persistence. ray, the ilb neural cell in this system when sampled at n′. The output of the cell is e, ″.

eIt'-η1°+el'-'★β...The array of nodes 9 to be added is weighted. output el, .

e for each neuron, and the output of the neurons in this array is the total q neurons within the neighborhood of neurons in the previous array to give the neurons of the By storing the factors of α, , in the second part 6 of the cell (multiplier 7) is given after the weight, The stimuli for e and 'n are as follows.

Here, e I, l is the stimulation of jtb, one of the neighboring neurons of q.

Factor α2. , guides the utterance trajectory along a preferred path in the array. These paths consist of grammar weights that tend to be These are the ones that were made.

This therefore creates a path shape that is less dependent on listener changes or background noise. Ru. Conceptually, the first part 6 is a "ganglion" nerve cell. While it can be thought of as a ray, the geometrical order of "receptor neurons" You can think of it as an attached array. The second part 6 is standard sense. It should be noted that they are not ordered. In Figure 7 It is noted that eI,'' is used rather than e, , n-1. be.

Whether this is possible in practice depends on the latency of the system.

The benefit of this arrangement is that it allows an aiding system for remembering early grammar. It is to be.

The system operates as follows. Assuming α for all weight moments Then, the distance between the ganglion and the receptor becomes zero.

A common sequence of stimulation of the receptor array 5 is the neuron R2 (R is the receptor, G is the ganglion). ) is the neuron R1. followed by R2 of the receptor array Stimulation of R1 results in stimulation of nodes G1 and G2 of ganglion array 6. Receptor area The second input pattern for A5 is that neuron R4 is used as a fiery cell instead of R2. When transformed in some way, such as by stimulated accordingly.

Therefore, the system is powerless to resist incoming turn conversion or noise. be.

Now, it is assumed that α and 5□ are the maximum values.

Something happens when the second converted input cover turn is now fed into the receptor array 5. Considering that it is the nerve that tends to innervate node G4 of ganglion array 6, Stimulate cell R4. However, the previously stimulated receptor neuron R1 persists To the nerve cell G2 that is still stimulated to be sexually active and which is generated in order to be stimulated. Then, a strong signal is sent via G1 and G2.

Therefore, the entire sequence of stimuli in the ganglionic layer is caused by a transformed but uniform G2 input. So I headed towards G1, who was following me.

The acceptors for the ganglion connection weights can be memorized from the application of a large number of pattern sequences. Ru. A typical knowledge algorithm can be as follows.

α+, r’-α+, r’-1★70 to 2★e Ie*e+, 4 n-' where, n is the sample time index, γ is the forgetting function, R2 is the knowledge function, e, and e,1 as revealed above.

This applies to the same auxiliary computer processing the weight vector W, and a similar approach. can be more fulfilled.

An alternative embodiment with means for guiding the grammar in said system is as follows: This is shown schematically in FIG. In this case, neurons 1+J in each array are are combined by weights G1.8, G3.1 with surrounding neighbors. Set the value α appropriately. By determining the trajectory within a particular layer, a moderate stimulation of a neuron can be When processing signals, there is a great deal of weight placed on neighboring neurons that respond to those signals. The selection of An alternative path results.

A representative knowledge algorithm for this system is shown.

α17″+1−G1.’★γ+2★w,,x’−’★w,,xkHere, γ, R2 is as described above, n is the update rate of the weighting function α, and k is the input vector This is the update rate of torque X.

In many cases, the trajectories produced by first layer utterances are essentially continuous. Become. However, not all trajectories are fully continuous; instead, consists of a series of relatively short segments separated by For example, this is the vowel - stop consonant - vowel (V, -CV2) in the first layer so that it receives the utterance May occur.

These do not result in neuronal firing during uniform cessation by the definition of silence. . The two vowels on either side of the stop are generally not phonetically adjustable, so this The trajectories in which they occur are not adjacent on the array. However, in certain languages The permissible Vl-C-V2 sequence in is restricted, and the trajectory associated with (Vl-C) The occurrence of is located on the trajectory for (C-V2) and other neuronal groups The nerve cells that make them more suitable to be stimulated are used “first”. It is desirable to take advantage of this coercion for selfish purposes, such as being forced to do so.

The schematic is shown in Figure 9, where the spherical firing pattern of the tall layer 9 is along the force vector as an input vector to layer 8. is expressed in The above outline is based on the trajectory of (V+C) and (CV2) in the following method. Operate. The generation of V, -C trajectories on the layer 8 is caused by layer 9, resulting in a compressed trajectory. This compressed trajectory is a vector F is encoded in layer F, where F is a feed with delay for the input of layer 8. back and form all the inputs in layer 9 of (X:F)-Q. It is added to the force vector X. In Figure 9, the delay 10 is the system Although shown as a separate element, it is actually an absolute and predetermined component of the neuronal array itself. It is thought of. A feedback delay is applied to the input of layer 8 at the same time as the stop time between Vl and V2. actually receive it. Layer 8 Self-Organization Takes Place in Real Feedback Pathways Then, its existence is necessary to cause the trajectory (C-V2) to occur. should be the input component, i.e. with a feedback vector without substance. The input vector Only insufficient stimulation of (CV2) results in a negative input vector. However However, in reality, the occurrence of junction between feedback F and input Therefore, this should be the input of the system that has been trained. Ru.

Because recognition of a particular word recognizes the initial sound associated with the expected next word. This feedback also has the effect of pre-treating this lower layer.

The intention of this trend to promote incomplete recognition reveals the input pattern .

In an ideal system, the output from each layer would serve as input to every previous layer. Feedback will be given. However, in reality, this does not apply to each neuron. This leads to a large number of connections and inputs that are overall unwieldy. This ideal arrangement In order to achieve closeness, in a preferred example, each output signal from each of the first layers described above is In whole or in part, the output of a higher layer indirectly influences the input to all previous layers. It is returned to the aforementioned layer for receiving.

The operation of the system consists of a high layer of recognized words and a low layer of recognized phonemes. It can be understood by considering. Certain words are It can be said that it has been properly recognized. This information is fed back and At the same time, the input is added to the phoneme layer as a pattern corresponding to the phoneme of the next word. parts are formed.

The phoneme layer is a combination of a feedback pattern and an introductory pattern. sufficient of the phoneme array neurons trained and associated with the second word phoneme. It is necessary to produce a stimulus.

Now, the second word is heavily inverted or input to said phoneme layer. Assume that you have not even reached it.

This layer further receives partial input from the pattern returning from the word layer and Once this has been achieved, the sequence of phonemes in the second word will be sufficiently stimulated. It tends to stimulate the nerve cells that are exposed to the body. Therefore, it does not exist or has been transformed. An insufficient stimulus representing the second word is generated. Error correction using knowledge of something This form of positive recognition of patterns aids in the recognition of poorly revealed patterns. Preferably occurs in series.

These methods eliminate speeds by using parallel process arrays. cannot be satisfied in real time for processes such as process recognition.

FIG. 10 shows the architecture of a neuron array structure having m neurons or computational elements N1. architecture, each associated with a respective local RAM 20. . Each RAM stores synaptic weights for associated neuronal elements.

The arrangement shown in FIG. It should be noted that allowing the output signal from each neuronal element of the array Ru.

In FIG. 10, a portion of the previous array 22 and the next array 23 are shown, as well as one Two arrays 21 are fully shown.

Before explaining what the system operation is, let's take a look at the required outputs of layer 21. Reveal power.

Here, X is the input vector from layer 24 and below (or the input vector for the first layer) ), y is the feedback vector from above layer 23, and y is the feedback vector from above layer 23, Then, β, Wl, and C1r are the persistent weights and fields included in the local RA M, respectively. is the weight of forward and feedback synapses.

The operation of the system is as follows. Neurons from N1 to Nff1 are , these are stored in respective accumulators of neuronal elements (not shown). Assume that you have just finished calculating the respective outputs η,°−1 to η,°−1, and switch It is assumed that the switch S2 is located and set. The switches S31 to Sgm are Since the neuron output is loaded into the storage element (T) of the output bus 24 of the layer 21, the position It is set to position a. At the same time, switches S11 to Sea+ switch the output of each neuron. η1°−1 is connected to its input along a persistent path 25, and said accumulation is set to position for the rater to be revealed. At this time, all of the neurons Then we start computing the first component of these outputs - the persistence term one, which is That for transneumnium N, is ηn-1★β. Therefore, each nerve cell The first term of the above equation, which is stored in the numulator, is just calculated. β is associated with each neuron obtained from the RAM 20.

Appropriate synaptic weights W1□ to Wm can be obtained from the local RAM of each neuron. On the other hand, the first value x1 stored in the feedforward bus of layer 22 is simultaneously transferred to layer 21. switch S31 of all arrays so that the inputs of all neurons in , Sl, , , move to the current position, and switch Sl is moved from Stt to the current position. is switched to a. (At the same time, the first value of the feedford bus in layer 21 is via a stack of arrays to the neuronal inputs such as layer 23).

At this point, the next component of each neuron is calculated inside said neuron, which is Xl , Wll, and is appended to the contents of the accumulator.

The next value x2 from said array is supplied to all neurons of the next array, and The output of a portion of X2. W1□ is the front part of each neuron's accumulator feedforward bus 2 of each array to be calculated and added to the output of 4, "clocked" along one location. This process This continues until the feedforward bus is completely cycled.

Then switch S2 is moved to position a and said process is returned as input y Iterated from l to y. These are the higher layers along the feedback path 26. This is the value fed back from the output bus.

Upon completion of this process, each neuron has a new output value stored in the accumulator. Calculate ηl″ and start the whole process again.

The non-linear processor 27 that follows the output bus 24 of each array is optional.

If included, the function would be to establish a threshold for the values stored on said bus. something that enhances or enhances power.

The system shown in FIG. 10 uses, for example, all It can be realized digitally using synaptic weights, and neuron output and input signals can be realized digitally. The number is also represented by an 8-bit word. The output bus, feedback path and circulation of persistent pathway data to minimize the amount of hardware associated with each neuron. Everything is done sequentially to minimize the number of internal connections between neurons.

Therefore, each storage element T on output bus 24 is an 8-bit 5ISO shift register. Consists of.

Figure 10 shows that the outputs of all neurons in the array form the inputs for the next layer. Assume representatively. As mentioned above, this is often unrealistic, but the first The architecture of Figure 0 separates the buses commensurate with the local arrangement shown in Figure 4. can be easily modified by connecting groups of neurons to .

Figure 11 shows an analog of the computational elements that can be used in the arrangement of Figure 10. /Digital hybrid implementation. In this case, the weights of all synapses are The information is digitally recorded in the RAki 20 addressed from the control circuit 30 to the bus 31. be done. The digital value from RAM20 is for each switch S++ 51m. The analog input signals from the multipliers are received via input line 33. The signal is supplied to a analog converter (DAC) 32. The operation of the DAC32 is It is controlled by the tarlock signal generated by the control circuit 30, and the control circuit 30 is fed along the inlet 34. D A C32 is, for example, switched R-2 It can be realized by an R ladder or a ramp comparator. In this case, the above The accumulator 35 associated with the calculation element is formed by an R-C leakage integrator. be done.

In this example, each storage element T of the serial bus is supplied from a control circuit 30. A single stage clocked analog shift register for a clock signal becomes more realistic.

This shift register can be a charge-coupled device, a bucket relay line, or a sample hoop. It can be a field circuit. In addition, all signal values in this system are converted to analog voltages. The feedback path in pulse amplitude modulation and serial The output bus 24 is calculated.

FIG. 12 shows switch 3 controlled by control circuit 30 via control line 37. of the accumulator 35 formed by an analog integrator resettable at 6 By way of example, a modification of FIG. 11 is shown.

FIG. 13 shows an embodiment of a pure analogue of neuronal elements. In this opinion Therefore, the serial bus structure of FIG. 10 is abandoned. The output of neurons in one layer is is assumed to be physically connected to the neuronal inputs of the layers. of said stack The voltage representing each element of the pattern vector applied to the bottom array is It is supplied in parallel to the input of each neuron, that is, the pattern vector is 3 If the bottom array has 30 parallel inputs, then each neuron has 30 parallel inputs. It becomes something you have to do. The analog circuit configuration is an auxiliary combination. Enables synaptic weights to be updated in real time without using data. Before This system will be completely independent.

The signal inputs to the nerve cells are xl...xN.

These voltages flow to the summing point P, resulting in a current 11 to IN. of each current The size is IH-x4/Ros+ and here RD5. is the drain-source resistance of transistor T2j. Seki The number 1/RDs, therefore, effectively weights the j@th synapse of the neuron. , and the total current at the summing point P is therefore characteristic of the output of the neuron.

(This total current is passed through resistor R0 and converted to voltage).

The magnitude of the resistance RDSI is T1. A motor controlled by a current flowing through the Transistor T2. controlled by the voltage supplied to the gate of T 1. The current through is in the turn supplied from the signal input voltage X, via To. depends on the amount of charge stored on its gate.

Therefore, if X is periodically large, T1. The gate of T2. noge There is an increased current flow i, which increases the drain voltage of this transistor. decreases the source-to-source resistance, which increases the effective value of the weight of the j-th synapse. do. For all neurons, the weight vector of the synapse ( After time Δt+ to become W (t + Δ t) − W (t) + L Δ t will be corrected.

Now, according to the arranged algorithm of said array, those neurons are simply The synaptic weight vector is set near the most strongly stimulated neuron to be modified. It's next door. This is the gate that controls the flow charge on the gate of Tll. ) to control the voltage of T31, the output of adjacent neurons of said array (i.e. (i.e., those in the vicinity of the neuron) and the output of the neuron (hydration unit A) By using the sum of T1. to control the flow of current to the gate of This is achieved with this circuit. In contrast, the function in the previous equation is is the total number of all nerve cells.

The second point to consider is that the magnitude of the synaptic weight vector is always uniform. If retained, a well-organized algorithm is to specify only the effects. child This produces an analog voltage by drawing IN from the current 11 from one current source. This is accomplished in a flow, and I becomes like this.

The weight of the synapse is proportional to I, which means that the weight vector of The size is in C1ty Block sense rather than Euclidean sense. This means that the value is constant. However, the simulation is satisfactory As shown here.

A number of knowledge processes have been mentioned above. Nevertheless, in principle it can be enabled to run at least some of them at the same time, and In this organized approach, the weight W of the neuron is equal to the grammar weight α (if not used). (e.g.) It has been observed that and each layer is trained before higher layers are trained. follow It is also possible to have such a series.

(a) Remember the weight Wl of the first layer.

(b) Remember the weight αI of the first layer.

(C) Remember the weight W of the second layer.

(d) Remember the weight α1 of the second layer.

(e) feedback between the second and first layers (if feedback is provided); Memorize the weight C.

(f) Lasts for the third and subsequent calendar months.

Ohtake's Fire Engine (Sushishi←NinlS7ta) , ha international search report 1++1mjll@R甐C＾輾-”””’PCT/GB88100710+-den ---! 1-m^5eh--+-x-, PCT/GB88100710 International Search report

Claims (18)

[Claims] 1. An input to receive a time series of input signal vectors and multiple reference vectors. storage means for storing and for receiving the same current input vector; an output signal indicating the degree of similarity between the input vector and each of said reference vectors; multiple similar comparison elements arranged during operation to generate For each element, compare between its reference vector and at least the previous input vector output modified depending on the output signal produced by that element as a result of a means of generating; the modified vector displaying a relatively high similarity between the input vector and the reference vector; The pattern represented by the output and the reference information are combined into the pattern represented by the time series. Recognition means for confirming turns and A pattern recognition device comprising: 2. The modifying means is configured such that the output is a current output of the element and a plurality of initial outputs of the element. The initial output and the current output are arrayed to generate for each element which is the sum of the forces Claims each multiplied by a value that is a monotonically decreasing function of the time elapsed between the forces 1. The pattern recognition device according to 1. 3. The function is a decay index, and the modified signal is e1n=e1n-1★β+η1 n where e1n and η1n are n1n samples, respectively. is the modified and unmodified output of the i1n element in time, β is a decaying constant 3. The pattern recognition device according to claim 2, wherein the pattern recognition device determines the validity time of the pattern recognition device. 4. The storage means is generated by supplying a training sequence of signal vectors to said input. contains the reference vector, and for each input vector, Check the element with the output displaying the maximum similarity between the input and reference vectors death, In the conceptual array space, let the weights and the similarity between the input signal vector be Adjustment of sense such as to increase exists within a limited neighborhood of said element supplying those elements and the reference vector of the identified element; This allows the reference vector to be a geometric map of the statistics of the training sequence. 4. The pattern recognition device according to claim 1, 2 or 3. 5. For each input vector, Check the element with the output displaying the maximum similarity between the input and reference vectors and between its weight and the input signal vector in the conceptual array space. Modified neighbors of said elements such that sense adjustments are made to increase said similarity of said elements. and those elements present in the reference vector of the identified element. applying a training sequence of signal vectors to said input to perform control means arranged in operation between the This allows the reference vector to be a geometric map of the statistics of the training sequence. 4. The pattern recognition device according to claim 1, 2 or 3. 6. The sum of the modified outputs of the element and the weight of the modified outputs of other elements. includes means for producing for each element a further modified output that is the found sum. A pattern recognition device according to any one of the preceding claims. 7. The weights using the shape of the weighted sum include a signal vector at the input. training sequences for each element and each of the other elements. The value of the weight to be used between 7. The effect of claim 6 is generated by iterative adjustment by a quantity dependent on the product of forces. pattern recognition device. 8. Each element and the above by a quantity that depends on the production of the output produced by the two elements. Iteratively adjust the weight values for use with each other element. During the application of the training sequence of signal vectors to the input, the 7. The pattern recognition apparatus according to claim 6, further comprising a control means for controlling the pattern recognition apparatus. 9. The modified or further modified outputs are lower compared to similar displays. The foregoing claim is directed to a non-linear operation to increase the value of a higher similar representation compared to The pattern recognition device according to any one of the above. 10. The recognition means comprises at least one reference vector having a respective stored reference vector. a further array of 3 comparison elements, each of which has its reference vector and previous said or previous generating an output signal indicative of a similarity between groups of outputs. each group of said modified (or further modified) outputs of said array; pattern recognition according to any one of the preceding claims, connected to receive the recognition device. 11. the input signal vector or group of signals feeding the previous array; the output signal generated by one or more of the arrays to modify the output signal; 11. The pattern recognition device according to claim 10, wherein the pattern recognition device is arranged for feedback. 12. The reference vector varies depending on the input vector and the magnitude of the input vector. said element that produces an output displaying the maximum similarity between it and its reference vector that does not Pattern recognition according to any one of the preceding claims, wherein the pattern recognition is standardized such as the identity of Device. 13. Each element is the double product of the respective reference vector and the input vector. A claim for producing an output that is or is a function of 13. The pattern recognition device according to 12. 14. multiple input signals, each having an input for receiving a time series of input signal vectors and an output. input of the other comparison element to receive the number comparison element and the current input vector, respectively. An array with multiple inputs connected in parallel with a force and multiple similar bases. It is for storing quasi-vectors, and each comparison element is the input vector and the reference to produce at its output a signal indicating the similarity between each one of the vectors. storage means arranged during operation for the purpose of The output is the input vector and its reference which does not change with the magnitude of the input vector. The identity of the elements will produce an output that displays the maximum similarity between the vector and is the double product of the reference vector and the input and reference vectors standardized to A pattern recognition device which is a function of or a function thereof. 15. Each element is for receiving an N-dimensional signal vector, input and a variable conductive path between the input and a current summing node for forming an output; The conductivity of the path depends on the amount of charge stored in the gate of the second transistor. Consists of insulated gate field effect transistors arranged to control electrical conductivity. a memory transistor, adjusting the stored charge depending on the output of the element and the output of neighboring elements; means and The pattern recognition method according to claim 13 or 14, comprising a plurality of N sections each having recognition device. 16. The conductive path is further formed by a transistor and is connected to the storage transistor. The star injects a single constant current into the array to control the conductivity of the further transistors. 16. A pattern recognition device according to claim 15 connected to supply said current ratio from a source. recognition device. 17. The speech input is connected to a plurality of parameters forming the components of the signal vector. the said claim comprising speech processing means operable to generate from each of the following periods; The speech recognizer according to any one of paragraphs. 18. The duration of the period of the speech is greater than the rate of change of the parameter. as claimed in claim 17, such as for representing oversampling of a channel. Pattern recognition device.