JP4539221B2 - Mutual associative memory and its software - Google Patents

Description

  The present invention relates to image recognition having a noise removal and restoration function for degraded images, various pattern recognitions, security systems such as unauthorized access prevention, or a mutual association using a multilayer neural network applicable to high-speed memory search. (Hetero-Associative) relates to a storage device.

  Types of associative memories using neural networks include self-associative memories and mutual (hetero) associative memories. Details are described in literature, Hopfield, JJ, Proc. Nationl. Acad. Sci. USA, 79, pp. 2554-2558, 1982, Associative Neural Memories Theory and Implementation Edited by Mohamad H. Hassoun OXFORD University Press 1993, etc. Yes.

  FIG. 2 shows a configuration example of a self-associative memory using one neural network as a conventional technique. A two-layer neural network composed only of an input layer and an output layer, a three-layer neural network (hourglass neural network), or an interconnection network type neural network known as a hop field network is used. These are configured such that the output signal of the self-associative neural network is directly fed back to the input.

The operation of the self-associative memory according to the prior art shown in FIG. 2 will be briefly described in the case where a two-layer neural network of the self-associative neural network 22 is used.
First, under the control of the associative loop control processing unit 20, an associative input signal from the input terminal 29 is input to the self-associative neural network 22 via the changeover switch 10 and the terminal 30. When the output layer output signal of the learned two-layer neural network as the self-associative neural network 22 is obtained at the output terminal 31, it is transmitted via the input changeover switch 10 under the control of the associative loop control processing unit 20. Feedback is provided to the input terminal 30 of the two-layer neural network. For this reason, the number of input layer units is the same as the number of output layer units. Therefore, the learning input data (learning attractor) and the teacher signal are the same and are learned. In this case, the weighting factor is often obtained by calculation.

The associative loop control processing unit 20 feeds back the output signal from the self-associative neural network 22 to the self-associative neural network 22 through the input changeover switch 10 and the input terminal 30 and obtains the output signal from the self-associative neural network 22 again. . This series of feedback processing is called associative loop processing, and the number of feedbacks is defined as the number of associative loops n.
Here, the input signal to the input layer of the self-associative neural network 22 at the input terminal 30 in the associative loop count n is X (n), the output signal of the self-associative neural network 22 is X (n + 1), In the associative loop processing of the associative loop count n + 1, X (n + 1) is fed back to the input terminal 30 again.

When this associative loop processing is repeated, the output signal is the same as the previous associative loop processing, and the state that does not change is called the attractor convergence state, the output signal at that time is called the attractor, and is output as the attractor output signal Obtained from terminal 31.
The attractor that can be converged may be a learning attractor (teacher signal) that is learning input data having characteristics closest to the associative input signal, or may be a spurious attractor (false attractor). Spurious attractors are not learning input data, and in most cases are composed of data containing a lot of noise and do not have clear features.

  A series of operations from when the associative input signal of the input terminal 29 is input to the attractor convergence state will be described below. First, the input changeover switch 10 is initially set (when n = 0) by the associative loop control from the associative loop control processing unit 20, and the input terminal 29 and the input terminal 30 are connected. Also, the internal state of the attractor convergence state identification processing unit 21 is reset. The associative input signal X (0) at the input terminal 29 is input to the self-associative neural network 22 as an input signal. An output signal X (1) is obtained from the self-associative neural network 22 and is stored in the attractor convergence state identification processing unit 21 for convergence identification processing for identifying whether the output signal X (1) has converged to attractor. Is done.

  Furthermore, the associative loop control processing unit 20 sets the number of associative loops n = 1, switches the input changeover switch 10 and feeds the output signal X (1) in order to feed back the output signal X (1) and input it to the self-associative neural network 22. 1) is fed back to the self-associative neural network 22 via the input terminal 30 to obtain a new output signal X (2) and input to the attractor convergence state identification processing unit 21. Such an associative loop process is repeated.

In the attractor convergence state identification processing unit 21, the output signal X (n) of the self-associative neural network 22 at the accumulated associative loop count n and the output of the associative neural network obtained by the associative loop count n + 1 associative loop processing. Compared with the signal X (n + 1), coincidence is detected. That is, generally, coincidence detection of X (n) and X (n + 1) is performed. However, n ≧ 0.
If they match, it is assumed that the attractor has converged, and an attractor convergence identification signal is sent to the output terminal 32, and an associative loop process completion signal is sent to the associative loop control processing unit 20 to stop the associative loop process. The output signal of the self-associative neural network 22 at this time is sent to the output terminal 31 as an attractor output signal. The state converged on this attractor is called a recall state.

If they do not match, it is considered that they have not converged, and an associative loop processing request signal is sent to request the associative loop control processing unit 20 again for associative loop processing.
When the associative loop control processing unit 20 receives the associative loop processing request signal, the associative loop number n = n + 1 is executed again if the associative loop processing number n is smaller than the maximum associative loop number N specified in advance. The output signal X (n) is fed back to the input terminal 30 via the input changeover switch 10. On the other hand, when the associative loop process completion signal is received or when the associative loop count n becomes equal to or greater than the maximum associative loop count N, the associative loop process is terminated.

When the associative loop processing is finished in the state where the number of associative loops n = N and the associative loop processing request signal is received from the attractor convergence state identification processing unit 21, the attractor divergence identification signal transmission request signal is transmitted to the attractor convergence state identification processing unit. Send to 21. At this time, the attractor convergence state identification processing unit 21 sends an attractor divergence identification signal to the output terminal 32.
In this state, even if the number of associative loops n is the specified maximum allowable number N, the attractor convergence state identification processing unit 21 can obtain the coincidence state of X (n) and X (n + 1). Means no state. This attractor divergence identification state is called a recollection state.

The associative loop control processing unit 20 initializes the input selector switch 10 and the attractor convergence state identification processing unit 21 every time a new associative input signal X (0) is input to the input terminal 29, and the series of the above-described series. The process is repeated for a new associative input signal X (0).
The self-associative neural network 22 is a learned neural network in which a weighting factor is set in advance. In the weighting factor learning process, an associative input signal given as a learning attractor at the input terminal 29 is the same as its output signal. The weighting coefficient is obtained and set based on the relationship that becomes the learning attractor. The details of how to obtain this weighting factor are described in the above-mentioned document.

  If the attractor output signal obtained from the output terminal 31 is learning input data (learning attractor), the application processing system connected thereafter using this learning attractor can be operated correctly, and there is no problem. However, in the case of a fake attractor called a spurious attractor, which will be described later, it causes a malfunction in the application processing system.

  When the associative input signal at the input terminal 29 is different from the learning attractor, an associative input signal similar to the learning attractor may reach the attractor convergence state through a plurality of associative loop processes and be recalled. For example, an associative loop process may be repeated for an unknown associative input signal including noise or the like to converge to a learning attractor and recall may be successful. Such an operation acts as a filter action or a content addressable memory.

  On the other hand, an attractor convergence identification signal is sent from the output terminal 32 of the attractor convergence state identification processing unit 21, and at the same time, an attractor output signal different from the learning attractor is sent to the output terminal 31, which is called a spurious attractor (false attractor). It may be recalled in the attractor convergence state. That is, it is an attractor convergence state for an associative input signal containing a lot of noise and distortion, but it is a spurious attractor that has almost no significant meaning completely different from a normal learning attractor. It is sent out as a learning attractor output signal.

  This is a spurious attractor that is indistinguishable from the learning attractor in the conventional method, is automatically generated greatly influenced by the learning input data stored in the weighting factor, and is unexpected and misleading. Not done. When converged to the spurious attractor, an error occurs in the application processing system using the attractor output signal from the output terminal 31 or the attractor convergence state identification signal from the terminal 32.

  In order to succeed in correct recall with the conventional self-associative neural network 22 as much as possible and to suppress the occurrence of spurious attractors, the learning input data as learning attractors must have orthogonal relationships that are not correlated with each other. That is, if a relatively similar associative input signal is calculated by calculating and setting a weighting factor for learning and storing as learning input data, interference between learning attractors occurs, and the case of convergence to spurious attractors increases, or There are more cases of non-convergent and non-convergent divergence.

  Therefore, the number of learning input data stored by the optimum weighting coefficient of the self-associative neural network 22, that is, the number of learning attractors is greatly limited. Further, the self-associative neural network 22 does not make much use of the generalization ability, which is one of the features of the neural network, and causes the attractor to converge only by the feedback loop in the associative loop processing. This means that it does not converge very easily and tends to be in an attractor divergence state, but even if it converges to the attractor, it tends to become a spurious attractor.

Furthermore, in such a conventional method, unless the spurious attractor output signal is compared with all the stored learning attractors, it cannot be distinguished whether it is a learning attractor or a spurious attractor. A processing amount is required.
The biggest drawback of the self-associative memory using such a self-associative neural network 22 is a small storage capacity due to a large restriction on the possible weighting factor. According to the above document, this storage capacity is theoretically given by about 0.12 × number of input layer units P.
Furthermore, when learning input data similar to each other is learned, there is a drawback that it is difficult to obtain an attractor convergence state from mutual interference. Further, in order to reduce spurious attractors in the attractor converged state that is erroneously recalled, the storage capacity is further reduced. Therefore, attractor convergence becomes more difficult as the number of learned input data approaches the theoretical limit.

  As another prior art, there is a method called a mutual (hetero) associative memory using two cascaded neural networks. As shown in FIG. 3, this method is composed of a forward three-layer neural network 23 and a reverse three-layer neural network 24 that are trained using teacher signals. Conventionally, a coarse code output type forward three-layer neural network 23 is used as the forward three-layer neural network 23.

  As an output signal of the coarse code output type forward three-layer neural network 23, only one of the units becomes 1, and the other K-1 units become 0. A binary output signal format of K units, that is, One A coarse encoded code called out of K is used. Since it is easy to associate the teacher signal corresponding to the learning input data of the neural network with the classification category, it is adopted as the output signal format of most multilayer neural networks and is learned.

  On the other hand, in the reverse three-layer neural network 24, the teacher signal of the coarse code output type forward three-layer neural network 23 is learned as learning input data. As a teacher signal of the backward three-layer neural network 24, the forward associative input signal X (0) from the input terminal 1 is used for feedback input to the coarse code output type forward three-layer neural network 23 via the input changeover switch 10. It is necessary to have the output format of the same number of input units as the learning input data of the coarse code output type forward three-layer neural network 23, that is, the learning attractor.

  That is, each learning is performed so that learning input data to the coarse code output type forward three-layer neural network 23 is reproduced as an output signal of the reverse three-layer neural network 24. The weighting factors learned in this way are set as weighting factors for the coarse code output type forward three-layer neural network 23 and the reverse three-layer neural network 24, respectively.

  Here, these operations will be briefly described. The input selector switch 10, the associative loop control processing unit 20, and the attractor convergence state identification processing unit 21 in FIG. 3 have the same configuration as that in FIG. As in the case of FIG. 2, the input changeover switch 10 is controlled by a control signal from the associative loop control processing unit 20, and as a default setting, the coarse code output type forward 3 in which the forward associative input signal of the input terminal 1 has been learned. Connection is performed so that the signal is input via the input terminal 2 of the layer neural network 23. The coarse code output type forward three-layer neural network 23 outputs an output signal corresponding to the forward associative input signal X (0) from the output terminal 3 and is further input to the learned reverse three-layer neural network 24 via the terminal 4. The The corresponding output signal is fed back to the input terminal 2 via the input changeover switch 10 as an associative loop process under the control of the associative loop control processing unit 20, and is also input and stored in the attractor convergence state identification processing unit 21. The

  In the attractor convergence state identification processing unit 21, the stored output signal X (n) of the backward three-layer neural network 24 at the time of the association loop number n-1 and the output signal X (n +) of the association loop number n. When 1) matches, it is considered that the beam has converged on the attractor, and the attractor convergence identification signal is sent from the output terminal 32, and the output signal of the reverse three-layer neural network 24 is sent from the output terminal 5 as the attractor output signal. Even if the number of associative loops n = N, an attractor divergence identification signal is transmitted from the output terminal 32 in an attractor divergence state that does not converge on the attractor.

  In this conventional associative memory configuration, the learning input data as the necessary K learning attractors and the teacher signals in the One out of K output signal format corresponding to each of them are learned. Can be increased. However, when the learning input data number K as the learning attractor is very large, the input layer of the backward three-layer neural network 24 having the same number of units as the K output layer units of the forward three-layer neural network 23. The number of units will inevitably become very large. Therefore, the scale of the coarse code output type forward three-layer neural network 23 and the reverse three-layer neural network 24 becomes very large, and there is a drawback that the amount of calculation of the learning process becomes enormous. Also, since a sparse output signal format consisting of K units of coarse codes is used, the number of cases of convergence to a spurious attractor is extremely high.

Even in this conventional method, since the convergence state to the spurious attractor cannot be detected, the learning attractor cannot be distinguished from the spurious attractor, and the attractor convergence state identification processing unit 21 sends out the attractor convergence identification signal. In this case, the application processing system using the attractor output as the learning attractor output signal from the output terminal 5 has a drawback of causing a malfunction. However, when the number of learning attractors is large, the number of attracted attractor divergent states that do not reach the attractor converged state is not as large as that of the self-associative memory.
In the above description, the coarse code output type forward three-layer neural network 23 that learns using a teacher signal has been described. In addition to this, although the performance is deteriorated, a self-organizing network may be used. Have both.

  As described above, since the self-associative memory has the same number of input / output units and is limited, it has a drawback limited to application to recognition of degraded image data and the like. Moreover, it does not have a category classification function. Furthermore, in order to converge to the learning attractor which is learning input data, there exists a fault that a weighting coefficient has restrictions and many learning input data cannot be memorize | stored. For example, the theoretical maximum learning input data number that can be stored in a two-layer neural network is 0.12 × input layer unit number, and has a disadvantage that the storage amount of the weight coefficient is very small.

  Inevitably, an output signal consisting of a large number of units identical to the input signal has a coarse output signal format because the number of learning input data to be stored is small. For this reason, after an associative input signal having a relatively small noise deterioration is input, the output signal is fed back to the input terminal 2 and the associative loop processing is repeated, so that the attractor convergence state identification processing unit 21 receives a signal other than the learning attractor. Convergence occurs very often in spurious attractor states that are not recalled correctly, and for an associative input signal that is greatly degraded in noise, there are very many cases where the attractor divergence state does not reach the attractor convergence state. There are also disadvantages such as.

  As described above, the conventional self-associative memory and the mutual associative memory have many drawbacks. In the self-associative neural network 22, the storage capacity of the weight coefficient is very small, and many learning attractors cannot be stored. This storage capacity is theoretically given by about 0.12 × number of input layer units, and the neural network becomes very complicated in order to increase the storage capacity. Also, in the associative loop processing, there is a drawback that the attractor does not converge so much and is likely to diverge the attractor, but even if it converges to the attractor, it is not a desired learning attractor but a spurious attractor. is there. In addition, the configuration has the same number of input / output units and is limited, and has the disadvantage that it does not have a category classification function and its application is limited.

On the other hand, in the conventional associative memory using the three-layer neural network, the coarse code output type forward three-layer neural network 23 is used. Therefore, in order to store many learning attractors, the output layer of the forward three-layer neural network 23 is used. The number of units has to be very large, the forward and backward three-layer neural network scale becomes very large, naturally, the amount of computation becomes enormous, and correct learning is difficult. Also, for the same reason, it has a drawback that it is very easy to converge on the spurious attractor.
Further, since there is no spurious / attractor detection means, there is a disadvantage that it causes a malfunction in the application processing system using the attractor output signal or the attractor convergence identification signal.

  The object of the present invention is to solve the above-mentioned problems, and can have a very large storage capacity as compared with a self-associative memory and a mutual associative memory using a conventional neural network, and falls into a spurious attractor. Very few high-performance multi-layer neural networks with attractor detection / discriminating function and associative input learning / non-learning input data discriminating function The object is to provide an associative memory device.

  In order to solve the above-mentioned problem, as a first means, a forward neural network means that has been trained to transmit a teacher signal that is a learning attractor corresponding to the classification category to learning input data, Backward neural network means learned so that the learning input data can be sent to the teacher signal is cascade-connected, and the forward neural network is connected via an input changeover switch controlled based on the number of associative loops of the forward associative input signal. Once input to the network means (associative loop number n = 0) and obtaining the output signal of the backward neural network means, the output switch of the backward neural network means is input to the forward neural network means. And the output signal of the forward neural network means And an associative loop process (associative loop number n ≧ 1) for sequentially obtaining an output signal of the backward neural network means, and an associative memory device having an associative loop control processing means for performing associative loop number counting and control, By comparing the classification category number region prepared from the classification category numbers corresponding to all of the teacher signals with the classification category number corresponding to the output signal of the forward neural network means, the learning signal of the output signal is detected. An associative memory device comprising at least learning internal / external category identifying means for identifying a category is configured.

  As a second means, forward neural network means learned so as to be able to send a teacher signal which is a learning attractor corresponding to the classification category to the learning input data, and the learning input data to the teacher signal Reciprocal neural network means learned so that it can be transmitted, the forward neural network means is subordinately connected to the reverse neural network means, and associative loop control processing means for performing associative loop count counting and control In the storage device, the reverse associative input signal is temporarily input to the reverse neural network means via the input changeover switch (number of associative loops n = 0) to obtain the output signal of the forward neural network means, and then the forward The output signal of the neural network means is converted to the backward neural network. Performing an associative loop process (associative loop number n ≧ 1) for sequentially obtaining an output signal of the backward neural network means and an output signal of the forward neural network means by feeding back to the input of the network means via the input changeover switch, A learning internal / external category of the output signal is obtained by a comparison process between a classification category number area prepared from classification category numbers corresponding to all of the teacher signals and a classification category number corresponding to the output signal of the forward neural network means. A mutual associative memory device characterized by having at least a learning internal / external category identifying means for identifying.

  As a third means, to the learning input data, the learning input data is sent to the forward neural network means that has been learned so that the learning signal corresponding to the classification category can be transmitted to the learning input data. The backward neural network means trained so that it can be transmitted is cascade-connected, and the forward associative input signal is temporarily input to the forward neural network means via the input changeover switch (number of associative loops n = 0). After obtaining the output signal of the network means, the output signal of the backward neural network is fed back to the input of the forward neural network means via the input changeover switch, and the output signal of the forward neural network means and the backward neural network means Associative loop processing that sequentially obtains the output signal An associative loop control processing means for performing associative loop count n ≧ 1) and performing associative loop count counting and control, and a plurality of parallel-connected teacher signals for the classification category and Is a multi-layer neural network that has learned teacher signals for parallel connection having different allocation arrangements, and an output that inversely converts each output signal from the multi-layer neural network into an output signal of an output code system that is the basis of the teacher signal Learning that is composed at least of an inverse transform processing means and an output composite means for selectively sending out one output signal from the output signal of the output inverse transform processing means, and can send the teacher signal to the learning input data Using the first forward parallel complex neural network means, the forward neural network neural network. Class means corresponding to all of the teacher signals and a classification category number prepared in advance and a classification category number corresponding to the output signal of the first forward parallel composite neural network means And a learning internal / external category identifying means for identifying the learning internal / external category of the output signal by the comparison process.

  As a fourth means, forward neural network means learned so that a teacher signal which is a learning attractor corresponding to the classification category can be sent to the learning input data, and the learning input data is sent to the teacher signal. Reciprocal neural network means trained so that it can be transmitted, the forward neural network is connected to the reverse neural network as a subordinate, and associative memory having associative loop control processing means for counting and controlling associative loops A multi-layer neural network in which each parallel connection teacher signal having a different allocation arrangement from the teacher signal for the classification category connected in parallel is learned, and each of the multi-layer neural networks The output signal is output from the output code system that is the basis of the teacher signal. An output inverse transform processing means for inversely transforming into a signal, and an output composite means for selectively sending out one output signal from the output signal of the output inverse transform processing means, and the teacher signal for the learning input data Using the learned first forward parallel complex neural network means that can be transmitted, at least the forward neural network means is configured, and a reverse associative input signal is temporarily input to the reverse neural network means via an input changeover switch ( The number of associative loops n = 0) and obtaining the output signal of the first forward parallel composite neural network means, the output signal of the first forward parallel composite neural network means is input to the reverse neural network means as the input Feedback neural network means fed back via an input changeover switch Associative loop processing (associative loop count n ≧ 1) for sequentially obtaining an output signal and an output signal of the first forward parallel complex neural network means, and preparing in advance composed of classification category numbers corresponding to all of the teacher signals At least learning internal / external category identifying means for identifying a learning internal / external category of the output signal by a comparison process between the classification category number region thus determined and the classification category number corresponding to the output signal of the first forward parallel complex neural network means The mutual associative memory device is characterized by having.

  As a fifth means, the learning input data is input to the forward neural network means that has been trained so as to be able to transmit a teacher signal that is a learning attractor corresponding to the classification category to the learning input data. The backward neural network means trained so that it can be transmitted is cascade-connected, and the forward associative input signal is temporarily input to the forward neural network means via the input changeover switch (number of associative loops n = 0). After obtaining the output signal of the network means, the output signal of the backward neural network is fed back to the input of the forward neural network means via the input changeover switch, and the output signal of the forward neural network means and the backward neural network means Associative loop processing that sequentially obtains the output signal An associative memory device having an associative loop control processing means for performing associative loop count n ≧ 1) and performing associative loop count counting and control, wherein the learning input data is converted into different parallel learning input data. A conversion processing means; a plurality of parallel connected multi-layer neural networks that learn the parallel learning input data and the teacher signal; and an output composite that selectively transmits one output signal from the output signals of the multi-layer neural network. And at least the forward neural network means using the trained second forward parallel complex neural network means capable of transmitting the teacher signal to the learning input data, and Prepared classification categories composed of classification category numbers corresponding to all teacher signals. It has at least learning internal / external category identifying means for identifying a learning internal / external category of the output signal by a comparison process between a Gori number area and a classification category number corresponding to the output signal of the second forward parallel composite neural network means. The characteristic associative memory device is constructed.

  A sixth means is a forward neural network means that learns to be able to send a teacher signal that is a learning attractor corresponding to the classification category to the learning input data, and the learning input data is sent to the teacher signal. Reciprocal neural network means trained so that it can be transmitted, the forward neural network is connected to the reverse neural network as a subordinate, and associative memory having associative loop control processing means for counting and controlling associative loops An input conversion processing means for converting the learning input data into different parallel learning input data, and a plurality of parallel connected multi-layered neural circuits that have learned the parallel learning input data and the teacher signal One output signal from the output signal of the network and the multilayer neural network And at least the forward neural network means using the learned second forward parallel parallel neural network means capable of sending the teacher signal to the learning input data. The reverse associative input signal is temporarily input to the reverse neural network means via the input changeover switch (number of associative loops n = 0) to obtain the output signal of the second forward parallel complex neural network means, The output signal of the second forward parallel complex neural network means is fed back to the input of the reverse neural network means via the input changeover switch, and the output signal of the backward neural network means and the second forward parallel complex neural network are fed back. Associative loop to obtain the output signal of the means sequentially (Associative loop count n ≧ 1), the prepared classification category number region composed of the classification category numbers corresponding to all of the teacher signals and the output signal of the second forward parallel complex neural network means A mutual associative memory device characterized by comprising at least a learning internal / external category identifying means for identifying a learning internal / external category of the output signal by a comparison process with a corresponding classification category number.

  As a seventh means, in the mutual associative memory device of any one of the first, third and fifth means, the two input signals of the forward neural network means obtained through continuous associative loop processing A mutual associative memory device comprising attractor convergence state identifying means for identifying an attractor convergence state based on a matching state between two output signals corresponding to each other and between two output signals of the backward neural network means Configure.

  As an eighth means, there is the mutual associative memory device of the second, fourth and sixth means, and the two input signals of the backward neural network means obtained and correspondingly obtained through continuous associative loop processing The mutual associative memory device is characterized by having attractor convergence state identifying means for identifying the attractor convergence state based on the matching state between the two output signals and between the two output signals of the forward neural network means.

  As ninth means, in the attractor convergence state identifying means of the seventh means, the forward associative input signal is inputted to the forward neural network means, and a corresponding output signal and an output signal of the backward neural network means are obtained. Associative loop processing with n = 0 associative loops respectively obtained, and then the output signal of the backward neural network means is fed back via the input changeover switch and input to the forward neural network means, and the corresponding output signal and the The matching state between the two input signals of the forward neural network means obtained through the associative loop processing of the associative loop number n = 1 for obtaining the output signals of the backward neural network means and the learning internal / external category identifying means The forward neural network according to the learning internal / external category identification signal Constituting the mutual associative memory apparatus characterized by learning and non-learning input data identification of the forward associative input signals input to the stage.

  The tenth means is an attractor convergence state identification means of the eighth means, wherein the reverse associative input signal is inputted to the reverse neural network means, and the corresponding output signal and the output signal of the forward neural network means are respectively input. Associative loop processing with n = 0 associative loops to be obtained, and the output signal of the forward neural network means is fed back via an input changeover switch and input to the reverse neural network means, and the corresponding output signal and the forward neural network The matching state between the two input signals of the backward neural network means obtained through the associative loop processing of the associative loop number n = 1 for obtaining the output signal of each means, and the learning inside / outside category of the learning inside / outside category identifying means Based on the identification signal, it is input to the backward neural network means. Further, a learning / non-learning input data of the backward associative input signal is identified, and the mutual associative memory device is characterized in that it is discriminated from the learning input data only when the matching state is the in-learning category identification state. .

  The eleventh means is the mutual associative memory device of any one of the seventh, eighth, ninth and tenth means, wherein the learning internal / external category for the output signal of the forward neural network means from the learning internal / external category identifying means Based on the identification signal and the attractor convergence identification signal from the attractor convergence state identification means, in the attractor convergence state, if it is a non-learning category identification state, the spurious attractor identification and the learning / attractor identification if it is an in-learning category identification state. The mutual associative memory device is characterized by having at least attractor identification processing means.

  As a twelfth means, in the attractor identification processing means of the eleventh means, from the attractor convergence identification signal and the learning inside / outside category identification signal, when the association loop number n = 1, If it is a category identification state, it is a learning attractor identification, if it is a non-learning category identification state, if it is One-Shot spurious attractor identification, the number of associative loops n> 1, if it is an attractor convergence state, if it is an in-learning category identification state, If the generalized learning / attractor identification and the non-learning category identification states are set, a generalized spurious / attractor identification is set.

  As thirteenth means, forward neural network processing learned so as to be able to send out a teacher signal, which is a learning attractor corresponding to the classification category, to the learning input data, and the learning input data for the teacher signal And a backward neural network processing learned so that it can be transmitted, a forward associative input signal is processed by the forward neural network processing to obtain an output signal, and the output signal is further input to the backward neural network. An output signal is obtained by processing, and then the output signal is fed back via a reverse neural network output signal input setting process, and an associative loop process is performed to obtain an output signal by the forward neural network process, and the number of associative loops is counted. Interactive Associative Memory Processing with Associative Loop Count Determination Processing In the software, by a comparison process between a classification category number corresponding to the output signal of the forward neural network processing and a classification category number region configured from classification category numbers corresponding to all the teacher signals of the forward neural network processing The learning internal / external category identifying process for identifying the learning internal / external category of the output signal and the continuous associative loop process between the two input signals of the forward neural network processing and the corresponding two output signals, the reverse Attractor convergence state identification processing for identifying the attractor convergence state based on the respective coincidence states between the two output signals of the neural network processing, and the forward neural network processing for the forward associative input signal and the association loop number n = 1 Between the two input signals and the learning internal / external Learning associated with the forward associative input signal and non-learning input data for identifying the forward associative input signal based on the learning internal / external category identification signal for the Gori identification process, and the learning from the learning internal / external category identifying process Based on the inside / outside category identification signal and the attractor convergence identification signal from the attractor convergence state identification process, if the attractor is in a non-learning category identification state, the spurious attractor identification and the learning / attractor identification is in a learning internal category identification state. And an associative memory processing software comprising at least an attractor identifying process.

  As a fourteenth means, forward neural network processing learned so that a teacher signal that is a learning attractor corresponding to the classification category can be sent to the learning input data, and the learning input data is sent to the teacher signal. In the associative memory processing software having at least a reverse neural network process learned so that it can be transmitted and an associative loop number determination process for counting the number of associative loops, the reverse associative input signal is processed by the reverse neural network process The output signal is input, the output signal is input and the output signal is obtained by the forward neural network processing, and then the output signal is fed back via the forward neural network output signal input setting processing to input the reverse neural network. Associating output signals with network processing A comparison process between the classification category number corresponding to the output signal of the forward neural network processing and the classification category number region formed of the classification category numbers corresponding to all the teacher signals of the forward neural network processing. The learning internal / external category identification process for identifying the learning internal / external category of the output signal, and the two input signals of the backward neural network processing and the corresponding two output signals obtained through the continuous associative loop process, Attractor convergence state identification processing for identifying attractor convergence state based on respective coincidence states between two output signals of forward neural network processing, and the backward neural network in the backward associative input signal and the number of associative loops n = 1 Processing input signal coincidence state between two input signals, and the learning internal and external categories The learning associative input signal learning / non-learning input data identification process for identifying the learning associative input signal learning / non-learning input data of the forward associative input signal and the learning internal / external category identification process from the learning internal / external category identification process, by the learning internal / external category identification signal Based on the category identification signal and the attractor convergence identification signal from the attractor convergence state identification process, if the attractor is in a non-learning category identification state in the convergence state, the spurious attractor identification and the learning / attractor identification is in the learning category identification state. The associative memory processing software is characterized by having at least attractor identification processing.

  As a fifteenth means, in the attractor identification process of the thirteenth means and the fourteenth means, when the associative loop count n = 1 from the attractor convergence identification signal and the learning inside / outside category identification signal, the attractor convergence state In the learning category identification state, learning / attractor identification, and non-learning category identification state, One-Shot spurious / attractor identification, associative loop count n> 1, the category classification in learning in the attractor convergence state If it is in the state, generalized learning / attractor identification, and if it is in the non-learning category identification state, generalized spurious / attractor identification is set.

  The mutual associative memory according to the multilayer neural network of the present invention uses a distributed output format in which a dense binary code is assigned as a teacher signal in the output signal of the forward multilayer neural network, and further the binary of the forward multilayer neural network. Learning internal / external category identification for output signal, learning / non-learning input data identification of associative input signal, and associative loop processing, between input signals and output signals of forward multi-layer neural network, and further output signal of reverse multi-layer neural network Attractor convergence state identification and attractor identification are performed by comparing each match between the two.

  As described above, the associative memory device of the present invention uses a binary code in which classification category numbers are densely encoded as a binary teacher signal of a forward multi-layer neural network. In addition, learning / attractor identification of the forward associative input signal and One-Shot spurious / attractor identification, and learning / non-learning input data can be instantly identified. Further, in the associative loop processing, generalization learning / attractor identification and generalized spurious / attractor identification in the attractor convergence state can be performed by detecting the generalization state of the forward multilayer neural network and the backward multilayer neural network. In addition, the attractor convergence state is achieved based on very few associative loop processes, and spurious attractors are very rarely generated. In addition, a much larger amount of learning attractors can be stored freely and stably, and a much more excellent associative memory that is not available in conventional devices is realized.

  For these reasons, the excellent associative memory device having a large capacity of the present invention is applicable to security systems such as high-speed data retrieval, unauthorized access detection by unknown input data, network failure detection system, etc. Furthermore, it can be widely applied to pattern recognition such as recognition of large-capacity image data degraded by a large amount of noise and filtering for degradation removal.

  An example of an associative memory using a three-layer neural network according to an embodiment of the present invention is shown in FIG. In the following embodiment of the mutual (hetero) associative memory using the multilayer neural network of the present invention, a three-layer neural network using a binary teacher signal is taken as an example to simplify the description of the basic configuration. The configuration and operation thereof will be described in detail. The present invention is not limited to a three-layer neural network, and any neural network that learns using a teacher signal such as a multilayer neural network having four or more layers may be used.

  Either a forward associative input signal input to the input terminal 1 or an output signal from the backward three-layer neural network 13 is selected by a control signal from the associative loop control processing unit 11, and the forward three-layer through the input terminal 2 An associative loop control process that sends the input to the neural network 12 and the input changeover switch 10 sent to the attractor convergence state identification processing unit 14 and the forward associative input signal from the input terminal 1 and the reverse associative input signal from the input terminal 18 A binary output signal is obtained for the input signal from the input changeover switch 19 and the input terminal 2 which is switched and selected by the control signal from the unit 11 and sent to the attractor convergence state identification processing unit 14 and sent out from the output terminal 3 , And output as a category category output signal from the output terminal 9, and the input selector switch 17, the learning internal / external category identification Either an advance three-layer neural network 12 to be sent to the processing unit 15 or the attractor convergence state identification processing unit 14, a binary output signal from the forward three-layer neural network 12, and a backward associative input signal from the terminal 18 are associative loops. An input changeover switch 17 that is switched and selected under the control of the control processing unit 11 and is input to the reverse three-layer neural network 13 through the input terminal 4, and an output signal corresponding to the input signal at the input terminal 4 is input to the input changeover switch 10 and attractor convergence. Each of the signals is sent to the state identification processing unit 14, and the binary output signal from the backward three-layer neural network 13 and the forward three-layer neural network 12 sent from the output terminal 5 as the attractor output signal is input, and the learning internal / external category identification signal is input. The attractor convergence state identification processing unit 14 and the attractor The learning internal / external category identification processing unit 15 and the associative loop control processing unit 11 which are respectively sent to the data identification processing unit 16 are initialized, and the associative input signal from the input changeover switch 19 and the input signal and output of the forward three-layer neural network 12 are output. The signal and the output signal of the reverse three-layer neural network 13 are input, the attractor convergence identification signal is transmitted to the attractor identification processing unit 16 and the output terminal 8, and the learning / non-learning input data identification signal is further transmitted to the output terminal 7. , An attractor convergence state identification processing unit 14 for sending an associative loop processing request signal or attractor convergence signal to the associative loop control processing unit 11, and a learning internal / external category identification signal and attractor convergence state identification processing unit from the learning internal / external category identification processing unit 15. 14, an attractor convergence identification signal is input, and an associative loop control processing unit 11 When an attractor identification signal transmission request signal or an attractor divergence identification signal transmission request signal is input from, and the attractor identification processing unit 16 that transmits the attractor identification signal to the output terminal 6 and the forward associative input signal are input from the input terminal 1 Alternatively, when a reverse associative input signal is input from the input terminal 18, switching setting control of the input changeover switches 10, 17, and 19 is performed, and the initial setting of the attractor convergence state identification processing unit 14 and the attractor identification processing unit 16 are attracted. It comprises an associative loop control processing unit 11 that sends an identification signal transmission request signal or attractor divergence identification signal transmission request signal, and further controls the start and end of the associative loop process.

  The operation of these associative loop processes will be described below. Prior to the associative loop processing, the forward three-layer neural network using learning data including a learning attractor as learning input data of the forward three-layer neural network 12 and a binary code as a binary teacher signal assigned to the classification category. 12 is learned in advance, and further, the backward three-layer neural network 13 is learned using learning data having an input / output relationship opposite to the learning data of the forward three-layer neural network 12, and each is set. . Here, the binary output signal of the forward neural network 12 is a distributed output format of a binary code representing a classification category.

  The number of classification categories (the number of teacher signals) of the forward three-layer neural network 12 is composed of the classification category numbers of serial numbers from 0 to M-1 corresponding to binary teacher signals assigned to the M classification categories. The classified category number area is provided, and the maximum value (M-1) of this area is set as the maximum classified category number Maxcatg in the learning internal / external category identification processing unit 15 in advance. Further, a distributed binary code as a binary teacher signal is associated with a category number from 0 to M-1.

Here, first, the operation in the normal case where the forward associative input signal is input to the input terminal 1 will be described. The associative loop control processing unit 11 initializes the associative loop count n = 0, and the associative loop control signal is stored in the input changeover switches 10, 17, and 19 and the attractor convergence state identification processing unit 14 stores the associative loop count. Transmitted for initial setting of I / O signal storage. When the forward associative input signal X (0) is input to the terminal 1, the input selector switch 10 is connected to the input terminal 1 and the input terminal 2, and the input selector switch 17 is always connected to the output terminal 3. Further, the input changeover switch 19 is set and controlled so that the forward associative input signal X (0) is input to the attractor convergence state identification processing unit 16 so that the terminal 4 is connected, and associative loop processing is executed.
Image data including a lot of noise at the input terminal 1 or image data consisting of a part of information, or a forward associative input signal X (0) as memory high-speed search data is learned via the input changeover switch 10 12 and further input to and stored in the attractor convergence state identification processing unit 14.

  When the number of associative loops n = 0, the binary output signal Y (0) from the forward three-layer neural network 12 and the output signal X from the reverse three-layer neural network 13 corresponding to the forward associative input signal X (0). (1) is obtained. When the number of associative loops is n (where n ≧ 1), the reverse three-layer neural network 13 is connected to the input terminal 2 via the input changeover switch 10 based on the control signal from the associative loop control processing unit 11. An output signal X (n) is input, and further, an output signal Y (n) is obtained from the forward three-layer neural network 12, and an output signal X (n + 1) is obtained from the reverse three-layer neural network 13, respectively. The attractor convergence state identification processing unit 14 is inputted and stored respectively.

  In the learning internal / external category identification processing unit 15, the integer value converted from the binary code in the distributed output format which is the binary output signal Y (n) of the forward three-layer neural network 12, that is, the classification category number IcatgY (n) is obtained. obtain. This is compared with the maximum classification category number Maxcatg.

If the classification category number IcatgY (n) is greater than the previously given maximum classification category number Maxcatg,
That is,
IcatgY (n)> Maxcatg (1)
Is satisfied, the binary output signal Y (n) of the forward three-layer neural network 12 is identified as the binary output signal of the outside learning category, and the outside learning category identification signal is used as the outside learning category identification signal. The data is sent to the attractor convergence state identification processing unit 14 and the attractor identification processing unit 16, respectively.

On the other hand, if the maximum category number is less than Maxcatg,
That is,
IcatgY (n) ≦ Maxcatg (2)
If the condition is satisfied, the binary output signal Y (n) of the forward three-layer neural network 12 is identified as the output signal of the in-learning category, and the attractor convergence state identification process is performed using the in-learning category identification signal as the learned in-out category identification signal Are sent to the unit 14 and the attractor identification processing unit 16, respectively.
In this way, the learning inside / outside category identification processing unit 15 can easily identify the learning inside / outside category by the arithmetic processing via the classification category number according to the assignment rule between the binary teacher signal and the classification category number.

In the associative loop process of the associative loop count n, the output signal X (n) from the reverse three-layer neural network is fed back via the input changeover switch 10 and input to the forward three-layer neural network 12. The binary output signal Y (n) from the forward three-layer neural network 12 corresponding to this is input to the backward three-layer neural network 13 via the input terminal 4 and further to the learning internal / external category identification processing unit 15 and attractor convergence state identification processing. Each is sent to the unit 14. Further, it is output to the output terminal 9 as a category category output signal.
The reverse three-layer neural network 13 outputs an output signal X (n + 1) corresponding to the input binary output signal Y (n) and sends it to the attractor convergence state identification processing unit 14 and the input changeover switch 10 respectively.

  As shown in Table 1, the attractor convergence state identification processing unit 14 compares the stored input / output signals of the forward and backward three-layer neural networks 12 and 13 to identify the attractor's convergence state, and the One-Shot attractor. One of the convergence identification signal, the identification signal immediately before the attractor convergence, and the attractor non-convergence identification signal is sent from the output terminal 8 and sent to the attractor identification processing unit 16. Furthermore, if the attractor has not converged, the associative loop processing request signal is sent to the associative loop control processing unit 11, and if the attractor has converged or just before the generalized attractor has converged, the attractor convergence signal is sent. Further, in the associative loop processing with the association loop count n = 1, the learning input as the learning / non-learning input data identification signal is output from the output terminal 7 based on the learning internal / external category identification signal from the learning internal / external category identification processing unit 15. Either a data identification signal or a non-learning input data identification signal is transmitted.

  In the attractor identification processing unit 16, the attractor convergence state identification processing unit 14 receives the attractor convergence identification signal, the associative loop control processing unit 11 receives the attractor identification signal transmission request signal or the attractor divergence identification signal transmission request signal, and learning internal / external category identification processing. A learning internal / external category identification signal is input from the unit 15. Based on these signals, as the attractor identification signal of the converged attractor, learning / attractor identification signal, generalized learning / attractor identification signal, One-Shot spurious attractor identification signal, generalized spurious attractor identification signal and attractor divergence identification signal Is sent from the output terminal 6.

  Furthermore, the binary output signal from the reverse three-layer neural network 13 is a learning / attractor output signal, generalized learning / attractor output signal, One-Shot spurious attractor output signal, generalized spurious attractor output signal, or attractor unconvergence. The output signal is sent from the output terminal 5 as any one of the output signals.

  Here, when the forward associative input signal X (0) is learning input data, the binary output signal Y (0), which is a binary teacher signal corresponding to the number of associative loops n = 0, is always obtained. Then, the learning inside / outside category identification processing unit 15 satisfies the expression (2), so that it is identified as a learning category. Further, when the input selector switch 10 is switched and the associative loop processing with the associative loop count n = 1 is performed, the expression (2) is similarly satisfied and the category within learning is identified.

In the attractor convergence state identification processing unit 14, as shown in Table 1, the input / output relationship between the forward and backward three-layer neural networks 12 and 13 is X (0) = X (1), Y (0) = Y (1). , X (1) = X (2), the One-Shot attractor convergence state is established, and the One-Shot attractor convergence identification signal is transmitted from the output terminal 8 as the attractor convergence identification signal.
Further, since the learning inside / outside category identification processing unit 15 identifies Y (1) as the learning inside category and the forward associative input signal X (0) = X (1), the forward associative input signal X (0) Is the learning input data. As a result, the learning input data identification signal is sent to the output terminal 7 as a learning / non-learning input data identification signal. In this way, at the time when the number of associative loops n = 1, the forward associative input signal X (0) is identified as learning input data or non-learning input data.

Further, in the attractor identification processing unit 16, based on the attractor identification signal transmission request signal from the associative loop control processing unit 11, the One-Shot attractor convergence identification signal as the attractor convergence identification signal and the learning inside / outside category identification signal The attractor is identified from the learning category identification signal, and the learning / attractor identification signal is sent to the output terminal 6 as the attractor identification signal.
Further, a learning / attractor output signal which is an output signal X (1) from the reverse three-layer neural network 13 is sent to the output terminal 5 as an attractor output signal.

  On the other hand, when unknown input data that is non-learning input data other than the learning input data is input as the forward associative input signal X (0), the learning internal / external category identification processing unit 15 uses the associative loop count n = 1. If IcatgY (1) is identified as a non-learning category, the forward associative input signal X (0) is non-learning input data and is immediately identified. At this time, as shown in Table 1, in the One-Shot attractor convergence state satisfying the relationship of X (0) = X (1), Y (0) = Y (1), X (1) = X (2) , One-Shot spurious attractor.

  In the attractor convergence state identification processing unit 14, X (n) = X (n + 1), Y (n) = Y (n + 1), where the number of associative loops is n + 1 (where n ≧ 1). When a generalized attractor convergence state satisfying the relationship X (n + 1) = X (n + 2) is satisfied, the forward or backward three-layer neural networks 12 and 13 are always transmitted in the associative loop count n immediately before that. If a generalized state occurs in any of the cases and the forward three-layer neural network 12 is in a generalized state, X (n-1) ≠ X (n), Y (n-1) = Y (n) , X (n) = X (n + 1) is satisfied. On the other hand, when the backward three-layer neural network 13 is in a generalized state, X (n−1) ≠ X (n), Y (n−1) ≠ Y (n), X (n) = X (n + The relationship of 1) is satisfied. In any case, the state immediately before the generalized attractor converges, and an identification signal just before the attractor convergence is sent from the output terminal 8.

For example, when the number of associative loops n = 1, the attractor convergence state identification processing unit 14 determines X (0) ≠ X (1), Y (0) = Y (1) depending on the generalization state of the forward three-layer neural network 12. , X (1) = X (2) relation, a generalized attractor just before convergence state (when n = 2, a generalized attractor converged state) is detected, an identification signal just before attractor convergence is sent from the output terminal 8 Is done. At this time, when the output signal Y (1) of the forward three-layer neural network 12 is identified as a learning category, the attractor identification processing unit 16 is determined to be a generalized learning / attractor, and the generalized learning / attractor is identified. An identification signal is sent from the output terminal 6.
On the other hand, when the output signal Y (1) of the forward three-layer neural network 12 is identified as a non-learning category, the attractor identification processing unit 16 turns out to be a generalized spurious attractor, and generalized spurious attractor identification. A signal is sent out from the output terminal 6. In any case, the forward associative input signal is non-learning input data.

  In the associative loop control processing unit 11, when an attractor convergence signal is input from the attractor convergence state identification processing unit 14, the associative loop processing is terminated and an attractor identification signal transmission request signal is transmitted to the attractor identification processing unit 16. . In addition, the attractor identification processing unit 16 converges to the generalized learning / attractor if the binary output signal Y (n) is in the learning category, and on the other hand to the generalized spurious attractor in the case of the non-learning category. It is found that they have converged, and a generalized learning / attractor identification signal and a generalized spurious / attractor identification signal are transmitted from the output terminal 6, respectively.

In addition, when the number of associative loops n is equal to or less than the specified maximum allowable number of associative loops N, the state immediately before the generalized attractor convergence does not occur, that is, X (n-1) ≠ X (n), Y (n-1) ≠ Y ( In the case of the relationship of n), X (n) ≠ X (n + 1), it is found that the attractor has not converged and is a recollection state. The attractor convergence state identification processing unit 14 sends an associative loop processing request signal to the associative loop control processing unit 11, and sends an attractor non-convergence identification signal to the output terminal 8 and the attractor identification processing unit 16.
If the associative loop count n is less than N, the associative loop control processing unit 11 sets the associative loop count n to n + 1 and continues the associative loop processing.

At this time, if the number of associative loops n> N in the associative loop control processing unit 11, an attractor divergence identification signal transmission request signal is transmitted to the attractor identification processing unit 16, and the associative loop processing is terminated. In the attractor identification processing unit 16, the attractor divergence identification signal is transmitted via the output terminal 6 in response to the attractor divergence identification signal transmission request signal from the associative loop control processing unit 11.
This means that, for the forward associative input signal X (0), the number of specified associative loops n is N or less and the attractor convergence state is not obtained, and a recollection state is entered.
When a new forward associative input signal X (0) is input to the input terminal 1, the number of associative loops n = 0 is set again, and the above-described series of associative loop processing is performed.

  Here, in the attractor convergence state identification processing unit 14, the attractor convergence state at the number of associative loops n in Table 1 and the input signals X (n−1) and X (n ), Between the output signals Y (n−1) and Y (n), and between the output signals X (n) and X (n + 1) of the reverse three-layer neural network 13 are summarized below.

In the associative loop processing of the associative loop number n (where N ≧ n ≧ 1),
X (n-1) = X (n), Y (n-1) = Y (n), X (n) = X (n + 1) (3)
Is the attractor convergence state.
As shown in Table 1, in particular, when Equation (3) holds when n = 1, the One-Shot attractor converges. If the forward associative input signal X (0) = X (1) and Y (1) is still within the learning category, it turns out that it is learning input data and converges to the learning / attractor. On the other hand, in the non-learning category, the forward associative input signal X (0) is non-learning input data and converges to a spurious attractor called a One-Shot spurious attractor.
When N ≠ 1, the generalized attractor converges to the attractor through the generalized state immediately before.

X (n-1) ≠ X (n), Y (n-1) = Y (n), X (n) = X (n + 1) (4)
In this state, the forward three-layer neural network 12 is in a generalized state.
Also,
X (n-1) ≠ X (n), Y (n-1) ≠ Y (n), X (n) = X (n + 1) (5)
In this state, the reverse three-layer neural network 13 is in a generalized state.
X (n-1) ≠ X (n), Y (n-1) ≠ Y (n), X (n) ≠ X (n + 1) (6)
In this state, the attractor has not converged.

  In the learning inside / outside category identification processing unit 15 of FIG. 1 in the embodiment of the present invention, all dense binary codes which are binary teacher signals are converted into consecutive serial numbers, that is, from 0 to the maximum classification. A classification category number area composed of serial numbers of category number Maxcatg (= M-1) is provided, and the binary output signal Y (n) of the forward three-layer neural network 12 is converted into a classification category number for comparison processing. The category identification method was explained. Here, it is also possible to use a binary output signal having a distributed output format of a binary code that is equal to or larger than the minimum number of bits required to represent the above category category number region.

That is, if the number of bits for representing a binary code in a distributed output format is Q,
Q ≧ Int [log ₂ M] (7)
Q may be used. Here, Int [x] represents an integer value obtained by rounding up x.
However, by using the smallest integer value satisfying Equation (7) as Q, almost no out-of-learning category is generated, and the occurrence of spurious attractors can be minimized. In particular, when the number of binary teacher signals M is a power of 2 (M = 2 ** Q), by using the required minimum number of bits Q, a non-learning category is not generated and is always a learning attractor. A very good characteristic that converges to the generalized learning attractor and does not generate a one-shot or generalized spurious attractor is realized.

In addition, in the learning inside / outside category identification processing unit 15, the classification category number corresponding to the binary teacher signal is from 0 to the inside / outside identification of the continuous classification category number area composed of the maximum classification category number Maxcatg. The inside / outside region identification may be performed for continuous classification category number areas composed of consecutive serial numbers from the minimum classification category number to the maximum classification category number.
Furthermore, instead of the classification category number area composed of the consecutive serial numbers described above, a discontinuous classification category number area composed only of the specified discontinuous integer values is provided in the continuous integer numbers, and the learning inside and outside are provided. In order to perform category identification, the forward three-layer neural network 12 may be learned using these binary codes as binary teacher signals, and the backward three-layer neural network 13 may be learned using them as learning input data.

For example, the required number of binary teacher signals M is a discontinuous classification category in which classification category numbers are prepared for every designated number (for example, in the case of every two, 0, 3, 6, 9,...). A number area (all categories other than these classification category numbers are non-learning categories) may be provided, and the classification category numbers in this classification category number area may be binary code converted and assigned to the binary teacher signal.
In other words, a binary code is assigned to a classification category number in a regular classification category number area composed of a specific integer value range or a specific integer value for a binary teacher signal, and these are converted into binary values. It may be a teacher signal.

  Further, in the coarse code output type forward three-layer neural network 23 used in the conventional method, only one output layer unit is 1 and all others are 0. One out of K (a winner takes all) May be used in the forward three-layer neural network 12. However, in this case, the scales of the forward and backward neural networks 12 and 13 become very large, and spurious attractors are generated so much that the performance as an associative memory is not necessarily high.

When a binary teacher signal in the output format of One out of K is used, the learning inside / outside category identification processing unit 15 repeats Modulo2 arithmetic processing of the classification category number converted from the binary output signal up to K times. If a result other than 0 is obtained even once, the classification category number is identified as a non-learning category, and if all are 0, a learning internal / external category identification method may be used. Alternatively, when there are a plurality of output layer units of one binary output signal Y (n), a learning internal / external category identification method for identifying a non-learning category and a learning internal category in the case of one output layer unit is provided. It may be used. That is, a binary output signal other than the One out of K output format may be identified as a non-learning category, and a binary output signal compliant with this output format may be identified as an in-learning category.
The attracted attractors are identified as the learning attractor and generalized learning attractor, and the One-Shot spurious attractor and generalized spurious attractor, respectively. .

  The above description in the embodiment of the present invention relates to the operation of the hetero-associative memory when the forward associative input signal X (0) at the input terminal 1 is input to the forward three-layer neural network 12 via the input changeover switch 10. Is. In addition, the operation of the hetero-associative memory for inputting the reverse associative input signal Y (0) of the input terminal 18 of FIG. 1 to the reverse three-layer neural network 13 via the input changeover switch 17 will be briefly described.

  The backward associative input signal Y (0) is a binary code indicating a classification category having the same format as the output signal of the forward three-layer neural network 12. Corresponding to the backward association input signal Y (0) indicating an ambiguous classification category, when the correct classification category and its learning attractor are extracted from the output terminals 9 and 5 through the associative loop processing, or the teacher signal of the classification category The learning attractor, that is, the learning input data is read out from the backward associative input signal Y (0).

In order to input the reverse associative input signal Y (0) and perform the associative loop processing, the input selector switch 10 is set so that the output signal of the reverse three-layer neural network 13 is always input to the input terminal 2. The input changeover switch 19 sets conditions for attractor convergence state identification shown in Table 2. Further, the input changeover switch 17 is set so that the reverse associative input signal Y (0) from the input terminal 18 is input to the reverse three-layer neural network 13 through the input terminal 4 at the associative loop count n = 0. . In the subsequent associative loop count n> 0, the output terminal 3 and the input terminal 4 are connected, and the output signal of the forward three-layer neural network 12 is set to be input.
In the attractor convergence state identification processing unit 14, unlike Table 1 in the case of the forward associative input signal, as shown in Table 2, the attractor convergence state identification is performed from the input / output relationship of the forward and backward three-layer neural networks 12 and 13. do. Other operations are the same as in the case of the associative loop processing of the forward associative input signal X (0).

  Next, FIG. 6 shows an associative process flow when the forward associative input signal X (0) is given with respect to the mutual associative memory process shown in FIG. 1 in the embodiment of the present invention. Here, the processing flow of the forward three-layer neural network and the reverse three-layer neural network will be described on the assumption that learning has already been completed in the same manner as described above.

The forward associative initial setting process 100 includes a maximum classification category number Maxcatg in the learning internal / external category identification process 150, an associative loop process maximum allowable number N in the associative loop number determination process 180, and a table to the attractor convergence state identification process 160. 1. Set initial parameters such as setting the conditions for identifying the attractive convergence state shown in FIG.
The associative loop process initial setting process 110 sets n = 0 and the initial state of the attractor convergence state identification process 160 as an initial setting of the associative loop count n.
The forward associative input signal input setting process 120 performs input setting of the forward associative input signal to the learned forward three-layer neural network process 130.

The forward three-layer neural network processing 130 is an associative loop number n = 0 and the input forward associative input signal X (0) or, in the case of the associative loop number n, a reverse three-layer neural network output signal input setting process. The output signal X (n) input through 200 is processed, and binary output signals Y (0) and Y (n) are sent out, respectively.
The learning internal / external category identification processing 150 performs the binary output signal Y (0) of the forward neural network when the association loop number n = 0, or the binary output signal Y (n) when the association loop number n. Are converted into classification category number IcatgY (0) or IcatgY (n), respectively.

  Of the consecutive classification category numbers obtained by converting the binary teacher signal, comparison processing is performed with the maximum classification category number Maxcatg given in advance, and if equation (2) holds, IcatgY (0), IcatgY (n ) Match the classification category number of any teacher signal, and are considered to be in-learning categories, and if the formula (1) holds, they do not match the classification category number of any teacher signal and are assigned. Since there is no classification category number, it is identified as a non-learning category, and a learning inside / outside category identification signal is transmitted.

  In the backward three-layer neural network processing 140, the binary output signal Y (0) of the forward three-layer neural network processing 130 is obtained when the association loop number n = 0, or Y (n) is obtained when the association loop number n. Each is input to a learned reverse three-layer neural network, and its output signals X (1) and X (n + 1) are obtained, respectively.

  In the attractor convergence state identification process 160, when the association loop number n = 0, the attractor convergence state identification is not performed, the attractor non-convergence state is set, and the process proceeds to the attractor convergence state determination process 170 as it is. When the number of associative loops n ≠ 0, the input signals X (n−1) and X (X () respectively input to the forward three-layer neural network in two consecutive associative loop processes in the case of the number of associative loops n−1 and n. n), a binary neural network that processes binary output signals Y (n-1) and Y (n), and these binary output signals Y (n-1) and Y (n) as input signals. The output signals X (n) and X (n + 1) are compared, and if Equation (3) holds when the number of associative loops n = 1, the One-Shot attractor convergence state and the number of associative loops n ≧ 1 When either formula (4) or formula (5) is established, it is identified as a state immediately before the generalized attractor converges.

  When Expression (6) is satisfied, the attractor is not yet converged, and attractor converged state determination processing 170 is performed.

In the attractor convergence state determination process 170, if the attractor has not converged, the process proceeds to the associative loop number determination process 180. If the attractor is in any other convergence state, the process proceeds to the association loop number n = 1 determination process 210.
When the association loop count n = 1 determination process 210 detects the state of the associative loop count n = 1, in the forward associative input signal learning / non-learning input data identification process 220, X (0) = X (1) In the learning internal / external category identification processing 150, when Y (0) is identified as the learning internal category and the One-Shot attractor converges, the forward associative input signal X (0) is used as the learning input data, otherwise Is identified as non-learning input data. Learning / non-learning input data identification of the forward associative input signal X (0) is performed with the association loop count n = 1. When the number of associative loops n ≠ 1, attractor identification processing 230 is directly performed.

In the associative loop count determination process 180, if the associative loop count n is less than the maximum associative loop count N, the associative loop count n is incremented by 1 in the n ← n + 1 setting process 190.
In the backward three-layer neural network output signal input setting process 200, X (n + 1) is set as the input of the forward neural network. Thereafter, as in the case of the previous associative loop count n, forward three-layer neural network processing 130, learning internal / external category identification processing 150, reverse three-layer neural network processing 140, attractor convergence state identification processing 160, and attractor convergence state determination processing 170. To implement.

  On the other hand, in the associative loop number determination process 180, if the attractor convergence state determination process 170 cannot obtain the attractor convergence state or the state just before the generalized attractor convergence, and if the associative loop number n ≧ N, the specified number N of associations In the loop processing, if the forward associative input signal X (0) does not converge to the attractor and becomes unrecognized in the attractor diverging state, the associative loop processing is stopped and the attractor identification processing 230 is performed.

  In the attractor identification process 230, in the associative loop number determination process 180, when n ≧ N, the attractor divergence is identified if the attractor has not converged. In the One-Shot attractor convergence identification state, if it is identified as a learning category by the learning inside / outside category identification processing 150, it is identified as a learning / attractor. If it is identified as a non-learning category, it is identified as a One-Shot spurious attractor. In the state immediately before the convergence of the generalized attractor, when the learning internal / external category identification processing 150 identifies the category within the learning, generalized learning / attractor, and when the generalized attractor is identified as the non-learning category, the generalized spurious attractor. Identify each.

  In the final forward associative input signal determination process 240, it is determined whether the forward associative input signal subjected to the associative loop process is the final forward associative input signal. If it is the final forward associative input signal, a series of associative loop processing is completed. Otherwise, the associative loop process initial setting process 110 is performed, and the associative loop process is continued for the new forward associative input signal.

The above description of the associative processing flow is for the case where the forward associative input signal X (0) is given. Here, FIG. 7 shows an associative process flow when the backward associative input signal Y (0) is given. In the associative loop process of the backward associative input signal Y (0), in the forward associative initial setting process 300, the maximum classification category number Maxcatg and the associative loop number determination process 180 in the learning internal / external category identification process 150 are performed. Initial parameter settings such as the maximum allowable number N of associative loop processing and the attractor convergence state identification process 320 shown in Table 2 for attractor convergence state identification processing 320 are performed.
Here, when the processing number in FIG. 7 is the same as the processing number in FIG. 6, the processing is the same as that in FIG.

In the reverse associative input signal input setting processing 310, the reverse associative input signal Y (0) is input and set to the reverse three-layer neural network processing 140. Thereafter, the attractor convergence state identification process 320 is performed through the processes 130 and 140. In this process, the attractor convergence state is identified according to Table 2. However, when the number of associative loops n = 0, the attractor convergence state determination process 170 is performed as an attractor unconvergence state without performing identification.
If the number of associative loops N ≧ n ≧ 1 when the attractor has not converged, the forward three-layer neural network output signal input setting processing 330 converts the forward three-layer neural network output signal into a reverse three-layer neural network. Set the input.

  In the backward associative input signal Y (0) learning / non-learning input data identification processing 340, when the number of associative loops n = 1 in the attractor convergence state identification processing 320, the attractor convergence identification state is the One-Shot attractor convergence state. Yes, if the output signal Y (1) of the forward three-layer neural network in the learning internal / external category identification processing 150 is identified as the learning internal category, the backward associative input signal Y (0) indicates the classification category for which learning has been performed. Since it becomes a teacher signal, it is identified as learning input data.

  On the other hand, when it is identified as a non-learning category, the backward associative input signal Y (0) is identified as non-learning input data. After performing the attractor identification process 230 according to Table 2, in the final reverse association input signal determination process 350, if the reverse association input signal Y (0) is not the final reverse association input signal Y (0), a new reverse association The input signal Y (0) is set and the associative loop process is repeated. If it is the final reverse associative input signal Y (0), the associative loop processing is completed.

  In the embodiment of the present invention, it is assumed that when a generalized state occurs, the state immediately before the generalized attractor convergence is identified by formula (4) or formula (5) in order to identify the attractor converged state. However, a signal indicating the generalized state of the forward or backward three-layer neural network may be sent to the output terminal. Further, as shown in Tables 1 and 2, instead of using the identification signal immediately before attractor convergence as the attractor convergence state identification according to Equation (4) or Equation (5), Equation (3) ) Holds (the associative loop process increases about once from the state immediately before the generalized attractor convergence), the generalized attractor convergence identification signal may be used as the generalized attractor convergence state. In this case, however, it is unclear whether the generalized state occurs in the forward or backward three-layer neural network.

  Further, in the forward three-layer neural network 12 and the forward three-layer neural network processing 130, a parallel composite neural network having very good generalization characteristics shown in FIG. 4 or 5 may be used instead. In the forward parallel composite neural network 40, for example, three-layered three-layer neural networks 41, 42 and 43 connected in parallel are binary codes of a basic output code system representing the same classification category assignment arrangement corresponding to the learning input data. On the other hand, this is a three-layer neural network in which learning is performed by using parallel teacher signal assignment of binary codes converted into different output code systems.

  In the output inverse transform processing units 44, 45, and 46, three binary output signals from these parallel neural networks 41, 42, and 43 with respect to the input signal X (n) at the terminal 2 are respectively input and assigned to the same classification category. The binary code of the original code system representing the arrangement is converted back and transmitted. The output composite processing unit 47 uses, for example, majority vote (Majority Voting) as an output composite method for these three binary codes obtained by inverse conversion to the original binary code system. A single binary code is selected and transmitted from the terminal 3 as a binary output signal Y (n). Alternatively, as the output composite method, final output selection may be performed based on the Hamming distance between the input signal and the intermediate layer binary output based on the intermediate layer binary output signal for the learning input data.

  In the parallel composite neural network 40 of FIG. 4, the parallel-connected three-layer neural network is trained by using different binary code parallel teacher signals for the same classification category assignment arrangement. In the composite neural network 50, for example, the three-layered neural networks 41, 42, and 43 connected in parallel have the original learning input data for the same teacher signal of the binary code representing the classification category assignment arrangement, for example, Learning is performed by associating different parallel learning input data converted via the input conversion processing units 51, 52, and 53 using a hash function or the like.

  Since these complex parallel complex neural networks 40 and 50 have a generalization capability far superior to that of a single three-layer neural network, attracting attractor by using at least the forward three-layer neural network 12 instead. The force is remarkably improved, and the spurious attractor and attractor divergence state hardly occur even for the forward associative input signal far away from the learning input data. Although it is complicated, a reverse parallel complex neural network having the same structure may be used instead of the reverse three-layer neural network 13 for further improvement.

  In the associative memory of the above embodiment, when the number of learning attractors as learning input data of the forward three-layer neural network 12 corresponding to the storage amount is very large, that is, when the required storage amount of the associative memory is very large. In addition, by increasing the number of intermediate layer units in the three-layer neural network, learning can be reliably converged and mutual associative memory can be realized. Therefore, the storage capacity can be dramatically increased as compared with the conventional self-associative memory.

  In the conventional self-associative memory, the learning attractor that can be stored due to the limitation of the storage capacity is limited to a representative one, and there is a restriction that the learning attractors are not much correlated with each other. In addition to these typical learning attractors, this mutual associative memory can store a large amount of complex input data with a large correlation similar to them as learning attractors, but it can also store complex forward associative input signals and backward associative input signals. In contrast, learning or generalized learning attractors can be converged.

  In the embodiment of the present invention, the case where a three-layer neural network is used has been described, but a multilayer neural network having three or more layers may be used. Furthermore, the present invention is not limited to a multilayer neural network, and may be a neural network trained using a binary teacher signal having the same input / output format.

Further, a forward three-layer neural network having a high generalization ability using a dense distributed binary code for the classification category is used, and the generalization ability to the teacher signals of the forward and backward three-layer neural networks 12 and 13 is used. High attractor suction is achieved.
Therefore, it is possible to easily converge the forward associative input signal away from the learning attractor to the learning attractor. In particular, in the associative memory device of the present invention, a continuous serial number converted from a binary teacher signal using a dense distributed binary output format as a binary teacher signal of the forward three-layer neural network 12 is a classification category number. Therefore, the convergence ability to the generalized learning attractor in the associative loop processing is much higher than that of the conventional method, and spurious attractors are hardly generated.

Furthermore, by using the parallel composite neural network having excellent generalization characteristics shown in FIG. 4 and FIG. 5 in place of the forward three-layer neural network 12, a further excellent associative memory can be realized.
In addition, the forward associative input signal X (0) is a learning input data, whether it is a learning attractor itself or non-learning input data, and has a feature that can be identified by an associative loop process of associative loop count n = 1. Unlike self-associative memory, it has a feature that enables accurate categorization.

  A learning attractor identified by the association loop count n = 1, a generalized learning attractor that has converged by repeating associative loop processing, or an attractor that easily identifies one-shots or generalized spurious attractors that do not have significant information By performing the identification, it is possible to prevent malfunctions in the application processing system using the attractor output signal and the classification category output signal, and to send a malfunction alarm to the application processing system.

As described above, the conventional self-associative memory has a very small data storage capacity.
In addition, although the conventional associative memory device can increase the data storage capacity, a lot of spurious attractors are generated, and the attractor output signal from the backward multi-layer neural network can be distinguished from the desired learning attractor or spurious attractor. Therefore, malfunctions are unavoidable in the application processing system using this output signal.

Furthermore, since K output layer units in the forward neural network are trained using a binary teacher signal of coarse binary code in the one out of K output format, the correct category category output signal has low generalization ability. However, the attractor output signal is often pulled into the spurious attractor. Further, the attracting force (convergence force) in the associative loop process is small, and the attractor diverges in many cases.
Furthermore, if the number of binary teacher signals is increased in order to increase the number of stored data, there are a number of disadvantages such as the scale of forward and backward neural networks becomes very large due to the use of coarse binary codes.

  On the other hand, the mutual associative memory device according to the present invention, as described above, allows the forward three-layer neural network to learn a binary teacher signal having a dense binary code distributed output format. The capability is high and the required three-layer neural network scale can be reduced. In addition, the convergence to the spurious attractor and the attractor divergence state are very rare, and the high speed convergence to the generalized learning attractor is possible.

  Furthermore, by increasing the number of intermediate layer units, it is possible to realize mutual associative memory that can correctly store a large amount of learning attractors without increasing the generation of spurious attractors and without affecting the attracting force of attractors. Is done. In addition, it has advanced associative input signal learning / non-learning input data identification, converged attractor learning / detailing ability between attractors and spurious attractors, so it has much better characteristics for practical use. it can. Further, the present invention can be applied to a wide range of application processing systems that use not only the attractor output signal but also these identification output and classification category output signal.

As described above, the structure and characteristics of the hetero associative memory are very excellent, and a free design according to various applications is possible, and the reliability is also excellent. Therefore, the conventional self-associative memory device has various practical restrictions on not only the storage capacity but also the attractor convergence speed (suction force) and the generation of many spurious attractors. Can solve the problem.
Furthermore, the classification category output signal can be obtained by the discrimination ability of the convergence attractor, the high generalization ability of the forward three-layer neural network for the forward associative input signal, and the high-precision category classification ability. It can also be connected to the applied processing system.

  Because of the above-mentioned wide range of effects, the associative memory device having a large capacity of the present invention is applicable to security management systems and security systems such as unauthorized access detection, various high-performance pattern recognition in biometrics, Application to recognition of large-capacity image data degraded by a lot of noise, etc., application to high-speed data retrieval, etc., and application to network fault detection systems such as abnormal detection of network fault status, etc. It has the characteristics that can be.

It is the 1st structural example in embodiment of the mutual associative memory device using the three-layer neural network which has an attractor identification function of this invention. It is one structural example of the self-associative memory device by the self-associative neural network 22 in the conventional system. It is one structural example of the mutual associative memory device using the multilayer neural network in a conventional system. It is one configuration of a forward parallel complex neural network with output inverse transformation. 1 is a configuration of a forward parallel complex neural network with input conversion. It is an associative loop process flow figure of the mutual associative memory with respect to the forward associative input signal in embodiment of this invention. It is an associative loop process flow figure of the mutual associative memory with respect to the reverse associative input signal in embodiment of this invention.

Explanation of symbols

1 Input terminal of forward associative input signal 2 Input terminal of forward neural network 3 Output terminal of forward neural network 4 Input terminal of reverse neural network 5 Output terminal of attractor output signal 6 Output terminal of attractor identification signal 7 Output terminal of learning / non-learning input data identification signal of forward associative input signal 8 Output terminal of attractor convergence identification signal 9 Classification category output signal Output terminal 10 input selector switch 11 associative loop control processing unit 12 forward three-layer neural network 13 reverse three-layer neural network 14 attractor convergence state identification processing unit 15 learning internal / external category identification processing unit 16 attractor identification processing unit 17 input changeover switch 18 reverse Associative input signal input terminal 19 Input changeover switch 20 Associative loop control processing unit 21 Attractor convergence state identification processing unit 22 Self-associative neural network 23 Coarse code output type forward three-layer neural network 24 Reverse three-layer Neural network 29 Input terminal of associative input signal 30 Input terminal of self-associative neural network 31 Output terminal of attractor output signal 32 Output terminal of attractor convergence identification signal 40 Forward parallel composite neural network processing unit 41 Three-layer neural network 42 Three-layer neural network 43 three-layer neural network 44 output conversion processing unit 45 output conversion processing unit 46 output conversion processing unit 47 output composite processing unit 50 forward parallel composite neural network processing unit 51 input conversion processing unit 52 input conversion processing unit 53 input conversion processing unit 100 forward Associative initial setting processing 110 Associative loop processing initial setting processing 120 Forward associative input signal input setting processing 130 Forward three-layer neural network processing 140 Reverse three-layer neural network processing 150 Learning internal / external category identification processing 160 Attractor convergence state identification processing 170 Attractor convergence state determination processing 180 Association loop number determination processing 190 Association loop number n = n + 1 setting processing 200 Reverse three-layer neural network output signal input setting processing 210 Association loop number n = 1 determination processing 220 forward associative input signal learning / non-learning input data identification processing 230 attractor identification processing 240 final forward associative input signal determination processing 300 reverse associative initial setting processing 310 forward associative input signal input setting processing 320 attractor convergence state identification processing 330 Forward three-layer neural network output signal input setting process 340 Reverse associative input signal learning / non-learning input data identification process 350 Final reverse associative input signal determination process

Claims (15)

The learning input data can be sent to the forward neural network means 12 which has learned so that the training signal corresponding to the classification category can be sent to the learning input data. Subordinately connecting the backward neural network means 13 learned as described above,
The forward associative input signal is temporarily input to the forward neural network means 12 via the input changeover switch 10 controlled based on the number of associative loops (the number of associative loops n = 0), and the output signal of the backward neural network means 13 is input. After being obtained, the output signal of the backward neural network means 13 is fed back to the input of the forward neural network means 12 via the input changeover switch 10, and the output signal of the forward neural network means 12 and the backward neural network means 13 are fed back. An associative loop process (associative loop number n ≧ 1) for sequentially obtaining output signals of the associative loop, and an associative memory device having an associative loop control processing means 11 for counting and controlling the associative loop number,
Learning of the output signal is performed by comparing the prepared category category number region composed of the category category numbers corresponding to all of the teacher signals with the category category number corresponding to the output signal of the forward neural network means 12. A mutual associative memory device comprising at least learning internal / external category identifying means 15 for identifying an internal / external category. The forward neural network means 12 that has learned so as to be able to send a teacher signal, which is a learning attractor corresponding to the classification category, to the learning input data, and the learning input data can be sent to the teacher signal. The associative memory having the associative loop control processing means 11 for counting and controlling the number of associative loops by using the backward neural network means 13 learned as described above, the forward neural network means 12 being subordinately connected to the backward neural network means 13 In the device,
The backward associative input signal is temporarily input to the backward neural network means 13 via the input changeover switch 17 (the number of associative loops n = 0) to obtain the output signal of the forward neural network means 12, and then the forward neural network means. 12 associative loop processing for feeding back the output signal of the backward neural network means 13 and the output signal of the forward neural network means 12 sequentially by feeding back the 12 output signals to the input of the backward neural network means 13 via the input changeover switch 17. (Associative loop count n ≧ 1)
Learning of the output signal is performed by comparing the prepared category category number region composed of the category category numbers corresponding to all of the teacher signals with the category category number corresponding to the output signal of the forward neural network means 12. A mutual associative memory device comprising at least learning internal / external category identifying means 15 for identifying an internal / external category. The learning input data can be sent to the forward neural network means 12 which has learned so that the training signal corresponding to the classification category can be sent to the learning input data. The backward neural network means 13 trained as described above is cascade-connected, and the forward associative input signal is temporarily input to the forward neural network means 12 via the input changeover switch 10 (number of associative loops n = 0). After the output signal of 13 is obtained, the output signal of the backward neural network 13 is fed back to the input of the forward neural network means 12 via the input changeover switch 10, and the output signal of the forward neural network means 12 and the reverse The output signal of the neural network means 13 Follow obtain associative loop (associative loop count n ≧ 1), be in mutually associative memory device having an associative loop control processing means 11 for performing the control and associative loop count counting,
A plurality of parallel neural networks 41, 42, 43 that have learned respective parallel connection teacher signals having an allocation arrangement different from the teacher signal for the classification category, and the multilayer neural networks 41, 42 , 43 are inversely transformed into output signals of the output code system that is the basis of the teacher signal, and output of the output inverse transformation processing means 44, 45, 46 A first forward parallel complex neural network means 40 that is trained to be capable of sending the teacher signal to the learning input data, comprising at least an output composite means 47 that selectively sends out one output signal from the signal. Using at least the forward neural network means 12;
A comparison process between a classification category number area prepared from classification category numbers corresponding to all of the teacher signals and a classification category number corresponding to the output signal of the first forward parallel composite neural network means 40, A mutual associative memory device comprising at least learning internal / external category identifying means 15 for identifying a learning internal / external category of the output signal. The forward neural network means 12 that has learned so as to be able to send a teacher signal, which is a learning attractor corresponding to the classification category, to the learning input data, and the learning input data can be sent to the teacher signal. The associative memory device having the associative loop control processing means 11 for counting and controlling the number of associative loops by using the backward neural network means 13 learned as described above, the forward neural network 12 being cascade-connected to the backward neural network 13 There,
A plurality of parallel neural networks 41, 42, 43 that have learned respective parallel connection teacher signals having an allocation arrangement different from the teacher signal for the classification category, and the multilayer neural networks 41, 42 , 43 are inversely transformed into output signals of the output code system that is the basis of the teacher signal, and output of the output inverse transformation processing means 44, 45, 46 A first advanced parallel complex neural network means 40 that has been learned and is composed of at least an output composite means 47 that selectively sends out one output signal from the signal, and that can send the teacher signal to the learning input data. At least the forward neural network means 12;
After the reverse associative input signal is once inputted to the reverse neural network means 13 via the input changeover switch 17 (the number of associative loops n = 0) and the output signal of the first forward parallel complex neural network means 40 is obtained, The output signal of the first forward parallel complex neural network means 40 is fed back to the input of the backward neural network means 13 via the input changeover switch 17, and the output signal of the backward neural network means 13 and the first advance Perform an associative loop process (associative loop count n ≧ 1) for sequentially obtaining the output signal of the parallel composite neural network means 40,
A comparison process between a classification category number area prepared from classification category numbers corresponding to all of the teacher signals and a classification category number corresponding to the output signal of the first forward parallel composite neural network means 40, A mutual associative memory device comprising at least learning internal / external category identifying means 15 for identifying a learning internal / external category of the output signal. The learning input data can be sent to the forward neural network means 12 which has learned so that the training signal corresponding to the classification category can be sent to the learning input data. The backward neural network means 13 trained as described above is cascade-connected, and the forward associative input signal is temporarily input to the forward neural network means 12 via the input changeover switch 10 (number of associative loops n = 0). After the output signal of 13 is obtained, the output signal of the backward neural network 13 is fed back to the input of the forward neural network means 12 via the input changeover switch 10, and the output signal of the forward neural network means 12 and the reverse The output signal of the neural network means 13 Follow obtain associative loop (associative loop count n ≧ 1), be in mutually associative memory device having an associative loop control processing means 11 for performing the control and associative loop count counting,
Input conversion processing means 51, 52, 53 for converting the learning input data into different parallel learning input data, and a multi-layer neural network that learns the parallel learning input data and the teacher signal connected in parallel. Network 41, 42, 43, and output composite means 47 for selectively sending out one output signal from the output signals of the multilayer neural networks 41, 42, 43, and the teacher signal for the learning input data At least the forward neural network means 12 is configured with a learned second forward parallel complex neural network means 50 that can be delivered;
By comparing the prepared category category number area composed of the category category numbers corresponding to all of the teacher signals with the category category number corresponding to the output signal of the second forward parallel composite neural network means 50, A mutual associative memory device comprising at least learning internal / external category identifying means 15 for identifying a learning internal / external category of the output signal. The forward neural network means 12 that has learned so as to be able to send a teacher signal, which is a learning attractor corresponding to the classification category, to the learning input data, and the learning input data can be sent to the teacher signal. The associative memory device having the associative loop control processing means 11 for counting and controlling the number of associative loops by using the backward neural network means 13 learned as described above, the forward neural network 12 being cascade-connected to the backward neural network 13 There,
Input conversion processing means 51, 52, 53 for converting the learning input data into different parallel learning input data, and a multi-layer neural network that learns the parallel learning input data and the teacher signal connected in parallel. Network 41, 42, 43, and output composite means 47 for selectively sending out one output signal from the output signals of the multilayer neural networks 41, 42, 43, and the teacher signal for the learning input data At least the forward neural network means 12 is configured with a learned second forward parallel complex neural network means 50 that can be delivered;
After a reverse associative input signal is input once to the reverse neural network means 13 via the input changeover switch 17 (number of associative loops n = 0) and an output signal of the second forward parallel complex neural network means 50 is obtained, The output signal of the second forward parallel complex neural network means 50 is fed back to the input of the backward neural network means 13 via the input changeover switch 17, and the output signal of the backward neural network means 13 and the second forward Perform an associative loop process (associative loop count n ≧ 1) to obtain the output signal of the parallel composite neural network means 50 sequentially,
By comparing the prepared category category number area composed of the category category numbers corresponding to all of the teacher signals with the category category number corresponding to the output signal of the second forward parallel composite neural network means 50, A mutual associative memory device comprising at least learning internal / external category identifying means 15 for identifying a learning internal / external category of the output signal. In the mutual associative memory device according to any one of claims 1, 3 and 5,
Through successive associative loop processing, the respective agreements between the two input signals of the forward neural network means 12, 40, 50 obtained and between the corresponding two output signals and between the two output signals of the backward neural network means 13 are obtained. An associative memory device comprising attractor convergence state identification means for identifying an attractor convergence state based on a state. In the mutual associative memory device according to claim 2, 4 and 6,
Through a continuous associative loop process, the respective agreements between the two input signals of the backward neural network means 13 and the corresponding two output signals obtained, and between the two output signals of the forward neural network means 12, 40, 50 are obtained. An associative memory device comprising attractor convergence state identification means for identifying an attractor convergence state based on a state. In the attractor convergence state identifying means 14 of claim 7,
Associative loop processing of n = 0 associative loops for inputting the forward associative input signal to the forward neural network means 12 and obtaining the corresponding output signal and the output signal of the backward neural network means 13, respectively, and then the backward neural network The number of associative loops in which the output signal of the means 13 is fed back via the input changeover switch 10 and inputted to the forward neural network means 12 and the corresponding output signal and the output signal of the backward neural network means 13 are obtained respectively. The forward neural network means based on the matching state between the two input signals of the forward neural network means 12 obtained through the associative loop processing of 1 and the learning internal / external category identification signal of the learning internal / external category identifying means 15 The forward associative input signal input to 12 Mutual associative memory apparatus characterized by learning and non-learning input data identification. In the attractor convergence state identifying means 14 of claim 8,
Associative loop processing of n = 0 associative loops for inputting the reverse associative input signal to the reverse neural network means 13 and obtaining the corresponding output signal and the output signal of the forward neural network means 12, respectively, and the forward neural network means The twelve output signals are fed back via the input changeover switch 17 and input to the backward neural network means 13 so that the corresponding output signal and the output signal of the forward neural network means 12 are obtained. Through the associative loop processing, the backward neural network means 13 receives the matching state between the two input signals of the backward neural network means 13 obtained and the learning internal / external category identification signal of the learning internal / external category identifying means 15. Learning of the input reverse associative input signal Training input data identified in the matching state of, when it is learned the category identification state only, mutual associative memory apparatus is characterized in that identifying the learning input data. In the mutual associative memory device according to any one of claims 7, 8, 9 and 10,
Learning in the attractor convergence state based on the learning inside / outside category identification signal for the output signal of the forward neural network means 12 from the learning inside / outside category identification means 15 and the attractor convergence identification signal from the attractor convergence state identification means 14 An associative memory device comprising: an attractor identification processing means 16 for performing at least the learning / attractor identification in the outside category identification state and the learning / attractor identification in the learning category identification state.   In the attractor identification processing means 16 of claim 11, from the attractor convergence identification signal and the learning inside / outside category identification signal, when the association loop count n = 1, if the attractor is in the convergence state and the in-learning category identification state, If learning / attractor identification, non-learning category identification state, one-shot spurious / attractor identification, associative loop count n> 1, generalized learning / attractor if attractor converged state and in-learning category identification state A mutual associative memory device characterized by performing generalized spurious attractor identification in the case of identification and non-learning category identification states. The learning input data can be sent to the forward neural network processing 130 that has been learned so that a teacher signal that is a learning attractor corresponding to the classification category can be sent to the learning input data, and the teacher signal. And at least a backward neural network processing 140 learned as follows:
A forward associative input signal is processed by the forward neural network processing 130 to obtain an output signal. Further, the output signal is inputted and an output signal is obtained by the backward neural network processing 140, and then the output signal is converted to a backward neural network. Interactive associative memory processing software having an associative loop number determination process 180 for performing an associative loop process in which feedback is input via the network output signal input setting process 200 and an output signal is obtained by the forward neural network process 130, and counting the number of associative loops There,
By comparing the classification category number corresponding to the output signal of the forward neural network processing 130 with the classification category number region composed of the classification category numbers corresponding to all the teacher signals of the forward neural network processing 130, the output Learning internal / external category identification processing 150 for identifying the learning internal / external category of the signal;
Through the continuous associative loop processing, based on the respective coincidence states obtained between the two input signals of the forward neural network processing 130 and the corresponding two output signals, and between the two output signals of the backward neural network processing 140. Attractor convergence state identification processing 160 for identifying the attractor convergence state;
The coincidence state between the two input signals of the forward associative input signal and the input signal of the forward neural network processing 130 at the associative loop number n = 1, and the learning internal / external category identification signal of the learning internal / external category identification processing 150, A forward associative input signal learning / non-learning input data identification process 220 for identifying the forward associative input signal learning / non-learning input data;
Based on the learning inside / outside category identification signal from the learning inside / outside category identification processing 150 and the attractor convergence state identification signal from the attractor convergence state identification processing 160, if the attracting state is a non-learning category identification state, spurious attractor identification is performed. And an associative memory processing software characterized by having at least an attractor identifying process 230 for at least learning / attractor identification in the category identification state within learning. The learning input data can be sent to the forward neural network processing 130 that has been learned so that a teacher signal that is a learning attractor corresponding to the classification category can be sent to the learning input data, and the teacher signal. In the associative memory processing software having at least the backward neural network processing 140 learned as described above and the associative loop number determination processing 180 for counting the number of associative loops,
A reverse associative input signal is processed by the reverse neural network processing 140 to obtain an output signal. Further, the output signal is input to obtain an output signal by the forward neural network processing 130, and then the output signal is converted to the forward neural network. Perform an associative loop process to input feedback through the network output signal input setting process 330 and obtain an output signal in the backward neural network process 140;
By comparing the classification category number corresponding to the output signal of the forward neural network processing 130 with the classification category number region composed of the classification category numbers corresponding to all the teacher signals of the forward neural network processing 130, the output Learning internal / external category identification processing 150 for identifying the learning internal / external category of the signal;
Through the continuous associative loop processing, based on the respective coincidence states obtained between the two input signals of the backward neural network processing 140 and the corresponding two output signals, and between the two output signals of the forward neural network processing 130. Attractor convergence state identification processing 320 for identifying the attractor convergence state;
The coincidence state between the two input signals of the reverse associative input signal and the input signal of the reverse neural network processing 140 at the associative loop number n = 1, and the learning internal / external category identification signal of the learning internal / external category identification processing 150, Reverse associative input signal learning / non-learning input data identification processing 340 for identifying the forward associative input signal learning / non-learning input data;
On the basis of the learning internal / external category identification signal from the learning internal / external category identification processing 150 and the attractor convergence identification signal from the attractor convergence state identification processing 320, the spurious / attractor identification is performed in the attractor convergence state and the non-learning category identification state. And an associative memory processing software characterized by having at least an attractor identifying process 230 for at least learning / attractor identification in the category identification state within learning. 15. In the attractor identification processing 230 according to claim 13 and claim 14, from the attractor convergence identification signal and the learning inside / outside category identification signal, when the associative loop count n = 1, the attractor convergence state and the in-learning category identification state If there is a learning / attractor identification and non-learning category identification state, if one-shot spurious / attractor identification and associative loop count n> 1, the attractor is converged, and if it is in-learning category identification state, generalization Interactive associative memory software characterized by generalized spurious attractor identification in the learning / attractor identification and non-learning category identification states.

Images