JPH04505678A - Parallel distributed processing network featuring information storage matrix - Google Patents

Abstract

(57) [Summary] This bulletin contains application data before electronic filing, so abstract data is not recorded.

Description

[Detailed description of the invention] Features an information storage matrix parallel distributed processing network field of invention The present invention provides an [NXN] information storage matrix in which connection weights satisfy the following determinant: Regarding a parallel distributed processing network characterized by being defined by [A] .

[A] [TI = [TI [Δ] (1) However, [Δ] means that the element is [TI is a diagonal matrix [N The column is formed from a certain number M of target vectors (where M<=N), and the remaining is formed from a predetermined number Q slack vectors (where Q = N-M) with columns of [N It consists of

Background of the invention Parallel distributed processing network (also widely called ``Nuron network'') ) has proven to be useful in solving a wide range of complex problems in an analog manner. is known. These networks consist of multiple linear and nonlinear amplifiers is a type of highly parallel computing circuit with The transfer function that defines the input-output relationship in the network connected to the input of the amplifier is I have it. This type of network consists of hardware (discrete components or integrated circuits) or traditional von Neumann-type devices. This is realized by a simulation using a digital combination computer.

For certain problems, this type of network can Digital combinations are considered more suitable than Niichiyo. parallel distributed processing network Examples of the kinds of problems where networks are used include associative memory, classification applications, and so on. These include processing, feature extraction, pattern recognition, and logic circuit implementation. These applications are commonly found in systems intended for process control, signal and data processing. It is something. For example, if the inventor of this application ), pending U.S. patent application no. 07/316,717, (Assigned to the Assignee of the Invention) is being developed using a learning parallel distributed processing network. Discloses an apparatus and method for controlling a process.

Parallel distribution is traditionally used to solve problems in the areas listed above. Many types of processing networks are known. commonly used Hopffield's design philosophy for parallel distributed processing networks is ield) algorithm (J, J. Hopfield, ``Response graded The neuron has set computational properties similar to those of the two-state neuron. ceeclingNational Academy of Science, U SA Vol, 81. pages3088-3092. May 1984゜Bi physics) and backpropagation algorithms (e.g. Rumelhart, H. inton, and Williams, “Learning internal representations by error propagation, J.P. Arallel Distributed Processing Explorer ations in the Microstructure of Cognit ion Volume I.

Foundations, Rumelhart and McClelland editors.

MIT Press, Cambridge.

Massachusetts (1986)). Ru.

A parallel distributed processing network is divided into one or more areas surrounded by a basin. N-dimensional vector of topology shape consisting of localized energy minimum points or equilibrium points on It can be conceptualized as a spatial representation, and when unknown inputs are given, network behavior is energy efficient. It is known that it is advantageous to concentrate on the minimum point or equilibrium point. There is. Furthermore, matrix mathematics accurately predicts the properties of n-dimensional vectors in other fields. Since it has already been demonstrated that It is advantageous to characterize it and analyze its behavior using traditional matrix techniques. It is also known that

Both Hopfield and backpropagation networks share the principles described below. It is designed to use the same algorithm. (1) Required output amount (location) the desired output vector (also called the goal vector) The behavior is that given some (or any) input code (input vector), The network outputs one of the target vectors. (2) Net The network is a linear operator (A) (which is a matrix of constant coefficients) and V can be characterized by a nonlinear thresholding device denoted by (,) The coefficients of the 0 matrix (A) determine the connection weights between amplifiers in the network, ν(, ) represents the synaptic action on the input and output sides of each amplifier. This kind of parallelism A fundamental problem in designing distributed processing networks is that some (or any) input When the vector X In is given, a certain desired output X0 from [A] and V(,) The goal is to find a linear operator (A) that yields . That is, [A] X, , l->X, (2) In the above equation, the operation using ν(,) is implicitly assumed.

Hopfield and backpropagation models perform matrix operations using several different techniques. I am pregnant with my child [A].

In the Hopfield algorithm, the operator [A1 basically represents the desired output vector. calculate the sum of the matrices obtained by the cross product operation for the vector or target vector obtained by That is, Xol,X,,21. ,,Xon reaches the desired target base. If it is a vector, it will be as follows.

A=X-1X”-+ X-*Xtat +, , , +X, rlXt-7 Then, Xt61 is the arrangement matrix of X01, and XalX”61 is 1=11− 0. + represents the cross product when n. In this case, operator [A] is Changed by placing . When such a matrix operator [A] is created, Given an input vector X-, we can use an iterative procedure to obtain the desired output. can. That is, [A]XII, =x1 [A] X, =X! (4) In the case of the above equation as well, the operation using ν(,) is implicitly assumed.

Unfortunately, this algorithm can converge to a result that is not the desired target vector. This is because this algorithm is designed that way. Sara Another limitation of this algorithm is that the network is When the network behaves as a virtual memory, convergence to a stable state depends on its input. Only when the force is close to the steady state (Hamming distance). This is a relief. This is one of the important limitations of the Pfield network. Furthermore, the results converge There is very little control over how quickly and how it is done. This means that the coefficients in the matrix ( The connection weights between amplifiers in a parallel distributed processing network are This is because it is determined "fixed" by calculation. In other words, flexibly reconstruct [A] This is because it cannot be done. In particular, (when performing a cross product operation, [A] always becomes symmetrical). That is.

U.S. Patent No. 4.660,166 (Hopfield), the associative memory network is an interconnected The output of each amplifier is connected to the inputs of all other amplifiers except that one. is connected to. In the Hopfield network disclosed in the above patent, requires that the connection matrix characterizing the connection weights be symmetrical, and the diagonal We need the elements above to be equal to zero. (Hopfield paper cited above In Figure 2, an amplifier is connected to its input with reverse current, which is incorrect. It seems to be. Figure 2 of the above patent corrects Hopfield's interconnection scheme. This seems to be a good representation. ) Hopfield's network is the basis for a variety of applications. For example, U.S. Pat. No. 4,719,591 (Hopfield and Tank ), networks are applied to the problem of decomposing signals into component signals. . U.S. Patent Nos. 4,731,747 and 4,737,929 (both Denke r) to control the convergence speed to improve the Hopfield network. Adjust the amplifier time constant so that the Clicks with only two values allow the construction of networks with narrow leads. The connection matrix that was created is used.

U.S. Patent No. 4,752,906 (Kleinfeld) is fed back into the input interconnection network. Temporary associations cannot be made by using delayed elements in the output. It eliminates the shortcomings of Hopfield's network. U.S. Patent No. 4,755 , No. 963 (Denker, Howard, and Jackel) We are further expanding the range of problems that can be solved with field networks.

The backpropagation network algorithm uses a multilayer feedforward network. A network is constructed and this network evaluates A using performance criteria ( The coefficients in A are adjusted to minimize the output error), and this method yields good results. However, this is a computationally intensive method. This means that convergent learning takes time. means to earn money. Backpropagation networks are extremely difficult to learn information to be memorized. It takes time. Many techniques have been developed to reduce this learning time. . For example, pending U.S. patent application Ser. No. 07/285.34 filed by the present inventor (filed on December 16, 1988 (ED-9367)) describes a nonlinear differential equation. We are using this to train backpropagation networks.

Summary of the invention The present invention consists of multiple amplifiers, or nodes, connected in a single layer, each amplifier A spreader relates to a parallel distributed processing network that has an input and a power. These The output of each node is connected to the net by each line with a predetermined connection weight. Some or all of the inputs of other nodes in the work (including their own inputs that are fed back) (including power). The connection weights are called the "information storage matrix" is defined by the [N　X　Nl matrix [A]. in this matrix The elements of the information storage matrices A and J are the fifth input node and the first output node. connection weight between the nodes.

According to the present invention, the information storage matrix [A] satisfies the following determinant.

[A] [:T] = [T] [Δ] (1) Matrix [A] is [NXN ] matrix, which is called a similarity deformation matrix, whose columns are a predetermined number of columns. In the [NXl]Xl hector of (M), a predetermined number (Q) of [NX1] undefined or rack vector. Each target vector is parallel distributed The value of 0M representing one of the outputs of the processing network is <=N and Q=(N -M) can be taken. Preferably, a vector in a similar deformation matrix each is linearly independent from all other vectors in that matrix. Ru. Each vector in a similar deformation matrix is There are cases where it is orthogonal to the vector of , and cases where it is not orthogonal to the vector.

If matrix [T] is not singular and matrix [T]-' exists, The information storage matrix [A] is defined as a matrix product.

[A] = [T] [A] [T] -’ (5) Matrix [A] is [N XN] is a diagonal matrix, where each element on the diagonal represents the location of the target or slack vector. corresponds to one of the [A] The relative value of each element on the diagonal of the matrix is The speed at which the output of a parallel distributed processing network converges toward the corresponding target vector It corresponds to the degree. Generally, the value of the element of the diagonal matrix corresponding to the target vector is preferably larger than the value of the element of the diagonal matrix corresponding to the slack vector . Specifically, the elements of the diagonal matrix corresponding to the target vector have an absolute value greater than 1. , whereas the elements of the diagonal matrix corresponding to the slack vector are smaller than l It has an absolute value.

The network according to the present invention has several advantages over the conventional network described above. have points.

Information storage matrix [A] is more generalized. In other words, make it symmetrical There is no need, and there is no need to make it approximately symmetrical. Also, the elements on the diagonal are , does not need to be equal to zero as in the Hopfield network. this This means that hardware implementation is also becoming more common. information The recognition-based behavior of the memory matrix is easier to understand than previous techniques. There is. An input vector is given to the network, and the network When converges to a solution that is not the target vector, it means that the solution based on recognition has been reached. , which is generally a linear combination of target vectors.

Including an indeterminate vector in the formation of a similar deformation matrix reduces the area of the target vector. It can be created with flexibility. Such a slack vector is It does not exist in Pfield's network.

When forming the information storage matrix, the matrix [ΔJ can be included as one of the factors. This is a feature not found in Hopfield's network. of a certain goal The speed of convergence to a solution can be controlled by selecting the values of the [A1 matrix.

In addition to the above, the calculation of the components of the information storage matrix is performed using the back propagation algorithm. It is faster and more computationally efficient than a connection matrix using a system. This is the back The surface propagation algorithm adopts the conceptualized delta rule to construct the connectivity matrix. This is because there is. However, this rule is not used in the information storage matrix. It is at least several times more computationally intensive than numerical methods.

Brief description of the drawing The present invention will now be described in detail with reference to the accompanying drawings, which are a part of this specification. Ru.

FIG. 1 is characterized by the components of an information storage matrix according to the invention. FIG. 2 is a schematic conceptual diagram showing a part of a parallel distributed processing network with connection weights.

Figure 2A shows any increment corresponding to elements in the network with values greater than zero. FIG. 2 is a schematic diagram showing a width transducer together with a feedback resistor and a bias resistor.

Figure 2B shows any increment corresponding to elements in the network with values less than zero. FIG. 2 is a schematic diagram showing a width transducer together with a feedback resistor and a bias resistor.

Figure 3 shows a nonlinear threshold amplifier that realizes the synaptic action of the function ν(,). FIG.

FIG. 4 shows a parallel distributed processing network according to the invention used in Example II. FIG.

The appendix, which is part of this specification, contains information on traditional von Neumann digital computers. A program for implementing a parallel distributed processing network according to the present invention on a computer A code list of the program written in Fortran language is provided. this code - The list is a coding of the network used in Example II. Ru.

Detailed description of the invention In the following detailed description, similar elements in the figures are designated with similar reference numerals. ing.

First, we will explain the underlying theory and mathematics of the parallel distributed processing network according to the present invention. We will explain it from the perspective of the method, and then show various examples of parallel distributed processing networks. This will be explained with reference to the drawings. After that, a parallel distributed processing network according to the present invention The operation of is explained using an example.

As mentioned above, we conceptualize a parallel distributed processing network in the form of an N-dimensional vector space. This is known to be convenient. Each such space is surrounded by an area. It has a topological structure consisting of one or more local equilibrium points, When an unknown input is given, the network behavior becomes concentrated at its equilibrium point. ing. Inputs are typically provided to the network in the form of digital codes. It is. An N-dimensional space can accommodate 2N input codes.

The network according to the invention consists of an [NXN] matrix (hereinafter "information storage matrix") connection between amplifiers to realize a parallel distributed processing network. The feature is that the weight can be specified. Uses information storage matrix When doing so, the movements are symmetrical, so only 2N input commands can be identified. only half of the code. The remaining codes (2N-1) are supplementary.

Generally, the information storage matrix [A] satisfies the following determinant [N　X　N1 There is a line.

[A] [T] = [T] [Δ] (1) Equation (1) is based on each λ, that is, [Δ ] Each element of the matrix is an eigenvalue, and each column vector in the similar transformation matrix [T] is an associated eigenvalue. We define the problem of eigenvalues, which are eigenvectors. Equation (1) has the maximum n can have solution bears. This determinant can be calculated using Gaussian elimination or delta You can solve the problem using rules.

When [T]-' exists, the information storage matrix [A] is obtained by the following matrix product It is formed.

[A] = [T] [L] [T] -' (5) Matrix [T] is a "similar transformation matrix" called [N XN] matrix. Each target vector is contained in an N-dimensional space representing the network. It is in the form of one of 2N possible codes. Each target vector is processed in parallel and distributed represents one of the desired outputs, ie, target values, of the physical network. Each target vector It is desirable to store it in some way and retrieve it at some point in the future. It contains new information, and the M=N target vector [X+, Xs, -, -, XM ] is the basis of network information. Preferably in a set Since each target vector in is linearly independent from other target vectors, it is N-dimensional. Any vector xl in space is a linear combination of a set of target vectors. It can be expressed as In this case, the reciprocal of the similar transformation matrix [T] exists do. Some or all of the M target vectors may be made orthogonal to each other as necessary. You can also make them orthogonal.

The number M of target vectors can be less than or equal to N, which is the number of dimensions of the information storage matrix. can. When the number of target vectors is specified as N or less (that is, M<N), the type The remaining target vectors of the similar deformation matrix (T) are [NX1] indefinite vectors of a predetermined number Q. , that is, completed by the slack vector (where Q=NM).

Slack vectors do not require a data format to represent the target vector; When viewed from the perspective of memory, it is imaginary. However, many (applications) The slack vector has been demonstrated to be important in this application. slack ・Select the vector so that the slack vector is the best code for the network. It is done in such a way that it does not represent one of the For example, in a typical case , the target vectors are each n bits long (i.e., binary 1 and -1, i.e. , [1-11,, -1-11]) Ru. Forming slack vectors from the same binary digits suppresses the corresponding code. It will be. Therefore, it is a binary number that is clearly distinguished from the 2N target vectors. A slack vector needs to be formed. Generally, values after the decimal point, zero formed by one (or more) of elements with positive or negative integer values is possible.

The slack vector is important because it delineates the topology and This is because the area of N-dimensional space can be formed accordingly.

To summarize the above, depending on the value of the number M, the target vector is the information record of the matrix [A]. can form all or part of the storage spectrum. Set the target vector to N or more. When specified below, the remaining vectors in the deformation matrix are indeterminate or slack vectors. Becomes Kutle. In all cases, the vectors in the similarity transformation matrix [T] are Form the information storage matrix [A] into a geometric spectral shape (i.e. [A ).

[The A3 matrix represents the set of all eigenvalues of the information storage matrix [A] [N　X　N] is a diagonal matrix and is known as the algebraic spectrum of [A].

[Each element of the A1 matrix corresponds to a respective one of the objectives or slack vectors are doing. [The convergence characteristic of the network is determined by the values assigned to the elements of the A1 matrix.] Gender is determined. [Since there is a degree of freedom in selecting the value of the A1 matrix, the speed of the network can be Can be controlled. Therefore, after the initial setup, the network has to reach a certain decision. The time or the time until convergence to a certain target value is determined by selecting the value of the A1 matrix as appropriate. Control is possible by doing this.

[The values assigned to the elements of the A1 matrix have influence in the network of the present invention.] ing. If the pre-assigned value is λ1〉1, then the corresponding eigenvector T , (which contains the desired output information) determines the asymptote in the N-dimensional information space. Therefore, the occurrence of the desired event is stimulated. The pre-assigned value is λ1 1, then the asymptote in the N-dimensional information space is determined by the corresponding eigenvector T. will be determined, and the occurrence of the event will be suppressed. The pre-assigned value is λ1 〉〉1, the network rapidly converges to the corresponding target vector and back propagation This approximates the feedforward effect of a network.

Preferably, the values assigned to the elements of the [A1 matrix corresponding to the target vector are It is larger than the value of the element of the [A1 matrix corresponding to the slack vector.

Specifically, the elements of the diagonal matrix corresponding to the target vector have an absolute value greater than 1. , and the elements of the diagonal matrix corresponding to the slack vector also have an absolute value less than 1. ing.

To summarize the above, by assigning λ, and choosing the slack vector, , the convergence speed of the network is increased and the area associated with a fixed equilibrium point is made more flexible. It can be formed by

Two methods can be used to evaluate the information storage matrix [A]. these Methods include the Gaussian removal method and the delta rule method.

We first explain the evaluation of the information storage matrix [A] using the Gaussian removal method. I will clarify.

X + + X * +, -I Let s be the basis of RN (i.e., Xl represents the target vector and ZI is the slack vector). However, RN represents the N-dimensional real vector space. There is.

Using this basis structure, the similarity transformation matrix [T]=EX1. Xs9. ,,XM, Zm-+, , , Zs]. If we relate this to a diagonal matrix, we get the following Ru.

This diagonal matrix contains predefined eigenvalues for each element in the basis.

Next, create the determinant as follows, [A] [T] = [T] [Δ] (1) Solution of [A] using Gaussian removal method I put it on. This is very useful in solving problems.

[T] t [A]’= [Δ] [T]” (6) Transfer equation (1) as above. This is because the matrix coefficients take a natural form with respect to the coefficients of [A]. Formula (1 ) or from equation (6), an N2 linear combination equation is obtained, which allows N2 of [A] The coefficient is calculated.

Another method is the delta rule method. Next is a set of linear equations.

A　X　l=See IXI AXM=λ,X, (7) AZw+r=λ No, λ, Z. 1 AZi=λNZ? 1 In the above equation, λ1 is a predefined eigenvalue.

Now, if we repeatedly apply the delta rule to the linear equation [7], we can evaluate [A]. can be valued.

The Delta Rule is by W. P. Jones and J. Ho5kins rBack − "propagatton", Byte, pages 155-162, 0ctob er 1987.

As you can see by comparing the above two methods, the Gaussian removal method is better than the delta rule. It is also fast. [T] If the reciprocal of the matrix exists, the information storage matrix can be expressed as It can be obtained by calculating the matrix product in (5).

FIG. 1 is a schematic diagram conceptualizing a part of the parallel distributed processing network according to the present invention. be. The network, generally designated 10, includes a plurality of amplifiers or nodes. 12 are connected within a single layer 14. The network 10 includes N amplifiers 12 -1 to 12-N. However, N is the information storage matrix obtained using the method described above. It corresponds to the number of dimensions of the box [A].

Only four amplifiers 12 are shown in FIG. That is, the first amplifier 12 -1, i-th amplifier 12-i, j-th amplifier 12-j, last amplifier 12- It is N. Connection relationship of other amplifiers among the N amplifiers configuring the network This should naturally be understood by looking at Figure 1. The more concrete example is FIG. 4 is a schematic diagram of FIG. Figure 4 shows the parallel distributed processing network used in Example 2. A specific example of H.10 is shown. In this case, N is equal to 4 and the fully connected A network 10 is shown. Resistors used in the network shown in Figure 4 Specific values of are also shown.

Each amplifier 12 has an inverting input port 16, a non-inverting input port 18, and an output port 2. 0. The output port 20 of each amplifier 12 has a feedback resistor 22 is connected to its own inverting input port 16 via the line where it is placed.

Additionally, the output port 20 of each amplifier 12 may be affected by the threshold nonlinearity or It is connected to a squasher 26 having a synaptic squash function ν(,). child A detailed view of the squasher 26 is shown in FIG. Each of the N amplifiers 12 The signal appearing at the output port 20 of is processed by the squasher 26, and then Some or all of the other amplifiers 12 in the network 10 through the connection line 3o is connected to either the inverting input port 16 or the non-inverting input port 18 of. to this This will be discussed later.

The connection between the output of one amplifier and the input of another amplifier is connected to the information storage matrix [A] Determined by the value of the corresponding element in , an element value of zero means no connection. , when the element value is positive, it means that it is connected to the non-inverting input, and when the element value is negative, it means that it is connected to the non-inverting input. , means connected to the inverting input.

Each connection line 30 has a connection resistance 34, which has the same variables i and j subscripts. It is shown with This subscript indicates that the connection resistance 34 with that subscript is the jth is connected to the line 30 connecting the input amplifier of and the i-th output amplifier. It means that. The value of the connection resistor 34 is determined by the corresponding addition in the information storage matrix. It is associated with a labeled variable. This will be discussed later.

Each of the lines 34 is also provided with a delay element 38. This delay element is A signal delay time of allow the time sequence of each iteration necessary to realize This is the time it takes for a certain amplifier 12 to reach its output state. delay The line has the same subscript variables as the connecting line and its resistance. It is. As mentioned above, the value assigned to the eigenvalue in the matrix [Δ] is The time (or number of iterations) required for the network 10 to reach a certain decision It corresponds to

Corresponding elements of the information storage matrix [A] and amplifiers in the network 10 Regarding how the value of the connection resistance 34 is realized based on the gain of 12 , will be explained below with reference to FIGS. 2A and 2B.

The input vector input to the network 10 has the following format.

When the information storage matrix [A] is evaluated using the method described above, it has the following form: Become.

In the above formula, each element A +, J of the information storage matrix [A] is positive or negative. Is a real constant or zero.

A certain element A of the information storage matrix [A] (when J is positive, that element A1. , the feedback resistance 22 (RF) and the connection resistance 34 (R, , , ) can be understood from FIG. 2A. Resistor 34 (R1, J) is connected to the non-inverting input port 16 of the amplifier 12 and the amplification line 30 The gain of the device 12 is obtained from the following equation.

eo / XJ = [1 + (RF / R1, J)] = IA,, JI In the above equation, eo is the voltage of the output signal appearing at the output port of amplifier 12-j. In a typical example, the value of the feedback resistor 22 for the entire network is It is fixed at a fixed value, and the values of connection resistance R+ and J can be easily determined from equation (8). can.

When a certain element AIJ of the information storage matrix [A] is negative, that element AI Value of J, feedback resistance 22 (Rr) and connection resistance 34 (R1, J) The relationship can be understood from FIG. 2B. Resistor 341, in this case , line 30 connected to the inverting input port 18 of amplifier 12 and The gain is obtained from the following equation.

eo /XJ = RF /R1, J = l A1. J1 In the above formula, e o is the voltage of the output signal appearing on the output port 20 of the amplifier 12. In this case too, The value of the feedback resistor 22 of the entire network is fixed at a predetermined constant value. Therefore, the value of the connection resistance R1 can be easily determined from equation (9).

As can be understood from the above, the values of elements A + and J of the information storage matrix are If it is between zero and one, you can use hardware or software techniques to resolve the problem. (That value can be achieved. For example, when using software methods, A6. The coefficients may be adjusted so that the value of , becomes larger than 1. hardware hands When using the method, two inverting amplifiers can be connected in cascade to A positive value will be obtained.

When a certain element AI of the information storage matrix [A] is zero, the fifth input node This means that the node and the first output node are not connected.

Figure 3 shows a nonlinear threshold setting device, that is, a squash system that generates a function destination (,). FIG. The squasher 26 sets the output of the node 12 to a predetermined upper limit value and a predetermined upper limit value. The network is defined to be limited to the range defined between the lower limit of .

Typically, the upper limit value and lower limit value are +1 and -1, respectively.

As understood from the above, the network 10 shown in the schematic diagrams of FIGS. 1 to 4 is , it is possible to physically construct it in any convenient form. That is, Network 10 can be implemented in electronic hardware or optical hardware. It is within the technical scope of the present invention to implement the It belongs to When implemented in electronic hardware, discrete Analog devices, amplifiers, resistors, and delay elements such as capacitors and RC networks integrated circuits by interconnecting network components using by interconnecting the components or by using a suitable base, indicated by the letter S in the figure. The entire network can be integrated on the board using any integrated circuit manufacturing technology. It is also possible to realize this by In addition, the network Such as Paracard Vectra, DEC VAX, Klee X-MP, etc. It is executed using a general-purpose digital computer that operates according to a program. It is also possible to express Regarding this, there is a code written in Fortran language in the appendix. A code list is posted. Network 10 follows this code list. It is implemented in DEC's VAX. The network shown in Example II uses this code ・Implemented as a list.

Tradition Below, we will explain the operation of the parallel distributed processing network of the present invention using an example and an example II. Refer to and explain.

This example is an example of using a parallel distributed processing network as a classifier. A big plan The business requires the collection and processing of personal data files of its employees. There is. The data collected reflects personal profiles such as:

This profile is entered into a computer file each time a new employee joins the company. It will be done. If you are a new employee, married, never divorced, have a college degree, are male, and have no children. The employee item has the following format:

full name: John Doe Employees: Male/Female: Single/Married Divorced or not 2 Children or not 2 University graduate or not: Therefore, each employee has a 6-bit code and a personal profile associated with their name. It describes the rules. The name and code are entered together into the data file.

The “employee” item was included because it is a network characterized by the information storage matrix. This is to operate the rack 10 symmetrically.

This company employs thousands of people and employees according to the information provided in their profile codes. We need a high-speed parallel distributed processing network to classify. Therefore, this company wants to know the names of each employee belonging to the following groups. In other words, (1) male Single by gender, (2) single by female, and (3) married by female. List these three groups Then, it becomes as follows.

Category 1: Male and single In the above table, "dc" means "ignored".

Furthermore, it is assumed that the following classification is also necessary.

Classification 2: Classification 3: Classification 4: Using the above four tables, the target vector can be generated. 4 classifications Examining the situation of the man, we obtain the target vector.

Similarly, examining the four categories of women's situations yields two additional target vectors. It will be done.

As is clear, there are three target vectors and six dimensions of information. follow Therefore, three more slack pects are required. Therefore, in the following matrix, R' The standard basis set in is defined as follows.

Next, add Z4 = e4% Zs = es, Zs = es to three additional slacks ・Select it as a vector and create a similar transformation matrix as follows.

T,: [x,,X,,X,,Z,,zs,Z,] Also, select the next diagonal matrix do.

An information storage matrix is obtained from T1 and 8. As long as this matrix is possible for the code (i.e. 32 elements in the code under consideration) When executed, the four zones shown in Table 1 are obtained. The first area in Table 1 is the target It shows all elements of the code that converge to the value X1. Similarly, the third and fourth The area is responsible for X and X. Each code belonging to these target areas is Increment by an appropriate counter. However, the second area is of interest here. The solution C = [1, 1゜-1, 1-1, 11'- by recognition that is not (In other words, from C, the classification is employee, male, married, divorced, with children, and university graduate. ).

When the codes associated with an employee's name converge to X, the employee's name is male. I'm a big fan of the single class. Similarly, if this code converges to X2 or xs , the name can be entered into Women and Single or Women and Married. However, the code converges to C In this case, the name is ignored. The arrows in Table 1 indicate common information within each area. Ru. Therefore, the information storage matrix of a parallel distributed processing network that executes a function of classification 1 is T1 and A are used to design the lix.

Furthermore, using [Δ] in the following similar transformation, Tm = [Xl, Xz, Xss Z 4. Za, zal However, Z 4 = ez, Zs = es s Zs:Q .

T-=[X3. x,,Xs,Z4. Zs, zal However, Z　a　=em　s Zs = e4. Zs = esT4 = [X3. Xs, X-, Z4. Zs, Za l However, Za = es + Zs = 84. Zs = es Tables 2.3 and 4 is obtained. These tables give classifications 2.3 and 4, respectively.

Next, assume that it is necessary to obtain the information shown in categories 5.6 and 7 below.

Classification 5: Classification 6: Classification 7: Therefore, the same target vector and [Δ] can be used.

The new analogous transformation is as follows.

TI = [Xl, xs, xs, z4. zs, zsl However, Z4 = e ,,Zs=94. Ze = esTs = [Xl, Xs, xs, Z4. Zs , Zako However, Z 4”6m + Zs = es + Zs = e*T, = [Xl, X-, Xs, Z4. Zs, Z6] However, Za; e*, Zs The results for the = es + Za = e4 area are shown in Tables 5.6 and 7.

Example II Realization of logic circuit For the case where the 24-bit left-right symmetric complementary code is as follows: I will consider it.

Next, when C=D, state X,=[1,1,-1°-1]t is executed, and C is When it is not equal to D, the state X, = [1, -1, 1, -1]'' is obtained. It is necessary to design a logical circuit. For this purpose, consider the following similar transformation.

T=[X,,Xa+Zl,Z4], where Zs”e++Z4= From e* and these matrices, the connection pattern shown in the following information storage matrix is obtained.

When this matrix is iterated and squashed, as specified in the figure, The target values X and XY area shown below are obtained.

11-1-1 Goal 11-11 1-1-1-1-1 1-1 1-L target aircraft, this logic is analog is executed and realized by the programming circuit. In this network, there is no solution based on recognition. It should be noted that

A schematic diagram of the parallel distributed processing network used in this Example II is shown in Figure 4. It is. Therefore, [Y] = [A] [Xl, So, Y, = +, 5Xl -2,5X.

y, = -15Xx -2,5Xs The value of the resistor (assuming an IK feedback resistor) is determined using equations (8) and (9). I can be treated. The Fortran code list shown in the appendix is shown in Figure 4. I coded a network to run on a DEC VAX computer. It is something that

The various embodiments described above can be modified in various ways, but such changes may be made upon request. It belongs to the technical scope of the present invention as defined in the scope of the invention.

(Margin below) 10:lO-0-5-0-5 Lambda -2, CIo 2.00 -0.50 -0+50code e lea+ent: xlXiXiXN international search report +nlmmmnml1llall+l1lfi Me, pc’r Hachiko90102 699

Claims (1)

[Claims] 1. It consists of multiple nodes connected in a single layer, each node has an input and an output, and the output of each node connects to a given connection to the input of a given number of nodes in the network. In a parallel distributed processing network connected by connection weights, the connection weights are defined by the [N×N] information storage matrix [A], and the elements Ai,j of the information storage matrix [A] are connected to the j-th input node and i It is the connection weight between the output nodes of It is an [N×N] similarity deformation matrix whose columns are formed by adding a predetermined number (M) of [N×1] target vectors to a predetermined number (Q) of [N×1] indefinite vectors. A vector represents one of the outputs of a parallel distributed processing network. (where M<=N and Q=(NM)), "Λ" is a [N×N] diagonal matrix, and each element on the diagonal of the matrix [Λ] is a similar transformation matrix. , and each element on the diagonal of the matrix [Λ] The relative value of a parallel distributed processing network corresponds to the speed at which the parallel distributed processing network converges toward the corresponding target vector. 2. 2. The parallel distributed processing network of claim 1, wherein the output of each node is connected to all inputs of the nodes in the network, including itself. 3. 2. The parallel distributed processing network of claim 1, wherein all of the vectors in the similar transformation matrix are linearly independent. 4. According to claim 3, all of the vectors in the similar transformation matrix are orthogonal. parallel distributed processing network. 5. 3. The parallel distributed processing network according to claim 2, wherein all of the vectors in the similar transformation matrix are linearly independent. 6. According to claim 5, all of the vectors in the similar transformation matrix are orthogonal. parallel distributed processing network. 7. Each element of the diagonal matrix [Λ] corresponding to the indefinite vector in the similar transformation matrix is equal to the value of each element of the diagonal matrix corresponding to the target vector in the similar transformation matrix. 6. The parallel distributed processing network according to claim 5, wherein the parallel distributed processing network has a value that is even smaller than that of . 8. Each element of the diagonal matrix [Λ] corresponding to the indefinite vector in the similar transformation matrix is equal to the value of each element of the diagonal matrix corresponding to the target vector in the similar transformation matrix. 4. The parallel distributed processing network according to claim 3, wherein the parallel distributed processing network has a value that is even smaller than that of the network. 9. Each element of the diagonal matrix [Λ] corresponding to the indefinite vector in the similar transformation matrix is equal to the value of each element of the diagonal matrix corresponding to the target vector in the similar transformation matrix. 3. The parallel distributed processing network according to claim 2, wherein the parallel distributed processing network has a value that is even smaller than . 10. Each element of the diagonal matrix [Λ] corresponding to the indefinite vector in the similar transformation matrix has a value smaller than the value of each of the components of the diagonal matrix corresponding to the target vector in the similar transformation matrix. Parallel distributed processing network according to claim 1 nine. 11. Each element of the diagonal matrix [Λ] corresponding to the indefinite vector in the similar transformation matrix has an absolute value smaller than 1, and each of the components of the diagonal matrix corresponding to the target vector in the similar transformation matrix 6. The parallel distributed processing network according to claim 5, wherein: has an absolute value greater than 1. 12. Each element of the diagonal matrix [Λ] corresponding to the indefinite vector in the similar transformation matrix has an absolute value smaller than 1, and each of the components of the diagonal matrix corresponding to the target vector in the similar transformation matrix 4. The parallel distributed processing network according to claim 3, wherein: has an absolute value greater than 1. 13. Each element of the diagonal matrix [Λ] corresponding to the indefinite vector in the similar transformation matrix has an absolute value smaller than 1, and each of the components of the diagonal matrix corresponding to the target vector in the similar transformation matrix 3. The parallel distributed processing network according to claim 2, wherein: has an absolute value greater than 1. 14. Each element of the diagonal matrix [Λ] corresponding to the indefinite vector in the similar transformation matrix has an absolute value smaller than 1, and each of the components of the diagonal matrix corresponding to the target vector in the similar transformation matrix 2. The parallel distributed processing network according to claim 1, wherein: has an absolute value greater than 1. 15. The parallel distributed processing network according to claim 2, further comprising a delay element placed between the output of each node and the input of the other node to which the output is connected. -k. 16. The parallel distributed processing network according to claim 1, further comprising a delay element placed between the output of each node and the input of the other node to which the output is connected. -k. 17. Connected to each output of a node to lower the value of the node's output to a predetermined upper limit. The claim further comprising a network limiting the range defined between the limits. Parallel distributed processing network according to item 16. 18. Connected to each output of a node to lower the value of the node's output to a predetermined upper limit. The claim further comprising a network limiting the range defined between the limits. Parallel distributed processing network according to item 15. 19. Connected to each output of a node to lower the value of the node's output to a predetermined upper limit. The claim further comprising a network limiting the range defined between the limits. Parallel distributed processing network according to item 2. 20. Connected to each output of a node to lower the value of the node's output to a predetermined upper limit. The claim further comprising a network limiting the range defined between the limits. Parallel distributed processing network according to item 1. 21. Each of the nodes has an inverting input terminal and a non-inverting input terminal, and the i-th output node is the j-th output node when the corresponding element Ai,j of the information storage matrix is greater than zero. 21. The parallel circuit according to claim 20, which is connected to the non-inverting input port of the second input node. Column distributed processing network. 22. The jth input node is connected between its output port and its non-inverting input port. a feedback resistor connected to its non-inverting input port, and a connection resistor connected to its non-inverting input port. Well, the feedback resistance and connection resistance are determined by the information storage matrix by the following relational expression. The arrangement according to claim 21 related to the corresponding element Ai,j of the Column distributed processing network. |Ai,j|=[1+(Rr/Ri,j)]23. Each of the nodes has an inverting input terminal and a non-inverting input terminal, and the i-th output node is the corresponding one of the information storage matrix. 20. The parallel distributed processing network according to claim 19, which is connected to the non-inverting input port of the j-th input node when the element Ai,j is greater than zero. 24. The jth input node is connected between its output port and its non-inverting input port. a feedback resistor connected to its non-inverting input port, and a connection resistor connected to its non-inverting input port. Well, the feedback resistance and connection resistance are determined by the information storage matrix by the following relational expression. The arrangement according to claim 23 related to the corresponding element Ai,j of the Column distributed processing network. |Ai;j|=[10(Rr/Ri,j)]25. Each of the nodes has an inverting input terminal and a non-inverting input terminal, and the i-th output node is the corresponding one of the information storage matrix. element Ai. 21. The parallel distributed processing network according to claim 20, which is connected to the inverting input port of the j-th input node when j is less than zero. 26. The jth input node is connected between its output port and its non-inverting input port. a feedback resistor connected to its non-inverting input port, and a connection resistor connected to its non-inverting input port. Well, the feedback resistance and connection resistance are determined by the information storage matrix by the following relational expression. 26. The parallel distributed processing network according to claim 25, wherein the parallel distributed processing network is associated with a corresponding element Ai,j of a class. −|Ai,j|=Rr/Ri,j 27. Each of the nodes has an inverting input terminal and a non-inverting input terminal, and the i-th output node is the j-th output node when the corresponding element Ai,j of the information storage matrix is less than zero. 20. The parallel fraction processing network of claim 19, wherein the parallel fraction processing network is connected to the inverting input port of the second input node. 28. The jth input node is connected between its output port and its non-inverting input port. a feedback resistor connected to its non-inverting input port, and a connection resistor connected to its non-inverting input port. Well, the feedback resistance and connection resistance are determined by the information storage matrix by the following relational expression. The arrangement according to claim 27 related to the corresponding element Ai,j of the Column distributed processing network. −|Ai,j|=Rr/Ri,j 29. According to claim 1, the network is implemented in a hardware element. parallel distributed processing network. 30. A network is a general-purpose digital computer that operates according to a program. The parallel distributed processing network according to claim 1, which is executed using a computer. 31. According to claim 2, the network is implemented in a hardware element. parallel distributed processing network. 32. A network is a general-purpose digital computer that operates according to a program. 3. The parallel distributed processing network according to claim 2, which is executed using a computer. 33. The parallel distributed processing network according to claim 1, wherein the information storage matrix [A] is the following matrix product when [T]-1 of the matrix [T] exists. [A]=[T1[Λ][T]-1 34. 3. The parallel distributed processing network according to claim 2, wherein the information storage matrix [A] is the following matrix product when [T]-1 of the matrix [T] exists. [A]=[T][Λ][T]-1 35. 6. The parallel distributed processing network according to claim 5, wherein the information storage matrix [A] is the following matrix product when [T]-1 of the matrix [T] exists. [A]=[T][Λ][T]-1 36. 4. The parallel distributed processing network according to claim 3, wherein the information storage matrix [A] is the following matrix product when [T]-1 of the matrix [T] exists. [A]=[T][Λ][T]-1 37. 10. The parallel distributed processing network according to claim 9, wherein the information storage matrix [A] is the following matrix product when [T]-1 of the matrix [T] exists. [A]=[T][Λ][T]-1 38. 11. The parallel distributed processing network according to claim 10, wherein the information storage matrix [A] is the following matrix product when [T]-1 of the matrix [T] exists. [A]=[T][Λ][T]-1 39. 14. The parallel distributed processing network according to claim 13, wherein the information storage matrix [A] is the following matrix product when [T]-1 of the matrix [T] exists. [A]=[T][Λ][T]-1 40. 15. The parallel distributed processing network according to claim 14, wherein the information storage matrix [A] is the following matrix product when [T]-1 of the matrix [T] exists. [A]=[T][Λ][T]-1 41. 17. The parallel distributed processing network according to claim 16, wherein the information storage matrix [A] is the following matrix product when [T]-1 of the matrix [T] exists. [A]=[T][Λ][T]-1 42. 16. The parallel distributed processing network according to claim 15, wherein the information storage matrix [A] is the following matrix product when [T]-1 of the matrix [T] exists. [A]=[T][Λ][T]-1 43. 21. The parallel distributed processing network according to claim 20, wherein the information storage matrix [A] is the following matrix product when [T]-1 of the matrix [T] exists. [A]=[T][Λ][T]-1 44. 20. The parallel distributed processing network according to claim 19, wherein the information storage matrix [A] is the following matrix product when [T]-1 of the matrix [T] exists. [A]=[T][Λ][T]-1 45. 24. The parallel distributed processing network according to claim 23, wherein the information storage matrix [A] is the following matrix product when [T]-1 of the matrix [T] exists. [A]=[T][Λ][T]-1 46. 22. The parallel distributed processing network according to claim 21, wherein the information storage matrix [A] is the following matrix product when [T]-1 of the matrix [T] exists. [A]=[T][Λ][T]-1 47. 28. The parallel distributed processing network according to claim 27, wherein the information storage matrix [A] is the following matrix product when [T]-1 of the matrix [T] exists. [A]=[T][Λ][T]-1 48. 26. The parallel distributed processing network according to claim 25, wherein the information storage matrix [A] is the following matrix product when [T]-1 of the matrix [T] exists. [A]=[T][Λ][T]-1 49. 34. The parallel distributed processing network of claim 33, wherein the network is implemented in hardware elements. 50. A network is a general-purpose digital computer that operates according to a program. 34. The parallel distributed processing network according to claim 33, wherein the parallel distributed processing network is executed using a computer. 51. 35. The parallel distributed processing network of claim 34, wherein the network is implemented in hardware elements. 52. A network is a general-purpose digital computer that operates according to a program. 35. The parallel distributed processing network according to claim 34, wherein the parallel distributed processing network is executed using a computer. 53. A parallel distributed processing system consisting of multiple nodes, each node having an input and an output. A method for constructing a logical network, which consists of the following steps: The column is a predetermined number (M) of [N×1] target vectors and a predetermined number (Q) of [N×1] indefinite vectors. each target vector is the output of the parallel distributed processing network. Expressing one of the forces (M <= N and Q = (NM)) [N × N] Create a similar deformation matrix, and each element on the diagonal is a vector in the similar deformation matrix. corresponds to a given one of The relative value of each element on the diagonal of is the target vector corresponding to the parallel distributed processing network. Create an [N×N] diagonal matrix [Λ] that corresponds to the speed of convergence toward A] [T] = [T] [Λ] Evaluate the information storage matrix to determine the values of the elements of the information storage matrix [A]. The output of each node is connected to a predetermined number of nodes in the network by a predetermined connection weight, and the element Ai,j of the information storage matrix [A] is connected to the j-th input node and the i-th input node. A parallel distributed processing network is constructed by arranging the nodes in a single layer to define the connection weights between the output nodes. 54. 54. The method of claim 53, wherein the step of evaluating is performed by a Gaussian subtraction method. 55. 54. The method of claim 53, wherein the step of evaluating is performed by a delta rule method. 56. 54. The method of claim 53, wherein in the step of constructing, the output of each of the nodes, including itself, is connected to all inputs of the nodes in the network. 57. The building steps are a general-purpose digital computer that operates according to the program. 54. The method of claim 53, which is performed using a computer. 58. The building steps are performed by interconnecting hardware components. 57. The method of claim 56, wherein the method is performed. 59. The building steps are a general-purpose digital computer that operates according to the program. 57. The method of claim 56, which is performed using a computer. 60. The building steps are performed by interconnecting hardware components. 57. The method of claim 56, wherein the method is performed.