JPH0329064A - Pattern recognizing device - Google Patents

Abstract

PURPOSE:To facilitate the weight correcting calculation by inputting the simplified data to a neural network and approximating the simplified data to the teacher data in accordance with an error between the output data of the neural network and the teacher data. CONSTITUTION:In a teaching step, the final target teacher data is computed into the simple data for calculation of the simplified data. This simplified data is inputted to a neural network 2 to obtain the output data. An arithmetic control part 4 functions to approximate the simplified data to the final target teacher data in accordance with an error between the obtained output data and the final target teacher data. A memory 3 stores the teacher data as well as the weight given to each coupling part 6 which connects the neurons 5 when the coincidence is obtained between the final target teacher data and the simplified data. Then an input port 4c transfers the pattern data between the part 4 and the outside of a pattern recognizing device. In such a constitution, the weight correcting calculation is facilitated and no extension of the network 2 is needed.

Description

[Detailed Description of the Invention] [Industrial Application Field] The present invention compares and refers pattern input data with a plurality of teacher data stored in a device, and specifies the pattern input data as one of the teacher data. Regarding pattern recognition devices. [Prior art] The human brain is approximately 140 r! ! It is well known that the brain is composed of individual neurons (brain cells), and each neuron forms a network through synaptic connections. Furthermore, the brain is thought to perform memory and learning by changing the connection state of the network. The human brain is particularly good at pattern recognition, and its information processing form is based on parallel processing using dark value calculations, and its memory form is said to be associative. The well-known berseptron is a mathematical model of these networks. Furthermore, in recent years, against the backdrop of the realization of massively multiplexed parallel processing due to improvements in hardware performance and the significant increase in memory capacity, computer-based pattern recognition processing using a model of a neural network (hereinafter referred to as a neural network) consisting of a large number of neurons has been developed. Attempts are being made to realize this. Figure 8 shows a biological model of a neuron, and Figure 9 shows its mathematical model. 5. is a neuron model, 6
1 is a model of an axon, and 10L is a model of synaptic connections. As shown in FIG. 9, the mathematical model of the neuron 5 is subjected to a weighting process R at the connection part of the connection part 6 with the neuron 5, which corresponds to the synaptic connection 10 of the biological pseudo model.
Each pattern input data X is input into the neuron 5. The weighted pattern input data is then subjected to a summation process S and then subjected to a dark value process T. In the dark value processing T, an S-shaped function such as a ngmoid function is usually used. Then, the value y subjected to the dark value processing is outputted to the output side connecting section 6 toward the subsequent neuron 5 as the output value of the neuron.

These models of neural insolence work can be broadly divided into automatic association type and Bakun association type based on the difference in the horizontal construction of neural work or the difference in pattern memory method. Although not shown here, the automatic associative network has a common input neuron and an output neuron, all neurons in the network are interconnected, and multiple pattern data are stored in the network. It is a neurotower that outputs pattern data that is most similar to the input pattern, and the stenowork that teaches the pattern data changes the strength of the connections between neurons in order to distinguish between similar input patterns. Is this kind of network work an R-commandation of combinatorial problems or ambiguity? It is said to be excellent at associating things. On the other hand, in the pattern-associative neural network 2■, as shown in FIG. 10, a plurality of neurons 5 are connected to the input layer. It is hierarchically divided into an intermediate layer and an input layer, and neurons 5 between the layers are connected via a connecting part 6 from the input layer to the output layer. Furthermore, the neurons 5 in the same layer are not connected, and the neurons 5 in the input layer and the neurons 5 in the output layer are independent.

In learning in the pattern associative neural network 2a, it is well known that a pack propagation algorithm is applied. Below, we will explain the neural network using this bank propagation algorithm. The backbropage algorithm was published in 1986, and prior to system operation, it first requires a pattern teaching step, and teaching data must be given to neurons 5 in the input layer. When each neuron 5 is given a pattern input data X for academic purposes, this signal is converted by each neuron 5. Subsequently, it is transmitted to each neuron 5 in the intermediate layer, and finally output as pattern output data y from each neuron 5 in the output layer (arrow F).

As described above, the coupling section 6 adds a dwarf to each signal upon input to each neuron 5 of each layer. Then, the output value from the output layer and a preset desired output value are compared, and the difference is reduced.
The strength (weight) of the coupling between the output layer and the hidden layer is changed. This range change is sequentially repeated until the signal propagation direction (arrow F) of the neural network 2 converges in the backward direction (arrow M).

In this way, the neural node I work 2a based on the bank-propagating algorithm repeatedly changes the strength of the connections between the neurons 5 until the difference between the obtained output y and the desired output A approaches O. According to this method, the data to be classified is extracted and made into pattern data, and then... By setting the representative value of the classification result as master data, it is possible to classify the ζ data by checking which representative value any data in the data group outputs a value close to.

Further, as a version of the neural network 21 with different input characteristics, there is a neural network 2b shown in FIG. That is, expanding the input feature using a tensor or trigonometric function, for example, for the input value X1 in the figure, χ. By simultaneously inputting the trigonometric function of the It number as a variable, the learning efficiency is greatly improved. In this way, pattern associative neural networks 2■, 2b using bank propagation algorithms
It is possible to inculcate pattern recognition functions and knowledge processing functions into the network, and it is possible to reliably process pattern input data that cannot be affected by linear separation processing, which is a weak point of the Versebutron. It is useful. [Problems to be solved by the invention] However, neural dysfunction and twerking21 as described above
In the pattern L2 recognition device equipped with the above, if the target pattern according to the teacher data is complex, it may be necessary to repeat the f4'JX a large number of times for the weight correction in the teaching step, or it may be necessary to repeat the weight correction. Even if it is repeated, there is a problem that, for example, if the error potential for the weight is multimodal, it will fall into a local optimum value and will be unable to produce an output close to the teaching data. In other words, depending on the data, it may converge easily or it may not. Further, if the neural network 2I is provided, the network itself becomes complicated, which increases the amount of calculations and requires a large number of hardware components.

Therefore, an object of the present invention is to enable convergence or speed up weight correction calculations without increasing the number of neurons in the neural network even if the target pattern is complex. The objective is to provide a pattern recognition device. [Means for Solving the Problems] In order to achieve the above object, the main means adopted by the present invention is a system having a data input section, a calculation section, and an output section.
or a connecting section for connecting a plurality of information processing sections and an output section and an input section of the information processing sections, and for adding weight to output information from the output section and transmitting the weighted information to the input section. The pattern recognition device includes a memory means for storing preset target teaching data, and input data is input to the pattern recognition device and is operated so that the output data is close to or close to the preset initial target teaching data. a first weight determining means that performs learning while changing the weight until a match is made, and determines the weight of each of the connected parts when they are close or match; and stepwise changing the initial target teacher data at least once or more. By learning while changing the weights until the output data at each stage approaches or matches the target teaching data at each stage, the final result when the output data approaches or matches the final target teaching data This pattern recognition device is characterized in that it includes a second weight determining means for determining weights. [Operation] In the pattern recognition device according to the present invention, when teaching a pattern to the pattern recognition device, first, the final goal teaching data set in advance and stored in the memory means is converted into an initial target teacher that is simplified and easy to calculate. The data is changed and set. Then, the input data is input to the pattern recognition device, the obtained output data is compared with the initial target teaching data, and learning is performed by repeatedly changing the weights until they are close to or coincide with each other, and the output data is The first weight determining means determines the weight of each of the connecting portions when the connecting portion is close to or coincides with the initial target teacher data.

Next, the initial target teacher data is set to approximate the final target teacher data in at least one step or more, and the above-described process is performed so that the initial target teacher data approaches or approaches the target teacher data at each stage. Similarly, when the output data is close to or coincides with the final target teacher data, the final weight of each of the connected parts is determined by the second weight determining means. As a result, weight correction calculation becomes easy and there is no need to increase the number of networks. [Examples] Examples embodying the present invention will be described below with reference to the attached drawings to provide an understanding of the present invention. here the first
The figure shows pattern recognition according to an embodiment of the present invention! 2 is a conceptual diagram showing the neural network included in the pattern recognition device, and FIG. 3 is a schematic structure showing the information processing and connection parts of the neural network. Fig. 4 shows the same two eral net 1.
A graph showing the relationship between the input data to the workpiece and the teacher data corresponding to the input data, and FIG. 5 is the relationship between the error and the weight used to explain the state in which the weight correction calculation by the pattern recognition device converges. Figure 6 is a graph showing the changes in each point on the pattern when the simplified pattern data shown by the broken line in Figure 4 approaches the final target teacher data shown by the solid line. This is a graph showing the number of loops required to converge the calculation for each stem in the figure. The examples described below are not intended to limit the technical scope of the present invention. As shown in FIG. 1, the pattern recognition device 1 according to the present embodiment includes a pattern associative neural network 2 that performs pattern Elm using the Banokubu pagination algorithm, and a neural network 2 that is always connected to the neural network 2. At the same time, in the teaching step, the final target teacher data is calculated into simplified data to calculate simplified data, and the simplified data is inputted to the neural network 2, resulting in output data and the final target. A calculation control unit 4 that brings the simplified data closer to the final target teacher data according to the error value with respect to the teacher data, and a neuron 5 that connects the simplified data and the final target teacher data when the simplified data matches the final target teacher data. It is monitored by a memory 3 that stores the weights assigned to the connection unit 6 and the final target teacher data, and an input port 4c that exchanges pattern data between the arithmetic and control unit 4 and the outside of the device. ．． As shown in FIG. 2, the neural network 2 has an input layer consisting of two neurons 5, an intermediate layer consisting of 12 neurons 5, and an output layer consisting of one neuron 5. They are connected by a connecting part 6. As shown in FIG. 3, the neuron 5 (information processing section) includes an input section 7 that is connected to the connection section 6 and receives weighted pattern input data from the connection section 6; The weighted pattern input data input to the neuron 5 are summed up, the resulting summation value is subjected to threshold processing, and when the summation value exceeds a threshold set in advance for each neuron 5, excitement (output) is generated. It consists of an operation section 8 configured to do the following: and an output section 9 that outputs pattern output data y to the neuron 5 in the downstream layer in the data processing direction (arrow F) when the operation section 8 is excited. There is. It goes without saying that the arithmetic unit 8 changes and determines one weight during the teaching process, and performs weight learning sequentially for each layer on the upstream side in the data processing direction.

In the pattern recognition device I configured as described above, when recognizing the pattern data A shown by the solid line connecting points a-t in FIG.
Simplified pattern data B (initial target teacher data) that approximates (final target teacher data) is set by the arithmetic control section 4. Such simplified pattern data B may be stored in the memory 3 in advance. The simplified pattern data B may be appropriately determined so that convergence in learning to be described later is performed efficiently.In this case, point a in the figure
-i' is set as shown by the broken line.
Here, for convenience, a value of 1 is always input to 5 of the neurons 5 in the input layer of the neural network 2 as a value that does not change. Of course, values according to appropriate rules may be input as necessary, and the pattern input data to the neural network 2 is a value corresponding to each pattern azi, which is 0.1 here. It is indicated by a value of ~0.9. Next, the first learning by the neural network 2
As a step, the pattern input data X is input to the pattern recognition device it1, and output data y is calculated.
At this time, learning (operation X) is performed to modify the weights so that the output data y approaches the preset initial target teacher data B. This learning is repeated for all input data, and the weights given by the connection unit 6 when all the output data and the corresponding initial teacher data are close to each other or coincide with each other are calculated as the first llIJ! Adopt as weight. That is, the first weight determining means realizes m functions until the initial weights are determined by the arithmetic control section 4 and the arithmetic control section 8. For example, pattern input data X to neuron 5 of the input layer
as 0. When l is input, the pattern input data x (0.1) is calculated by a predetermined calculation formula in the calculation unit 8, and the calculated value is sent to each neuron 5 of the intermediate layer via the connection unit 6. It is transmitted to At this time, the connecting part 6
In , a certain weight is added to the expression I{a. Then, the weighted calculation value is inputted to the calculation unit 8 via the input unit 7 of the intermediate layer New A 5, and is calculated by the summation processing and the threshold processing. The operation X value calculated in each of the intermediate layers;
The weighting, summation, and
Threshold processing is performed, and the calculated value is output from the output section 9 of the 221-ron 5 in the output layer as output data y. Then, the output data y and the pattern input data X (0
．． 1) is compared with the initial target training data B (0.8), and in the case of unmanifold, first the weight in the connection part 6 between the output layer and the hidden layer is changed, and then the weight in the connection part 6 between the output layer and the input layer is changed. The weights in connections 6 between layers are changed. Then, the process based on the above procedure is performed again, and the weight of each connecting part 6 is changed so that the output data y corresponding to the pattern input data x (0.l) approaches the initial target teaching data B (0.8). ．． Subsequently, the same processing as that performed on the pattern input data x (0.1.) is performed on the next pattern input data x (0.2). At this time, the weight of each connection part 6 obtained in the pattern input data x (0.1) is initially used. Similarly, pattern input data X is 0 below.
．． 3, 0.4, . . , 0.8, 0.9, the above process is also executed. If such pattern input data X is O. The iterative processing performed from l to 0.9 finally yields for all pattern input data X,
The process is executed until the corresponding output data y approaches or matches the initial target teacher data B. Such learning ends when all the output data y are close to or coincide with the corresponding initial target teacher data B, and step 1 shown in the table of FIG. 6 is completed, and the weights at that time are Determined by 8. Also, the number of loops N required in this case is as shown by 31 in FIG.
Error F. <0. Using 01 as the convergence judgment condition, it took just under 50,000 times. The above is the process of the first step, and the weight of each connection section 6 indicating the strength of the connection between the neurons 5 of the neural network 2 at that time is determined by the calculation section 8 and stored in the memory 3. Next, a second stage of initial target teacher data is set in which the initial target teacher data B is brought closer to the final target teacher data A by an arbitrary amount. Here, point a and point a', point C and point C'. Since the values of point i and point i' are different, point a, point c' i/
The initial teacher data B at point is converted to the final target teacher data A (point a 0
．． 2. 0.2 at point c and 0.8 at point i, respectively. The current initial target teacher data is set using the value that is brought closer by 1 and the remaining value of the previous initial target teacher data B. Note that if the arbitrary amount is too large, the weight correction calculation may not converge, and if it is too small, the calculation amount for the neural network 2 as a whole will be enormous, so the initial target teacher data B and the final It may be determined appropriately depending on the difference from the target teaching data A. In addition, here, only the initial target teacher data B corresponding to the input data x (0.1.0.3 0.9) is changed, but this is not limited to changing the initial target corresponding to other input data X. The teacher data B may be adjusted by an appropriate amount, and the initial target teacher data B may be adjusted so as to approach the final target teacher data A as a whole. Once this initial target teacher data is set, the same processing as in the previous step 1 is performed for each input data x (0.1 to 0.9), and the output data y is close to the current initial target teacher data. Otherwise, the weights are learned by changing each weight until they match (Step 2 in Figure 6). Note that the number of loops N this time was approximately 35,000 times (Figure 7, 32). Thereafter, the same processes as Step 1 and Step 2 described above are executed step by step. In this example. Processing up to stencil 7 has been completed (steps 3 to 7 in FIG. 6 and 53 to S7 in FIG. 7). Here, in Step 7, the above-described learning is performed on the final target teacher data A, and all pattern input data x (0.
1-0. 9) Each output data y corresponding to
If it is close to or coincides with the final r1+=teacher data A, all teaching steps are completed. Therefore, the second weight determining means realizes the functions achieved in steps 2 to 7 of teaching. The weight of each connection part 6 obtained through the above-described series of learning is stored in the memory 3 and used as the final "weight" L7 for recognizing the pattern data A. 12It
In the neural network 2 in which the weights are set in the connection unit 6, all or any number of pattern input data X are input to the neurons 5 of the input layer, and the obtained output data y is the final target teacher data A. If it matches, pattern A is recognized. Furthermore, by setting the weights and changing the strength of the connections in the neural network 2 for various and large numbers of pattern data, the output data for a certain pattern input data and the teacher data can be compared, and which pattern You can lm whether it is. As described above, the pattern recognition device I according to this embodiment can converge the pattern data step by step as shown by the curve H shown in FIG. 7 even if the pattern data is complex and difficult to converge. possible, and the number of loops at that time N
You don't need much either. Incidentally, in the conventional pattern recognition device using only the backpropagation algorithm, as shown by the curve G, the convergence criterion condition E<0.01 was not reached with a large error E and the convergence was not achieved.

In this invention, the pattern input data X and the initial target teacher data B are used, and the final target teacher data is not used. If complex data, for example multimodal data C, is used from the beginning, as shown in Fig. Even if the calculation is repeated to reach the true optimal point P, when the local optimal point n is reached, it is judged as if it had converged, and the process does not converge with an error less than the local optimal point n. This is because there is. Therefore, data D that approximates data C is set, and the calculation is repeated from the calculation start point q of data D that corresponds to the calculation start point m of data C, to find the point r that is the optimal solution for data D. Then, by returning to point 0 of data C corresponding to the point r and repeating the calculation again, point P, which is the optimal solution for data C, can be found. The above is an abstract explanation of a part of the outline of the present invention. In the above-mentioned embodiment, there is one neuron 5 in the input layer that can substantially input the pattern data Good too.

Furthermore, the neurons 5 in the intermediate layer may be omitted, and conversely, the intermediate layer may have a plurality of layers.

Furthermore, in the neural network 2 of the above embodiment, the connecting part 6 connects the neurons 5 of the upper and lower layers, and may be one in which the neurons 5 of the input layer and the output layer are directly connected, It is also possible to adopt a structure in which the input section 7 and output section 9 of the neuron 5 of No. 1 are self-connected (as shown by the two-dot chain line 6' in Fig. 3). In this embodiment, the condition of error E<0.01 was used as the convergence judgment condition for the weight correction calculation, but it should be noted that the condition is not limited to this and any convergence judgment condition can be applied. [Effects of the Invention] According to the present invention, one or more information processing sections each having a data input section, a calculation section, and an output section, and an output section and an input section of the information processing section are connected, and a A pattern recognition device comprising a connection section for adding weight to output information of and transmitting it to the input section, comprising a memory means for storing preset target teacher data, and a memory means for storing the input data to the pattern recognition section. By inputting the data into the device and performing calculations, learning is performed while changing the weight until the output data approaches or matches the preset initial target teaching data, and determines the weight of each of the connected parts when the output data approaches or matches the initial target teaching data set in advance. a first weight determining means for changing the initial target teacher data stepwise at least once or more to bring it closer to the final target teacher data, and changing the initial target teacher data at least once until the output data at each stage is close to or coincides with the target teacher data at each stage. This is a pattern recognition device characterized by comprising a second weight determining means that determines a final weight when the output data is close to or coincides with the final target teaching data by learning while changing the weight. , it is possible to reliably and quickly converge weight correction calculations without increasing the neural network more than necessary.

[Brief explanation of drawings]

FIG. 1 is a block configuration diagram showing a pattern recognition device according to an embodiment of the present invention, FIG. 2 is a conceptual diagram showing a neural network included in the pattern recognition device, and FIG. 3 is a block diagram showing the neural network installed in the pattern recognition device. Figure 4 is a graph showing the relationship between the input data to the neural network and the teacher data corresponding to the input data, and Figure 5 is the weight correction by the pattern recognition device. A graph showing the relationship between error and weight used to explain the state in which calculation converges. Figure 6 shows when the simplified pattern data shown by the broken line in Figure 4 approaches the final target teacher data shown by the solid line. Figure 7 is a graph showing the number of loops required for convergence of calculation for each stem in Figure 6, Figure 8 shows the connection state between neurons and connections. Conceptual diagram, Figure 9 is an explanatory diagram showing the mathematical model of the neuron and the connection part, Figure 10 is a state diagram showing an outline of the calculation processing of the neural network based on the backblowing algorithm, and Figure 1l is an explanatory diagram showing the mathematical model of the neuron and the connection part. This is a network configuration diagram showing an example of a network. [Explanation of symbols] l...Pattern recognition devices 22a, 2b...Neural network 3...Memory 4...Arithmetic system I1 section (first and second weight setting means) 5
．． 5,... Nieron (Information Processing Department) 6.6'...
Connecting unit 7...Input unit 8...Calculating unit (first and second weight determining means) 9...
Output section

Claims (1)

[Claims] (1) 1 having a data input section, a calculation section, and an output section
or a connecting section for connecting a plurality of information processing sections and an output section and an input section of the information processing sections, and for adding weight to output information from the output section and transmitting the weighted information to the input section. A pattern recognition device includes a memory means for storing preset target teaching data, and input data is inputted to the pattern recognition device and calculated so that the output data is close to or coincides with the preset initial target teaching data. a first weight determining means that learns while changing the weight until the first weight is determined, and determines the weight of each of the connected parts when they are close or coincident; The final weight when the output data approaches or matches the final target teaching data by learning while changing the weights until the output data at each stage approaches or matches the target teaching data at each stage. A pattern recognition device comprising second weight determining means for determining.