JPH03240827A - Learning method for neural network - Google Patents

Abstract

PURPOSE:To prevent the excessive execution of the learning for a learning pattern by setting a weight changing constant for connection of a neural network which is used for the learning applying an error or stopping the learning of the learning pattern. CONSTITUTION:A neural network identifies the character images consisting of numeric characters 0 - 9. The comparison value is obtained between the output data obtained when a learning pattern is inputted to the neural network and the teacher data showing the category which includes the learning pattern. Based on the comparison value, a weight changing constant is set for connection of the neural network used for learning the learning pattern or the learning of the learning pattern is stopped.

Description

DETAILED DESCRIPTION OF THE INVENTION <Industrial Application Field> The present invention relates to a neural network learning method by learning using errors.

<Prior Art> Recently, multilayer persebutron-type neural networks that learn using errors, such as error backpropagation learning, have come to be used in the fields of speech recognition and character recognition.

Conventionally, a neural network commonly used in these fields is a three-layer persebutron type neural network. This three-layer bersebutron type neural network has an input layer 1 as shown in Figure 1.
．． It is composed of three layers: an intermediate layer 2 and an output layer 3.

The input layer l has a plurality of nodes 5.5. , the intermediate layer 2 has a plurality of no F 6 , 6, -, and the output layer 3
has multiple notes 7, . It has 8. And the input layer l
Each note of 5.5. and all nodes of middle tier 2 6.6.
are connected by nonapse connections, and similarly each note 6.6 . - and all notes 7 of output layer 3, . 8 and is connected to each other through nnabus connections to form a network. In the three-layer persebutronic neural network configured as described above, when input data is input to each node 5.5° of the input layer 1, output data corresponding to the structure of this network is sent to each node 7 of the output layer 3.・、8
It outputs from.

Each node that constitutes the intermediate layer 2 and output layer 3 has a part that receives and receives input data from other nodes, a part that converts the input data according to predetermined rules, and a part that outputs the converted results. Consists of parts.

A weight representing the strength of the connection is added to the nonapse connection with other notes. This weight represents the strength of the connection between notes, and changing the value of this weight changes the structure of the network, allowing it to output different output data for the same human data. The above-mentioned error backpropagation learning means that when learning data representing a learning pattern belonging to a certain category is input to the input layer I, the data output from the output layer 3 becomes output data representing the category to which this learning pattern belongs. The key is to set the weight of the connection between each note so that Therefore, when a neural network trained by error backpropagation learning is inputted with data representing a feature pattern of an audio signal or character image in the input layer l, it identifies the category to which the feature pattern belongs and displays the identification result. It outputs data.

The above error backpropagation learning is performed as follows. Each node of input layer l 5.5. - Input the learning data representing the learning pattern. Furthermore, each node 7 of the output layer 3, -.
In step 8, teacher data representing the Rikitogori to which the above learning pattern belongs is presented. Then, each node of the intermediate layer 2 and the output layer 3 has the actual output data that corresponds to the training data and the desired output data (such as
The weights of the nonapse connections are changed to minimize the error with the value of the training data). Such a learning process is repeatedly performed on sets of learning data and teacher data in various learning patterns.

FIG. 2 shows the output values of the nodes and the weights of the Nnabus connections during learning by error backpropagation learning of a multilayer persebutron type neural network.

In FIG. 2, the connection weight W added to the node in the intermediate layer changes depending on the input value X from the lower note based on the learning data, the learning output value δ based on the output data O and the teacher data T. At that time, the changes in the weights of nonapse connections for each learning process are as follows.

α・△−N, N−1(n)・(+) l Here, △−, N−1(n.1) 3 The first note of stage N in the (n−1)th study and ( Ni) Amount of change in connection weight between the j-th node of the stage and the node of the i-th node of the N-th stage and the j-th node of the (N-1) stage in the n-th learning Amount of change in connection weight between δ8.Learning output value of the 1st node in the Nth stage , a stabilizing constant (e.g. α-0,9) In other words, the amount of change in weight in the (n+1)th learning w(n・1) is the weight change amount from the lower note in the (n-1)th learning. The learning constant η times the value of the bond value between the input value L, which is added to the value multiplied by α, is also ).Thus, the learning output value is based on the error value between the output data O output from the output layer 3 in the n-th learning and the corresponding teacher data T. When the value of δ becomes smaller, the amount of change Δ1 in the weight value W during the (n-1)th learning becomes smaller, and the learning tends to converge.

The error backpropagation learning described above is completed as follows. That is, each node 7 in the output layer 3,...
The sum of squares E of the errors between the output data 0 from 8 and the teacher data T input to the corresponding node is calculated.

Here, OQ: the output value of the 0th node in the output layer 3 TQ: the element value of the teacher data for the 4th node in the output layer 3, and the sum of the values of the sum of squares E of errors for all learning patterns. When the value (usually using the sum of squared errors) becomes smaller than a predetermined threshold, it is determined that the error backpropagation learning has converged, and the error backpropagation learning is terminated.

<Problem to be solved by the invention> As mentioned above, in a three-layer perceptron neural network, the total value of the sum of squares E of errors related to all learning patterns is less than or equal to a predetermined threshold. to,
Since learning is continued using a constant value of the constant value η for all learning patterns, the following problem arises.

In other words, despite the fact that the error value between the output data of the output layer 3 and the teacher data, which are used as a measure of learning progress judgment, is small, it becomes a target for identification due to the learned neural and virtual work. If the new input cover pattern cannot be correctly identified, please try again. This problem occurs because learning for all learning patterns is continued when the sum of the squared sums E of errors for all learning patterns does not become equal to or less than a predetermined threshold. In other words, if the learning has already converged,
) There may be cases where the learning pattern is learned more than necessary. In such a case, the value of the learning constant η may remain set at a predetermined value. Second, even though the learning for identifying differences between categories that should have been learned has already been completed, Learning progresses at the rate of progress. As a result, the specific parts that are commonly seen in the learning pattern are learned more than necessary.1. At the time of identification, input patterns that should originally be identified in the same category end up being identified as being in different categories. As a result, even if the error between the output data and the teacher data for the learning pattern becomes small, the discrimination ability of the input pattern does not improve.

In addition, as mentioned above, all learning patterns are uniformly learned until the sum of the squared sums E of errors related to all learning patterns becomes equal to or less than a predetermined threshold, so it is possible to fall into a locally optimal solution during learning. There is also the problem that once a local optimal solution is reached, it is often impossible to get out of that state.

Furthermore, since learning is performed on all learning patterns until the total value of the sum of squares E of errors regarding all learning patterns becomes equal to or less than a predetermined threshold, there is also the problem that learning takes time.

Therefore, the purpose of the present invention is to prevent a specific learning pattern from being over-trained, to have a high ability to identify input patterns, to easily escape from a locally optimal solution, and to reduce the number of learning times until convergence. An object of the present invention is to provide a learning method for a neural network that can complete learning in a short time with fewer errors.

Means for Solving the Problems> In order to achieve the above object, a first invention is a method for learning a neural network by learning using errors, which provides a method for learning a neural network by learning a learning pattern into the neural network. A comparison value between the output data and training data representing the category to which the learning pattern belongs is obtained, and based on the comparison value, a change constant of the connection weight of the neural network used in learning regarding the learning pattern is set. Alternatively, the learning according to the learning pattern described above is stopped.

The second invention is a learning method for a neural network by learning using errors, in which a learning pattern is input to the neural network, and each element value of the output data is 12t, among the element values of the output data, calculate the ratio between the element value representing the category to which the learning button belongs and the maximum value of the other element values, and based on the value of the ratio, perform learning related to the learning pattern. The present invention is characterized by setting a change constant of the connection weight of the neural network used in the above-mentioned case, or by stopping learning related to the above-mentioned learning pattern.

Further, a third invention is a learning method for a neural network by learning using errors, which includes output data when a learning pattern is input to the neural network, and training data representing a category to which the learning pattern belongs. , calculate the average value for each same category in the above comparison values, and calculate the above neural neural It is characterized by setting a change constant for the weight of network connections or stopping learning related to the learning pattern.

<Example> Hereinafter, the present invention will be explained in detail with reference to illustrated embodiments.

FIG. 1 is a schematic diagram of a three-layer perceptron type neural network according to the present invention, and the structure of this neural network is as described above. The above neural network is a neural network that identifies character images consisting of the numbers "0° to "9", and each node 5.5 of the input layer l has a character image or a character image extracted by a known method. The element values of the image feature pattern are input.

Therefore, the number of nodes in the input layer l may be set according to the number of dimensions of the pattern of the input character image or image feature amount. Ten nodes 7, . . . , 8 of the output layer 3 are provided and assigned to numbers "0" to "9", respectively. The number of nodes in the middle layer 29 is arbitrarily set statically or dynamically.

Below, we will discuss the three-layer perceptron-type neural system shown in Figure 1.
The learning method of the present invention relating to networks will be described in detail.

The second invention is to adjust the value of the learning constant η used when calculating the amount of change △★ of the connection weight W using equation (1) at each intermediate @2 and output node 13 according to the progress of learning. By setting the information appropriately, learning to improve the discrimination ability can be performed efficiently.

[Example 1] The learning method of the neural network in Example 1 is a method of controlling the progress of learning according to the error value between the output data from the output layer 3 and the teacher data.

Embodiment 1a In this embodiment, when learning for a certain learning pattern converges, learning for that learning pattern is terminated.

Each node 5.5 of the input 11 of the neural network shown in FIG. −, the category to which it belongs (“0°～“
9"), a known character image characteristic pattern, or a learning pattern. Then, each node of the intermediate layer 2 and output layer 3 outputs a value X calculated according to equation (2).

Here, i: Output value of the first note in the N stage (corresponds to the output value OQ in the case of a node in the output layer) fl・Output function at the i-th node (for example, a ngmoid function) u: the i-th node in the N stage As a result, each node 7, . . . , 8 of the output layer 3 outputs an output value of 0. , (1≦Q≦L) is forced.

On the other hand, each node 7,..., 8 of the output layer 3 has a "
Error backpropagation learning is performed by inputting teacher data T that gives "0°" to other nodes as well as "0°" to other nodes.

In this embodiment, at this time, the sum of squares of errors E, which is a comparison value between the output data 0 and the corresponding teacher data T, is calculated using equation (3).

If the value of the sum of squares E of this error is smaller than the threshold value n, then the learning for that learning pattern 7 is terminated.

There are various ways to end the learning, but for example, the value of the learning constant η may be set to "0". In this way, by setting the value of the learning constant η to "Oo", the right-hand side of equation (1) becomes only the term σ・△w(n), and due to the action of the stabilizing constant α(<1), △v(n+1) becomes “
0". At that time, learning based on the term α・△w(n) continues for a short period of time. However,
Since the term α·Δw(n) is a continuation of the previous learning, even if learning based on this term is continued for a short period of time, the direction of learning will not deviate significantly.

By doing so, it is possible to promptly terminate learning for learning patterns for which learning has converged quickly, and to prevent excessive learning.

FIG. 3 is a flowchart of one learning procedure for one learning sample in neural network learning in this embodiment. The learning procedure in this embodiment will be described below with reference to FIG.

In step S1, the learning data of the Xth learning pattern is the input layer! are input to three nodes 5.5, . . .

In step S2, each node 6.6 . . . . and the output value at each node 7° . . . , 8 of the output layer 3 is calculated by equation (2).

In step S3, the value of the sum of squares E of the errors between the output data O from each node 7, . . . , 8 of the output layer 3 and the corresponding teacher data T is calculated using equation (3).

In step S4, it is determined whether the value of the sum of squares E of the errors calculated in step S3 is smaller than a threshold value . As a result, if it is smaller than the threshold value n, the process proceeds to step S6; otherwise, the process proceeds to step S5.

In step S5, the teacher data T is input to each node 7, . . . , 8 of the output layer 3, and learning is executed.

Then, using the output data 0 from each node 7, .
The amount of change in the weight percentage of synaptic connections Δ★ is calculated by equation (1). Then, the value of the connection weight W is updated according to the calculated amount of change ΔW.

In step S6, one round of learning regarding the Xth learning data is completed.

In this way, in this embodiment, learning for all learning patterns is terminated at once when the sum of the squared sums E of errors for all learning patterns becomes equal to or less than a predetermined threshold. Rather, for each learning pattern, when the value of the sum of squared errors E becomes smaller than the threshold value n, learning for that learning pattern is terminated.

In other words, for each learning pattern, when the learning converges, the study M for that learning Batano is completed, so that the learning related to the identification of the category that should have been learned as usual; Learning is not continued. Therefore, uniform learning is performed for all learning patterns, and as a result, the ability to identify input patterns can be improved.

At the same time, the subject matter related to the learning pattern for which learning has converged is completed, and learning is continued only for the learning pattern for which learning has not been completed, so that learning can be directed in an appropriate direction. Therefore, if learning falls into a locally optimal solution, it is easy to escape, and the number of times of learning can be reduced.

Furthermore, in addition to the small number of learning patterns as described above, the number of learning patterns decreases as learning progresses, so learning can be completed in a short time.

The value of the threshold value n in the above embodiment may be a constant value such as "O25", but it may also be set to change dynamically depending on the progress of learning, etc.

Practical Example 1b In this embodiment, the value of the learning constant η used in calculating the amount of change in the value of the connection weight W during learning is set as described below.

As in the case of Embodiment 1a, the learning curve is input to each note 5.5 of the input layer 1 of the neural network shown in FIG. Then, each no of middle layer 2)
6, 6, and output layer 3 valley no F' 7,
-, 8 calculate the output value using equation (2).As a result, each note 7, . 8 outputs the output value OQ.

In this embodiment, at that time, the sum of squares of errors E, which is a comparison value between the output data O and the corresponding teacher data T, is calculated using equation (3). Then, when the teaching data T is input to each node 7, . That's what I do.

η=n-E ・ (4) By doing this, if the value of the sum of squares E of the error between the output data O of the neural network and the teacher data T is small, the value of the learning constant η is reduced to avoid excess This can prevent negative learning.

FIG. 4 is a flowchart of one learning procedure for one learning pattern in neural network learning in this embodiment. The learning procedure in this embodiment will be explained below with reference to FIG.

In step Sll, the learning data related to the X-th learning pattern is used for each note 5, 5 . is input.

In step S12, each node 6.6. - and the output value at each node 7, . . . , 8 of the output layer 3 is calculated by equation (2).

In step S13, the value of the sum of squares E of the errors between the output data 0 from each node 7, . . . , 8 of the output layer 3 and the corresponding teacher data T is calculated using equation (3).

In step S14, the value of the learning constant η is set to "n-E".

In step 515, the teacher data T is input to each node 7, . . . , 8 of the output layer 3. Then, the amount of change ΔW in the weight W of the nonapse connection is calculated using equation (1). Then, the value of the connection weight W is updated in accordance with this calculated sea f two-change dragon Δ★.

In this way, when the value of the connection weight is updated, one learning process for one learning pattern is completed.

As described above, in this embodiment, the learning constant η used to calculate the weight change Δ★ of the connections applied during learning is set based on the sum of squares E of the errors. Therefore, when the value of the sum of squares of errors E becomes smaller (that is, when learning moves towards convergence), the amount of change in weight - △W can be reduced accordingly to prevent learning from progressing excessively. be. As a result, uniform learning is performed for all learning patterns,
The ability to identify input patterns can be improved.

In addition, the value of the learning constant η is set according to the progress of learning,
Since learning patterns that have not converged are emphasized during learning, learning can be directed in an appropriate direction. Therefore, if learning falls into a locally optimal solution, it is possible to escape easily, the number of times of learning can be reduced, and learning can be completed in a short time.

The value of the threshold value n in the above embodiment may be a constant value of, for example, 0, but may be set to dynamically change depending on the progress of learning, etc.

Two Embodiments The learning method of the neural network in the second embodiment is a method of controlling the progress of learning according to the output values from each of the notes 7, . . . , 8 of the output layer 3.

In this embodiment, the value of the learning constant η used when calculating the amount of change ΔW in the value of the connection weight W during learning is set as described below.

Each node of the input layer l of the neural network shown in FIG. 1 5.5. A learning pattern of a learning sample belonging to a certain category is input. Then, each note 6.6 of middle layer 2. ...and each node 7 of the output layer 3,
..., 8 calculates the output value using equation (2). As a result, from each node 7°..., 8 of the output layer 3, the output value O
Q force is applied.

In this embodiment, at that time, the output value Or of the node assigned to the category to which the learning pattern belongs in the output layer 3 and the maximum value O of the output value of the nearest node assigned to the other categories are determined. The value of ratio I] is set to (5
) is calculated using the formula.

H=Or70@-(5) In this case, as mentioned above, when learning the neural network, the learning pattern is assigned to the category to which it belongs (for example, "9") and the f2 output layer 3 node (for example, node 8 ) learning is performed so that the output value from ``Bi'' becomes ``Bi'' and the output value from other nodes becomes ``0''. Therefore, if the value of the ratio The output value from the assigned node is the maximum, which means that learning has been performed correctly.

In other words, the value of ratio H represents the degree of progress of learning.

Therefore, in this embodiment, if the value of the ratio H is greater than the threshold value 1, learning is terminated. On the other hand, the value of the ratio H is the threshold m
In the following cases, the value of the learning constant η is set as follows depending on the degree of progress of learning based on the value of the ratio H. That is, for example, when m=1.2, the learning constant η! 2kuH → 0 1.0<H≦1.2 - 0.08 08〈H≦1.0 → 0''OH≦0.8 - 0.12 By doing this, the output of the neural network As the output value Or from the node assigned to the category to which the learning pattern in layer 3 belongs becomes larger than the maximum value Ov of the output values from other nodes, the learning constant η
The value of can be reduced, and overfitting can be prevented.

FIG. 5 is a flowchart of one learning procedure for one learning pattern in neural network learning in this embodiment. The learning procedure in this embodiment will be explained below with reference to FIG.

In step S21''iq, learning data related to the X-th learning pattern is input to each node 5..5 of the input layer l.

In step S22, based on the learning data input to the input layer 1, each node 6.6. ...and the output value at each node 7, ..., 8 of output 13 (2)
Calculated by the formula.

Step 2 523 output @3. ) each node 7,...
Based on the output values of 8, the value of the ratio H is calculated by equation (5).

In step S24, the value of the learning constant η is set as described above according to the value of the ratio H calculated in step S23.

In step S25, each node 7, ..., 8 of the output layer 3
The teacher data T is input to the . Then, the amount of change ΔW in the weight W of the NNAPS connection is calculated by (1) 2. Then, the value of the connection weight W is updated according to the calculated amount of change Δ.

In this way, when the value of the connection weight W is updated, one learning session for one learning sample is completed.

In this manner, in this embodiment, the ratio H between the output value from the node to which the category to which the learning pattern belongs in the output layer 3 is assigned and the maximum value of the output values from other nodes is determined. Then, the value of the learning constant η is set according to the value of this ratio H. Therefore, when the value of the ratio H increases (that is, when learning tends to converge), the amount of change △W of the weight ☆ is decreased to terminate the learning accordingly, and the learning progresses in an overvariant manner. can be prevented.

As a result, the entire university! Uniform learning is performed on patterns, and the ability to identify input patterns can be improved.

In addition, learning for learning patterns that have converged is terminated, and for learning patterns for which learning has not yet been completed, the value of the learning constant η is set according to the progress of the subject to emphasize learning patterns for which learning has not converged. Since the students learn based on their own knowledge, they can direct their learning in an appropriate direction. Therefore, if learning falls into a local optimum solution, it is possible to easily escape from the problem and the number of times of learning can be reduced.

Furthermore, in addition to the small number of times of learning as described above, the number of learning patterns decreases as learning progresses, so learning can be completed in a short time.

The value of the threshold m used in the embodiment L may be a constant value, such as I2'', for example, but it may also be set to dynamically change depending on the progress of learning, etc.

In the above embodiment, the value of the learning constant η is changed stepwise in accordance with the value of the ratio H, but the invention is not limited to this. In other words, whether learning is terminated or continued depending on the threshold m (i.e., η-=0 or η
-ηI). Further, the learning constant η may be changed continuously.

[Example 3] The neural network learning method in Example 3 is based on the average value of the sum of squared errors between the output data from the output 13 and the teacher data for all learning patterns belonging to one category. It is a method of controlling the progress of learning.

Also in this embodiment, the value of the learning constant η used when calculating the amount of change ΔW in the connection weight 1 is set as described below.

The input layer of the neural network shown in Figure 1! Each node of 5.5. A learning pattern of a learning sample belonging to a certain category is input. Then, each note 6.6 of middle layer 2.・and each node 7 of the output layer 3, .
8 calculates the output value using equation (2). As a result, each node 7.8 of the output layer 3 outputs an output value OQ.

In this embodiment, at that time, the sum of squares of errors E, which is a comparison value between the output data O and the corresponding teacher data T, is calculated using equation (3). Similarly, the value of the sum of squares E of the errors between the output data O and the value T of the teacher data for all learning patterns belonging to the same category is calculated. Then, the average value E of the sum of squared errors E for all learning patterns belonging to the same category. Calculate. and,
Learning constant η used for each note in hidden layer 2 and output layer 3
The value of is set using equation (6).

η:n-E0 (6) By doing this, the average value E of the sum of squares E of errors. When learning a learning pattern that belongs to a category with a small value (that is, easy to identify), the value of the learning constant η can be set to a small value to prevent the gist from being carried out excessively.

FIG. 6 is a flowchart of one learning procedure for one learning pattern in neural network learning in this embodiment. The learning procedure for this embodiment will be described below with reference to FIG.

At step 531, a flag X representing the number of learning samples belonging to the same category is set to "0".

In step S32, the learning data related to the X-th learning pattern belonging to the same category is
．．・Entered in

In step S33, based on the learning data input to the input layer 1, each node 6.6. ... and the output value at each node 7, ..., 8 of the output layer 3 is calculated by (2).

In step S34, the value of the sum of squares E of the errors between the output data O from each node 7, . . . , 8 of the output layer 3 and the corresponding teacher data T is calculated using equation (3).

In step S35, it is determined whether the content of flag X is greater than or equal to the total number of learning patterns belonging to the same category "X". If the result is “X” or more, step S3
If not, the process proceeds to step S36.

In step S36, the contents of flag X are incremented and the process returns to step S32.

In step S37, the average value E0 of the sum of squares E of errors for all learning patterns belonging to the same category is calculated.

In step S3g, the value of the learning constant η is set to "n-Eo".

In step S39, learning data of an arbitrary learning pattern belonging to the same category is input.

In step S40, the teacher data T corresponding to the learning data input in step S39 is input to each node 7, . . . , 8 of the output layer 3. Then, (+
) is used to calculate the amount of change Δ1 in the weight 1 of the nonabs connection. Then, the value of the connection weight 1 is updated according to the calculated amount of change Δ1.

In this way, when the value of the connection weight 1 is updated, the learning is completed once for each learning sample.

In this way, in this example: Well, prior to learning,
Sum of squared errors for the learning pattern E,) L average value E for each category. The learning constant η used during learning is set to η−n−Eo using Eo in the category to which the learning pattern to be learned belongs. Therefore, the connection weight W depends on the ease of identifying the learning pattern.
This allows the amount of change ΔW to be set, and it is possible to prevent learning from progressing excessively. As a result, uniform learning is performed for all learning patterns, and the ability to identify input patterns can be improved.

First, the value of the learning constant η is set according to the degree of ease of identifying the category, and learning patterns belonging to categories that are difficult to identify are emphasized during learning, so that learning can be directed in an appropriate direction. I can do it. Therefore, if learning becomes a locally optimal solution, it is easy to escape, the number of times of learning can be reduced, and learning can be completed in a short time.

The value of the threshold value n in the above embodiment may be set to a constant value of, for example, 0, but it may also be set to dynamically change depending on the progress of learning, etc.

In the embodiment described above, first, the average value E0 of the sum of squares E of errors related to a certain category is calculated f2, and then learning patterns belonging to the same category are performed. However, the present invention is not limited to this, and the average value E of the sum of squares E of errors in all categories is determined in advance. There is no problem in calculating.

Average value E of the sum of squares E of errors in the above embodiment.

This concept also includes the total value of the sum of squares E of errors.

In Example 1a, Example 1b, and Example 3, the comparison value between output data O and teacher data T is calculated by the sum of squares of errors between corresponding elements of both data.

However, the present invention is not limited to this, and calculation may be performed using, for example, the average value of absolute values of errors.

Each of the above embodiments may be implemented alone, or some may be combined. That is, the above Example Ia and Example t
b, and control the value of the learning constant η according to the degree of convergence of learning based on the value of the sum of squares E of the error between the output data O of the neural network and the teacher data T, and -
Learning may be terminated when the value of E reaches a predetermined value or less.

Similarly, in Example 3, the average value E for each category of the sum of squares E of errors for the learning pattern. The learning may be terminated when the learning constant η=n-E or less than a predetermined value is reached.

In each of the above embodiments, a three-layer perceptron neural network is used, but a multilayer perceptron neural network having four or more layers may be used.

Further, the neural network according to the present invention is not limited to a multilayer perceptron type neural network. In short, a neural network of any structure may be used as long as it can perform learning using the amount of learning or error used.

<Effects of the Invention> As is clear from the above, in the neural network learning methods of the above-mentioned inventions, the learning method is based on the comparison value between the output data related to the learning pattern and the training data, or on the element value in the output data. Based on the value of the ratio, or the average value for each same category in the comparison value between the output data and the training data, the connection of the neural network used in learning using the error related to the learning pattern. Since the setting of the weight change constant or 1 is configured to stop the learning related to the learning pattern, it is possible to prevent the learning related to the learning pattern from being carried out unnecessarily. Therefore, the learning of the neural network is performed equally for all learning patterns, and as a result, the pattern discrimination ability of the neural network is improved.

First, learning can be carried out only for unconverged learning patterns or by emphasizing unconverged learning patterns, so even if you fall into a local optimal solution during learning, you can easily escape.
Reduce the number of times you study.

Roughly speaking, in addition to reducing the number of times you learn as mentioned above, you can also reduce the number of learning patterns as you advance in learning, so you can finish learning in a short time.

[Brief explanation of drawings]

Figure m1 is the neural network according to this invention.)
-'μ A schematic configuration diagram showing the example; Figure 2 is an explanatory diagram of the output values and nonapse connection weights in a multilayer pacetronic neural network; Figures 3 to 6 are based on one learning sample in each example. It is a flowchart showing a one-time learning procedure. 1. Human power layer, 2. Middle layer, 3. Output layer, 5, 6, 7.8 ・Note.

Claims (3)

[Claims] (1) A neural network learning method by learning using errors, which calculates a comparison value between output data when a learning pattern is input to the neural network and training data representing the category to which the learning pattern belongs. , a neural network characterized in that, based on the comparison value, a change constant of the connection weight of the neural network used during learning related to the learning pattern is set, or learning related to the learning pattern is stopped.・Network learning method. (2) A neural network learning method by learning using errors, which calculates each element value of the output data when a learning pattern is input to the neural network, and calculates the Calculate the ratio between the element value representing the category to which the learning pattern belongs and the maximum value of other element values, and based on the value of the ratio, determine the connection weight of the neural network used during learning related to the learning pattern. A neural network learning method characterized by setting a change constant of or stopping learning related to the learning pattern. (3) A neural network learning method by learning using errors, which calculates a comparison value between output data when a learning pattern is input to the neural network and training data representing the category to which the learning pattern belongs. , calculate the average value of the comparison values for each same category, and calculate the connection weight of the neural network used in learning for learning patterns belonging to the same category based on the average value of the comparison values for each same category. A neural network learning method characterized by setting a change constant of or stopping learning related to the learning pattern.