JP5002821B2 - Neural network device and method thereof - Google Patents

Description

  The present invention relates to a neural network device that realizes an information processing mechanism created by mimicking the mechanism of a brain, and more particularly to efficiently obtain a global solution rather than a local solution of a weighting factor between an input layer and a hidden layer. The present invention relates to a neural network device capable of

  A neural network is an information processing mechanism that mimics the mechanism of the human brain. The human brain has an excellent ability to easily achieve even processing that is difficult to achieve with a computer. Currently, the computer we use is called the Neumann type, which is very fast and is extremely powerful when solving formulated problems. When solving such problems, it is difficult to formulate the problem and it is very difficult to implement. Therefore, a neural network was born in order to realize processes such as recognition, storage, and judgment, which are human basic functions, on a computer, using the information processing mechanism in the human brain as a hint. Neural networks have a simplified structure of information processing mechanisms in the brain. There are about 14 billion nerve cells in the human brain, and these nerve cells are connected to each other. Each nerve cell has a function of receiving an input signal from another cell and sending an output signal to the other cell when the sum of the input signals exceeds a certain value. This propagation of information between cells makes it possible to perform processes such as recognition, memory, and judgment that are usually performed by humans. The biggest feature of information processing in the brain is that a large number of cells that have only simple functions are gathered to realize complex and advanced processing as a whole, and neural networks take advantage of this advantage. It has become. Specifically, a nerve cell in the brain is modeled as an element called “neuron”, and a network is constructed by arranging and connecting a large number of neurons. When actually applied, various problems can be dealt with by changing (learning) the parameters of each neuron according to the problem to be applied.

  FIG. 10A is a schematic diagram of a neuron. The cell body (soma) is the part that contains the nucleus and can be said to be the main body of the neuron, and the dendrite (dendrite) is the part of many branches that come out of the cell body, the input of the neuron The axon extends from the cell body and corresponds to the output terminal of the neuron, and the synapse is in contact with other neurons with a small thick leg at the end of the branch. It plays a role of transmitting information.

  When modeled, the result is as shown in FIG. 10B, which can be expressed by the following formula.

Combining a large number of them enables complex calculations. FIG. 10C shows a three-layer hierarchical neural network (arbitrary mapping can be realized).
Learning of the neural network is performed as shown in FIG.
FIG. 11A shows an error (error).

新しい重みは、次式となる。

The new weight is:

このようにすると極小値になる。

In this way, the minimum value is obtained.

When changing so as to satisfy these equations, the error E does not increase with respect to the time t.
There is a way to determine the weight of a neural network by exemplifying a lot of learning data (a set of inputs and desired outputs). The learning method based on the steepest descent method is called error back propagation.
FIGS. 11B and 11C are diagrams for explaining the structure of error back propagation.
The total input to the jth unit in the output layer is

であり、誤差(エラー)Ｅが小さくなるようにｗを決める。

And w is determined so that the error (error) E becomes small.

に従うように重みｗを変化させつづければよい。

The weight w may be continuously changed so as to follow.

を求める。
図１１（ｄ）参照。

Ask for.
Refer to FIG.

ここで（３）より

Here from (3)

これを、

this,

で微分すると、

Differentiated by

だけが残る。

Only remains.

よって（１３）は、

Therefore (13)

を求める。
図１１（ｅ）参照。

Ask for.
Refer to FIG.

First,

next,

を伝搬しながら傾きReverse the above calculation from the output side to the input side,

Tilt while propagating

The calculation proceeds. This is called error back propagation learning.
The characteristics of neural networks are as follows.
(1) Learning ability: Necessary functions can be automatically formed based on the input / output samples presented.
(2) Non-linearity: By learning, even complex mapping relationships that are difficult to formulate can be easily constructed.
(3) Parallel processing: The input signal is sent to various neurons through connection and processed in parallel.

  In a hierarchical neural network based on error back-propagation learning, optimization learning based on the steepest descent method is performed, so there is a risk of falling into a local solution rather than guaranteeing a global optimal solution. There is simulated annealing as a learning method to reach the global optimal solution, but it takes time to learn and the input data set and the desired output data set (referred to as the training data set). There is a problem that it is necessary to prepare a large amount and is not realistic.

  The present invention has been made to solve the above problems, and an object of the present invention is to provide a neural network device capable of reaching a global optimum solution by learning in a short time with a small number of training data sets. .

  As a result of the inventor's earnest research on the neural network, it has been found that the input data becomes highly correlated with the weighting coefficient connecting the input layer and the hidden layer as the learning progresses. Although it depends on the number of hidden layer nodes, if the number of input layer and the number of weighting coefficients is the same (if there is one hidden layer node, all the input layer nodes will be The number of weighting factors is the same as the number of nodes in the input layer), and the correlation between the input and the weighting factor when learning proceeds with the desired output,

Becomes higher. Similarly, when the number of hidden layers increases, the correlation coefficient of the equation can be calculated by thinning out or by performing an averaging operation to match the number of input layers, and the progress of learning can be checked. it can. However, the method of obtaining the correlation coefficient is not limited to the above formula, and any formula that can be quantified to the degree of correlation can be used.

  The present invention is based on a hierarchical neural network with error back-propagation learning based on the steepest descent method, and the progress of learning is investigated by examining the correlation coefficient between the input data and the weight coefficient between the input layer and the hidden layer. The situation is examined, and it is determined whether the solution falls into a local solution or is a suboptimal solution close to the global optimum solution. Also, even if error back-propagation learning converges, if the correlation coefficient is low, that is, if it falls into a local solution, use a learning method that changes the initial value and restarts learning. Also included are neural network devices.

The neural network device according to the present invention includes an input layer having nodes, a hidden layer, and an output layer. The input data is input to the input layer and the output data output from the output layer and the input data are prepared in advance. Compared with teacher data, using the error that is the comparison result, update the weighting coefficient between the nodes of the output layer and the hidden layer and the weighting coefficient between the nodes of the hidden layer and the input layer. Hierarchical neural network device based on which the learning has converged when the learning convergence condition that the correlation coefficient between the input data and the weighting coefficient between the nodes of the input layer and the hidden layer is equal to or greater than a predetermined threshold is met A second learning convergence determination unit is provided.

  As described above, according to the present invention, the weight coefficient is updated using the error between the output data generated by the neural network and the output data of the teacher data, and when the error becomes small, the learning is not finished. Since the learning convergence is judged based on the correlation coefficient between the input data and the weighting factors of the input layer and the hidden layer, the input of the learning is not judged as the learning convergence state just by updating the weighting factor. The convergence of learning is judged using the correlation coefficient between the data and the weighting coefficients of the input layer and hidden layer, and the learning is terminated by deriving the global optimal solution without ending the learning with the local solution. There is an effect that can be.

The neural network device according to the present invention includes an input layer having nodes, a hidden layer, and an output layer. The input data is input to the input layer, and the output data output from the output layer and the input data are prepared in advance. Error propagation that learns by comparing the weighted coefficient between the nodes of the output layer and the hidden layer and the weight coefficient between the nodes of the hidden layer and the input layer using the error that is the comparison result A hierarchical neural network device based on learning, wherein a first learning convergence determination unit that determines convergence of learning based on an error of the comparison result of learning, and weights between nodes of input data and input layers and hidden layers and a second learning convergence judging section correlation coefficient between the coefficient is determined to learning when true learning convergence condition that is greater than or equal to a predetermined threshold value has converged, the first learning convergence judgment unit In which the second learning convergence judgment unit both learning terminates learning when you are determined to have converged.
More specifically, the first learning convergence determination unit looks at the fluctuation of the error of the comparison result obtained so far, and determines that the learning has converged when the fluctuation is almost eliminated.

The neural network device according to the present invention includes an input layer having nodes, a hidden layer, and an output layer. The input data is input to the input layer, and the output data output from the output layer and the input data are prepared in advance. Error propagation that learns by comparing the weighted coefficient between the nodes of the output layer and the hidden layer and the weight coefficient between the nodes of the hidden layer and the input layer using the error that is the comparison result A hierarchical neural network device based on learning, wherein a first learning convergence determination unit that determines convergence of learning based on an error of the comparison result of learning, and weights between nodes of input data and input layers and hidden layers and a second learning convergence judging section correlation coefficient between the coefficient is determined to learning when true learning convergence condition that is greater than or equal to a predetermined threshold value has converged, the first learning convergence judgment unit After it is determined that the learning has converged, the second learning convergence judgment unit judges the convergence of the learning, when the second learning convergence judgment unit determines that the learning has converged, is to end the learning.

  As described above, according to the present invention, learning convergence is first determined from the error between the output data generated by the neural network and the output data of the teacher data. The output of the output data generated by the neural network and the output of the teacher data is determined. The error with the data is indispensable for the reflection of the weight coefficient in error back propagation learning, and before judging the learning convergence from the correlation coefficient between the input data and the weight coefficient of the input layer and the hidden layer. It has the effect that it is efficient to carry out the determination of the learning convergence by the error using the already determined error.

Also, the neural network system according to the present invention, an input layer having nodes consists hidden layer及 beauty output layer, and input to the input layer of input data corresponding to the output data and input data output from the output layer Compared with teacher data prepared in advance , learning is performed by updating the weighting coefficient between the nodes of the output layer and the hidden layer and the weighting coefficient between the nodes of the hidden layer and the input layer by using the error as the comparison result. Hierarchical neural network device based on backpropagation learning, learning when the correlation coefficient between the input data and the weighting coefficient between the nodes of the input layer and hidden layer satisfies the learning convergence condition where the increasing tendency reaches saturation There are those comprising a second learning convergence judging section for judging to have converged.
Also, the neural network system according to the present invention, an input layer having nodes consists hidden layer及 beauty output layer, and input to the input layer of input data corresponding to the output data and input data output from the output layer Compared with teacher data prepared in advance , learning is performed by updating the weighting coefficient between the nodes of the output layer and the hidden layer and the weighting coefficient between the nodes of the hidden layer and the input layer by using the error as the comparison result. A hierarchical neural network device based on error back propagation learning, a first learning convergence determination unit for determining convergence of learning based on an error of a comparison result of learning, and nodes of input data, an input layer, and a hidden layer and a second learning convergence judging section for judging the learning has converged when the correlation coefficient between the weighting factors increasing tendency corresponds to learning convergence conditions reach the saturation between the first learning convergence judgment unit When 2 of the learning convergence judgment unit are both learning is to end the training if you are determined to have converged.
Also, neural network system according to the present invention, an input layer having nodes consists hidden layer及 beauty output layer, and input to the input layer of input data corresponding to output data and input data output from the output layer An error that is learned by comparing the teacher data prepared in advance and updating the weighting coefficient between the nodes of the output layer and the hidden layer and the weighting coefficient between the nodes of the hidden layer and the input layer using the error that is the comparison result A hierarchical neural network device based on back propagation learning , wherein a first learning convergence determination unit that determines learning convergence based on an error of a comparison result of the learning, and between input data and nodes of an input layer and a hidden layer comprising of a second learning convergence judging section for judging the learning has converged when the correlation coefficient between the weighting factors increasing tendency corresponds to learning convergence conditions reach the saturation, the first learning convergence judgment unit After learning is determined that converges the second learning convergence judgment unit judges the convergence of the learning, when the second learning convergence judgment unit determines that the learning has converged, is to end the learning.
Also, neural network system according to the present invention, if necessary, the second learning convergence judgment unit, if not applicable to the learning convergence conditions are those re-relearned initialize the weighting factor.

  Thus, according to the present invention, when it is determined that learning has converged from the error between the output data generated by the neural network and the output data of the teacher data, and the input data and the weighting factors of the input layer and the hidden layer, If it is determined that the learning has not converged based on the correlation coefficient, it means that the global optimal solution is not found and falls into the local solution, and the weight coefficient is initialized again and error propagation propagation learning is performed. This has the effect that a global optimum solution can be obtained. Since it is difficult to get out of the local solution and reach the global optimal solution no matter how many times learning is performed in the state of falling into the local solution, it is desirable to initialize the weighting factor.

Also, if the neural network system according to the present invention should, when said weighting factors to initialize, the correlation coefficient of the weighting coefficient before initializing the weighting factors are initialized this phase relationship When the number is equal to or greater than a predetermined threshold, the weighting coefficient is initialized again.
Thus, according to the present invention, even if the weighting factor is initialized, if it is the same as the still weighting factor, there is a high possibility of falling into the same local solution when error back propagation learning is performed again. In such a case, there is an effect that by reinitializing, unnecessary learning can be avoided and a global optimum solution can be efficiently obtained.

Also, the backpropagation learning method of a hierarchical neural network system according to the present invention, the teacher data is input to the input layer of input data are prepared in advance corresponding to the output data and the input data output from the output layer And the error inverse of the hierarchical neural network device that learns by updating the weighting coefficient between the nodes of the output layer and the hidden layer and the weighting coefficient between the nodes of the hidden layer and the input layer using the error that is the comparison result In this propagation learning method, it is determined that learning has converged when a correlation coefficient between input data and a weighting coefficient between nodes in the input layer and the hidden layer is equal to or greater than a predetermined threshold .
Also, the error back propagation learning method of the hierarchical neural network device according to the present invention includes: input data input to an input layer; output data output from an output layer; and teacher data prepared in advance corresponding to the input data. comparison, an error reverse hierarchical neural network system that learns by updating the weighting factor between the nodes of the weighting factors and the hidden layer and input layer between nodes in the output layer and the hidden layer using an error as a result of the comparison Propagation learning method that determines that learning has converged when the correlation coefficient between the input data and the weighting coefficient between nodes in the input layer and hidden layer satisfies the learning convergence condition where the increasing tendency reaches saturation. is there.
The error back propagation learning program according to the present invention includes an input layer having nodes, a hidden layer, and an output layer. The input data is input to the input layer and corresponds to the output data and the input data output from the output layer. comparing the previously prepared teaching data, learning users update the weighting factor between the nodes of the weighting factors and the hidden layer and input layer between nodes of the output layer and a hidden layer with an error as a result of the comparison to a Sagyaku propagation learning program erroneously causing a computer to function as a hierarchical neural network system, the learning when the correlation coefficient between the weighting coefficients between the nodes of the input layer and the hidden layer to the input data is equal to or greater than a predetermined threshold value The computer is caused to function as a second learning convergence determination unit that determines that it has converged .
The error back propagation learning program according to the present invention includes an input layer having nodes, a hidden layer, and an output layer. The input data is input to the input layer and corresponds to the output data and the input data output from the output layer. comparing the previously prepared teaching data, learning users update the weighting factor between the nodes of the weighting factors and the hidden layer and input layer between nodes of the output layer and a hidden layer with an error as a result of the comparison to a Sagyaku propagation learning program erroneously causing a computer to function as a hierarchical neural network system, the learning convergence conditions the correlation coefficient of a weighting factor between the nodes of the input layer and the hidden layer to the input data is increasing tendency reaches saturation In this case, the computer is caused to function as a second learning convergence determination unit that determines that learning has converged .
Thus, the present invention can be grasped as a method or a program .
These outlines of the invention do not enumerate the features essential to the present invention, and a sub-combination of these features can also be an invention.

1 is a block configuration diagram of a neural network device according to a first embodiment of the present invention. 1 is a hardware configuration diagram of a computer in which a neural network device according to a first embodiment of the present invention is constructed. It is an operation | movement flowchart at the time of learning of the neural network apparatus which concerns on the 1st Embodiment of this invention. It is a partial substitution flowchart regarding initialization of the neural network device concerning a 1st embodiment of the present invention. It is an observation image and actual measurement value of 17:11 on May 20, 2002 which concern on an Example. It is an observation image and actual measurement value of 16:45 on November 30, 2004 which concern on an Example. FIG. 7 is a transition of the solution for the input and the desired output of FIG. FIG. 6 is the transition of the solution for the input and the desired output of FIG. FIG. 6 is the transition of the solution for the input and the desired output of FIG. It is explanatory drawing of the neural network of background art. It is explanatory drawing of the neural network of background art.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Input part 20 Neural network mechanism part 30 Output part 40 Error back propagation mechanism 41 Error calculation part 42 Weight coefficient reflection part 43 Error convergence determination part 50 Correlation coefficient mechanism 51 Correlation coefficient calculation part 52 Correlation coefficient condition determination part 53 Initial stage Conversion unit 100 Computer 111 CPU
112 RAM
113 ROM
114 Flash memory 115 HD
116 LAN card 117 mouse 118 keyboard 119 video card 119a display 120 sound card 120a speaker 121 drive

(First embodiment of the present invention)
(1) Block Configuration FIG. 1 is a block configuration diagram of a neural network device according to this embodiment.
The neural network device according to the present embodiment includes an input unit 10 that captures input data to be processed, a neural network mechanism unit 20 that processes the captured input data and generates output data, and an output unit that sends the generated output data 30 and obtaining an error from the output data generated by the neural network mechanism 20 during learning and the output data of the teacher data, determining whether the error has converged from the obtained error, and determining that the error has not converged. In this case, the error back-propagation mechanism 40 that performs error back-propagation and updates the weighting coefficient, and the correlation coefficient between the input data and the hidden coefficient between the input layer and the hidden layer are obtained during learning, and the error back-propagation mechanism 40 converges. If it is determined that the latest correlation coefficient is lower than the predetermined threshold, the correlation coefficient is increasing and reaches saturation. If the latest correlation coefficient is lower than the predetermined threshold, if the correlation coefficient is not increasing, or if the correlation coefficient does not reach saturation even if the correlation coefficient is increasing, the neural network mechanism unit And a correlation coefficient mechanism 50 that initializes 20.

  The error back propagation mechanism 40 includes an error calculation unit 41 that calculates an error between the output data and the output data of the teacher data for each input of input data during learning, and the neural network mechanism unit 20 using the error calculated by the error calculation unit 41. The weight coefficient reflecting section 42 for updating the weight coefficient and the error convergence determining section 43 for determining whether or not the error has been converged from the error variation obtained by the error calculating section 41. The weight coefficient reflecting unit 42 can be configured to reflect the weight coefficient only when the error convergence determining unit 43 determines that the error does not converge, or the error calculating unit 41 obtains an error regardless of the error convergence determining unit 43. In such a case, the weight coefficient reflecting unit 42 may be configured to reflect the weight coefficient. Even in the case of adopting the latter configuration, if the error convergence determination unit 43 determines that the convergence has occurred and if a global optimum solution has been obtained, the processing in the error calculation unit 41 and the weight coefficient reflection unit 42 is performed thereafter. Absent.

  The correlation coefficient mechanism 50 includes a correlation coefficient calculation unit 51 that obtains a correlation coefficient between the input data and the weight coefficient of the hidden layer and the hidden layer for each input of the input data during learning, and the latest correlation coefficient is a predetermined threshold value. A correlation coefficient condition determining unit 52 for determining whether the correlation coefficient is increasing and reaching a saturation level, and the correlation coefficient condition determining unit 52 includes the latest correlation coefficient Is determined to be lower than the predetermined threshold, the correlation coefficient is determined not to increase, or the neural network mechanism unit 20 is initialized when it is determined that the correlation coefficient does not reach saturation even when the correlation coefficient is increasing. And an initialization unit 53 to be converted.

(2) Hardware Configuration FIG. 2 is a hardware configuration diagram of a computer in which the neural network device according to this embodiment is constructed.
The computer 100 in which the neural network is constructed includes a CPU (Central Processing Unit) 111, a RAM (Random Access Memory) 112, a ROM (Read Only Memory) 113, a flash memory (Flash memory) 114, and an HD (external storage device). Hard disk) 115, LAN (Local Area Network) card 116, mouse 117, keyboard 118, video card 119, display 119a which is a display device electrically connected to the video card 119, sound card 120, and sound card 120 It comprises a speaker 120a, which is an electrically connected sound output device, and a drive 121 for reading and writing a storage medium such as a flexible disk, CD-ROM, DVD-ROM or the like. A so-called person skilled in the art can slightly change the components of the hardware, and can construct one neural network for a plurality of computers. One or a plurality of modules, not all, can be constructed for each computer to achieve load distribution. Of course, a neural network can also be constructed on the grid computer system.

(3) Operation FIG. 3 is an operation flowchart during learning of the neural network device according to the present embodiment.
The input unit 10 captures teacher data composed of input data and output data, the captured input data is processed by the neural network mechanism 20, and the output unit 10 outputs the generated output data. The error calculation unit 41 calculates an error from the output data and the output data of the teacher data (step 201). The weight coefficient reflection unit 42 updates the weight coefficient of the neural network using the calculated error (step 202). The correlation coefficient calculation unit 51 calculates a correlation coefficient from the input layer and hidden layer weight coefficients and the input data (step 211). The error convergence determination unit 43 determines whether the error has converged using the error obtained by the error calculation unit 41 (step 221). If it is determined in step 221 that the convergence has not occurred, the process returns to step 100. If it is determined in step 221 that convergence has occurred, the correlation coefficient condition determination unit 52 uses the correlation coefficient obtained by the correlation coefficient calculation unit 51 and the latest correlation coefficient is lower than the predetermined threshold value. Whether or not (step 231). If it is determined in step 231 that the most recent correlation coefficient is low, it is determined whether the correlation coefficient is increasing and reaches saturation (step 232). If it is determined in step 231 that the correlation coefficient is not low, or if it is determined in step 232 that the correlation coefficient tends to increase and reaches saturation, the learning of the neural network is terminated as a global optimal solution has been obtained. To do. If it is determined in step 232 that the correlation coefficient is increasing and has not reached saturation, it is determined that a local solution has been obtained, and the initial value of the neural network mechanism 20 is reset (step 241). Return.

(4) Effects of the present embodiment As described above, according to the neural network device according to the present embodiment, the error between the output data generated by the neural network and the output data of the teacher data is obtained and the error coefficient is propagated back to determine the weight coefficient. Update, accumulate the obtained error, judge the convergence of the error from the fluctuation of the error, and determine whether the fluctuation of the correlation coefficient obtained continuously when the error is converged satisfies the predetermined condition In this case, the predetermined condition is that the correlation coefficient is not smaller than the predetermined threshold, and that the correlation coefficient is increasing and has reached saturation. If the condition is not met, the neural network is initialized and learned again if the condition is not met, so the steepest descent method is applied to learn quickly. In addition, the neural network that has been learned by the steepest descent method determines whether or not a global optimal solution has actually been obtained. It can be in the desired state.

(Other embodiments)
[Judgment by error and judgment by correlation coefficient]
In the neural network device according to the first embodiment, the fluctuation of the error between the output data and the output data of the teacher data is first determined, and then the correlation coefficient is determined. It is also possible to determine the variation in error after first executing the above determination. However, since the error between the output data and the output data of the teacher data is a value that must be derived for learning in error back-propagation learning, it is preferable to determine the error variation first. In addition, when it is determined that learning has converged from fluctuations in the error between the output data and the output data of the teacher data, at least the local solution is obtained, whereas when it is determined that learning has converged from the correlation coefficient There is a possibility that even a local solution has not been obtained, and it is desirable to first determine the convergence of learning from the variation in error between the output data and the output data of the teacher data.

[Initialization of weighting factor]
In the neural network device according to the first embodiment, for example, initialization is performed using a random number. However, there is a possibility that the initialization using the random number may be substantially the same as the weighting factor before initialization. If the weighting coefficients are almost the same even after initialization, there is a high possibility that they will fall into the same local solution again, and in order to prevent unnecessary learning, the weighting coefficients after initialization and the If the correlation coefficient of the weighting coefficient is equal to or greater than a predetermined threshold, it can be re-initialized.

  For example, as shown in FIG. 4 instead of step 241, the current weighting factor is recorded (step 2411), the weighting factor is initialized (step 2412), and the recorded weighting factor before initialization and weight after initialization are recorded. A correlation coefficient of the coefficient is obtained (step 2413), and it is determined whether or not this correlation coefficient is equal to or greater than a predetermined threshold (step 2414). If it is determined that the correlation coefficient is equal to or greater than the predetermined threshold, the process returns to step 2411. If it is determined that the value is smaller, the process returns to step 100.

A hierarchical neural network was used to estimate the sea surface temperature (SST) using NOAA / AVHRR (High Spatial Resolution Visible Thermal Infrared Radiometer) data. FIG. 5 or FIG. 6 is a thermal infrared image of NOAA / AVHRR bands 4 and 5 and a sea surface temperature estimated by a method called MCSST (multi-channel sea surface temperature). FIG. 5A or 6A is a thermal infrared image of the band 4, and FIG.
(B) or FIG. 6 (b) is a thermal infrared image of band 5, and FIG. 5 (c) or FIG. 6 (c) is an estimated actual measurement value by MCSST. FIG. 5 is that at 17:11 on May 20, 2002, and FIG. 6 is that at 16:45 on November 30, 2004.

  Error back-propagation learning of a hierarchical neural network with FIG. 5 (a) (b) or 6 (a) (b) as input data and FIG. 5 (c) or 6 (c) as a desired output. Went. At this time, the average of the difference between the desired output and the actual output of the hierarchical neural network was evaluated as an average error. The initial value of the weighting factor is given by a uniform random number.

  An example of the course of learning is shown in FIGS. FIG. 7 shows the transition of the solution for the input and the desired output of FIG. 6, and FIG. 8 or FIG. 9 is the case for FIG. 7 and 8 relate to the image area off Miyazaki Prefecture, and FIG. 9 relates to the image area off Kokura.

  FIG. 7 (a) shows that the correlation coefficient converges at a low part without turning to an increasing trend, and FIG. 7 (b) shows that the error decreases as the number of learning passes and the decreasing tendency also converges. Indicates that you are heading toward Therefore, it can be seen from FIG.

  FIG. 8A shows that the correlation coefficient tends to converge through an increasing tendency, and FIG. 8B shows that the error decreases as the number of learning passes, and the decreasing tendency tends to converge. Indicates that Therefore, it can be seen from FIG. 8 that a global optimum solution has been obtained.

  FIG. 9 (a) shows that the correlation coefficient maintains an increasing trend, and FIG. 9 (b) shows that the error decreases as the number of learning passes, and the decreasing trend is also in the direction of convergence. It shows that. Therefore, it can be seen from FIG. 9 that a global optimum solution has been obtained.

  When these are seen, the correlation coefficient of the weight coefficient between an input, an input layer, and a hidden layer becomes high, and the tendency for an average error to decrease is seen. That is, the correlation coefficient can be used as an index for learning convergence.

  Although the present invention has been described with the above embodiments, the technical scope of the present invention is not limited to the scope described in the embodiments, and various modifications or improvements can be added to these embodiments. . And embodiment which added such a change or improvement is also contained in the technical scope of the present invention. This is apparent from the claims and the means for solving the problems.

Claims (12)

It consists of an input layer with nodes, a hidden layer, and an output layer. Input data is input to the input layer, the output data output from the output layer is compared with teacher data prepared in advance corresponding to the input data, and the comparison result A hierarchical neural network device based on error back-propagation learning that learns by updating the weighting coefficient between nodes in the output layer and hidden layer and the weighting coefficient between nodes in the hidden layer and input layer using the error ,
A second learning convergence determination unit configured to determine that learning has converged when a learning convergence condition that a correlation coefficient between input data and a weighting coefficient between nodes of the input layer and the hidden layer is equal to or greater than a predetermined threshold is satisfied; A hierarchical neural network device based on error back-propagation learning. It consists of an input layer with nodes, a hidden layer, and an output layer. Input data is input to the input layer, the output data output from the output layer is compared with teacher data prepared in advance corresponding to the input data, and the comparison result A hierarchical neural network device based on error back-propagation learning that learns by updating the weighting coefficient between nodes in the output layer and hidden layer and the weighting coefficient between nodes in the hidden layer and input layer using the error ,
A first learning convergence determination unit that determines learning convergence based on an error of the learning comparison result;
A second learning convergence determination unit that determines that learning has converged when a learning convergence condition that a correlation coefficient between input data and a weighting coefficient between nodes of the input layer and the hidden layer is equal to or greater than a predetermined threshold is satisfied; Prepared,
A hierarchical neural network device based on error back propagation learning that terminates learning when both the first learning convergence determination unit and the second learning convergence determination unit determine that learning has converged. It consists of an input layer with nodes, a hidden layer, and an output layer. Input data is input to the input layer, the output data output from the output layer is compared with teacher data prepared in advance corresponding to the input data, and the comparison result A hierarchical neural network device based on error back-propagation learning that learns by updating the weighting coefficient between nodes in the output layer and hidden layer and the weighting coefficient between nodes in the hidden layer and input layer using the error ,
A first learning convergence determination unit that determines learning convergence based on an error of the learning comparison result;
A second learning convergence determination unit that determines that learning has converged when a learning convergence condition that a correlation coefficient between input data and a weighting coefficient between nodes of the input layer and the hidden layer is equal to or greater than a predetermined threshold is satisfied; Prepared,
After the first learning convergence determination unit determines that the learning has converged, the second learning convergence determination unit determines the learning convergence,
A hierarchical neural network device based on error back propagation learning that terminates learning when the second learning convergence determination unit determines that learning has converged. It consists of an input layer with nodes, a hidden layer, and an output layer. Input data is input to the input layer, the output data output from the output layer is compared with teacher data prepared in advance corresponding to the input data, and the comparison result A hierarchical neural network device based on error back-propagation learning that learns by updating the weighting coefficient between nodes in the output layer and hidden layer and the weighting coefficient between nodes in the hidden layer and input layer using the error ,
A second learning convergence determination unit configured to determine that learning has converged when the correlation coefficient between the input data and the weighting coefficient between nodes in the input layer and the hidden layer corresponds to a learning convergence condition in which the increasing tendency reaches saturation; A hierarchical neural network device based on error back-propagation learning. It consists of an input layer with nodes, a hidden layer, and an output layer. Input data is input to the input layer, the output data output from the output layer is compared with teacher data prepared in advance corresponding to the input data, and the comparison result A hierarchical neural network device based on error back-propagation learning that learns by updating the weighting coefficient between nodes in the output layer and hidden layer and the weighting coefficient between nodes in the hidden layer and input layer using the error ,
A first learning convergence determination unit that determines learning convergence based on an error of the learning comparison result;
A second learning convergence determination unit that determines that learning has converged when the correlation coefficient between the input data and the weighting coefficient between nodes of the input layer and the hidden layer corresponds to a learning convergence condition in which the increasing tendency reaches saturation; Prepared,
A hierarchical neural network device based on error back propagation learning that terminates learning when both the first learning convergence determination unit and the second learning convergence determination unit determine that learning has converged . It consists of an input layer with nodes, a hidden layer, and an output layer. Input data is input to the input layer, the output data output from the output layer is compared with teacher data prepared in advance corresponding to the input data, and the comparison result A hierarchical neural network device based on error back-propagation learning that learns by updating the weighting coefficient between nodes in the output layer and hidden layer and the weighting coefficient between nodes in the hidden layer and input layer using the error ,
A first learning convergence determination unit that determines learning convergence based on an error of the learning comparison result;
A second learning convergence determination unit that determines that learning has converged when the correlation coefficient between the input data and the weighting coefficient between nodes of the input layer and the hidden layer corresponds to a learning convergence condition in which the increasing tendency reaches saturation; Prepared,
After the first learning convergence determination unit determines that the learning has converged, the second learning convergence determination unit determines the learning convergence,
A hierarchical neural network device based on error back propagation learning that terminates learning when the second learning convergence determination unit determines that learning has converged . 7. The hierarchical neural network based on error back propagation learning according to claim 1, wherein, when the learning convergence condition is not satisfied, the second learning convergence determination unit initializes a weighting coefficient and performs learning again. Equipment . When initializing the weighting coefficient, a correlation coefficient between the initialized weighting coefficient and the weighting coefficient before initialization is obtained, and when the correlation coefficient is equal to or greater than a predetermined threshold, the weighting coefficient is initialized again. Do
A hierarchical neural network device based on error back propagation learning according to claim 7. Input data is input to the input layer and the output data output from the output layer is compared with the teacher data prepared in advance corresponding to the input data. An error back-propagation learning method of a hierarchical neural network device that learns by updating a weighting factor of a node and a weighting factor between nodes of a hidden layer and an input layer,
An error back-propagation learning method for a hierarchical neural network device that determines that learning has converged when a correlation coefficient between input data and a weight coefficient between nodes of an input layer and a hidden layer is equal to or greater than a predetermined threshold. Input data is input to the input layer and the output data output from the output layer is compared with the teacher data prepared in advance corresponding to the input data. An error back-propagation learning method of a hierarchical neural network device that learns by updating a weighting factor of a node and a weighting factor between nodes of a hidden layer and an input layer,
Error back-propagation in a hierarchical neural network device that judges that learning has converged when the correlation coefficient between the input data and the weighting factor between nodes in the input layer and hidden layer satisfies the learning convergence condition where the increasing tendency reaches saturation Learning method. It consists of an input layer with nodes, a hidden layer, and an output layer. Input data is input to the input layer, the output data output from the output layer is compared with teacher data prepared in advance corresponding to the input data, and the comparison result Error propagation propagation learning that makes the computer function as a hierarchical neural network device that learns by updating the weighting coefficient between the nodes in the output layer and the hidden layer and the weighting coefficient between the nodes in the hidden layer and the input layer using the error A program,
Error back propagation learning that causes the computer to function as a second learning convergence determination unit that determines that learning has converged when the correlation coefficient between the input data and the weighting coefficient between nodes in the input layer and the hidden layer is equal to or greater than a predetermined threshold. program. It consists of an input layer with nodes, a hidden layer, and an output layer. Input data is input to the input layer, the output data output from the output layer is compared with teacher data prepared in advance corresponding to the input data, and the comparison result Error propagation propagation learning that makes the computer function as a hierarchical neural network device that learns by updating the weighting coefficient between the nodes in the output layer and the hidden layer and the weighting coefficient between the nodes in the hidden layer and the input layer using the error A program,
A computer serving as a second learning convergence determination unit that determines that learning has converged when the correlation coefficient between the input data and the weighting coefficient between nodes in the input layer and the hidden layer corresponds to a learning convergence condition in which the increasing tendency reaches saturation An error back propagation learning program that works.

Images