JPH08212181A - Construction system for neural network - Google Patents

Abstract

PURPOSE: To provide a construction system for neural network which can decide an optimum neural network parameter in a small number of trial-and-error. CONSTITUTION: A preprocessing part 10 sets the input items, a learning rule, a transfer function and the learning frequency, i.e., the parameters of a neural network and also stores the learning data and the recollection data. A neural network 20 carries out the learning and the recollection based on those parameters and data. A postprocessing part 30 displays a table that shows an average square error of the recollection results and a table and a graph that show the results of comparison between the learning and recollection results and each desirable output result. Thus, the quality of parameters can be studied in the order of the input items, the learning rule, the transfer function and the learning frequency. When the input items are studied, each of different items is deleted among all conceivable input items and the learning and recollection of the network 20 are carried out to study the influences against the output results of deleted items. Then an optimum input item is decided based on the result of the preceding study of influences.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a neural network construction system, and more particularly to a pre-processing and post-processing system for a neural network construction system.

[0002]

2. Description of the Related Art When a new neural network is constructed, in order to accurately predict output data by the neural network, for example, the type of input items of the neural network, the type of input data and output data, that is, the value itself is used. It is necessary to consider whether to use, take deviation, or make logarithm. In addition, a learning rule that is a method of changing the weight of coupling, a transfer function that is an output function of each layer,
It is necessary to consider the number of times of learning how many times the connection weight is changed.

Further, since these input items, learning rules, transfer functions, and number of learnings are related to each other, if one parameter is changed, another parameter must be considered and the study procedure is complicated. It is necessary to make a good plan for the experiment in order to avoid unnecessary examination. Therefore, in the conventional neural network construction system, as preprocessing, the correlation coefficient between input data and output data, multiple regression coefficient, etc. are obtained and the relationship between each input data and output data is shown in a graph. We are trying to show the effect on output data and help determine the input data for the user. In addition, as post-processing, it is configured to display a graph of the mean square error of the learned data, a graph showing the strength of the connection weight, a correlation diagram between the output value of the neural network and the desired output value, etc. There is.

[0004]

However, in the above-mentioned preprocessing, it is difficult to actually determine the input data that affects the output data. That is, the correlation coefficient and the multiple regression coefficient calculated in the pre-processing are statistical methods that treat data as being linear, whereas the neural network considers a data set to be non-linear. Therefore, no matter how much the input data has a very strong correlation with the output data due to the correlation coefficient or the multiple regression coefficient, it will be the input data having a very strong correlation with the output data in the neural network. Not limited to this, conversely, even when the result of the input data having almost no correlation with the output data due to the correlation coefficient or the multiple regression coefficient, it may be a very important input factor in the neural network.

Further, since it is rare to use a neural network for an event in which input data and output data are displayed on one graph and the relationship between them can be easily grasped, trial and error can be made even if the graph is viewed. You cannot reduce the number of times. Also in the post-processing, from the graph of the mean square error of the learning data and the display of the correlation diagram between the output value of the neural network and the desired output value, only the operation accuracy of the neural network can be known. I don't know what to do and the problem of overrunning is not solved. In addition, even if the strength of the connection weight is displayed, it is not useful because the influence of each input data on each output data is often unknown when trying to determine the parameters of the neural network.

The present invention has been made in view of the above circumstances, and an object of the present invention is to provide a neural network construction system capable of determining optimal neural network parameters with a small number of trials and errors.

[0007]

In order to achieve the above object, the present invention is directed to a neural network for constructing the neural network by examining parameters such as input items of the neural network, learning rules, transfer functions, and the number of learnings. A construction system, wherein an input item examination system for examining the input items includes: an input unit for inputting all input items assumed as input items of the neural network; and all input items input from the input unit. From the input item generation unit that generates a combination of arbitrary input items from the input unit, all input items input from the input unit and the input items generated by the input item generation unit Learning based on learning data composed of input data to be used and desired output data, After that, a neural network that recollects and calculates the output data based on recollection data composed of input data used for recollection and its desired output data, and output data calculated by the neural network and its desired data And a post-processing unit that outputs information indicating the calculation accuracy of the neural network based on the output data.

[0008]

According to the present invention, when considering the input items of the neural network, a combination of arbitrary input items from all input items assumed as the input items of the neural network input from the input unit is selected. Generated by the input item generator. Then, with respect to all of these input items and each combination of the input items, the neural network is learned and recalled based on the learning data and the recall data prepared in advance, and the output data is calculated and output to the post-processing unit. The post-processing unit outputs information indicating the calculation accuracy of the neural network based on the output data calculated by the neural network and the desired output data. As a result, it is possible to know the degree of influence of the input data of each input item on the output data. Then, the input item of the neural network can be determined by selecting the input item having a strong influence on the output data.

[0009]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS A preferred embodiment of a neural network construction system according to the present invention will be described in detail below with reference to the accompanying drawings. FIG. 1 is a block diagram showing a construction system of a neural network according to the present invention. As shown in the figure, the construction system of the neural network is roughly divided into a pre-processing unit 10, a neural network 20, and a post-processing unit 30, and the operation of each unit will be described below.

The preprocessing section 10 includes a first input section 10A, a second input section 10B and an input item generation section 1 as shown in FIG.
0C, a learning rule / transfer function setting unit 10D, and a learning number setting unit 10E. The first input unit 10A uses learning data and recollection data (recollection data refers to learning) used for learning and recollection of the neural network 20 from a user (recollection refers to calculating output data by the learned neural network). Input data different from the learning data used for the above and desired output data corresponding thereto) are stored and these data are stored, and the learning data or the recall data is used as the second data when learning and recalling the neural network 20. Output to the input unit 10B. The learning data and the recall data of the first input unit 10A are recorded with the input data for all the input items that are considered to be related to the output data.

In the input item generating section 10C, the first input section 10
Input data to be input to the neural network 20 from among the input items of the learning data and the recall data stored in A (hereinafter, the input items of the learning data and the recall data stored in the first input unit 10A are referred to as all input items). Determine the input items for. The second input unit 10B extracts only the input data of the learning data and the recall data corresponding to the input item determined by the input item generation unit 10C from the learning data and the recall data stored in the first input unit 10A. To the neural network 20.

The learning rule / transfer function setting unit 10D determines the learning rule and the transfer function, and the learning number setting unit 10E determines the learning number. Then, the input item generation unit 10
C, learning rule / transfer function setting unit 10D, learning number setting unit 1
Learning of the neural network 20 is performed using the parameters (input item, learning rule, transfer function of each layer, number of learning) determined in 0E.

The details of the method of determining the input items, the learning rule and the transfer function, and the learning number in the input item generating section 10C, the learning rule / transfer function setting section 10D, and the learning number setting section 10E will be described later. The neural network 20
Learns based on the parameters determined by the preprocessing unit 10,
After the learning is finished, in order to evaluate the quality of the learning, the input data of the learning data and the recall data is input from the second input unit 10B and the output data is calculated. And
The calculated output data is output to the post-processing unit 30.

The post-processing unit 30 is a first post-processing unit 30A,
The second post-processing unit 30B includes the first post-processing unit 30A.
Input data of recall data to neural network 2
The mean square error (with respect to the desired output data (the output data of the recall data recorded in the first input unit 10A)) of the output data calculated by inputting to 0 is calculated, and the result is tabulated and displayed on a monitor or printed. .

Further, the second post-processing unit 30B uses the neural network 2 for each of the learning data and the recall data.
A graph in which the output data calculated by 0 is compared with the desired output data and a correlation diagram are displayed on a monitor or printed. Details of the tables, graphs, and correlation diagrams displayed by the post-processing unit 30 will be described later. FIG. 2 is a flowchart showing an outline of the construction procedure in the construction system of the neural network shown in FIG.

When constructing a neural network, in order to obtain better calculation accuracy, the parameters of the neural network are input items, learning rules (generalized delta rules, etc.), transfer functions (sigmoid functions, etc.), learning. It is necessary to examine the number of times (the number of learning iterations with the same learning data). In this embodiment, among these parameters, those having a strong influence on the construction of the neural network are examined and determined in order. That is, the input item, the learning rule and the transfer function, and the learning frequency are determined in this order.

Therefore, as shown in FIG. 2, first, the input item of the neural network most suitable for the calculation of the output data is determined (step S10). After the input item is determined, the learning rule and the transfer function of each layer that are most suitable for calculating the output data are determined for this input item (step S).
20). And finally, step S10 and step S2
In the input item, learning rule, and transfer function determined in 0, the number of times of learning most suitable for calculation of output data is determined (step S30).

The above is the outline of the procedure for constructing the neural network. Next, referring to the flowchart of FIG.
The details of the procedure for constructing the neural network shown in FIG. 3 will be described (details of step S10 of FIG. 2 are shown in FIG. 3). First, the input data and output data for all input items that are considered to be related to the output data are first input unit 10A for learning data and recall data.
Is input and stored (step S100). Then, in the input item generation unit 10C, the one to be the input item of the neural network 20 is selected from all the input items stored in the first input unit 10A. At this time, in the first learning and recall of the neural network 20, all the input items are set as the input items of the neural network 20 (step S102). The learning rule / transfer function setting unit 10D sets the learning rule and the transfer function, and the learning number setting unit 10E sets the learning number. However, the learning rule, transfer function, and number of learnings are not examined until the most suitable input item for calculating the output data is determined. Until then, the learning rule is a generalized delta rule, the transfer function is a sigmoid function, 100 times / piece is applied to the number of times of learning.

Next, the neural network 20 is learned based on the parameters of the neural network 20 set in the input item generation unit 10C, the learning rule / transfer function setting unit 10D, and the learning number setting unit 10E (step S104). Learning of the neural network 20 is performed by the back propagation method using, for example, the input data of the learning data output from the second input unit 10B and the teacher data that is the desired output data.
Then, after the learning of the neural network 20 is finished, the input data of the learning data and the recall data is input to the neural network 20 from the second input unit 10B to calculate the output data (recall of the neural network), and this calculation The result is output to the post-processing unit 30 (step S106).

After learning and recalling the neural network using all the input items as parameters as described above, the process returns to step S102 through the processes of steps S108 and S110, and the neural network 2
Set the input item of 0 again. At this time, the input item of the neural network 20 is set to one obtained by deleting one item from all the input items. Then, the neural network 20 is learned and recalled in the same manner as described above (the processing from step S102 to step S106 is performed).

Incidentally, step S108 and step S110
In order to repeat the processing from step S102 to step S106 for all combinations in which one item is deleted from all input items, step S
The processing from 102 to step S106 is repeated for the number of all input items (the number of combinations of one input item deleted from all input items is equal to the number of all input items).

As described above, the learning and recall results of the neural network 20 for all combinations of one input item deleted from all input items are output to the post-processing unit 30. Then, the first post-processing unit 30A of the post-processing unit 30
Calculates the mean square error of the output data from the output data of the recall data which is the recall result of the neural network 20 and the desired output data thereof, and makes a list of all the combinations of the input items for monitor or print display (step S112). ).

FIG. 4 is an example of a list displayed in the first post-processing section 30A at this time. As shown in the figure,
In the list, a combination of input items (the input items marked with circles in the figure show the input items selected as parameters of the neural network 20) and the average of the output data of the neural network 20 learned by the parameters. The squared error is shown.

From the figure, it is possible to read how much the input items deleted from all the input items affect the output data. That is, for the mean square error of the output data when learning and recalling using all input items,
When the mean square error of the output data when learning and recalling by deleting one input item does not change much, it can be read that the input item deleted at this time has almost no effect on the output data. On the contrary, when the mean square error of the output data when learning and recalling using all input items is greatly changed, the mean square error of the output data when one input item is deleted and learning and recalling changes significantly. , It can be seen that the input items deleted at this time have a great influence on the output data.

Therefore, from the list of FIG. 4, it is possible to select the input item that is really necessary (most suitable) for calculating the output data among all the input items. The mean square error of the output data when one input item is deleted and the mean square error of the output data when the learning and recall are performed using all the input items is large. Whether it is small or not is determined by the ratio of the amount of change to the mean square error of the output data when learning and recalling using all input items, and the most suitable input item is automatically selected. Good.

If necessary, in the second post-processing unit 30B, a graph displaying the output data of the neural network 20 and the desired output data concerning the learning data and the recall data as shown in FIGS. 5, 6), a correlation diagram of these output data (FIGS. 7, 8) and a table (FIG. 9) showing the absolute error and the relative error of the output data can be displayed on a monitor or printed.

As described above, the post-processing unit 30 determines the most suitable input item for calculating the output data by referring to the tables and graphs shown in FIGS. 4 and 5 to 9 (step S1).
14). Then, this input item is input item generation unit 10C.
Set as the final input item in. After determining the most suitable input items for calculating output data,
Consider the optimal learning rule and transfer function as shown in.
At the time of this examination, learning and recall of the neural network 20 are performed using candidate learning rules and transfer functions.
Further, a list of mean square errors of the output data as shown in FIG. 5, a graph showing the output data of the neural network 20 and the desired output data, and a correlation diagram are displayed on a monitor or printed. Then, the most suitable learning rule and transfer function are determined from these display results. Also,
Regarding the number of times of learning, after the learning rule and the transfer function are determined, they are determined in the same manner as the method of determining the learning rule and the transfer function.

When determining the input items of the neural network 20, one input item was deleted from all the input items, and the degree of influence of the deleted input items on the output data was examined. It is not always necessary to delete one item at a time, and the input item may be divided into some blocks and deleted for each block for consideration. The neural network construction system described above can determine all the parameters of the neural network based on the mean square error, and the procedure of determining all the parameters can be automated. Alternatively, the parameters may be interactively examined, and the final determination of the parameters may be left to the user's judgment.

[0029]

As described above, according to the neural network construction system of the present invention, it is decided that the parameters of the neural network that have a great influence on the output data will be examined first. Learning the neural network took less time by evaluating the mean square error. Also, since the learning results and recall results are displayed in the output graph, it became possible to understand at a glance whether the given data could be learned well and what the neural network represents.

[Brief description of drawings]

FIG. 1 is a block diagram showing a construction system of a neural network according to the present invention.

2 is a flowchart showing an outline of a construction procedure in the neural network construction system shown in FIG. 1;

FIG. 3 is a flowchart showing details of a procedure for constructing a neural network.

FIG. 4 is an example of a list showing a mean square error of output data calculated in a neural network.

FIG. 5 is a graph of output data related to learning data displayed on the post-processing unit 30B.

FIG. 6 is a graph of output data regarding recall data displayed in the post-processing unit 30B.

FIG. 7 is a correlation diagram between predicted values and actual measured values regarding learning data displayed in the post-processing unit 30B.

FIG. 8 is a correlation diagram between a predicted value and an actually measured value regarding learning data displayed in the post-processing unit 30B.

FIG. 9 is a table showing absolute errors and relative errors of output data displayed in the post-processing unit 30B.

[Explanation of symbols]

 10 ... Pre-processing unit 10A ... First input unit 10B ... Second input unit 10C ... Input item generation unit 10D ... Learning rule / transfer function setting unit 10E ... Learning number setting unit 20 ... Neural network 30 ... Post-processing unit 30A ... 1 post-processing section 30B ... second post-processing section

Claims (4)

[Claims] 1. A neural network construction system for constructing the neural network by examining parameters such as an input item of a neural network, a learning rule, a transfer function, and a learning frequency, and an input item examination for examining the input item. The system includes an input unit for inputting all input items assumed as input items of the neural network, and an input item generation for generating a combination of arbitrary input items from all input items input from the input unit. Section, with respect to each of all the input items input from the input unit and the input items generated by the input item generation unit,
Learning based on learning data composed of input data used for learning prepared in advance and desired output data thereof, and after the learning, recall composed of input data used for recall prepared in advance and its desired output data A neural network that recollects the output data based on the data, and a post-processing unit that outputs information indicating the calculation accuracy of the neural network based on the output data calculated by the neural network and the desired output data, A system for constructing a neural network, which comprises: 2. The system for constructing a neural network according to claim 1, wherein the input item generated by the input item generating unit is one in which each input item is deleted from all the input items. . 3. The system for constructing a neural network according to claim 1, wherein the input item examination system first decides an input item and then sequentially examines and decides the learning rule, the transfer function, and the learning frequency. 4. The neural network according to claim 1, wherein the post-processing unit uses a mean square error based on the learning result and the recall result output from the arithmetic unit of the neural network to display in a table. Building system.