JP6458073B2 - Linear polynomial model creation system, creation method, and creation program for multilayer neural network (deep learning) - Google Patents

Description

  The present invention relates to a linear polynomial model creation system, creation method, and creation program in a multilayer neural network (deep learning).

  Multi-layer neural network is a type of machine learning technology with a structure that simulates the human brain, and has an algorithm that automatically acquires important features from a large number of input items, and has excellent data classification and continuous value estimation capabilities. Application to many fields has been attempted (see Patent Document 1).

  A multilayer neural network is roughly divided into three layers (an input layer, an intermediate layer, and an output layer). Each layer is composed of a plurality of nodes, and the nodes between the layers are connected with different connection weights. The input data input to the input layer passes through the nodes in the intermediate layer having different connection weights, and the input items are synthesized according to the connection weights, passed through the output layer, processed as output data, and an output result is created. Such a series of input to output flows enables recognition judgment.

  Conventionally, many types of neural networks have been proposed. A multi-layer neural network having a plurality of intermediate layers has not been realized due to the computing ability of a computer, but has been realized by the recent improvement of computer ability.

  Multilayer neural networks can achieve a level of accuracy that cannot be achieved with other machine learning techniques. Machine learning is a technology in which a machine learns rules from data without human being giving rules explicitly. Usually, learning is performed with input data and teacher labels that indicate what the input data means. The result model is created.

JP 2015-210747 A

  An object of the present invention is to provide a linear polynomial model creation system, method, and program capable of scoring the contribution of input data items used for model creation based on a model created by a multilayer neural network. There is.

  Another object of the present invention is to reduce a calculation cost by approximating linearly an algorithm of a model created by a multilayer neural network, and to create a model that can improve a processing performance. To provide a creation system, a creation method, and a creation program.

  In order to solve the above problems, the linear polynomial model creation system, creation method, and creation program of the present invention provide the following solution.

(1) A linear polynomial model creation system according to the present invention includes a contribution calculation unit that calculates a contribution for each input item based on a connection weight between nodes in each layer constituting a multilayer neural network, and a calculated contribution And a linear polynomial generator for generating a linear polynomial based on the above.

(2) In the above (1), a high contribution item extraction unit that extracts items whose absolute value of the contribution score exceeds a predetermined threshold based on the contribution is provided, and the linear polynomial generation unit includes the extraction A linear polynomial is generated based on the high-contribution item.

(3) In the above (1) or (2), a preprocessing unit that preprocesses original data to create learning data, a model generation unit that generates a model based on the learning data, and the accuracy of the model is evaluated And an evaluation unit.

(4) In any one of the above (1) to (3), a contribution degree visualization unit that visualizes contribution degree for each item based on the extracted high contribution degree item is included.

(5) A method for creating a linear polynomial model of the present invention includes a contribution calculation step in which a computer calculates a contribution for each input item based on a connection weight between nodes in each layer constituting a multilayer neural network, and a calculation And a linear polynomial generation step of generating a linear polynomial based on the contribution degree.

(6) A linear polynomial model creation program according to the present invention includes: a calculation step for calculating a contribution for each item based on a connection weight between nodes in each layer constituting a multilayer neural network; And a linear polynomial generation step of generating a linear polynomial based on the contribution degree.

  According to the present invention, a linear polynomial model creation system, method, and program capable of scoring the contribution of input data items used for model creation based on a model created by a multilayer neural network are provided. be able to.

  In addition, it is possible to provide a system, method, and program for creating a linear polynomial model in a multilayer neural network that can reduce calculation cost and improve processing performance.

  A system for creating a linear polynomial model in a multi-layer neural network can increase installation opportunities, including a system with limited calculation resources, by reducing processing speed and speed.

  In order to install deep learning, it is necessary to reduce the amount of calculation. However, according to the present invention, matrix calculation can be converted into a method with a small amount of calculation called a first-order polynomial. Further, by selecting input items used in the calculation when creating the first order polynomial, further weight reduction can be achieved.

It is a figure which shows the function structure of the multilayer neural network structure analysis system which concerns on a high contribution item. 2 is a flowchart of extracting a high contribution item for improving the performance of a multilayer neural network (deep learning) in the multilayer neural network structure analysis system shown in FIG. It is a graph which shows the relationship of the contribution with respect to a ranking. It is a figure which shows the function structure of the multilayer neural network structure analysis system which concerns on preparation of a linear polynomial model. 5 is a flowchart for creating a linear polynomial model in a multilayer neural network (deep learning) in the multilayer neural network structure analysis system shown in FIG. It is a conceptual diagram of a determination process. It is a conceptual diagram of a multilayer neural network.

  DESCRIPTION OF EMBODIMENTS Hereinafter, embodiments for carrying out the present invention (hereinafter referred to as embodiments) will be described in detail with reference to the accompanying drawings. In the subsequent drawings, the same numbers or symbols are assigned to the same elements throughout the description of the embodiments. In the functional configuration diagram, an arrow between functional blocks represents a data flow direction or a process flow direction.

  As shown in FIG. 1, the multilayer neural network structure analysis system 100 includes a preprocessing unit 110, a model generation unit 120, an evaluation unit 130, a contribution calculation unit 140, a high contribution item extraction unit 150, a data set, and the like. A generation unit 170. The multilayer neural network structure analysis system 100 is configured to include a high contribution item extraction system for improving the performance of the multilayer neural network. The multilayer neural network structure analysis system 100 may include a high contribution visualization unit 160.

  The multilayer neural network structure analysis system 100 is embodied as, for example, an electronic device such as a personal computer, or a computer program implemented therein, and an input signal for inputting an input signal from the outside. Unit 101 and output signal unit 102 for outputting an output signal to the outside.

  Hereinafter, each configuration of the multilayer neural network structure analysis system 100 and an analysis method thereof will be described with reference to FIGS. 1 and 2.

  The preprocessing unit 110 performs preprocessing on the original data 200 to generate learning data and test data (step ST01). The original data 200 is composed of a pair of input data and a teacher label for the input data, and the original data 200 includes noise or the distribution of the original data 200 is biased. Therefore, the preprocessing unit 110 performs noise removal that removes a group far from the average.

  The model generation unit 120 generates a model as shown in FIG. 7, for example, based on the learning data (step ST03). As shown in FIG. 7, the multilayer neural network is composed of an input layer 10, intermediate layers 20 and 30, and an output layer 40. The intermediate layers 20 and 30 are characterized by using a multilayer auto encoder or the like. A model is generated by performing synthesis and extraction.

  Evaluation unit 130 evaluates the accuracy of the model (step ST05). For model evaluation, input data of the original data 200 received by the input signal unit 101 is input to the model, and it is determined for each set whether or not the output value from the output signal unit 102 matches the teacher data. In the original data 200, if the number or ratio of matches is greater than a predetermined value, it is evaluated that the accuracy is high (good).

At the time of evaluation, it is preferable to visualize with the high contribution degree visualization unit 160. The high contribution visualization unit 160 preferably outputs the visualized evaluation to an output device such as a monitor.
If the accuracy is poor as a result of the evaluation, the process proceeds to step ST03 to generate a model again.

  When the evaluation unit 130 makes a high (good) evaluation, the contribution calculation unit 140 calculates the contribution that the input item gives to the output result (step ST07). The contribution degree means the degree of influence of each input item on the output result of each output node. The connection weight between nodes in each layer constituting the multilayer neural network is extracted, and based on this, the input layer 10 to the output layer 40 are extracted. Is calculated by sequentially integrating the matrix representing the coupling between the layers (described later). FIG. 3 shows a diagram in which items are rearranged by the calculated contribution value.

  Then, the high contribution item extraction unit 150 extracts items that greatly contribute to “false” or “true” based on the calculated contribution (step ST09). The method of extracting items that contribute greatly is a method of selecting an item whose absolute value of contribution is greater than or equal to a predetermined value, or, for example, in ranges A and B where the absolute value of contribution is large, as shown in FIG. There is a method of extracting a range when the areas a and b represented by diagonal lines surrounded by a contribution broken line from the end of the horizontal axis become an area of a predetermined range. In the region C near the center of the horizontal axis, the polygonal line of contribution overlaps the horizontal axis and is substantially parallel, so the area between the polygonal line of contribution and the horizontal axis is small and contributes to the output result. Therefore, it is not an extraction target as a high-contribution item.

  The high contribution level visualization unit 160 displays, for example, a graph as shown in FIG. 3 so that an item that greatly contributes to “false” or “true” can be easily grasped visually. In the area A of the left dotted line frame of the graph, it is shown that the higher the contribution level, the greater the contribution to “false”. Similarly, in the area B of the right dotted line frame of the graph, it is shown that the lower the contribution level, the greater the contribution to “true”. On the other hand, it is shown that the item with the middle degree of contribution does not contribute much to “false” or “true”.

  Based on the extracted high-contribution input items, the data set generation unit 170 extracts only the columns that make a significant contribution to “true” or “false” from among the columns that make up the original data 200, and creates new data A set is generated (data set generation step, step ST11).

  The data set created by the data set generation unit 170 is a high contribution item data set, and low contribution items are removed from the learning data. For example, it can be used for the purpose of creating a model again using the high contribution item data set. As a result, in an environment where calculation resources are limited, calculation resources can be saved by creating a model with a data set in which low contribution items are removed.

  In addition, in statistical analysis for clarifying the nature of learning data, it can be used for applications in which analysis is performed with high contribution items as the starting point. This makes it easy to express the properties of the data when analyzed with items that greatly contribute to the results of the analysis.

  Next, each configuration of the multilayer neural network structure analysis system 100a and its analysis method will be described with reference to FIGS. The multilayer neural network structure analysis system 100 a includes a linear polynomial generation unit 180 instead of the data set generation unit 170 of the multilayer neural network structure analysis system 100. The multilayer neural network structure analysis system 100a includes a system for creating a linear polynomial model in a multilayer neural network.

In the following description, only the parts specific to the multilayer neural network structure analysis system 100a will be described.
The multilayer neural network structure analysis system 100a includes a preprocessing unit 110, a model generation unit 120, an evaluation unit 130, a contribution calculation unit 140, and a linear polynomial generation unit 180. The multilayer neural network structure analysis system 100a may include a high contribution item extraction unit 150 and may include a high contribution visualization unit 160.

  The linear polynomial generator 180 generates a linear polynomial model (step ST17).

  The linear polynomial generation unit 180, based on the contribution calculated by the contribution calculation unit 140, or in the case of including the high contribution item extraction unit 150, the item extracted by the high contribution item extraction unit 150 and its contribution Based on the above, a linear polynomial model is generated. In general, a model formed by a multilayer neural network tends to have a large amount of calculation, but the linear polynomial generated by the linear polynomial generator 180 is approximately linear to the algorithm of the model formed by the multilayer neural network. It is possible to reduce the calculation cost.

  As shown in FIG. 6, in the conventional determination processing method using deep learning, when input data is determined, a determination request is made to a model created by deep learning that operates in an environment capable of large-scale operations such as a cloud environment. And receive the determination result. For this reason, the environment in which the model created by deep learning operates is required to have high specifications. Therefore, the linear polynomial generator 180 generates a linear polynomial approximated to a deep learning model in advance, and incorporates it as a scoring logic in the determination rule of the determination process. Even a low-spec terminal can perform the determination process.

  FIG. 7 illustrates a multilayer neural network 1 having an input of an input signal and an output of an output signal. The input data is composed of one or a plurality of sets. Each set includes, for example, a numeric string (for example, 1 and 2) corresponding to two input items and two items (true, False).

  The multilayer neural network 1 is roughly divided into three layers (input layer 10, intermediate layers 20, 30 and output layer 40). Each layer is composed of a plurality of nodes, and the nodes between the layers are connected with different connection weights. ing. Input data input to the input layer 10 passes through the input layer 10 and passes through nodes in the intermediate layers 20 and 30 having different connection weights, and the input items are synthesized in accordance with the connection weights and output through the output layer 40. It is processed as data and an output result is created. Such a series of input to output flows enables recognition judgment. The intermediate layers 20 and 30 can be expressed by matrix weights. Therefore, in the linear polynomial, by calculating the weights of all the matrices of the intermediate layers 20 and 30 in advance, it is possible to generate a prediction calculation with a linear polynomial of the input value and the calculated weight.

  According to the multi-layer neural network structure analysis systems 100 and 100a, a deep learning model (model created by a multi-layer neural network) created using customer behavior history and attribute data for a specific product in the marketing field. By visualizing (visualizing), it is possible to implement effective measures by specifying items to be focused on in marketing.

  In addition, by visualizing deep learning models created using insurance product data and insurance claim data in the field of handling various insurance products, it is possible to grasp the characteristics of insurance products that are unlikely to be charged. It will be possible to develop insurance products with high profitability.

  In addition, by visualizing the deep learning model created by using data of customer companies with healthy financial conditions and deteriorated customer companies in financial institutions, it is possible to detect in advance customer companies with concerns about financial deterioration. Become.

  In addition, by installing a lightweight linear deep learning model (linear polynomial), it is possible to achieve both the calculation speed and detection accuracy at the site, and improve the performance of measuring instruments composed of low-spec hardware. can do.

  As mentioned above, although this invention was demonstrated using embodiment, it cannot be overemphasized that the technical scope of this invention is not limited to the range as described in the said embodiment. It will be apparent to those skilled in the art that various modifications or improvements can be added to the above embodiment. Further, it is apparent from the description of the scope of claims that embodiments with such changes or improvements can be included in the technical scope of the present invention.

  In the above embodiment, the present invention is a product invention, and the system for creating a linear polynomial model in a multilayer neural network is mainly described. However, the present invention is a method invention (creation of a linear polynomial model in a multilayer neural network). Method) or the invention of the above method (a method for creating a linear polynomial model in a multilayer neural network), or a program invention (a program for creating a linear polynomial model in a multilayer neural network) that causes a computer to execute each stage.

  In the above embodiment, a high contribution item extraction system, method, and program for improving the performance of a multilayer neural network are also disclosed.

DESCRIPTION OF SYMBOLS 1 Multilayer neural network 10 Input layer 20, 30 Intermediate layer 40 Output layer 100, 100a Multilayer neural network structure analysis system 101 Input signal unit 102 Output signal unit 110 Preprocessing unit 120 Model generation unit 130 Evaluation unit 140 Contribution calculation unit 150 High Contribution item extraction unit 160 High contribution visualization unit 170 Data set generation unit 180 Linear polynomial generation unit 200 Original data

Claims (6)

A contribution calculating unit that calculates a contribution for each input item based on a connection weight between nodes in each layer constituting the multilayer neural network;
Based on the calculated contribution extracting input items, and a linear polynomial generator for generating a linear polynomial based on the extracted input item, creation system of the linear polynomial model. Wherein said calculated in the contribution degree calculating section on the basis of the contribution, includes high contribution item extraction unit extracting input items which the absolute value of the contribution score exceeds a predetermined threshold value,
The linear polynomial generating unit generates a linear polynomial based on the input item of the high contribution extracted in the high contribution item extraction section, creating system of a linear polynomial model according to claim 1. Based on the input items of the high contribution extracted in the high contribution item extraction section includes a contribution visualizing unit that performs visualization of the contribution of each input item, creating a linear polynomial model according to claim 2 system. A preprocessing unit that preprocesses original data to create learning data, a model generation unit that generates a model based on the learning data,
The system for creating a linear polynomial model according to claim 1 , further comprising: an evaluation unit that evaluates the accuracy of the model. Computer
A contribution calculation step for calculating a contribution for each input item based on a connection weight between nodes in each layer constituting the multilayer neural network;
Based on the calculated contribution extracting input fields, performing a linear polynomial generation step of generating a linear polynomial based on the extracted input item, how to create a linear polynomial model. On the computer,
A contribution calculation step for calculating a contribution for each input item based on a connection weight between nodes in each layer constituting the multilayer neural network;
Based on the calculated contribution extracting input items, to execute a linear polynomial generation step of generating a linear polynomial based on the extracted input item, the linear polynomial model creation program.

Images