JP2019053491A - Neural network evaluation apparatus, neural network evaluation method, and program - Google Patents

Abstract

To provide a neural network evaluation device capable of providing a result of classifying a sample set of input data of a neural network into clusters according to information on a fluctuation of an output value when an input value of input data is fluctuated regarding the output value calculated based on a value of an output node of the neural network, neural network evaluation method, and program.SOLUTION: The neural network evaluation apparatus includes: a fluctuation deriving unit; a grouping unit; and an output unit. The fluctuation deriving unit obtains a fluctuation of an output value when an input value contained in a sample of input data input to a neural network is fluctuated concerning the output value calculated based on a value of an output node of the neural network. The grouping unit classifies the samples into groups based on the fluctuation of the output value. The output unit outputs the result of classifying the sample.SELECTED DRAWING: Figure 2

Description

  Embodiments described herein relate generally to a neural network evaluation apparatus, a neural network evaluation method, and a program.

  In recent years, machine learning techniques have begun to be used in various fields. When trying to use a model obtained by machine learning for important decision / judgment, etc., if the model remains in the black box, it is not trusted by the user, and the user may hesitate to make a decision / judgement based on the model. There is sex. Therefore, it is desirable to be able to provide information that helps the user's understanding of the relationship between data captured by the model and the basis of the results output by the model. In addition, understanding the relationship captured by the model itself may be useful for some decision / judgement or improvement of the user.

  Among machine learning techniques, neural networks are particularly popular because of their potential for obtaining highly accurate models. However, the relationship between the data captured by the neural network is complicated and difficult for the user to understand as it is. Therefore, in order to make it easy for the user to understand the model of the neural network, a technique for explaining the basis of the model output, a technique for simplifying the model, a technique for summarizing the model, and the like are important.

  With regard to machine learning, in medical diagnosis, there is known a technique that presents, for each diagnosis target data, a determination result of a model and a contribution that is a factor that led to calculation of the determination.

Further, the effectiveness W of each variable is obtained by a learning algorithm, and the obtained effectiveness W of each variable is arranged in descending order. From the top, there is known a technique for selecting a variable such that the sum of the effectiveness of variables is equal to or greater than a certain ratio of the sum of the effectiveness of all variables.
In the conventional technique, in order for the user to grasp the properties of the entire model, it is necessary to confirm information about a large number of samples, and the burden on the user may be heavy. In addition, it may be difficult to sufficiently simplify the model while maintaining the properties of the model.

JP 2006-43007 A JP 2012-73761 A

  The problem to be solved by the present invention is that the output when the input value of the input data is varied in the sample set of the input data of the neural network with respect to the output value calculated based on the value of the output node of the neural network. To provide a neural network evaluation apparatus, a neural network evaluation method, and a program capable of providing a result classified into clusters based on a change in value.

  The neural network evaluation apparatus according to the embodiment includes a variation deriving unit and a grouping unit. The fluctuation derivation unit obtains a fluctuation of the output value when an input value of a sample of input data inputted to the neural network is fluctuated with respect to an output value calculated based on a value of an output node of the neural network. . The grouping unit classifies the samples into groups based on fluctuations in the output value.

The figure which shows an example of the neural network evaluation system of 1st Embodiment. The block diagram which shows an example of the neural network evaluation apparatus of 1st Embodiment. The figure which shows an example of a sample matrix. The figure which shows an example of a fluctuation matrix. The figure which shows an example of the clustering result in the neural network evaluation apparatus of 1st Embodiment. The figure which shows an example of model summary information. The flowchart which shows an example of operation | movement of the neural network evaluation apparatus of 1st Embodiment. The figure which shows an example of a fluctuation matrix. The block diagram which shows an example of the neural network evaluation apparatus of 2nd Embodiment. The figure which shows an example of the information of the intercept in the neural network evaluation apparatus of 2nd Embodiment. The figure which shows an example of model summary information. The flowchart which shows an example of operation | movement of the neural network evaluation apparatus of 2nd Embodiment. The flowchart which shows an example of operation | movement of the neural network evaluation apparatus of 2nd Embodiment.

  Hereinafter, a neural network evaluation device, a neural network evaluation method, and a program according to embodiments will be described with reference to the drawings. In the following description, the same reference numerals are given to configurations having the same or similar functions. And the description which overlaps those structures may be abbreviate | omitted.

  Further, “based on XX” in the present application means “based on at least XX”, and includes a case based on another element in addition to XX. Further, “based on XX” is not limited to the case where XX is directly used, but also includes the case where it is based on an operation or processing performed on XX. “XX” is an arbitrary element (for example, arbitrary information).

(First embodiment)
[Neural network evaluation system]
FIG. 1 is a diagram illustrating an example of a neural network evaluation system according to the first embodiment. The neural network evaluation system 1 according to the first embodiment includes a neural network evaluation device 100 and a terminal device 200. Neural network evaluation apparatus 100 and terminal apparatus 200 are connected via network 50.

  The neural network evaluation apparatus 100 outputs the neural network when each of the input values included in the sample is minutely changed with respect to the sample of input data (hereinafter referred to as “sample”) with respect to the specified neural network. The fluctuation of the output value calculated based on the value of the node (hereinafter simply referred to as “output value”) is calculated. Hereinafter, a vector obtained by arranging the fluctuation of the output value when each of the plurality of input values is minutely varied for all the input variables is referred to as a variation vector. The output value may be the value of any one of the output nodes of the neural network, or may be calculated by applying some operation to the value of one or more nodes of the output nodes. It may be a value.

  The neural network evaluation apparatus 100 calculates variation vectors for a plurality of sample sets designated by the user, and arranges the calculated variation vectors. Hereinafter, a matrix obtained by arranging the variation vectors is referred to as a variation matrix. Here, each of the plurality of samples is associated with an element included in the variation matrix.

  The neural network evaluation apparatus 100 groups each sample based on the variation matrix. In the present embodiment, a description will be continued for a case where clustering is applied as an example of grouping. In this case, the neural network evaluation apparatus 100 clusters each sample based on the variation matrix. The neural network evaluation apparatus 100 generates model summary information for each cluster. Here, the model summary information includes a ranking in which input variables are arranged in order of strong influence on output values for each cluster, and a model that approximates the relationship between output values and input variables for each cluster in a simple form (for example, a linear model). ) Etc.

  The terminal device 200 receives the model summary information transmitted by the neural network evaluation device 100 and presents the received summary information to the user.

(Neural network evaluation device)
FIG. 2 is a block diagram illustrating an example of the neural network evaluation apparatus according to the first embodiment. The neural network evaluation apparatus 100 includes, for example, a communication unit 102, a storage unit 110, and an information processing unit 130.

  The communication unit 102 is realized by a communication module. The communication unit 102 communicates with the terminal device 200 via the network 50. In addition, the communication unit 102 transmits the model summary information to the terminal device 200.

  The storage unit 110 is realized by, for example, a random access memory (RAM), a read only memory (ROM), a hard disk drive (HDD), a flash memory, or a hybrid storage device in which a plurality of these are combined. Instead of being provided as a part of the neural network evaluation apparatus 100, a part or all of the storage unit 110 is replaced by a processor of the neural network evaluation apparatus 100 such as a NAS (Network Attached Storage) or an external storage server. It may be realized by an external device that can be accessed via the network. The storage unit 110 stores a program 112 executed by the information processing unit 130 and setting data 114. The setting data 114 includes contents set by the user regarding processing executed by the information processing unit 130. Specifically, the contents set by the user include model data, sample sets, various settings for grouping sample sets, and the like.

[Model data]
Model data will be described. The model data is neural network data for which summary information is generated. The model data includes, at a minimum, information necessary for calculating the value of each output node of the neural network for given input data. Here, information necessary for calculating the value of each output node is various parameters such as the weight of each edge, the value of bias, and other parameters of the activation function. In the first embodiment, the description will be continued for the case where model data is given in advance to the neural network evaluation apparatus 100 by the user. It should be noted that the description will be continued for the case where the neural network that is the target for generating the summary information is learned by a learning data set that is standardized or normalized for each input variable.

[Sample set]
A sample set will be described. The sample set is a set of samples of data input to the neural network. Here, the sample is a collection of specific values for each input variable of the neural network for all input variables. For example, if a sample of one input data is expressed as a vector in which input variables are arranged side by side in order, the sample set is a matrix of size “number of samples × number of input variables” in which vectors of all samples are arranged vertically. Can be expressed as This matrix is hereinafter referred to as a “sample matrix”.

FIG. 3 is a diagram illustrating an example of a sample matrix. FIG. 3 is an example of a sample matrix when there are three input variables (x ₁ , x ₂ , x ₃ ). Specifically, Sample 1 includes 0.45, 0.12, and 0.81, and Sample 2 includes 0.18, 0.29, and 0.03. Sample 3 includes -0.23, 0.32, and 0.53, ... Sample N includes -0.81, -0.50, and -0.37. And are included. In the first embodiment, the description will be continued for a case where a sample set is given in advance to the neural network evaluation apparatus 100 by the user. Specifically, a sample set of learning data when learning the neural network is given to the neural network evaluation apparatus 100. Each sample included in the sample set is assumed to have been converted for each input variable with the same coefficient as the standardization or normalization performed on the learning data set before learning of the neural network.

  Returning to FIG. 2, the description will be continued. The information processing unit 130 is, for example, a functional unit (hereinafter referred to as “software functional unit”) realized by a processor such as a CPU (Central Processing Unit) executing the program 112 stored in the storage unit 110. . The information processing unit 130 may be realized by hardware such as LSI (Large Scale Integration), ASIC (Application Specific Integrated Circuit), or FPGA (Field-Programmable Gate Array). It may be realized by a combination of The information processing unit 130 includes, for example, a variation matrix calculation unit 132, a grouping unit 134, and a summary information generation unit 136.

  The fluctuation matrix calculation unit 132 performs settings for calculating the fluctuation matrix based on the setting data 114 stored in the storage unit 110. Specifically, when the neural network for which summary information is generated includes a plurality of output nodes, and the value of one of the output nodes is set as an output value to calculate the variation matrix, the variation matrix calculation unit 132 is The target output node for calculating the variation matrix is set. In addition, when a calculation is performed on the value of one or a plurality of nodes among the output nodes of the neural network, and a value obtained thereby is set as an output value to calculate a variation matrix, the calculation method is set.

  Further, the variation matrix calculation unit 132 refers to the model data and the sample set included in the setting data 114, and calculates the variation matrix according to the setting. Specifically, the variation matrix calculation unit 132 calculates a gradient vector of output values for each sample. Specifically, this can be performed as follows.

  Here, “→” above the variable name indicates that the variable is a vector. The variation matrix calculation unit 132 calculates the value of each output node of the neural network based on the model data. The variation matrix calculation unit 132 acquires the output value by selecting the value of the output node specified in the setting data 114 or by applying the calculation method set in the setting data 114 to the value of the output node. . At this time, the gradient vector with respect to a certain input data sample regarding the acquired output value is represented by Formula (1).

  The variation matrix calculation unit 132 calculates a gradient vector for the entire sample set, and outputs a matrix in which the calculated gradient vectors are arranged to the grouping unit 134 as a variation matrix. Here, the partial differential value constituting the gradient vector can be calculated using a chain rule or the like. The chain rule refers to a relational expression that, when differentiating a composite function obtained by combining a plurality of functions, the derivative is given by the product of the respective derivatives.

  FIG. 4 is a diagram illustrating an example of a variation matrix. FIG. 4 shows an example in which there are three input variables. In FIG. 4, each row of the variation matrix is arranged in the order of the samples described with reference to FIG. 3 so that the correspondence between the gradient vector of each row of the variation matrix calculated from which sample of the sample set can be understood. Shown side by side. Specifically, the gradient vector of sample 1 is 0.66, 0.01, 0.24, the gradient vector of sample 2 is 0.02, 0.74, −0.71, and sample 3 Are gradient vectors of 0.01, 0.63, −0.65,..., And the gradient vectors of sample N are 0.71, 0.01, 0.27.

  The grouping unit 134 acquires the variation matrix output from the variation matrix calculation unit 132. The grouping unit 134 groups the samples included in the sample set into groups according to the settings included in the setting data 114 based on the obtained variation matrix. In the present embodiment, a description will be continued for a case where clustering is applied as an example of grouping. In this case, the grouping unit 134 clusters the samples included in the sample set based on the obtained variation matrix. Specifically, the grouping unit 134 clusters the gradient vectors of each row of the variation matrix using a clustering algorithm such as a k-means method. Each row of the variation matrix is associated with each sample included in the sample set. For this reason, each sample is clustered by clustering the gradient vectors of each row of the variation matrix.

  The k-means method will be described. The k-means method is one of non-hierarchical clustering algorithms. The k-means method uses an average of clusters and classifies a sample set into k given clusters. An example of the procedure of the k-means method will be described. Let n be the number of data and k be the number of clusters. A cluster is randomly allocated for each data. Calculate the center of each cluster based on the allocated data. The distance between each data and the average of each cluster is obtained, and each data is reassigned to the nearest central cluster. If the allocation of all data clusters has not changed in the above processing, or if the amount of change has fallen below a predetermined threshold value, it is determined that it has converged, and the processing ends. Otherwise, the center of each cluster is recalculated from the newly allocated cluster and the above processing is repeated.

  The user stores information indicating the number of clusters to be clustered using the k-means method in the setting data 114. The grouping unit 134 outputs a clustering result including information indicating a cluster to which each sample included in the sample set belongs to the summary information generation unit 136.

  FIG. 5 is a diagram illustrating an example of a clustering result in the neural network evaluation apparatus according to the first embodiment. FIG. 5 shows a clustering result obtained when the sample set includes N samples (N is an integer of N> 0). The clustering result includes the cluster number of sample 1−the cluster number of sample N. Specifically, sample 1 has a cluster number of 1, sample 2 has a cluster number of 2, sample 3 has a cluster number of 2,..., Sample N has a cluster number of 1.

  The summary information generation unit 136 acquires the variation matrix output from the variation matrix calculation unit 132 and the clustering result output from the grouping unit 134. Hereinafter, in the first embodiment, the description will be continued assuming that, for each cluster output by the grouping unit 134, a ranking in which input variables are arranged in descending order of the degree of influence on the output value is created as model summary information. The summary information generation unit 136 calculates the influence of the input variable on the output value for each cluster (for each group) based on the acquired clustering result. The summary information generation unit 136 creates a ranking in which input variables are arranged in descending order of the degree of influence on the output value. Then, the summary information generation unit 136 outputs the created ranking to the communication unit 102 as model summary information.

  Specifically, the summary information generation unit 136 generates a matrix in which a row corresponding to a sample belonging to the cluster is extracted from the variation matrix for a certain cluster. The summary information generation unit 136 performs summation, average value, median value, maximum value, minimum value, and maximum value for each column for the generated matrix or the matrix obtained by taking the absolute value of each element of the generated matrix. Calculate statistics such as frequent values. The summary information generation unit 136 regards the calculated statistic as an influence degree for each input variable. The summary information generation unit 136 sorts the input variables in descending order of influence, and generates all input variables or the number of input variables specified in the setting data 114 from the top as rankings. The summary information generation unit 136 performs processing for generating rankings in all clusters, and outputs the ranking for each cluster to the communication unit 102 as model summary information. The communication unit 102 acquires the model summary information output from the summary information generation unit 136 and transmits the acquired model summary information to the terminal device 200. The terminal device 200 receives the model summary information transmitted from the neural network evaluation device 100 and presents the received model summary information to the user.

FIG. 6 is a diagram illustrating an example of model summary information. In the neural network evaluation system 1 of the first embodiment, the terminal summary 200 displays the model summary information. In the example shown in FIG. 6, the ranking of the input variables is displayed for the cluster selected by the user among cluster 1, cluster 2, cluster 3, and cluster 4. In the example shown in FIG. 6, cluster 1 is selected, and the ranking of input variables is shown for cluster 1. Specifically, variable names x ₁₁ , x ₄ , x ₈ , x ₅ , x ₁ , x ₇ are shown in descending order of influence. By configuring in this way, the neural network evaluation apparatus 100 can inform the user which input variable has a strong influence on the output value for each cluster. For this reason, the user can deepen understanding of the model. As model summary information, instead of ranking the input variables for each cluster, generate information that approximates the relationship between the input variables and output values of the neural network for each cluster with a simple model (such as a linear model). May be.

(Operation of the neural network evaluation device)
FIG. 7 is a flowchart illustrating an example of the operation of the neural network evaluation apparatus according to the first embodiment.

  (Step S <b> 101) The fluctuation matrix calculation unit 132 performs settings for calculating the fluctuation matrix based on information stored in the setting data 114 stored in the storage unit 110. The variation matrix calculation unit 132 refers to the model data and the sample set, and calculates the variation matrix according to the setting. The variation matrix calculation unit 132 outputs the calculated variation matrix to the grouping unit 134.

  (Step S102) The grouping unit 134 of the neural network evaluation apparatus 100 acquires the variation matrix output from the variation matrix calculation unit 132, and sets the sample set according to the setting included in the setting data 114 based on the acquired variation matrix. Cluster samples included in. The grouping unit 134 outputs the clustering result to the summary information generation unit 136.

  (Step S103) The summary information generation unit 136 of the neural network evaluation apparatus 100 acquires the clustering result output from the grouping unit 134. The summary information generation unit 136 calculates the influence of the input variable on the output value for each cluster based on the acquired clustering result. The summary information generation unit 136 creates a ranking in which input variables are arranged in descending order of the degree of influence on the output value.

  (Step S104) The summary information generation unit 136 of the neural network evaluation apparatus 100 outputs information indicating the calculated ranking to the communication unit 102 as model summary information. The communication unit 102 transmits the model summary information output from the summary information generation unit 136 to the terminal device 200. The terminal device 200 receives the model summary information transmitted from the neural network evaluation device 100 and presents the received model summary information to the user.

  In the above-described embodiment, the case where the sample set, the variation information, and the clustering result are represented by a matrix has been described. For example, a sample set, variation information, and clustering result may be represented by vectors.

  In the above-described embodiment, the case where the model data is given in advance to the neural network evaluation apparatus 100 by the user has been described, but this is not restrictive. For example, a function for learning a neural network may be added to the neural network evaluation apparatus 100, and a model learned by the function for learning the added neural network may be a target for generating summary information.

  As described in the above-described embodiment, in addition to selecting any value of the output node of the neural network as an output value as it is, some operation is performed on the value of one or a plurality of nodes among the output nodes, thereby The fluctuation matrix may be calculated by setting the obtained value as the output value. The “some operation” at this time may be, for example, applying a function such as softmax to the value of the output node.

  In the above-described embodiment, the case where the rows of the variation matrix are arranged in accordance with the order of the samples included in the sample set has been described. For example, the rows of the variation matrix may be arranged in the order opposite to the order of the samples included in the sample set.

  In the embodiment described above, the case where the variation matrix calculation unit 132 derives the gradient vector by calculating the partial differentiation with reference to the model data included in the setting data 114 has been described. For example, the model data itself has a function of calculating a gradient vector for given input data, and the variation matrix calculation unit 132 may only refer to the gradient vector calculated by the model data. Further, the variation matrix calculation unit 132 may derive the gradient vector by a difference method or the like instead of calculating the partial differentiation.

  In the above-described embodiment, the grouping unit 134 has been described as performing clustering using a clustering algorithm such as the k-means method, but this is not restrictive. For example, any algorithm can be used as the clustering algorithm. Further, the grouping unit 134 may include a plurality of grouping algorithms so that the user can select. In addition, the grouping unit 134 generates a model (such as a linear model) designated for each group after grouping by some clustering algorithm, evaluates the degree of fit to those models, and obtains the evaluation result of the degree of fit. Based on this, each sample may be redistributed. Specifically, for each sample, the smallness of the difference when comparing the result applied to the linear model of each group with the result obtained by the neural network is evaluated as the degree of fit to the linear model. A method of redistributing each sample to a group having a linear model having the highest degree of fit is conceivable. Further, the generation and redistribution of the linear model may be alternately performed a plurality of times.

  In the above-described embodiment, the case where the grouping unit 134 outputs the clustering result to the summary information generation unit 136 has been described, but this is not restrictive. For example, the grouping unit 134 may output the clustering result to the communication unit 102. Then, the communication unit 102 may acquire the clustering result output from the grouping unit 134 and transmit the acquired clustering result to the terminal device 200. The terminal device 200 may acquire the clustering result transmitted by the neural network evaluation device 100 and present the acquired cluster link result to the user.

  In the above-described embodiment, the case where the grouping unit 134 performs clustering using all the elements of the variation matrix has been described, but the present invention is not limited to this example. For example, the grouping unit 134 may perform clustering using some elements of the variation matrix. Specifically, the grouping unit 134 may extract the elements of some columns of the variation matrix and perform clustering using the extracted elements. At this time, the column extracted from the variation matrix may be determined based on the content specified by the user. Further, the grouping unit 134 calculates, for each column of the matrix, the sum, average value, median value, maximum value, minimum value, and mode of the variation matrix or a matrix obtained by taking the absolute values of all the elements of the variation matrix. A statistic such as a value may be calculated, and a certain number of column elements may be extracted from the variation matrix in descending order of the calculated statistic. Then, the grouping unit 134 may perform clustering using the extracted elements.

  In the above-described embodiment, the grouping unit 134 may perform clustering using both the variation matrix and the sample matrix. Specifically, the grouping unit 134 may use a matrix obtained by concatenating the variation matrix and the sample matrix as a new variation matrix.

  FIG. 8 is a diagram illustrating an example of a variation matrix. In the example shown in FIG. 8, a variation matrix is newly created by concatenating a sample matrix and a variation matrix. Here, instead of the variation matrix, a matrix obtained by extracting some columns of the variation matrix may be used, or a matrix obtained by extracting some columns of the sample matrix may be used instead of the sample matrix. Good.

  In the above-described embodiment, the grouping unit 134 performs multi-stage clustering, such as clustering with a sample matrix and then subdividing the clusters by clustering with a variation matrix for each cluster obtained by the clustering. Also good. By using the sample matrix for clustering, clustering considering the similarity of input variables is possible, and the continuity of cluster distribution in the space of input variables can be improved.

  In the embodiment described above, the case where the neural network evaluation apparatus 100 acquires the sample set included in the setting data 114 of the storage unit 110 and transmits the model summary information to the terminal apparatus 200 has been described. I can't. For example, the neural network evaluation apparatus 100 includes an input device such as a keyboard and a mouse and a display device such as a display, acquires a sample input by the user operating the input device, and stores model summary information on the display device. You may make it output.

  According to at least one embodiment described above, the variation matrix calculation unit 132 calculates information indicating the relationship between the input variable and the output value for each sample. By configuring in this way, the neural network evaluation apparatus 100 can be used in a faster method such as partial differentiation or difference method compared to the case where similar information is calculated by learning a linear model or the like for each sample. The relationship between the input variable and the output value can be calculated. Further, the neural network evaluation apparatus 100 can handle data having a large number of samples and a large number of input dimensions.

  Further, according to at least one embodiment described above, the grouping unit 134 collects samples having similar relationships between input variables and output values based on a variation matrix. By configuring in this way, the neural network evaluation apparatus 100 can reduce the labor required for the user to grasp the information as compared with the case of presenting information about individual samples. Further, by collecting samples having similar properties into clusters, the neural network evaluation apparatus 100 can express the relationship between input variables and output values with a simpler model than the original neural network in each cluster. Also, compared to a technique that simplifies a model collectively without considering the difference in model characteristics between samples, the neural network evaluation apparatus 100 presents information for each cluster. Information can be presented while maintaining

  Further, in the technique of preferentially selecting and presenting samples whose output values depend on a large number of input variables, there is a possibility that samples whose output values strongly depend only on a small number of input variables are not presented. On the other hand, if the neural network evaluation apparatus 100 can appropriately perform grouping such as clustering, a set of samples whose output values strongly depend only on a small number of input variables are output values on many input variables. Forms a different cluster than the sample on which it depends. Therefore, the neural network evaluation apparatus 100 can present model summary information to the user even for a set of samples whose output values strongly depend only on a small number of input variables.

  Further, according to at least one embodiment described above, the summary information generation unit 136 approximates the ranking of input variables having a large influence on the output value and the relationship between the input variable and the output value for each cluster. Create a linear model. By configuring in this way, the user can specify a notable variable for each cluster, and can grasp the relationship between each input variable and the output value with a simple model.

(Second Embodiment)
(Neural network evaluation system)
FIG. 1 can be applied to an example of the neural network evaluation system according to the second embodiment. However, the neural network evaluation system 1A includes a neural network evaluation apparatus 100A instead of the neural network evaluation apparatus 100.

  The neural network evaluation apparatus 100A approximates the relationship between the input data and the output value by a linear model for each cluster, and outputs the relationship between the input data and the output value obtained by the approximation as model summary information. The neural network evaluation apparatus 100A classifies unknown samples into clusters, and predicts output values using a linear model corresponding to the clusters.

(Neural network evaluation device)
FIG. 9 is a block diagram illustrating an example of a neural network evaluation apparatus according to the second embodiment. Compared to the neural network evaluation device 100, the neural network evaluation device 100A includes an information processing unit 130A instead of the information processing unit 130. In addition, the communication unit 102 receives an unknown sample transmitted by the terminal device 200.

  The information processing unit 130 </ b> A is a software function unit that is realized when a processor such as a CPU executes the program 112 stored in the storage unit 110. The information processing unit 130A may be realized by hardware such as LSI, ASIC, or FPGA, or may be realized by a combination of a software function unit and hardware. The information processing unit 130A includes, for example, a variation matrix calculation unit 132, a grouping unit 134A, a summary information generation unit 136A, a classification rule generation unit 138, and a prediction unit 140.

  The grouping unit 134A acquires the variation matrix output from the variation matrix calculation unit 132. The grouping unit 134A refers to the model data included in the setting data 114 and the sample set, and calculates the intercept of the surface in contact with the model of the neural network for each sample using Expression (2).

  FIG. 10 is a diagram illustrating an example of intercept information in the neural network evaluation apparatus according to the second embodiment. In the example illustrated in FIG. 10, the grouping unit 134A calculates intercepts for all samples in the sample set, and arranges the calculated intercepts. Specifically, the intercept of sample 1 is -0.52, the intercept of sample 2 is -0.20, the intercept of sample 3 is -0.23, ..., the intercept of sample N is 0.36.

  The grouping unit 134A groups each sample into a group based on the variation matrix and the intercept. In the present embodiment, a description will be continued for a case where clustering is applied as an example of grouping. In this case, the grouping unit 134A clusters each sample based on the variation matrix and the intercept. Specifically, the grouping unit 134A clusters each sample based on the variation matrix. The first embodiment can be applied to the process of clustering each sample based on the variation matrix. For each cluster obtained by clustering, the grouping unit 134A further clusters the samples belonging to the cluster based on the similarity of the intercepts. Here, as described in the first embodiment, the grouping unit 134A may use a matrix obtained by extracting some columns from the variation matrix for clustering. Further, the grouping unit 134A generates a new matrix by concatenating the matrix obtained by arranging the intercepts described with reference to FIG. 10 to the variation matrix, and performs clustering based on the generated matrix. You may go. The grouping unit 134A newly generates and generates a matrix by concatenating a matrix obtained by arranging the intercept described with reference to FIG. 10 to a matrix obtained by extracting some columns of the variation matrix. Clustering may be performed based on the matrix. In addition, when the matrix obtained by arranging the intercept described with reference to FIG. 10 is connected to the matrix obtained by extracting the variation matrix or a part of the matrix, the elements of each matrix are multiplied by a constant, for example. You may connect after weighting.

  When clustering is performed based only on the variation matrix, samples having similar inclinations are clustered into the same cluster based only on the inclination of the surface in contact with the model of the neural network. For this reason, if samples having significantly different intercepts are mixed in each cluster, the relationship between the input data and the output value may not be approximated with a single linear model. The neural network evaluation apparatus 100A performs clustering based on the intercept, so that samples with greatly different intercepts are divided into different clusters. By configuring in this way, it becomes possible to fit a linear model more appropriately. The grouping unit 134A outputs the clustering result and the intercept information for each sample to the summary information generation unit 136A.

  The summary information generation unit 136A acquires the variation matrix calculated by the variation matrix calculation unit 132, the clustering result output by the grouping unit 134A, and the intercept information for each sample. The summary information generation unit 136A generates a linear model that approximates the relationship between the input data and the output value for each cluster based on the acquired clustering result. The summary information generation unit 136A performs the derivation of the linear model by the following method, for example. For a certain cluster c, a linear model that approximates an output value with respect to input data is obtained by Expression (3).

  In Equations (4) and (5), if the parentheses in the mean operation are vectors, an average vector (a vector obtained by taking an average value for each element) is taken, and if it is a scalar, an average value is taken. . The summary information generation unit 136A obtains a linear model for all clusters, and outputs the obtained linear model for each cluster to the classification rule generation unit 138 as model summary information. The summary information generation unit 136A may output the model summary information to the communication unit 102. In this case, the communication unit 102 transmits the model summary information output from the summary information generation unit 136 to the terminal device 200. The terminal device 200 receives the model summary information transmitted by the neural network evaluation device 100A, and presents the received model summary information to the user.

  The classification rule generation unit 138 generates a classification rule for determining a cluster to which an unknown sample belongs. Specifically, the classification rule generation unit 138 generates a classification rule by learning an arbitrary identification model using each row of the sample matrix as an input variable and the clustering result corresponding to each row as teacher data. The classification rule generation unit 138 outputs the generated classification rule to the prediction unit 140. The classification rule generation unit 138 may output the classification rule to the communication unit 102. In this case, the communication unit 102 transmits the classification rule output from the classification rule generation unit 138 to the terminal device 200. The terminal device 200 receives the classification rule transmitted by the neural network evaluation device 100A and presents the received classification rule to the user.

  The prediction unit 140 acquires the classification rule output from the classification rule generation unit 138. The prediction unit 140 acquires an unknown sample by controlling the communication unit 102. Here, the unknown sample is assumed to have been converted for each input variable with the same coefficient as that of normalization or normalization performed on the learning data set before learning of the neural network. The prediction unit 140 acquires an unknown sample, refers to the classification rule, and determines which cluster the unknown sample belongs to. The prediction unit 140 predicts an output value using a linear model of a cluster to which an unknown sample belongs, and outputs the prediction value to the communication unit 102. The communication unit 102 transmits the predicted value output by the prediction unit 140 to the terminal device 200. The terminal device 200 receives the predicted value transmitted by the neural network evaluation device 100A, and presents the received predicted value to the user.

FIG. 11 is a diagram illustrating an example of model summary information. In the neural network evaluation system 1A of the second embodiment, the model summary information is displayed by the terminal device 200. In the example shown in FIG. 11, the weight for each input variable of the linear model is displayed for the cluster selected by the user among the cluster 1, cluster 2, cluster 3, and cluster 4. In the example shown in FIG. 11, cluster 1 is selected, and for cluster 1, the weight of the linear model for each input variable is shown. Specifically, variable names x ₁₁ , x ₄ , x ₈ , x ₅ , x ₁ , x ₇ are shown in descending order of the absolute value of the weight of the linear model. By configuring in this way, the neural network evaluation apparatus 100A can allow the user to grasp how much the output value increases or decreases or how much the amount of change when each input variable changes for each cluster. For this reason, the user can deepen understanding of the model.

  In FIG. 11, the classification rule generated by the classification rule generation unit 138 is also displayed. Here, as an example, the case where the classification rule is a decision tree is shown. With this configuration, the user can know the correspondence between each cluster and the condition of the value of the input variable by looking at the classification rule decision tree. This deepens understanding of neural networks. Further, when the user selects a variable name, the relationship may be shown by displaying a scatter diagram of the selected variable and output value for the sample in the cluster being displayed.

(Operation of the neural network evaluation device)
FIG. 12 is a flowchart illustrating an example of the operation of the neural network evaluation apparatus according to the second embodiment. For step S201, step S101 described with reference to FIG. 7 can be applied.

  (Step S202) The grouping unit 134A of the neural network evaluation apparatus 100A acquires the variation matrix output by the variation matrix calculation unit 132. The grouping unit 134A refers to the model data included in the setting data 114, the sample set, and the variation matrix output from the variation matrix 132, and calculates the intercept of the surface in contact with the neural network model in each sample. The grouping unit 134A performs clustering based on the variation matrix and the intercept. The grouping unit 134A outputs the clustering result to the summary information generation unit 136A.

  (Step S203) The summary information generation unit 136A of the neural network evaluation apparatus 100A acquires the variation matrix output by the variation matrix calculation unit 132, the clustering result output by the grouping unit 134A, and the intercept information for each sample. . The summary information generation unit 136A generates a linear model that approximates the relationship between the input data and the output value for each cluster based on the acquired clustering result. The summary information generation unit 136A outputs information indicating the generated linear model to the classification rule generation unit 138.

  (Step S204) The classification rule generation unit 138 of the neural network evaluation apparatus 100A acquires the clustering result output by the grouping unit 134A and the sample set (sample matrix) included in the setting data 114. The classification rule generation unit 138 generates a classification rule by learning an arbitrary identification model using each row of the sample matrix as an input variable and the clustering result corresponding to each row as teacher data.

  (Step S205) The classification rule generation unit 138 of the neural network evaluation apparatus 100A outputs information indicating the generated classification rule to the communication unit 102. The communication unit 102 transmits information indicating the classification rule output from the classification rule generation unit 138 to the terminal device 200. The terminal device 200 receives the information indicating the classification rule transmitted by the neural network evaluation device 100A, and presents the information indicating the received classification rule to the user.

  FIG. 13 is a flowchart illustrating an example of the operation of the neural network evaluation apparatus according to the second embodiment. In the example shown in FIG. 13, the operation after the classification rule generation unit 138 of the neural network evaluation apparatus 100A generates a classification rule is shown.

  (Step S301) The prediction unit 140 of the neural network evaluation apparatus 100A acquires the classification rule output by the classification rule generation unit 138 and the linear model of each cluster generated by the summary information generation unit 136A. In addition, the prediction unit 140 controls the communication unit 102 to acquire an unknown sample, refers to a classification rule, and determines a cluster to which the acquired unknown sample belongs. The prediction unit 140 predicts an output value using the determined linear model of the cluster.

  (Step S302) The prediction unit 140 of the neural network evaluation apparatus 100A outputs the prediction value to the communication unit 102. The communication unit 102 transmits the predicted value output by the prediction unit 140 to the terminal device 200. The terminal device 200 receives the predicted value transmitted by the neural network evaluation device 100A, and presents the received predicted value to the user.

  In the above-described embodiment, the grouping unit 134A can apply an arbitrary algorithm to clustering.

  In the above-described embodiment, the case where the sample set, the variation information, the clustering result, and the intercept are represented by a matrix has been described. However, the present invention is not limited to this example. For example, the sample set, variation information, clustering result, and intercept may be represented by vectors.

  In the above-described embodiment, the case where the grouping unit 134A performs clustering based on the variation matrix and the intercept has been described, but the present invention is not limited thereto. For example, the grouping unit 134A may perform clustering based on the sample matrix in addition to the variation matrix and the intercept. The grouping unit 134A may perform clustering using the clustering method described in the first embodiment, or may perform clustering by combining the clustering methods described above.

  In the above-described embodiment, the case has been described in which the summary information generation unit 136A obtains linear models for all clusters and outputs the obtained linear models for each cluster as model summary information. However, the present invention is not limited to this example. For example, the summary information generation unit 136A may obtain an arbitrary model other than the linear model for all clusters or a part of the clusters.

  In the above-described embodiment, when obtaining a linear model for a cluster, the summary information generating unit 136A may obtain a linear model for each cluster by performing learning again. In addition, the summary information generation unit 136A may redistribute the cluster to which each sample belongs based on the linear model obtained for each cluster. For example, the summary information generation unit 136A may redistribute each sample into a cluster including a linear model that outputs a value closest to the output value of the neural network. The neural network evaluation apparatus 100A may update the clustering result by alternately performing cluster redistribution and linear model calculation a plurality of times.

  In the above-described embodiment, the case where the neural network evaluation apparatus 100A acquires the sample set included in the setting data 114 of the storage unit 110 and transmits the model summary information to the terminal apparatus 200 has been described. I can't. For example, the neural network evaluation apparatus 100A includes an input device such as a keyboard and a mouse and a display device such as a display, acquires a sample input by the user operating the input device, and stores model summary information on the display device. You may make it output. The same applies to an unknown sample given to the prediction unit 140.

  According to at least one embodiment described above, unknown samples can be classified into clusters, and output values can be predicted using a simple model of the corresponding cluster. By configuring in this way, the processing time of the prediction process for unknown data is shortened compared to using the original neural network. In addition, by making the classification rule a model that is easy for the user to understand, such as a decision tree, the user can understand how the generated cluster is formed. In the above-described embodiment, the fluctuation matrix calculation unit is an example of a fluctuation derivation unit.

  As mentioned above, although embodiment of this invention and its modification were demonstrated, these embodiment and its modification are shown as an example, and are not intending limiting the range of invention. These embodiments and modifications thereof can be implemented in various other forms, and various omissions, replacements, changes, and combinations can be made without departing from the spirit of the invention. These embodiments and modifications thereof are included in the scope and gist of the invention, and are also included in the invention described in the claims and the equivalents thereof.

  DESCRIPTION OF SYMBOLS 1, 1A ... Neural network evaluation system, 50 ... Network, 100, 100A ... Neural network evaluation apparatus, 102 ... Communication part, 110 ... Memory | storage part, 112 ... Program, 114 ... Setting data, 130, 130A ... Information processing part, 132 ... fluctuation matrix calculation unit, 134, 134A ... grouping unit, 136, 136A ... summary information generation unit, 138 ... classification rule generation unit, 140 ... prediction unit, 200 ... terminal device

Claims (12)

A fluctuation derivation unit that obtains fluctuations of the output value when the input value included in the sample of the input data of the neural network is fluctuated with respect to the output value calculated based on the value of the output node included in the neural network. ,
A neural network evaluation apparatus comprising: a grouping unit that classifies the samples into groups based on fluctuations in the output value.   The fluctuation derivation unit obtains a fluctuation matrix including a matrix obtained by arranging fluctuations of the output value when the input value input to the neural network is fluctuated as information on the fluctuation of the output value. The neural network evaluation apparatus according to claim 1.   The neural network according to claim 1 or 2, wherein the variation is a vector, and is a gradient vector obtained by partial differentiation of the output value with the input value, or a gradient vector calculated by a difference method. Evaluation device.   The neural network evaluation apparatus according to claim 2, wherein the grouping unit extracts some elements of the variation matrix and classifies the samples into groups based on the extracted some elements.   The neural network evaluation according to claim 2, wherein the grouping unit obtains an intercept of a surface in contact with the neural network at a point corresponding to the sample, and further classifies the samples into groups based on the obtained intercept. apparatus.   6. The neural network evaluation device according to claim 1, further comprising: a summary information generation unit that generates information for summarizing the neural network for each of the groups classified by the grouping unit. .   The neural network evaluation apparatus according to claim 6, wherein the information to be summarized is information relating to an influence of the input value on the output value, or a model that approximates a relationship between the input value and the output value.   The summary information generation unit extracts an element corresponding to the input value from a variation matrix, and for the extracted element or an element obtained by taking an absolute value for the extracted element, a sum, an average value, a center The neural network evaluation apparatus according to claim 6, wherein an influence of the input value on the output value is obtained by calculating one of a value, a maximum value, a minimum value, and a mode value.   9. The classification rule generation unit according to claim 1, further comprising a classification rule generation unit configured to generate a rule for classifying unknown samples into groups based on the group classified by the grouping unit. Neural network evaluation device. The prediction unit according to claim 9, further comprising: a prediction unit that classifies the unknown samples into groups according to the rules generated by the classification rule generation unit, and predicts an output value based on the group into which the unknown samples are classified. The neural network evaluation apparatus described. Computer
Regarding the output value calculated based on the value of the output node included in the neural network, the variation of the output value when the input value included in the sample of the input data of the neural network is varied,
Classifying the samples into groups based on variations in the output values;
Outputting the result of classifying the sample;
Neural network evaluation method. On the computer,
Regarding the output value calculated based on the value of the output node included in the neural network, the variation of the output value when the input value included in the sample of the input data of the neural network is varied,
Classifying the samples into groups based on variations in the output values;
Outputting the result of classifying the sample;
program.

Images