JP6948851B2 - Information processing device, information processing method - Google Patents

Description

The present invention relates to an information processing technique using a hierarchical neural network.

In recent years, there are services that analyze the activity patterns of people and crowds from images and videos acquired by surveillance cameras, and detect and report specific events. In order to realize the service, the attributes of the object such as whether it is a person or a car, the type of action such as walking or running, and whether it is a bag or a basket are determined from the moving image taken by the surveillance camera. It is indispensable to have a recognition technology using machine learning that can recognize the type of personal belongings such as a camera. Deep Neural Network (hereinafter abbreviated as DNN) is attracting attention as a machine learning method that realizes highly accurate recognition. The services described above are used in various environments such as nursing care facilities, general households, public facilities such as stations and urban areas, and stores such as supermarkets and convenience stores. On the other hand, the learning data for learning the DNN is often acquired in an environment different from the environment in which the service is actually used. For example, learning data may be obtained from a developer's performance in the laboratory. The recognizer learned using such learning data depends on the feature amount peculiar to the learning data, and there is a problem that the performance is not sufficiently exhibited in the environment where the surveillance camera is actually installed. Therefore, there is an increasing demand for specifying the feature amount used for recognition by the learned DNN.

In Non-Patent Document 1, among the learned feature maps of the specific layer of DNN, those having high activity with respect to the input image data for evaluation are selected, and the feature map is inversely converted between the polling layer and the convolution layer. Is visualized by sequentially applying and returning to the input layer.

Further, in Non-Patent Document 2, the image data for evaluation is divided, and the partial image from which each region is removed is input to the learned DNN. Then, the area on the image that contributes to the recognition is selected based on the change in the recognition accuracy of the DNN when each partial image is input to the DNN.

Further, Non-Patent Document 3 proposes a method called Dropout, which learns DNN while adding zero or noise to the value of a randomly selected neuron. By this method, the number of activated DNN neurons can be suppressed so as to avoid excessive adaptation to the training data while improving the recognition accuracy.

Visualizing and Understanding Convolutional Networks, M.D. Ziler and R. Fergus, European Conference on Computer Vision (ECCV), 2014 Object Detectors Emerge in Deep Scene CNNs, B. Zhou, A. Khosla, A. Lapedriza, A. Oliva and A. Torralba, International Conference on Learning Representations (ICLR), 2015 Dropout: A Simple Way to Prevent Neural Networks from Overfitting, N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov, Journal of Machine Learning Research 15 (2014) 1929-1958.

However, the method described in Non-Patent Document 1 does not visualize the feature map that contributed to the recognition of the image data for evaluation. Specifically, the information of the feature map with high activity visualized in Non-Patent Document 1 disappears due to a small weighting coefficient and cancellation with other feature maps in the process of being propagated to the output layer of DNN. May be done. In that case, the feature map with high activity does not contribute to recognition. On the contrary, in the process of propagating the information of the feature map having low activity to the output layer, it may be strengthened by a large weighting coefficient or a synergistic effect with other feature maps. In that case, the feature map with low activity contributes to recognition. Therefore, with the method described in Non-Patent Document 1, the user cannot grasp to what extent the visualized feature map is utilized for recognition. Further, by the method described in Non-Patent Document 1, the user cannot grasp whether or not there is a feature map other than the visualized feature map that contributes to recognition.

On the other hand, in Non-Patent Document 2, it is possible to visualize the area on the image data that contributes to the recognition accuracy. As a result, the user can grasp which area on the image data contributes to the recognition and how much. However, since the visualization method of Non-Patent Document 2 does not visualize the feature map, it is not clear which feature on the selected image data area is actually used by DNN for recognition. For example, when a plurality of objects exist in the same area, it is not known which object's information contributes to recognition. Also, when a human face is selected, it is not clear whether it is the facial expression, color, size, shape, or parts such as hair, eyes, and mouth that contribute to recognition. Further, the method described in Non-Patent Document 2 has a problem that it takes time to calculate because it is necessary to obtain the output value of DNN for each partial image created by removing the region.

On the other hand, in the method described in Non-Patent Document 3, DNN can be learned so that a limited number of neurons contribute to recognition. However, in the method described in Non-Patent Document 3, neurons that contribute to recognition are not explicitly selected. Therefore, in order to understand the contributing neurons, it is necessary for experts to analyze the activation status of neurons with respect to various evaluation data. In other words, a separate method for identifying neurons that contribute to recognition is required.

Further, in the method described in Non-Patent Document 3, neurons that contribute to recognition are acquired based on learning data, but the neurons are not always useful in actual recognition. As described above, the learning data acquired in a specific environment may include a bias peculiar to the environment, and the neurons that contribute to the recognition acquired using the learning data are not necessary for the original recognition. There is a possibility that the feature amount is expressed incorrectly. For example, in the learning data of motion recognition of "walking" and "running", "desk" is always reflected in the "walking" data, and no "desk" is reflected in the learning data of "running". Suppose there is a bias. In that case, in the method described in Non-Patent Document 3, a neuron corresponding to the feature amount of the "desk" is acquired as a neuron that contributes to recognition. However, in a general environment in which the actually learned DNN is used, there is no such bias, so that the neuron is not useful, but rather may adversely affect cognition. For example, the DNN may mistakenly recognize "walking" when a "desk" is shown in the image of the "running" motion.

As described above, in the method described in Non-Patent Document 3, when the learning data is biased, the neurons contributing to recognition have a problem of expressing an erroneous feature amount, and the user can easily confirm the problem. There is a problem that it cannot be done.

The present invention has been made in view of such a problem, and provides a technique for identifying a DNN feature map or a neuron that contributes to the recognition of evaluation data.

The uniformity of the present invention includes a first neural network that outputs the probability that an object belonging to the category is included in the input data as an output value for each of the plurality of categories.
A second neural network in which a specific unit is modified in the first neural network , and the probability that the input data includes an object belonging to the category is output as an output value for each of the plurality of categories. With the second neural network
For each of the plurality of categories, the difference information of the first neural network representing the difference between the output value the output value output to the category second neural network is output to the category And the calculation method to find
It is characterized by including an output means for outputting to a display device information indicating the contribution of a specific unit to the output value with respect to the input data based on the difference information obtained by the calculation means.

According to the configuration of the present invention, it is possible to identify a DNN feature map or a neuron that contributes to the recognition of evaluation data.

The figure which shows the configuration example of the recognition learning system 1. The figure which shows an example of the information which a storage part M1 stores. The figure which shows an example of the network structure of DNN. The figure which shows an example of the information which a storage part M2 stores. The figure explaining how to obtain the change information. The figure which shows the display example of GUI. The figure which shows the display example of GUI. The flowchart of the operation of the recognition learning system 1. The figure which shows the configuration example of the recognition learning system 1a. The figure which shows the display example of GUI. The figure which shows an example of the dropout ratio. The flowchart of the operation of the recognition learning system 1a. The figure which shows the configuration example of the recognition learning system 1b. The figure which shows the hardware configuration example of a computer apparatus. The figure explaining how to obtain the change information. The figure explaining how to obtain the change information.

Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. In addition, the embodiment described below shows an example when the present invention is concretely implemented, and is one of the specific examples of the configuration described in the claims.

[First Embodiment]
In this embodiment, an example of an information processing apparatus having the following configuration will be described. The information processing device obtains the output value of the first neural network corresponding to each category for the input data (first calculation). Here, the output value of the second neural network in which the unit specified in the first neural network corresponding to each category for the input data is changed is obtained (second calculation). Then, for each category, change information representing the change between the output value obtained in the first calculation and the output value obtained in the second calculation is obtained (third calculation), and is obtained in the third calculation. Based on the change information, the information indicating the contribution of the specified unit is output to the display device.

In the present embodiment, a case where such an information processing device is applied to the recognition learning device 10 in the recognition learning system 1 as shown in FIG. 1 will be described. As shown in FIG. 1, the recognition learning system 1 includes a recognition learning device 10 and a terminal device 100, and the recognition learning device 10 and the terminal device 100 communicate data with each other via a wireless or wired network. Is configured to be possible. For example, a fixed telephone line network, a mobile phone line network, or the Internet can be applied to this network. Although the recognition learning device 10 and the terminal device 100 are shown as separate devices in FIG. 1, the recognition learning device 10 and the terminal device 100 may be integrated into one device.

In the present embodiment, the user of the recognition learning system 1 confirms whether or not unnecessary feature quantities in the learned DNN are used for recognition of the learning image or video (hereinafter referred to as learning data). The case to do is explained. Specifically, the recognition learning system 1 identifies the feature amount of DNN that contributed to the recognition of the image or video (hereinafter, evaluation data) used for evaluation, and the information indicating the feature amount is used as the evaluation data. Display in layers. Here, the unnecessary feature amount is a feature amount that is unexpectedly reflected when the learning data is acquired and depends on an object or an event peculiar to the learning data acquisition environment. For example, if the learning data is data obtained by shooting a performance in a laboratory, the experimental equipment unique to the laboratory and the habits, clothes, postures, etc. unique to the performer are the learning data. Corresponds to objects and events specific to the acquisition environment. Here, the user is, for example, a research and developer who develops this system, or a system integrator who adjusts DNN in order to provide this system to an end user together with a surveillance camera. Further, the recognition target of DNN is a state of an object that can be conceptualized and verbalized, and is characterized by label information that linguistically indicates the state. The recognition targets include, for example, the attributes of objects such as "people" and "cars", the behavior of objects such as "walking" and "running", and the behavior of people such as "bags" and "baskets". Includes personal belongings. Here, the DNN includes a Convolution Neural Network (hereinafter abbreviated as CNN) proposed in the following documents.
-ImageNet Classification with Deep Convolutional Neural Networks, A. Krizhevsky, I. Sutskever and GE Hinton, Advances in Neural Information Processing Systems 25 (NIPS 2012)

First, the terminal device 100 will be described. The terminal device 100 is a device having a display unit DS for displaying various information and an operation detection unit OP for detecting a user's operation performed on the display unit DS. For example, a PC (Personal Computer), a tablet PC, a smartphone, a future phone, or the like can be applied to the terminal device 100.

The display unit DS includes an image display panel such as a liquid crystal panel or an organic EL panel, and displays various information received from the recognition learning device 10. Although the details will be described later, the display unit DS provides evaluation data, unit visualization information for visualizing the feature amount generated by the visualization unit 15 described later, and the degree of contribution in recognizing the feature amount generated by the detection unit 13. The change information shown is displayed. In addition, the display unit DS displays a list of feature maps and neuron-identifying unit IDs constituting the DNN stored in the recognition learning device 10 described later, and category IDs for identifying categories to be recognized.

The operation detection unit OP includes a touch sensor arranged on the image display panel of the display unit DS, detects the user's operation based on the movement of the user's finger or the touch pen, and operates information indicating the detected operation. Is transmitted to the recognition learning device 10. The operation detection unit OP may include an input device such as a controller, a keyboard, and a mouse, and may acquire operation information indicating the user's operation on the image displayed on the image display panel. This operation information includes, for example, an instruction for selecting evaluation data, an instruction for executing visualization, an instruction for selecting a unit ID and a category ID, and the like. When the operation detection unit OP detects "execution of visualization" as the operation information, the operation detection unit OP transmits the evaluation data stored in the terminal device 100 to the recognition learning device 10. When the operation detection unit OP detects the selection of the unit ID and the category ID as the operation information, the operation detection unit OP receives the unit visualization information and the change information corresponding to the unit ID and the category ID from the recognition learning device 10 and uses them as evaluation data. It is superimposed and displayed on the display unit DS.

Next, the recognition learning device 10 will be described. The storage unit M1 stores the following information in association with the category ID that identifies the category to be recognized. That is, a layer ID that identifies each layer of the DNN, a layer name information that indicates the name of the layer of the layer ID, a lower layer ID that identifies the layer immediately below the layer, and an upper layer ID that identifies the layer immediately above the layer. , Processing parameter information indicating the processing method and processing parameter in the layer is stored. FIG. 2 shows an example of the information stored in the storage unit M1.

In FIG. 2, the category ID and the hierarchical ID to be recognized are represented as character strings composed of alphabets and numbers, but the expression method of the category ID and the hierarchical ID is not limited to a specific expression method. In the case of FIG. 2, there are two categories to be recognized, and the two categories are identified by the category ID “C01” and the category ID “C02”.

In FIG. 2, in association with the layer ID “L01”, the layer name “input layer”, the lower layer ID “Null” (indicating that there is no layer lower than the layer of the layer ID “L01”), and the upper layer ID. "L02" and the processing parameter "processing method: data input" are stored. This is because the layer having the layer ID "L01" is the "input layer", there is no layer lower than the input layer, and the layer ID one layer above the input layer is "L02". Indicates that the processing method performed in the input layer is data input. That is, the input layer is a layer for inputting data such as an image or a video into the DNN and transferring the data to the layer having the layer ID "L02".

Further, in FIG. 2, the layer name “Convolution 1 layer”, the lower layer ID “L01”, the upper layer ID “L03”, and the processing parameter “processing method: Convolution ...” are stored in association with the layer ID “L02”. This is because the layer name of the layer having the layer ID "L02" is the Convolution 1 layer, the layer one level below the Convolution 1 layer is the "input layer", and the layer ID of the layer one layer above the Convolution 1 layer is "". It indicates that it is "L03". Further, the processing method performed in the Convolution 1 layer indicates that the data input from the input layer is subjected to a convolution operation using a weighting coefficient and a bias term as processing parameters. That is, the Convolution 1 layer performs a convolution operation using a weighting coefficient and a bias term on the data input from the input layer, and the result of the convolution operation is applied to the layer (Polling 1 layer) whose layer ID is "L03". It is a hierarchy to output. In addition to data input and Convolution, the processing methods held by this processing parameter include those described in the following documents. That is, there are Pooling for calculating the maximum value for each filter, InnerProduct for calculating the inner product of the input data and the weighting coefficient, and softmax for calculating the probability that the evaluation data belongs to the category.
・ J. Yangging et al., Caffe: COnvolutional Architecture for Fast Feature Embedding, 2014

Further, this processing parameter includes the size, number and stride width of the filter used for processing each layer, the weighting coefficient used in the Convolution layer and the InnerProduct layer, and the value of the bias term.

FIG. 3 shows an example of the network structure of the DNN defined by the information stored in the storage unit M1. The DNN illustrated in FIG. 3 is composed of an input layer 301, a Convolution 1 layer 302, a Polling 1 layer 303, a Convolution 2 layer 304, a Pooling 2 layer 305, an Inner product layer 306, and an output layer 307. The process performed between the input layer 301 and the Convolution 1 layer 302 is the "Convolution process 311" defined in the process parameter information corresponding to the Convolution 1 layer 302. Further, the process performed between the Composition 1 layer 302 and the Pooling 1 layer 303 is the “Polling process 312” defined in the process parameter information corresponding to the Pooling 1 layer 303. Further, the process performed between the Polling 2 layer 305 and the Inner product layer 306 is the "Inner Product process 313" defined in the process parameter information corresponding to the Inner product layer 306. Further, the process performed between the Inner product layer 306 and the output layer 307 is the "softmax process 314" defined in the process parameter information corresponding to the output layer 307.

Further, in FIG. 3, a plurality of feature maps exist in the Convolution layer and the polling layer, and a plurality of neurons exist in the InnerProduct layer and the output layer. Units such as feature maps and neurons are then identified by the unit ID. For example, the two feature maps in the Convolution 1 layer 302 are identified by the unit ID "F02001" 321 and the unit ID "F02002" 322. In addition, the two neurons in the InnerProduct layer 306 are identified by the unit ID "F06001" 323 and the unit ID "F06002" 324. Further, in FIG. 3, the categories ID = C01 and C02 to be recognized are assigned to the two neurons in the output layer 307, respectively. That is, as will be described in detail later, the output value from the neuron of category ID = C01 is the output score information corresponding to the category ID = C01, and the output value from the neuron of category ID = C02 becomes category ID = C02. Corresponding output score information.

As described above, the information stored in the storage unit M1 defines the network structure of the DNN. Therefore, in the following, the information stored in the storage unit M1 may be referred to as the structural information of the DNN. ..

The storage unit M2 stores unit state information indicating the state of the unit which is the processing result of each layer of DNN for the evaluation data, and output score information indicating the output score of DNN for each category to be recognized. Specifically, the storage unit M2 stores the output score information of the DNN for each category in association with the category ID that identifies each category. Further, the storage unit M2 stores a unit ID that identifies a unit such as a feature map or a neuron in the hierarchy and unit state information indicating the state of the unit in association with the hierarchy ID that identifies the hierarchy of the DNN. .. FIG. 4 shows an example of the information stored in the storage unit M2.

In FIG. 4, the unit ID is represented as a character string composed of alphabets and numbers, but the expression method of the unit ID is not limited to a specific expression method. The unit ID is generated based on the hierarchy ID of the hierarchy to which the unit belongs and the order of the units in the hierarchy. For example, the unit ID of the first unit of the layer ID "L02" is "F02001". The unit ID of the second unit in the same layer is "F02002".

Further, in FIG. 4, "10.5" is stored as the output score information of the category ID "C01", and "3.8" is stored as the output score information of the category ID "C02". Further, the unit ID "F012001" and the matrix of the feature map are stored as the unit state in association with the layer ID "L01". Further, the unit ID "F06001" and the value of the neuron as the unit state are stored in association with the hierarchical ID "L06".

Returning to FIG. 1, the processing unit 11 calculates the output score information of each recognition target category of DNN for the evaluation data, and stores the unit state information of each unit obtained in the calculation process in the storage unit M2. .. Specifically, the processing unit 11 reads from the storage unit M1 the category ID of the category to be recognized by the DNN, the lower layer ID associated with each layer ID, the upper layer ID, and the processing parameter information. Then, the processing unit 11 constructs a DNN based on the read structural information, and provides processing parameter information corresponding to each layer in the order of the lowest layer to the highest layer with respect to the evaluation data received from the terminal device 100. Apply and process. Then, the processing unit 11 associates the output score information corresponding to the category ID read from the storage unit M1 among the outputs (output score information) from the uppermost layer of the DNN with the category ID to the storage unit M2. Store.

In the present embodiment, an image is used as the evaluation data, but the evaluation data is not limited to the image. For example, as proposed in the following documents, an image can be recognized.
・ Two-stream convlutional networks for action recognition in videos, K. Simonyan and A. Zisserman, Advances in Neural Information Processing System 25 (NIPS), 2014.
・ 3D Convlutional Neural Networks for Human Action Recognition, S. Ji, W. Xu, M. Yang and K. Yu, Pattern Analysis and Machine Intelligence, vol. 35, no. 1, pp. 221-231, 2012

The processing unit 11 associates the unit status information of each unit in the process from inputting the evaluation data to the input layer to obtaining the output of the highest layer with the layer ID of the layer to which the unit belongs and the unit ID of the unit. And store it in the storage unit M2. Then, the processing unit 11 outputs a trigger to the processing unit 12.

The processing unit 12 reads the category ID to be recognized and the lower layer ID, upper layer ID, and processing parameter information associated with the layer ID from the storage unit M1 in response to the input of the trigger from the processing unit 11. .. Further, the processing unit 12 reads the output score information associated with the category ID and the unit state information associated with the hierarchical ID and the unit ID from the storage unit M2. Then, the processing unit 12 performs a predetermined process for the unit status information corresponding to the specific unit ID among the read unit IDs. Here, the specific unit ID is a unit ID designated (set) in advance by the user as a unit ID for identifying the unit to be visualized (the unit to be visualized). For example, when the user wants to visualize the first feature map of the Convolution 1 layer, "F02001" is set as a specific unit ID. Further, when the user wants to visualize all the feature maps of the Convolution 1 layer, "F02 *" is set as a specific unit ID by using a wild card. In addition, various processes can be considered for the "specified process" performed for the unit status information corresponding to the specific unit ID. For example, the following two types of processes (first process and second process) are considered. Can be considered.

In the first process, the processing unit 12 sets another set having the same size as the set of numerical values represented by the unit state information corresponding to the specific unit ID among the unit state information read from the storage unit M2 and having all zero elements. Generated as additional unit information. For example, when the unit state information represents a matrix of the feature map, a matrix having the same size as the matrix and having all elements 0 is generated as additional unit information. When the unit state information is a neuron value, a neuron value having a value of 0 is generated as additional unit information. In the following, the unit state information corresponding to a specific unit ID is replaced with a unit (feature map or neuron) having all zero elements, so that the output from the unit becomes 0, and the unit is pseudo on the DNN. It will be in the deleted state.

In the second process, the processing unit 12 is a separate set having the same size as the set of numerical values represented by the unit state information corresponding to the specific unit ID among the unit state information read from the storage unit M2, and all the elements are random values. Is generated as additional information. Random values follow, for example, a normal distribution or a Laplace distribution independently and identically. For example, when the unit state information represents a matrix of the feature map, a matrix having the same size as the matrix and all elements having random values is generated as additional information. When the unit state information is a neuron value, a neuron value having a random value is generated as additional information. Then, the processing unit 12 generates the additional unit information by adding the additional information to the unit state information corresponding to the specific unit ID (addition for each corresponding element).

Then, the processing unit 12 outputs a specific unit ID (unit ID of the unit status information subject to the specified processing) and additional unit information to the detection unit 13.

The detection unit 13 reads the category ID to be recognized and the lower layer ID, upper layer ID, and processing parameter information associated with the layer ID from the storage unit M1. Further, the detection unit 13 reads the output score information associated with the category ID and the unit state information associated with the hierarchical ID and the unit ID from the storage unit M2. Then, the detection unit 13 calculates the output score information of each recognition target category of DNN with respect to the evaluation data in the same manner as the processing unit 11, but at that time, it is added as the unit state information corresponding to the specific unit ID. Use unit information. Further, the detection unit 13 does not need to recalculate the unit state information associated with the layer ID corresponding to the layer lower than the layer ID corresponding to the specific unit ID, and is stored in the storage unit M2. The unit status information may be used. For example, when the specified processing is performed on the unit of the Convolution 2 layer, the unit state information of the Convolution 1 layer and the Pooling 1 layer is reused for the calculation of the output score information.

In this way, the detection unit 13 calculates the output score information of each recognition target category for the DNN evaluation data when the unit corresponding to the specific unit ID is replaced with the additional unit information. The detection unit 13 indicates a change in the calculated output score information with respect to the output score information stored in the storage unit M2 (change in the output score information due to replacement of the unit corresponding to the specific unit ID with the additional unit information). Change information) is calculated for each category. Further, various calculation processes can be considered for the calculation process of change information. For example, the following two calculation processes (first calculation process and second calculation process) can be considered.

In the first calculation process, the detection unit 13 outputs the output score information for the DNN evaluation data when the unit status information corresponding to the specific unit ID is replaced with the additional unit information, and the output stored in the storage unit M2. The difference between the score information and the score information is obtained as change information. In the first calculation process, for example, change information is obtained according to the following equation (1). As in the equation (1), this difference may take an absolute value so as not to take a negative value.

In the equation (1), ΔS _{c and u} are change information obtained for the category c when the unit state information of the unit ID = u is replaced with the additional unit information. Sc is the output score information of the category c read from the storage unit M2, and Sc _{and u} are the output score information output from the DNN for the category c when the unit state information of the unit ID = u is replaced with the additional unit information. Is.

In the second process, the detection unit 13 was used to generate the output score information for the DNN evaluation data when the unit status information corresponding to the specific unit ID was replaced with the additional unit information, and the additional unit information. The correlation coefficient with the additional information is obtained as change information. In this case, the processing unit 12 needs to further output additional information to the detection unit 13. Specifically, the processing unit 12 and the detection unit 13 perform the following processing for each visualization target unit (or a part thereof). That is, the processing unit 12 adds additional information to the unit status information of the visualization target unit to generate additional unit information, and the detection unit 13 uses the additional unit information instead of the unit status information of the visualization target unit. Calculate the output score information of. Then, the detection unit 13 calculates the correlation coefficient by calculating the following equation (2) using the set of each calculated output score information and the additional information used for calculating the output score information. Calculate as change information.

In equation (2), N indicates the number of repetitions (number of pairs). _{Sc, u, and i} indicate the output score information of the category c when the unit state information of the unit ID = u is replaced with the additional unit information generated by the i-th specified process. a _i is the i-th additional information.

A method of obtaining change information when the processing unit 12 performs the first processing described above will be described with reference to FIG. In FIG. 5, the units included in the Convolution 1 layer 501 and the Convolution 2 layer 502 of the DNN (units 511 and 512, respectively) are set as the visualization target units. In such a case, the processing unit 12 generates a unit (additional unit information) 531 having the same size as the unit 511 and having all elements 0, and a unit having the same size as the unit 512 and having all elements 0 (additional unit information) 531. Additional unit information) 532 is generated.

Then, in FIG. 5, the detection unit 13 obtains the DNN output score information when the unit 511 is replaced with the unit 531 (DNN output score information when the unit 512 is replaced with the unit 532) as 8.5. .. Further, in FIG. 5, the output score information of the DNN before replacement is set to 10.5. However, the change information is 2. The calculation process of such change information is performed for each category.

In this way, the processing unit 12 and the detection unit 13 can calculate the change information for each category for each visualization target unit. That is, the following series of processing is performed for each visualization target unit. That is, the output score information for each category of the DNN in which the unit state information of the visualization target unit is replaced with the corresponding additional unit information is calculated, and the change information with respect to the output score information for each category of the DNN before the replacement is obtained.

Then, the detection unit 13 outputs a set of a specific unit ID, change information, and unit state information to the selection unit 14 for each category ID. That is, the detection unit 13 outputs to the selection unit 14 a set _{of change information ΔS c and u} for each category c calculated by the equation (1) or the equation (2).

The selection unit 14 selects the unit ID of the unit having a high degree of contribution to recognition for each of the input category IDs based on the input change information. As a method of selecting the unit ID, the selection unit 14 selects a unit ID having a large value of change information as a unit ID of a unit having a high degree of contribution for each category ID. Specifically, for example, the selection unit 14 selects all unit IDs having change information equal to or greater than the threshold value for each category ID. Further, the selection unit 14 selects a unit ID having a specified number of change information from the beginning in descending order of the value of the change information for each category ID. Then, the selection unit 14 outputs a set of the selected unit ID and the change information to the visualization unit 15 for each category ID. The selection unit 14 may select units that contribute to recognition for all categories, not for each category. For example, the selection unit 14 obtains a statistical value such as an average value, a total or a maximum value of change information of all categories of a specific unit ID, and selects a unit having a large statistical value.

The threshold value to be compared with the change information and the number of unit IDs to be selected can be set by, for example, a person adjusting the numerical value displayed on the display unit DS of the terminal device 100. Further, the operation detection unit OP detects an operation indicating a change in the numerical value by a person, and outputs the numerical value and the operation information to the recognition learning device 10. The recognition learning device 10 stores the numerical value as a threshold value or the number of unit IDs to be selected in a memory (not shown) in the recognition learning device 10 in response to input of the numerical value and operation information from the terminal device 100. ..

The visualization unit 15 generates information for visualizing the unit corresponding to the unit ID received from the selection unit 14 as unit visualization information. Specifically, the visualization unit 15 reads the lower layer ID, the upper layer ID, and the processing parameter information associated with each layer ID from the storage unit M1. Then, the visualization unit 15 generates unit visualization information based on the lower layer ID, the upper layer ID, and the processing parameter information read from the storage unit M1. For example, as described in Non-Patent Document 1, it is possible to use a method of returning the unit state information to the input layer by sequentially performing the inverse transformation of the lower polling layer and the convolution layer. Thereby, the target (feature) corresponding to the visualization target unit can be specified on the image as the evaluation data. The information indicating the area on the image of the specified object (feature) and the object to be arranged in the area is the unit visualization information.

Then, the visualization unit 15 transmits the unit ID and change information received from the selection unit 14, the layer ID corresponding to the unit ID, the category ID, and the unit visualization information to the terminal device 100.

The GUI (graphical user interface) illustrated in FIG. 6 is displayed on the display unit DS of the terminal device 100. In this GUI, DS1 is an image as evaluation data held by the terminal device 100. The DS2 is a display area for displaying a list of unit IDs received from the visualization unit 15 and a hierarchical ID. The DS3 is a display area for displaying a list of category IDs received from the visualization unit 15. As illustrated in FIG. 7 in such a GUI, it is assumed that the operation detection unit OP detects that one of the unit IDs listed in the DS2 is specified by the operation US1 by the user. Further, it is assumed that the operation detection unit OP detects that one of the category IDs listed on the DS3 is specified by the operation US2 by the user. Then, as shown in FIG. 7, the change information corresponding to the designated unit ID and category ID among the change information received from the recognition learning device 10 is displayed as the contribution degree DS 102 on the display unit DS of the terminal device 100. Further, on the display unit DS, the object DS101 indicated by the unit visualization information is displayed in the area (head area) indicated by the unit visualization information corresponding to the designated unit ID. Both the contribution degree DS102 and the object DS101 are superimposed and displayed on the image as evaluation data. However, the GUI layout is not limited to the layout shown in FIG. However, when displaying the contribution degree DS102 and the object DS101, it is not necessary to superimpose them on the image as evaluation data. As the information representing the contribution of the specified unit, the change information may not be displayed as it is as the contribution degree, but may be normalized to a value of an appropriate size, or may be represented by a level for each predetermined range. It may be expressed as a graph.

Next, the operation of the above-mentioned recognition learning system 1 will be described with reference to the flowchart of FIG. FIG. 8 is a flowchart showing an example of visualization of a feature amount that contributes to recognition processing in DNN. Since the details of each process shown in FIG. 8 are as described above, they will be briefly described below.

First, the display unit DS of the terminal device 100 displays a list of evaluation data (V101). The list of evaluation data may be, for example, a list of thumbnails of images or a list of previews of images. Here, when the operation detection unit OP detects that the user has selected one from the list of evaluation data and has input the instruction of "execution of visualization", the terminal device 100 is selected from the list. The evaluation data is transmitted to the recognition learning device 10 (V102). The processing unit 12 of the recognition learning device 10 receives the evaluation data transmitted from the terminal device 100 (V102).

Next, the processing unit 11 of the recognition learning device 10 reads the category ID of the category to be recognized by the DNN, the lower layer ID, the upper layer ID, and the processing parameter information associated with each layer ID from the storage unit M1 ( V103).

Next, the processing unit 11 applies the processing parameter information corresponding to each layer in the order from the lowest layer to the highest layer to the evaluation data received from the terminal device 100 based on the read structural information. Obtain output score information for each category (V104).

Then, the processing unit 11 stores the output score information corresponding to the category ID read from the storage unit M1 among the outputs (output score information) from the uppermost layer of the DNN in the storage unit M2 in association with the category ID. (V105). Further, the processing unit 11 stores the unit status information of each unit in the storage unit M2 in association with the layer ID of the hierarchy to which the unit belongs and the unit ID of the unit (V105). Then, the processing unit 11 outputs a trigger to the processing unit 12.

Next, the processing unit 12 initializes the value of the counter variable i for counting the number of units to be visualized to 0 (V106). Further, the processing unit 12 reads the category ID to be recognized, the lower layer ID associated with the layer ID, the upper layer ID, and the processing parameter information from the storage unit M1 (V106). Further, the processing unit 12 reads the output score information associated with the category ID and the unit state information associated with the hierarchical ID and the unit ID from the storage unit M2 (V106).

Assuming that the number of specific unit IDs among the read unit IDs is N (N is an integer of 2 or more), the processing unit 12 performs a predetermined process for the unit status information corresponding to the i-th specific unit ID. Then, the additional unit information is generated (V107). Then, the processing unit 12 outputs the i-th specific unit ID and the additional unit information generated for the i-th specific unit ID to the detection unit 13 (V107).

The detection unit 13 calculates the output score information of each recognition target category of DNN for the evaluation data in the same manner as the processing unit 11, but at that time, instead of the unit status information corresponding to the i-th specific unit ID, Additional unit information is used (V108).

Then, the detection unit 13 obtains a change between the output score information calculated by V108 and the output score information stored in the storage unit M2 for each category (V109). Then, the detection unit 13 increments the value of the counter variable i by one (V110). If the value of the counter variable i after increment is N or more, the process proceeds to V112 via V111, and if it is less than N, the process returns to V107 via V111.

The selection unit 14 selects the unit ID of the unit having a high contribution to recognition for each category ID, and outputs the set of the selected unit ID and the change information to the visualization unit 15 for each category ID (for each category ID). V112).

The visualization unit 15 generates information for visualizing the unit corresponding to the unit ID received from the selection unit 14 as unit visualization information (V113). Then, the visualization unit 15 transmits the unit ID and change information received from the selection unit 14, the layer ID corresponding to the unit ID, the category ID, and the unit visualization information to the terminal device 100 (V113).

The display unit DS of the terminal device 100 displays an image as evaluation data held by the terminal device 100, a list of unit IDs and hierarchical IDs received from the visualization unit 15, and a list of category IDs received from the visualization unit 15. There is. In such a state, it is assumed that the user specifies the unit ID and the category ID on the GUI. Then, the display unit DS superimposes each of the contribution degree represented by the change information corresponding to the designated unit ID and the category ID and the object indicated by the unit visualization information corresponding to the designated unit ID on the evaluation data. Display (V114).

As described above, according to the present embodiment, it is possible to visualize the information of the unit such as the feature map or the neuron that contributes to the recognition of DNN with respect to the evaluation data. However, the user can confirm whether or not unnecessary features such as features peculiar to the learning data are used for recognition. Then, if it is found that the DNN uses an unnecessary feature amount for recognition, the user can delete the data including the feature amount from the learning data and relearn the DNN. As a result, the user can acquire a DNN that does not use unnecessary features.

Further, in the present embodiment, when detecting a change in the output score information, the already calculated state of each unit is reused. As a result, a unit that contributes to recognition can be obtained at high speed. In particular, the higher the level, the more units in the lower layer that can be reused, so it can be obtained at higher speed. Therefore, the user can confirm the feature amount that contributes to the recognition of DNN by using more evaluation data.

In this embodiment, a case where a unit that contributes to recognition is selected based on an independent change in the output score information for each unit has been described. However, these series of processes may be performed in consideration of the co-occurrence of a plurality of units. For example, a combination of units that approximately maximizes the change in output score may be selected using the Forward Selection or Backward Selection described in the following documents.
・ Feature Selection for Reinforcement Learning: Evaluating Implicit State-Reward Dependency via Conditional Mutual Information, H. Hachiya & M. Sugiyama, ECML2010

[Second Embodiment]
In each of the following embodiments including this embodiment, the differences from the first embodiment will be mainly described, and unless otherwise specified below, the same as the first embodiment. A configuration example of the recognition learning system 1a according to the present embodiment will be described with reference to FIG. In the recognition learning system 1a according to the present embodiment, the user confirms whether the learned DNN is using unnecessary features for recognition, and if they are used, the importance of the features is set low. It has a configuration for re-learning the recognizer. That is, it differs from the first embodiment in that the recognition learning device 10a relearns the DNN based on the operation information indicating the feedback from the user with respect to the visualized feature amount.

The recognition learning system 1a according to the present embodiment has a recognition learning device 10a and a terminal device 100a, and between the recognition learning device 10a and the terminal device 100a, as in the first embodiment, a wired or wireless connection is used. It is configured to allow data communication with each other via a network.

The operation detection unit OP of the terminal device 100a detects the operation information for the display unit DS of the user as in the first embodiment. In the present embodiment, the operation detection unit OP further detects an instruction for setting importance information, which will be described later, and an instruction for executing DNN re-learning.

In the present embodiment, the display unit DS displays the GUI illustrated in FIG. 10 instead of the GUI shown in FIG. In the GUI of FIG. 10, "F04001" is selected as the unit ID from DS2, and "C02" is selected as the category ID from DS3. As a result, the object DS101 (with the samurai district indicated by the unit visualization information) is superimposed and displayed on the building (the area indicated by the unit visualization information) in the background. Further, in the GUI of FIG. 10, the pull-down menu DS401 of the importance for acquiring the feedback operation US3 from the user for the unit (feature amount) shown by the object DS101 and the relearning execution button DS402 are displayed in the display area DS4. It is displayed. The pull-down menu DS401 lists a plurality of importance (for example, a real value between 0 and 1, the larger the value, the higher the importance, and the smaller the value, the lower the importance) by instructing. Since it is displayed, the user can select and instruct one importance from the list. The execution button DS402 can instruct the recognition learning device 10a to relearn by instructing it.

The operation detection unit OP detects operation information indicating an operation on the pull-down menu DS401 or the execution button DS402 by the user. When the operation information is "input of importance using the pull-down menu DS401", the terminal device 100a converts the input importance information indicating the importance into the unit ID of the visualization target unit corresponding to the object DS101. Associate and memorize. On the other hand, when the operation information is the "instruction of the execution button DS402", the terminal device 100a relearns the stored importance information and the unit ID stored in association with the importance information. Is transmitted to the recognition learning device 10a together with the execution instruction of. If the user has not set the importance using the pull-down menu DS401, the importance information indicating the default importance will be transmitted. The importance of this default is not limited to a specific importance, but is 1, for example. Further, the value of the change information input from the recognition learning device 10a in association with the unit ID may be set as the default importance.

On the other hand, when the re-learning unit 16 of the recognition learning device 10a receives a re-learning execution instruction from the terminal device 100a, the re-learning unit 16 learns the DNN based on the importance information using the learning data. Specifically, in response to the input of the combination of the unit ID and the importance information from the terminal device 100a, the re-learning unit 16 associates the storage unit M1 with the category ID to be recognized by the DNN and each layer ID. The lower layer ID, upper layer ID, and processing parameter information are read. Then, the re-learning unit 16 minimizes the identification error of the DNN with respect to the learning data by using the learning method with importance based on the structural information of the DNN read from the storage unit M1 and the importance information received from the terminal device 100a. Update the processing parameter information so that Here, the processing parameter information to be updated is, for example, the weight coefficient and the value of the bias term of the Convolution processing and the InnerProduct processing. This learning data is data composed of a plurality of sets of input data such as images and videos and category IDs to which the input data belong, and is created in advance. Further, the learning method with importance includes, for example, the following two learning methods.

As the first learning method, the re-learning unit 16 sets the dropout ratio of each unit of the DNN structural information read from the storage unit M1 based on the unit ID and the importance information received from the terminal device 100a. .. As proposed in Non-Patent Document 3 above, dropout is a process of temporarily disconnecting randomly selected units from the network in each iteration of the learning process, and is a process related to the dropped out unit. The parameter information is not updated in the iteration.

The ratio of each unit to which this dropout is performed is usually set to a fixed value of 0.5 (above document) or the like, but in this first learning method, the ratio is set as the following equation (3). Is set based on the input importance information.

In equation (3), r is the dropout rate and I is the importance represented by the importance information. For example, if the importance I is 1, the dropout rate is set to the normal rate of 0.5. However, for units of importance I of 0.1, the dropout rate is set to a higher value than the normal rate, for example 0.95. As a result, the unit of low importance is dropped out frequently, so that the processing parameter information of the unit is difficult to be updated. Therefore, the contribution of the unit to recognition is relatively small.

FIG. 11 shows an example of the dropout ratio set for each unit. In FIG. 11, the dropout ratios of 0.5 and 0.95 are set for the feature map 1202 and the feature map 1203 of the Convolution 2 layer 1201, respectively. Also, in FIG. 11, the dropout rate of neuron 1205 in Inner product layer 1204 is set to 0.7.

As a second learning method, the re-learning unit 16 adds a penalty term based on the unit ID and importance information received from the terminal device 100a to the identification error that is minimized as in the following equation (4). ..

In equation (4), θ is a vector having the processing parameter information of each unit of DNN as an element, E (θ) is the DNN identification error with respect to the training data, and λ is a coefficient for balancing the error and the penalty term of importance. , U is a matrix having the reciprocal of the importance of each unit as a diagonal component. For example, when the importance of the i-th unit is 0.5, the element _{Uii of the} matrix U is 2. Here, the less important the unit, the stronger the penalty for the processing parameter information of the unit. Therefore, the DNN learned to minimize the equation (4) is learned not to use the less important unit. NS.

Although details are omitted, in the first and second learning methods, the processing parameter information of each layer is initialized first, and then the identification error is minimized. For this purpose, gradient methods such as Stochastic Gradient Descent (SGD) and Adadelta (J. Yangging et al., Caffe: COnvolutional Architecture for Fast Feature Embedding, 2014) are used.

Then, the re-learning unit 16 stores the updated processing parameter information in the storage unit M1 in association with the layer ID. As a result, the structural information of the DNN stored in the storage unit M1 is updated by re-learning.

Next, the operation of the recognition learning system 1a according to the present embodiment will be described with reference to the flowchart of FIG. In FIG. 12, the same processing steps as those shown in FIG. 8 are assigned the same step numbers, and the description of the processing steps will be omitted.

It is assumed that the user performs "input of importance using the pull-down menu DS401" after the processing of V114. At this time, the terminal device 100a stores the input importance information indicating the importance in association with the unit ID of the visualization target unit corresponding to the object DS101 (F101). On the other hand, when the user gives an "instruction of the execution button DS402", the terminal device 100a transmits the importance information and the unit ID stored in association with the importance information to the recognition learning device 10a. (F101).

Next, the re-learning unit 16 sets the dropout ratio of each unit of the DNN structural information read from the storage unit M1 based on the unit ID received from the terminal device 100a and the importance information (F102). Next, the re-learning unit 16 initializes the processing parameter information, and then updates the processing parameter information by using a gradient method such as SGD or AdaDelta so as to minimize the identification error (F103). Next, the re-learning unit 16 stores the processing parameter information updated in F103 in the storage unit M1 in association with the corresponding hierarchical ID (F104).

As described above, according to the present embodiment, in addition to the effect according to the first embodiment, if it is found that the DNN uses an unnecessary feature amount for recognition, the user applies the feature amount to the feature amount. You can relearn the DNN with a lower importance. As a result, the user can acquire the DNN without using unnecessary features by intuitive and simple operation.

[Third Embodiment]
A configuration example of the recognition learning system 1b according to the present embodiment will be described with reference to FIG. The recognition learning system 1b according to the present embodiment has a configuration in which feature maps and neurons having a low contribution in recognition of evaluation data prepared by the user are selected and deleted from the DNN. Here, the evaluation data is, for example, an image composed of a plurality of images or a plurality of clips of a specific domain. A domain is an environment in which this system is assumed to be used, for example, a nursing care facility, a general household, a station or city of a public facility, a store, or the like.

The recognition learning system 1b according to the present embodiment has a recognition learning device 10b and a terminal device 100, and the recognition learning device 10a and the terminal device 100 are connected between the recognition learning device 10a and the terminal device 100 by wire, wireless, or the like as in the first embodiment. It is configured to allow data communication with each other via a network.

The selection unit 14b selects the unit ID of the unit having a low contribution to recognition for each category ID input based on the change information input from the detection unit 13. As a method of selecting the unit ID, the selection unit 14b selects a unit ID having a small change information as a unit ID of a unit having a low contribution degree for each category ID. For example, the selection unit 14 obtains the average of the change information for various evaluation data for each unit ID for each category ID, and selects all the unit IDs having the change information whose average is less than the threshold value. Further, the selection unit 14b selects, as the selection unit ID, the unit ID corresponding to the average of the specified number from the beginning in ascending order of the average for each category ID. Then, the selection unit 14b outputs a set of the selection unit ID and the change information to the visualization unit 15 and the deletion unit 17 for each category ID.

The deletion unit 17 deletes the unit corresponding to the selected unit ID from the DNN. Specifically, in response to the input of the set of the selection unit ID and the change information from the selection unit 14b, the deletion unit 17 receives the category ID to be recognized by the DNN and each layer ID from the storage unit M1. Read the lower layer ID, upper layer ID, and processing parameter information associated with. Then, the deletion unit 17 updates the structural information of the DNN by an update method based on the selection unit ID input from the selection unit 14b. As an update method, for example, the unit is deleted by setting the weighting coefficient and the bias term of the unit of the selected unit ID to 0, which are included in the processing parameter information. Further, the deletion unit 17 reduces the number of filters held in the processing parameter information of the hierarchy to which the unit of the selected unit ID belongs according to the number of deleted units. Then, the deletion unit 17 stores the updated structural information in the storage unit M1.

The visualization unit 15 generates unit visualization information for visualizing the unit corresponding to the selected unit ID. Then, the terminal device 100 displays the object on the display unit DS based on the generated unit visualization information. As a result, the user can confirm the deleted unit by the recognition learning device 10b.

The deletion unit 17 may hold processing parameter information such as the weighting coefficient and the bias term of the deleted unit in the recognition learning system 1b. Then, the terminal device 100 displays the "recovery" button on the display unit DS together with the unit visualization information of the deleted unit. Then, when the operation detection unit OP of the terminal device 100 detects the operation information indicating the selection of the unit visualization information by the user and the operation for the "recovery" button, the terminal device 100 responds to the deletion unit 17 of the recognition learning device 10b. And send the operation information. In response to receiving the operation information from the terminal device 100, the deletion unit 17 selects the processing parameter information corresponding to the unit ID corresponding to the unit visualization information selected by the user, which is stored in the own device. Then, the processing parameter information is added to the storage unit M1. As a result, the user can confirm the deleted unit by the recognition learning device 10b, and if it is found that the important unit has been deleted, the user can restore the unit to DNN.

Thus, according to the present embodiment, feature maps or neurons that do not contribute to the recognition of DNN for evaluation data in a specific domain can be deleted. As a result, in a specific domain, DNN can recognize lightly and at high speed while maintaining recognition accuracy. For example, it is possible to learn a DNN that can correspond to various environments using learning data including various domains, and adjust the DNN according to a specific domain in which this system is actually used. ..

[Fourth Embodiment]
Various processes can be considered as the "specified process" performed for the unit status information corresponding to the specific unit ID. For example, the following processes (third process, fourth process) can also be considered.

As a third process, the processing unit 12 uses the unit status information associated with any unit ID in the same hierarchy as the unit corresponding to the specific unit ID among the unit status information read from the storage unit M2 as additional unit information. Generate. Here, the arbitrary unit ID corresponds to, for example, a unit ID adjacent to a specific unit ID, an ID of a randomly selected unit, a fixed unit ID, or the like. Here, the random unit IDs are selected according to a uniform distribution, for example, from the unit IDs in the same hierarchy. As the "specified processing", processing such as four arithmetic operations such as adding additional unit information to the predetermined unit status information may be performed.

As a fourth process, the processing unit 12 is a feature map having the same size as a set of numerical values represented by the unit state information corresponding to a specific unit ID among the unit state information read from the storage unit M2 and having a predetermined value for the element. Or generate additional unit information indicating a neuron. Here, the predetermined value is, for example, a predetermined fixed numerical pattern.

The processing information required for this "specified processing" is stored in a storage device inside or outside the own device. For example, the processing information is stored as a part of the structural information of the DNN of the storage unit M1 in the own device. This processing information includes, for example, an ID indicating "specified processing", additional unit information, information on a probability distribution for generating a random value, and processing information for additional unit information such as replacement and four arithmetic operations and specific unit information. and so on.

Further, the "specified processing" may be performed as a part of the structure of the DNN. Specifically, the processing unit 12 provides DNN structure information indicating a structure in which a unit addition processing layer for performing "specified processing" is inserted between a predetermined layer to be processed and a layer one level higher. Generate. Here, each unit information of the unit addition processing layer corresponds to the addition unit information as will be described later in FIG. 16, and "specified processing" is applied to each unit information of the next lower layer. The processing parameters of the DNN structure information are set so as to be performed. Then, the processing unit 12 stores the generated DNN structure information in the storage unit M1.

FIG. 15 is a diagram showing an example of applying the third process to the unit to be visualized by DNN. First, FIG. 15 describes a case where the units included in the Convolution 1 layer 501 and the Convolution 2 layer 502 of the DNN stored in the storage unit M1 are set as the visualization target units. Specifically, in FIG. 15, as the third process, units 531-2 and 532-2 adjacent to the units 511 and 512 in the same hierarchy are selected as additional unit information, and the unit status information of the units 511 and 512 is selected. Are shown to be replaced (541-2, 542-2) or added, respectively.

FIG. 16 is a diagram showing an example in which "specified processing" is applied as processing of the DNN hierarchy. First, FIG. 16 describes a case where the units included in the Convolution 1 layer 501 and the Convolution 2 layer 502 of the DNN stored in the storage unit M1 are set as the visualization target units. Specifically, in FIG. 16, the outputs of the Convolution 1 layer 501 and the Convolution 2 layer 502 are input to the unit addition process 1 layer 501-3 and the unit addition process 2 layer 502-3, respectively, and the first to fourth layers described above are described. Indicates that the process is applied. For example, the additional unit information having the unit ID of the unit addition process 1 layer is applied to the unit having the unit ID of the Convolution 1 layer of F0201. Further, the additional unit information having the unit ID of the unit addition process 2 layer F06003 is applied to the unit having the unit ID of the Convolution 2 layer F0503. For example, when the fourth process is used in the unit addition process 1 layer, the addition unit information F03001 is set so that the element has a predetermined value with the same size as the unit whose unit ID is F012001, and the unit ID is F022001. Apply four arithmetic operations such as replacing or adding units in.

[Fifth Embodiment]
In the step between step V106 and step V107 of FIG. 8, what kind of processing is to be performed as "specified processing" may be set. In that case, the process for realizing the set "specified process" will be performed in each subsequent step. For example, the processing unit 12 sets the "specified processing" based on the processing information read from the storage unit M1. For example, when the "specified processing" is processed as a part of the DNN structure as described above in the description of FIG. 16, the structure in which the unit addition processing layer is inserted and the processing parameters corresponding to the "specified processing" Generates DNN structure information indicating. Then, the processing unit 12 stores the generated DNN structure information in the storage unit M1.

Further, in each of the above embodiments, the problem of identifying a plurality of states has been described as an example, but the present invention is not limited to this, and can be applied to a general identification problem. It can be applied to the problem of anomaly detection to identify.

Further, in each of the above embodiments, it has been described that each of the recognition learning devices 10, 10a and 10b includes a storage unit M1 and a storage unit M2. It may be an external device capable of communicating with 10a and 10b. For example, the storage unit M1 and the storage unit M2 may include the storage unit M1 and the storage unit M2 on a server capable of data communication with the recognition learning devices 10, 10a and 10b via a network, or another device may include the storage unit M1 and the storage unit M2. This also applies to other functional parts.

In addition, some or all of the configurations of the above-described embodiments and modifications can be used in combination as appropriate, and some or all of the configurations of the above-described embodiments and modifications can be used in combination. May be used selectively.

[Sixth Embodiment]
Each functional part constituting the recognition learning devices 10, 10a and 10b may be implemented by hardware, but each part other than the storage unit M1 and the storage unit M2 may be implemented by software (computer program). .. In such a case, a computer device capable of executing this software (having a storage unit M1 and a storage unit M2 or capable of data communication with the storage unit M1 and the storage unit M2) is applied to the recognition learning devices 10, 10a and 10b. It is possible. An example of a hardware configuration of such a computer device will be described with reference to the block diagram of FIG.

The CPU 901 performs processing using computer programs and data stored in the RAM 902 and the ROM 903. As a result, the CPU 901 controls the operation of the entire computer device, and executes or controls each of the above-described processes as performed by the recognition learning devices 10, 10a, and 10b to which the computer device is applied.

The RAM 902 has an area for storing computer programs and data loaded from the ROM 903 and the external storage device 906, and data received from the outside via the I / F (interface) 907. Further, the RAM 902 has a work area used by the CPU 901 to execute various processes. As described above, the RAM 902 can appropriately provide various areas. The ROM 903 stores setting data, a boot program, and the like of the computer device that do not need to be rewritten.

The operation unit 904 is configured by a user interface such as a mouse or a keyboard, and various instructions can be input to the CPU 901 by the user operating the operation unit 904. For example, the user can input setting information such as a threshold value to the computer device by operating the computer device.

The display unit 905 is composed of a CRT, a liquid crystal screen, or the like, and can display the processing result by the CPU 901 with an image, characters, or the like. The display unit 905 may be a projection device that projects an image or characters onto the projection surface. The touch panel screen may be configured by integrating the operation unit 904 and the display unit 905.

The external storage device 906 is a large-capacity information storage device typified by a hard disk drive device. The external storage device 906 stores computer programs and data for causing the CPU 901 to execute or control each of the above-mentioned processes as performed by the OS (operating system) and the recognition learning devices 10, 10a, and 10b. This computer program includes a computer program for causing the CPU 901 to execute or control the functions of the functional units of the recognition learning devices 10, 10a, and 10b excluding the storage unit M1 and the storage unit M2 in FIGS. 1, 9, and 13. ing. Further, the data stored in the external storage device 906 includes data (threshold value and the like) handled by the recognition learning devices 10, 10a and 10b as known information. Further, the storage unit M1 and the storage unit M2 may be provided in the external storage device 906. The computer programs and data stored in the external storage device 906 are appropriately loaded into the RAM 902 according to the control by the CPU 901, and are processed by the CPU 901.

The I / F 907 functions as an interface for performing data communication with an external device. For example, data communication with the terminal device 100 (100a) is performed via the I / F 907.

The CPU 901, RAM 902, ROM 903, operation unit 904, display unit 905, external storage device 906, and I / F 907 are all connected to the bus 908. The configuration of the computer device shown in FIG. 14 can also be applied to the terminal device 100 (100a). In this case, the display unit 905 functions as the display unit DS, and the operation detection unit OP can be mounted by the operation unit 904.

As described above, according to each of the above-described embodiments and modifications, it is possible to visualize the feature amount of DNN that contributes to the recognition of the evaluation data. Therefore, the user can confirm whether or not the DNN uses the feature amount peculiar to the learning data. In addition, the DNN can be relearned based on the feedback from the user of the importance of the visualized features. Therefore, the user can control the DNN so as not to use the feature amount peculiar to the learning data. In addition, the DNN feature amount that does not contribute to the recognition of the evaluation data can be deleted. Therefore, the speed and weight of the DNN can be reduced according to the usage environment.

(Other Examples)
The present invention supplies a program that realizes one or more functions of the above-described embodiment to a system or device via a network or storage medium, and one or more processors in the computer of the system or device reads and executes the program. It can also be realized by the processing to be performed. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

11: Processing unit 12: Processing unit 13: Detection unit 14: Selection unit 15: Visualization unit

Claims (18)

For each of the plurality of categories, a first neural network that outputs the probability that the target belonging to the category is included in the input data as an output value, and
A second neural network in which a specific unit is modified in the first neural network , and the probability that the input data includes an object belonging to the category is output as an output value for each of the plurality of categories. With the second neural network
For each of the plurality of categories, the difference information of the first neural network representing the difference between the output value the output value output to the category second neural network is output to the category And the calculation method to find
An information processing device including an output means for outputting to a display device information indicating the contribution of a specific unit to the output value with respect to the input data based on the difference information obtained by the calculation means. The information processing apparatus according to claim 1, wherein the second neural network is a neural network in which the output values of all neurons in the specific unit of the first neural network are changed to 0. The information processing apparatus according to claim 1, wherein the second neural network is a neural network in which a predetermined value is added to an output value of each neuron in the specific unit of the first neural network. .. The first and second neural networks are neural networks having a plurality of layers.
The second neural network uses the processing result of the lower layer obtained when the first neural network obtains the output value as the processing result of the layer lower than the layer to which the specific unit belongs. The information processing apparatus according to any one of claims 1 to 3. The information processing apparatus according to any one of claims 1 to 4, wherein the calculation means obtains the difference as the difference information. The calculation means according to any one of claims 1 to 4, wherein the calculation means obtains the difference information based on the output value of the first neural network and the information used for the change. Information processing device. The information according to any one of claims 1 to 6, wherein the second neural network is a neural network in which a plurality of specific units in the first neural network are sequentially changed. Processing equipment. Said output means, for each of the plurality of categories, and defining the number of the difference information in descending order, according to claim 1 to 7, characterized in that the output information representative of a particular unit corresponding to the difference information, the The information processing apparatus according to any one of the above items. The output means specifies a specified number of specific units in ascending order of the average of the difference information with respect to the plurality of input data for each of the plurality of categories , and the calculation means obtains the specified specific units. The information processing apparatus according to any one of claims 1 to 7, wherein the difference information and the information representing the specified specific unit are output. In addition
The information processing apparatus according to claim 9, further comprising means for deleting the specified specific unit from the first neural network. The output means further comprises any one of claims 1 to 10, wherein the output means outputs information representing the characteristics of the input data that contributes to the output value of the specific unit to the display device. The information processing device described. In addition, it is equipped with a means for accepting the selection of units by the user.
The output means is characterized in that the unit selected by the user is the specific unit, and information representing the characteristics of the input data that contributes to the output value of the specific unit is displayed on the display device. The information processing device according to claim 11. Further, a means for accepting the selection of the category by the user is provided.
12. The output means is characterized in that the display device displays information representing the characteristics of the input data that contributes to the output value of the category selected by the user in the unit selected by the user. The information processing device described in. It said output means further among the elements included in the input data, and outputs the information representing the elements contributing to the output value of the particular unit, to the display device, contribute to the output value of the particular unit The information processing apparatus according to any one of claims 1 to 13, wherein the elements to be processed are identified and displayed. In addition
When importance for the elements displayed in the display device is input, and characterized by using the importance with learning method using the importance comprises means for performing re-learning of the first neural network The information processing apparatus according to claim 14. The information processing apparatus according to any one of claims 1 to 15, wherein the unit is a feature map of a neural network or a neuron. For each of the plurality of categories, a first neural network that outputs the probability that the target belonging to the category is included in the input data as an output value, and
A second neural network in which a specific unit is modified in the first neural network, and the probability that the input data includes an object belonging to the category is output as an output value for each of the plurality of categories. With the second neural network
It is an information processing method performed by an information processing device having
For each of the plurality of categories, the calculation means of the information processing device outputs an output value for the category by the first neural network and an output value output for the category by the second neural network. a calculation step of obtaining difference information representing a difference between,
The output means of the information processing device includes an output step of outputting to a display device information indicating the contribution of a specific unit to the output value to the input data based on the difference information obtained in the calculation step. An information processing method characterized by the fact that. A computer program for causing a computer to function as each means of the information processing apparatus according to any one of claims 1 to 16.

Images