JP2021111299A - Learning device, learning method, learning program, identification device, identification method, and identification program - Google Patents

Abstract

To present more useful explanatory materials for the judgment basis of neural networks.SOLUTION: A learning device includes an input unit configured to acquire learning data and a correct answer value, an important area estimation unit configured to estimate one or more important areas based on the learning data, a trimming processing unit configured to trim the one or more important areas based on the learning data and information indicating each of the one or more important areas and output the one or more important areas, a feature quantity extraction unit configured to extract a feature quantity based on the one or more important areas and a first neural network, a similarity calculation unit configured to calculate and output a similarity between the feature quantity and a prototype, an inference unit configured to output an inference value based on the similarity, an evaluation unit configured to evaluate the inference value based on the correct answer value and obtain an evaluation result, and an update unit configured to update a weight parameter of the first neural network and the prototype based on the evaluation result.SELECTED DRAWING: Figure 1

Description

The present invention relates to a learning device, a learning method, a learning program, an identification device, an identification method and an identification program.

In general, a multi-layer neural network having high performance in today's image recognition and the like is composed of a huge number of parameters and a complicated model. However, while this type of machine learning system exhibits excellent performance, there is a problem that it is difficult to interpret the judgment basis of the neural network. In order to solve this problem, some methods have been proposed to present explanatory materials for the judgment basis of the neural network. For example, there is known a method of presenting a similar example to the input data as an example of an explanatory material for the judgment basis of the neural network.

The method described in Patent Document 1 searches a case database for cases (similar cases) corresponding to image features based on image features extracted from images by machine learning, mainly in the diagnosis of medical images, and is similar. It is a method of presenting a case.

The method described in Non-Patent Document 1 learns typical examples (prototypes) of each of a plurality of classification classes in image classification, and determines the prototype having the highest degree of similarity to the features of identification data at the time of inference by a neural network. This is a method presented as an example of materials for explaining the rationale. In such a method, since a layer for calculating the feature amount of the identification data and the similarity with the prototype is embedded in the model, the neural network learns to perform the classification based on the similarity.

The method described in Patent Document 2 is a method of learning typical examples (prototypes) of each of a plurality of classification classes, similarly to the method described in Non-Patent Document 1. However, the method of Patent Document 2 does not present the learned prototype as it is, but searches for the data having the feature amount closest to the prototype from the learning data and presents the found learning data. At this time, a part of the learning data may be presented as a similar part instead of the whole of the found learning data (for example, an image).

JP-A-2019-125240

Oscar Li, 3 others, "Deep Learning for Case-Based Reasoning through Prototypes: ANeural Network that Explains Its Predictions", [online], [Searched December 26, 1st year of Reiwa], Internet <https://arxiv. org / abs / 1710.04806 ＞ Chaofan Chen, 5 others, "This LooksLike That: Deep Learning for Interpretable Image Recognition", [online], [Searched December 26, 1st year of Reiwa], Internet <https://arxiv.org/abs/1806.10574> Wei Liu, 6 others, "SSD: Single Shot MultiBox Detector", [online], [Searched on December 26, 1st year of Reiwa], Internet <https://arxiv.org/abs/1512.02325>

However, in the method described in Patent Document 1 and the method described in Non-Patent Document 1, similar examples of data input to the neural network are presented as an entire image. Therefore, in such a method, it is difficult to interpret which part of the whole image presented as a similar example is similar to the data input to the neural network.

In the method described in Non-Patent Document 2, a range (similar range) similar to the input image calculated on the feature space is upsampled to the same size as the input image, so that the input image corresponding to the similar range is obtained. It is possible to present the site. However, the relationship between the position on the feature space and the position on the input image is different from the mere enlargement relationship or reduction relationship. Therefore, there is a possibility that the neural network determines a part other than the presented part as a similar part. That is, the method described in Non-Patent Document 2 does not always present the correct similar site.

Therefore, it is desired to provide a technique capable of presenting a more useful explanatory material for the judgment basis of the neural network.

In order to solve the above problem, according to a certain viewpoint of the present invention, an input unit for acquiring learning data and a correct answer value, and an important area estimation for estimating one or more important areas based on the learning data. A trimming processing unit that trims the one or more important regions based on the unit, the learning data, and information indicating each of the one or a plurality of important regions, and outputs the one or a plurality of important regions. A feature extraction unit that extracts a feature amount based on the one or a plurality of important regions and a first neural network, a similarity calculation unit that calculates and outputs the similarity between the feature amount and the prototype, and the like. An inference unit that outputs an inference value based on the similarity, an evaluation unit that evaluates the inference value based on the correct answer value and obtains an evaluation result, and the first neural network based on the evaluation result. A learning device is provided that includes an update unit that updates the weight parameter and the prototype.

The important region estimation unit estimates the one or a plurality of important regions based on the learning data and the second neural network, and the update unit estimates the second neural network based on the evaluation result. The weight parameter may be updated.

The inference unit outputs the inference value based on the similarity and the third neural network, and the update unit updates the weight parameter of the third neural network based on the evaluation result. May be good.

The size of each of the one or more important regions may be variable.

Predetermined constraints may be imposed on the size of each of the one or more important areas.

The size of the feature amount may be variable.

The number of channels of the feature amount is the same as the number of channels of the prototype, and the similarity calculation unit calculates the similarity between one or more of the channel data of the feature amount and the channel data of the prototype. The highest similarity may be output to the inference unit as the similarity corresponding to the channel.

The similarity calculation unit saves the similarity output to the inference unit and the feature amount corresponding to the similarity in a part or all of the plurality of learning data as storage data for each channel. The update unit detects a feature amount having the highest degree of similarity to the prototype as a similar feature amount from the stored data for each channel, and the region corresponding to the similar feature amount of the learning data from which the similar feature amount is extracted. Data may be associated with the prototype for each channel.

The update unit may overwrite the prototype for each channel by the similar feature amount.

When the prototype is overwritten by the similar feature amount in the middle of learning, the update unit may stop updating the prototype.

Further, according to another aspect of the present invention, acquisition of training data and correct answer values, estimation of one or more important regions based on the training data, and the training data and the above. Trimming the one or more important areas based on the information indicating each of the one or more important areas to output the one or more important areas, and the one or more important areas and the first Extracting the feature amount based on the neural network of 1, calculating and outputting the similarity between the feature amount and the prototype, outputting the inferred value based on the similarity degree, and the correct answer. A learning method including evaluating the inferred value based on the value to obtain an evaluation result, and updating the weight parameter of the first neural network and the prototype based on the evaluation result. Is provided.

Further, according to another aspect of the present invention, the computer is provided with an input unit for acquiring training data and a correct answer value, and an important region estimation unit for estimating one or more important regions based on the training data. A trimming processing unit that trims the one or more important areas based on the learning data and information indicating each of the one or more important areas and outputs the one or more important areas. A feature extraction unit that extracts a feature amount based on the one or a plurality of important regions and a first neural network, a similarity calculation unit that calculates and outputs the similarity between the feature amount and the prototype, and the similarity. An inference unit that outputs an inference value based on the degree, an evaluation unit that evaluates the inference value based on the correct answer value and obtains an evaluation result, and a weight parameter of the first neural network based on the evaluation result. A learning program for functioning as a learning device including an update unit for updating the prototype and the prototype is provided.

Further, according to another aspect of the present invention, the input unit for acquiring the identification data and the correct answer value, the important area estimation unit for estimating one or more important regions based on the identification data, and the identification. A trimming processing unit that trims the one or more important areas based on the data and the one or more important areas and outputs the one or more important areas, and the one or more important areas. A feature extraction unit that extracts a feature amount based on the above-mentioned feature amount and the first neural network, a similarity degree calculation unit that calculates and outputs the similarity between the feature amount and the prototype, and an inferred value based on the similarity degree. An identification device including an inference unit for output and a display control unit for controlling region data corresponding to the prototype of training data to be displayed for each channel is provided.

The display control unit may control so that the information regarding the region corresponding to the similarity output to the inference unit of the identification data is displayed for each channel.

The display control unit may control so that the similarity output to the inference unit or a value corresponding to the similarity is displayed as a score for each channel.

The display control unit may control so that a predetermined number of the area data are displayed in descending order of similarity output to the inference unit.

Further, according to another aspect of the present invention, the identification data and the correct answer value are acquired, one or a plurality of important regions are estimated based on the identification data, and the identification data and the above are described. Trimming the one or more important regions based on the one or more important regions to output the one or more important regions, and the one or more important regions and the first neural network. Extracting the feature amount based on the above, calculating and outputting the similarity between the feature amount and the prototype, outputting the inferred value based on the similarity degree, and the prototype of the training data. Identification methods are provided, including controlling the region data corresponding to the channel to be displayed on a channel-by-channel basis.

Further, according to another aspect of the present invention, the computer is provided with an input unit for acquiring identification data and a correct answer value, and an important region estimation unit for estimating one or more important regions based on the identification data. A trimming processing unit that trims the one or more important areas based on the identification data and the one or more important areas and outputs the one or more important areas, and the one or more important areas. Based on the feature extraction unit that extracts the feature amount based on the important region of the above and the first neural network, the similarity calculation unit that calculates and outputs the similarity between the feature amount and the prototype, and the similarity degree. An identification program for functioning as an identification device including an inference unit that outputs an inference value and a display control unit that controls so that area data corresponding to the prototype of training data is displayed for each channel is provided. NS.

As described above, the present invention provides a technique capable of presenting a more useful explanatory material for the judgment basis of the neural network.

It is a figure which shows the functional structure example of the learning apparatus which concerns on embodiment of this invention. It is a figure for demonstrating the detail of the function of the important area estimation part. It is a figure for demonstrating the detail of the function of the trimming processing part. It is a figure for demonstrating the detail of the function of the feature extraction part. It is a figure for demonstrating the detail of the function of the similarity calculation part. It is a figure for demonstrating the detail of the function of an inference part. It is a figure for demonstrating the detail of the function of the evaluation part. It is a figure for demonstrating the detail of the function of the update part. It is a flowchart which shows the operation example of the learning apparatus which concerns on this embodiment. It is a figure which shows the functional structure example of the identification device which concerns on the same embodiment. It is a figure which shows the example of the similar part presentation screen. It is a flowchart which shows the operation example of the identification device which concerns on this embodiment. It is a figure which shows the hardware configuration of the information processing apparatus as an example of the learning apparatus which concerns on this embodiment.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the present specification and the drawings, components having substantially the same functional configuration are designated by the same reference numerals, so that duplicate description will be omitted.

Further, in the present specification and the drawings, a plurality of components having substantially the same functional configuration may be distinguished by adding different numbers after the same reference numerals. However, if it is not necessary to distinguish each of a plurality of components having substantially the same functional configuration, only the same reference numerals are given. Further, similar components of different embodiments may be distinguished by adding different alphabets after the same reference numerals. However, if it is not necessary to distinguish each of the similar components of different embodiments, only the same reference numerals are given.

(1. Details of the embodiment)
Subsequently, the details of the embodiment of the present invention will be described. In the embodiment of the present invention, the learning device 10 (FIG. 1) that learns the neural network based on the combination of the training data and the correct answer value will be described, and then the trained neural network and the identification data (test data) will be described. The identification device 20 (FIG. 10) that outputs an inferred value based on the above will be described.

In the following, it is mainly assumed that the learning device 10 and the identification device 20 are realized by the same computer. However, the learning device 10 and the identification device 20 may be realized by different computers. In such a case, the trained neural network generated by the learning device 10 is provided to the identification device 20. For example, the trained neural network may be provided from the learning device 10 to the identification device 20 via a recording medium, or may be provided via communication.

(1-1. Configuration of learning device)
First, a configuration example of the learning device 10 according to the embodiment of the present invention will be described. FIG. 1 is a diagram showing a functional configuration example of the learning device 10 according to the embodiment of the present invention. As shown in FIG. 1, the learning device 10 according to the embodiment of the present invention includes an input unit 121, an important region estimation unit 122, a trimming processing unit 123, a feature extraction unit 124, and a similarity calculation unit 125. , The inference unit 126, the evaluation unit 140, and the update unit 150 are provided.

In the embodiment of the present invention, the input unit 121, the important region estimation unit 122, the trimming processing unit 123, the feature extraction unit 124, the similarity calculation unit 125, and the inference unit 126 are configured by the neural network 120. Mainly assume the case. Hereinafter, the neural network is also referred to as "NN".

More specifically, the feature extraction unit 124 includes a first neural network (hereinafter, also referred to as “feature extraction NN”), and the important region estimation unit 122 includes a second neural network (hereinafter, “important region estimation”). NN ”), and the inference unit 126 includes a third neural network (hereinafter, also referred to as“ inference NN ”). However, the input unit 121, the important area estimation unit 122, the trimming processing unit 123, the feature extraction unit 124, the similarity calculation unit 125, and the inference unit 126 may have any specific configuration. good.

These blocks include an arithmetic unit such as a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit), and a program stored in a ROM (Read Only Memory) is expanded into a RAM by the arithmetic unit and executed. The function can be realized by the above. At this time, a computer-readable recording medium on which the program is recorded may also be provided. Alternatively, these blocks may be composed of dedicated hardware or may be composed of a combination of a plurality of hardware. The data required for the calculation by the arithmetic unit is appropriately stored by a storage unit (not shown).

The data set 110, the weight parameter 131 of the important region estimation NN, the weight parameter 132 of the feature extraction NN, the weight parameter 133 of the inference NN, the prototype 134, and the stored data 160 are stored by a storage unit (not shown). Such a storage unit may be composed of a memory such as a RAM (Random Access Memory), a hard disk drive, or a flash memory.

In the initial state, initial values are set for each of the weight parameter 131 of the important region estimation NN, the weight parameter 132 of the feature extraction NN, the weight parameter 133 of the inference NN, and the prototype 134. For example, the initial values set in these may be random values, but may be any value. For example, the initial values set in these may be learned values obtained by learning in advance. On the other hand, nothing in particular may be set in the saved data in the initial state.

(Data set 110)
The data set 110 is configured to include a plurality of learning data (input data) and correct answer values of the plurality of learning data. In the embodiment of the present invention, it is mainly assumed that the learning data is image data (particularly, still image data). However, the type of learning data is not particularly limited. For example, the learning data may be moving image data including a plurality of frames or acoustic data.

(Input unit 121)
The input unit 121 sequentially acquires a combination of learning data and a correct answer value from the data set 110. The input unit 121 sequentially outputs the combination of the learning data and the correct answer value to the important area estimation unit 122 and the trimming processing unit 123, respectively. In each block after the input unit 121, each process is sequentially executed repeatedly based on the input from the block in the previous stage.

For example, when the input unit 121 has acquired all the combinations of the learning data and the correct answer values from the data set 110, the input unit 121 repeats the operation of reacquiring the combinations from the beginning and outputting them again a predetermined number of times. good. In such a case, even in the block after the input unit 121, each process may be repeatedly executed in sequence based on the re-input from the block in the previous stage.

(Important area estimation unit 122)
The important area estimation unit 122 estimates one or a plurality of important areas from the learning data based on the learning data output from the input unit 121 and the important area estimation NN. More specifically, the important area estimation unit 122 outputs data output from the important area estimation NN based on inputting learning data to the important area estimation NN, and information indicating one or a plurality of important areas (1). Or obtain as the position and size of each of the multiple important areas). The important area estimation unit 122 outputs information indicating each of the one or a plurality of important areas to the trimming processing unit 123. Here, the function of the important region estimation unit 122 will be described in more detail with reference to FIG.

FIG. 2 is a diagram for explaining the details of the function of the important region estimation unit 122. With reference to FIG. 2, the learning data G1 output from the input unit 121 is shown, and the learning data G1 shows a “dog” as an example of the subject. At this time, it is assumed that the correct answer value of the learning data G1 is "dog". However, the subject reflected in the learning data G1 is not limited to the "dog". The important area estimation unit 122 causes the important area estimation NN to input the learning data G1, and uses the weight parameter 131 to indicate information indicating each of the important areas R1 to R4 output from the important area estimation NN (each of the important areas R1 to R4). Position and size).

For example, the important area estimation unit 122 estimates a predetermined number of important areas from the learning data G1. The number of important regions is not limited, but in order to improve the accuracy of the prototype described later, it is desirable that the number is equal to or greater than the number of channels of the prototype. However, the number of critical regions may be less than the number of prototype channels. In general, the channel is the first dimension of the feature amount extracted by the neural network for one input data, but in the present specification, the feature is defined for one or more important regions. The first dimension of the feature amount extracted by the extraction unit 124 is called a channel. Therefore, the number of channels corresponds to the number of one or more important regions.

FIG. 2 shows an example in which the important area R1 (ear), the important area R2 (eye), the important area R3 (mouth), and the important area R4 (leg) are estimated from the learning data G1 by the important area estimation unit 122. It is shown. That is, an example in which four important regions are estimated by the important region estimation unit 122 is shown. However, the number of important regions estimated by the important region estimation unit 122 is not limited. The types of important regions estimated by the important region estimation unit 122 are also not limited to the ears, eyes, mouth and legs.

Here, if the training data is x, the position of the important region in the training data is t, the size of the important region is s, and the processing of the important region estimation NN is the function g (), the position t of the important region is used. , The relationship between the size s of the important region and the function g () indicating the processing of the important region estimation NN can be expressed by the following mathematical formula (1).

s, t = g (x) ... (1)

For example, when the training data is two-dimensional data such as image data, and the shape of the important region is rectangular, the position t of the important region is a predetermined point (for example, for example) of the rectangle in the two-dimensional data. It can be represented by a combination of the vertical axis coordinates and the horizontal axis coordinates of (such as the upper left vertex of a rectangle). The size s of the important area is represented by the length of each of the vertical and horizontal directions of the rectangle in the two-dimensional data. However, the shape of the important region is not limited to a rectangle, and may be another shape (for example, a circle).

The size s of the important region may be fixed or may not be fixed (may be variable). When the size s of the important region is variable, it is expected that the flexibility of the similar portion presented later based on the important region is increased. On the other hand, if the size s of the important region is too close to the learning data, the meaning of trimming the important region later by the trimming processing unit 123 is diminished. Therefore, a predetermined constraint may be imposed on the size of the important area so that the important area can be trimmed meaningfully.

For example, a function (for example, a sigmoid function) whose range is limited to a predetermined range may be multiplied by g (x) so that the size s of the important region falls within a predetermined range. For example, if g (x) is multiplied by a constant and a sigmoid function, the size s of the important region will be within the range from 0 to the constant. Alternatively, if the smaller value of g (x) and the constant is adopted as the size s of the important region, the size s of the important region will be within the constant. For example, if these constants are set to a predetermined ratio (for example, half) to the size of the training data, the size s of the important region becomes a size less than or equal to the predetermined ratio of the size of the training data. Be constrained to be.

Alternatively, if the size s of the important area (or the square of the size s of the important area) is added to the loss function and the weight parameter 131 of the important area estimation NN is updated by the update unit 150, the size s of the important area s. Learning will be performed so that The specific configuration of the important region estimation NN is not particularly limited. For example, as the important region estimation NN, a convolutional neural network (for example, the convolutional neural network described in Non-Patent Document 3 described above) may be used.

(Trimming processing unit 123)
The explanation will be continued by returning to FIG. The trimming processing unit 123 has the learning data G1 output from the input unit 121 and the important area R1 of the learning data G1 based on the information indicating each of the important areas R1 to R4 output from the important area estimation unit 122. ~ R4 is trimmed and important regions R1 to R4 are output to the feature extraction unit 124. Here, the function of the trimming processing unit 123 will be described in more detail with reference to FIG.

FIG. 3 is a diagram for explaining the details of the function of the trimming processing unit 123. With reference to FIG. 3, the learning data G1 output from the input unit 121 is shown, and information indicating each of the important regions R1 to R4 estimated by the important region estimation unit 122 (position and size of each of the important regions R1 to R4). )It is shown. The trimming processing unit 123 trims the important regions R1 to R4 from the learning data G1. As shown in FIG. 3, trimming for the important regions R1 to R4 may mean excluding the regions other than the important regions R1 to R4 from the learning data G1.

(Feature Extraction Unit 124)
The explanation will be continued by returning to FIG. The feature extraction unit 124 extracts a feature amount based on the important regions R1 to R4 output from the trimming processing unit 123 and the feature extraction NN. More specifically, the feature extraction unit 124 obtains the data output from the feature extraction NN as a feature amount based on the fact that the feature extraction NN is input with the important regions R1 to R4. The feature extraction unit 124 outputs the feature amount to the similarity calculation unit 125. Here, the function of the feature extraction unit 124 will be described in more detail with reference to FIG.

FIG. 4 is a diagram for explaining the details of the function of the feature extraction unit 124. With reference to FIG. 4, important regions R1 to R4 output from the trimming processing unit 123 are shown. The feature extraction unit 124 causes the feature extraction NN to input the important regions R1 to R4, and obtains the feature quantities F1 to F4 output from the feature extraction NN using the weight parameter 132.

The size of each of the feature quantities F1 to F4 may be fixed, but may not be fixed (may be variable). When the size of each of the feature amounts F1 to F4 is variable, it is expected that the flexibility of the similar portion presented later based on the feature amounts F1 to F4 will be increased. Each of the feature amounts F1 to F4 corresponds to the channel data of the feature amount. That is, in the embodiment of the present invention, it is mainly assumed that the number of channels of the feature amount is 4, but the number of channels of the feature amount is not limited.

Moreover, the specific configuration of the feature extraction NN is not limited. For example, as the feature extraction NN, a neural network composed of a plurality of convolutional layers may be used.

(Similarity calculation unit 125)
The explanation will be continued by returning to FIG. The similarity calculation unit 125 calculates the similarity between the feature amounts F1 to F4 output from the feature extraction unit 124 and the prototype 134. The dimensions of the feature quantities F1 to F4 output from the feature extraction unit 124 and the dimensions of the prototype 134 are set to be the same. Here, it is assumed that the feature quantities F1 to F4 and the prototype 134 are two-dimensional data (that is, three-dimensional data) divided into a plurality of channels, but the number of dimensions is not limited. Then, the similarity calculation unit 125 outputs the calculated similarity between the feature quantities F1 to F4 and the prototype 134 to the inference unit 126. Here, the function of the similarity calculation unit 125 will be described in more detail with reference to FIG.

FIG. 5 is a diagram for explaining the details of the function of the similarity calculation unit 125. With reference to FIG. 5, the feature quantities F1 to F4 and the prototypes P1 to P4 are shown. Each of the prototypes P1 to P4 corresponds to the channel data of the prototype. That is, in the embodiment of the present invention, it is mainly assumed that the number of channels of the prototype is 4, but the number of channels of the prototype is not limited.

The number of channels of the feature quantities F1 to F4 and the number of channels of the prototypes P1 to P4 are set to be the same. As a result, the similarity calculation unit 125 can calculate the similarity between the feature quantities F1 to F4 and the prototypes P1 to P4 for each channel. In the example shown in FIG. 5, the similarity calculation unit 125 calculates the similarity M1 between the feature amount F1 and the prototype P1, calculates the similarity degree M2 between the feature amount F2 and the prototype P2, and sets the feature amount F3. The degree of similarity M3 with the prototype P3 is calculated, and the degree of similarity M4 between the feature amount F4 and the prototype P4 is calculated.

The similarity between the feature quantity and the prototype in the corresponding channel may be calculated in any way. For example, when the size of the feature amount in the corresponding channel and the size of the prototype are the same, the similarity calculation unit 125 corresponds the similarity between the feature amount and the prototype in the corresponding channel in the feature amount and the prototype. It can be calculated using the sum of squares (L2 norm) of the differences between the elements. For example, the reciprocal of the L2 norm can be suitably used as the degree of similarity because the smaller the sum of squares of the differences between the elements (the closer the feature amount and the prototype are), the larger the reciprocal.

Alternatively, the size of the feature in the corresponding channel may be larger than the size of the prototype. In such a case, the data type of the feature amount output from the feature extraction unit 124 is expressed as C (channel) × H (height) × W (width), and the prototype data type is C (channel) × H'. It is expressed as (height) x W'(width) (however, H> H'and W> W').

At this time, let z be the feature amount, and let z'be a partial feature amount that can be cut out from the feature amount z and have the same size as the prototype p. Then, the similarity calculation unit 125 includes the feature amount z'of one or a plurality of features of the feature amount z (that is, all or a part of the partial feature amount z'that can be cut out from the feature amount z) and the prototype p. The highest similarity M among the similarity of the above may be output to the inference unit 126 as the similarity corresponding to the channel. That is, the similarity M may be calculated by the similarity calculation unit 125 as shown in the following mathematical formula (2).

The method of calculating the degree of similarity is not limited to such an example. For example, the function for calculating the similarity may be freely set according to the problem to be solved by the neural network as long as it is a function to which the backpropagation method in the neural network can be applied. The stored data 160 updated by the similarity calculation unit 125 will be described later.

(Inference unit 126)
The explanation will be continued by returning to FIG. The inference unit 126 makes an inference based on the similarity degrees M1 to M4 output from the similarity calculation unit 125, and obtains an inference value. Then, the inference unit 126 outputs the inference value to the evaluation unit 140. Here, the function of the inference unit 126 will be described in more detail with reference to FIG.

FIG. 6 is a diagram for explaining the details of the function of the inference unit 126. With reference to FIG. 6, the similarity degrees M1 to M4 output from the similarity degree calculation unit 125 are shown. The inference unit 126 causes the inference NN to input the similarity M1 to M4, and obtains the inference value output from the inference NN using the weight parameter 133. In this specification, obtaining data output from a neural network based on input of data to the neural network is broadly referred to as "inference". Therefore, the term "inference" is also used in the learning stage.

The specific configuration of the inference NN is not particularly limited. However, the output format of the inference NN should be set in accordance with the format of the correct answer value corresponding to the learning data. For example, when the correct answer value is a class of the classification problem, the output of the inference NN may be a one-hot vector having a length corresponding to the number of classes.

(Evaluation unit 140)
The explanation will be continued by returning to FIG. The evaluation unit 140 evaluates the inference value output from the inference unit 126 based on the correct answer value acquired by the input unit 121, and obtains an evaluation result. Then, the evaluation unit 140 outputs the evaluation result to the update unit 150. Here, the function of the evaluation unit 140 will be described in more detail with reference to FIG. 7.

FIG. 7 is a diagram for explaining the details of the function of the evaluation unit 140. With reference to FIG. 7, the inferred value output from the inference unit 126 is shown. Further, referring to FIG. 7, the correct answer value acquired by the input unit 121 is shown. In the embodiment of the present invention, it is assumed that the evaluation unit 140 calculates the loss function corresponding to the correct answer value and the inferred value as the evaluation result. Here, the loss function used in the embodiment of the present invention is not limited to a specific function, and a loss function similar to the loss function used in a general neural network may be used. For example, the loss function may be a mean square error based on the difference between the correct value and the inferred value.

(Update part 150)
The explanation will be continued by returning to FIG. The update unit 150 updates the weight parameter 131 of the important region estimation NN, the weight parameter 132 of the feature extraction NN, the weight parameter 133 of the inference NN, and the prototype 134 based on the evaluation result output from the evaluation unit 140. I do. As a result, the weight parameter 131 of the important region estimation NN, the weight parameter 132 of the feature extraction NN, the weight parameter 133 of the inference NN, and the prototype 134 so that the inference value output from the inference unit 126 approaches the correct answer value. Can be trained. Here, the function of the update unit 150 will be described in more detail with reference to FIG.

FIG. 8 is a diagram for explaining the details of the function of the update unit 150. With reference to FIG. 8, the evaluation result output from the evaluation unit 140 is shown. Each of the prototypes P1 to P4 corresponds to the channel data of the prototype. For example, the update unit 150 uses an error backpropagation method (backpropagation) based on the evaluation result output from the evaluation unit 140 to carry out the weight parameter 131 of the important region estimation NN, the weight parameter 132 of the feature extraction NN, and the inference NN. The weight parameter 133 and the prototype 134 may be updated.

In the embodiment of the present invention, it is assumed that the learning device 10 detects (searches) the learning data in which the feature amount (similar feature amount) similar to the prototype is extracted. Then, it is assumed that the identification device 20 presents the region data corresponding to the similar feature amount of the learning data detected by the learning device 10 as a similar part. This makes it easier to understand the similarity between the similar example and the test data site by site, rather than presenting the entire similar example.

More specifically, as described above, the similarity calculation unit 125 calculates the similarity between the feature amount and the prototype for each of the plurality of learning data for each channel and outputs the similarity to the inference unit 126. Therefore, the similarity calculation unit 125 stores the similarity output to the inference unit 126 and the feature amount corresponding to the similarity as storage data 160 for each channel at a predetermined timing. As an example, FIG. 8 shows a plurality of similarities (similarity: 50, similarity: 10, ..., Similarity: 20) output to the inference unit 126 and a plurality of similarities for the channel corresponding to the prototype P1. An example is shown in which the feature amount corresponding to each degree of similarity is stored as the stored data 160. However, the features and similarity of the channels corresponding to each of the prototypes P2 to P4 are also preserved.

In the embodiment of the present invention, after the learning by the learning device 10 using the data set 110 is repeatedly executed several times (for example, after being repeatedly executed four times), the next round (for example, 5). It is assumed that the similarity and the feature amount of (rounds, etc.) are preserved. However, the timing at which the similarity and the feature amount are stored is not limited. The similarity calculation unit 125 may store the similarity and the feature amount in a part or all of the plurality of learning data used for learning by the learning device 10.

When the similarity calculation unit 125 finishes storing the similarity and the feature amount, the update unit 150 detects the feature amount having the highest similarity with the prototype P1 as the similar feature amount from the stored data 160. In the example shown in FIG. 8, a feature having a similarity of "50" is detected as a similar feature. The update unit 150 associates the region data corresponding to the similar feature amount of the learning data from which the similar feature amount has been extracted with the prototype P1 as a similar part. Similarly, the update unit 150 associates the region data with the prototypes P2 to P4 as similar parts for each channel.

The prototype 134 may be continuously updated by the error backpropagation method (backpropagation) together with the weight parameters 131 to 133 until the learning is completed. However, in the embodiment of the present invention, it is assumed that the update unit 150 overwrites the prototype 134 for each channel by the detected similar feature amount. This can improve the consistency between the prototype 134 used to calculate the similarity and the presented similarity sites. From the viewpoint of such consistency, it is preferable that the update unit 150 stops updating the prototype 134 when the prototype 134 is overwritten by a similar feature amount in the middle of learning.

The update unit 150 determines whether or not the learning end condition is satisfied each time the update based on the learning data is completed. When it is determined that the learning end condition is not satisfied, the input unit 121 acquires the next learning data, and the important area estimation unit 122, the trimming processing unit 123, the feature extraction unit 124, and the similarity calculation unit 125. , The inference unit 126, the evaluation unit 140, and the update unit 150, respectively, re-execute their own processing based on the next input data. On the other hand, when the update unit 150 determines that the learning end condition is satisfied, the learning is ended.

The learning end condition is not particularly limited as long as it is a condition indicating that the neural network 120 has been learned to some extent. Specifically, the end of learning may include the condition that the value of the loss function is smaller than the threshold value. Alternatively, the learning end condition may include a condition that the change in the value of the loss function is smaller than the threshold value (a condition that the value of the loss function is in a converged state). Alternatively, the learning end condition may include the condition that the weight parameter is updated a predetermined number of times. Alternatively, when the evaluation unit 140 calculates the accuracy based on the correct answer value and the inferred value, the learning end condition may include a condition that the accuracy exceeds a predetermined ratio (for example, 90%).

The configuration example of the learning device 10 according to the embodiment of the present invention has been described above.

(1-2. Operation of learning device)
Subsequently, an operation example of the learning device 10 according to the embodiment of the present invention will be described. FIG. 9 is a flowchart showing an operation example of the learning device 10 according to the embodiment of the present invention. First, as shown in FIG. 9, the input unit 121 acquires a combination of learning data and a correct answer value from the data set 110. Further, the important region estimation unit 122 acquires the weight parameter 131, the feature extraction unit 124 acquires the weight parameter 132, the inference unit 126 acquires the weight parameter 133, and the similarity calculation unit 125 acquires the prototype 134. (S11).

The important area estimation unit 122 estimates one or more important areas from the learning data based on the learning data output from the input unit 121 and the important area estimation NN (S12). More specifically, the important area estimation unit 122 causes the important area estimation NN to input the learning data, and uses the weight parameter 131 to output the data output from the important area estimation NN to the information indicating each of the one or a plurality of important areas ( Obtained as the position and size of each of one or more important areas). The important area estimation unit 122 outputs information indicating each of the one or a plurality of important areas to the trimming processing unit 123.

The trimming processing unit 123 has one or a plurality of learning data based on the learning data output from the input unit 121 and the information indicating one or a plurality of important areas output from the important area estimation unit 122. Trimming is performed on the important area (S13). Then, the trimming processing unit 123 outputs one or a plurality of important regions to the feature extraction unit 124.

The feature extraction unit 124 extracts a feature amount based on one or a plurality of important regions output from the trimming processing unit 123 and the feature extraction NN (S14). More specifically, the feature extraction unit 124 causes the feature extraction NN to input an important region, and obtains a feature amount output from the feature extraction NN using the weight parameter 132. The feature extraction unit 124 outputs the feature amount to the similarity calculation unit 125.

The similarity calculation unit 125 calculates the similarity between the feature amount output from the feature extraction unit 124 and the prototype 134 (S15). The inference unit 126 makes an inference based on the similarity output from the similarity calculation unit 125 and obtains an inferred value (S16). More specifically, the inference unit 126 causes the inference NN to input the similarity, and obtains the inference value output from the inference NN using the weight parameter 133. Then, the inference unit 126 outputs the inference value to the evaluation unit 140.

The evaluation unit 140 evaluates the inference value output from the inference unit 126 based on the correct answer value acquired by the input unit 121, and obtains an evaluation result (S17). More specifically, the evaluation unit 140 calculates the loss function according to the correct answer value and the inferred value as the evaluation result. Then, the evaluation unit 140 outputs the evaluation result to the update unit 150. The update unit 150 updates the weight parameter 131 of the important region estimation NN, the weight parameter 132 of the feature extraction NN, the weight parameter 133 of the inference NN, and the prototype 134 based on the evaluation result output from the evaluation unit 140. (S18).

The update unit 150 determines whether or not the learning end condition is satisfied each time the update based on the learning data is completed (S19). When it is determined that the learning end condition is not satisfied (“NO” in S19), the operation is shifted to S11, the next learning data is acquired by the input unit 121, the important area estimation unit 122, trimming. The processing unit 123, the feature extraction unit 124, the similarity calculation unit 125, the inference unit 126, the evaluation unit 140, and the update unit 150 each re-execute their own processing based on the next input data. On the other hand, when the update unit 150 determines that the learning end condition is satisfied (“YES” in S19), the learning is ended.

The operation example of the learning device 10 according to the embodiment of the present invention has been described above.

(1-3. Configuration of identification device)
Subsequently, a configuration example of the identification device 20 according to the embodiment of the present invention will be described. FIG. 10 is a diagram showing a functional configuration example of the identification device 20 according to the embodiment of the present invention. As shown in FIG. 10, the identification device 20 according to the embodiment of the present invention includes a trained neural network 120 learned by the learning device 10. In addition, the identification device 20 includes a display control unit 220 and a display unit 230.

The display control unit 220 includes an arithmetic unit, and its function can be realized by the program stored in the ROM being expanded and executed in the RAM by the arithmetic unit. At this time, a computer-readable recording medium on which the program is recorded may also be provided. Alternatively, these blocks may be composed of dedicated hardware or may be composed of a combination of a plurality of hardware. The data required for the calculation by the arithmetic unit is appropriately stored by a storage unit (not shown).

The display unit 230 is composed of a display. The test data 210, the weight parameter 131 of the important region estimation NN, the weight parameter 132 of the feature extraction NN, the weight parameter 133 of the inference NN, and the prototype 134 are stored by a storage unit (not shown). Such a storage unit may be composed of a memory such as a RAM, a hard disk drive, or a flash memory.

(Test data 210)
The test data 210 corresponds to identification data. In the embodiment of the present invention, it is mainly assumed that the test data 210 is image data (particularly, still image data) like the learning data. However, the type of test data 210 is not particularly limited. For example, the test data 210 may be moving image data including a plurality of frames or acoustic data, as in the case of learning data.

(Input unit 121-Inference unit 126)
The input unit 121 acquires the test data 210. The input unit 121 outputs the test data 210 to the important area estimation unit 122 and the trimming processing unit 123, respectively. The important area estimation unit 122 estimates one or more important areas from the test data 210 based on the test data 210 output from the input unit 121 and the important area estimation NN. The method of estimating one or more important regions from the test data 210 is the same as the method of estimating the important region from the learning data by the important region estimation unit 122 in the learning device 10.

The trimming processing unit 123 has one or a plurality of important areas of the test data based on the test data output from the input unit 121 and the information indicating each of the one or a plurality of important areas output from the important area estimation unit 122. Is trimmed and one or a plurality of important regions are output to the feature extraction unit 124. The method of trimming one or more important areas of the test data 210 is the same as the method of trimming the one or more important areas of the learning data by the trimming processing unit 123 in the learning device 10.

The feature extraction unit 124 extracts a feature amount based on one or a plurality of important regions output from the trimming processing unit 123 and the feature extraction NN. The method in which the feature extraction unit 124 extracts the feature amount is the same as the method in which the feature extraction unit 124 in the learning device 10 extracts the feature amount. The similarity calculation unit 125 calculates the similarity between the feature amount output from the feature extraction unit 124 and the prototype 134. The method in which the similarity calculation unit 125 calculates the similarity is the same as the method in which the similarity calculation unit 125 in the learning device 10 calculates the similarity.

The inference unit 126 makes an inference based on the similarity output from the similarity calculation unit 125 and obtains an inferred value. The method in which the inference unit 126 makes an inference is the same as the method in which the inference unit 126 in the learning device 10 makes an inference. For example, when a "dog" is shown as a subject in the test data, it is assumed that the "dog" is output as an inference value corresponding to the test data. At this time, it is conceivable that the entire image of the "dog" is presented as a similar example.

However, in the embodiment of the present invention, it is assumed that the identification device 20 presents the region data corresponding to the similar feature amount as the similar part. This makes it easier to understand the similarity between the similar example and the test data site by site, rather than presenting the entire image of the "dog" as a similar example. Hereinafter, the functions of the display control unit 220 and the display unit 230 will be described as blocks for presenting similar parts.

(Display control unit 220 to display unit 230)
As described above, the region data corresponding to the similar feature amount of the learning data from which the feature amount having the highest degree of similarity to the prototype (similar feature amount) is extracted is associated with the prototype for each channel as a similar part. .. Therefore, the display control unit 220 controls the display unit 230 so that similar parts corresponding to the prototypes of the learning data are displayed for each channel. In the following, the presentation of similar parts will be described in detail with reference to FIG.

FIG. 11 is a diagram showing an example of a similar part presentation screen. With reference to FIG. 11, a similar site presentation screen D1 is shown. The display control unit 220 controls the display unit 230 so that the similar portion presentation screen D1 is displayed by the display unit 230. In addition to the test data G2, the similar site presentation screen D1 includes similar sites R21 (ears), similar sites R22 (eyes), similar sites R23 (mouth), and similar sites R24 (legs) corresponding to each channel. include.

By presenting the similar sites R21 to R24 in this way, the similarity between the "dog" as a similar example and the test data G2 can be easily understood for each site. For example, if attention is paid to the similar parts R21 to R24, it becomes easier to understand the reason why the similar example and the test data G2 are similar. In the example shown in FIG. 11, all of the similar parts R21 to R24 are displayed, but the display control unit 220 determines only a part of the similar parts R21 to R24 (for example, in descending order of similarity). (Only the number) may be displayed on the display unit 230. As a result, a prototype with a large contribution to inference can be grasped. Alternatively, the display control unit 220 may display a predetermined number) on the display unit 230 in ascending order of similarity. As a result, a prototype with a small contribution to inference can be grasped.

Further, the display control unit 220 controls the display unit 230 so that the value corresponding to the similarity output to the inference unit 126 of the test data G2 is displayed as a score for each channel. The score is obtained by multiplying the similarity (for example, the above formula (1)) by a function (for example, a sigmoid function) whose range is limited to a predetermined range so that the score falls within a predetermined range. You may. Alternatively, the score may be obtained by adding a constant other than 0 to the denominator of the similarity so that the similarity does not diverge to infinity. Alternatively, the score may be the similarity itself.

On the similar part presentation screen D1, "50" is displayed as a score corresponding to the similar part R21, "20" is displayed as a score corresponding to the similar part R22, and "30" is displayed as a score corresponding to the similar part R23. It is displayed, and "70" is displayed as a score corresponding to the similar portion R24. By displaying the score in this way, it becomes possible to understand how similar the region of the test data G2 and the similar sites R21 to R24 are.

Further, as shown in FIG. 11, the display control unit 220 displays the information (position and size of the area) regarding the area corresponding to the similarity output to the inference unit 126 of the test data G2. It is good to control 230. This makes it easier to understand which region of the test data G2 corresponds to a similar site. In the example shown in FIG. 11, information R31 to R34 regarding the regions corresponding to the four similarities output to the inference unit 126 of the test data G2 is displayed.

The configuration example of the identification device 20 according to the embodiment of the present invention has been described above.

(1-4. Operation of identification device)
Subsequently, an operation example of the identification device 20 according to the embodiment of the present invention will be described. FIG. 12 is a flowchart showing an operation example of the identification device 20 according to the embodiment of the present invention. First, as shown in FIG. 12, the input unit 121 acquires test data. Further, the important region estimation unit 122 acquires the weight parameter 131, the feature extraction unit 124 acquires the weight parameter 132, the inference unit 126 acquires the weight parameter 133, and the similarity calculation unit 125 acquires the prototype 134. (S31).

The important area estimation unit 122 estimates one or more important areas from the test data based on the test data output from the input unit 121 and the important area estimation NN (S32). More specifically, the important area estimation unit 122 causes the important area estimation NN to input test data, and uses the weight parameter 131 to output data from the important area estimation NN to indicate information (1) indicating each of one or a plurality of important areas. Or obtain as the position and size of each of the multiple important areas). The important area estimation unit 122 outputs information indicating each of the one or a plurality of important areas to the trimming processing unit 123.

The trimming processing unit 123 has one or a plurality of important learning data based on the test data output from the input unit 121 and the information indicating each of the one or a plurality of important areas output from the important area estimation unit 122. Trimming is performed on the area (S33). Then, the trimming processing unit 123 outputs one or a plurality of important regions to the feature extraction unit 124.

The feature extraction unit 124 extracts a feature amount based on one or a plurality of important regions output from the trimming processing unit 123 and the feature extraction NN (S34). More specifically, the feature extraction unit 124 causes the feature extraction NN to input an important region, and obtains a feature amount output from the feature extraction NN using the weight parameter 132. The feature extraction unit 124 outputs the feature amount to the similarity calculation unit 125.

The similarity calculation unit 125 calculates the similarity between the feature amount output from the feature extraction unit 124 and the prototype 134 (S35). The inference unit 126 makes an inference based on the similarity output from the similarity calculation unit 125 and obtains an inferred value (S36). More specifically, the inference unit 126 causes the inference NN to input the similarity, and obtains the inference value output from the inference NN using the weight parameter 133. Then, the inference unit 126 outputs the inference value.

Further, the display control unit 220 controls the display unit 230 so that similar parts corresponding to the prototypes of the learning data are displayed for each channel. Further, the display control unit 220 controls the display unit 230 so that the value corresponding to the similarity output to the inference unit 126 of the test data is displayed as a score for each channel (S37). Further, the display control unit 220 controls the display unit 230 so that the information (position and size of the area) regarding the area corresponding to the similarity output to the inference unit 126 of the test data is displayed.

The operation example of the identification device 20 according to the embodiment of the present invention has been described above.

(2. Hardware configuration example)
Subsequently, a hardware configuration example of the learning device 10 according to the embodiment of the present invention will be described. However, a hardware configuration example of the identification device 20 according to the embodiment of the present invention can be realized in the same manner.

Hereinafter, as a hardware configuration example of the learning device 10 according to the embodiment of the present invention, a hardware configuration example of the information processing device 900 will be described. The hardware configuration example of the information processing device 900 described below is only an example of the hardware configuration of the learning device 10. Therefore, as for the hardware configuration of the learning device 10, an unnecessary configuration may be deleted from the hardware configuration of the information processing apparatus 900 described below, or a new configuration may be added.

FIG. 13 is a diagram showing a hardware configuration of an information processing device 900 as an example of the learning device 10 according to the embodiment of the present invention. The information processing device 900 includes a CPU (Central Processing Unit) 901, a ROM (Read Only Memory) 902, a RAM (Random Access Memory) 903, a host bus 904, a bridge 905, an external bus 906, and an interface 907. , An input device 908, an output device 909, a storage device 910, and a communication device 911.

The CPU 901 functions as an arithmetic processing device and a control device, and controls the overall operation in the information processing device 900 according to various programs. Further, the CPU 901 may be a microprocessor. The ROM 902 stores programs, calculation parameters, and the like used by the CPU 901. The RAM 903 temporarily stores a program used in the execution of the CPU 901, parameters that are appropriately changed in the execution, and the like. These are connected to each other by a host bus 904 composed of a CPU bus or the like.

The host bus 904 is connected to an external bus 906 such as a PCI (Peripheral Component Interconnect / Interface) bus via a bridge 905. It is not always necessary to separately configure the host bus 904, the bridge 905, and the external bus 906, and these functions may be implemented in one bus.

The input device 908 includes input means for the user to input information such as a mouse, keyboard, touch panel, buttons, microphone, switch, and lever, and an input control circuit that generates an input signal based on the input by the user and outputs the input signal to the CPU 901. And so on. By operating the input device 908, the user who operates the information processing device 900 can input various data to the information processing device 900 and instruct the processing operation.

The output device 909 includes, for example, a CRT (Cathode Ray Tube) display device, a liquid crystal display (LCD) device, an OLED (Organic Light Emitting Node) device, a display device such as a lamp, and an audio output device such as a speaker.

The storage device 910 is a device for storing data. The storage device 910 may include a storage medium, a recording device for recording data on the storage medium, a reading device for reading data from the storage medium, a deleting device for deleting the data recorded on the storage medium, and the like. The storage device 910 is composed of, for example, an HDD (Hard Disk Drive). The storage device 910 drives a hard disk and stores programs and various data executed by the CPU 901.

The communication device 911 is, for example, a communication interface composed of a communication device or the like for connecting to a network. Further, the communication device 911 may support either wireless communication or wired communication.

The hardware configuration example of the learning device 10 according to the embodiment of the present invention has been described above.

(3. Summary)
As described above, according to the embodiment of the present invention, there is provided a technique capable of presenting a more useful explanatory material for the judgment basis of the neural network. More specifically, according to the embodiment of the present invention, the important region is trimmed to exclude the region other than the important region, and then the feature amount is extracted, and the similarity between the feature amount and the prototype is calculated. Thereby, a similar part corresponding to the similar feature amount of the learning data from which the feature amount similar to the prototype (similar feature amount) is extracted can be presented. Therefore, the similarity between the similar example and the test data is easier to understand for each site than the whole similar example is presented.

Although the preferred embodiments of the present invention have been described in detail with reference to the accompanying drawings, the present invention is not limited to such examples. It is clear that a person having ordinary knowledge in the field of technology to which the present invention belongs can come up with various modifications or modifications within the scope of the technical ideas described in the claims. , These are also naturally understood to belong to the technical scope of the present invention.

For example, in the above description, the case where the display control unit 220 in the identification device 20 causes the display unit 230 to display information regarding a similar portion, a score, and a test data area has been mainly described. However, like the identification device 20, the learning device 10 has a similar part corresponding to the prototype of the learning data, a similarity or a value (score) corresponding to the similarity output to the inference unit 126, and inference of the learning data. Information about the area corresponding to the similarity output to the unit 126 may be displayed on the display unit 230 for each channel.

10 Learning device 110 Data set 120 Neural network 121 Input unit 122 Important area estimation unit 123 Trimming processing unit 124 Feature extraction unit 125 Similarity calculation unit 126 Inference unit 131-133 Parameters 134 Prototype 140 Evaluation unit 150 Update unit 160 Saved data 20 Identification Device 220 Display control unit 230 Display unit

Claims (18)

Input part to acquire learning data and correct answer value,
An important area estimation unit that estimates one or more important areas based on the learning data, and an important area estimation unit.
A trimming processing unit that trims the one or more important areas based on the learning data and information indicating each of the one or more important areas and outputs the one or more important areas.
A feature extraction unit that extracts features based on the one or more important regions and the first neural network, and
A similarity calculation unit that calculates and outputs the similarity between the feature amount and the prototype, and
An inference unit that outputs an inference value based on the similarity, and an inference unit.
An evaluation unit that evaluates the inferred value based on the correct answer value and obtains an evaluation result,
Based on the evaluation result, an update unit that updates the weight parameter of the first neural network and the prototype, and an update unit.
A learning device equipped with. The important region estimation unit estimates the one or a plurality of important regions based on the learning data and the second neural network.
The update unit updates the weight parameter of the second neural network based on the evaluation result.
The learning device according to claim 1. The inference unit outputs the inference value based on the similarity and the third neural network.
The update unit updates the weight parameter of the third neural network based on the evaluation result.
The learning device according to claim 1 or 2. The size of each of the one or more important regions is variable.
The learning device according to any one of claims 1 to 3. Predetermined constraints are imposed on the size of each of the one or more important areas.
The learning device according to any one of claims 1 to 4. The size of the feature amount is variable.
The learning device according to any one of claims 1 to 5. The number of channels of the feature amount is the same as the number of channels of the prototype.
The similarity calculation unit gives the inference unit the highest degree of similarity between one or more of the channel data of the feature amount and the channel data of the prototype as the degree of similarity corresponding to the channel. Output,
The learning device according to any one of claims 1 to 6. The similarity calculation unit saves the similarity output to the inference unit and the feature amount corresponding to the similarity in a part or all of the plurality of learning data as storage data for each channel.
The update unit detects a feature amount having the highest degree of similarity to the prototype as a similar feature amount from the stored data for each channel, and the region corresponding to the similar feature amount of the learning data from which the similar feature amount is extracted. Associate data with the prototype for each channel,
The learning device according to any one of claims 1 to 7. The update unit overwrites the prototype for each channel with the similar features.
The learning device according to claim 8. When the prototype is overwritten by the similar feature amount in the middle of learning, the update unit stops updating the prototype.
The learning device according to claim 9. Acquiring training data and correct answer values,
Estimating one or more important regions based on the training data,
Trimming the one or more important areas based on the learning data and the information indicating each of the one or more important areas, and outputting the one or more important areas.
Extracting features based on the one or more important regions and the first neural network,
To calculate and output the similarity between the feature amount and the prototype,
To output the inferred value based on the similarity,
To obtain the evaluation result by evaluating the inferred value based on the correct answer value,
Based on the evaluation result, the weight parameter of the first neural network and the prototype are updated, and
Learning methods, including. Computer,
Input part to acquire learning data and correct answer value,
An important area estimation unit that estimates one or more important areas based on the learning data, and an important area estimation unit.
A trimming processing unit that trims the one or more important areas based on the learning data and information indicating each of the one or more important areas and outputs the one or more important areas.
A feature extraction unit that extracts features based on the one or more important regions and the first neural network, and
A similarity calculation unit that calculates and outputs the similarity between the feature amount and the prototype, and
An inference unit that outputs an inference value based on the similarity, and an inference unit.
An evaluation unit that evaluates the inferred value based on the correct answer value and obtains an evaluation result,
Based on the evaluation result, an update unit that updates the weight parameter of the first neural network and the prototype, and an update unit.
A learning program to function as a learning device equipped with. An input unit that acquires identification data and correct answer values,
An important area estimation unit that estimates one or more important areas based on the identification data, and an important area estimation unit.
A trimming processing unit that trims the one or more important areas based on the identification data and the one or more important areas and outputs the one or more important areas.
A feature extraction unit that extracts features based on the one or more important regions and the first neural network, and
A similarity calculation unit that calculates and outputs the similarity between the feature amount and the prototype, and
An inference unit that outputs an inference value based on the similarity, and an inference unit.
A display control unit that controls so that the area data corresponding to the prototype of the training data is displayed for each channel, and
An identification device. The display control unit controls so that information about a region corresponding to the similarity output to the inference unit of the identification data is displayed for each channel.
The identification device according to claim 13. The display control unit controls so that the similarity output to the inference unit or a value corresponding to the similarity is displayed as a score for each channel.
The identification device according to claim 13 or 14. The display control unit controls so that a predetermined number of the area data are displayed in descending order of similarity output to the inference unit.
The identification device according to any one of claims 13 to 15. Acquiring identification data and correct answer value,
Estimating one or more important regions based on the identification data
To output the one or more important areas by trimming the one or more important areas based on the identification data and the one or more important areas.
Extracting features based on the one or more important regions and the first neural network,
To calculate and output the similarity between the feature amount and the prototype,
To output the inferred value based on the similarity,
Control so that the area data corresponding to the prototype of the training data is displayed for each channel, and
Identification methods, including. Computer,
An input unit that acquires identification data and correct answer values,
An important area estimation unit that estimates one or more important areas based on the identification data, and an important area estimation unit.
A trimming processing unit that trims the one or more important areas based on the identification data and the one or more important areas and outputs the one or more important areas.
A feature extraction unit that extracts features based on the one or more important regions and the first neural network, and
A similarity calculation unit that calculates and outputs the similarity between the feature amount and the prototype, and
An inference unit that outputs an inference value based on the similarity, and an inference unit.
A display control unit that controls so that the area data corresponding to the prototype of the training data is displayed for each channel, and
An identification program for functioning as an identification device.

Images