JP6991960B2 - Image recognition device, image recognition method and program - Google Patents

Description

The present invention relates to an image recognition device, an image recognition method and a program, and more particularly to a technique for reducing the load of image recognition processing.

In recent years, a technique for recognizing a class of an object from an image using deep learning, which is a kind of neural network, has been put into practical use. Among such techniques, the structure of a neural network containing more layers has been proposed in order to improve the recognition accuracy. In a neural network, more sophisticated and complex features can be extracted by stacking layers. Therefore, deepening the layer plays an important role in improving the recognition accuracy of machine learning models using neural networks.

On the other hand, as the layer of the neural network becomes deeper, the amount of calculation at the time of executing the recognition process increases, and the computing power required to execute the recognition process tends to increase. For this reason, it may be difficult to execute a machine learning model with a deep neural network layer in a device having a relatively small computational resource such as an IoT (Internet Of Things) device.

In order to deal with this problem, for example, in Non-Patent Document 1, a technique is proposed in which output layers are provided in a plurality of layers constituting a machine learning model, and an output layer is selected according to fluctuations in computational resources to perform image recognition. Has been done.

Gao Huang, et al., Multi-Scale Dense Convolutional Networks for Resource Efficient Image Classification, International Conference on Learning Representations (ICLR) 2018.

In the technique according to Non-Patent Document 1, the recognition accuracy decreases as the calculation for recognition is completed in the output layer of the shallow layer (that is, the layer close to the input layer) of the neural network. Further, since the neural network of the technique according to Non-Patent Document 1 has a layered structure that spreads two-dimensionally, the size of the machine learning model becomes large and learning may become difficult. As described above, in the image recognition processing using the machine learning model of the neural network, there is room for improvement in reducing the processing load.

Therefore, the present invention has been made in view of these points, and an object of the present invention is to provide a technique for reducing a processing load in an image recognition process using a machine learning model of a neural network.

The first aspect of the present invention is an image recognition device. This device is a machine learning model that includes a plurality of filters as model parameters, and information indicating which of a plurality of predetermined recognition targets the subject included in the image data to be processed is the recognition target. To recognize which subject group the subject included in the image data belongs to among a plurality of predetermined subject groups among the plurality of filters and the model acquisition unit that acquires the machine learning model that outputs the data. By applying the pre-stage filter group to the image data, the group recognition unit that identifies the subject group, the application result of the pre-stage filter group, and the filter group excluding the pre-stage filter group from the plurality of filters. Based on the application result of a certain latter stage filter group, the individual recognition unit that specifies the recognition target included in the image data, the subject group specified by the group recognition unit, and each filter constituting the latter stage filter group are set. A filter selection unit for selecting a filter to be applied to the individual recognition unit among the subsequent filter groups is provided based on at least the priority.

The filter selection unit selects filters to be applied to the individual recognition unit in descending order of priority set for each filter constituting the subsequent filter group within a range allowed by the calculation resource of the image recognition device. May be good.

The image recognition device has a weight index calculation unit that calculates an index indicating the magnitude of the weight coefficient of each of the plurality of filters, and a filter that indicates that the weight coefficient is large by the index has a small weight coefficient. It may be further provided with a priority setting unit for setting the priority higher than the filter showing the above.

The image recognition device has a similarity calculation unit that calculates the similarity with other filters for each of the plurality of filters, and a filter that does not have other similar filters has a filter that has other similar filters. Further may be provided with a priority setting unit for setting the priority higher.

The priority setting unit may change the priority set for each filter constituting the subsequent filter group for each of the plurality of subject groups.

When the equal priority is set for each of the two or more filters constituting the subsequent filter group, the filter selection unit may randomly select a filter from the filters set with the same priority.

The image recognition device is based on learning data in which a plurality of image data, a subject included in each of the plurality of image data, and a subject group of the subject are associated with each other, and the machine is subjected to machine learning by machine learning using a neural network. A learning unit that generates a learning model may be further provided, and the learning unit may include a pre-stage learning unit that generates the pre-stage filter group, and a pre-stage learning unit that generates the pre-stage filter group and then moves to the pre-stage filter group. It may be provided with a post-stage learning unit that generates the post-stage filter group by using the application result of the pre-stage filter group without performing the error back propagation.

The second aspect of the present invention is an image recognition method. In this method, the processor is a machine learning model that includes a plurality of filters as model parameters, and which of the plurality of predetermined recognition targets the subject included in the image data to be processed is the recognition target. A step of acquiring a machine learning model that outputs information indicating the above, and recognition of which subject group among the plurality of filters the subject included in the image data belongs to a plurality of predetermined subject groups. By applying the pre-stage filter group for the purpose to the image data, a step of specifying the subject group, the specified subject group, and a post-stage filter which is a filter group excluding the pre-stage filter group from the plurality of filters. A step of selecting one or more filters from the filters constituting the latter-stage filter group, an application result of the first-stage filter group, and selection based on at least the priority set for each filter constituting the group. Based on the application result of the filter, the step of specifying the recognition target included in the image data is executed.

A third aspect of the present invention is a program. This program is a machine learning model that includes multiple filters as model parameters in a computer, and which of the plurality of predetermined recognition targets the subject included in the image data to be processed is the recognition target. A function to acquire a machine learning model that outputs information indicating the above, and recognition of which subject group the subject included in the image data belongs to among a plurality of predetermined subject groups among the plurality of filters. By applying the pre-stage filter group for the purpose to the image data, the function of specifying the subject group, the specified subject group, and the post-stage filter which is a filter group excluding the pre-stage filter group from the plurality of filters. A function to select one or more filters from the filters constituting the latter-stage filter group based on at least the priority set for each filter constituting the group, an application result of the first-stage filter group, and selection. Based on the application result of the filter, the function of specifying the recognition target included in the image data is realized.

According to the present invention, it is possible to reduce the processing load in the image recognition processing using the machine learning model of the neural network.

It is a figure which shows typically the general functional structure of a convolutional neural network. It is a figure which shows typically the layer structure of the machine learning model of the neural network known as AlexNet. It is a figure which shows typically the layer structure of the convolutional neural network used by the image recognition apparatus which concerns on embodiment. It is a figure for demonstrating the learning process of the neural network which concerns on embodiment. It is a figure for demonstrating the recognition process of the neural network which concerns on embodiment. It is a figure which shows typically the functional structure of the image recognition apparatus which concerns on embodiment. It is a figure which shows typically the priority of the filter for each subject group in a tabular form. It is a figure which shows typically the functional structure of the learning part which concerns on embodiment. It is a flowchart for demonstrating the flow of the image recognition processing executed by the image recognition apparatus which concerns on embodiment.

<Convolutional neural network>
The image recognition device according to the embodiment is a device for executing image recognition processing using a machine learning model of a neural network. The image recognition device according to the embodiment uses a machine learning model of a convolutional neural network (CNN) as a main example. Therefore, as a prerequisite technique of the information processing apparatus according to the embodiment, first, a convolutional neural network will be briefly described.

FIG. 1 is a diagram schematically showing a general functional configuration of a convolutional neural network. Currently, neural networks with various configurations have been proposed, but these basic configurations are common. The basic configuration of a neural network is represented by a superposition (or graph structure) of a plurality of types of layers. The neural network learns the model parameters so that the output result for the input data is an appropriate value. In other words, the neural network learns the model parameters to minimize the loss function defined so that the output result for the input data is of appropriate value.

FIG. 1 shows a machine learning model trained to output the types of subjects included in the input image I. In the example shown in FIG. 1, the input image I input to the input layer Li is processed in the order of the first convolution layer C1 and the second convolution layer C2, and the pooling layer P, the first fully connected layer F1, and the second. It is configured to reach the fully connected layer F2 and the output layer Lo. The output layer outputs an identification label B indicating the type of the subject included in the input image I.

For example, when the machine learning model shown in FIG. 1 is a machine learning model for recognizing a plurality of animals such as dogs, cats, and monkeys, an identification label B for identifying an animal to be identified is assigned in advance. There is. When the input image I is input to the input layer Li of the machine learning model, the output layer Lo outputs an identification label B indicating which of the plurality of predetermined recognition targets is the recognition target. The identification label B is, for example, a bit string uniquely assigned to each of the plurality of recognition targets.

In a neural network, the output of the front layer becomes the input of the rear layer adjacent to the front layer. Each convolutional layer in the convolutional neural network applies a filter to the signal input from the previous layer, and the output of the filter becomes the output of that layer.

FIG. 2 is a diagram schematically showing the layer structure of a machine learning model of a neural network known as AlexNet. As shown in FIG. 2, AlexNet has an input layer Li and five folding layers (first folding layer C1, second folding layer C2, third folding layer C3, fourth folding layer C4, and a second folding layer. 5 convolution layer C5), two fully connected layers (first fully connected layer F1 and second fully connected layer F2), and an output layer Lo, and the final layer outputs 1000 kinds of identification labels B. It is configured in. That is, AlexNet is a convolutional neural network for recognizing 1000 types of recognition targets.

Although not shown, a net structure with a deeper layer has been proposed in order to improve recognition accuracy. For example, a machine learning model of a neural network known as ResNet has a layered structure consisting of 152 layers. In a neural network, it is possible to extract more sophisticated and complicated features by stacking layers, so it is considered that deepening the layers plays an important role in improving recognition accuracy.

<Neural network according to the embodiment>
FIG. 3 is a diagram schematically showing the layer structure of the convolutional neural network N used by the image recognition device according to the embodiment. Hereinafter, the neural network N according to the embodiment will be described with reference to FIG.

The machine learning model of the convolutional neural network N used by the image recognition device according to the embodiment includes a plurality of convolutional layers as in the conventional convolutional neural network, and the subject included in the input image I is a plurality of recognition targets. Of these, which recognition target is output. Therefore, the machine learning model of the convolutional neural network N used by the image recognition device 1 according to the embodiment includes a plurality of filters as model parameters.

In a convolutional neural network, more sophisticated and complicated features can be extracted by stacking layers, so that even similar recognition objects that are difficult to distinguish can be recognized by deepening the layers. On the contrary, if the recognition targets are significantly different from each other, it can be recognized even if the number of layers of the convolutional neural network is small. These events suggest that the large feature of the recognition target is recognized in the front part of the convolutional neural network, and the detailed feature peculiar to each recognition target is recognized in the latter part. A similar phenomenon can be realized by changing the number of filters in each layer. That is, by increasing the number of filters in each layer, even similar recognition targets that are difficult to distinguish can be recognized. On the contrary, if the recognition targets are significantly different from each other, recognition can be performed even if the number of filters in each layer is small.

Therefore, the filters that make up the machine learning model are those that contribute to the recognition of all recognition targets and those that play an important role in recognizing one recognition target but contribute less to the recognition of another recognition target. It is considered that there is a filter that does not. The former filter includes a filter for recognizing a feature commonly included in a plurality of recognition targets, and the latter filter includes a filter for distinguishing a specific recognition target having similar features.

For example, recognition targets of a machine learning model include mammals such as cats, dogs, monkeys, or cows, plants such as mandarins, carrots, radishes, or cabbage, and artificial objects such as automobiles and buildings. And. In this case, for example, a filter that strongly reacts to the "eyes" of mammals is considered to be a filter for capturing a large feature of whether or not the recognition target is a mammal, and thus is considered to be a filter that contributes to the recognition of all recognition targets. ..

On the other hand, for example, the filter used to distinguish between "dog" and "cat" is considered to greatly contribute to recognition when the recognition target is a dog or a cat, but the recognition target is a cat or a dog. If it is a "cat" or "automobile" other than the above, it is considered that it does not contribute much to recognition. That is, it is considered that there is a filter that plays an important role in recognizing a certain recognition target but does not contribute much to the recognition of another recognition target.

As shown in FIG. 3, the machine learning model of the convolutional neural network N used by the image recognition device according to the embodiment includes two filter groups, a front-stage filter group and a rear-stage filter group. In the example shown in FIG. 3, the front-stage filter group consists of five folding layers (first front-stage folding layer C _f 1, second front-stage folding layer C _f 2, third front-stage folding layer C _f 3, fourth. It is a filter included in the pre-stage convolutional layer C _f4 and the fifth pre-stage convolutional layer C _f5 ) and two pre-bonding layers (first fully-bonded layer F1 and second fully-bonded layer F2). In addition, the posterior filter group consists of five convolutional layers (first posterior convolutional layer _Cr 1, second posterior convolutional layer _Cr 2, third posterior convolutional layer _Cr 3, and fourth posterior convolutional layer _Cr 3. 4 and the filter included in the 5th post-stage convolutional layer _Cr 5).

For example, when the present embodiment is applied to AlexNet shown in FIG. 2, the filter provided in each layer is equally divided into a front-stage filter group and a rear-stage filter group. Specifically, as the neural network structure, the first convolution layer of the front-stage filter group and the rear-stage filter group is a (55 × 55) node × 48 filter. Similarly, for the second convolutional layer to the fifth convolutional layer, a (27 × 27) node × 128 filter, a (13 × 13) node × 192 filter, and a (13 × 13) node × 192 filter, respectively. 13 x 13) Node x 128 filter. Further, the first fully connected layer F1, the second fully connected layer F2, the third fully connected layer F3, and the fourth fully connected layer F4 all have 2048 nodes.

[Learning process]
First, the learning process of the neural network N according to the embodiment will be described.

4 (a)-(c) are diagrams for explaining the learning process of the neural network N according to the embodiment. Specifically, FIGS. 4 (a) and 4 (b) show the first-stage learning and the second-stage learning of the first-stage filter group, respectively, and FIG. 4 (c) shows the learning of the second-stage filter group. ing. Here, the pre-stage filter group is a filter group used for recognizing which subject group among a plurality of predetermined subject groups the subject included in the input image I belongs to. The latter-stage filter group is a filter group excluding the filters constituting the first-stage filter from the filters constituting the machine learning model.

As shown in FIG. 4A, in the first stage learning of the pre-stage filter group, when the learning data which is the image data for learning is input, the identification label B of the subject associated with the image data is output. The first pre-stage filter group is generated. That is, the first-stage learning of the pre-stage filter group is the same as the normal machine learning process.

As shown in FIG. 4B, in the second stage learning of the pre-stage filter group, when the training data is input, a machine learning model that outputs a group identification label indicating the subject group to which the image data belongs is generated. do. Specifically, by fine-tuning the first fully connected layer F1 and the second fully connected layer F2 included in the first-stage filter group, a second first-stage filter group that recognizes the group to which the image data belongs is generated.

In FIG. 4B, the first fully bonded layer F1 and the second fully bonded layer F2 after fine tuning are described as the first fully bonded layer F1'and the second fully bonded layer F2', respectively. Therefore, the convolutional layer of the first pre-stage filter group and the convolutional layer of the second pre-stage filter group are common. Since most of the first pre-stage filter group and the second pre-stage filter group are common, they are simply referred to as the pre-stage filter group unless the two are particularly distinguished.

As shown in FIG. 4C, the learning of the latter-stage filter is learned together with the machine learning model generated in the first learning stage of the first-stage filter group. That is, the rear filter group is learned so as to output the identification label B of the training data by using the output of the first front filter group and the output of the rear filter group. At the time of learning the latter-stage filter group, error back propagation is not performed to the first-stage filter group, and the latter-stage filter group is learned using the application result of the first-stage filter group.

[Recognition process]
Subsequently, the learning process of the neural network N according to the embodiment will be described.

5 (a)-(b) are diagrams for explaining the recognition process of the neural network N according to the embodiment. The recognition process of the neural network N according to the embodiment is composed of two stages, a group recognition stage for recognizing the group to which the subject included in the input image I belongs and a subject recognition stage for recognizing the subject itself. ing.

FIG. 5A is a diagram for explaining group recognition of the input image I. Group recognition is performed using the second pre-stage filter group generated in the second-stage learning of the pre-stage filter group. By applying the second pre-stage filter group to the input image I, the subject group to which the subject included in the input image I belongs is recognized.

The image recognition device according to the embodiment can be expected to recognize the recognition target with the highest recognition accuracy by applying all the filters included in the subsequent filter group. However, it is possible to maintain a constant recognition accuracy without applying all the filters included in the subsequent filter group. Therefore, in the image recognition device, for example, the calculation resource of the image recognition device is originally small, or even if the calculation resource is large, the processing load of other operations is large and the calculation resource temporarily allocated to the application of the machine learning model is small. In this case, the feasibility of the calculation and the recognition accuracy can be balanced by selecting the post-stage filter to be applied according to the calculation resource.

Although the details will be described later, the image recognition device according to the embodiment selects a filter to be actually applied from the filters constituting the subsequent filter group based on the subject group to which the subject included in the input image I belongs. Then, as shown in FIG. 5B, the output of the first front-stage filter group and the output of the selected rear-stage filter group are combined to output the identification label B indicating the subject included in the input image I. Ru.

Here, if the amount of increase in the amount of calculation required for group recognition of the input image I exceeds the amount of reduction in the amount of calculation due to the selection of the filters in the subsequent filter group, the filters in the subsequent filter group are selected. meaningless. However, in the neural network N according to the embodiment, the convolutional layer of the first pre-stage filter group and the convolutional layer of the second pre-stage filter group are common. Therefore, the calculation result of the convolutional layer of the second pre-stage filter group executed for recognizing the subject group to which the subject included in the input image I belongs can be diverted to the calculation of the convolutional layer of the first pre-stage filter group. .. As a result, in the recognition process of the neural network N according to the embodiment, the calculation cost that increases for recognizing the subject group to which the subject included in the input image I belongs is substantially the calculation of the first fully connected layer F1'. And only the operation of the second fully connected layer F2'. Therefore, it can be expected that the amount of increase in the amount of calculation required for group recognition of the input image I is sufficiently smaller than the amount of decrease in the amount of calculation due to the selection of filters in the subsequent filter group.

As described above, the image recognition device according to the embodiment can reduce the processing load of the recognition process by selecting the filter applied to the input image I.

<Functional configuration of the image recognition device 1 according to the embodiment>
FIG. 6 is a diagram schematically showing a functional configuration of the image recognition device 1 according to the embodiment. The image recognition device 1 includes a storage unit 2 and a control unit 3. In FIG. 6, the arrows indicate the main data flows, and there may be data flows not shown in FIG. In FIG. 6, each functional block shows not a hardware (device) unit configuration but a functional unit configuration. Therefore, the functional block shown in FIG. 6 may be mounted in a single device, or may be mounted in a plurality of devices separately. Data can be exchanged between functional blocks via any means such as a data bus, a network, and a portable storage medium.

The storage unit 2 includes a ROM (Read Only Memory) that stores a BIOS (Basic Input Output System) of a computer that realizes the image recognition device 1, a RAM (Random Access Memory) that is a work area of the image recognition device 1, and an OS (OS). It is a large-capacity storage device such as an HDD (Hard Disk Drive) or SSD (Solid State Drive) that stores an Operating System), an application program, and various information referred to when the application program is executed.

The control unit 3 is a processor such as a CPU (Central Processing Unit) or GPU (Graphics Processing Unit) of the image recognition device 1, and the image acquisition unit 30 and the model acquisition unit are executed by executing a program stored in the storage unit 2. It functions as 31, a group recognition unit 32, a filter selection unit 33, an individual recognition unit 34, a resource acquisition unit 35, a weight index calculation unit 36, a priority setting unit 37, a similarity calculation unit 38, and a learning unit 39.

Note that FIG. 6 shows an example in which the image recognition device 1 is composed of a single device. However, the image recognition device 1 may be realized by a plurality of processors, memory, and other computational resources, such as a cloud computing system. In this case, each unit constituting the control unit 3 is realized by executing a program by at least one of a plurality of different processors.

The image acquisition unit 30 acquires the input image I, which is the image data to be processed by the image recognition device 1. The model acquisition unit 31 acquires a machine learning model including a plurality of filters as model parameters. In the machine learning model acquired by the model acquisition unit 31 according to the embodiment, which of the plurality of predetermined recognition targets the subject included in the input image I, which is the image data to be processed, is the recognition target. It is a machine learning model that outputs information indicating.

The group recognition unit 32 is a pre-stage filter group for recognizing which subject group the subject included in the input image I belongs to among the plurality of predetermined subject groups among the plurality of filters included in the learning model. (That is, the above-mentioned second pre-stage filter group) is applied to the input image I to specify the subject group.

The filter selection unit 33 selects one or more filters from the latter-stage filter group, which is a filter group excluding the first-stage filter group, from among the plurality of filters included in the machine learning model. Here, the filter selection unit 33 selects a filter based on at least the subject group specified by the group recognition unit 32 and the priority set for each filter constituting the subsequent filter group. Details of filter selection by the filter selection unit 33 will be described later.

The individual recognition unit 34 identifies the recognition target included in the input image I based on the application result of the first pre-stage filter group and the result of applying the filter selected by the filter selection unit 33 to the input image I. In this way, the image recognition device 1 selects a filter that is actually applied to the input image I from the post-stage filter group provided in the machine learning model. As a result, the image recognition device 1 can reduce the processing load in the image recognition process as compared with the case where all the filters included in the machine learning model are used.

The resource acquisition unit 35 acquires the calculation resource of the image recognition device 1. Here, the "calculation resource" acquired by the resource acquisition unit 35 is image recognition by the image recognition device 1 such as the power of a processor such as a CPU and a GPU included in the image recognition device 1 and the capacity of the main storage device included in the image recognition device 1. Computational power that can be assigned to processing. The calculation resource of the image recognition device 1 differs depending on the type of the image recognition device 1, for example, when the image recognition device 1 is a blade server and when it is an IoT device. Further, even in the same image recognition device 1, the calculation resources that can be allocated to the image recognition process change depending on the size of the calculation resources allocated to other processes that are executed in parallel during the image recognition process. sell. The resource acquisition unit 35 acquires computational resources that can be used when the image recognition device 1 performs image recognition processing using a machine learning model.

The filter selection unit 33 selects filters to be applied to the individual recognition unit 34 in descending order of priority set in each filter constituting the subsequent filter group within the range allowed by the calculation resource of the image recognition device 1. Generally, the larger the number of filters applied in the image recognition process, the higher the recognition accuracy can be expected. The image recognition device 1 performs image recognition processing that can be expected to have the highest recognition accuracy within the range that the image recognition device 1 can execute by selecting a filter of a post-stage filter group to be applied according to the calculation resource of the image recognition device 1. Can be executed.

[Filter selection process]
Subsequently, the filter selection process in the image recognition device 1 according to the embodiment will be described.

As described above, in the neural network, the output of the front layer becomes the input of the rear layer adjacent to the front layer. Each convolutional layer in the convolutional neural network applies a filter to the signal input from the previous layer, and the output of the filter becomes the output of that layer. Therefore, the larger the absolute value of the weighting factor of the filter in the convolutional layer, the larger the absolute value of the input signal of the next layer can be.

That is, the magnitude of the weighting factor of the filter of one convolutional layer can be an index value of the activity of the corresponding unit in the next layer. In a neural network, it is said that among the units constituting the layer, the unit having a high activity contributes more to the cognitive ability than the unit having a low activity.

Therefore, the weight index calculation unit 36 calculates an index indicating the magnitude of the weight coefficient of each of the plurality of filters included in the subsequent filter group. Here, the "index indicating the magnitude of the weighting coefficient" is, for example, a value obtained by dividing the sum of the absolute values of the weighting coefficients of the filter by the number of weighting coefficients of the filter. Alternatively, it may be a value obtained by dividing the sum of the squares of the weighting coefficients of the filter by the number of weighting coefficients of the filter. In any case, the larger the index indicating the magnitude of the weighting coefficient of the filter, the larger the weighting coefficient included in the filter.

The priority setting unit 37 sets the priority of the filter indicating that the weighting coefficient is large by the index calculated by the weighting index calculating unit 36 to be higher than that of the filter indicating that the weighting coefficient is small. As a result, the image recognition device 1 can preferentially select a filter that is considered to have a large contribution to the recognition ability of the machine learning model.

Here, the priority stage set for each filter included in the subsequent filter group in which the priority setting unit 37 is located is arbitrary up to the number of filters included in the latter filter group. For example, if the priority level is the same as the number of filters, the filters contained in the convolutional layer can be ordered using the priority.

Alternatively, when the number of filters is two or more, the priority stage may be two stages of "high" and "low". In this case, the priority setting unit 37 sets a predetermined threshold value A, sets the priority to "high" when the index indicating the magnitude of the weighting factor of each filter exceeds the threshold value A, and sets the priority to "high" when the index indicates the magnitude of the weighting factor of each filter is less than the threshold value A. The priority may be set to "low".

Further, the priority setting unit 37 may change the priority set for each filter based on the similarity between the filters included in the subsequent filter group. In general, the closer the weighting factors of two filters are, the more the two filters are considered to extract similar features. Therefore, when two filters are similar, if one of the filters extracts the features, it is considered that the change in the final recognition accuracy is small even if the other filter is not used. Conversely, a filter for which no other similar filter exists may be able to extract features that the filter cannot extract with other filters.

Therefore, the similarity calculation unit 38 calculates the similarity with other filters for each of the plurality of filters included in the subsequent filter group. The priority setting unit 37 sets the priority of the filter having no other similar filter higher than that of the filter having other similar filters. Thereby, the image recognition device 1 can select a filter for extracting different features within the range allowed by the calculation resource of the image recognition device 1.

Here, the similarity calculation unit 38 may calculate the "distance" between the filters as the similarity between the filters. The "distance" between the filters calculated by the similarity calculation unit 38 may be any quantity as long as the axiom of the distance is satisfied, and is, for example, the Euclidean distance between the filters. Specifically, the similarity calculation unit 38 calculates the similarity D (i, j) between the i-filter and the j-filter using the following equation (1).

ここで、Ｉ（ｍ，ｎ，ｆ）は、３次元の第ｉフィルタの縦ｍ、横ｎ、高さｆにおける要素であり、Ｊ（ｍ，ｎ，ｆ）は、第ｊフィルタの縦ｍ、横ｎ、高さｆにおける要素である。式（１）は、２つのフィルタのユークリッド距離を、フィルタの要素数（重み係数の数）で正規化した量であることを示している。ある２つのフィルタ間の非類似度Ｄの値が小さいほど、そのフィルタ同士は類似していることを示している。この他にも、類似度算出部３８は、例えばコサイン類似度を用いてフィルタ間の類似度を算出してもよい。

Here, I (m, n, f) is an element in the vertical m, the horizontal n, and the height f of the three-dimensional i-filter, and J (m, n, f) is the vertical m of the j-th filter. , Lateral n, height f. Equation (1) shows that the Euclidean distance between the two filters is a quantity normalized by the number of elements of the filter (the number of weighting coefficients). The smaller the value of the degree of dissimilarity D between two filters, the more similar the filters are. In addition to this, the similarity calculation unit 38 may calculate the similarity between filters using, for example, the cosine similarity.

The similarity calculation unit 38 calculates S (i) = ΣD (i, j) (j = 1, ..., Nf; Nf is the number of filters) as the similarity S (i) of the i-filter. , Higher priority may be assigned to filters with higher values (ie no other similar filters).

Similar to the weight index calculation unit 36, the priority setting unit 37 may also have two levels of similarity, “similar” and “dissimilar”. In this case, the priority setting unit 37 may set a predetermined threshold value B, and if the similarity between the filters exceeds the threshold value B, it may be regarded as “similar”, and if it is less than the threshold value B, it may be regarded as “dissimilar”. .. Further, the priority setting unit 37 may obtain the similarity between the filters for the filters having the lower priority when the weight index calculation unit 36 sets the priority to two stages. In this case, the priority setting unit 37 may set the priority of the filter that is dissimilar to any of the filters to “high”.

As described above, it is considered that there are filters that make up a machine learning model that play an important role in recognizing one recognition target but do not contribute much to the recognition of another recognition target. .. Therefore, the importance of the filter for recognizing the subject may change depending on the type of the subject.

Therefore, the priority setting unit 37 changes the priority set for each filter constituting the subsequent filter group for each of a plurality of predetermined subject groups. Hereinafter, the setting of the priority of the filter for each subject group executed by the priority setting unit 37 will be specifically described.

Now, it is assumed that the type of the subject group is P type (P is an integer of 2 or more) and the number of filters included in the subsequent filter group is Q (Q is an integer of 2 or more). The qth filter is the filter f _q (1 ≦ q ≦ Q), and the order of importance of the filter f _q in the pth group (1 ≦ p ≦ P) is W _pq .

Step 1: The priority setting unit 37 sets p to 1.
Step 2: The priority setting unit 37 acquires the test data Tp of the _p -th subject group.
Step 3: The priority setting unit 37 applies the front-stage filter group and all the rear-stage filter groups to the test data _{Tp, and calculates the recognition rate R p .}

Step 4: The priority setting unit 37 sets q to 1.
Step 5: The priority setting unit 37 calculates the recognition rate R _pq of the test data T _p when the filter f _q is excluded from the filters included in the subsequent filter group and applied.
Step 6: The priority setting unit 37 calculates a reduction amount C _q of the recognition rate, which is a value obtained by subtracting the recognition rate R _pq from the recognition rate R _p . The amount of decrease C _q indicates the amount of decrease in the recognition rate due to the exclusion of the filter f _q . That is, it shows the contribution of the filter f _q to the recognition rate.

Step 7: The priority setting unit 37 updates the value of q to q + 1.
Step 8: The priority setting unit 37 repeats the processes of steps 4 and 5 until q exceeds Q.
Step 9: The priority setting unit 37 sorts the Q reduction amounts C _q in descending order. At this time, the order of the subscripts q of the reduction amount C _q is the important order W _pq of the filter f _q in the p-th subject group.

Step 10: The priority setting unit 37 updates the value of p to p + 1.
Stap 11: The priority setting unit 37 repeats the processes from step 2 to step 8 until p exceeds P.

By the above processing, the priority setting unit 37 can obtain the order of importance of each filter constituting the subsequent filter group for each predetermined subject group. The priority setting unit 37 raises the priority of the filter having higher importance for each predetermined subject group. As a result, the priority setting unit 37 can change the priority set for each filter constituting the subsequent filter group for each of a plurality of predetermined subject groups.

As with the similarity of the filter, the priority setting unit 37 may have two levels of importance or not. In this case, the priority setting unit 37 sets a predetermined threshold value C, the filter f _q is important when the recognition rate R _pq is smaller than the threshold value C, and the filter f _q is important when the recognition rate R _pq is equal to or higher than the threshold value C. Should not be important.

FIG. 7 is a diagram schematically showing the priority of the filter for each subject group in a table format. Specifically, FIG. 7 shows the data structure of a priority database that stores the magnitude, similarity, order of importance, and priorities of the weighting factors for each filter for the first subject group. The priority database is stored in the storage unit 2 and managed by the priority setting unit 37. By referring to the priority database, the filter selection unit 33 informs the individual recognition unit 34 of the latter-stage filter group based on at least the subject group and the priority set for each filter constituting the latter-stage filter group. You can select the filter to apply.

Here, when the priority setting unit 37 sets the priority level set for each filter included in the subsequent filter group to less than the number of filters, it is possible that a plurality of filters have the same priority. Therefore, when the same priority is set for each of the two or more filters constituting the previous stage filter group, the filter selection unit 33 may randomly select a filter from the filters set with the same priority. As a result, the filter selection unit 33 can select the filter without referring to an index other than the priority.

The learning unit 39 generates a machine learning model by machine learning using a neural network based on learning data in which a plurality of image data, a subject included in each of the plurality of image data, and a subject group of the subject are associated with each other. do.

FIG. 8 is a diagram schematically showing the functional configuration of the learning unit 39 according to the embodiment. As shown in FIG. 8, the learning unit 39 includes a front-stage learning unit 390 and a rear-stage learning unit 391.

First, the image acquisition unit 30 acquires a plurality of image data in which the subject included in the image data and the subject group to which the subject belongs are known. The pre-stage learning unit 390 uses a plurality of image data acquired by the image acquisition unit 30 as training data, and as described with reference to FIGS. 4 (a)-(b), the pre-stage filter is performed by machine learning using a neural network. Generate a swarm.

As described with reference to FIG. 4C, the rear-stage learning unit 391 applies the front-stage filter group without performing error back propagation to the front-stage filter group after the front-stage learning unit 390 generates the front-stage filter group. The result is used to generate a group of subsequent filters.

Specifically, the latter-stage learning unit 391 updates the weights of the filters constituting each layer by back-propagating the error between the output of the output layer Lo and the identification label B associated with the input image I. At this time, the rear-stage learning unit 391 includes a first front-stage folding layer C _f 1, a second front-stage folding layer C _f 2, a third front-stage folding layer C _f 3, a fourth front-stage folding layer C _f 4, and the like. The weight of the filter included in the fifth front convolution layer C _f 5 is fixed, and its update is prohibited. As a result, the rear-stage learning unit 391 can generate the rear-stage filter group while keeping the front-stage filter group fixed.

<Processing flow of the image recognition method executed by the image recognition device 1>
FIG. 9 is a flowchart for explaining the flow of the image recognition process executed by the image recognition device 1 according to the embodiment. The process in this flowchart starts, for example, when the image recognition device 1 is activated.

The model acquisition unit 31 is a machine learning model that includes a plurality of filters as model parameters, and recognizes the subject included in the input image I, which is the image data to be processed, among a plurality of predetermined recognition targets. Acquire a machine learning model that outputs information indicating whether or not it is a target (S2).

The group recognition unit 32 uses the input image I as a pre-stage filter group for recognizing which of the plurality of predetermined subject groups the subject included in the input image I belongs to among the plurality of filters. By applying, the subject group is specified (S4).

The filter selection unit 33 is one or more of the filters constituting the subsequent filter group based on at least the subject group specified by the model acquisition unit 31 and the priority set for each filter constituting the subsequent filter group. Select the filter of (S6).

The individual recognition unit 34 identifies the recognition target included in the input image I based on the application result of the previous stage filter group and the application result of the selected filter (S8).

<Effects of the image recognition device 1 according to the embodiment>
As described above, according to the image recognition device 1 according to the embodiment, it is possible to reduce the processing load in the image recognition processing using the machine learning model of the neural network.

Although the present invention has been described above using the embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments, and various modifications and changes can be made within the scope of the gist. be. For example, the specific embodiment of the distribution / integration of the device is not limited to the above embodiment, and all or a part thereof may be functionally or physically distributed / integrated in any unit. Can be done. Also included in the embodiments of the present invention are new embodiments resulting from any combination of the plurality of embodiments. The effect of the new embodiment produced by the combination has the effect of the original embodiment together.

1 ... Image recognition device 2 ... Storage unit 3 ... Control unit 30 ... Image acquisition unit 31 ... Model acquisition unit 32 ... Group recognition unit 33 ... Filter selection unit 34 ... -Individual recognition unit 35 ... Resource acquisition unit 36 ... Weight index calculation unit 37 ... Priority setting unit 38 ... Similarity calculation unit 39 ... Learning unit 390 ... Pre-stage learning unit 391.・ ・ Post-stage learning unit N ・ ・ ・ Convolutional neural network

Claims (9)

A machine learning model that includes a plurality of filters as model parameters, and outputs information indicating which of a plurality of predetermined recognition targets the subject included in the image data to be processed is. The model acquisition unit that acquires the learning model, and
By applying to the image data a pre-stage filter group for recognizing which of the plurality of predetermined subject groups the subject included in the image data belongs to among the plurality of filters. A group recognition unit that identifies the subject group and
Individual recognition that identifies the recognition target included in the image data based on the application result of the pre-stage filter group and the application result of the post-stage filter group, which is a filter group excluding the pre-stage filter group among the plurality of filters. Department and
A filter to be applied to the individual recognition unit among the rear filter groups is selected based on at least the subject group specified by the group recognition unit and the priority set for each filter constituting the rear filter group. Filter selection part and
Image recognition device. The filter selection unit selects filters to be applied to the individual recognition unit in descending order of priority set for each filter constituting the subsequent filter group within a range allowed by the calculation resource of the image recognition device.
The image recognition device according to claim 1. A weight index calculation unit that calculates an index indicating the magnitude of the weight coefficient of each of the plurality of filters, and a weight index calculation unit.
The filter indicating that the weighting factor is large by the index includes a priority setting unit that sets the priority higher than the filter indicating that the weighting factor is small.
The image recognition device according to claim 1 or 2, further comprising. A similarity calculation unit that calculates the similarity with other filters for each of the plurality of filters,
A filter having no other similar filter further includes a priority setting unit for setting the priority higher than that of a filter having another similar filter.
The image recognition device according to any one of claims 1 to 3. The priority setting unit changes the priority set for each filter constituting the subsequent filter group for each of the plurality of subject groups.
The image recognition device according to claim 3 or 4. When the equal priority is set for each of the two or more filters constituting the subsequent filter group, the filter selection unit randomly selects a filter from the filters set with the same priority.
The image recognition device according to any one of claims 1 to 5. Learning to generate the machine learning model by machine learning using a neural network based on learning data in which a plurality of image data, a subject included in each of the plurality of image data, and a subject group of the subject are associated with each other. With more parts,
The learning unit
The pre-stage learning unit that generates the pre-stage filter group and
After the pre-stage learning unit generates the pre-stage filter group, the post-stage learning unit that generates the post-stage filter group using the application result of the pre-stage filter group without performing error back propagation to the pre-stage filter group. Prepare, prepare
The image recognition device according to any one of claims 1 to 6. The processor,
A machine learning model that includes a plurality of filters as model parameters, and outputs information indicating which of a plurality of predetermined recognition targets the subject included in the image data to be processed is. The steps to get the learning model and
By applying to the image data a pre-stage filter group for recognizing which of the plurality of predetermined subject groups the subject included in the image data belongs to among the plurality of filters. The step of identifying the subject group and
Based on at least the specified subject group and the priority set for each filter constituting the rear filter group, which is a filter group excluding the front filter group among the plurality of filters, the rear filter group is provided. A step to select one or more filters from the constituent filters, and
A step of specifying a recognition target included in the image data based on the application result of the previous stage filter group and the application result of the selected filter.
Image recognition method to perform. On the computer
A machine learning model that includes a plurality of filters as model parameters, and outputs information indicating which of a plurality of predetermined recognition targets the subject included in the image data to be processed is. With the ability to get a learning model,
By applying to the image data a pre-stage filter group for recognizing which of the plurality of predetermined subject groups the subject included in the image data belongs to among the plurality of filters. The function to specify the subject group and
Based on at least the specified subject group and the priority set for each filter constituting the rear filter group, which is a filter group excluding the front filter group among the plurality of filters, the rear filter group is provided. A function to select one or more filters from the constituent filters, and
A function of specifying a recognition target included in the image data based on the application result of the previous stage filter group and the application result of the selected filter.
A program that realizes.

Images