JP2020107185A - Image recognition device, image recognition method and program - Google Patents

Abstract

To reduce a processing load in image recognition processing using a machine learning model of a neural network.SOLUTION: A model acquisition unit 31 acquires a machine learning model that recognizes a subject included in image data to be processed. A group recognition unit 32 identifies, from a plurality of filters included in machine learning, a subject group by applying to the image data a front-stage filter group for recognizing which subject group a subject included in the image data belongs to. An individual recognition unit 34 identifies, based on an application result of the front-stage filter group and an application result of a rear-stage filter group which is a filter group excluding the front-stage filter group from the plurality of filters, a recognition object included in the image data A filter selection unit 33 selects, based on the subject group identified by the group recognition unit 32 and the priority set for each filter that constitutes the rear-stage filter group, a filter to be applied to the individual recognition unit 34 from the rear-stage filter group.SELECTED DRAWING: Figure 6

Description

The present invention relates to an image recognition device, an image recognition method and a program, and more particularly to a technique for reducing the load of image recognition processing.

In recent years, a technique for recognizing an object class from an image using deep learning, which is a kind of neural network, has been put into practical use. Among such techniques, a structure of a neural network including more layers has been proposed in order to improve recognition accuracy. In the neural network, it becomes possible to extract higher and more complex features by stacking layers. Therefore, making the layers deep plays an important role in improving the recognition accuracy of machine learning models using neural networks.

On the other hand, the deeper the layer of the neural network, the more the amount of calculation at the time of executing the recognition process increases, and the calculation capacity required for executing the recognition process tends to increase. Therefore, it may be difficult to execute a machine learning model having a deep neural network layer in a device having relatively small calculation resources such as an IoT (Internet Of Things) device.

In order to deal with this problem, for example, in Non-Patent Document 1, a technique is proposed in which an output layer is provided in a plurality of layers forming a machine learning model, and the output layer is selected according to the fluctuation of calculation resources to perform image recognition. Has been done.

Gao Huang, et al., Multi-Scale Dense Convolutional Networks for Resource Efficient Image Classification, International Conference on Learning Representations (ICLR) 2018.

In the technique according to Non-Patent Document 1, the recognition accuracy decreases as the calculation for recognition is completed in the output layer of the shallow layer (that is, the layer close to the input layer) of the neural network. Further, since the neural network of the technique according to Non-Patent Document 1 has a layered structure that spreads two-dimensionally, the size of the machine learning model may be large and learning may be difficult. Thus, there is room for improvement in reducing the processing load in the image recognition processing using the machine learning model of the neural network.

Therefore, the present invention has been made in view of these points, and an object thereof is to provide a technique for reducing the processing load in image recognition processing using a machine learning model of a neural network.

A first aspect of the present invention is an image recognition device. This device is a machine learning model that includes a plurality of filters as model parameters, and information indicating which of the plurality of predetermined recognition targets the subject included in the image data of the processing target is a recognition target. A model acquisition unit that acquires a machine learning model that outputs a machine learning model, and to recognize which one of a plurality of predetermined object groups the object included in the image data belongs to among the plurality of filters. By applying the pre-stage filter group to the image data, a group recognition unit that identifies the subject group, an application result of the pre-stage filter group, and a filter group excluding the pre-stage filter group among the plurality of filters. Based on the application result of a certain post-stage filter group, an individual recognition unit that identifies the recognition target included in the image data, a subject group identified by the group recognition unit, and set to each filter that configures the post-stage filter group A filter selecting unit that selects a filter to be applied to the individual recognizing unit in the post-stage filter group based on at least the assigned priority.

The filter selection unit selects a filter to be applied to the individual recognition unit in descending order of priority set for each filter forming the latter-stage filter group in a range permitted by the calculation resource of the image recognition device. Good.

In the image recognition device, the weighting factor calculation unit that calculates the index indicating the magnitude of the weighting factor of each of the plurality of filters, and the filter that indicates that the weighting factor is large by the index have a small weighting factor. And a priority setting unit for setting the priority higher than that of the filter indicating.

The image recognition device, a similarity calculation unit that calculates the similarity of each of the plurality of filters with other filters, and a filter in which no other similar filter exists, than a filter in which another similar filter exists. Also, a priority setting unit that sets the priority higher may be further provided.

The priority setting unit may change, for each of the plurality of subject groups, the priority set for each filter included in the post-stage filter group.

When equal priority is set to each of the two or more filters forming the latter filter group, the filter selection unit may randomly select a filter from filters having equal priorities.

The image recognition device performs the machine learning by machine learning using a neural network based on a plurality of image data, a subject included in each of the plurality of image data, and learning data in which a subject group of the subject is associated. The learning unit may further include a learning unit that generates a learning model, wherein the learning unit includes a pre-stage learning unit that generates the pre-stage filter group, and the pre-stage learning unit that generates the pre-stage filter group and then moves to the pre-stage filter group. And a post-stage learning unit that generates the post-stage filter group using the application result of the pre-stage filter group without performing the error back propagation.

A second aspect of the present invention is an image recognition method. In this method, the processor is a machine learning model including a plurality of filters as model parameters, and which one of a plurality of predetermined recognition targets is a subject included in the image data to be processed is a recognition target. And a step of acquiring a machine learning model that outputs information indicating that which of the plurality of filters the subject included in the image data belongs to among a plurality of predetermined subject groups is recognized. Applying a pre-stage filter group to the image data to identify the subject group, the identified subject group, and a post-stage filter that is a filter group excluding the pre-stage filter group from the plurality of filters. Selecting at least one filter from the filters forming the latter filter group based on at least the priority set for each filter forming the group, the application result of the former filter group, and the selection And specifying a recognition target included in the image data based on the applied result of the filter.

A third aspect of the present invention is a program. This program is a machine learning model in which a computer includes a plurality of filters as model parameters, and which one of a plurality of predetermined recognition targets is a subject included in image data to be processed is a recognition target. And a function of acquiring a machine learning model that outputs information indicating that which of the plurality of filters the subject included in the image data belongs to among a plurality of predetermined subject groups is recognized. By applying a pre-stage filter group to the image data, a function of identifying the subject group, the identified subject group, and a post-stage filter that is a filter group excluding the pre-stage filter group from the plurality of filters A function of selecting one or more filters from the filters forming the latter-stage filter group based on at least the priority set for each filter constituting the group, an application result of the preceding-stage filter group, and a selection And a function of specifying a recognition target included in the image data based on the applied result of the filter.

According to the present invention, it is possible to reduce the processing load in image recognition processing using a machine learning model of a neural network.

It is a figure which shows typically the general function structure of a convolutional neural network. It is a figure which shows typically the layered structure of the machine learning model of the neural network known as AlexNet. It is a figure which shows typically the layered structure of the convolutional neural network which the image recognition apparatus which concerns on embodiment uses. It is a figure for explaining the learning process of the neural network concerning an embodiment. It is a figure for demonstrating the recognition process of the neural network which concerns on embodiment. It is a figure which shows typically the functional structure of the image recognition apparatus which concerns on embodiment. It is a figure which shows typically the priority of the filter for every subject group in a tabular form. It is a figure which shows typically the functional structure of the learning part which concerns on embodiment. 6 is a flowchart for explaining the flow of image recognition processing executed by the image recognition device according to the embodiment.

<Convolutional neural network>
The image recognition device according to the embodiment is a device for executing an image recognition process using a machine learning model of a neural network. The image recognition apparatus according to the embodiment uses a machine learning model of a convolutional neural network (CNN) as a main example. Therefore, as a prerequisite technique of the information processing apparatus according to the embodiment, first, a convolutional neural network will be briefly described.

FIG. 1 is a diagram schematically showing a general functional configuration of a convolutional neural network. Currently, various configurations of neural networks have been proposed, but these basic configurations are common. The basic configuration of the neural network is expressed by superimposing a plurality of types of layers (or a graph structure). The neural network learns the model parameters so that the output result for the input data has an appropriate value. In other words, the neural network learns the model parameters so as to minimize the loss function defined so that the output result with respect to the input data becomes an appropriate value.

FIG. 1 shows a machine learning model learned so as to output the types of subjects included in the input image I. In the example shown in FIG. 1, the input image I input to the input layer Li is processed in the order of the first convolutional layer C1 and the second convolutional layer C2, and the pooling layer P, the first fully connected layer F1, and the second It is configured to reach the full coupling layer F2 and the output layer Lo. The output layer outputs the identification label B indicating the type of subject included in the input image I.

For example, when the machine learning model shown in FIG. 1 is a machine learning model for recognizing a plurality of animals such as dogs, cats, and monkeys, an identification label B for identifying an animal to be identified is assigned in advance. There is. When the input image I is input to the input layer Li of this machine learning model, the output layer Lo outputs the identification label B indicating which of the plurality of predetermined recognition targets is the recognition target. The identification label B is, for example, a bit string uniquely assigned to each of the plurality of recognition targets.

In the neural network, the output of the preceding layer becomes the input of the succeeding layer adjacent to the preceding layer. Each convolutional layer in the convolutional neural network applies a filter to the signal input from the previous stage layer, and the output of the filter becomes the output of that layer.

FIG. 2 is a diagram schematically showing the layer structure of a machine learning model of a neural network known as AlexNet. As shown in FIG. 2, AlexNet includes an input layer Li, five convolution layers (first convolution layer C1, second convolution layer C2, third convolution layer C3, fourth convolution layer C4, and fourth convolution layer C4). 5 convolutional layer C5), two fully coupled layers (first fully coupled layer F1 and second fully coupled layer F2), and output layer Lo, and the final layer outputs 1000 kinds of identification labels B. Is configured. That is, AlexNet is a convolutional neural network for recognizing 1000 types of recognition targets.

Although not shown, a net structure having a deeper layer has been proposed to improve recognition accuracy. For example, a neural network machine learning model known as ResNet has a layered structure of 152 layers. Since it is possible to extract higher and more complicated features with each layer of the neural network, it is considered that making the layers deep plays an important role in improving the recognition accuracy.

<Neural network according to the embodiment>
FIG. 3 is a diagram schematically showing the layer structure of the convolutional neural network N used by the image recognition apparatus according to the embodiment. The neural network N according to the embodiment will be described below with reference to FIG.

The machine learning model of the convolutional neural network N used by the image recognition apparatus according to the embodiment includes a plurality of convolutional layers as in the conventional convolutional neural network, and the subject included in the input image I is a plurality of recognition targets. Which of the recognition targets is output. Therefore, the machine learning model of the convolutional neural network N used by the image recognition device 1 according to the embodiment includes a plurality of filters as model parameters.

In the convolutional neural network, higher and more complicated features can be extracted each time a layer is overlaid. Therefore, by making layers deeper, even similar recognition targets that are difficult to distinguish can be recognized. On the contrary, if the recognition targets are significantly different from each other, the recognition can be performed even if the number of layers of the convolutional neural network is small. These phenomena suggest that large features of the recognition target are recognized in the front part of the convolutional neural network, and detailed features peculiar to each recognition target are recognized in the subsequent stages. The same phenomenon can be realized by changing the number of filters in each layer. That is, by increasing the number of filters in each layer, even similar recognition targets that are difficult to distinguish can be recognized. On the contrary, if the recognition targets are significantly different, the recognition can be performed even if the number of filters in each layer is small.

Therefore, the filters that make up the machine learning model contribute to the recognition of all recognition targets, and play an important role in recognizing one recognition target, but do not contribute much to the recognition of another recognition target. It is considered that there is a filter that does not. The former filter includes a filter for recognizing a feature that a plurality of recognition targets commonly include, and the latter filter includes a filter for distinguishing a specific recognition target having similar features.

For example, the recognition target of a certain machine learning model includes a mammal such as a cat, a dog, a monkey, or a cow, a plant such as a mandarin orange, a carrot, a radish, or a cabbage, and an artificial object such as a car or a building. And In this case, for example, a filter that strongly reacts to the "eyes" of mammals is considered to be a filter for capturing a large feature of whether or not the recognition target is a mammal, and thus is considered to be a filter that contributes to recognition of all recognition targets. ..

On the other hand, for example, a filter used for distinguishing “dog” from “cat” is considered to greatly contribute to recognition when the recognition target is a dog or a cat, but the recognition target is a cat or a dog. If it is a "cabbage" or "car" other than the above, it is considered that it does not contribute much to recognition. That is, it is considered that there exists a filter that plays an important role in recognizing a certain recognition target but does not contribute much to the recognition of another recognition target.

As shown in FIG. 3, the machine learning model of the convolutional neural network N used by the image recognition apparatus according to the embodiment includes two filter groups, a pre-stage filter group and a post-stage filter group. In the example illustrated in FIG. 3, the pre-stage filter group includes five convolution layers (first pre-stage convolution layer C _f 1, second pre-stage convolution layer C _f 2, third pre-stage convolution layer C _f 3, and fourth convolution layer C _f 1. The filters are included in the front convolutional layer C _f 4 and the fifth front convolutional layer C _f 5) and the two front coupling layers (the first fully coupling layer F1 and the second fully coupling layer F2). The post-stage filter group includes five convolutional layers (first post-stage convolutional layer C _r 1, second post-stage convolutional layer C _r 2, third post-stage convolutional layer C _r 3, and fourth post-stage convolutional layer C _{r ).} 4 and the fifth post-convolutional layer C _r 5).

For example, when the present embodiment is applied to AlexNet shown in FIG. 2, the filters included in each layer are equally divided into a pre-stage filter group and a post-stage filter group. Specifically, the neural network structure has (55×55) node×48 filters for the first convolution layer of the pre-stage filter group and the post-stage filter group. Similarly, for the second to fifth convolution layers, (27×27) node×128 filter, (13×13) node×192 filter, (13×13) node×192 filter, ( 13×13) node×128 filter. Further, the first total coupling layer F1, the second total coupling layer F2, the third total coupling layer F3, and the fourth total coupling layer F4 all have 2048 nodes.

[Learning process]
First, the learning process of the neural network N according to the embodiment will be described.

4A to 4C are diagrams for explaining the learning process of the neural network N according to the embodiment. Specifically, FIGS. 4A and 4B show the first-stage learning and the second-stage learning of the pre-stage filter group, respectively, and FIG. 4C shows the learning of the post-stage filter group. ing. Here, the pre-stage filter group is a filter group used for recognizing which one of a plurality of predetermined subject groups the subject included in the input image I belongs to. The post-stage filter group is a filter group excluding the filters constituting the pre-stage filter from the filters constituting the machine learning model.

As shown in FIG. 4A, in the first-stage learning of the pre-stage filter group, when learning data, which is image data for learning, is input, the identification label B of the subject associated with the image data is output. To generate a first pre-filter group. That is, the first stage learning of the pre-stage filter group is the same as the normal machine learning process.

As shown in FIG. 4B, in the second stage learning of the pre-stage filter group, when learning data is input, a machine learning model that outputs a group identification label indicating a subject group to which the image data belongs is generated. To do. Specifically, the first pre-combined layer F1 and the second full-combined layer F2 included in the pre-stage filter group are fine-tuned to generate a second pre-stage filter group that recognizes the group to which the image data belongs.

In FIG. 4B, the first fully coupled layer F1 and the second fully coupled layer F2 after fine tuning are described as a first fully coupled layer F1' and a second fully coupled layer F2', respectively. Therefore, the convolutional layer of the first pre-stage filter group and the convolutional layer of the second pre-stage filter group are common. Since most of the first pre-stage filter group and the second pre-stage filter group are common, the first pre-stage filter group is simply referred to as the pre-stage filter group unless a distinction is made between the two.

As shown in FIG. 4C, the learning of the second-stage filter is performed together with the machine learning model generated in the first learning stage of the first-stage filter group. That is, the post-filter group is learned so that the identification label B of the learning data is output using the output of the first pre-filter group and the output of the post-filter group. At the time of learning the post-stage filter group, back propagation of the error to the pre-stage filter group is not performed, but the post-stage filter group is learned using the application result of the pre-stage filter group.

[Recognition process]
Next, the learning process of the neural network N according to the embodiment will be described.

FIGS. 5A and 5B are diagrams for explaining the recognition process of the neural network N according to the embodiment. The recognition process of the neural network N according to the embodiment is composed of two stages: a group recognition stage for recognizing a group to which the subject included in the input image I belongs and a subject recognition stage for recognizing the subject itself. ing.

FIG. 5A is a diagram for explaining the group recognition of the input image I. Group recognition is performed using the second pre-stage filter group generated in the second-stage learning of the pre-stage filter group. By applying the second pre-filter group to the input image I, the subject group to which the subject included in the input image I belongs is recognized.

The image recognition apparatus according to the embodiment can be expected to recognize the recognition target with the highest recognition accuracy by applying all the filters included in the post-stage filter group. However, it is possible to maintain constant recognition accuracy without applying all the filters included in the latter filter group. Therefore, in the image recognition device, for example, the calculation resource of the image recognition device is originally small, or even if the calculation resource is large, the processing load of other calculations is large and the calculation resource temporarily allocated to the application of the machine learning model is small. In this case, it is possible to balance the feasibility of calculation with the recognition accuracy by selecting the post-stage filter to be applied according to the calculation resource.

Although details will be described later, the image recognition apparatus according to the embodiment selects a filter to be actually applied from the filters forming the post-stage filter group based on the subject group to which the subject included in the input image I belongs. Then, as shown in FIG. 5B, the output of the first front filter group and the output of the selected rear filter group are combined to output the identification label B indicating the subject included in the input image I. It

Here, if the amount of increase in the amount of calculation required for group recognition of the input image I exceeds the amount of decrease in the amount of calculation due to selection of filters in the post-stage filter group, it is possible to select filters in the post-stage filter group. meaningless. However, in the neural network N according to the embodiment, the convolutional layer of the first pre-stage filter group and the convolutional layer of the second pre-stage filter group are common. Therefore, the calculation result of the convolutional layer of the second pre-stage filter group executed for recognizing the subject group to which the subject included in the input image I belongs can be used for the calculation of the convolutional layer of the first pre-stage filter group. .. Accordingly, in the recognition process of the neural network N according to the embodiment, the calculation cost increased for recognizing the subject group to which the subject included in the input image I belongs is substantially the calculation of the first fully connected layer F1′. And the calculation of the second fully connected layer F2'. Therefore, it can be expected that the increase amount of the calculation amount required for the group recognition of the input image I is sufficiently smaller than the reduction amount of the calculation amount due to the selection of the filters of the post-stage filter group.

As described above, the image recognition apparatus according to the embodiment can reduce the processing load of the recognition processing by selecting the filters to be applied to the input image I.

<Functional configuration of the image recognition device 1 according to the embodiment>
FIG. 6 is a diagram schematically showing the functional configuration of the image recognition device 1 according to the embodiment. The image recognition device 1 includes a storage unit 2 and a control unit 3. In FIG. 6, arrows indicate main data flows, and there may be data flows not shown in FIG. In FIG. 6, each functional block does not have a hardware (device) unit configuration but a functional unit configuration. Therefore, the functional blocks shown in FIG. 6 may be implemented in a single device or may be separately implemented in a plurality of devices. Data transfer between the functional blocks may be performed via any means such as a data bus, a network, a portable storage medium, or the like.

The storage unit 2 includes a ROM (Read Only Memory) that stores a BIOS (Basic Input Output System) of a computer that implements the image recognition apparatus 1, a RAM (Random Access Memory) that is a work area of the image recognition apparatus 1, and an OS ( An operating system), an application program, and a large-capacity storage device such as an HDD (Hard Disk Drive) and an SSD (Solid State Drive) that stores various types of information referred to when the application program is executed.

The control unit 3 is a processor such as a CPU (Central Processing Unit) and a GPU (Graphics Processing Unit) of the image recognition device 1, and executes the program stored in the storage unit 2 to execute the image acquisition unit 30 and the model acquisition unit. 31, the group recognition unit 32, the filter selection unit 33, the individual recognition unit 34, the resource acquisition unit 35, the weight index calculation unit 36, the priority setting unit 37, the similarity calculation unit 38, and the learning unit 39.

Note that FIG. 6 shows an example in which the image recognition device 1 is configured by a single device. However, the image recognition device 1 may be realized by a plurality of computing resources such as a processor and a memory like a cloud computing system. In this case, each unit constituting the control unit 3 is realized by executing a program by at least one of the plurality of different processors.

The image acquisition unit 30 acquires an input image I which is image data to be processed by the image recognition device 1. The model acquisition unit 31 acquires a machine learning model including a plurality of filters as model parameters. In the machine learning model acquired by the model acquisition unit 31 according to the embodiment, which one of a plurality of predetermined recognition targets is a subject included in an input image I that is image data to be processed is a recognition target. Is a machine learning model that outputs information indicating

The group recognition unit 32 recognizes which of the plurality of predetermined object groups the object included in the input image I belongs to among the plurality of filters included in the learning model. (That is, the above-mentioned second pre-stage filter group) is applied to the input image I to identify the subject group.

The filter selection unit 33 selects one or more filters from the plurality of filters included in the machine learning model from the post-stage filter group that is the filter group excluding the pre-stage filter group. Here, the filter selection unit 33 selects a filter based on at least the subject group identified by the group recognition unit 32 and the priority set for each filter that configures the subsequent filter group. Details of filter selection by the filter selection unit 33 will be described later.

The individual recognition unit 34 identifies the recognition target included in the input image I based on the application result of the first pre-stage filter group and the result of applying the filter selected by the filter selection unit 33 to the input image I. In this way, the image recognition apparatus 1 selects a filter to be actually applied to the input image I from the post-stage filter group included in the machine learning model. As a result, the image recognition device 1 can reduce the processing load in the image recognition process, as compared with the case where all the filters included in the machine learning model are used.

The resource acquisition unit 35 acquires the calculation resource of the image recognition device 1. Here, the “computation resource” acquired by the resource acquisition unit 35 includes the power of a processor such as a CPU and a GPU included in the image recognition device 1 and the capacity of a main storage device included in the image recognition device 1 and the like. It is the computing power that can be assigned to a process. The calculation resource of the image recognition device 1 differs depending on the type of the image recognition device 1, such as when the image recognition device 1 is a blade server and when it is an IoT device. Even in the same image recognition apparatus 1, the calculation resources that can be allocated to the image recognition process change depending on the size of the calculation resources that are allocated to other processes that are executed in parallel during the image recognition process. sell. The resource acquisition unit 35 acquires a calculation resource that can be used when the image recognition device 1 performs an image recognition process using a machine learning model.

The filter selection unit 33 selects a filter to be applied to the individual recognition unit 34 in descending order of priority set for each filter that configures the post-stage filter group within a range permitted by the calculation resource of the image recognition device 1. Generally, the higher the number of filters applied in the image recognition processing, the higher the recognition accuracy can be expected. The image recognition device 1 selects the filter of the post-stage filter group to be applied according to the calculation resource of the image recognition device 1 to perform the image recognition process in which the highest recognition accuracy can be expected in the range in which the image recognition device 1 can execute. Can be executed.

[Filter selection processing]
Subsequently, a filter selection process in the image recognition device 1 according to the embodiment will be described.

As described above, in the neural network, the output of the preceding layer becomes the input of the following layer adjacent to the preceding layer. Each convolutional layer in the convolutional neural network applies a filter to the signal input from the previous stage layer, and the output of the filter becomes the output of that layer. Therefore, the larger the absolute value of the weighting coefficient of the filter in the convolutional layer, the larger the absolute value of the input signal of the next layer can be.

That is, the magnitude of the weighting coefficient of the filter of a certain convolutional layer can be an index value of the activity of the corresponding unit in the next layer. In a neural network, it is said that a unit having a high activity among units forming a layer has a greater contribution to a recognition ability than a unit having a low activity.

Therefore, the weight index calculation unit 36 calculates an index indicating the magnitude of the weight coefficient of each of the plurality of filters included in the latter filter group. Here, the “index indicating the magnitude of the weighting coefficient” is, for example, a value obtained by dividing the sum of absolute values of the weighting coefficients of the filter by the number of weighting coefficients of the filter. Alternatively, it may be a value obtained by dividing the sum of squares of the filter weighting factors by the number of filter weighting factors. In any case, the larger the index indicating the size of the weighting coefficient of the filter, the larger the weighting coefficient included in the filter.

The priority setting unit 37 sets a higher priority for a filter that indicates that the weighting coefficient is larger by the index calculated by the weighting index calculation unit 36 than a filter that indicates that the weighting coefficient is small. Thereby, the image recognition device 1 can preferentially select a filter that is considered to have a large contribution to the recognition ability of the machine learning model.

Here, the priority level set by the priority setting unit 37 for each filter included in the post-stage filter group is arbitrary with the upper limit being the number of filters included in the post-stage filter group. For example, if the priority level is the same as the number of filters, the filters included in the convolutional layer can be ordered by priority.

Alternatively, when the number of filters is two or more, the priority level may be set to two levels of “high” and “low”. In this case, the priority setting unit 37 sets a predetermined threshold A, sets the priority to “high” when the index indicating the size of the weighting coefficient of each filter exceeds the threshold A, and sets the priority to less than the threshold A. The priority may be “low”.

Further, the priority setting unit 37 may change the priority set for each filter based on the similarity between the filters included in the latter filter group. In general, it is considered that the closer the weighting coefficients of a certain two filters are, the more the two filters extract the similar features. Therefore, when two certain filters are similar to each other, if one of the filters extracts a feature, it is considered that the final change in the recognition accuracy is small without using the other filter. Conversely, a filter for which no other similar filter exists can potentially extract features that other filters cannot.

Therefore, the similarity calculation unit 38 calculates the similarity of each of the plurality of filters included in the latter filter group to other filters. The priority setting unit 37 sets a higher priority for a filter having no other similar filter than for a filter having another similar filter. As a result, the image recognition device 1 can select a filter for extracting different features within a range permitted by the calculation resources of the image recognition device 1.

Here, the similarity calculation unit 38 may calculate the “distance” between the filters as the similarity between the filters. The “distance” between the filters calculated by the similarity calculation unit 38 may be any amount as long as the distance axiom is satisfied, but is, for example, the Euclidean distance between the filters. Specifically, the similarity calculator 38 calculates the similarity D(i,j) between the i-th filter and the j-th filter using the following equation (1).

ここで、Ｉ（ｍ，ｎ，ｆ）は、３次元の第ｉフィルタの縦ｍ、横ｎ、高さｆにおける要素であり、Ｊ（ｍ，ｎ，ｆ）は、第ｊフィルタの縦ｍ、横ｎ、高さｆにおける要素である。式（１）は、２つのフィルタのユークリッド距離を、フィルタの要素数（重み係数の数）で正規化した量であることを示している。ある２つのフィルタ間の非類似度Ｄの値が小さいほど、そのフィルタ同士は類似していることを示している。この他にも、類似度算出部３８は、例えばコサイン類似度を用いてフィルタ間の類似度を算出してもよい。

Here, I(m,n,f) is an element in the vertical m, horizontal n, and height f of the three-dimensional i-th filter, and J(m,n,f) is the vertical m of the j-th filter. , Lateral n, and height f. Expression (1) indicates that the Euclidean distance between the two filters is a quantity normalized by the number of filter elements (the number of weighting factors). The smaller the dissimilarity D between two filters is, the more similar the filters are. In addition to this, the similarity calculation unit 38 may calculate the similarity between filters using, for example, the cosine similarity.

The similarity calculation unit 38 calculates S(i)=ΣD(i,j) (j=1,..., Nf; Nf is the number of filters) as the similarity S(i) of the i-th filter. , A filter whose value is large (that is, there is no other similar filter) may be assigned a high priority.

Similar to the weight index calculation unit 36, the priority setting unit 37 may have two levels of similarity, “similar” and “not similar”. In this case, the priority setting unit 37 sets a predetermined threshold value B, and if the similarity between the filters exceeds the threshold value B, the similarity is set, and if the similarity degree between the filters is less than the threshold value B, the priority setting section 37 sets the dissimilarity. .. Further, when the weighting index calculation unit 36 sets the priority to two levels, the priority setting unit 37 may obtain the similarity between the filters by targeting the filters having the low priority. In this case, the priority setting unit 37 may set the priority of the filter dissimilar to any of the filters to “high”.

As described above, it is considered that there are filters that make up a machine learning model that play an important role in recognizing one recognition target but do not contribute much to the recognition of another recognition target. .. Therefore, the importance of the filter for recognizing the subject may change depending on the type of the subject.

Therefore, the priority setting unit 37 changes the priority set for each of the filters forming the post-stage filter group for each of a plurality of predetermined subject groups. Hereinafter, the setting of the filter priority for each subject group performed by the priority setting unit 37 will be specifically described.

Now, it is assumed that there are P types of subject groups (P is an integer of 2 or more) and Q filters (Q is an integer of 2 or more) included in the post-stage filter group. Let the q-th filter be the filter f _q (1≦q≦Q), and the order of importance of the filter f _q in the p-th group (1≦p≦P) be W _pq .

Step 1: The priority setting unit 37 sets 1 to p.
Step 2: The priority setting unit 37 acquires the test data T _p of the p-th subject group.
Step 3: priority setting unit 37 applies a pre-stage filter group and all of the subsequent filter group to the test data T _p, calculates the recognition rate R _p.

Step 4: The priority setting unit 37 sets 1 to q.
Step 5: The priority setting unit 37 calculates the recognition rate R _pq of the test data T _p when the filter f _q is excluded from the filters included in the latter filter group and applied.
Step 6: The priority setting unit 37 calculates the reduction amount C _q of the recognition rate, which is a value obtained by subtracting the recognition rate R _pq from the recognition rate R _p . The reduction amount C _q indicates the reduction amount of the recognition rate due to the exclusion of the filter f _q . That is, the contribution of the filter f _{q to} the recognition rate is shown.

Step 7: The priority setting unit 37 updates the value of q to q+1.
Step 8: The priority setting unit 37 repeats the processing of steps 4 and 5 until q exceeds Q.
Step 9: The priority setting unit 37 sorts the Q reduction amounts C _q in descending order. At this time, the order of the subscript q of the reduction amount C _q is the important order W _pq of the filter f _q in the p-th subject group.

Step 10: The priority setting unit 37 updates the value of p to p+1.
Until the stub 11:p exceeds P, the priority setting unit 37 repeats the processing from step 2 to step 8.

Through the above processing, the priority setting unit 37 can obtain the order of importance of the filters forming the post-stage filter group for each predetermined object group. The priority setting unit 37 raises the priority for a filter having higher importance for each predetermined object group. As a result, the priority setting unit 37 can change the priority set for each of the filters forming the post-stage filter group for each of a plurality of predetermined subject groups.

As with the filter similarity, the priority setting unit 37 may set the level of importance to two levels of importance or not. In this case, the priority setting unit 37 sets a predetermined threshold value C, recognition rate R _pq cases the threshold C is smaller than the critical filter f _q, if the recognition rate R _pq is less than the threshold value C filter f _q Should not be important.

FIG. 7 is a diagram schematically showing the filter priority for each subject group in a table format. Specifically, FIG. 7 shows a data structure of a priority database that stores the magnitude of the weighting factor for each filter, similarity, order of importance, and priority for the first subject group. The priority database is stored in the storage unit 2 and managed by the priority setting unit 37. By referring to the priority database, the filter selection unit 33 causes the individual recognition unit 34 of the post-stage filter group to identify the individual group based on at least the subject group and the priorities set for the filters constituting the post-stage filter group. You can select the filter to apply.

Here, when the priority setting unit 37 sets the priority level set for each filter included in the subsequent filter group to less than the number of filters, it is possible that a plurality of filters have the same priority. Therefore, when equal priority is set to each of two or more filters that configure the pre-stage filter group, the filter selection unit 33 may randomly select the filters from the filters to which equal priority is set. As a result, the filter selection unit 33 can select a filter without referring to an index other than the priority.

The learning unit 39 generates a machine learning model by machine learning using a neural network based on learning data in which a plurality of image data, a subject included in each of the plurality of image data, and a subject group of the subject are associated with each other. To do.

FIG. 8 is a diagram schematically showing the functional configuration of the learning unit 39 according to the embodiment. As shown in FIG. 8, the learning unit 39 includes a pre-stage learning unit 390 and a post-stage learning unit 391.

First, the image acquisition unit 30 acquires a plurality of image data in which the subject included in the image data and the subject group to which the subject belongs are known. The pre-stage learning unit 390 uses the plurality of image data acquired by the image acquisition unit 30 as learning data, and as described with reference to FIGS. 4A and 4B, the pre-stage learning unit performs machine learning using a neural network. Generate a group.

As described with reference to FIG. 4C, the post-stage learning unit 391 applies the pre-stage filter group without performing back error propagation to the pre-stage filter group after the pre-stage learning unit 390 generates the pre-stage filter group. A post-stage filter group is generated using the result.

Specifically, the post-stage learning unit 391 updates the weights of the filters forming each layer by back-propagating the error between the output of the output layer Lo and the identification label B associated with the input image I. At this time, the second-stage learning unit 391 includes the first front-stage convolutional layer C _f 1, the second front-stage convolutional layer C _f 2, the third front-stage convolutional layer C _f 3, the fourth front-stage convolutional layer C _f 4, and The weight of the filter included in the fifth front convolutional layer C _f 5 is fixed and its update is prohibited. As a result, the post-stage learning unit 391 can generate the post-stage filter group while fixing the pre-stage filter group.

<Processing Flow of Image Recognition Method Performed by Image Recognition Device 1>
FIG. 9 is a flowchart for explaining the flow of the image recognition process executed by the image recognition device 1 according to the embodiment. The process in this flowchart is started, for example, when the image recognition device 1 is activated.

The model acquisition unit 31 is a machine learning model including a plurality of filters as model parameters, and a subject included in an input image I that is image data to be processed is recognized as one of a plurality of predetermined recognition targets. A machine learning model that outputs information indicating whether it is a target is acquired (S2).

The group recognition unit 32 sets, in the input image I, a pre-stage filter group for recognizing which of the plurality of predetermined object groups the object included in the input image I belongs to among the plurality of filters. By applying, the subject group is specified (S4).

The filter selection unit 33 selects one or more of the filters that form the post-stage filter group based on at least the subject group identified by the model acquisition unit 31 and the priority set for each filter that forms the post-stage filter group. Is selected (S6).

The individual recognition unit 34 identifies the recognition target included in the input image I based on the application result of the pre-stage filter group and the application result of the selected filter (S8).

<Effects of the image recognition device 1 according to the embodiment>
As described above, the image recognition device 1 according to the embodiment can reduce the processing load in the image recognition process using the machine learning model of the neural network.

Although the present invention has been described above using the embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments, and various modifications and changes are possible within the scope of the gist thereof. is there. For example, the specific embodiment of device distribution/integration is not limited to the above embodiment, and all or part of the device may be functionally or physically distributed/integrated in arbitrary units. You can Further, a new embodiment that occurs due to an arbitrary combination of a plurality of embodiments is also included in the embodiment of the present invention. The effect of the new embodiment produced by the combination also has the effect of the original embodiment.

1... Image recognition device 2... Storage unit 3... Control unit 30... Image acquisition unit 31... Model acquisition unit 32... Group recognition unit 33... Filter selection unit 34... Individual recognition unit 35... Resource acquisition unit 36... Weight index calculation unit 37... Priority setting unit 38... Similarity calculation unit 39... Learning unit 390... Previous learning unit 391. ..Lear learning unit N... Convolutional neural network

Claims (9)

A machine learning model including a plurality of filters as model parameters, which outputs information indicating which recognition target among a plurality of predetermined recognition targets the subject included in the image data to be processed is. A model acquisition unit that acquires a learning model,
By applying, to the image data, a pre-stage filter group for recognizing which one of the plurality of subject groups the subject included in the image data belongs to among the plurality of filters, among the plurality of filters, A group recognition unit that identifies the subject group,
Individual recognition for identifying a recognition target included in the image data based on the application result of the pre-stage filter group and the application result of the post-stage filter group which is a filter group excluding the pre-stage filter group among the plurality of filters. Department,
A filter to be applied to the individual recognition unit is selected from the post-stage filter group based on at least the subject group specified by the group recognition unit and the priority set for each filter forming the post-stage filter group. A filter selector,
An image recognition device including. The filter selection unit selects a filter to be applied to the individual recognition unit in descending order of priority set in each filter that configures the latter-stage filter group in a range that the calculation resource of the image recognition device allows,
The image recognition device according to claim 1. A weighting index calculation unit that calculates an index indicating the magnitude of the weighting factor of each of the plurality of filters,
The filter indicating that the weighting coefficient is large by the index, the priority setting unit that sets the priority higher than that of the filter indicating that the weighting coefficient is small,
The image recognition device according to claim 1, further comprising: A similarity calculation unit that calculates the similarity with other filters for each of the plurality of filters,
A filter having no other similar filter is further provided with a priority setting unit that sets the priority higher than a filter having another similar filter,
The image recognition device according to claim 1. The priority setting unit, for each of the plurality of subject groups, changes the priority set to each filter that configures the latter-stage filter group,
The image recognition device according to claim 3 or 4. When equal priority is set to each of two or more filters that form the latter stage filter group, the filter selection unit randomly selects a filter from the filters to which equal priority is set,
The image recognition device according to claim 1. Learning that generates the machine learning model by machine learning using a neural network based on a plurality of image data, a subject included in each of the plurality of image data, and learning data in which a subject group of the subject is associated. More parts,
The learning unit is
A pre-stage learning unit that generates the pre-stage filter group,
After the pre-stage learning unit generates the pre-stage filter group, a post-stage learning unit that generates the post-stage filter group using the application result of the pre-stage filter group without performing error back propagation to the pre-stage filter group, Prepare,
The image recognition device according to claim 1. The processor
A machine learning model including a plurality of filters as model parameters, which outputs information indicating which recognition target among a plurality of predetermined recognition targets the subject included in the image data to be processed is. Obtaining a learning model,
By applying, to the image data, a pre-stage filter group for recognizing which one of the plurality of subject groups the subject included in the image data belongs to among the plurality of filters, among the plurality of filters, Identifying the subject group,
Based on at least the specified subject group and the priority set for each filter constituting the post-stage filter group that is the filter group excluding the pre-stage filter group among the plurality of filters, the post-stage filter group is Selecting one or more filters from the constituent filters,
Based on the application result of the pre-stage filter group, and the application result of the selected filter, a step of identifying the recognition target included in the image data,
Image recognition method for executing. On the computer,
A machine learning model including a plurality of filters as model parameters, which outputs information indicating which recognition target among a plurality of predetermined recognition targets the subject included in the image data to be processed is. The ability to get a learning model,
By applying, to the image data, a pre-stage filter group for recognizing which one of the plurality of subject groups the subject included in the image data belongs to among the plurality of filters, among the plurality of filters, A function for specifying the subject group,
Based on at least the specified subject group and the priority set for each filter constituting the post-stage filter group that is the filter group excluding the pre-stage filter group among the plurality of filters, the post-stage filter group is A function to select one or more filters from the filters to configure,
Based on the application result of the pre-stage filter group and the application result of the selected filter, a function of specifying a recognition target included in the image data,
A program that realizes.

Images