JP7296270B2 - Image feature extraction device and its program - Google Patents

Description

The present invention relates to an image feature extracting device and its program for extracting features of an image using a convolutional neural network.

2. Description of the Related Art Conventionally, in the field of image recognition such as object recognition in an image and face recognition, a technique using a convolutional neural network (CNN) is known.
This CNN, like VGG (Visual Geometry Group), deepens the layers of the network and performs learning to improve recognition accuracy. FIG. 9 shows the basic structure of CNN. A conventional CNN, as shown in FIG. 9, has a network structure in which a plurality of (two layers in FIG. 9) convolutional layers are connected and an output H(x) is optimized with respect to an input x. In this case, the map F(x) to be learned is H(x) itself.
However, in conventional CNN, if the network layer is simply made deeper, the gradient during learning disappears or diverges, preventing correct learning and degrading recognition accuracy. It is known.

Therefore, in recent years, a technique using a residual neural network (ResNet: Residual Network) is known. FIG. 10 shows the basic structure of ResNet.
As shown in FIG. 10, this ResNet is a network structure in which a plurality of convolutional layers (two layers in FIG. 10) are connected and the output H(x), which is the sum of the output and the input x, is optimized. . In this case, the map F(x) to be learned is H(x)-x, which is the residual between the output H(x) and the input x.
In this way, ResNet provides a path that transmits the input x as it is, thereby transmitting information to the lower layers of the network, suppressing defects such as vanishing gradients during learning, and realizing a multi-layered network.

K. He, X. Zhang, S. Ren, and J. Sun,“Deep Residual Learning for Image Recognition”, in Proc. CVPR, 2015.

As explained in FIG. 10, the basic structure of ResNet provides a path through which the input x is passed directly to the lower layers of the network without convolution by the convolution layers.
When constructing a convolutional neural network that performs image recognition, etc. by multilayering the basic structure of this ResNet, the map F(x) to be learned and the input x that does not pass through the convolutional layer are the same in any layer. will be added by weight.
However, the importance of the input x may vary depending on the layer position, and uniform addition may not always contribute to network optimization.
Therefore, there has been a demand for further measures to optimize convolutional neural networks.

The present invention has been made in view of such problems, and it is possible to extract image features by adding weights to data transmitted to lower layers without passing through a plurality of convolutional layers of a convolutional neural network. An object of the present invention is to provide an image feature extraction device and its program.

In order to solve the above-described problems, an image feature extraction device according to the present invention is an image feature extraction device for extracting a feature amount of an image from image data, comprising convolution means, scaling means, and addition means. A multi-stage configuration is adopted.

In such a configuration, the image feature extracting device performs operations of a plurality of convolution layers on the input data by using a kernel learned in advance as a parameter of the convolution neural network by the convolution means of the basic component. Thereby, the convolution means sequentially convolves and extracts the features of the image.
Further, the image feature extracting device multiplies the input data by the scaling coefficient, which is learned in advance as a parameter of the convolutional neural network, by the scaling means of the basic component. As a result, the scaling means adds weights according to learning to the input data on paths that do not pass through the convolutional layers.
Then, the image feature extracting device adds the calculation result of the convolution means and the calculation result of the scaling means by the adding means of the basic component.
In addition, the image feature extraction device is equipped with this basic configuration unit in a multi-stage configuration, and by performing sequential operations, for data transmitted to lower layers without passing through a plurality of convolution layers of the convolutional neural network, each basic configuration unit can be added with pre-learned weights.

Also, the present invention can be realized by an image feature extraction program for causing a computer to function as the image feature extraction device.

ADVANTAGE OF THE INVENTION This invention has the outstanding effect shown below.
According to the present invention, in a convolutional neural network, weights can be added to data transmitted to lower layers without passing through a plurality of convolutional layers.
As a result, the present invention can extract image features with high accuracy using a convolutional neural network that learns the importance of input data differently depending on the position of the layer.

It is a network diagram showing the basic structure of CNN used in the image feature extraction device according to the embodiment of the present invention. 1 is a block diagram showing the configuration of a basic component of an image feature extraction device according to an embodiment of the present invention; FIG. FIG. 4 is a block diagram showing another configuration of the basic configuration unit of the image feature extraction device according to the embodiment of the present invention; 1 is a network diagram showing a CNN model used by an image feature extraction device according to an embodiment of the present invention; FIG. 1 is an overall configuration diagram showing the configuration of an image feature extraction device according to an embodiment of the present invention; FIG. 4 is a flow chart showing the operation of the image feature extraction device according to the embodiment of the present invention; FIG. 4 is a network diagram showing a CNN model used by a parameter learning device for learning parameters of an image feature extraction device; 1 is an overall configuration diagram showing the configuration of a parameter learning device for learning parameters of an image feature extraction device; FIG. 1 is a network diagram showing the basic structure of a conventional CNN; FIG. 1 is a network diagram showing the basic structure of a conventional ResNet; FIG.

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments of the present invention will be described with reference to the drawings.
<Outline of basic structure of convolutional neural network>
With reference to FIG. 1, an outline of a basic structure N of a convolutional neural network (hereinafter referred to as CNN) used in the image feature extraction device 1 (FIG. 5) according to the embodiment of the present invention will be described.
As shown in FIG. 1, the basic structure N of the CNN used in the image feature extraction device 1 (FIG. 5) is data obtained by connecting a plurality of (two layers in FIG. 1) convolution layers CL, CL and convolving the input x. It is a network model that optimizes the output H(x) obtained by adding (F(x)) and data (ax) obtained by scaling (a times) the input x in the scaling layer SL.
The basic structure N of the CNN is that the mapping F(x) performed by the multiple convolution layers CL, CL is H(x) which is the residual between the output H(x) and the data (ax) a times as large as the input x. −ax is a network model learned in advance.
In this way, the basic structure N of the CNN weights and adds data that has undergone convolutional operations in a plurality of convolutional layers CL and CL and data that is transmitted to lower layers without passing through the convolutional layers CL and CL. .
Note that the number of convolutional layers CL of the basic structure N of the CNN is not limited to two, and may be three or more.

<Basic Configuration of Image Feature Extraction Device>
Next, with reference to FIGS. 2 and 3, the basic components of the image feature extraction device 1 (FIG. 5) according to the embodiment of the present invention will be described.
FIG. 2 is a configuration diagram of the basic configuration unit 10 of the image feature extraction device 1 that embodies the basic structure N of the CNN described in FIG. 1 as the device configuration.
The basic configuration unit 10 performs a convolution operation on input data, extracts features, and outputs data.

Input data is a three-dimensional matrix (feature matrix) having a size of W×H×C. The columns W and rows H of the matrix represent the width and height of the feature amount, and the depth C of the matrix represents the number of channels (the number of feature amount types). Input data to the basic configuration unit 10 of the top layer of the image feature extraction device 1 is image data. In this case, the input data has a predetermined width W pixels, height H pixels, and the number of channels C (“3” for RGB).
The input data and the output data of the basic configuration unit 10 have the same matrix dimension. Here, the dimensions of the matrix refer to the number of arrays for the width, height, and number of channels of the matrix.
As shown in FIG. 2 , the basic configuration unit 10 includes convolution means 11 , scaling means 12 and addition means 13 .

The convolution means 11 performs a convolution operation on input data using a plurality of kernels having learned parameters (connection weight coefficients) of a predetermined size.
For example, the convolution means 11 applies a pre-learned 3×3 size kernel to C are used to add one pixel (for example, zero padding) to both ends of the input data in the width and height directions. Then, the convolution means 11 sequentially shifts the kernels with a stride of "1" to perform the convolution operation. As a result, the convolution means 11 generates data having the same dimensions as the matrix of the input data (width, height, number of channels).
The convolution means 11 performs this convolution layer operation for the number of a plurality of predetermined convolution layers CL of the CNN shown in FIG.
The convolution means 11 outputs the calculated data to the addition means 13 .

The scaling means 12 multiplies each element of the input data by a scaling coefficient, which is a pre-learned parameter. This scaling means 12 bypasses the operation of the convolutional layers.
The scaling factor is a scalar value learned in advance as a CNN parameter. This scaling factor indicates the weight of the path through convolution means 11 and the path through scaling means 12 .
The scaling means 12 multiplies each element of the matrix, which is input data of size W (width)×H (height)×C (channel), by a scaling factor.
The dimensions (width, height, number of channels) of the data calculated by the scaling means 12 are the same as the input data.
The scaling means 12 outputs the calculated data to the adding means 13 .

The adding means 13 adds the data calculated by the convolution means 11 and the data calculated by the scaling means 12 .
Since the data calculated by the convolution means 11 and the data calculated by the scaling means 12 input to the addition means 13 are data of the same dimension (width, height, number of channels), the addition means 13 , add the elements at the same position in the two matrices, respectively.
As a result, the adding means 13 generates output data with the same dimension as the input data and the feature quantity extracted.

Next, another configuration (basic configuration unit 10B) of the basic configuration unit of the image feature extraction device 1 will be described with reference to FIG.
As with the basic configuration unit 10 (FIG. 2), the basic configuration unit 10B performs a convolution operation on input data, extracts features, and outputs them as output data.
The input data and output data in the basic configuration unit 10 described with reference to FIG. 2 are matrices of the same dimension. However, the input data and the output data in the basic configuration unit 10B are matrices of different dimensions.

As shown in FIG. 3, the basic configuration unit 10B includes convolution means 11B, scaling means 12B, and addition means 13. As shown in FIG.
Since the adding means 13 has the same configuration as the basic configuration section 10 explained in FIG. 2, the explanation thereof is omitted.

The convolution means 11B performs a convolution operation on input data using a plurality of kernels having learned parameters (connection weight coefficients) of a predetermined size. Note that the convolution means 11B generates data obtained by changing the dimension of the matrix of the input data.
For example, the convolution means 11B applies two pre-learned kernels of 3×3 size to input data of size W (width)×H (height)×C (channel) as an operation of one convolution layer. xC pixels are used to add one pixel (for example, zero padding) to both ends of the input data in the width and height directions. Then, the convolution means 11B sequentially shifts the kernels with a stride of "2" to perform the convolution operation. As a result, the convolution means 11B halves the width and height of the matrix of the input data and generates data with double the number of channels.

Note that the convolution means 11B does not need to change the dimension of the matrix in all the operations of a plurality of convolution layers. Only the operation of the first convolution layer changes the dimension of the matrix. may be calculated without changing the dimension of .
The convolution means 11B performs this convolution layer operation for the number of predetermined convolution layers CL of the CNN shown in FIG.
The convolution means 11B outputs the data after the calculation to the addition means 13. FIG.

The scaling means 12B aligns the dimensions of the input data matrix with the dimensions of the data output from the convolution means 11B, and multiplies each element of the matrix by a scaling factor, which is a pre-learned parameter.
This scaling means 12B linearly projects the matrix of the input data onto the dimensions of the data output by the convolution means 11B.

For example, the convolution means 11B convolves input data having a size of W (width) x H (height) x C (channels) into a (W/2) x (H/2) x (2 x C) matrix. It is assumed that calculations are performed.
In this case, the scaling means 12B reduces the W×H data to (W/2)×(H/2) for each channel, further linearly interpolates the channels, and obtains a 2×C channel number matrix to generate
Then, the scaling means 12B multiplies each element of the matrix, which is input data of size W/2 (width)×H/2 (height)×2×C (channel), by the scaling factor.
As a result, the dimensions (width, height, number of channels) of the data calculated by the scaling means 12B can be matched with the dimensions of the data calculated by the convolution means 11B.
The scaling means 12B outputs the data after the calculation to the addition means 13. FIG.

As described above, the basic configuration units 10 and 10B can weight and add the operation results of a plurality of convolution layers for the input data in the convolution means 11 and 11B and the input data that does not pass through the convolution layers. can.

<Overview of CNN for image feature extraction device>
Next, with reference to FIG. 4, an overview of the CNN model M in which the image feature extraction device 1 (FIG. 5) operates will be described.

The model M is a convolutional neural network (CNN) that inputs the data of the image I and extracts the feature value V of the image I. The model M includes a plurality of predetermined convolution layers CL (here, [CL ₁ , CL ₂ ], [CL ₃ , CL ₄ ], [CL ₅ , CL ₆ ] and [CL ₇ , CL ₈ ]), scaling layers SL (SL ₁ to SL ₄ ) that bypass the convolutional layer CL Prepare.
Then, the model M adds the outputs of a plurality of predetermined convolutional layers CL and the outputs of the scaling layers SL that do not pass through the convolutional layers, and sequentially outputs the image features to the subsequent convolutional layers CL.
The final output of the model M is a feature V of predetermined dimensions.
In this way, the model M becomes a network in which the weights of routes that are not convolved are changed according to the depth of the network.

Note that this model M is merely an example, and the total number of convolutional layers CL and the number of convolutional layers CL bypassed by the scaling layer SL are not limited to this example. Also, an activation function (ReLU: Rectified Linear Units, etc.) may be applied to the output data of each convolutional layer CL and the data after addition.

<Configuration of image feature extraction device>
Next, the configuration of the image feature extraction device 1 according to the embodiment of the present invention will be described with reference to FIG.
The image feature extraction device 1 extracts a feature amount V from an image I using the model M described with reference to FIG.
As shown in FIG. 5, the image feature extraction device 1 includes a plurality of basic configuration units 10 and 10B and parameter storage means 20. In FIG.

The basic configuration units 10 and 10B perform a convolution operation on input data, extract features, and produce output data, which are the same as those described with reference to FIGS.
Here, the image feature extraction device 1 is configured in multiple stages in the order of basic configuration units 10 ₁ , 10B ₂ , 10B ₃ and 10B ₄ .

The basic configuration unit 10 ₁ includes convolution means 11 ₁ (11), scaling means 12 ₁ (12), and addition means 13 ₁ (13).
The convolution means 11 ₁ performs two convolution layer CL ₁ and CL ₂ calculations on an image I of W (width)×H (height)×C (channel). Note that “3×3 conv, 64” in the convolution layers CL ₁ and CL ₂ indicates that 64 kernels of 3×3 size are used to perform the convolution operation.
As a result, the convolution means ₁₁₁ generates data of 64 channels from the image I with the same W (width)×H (height).
Kernel parameters (connection weight coefficients) used in the calculations of the convolution layers CL ₁ and CL ₂ performed by the convolution means 11 ₁ are pre-stored in the parameter storage means 20 as learned parameters.
The convolution means _11-1 outputs the data after the operation to the addition means _13-1 .

The scaling means ₁₂₁ multiplies each element of the matrix of the image I of W (width)×H (height)×C (channel) by a scaling coefficient which is a pre-learned parameter. Note that the learned scaling coefficients are stored in the parameter storage means 20 in advance.
The scaling means _12-1 outputs the data after the calculation to the adding means _13-1 .

The addition means _13-1 adds the data calculated by the convolution means _11-1 and the data calculated by the scaling means _12-1 .
The addition means _13-1 outputs the data (feature matrix) of the addition result to the subsequent basic configuration section 10B- ₂ .

The basic configuration unit 10B ₂ includes convolution means 11B ₂ (11B), scaling means 12B ₂ (12B), and addition means 13 ₂ (13).
The convolution means 11B ₂ performs calculation of two convolution layers CL ₃ and CL ₄ on the feature matrix calculated by the basic component 10 ₁ . Note that “3×3 conv, 128” in the convolution layers CL ₃ and CL ₄ indicates that 128 kernels of 3×3 size are used to perform the convolution operation. Also, "/2" in the convolutional layer CL ₃ indicates that the kernel is shifted with a stride of "2". Note that other convolutional layers CL without "/2" have a stride of "1".
As a result, the convolution means _11B2 generates W/2 (width)×H/2 (height), 128 channel data from the feature matrix.
Kernel parameters (connection weight coefficients) used in the calculations of the convolution layers CL ₃ and CL ₄ performed by the convolution means 11B ₂ are pre-stored in the parameter storage means 20 as learned parameters.
The convolution means 11B- ₂ outputs the data after the calculation to the addition means _13-2 .

The scaling means 12B ₂ aligns the feature matrix calculated by the basic component 10 ₁ to the same dimension as the matrix output by the convolution means 11B ₂ , and applies a scaling coefficient, which is a pre-learned parameter, to each element of the matrix. is multiplied by Note that the learned scaling coefficients are stored in the parameter storage means 20 in advance.
The scaling means 12B- ₂ outputs the data after the calculation to the addition means _13-2 .

The adding means _13-2 adds the data calculated by the convolution means 11B- ₂ and the data calculated by the scaling means 12B- ₂ .
The addition means _13-2 outputs the data (feature matrix) of the addition result to the subsequent basic configuration section 10B- ₃ .

The basic configuration units 10B ₃ and 10B ₄ have the same configuration as the basic configuration unit 10B ₂ except for the size of the kernel, so the explanation is omitted.
The basic configuration unit _10B4 at the final stage outputs the final calculation result as the feature amount V of the image I. FIG.

The parameter storage means 20 stores in advance the structure of the model M described in FIG. 4 and the learned parameters. The parameter storage means 20 can be composed of a general storage medium such as a semiconductor memory.

As described above, the image feature extraction apparatus 1 uses a model learned by weighting the route passing through the data in the lower layer without convolution when extracting the feature amount V from the image I by the convolution operation. . As a result, the image feature extracting apparatus 1 can extract the feature amount of the image with higher accuracy by reflecting the feature of the depth of the network with the weight.
Note that the image feature extraction device 1 can be operated by an image feature extraction program for causing a computer to function as each means described above.

<Operation of image feature extraction device>
Next, the operation of the image feature extraction device 1 according to the embodiment of the present invention will be described with reference to FIG. It is assumed that the parameter storage means 20 stores pre-learned parameters. Further, since which of the basic configuration unit 10 and the basic configuration unit 10B is used differs depending on the structure of the model M (FIG. 4), only the basic configuration unit 10 will be basically described here, and the basic configuration unit 10 , 10B, the difference will be explained.

In step S1, the frontmost basic configuration unit 10 inputs an image I as data.
In step S<b>2 , the convolution unit 11 of the basic configuration unit 10 refers to the kernel connection weight coefficients, which are parameters stored in the parameter storage unit 20 , and performs convolution operations in a plurality of convolution layers. At this time, when the basic configuration unit is the basic configuration unit 10B, the dimensions of the matrix of the input data and the matrix of the data after the convolution operation are different.

In step S3, if the dimensions of the matrix are different before and after the convolution operation, that is, if the operation in step S2 is performed in the basic configuration unit 10B (Yes in step S3), in step S4, the scaling means 12B performs the Linear projection is performed so that the input data of is aligned with the dimensions of the matrix after the convolution operation, and dimension conversion is performed.
On the other hand, when the dimensions of the matrix are the same before and after the convolution operation, that is, when the operation of step S2 is performed by the basic configuration unit 10 (No in step S3), the scaling means 12 does not perform the operation of step S4 and performs step The operation proceeds to S5.

In step S5, the scaling means 12 converts the input data or the input data dimensionally transformed in step S4 to each element of the input data by referring to the scaling coefficients, which are parameters stored in the parameter storage means 20. , multiplied by the scaling factor.

In step S6, the addition unit 13 adds the matrix resulting from the convolution operation in step S2 and the matrix multiplied by the scaling factor in step S5 element by element.

In step S7, if the basic configuration unit 10 is connected to the subsequent stage (Yes in step S7), return to step S2, and the subsequent basic configuration unit 10 performs a convolution operation on the data of the addition result of step S6. conduct.
On the other hand, if the basic configuration unit 10 is not connected to the subsequent stage (No in step S7), the basic configuration unit 10 outputs the calculation result as the feature amount V in step S8.
The image feature extraction device 1 can extract the feature amount from the image I by the above operation.

Although the configuration and operation of the image feature extraction device 1 according to the embodiment of the present invention have been described above, the present invention is not limited to this embodiment.
Here, in order to simplify the explanation, the number of stages of the basic configuration unit 10 (10B) is "4", and the number of convolution layers in the basic configuration unit 10 (10B) is "2". It does not matter if it is other than this. For example, the number of stages of the basic configuration unit 10 (10B) may be about 10 to 50 stages used in general neural networks. Also, the number of convolution layers in the basic configuration unit 10 (10B) may be three or more.

Also, here, the basic configuration units 10 (10B) are physically connected. However, one basic configuration unit may repeatedly perform calculations in accordance with the model M, thereby performing the same calculation as in the processing in which the basic configuration units are configured in multiple stages.

Further, here, the basic configuration unit 10B is configured to convert the dimension of the matrix (convert the width to 1/2 and the height to 1/2) between the input data and the output data. However, the basic configuration unit 10B connects the pooling means (not shown) that performs a pooling layer operation (maximum pooling, etc.) that reduces the width and height of the matrix (to 1/2 each) and the basic configuration unit 10. may be configured.

Further, when the image feature extraction device 1 extracts image classification feature amounts for classifying persons and objects appearing in an image, the number of channels output from the basic configuration unit 10 (10B) at the final stage is A classification number for classification may be used. Then, the image feature extraction device 1 has a configuration in which a global average pooling means for performing calculation of the global average pooling (GAP) layer is connected to the final stage basic configuration unit 10 (10B), and as an image classification device Can be configured.

Also, although a scaling layer is added here based on ResNet, the present invention can be applied to all network structures of CNN. For example, in Inception-ResNet described in the following reference, a scaling layer may be provided in the path through which data is transmitted to lower layers without convolution.
(References)
C. Szegedy, S. Ioffe, V. Vanhoucke, and A. Alemi,“Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning”, in Proc. CVPR, 2016.

<Parameter learning device>
Next, an example of the parameter learning device 2 that learns the CNN parameters used by the image feature extraction device 1 will be described with reference to FIGS. 7 and 8. FIG.
The parameter learning device 2 uses the CNN model (learning model) M2 shown in FIG. to learn.
The model M2 is a model obtained by adding a fully connected layer FL to the model M (FIG. 4).

The learning data includes, for example, learning images (face images) LI obtained by photographing a plurality (approximately 100) of a large number (approximately 100,000) of people, and correct data LC, which are the labels of the persons appearing in the learning images LI. Paired data can be used.
The parameter learning device 2 learns the parameters of the model M that constitutes the model M2 by learning the parameters of the model M2.

As shown in FIG. 8, the parameter learning device 2 includes a plurality of basic configuration units 10 and 10B, parameter storage means 20, full coupling means 30, and error calculation means 40.
Here, the parameter learning device 2 has the same configuration as the image feature extraction device 1 and is configured in multiple stages in the order of basic configuration units 10 ₁ , 10B ₂ , 10B ₃ and 10B ₄ .

As in the configuration of the image feature extraction device 1, the basic configuration units 10 and 10B refer to the parameters stored in the parameter storage means 20, perform convolution operations on the input data, and It scales the input data and adds them together.
In addition, the basic configuration units 10 and 10B of the parameter learning device 2 update the coupling weight coefficients and scaling coefficients by the error backpropagation method based on the error input from the latter stage of the network, and propagate the error to the previous stage. .

The parameter storage means 20 stores the structure and parameters of the model M2 explained in FIG. Initial values such as random numbers are set in advance for the parameters. The parameters of the parameter storage means 20 are updated by the basic configuration units 10 and 10B and the total coupling means 30 .

The total combining means 30 refers to the parameters stored in the parameter storage means 20, and converts each element of the data sequentially convoluted by the basic configuration units 10 ₁ , 10B ₂ , 10B ₃ and 10B ₄ into one or more The fully-connected layer performs a fully-connected operation to generate a one-dimensional vector of a predetermined data length. This calculation result is the recognition result R obtained by recognizing (here, face recognition) the learning image LI.
Further, the full coupling means 30 updates the coupling weight coefficients by the error backpropagation method based on the error input from the error computing means 40, and propagates the error to the basic configuration section 10 in the previous stage.
The fully-connected means 30 has a fully-connected layer added at the final stage for aligning the dimensions with the data of the correct data LC.

The error computing means 40 computes the error between the output (recognition result R) of the full combining means 30 and the correct data LC. The error computing means 40 outputs the error to the full coupling means 30 . Note that the error computing means 40 repeats the operation of the basic configuration units 10 and 10B and the total coupling means 30 a predetermined number of times or until the degree of change in the parameter stored in the parameter storage means 20 falls below a predetermined threshold value. make it work.

As described above, the parameter learning device 2 can learn CNN parameters used by the image feature extraction device 1 by using learning data.
Thereby, the parameter learning device 2 can learn parameters of a model for extracting image features for recognizing a person's face.

Note that although an image and a label of a person appearing in the image are used as the learning data here, various learning data may be used according to the object from which the image feature is to be extracted. For example, if you want to learn parameters for identifying objects (eg, animals) in images, you can use images and corresponding object labels as learning data. In that case, the dimension of the fully connected layer added to the final stage of the fully connecting means 30 should be matched with the number of labels of the object.

Further, here, the total coupling means 30 is provided, but if the number of channels of the outputs of the final-stage basic configuration units 10 and 10B is the number of object labels, the global Pooling means may be provided for performing average pooling (GAP) layer operations. This reduces the number of parameters to learn.

<Performance evaluation when scaling layer is provided>
A performance comparison was performed between Inception-ResNet V2 (conventional method) described in the above-mentioned reference and CNN (this method) provided with a scaling layer in Inception-ResNet V2. In addition, both the conventional method and the present method perform learning with the same learning data.

For performance evaluation, an LFW (Labeled Faces in the Wild) dataset (http://vis-www.cs.umass.edu/lfw/) widely used in the field of face recognition was used.
This data set has about 6000 pairs of two face images and correct data indicating whether or not the person shown in the two images is the same person.
Here, a feature amount is extracted for each of the two face images, and if the distance of the feature amount (Euclidean distance, etc.) is equal to or less than a predetermined threshold value, it is determined that the two images are the same person. If it is larger than the threshold, it is determined that the persons appearing in both images are not the same. This determination result is compared with the correct data, and the rate of correct determination is defined as the recognition accuracy.
The results of recognition accuracy using the conventional method and this method are shown in the table below.

As shown in this table, the accuracy of face recognition using the present method was higher than that using the conventional method.
Thus, in this method, by providing a scaling layer in ResNet, the recognition accuracy is improved and the feature amount can be more accurately extracted from the image.

1 image feature extraction device 10, 10B basic configuration unit 11, 11B convolution means 12, 12B scaling means 13 addition means 20 parameter storage means 30 full connection means 40 error calculation means I image V feature quantity M model CL convolution layer SL scaling layer FL fully connected layer

Claims (4)

An image feature extraction device for extracting a feature amount of the image from image data,
A convolution means for performing operations of a plurality of convolution layers on input data using pre-learned kernels as parameters of the convolution neural network;
scaling means for multiplying the input data by the scaling factor, using a pre-learned scaling factor as a parameter of the convolutional neural network;
a basic configuration unit having a multi-stage configuration including addition means for adding the operation result of the convolution means and the operation result of the scaling means;
An image feature extracting apparatus characterized in that the calculation result of a final-stage basic configuration unit is used as the feature amount. At least one or more of the basic constituent parts
The convolution means generates a calculation result of a dimension different from the input data in the calculation of a plurality of convolution layers,
2. The image feature extracting apparatus according to claim 1, wherein said scaling means converts said input data into data of said dimensions by linear projection, and then multiplies said data by said scaling factor. 3. When the feature amount is used as the feature amount for image classification, the number of channels of the data output from the final-stage basic configuration unit is used as the classification number for image classification. An image feature extraction device as described. An image feature extraction program for causing a computer to function as the image feature extraction device according to any one of claims 1 to 3.

Images