JPWO2018207334A1 - Image recognition apparatus, image recognition method, and image recognition program - Google Patents

Abstract

The image recognition apparatus according to the present invention uses at least one or more first images having a predetermined attribute shared by authentication objects, and uses a second image that is an unknown image whether or not the attribute is present. Parameter calculating means for calculating one or more parameters for extracting the attribute-likeness or the feature representing the attribute-likeness, feature extracting means for extracting a feature from the input image using the parameters, and at least Determination means for determining whether or not the second image is an authentication target based on the feature extracted from the second image, and the feature extraction means combines calculation elements including predetermined parameters. In the obtained neural network, the number of calculation elements that form two or more layers from input to output and belong to at least one layer is the number of input layers to which image information is input. Using a small neural network neural network than the number of calculation elements, extracts feature.

Description

  The present invention relates to an image recognition apparatus, an image recognition method, and an image recognition program that learn a sample image and recognize an image.

  An example of an image recognition apparatus is described in Patent Document 1. In the method described in Patent Literature 1, first, the image input unit normalizes the position and size of the face of the target person with respect to the face image set, and smoothes the image histogram for correcting the luminance change. Thereafter, shading correction is performed to remove the influence of shadows, and image compression and normalization are performed. Next, each face image included in the face image set is subjected to KL expansion (Karhunen-Loeve expansion) to obtain an eigenvalue and a coefficient, and a coefficient value is input as a feature amount. The neural network is learned so that it can be output. In the method described in Patent Document 1, when determining whether or not an unknown image is a face image, the unknown image is expressed on the eigenvector space of the KL expansion obtained above and the obtained value ( Determination is performed by inputting a coefficient as a feature amount into the above-described neural network.

  The method described in Patent Document 1 uses only an image having a “face” as a recognition target to construct an eigenvector space that expresses the face-likeness, and obtains features used for learning of a neural network therefrom.

  In addition, as a technique related to the present invention, an appendix of Non-Patent Document 1 describes greedy layer training for each layer as an example of a learning method in deep learning such as a neural network. Non-Patent Document 2 describes an example of a neural network called an adversary autoencoder among neural networks called an autoencoder. In FIG. 7 of the present application, p. 2 disclosed in FIG. A schematic diagram of 1 is shown. Non-Patent Document 3 describes an example of a neural network called a denoising autoencoder.

Japanese Patent Laid-Open No. 11-25269

Yoshua Bengio, et al., "Greedy Layer-Wise Training of Deep Networks", proceedings on Neural Information Processing Systems, 2006, Appendix of the paper (p.5). Alireze Makhzani, et al., "Adversarial Autoencoder", Proceedings of International Conference on Learning Representations, 2016. P. Vincent, et al., "Extracting and Composing robust features with denoising autoencoders", Proceedings of International Conference on Machine Learning (ICML), 2008, pp.1097-1098.

  Although the method described in Patent Document 1 assumes recognition of a face image, it can be applied to any authentication target other than a face in principle. For example, as feature extraction for determining whether or not a specific product produced in a factory is normal from a product image, it is also possible to construct an eigenvector space that expresses the quality of a normal product from a set of normal product images .

  If the eigenvector space that expresses the quality of normal products in this method is constructed with high accuracy, even if it is difficult to obtain an abnormal sample image related to the recognition target object, learning is performed mainly on the normal sample image. Thus, it can be accurately determined whether or not it is a recognition target.

  In the present invention, an image that is determined not to be a recognition target, more specifically, an image that does not have a predetermined attribute that the recognition target has in common is referred to as an “abnormal sample image”. An image to be determined as a recognition target, more specifically, an image having a predetermined attribute shared by the recognition target is referred to as a “normal sample image”.

  However, since the method described in Patent Document 1 forms the eigenvector space by a simple method, there is a problem that the determination accuracy is deteriorated when the number of dimensions of the eigenvector space is small.

  For example, in the method described in Patent Document 1, when an image is reduced to an eigenvector space, only eigenvectors corresponding to the number of dimensions of the eigenvector space are used from the eigenvector having the larger eigenvalue. For this reason, the coefficient of the eigenvector used for determination is limited to the coefficient of the eigenvector. In such a method, when the eigenvalue corresponding to the eigenvector that has not been used is relatively large, the loss of information with respect to the original image becomes large, and the accuracy of the determination is reduced.

  In view of the above problems, an object of the present invention is to provide an image recognition apparatus, an image recognition method, and an image recognition program that can accurately recognize an image even in a situation where an abnormal sample image related to a recognition target is difficult to obtain.

  The image recognition apparatus according to the present invention uses at least one or more first images having a predetermined attribute shared by the authentication targets, and uses a second image which is an image whose unknown whether or not the attribute is present. Parameter calculating means for calculating one or more parameters for extracting the attribute-likeness or the feature representing the attribute-likeness, feature extracting means for extracting a feature from the input image using the parameters, and at least Determination means for determining whether or not the second image is an authentication target based on a feature extracted from the second image, and the feature extraction means is obtained by combining calculation elements including the parameter. The number of calculation elements that constitute two or more layers from input to output and that belong to at least one layer is the number of input layers to which image information is input. Using small neural network than the number of calculation elements, and extracting a feature.

  The image recognition method according to the present invention is an image in which the information processing apparatus uses at least one or more first images having a predetermined attribute shared by the authentication target and whether or not the attribute is unknown. Calculating one or more parameters for extracting the attribute-likeness or features representing the attribute-likeness from a second image, and using the parameters to extract features from the input image; A neural network obtained by determining whether or not a second image is an authentication target based on features extracted from an image and combining calculation elements including the parameters when extracting the features. The number of calculation elements that constitute two or more layers from the input to the output and that belong to at least one layer is smaller than the number of calculation elements in the input layer to which image information is input. Characterized by using network.

  The image recognition program according to the present invention uses at least one or more first images having a predetermined attribute shared by authentication objects in a computer, and the second image is an unknown image having the attribute. A parameter calculation process for calculating one or more parameters for extracting the attribute-likeness or the feature representing the attribute-lessness from the image, and a feature extraction process for extracting a feature from the input image using the parameter; And a determination process for determining whether or not the second image is an authentication target based on at least a feature extracted from the second image, and a calculation element including the parameter is combined in the feature extraction process A neural network obtained by the above method, in which the calculation elements form two or more layers from input to output, and the number of calculation elements belonging to at least one layer is an image. Information using a small neural network than the number of calculation elements of the input layer is inputted, characterized in that to extract features.

  According to the present invention, an image can be recognized with high accuracy even in a situation where an abnormal sample image related to a recognition target is difficult to obtain.

It is a block diagram showing an example of image recognition device 10 of a 1st embodiment. It is explanatory drawing which shows the example of the neural network which the feature extraction means 11 uses. It is a flowchart which shows an example of operation | movement of learning step ST1 of 1st Embodiment. It is a flowchart which shows an example of operation | movement of determination step SD1 of 1st Embodiment. It is explanatory drawing which shows the example of determination of th in the determination means. It is a block diagram which shows the example of the image recognition apparatus 20 of 2nd Embodiment. It is explanatory drawing which shows typically the structural example of a hostile self-encoder. It is a flowchart which shows an example of operation | movement of learning step ST2 of 2nd Embodiment. It is a flowchart which shows an example of operation | movement of determination step SD2 of 2nd Embodiment. It is a block diagram which shows the example of the image recognition apparatus 30 of 3rd Embodiment. It is a flowchart which shows an example of operation | movement of learning step ST3 of 3rd Embodiment. It is a flowchart which shows an example of operation | movement of determination step SD3 of 3rd Embodiment. It is a schematic block diagram which shows the structural example of the computer concerning embodiment of this invention. It is a block diagram which shows the outline | summary of this invention.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

Embodiment 1. FIG.
[Description of configuration]
FIG. 1 is a block diagram illustrating an example of an image recognition apparatus 10 according to the present embodiment. The image recognition apparatus 10 illustrated in FIG. 1 includes a feature extraction unit 11, a parameter calculation unit 12, and a determination unit 13.

  The feature extraction unit 11 extracts features from an input image to be recognized (hereinafter referred to as a test image). The feature extraction unit 11 has one or more parameters 111 for extracting features for determining the object likeness to be determined, and performs a predetermined calculation using the parameters 111 on the test image. The feature is extracted by performing. Here, the parameters 111 are not particularly limited as long as the calculation characteristics of the features in the feature extraction unit 11 change according to the values. Hereinafter, the parameter 111 may be expressed as a feature extraction parameter. In addition, the feature extracted from the image by the feature extraction unit 11 may be expressed as an image feature.

  The parameter calculation unit 12 calculates each value of the parameter 111 used by the feature extraction unit 11 for feature extraction. The value of the parameter 111 is generally calculated using a set of learning images prepared in advance such as a normal sample image and an abnormal sample image, but is not limited thereto. In addition, the calculation method of the parameter 111 of this embodiment is mentioned later.

  The determination unit 13 determines whether or not the test image is a recognition target based on the image feature extracted by the feature extraction unit 11.

  Hereinafter, in the present embodiment, an example in which an object (object, part, part, etc.) having a predetermined attribute is set as a recognition target will be described. In this case, the predetermined attribute is a property common to the object of interest in the image, and can be arbitrarily set. Examples of the predetermined attributes include “person (in the sense of a specific species in animals and plants)”, “specific individual”, and “(part of a person having a predetermined part such as eyes, nose and mouth) with respect to the subject. No.) ”and“ non-defective product ”(in the sense of non-defective product quality) for specific objects produced at the factory. In the following, a case where “a non-defective product of a specific object produced in a factory” is a recognition target will be described as an example.

  When attention is paid to a predetermined attribute shared by such authentication targets, the relationship between the constituent elements can be explained as follows, for example. That is, the parameter calculation unit 12 determines the value of the parameter 111 included in the feature extraction unit 11 using an image set (normal sample image set) having a predetermined attribute, and the feature extraction unit 11 determines the determined parameter 111. Based on the value of the test image, a feature for determining the likelihood of a predetermined attribute of the image is extracted from the test image, and the determination unit 13 determines the test image based on the image feature obtained from the test image. It is determined whether it has the attribute of. If it is determined that the test image has a predetermined attribute, the test image is determined to be an authentication target. If the test image is determined not to have the predetermined attribute, the test image is not an authentication target. It is determined.

  At this time, the parameter calculation unit 12 learns a function for converting the input image into a feature space that best restores the input image according to the target object (in this example, having a predetermined attribute), and obtains it as a learning result. The value of the parameter 111 may be determined from the obtained function. The feature extraction unit 11 may convert the learning image and the test image into the feature space based on the value of the parameter 111 obtained as a result of learning. Further, the determination unit 13 may determine whether the object shown in the test image is a target object based on the proximity between them in the feature space converted by the feature extraction unit 11. Good.

  A preferred example of the feature extraction method in the feature extraction means 11 is a method using a neural network. Note that the feature extraction unit 11 itself may be realized by a program constituting a neural network (more specifically, a processor that operates according to the program).

A neural network is a calculation model obtained by combining calculation elements including specific parameters. As the calculation element, for example, an input signal x = (x1, x2,..., Xp) using a weight wa = (wa1, wa2,..., Wap) ^T which is a parameter specific to the element and a bias ba. ) To calculate and output the value of f (wa.x + ba). Here, a represents the unit number of the calculation element. T represents vector transposition. p represents the number of input signals. “·” Represents an inner product of vectors. Further, f is called an activation function, and for example, a sigmoid function, a ReLU function, or the like is used.

  Here, a set of calculation elements included in the neural network can be divided into a plurality of subsets to form a hierarchy. Calculation of parameters (weights, biases) of calculation elements included in such a hierarchical neural network is performed by a set of “learning samples and a pair of teacher signals for learning samples” (hereinafter referred to as learning sample pairs). (Hereinafter referred to as a learning sample set). This parameter calculation is sometimes called learning. Therefore, it can be said that the parameter calculation means is learning means for learning the neural network.

  One specific parameter calculation method is the error back propagation method. The error back propagation method uses known optimization techniques such as steepest descent so that the final output of the hierarchical neural network is as small as possible with the teacher signal. (Bias) update. In principle, the parameter has the property of approaching the optimum value by updating the parameter multiple times. Here, the learning sample used for one calculation does not necessarily need to be all image sample pairs belonging to the learning sample set, but only some image sample pairs (partial image sample pair set) belonging to the learning sample set. May be used.

  In addition, when the parameter calculation is performed using the partial image sample pair set, the method of taking the partial image sample pair set may be randomly selected at each iteration. For example, when the steepest descent method is used as an optimization method, it is called a stochastic gradient method. If the calculation results of the parameters of the calculation element finally obtained in C iterations are compared with the results of two independent trials, the parameters are calculated based on different partial image sample pair sets. Generally, even if they generally match, they do not exactly match. In addition, the initial value of the calculation element parameter may be randomly given before the parameter calculation. In this case, the final calculation result of the calculation element parameter does not exactly match. In addition, the final calculation results of the parameters of a plurality of calculation elements using different optimization methods are generally different. These are examples where the parameter calculation method of the calculation element generates a probabilistic result. As described above, the parameter calculation means 12 may obtain the parameters by a probabilistic method such as random selection of the initial value of the weight of the neural network or the learning sample to be selected during learning, or using a different optimization method. Good. Modifications that can be configured using such properties will be described later separately. The parameter calculation unit 12 can use a known learning method other than the above.

  Hereinafter, the neural network includes two or more layers including an input layer for inputting image information, and the number of calculation elements in at least one layer is smaller than the number of calculation elements in the input layer. An example of a configured auto encoder type neural network will be described. In this case, the feature obtained from the image is information output from at least one layer of the auto-encoder neural network.

  FIG. 2 is an explanatory diagram illustrating an example of a neural network used by the feature extraction unit 11 when extracting features. The example neural network shown in FIG. 2 includes seven layers including an input layer and an output layer. Each layer includes one or more computing elements. Circles represent computing elements. When such a neural network is used, the feature extraction unit 11 may use the output of the fourth layer counted from the input layer as an image feature. The neural network shown in this example is a type of neural network called the above-described auto encoder type, that is, a self-encoder. Although the self-encoder in a narrow sense is composed of three layers, in recent years, a configuration expanded to multiple layers has been proposed. Therefore, also in this specification, the network configuration of the self-encoder is not limited to three layers, and generally has a plurality of layers, and that the number of calculation elements of at least one layer is smaller than the number of elements of the input layer. This is a configuration requirement.

  When using an auto-encoder type neural network, as shown in FIG. 2, it is a preferable example to use the output value of the intermediate layer having the minimum number of elements as the image feature. It can also be used as a feature.

  Regarding the calculation of the value of the parameter 111, the parameter calculation unit 12 preferably uses the image set having a predetermined attribute to learn the parameter of each calculation element in the neural network as described above and determine the value. It is. In the above example, the parameter calculation means 12 may calculate the value of the parameter 111 using a set of pairs of normal sample images as learning samples and normal sample images as teacher signals.

  A suitable example of the determination method in the determination unit 13 is to calculate a distance between an image feature obtained from an image having a predetermined attribute and an image feature obtained from a test image, and perform a test based on the distance. This is a method for determining whether an image has a predetermined attribute. As the distance in this case, an existing distance may be used. For example, Euclidean distance, city area distance, and the like can be used, but the distance is not limited thereto. It is also possible to use similarity instead of distance. As the degree of similarity, for example, an inner product between vectors when the feature is regarded as a vector and an angle formed by the vector can be used, but are not limited thereto. In the example described later, the Euclidean distance is used for the determination. Further, when using the similarity, the determination criterion may be reversed as compared with the case of distance, but the description is omitted because it is self-evident.

  Note that the determination unit 13 extracts a predetermined feature amount (for example, proximity between them in the feature space) based on each feature from each of the image set having the predetermined attribute and the test image, Based on the extracted feature quantity, it may be determined whether or not the test image has a predetermined attribute.

[Description of operation]
Next, the operation of this embodiment will be described. The operation of this embodiment is roughly divided into a learning step ST1 and a determination step SD1. Here, it is assumed that the learning step ST1 is performed prior to the determination step SD1.

  FIG. 3 is a flowchart showing an example of the operation of the learning step ST1 of the present embodiment. In the learning step ST1, the value of the parameter 111 is determined mainly using a learning image having a predetermined attribute. In the example shown in FIG. 3, first, the parameter calculation means 12 calculates the value of the parameter 111 using a set of learning images given in advance (step ST11).

  In step ST11, the parameter calculation means 12 performs learning such that, for example, when a learning image is given to the input layer of the neural network, the output layer becomes the learning image itself. The learning method may be a known method. For example, when the number of layers of the neural network is 3, learning may be performed using a known method such as an error back propagation method. When the number of layers is 4 or more, for example, greedy learning for each layer described in Non-Patent Document 1 can be used.

  In the method described in Non-Patent Document 1, first, three layers are formed in a manner partially described above with respect to three layers close to the input layer (input layer (= 1 layer), second layer, and third layer). The parameters of the neural network computing element are determined. Thereafter, the bias of the first layer and the coupling weights of the first layer to the second layer are fixed, and parameters are similarly obtained for the second layer, the third layer, and the fourth layer. When this procedure is finally repeated until the third layer becomes the output layer, the learning ends. After learning is completed, the calculated element parameter values are recalculated from the input layer to all the output layers using the error back propagation method, etc. (generally called Fine Tuning). May be.

  Next, the feature extraction unit 11 calculates an image feature for each learning image using the value of the parameter 111 calculated by the parameter calculation unit 12 (step ST12). For example, when the neural network shown in FIG. 2 is obtained by learning, the feature extraction unit 11 can use the output values of the two calculation elements in the fourth layer as image features as described above.

  FIG. 4 is a flowchart showing an example of the operation of the determination step SD1 of the present embodiment. In the determination step SD1, an image feature is calculated from the test image based on the determined value of the feature extraction parameter to determine the test image. In the example shown in FIG. 4, first, the feature extraction unit 11 extracts an image feature from the test image using the value of the parameter 111 calculated by the parameter calculation unit 12 in the learning step ST1 (step SD11).

  Next, the determination unit 13 determines whether the test image has a predetermined attribute by comparing the image feature of the learning image obtained in step ST12 with the learning feature of the test image obtained in step SD11. (Step SD12).

  As an attribute determination method, for example, there are the following methods. That is, the determination unit 13 uses the image feature of the learning image whose distance from the image feature of the test image is nth (n is an integer equal to or greater than 1), and the distance Dist_n between the image features is smaller than the real number th. In some cases (or less than th), it may be determined that the test image has a predetermined attribute. Here, the values of n and th are arbitrarily determined. When n = 1, the above method corresponds to determination using the nearest learning image.

  Further, n and th according to the determination method can be determined by the following method. For example, when n is fixed to one arbitrary value, when all learning images are used as test images while th is gradually decreased from a large value to a small value, the test image (learning image) is correctly determined. Th is obtained so that the rate of detection (detection rate) is 100%. The smallest th among such th is used. For example, when a learning image (normal sample image) having a predetermined attribute is used as a test image, a rate at which the test image (learning image) has a predetermined attribute may be obtained as a detection rate.

  In the above method, if th is reduced, the result of the determination by the determination means 13 having the predetermined attribute is less likely to include those that do not have the predetermined attribute, but there is a tendency for many leaks to occur. By using the smallest th among ths that satisfy the detection rate of 100%, it is expected that the determination result of the test image including the image not having the predetermined attribute will be omitted as much as possible and that the determination result does not include an error. The

  FIG. 5 is an explanatory diagram illustrating an example of determining th in the determination unit 13. FIG. 5A is an explanatory diagram showing an example of determining th when n = 1 is fixed, and FIG. 5B is a case where n is not fixed, that is, when both n and th are varied. It is explanatory drawing which shows the example of determination of n and th.

  FIG. 5A shows the detection rate when the value of th is decreased from 0.05 to 0.05 in increments of 0.05 while the value of n is fixed to 1. In this example, th that achieves a detection rate of 100% is three, 0.2, 0.15, and 0.1. Therefore, the smallest th = 0.1 is adopted.

  Further, as shown in FIG. 5B, both n and th can be varied. In this case, a value in the vicinity of the boundary between an area indicating a set of values where the detection rate is 100% and an area indicating a set of values not being 100% may be employed. For example, in the example shown in FIG. 5B, a set of values indicated by a dashed frame, that is, (n, th) = (1, 0.1), (51, 0.15), (101, 0.15). ), (151, 0.2) or the like may be used. Further, as shown in the first modification, determination may be tried independently using a plurality of value sets, and the final determination result may be obtained by summing up the results.

  Further, an image other than the learning image may be used as the test image for determining the values of n and th. In addition, only an image having no predetermined attribute may be used as a test image. In this case, it is only necessary to obtain th that achieves 100% as a detection rate that does not have a predetermined attribute. And the largest th among such th may be adopted. In this case, the determination unit 13 determines whether or not the test image has a predetermined attribute.

  Of course, the values of n and th can be determined using an image set in which an image having no predetermined attribute and an image having the predetermined attribute are mixed. In that case, it is desirable to have a label relating to the correct answer (whether or not it has an attribute) for each image in the image set.

[Description of effects]
In the present embodiment, the parameter 111 is preferably learned even when the learning image is only an image having a predetermined attribute (normal sample). Therefore, even when only a small number (abnormal samples) having no predetermined attribute is available or not available at all, it is possible to determine whether the unknown sample is normal or abnormal with high accuracy.

  In particular, when the function of the feature extraction means 11 for calculating the feature from the image is regarded as a mathematical function by using a neural network for learning of the feature extraction parameter, a more complicated function than the case of the principal component analysis is obtained. Can learn. For this reason, more accurate determination can be realized. In general, the distance between the image feature space and the actual image dissimilarity does not necessarily match, but if the image features of the test image and the learning image are close to each other, an arbitrary distance value such as the Euclidean distance is set. Using this method to approximate the degree of difference between images is a commonly performed operation, and in principle, highly accurate attribute determination can be expected.

Embodiment 2. FIG.
[Description of configuration]
Next, a second embodiment of the present invention will be described. Since a part of the configuration of the present embodiment is the same as that of the first embodiment, different components will be mainly described below. FIG. 6 is a block diagram illustrating an example of the image recognition apparatus 20 according to the second embodiment. The image recognition apparatus 20 illustrated in FIG. 6 includes a feature extraction unit 21, a parameter calculation unit 22, and a determination unit 23.

  The feature extraction unit 21 extracts a feature (image feature) from an input image (test image) to be recognized, and calculates a distance value to be described later. The feature extraction unit 21 has one or more parameters 211 as in the first embodiment, and extracts image features by performing a predetermined calculation using the parameters 211 on the test image. .

  The feature extraction unit 21 of the present embodiment extracts features using a neural network called a hostile self-encoder as shown in Non-Patent Document 2 among auto-encoder neural networks.

  The feature of the hostile self-encoder is that learning (that is, calculation of the parameter 211) according to a pre-specified distribution such as an m-dimensional normal distribution (m is an integer of 1 or more) or an m-dimensional mixed normal distribution can be performed. It is. Therefore, if a hostile self-encoder is used, it is possible to obtain image features according to a distribution designated in advance, such as an m-dimensional normal distribution (m is an integer of 1 or more) and an m-dimensional mixed normal distribution.

  FIG. 7 is an explanatory diagram schematically illustrating a configuration example of a hostile self-encoder disclosed in Non-Patent Document 2. The feature extraction means 21 may extract features using, for example, a hostile self-encoder as shown in FIG. In FIG. 7, p (z) represents a positive sample. Q (Z) represents a negative sample. In addition, “Adversary cost” in the figure is a cost for distinguishing a negative sample from a positive sample. The upper part of FIG. 7 corresponds to a self-encoder, and the lower part corresponds to an identifying network described later.

  The calculation (learning) method of the calculation element parameter of the neural network having such a configuration includes a restructuring phase and a regularization phase. In the reconstruction phase, for example, when a learning sample is input, the self-encoding inherent in the hostile self-encoder so that the output becomes the learning sample itself so that an output that reconstructs the input image can be obtained. Learn the vessel. In the regularization phase, the discriminative network inherent in hostile self-coding (a network that identifies whether the input sample is a sample resulting from a commanded distribution or a sample generated by a self-encoder) To learn the identification network.

  The parameter calculation unit 22 obtains each value of the parameter 211 used for feature extraction by the feature extraction unit 21 by the calculation (learning) method proposed for the hostile self-encoder described above. Details of the configuration of the hostile self-encoder and the parameter calculation method are described in Non-Patent Document 2 above.

  Hereinafter, a case where an m-dimensional normal distribution is designated for the hostile self-encoder will be described as an example.

  The determination means 23 can use the determination means 13 of the first embodiment, but in the following, the characteristics of the image features extracted by the feature extraction means 21, that is, the predetermined distribution used by the hostile self-encoder for learning. A determination method using the property of will be described.

  For example, when an image feature follows an m-dimensional normal distribution, a distance such as an Euclidean distance or a Mahalanobis distance can be calculated between a point on the space and an average vector. The determination unit 23 is based on the distance between the point in the m-dimensional space where the image feature calculated for the test image exists and the point in the m-dimensional space that is the average vector of the image feature calculated for the learning image. Thus, it may be determined whether the test image has a predetermined attribute. In addition to the average vector, for example, a value of a variance covariance matrix can be used. In addition to the distance, it is also possible to use an arbitrary value (index) obtained from the value of the average vector or the variance-covariance matrix. In this way, a value indicating the probability of occurrence of the image feature of interest with respect to the probability for attribute determination, more specifically, the probability distribution assumed to follow the set of image features is obtained. May be.

  In particular, in an m-dimensional normal distribution, the probability can be calculated for an area in an m-dimensional space, so a statistical test with a null hypothesis that a test image has a predetermined attribute is performed. It becomes possible to do.

  As described above, when the image feature follows a predetermined distribution, the determination unit 23 uses parameters in the predetermined distribution (parameters that determine the shape of the distribution such as an average vector of an m-dimensional normal distribution or a variance-covariance matrix). The presence / absence of an attribute may be determined using any calculated index.

  In this embodiment, the determination unit 23 may determine the presence / absence of an attribute with respect to the distance by applying a method for determining th when n is fixed to an arbitrary value. Further, when the determination unit 13 of the first embodiment is used as the determination unit 23, the determination unit 23 assumes that the learning algorithm is probabilistic, for example, a plurality of determination results for one test image. Alternatively, a plurality of indices may be obtained, and the test image may be determined based on the obtained determinations or the plurality of indices. Note that the method of obtaining a plurality of determination results or a plurality of indices may calculate a parameter each time and perform a plurality of determinations, or include a plurality of pairs of parameter calculation means and feature extraction means, and test each of them. An image may be input to perform feature extraction, and a plurality of determination results or a plurality of indexes may be obtained based on the result. It is also possible to provide a plurality of image recognition means for performing from parameter calculation to feature extraction and determination, and obtaining a determination result by inputting a test image for each. The same applies to other embodiments.

  The determination means 23 determines the presence or absence of an attribute using one or more of these methods.

[Description of operation]
Next, the operation of this embodiment will be described. The operation of this embodiment is roughly divided into a learning step ST2 and a determination step SD2. Also in the present embodiment, the learning step ST2 is performed prior to the determination step SD2.

  FIG. 8 is a flowchart showing an example of the operation of the learning step ST2 of the present embodiment. In the learning step ST2, the value of the parameter 211 is determined mainly using a learning image having a predetermined attribute. In the example shown in FIG. 8, the parameter calculation means 22 calculates the value of the parameter 211 using a set of learning images given in advance (step ST21). Thereafter, the feature extraction unit 21 may calculate an average vector in a predetermined distribution (for example, an m-dimensional normal distribution) as an image feature of the learning image set using the calculated value of the parameter 211.

  FIG. 9 is a flowchart illustrating an example of the operation of the determination step SD2 of the present embodiment. In the determination step SD2, the distance from the average vector of the test image is calculated based on the determined value of the parameter 211 to determine the test image. In the example shown in FIG. 9, the feature extraction unit 21 first extracts an image feature from the test image using the value of the parameter 211 calculated by the parameter calculation unit 12 in the learning step ST2 (step SD21). In the present embodiment, the feature extraction unit 21 extracts image features according to the m-dimensional normal distribution, more specifically, points on the m-dimensional section.

  Next, the determination unit 223 calculates a distance from the average image feature vector of the learning image from the image feature of the test image obtained in step SD221, and when the distance is smaller than a predetermined value, the test image Is determined to have a predetermined attribute (step SD22).

[Description of effects]
In the present embodiment, the feature extraction parameter is calculated so that the image feature calculated by the feature extraction unit 21 follows a probability distribution specified in advance. Then, based on the distance calculated on the probability distribution for the image feature extracted from the test image using such a feature extraction parameter, a predetermined attribute likelihood for the test image is determined. For this reason, according to the present embodiment, it is possible to give predetermined attribute-likeness with a probabilistic index.

Embodiment 3. FIG.
[Description of configuration]
Next, a second embodiment of the present invention will be described. Since a part of the configuration of the present embodiment is the same as that of the first embodiment, different components will be mainly described below. FIG. 10 is a block diagram illustrating an example of the image recognition device 30 according to the third embodiment. The image recognition apparatus 30 shown in FIG. 10 includes a feature extraction unit 31, a parameter calculation unit 32, and a determination unit 33.

  The feature extraction unit 31 extracts a noise component as a feature (image feature) from an input image (test image) to be recognized. The feature extraction unit 31 has one or more parameters 311 as in the first embodiment, and extracts image features by performing a predetermined calculation using the parameters 311 on the test image. .

  The feature extraction means 31 of the present embodiment extracts features (noise components) from the test image using a neural network called a denoising encoder among the auto encoder type neural networks.

  The denoising encoder is configured to output the original data from the input data when the input data is data that is abnormal (damaged, etc.) in part due to noise added to the original data. Is done. In this denoising auto encoder, the calculation element parameters are calculated by adding the noise to the original learning sample as the learning sample for the denoising encoder, and the teacher signal as the learning sample before adding noise. A known method such as a back propagation method may be used. Accordingly, the denoising auto encoder can have a feature that can remove noise in the input image in addition to the feature of the auto encoder type neural network. At this time, the noise component removed by the denoising auto encoder corresponds to a component recognized as a noise component by learning using one or more normal sample images or a component extracted by a method similar to the component. It can be said that it is one of the features that express the lack of attributes.

  The parameter calculation unit 32 obtains each value of the parameter 311 used by the feature extraction unit 31 for feature extraction so as to remove noise from the input image. More specifically, the parameter calculation means 32 calculates | requires with the calculation (learning) method proposed for the denoising encoder. The configuration of a neural network called a denoising encoder is described in Non-Patent Document 3 above, for example.

  The parameter calculation unit 32 calculates, for example, a feature extraction parameter so that a learning image obtained by artificially adding noise to a learning image is input and a learning image before adding noise is output as an output. I do. At this time, if it is possible to use a case where there is an abnormality in a part of an image having a predetermined attribute, the image may be learned as an input.

  The feature extraction unit 31 obtains a noise-removed image from the test image using the parameter 311 learned as described above, and obtains the difference image between the obtained noise-removed image and the test image. It may be extracted as a noise component. In other words, in this embodiment, difference information between the information (test image) input to the input layer of the learned denoising auto encoder and the information (noise-removed image) output from the output layer is used as an image feature. .

  The determination unit 33 determines whether the test image has a predetermined attribute based on the noise component (difference image) obtained from the test image by the feature extraction unit 31. The determination unit 33 may determine the test image based on the magnitude of the difference. For example, the determination unit 33 may calculate the sum of the pixel values of the difference image and determine that the predetermined attribute has a predetermined attribute when it is equal to or less than a predetermined value, but is not limited thereto.

  The basic concept of this embodiment is to extract only the noise component by taking the difference between the input image and its noise-removed image, and if the extracted noise component is small, it is determined that it has a predetermined attribute, and vice versa. In other words, it is determined that it does not have a predetermined attribute.

[Description of operation]
Next, the operation of this embodiment will be described. The operation of the embodiment is also roughly divided into a learning step ST3 and a determination step SD3. Also in the present embodiment, the learning step ST2 is performed prior to the determination step SD2.

  FIG. 11 is a flowchart illustrating an example of the operation of the learning step ST3 of the present embodiment. In the learning step ST3, the value of the parameter 311 is determined mainly using a learning image having a predetermined attribute and a noise image corresponding to the learning image. In the example shown in FIG. 11, the parameter calculation means 32 first calculates the value of the parameter 311 using a learning image set having a predetermined attribute given in advance and a set of noise learning images corresponding to the learning image set (step ST31). ).

  FIG. 12 is a flowchart illustrating an example of the operation of the determination step SD3 of the present embodiment. In the determination step SD3, the test image is determined by taking the difference between the test image and the noise-removed image based on the determined value of the parameter 311.

  In the example shown in FIG. 12, first, the feature extraction unit 31 extracts a noise component as a feature of the test image based on the determined value of the parameter 311 and generates a noise-removed image of the test image (step SD31). .

  Next, the determination unit 33 calculates the difference between the noise-removed image generated in step SD31 and the test image, and determines whether the test image has a predetermined attribute based on the difference (difference image) (step) SD32).

[Description of effects]
In the present embodiment, it is configured to extract a noise component included in the test image and determine whether or not it has a predetermined attribute. For this reason, according to the present embodiment, it is possible to perform attribute determination that is easy to understand visually.

[Other Embodiments]
In each of the above embodiments, a known algorithm is used for learning the feature extraction parameters of the neural network. Among them, there is an algorithm in which a required parameter varies by changing the initial value of the parameter, changing the number of learning iterations, or changing the order in which learning samples are given. By using this variation, it is possible to make a more accurate determination.

[Modification 1]
For example, in each of the above embodiments, it is possible to independently determine whether or not a test image has a predetermined attribute by r times (r is an integer of 2 or more) and take a majority vote. For example, in the case of r = 3, when it is determined twice that it has a predetermined attribute and once it is determined that it does not have a predetermined attribute, the determination means finally determines based on these determination results You may determine with having a predetermined attribute.

  In this case, learning of the feature extraction parameters of the neural network failed at a low frequency due to the fact that the learning algorithm is probabilistic (effective features for attribute determination could not be obtained). However, it is possible to perform highly accurate determination as a whole.

  Note that the final determination method does not necessarily need to be a majority decision. For example, when it is determined that a predetermined attribute is obtained at an arbitrary number of trials of one or more, it is determined that the final determination method has the predetermined attribute as a whole. May be. In addition, a new index may be calculated, for example, by calculating an average value or a variance value from indices obtained by a plurality of trials, and a final determination may be made based on the value.

[Modification 2]
In the second embodiment, an example is described in which a probability for attribute determination is calculated using a parameter or the like in a probability distribution specified in advance, and a statistical test is configured. If such a change is also considered as a stochastic phenomenon, the probability for attribute determination can be set as the simultaneous probability of r trials, and a new statistical test can be configured.

  This is also due to the fact that the learning algorithm is probabilistic, and when learning of the feature extraction parameters of the neural network fails at a low frequency (features that are effective for attribute determination cannot be obtained) as a whole It is an example which can perform a highly accurate determination.

  In each of the above embodiments, the processing target data is described as an image, but the processing target data is not limited to an image. In other words, any data can be used as long as it can be converted into a signal format that can be input by a neural network.For example, data obtained by performing arbitrary image processing on an image, a combination of a plurality of types of images captured using different sensors, Data obtained by adding an audio signal, annotation information, or the like to an image may be a processing target. Any information propagation method on the neural network and learning method for the neural network may be used as long as they are not substantially different from the methods described in the above embodiments.

  Next, a configuration example of a computer according to the embodiment of the present invention will be shown. FIG. 13 is a schematic block diagram illustrating a configuration example of a computer according to the embodiment of the present invention. The computer 1000 includes a CPU 1001, a main storage device 1002, an auxiliary storage device 1003, an interface 1004, a display device 1005, and an input device 1006.

  The above-described image recognition apparatus may be implemented in the computer 1000, for example. In that case, the operation of each device may be stored in the auxiliary storage device 1003 in the form of a program. The CPU 1001 reads out the program from the auxiliary storage device 1003 and develops it in the main storage device 1002, and executes the predetermined processing in the above embodiment according to the program.

  The auxiliary storage device 1003 is an example of a tangible medium that is not temporary. Other examples of the tangible medium that is not temporary include a magnetic disk, a magneto-optical disk, a CD-ROM, a DVD-ROM, and a semiconductor memory connected via the interface 1004. When this program is distributed to the computer 1000 via a communication line, the computer that has received the distribution may develop the program in the main storage device 1002 and execute the predetermined processing in the above embodiment.

  The program may be for realizing a part of predetermined processing in each embodiment. Furthermore, the program may be a difference program that realizes the predetermined processing in the above-described embodiment in combination with another program already stored in the auxiliary storage device 1003.

  The interface 1004 transmits / receives information to / from other devices. The display device 1005 presents information to the user. The input device 1006 accepts input of information from the user.

  Further, depending on the processing contents in the embodiment, some elements of the computer 1000 may be omitted. For example, if the device does not present information to the user, the display device 1005 can be omitted.

  In addition, some or all of the constituent elements of each device are implemented by general-purpose or dedicated circuits (Circuitry), processors, and the like or combinations thereof. These may be constituted by a single chip or may be constituted by a plurality of chips connected via a bus. Moreover, a part or all of each component of each device may be realized by a combination of the above-described circuit and the like and a program.

  When some or all of the constituent elements of each device are realized by a plurality of information processing devices and circuits, the plurality of information processing devices and circuits may be centrally arranged or distributedly arranged. Also good. For example, the information processing apparatus, the circuit, and the like may be realized as a form in which each is connected via a communication network, such as a client and server system and a cloud computing system.

  Next, the outline of the present invention will be described. FIG. 14 is a block diagram showing an outline of the present invention. An image recognition apparatus 500 illustrated in FIG. 14 includes a parameter calculation unit 501, a feature extraction unit 502, and a determination unit 503.

  The parameter calculation unit 501 (for example, the parameter calculation unit 12, 22, 32) uses at least one or more first images having a predetermined attribute shared by the authentication target to determine whether the attribute calculation unit 501 has the attribute. One or more parameters are extracted for extracting the attribute-likeness or the feature representing the attribute-likeness from a second image that is an unknown image.

  The feature extraction unit 502 (for example, the feature extraction unit 11, 21, 31) extracts a feature from the input image using the parameter calculated by the parameter calculation unit 501. More specifically, the feature extraction unit 502 is a neural network obtained by combining calculation elements including the parameters, and the calculation elements form two or more layers from input to output, and at least one layer A feature is extracted using a neural network in which the number of computing elements belonging to the number is smaller than the number of computing elements in the input layer to which image information is input.

  The determination unit 503 (for example, the determination unit 13, 23, 33) determines whether or not the second image is an authentication target based on at least the feature extracted from the second image.

  With such a configuration, even when it is difficult to obtain an abnormal sample image related to the recognition target, the image can be recognized with high accuracy.

  Although the present invention has been described with reference to the embodiments and examples, the present invention is not limited to the above embodiments and examples. Various changes that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention.

  The present invention can be used, for example, as an inspection apparatus that detects a foreign matter mixed product or a poor quality product among manufactured products in a factory. Further, the present invention can be used not only as a factory-produced product, but also as an abnormality detection device that detects an abnormality of a general object. Further, the present invention is, for example, as a part of a biometric authentication device used for a security gate or the like, as an inspection device for confirming whether an input image is actually a part (face, human body, etc.) to be authenticated. Is also available. Further, the present invention provides, for example, an image recognition unit that identifies the identity of an object among a plurality of frames with respect to an object such as a face, a human body, an object, or the like shown in the video in a tracking device that tracks a specific person in the video. Can also be used.

10, 20, 30 Image recognition device 11, 21, 31 Feature extraction means 111, 211, 311 Parameter 12, 22, 32 Parameter calculation means 13, 23, 33 Determination means 1000 Computer 1001 CPU
1002 Main storage device 1003 Auxiliary storage device 1004 Interface 1005 Display device 1006 Input device 500 Image recognition device 501 Parameter calculation means 502 Feature extraction means 503 Determination means

Claims (10)

Using at least one or more first images having a predetermined attribute shared by the authentication object, the attribute-likeness or the attribute-likeness from the second image, which is an image whose unknown or not, is unknown. Parameter calculating means for calculating one or more parameters for extracting features representing
Feature extraction means for extracting the features from the input image using the parameters;
Determination means for determining whether or not the second image is an authentication target based on at least the feature extracted from the second image;
The feature extraction means is a neural network obtained by combining calculation elements including the parameters, wherein the calculation elements constitute two or more layers from input to output, and the calculation elements belonging to at least one layer An image recognition apparatus characterized in that the feature is extracted using a neural network whose number is smaller than the number of calculation elements in the input layer to which image information is inputted. The image recognition apparatus according to claim 1, wherein the characteristic is information output from an intermediate layer of the neural network. The feature extraction means extracts the feature from each of the first images and the second image using the neural network including the parameters,
The determination unit calculates a distance between the feature extracted from each of the first images and a feature extracted from the second image, and is predetermined when the distances are arranged on the basis of the size. The image recognition apparatus according to claim 1, wherein the second image is determined based on a distance having a rank of. The parameter calculation means calculates the parameter so that the extracted feature follows a pre-specified distribution,
The determination means indicates whether or not the second image calculated based on at least the feature extracted from the second image using the second parameter that defines the shape of the distribution has the attribute. The image recognition apparatus according to claim 1, wherein the second image is determined based on the image. The parameter calculation means removes a noise component that is regarded as noise in the first image from the second image by using at least one or more first images having the attribute and including noise. Calculate the parameters as
The feature extraction means extracts the noise component as the feature from at least the second image using the parameter,
The image recognition apparatus according to claim 1, wherein the determination unit determines the second image based on the noise component. The image recognition apparatus according to claim 5, wherein the determination unit determines that the second image is a certification target when the noise component is other than a predetermined threshold value. The parameter calculation means calculates the parameter by a probabilistic method,
The determination means determines the second image based on a plurality of determination results obtained by performing feature extraction a plurality of times using the parameters calculated individually for one second image. The image recognition device according to any one of claims 1 to 6. The parameter calculation means calculates the parameter by a probabilistic method,
The determination means determines the second image based on a plurality of indices obtained by performing feature extraction a plurality of times using the parameters calculated individually for one second image. The image recognition apparatus according to claim 4. Information processing device
Using at least one or more first images having a predetermined attribute shared by the authentication object, the attribute-likeness or the attribute-likeness from the second image, which is an image whose unknown or not, is unknown. Calculate one or more parameters for extracting features representing
Extracting the features from the input image using the parameters;
Determining whether or not the second image is an authentication target based on at least the feature extracted from the second image;
A neural network obtained by combining calculation elements including the parameters when extracting the features, wherein the calculation elements constitute two or more layers from input to output and belong to at least one layer An image recognition method using a neural network in which the number of elements is smaller than the number of calculation elements in an input layer to which image information is input. On the computer,
Using at least one or more first images having a predetermined attribute shared by the authentication object, the attribute-likeness or the attribute-likeness from the second image, which is an image whose unknown or not, is unknown. A parameter calculation process for calculating one or more parameters for extracting features representing
Whether the second image is an authentication target based on a feature extraction process for extracting the feature from the input image using the parameter, and at least the feature extracted from the second image To execute the determination process to determine
A neural network obtained by combining calculation elements including the parameters in the feature extraction process, wherein the calculation elements constitute two or more layers from input to output, and the calculation elements belonging to at least one layer An image recognition program characterized in that the feature is extracted by using a neural network whose number is smaller than the number of calculation elements in the input layer to which image information is inputted.

Images