JP6699048B2 - Feature selecting device, tag related area extracting device, method, and program - Google Patents

Description

  The present invention relates to a feature selection device, a tag related area extraction device, a method, and a program.

  Distribution on the network due to the improvement of communication environment, the spread of devices equipped with shooting functions (digital cameras, smartphones, tablets, etc.) and the accompanying development of SNS (social networking service) sites and EC (electronic commerce) sites. The number of image contents to be played is enormous. In order to efficiently organize and search such enormous contents, there is an increasing demand for a technique for automatically analyzing images.

  As one of analysis techniques, there is a technique of automatically extracting a region of a specific object included in an image by a detector learned in advance (for example, Non-Patent Documents 1 and 2).

  Further, there is a technique for determining whether or not an object is included in each image area by extracting an image area that seems to be an object and inputting the extracted area to Deep Convolutional Neural Networks (DCNN) that has already been learned for object recognition. It is known (for example, nonpatent literature 3).

Felzenszwalb, P., McAllester, D., & Ramanan, D., (2008, June). "A discriminatively trained, multiscale, deformable part model.", In Computer Vision and Pattern Recognition, 2008. CVPR 2008. IEEE Conference on (p.1-8). IEEE. Malisiewicz, T., Gupta, A., & Efros, AA, (2011, November). "Ensemble of exemplar-svms for object detection and beyond.", In Computer Vision (ICCV), 2011 IEEE International Conference on (p. 89-96). IEEE. Girshick, R. (2015). "Fast r-cnn.", In Proceedings of the IEEE International Conference on Computer Vision (p.1440-1448).

  In order to generate a detector as described in Non-Patent Document 1 or Non-Patent Document 2, an image including a specific object and area information of the specific object in the image are associated with each other. A large amount of learning data is required.

  Further, learning of the DCNN for object recognition as described in Non-Patent Document 3 requires a large amount of image data and a set of tags. This learning data does not explicitly request a specific object area in the image, but basically an image showing a single object is used, and learning that includes area information of a specific object Data is needed.

  As described above, the SNS site has a large amount of image data posted by the user by adding tags, and the EC site has a large amount of image data of clothes to which the content provider has added tags such as colors and patterns. However, it takes a lot of manpower to generate the learning data that has been conventionally required, that is, the data in which the region information of the tag is linked in addition to the image and the tag.

  Further, there is a problem that it is unclear what image feature is effective for identifying the area information of the tag in the image.

  The present invention has been made to solve the above problems, and an object of the present invention is to provide a feature selection apparatus, method, and program capable of acquiring image features effective for image identification.

  Another object of the present invention is to provide a tag-related area extraction device, method, and program capable of accurately extracting an area associated with a tag in an image.

  In order to achieve the above object, a video feature selection apparatus according to the present invention is a set of images to which a tag representing a specific object included in an image is attached, and a set of images to which the tag is not attached. Based on the negative set, for each of the plurality of image features obtained from the image, a histogram representing the distribution of the image features for each of the images included in the positive set, and the images included in the negative set. An image feature generator that generates a histogram representing the distribution of the image features for each of the image features, and the histogram of the positive set and the histogram that is generated by the image feature generator for each of the plurality of image features. An image feature distribution comparison unit that calculates a distance between the negative set and the histogram; a feature descriptor selection unit that selects the top N image features of the distance calculated by the image feature distribution comparison unit; It is configured to include.

  Further, in the feature selection method according to the present invention, the image feature generation unit uses a positive set, which is a set of images to which a tag representing a specific object included in the image is added, and a set of images to which the tag is not added. Based on a certain negative set, for each of the plurality of image features obtained from the image, a histogram representing the distribution of the image features for each of the images included in the positive set, and of the images included in the negative set. A step of generating a histogram representing a distribution of the image features for each, and an image feature distribution comparison unit, the image of the plurality of image features generated by the image feature generation unit, the positive set of the Calculating a distance between a histogram and the histograms of the negative set, and a feature descriptor selecting unit selecting the top N image features of the distance calculated by the image feature distribution comparing unit. It is characterized in that it is executed by including and.

  Further, the image feature generation unit of the present invention, based on the positive set and the negative set, and a neural network that has been learned in advance, each of the images included in the positive set and the image included in the negative set. A distribution of the output of the unit for each of the images included in the positive set with respect to the output of each unit of the neural network as each of a plurality of image features obtained from the image. And a histogram representing the distribution of the output of the unit for each of the images included in the negative set, wherein the feature descriptor selection unit includes the distance calculated by the image feature distribution comparison unit. The output of the top N units may be selected.

  The tag-related area extraction device of the present invention is selected by the feature selection device, a mask generation unit that generates a plurality of masks of different sizes for masking an input image, and the feature descriptor selection unit. The mask image, which is the input image masked by each of the plurality of masks generated by the mask generation unit, is input to the neural network with respect to each output of the generated units. For each output of each of the units selected by the feature descriptor selecting unit, an image representing an average of the output of the unit obtained from each of the A feature descriptor normalization unit that generates a normalized image feature descriptor obtained by normalizing the image feature descriptor generated by the feature descriptor generation unit, and each of the units selected by the image feature distribution comparison unit Based on each of the distances obtained for each of the outputs and each of the normalized image feature descriptors generated by the feature descriptor normalizer for each output of the selected units. A tag relevance calculator that calculates the relevance between the tag and the pixel for each pixel of the input image by adding the respective weighted image feature descriptors with a weight according to the distance; , Is included.

  Further, the tag relevance calculating unit may further extract an area formed of an image having the relevance of a predetermined value or more as an area associated with the tag.

  Moreover, you may make it use CNN(Convolutional Neural Network) for the said neural network.

  A program according to the present invention is a program for causing a computer to function as each unit of the feature selection device or the tag-related area extraction device.

  According to the feature selection apparatus, the method, and the program of the present invention, based on the positive set and the negative set, for each of the plurality of image features obtained from the image, a histogram representing the distribution of the image features of the images of the positive set, and Generate a histogram representing the distribution of image features of the negative set of images, calculate the distance between the positive set histogram and the negative set histogram for each of the plurality of image features, and for the calculated distance By selecting the top N image features as the image feature descriptor, it is possible to obtain the effect that the image features effective for image identification can be acquired.

  Further, according to the tag-related area extracting device, method, and program of the present invention, each of the images included in the positive set and each of the images included in the negative set are input to the neural network, and the output of each unit of the neural network is output. In contrast, the distance between the positive set histogram and the negative set histogram is calculated, and the output of the top N units for the calculated distance is selected, and the sizes for masking the input image are different. A unit obtained by generating a plurality of masks, inputting each mask image, which is an input image masked by each of the plurality of masks, to a neural network for each output of each selected unit, and obtaining each mask image. Generate an image representing the average of the outputs of the image feature descriptors, generate a normalized image feature descriptor by normalizing the generated image feature descriptors, and obtain the distance obtained for each output of the selected unit. , And each of the normalized image feature descriptors generated for each output of each of the selected units by adding each of the normalized image feature descriptors with a distance-dependent weight. By calculating the degree of association between the tag and the pixel for each pixel of the image, it is possible to obtain an effect that the region related to the tag in the image can be accurately extracted.

It is a block diagram which shows the structure of the tag relevant area extraction device which concerns on embodiment of this invention. It is a figure showing an example of 1 composition of a tag relation field extraction part of a tag relation field extraction device concerning an embodiment of the invention. It is a flow chart which shows a tag related field extraction processing routine in a tag related field extraction device concerning an embodiment of the invention. It is a figure which shows the example of the experimental result which used the embodiment of this invention. It is a figure which shows the example of the experimental result which used the embodiment of this invention.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

<Outline of Embodiment of the Present Invention>

  In the embodiment of the present invention, a region strongly associated with the tag is extracted from the image by using the change in the distribution of the image features depending on the presence or absence of the tag and the difference in the feature descriptor obtained from each region of the image.

  Specifically, some kind of local feature descriptor is applied to the image set, and the distribution of image features obtained from the image set tagged and the distribution of image features obtained from the image set not tagged. The distance between distributions is calculated between and, and the tag with the longest distance is selected as the tag that is easy to recognize. In this case, by preparing a large number of feature descriptors, it is possible to obtain a tag having a large distance between image feature distributions and a feature descriptor having a large distance between image feature distributions. This can be seen as a feature descriptor useful in visual recognition.

  Further, using this, it is possible to obtain the image feature description characteristic of the image set to which the tag is added. By comparing the feature descriptor thus obtained with the characteristic image feature description and the feature description obtained from each region of the image, it is possible to obtain a region strongly associated with the tag.

  In the following, a set of a fashion coordinate image posted on the SNS as a set of images to which tags are attached and related tags is assumed.

<Configuration of Tag Related Area Extraction Device According to Embodiment of Present Invention>

  Next, the configuration of the tag-related area extraction device according to the embodiment of the present invention will be described. As shown in FIG. 1, a tag-related area extraction device 100 according to an embodiment of the present invention includes a CPU, a RAM, and a ROM that stores a program and various data for executing a tag-related area extraction processing routine described later. And a computer including and. The tag-related area extraction device 100 is functionally provided with an input unit 10, a calculation unit 20, and an output unit 40 as shown in FIG. The tag-related area extraction device 100 extracts an area related to a tag from the image set to which the tag is added.

The input unit 10 receives the tagged image set D. The tag indicates that the image contains a specific object. The tagged image set D includes, for each tag u, a positive set D _u ⁺ which is a set of images to which the tag u is added and a negative set D _u ⁻ which is a set of images to which the tag u is not added. include. The input unit 10 also receives, for each tag u, the input image I(x, y) to which the tag is attached.

  The calculation unit 20 includes an image set database 22, an image feature generation unit 24, an image feature distribution comparison unit 26, a feature descriptor selection unit 28, and a tag related area extraction unit 30.

The image set database 22 stores the tagged image set D including the positive set D _u ⁺ and the negative set D _u ^{− of} each tag u received by the input unit 10.

The image feature generation unit 24, for each tag u, based on the positive set D _u ⁺ and the negative set D _u ⁻ stored in the image set database 22 and the neural network learned in advance, the positive set D _u. Image features are generated for all images in the ⁺ and negative set D _u ⁻ .

Specifically, first, the image feature generation unit 24 preliminarily learns each of the images included in the positive set D _u ⁺ and the negative set D _u ⁻ stored in the image set database 22. Input to the neural network.

  In the present embodiment, the case where the output of each unit of the neural network learned in advance is used as each image feature obtained from the image will be described as an example. Further, a case where a Convolutional Neural Network (CNN) is used as a neural network learned in advance will be described as an example. CNN can be viewed as a local feature descriptor. The CNN holds many filters inside, and the output of each filter can be used as a different feature descriptor. In this embodiment, the CNN filter used is one that has been learned in advance using an image data set or the like for object recognition. Below, the filter used in each layer of CNN is called a unit.

Next, the image feature generation unit 24, for each unit i of the neural network as each image feature obtained from the image, a histogram that represents the distribution of the output of that unit for each image included in the positive set D _u ^+. Generate P _i ⁺ and a histogram P _i ⁻ representing the distribution of the output of the unit i for each of the images included in the negative set D _u ⁻ .

Image feature distribution comparing unit 26, for each tag u, for each of a plurality of units i, positive set generated by the image feature generation unit 24 D _u ⁺ histogram P _i ⁺ and negative set of D _u ^- histogram P _i ⁻ Calculate the distance between and.

In the present embodiment, a case will be described as an example in which the Kullback-Leibler distance (hereinafter, referred to as KL distance) is used as the distance between the histogram P _i ⁺ and the histogram P _i ⁻ .

The KL distance S _i (u|D) between the positive set D _u ⁺ and the negative set D _u ⁻ with respect to the tag u, which is obtained from the tagged image set D stored in the image set database 22, is represented by each histogram. It is calculated as follows, where bin is x. In addition, x represents the value output from each unit.

For example, if u is a tag not visually recognized easily, the distribution of positive set D _u ⁺ of the image features is close to random, positive set D _u ⁺ of the image features of the distribution and negative set D _u ^- image features The difference from the distribution of is small. On the other hand, when u is a tag that can be easily recognized visually, the difference between the distribution of image features of the positive set D _u ⁺ and the distribution of image features of the negative set D _u ⁻ becomes large.

Therefore, in the case of tags with color names such as “red” and “white” and texture tags such as “border” and “floral pattern”, the value of the KL distance S _i (u|D) becomes large.

Then, the image feature distribution comparison unit 26 outputs the KL distance S _i (u|D) for each of the plurality of units.

  The feature descriptor selection unit 28 selects, for each tag u, the top N units of the distances calculated by the image feature distribution comparison unit 26 as image features.

Specifically, the feature descriptor selection unit 28 receives the KL distance S _i (u|D) calculated by the image feature distribution comparison unit 26 as an input, and determines the one with the larger value of the KL distance S _i (u|D). From the above, N units are selected as image features, and a set of upper N units of the selected KL distance S _i (u|D) is defined as θ _u .

For each tag u, the tag-related area extraction unit 30 determines the tag based on the set θ _u composed of the upper N units of the KL distance S _i (u|D) selected by the feature descriptor selection unit 28. An area associated with the tag is extracted from the provided input image. The tag-related area extraction unit 30 includes a mask generation unit 32, a characteristic descriptor generation unit 34, a characteristic descriptor normalization unit 36, and a tag relation degree calculation unit 38.

  The mask generation unit 32 generates a plurality of masks having different sizes for masking the input image I(x, y) received by the input unit 10. As the mask, an average image of the data set (same resolution as the image to be masked) cut out from the corresponding corresponding position is used. The size of the mask is, for example, one-tenth, one-fifth, one-third, or the like of the input image I(x,y).

The characteristic descriptor generation unit 34 generates, for each tag u, a plurality of units generated by the mask generation unit 32 with respect to the output of each unit of the set θ _{u including} the units selected for the tag u by the characteristic descriptor selection unit 28. Each of the mask images, which is the input image masked by each of the masks, is input to the neural network. Then, the feature descriptor generation unit 34, for the output of each unit of the set θ _u consisting of the unit selected for the tag u, an image representing the average of the output of the unit, obtained from each of the mask image, Generate as an image feature descriptor.

The feature descriptor generating unit 34 hides a partial area of the image with a mask defined by (x, y) for the input image I(x, y), and hides a partial area of the output of the unit i. An image feature descriptor A _i (x, y) is generated from the generated image. A function for generating an image feature descriptor from such a mask is defined as a _i (x, y). A _i (x, y) is the i-th feature descriptor (here, the output of the i-th unit), and is an average of outputs obtained corresponding to a plurality of masks.

The feature descriptor normalization unit 36 generates an image generated by the feature descriptor generation unit 34 for each output of the unit i of the set θ _{u including} the units selected by the feature descriptor selection unit 28 for each tag u. A normalized image feature descriptor R _i (x, y) is generated by normalizing the feature descriptor A _i (x, y).

Specifically, first, the feature descriptor normalization unit 36 calculates an average image of the image feature descriptors A _i (x, y) of each unit i. The feature descriptor normalizing unit 36, for each unit i, the image characteristic descriptor A _{i (x,} y) enter the image feature descriptors A _{i (x,} y) the difference between the average image and the When the value of the difference is negative when the maximum value is obtained, the entire image feature descriptor A _i (x, y) is multiplied by -1. Then, the feature descriptor normalization unit 36 normalizes the image feature descriptor A _i (x, y) of each unit i to a value between 0 and 1, and normalizes the image feature descriptor R _i ( x, y).

The tag relevance calculator 38 calculates the KL distance S _i (u|D) obtained for each output of the unit i of the set θ _{u including} the units selected by the feature descriptor selector 28 for each tag u. A normalized image feature descriptor based on each of the normalized image feature descriptors R _i (x, y) generated for each output of the unit i selected by the feature descriptor normalizer 36; By adding each R _i (x, y) with a weight corresponding to the KL distance S _i (u|D), the tag and pixel between each pixel of the input image I(x, y) Calculate the degree of association of.

  The tag relevance calculator 38 calculates the tag relevance region M(x, y|u, I) representing the relevance between the tag u and the pixel for each pixel of the input image I(x, y) as follows. It is calculated by the equation (2).

_{Here, S} i _(u│D) is obtained from the tagged image set D, to the tag u, positive set _D ^{u +} histogram _P ^{i +} and negative set _{D u} ^- KL between ^- histogram _{P i} is the distance, Z is one in which the sum of _S i _(u│D) from large units _S i _(u│D) to N units sequentially.

  Further, the tag relevance calculating unit 38 may further extract a region including an image having a relevance calculated according to the above formula (2) that is equal to or more than a predetermined value as a region related to the tag.

<Operation of Tag Related Area Extraction Device According to Embodiment of Present Invention>

  Next, the operation of the tag-related area extraction device 100 according to the embodiment of the present invention will be described. When the input unit 10 receives the tagged image set D, stores it in the image set database 22, and inputs the input image I(x, y), the tag-related area extraction device 100 displays the tag u for each tag u in FIG. The tag related area extraction processing routine shown is executed.

First, in step S100, the image feature generation unit 24 acquires the positive set D _u ⁺ and the negative set D _u ⁻ stored in the image set database 22.

Next, in step S102, the image feature generation unit 24 inputs each of the images included in the positive set D _u ⁺ and each of the images included in the negative set D _u ⁻ acquired in step S100 to the CNN. Then, the image feature generation unit 24, for each unit i of the CNN, the histogram P _i ⁺ representing the distribution of the output of the unit for each of the images included in the positive set D _u ^+, negative set function D _u ^- in And a histogram P _i ⁻ representing the distribution of the output of the unit i in question for each of the included images.

In step S104, the image feature distribution comparing unit 26 generated in step S102, for each of a plurality of units, positive set _D ^{u +} histogram _P ^{i +} and negative set _{D u} ^- histogram _{P i} ^- and The distance between is calculated according to equation (1) above.

In step S106, the feature descriptor selection unit 28 selects the upper N units of the distance calculated in step S104 and sets them as the unit set θ _u .

  In step S108, the mask generation unit 32 generates a plurality of masks having different sizes for masking the input image I(x, y) received by the input unit 10.

In step S110, the feature descriptor generation unit 34 masks the output of each unit i of the set θ _{u including} the units selected in step S106 by each of the plurality of masks generated in step S108. Each of the mask images, which are the input images, is input to the neural network. Then, the feature descriptor generation unit 34 sets, as an image feature descriptor A _i (x, y), an image representing the average of the outputs of the unit i obtained from each of the mask images for each output of the unit i. To generate.

In step S112, the feature descriptor normalization unit 36, for each output of the unit i of the set θ _{u including} the units selected in step S106, generates the image feature descriptor A _i (x , Y) is normalized to generate a normalized image feature descriptor R _i (x, y).

In step S114, the tag relevance calculating unit 38 selects each of the KL distances S _i (u|D) obtained for each output of the unit i of the set θ _{u including} the units selected in step S106. Based on each of the normalized image feature descriptors R _i (x, y) generated in step S112 for each output of each unit i, according to equation (2) above, the normalized image feature descriptor R By adding each _i (x, y) with a weight according to the KL distance S _i (u|D), for each pixel of the input image I(x, y), between the tag and the pixel, Calculate the degree of association.

  In step S116, the output unit 40 outputs the degree of association between the tag and the pixel for each pixel of the input image I(x, y) calculated in step S114 as a result, and ends the process.

<Experimental example>
FIG. 4 is an example of a tag-related area obtained by manually extracting tags from an actual image and a tag-related area obtained by the tag-related area extracting apparatus according to the embodiment of the present invention. In addition, FIG. 5 is an extracted tag-related region when the value of the number N of units selected from the larger distance S _i (u|D) between histograms is changed. The optimum value of the number N of units depends on the tag. The calculation amount increases as the value of the number N of units increases, but the tag-related area is extracted in more detail.

  As described above, according to the tag-related area extraction device of the embodiment of the present invention, each of the images included in the positive set and each of the images included in the negative set are input to the CNN, and each unit of the CNN is input. Size for calculating the distance between the positive set histogram and the negative set histogram, and selecting the output of the top N units for the calculated distance and masking for the input image A plurality of different masks are generated, and for each output of the selected unit, each of the mask images, which is an input image masked by each of the plurality of masks, is input to CNN and is obtained from each of the mask images. An image representing the average of the outputs of the units is generated as an image feature descriptor, a normalized image feature descriptor obtained by normalizing the generated image feature descriptor is generated, and obtained for each output of the selected unit. Based on each of the distances and each of the normalized image feature descriptors generated for each output of the selected units, by adding each of the normalized image feature descriptors with a weight depending on the distance, By calculating the degree of association between the tag and the pixel for each pixel of the input image, the region related to the tag in the image can be accurately extracted.

  Further, according to the embodiment of the present invention, although such area information is not included, a related area for each tag can be learned from data in which an image and a tag included in the image are associated with each other.

  The present invention is not limited to the above-described embodiments, and various modifications and applications can be made without departing from the scope of the present invention.

  For example, the above embodiment has been described by taking the case where the present invention is applied to the tag-related area extracting device as an example, but the present invention is not limited to this and the present invention may be applied to a feature selecting device. In that case, the feature selection device is configured to include an image feature generation unit 24, an image feature distribution comparison unit 26, and a feature descriptor selection unit 28.

  According to this feature selection device, based on the positive set and the negative set, for each of the plurality of image features obtained from the image, a histogram representing the distribution of the image features of the image of the positive set, and the image of the image of the negative set. And a histogram representing the distribution of the features, calculating the distance between the positive set histogram and the negative set histogram for each of the plurality of image features, and calculating the top N image features for the calculated distances. By selecting the image feature descriptor, it is possible to acquire the image feature effective for image identification.

  Further, in the above embodiment, the case where the output of each unit of the neural network learned in advance is used as each image feature obtained from the image has been described as an example, but the present invention is not limited to this and other Image features may be used.

  Further, in the above-described embodiment, the case where the CNN is used as the neural network has been described as an example, but the present invention is not limited to this, and another neural network may be used.

  Further, in the above-described embodiment, the case where the Kullback-Leibler distance is used as the distance between the histograms has been described as an example, but the present invention is not limited to this, and another distance may be used.

  The above-described tag-related area extraction device 100 has a computer system inside, but the “computer system” also includes a homepage providing environment (or display environment) if a WWW system is used. .

  Further, in the specification of the present application, the embodiment in which the program is preinstalled has been described, but the program can be stored in a computer-readable recording medium and provided.

10 input unit 20 operation unit 22 image set database 24 image feature generation unit 26 image feature distribution comparison unit 28 feature descriptor selection unit 30 tag related region extraction unit 32 mask generation unit 34 feature descriptor generation unit 36 feature descriptor normalization unit 38 tag degree-of-association calculation unit 40 output unit 100 tag-related area extraction device

Claims (7)

Of a plurality of image features obtained from the image based on a positive set that is a set of images to which a tag representing a specific object included in the image is attached and a negative set that is a set of images to which the tag is not attached. Image features for each generating a histogram representing the distribution of the image features for each of the images included in the positive set and a histogram representing the distribution of the image features for each of the images included in the negative set A generator,
An image feature distribution comparison unit that calculates a distance between the histogram of the positive set and the histogram of the negative set, for each of the plurality of image features generated by the image feature generation unit,
A feature descriptor selection unit that selects the top N image features for the distance calculated by the image feature distribution comparison unit;
Only including,
The image feature generation unit, based on the positive set and the negative set, and a neural network that has been learned in advance, the neural network for each of the images included in the positive set and each of the images included in the negative set. A histogram representing the distribution of the output of the unit for each of the images included in the positive set, with respect to the output of each unit of the neural network as each of a plurality of image features obtained from the image. And a histogram representing the distribution of the output of the unit for each of the images included in the negative set,
The feature descriptor selection unit selects the output of the top N units for the distance calculated by the image feature distribution comparison unit,
Feature selection device. A feature selection apparatus according to claim 1 ;
A mask generation unit that generates a plurality of masks of different sizes for masking the input image;
For each output of each of the units selected by the feature descriptor selection unit, the mask image that is the input image masked by each of the plurality of masks generated by the mask generation unit is set to the neural network. A feature descriptor generation unit that inputs to the network and generates an image representing an average of the outputs of the units obtained from each of the mask images, as an image feature descriptor,
For each output of the unit selected by the feature descriptor selection unit, a feature descriptor normalization for generating a normalized image feature descriptor obtained by normalizing the image feature descriptor generated by the feature descriptor generation unit. Akabe,
Each of the distances obtained for the output of each of the selected units by the image feature distribution comparison unit and the generated for each output of each of the selected units by the feature descriptor normalization unit. The tag and the pixel for each pixel of the input image by adding each of the normalized image feature descriptors with a weight according to the distance based on each of the normalized image feature descriptors. A tag relevance calculator that calculates a relevance between
An apparatus for extracting a tag-related area including. The tag-related area extraction device according to claim 2 , wherein the tag-related area calculating unit further extracts an area formed of an image in which the degree of association is a predetermined value or more as an area associated with the tag. The feature selecting apparatus according to claim 1 , wherein a CNN (Convolutional Neural Network) is used for the neural network. The CNN (Convolutional Neural Network) is used for the neural network, The tag related area|region extraction apparatus of Claim 2 or Claim 3 . The image feature generation unit obtains from the image based on a positive set that is a set of images to which a tag representing a specific object included in the image is attached and a negative set that is a set of images to which the tag is not attached. A histogram representing the distribution of the image features for each of the images included in the positive set and a histogram representing the distribution of the image features for each of the images included in the negative set for each of the plurality of image features The steps of generating and
An image feature distribution comparing unit calculates a distance between the histogram of the positive set and the histogram of the negative set for each of the plurality of image features generated by the image feature generating unit; ,
A feature descriptor selecting unit selects the top N image features of the distance calculated by the image feature distribution comparing unit;
Only including,
The step of generating the histogram by the image feature generation unit is included in each of the images included in the positive set and the negative set based on the positive set and the negative set, and a neural network that has been learned in advance. Inputting each of the images to the neural network and outputting the output of each unit of the neural network as each of a plurality of image features obtained from the image to the output of each unit of the images included in the positive set And a histogram representing the distribution of the output of the unit for each of the images included in the negative set,
The step of selecting the image features by the feature descriptor selection unit selects the output of the top N units for the distance calculated by the image feature distribution comparison unit,
Feature selection method. Computer, according to claim 1 or feature selection device according to claim 4, or claim 2, claim 3, or a program to function as each section of the tag associated region extraction apparatus according to claim 5.