JP2018013887A - Feature selection device, tag relevant area extraction device, method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To make it possible to acquire an image feature effective in identifying images.SOLUTION: An image feature creation unit 24 is configured to create a histogram expressing a distribution of an image feature as to each of images included in a positive set and a histogram expressing a distribution of an image feature included in a negative set with respect to each of a plurality of image features to be obtained from an image on the basis of the positive set and the negative set. An image feature distribution comparison unit 26 is configured to calculate a distance between the histogram of the positive set and the histogram of the negative set as to each of the plurality of image features. A feature descriptor selection unit is configured to select top N pieces of image features about the distance calculated by the image feature distribution comparison unit 26.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a feature selection device, a tag related region extraction device, a method, and a program.

  Distributed on the network by improving the quality of the communication environment, the widespread use of devices equipped with shooting functions (digital cameras, smartphones, tablets, etc.) and the development of social networking service (SNS) sites and EC (electronic commerce) sites. The number of image contents to be processed is enormous. In order to efficiently organize and search such enormous contents, there is an increasing demand for a technique for automatically analyzing images.

  As one of analysis techniques, there is a technique of automatically extracting a region of a specific object included in an image by a previously learned detector (for example, Non-Patent Documents 1 and 2).

  Also, there is a technique for extracting an image area that is likely to be an object, and determining whether each image area includes an object by inputting the extracted area to Deep Convolutional Neural Networks (DCNN) that has been learned for object recognition. It is known (for example, Non-Patent Document 3).

Felzenszwalb, P., McAllester, D., & Ramanan, D., (2008, June). "A discriminatively trained, multiscale, deformable part model.", In Computer Vision and Pattern Recognition, 2008. CVPR 2008. IEEE Conference on (p.1-8). IEEE. Malisiewicz, T., Gupta, A., & Efros, AA, (2011, November). "Ensemble of exemplar-svms for object detection and beyond.", In Computer Vision (ICCV), 2011 IEEE International Conference on (p. 89-96) .IEEE. Girshick, R. (2015). "Fast r-cnn.", In Proceedings of the IEEE International Conference on Computer Vision (p.1440-1448).

  In order to generate a detector as described in Non-Patent Document 1 or Non-Patent Document 2 described above, an image including a specific object is associated with region information of the specific object in the image. A large amount of learning data is required.

  In addition, learning of DCNN for object recognition as described in Non-Patent Document 3 requires a large amount of image data and tag sets. This learning data does not explicitly require a specific object area in the image, but basically an image showing a single object is used, and learning that includes the area information of the specific object Data is required.

  As described above, the SNS site has a large amount of image data posted by the user with tags, and the EC site has a large amount of image data of clothes to which the content provider has added tags such as colors and patterns. However, it takes a lot of manpower to generate learning data that is conventionally required, that is, data in which tag region information is linked in addition to images and tags.

  In addition, there is a problem that it is not clear what image features are effective for identifying tag region information in an image.

  The present invention has been made to solve the above-described problems, and an object of the present invention is to provide a feature selection device, method, and program capable of acquiring image features effective for image identification.

  It is another object of the present invention to provide a tag related area extracting apparatus, method, and program capable of accurately extracting an area related to a tag in an image.

  In order to achieve the above object, a video feature selection device according to the present invention includes a positive set, which is a set of images to which a tag representing a specific object included in an image is assigned, and a set of images to which the tag is not assigned. A histogram representing a distribution of the image features for each of the images included in the positive set, and an image included in the negative set, for each of a plurality of image features obtained from the image based on the negative set An image feature generation unit that generates a histogram representing the distribution of the image features for each of the image feature generation unit, and the histogram of the positive set for each of the plurality of image features generated by the image feature generation unit and the histogram An image feature distribution comparison unit for calculating a distance between the histogram and a negative set; It is configured to include a characteristic descriptor selector for selecting top N of the image features on the distance calculated by the feature distribution comparing unit.

  In the feature selection method according to the present invention, the image feature generation unit includes a positive set that is a set of images to which a tag representing a specific object included in the image is assigned and a set of images to which the tag is not assigned. Based on a negative set, for each of a plurality of image features obtained from the image, a histogram representing a distribution of the image features for each of the images included in the positive set, and an image of the image included in the negative set Generating a histogram representing the distribution of the image features for each, and an image feature distribution comparison unit generated by the image feature generation unit for each of a plurality of the image features of the positive set Calculating a distance between a histogram and the histogram of the negative set; and Child selecting unit, and executes comprise the steps of selecting the top N of the image features on the distance calculated by the image feature distribution comparing unit.

  In addition, the image feature generation unit of the present invention may be configured so that each of the images included in the positive set and the images included in the negative set are based on the positive set, the negative set, and a neural network learned in advance. Distribution of the output of the unit for each of the images included in the positive set with respect to the output of each unit of the neural network as each of a plurality of image features obtained from the image, each input to the neural network And a histogram representing the distribution of the output of the unit for each of the images included in the negative set, and the feature descriptor selection unit calculates the distance calculated by the image feature distribution comparison unit The output of the top N units It may be-option.

  Further, the tag related area extracting device of the present invention is selected by the feature selecting device, a mask generating unit that generates a plurality of masks having different sizes for masking an input image, and the feature descriptor selecting unit. For each output of the unit, the mask image, which is the input image masked by each of the plurality of masks generated by the mask generation unit, is input to the neural network, and the mask image For each of the units selected by the feature descriptor selection unit and a feature descriptor generation unit that generates an image representing an average of the output of the units obtained from each of the unit Feature description for generating a normalized image feature descriptor obtained by normalizing the image feature descriptor generated by the feature descriptor generation unit A child normalization unit, each of the distances obtained for the output of each of the units selected by the image feature distribution comparison unit, and each of the units selected by the feature descriptor normalization unit. Based on each of the normalized image feature descriptors generated for output, each of the normalized image feature descriptors is added with a weight according to the distance, so that for each pixel of the input image And a tag relevance calculating unit that calculates a relevance between the tag and the pixel.

  In addition, the tag relevance calculation unit may further extract an area composed of images having the relevance greater than or equal to a predetermined value as an area related to the tag.

  The neural network may be a CNN (Convolutional Neural Network).

  The program according to the present invention is a program for causing a computer to function as each part of the feature selection device or the tag related region extraction device.

  According to the feature selection device, method, and program of the present invention, for each of a plurality of image features obtained from an image based on a positive set and a negative set, a histogram representing the distribution of the image features of the positive set image, Generating a histogram representing the distribution of image features of the negative set image, calculating a distance between the positive set histogram and the negative set histogram for each of the plurality of image features, and calculating the calculated distance By selecting the top N image features as image feature descriptors, it is possible to obtain an effect that image features effective for image identification can be acquired.

  In addition, according to the tag related region extraction apparatus, method, and program of the present invention, each of the images included in the positive set and each of the images included in the negative set are input to the neural network, and the output of each unit of the neural network is output. In contrast, the distance between the positive set histogram and the negative set histogram is calculated, the output of the top N units is selected for the calculated distance, and the size for masking the input image is different. A unit that generates a plurality of masks and inputs each mask image, which is an input image masked by each of the plurality of masks, to the neural network for each output of the selected unit, and is obtained from each of the mask images An image representing the average of the output of the image is generated as an image feature descriptor Generate a normalized image feature descriptor that is a normalization of the generated image feature descriptor, and generate each of the distances obtained for each output of the selected unit and each output of the selected unit. Based on each of the normalized image feature descriptors, the degree of association between the tag and the pixel for each pixel of the input image by adding each of the normalized image feature descriptors with a weight corresponding to the distance. By calculating, it is possible to extract an area related to the tag in the image with high accuracy.

It is a block diagram which shows the structure of the tag relevant area extraction apparatus which concerns on embodiment of this invention. It is a figure which shows the example of 1 structure of the tag relevant area extraction part of the tag relevant area extraction apparatus which concerns on embodiment of this invention. It is a flowchart which shows the tag relevant area extraction processing routine in the tag relevant area extraction apparatus which concerns on embodiment of this invention. It is a figure which shows the example of the experimental result using embodiment of this invention. It is a figure which shows the example of the experimental result using embodiment of this invention.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

<Outline according to Embodiment of the Present Invention>

  In the embodiment of the present invention, a region strongly associated with a tag is extracted from an image by using a change in distribution of image features depending on the presence or absence of a tag and a difference in feature descriptors obtained from each region of the image.

  Specifically, applying some local feature descriptor to the image set, the distribution of image features obtained from the image set with the tag attached, and the distribution of image features obtained from the image set without the tag attached The distance between the distributions is calculated between them, and the tag that is the most distant is selected as the tag that can be easily recognized. In this case, by preparing a large number of feature descriptors, it is possible to obtain a feature descriptor having a large distance between image feature distributions simultaneously with a tag having a large distance between image feature distributions. This can be seen as a feature descriptor useful in visual recognition.

  Furthermore, using this, it is possible to obtain an image feature description characteristic of the image set to which the tag is attached. By comparing the feature descriptor thus obtained, the characteristic image feature description, and the feature description obtained from each region of the image, a region strongly associated with the tag can be obtained.

  In the following, a set of a fashion coordinated image posted to the SNS as an image set to which a tag is attached and a related tag is assumed.

<Configuration of Tag Related Area Extraction Device According to Embodiment of the Present Invention>

  Next, the configuration of the tag related area extracting apparatus according to the embodiment of the present invention will be described. As shown in FIG. 1, a tag related area extraction device 100 according to an embodiment of the present invention includes a CPU, a RAM, and a ROM that stores a program and various data for executing a tag related area extraction processing routine described later. And a computer including the above. Functionally, the tag related area extracting apparatus 100 includes an input unit 10, a calculation unit 20, and an output unit 40 as shown in FIG. The tag related area extraction apparatus 100 extracts an area related to a tag from an image set to which the tag is assigned.

The input unit 10 receives the tagged image set D. The tag represents that a specific object is included in the image. The tagged image set D includes, for each tag u, a positive set D _u ⁺ that is a set of images to which the tag u is assigned and a negative set D _u ⁻ that is a set of images to which the tag u is not assigned. include. Further, the input unit 10 receives an input image I (x, y) to which the tag is assigned for each tag u.

  The calculation unit 20 includes an image set database 22, an image feature generation unit 24, an image feature distribution comparison unit 26, a feature descriptor selection unit 28, and a tag related region extraction unit 30.

The image set database 22 stores the tagged image set D including the positive set D _u ⁺ and the negative set D _u ^{− of} each tag u, which is received by the input unit 10.

The image feature generation unit 24, for each tag u, based on the positive set D _u ⁺ and the negative set D _u ⁻ stored in the image set database 22 and the previously learned neural network, the positive set D _u. Image features are generated for all images included in the ⁺ and negative set D _u ⁻ .

Specifically, first, the image feature generation unit 24 learns in advance each of the images included in the positive set D _u ⁺ and each of the images included in the negative set D _u ⁻ stored in the image set database 22. Input to the neural network.

  In the present embodiment, a case where the output of each unit of a neural network learned in advance is used as each image feature obtained from an image will be described as an example. A case where a convolutional neural network (CNN) is used as a previously learned neural network will be described as an example. The CNN can be viewed as a local feature descriptor. The CNN has a large number of filters inside, and the output of each filter can be used as a different feature descriptor. In the present embodiment, a CNN filter that has been learned in advance using an image data set for object recognition or the like is used. Below, the filter used in each layer of CNN is called a unit.

Next, for each unit i of the neural network as each image feature obtained from the image, the image feature generation unit 24 represents a histogram representing the output distribution of the unit for each of the images included in the positive set D _u ^+. P _i ⁺ and a histogram P _i ⁻ representing the output distribution of the unit i for each of the images included in the negative set D _u ⁻ are generated.

Image feature distribution comparing unit 26, for each tag u, for each of a plurality of units i, positive set generated by the image feature generation unit 24 D _u ⁺ histogram P _i ⁺ and negative set of D _u ^- histogram P _i ^- calculating the distance between the.

In the present embodiment, a case will be described as an example in which a Cullback-Lailer distance (hereinafter referred to as KL distance) is used as the distance between the histogram P _i ⁺ and the histogram P _i ⁻ .

The KL distance S _i (u | D) between the positive set D _u ⁺ and the negative set D _u ⁻ for the tag u, obtained from the tagged image set D stored in the image set database 22, is calculated for each histogram. It is obtained as follows with bin as x. Note that x represents a value output from each unit.

For example, if u is a tag not visually recognized easily, the distribution of positive set D _u ⁺ of the image features it is close to random, positive set D _u ⁺ of the image features of the distribution and negative set D _u ^- image features The difference from the distribution of is small. On the other hand, when u is a visually recognizable tag, the difference between the distribution of image features of the positive set D _u ⁺ and the distribution of image features of the negative set D _u ⁻ becomes large.

Therefore, for example, in the case of a tag with a color name such as “red” or “white” or a texture tag such as “border” or “floral pattern”, the value of the KL distance S _i (u | D) increases.

Then, the image feature distribution comparison unit 26 outputs the KL distance S _i (u | D) for each of the plurality of units.

  The feature descriptor selection unit 28 selects, for each tag u, the top N units for the distance calculated by the image feature distribution comparison unit 26 as image features.

Specifically, the feature descriptor selection unit 28 uses the KL distance S _i (u | D) calculated by the image feature distribution comparison unit 26 as an input, and has a larger value of the KL distance S _i (u | D). N units are selected as image features, and a set consisting of the top N units of the selected KL distance S _i (u | D) is defined as θ _u .

For each tag u, the tag related region extraction unit 30 determines that the tag is based on a set θ _u composed of the top N units of the KL distance S _i (u | D) selected by the feature descriptor selection unit 28. A region related to the tag is extracted from the given input image. The tag related area extraction unit 30 includes a mask generation unit 32, a feature descriptor generation unit 34, a feature descriptor normalization unit 36, and a tag association degree calculation unit 38.

  The mask generation unit 32 generates a plurality of masks having different sizes for masking the input image I (x, y) received by the input unit 10. As the mask, an image cut from the same corresponding position of the average image (the same resolution as the image to be masked) of the data set is used. The size of the mask is, for example, one tenth, one fifth, one third, or the like of the input image I (x, y).

For each tag u, the feature descriptor generator 34 generates a plurality of units generated by the mask generator 32 for the output of each unit of the set θ _u composed of units selected for the tag u by the feature descriptor selector 28. Each mask image, which is an input image masked by each of the masks, is input to the neural network. Then, the feature descriptor generation unit 34 obtains an image representing the average of the outputs of the units obtained from each of the mask images for the output of each unit of the set θ _u composed of the units selected for the tag u. Generated as an image feature descriptor.

The feature descriptor generation unit 34 hides a partial area of the image with the mask defined by (x, y) with respect to the input image I (x, y) and hides the partial area with respect to the output of the unit i. An image feature descriptor A _i (x, y) is generated from the processed image. A function for generating an image feature descriptor from such a mask is defined as a _i (x, y). A _i (x, y) is an i-th feature descriptor (here, the output of the i-th unit), and is an average of outputs obtained corresponding to a plurality of masks.

The feature descriptor normalization unit 36 generates an image generated by the feature descriptor generation unit 34 for each output of the unit i of the set θ _u composed of units selected by the feature descriptor selection unit 28 for each tag u. A normalized image feature descriptor R _i (x, y) obtained by normalizing the feature descriptor A _i (x, y) is generated.

Specifically, first, the feature descriptor normalization unit 36 calculates an average image of the image feature descriptors A _i (x, y) of each unit i. The feature descriptor normalizing unit 36, for each unit i, the image characteristic descriptor A _{i (x,} y) enter the image feature descriptors A _{i (x,} y) the difference between the average image and the When the value of the difference when taking the maximum value is negative, the entire image feature descriptor A _i (x, y) is multiplied by −1. Then, the feature descriptor normalization unit 36 normalizes the image feature descriptor A _i (x, y) of each unit i to a value between 0 and 1, and the normalized image feature descriptor R _i ( x, y).

For each tag u, the tag relevance calculation unit 38 calculates the KL distance S _i (u | D) obtained for each output of the unit i of the set θ _u composed of units selected by the feature descriptor selection unit 28. Based on each and each of the normalized image feature descriptors R _i (x, y) generated for each output of the unit i selected by the feature descriptor normalization unit 36, By adding each of R _i (x, y) with a weight according to the KL distance S _i (u | D), between the tag and the pixel for each pixel of the input image I (x, y). The relevance of is calculated.

  The tag relevance calculating unit 38 calculates a tag related region M (x, y | u, I) representing the relevance between the tag u and the pixel for each pixel of the input image I (x, y) as follows. (2) is calculated.

_{Here, S} i _(u│D) is obtained from the tagged image set D, to the tag u, positive set _D ^{u +} histogram _P ^{i +} and negative set _{D u} ^- KL between ^- histogram _{P i} Z is a sum of S _i (u | D) from the unit with the largest S _i (u | D) up to N units.

  In addition, the tag relevance calculation unit 38 may further extract an area including images whose relevance calculated according to the above equation (2) is equal to or greater than a predetermined value as an area related to the tag.

<Operation of Tag Related Area Extraction Device According to Embodiment of Present Invention>

  Next, the operation of the tag related region extraction device 100 according to the embodiment of the present invention will be described. When the input unit 10 receives the tagged image set D and stores it in the image set database 22, and the input image I (x, y) is input, the tag related area extracting device 100 performs the process shown in FIG. The tag related area extraction processing routine shown is executed.

First, in step S ^< _b ^> 100, the image feature generation unit 24 obtains a positive set D _u ⁺ and a negative set D _u ⁻ stored in the image set database 22.

Next, in step S102, the image feature generation unit 24 inputs each of the images included in the positive set D _u ⁺ acquired in step S100 and each of the images included in the negative set D _u ⁻ to the CNN. Then, the image feature generation unit 24, for each unit i of the CNN, the histogram P _i ⁺ representing the distribution of the output of the unit for each of the images included in the positive set D _u ^+, negative set function D _u ^- in A histogram P _i ⁻ representing the output distribution of the unit i for each of the included images is generated.

In step S104, the image feature distribution comparison unit 26 generates the positive set D _u ⁺ histogram P _i ⁺ and the negative set D _u ⁻ histogram P _i ⁻ for each of the plurality of units generated in step S102. Is calculated according to the above equation (1).

In step S106, the feature descriptor selection unit 28 selects the top N units for the distance calculated in step S104, and sets it as a unit set θ _u .

  In step S <b> 108, the mask generating unit 32 generates a plurality of masks having different sizes for masking the input image I (x, y) received by the input unit 10.

In step S110, the feature descriptor generation unit 34 masks each output of the unit i of the set θ _u composed of the units selected in step S106 with each of the plurality of masks generated in step S108. Each of the mask images as input images is input to the neural network. Then, the feature descriptor generation unit 34 sets, as an image feature descriptor A _i (x, y), an image representing the average of the output of the unit i obtained from each of the mask images for each output of the unit i. Generate.

In step S112, the feature descriptor normalization unit 36 generates the image feature descriptor A _i (x) generated in step S110 for each output of the unit i of the set θ _u composed of the units selected in step S106. , Y) to generate a normalized image feature descriptor R _i (x, y).

In step S114, the tag relevance calculation unit 38 selects each of the KL distances S _i (u | D) obtained for each output of the unit i of the set θ _{u including} the units selected in step S106, and selects Based on each of the normalized image feature descriptors R _i (x, y) generated in step S112 for each output of the unit i, the normalized image feature descriptor R according to the above equation (2). By adding each of _i (x, y) with a weight according to the KL distance S _i (u | D), between the tag and the pixel for each pixel of the input image I (x, y) Calculate relevance.

  In step S116, the output unit 40 outputs the degree of association between the tag and the pixel for each pixel of the input image I (x, y) calculated in step S114, and ends the process.

<Experimental example>
FIG. 4 is an example of a tag-related region obtained by manually extracting tags from an actual image and a tag-related region obtained by the tag-related region extraction device according to the embodiment of the present invention. FIG. 5 shows extracted tag-related regions when the value of the number N of units to be selected from the one having the larger distance S _i (u | D) between the histograms is changed. The value of the optimal number N of units depends on the tag. As the number N of units increases, the amount of calculation increases, but tag-related areas are extracted in more detail.

  As described above, according to the tag related region extracting device according to the embodiment of the present invention, each of the images included in the positive set and each of the images included in the negative set are input to the CNN, and each unit of the CNN is input. The size for calculating the distance between the positive set histogram and the negative set histogram, selecting the output of the top N units for the calculated distance, and masking the input image A plurality of different masks are generated, and for each output of the selected unit, each of the mask images, which are input images masked by each of the plurality of masks, is input to the CNN and obtained from each of the mask images. An image representing the average output of the unit is generated as an image feature descriptor, and the generated image feature descriptor is normalized. Based on each of the distances obtained for each output of the selected unit and each of the normalized image feature descriptors generated for each output of the selected unit. By adding each of the normalized image feature descriptors with a weight corresponding to the distance, the degree of association between the tag and the pixel for each pixel of the input image is calculated, thereby associating with the tag in the image. The area to be performed can be extracted with high accuracy.

  Further, according to the embodiment of the present invention, such region information is not included, but a related region for each tag can be learned from data associated with an image and a tag included in the image.

  The present invention is not limited to the above-described embodiment, and various modifications and applications can be made without departing from the gist of the present invention.

  For example, in the above-described embodiment, the case where the present invention is applied to the tag-related region extraction device has been described as an example. However, the present invention is not limited to this, and the present invention may be applied to a feature selection device. In this case, the feature selection device includes an image feature generation unit 24, an image feature distribution comparison unit 26, and a feature descriptor selection unit 28.

  According to this feature selection apparatus, for each of a plurality of image features obtained from an image based on a positive set and a negative set, a histogram representing the distribution of image features of the positive set image and an image of the negative set image Generating a histogram representing the distribution of features, calculating a distance between the positive set histogram and the negative set histogram for each of the plurality of image features, and calculating the top N image features for the calculated distance. By selecting as an image feature descriptor, an image feature effective for image identification can be acquired.

  In the above embodiment, the case where the output of each unit of the neural network learned in advance is used as each image feature obtained from the image is described as an example. However, the present invention is not limited to this. Image features may be used.

  In the above-described embodiment, the case where CNN is used as the neural network has been described as an example. However, the present invention is not limited to this, and another neural network may be used.

  Further, in the above-described embodiment, the case where the Cullback / Librer distance is used as the distance between the histograms has been described as an example, but the present invention is not limited to this, and other distances may be used.

  The tag related area extracting apparatus 100 described above has a computer system inside, but the “computer system” includes a homepage providing environment (or display environment) if a WWW system is used. .

  In the present specification, the embodiment has been described in which the program is installed in advance. However, the program can be provided by being stored in a computer-readable recording medium.

DESCRIPTION OF SYMBOLS 10 Input part 20 Operation part 22 Image set database 24 Image feature generation part 26 Image feature distribution comparison part 28 Feature descriptor selection part 30 Tag related area extraction part 32 Mask generation part 34 Feature descriptor generation part 36 Feature descriptor normalization part 38 Tag Relevance Degree Calculation Unit 40 Output Unit 100 Tag Related Area Extraction Device

Claims (8)

Based on a positive set which is a set of images to which a tag representing a specific object included in the image is assigned and a negative set which is a set of images to which the tag is not assigned, a plurality of image features obtained from the image Image features that generate, for each, a histogram representing the distribution of the image features for each of the images included in the positive set and a histogram representing the distribution of the image features for each of the images included in the negative set. A generator,
An image feature distribution comparison unit that calculates a distance between the histogram of the positive set and the histogram of the negative set for each of a plurality of the image features generated by the image feature generation unit;
A feature descriptor selection unit that selects the top N image features for the distance calculated by the image feature distribution comparison unit;
A feature selection device. The image feature generation unit is configured to convert each of the images included in the positive set and each of the images included in the negative set based on the positive set and the negative set and a previously learned neural network to the neural network. A histogram representing the distribution of the output of the unit for each of the images included in the positive set, for the output of each unit of the neural network as each of a plurality of image features obtained from the image; Generating a histogram representing the distribution of the output of the unit for each of the images included in the negative set;
The feature selection apparatus according to claim 1, wherein the feature descriptor selection unit selects the outputs of the top N units for the distance calculated by the image feature distribution comparison unit. A feature selection device according to claim 2;
A mask generation unit that generates a plurality of masks having different sizes for masking an input image;
For each output of the unit selected by the feature descriptor selection unit, each of the mask images as the input image masked by each of the plurality of masks generated by the mask generation unit is converted into the neural network. A feature descriptor generator for generating an image representing an average of the output of the unit, which is input to the network and obtained from each of the mask images, as an image feature descriptor;
A feature descriptor normal that generates a normalized image feature descriptor obtained by normalizing the image feature descriptor generated by the feature descriptor generation unit for each output of the unit selected by the feature descriptor selection unit. And
Each of the distances obtained for the output of each of the selected units by the image feature distribution comparison unit, and the output of each of the units selected by the feature descriptor normalization unit. The tag and the pixel for each pixel of the input image by adding each of the normalized image feature descriptors with a weight according to the distance based on each of the normalized image feature descriptors A tag relevance calculator that calculates the relevance between
A tag-related area extracting device including: The tag related region extraction device according to claim 3, wherein the tag relevance calculation unit further extracts a region formed of an image having a relevance greater than or equal to a predetermined value as a region related to the tag. The feature selection device according to claim 2, wherein a CNN (Convolutional Neural Network) is used for the neural network. The tag-related region extraction device according to claim 3 or 4, wherein a CNN (Convolutional Neural Network) is used for the neural network. The image feature generation unit is obtained from the image based on a positive set that is a set of images to which a tag representing a specific object included in the image is assigned and a negative set that is a set of images to which the tag is not assigned. A histogram representing the distribution of the image features for each of the images included in the positive set and a histogram representing the distribution of the image features for each of the images included in the negative set. Generating and
An image feature distribution comparison unit calculating a distance between the histogram of the positive set and the histogram of the negative set for each of the plurality of image features generated by the image feature generation unit; ,
A feature descriptor selection unit selecting the top N image features for the distance calculated by the image feature distribution comparison unit;
A feature selection method including:   For causing a computer to function as each part of the feature selection device according to claim 1, claim 2, or 5, or the tag related region extraction device according to claim 3, claim 4, or claim 6 program.