JP2015109024A - Image dictionary generation device, image dictionary generation method and computer program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To improve accuracy of an image dictionary.SOLUTION: Feature amounts are extracted for a plurality of learning images corresponding to a certain semantic label, full station representative information item indicating features of the plurality of learning images corresponding to the semantic label is extracted on the basis of the feature amounts, and the feature amounts are clustered. For each cluster, local representative information is extracted on the basis of the feature amount included in the cluster, and on the basis of the full station representative information and the local representative information, the learning images and the clusters are associated with each other. On the basis of the association between the clusters and the plurality of learning images, identifiers of the clusters are generated, and a combination of the identifiers is acquired as an image dictionary of the semantic label.

Description

  The present invention relates to a technique for generating an image dictionary.

There are the following methods as a conventional image dictionary generation method.
First, a plurality of topic models are obtained from learning image groups related to all semantic labels that are targets of the image dictionary. Next, the similarity between each learning image and the topic model is calculated. Next, learning images are associated with a plurality of topics in descending order of similarity. Next, when an image dictionary related to a certain meaning label is constructed, learning images assigned to the meaning label are collected from the learning image group belonging to each topic. Next, an identification model for each topic is constructed using a machine learning technique. And the image dictionary of the meaning label is produced | generated by matching the identification model for every topic (refer nonpatent literature 1).

TRECVID 2012 Semantic Video Concept Detection by NTT-MD-DUTY.Sun, K. Sudo, Y. Taniguchi-NTT Media Intelligence Laboratories, Japan H. Li, L. Yi, Y. Guan-School of Software, Dalian University of Technology, China, http://www-nlpir.nist.gov/projects/tvpubs/tv12.papers/ntt.pdf

However, the image dictionary generation method shown in Non-Patent Document 1 has the following problems.
A semantic label may be composed of a combination of a plurality of elements (topics) in an image. For example, the meaning label “beach” is composed of a combination of a plurality of topics such as “sea”, “sun”, “sand”, and “people”. As a result, it is necessary to reconstruct the image content using multiple topics. When there are many topics, if only some topics are used, the use of information in the learning image group (all station information) becomes insufficient. Therefore, there is a problem that the accuracy of the image dictionary is lowered.

  In addition, in the conventional technique for expressing a topic with an average feature vector of an image, the image dictionary has insufficient expression power. As a result, there has been a problem that the accuracy of meaning labeling is lowered.

  In addition, when the meaning of an image is composed of a plurality of topics, the contribution degree of each topic may be different. For example, the meaning of the image “beach” is expressed by a combination of a plurality of topics “sea”, “sun”, “sand”, and “people”. However, in a single “beach” image in which “sea” is projected large and “people” is projected small, the topic “sea” and topic “people” have different degrees of contribution. Since the degree of contribution for each topic is not reflected in the process, there is a problem that the accuracy of meaning label assignment is lowered.

  In view of the above circumstances, an object of the present invention is to provide a technique that can improve the accuracy of an image dictionary.

  According to one aspect of the present invention, a feature amount extraction unit that extracts a feature amount for a plurality of learning images associated with a semantic label, and a plurality of learning images associated with the semantic label based on the feature amount An all-station representative information extracting unit that extracts all-station representative information indicating the features of the local area, and a local representative information extracting unit that clusters the feature amounts and extracts local representative information based on the feature amounts included in the clusters for each cluster; A learning image associating unit for associating the learning image with the cluster based on the all-station representative information and the local representative information, and a classifier for each cluster based on the association between the cluster and a plurality of learning images. And an image dictionary generation unit that acquires the combination of the discriminators as an image dictionary of the semantic labels.

  One aspect of the present invention is the image dictionary generation device described above, wherein the learning image association unit calculates a reconstruction error between each learning image and the cluster based on the all-station representative information and the local representative information. Then, the learning image is associated with the cluster in ascending order of the reconstruction error.

  According to one aspect of the present invention, a feature amount extracting step of extracting a feature amount for a plurality of learning images associated with a certain meaning label, and a plurality of learning images associated with the meaning label based on the feature amount All-station representative information extracting step for extracting all-station representative information indicating the characteristics of the above; and local representative information extracting step for clustering the feature amounts and extracting local representative information based on the feature amounts included in the clusters for each cluster; A learning image associating step for associating the learning image with the cluster based on the all-station representative information and the local representative information, and an identifier for each cluster based on the association between the cluster and a plurality of learning images. And an image dictionary generating step for acquiring a combination of the discriminators as an image dictionary of the semantic labels. It is a generation method.

  One aspect of the present invention is a computer program for causing a computer to operate as the image dictionary generation apparatus.

  According to the present invention, it is possible to improve the accuracy of the image dictionary.

It is a schematic block diagram which shows the structure of an image dictionary production | generation apparatus. It is a flowchart which shows the specific example of a process of an image dictionary production | generation apparatus. It is a flowchart which shows the specific example of the process in which a learning image matching part matches a learning image with a topic. It is a flowchart which shows the specific example of the process in which an image dictionary production | generation part produces | generates an image dictionary. It is a schematic block diagram which shows the structure of an image meaning label provision apparatus. It is a flowchart showing the specific example of the process which assign | provides a semantic label to an image or an image | video using the image dictionary produced | generated by the image dictionary production | generation apparatus. It is the schematic showing the outline of the process which provides a semantic label to an image and an image | video using an image dictionary.

The details of an embodiment of the image dictionary generating apparatus according to the present invention will be described.
FIG. 1 is a schematic block diagram showing the configuration of the image dictionary generation device 10. The image dictionary generation device 10 includes a CPU (Central Processing Unit), a memory, an auxiliary storage device, and the like connected by a bus, and executes an image dictionary generation program. The image dictionary generation device 10 executes the image dictionary generation program by storing the storage unit 1, the learning image group collection unit 2, the feature amount extraction unit 3, the representative information extraction unit 4, the learning image association unit 5, and the image dictionary generation unit 6. It functions as a device provided with. Note that all or some of the functions of the image dictionary generation device 10 may be realized using hardware such as an application specific integrated circuit (ASIC), a programmable logic device (PLD), or a field programmable gate array (FPGA). good. The image dictionary generation program may be recorded on a computer-readable recording medium. The computer-readable recording medium is, for example, a portable medium such as a flexible disk, a magneto-optical disk, a ROM, a CD-ROM, or a storage device such as a hard disk built in the computer system.

  The storage unit 1 is configured using a storage device such as a magnetic hard disk device or a semiconductor storage device. The accumulating unit 1 accumulates learning images prepared in advance for each semantic label. The association between the learning image and the semantic label may be performed by any process. For example, a learning image may be associated with a semantic label by a human hand. For example, images and tags that exist on the Web may be acquired as associated learning images and semantic labels. When the storage unit 1 receives designation of a semantic label from the learning image group collection unit 2, the accumulation unit 1 outputs a plurality of learning images associated with the designated semantic label to the learning image group collection unit 2.

  The learning image group collection unit 2 outputs a plurality of learning images associated with the designated semantic label by outputting the designation of the semantic label to the storage unit 1. The learning image group collection unit 2 outputs the received plurality of learning images (hereinafter referred to as “learning image group”) to the feature amount extraction unit 3. The learning image group also includes information on the meaning label of each learning image (hereinafter referred to as “semantic label information”).

  The feature quantity extraction unit 3 receives a learning image group related to a semantic label from the learning image group collection unit 2. The feature amount extraction unit 3 extracts a feature amount for each learning image included in the received learning image group. The feature quantity extraction unit 3 may extract feature quantities (physical feature quantities) such as color histograms and pattern histograms, for example. The feature amount extraction unit 3 may divide the learning image and extract the feature amount from each region. The feature amount extraction unit 3 may extract SIFT feature points from the learning image and extract local feature amounts such as SIFT. The feature quantity extraction unit 3 outputs the extracted feature quantity to the local representative information extraction unit 42.

  The representative information extraction unit 4 extracts representative information. The representative information extraction unit 4 includes an all-station representative information extraction unit 41 and a local representative information extraction unit 42.

The all-station representative information extraction unit 41 receives the feature amount of the learning image group extracted from the feature amount extraction unit 3. The all-station representative information extraction unit 41 extracts all-station representative information based on the received feature amount. The all-station representative information is information that simply and accurately represents the entire information of the learning image group associated with one semantic label. For example, the all-station representative information extraction unit 41 may extract all-station representative information based on local feature amounts at all SIFT feature points of the learning image group image. For example, a sparse representation method may be applied to this processing. The technique disclosed in Reference Document 1 shown below may be applied to the sparse representation method.
Reference 1: http://www.stanford.edu/class/cs294a/sparseAutoencoder_2011new.pdf

The base D ^S (matrix N × M composed of a plurality of feature amounts, N is the number of feature dimensions, M> N) obtained by this method is representative information of all stations. The all-station representative information extraction unit 41 stores the extracted all-station representative information in the storage unit 1. Further, the all-station representative information extraction unit 41 outputs the extracted all-station representative information to the learning image association unit 5.

  The local representative information extraction unit 42 receives a plurality of feature amounts extracted by the feature amount extraction unit 3. The local representative information extraction unit 42 performs a clustering process on the received plurality of feature amounts to generate a plurality of clusters. Each generated cluster corresponds to one topic. For example, the clustering process may be executed using a k-means method, an LDA (Latent Dirichlet Allocation) method, or the like.

In the clustering process, an image group with similar image similarity calculated from the feature amount is output as one cluster (topic). The local representative information is information that simply and accurately represents the information of the learning image included in each cluster. For example, the local representative information extraction unit 42 may extract the local representative information based on the local feature amount at the SIFT feature point of the learning image included in the cluster. For example, a sparse representation method may be applied to this processing. Hereinafter, the local representative information is represented as D ^I _c . c = 1, 2. K is the number of topics.

  The local representative information extraction unit 42 stores the extracted local representative information in the storage unit 1. Further, the local representative information extraction unit 42 outputs the extracted local representative information to the learning image association unit 5.

  The learning image association unit 5 receives all station representative information from the all station representative information extraction unit 41. The learning image association unit 5 receives local representative information from the local representative information extraction unit 42. In the following description, overall representative information and local representative information are collectively referred to as “representative information”. The learning image association unit 5 acquires the reconstruction error of each image in the learning image group using the representative information. The learning image association unit 5 associates each learning image with a plurality of topics in order from the image with the smallest reconstruction error. The learning image association unit 5 outputs the correspondence between the topic and the learning image to the image dictionary generation unit 6.

  The image dictionary generation unit 6 receives a learning image for each topic from the learning image association unit 5. The image dictionary generation unit 6 collects learning images assigned to semantic labels (hereinafter referred to as “processing target semantic labels”) for which an image dictionary is to be generated, and uses a machine learning method to determine an identification model for each topic. To construct. Then, the image dictionary generation unit 6 generates an image dictionary of the processing target semantic labels by combining the identification models for each topic. The image dictionary generation unit 6 stores the generated image dictionary in the storage unit 1.

Next, all-station representative information and local representative information will be described.
When sparse representation is used, all-station representative information and local representative information are expressed by the following equations 1 and 2, respectively.

In Equation 1, D _c is a base representing representative information composed of all-station representative information and local representative information. D ^S is a base representing all-station representative information. D ^I _c is a base representing local representative information.

In Equation 2, G _c is a coefficient corresponding to D _c . G ^S _c is a coefficient corresponding to the ^{D S} when the image is reconstructed by ^{D S.} G ^I _c is a coefficient corresponding to D ^I _c when the image is reconstructed with a certain D ^I _c (c = 1, 2,...).

D _c combines the sparse coding bases of the learning images and the bases of the individual topics. In other words, [D ^S , D ^I _c ] is a matrix in which the matrix is arranged horizontally, and G _c is a coefficient corresponding to the base.
The topic model in Reference 1 is represented by a set of parameters (such as an average feature amount by k-means). On the other hand, in this embodiment, a topic model is represented using all-station representative information and local representative information. Since the entire information and local information are used in this way, the content of the topic can be expressed accurately with little loss of information.

  FIG. 2 is a flowchart illustrating a specific example of processing of the image dictionary generation device 10. Hereinafter, a processing example of the image dictionary generation device 10 will be described with reference to FIG.

  First, the learning image group collection unit 2 reads all the learning images (learning image group) associated with the semantic labels from the storage unit 1 (step S201). Next, the feature amount extraction unit 3 extracts a feature amount for each learning image in the read learning image group (step S202). Next, the all station representative information extraction unit 41 extracts all station representative information (step S203). Next, the local representative information extraction unit 42 performs a clustering process on the feature amount extracted by the feature amount extraction unit 3. And the local representative information extraction part 42 extracts the local representative information of each cluster (step S204).

  Next, the learning image association unit 5 associates each learning image in the learning image group with each topic based on the representative information (step S205).

  FIG. 3 is a flowchart illustrating a specific example of processing in which the learning image association unit 5 associates the learning image with the topic. First, the learning image associating unit 5 reads out a learning image group (including N learning images: N is an integer of 1 or more) related to all semantic labels (step S301). Next, the learning image association unit 5 reads the representative information extracted in step S203 and step S204 (step S302).

Next, the learning image associating unit 5 substitutes 1 for the variable n (step S303). The learning image associating unit 5 calculates a reconstruction error of the nth learning image (feature amount is x _i ) using the K topic models (step S304).
When Sparse coding is used, the mathematical formula reconstructed using the c-th topic model is expressed as follows.

g _ci is a coefficient of the i-th column in the coefficient G _c . For the nth learning image, the reconstruction error using the cth topic model is expressed by the following mathematical formula.

  Next, the learning image association unit 5 determines whether or not the calculation of the reconstruction error has been completed for all the learning images included in the learning image group (step S305). Specifically, the learning image association unit 5 determines whether or not the variable n is smaller than the number N of learning images. When the variable n is smaller than N (step S305—YES), the learning image association unit 5 increments the variable n and returns to the process of step S304 (step S307). On the other hand, when the variable n is greater than or equal to N (step S305—NO), the learning image associating unit 5 determines, for each topic, the top P (P is an integer equal to or greater than 1) in order from the smallest calculated reconstruction error. ) Is associated with the topic (step S306).

The value of P may be set as follows, for example. First, the reconstruction errors ε _ic are arranged in a line in ascending order. For example, reconstruction errors are arranged like (0.58, 0.6, 0.62, 0.95, 0.96, 0.98). Next, the reconstruction error epsilon _ics, divided by the previous reconstruction error epsilon _ics. The values obtained in this way are compared before and after, and the number of topics processed up to the point when the numerical value suddenly increases is adopted as P. In the above example, the topic before the topic to which 0.95 belongs is adopted. By performing such processing, it is possible to automatically determine the top P.

  Through the above processing, the learning image is associated with a plurality of topics. For example, when SIFT points and area feature quantities are used, the learning image associated with the semantic label “beach” is associated with topics such as “sun”, “people”, and “sea”.

  Returning to the description of FIG. After the process of step S205, the image dictionary generation unit 6 generates an image dictionary related to the semantic label based on the learning image associated with each topic (step S206).

  FIG. 4 is a flowchart showing a specific example of processing in which the image dictionary generating unit 6 generates an image dictionary. First, the image dictionary generation unit 6 reads a plurality of learning images associated with each topic model for each of M topic models (step S401). Next, the image dictionary generating unit 6 substitutes 1 for the variable c (step S402). Next, the image dictionary generation unit 6 substitutes 1 for the variable m (step S403).

  Next, the image dictionary generation unit 6 acquires a learning image assigned to the c-th semantic label from learning images associated with the m-th topic model (step S404). In this process, the image dictionary generation unit 6 acquires all learning images that meet such a condition. Next, the image dictionary generation unit 6 determines whether or not the number of acquired learning images is equal to or greater than a predetermined threshold (step S405). When the number of acquired learning images is equal to or greater than a predetermined threshold (step S405—YES), the image dictionary generation unit 6 uses the machine learning technique to identify the mth topic classifier relating to the cth semantic label. Generate (step S406). A specific example of the machine learning method is SVM (Support Vector Machine). Regarding the feature amount used when generating the discriminator, when the feature amount extracted in step S202 is a physical feature amount (color, pattern, etc.), the feature amount may be used as it is. When the feature amount extracted in step S202 is a SIFT feature point or a region feature amount, a feature point or a region feature amount whose similarity to the mth topic model in the nth learning image is equal to or greater than a certain value is used as the feature amount. It may be used as.

  In the process of step S405, when the number of acquired learning images is less than the predetermined threshold (step S405-NO), or after the process of step S406, the image dictionary generation unit 6 relates to the c-th semantic label. It is determined whether or not the processing of steps S404 to S406 has been completed for all topics (step S407). If there is a topic that has not been processed, that is, if the variable m is smaller than M (step S407: YES), the image dictionary generation unit 6 increments m (step S410) and returns to the process of step S404. .

  On the other hand, in the process of step S407, when the processes of steps S404 to S406 are completed for all topics (step S407-NO), that is, when the variable m is M or more, the image dictionary generation unit 6 identifies each topic. Is stored in the storage unit 1 as an image dictionary of the c-th semantic label (step S408).

  Next, the image dictionary generation unit 6 determines whether or not the process of step S408 has been performed on all semantic labels (step S409). When there is a meaning label that has not been subjected to the process of step S408 (step S409-YES), that is, when the variable c is smaller than C, the image dictionary generation unit 6 increments the variable c and proceeds to the process of step S403. Return. On the other hand, when the process of step S408 is completed for all semantic labels (step S409-NO), that is, when the variable c is C or more, the image dictionary generation unit 6 ends the process.

  FIG. 5 is a schematic block diagram showing the configuration of the image meaning label assigning device 20. The image meaning label assigning device 20 includes a CPU, a memory, an auxiliary storage device, and the like connected by a bus, and executes an image meaning label assigning program. The image meaning label assigning device 20 includes an accumulating unit 21, an image collecting unit 22, a feature amount extracting unit 23, an associating unit 24, a similarity integrating unit 25, and an image meaning label assigning unit 26 by executing an image meaning label assigning program. It functions as a device provided. Note that all or some of the functions of the image meaning label assigning apparatus 20 may be realized using hardware such as an ASIC, a PLD, or an FPGA. The image dictionary generation program may be recorded on a computer-readable recording medium. The computer-readable recording medium is, for example, a portable medium such as a flexible disk, a magneto-optical disk, a ROM, a CD-ROM, or a storage device such as a hard disk built in the computer system.

  The storage unit 21 is configured using a storage device such as a magnetic hard disk device or a semiconductor storage device. The storage unit 21 stores data such as images to be classified, representative information, and image dictionaries.

  The image collection unit 22 receives the analysis target image stored in the storage unit 21 by outputting the designation of classification to the storage unit 21. The image collection unit 22 outputs the received analysis target image to the feature amount extraction unit 23.

  The feature amount extraction unit 23 receives the analysis target image from the image collection unit 22. The feature amount extraction unit 23 extracts a feature amount from the received analysis target image. The feature quantity extraction unit 23 may extract feature quantities (physical feature quantities) such as color histograms and pattern histograms, for example. The feature amount extraction unit 23 may divide the learning image and extract the feature amount from each region. The feature amount extraction unit 23 may extract SIFT feature points from the learning image and extract local feature amounts such as SIFT. The algorithm used when the feature quantity extraction unit 23 extracts the feature quantity is preferably the same algorithm as the feature quantity extraction unit 3 of the image dictionary generation device 10. The feature quantity extraction unit 23 outputs the extracted feature quantity to the association unit 24.

  The association unit 24 calculates the reconstruction error of the analysis target image based on the feature amount received from the feature amount extraction unit 23 and the representative information received from the storage unit 21. The association unit 24 associates analysis target images with a plurality of topics in ascending order of reconstruction error. The association unit 24 outputs information indicating the result of the association (association information) to the similarity integration unit 25.

  The similarity integration unit 25 receives the association information from the association unit 24. The similarity integration unit 25 receives the image dictionary from the storage unit 21. The similarity integration unit 25 calculates the similarity between the analysis target image and the classifier of each topic. The similarity integration unit 25 integrates the similarity using the reconstruction error for each topic received from the association unit 24. The similarity integrating unit 25 outputs information (integrated information) indicating the integrated result to the image meaning label attaching unit 26.

  The image meaning label assigning unit 26 assigns a meaning label to the analysis target image using the integrated information received from the similarity integrating unit 25. The image meaning label assigning unit 26 stores the image assigned the meaning label in the storage unit 21.

  FIG. 6 is a flowchart illustrating a specific example of processing in which the image meaning label assigning device 20 assigns a meaning label to an image or video using the image dictionary generated by the image dictionary generating device 10. FIG. 7 is a schematic diagram showing an outline of a process for assigning a semantic label to an image or video using an image dictionary. Hereinafter, a specific example of processing for assigning a semantic label to an image or video using an image dictionary will be described.

First, the semantic label assigning device 20 acquires all-station representative information and local representative information from the storage unit 1 (step S501). Next, the semantic label assigning device 20 reads an image (for example, J images) to which a semantic label is assigned (step S502). Next, the semantic label assigning apparatus 20 substitutes 1 for the variable j (step S503). Next, the semantic label assignment apparatus 20 calculates a reconstruction error ε _jc using the j-th image and the topic model Dc (c = 1,..., K). The semantic label assigning device 20 associates the topic model Dc for which a reconstruction error equal to or less than a predetermined threshold is calculated with the j-th image. For example, in FIG. 7, topic models (T1 and Tm−1) displayed as a cross “×” are associated with the image j.

Next, the semantic label assignment apparatus 20 calculates, for each column in the table of FIG. 7, the feature amount related to the topic model associated with j and the similarity sc between the individual classifiers in the column. Then, the semantic label assigning device 20 uses the reconstruction error calculated for each semantic label Ci (i = 1,..., L: i and L are integers of 1 or more) as weights, and calculates the calculated similarity sc. By integrating, the similarity between the image j and the semantic label Ci is acquired. When the associated reconstruction error is ε _jc ′, the weighting wjc is calculated by the following equation (5).

Q represents the number of topics to which the image j corresponds for each row. The similarity between the image j and a certain meaning label is calculated by Equation 6.

  Next, the semantic label assigning device 20 assigns up to R semantic labels to the image j in order from the highest similarity acquired in step S505 (step S506). Next, the semantic label assigning device 20 determines whether or not semantic labels have been assigned to all images (step S507). If there is an image to which no semantic label is assigned (step S507: YES), that is, if the variable j is smaller than J, the semantic label assigning device 20 increments the variable j and returns to the process of step S504. On the other hand, if there is no image to which no semantic label is assigned (step S507—NO), that is, if the variable j is equal to or greater than J, the semantic label assignment device 20 ends the process.

  In the image dictionary generation device 10, an image is associated with a plurality of topics using all-station representative information representing the entire information of the image group and local representative information of the topics divided into the image group. Also, the degree of contribution of each topic to the image is given by the reciprocal of the reconstruction error. Using the degree of contribution of each topic to the image, the identification results for each topic are integrated. With these processes, it is possible to generate an image dictionary with high accuracy.

  Also, the image dictionary generation device 10 generates a topic model common to all semantic labels based on learning image groups (image groups having various meanings) related to all semantic labels. Therefore, it is possible to generate a topic model that is more varied. For example, “Beach”, “Festival”, “Meeting”, “Wild”, “Drinking Party”, etc. are more abundant than when generating “People” topic models only from images related to the meaning label “Beach”. If a “person” topic model is generated from a group of images related to various semantic labels, a more accurate and versatile topic model can be generated.

In addition, since a machine learning method is used when generating an image dictionary, there has been a problem that processing time increases as learning data increases. For example, in an image dictionary generation method in which one identification model is calculated using all learning data for one semantic label as in Non-Patent Document 1, an enormous processing time is required depending on the amount of learning images. There was a problem that it was necessary. In order to deal with such a problem, the image dictionary generation device 10 generates a common topic model for all semantic labels, so that it is possible to suppress an increase in processing time when the amount of learning images increases. .
In addition, when there are a large number of semantic labels, or when there are a large number of images to which a semantic label is to be applied, it is possible to apply the semantic labels more efficiently.

<Modification>
In the process of step S306 in the flowchart of FIG. 3, the learning image association unit 5 may associate the topic with the learning image based on another criterion based on the reconstruction error. For example, the learning image associating unit 5 may associate all the images of reconstruction errors that are equal to or less than a predetermined threshold with respect to each topic.

  The embodiment of the present invention has been described in detail with reference to the drawings. However, the specific configuration is not limited to this embodiment, and includes designs and the like that do not depart from the gist of the present invention.

DESCRIPTION OF SYMBOLS 10 ... Image dictionary production | generation apparatus, 1 ... Accumulation part, 2 ... Learning image group collection part, 3 ... Feature-value extraction part, 4 ... Representative information extraction part, 41 ... All station representative information extraction part, 42 ... Local representative information extraction part, DESCRIPTION OF SYMBOLS 5 ... Learning image matching part, 6 ... Image dictionary production | generation part, 20 ... Image meaning label provision apparatus, 21 ... Accumulation part, 22 ... Image collection part, 23 ... Feature-value extraction part, 24 ... Association part, 25 ... Similarity Degree integration unit, 26 ... image meaning label assigning unit

Claims (4)

A feature amount extraction unit that extracts feature amounts of a plurality of learning images associated with a certain meaning label;
An all-station representative information extraction unit that extracts all-station representative information indicating features of a plurality of learning images associated with the semantic label, based on the feature amount;
Clustering the feature quantities, and for each cluster, a local representative information extraction unit that extracts local representative information based on the feature quantities included in the cluster;
A learning image association unit that associates the learning image with the cluster based on the all-station representative information and the local representative information;
An image dictionary generation unit that generates a classifier of each cluster based on the association between the cluster and a plurality of learning images, and acquires a combination of the classifiers as an image dictionary of the semantic labels;
An image dictionary generation device comprising:   The learning image association unit calculates a reconstruction error between each learning image and the cluster based on the all-station representative information and the local representative information, and the learning image and the cluster in ascending order of the reconstruction error. The image dictionary generation device according to claim 1, which is associated with the image dictionary. A feature amount extraction step for extracting feature amounts for a plurality of learning images associated with a certain meaning label;
All-station representative information extraction step for extracting all-station representative information indicating the characteristics of a plurality of learning images associated with the semantic label based on the feature amount;
Clustering the feature amounts, and extracting local representative information based on the feature amounts included in the clusters for each cluster; and
A learning image association step for associating the learning image with the cluster based on the all-station representative information and the local representative information;
An image dictionary generating step for generating a classifier of each cluster based on the association between the cluster and a plurality of learning images, and acquiring a combination of the classifiers as an image dictionary of the semantic labels;
An image dictionary generation method comprising:   A computer program for operating a computer as the image dictionary generation device according to claim 1.

Images