JP2017027526A - Feature value generation method, feature value generation device, and feature value generation program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a feature value generation method, a feature value generation device, and a feature value generation program that make it possible to discover a semantically associated signal content with high accuracy.SOLUTION: The method includes: generating a correspondence relation between each word and a word vector on the basis of co-occurrence of a word included in a document so that word vectors are closer to each other for words that are more likely to co-occur; extracting the document feature value of a document using a learned word vector; extracting the initial feature value of a signal content; finding the relative geometrical relation of the signal content and the document using at least one of the word vector and/or the document feature value; and generating, on the basis of the initial feature value, the relative geometrical relation, and a relation indicator, a feature value conversion function that converts the initial feature value to a low-order feature value and outputting the generated function.SELECTED DRAWING: Figure 1

Description

  The present invention enables the discovery of signal content more semantically related by using a document when generating a feature amount for executing search and recognition of signal content such as image, audio, and video. The present invention relates to a feature quantity generation method, a feature quantity generation device, and a feature quantity generation program for generating quantities.

  The number of media contents (images, videos, sounds, etc.) distributed on the network has become enormous due to sophistication and high quality of communication environments, computers, distributed processing infrastructure technologies, and the like. For example, a search engine is said to have trillions of web pages indexed. Some sites report that 350 million images are uploaded every day, and some sites report that 64 hours of video per minute are newly released. .

  Hereinafter, for the sake of convenience, content made up of signal media such as images, video, and audio will be referred to as signal content.

  While such a huge amount of signal content is a rich source of information for users, it also brings about the problem that it is increasingly difficult to quickly access the signal content desired to be viewed. In such a trend, there is an increasing demand for media analysis technology for efficiently searching for signal contents to be browsed and viewed.

  In the analysis of signal content, procedures for finding semantically related signal content play an important role. For example, considering the case of classifying signal contents, it is usual to classify signal contents belonging to the same semantic concept into the same category. Alternatively, when searching for signal content, when signal content is given as a query, it is a basic requirement to search for signal content that is semantically related to the signal content. In addition, in content recommendation, a signal content similar to the signal content that the user has browsed / viewed so far is discovered and recommended, and this is also semantically duplicated in the case of content summarization. It is necessary to organize the contents into a non-content.

  Describe a typical procedure for discovering semantically relevant signal content. First, signal content is expressed by a certain feature amount. Next, the similarity is calculated by measuring the closeness of the feature quantities, and the closer the similarity is, the more the signal content is considered to be semantically related. To give a simple example, if the signal content is an image or video, the degree of similarity can be measured using the color histogram of the image (video frame) as a feature amount. In the case of an audio signal, the degree of similarity can be measured using a characteristic (such as Mel-Frequency Cepstral Coefficient) obtained by analyzing the frequency characteristics of the waveform of the audio signal. Needless to say, if the number of contents is 1,000, it is necessary to calculate the similarity for each of the 1,000 contents, and to pick up the content having a high similarity as a similar content.

However, when measuring the similarity of signal content, there are two important problems shown in (1) and (2) below.
(1) Requires enormous calculation time (2) Difficult to find semantically similar signal content

  Hereinafter, the important problems shown in the above (1) and (2) will be specifically described.

  (1) Usually, the dimension of the feature amount (vector) of the signal content is often high, and the calculation of the degree of similarity requires a huge amount of time. Taking images and videos as an example, even simple features such as color histograms are typically hundreds to thousands of dimensional real-valued vectors, and in recent feature representations using neural networks, A feature representation based on a thousand dimensions, a sparse representation or a Fisher kernel can be a vector of hundreds of thousands to millions of dimensions. Furthermore, since it is necessary to calculate the degree of similarity for all content sets, the calculation time is proportional to the dimension D of the feature amount and the number N of contents, regardless of what degree of similarity calculation means is used. Cost.

(2) As represented by the image feature values such as the color histogram described above, the feature representation of signal content such as images, video, and audio generally represents physical properties in general. Naturally, just because physical features are close does not necessarily mean that the signal content is semantically related. For example, you want content similar to the “(red) apple” image, not “red candy”, but the same fruits “green apple” and “orange”, but these are at least close to the color histogram. Cannot be evaluated correctly.

  In view of the above background, there is a demand for a technique that can generate (1) a feature quantity of signal content that enables the discovery of semantically related content while being (1) fast.

  In the past, several inventions have been made and disclosed regarding such techniques.

  For example, in the technique disclosed in Non-Patent Document 1, a large number of images and associated semantic labels (that is, labels indicating what semantic category each image belongs to) are given. Below, a method for learning the relationship between images and semantic labels and converting them into features using the Convolutional Neural Network (CNN) is disclosed.

  Further, in the technology disclosed in Patent Document 1, in the compression of the feature amount of two types of signal content that co-occurs simultaneously, one or both of the feature amounts are missing and the signal content that has not been co-occurrence is detected. In such a case, a feature quantity generation technique for reducing the dimension of the original feature quantity to reduce the dimension is disclosed.

JP 2010-282277 A

Alex Krizhevsky, Ilya Sutskever, Geoffrey E. Hinton, "ImageNet Classification with Deep Convolutional Neural Networks." In Proc. Advances in Neural Information Processing Systems (NIPS) ", Pages. 1097-1105, 2012.

  In the technique disclosed in Non-Patent Document 1, it is possible to obtain a semantic image feature amount by learning a relationship between an image and a semantic label based on a pair of an image and a semantic label. However, this technique is based on the assumption that an enormous amount of images (for example, approximately 1.2 million images in the example disclosed in Non-Patent Document 1) and semantic labels for each of them are known. In many cases, when a semantic label is assigned to an image, it must be given manually, and it is practically difficult to assign a semantic label to such a large amount of images. There were many cases where it was difficult to use the technology. Further, this technique is a technique that can be applied only to images, and cannot be applied to other signal contents such as audio.

  The technique disclosed in Patent Document 1 is a technique for generating a new low-dimensional feature amount using a correlation between two types of content that co-occur simultaneously. Unlike the technique disclosed in Non-Patent Document 1, it is a feature that it is not necessary to directly attach a semantic label to an image.

  In the technique of Patent Document 1, the feature amount is generated by learning based on a statistical amount (correlation) between the feature amount of the signal content and the feature amount of the document. However, the simple correlation between the physical feature amount of the signal content and the semantic feature amount of the document is often not significant, and as a result, the feature amount can be used to discover the semantically related signal content. It was often difficult to obtain. In particular, this technique requires a large number of signal content and document pairs that co-occur at the same time, and it is difficult to obtain sufficient accuracy if a sufficient number of pairs cannot be collected. It was.

  The present invention has been made in view of the circumstances as described above, and is capable of generating low-dimensional feature quantities of signal content and has a characteristic geometric characteristic of a document feature quantity corresponding to the semantic contents of the document. Capturing characteristics and learning the relevance between signal content and documents based on geometric characteristics ensures accurate signal content that is semantically related, even when there are few pairs of signal content and documents It is an object of the present invention to provide a feature quantity generation method, a feature quantity generation device, and a feature quantity generation program that can be found well.

  In order to achieve the above object, according to the feature value generation method of the present invention, one or more desired types of signal contents and documents are respectively given, and the relationship between one or more sets of the signal contents and the documents. A feature value generation method for learning a feature value conversion function for generating a low-dimensional feature value of the signal content when a relation indicator indicating presence / absence of a signal is given, and a co-occurrence of words included in the document Based on the above, the word vector learning step for generating the correspondence between each word and the word vector so that the more likely words to be co-occurred, the word vector learning step, and using the learned word vector, A document feature extracting step for extracting a document feature, an initial feature extracting step for extracting an initial feature of the signal content, a small one of the word vector and the document feature The relative geometric relationship between the signal content and the document is obtained using at least one, and the initial feature amount is reduced to a low-dimensional feature amount based on the initial feature amount, the relative geometric relationship, and a relation indicator. Generating a feature amount conversion function to be converted into a feature amount conversion function and outputting the feature amount conversion function.

  In addition, when the signal content of the desired type is given, the feature amount generation method for generating the low-dimensional feature amount of the signal content, the initial feature extracting the initial feature amount of the signal content There may be included a quantity extraction step and a reduction order step of reducing the initial feature quantity and outputting it based on the feature quantity conversion function generated by the feature quantity generation method.

  In addition, the feature value generation apparatus of the present invention provides one or more desired types of signal content and document, and a relationship instruction that indicates whether or not there is a relationship between one or more sets of the signal content and the document. A feature value generation device that learns a feature value conversion function that generates a low-dimensional feature value of the signal content when a child is given, based on co-occurrence of words included in the document A word vector learning unit that generates a correspondence between each word and the word vector so that the easier the word is to be closer to each other, and the document feature amount of the document is extracted using the learned word vector A document feature extraction unit, an initial feature amount extraction unit that extracts an initial feature amount of the signal content, and at least one of the word vector and the document feature amount, and the signal content. And determining a relative geometric relationship of the document, and generating a feature amount conversion function for converting the initial feature amount into a low-dimensional feature amount based on the initial feature amount, the relative geometric relationship, and a relation indicator. And a feature quantity conversion function generation unit to output.

  In addition, when a desired type of signal content is given, a feature amount generation device that generates a low-dimensional feature amount of the signal content, an initial feature amount extraction unit that extracts the initial feature amount of the signal content; A lower-order unit that lowers the initial feature value based on the feature value conversion function generated by the feature value generation device and outputs the reduced feature value.

  The feature value generation program of the present invention is a program for causing a computer to execute each step of the feature value generation method.

  According to the present invention composed of the above features, by capturing the geometric characteristics of the document features, the semantic content of the document can be captured more accurately, and the relationship between the signal content and the document can be determined using the geometric characteristics. By learning, even when there are few pairs of signal contents and documents, it is possible to generate low-dimensional feature quantities of signal contents that enable more accurate detection of signal contents that are more semantically related. A feature value generation method, a feature value generation device, and a feature value generation program can be provided.

 Furthermore, since the feature amount of the signal content generated by the present invention is very low in comparison with the original initial feature amount, it is possible to find similar content at high speed. In other words, it is possible to respond to usage that requires more real-time performance, and as a specific usage scene that makes use of these effects, places and products that you are interested in walking around the city can be viewed on mobile devices. There is an advantage that it is possible to take a picture and search for similar places and products.

 According to the features of the above two points, according to the present invention, (1) it is possible to generate a feature quantity of signal content that enables (1) high speed but also (2) discovery of semantically similar content.

It is a block diagram which shows the structure of the feature-value production | generation apparatus which concerns on 1st Embodiment. It is a schematic diagram for demonstrating the production | generation method of the feature-value conversion function which concerns on embodiment. It is a flowchart which shows the flow of the feature-value conversion function learning process performed by the feature-value production | generation apparatus which concerns on embodiment. It is a flowchart which shows the flow of the feature-value conversion process performed by the feature-value production | generation apparatus which concerns on embodiment. It is a block diagram which shows the structure of the feature-value production | generation apparatus which concerns on 2nd Embodiment.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

<< Overall structure >>

  FIG. 1 is a functional block diagram illustrating an example of a configuration of a feature quantity generation device 1 according to an embodiment of the present invention.

  The feature value generation apparatus 1 shown in FIG. 1 includes a word vector learning unit 11, a document feature value extraction unit 12, an initial feature value extraction unit 13, a feature value conversion function generation unit 14, and a reduction dimension unit 15. The feature quantity generation device 1 includes a storage unit 3 as a storage device.

  Also, the feature quantity generation device 1 is connected to the content database 2 via communication means and communicates information with each other. The feature quantity generation device 1 generates a feature quantity conversion function 31 based on the signal content 21, the document 22, and the relation indicator 23 registered in the content database 2 and stores them in the storage unit 3. Execute. In addition, the feature quantity generation device 1 executes a feature quantity conversion process for generating a new low-dimensional feature quantity 5 based on the initial feature quantity of the signal content 4 using the learned feature quantity conversion function 31.

  In the figure, solid line arrows indicate data communication and its direction during the feature amount conversion function learning process, and broken line arrows indicate data communication and its direction during the feature amount conversion process.

  The content database 2 may be inside or outside the feature quantity generation device 1. Any known communication means can be used. In the present embodiment, it is assumed that the content database 2 is external to the feature quantity generation device 1, and the feature quantity generation device 1 is connected to the content database 2 via communication means that communicates via the Internet or TCP / IP. And In the present embodiment, the content database 2 includes a so-called RDBMS (Relational Database Management System), but is not limited to this, and is a database using another management system. May be.

  As shown in the figure, the content database 2 stores a signal content 21, a document 22, and a relationship indicator 23. The signal content 21 is constituted by a set of a plurality of content files. For example, when the type of the signal content 21 is an image, the signal content 21 is configured by a set of image files. When the type of the signal content 21 is sound, the signal content 21 is configured as a set in a sound file. When the type of the signal content 21 is video, the signal content 21 is configured by a set of video files. On the other hand, the document 22 is composed of a set of document files.

  In the content database 2, an identifier (for example, an ID by a file-specific serial number) that is uniquely identifiable is associated with the content file of each signal content 21 and the document file of each document 22. Any file can be referred to by specifying an identifier.

  The relationship indicator 23 indicates the “relation” between the content file of each signal content 21 and the document file of each document 22, and the content of the signal content 21 determined to have “relation”. A set of files and a document file of the document 22 is described as a set of identifiers. The “relationship” here is preferably the semantic content relationship of the signal content 21 or the document 22. Any method can be adopted as a method for generating a set of identifiers. For example, the identifier set may be generated manually, the identifier set may be generated mechanically, or both may be generated.

  For example, consider the case where the signal content 21 is an image. For example, when generating a set of identifiers manually, the contents of the third image file having the identifier “image 3” and the eighth document file having the identifier “document 8” are viewed manually. When it is determined that the contents are related to each other, based on the user's instruction, a set of the identifier of the third image file and the identifier of the eighth document file {“image 3”, “ Information indicating the document 8 ″} is stored in the content database 2 as the relationship indicator 23.

  In addition, for example, when a set of identifiers is mechanically generated, a case where an image file is collected from a web page is given as an example. Most simply, the image file A and the document file B in the same web page are considered to be related, and are a set of an identifier of the image file A and an identifier of the document file B {“image A”, Information indicating “document B”} is stored in the content database 2 as the relationship indicator 23. Alternatively, it is assumed that the image file and the document file in the vicinity as the layout on the web page are related to each other, and information indicating the pair of the identifier of the image file and the identifier of the document file is the relation indicator 23. May be stored in the content database 2. When generating a set of identifiers mechanically, there is an advantage that the relation indicator 23 can be obtained without manpower.

  The relation indicator 23 does not necessarily need to be assigned to all the signal contents 21 stored in the content database 2, and may be given to a part of the signal contents 21. In particular, the technique of the present embodiment is characterized in that it can capture the relative geometric relationship in the document feature space that is highly correlated with the meaning of the document. The given relationship indicator 23 is, for example, It is sufficient if it is given to about half of the signal contents 21.

  In addition, as metadata, for example, data representing the content of the signal content 21 (title of the signal content 21, summary sentence, keyword, etc.), data related to the format of the signal content 21 (data amount of the signal content 21, signal content 21 Or the like, such as the size of a thumbnail or the like to be represented). In the present embodiment, a case where metadata is not used will be described.

  Here, the above-described geometric relationship will be specifically described. For example, the geometric relationship is expressed as a matrix W. The i-th and j-th elements of the geometric relationship W represent the closeness between the i-th document feature value and the j-th document feature value, and the larger this value, the closer. As an example, as shown in FIG. 2, document feature values 8A and document feature values 8B having mutually large values as elements in the geometric relationship W are considered to (implicitly) constitute a group. It is done. However, the grouping process is not actually performed, and the document feature amounts 8A and the document feature amounts 8B having mutually large values as elements in the geometric relationship W are represented by a certain kind of software here for convenience. Expressed as a group. Since the geometric relationship W represents a semantic relationship, document feature quantities belonging to the same group also have a high semantic relationship.

  Under such a premise, it can be considered that the document feature value 8A of the same group and the initial feature value 9A connected through the relationship indicator 23 also belong to the same group. Further, it can be considered that the document feature value 8B of the same group and the initial feature value 9B connected through the relationship indicator 23 also belong to the same group. Through this, it is considered that the initial feature quantities 9A and the initial feature quantities 9B also have groups that reflect the semantic relevance of the document feature quantities.

  That is, a feature quantity conversion function for reducing the dimension is obtained so that the initial feature quantities 9A and 9B belonging to the same group are arranged in the same dimension, and the initial feature quantities belonging to different groups are arranged in different dimensions. Details of the feature amount conversion function will be described later.

  Each unit and the content database 2 included in the feature quantity generation device 1 are configured by a computer, a server, or the like that includes an arithmetic processing device, a storage device, and the like, and processes executed by each unit of the feature quantity generation device 1 are executed by various programs. . In the present embodiment, various programs are stored in a storage device included in the feature value generation device 1, but the storage destination of the various programs is not limited to this, and is recorded in a recording medium such as a magnetic disk, an optical disk, or a semiconductor memory. Or may be provided through a network. Further, any other components are not necessarily realized by a single computer or server, and may be realized by being distributed by a plurality of computers connected by a network.

  The search device 6 is not an essential component for realizing the present embodiment, but is a device provided for using the feature quantity generation device 1 according to the present embodiment for the purpose of searching for signal content that is semantically related. is there. The search device 6 is connected to the feature value generation device 1 and the content database 2 so as to communicate with each other.

<< Processor >>

  Next, each processing unit of the feature quantity generation device 1 in the present embodiment will be described.

  The word vector learning unit 11 acquires the document 22 from the content database 2, learns and generates a word vector 32 that is a vector characterizing the word for each of one or more words included in the document, The data is output to the document feature amount extraction unit 12 and stored in the storage unit 3.

  The document feature amount extraction unit 12 acquires the word vector 32 and the document 22 acquired from the content database 2 from the word vector learning unit 11, and extracts the document feature amount of each document file of the document 22 based on the word vector 32. To do. In addition, the document feature amount extraction unit 12 outputs the extracted document feature amount to the feature amount conversion function generation unit 14.

  The initial feature quantity extraction unit 13 acquires the signal content 21 stored in the content database 2 in the feature quantity conversion function learning process, analyzes the acquired signal content 21, and extracts the initial feature quantity of the signal content 21 And output to the feature amount conversion function generation unit 14. Further, the initial feature quantity extraction unit 13 acquires the signal content 4 input from the outside via the communication means described above in the feature quantity conversion process, analyzes the acquired signal content 4, and performs initial feature of the signal content 4. The amount is extracted and output to the reduction unit 15.

  The feature amount conversion function generation unit 14 acquires the document feature amount from the document feature amount extraction unit 12, acquires the initial feature amount from the initial feature amount extraction unit 13, acquires the relation indicator 23 from the content database 2, and stores it. The word vector 32 is read from the part 3. The feature quantity conversion function generation unit 14 uses the document feature quantity, the initial feature quantity, the relation indicator 23, and the word vector 32 to convert the initial feature quantity into a new low-dimensional feature quantity 5. 31 is generated by learning and stored in the storage unit 3.

  The reduction unit 15 acquires the initial feature amount from the initial feature amount extraction unit 13, reads the feature amount conversion function 31 from the storage unit 3, and uses the feature amount conversion function 31 to convert the initial feature amount into the low-dimensional feature amount 5. The low-dimensional feature quantity 5 is generated by converting to. Then, the dimension reduction unit 15 stores the generated reduced dimension feature quantity 5 in the content database 2. As a result, the low-dimensional feature value 5 is stored in the content database 2 after being associated with each signal content 21 stored in the content database 2.

  Further, when the feature quantity generation device 1 is connected to the search device 6, the reduction unit 15 outputs the generated low-dimensional feature value 5 to the search device 6. When the signal content 4 is input according to a user instruction, the search device 6 acquires a low-dimensional feature value 5 corresponding to the signal content 4 from the feature value generation device 1. Further, the search device 6 searches the signal content 21 similar to the low-dimensional feature quantity 5 corresponding to the signal content 4 using the content database 2, acquires the search result 7 from the content database 2, and inputs the signal content 4. Output to the original.

<< Process overview >>

  Next, each process of the feature value generation apparatus 1 in the present embodiment will be roughly described. The feature value generation apparatus 1 according to the present embodiment executes a feature value conversion function learning process that learns and generates a feature value conversion function, and a feature value conversion process that converts an initial feature value into a low-dimensional feature value.

  First, the feature amount conversion function learning process will be described. FIG. 3 is a flowchart showing the flow of the feature amount conversion function learning process. The feature quantity conversion function learning process is a process that is performed at least once before the feature quantity conversion process is performed on the signal content 4 to be searched, for example, and the timing when the user's instruction is input Is executed.

  In step S201, the word vector learning unit 11 acquires the document 22 from the content database 2, and based on the co-occurrence of words included in the document 22, the words that are likely to co-occur are closer to each other. The word vector 32 is learned and generated, and the word vector 32 is stored in the storage unit 3.

  In the next step S <b> 202, the document feature amount extraction unit 12 acquires the document 22 from the content database 2 and acquires the word vector 32 from the storage unit 3. In addition, the document feature amount extraction unit 12 then performs feature extraction processing on each document file included in the document 22 or a part of the document file based on the acquired word vector 32 to obtain the document feature. The amount is extracted and output to the feature amount conversion function generation unit 14.

  In the next step S203, the initial feature amount extraction unit 13 acquires the signal content 21 from the content database 2, and performs feature extraction for each of the content files included in the acquired signal content 21 or a part of the content file. Processing is performed to extract an initial feature value and output to the feature value conversion function generation unit 14.

  In the next step S204, the signal content conversion function generation unit 14 uses at least one of the word vector acquired from the storage unit 3 and the document feature amount acquired from the document feature amount extraction unit 12 to generate signal content. 21, the relative geometric relationship between the document 22 and the document 22 is obtained, the feature amount based on the initial feature amount obtained from the initial feature amount extraction unit 13, the obtained relative geometric relationship, and the relation indicator 23 obtained from the content database 2. A conversion function 31 is generated, and the feature amount conversion function 31 is stored in the storage unit 3.

  Through the processing as described above, the feature quantity generation device 1 generates the feature quantity conversion function 31 from the signal content 21, the document 22, and the relation indicator 23 stored in the content database 2. Details of various processes executed in each step will be described later.

  Next, the feature amount conversion process will be described. FIG. 4 is a flowchart showing the flow of the feature amount conversion process. The feature amount conversion process is a process for reducing the initial feature amount of the signal content 4 using the feature amount conversion function 31 stored in the storage unit 3. The feature amount conversion processing is executed at the timing when the user's instruction is input after the signal content 4 is designated by the user.

  In step S301, the initial feature amount extraction unit 13 acquires the signal content 4 designated by the user via the communication unit described above, extracts the initial feature amount of the acquired signal content 4, and obtains the initial feature amount. Output to the reduction unit 15. In the present embodiment, the signal content 4 indicated by the user is acquired. However, the acquisition method of the signal content 4 is not limited to this, and when the signal content 4 is stored in the storage unit 3, the storage unit 3. You may get from.

  In the next step S302, the reduction unit 15 reduces the initial feature amount acquired from the initial feature amount extraction unit 13 based on the feature amount conversion function 31 acquired from the storage unit 3 to reduce the low-dimensional feature amount 5. Convert to and output.

  Through the processing as described above, the feature quantity generation device 1 obtains the low-dimensional feature quantity 5 of the signal content 4 designated by the user.

<< Details of each process >>

  Hereinafter, an example of the present embodiment will be described in detail for each of the processes described above.

[Learning word vectors]

  The word vector 32 is a finite-dimensional vector that is uniquely determined for a word. Assuming the dimension Dy of the word vector 32, an arbitrary integer value can be designated for the dimension Dy. For example, Dy = 100, Dy = 1000, etc. are preferable. Here, it is assumed that the conversion for converting the word v included in the vocabulary Voc into the word vector 32 is represented by vec (v). In this case, the conversion vec (v) is configured by, for example, a hash or lookup table using the word v as a key and the word vector 32 as its value.

  Various known methods can be applied as a method for generating the word vector 32 of an arbitrary word v included in the vocabulary Voc. In this embodiment, a method using singular value decomposition (SVD) is used. Apply. In this method, the word vector 32 is generated by the following procedure.

  (1) List words that appear in the document without duplication. At this time, a word that can be regarded as substantially the same word by absorbing notation shaking or performing a stemming process that is a process of matching with the stem of a word whose word shape changes is preliminarily made into one word You can put it together. At this time, words having a very large number of appearances, words having a very small number of appearances, and the like may be removed. In the case of removing a word based on the appearance frequency, for example, a word frequency inverse document frequency (Term-frequency, Inverse Document Frequency: TF-IDF) score is obtained for each word, and a word with a low score is removed. good. A set of words finally enumerated after performing these processes is defined as a vocabulary Voc.

  (2) For all words in the vocabulary Voc, the appearance frequency in each document is counted, and a matrix of the number of documents × word number size having the appearance frequency as each element is generated. Note that the value of each element is not limited to the appearance frequency, and may be, for example, a TF-IDF score value.

  (3) SVD is applied to the generated matrix. The generated matrix can be decomposed into “number of documents × Dy matrix”, “Dy × Dy matrix”, and “Dy × word number matrix” by applying SVD. Of these matrices obtained by decomposition, each column of the last Dy × word number matrix represents a word vector 32 of all the words included in the vocabulary Voc.

  As described above, in the present embodiment, the method using SVD is applied as the method for generating the word vector 32. However, since there are many methods other than the method using the SVD as the method for generating the word vector 32, the method using the SVD. Other methods may be applied. In that case, it is preferable to apply a method using Skip-gram (SG), Continuous Bag-of-Words (CBOW) described in Reference Document 1 below.

  [Reference 1] T. Mikolov, I. Sutskever, K. Chen, GS Corrado, and J. Dean, "Distributed Representations of Words and Phrases and Their Compositionality," In Proc., Advances in Neural Information Processing Systems (NIPS) , 2013.

  In the method using SG, CBOW or the like, the word vector 32 is learned based on the context of words appearing in the document. In the method using SG, when each word in a document is given as input information, a three-layer neural network type function that predicts words appearing before and after the given word is learned. When the number of nodes in the second layer of this neural network type function model is Dy, the network connection weight between the second layer and the first layer is finally expressed as a matrix of Dy × number of words. Is a word vector 32.

  In the method using CBOW, unlike the method using SG, a three-layer neural network type function that predicts a word to be predicted is given as input information with words appearing before and after the word to be predicted. learn. Similarly to the method using SG, the network connection weight between the second layer and the first layer is set as the word vector 32.

  According to the above-mentioned reference document 1, it is shown that the word vector 32 generated in this way has a significantly higher meaning and content capturing capability compared to the word vector 32 such as SVD. For example, vec ( It has been shown that it has a characteristic geometric characteristic of “Berlin”) − vec (“Germany”) + vec (“France”) ≈vec (“Paris”). Therefore, the object of the present embodiment is that there is a characteristic relationship in the correspondence between the meaning relationship of the words and the geometric relationship between the word vectors by the method of generating the word vector 32 described in the above-mentioned reference 1. Can be obtained.

  Through the above procedure, the feature quantity generation device 1 learns and generates the word vector 32 from the input document 22.

[Extract document features]

  Next, an extraction method for extracting the document feature amount of the document from an arbitrary document based on the generated word vector 32 will be described in detail.

  This procedure is extremely simple, and all the words included in the vocabulary Voc among the words appearing in the document are listed, and the statistic of the word vector 32 corresponding to each listed word may be used as the document feature amount. . A document that contains one word even if it is a document that contains a plurality of words because the statistic here should be valid without depending on the number of words contained in the document In other words, the statistic is preferably a statistic that can be expressed as data in a word vector space having the same dimensions as the original word vector 32 itself. For example, any statistic such as a sum for each dimension, a maximum value for each dimension, a median value for each dimension, a mode value for each dimension, and the like can be used for each dimension. However, most simply, it is preferable to obtain an average vector of all the word vectors 32 and use the obtained average vector as the document feature amount.

  Through the procedure described above, the feature quantity generation device 1 obtains the document feature quantity of the document 22 stored in the content database 2.

[Initial feature extraction]

  Next, an initial feature quantity extraction method for extracting the initial feature quantity of the signal content 21 will be described.

  In the initial feature quantity extraction process, the initial feature quantity that can be extracted differs depending on the type (image, sound, video, etc.) of the signal content 21. However, what kind of initial feature value is extracted from each type of signal content 21 is not important as a requirement of this embodiment, and a known initial feature value may be extracted using a known feature extraction process. Absent.

  Specifically, the initial feature amount of the signal content 21 is valid if it is numeric data (scalar or vector) having dimensions extracted from the signal content 21. Here, an example of an initial feature extraction process for various signal contents 21 suitable for an example of the present embodiment will be described.

  When the type of the signal content 21 is an image, for example, feature amounts such as brightness features, color features, texture features, concept features, and landscape features are extracted from the signal content 21 as initial feature amounts.

  When extracting the brightness feature, the V value in the HSV color space is counted and extracted as a histogram. In this case, each image included in the signal content 21 is represented as a vector having the same number of dimensions as the number of V-value quantizations (for example, 256 gradations for 16-bit quantization).

  When extracting color features, the value of each axis (L *, a *, b *) in the L * a * b * color space is counted up and extracted as a histogram. At this time, the number of histogram bins for each axis may be set to 4 for L *, 14 for a *, 14 for b *, and the like. In this case, each image included in the signal content 21 is expressed as a total of three axes of bins as 4 × 14 × 14 = 784, that is, a 784-dimensional vector.

  When extracting a texture feature, a statistic (contrast) of a density histogram, a power spectrum, and the like are extracted. Or you may extract a local feature-value. When extracting a local feature amount, it is preferable because it can be extracted as a histogram as in the case of color, movement and the like. As the local feature amount, for example, SIFT (Scale Invariant Feature Transform) described in Reference Document 2 below, SURF (Speeded Up Robust Features) described in Reference Document 3 below, and the like can be used.

[Reference 2] D.G. Lowe, "Distinctive Image Features from Scale-Invariant Keypoints", International Journal of Computer Vision, pp.91-110, 2004.

[Reference 3] H. Bay, T. Tuytelaars, and L.V. Gool, "SURF: Speeded Up Robust Features", Lecture Notes in Computer Science, vol. 3951, pp.404-417, 2006.

  The local feature amount extracted in this way is expressed as a 128-dimensional real value vector, for example. This vector is converted into a code with reference to a code length generated by learning in advance, and a histogram is generated by counting the number of codes. In this case, the number of bins in the histogram matches the code number of the code length. Alternatively, the sparse expression described in Reference Document 4, the feature expression based on the Fisher kernel described in Reference Documents 5 and 6, and the like may be used as the local feature amount.

  [Reference 4] Jinjun Wang, Jianchao Yang, Kai Yu, Fengjun Lv, Thomas Huang, and Yihong Gong, "Locality-constrained Linear Coding for Image Classification", IEEE Conference on Computer Vision and Pattern Recognition, pp. 3360-3367, 2010.

  [Reference 5] Florent Perronnin, Jorge Sanchez, Thomas Mensink, "Improving the Fisher Kernel for Large-Scale Image Classification", European Conference on Computer Vision, pp. 143-156, 2010.

  [Reference 6] Herve Jegou, Florent Perronnin, Matthijs Douze, Jorge Sanchez, Patrick Perez, Cordelia Schmid, "Aggregating Local Image Descriptors into Compact Codes", IEEE Trans. Pattern Recognition and Machine Intelligence, Vol. 34, No. 9, pp. 1704-1716, 2012.

  Regardless of which local feature is used, the resulting initial feature is a real-valued vector having a length that depends on the number of codes of the code length.

  When extracting concept features, feature amounts such as an object included in the image and an event captured by the image are extracted as initial feature amounts. Any object, event, or the like may be used as the object to be extracted, such as “sea”, “mountain”, “ball”, and the like. If “sea” is reflected in the image, it is determined that the image belongs to the concept of “sea”. Whether each image belongs to each concept is identified using a concept classifier. Normally, one concept discriminator is prepared for each concept, and when an image feature amount is input, whether or not the image belongs to a concept to be identified is output as an attribution level. The concept discriminator is acquired by learning the relationship between the feature amount of the image (for example, the above-described local feature amount) and the correct answer label indicating in advance which concept the image belongs to. Is done. For example, a support vector machine may be used as the learning device. When extracting concept features, the level of belonging to each concept is collectively expressed as a vector. In this case, the generated initial feature amount is a vector having the same number of dimensions as the number of concepts.

  A landscape feature is a feature amount that represents a landscape or scene of an image. When extracting a landscape feature, for example, the GIST descriptor described in Reference Document 7 below can be used. The GIST descriptor is expressed by a coefficient when an image is divided into a plurality of areas and a filter having a certain orientation is applied to each of the divided areas. However, in this case, the generated initial feature amount is a vector having a length depending on the type of filter (the number of regions to be divided and the number of orientations).

[Reference 7] A. Oliva and A. Torralba, "Building the gist of a scene: the role of global image features in recognition", Progress in Brain Research, 155, pp. 23-36, 2006.

  Further, the feature amount by CNN described in Non-Patent Document 1 may be extracted as the initial feature amount.

  Next, when the type of the signal content 21 is a sound, for example, an initial feature amount such as a pitch feature, a sound pressure feature, a spectrum feature, a rhythm feature, an utterance feature, a music feature, a sound event feature, etc. is extracted from the signal content 21. Extract as

  When extracting pitch features, for example, feature values of pitch (pitch) may be extracted from the sound file included in the signal content 21. As an extraction method, for example, a method described in the following reference website can be applied. In this case, the pitch may be expressed as a one-dimensional vector (scalar), or the pitch may be quantized into a plurality of dimensions and expressed as a vector having a plurality of dimensions.

[Reference website] http://en.wikipedia.org/wiki/Pitch_detection_algorithm

  When extracting the sound pressure feature, the feature value of the amplitude value of the speech waveform data may be extracted from the sound file included in the signal content 21 as the initial feature value. Alternatively, a short-time power spectrum of voice waveform data may be extracted from a sound file included in the signal content 21, and an average power in an arbitrary band may be calculated to obtain a feature value, which may be used as an initial feature value. Whether the amplitude value of the speech waveform data is extracted or the short-time power spectrum is extracted, the generated initial feature amount is a vector whose length depends on the number of bands for calculating the sound pressure.

  When extracting spectral features, for example, a feature value of Mel-Frequency Cepstral Coefficients (MFCC) may be extracted from the sound file included in the signal content 21 as an initial feature value.

  When extracting rhythm features, for example, a tempo feature value may be extracted from the sound file included in the signal content 21 as an initial feature value. When extracting the tempo, for example, the method described in Reference Document 8 below can be applied.

[Reference 8] E.D. Scheirer, "Tempo and Beat Analysis of Acoustic Musical Signals", Journal of Acoustic Society America, Vol. 103, Issue 1, pp.588-601, 1998.

  The utterance feature and the music feature represent the presence or absence of utterance and the presence or absence of music, respectively. When extracting an utterance feature or a music feature, a section in which an utterance or music is present may be extracted as a feature amount from a sound file included in the signal content 21. In order to identify a section where speech or music exists, for example, the method described in Reference Document 9 below can be applied.

[Reference 9] K. Minami, A. Akutsu, H. Hamada, and Y. Tonomura, "Video Handling with Music and Speech Detection", IEEE Multimedia, vol. 5, no. 3, pp. 17-25, 1998 .

  When extracting sound event features, for example, emotional sounds such as laughter and loud voice, or environmental sounds such as gunshots and explosion sounds are detected as sound events. May be extracted as the initial feature amount. When detecting such a sound event, for example, a method described in Reference Document 10 below can be applied.

[Reference 10] International Publication No. 2008/032787

  When the type of the signal content 21 is a video, since the video is generally an image and sound stream, the initial feature amount can be extracted using the above-described image feature and sound feature. Regarding which image section in the video to analyze or which sound section to analyze, for example, the video is divided into a plurality of sections in advance, and one image is extracted for each section. Extract the amount. Further, the video is divided into a plurality of sections in advance, and the sound feature amount is extracted for each section. In this way, the initial feature extraction process is performed.

  In addition, when dividing | segmenting an image | video into a several area, you may divide | segment video at a predetermined fixed interval, for example, the division | segmentation method etc. of the following reference literature 11 etc. are applied, and an image | video breaks discontinuously. You may divide | segment by the cut point which is a point. Preferably, the latter division method is applied. As a result of dividing the video into a plurality of sections, a start point (start time) and an end point (end time) of each section are obtained, and may be handled as separate initial feature values for each time.

[Reference 11] Y. Tonomura, A. Akutsu, Y. Taniguchi, and G. Suzuki, "Structured Video Computing", IEEE Multimedia, pp.34-43, 1994.

  The initial feature quantity extracted as described above may be any one of the extracted feature quantities, or may be a feature quantity calculated from a plurality of feature quantities. The initial feature value is not limited to the feature value extracted by the above-described method, and a feature value acquired by another known extraction method may be used as the initial feature value.

[Generate feature conversion function]

  Next, a method for generating a feature quantity conversion function will be described in detail.

  An initial feature amount extracted from the signal content i of the plurality of signal contents 21 is represented as xiεRD. The initial feature dimension of the signal content is Dx.

  At this time, a feature amount conversion function f: RDx → Rd for reducing the signal content to d dimension (d ≦ Dx) is obtained. In the present embodiment, by learning and utilizing the relationship between the signal content 21 and the document 22, the semantic content of the document 22 is transferred to the initial feature amount of the signal content 21, and the semantic similarity is more reflected. The generated low-dimensional feature value is generated. To achieve this, the following two policies are adopted.

  (1) Saving of the relationship between the initial feature quantity and the document feature quantity: In order to transfer the semantic content captured from the document feature quantity to the initial feature quantity, the signal content 21 (initial feature quantity) and the document 22 (document feature quantity) are stored. ) Is generated so as not to break the relationship. In this embodiment, since the relationship between the signal content 21 and the document 22 is given as the relationship indicator 23, the feature amount conversion function f is learned so as to store the relationship indicated by the relationship indicator 23.

  (2) Preservation of geometric relationship of document feature amount space: As described above, the word vector 32 includes, for example, “vec (“ Berlin ”) − vec (“ Germany ”) + vec (“ France ”) in SG and CBOW. It is known that there is a clear relationship between the meaning of a word, such as ≈vec (“Paris”), and the geometric relationship of the word vector 32. However, if the feature quantity conversion function f is learned so as to preserve the geometric relationship in the word vector space, the feature quantity conversion function f that effectively preserves the semantic relationship of the word vector can be obtained. . Unlike the case of using a simple correlation as in the method of reducing the dimension described in Patent Document 1, it is effective even when the amount of the document 22 is small by capturing the geometric relationship related to the semantic content. This is advantageous in that low-dimensional feature values can be configured.

  Thereafter, without losing generality, the number of sets of the signal content 21 and the document 22 to which the relation indicator 23 is given is N, and the initial feature amount xi (i = 1, 2,... .., N) and the document feature quantity yi (i = 1, 2,..., N), a pair having the same index (for example, x1 and y1, x2 and y2, etc.) is instructed by the relation indicator 23. Suppose that it is a pair. These are normalized to an average of 0. That is, the average vector of the initial feature quantity xi is a zero vector.

  Next, a specific procedure for generating the feature quantity conversion function f will be described.

  First, a process for capturing a geometric relationship in the word vector space will be described. In order to realize this, it is necessary to examine how each document feature amount yi has a geometric relationship with other document feature amounts yj ≠ i. In general, this geometric relationship outputs the geometric relationship W between the document feature value yi and the document feature value yj with N document feature value vectors yi (i = 1, 2,..., N) as input information. Is obtained by solving the following optimization problem.

  Here, the geometrical relationship W = {wij} is an N × N matrix, and the i-th row geometrical relationship wi represents a geometrical relationship between the document feature amount yi and another document feature amount yj. More specifically, the geometric relationship W described here represents the document feature amount yi as a linear sum of other document feature amounts yj (j = 1, 2,..., N, i ≠ j). Represents the linear combination weight. This calculation method is a so-called least square method, which can be solved using a known gradient method or matrix decomposition. The resulting geometric relationship W is larger as the element corresponding to the relationship between the documents 22 having more semantically related contents (that is, the element wij corresponding to the relationship between the document feature amount yi and the document feature amount yj). Tend to have a value.

  As an easy-to-understand example, it is assumed that the document feature amount yi is the document feature amount of the document 22 describing the apple. Further, it is assumed that there is a document feature amount yj of the document 22 describing the green apple and a document feature amount yk of the document 22 describing the car. At this time, since the document feature amount yj is more similar to the document feature amount yi than the document feature amount yk, the geometric relationship wk corresponding to the document feature amount yk in the geometric relationship W has almost the same value. The geometric relationship wk corresponding to the document feature amount yj is 0, and most of the values are close to 1.

  Alternatively, each document feature amount yi has an effective geometric relationship W with only a very small number of other document feature amounts, that is, the document 22 semantically related to the document 22 having the document feature amount yi is N There are many cases where the number of documents is very limited. In this case, the geometric relationship W between the document feature value yi and the document feature value yj is obtained by using N document feature value vectors yi (i = 1, 2,..., N) as input information. It is obtained by solving the optimization problem of the following formulas (1) and (2) using the geometric relationship W between the quantity yi and the document feature quantity yj as output information.

  This optimization problem is a so-called sparse regression problem or sparse coding problem in which only a limited number of elements in the vector wi are regularized to have non-zero values. It can be solved by using some known numerical calculation algorithms such as (Least Angle Regression: LARS), Alternating Direction Method of Multipliers (ADMM).

  On the other hand, in this embodiment, it is not always necessary that the relationship indicator 23 be given to all the documents 22 registered in the content database 2. In particular, when the number of documents 22 to which the relationship indicator 23 is given is small, a sufficient amount of data (document feature amount) sufficient to capture the geometric relationship in the word vector space cannot be obtained. There are cases where acquisition fails, that is, the elements of the geometric relationship W between the documents 22 that are semantically related (not) should or should not be larger. Therefore, in the present embodiment, it is considered that the document feature amount of the document 22 to which the relationship indicator 23 is not given is also used to obtain a sufficient amount of data for capturing the geometric relationship.

  Assume that there are M documents 22 to which no relationship indicator 23 is given. This is expressed as a document feature ui (i = 1, 2,..., (N + M)) including the document 22 to which the relationship indicator 23 is not given (the first N items are given the relationship indicator 23). Document feature amount yi (i = 1,..., N)) of the document 22. In this case, similar to the above-described procedure, these geometrical relationships W may be obtained using N + M document feature quantities ui. Specifically, the problems corresponding to the equations (1) and (2) are the following equations (3) and (4), respectively.

  The geometric relationship W may be obtained by solving the above equations (3) and (4) by the same procedure as the procedure using the above equations (1) and (2). Note that the document feature amount of the document 22 that does not have the relationship indicator 23 is not necessarily the document feature amount extracted from the document 22. Specifically, the word vector 32 itself can be used as the document 22 that does not have the direct relationship indicator 23. The word vector 32 itself is a Dy-dimensional vector in the word vector space, and is special data that can be said to be a “base” that defines this space. Therefore, the word vector 32 is information that is particularly useful for capturing the geometric relationship W in the word vector space. Is effective.

  Through the procedure as described above, the feature quantity generation device 1 captures the geometric relationship W in the word vector space.

  Subsequently, an initial setting is made so that the geometric relationship W between the document feature quantities in the word vector space is stored as much as possible, and the relationship between the signal content 21 and the document 22 represented by the relationship indicator 23 is stored as much as possible. A feature quantity conversion function f for converting the feature quantity into a low-dimensional feature quantity having a dimension lower than that of the initial feature quantity is generated. Specifically, the feature quantity conversion function f is generated according to the following procedure.

  In the present embodiment, a feature quantity conversion function f that takes the following form is considered.

  In the above equation (5), αk, t is a parameter, and κ (xt, x) is a kernel function. The kernel function is a function shown in the following formula (6), and satisfies the following formula (7) for N initial feature quantities {x1,..., XN}, and is set to arbitrary real numbers αi and αj. On the other hand, it is a function that satisfies the following equation (8).

  There are an infinite number of such functions. For example, the following formulas (9), (10), and (11) exist. However, β and γ are positive real value parameters, and p is an integer parameter, which may be determined as appropriate.

  When the kernel function of the above equation (11) is used and p = 1 and γ = 0, the obtained feature quantity conversion function f is reduced to a simple linear function, and processing for obtaining a low-dimensional feature quantity is performed. There is an advantage that the processing amount for obtaining the feature amount conversion function f is greatly reduced. On the other hand, in other cases, the feature quantity conversion function f becomes a non-linear function and the processing amount increases, but the information amount of the document 22 that can be transferred to the finally obtained low-dimensional feature amount increases, and as a result There is an advantage that the accuracy obtained is improved.

  In the above equation (5), since bk is a constant determined by calculating the following equation (12), that is, calculating the average value, the above equation (5) is converted into an inner product form such as the following equation (13). be able to. However, ak and κ (x) are expressed by the following equation (14).

  Here, T is a constant that determines the feature amount conversion function f, specifically, the kernel vector mapping κ (x). The constant T may be set to an arbitrary value in the range of T ≦ N. For example, assuming that T = 300, T initial feature quantities may be randomly selected from all the initial feature quantities {x1,..., XN} and used to generate the feature quantity conversion function f. A representative vector determined using a clustering method such as -means may be used to generate the feature amount conversion function f.

  However, in order to obtain the feature amount conversion function f, the parameter {αk} may be determined. Therefore, a method for determining the parameter {αk} will be described below.

  For convenience, document feature values {u1,..., U (N + M)} including document feature values having no relation designator 23 (where u1 = y1,..., UN = yN) are defined. . However, the document feature amount not having the relation indicator 23 is not essential, and in this case, M = 0 may be set. Further, a kernel vector map ρ (ui) corresponding to κ (xi) is obtained by the same procedure as the initial feature value xi.

  A matrix K = [κ (x1, 1), in which κ (xi) (i = 1, 2,..., N) and κ (ui) (i = 1, 2,..., N + M) are arranged. .., Κ1 (x1, N)], Ρ = [κ (u1),..., Κ (uN + M)] are defined. Further, in addition to the feature amount conversion function f, a conversion function g for reducing the document feature amount having the completely same format is prepared, and a parameter corresponding to αk in the above equation (13) is set to θk (k = 1, 2, ..., d). Note that g and θk are used for convenience for mathematical reasons, and are not used as the feature amount conversion function f in this embodiment. A matrix in which αk is arranged is defined as Α = [α1,..., αd], and a matrix in which θk (k = 1, 2,..., d) is arranged is Θ = [θ1,. ] And solve the following equation.

  Here, the first term of the above equation (15) is supported by the relation indicator 23 in the low-dimensional feature value obtained when the initial feature value and the document feature value are reduced by A and Θ. This is a term that requests to maximize the linear or non-linear correlation between pairs. The second term is a term that requests that the geometric relationship W on the word vector 32 captured by the geometric relationship W be preserved even after the reduction in dimension. That is, the feature quantity conversion function f obtained by solving the above equation (15) is the relationship between the low-dimensional feature quantity of the signal content 21 and the reduced document feature quantity, and the meaning on the word vector 32. Is a term that preserves both of the geometric relationship W reflecting the maximum.

  When a simple algebraic modification is applied to the above equation (15), the following equation (16) is obtained.

Here, since the following equation (17) is convex with respect to G, by differentiating G and taking its extreme value, it can be reduced to a generalized eigenvalue problem as shown in the following equation (18).

  A solution of such a generalized eigenvalue problem can be obtained by a known numerical calculation algorithm such as an iterative method. Since the element of G includes the parameter A to be obtained, the feature quantity conversion function f can be obtained using the element of G.

  As described above, the feature quantity generation device 1 obtains the feature quantity conversion function f that optimally satisfies the target property.

[Lower dimension]

  After the feature amount conversion function f is obtained, the initial feature amount for any signal content 21 can be reduced. Specifically, after shifting so that the average becomes 0 with respect to the initial feature value x, a new low-dimensional feature value is calculated by the following equation (19).

<< Application to discovery of semantically related content >>

  An example will be described in which the feature value generation apparatus 1 according to the present embodiment described above is used for the purpose of searching for signal content 21 that is semantically related. Here, a case where the signal content 21 is an image will be described as an example.

  For example, it is assumed that N database images are stored in the content database 2. It is assumed that the feature quantity conversion function f is obtained through the above-described feature quantity conversion function learning process and stored in the storage unit 3, and further, the low-dimensional feature quantity Z = {z 1, corresponding to the N database images. ..., zN} is obtained. At this time, when a new query image is given by the user, the purpose is to find a database image that is semantically related to the query image.

  First, an initial feature quantity extraction process is performed on the query image to extract an initial feature quantity xq. Thereafter, the feature quantity xq is reduced in dimension based on the above equation (19) to obtain a low-dimensional feature quantity zq.

  The distance between each of the low-dimensional feature quantity zq and the low-dimensional feature quantity Z is calculated, and a database image with a small distance is output as a database image that is semantically related. Since both the low-dimensional feature value zq and the low-dimensional feature value Z are low-dimensional, the time required for the distance calculation is shorter than when using the initial feature value before the reduction in dimension, and the calculation is performed at high speed. can do. In addition, unlike the initial feature quantity that is a physical feature quantity, the low-dimensional feature quantity obtained by the present embodiment is obtained by using the feature quantity conversion function f learned so as to be highly related to the semantic content of the document 22. Therefore, it is possible to find a database image having high semantic relevance with high accuracy.

  The above is an example in the case of using for the purpose of searching for the signal content 21 that is semantically related in the present embodiment.

  As described above, according to the present embodiment, by capturing the geometric characteristics of the document feature amount, the semantic content of the document 22 is captured more accurately, and using this, the relationship between the signal content 21 and the document 22 is determined. By learning the nature, it is possible to discover the signal content 21 more semantically related even when there are few pairs of the signal content 21 and the document 22.

  Furthermore, since the low-dimensional feature amount of the signal content 21 generated in the present embodiment is very low compared to the original initial feature amount, it is possible to find similar content at high speed. (1) While being fast, (2) it is possible to generate a feature amount of the signal content 21 that enables the discovery of semantically similar content.

  In addition, it cannot be overemphasized that it can take arbitrary uses and a structure within the range with which the main characteristics in this embodiment are satisfy | filled. For example, the feature quantity conversion function generation process and the feature quantity conversion process can be separated, and the devices that make up each can be separated, so the feature quantity conversion function generation process executed in batch processing is executed on the server device by online processing It is also possible to adopt a configuration in which the feature amount conversion processing to be performed is incorporated into a client device such as a smartphone. An example of the apparatus configuration in this case is shown in FIG.

  As illustrated in FIG. 5, the server device 100 includes a word vector learning unit 101, a document feature extraction unit 102, an initial feature amount extraction unit 103, a feature amount conversion function generation unit 104, a reduction dimension unit 105, and a storage unit 300. Prepare. The storage unit 300 stores a feature amount conversion function 301 and a word vector 302. The content database 200 stores signal content 201, a document 202, a relation indicator 203, and a low-dimensional feature amount 204. The server device 100 is connected to the content database 200.

  On the other hand, as illustrated in FIG. 5, the client device 400 includes an initial feature amount extraction unit 401, a low-dimensionalization unit 402, and a storage unit 500. The storage unit 500 stores a feature amount conversion function 501.

  Here, in the server apparatus 100 and the client apparatus 400, common components (initial feature quantity extraction unit, storage unit) are configured to have the same functions, and are the same as the respective components described in FIG. The name may have the same function as in FIG. Furthermore, the feature amount conversion function 301 stored in the storage unit 300 of the server device 100 and the feature amount conversion function 501 stored in the storage unit 500 of the client device 400 are each appropriately synchronized by some communication means. Shall.

  Further, a search device 800 is provided. The search device 800 may be incorporated in the server device 100 or may be connected to the server device 100 from the outside.

  The outline of processing in this apparatus configuration is as follows.

  First, the server apparatus 100 performs the above-described feature quantity conversion function learning process, appropriately generates a feature quantity conversion function, and synchronizes with the feature quantity conversion function of the client apparatus 400. Further, the same processing as described above is performed on the content in the content database 200 to generate a low-dimensional feature amount 204 and store it in the content database 200.

  On the other hand, when the client apparatus 400 receives a search request from the user, that is, an input of the new signal content 600, the client apparatus 400 generates a low-dimensional feature quantity 700 for the signal content 600 and outputs it to the search apparatus 800.

  When the search device 800 receives the low-dimensional feature value 700 from the client device 400, the search device 800 searches the content database 200 using the low-dimensional feature value and semantically related signal content based on the low-dimensional feature value 700. And the found signal content is output as a search result 900 to the user.

  By configuring the server device 100 and the client device 400 in this way, the server device 100 can perform the feature amount conversion function learning processing, and the client device 400 can be configured to perform only the feature amount conversion processing.

  Here, the merit of configuring the server apparatus 100 and the client apparatus 400 in this way will be described. In general, a client device (PC, smartphone, etc.) has poor calculation capability as compared to a server device, and therefore may not be suitable for processing with a relatively large amount of calculation such as feature amount conversion function generation. However, with this configuration, the feature amount conversion function learning process can be appropriately performed by a server device having high calculation capability, and only the feature amount conversion process having a small amount of operation can be performed by the client device. In addition, when transmitting information with a large amount of data through communication via a network, there is a problem that it takes a long time to transmit. However, due to this configuration, only low-dimensional feature values with a small amount of information are transmitted. Can be processed, and the quick response to the search can be improved.

DESCRIPTION OF SYMBOLS 1 Feature-value production | generation apparatus 2 Content database 3 Memory | storage part 4 Signal content 5 Low-dimensional feature-value 11 Word vector learning part 12 Document feature-value extraction part 13 Initial feature-value extraction part 14 Feature-value conversion function generation part 15 Low-dimensionalization part 21 Signal Content 22 Document 23 Relation indicator 31 Feature amount conversion function 32 Word vector

Claims (5)

If the signal content and document of the desired type are each given one or more, and the relation indicator indicating the presence or absence of the relationship between one or more sets of the signal content and the document is given, the signal A feature value generation method for learning a feature value conversion function for generating a low-dimensional feature value of content,
Based on the co-occurrence of words included in the document, a word vector learning step for generating a correspondence between each word and the word vector so that words that are more likely to co-occur are closer to each other.
A document feature extraction step of extracting a document feature amount of the document using the learned word vector;
An initial feature amount extracting step for extracting an initial feature amount of the signal content;
A relative geometric relationship between the signal content and the document is obtained using at least one of the word vector and the document feature amount, and based on the initial feature amount, the relative geometric relationship, and a relation indicator. Generating a feature quantity conversion function for converting the initial feature quantity into a low-dimensional feature quantity, and outputting a feature quantity conversion function generating step;
A method for generating a feature quantity. A feature value generation method for generating the low-dimensional feature value of the signal content when the desired type of signal content is given,
An initial feature extraction step for extracting the initial feature of the signal content;
Based on the feature quantity conversion function generated by the feature quantity generation method according to claim 1, a reduction order step for reducing and outputting the initial feature quantity, and
A method for generating a feature quantity. If the signal content and document of the desired type are each given one or more, and the relation indicator indicating the presence or absence of the relationship between one or more sets of the signal content and the document is given, the signal A feature value generation device for learning a feature value conversion function for generating a low-dimensional feature value of content,
Based on the co-occurrence of words included in the document, a word vector learning unit that generates a correspondence between each word and the word vector so that words that are more likely to co-occur are closer to each other,
A document feature extraction unit that extracts a document feature amount of the document using the learned word vector;
An initial feature amount extraction unit for extracting an initial feature amount of the signal content;
A relative geometric relationship between the signal content and the document is obtained using at least one of the word vector and the document feature amount, and based on the initial feature amount, the relative geometric relationship, and a relation indicator. Generating a feature value conversion function for converting the initial feature value into a low-dimensional feature value, and outputting a feature value conversion function generating unit;
A feature amount generating apparatus. A feature value generation device that generates a low-dimensional feature value of a signal content when a desired type of signal content is given,
An initial feature amount extraction unit for extracting an initial feature amount of the signal content;
Based on the feature value conversion function generated by the feature value generation device according to claim 3, a reduction unit for reducing and outputting the initial feature value,
A feature amount generating apparatus.   A feature quantity generation program for causing a computer to execute each step of the feature quantity generation method according to claim 1 or 2.

Images