JP2013109479A - Content conversion method, content conversion device, and content conversion program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a content conversion method, a content conversion device, and a content conversion program capable of highly accurately detecting a similar content from a large amount of multimedia contents at a high speed and with a memory saving type.SOLUTION: The content conversion method includes: distributing respective feature amounts xof a plurality of contents i on a feature amount space Ras feature points and classifying them at least two groups, obtaining a straight line having an equal vertical distance from all the feature points of each group, and generating a hash function hfor converting the content i into binary values on the basis of a fact in which two space sides divided by the straight line the feature points are present.

Description

  The present invention relates to a content conversion method, a content conversion device, and a content conversion program used when searching for similar content from a large amount of multimedia content.

  With the advancement of communication networks, storage, and distributed environments, the number of multimedia contents distributed online has become enormous.

  For example, the number of web pages that a search engine can search is said to be billions or trillions. Wikipedia, famous as a collective intelligence encyclopedia, can browse over 3.5 million articles.

  One social media site reports that 2.5 billion images are uploaded every month, and another video sharing site reports that 48 hours of video per minute continues to be released. is there.

  While such a huge amount of multimedia content is a rich choice for viewers, it is difficult to quickly access the content that you want to view or browse, or to quickly grasp the contents by listing them. It also brings serious challenges.

  Therefore, there is an increasing demand for content search / recommendation for efficiently searching for content to be browsed / viewed, and content summaries that are shortened by omitting redundant portions in the content.

  One of the most basic and important functions for realizing such content search, recommendation, and summarization is a function for finding similar content or a part of the content.

  For example, when searching for content, when a certain content is given as a query, it is a basic requirement to search for content similar to this content. Similarly, in the case of recommendation, content similar to the content that the user has browsed or browsed so far is found and recommended. Further, even in the case of summarization, since it is redundant to present similar content, a function for finding and omitting it is necessary.

  Here, a well-known similar content discovery method will be described. It is assumed that the content or a part thereof is expressed by a certain feature amount. At this time, the similarity is calculated by measuring the proximity of the feature quantities, and similar content is found based on the similarity.

  For example, if the content is an image, the similarity can be measured using the color histogram of the image as a feature amount. If the content is a document, the degree of similarity can be measured by using a histogram of the appearance frequency of words (referred to as a Bag-of-Words histogram).

  Needless to say, if the number of contents is 10,000, it is necessary to calculate the similarity for each of the 10,000 contents, and as a result, it is necessary to pick up the contents with high similarity as similar contents.

  However, as described above, when trying to target a large amount of content, there are two important issues: (1) it takes a long time to calculate and (2) consumes a large amount of memory.

  Usually, the feature amount of content is often multidimensional, and it takes time to calculate the similarity. In general, the dimension of a Bag-of-Words histogram of a document is the same dimension as the type of word (vocabulary), and the color histogram of an image is generally a real-valued vector of hundreds to thousands of dimensions.

  Furthermore, since it is necessary to calculate the degree of similarity for all content sets, the amount of calculation of O (N) is required if there are N pieces of content no matter what kind of similarity degree calculation means is used. .

Further, in order to execute an immediate search, it is preferable to store the feature amount or its similarity in a memory, but in order to perform this, an O (N ² ) memory is required.

  In order to deal with such problems, efforts have been made on techniques for expressing contents with low-capacity feature amounts and efficiently finding similar contents. In order to solve this problem, several inventions have been made and disclosed.

  In the technology disclosed in Patent Document 1, the feature amount of the content is dimensionally compressed by principal component analysis to reduce the dimension, and by measuring the distance between the low-dimensional feature amounts, the feature amount is reduced. We are trying to speed up.

  In the technique disclosed in Non-Patent Document 1, a hash function group is generated such that the similarity between the original feature value and the collision probability are equal in any two adjacent contents (feature values).

  A cosine similarity is considered as a typical similarity. In this case, a basic method for generating a hash function is to generate a plurality of random hyperplanes in a feature amount space (called random projection).

  By hashing the feature amount depending on which side of each hyperplane the feature amount exists, similar content can be found approximately without obtaining similarity between all the contents.

  The technique disclosed in Non-Patent Document 2 is a hash function generation technique that considers the similarity by Shift-Invariant Kernel, unlike the cosine similarity considered by Non-Patent Document 1.

  The basic procedure is similar to that of Non-Patent Document 1, and a random map is also generated, and the feature value is hashed based on this. On the other hand, the non-patent document 1 is different from the non-patent document 1 in that the non-patent document 1 “generates a hash function group in which the similarity of the original feature amount and the collision probability are equal”. 2 generates a hash function in which the Hamming distance between hash values is suppressed by bounds (upper and lower bounds) that depend on the degree of similarity using Shift-Invariant Kernel.

  It should be noted that both of the non-patent documents 1 and 2 are assigned a 1-bit binary value per hash function. That is, if the number of hash functions is B, the hash value is Bbit.

Japanese Patent No. 3730179

M. Datar, N. Immorlica, P. Indyk, V.S. Mirrokni, “Locality-Sensitive Hashing Scheme based on p-Stable Distributions”, In Proceedings of the Twentieth Annual Symposium on Computational Geometry, 2004, p.253-262 M. Raginsky, S. Lazebnik, “Locality-Sensitive Binary Codes from Shift-Invariant Kernels”, Advances in Neural Information Processing Systems 22, 2009, p.1509-1517

  Although the technique described in Patent Document 1 expresses feature quantities in a compressed manner, it is necessary to obtain the similarity between the compressed feature quantities by using the Euclidean distance, and thus it has not been possible to achieve a significant reduction in calculation time.

  In the techniques disclosed in Non-Patent Documents 1 and 2, since the generation of the hash function (hyperplane) is random, the number of hashes is sufficient to generate a hash function that reflects the similarity of content. It was necessary to create a large number of hash functions.

  The present invention has been made in view of this problem. A content conversion method, a content conversion device, and a content conversion method capable of finding high-precision and similar content from a large amount of multimedia content while being high-speed and memory-saving. It is an object to provide a content conversion program.

  A content conversion method according to a first aspect of the present invention is a content conversion method for converting content into one or more binary values. The computer reads out the content from the storage means and extracts each feature quantity of the plurality of contents. And extracting each feature quantity as a feature point in the feature quantity space and classifying it into at least two groups, obtaining a straight line or a hyperplane having the same vertical distance from all the feature points of each county, And generating a hash function for converting the content into a binary value based on which space side is divided by the figure.

In the content conversion method according to the second aspect of the present invention, the generating step uses trace {w ^T (S _b −S _w) using the inter-group variance value S _b and the intra-group variance value S _w of the two groups as variables. ) W} (where w ^T w = 1), and the figure is obtained by using the value of w that maximizes the calculated value as the slope.

  In the content conversion method according to the third aspect of the present invention, when the generation step generates a plurality of hash functions, the distribution state of the feature points is changed so that the degree of correlation with other hash functions becomes low. It is characterized by.

  The content conversion method according to a fourth aspect of the present invention further includes a conversion step of converting the content into one or more binary values using one or more hash functions.

  According to a fifth aspect of the present invention, there is provided a content conversion apparatus according to the present invention, wherein the content conversion apparatus converts the content into one or more binary values, reads out the content from the storage unit, and extracts each feature quantity of the plurality of contents The feature amounts are distributed as feature points on the feature amount space and classified into at least two groups, and a straight line or a hyperplane having the same vertical distance from all feature points of each group is obtained, and the feature points are represented by the figure. Generating means for generating a hash function for converting the content into a binary value based on which of the two divided spaces is present.

In the content conversion device according to the sixth aspect of the present invention, the generating means uses trace {w ^T (S _b −S _w) using the inter-group variance value S _b and the intra-group variance value S _w of the two groups as variables. ) W} (where w ^T w = 1), and the figure is obtained by using the value of w that maximizes the calculated value as the slope.

  In the content conversion device according to the seventh aspect of the present invention, when the generation unit generates a plurality of the hash functions, the distribution unit changes the distribution state of the feature points so that the degree of correlation with other hash functions becomes low. It is characterized by.

  A content conversion program according to an eighth aspect of the present invention causes a computer to execute the content conversion method according to the first to fourth aspects.

  As described above, according to the present invention, the feature quantities of a plurality of contents are distributed as feature points in the feature quantity space and classified into at least two groups, and the vertical distances from all the feature points of each group are equal to each other. Since a hash function that converts the content into a binary value based on which space the feature point is divided into by the figure is found, a hash function that is robust against fluctuations can be generated. Thus, it is possible to provide a content conversion method, a content conversion apparatus, and a content conversion program capable of finding similar contents with high accuracy while being high-speed and memory-saving from a large amount of multimedia contents.

  In addition, according to the present invention, when generating a plurality of hash functions, the distribution state of the feature points is changed so that the degree of correlation with other hash functions is lowered, so that hash functions having low correlation with each other can be generated. As a result, it is possible to provide a content conversion method, a content conversion apparatus, and a content conversion program that can perform high-accuracy similar content search with a smaller number of hash functions (number of bits).

  According to the present invention, it is possible to provide a content conversion method, a content conversion apparatus, and a content conversion program capable of finding similar content with high accuracy while being high speed and memory-saving from a large amount of multimedia content.

It is a figure showing an example of functional block composition of an information processor. It is a flowchart which shows the flow of the example of a hash function production | generation process. It is a flowchart which shows the flow of the example of a hashing process. It is a figure explaining the geometrical meaning of a hash function. It is a figure explaining the hash function which divides | segments two similar groups. It is a figure which shows the comparison result of similar search precision.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[Configuration of information processing device]
FIG. 1 is a diagram showing a functional block configuration of the information processing apparatus according to the present embodiment. An information processing apparatus 1 shown in FIG. 1 is a content conversion apparatus that converts content into one or more binary values, and includes an input unit 11, a feature extraction unit 12, a hash function generation unit 13, and a hash function storage unit. 14, a hashing unit 15, and an output unit 16.

  The information processing apparatus 1 is connected to the content database 2 via a communication unit, communicates information with each other via the input unit 11 and the output unit 16, and generates a hash function based on the content registered in the content database 2. A hash function generation process and a hash process for converting the content into one or more binary values using the generated hash function are performed.

  Each unit included in the information processing device 1 may be configured by a computer including an arithmetic processing device, a storage device, and the like, and the processing of each unit may be executed by a program. This program is stored in a storage device included in the information processing apparatus 1, and can be recorded on a recording medium such as a magnetic disk, an optical disk, or a semiconductor memory, or can be provided through a communication network.

  The content database 2 may be inside or outside the information processing apparatus 1. Although any known communication means can be used, in the present embodiment, it is assumed that the communication means is externally connected to communicate via the Internet or TCP / IP.

  Further, the content database 2 may be configured by a so-called RDBMS (Relational Database Management System) or the like.

  The content database 2 stores at least data of the content itself (hereinafter, content data) or an address uniquely indicating the location of the data.

  The content data is, for example, a document file for a document, an image file for an image, a sound file for sound, a video file for video, and the like. Preferably, the content database 2 includes an identifier that can uniquely identify each content.

  In addition, as metadata, for example, content that expresses the content content (for example, content title, summary sentence, keyword, etc.), content format (for example, content data size, thumbnail size, etc.), etc. It does not matter.

[Functions of each part of the information processing device]
Subsequently, functions of each unit of the information processing apparatus 1 will be described.

  The input unit 11 acquires content data from the content database 2 and transmits it to the feature extraction unit 12.

  The feature extraction unit 12 analyzes the content data obtained from the input unit 11 and extracts feature amounts that characteristically represent the content. The extracted feature amount is transmitted to the hash function generation unit 13 during the hash function generation process, and is transmitted to the hashing unit 15 during the hashing process.

  The hash function generation unit 13 generates one or more hash functions based on the feature amounts transmitted from the feature extraction unit 12 and stores them in the hash function storage unit 14.

  The hash function storage unit 14 stores one or more hash functions generated by the hash function generation unit 13.

  Based on one or more hash functions stored in the hash function storage unit 14, the hashing unit 15 converts the feature amount of the content transmitted from the feature extraction unit 12 into one or more hash values that are binary values. The data is converted and transmitted to the output unit 16.

  The output unit 16 transmits the hash value converted by the hashing unit 15 to the content database 2 and stores it.

[Processing operation of information processing device]
Next, the processing operation of the information processing apparatus 1 will be described. The information processing apparatus 1 executes a hash function generation process for generating a hash function and a hashing process for hashing the feature amount of the content. Hereinafter, these two processes will be described.

  First, the hash function generation process will be described. FIG. 2 is a flowchart showing the flow of the hash function generation process. The hash function generation process is a process that is performed at least once before the content data is actually hashed.

  First, the input unit 11 acquires content data from the content database 2 and transmits it to the feature extraction unit 12 (step S101).

  Subsequently, the feature extraction unit 12 extracts a feature amount from the content data and transmits it to the hash function generation unit 13 (step S102).

  Finally, the hash function generation unit 13 generates one or more hash functions based on the feature amount, and stores them in the hash function storage unit 14 (step S103).

  Through the above processing, one or more hash functions can be generated from each content data stored in the content database 2. Details of the feature amount extraction processing and the hash function generation processing will be described later.

  Next, the hashing process will be described. FIG. 3 is a flowchart showing the flow of the hashing process. The hashing process is a process for hashing the feature amount of the content using the hash function stored in the hash function storage unit 14.

  First, the input unit 11 acquires content data from the content database 2 and transmits it to the feature extraction unit 12 (step S201).

  Subsequently, the feature extraction unit 12 extracts a feature amount from the content data and transmits it to the hashing unit 15 (step S202). The processes of steps S201 and S202 are the same as steps S101 and S102 of the hash function generation process, respectively.

  Then, the hashing unit 15 converts the feature amount of the content into a hash value using one or more hash functions stored in the hash function storage unit 14, and transmits the hash value to the output unit 16 (step S203).

  Since one hash function is converted into a 1-bit hash value, when B hash functions are stored in the hash function storage unit 14, the content is converted into a B-bit hash value.

  Finally, the output unit 16 transmits the hash value to the content database 2 and stores it (step S204).

  Through the above process, the hash value of the content registered in the content database can be obtained.

[Feature extraction processing]
Next, feature amount extraction processing will be described. The process of extracting the content feature amount depends on the type of content. For example, the feature quantity that can be extracted or extracted varies depending on whether the content is a document, an image, a sound, or a video.

  Here, what kind of feature value is extracted is not important as a requirement of the present invention, and a publicly known feature extraction process may be used. Therefore, here, an example of feature extraction processing for various contents suitable for an example of the present embodiment will be described.

  When the content is a document, the appearance frequency of words appearing in the document can be used. For example, the appearance frequency may be counted for each word corresponding to a noun, an adjective, or the like using a known morphological analysis.

  When the content is an image, for example, brightness features, color features, texture features, concept features, landscape features, and the like are extracted.

  The brightness feature can be extracted as a histogram by counting the V values in the HSV color space.

  The color feature can be extracted as a histogram by counting the values of the respective axes (L *, a *, b *) in the L * a * b * color space. The number of histogram bins on each axis may be, for example, 4 for L *, 14 for a *, 14 for b *, etc. In this case, the total number of bins for 3 axes is 4 X14x14 = 784.

  As a texture feature, a statistic (contrast) of a density histogram, a power spectrum, or the like may be obtained. Alternatively, it is preferable to use a local feature amount because it can be extracted in the form of a histogram as in the case of color and movement.

As the local feature, for example, SIFT (Scale Invariant Feature Transform) described in the following Reference 1 or SURF (Speeded Up Robust Features) described in the following Reference 2 can be used.
[Reference 1] DG Lowe, “Distinctive Image Features from Scale-Invariant Keypoints”, International Journal of Computer Vision, pp. 91-110, 2004 [Reference 2] H. Bay, T. Tuytelaars, LV Gool, “ SURF: Speeded Up Robust Features ”, Lecture Notes in Computer Science, vol. 3951, pp.404-417, 2006

  The local feature extracted by these becomes, for example, a 128-dimensional real value vector. This vector is converted into a code with reference to a code length that has been learned and generated in advance, and a histogram can be generated by counting the number of the codes. In this case, the number of bins in the histogram matches the code number of the code length. Any number of codes may be used. For example, 512 or 1024 may be used.

  A concept feature is an object included in an image or an event captured by the image. Anything may be used, but examples include “sea”, “mountain”, “ball”, and the like. If “sea” appears in an image, the image belongs to the “sea” concept.

  Whether or not the image belongs to each concept can be determined using a concept classifier. Usually, one concept discriminator is prepared for each concept, and the feature amount of the image is input, and whether or not the image belongs to a certain concept is output as an attribution level.

  The concept discriminator is acquired by learning in advance, and represents the concept of the image to which the image belongs, based on the predetermined image features, for example, the local features described above and the human hand beforehand. Acquired by learning the relationship with the correct answer label.

  For example, a support vector machine may be used as the learning device. Concept features can be obtained by expressing the attribution levels for each concept together as a vector.

A landscape feature is a feature amount that represents a landscape or scene of an image. For example, the GIST descriptor described in Reference Document 3 below can be used.
[Reference 3] A. Oliva, A. Torralba, “Building the gist of a scene: the role of global image features in recognition”, Progress in Brain Research, 155, pp. 23-36, 2006

  When the content is a sound, a pitch feature, a sound pressure feature, a spectrum feature, a rhythm feature, an utterance feature, a music feature, a sound event feature, and the like are extracted.

The pitch feature may be a pitch, for example, and can be extracted using a method described in Reference Document 4 below.
[Reference 4] Sadaaki Furui, “Digital Audio Processing, 4.9 Pitch Extraction”, pp.57-59, 1985

  As the sound pressure feature, an amplitude value of speech waveform data may be used, or a short-time power spectrum may be obtained, and an average power in an arbitrary band may be calculated and used.

  As the spectral feature, for example, Mel-Frequency Cepstral Coefficients (MFCC) can be used.

As the rhythm feature, for example, a tempo may be extracted. In order to extract the tempo, for example, a method described in Reference Document 5 below can be used.
[Reference 5] ED Scheirer, “Tempo and Beat Analysis of Acoustic Musical Signals”, Journal of Acoustic Society America, Vol. 103, Issue 1, pp.588-601, 1998

The utterance feature and the music feature represent the presence / absence of utterance and the presence / absence of music, respectively. In order to find a section in which speech / music exists, for example, a method described in Reference Document 6 below may be used.
[Reference 6] K. Minami, A. Akutsu, H. Hamada, Y. Tonomura, “Video Handling with Music and Speech Detection”, IEEE Multimedia, vol. 5, no. 3, pp. 17-25, 1998

As the sound event feature, for example, emotional sound such as laughter and loud voice, or occurrence of environmental sound such as gunshot and explosion sound may be used. In order to detect such a sound event, for example, a method described in Reference Document 7 below may be used.
[Reference 7] International Publication No. 2008/032787

  When the content is a video, since the video is generally a stream of images and sounds, the image features and sound features described above can be used. As to which image and sound information in the video is analyzed, for example, the video is divided into several sections in advance, and feature extraction is performed from one image and sound for each section.

In order to divide the video into sections, the video may be divided at predetermined intervals, for example, the video may be discontinuously cut using the method described in Reference Document 8 below. It is good also as what divides | segments by the cut point which is a point.
[Reference 8] Y. Tonomura, A. Akutsu, Y. Taniguchi, G. Suzuki, “Structured Video Computing”, IEEE Multimedia, pp.34-43, 1994

  Desirably, the latter method is adopted. As a result of the video section division process, the start point (start time) and end point (end time) of the section are obtained, and may be handled as separate feature quantities at each time.

  One or a plurality of feature quantities extracted as described above may be used, or other known feature quantities may be used.

[Hash function generation processing]
Next, a hash function generation process will be described. It is assumed that the feature quantity extracted from the content i (i = 1, 2,..., N) is x _i, and that the feature quantity x _i belongs to the feature quantity space R ^d (x _i ∈R ^d ).

At this time, in step S103 described above, a set of hash functions h satisfying h: R ^d → {0, 1} is obtained. Since the feature quantity x _i is mapped to a binary value that takes 0 or 1 depending on each h, _B binary values, that is, Bbit of the Bbit by the hash function set H = {h ₁ , h ₂ ,. It will be converted to a hash value.

An object of the present invention is to make it possible to omit time-consuming similarity calculation between contents by using this hash value. Therefore, the hash function is required to be a mapping to the hash value related to the similarity s: R ^d × R ^d → R between the original feature values x _i, and the hash value becomes higher as the content has a higher similarity. It is preferable that the distance (Hamming distance) becomes close.

The hash function generation process will be described in detail below. First, as a hash function h _k (k = 1, 2,..., B) generated by the hash function generation unit 13, a hash function based on a linear function as shown in equations (1) and (2) is considered.

Here, sign (x) is a sign function having w _k εR ^d and b _k εR as parameters and taking 1 when x ≧ 0 and -1 when x <0. Since the hash function is based on such a sign function, the feature amount x _i of the content i is converted into a binary value of 0 or 1.

In this hash function h _k , there are only two unknown parameters, w _k and b _k . Here, if, when the characteristic amounts x _i of the content i is normalized to zero mean without loss of generality as a b k _{= 0.}

Since the normalizing characteristic amounts x _i 0 is than the average m of the feature x _i may be subtracted from each feature amount x _i, which is possible in x _i ∈R ^d, b _k = 0.

Therefore, hereinafter, it is assumed that the average of the feature amounts x _i is normalized to 0, and the equation (2) is redefined as the equation (3).

In the hash function h _k of the expressions (1) and (3), defining the parameters of w _k (k = 1, 2,..., B) is the purpose of this hash function generation process.

Here, the meaning of the hash function h _k expressed as equations (1) and (3) can be explained geometrically with reference to FIG. In the figure, feature quantities x _i extracted from each content i are distributed as feature points in the feature quantity space R ^d . In the figure, for the sake of convenience, it is shown as two-dimensional, but it is actually a d-dimensional space.

Here, φ _k (x _i ) constituting the hash function h _k represents a straight line (actually, a (d−1) -dimensional hyperplane) passing through the origin on the feature amount space R ^d . Since the hash function h _k (x _i ) is essentially a sign function as described above, any space side where the feature point indicating the feature quantity x _i is divided into two by this straight line φ _k (x _i ). 1 or 0 depending on whether or not

That is, the hash function h _k (x _i ) defined by the equations (1) and (3) is a function that divides the feature amount space R ^d into two regions of 1 and 0 by the straight line φ _k (x _i ). is there. The parameter w _k corresponds to the slope of the straight line φ _k (x _i ), and if the value of w _k changes, the angle to be divided changes.

Therefore, according to the present embodiment, it is possible to provide a method for generating the hash function h _k that rationally determines the parameter w _k based on the distribution of the feature quantity x _i of the content i. Hereinafter, the principle and processing contents will be described.

One of the most rational methods for obtaining the parameter w _k so as to achieve a hash function that meets the above-described purpose is to draw a straight line so that similar content groups are gathered on one side of the straight line in the example of FIG. (That is, determine the parameter w _k ).

Here, description will be made with reference to FIG. In the figure, a feature amount x _i, represented by white circles (○), there is the feature amount x _i, represented by a black circle (●). These Separately, a double circle (◎) feature amount x _i, represented by, but there may time being this is ignored.

Now, the feature x _i, represented by white circles and black circles are assumed to be the characteristic amounts of the contents that are similar to each other if the same color. At this time, a straight line may be drawn so that these similar content groups are gathered on one side of the straight line. Such a straight line is a straight line passing between the two groups, and actually exists infinitely. For example, there is a straight line 51.

However, such a straight line 51 lacks robustness (robustness). Here, the feature quantity x _i of the double circle that was ignored is considered. This is rather around the white circle group, and it can be seen at a glance that it is the feature amount of the content that should belong to the white circle group.

However, the double circle feature quantity x _i has been classified into the group of black circles by the straight line 51 and has been given an incorrect classification. If the content i having the double circle feature amount x _i is generated by fluctuation due to noise, a correct classification cannot be given.

Usually, since the feature amount x _i of the content i is usually be obtained with a fluctuation in the linear 51 drawn arbitrarily, no robustness against such fluctuations, which leads to loss of accuracy .

In contrast, in the present embodiment, a straight line having high robustness can be obtained from infinite straight lines. A highly robust straight line may be selected as a straight line that maximizes the margin, that is, a straight line in which the perpendicular distances from all the feature amounts x _i (feature points) of both groups are equal. .

The straight line 52 selected in this way is a straight line that is equally farthest from each other's group. Therefore, even if the feature amount x _i as a double circle, such as those generated fluctuates around the both groups may correct classification.

That is, the hash function generator 13, classified into at least two groups are distributed as a feature point on the characteristic amounts x _i characteristic amount space R ^d of the plurality of content i, from all feature points in each county A straight line having the same vertical distance is calculated, and a hash function for converting the content i into a binary value is generated based on which space the feature point is divided into two by the straight line.

Such linear 52 is, for example, determine the value of w that maximizes the calculated value of the objective function of their both groups intergroup variance S _b and expressions with county within variance S _w in a variable (4) Can be obtained. Note that “trace” is used to calculate the sum of diagonal components of an n-order square matrix in {}, and is generally used in the field of linear algebra.

Incidentally, intergroup variance _{S b,} county in variance _{S w,} respectively, equation (5) may be calculated using (6). Where m ^j is the average vector of group j (j = 0, 1), m is the overall average vector, N _j is the number of feature quantities x _i belonging to group j, and N is the number of overall feature quantities x _i . is there. As described above, when normalization is performed so that the average of the whole becomes zero, theoretically, m = 0.

Note that w that maximizes the expression (4) can be obtained as a solution (eigenvector) of the following eigenvalue problem by, for example, the Lagrange undetermined multiplier method.

  As a method for obtaining w by solving this eigenvalue problem, various known methods such as an iterative method may be used.

The above needs to know in advance which group the feature amount x _{i of the} content i belongs to. Hereinafter, this information is referred to as group instruction information. On the other hand, in the actual problem, the group instruction information is unknown. In other words, the above-described method cannot be used for many problems that exist in reality.

  Therefore, in the present embodiment, processing is performed that can generate a similar hash function even when the group instruction information is unknown. Hereinafter, an example of this process will be described in detail.

First, a similarity matrix V that stores the similarity between feature quantities x _{i is} obtained. The similarity matrix V may be any type, but may be obtained using, for example, Expression (8).

Also, a diagonal matrix D is calculated based on the similarity matrix V.

Then, a regularization term that does not depend on the group instruction information is introduced into Equation (4).

A second term that regularization term in equation (10), does not include the between-group variance S _b and counties within variance S _w, is computable term without any group instruction information.

The value of w that maximizes the calculated value of equation (10) can be obtained as a solution to the following eigenvalue problem by the Lagrange undetermined multiplier method. Incidentally, eta is a parameter, X is the set of the feature x _i when calculated hash function.

Since the solution of such an eigenvalue problem can be calculated by a known method such as an iterative method as described above, the inter-group variance value _Sb and the intra-group variance value _Sw are zero matrices (the elements are all zero). Matrix), w satisfying equation (11) can be obtained even if there is no group instruction information. Thus the w obtained, and _{w 1.}

Hash function one (i.e., to convert each feature amount x _i to the hash value of 1 bit) To accept, it may end the process of generating the hash function with a more procedures.

However, usually, a plurality of hash functions are prepared and converted into a multi-bit hash value. Here, assuming that B hash functions are prepared by the following procedure, w ₂ , w ₃ ,..., W _B are obtained sequentially from w ₁ .

Now, assuming that _k hash functions h _k are obtained, consider obtaining the k + 1th hash function h _{k + 1} . In this embodiment, by using a hash function h _k to k-th, to determine the pseudo-group instruction information to each feature quantity x _i. This assigning method may be any method. For example, pseudo group instruction information is given based on the following rule using the _k-th hash function h _k .

Here, δ is a parameter. By considering the pseudo group instruction information determined in this way as group instruction information, it is possible to calculate the inter-group variance value _Sb and the county variance value S _w using the equations (5) and (6). And w _{k + 1} satisfying equation (11) can be obtained.

Basically, the hash function h _k may be obtained sequentially by repeating the above processing as k = 1, 2,.

On the other hand, when “pseudo group instruction information is determined for each feature quantity x _i using the _kth hash functions h _k ”, the kth hash functions h _k and k + 1 are used. There is a high probability that the hash functions h _{k + 1} of the eyes are highly correlated (similar) to each other, which is inefficient.

  Therefore, in the present embodiment, correction processing for generating hash functions having low correlation with each other is introduced when a plurality of hash functions are generated so that high-precision hash function generation can be realized with a smaller number of hash functions.

For example, using a hash function _{h k} of the k-th set of characteristic amounts _{x i} when calculated hash function _{h k} of the k-th _{X k = {x 1, x} 2, ..., x N} , and Correct as follows. That is, the distribution state of feature points is changed so that the degree of correlation between hash functions is low.

By this correction, the direction component indicated by w _k ^T X _k is degenerated from the distribution of the feature amount, and as a result, the feature amount distribution has a low correlation with the already obtained hash function. By this correction, hash functions having a small correlation with each other are generated.

  By repeating the above processing, B hash functions can be obtained. An example of processing can be summarized as follows.

(Procedure 1) First, k = 1, S _b ^{k = 1} = 0, S _w ^{k = 1} = 0, and X _{k = 1} = X.

(Procedure 2) Next, the following eigenvalue problem is solved, and the eigenvector corresponding to the maximum eigenvalue is set to w _k .

(Procedure 3) Next, pseudo group indication information is calculated using equation (12).

(Procedure 4) Next, if k = B, the process is terminated; otherwise, the process proceeds to (Procedure 5). B can be set arbitrarily.

(Procedure 5) Next, S _b ^{k + 1} , S _w ^{k + 1} , and X _{k + 1} are _obtained using equations (15) to (17).

(Procedure 6) Return to (Procedure 2).

The hash function h _k generated by the above processing procedure, specifically, the parameter w _k (k = 1, 2,..., B) is stored in the hash function storage unit 14.

[Hashing]
If the above hash function generation processing has been completed, the hash function storage unit 14 stores B hash functions. By using this, an arbitrary content i expressed by the feature value x _i can be expressed by a hash value of B bits or less.

〔Example〕
Here, an example will be described in which a similar image is searched based on the hash value implemented in the present embodiment for an image database.

  In the image database of this embodiment, about 20,000 images are registered. Each image shows 10 different objects. In this embodiment, an object is to find an image showing the same object from images registered in the image database.

  In the case of the conventional method, feature amounts are extracted from all images registered in the image database, and images in the image database having a high similarity to the feature amount of the browse image are output as a recommendation result.

  On the other hand, in the present embodiment, this is performed using a hash value. Although any feature amount may be used, in this embodiment, the image is reduced to 28 pixels × 28 pixels = 784 pixels, and the luminance value (256 tone) of each pixel is used as it is.

  That is, it is a 784-dimensional integer vector and corresponds to 6,272 bits. This feature amount is extracted in advance from all images in the image database. Further, according to the present embodiment, a hash value is generated from the feature amount of the content and registered in the image database.

  In this example, the similarity search accuracy based on the hash value generated by the present embodiment was compared with the similarity search accuracy based on the hash value generated by the techniques of Non-Patent Documents 1 and 2. FIG. 6 shows the search accuracy when the number of bits of the hash value (number of hash functions) is changed.

  In any number of bits, the present embodiment obtains higher accuracy. According to the present embodiment, high accuracy can be realized even when a place where 6,272 bits are normally required per content is expressed by a very small amount of information such as 8 bits to 32 bits. From this, the high effect of the technique by a present Example can be confirmed.

As described above, according to the present embodiment, each feature quantity x _i of a plurality of contents i is distributed as feature points on the feature quantity space R ^d and classified into at least two groups, and all the features of each county are classified. More specifically, a straight line having the same vertical distance from the point is obtained, and a hash function h _k for converting the content i into a binary value is generated based on which space side the feature point is divided into two by the straight line. Since the hash function h _k is generated based on the inter-group distance and the intra-group distance of the feature quantity x _i , a robust hash function can be generated, and as a result, a large amount of multimedia content can be used. Provides a content conversion method, a content conversion apparatus, and a content conversion program capable of finding similar contents with high accuracy while being high-speed and memory-saving Kill.

In addition, according to the present embodiment, when generating a plurality of hash functions h _k , the hash functions are generated while sequentially correcting the distribution of the feature quantity x _i , so that hash functions having low correlation can be generated. As a result, high-accuracy similar content search can be performed with a smaller number of hash functions (number of bits).

1 Information processing device (content conversion device)
2 ... Content database 11 ... Input unit 12 ... Feature extraction unit (extraction means)
13 ... Hash function generator (generator)
14 ... Hash function storage unit (storage means)
15 ... Hashing unit 16 ... Output unit S101-S103, S201-S204 ... Step

Claims (8)

In a content conversion method for converting content into one or more binary values,
By computer
An extraction step of reading out the content from the storage means and extracting each feature amount of the plurality of contents;
Each feature amount is distributed as a feature point in the feature amount space and classified into at least two groups, and a straight line or a hyperplane having the same vertical distance from all feature points of each group is obtained, and the feature point is divided into two by the figure. Generating a hash function that converts the content into a binary value based on which spatial side is
A content conversion method characterized by comprising: The generating step includes
The calculated value of trace {w ^T (S _b −S _w ) w} (where w ^T w = 1) is the maximum using the inter-group variance value S _b and the county variance value S _w of the two groups as variables. The content conversion method according to claim 1, wherein the figure is obtained by using a value of w as a slope. The generating step includes
3. The content conversion method according to claim 1, wherein when generating a plurality of hash functions, the distribution state of the feature points is changed so that the degree of correlation with other hash functions is low.   4. The content conversion method according to claim 1, further comprising a conversion step of converting the content into one or more binary values using one or more of the hash functions. In a content conversion device that converts content into one or more binary values,
Extracting means for reading out content from the storage means and extracting each feature amount of the plurality of contents;
Each feature amount is distributed as a feature point in the feature amount space and classified into at least two groups, and a straight line or a hyperplane having the same vertical distance from all feature points of each group is obtained, and the feature point is divided into two by the figure. Generating means for generating a hash function for converting the content into a binary value based on which space side is provided;
A content conversion apparatus comprising: The generating means includes
The calculated value of trace {w ^T (S _b −S _w ) w} (where w ^T w = 1) is the maximum using the inter-group variance value S _b and the county variance value S _w of the two groups as variables. The content conversion apparatus according to claim 5, wherein the figure is obtained by using the value of w as a slope. The generating means includes
The content conversion apparatus according to claim 5 or 6, wherein when generating a plurality of hash functions, the distribution state of the feature points is changed so that the degree of correlation with other hash functions is low.   A content conversion program that causes a computer to execute the content conversion method according to claim 1.

Images