JP4544098B2 - Content search device - Google Patents

Description

  The present invention relates to a content search apparatus that searches a user's desired content from a recording medium on which the content is recorded.

  In recent years, many large-capacity recording media such as optical disks, hard disks, and semiconductor memories have been used. Some portable music playback apparatuses, which are examples of apparatuses using large-capacity recording media, can store music data of 10,000 songs or more in one apparatus.

  In order to efficiently search for a desired content from a large amount of content in this way, a method of searching using an impression value obtained by quantifying an impression on the content is proposed in Patent Document 1.

  In the search method of Patent Document 1, first, a feature value representing a physical feature of a signal is extracted from a content signal (a music piece in Patent Document 1) in a recording medium to be searched, and the feature value is extracted from the content. The impression value is converted into a numerical impression value. By mapping the impression value on a two-dimensional plane (“Impression space” in Patent Document 1), the similarity of the impression value is schematically shown.

  Next, as a search requirement, the user has a character string such as the artist name of the content to be searched, a numerical value indicating the degree of impression such as “strength”, and the presence or absence of a predetermined condition such as “I want to dance cheerfully” Enter the search method.

  Then, the Euclidean distance between the input predicted impression value of the content to be searched and the impression value of the content to be searched mapped in the impression space is calculated, and the content is fed back to the user in order from the content having the smallest Euclidean distance. The search results are presented.

  In the technique of Patent Document 2, a method is described in which a factor position calculated from a parameter for content impression is mapped to a three-dimensional coordinate space to perform a search on the coordinate space.

  Furthermore, in the technique of Patent Document 3, the sensitivity data including the evaluation value representing the user's evaluation for the video data stored in the recording medium as a numerical value is stored in the apparatus. And how to search. This evaluation value input method includes a method in which a degree for a word representing sensitivity, such as “I am impressed” or “I am energetic”, is designated by a numerical value of 0 to 1 or in a two-dimensional space as in Patent Document 1. There are mapping methods.

  In these techniques, when a user selects content mapped in space, a pointing device such as a mouse is used.

  In addition, when searching for impression words representing impressions as a query, there are a direct input method in which the user inputs and converts impression words as keywords, and a method for selecting impression words as keywords using check boxes and pull-down menus. It was taken.

According to the search methods of Patent Documents 1, 2, and 3, a desired content can be searched from a recording medium without the user knowing the title of the content.
JP 2002-278547 A Japanese Patent Laid-Open No. 9-34909 Japanese Patent Laid-Open No. 9-6802

  However, the method using a pointing device such as a mouse cannot be used with a portable terminal or the like, and there is a problem that the use environment is limited.

  In addition, the search method based on the direct input method has a problem that it is difficult to use in an environment where the keyboard cannot be used because it greatly depends on the input device.

  In addition, a search method using a check box, a pull-down menu, or the like generally has a problem that the number of items is very large, which causes confusion for the user and an image of desired content becomes ambiguous. Furthermore, there is a problem that all check items cannot be entered in a device with a small screen display portion such as a portable terminal.

  The present invention has been made in view of the above circumstances, and can be used in a device such as a portable terminal to which a screen display unit is small, an input device such as a mouse or a keyboard is not connected, and the usage method is simple. An object is to provide a content search device.

In order to achieve the above object, a content search device (1) according to claim 1 of the present invention is for searching for a desired content from a recording medium on which a plurality of contents are recorded. Part value storage means (31) for storing the part value that represents the shape of each part as a numerical value, and the range of the part value for each part in advance. Template storage means (32) for storing templates of representative facial expressions composed of each set part, and the impression level of each impression word representing facial expression preset for each template is digitized Template impression vector storage means (33) for storing the impression value as a template impression vector, and impression values for each of a plurality of contents recorded on the recording medium. The metadata storage means (34) for storing a certain recorded content impression vector as metadata and the shape of each part corresponding to the selected part value from the part values stored in the part value storage means (31) Image image generation means (21) for generating an image that represents the impression of the user's desired content by combining the image,
Template selection means (22) for selecting a template that matches the combination of part values of the image image generated by the image image generation means (21) from the templates stored in the template storage means (32), and template selection The template impression vector of the template selected by the means (22) is acquired from the template impression vector storage means (33), and an image image impression vector which is an impression value of the image image generated based on the acquired template impression vector is calculated. Image image impression vector calculating means (23) to perform,
Content search means (24) for calculating the distance between the calculated image image impression vector and each recorded content impression vector, and outputting desired content candidates from the content corresponding to the recorded content impression vector in ascending order of the distance. It is characterized by providing.

  Further, claim 2 is the content search device (2) according to claim 1, wherein the part value calculation means (2) calculates the part value of each part from the recorded content impression vector of the content selected as a representative from the recording medium. 25), and the image image generation means (21) generates an image image representing the impression of the desired content by combining the shape of each part corresponding to the part value calculated by the part value calculation means (25) It is characterized by doing.

  According to the content search apparatus of the present invention, the screen display unit is small and can be used in devices such as a portable terminal to which an input device such as a mouse or a keyboard is not connected. be able to.

<First Embodiment>
The configuration of the content search apparatus 1 according to the first embodiment of the present invention will be described with reference to FIG. The content search device 1 according to the present embodiment searches for content desired by a user from content stored in a recording medium (hereinafter referred to as “recorded content”). The configuration of the content search apparatus 1 according to the present embodiment includes an input unit 10, a control unit 20, a storage unit 30, and a display unit 40. In the present embodiment, a value obtained by quantifying an impression degree with respect to an impression word that is a phrase such as “brightness”, “power”, and “kindness” representing an impression is referred to as an impression vector.

  The input means 10 is a device for a user to input using the display means 40, and is a cross cursor or a click wheel that can be used on a portable terminal.

  The control unit 20 includes an image image generation unit 21, a template selection unit 22, an image image impression vector calculation unit 23, and a content search unit 24.

  The image image generation means 21 generates an image image representing the impression of the desired content of the user with the shape of the part selected by the user's instruction from the shape of the parts constituting the face image. Examples of parts include “Eye”, “Brow”, “Nose”, “Hair”, “Mouth”, “Outline”, “Ear”, “Wrinkle”, “Skin color”, etc. In the form, there are three parts to be used for an image image: “pupil”, “eyebrows”, and “mouth”. Each part is given a part value for identifying the shape.

  The template selection means 22 selects a template that matches a combination of each part value of the image image generated by the image image generation means 21 from a plurality of facial expression templates whose part value ranges are preset for each part.

  The image image impression vector calculation means 23 acquires a template impression vector which is an impression vector of the selected template, and calculates an image image impression vector which is an impression vector of the image image from this template impression vector.

  The content search means 24 calculates the distance between the recorded content impression vector, which is the impression vector of the recorded content, and the image image impression vector. As a result of the calculation, a recorded content impression vector whose distance from the image image impression vector is equal to or less than a predetermined value is extracted, and bibliographic information of the content in the recording medium corresponding to the extracted recorded content impression vector is output to the display means 40 as a search result. To do.

  The storage unit 30 is a memory, HDD, or the like, and includes a part value storage unit 31, a template storage unit 32, a template impression vector storage unit 33, a metadata storage unit 34, and a bibliographic information storage unit 35.

  The part value storage means 31 stores a part value in which the shape of each part is represented by a numerical value when the impression of each content is associated with each part constituting the human face.

  The template storage means 32 classifies the image image into a typical image set in advance such as “brave” or “fun” according to the facial expression obtained from the image image. A template of a plurality of representative expressions composed of preset parts is stored.

  The template impression vector storage means 33 stores an impression value obtained by quantifying the impression level of each impression word representing a facial expression preset for each facial expression template as a template impression vector.

  The metadata storage means 34 stores recorded content impression vectors that are impression vectors of the respective recorded contents.

The bibliographic information storage means 35 stores bibliographic information such as the title and artist name of recorded content, and content ID ₁ to content ID _j for identifying each content.

  The display unit 40 displays the bibliographic information of the recorded content output as the search result of the content search unit 24.

  Next, the operation of the content search apparatus 1 in the present embodiment will be described with reference to the flowchart of FIG.

  First, when the content search apparatus 1 is activated, a default image is generated by the image generation means 21 from preset default part values of each part (S1).

  An example of the shape of the part corresponding to each part value is shown in FIG. In this embodiment, the variation of the shape of the “pupil” is set in the range of the part values 0 to +10, and the variation of the shape of the “eyebrows” and the “mouth” is set in the range of the part values −10 to +10.

  The default part value of each part in the present embodiment is 6 for “pupil”, “eyebrows”, and “mouth”. The generated default image is displayed on the display means 40 as shown in FIG. 4 (S2).

  Next, each part value is changed by the user in order to match the displayed default image with an image for a user-desired content to be searched (hereinafter referred to as “search target content”) (S3).

  The part value changing process of the default image will be described. In the default image displayed on the display means 40 of FIG. 4, the values of the parts of the pupil, the eyebrows, and the mouth can be changed by a user operation.

  First, the part to be changed is designated by the part designation cursor 44 by the user. In FIG. 4, “eyebrows” indicated by the part designation cursor 44 indicates that the part is a change processing target.

  The part value increase / decrease button such as the cross cursor 50 as shown in FIG. 5 as the input means 10 or the click wheel 60 as shown in FIG. When the user operates 51 or 61, the part value of “eyebrows” is increased or decreased. The cross cursor 50 is operated by pressing a button, and the click wheel 60 is operated by sliding a finger.

  As the part value is increased or decreased by this operation, the shape of the “eyebrow” on the display means 40 is changed and displayed according to the shape for each part value shown in FIG.

  Next, the shape representing the image of the search target content is selected by the user from the variations of the shape of the “eyebrows” displayed on the display unit 40, and the part designation button 52 or 62 is pressed.

  When the part designation button 52 or 62 is pressed, the part designation cursor 44 moves to “pupil”.

  Parts values are changed for “pupil” and “mouth” in the same process as “eyebrows”. When all the parts have been changed, the user presses the enter button 53 or 63 to generate an image representing an image of the search target content (“Yes” in S4). When the changed part is further updated, the determination button 53 or 63 is not pressed and the process returns to step S2 to continue the change process (“No” in S4).

  An example in which the image image generated by the image image generating means 21 is displayed on the display means 40 is shown in FIG. In this embodiment, it is assumed that the part value of “eyebrows” is changed to -3, the part value of “pupil” is 5, and the part value of “mouth” is changed to 8. Thus, the default image changing process is completed.

  Next, in the template selection unit 22, a template that matches the generated image image is selected from templates stored in advance in the template storage unit 32 based on the facial expression represented by the combination of part values (S5). ). The reason why the “expression expressed by a combination” of the part values is used as a template is that a human facial expression does not make sense only by each part, and an image obtained from a plurality of parts is meaningful.

  FIG. 8 shows an example of a template stored in the template storage means 32 and a part value range of each part for matching with each template. In the present embodiment, a template A representing an “anger” image, a template B representing a “fun” image, a template C representing a “courageous” image, a template D representing a “relaxed” image, and a “parting” image. The range of part values of “eyebrows”, “pupil”, and “mouth” for each of the five templates E to be represented is preset.

  The value of each part of the content to be processed in the present embodiment is “-3” for “eyebrows”, “5” for “pupil”, and “8” for “mouth”. 2 to 5, “Mouth” A template D that matches one or more conditions and represents a “loose” image is selected.

  Next, a template impression vector related to the selected template D is acquired from the template impression vector storage means 33 (S6).

  An example of the template impression vector stored in the template impression vector storage means 33 is shown in FIG. In this embodiment, the template impression vector is a numerical value representing the degree of impression with respect to the components 1 to N, which are impression words representing impressions such as “brightness”, “strength”, and “kindness” for each of the templates A to E. It has been. In the present embodiment, component 1 = 0.30, component 2 = 0.00,... Component N = 0.00 of template D are acquired.

  Here, when there is a difference between each part value of the image image and the boundary value of the part value of the template selected in FIG. 8, the template impression vector value acquired in FIG. An image impression vector is calculated (S7).

  In the present embodiment, the part value of “eyebrows” in the image image is −3, and there is a difference of 2 from the boundary value “−1” of the part value of “eyebrows” in the template D. Further, the part value of the “mouth” of the image image is 8, and there is a difference of 7 from the boundary value 1 of the part value of the “mouth” of the template D.

From these differences, an image image impression vector is calculated using the following equation (1).

In Expression (1), t is an image image impression vector to be obtained, Z is each component of the template impression vector, α _i is a correction coefficient, Δp _i is a part value of part _i of the created image image, and It is the difference from the boundary value of the part value of part _i of the template, K is the total number of parts, and N is the number of impression vector components.

Here, if the desired impression vector component is n (1 ≦ n ≦ N) and the part is k (1 ≦ k ≦ K), Z _n represents the nth impression vector component of the template, and α _nk is the nth The correction coefficient for the vector component part k is represented, and Δp _k represents the difference between the part value of the part k of the image image and the boundary value of the part value of the part k of the template. The correction coefficient α _nk is set in advance.

  An example of the calculated image impression vector is shown in FIG. In the present embodiment, the image image impression vector is calculated from the difference between “eyebrows” and “mouth” by correcting the impression vector 0.30 of the component 1 of the template D in FIG. 9 to 0.2213. .

  Next, the content search unit 24 calculates the distance between the recorded content impression vector of each recording content recorded on the recording medium and the image image impression vector acquired from the image image impression vector calculation unit 23 (S8). ).

  Each recorded content impression vector is calculated from a method using a data conversion table for converting bibliographic information such as genre, artist gender, or lyrics into an impression vector, or a content signal described in JP-A-6-290574. The feature amount is calculated by a method using the feature amount and stored in the metadata storage unit 34 of the storage unit 30 in advance.

FIG. 11 shows an example of the recorded content impression vector. In the present embodiment, components 1 to N of impression vectors calculated for each of content ID ₁ to content ID _j are stored as recorded content impression vectors.

The distance between the recorded content impression vector and the image image impression vector is calculated using the following equation (2).

In Expression (2), d (j) (1 ≦ j ≦ S, S represents the total number of contents) is the distance between the calculated recorded content impression vector and the image image impression vector, and x _i is the image image impression vector. I _ij is the i-th impression vector component of the recorded content impression vector of content ID _j , i is the impression vector component, j is a number for identifying the content, N Is the dimension of the impression vector.

  As a result of the calculation, the content search unit 24 extracts a predetermined number of recorded content impression vectors in order from the one closest to the image image impression vector, and acquires a content ID corresponding to the recorded content impression vector.

  The recorded content in the recording medium corresponding to these acquired content IDs is a candidate for the content desired by the user. In order to present these content candidates to the user, the bibliographic information of the content is acquired from the bibliographic information storage unit 35 based on the acquired content ID and output to the display unit 40 (S9).

FIG. 12 shows an example of bibliographic information of each recorded content. In the present embodiment, the title, author / artist name, and genre of content ID ₁ to content ID _j are stored as bibliographic information.

  The data in the metadata storage unit 34 and the data in the bibliographic information storage unit 35 are associated with each other by the content ID, and the bibliographic information is based on the content ID of the recorded content extracted from the metadata storage unit 34 by the content search unit 24. Bibliographic information is acquired from the storage means 35 and output to the display means 40 for display.

  By the above processing, when a content having a “relaxed” image is searched based on the template D to which the image image of FIG. 6 matches, for example, a song has a slow tempo, and a video has no intense camera work. Content is extracted and displayed.

  Furthermore, the content data of the content corresponding to the displayed bibliographic information can be acquired from the recording medium according to a user instruction and can be reproduced by the display means 40.

  According to the first embodiment described above, since the search is performed based on the image image having facial expressions, the user can easily generate an image and can perform the search intuitively.

  Further, since the input is simple, the present invention can be applied to a small portable terminal device in which an input device is limited, a CD, MD, or DVD player.

  Furthermore, since it can be used without understanding the language, it is not necessary to translate it into other languages, and it can be easily used by children.

  In the present embodiment, the part value changing process is operated using a cross cursor or a click wheel. However, the present invention is not limited to this, and can be operated using a commonly used mouse.

  In the present embodiment, the recorded content impression vector is calculated by a method using a data conversion table for converting bibliographic information into an impression vector, or a feature calculated from a content signal described in Japanese Patent Laid-Open No. 6-290574. Although a method using quantity is used, the method is not limited to this, and a method in which a user or a designer directly inputs a vector component may be used.

Second Embodiment
A content search apparatus 2 according to a second embodiment of the present invention will be described. In the content search device 1 of the first embodiment, an image is generated based on the default image image. However, in the content search device 2 of the present embodiment, recording of the recorded content that is determined to be close to the desired content by the user is selected. An image is generated based on the content impression vector.

  The configuration of the content search device 2 according to the second embodiment of the present invention will be described with reference to FIG.

  The configuration of the content search apparatus 2 according to the present embodiment is the same as that of the first embodiment except that the control unit 20 includes the part value calculation unit 25, and thus detailed description thereof is omitted.

  The part value calculation means 25 acquires a content impression vector from the bibliographic information of the recorded content, and further calculates a part value from the content impression vector.

  The operation of the content search device 2 in this embodiment will be described with reference to the flowchart of FIG.

  First, the representative content is selected by the user based on the bibliographic information of the content in the recording medium stored in the bibliographic information storage unit 35 (S11). At this time, the user selects a content close to the desired content as the representative content.

  Next, in the part value calculation unit 25, based on the content ID of the selected representative content, a representative content impression vector that is an impression vector of the representative content is acquired from the metadata storage unit 34 (S12).

  Further, the part value calculation means 25 calculates the distance between the acquired representative content impression vector and the template impression vectors of the templates A to E shown in FIG. 9, and the template with the shortest distance is selected ( S13).

The distance between the representative content impression vector and each of the template impression vectors of the templates A to E is calculated using the following equation (3).

In Expression (3), d (j) is the calculated distance between the representative content impression vector and the template impression vector, x _ij is the i-th impression vector component of the template impression vector of template j, and y _i is This is the i-th impression vector component of the representative content impression vector, i indicates the number of the impression vector component, and j (1 ≦ j ≦ R, R is the total number of templates) is a number for identifying the template , N is the dimension of the impression vector.

The distance d (j) obtained by Expression (3) is obtained by adding the i-th impression vector component x _ij of the template impression vector of the template j, the i-th impression vector component y _{i of the} representative content impression vector, and the square error. Calculated.

  Further, when the template with the shortest impression vector distance is selected, the part value calculation means 25 obtains the part value of the corresponding template according to the template matching condition stored in the template storage means 32 shown in FIG. (S14).

Then, the difference between the impression vector component of the selected template and the impression vector component of the representative content is calculated, and using this difference, the part value of the template is finely adjusted according to the following equation (4), and the representative content impression vector A part value is obtained.

  In Expression (4), p is a vector whose component is the part value of the representative content impression vector to be obtained, M is the total number of parts, N is the number of components of the impression vector, and the part is m (1 ≦ m ≦ M) When the impression vector component is n (1 ≦ n ≦ N) th, pm indicates the value of the mth part of the template, βmn is a preset correction coefficient, and Δtn is the vector component of the template The difference between n and the vector component n of the representative content impression vector is shown.

  FIG. 15 shows an example of the part value obtained by Expression (4). And based on this part value, an image image is produced | generated by the image image production | generation means 21 (S15). Furthermore, a part value changing process is performed in the same manner as the process of step S2 of FIG. 2 (S16).

  Here, when reselection of the representative content is instructed by the user, when a predetermined button (not shown) is pressed by the user, the process returns to step S12 to reselect the representative content ("S17" Yes ").

  When the representative content is not reselected (“No” in S17), the subsequent processing in steps S18 to S23 is the same as the processing in steps S4 to S9 in FIG. Omitted.

  According to the second embodiment described above, since the search is performed by generating the image of the desired content from the image of the representative content similar to the desired content, the effort for generating the image is reduced. The

It is a block diagram which shows the content search device in 1st Embodiment of this invention. It is a flowchart which shows operation | movement of the content search apparatus in 1st Embodiment of this invention. It is a table | surface which shows the shape of each part corresponding to the part value set to the content search device in 1st Embodiment of this invention. It is a display figure which shows the default image produced | generated with the content search device in 1st Embodiment of this invention. It is a front view which shows the cross cursor used with the content search device in 1st Embodiment of this invention. It is a front view which shows the click wheel used with the content search apparatus in 1st Embodiment of this invention. It is a display figure which shows the image image produced | generated with the content search device in 1st Embodiment of this invention. It is a table | surface which shows the template adaptation conditions set to the content search device in 1st Embodiment of this invention. It is a table | surface which shows the template impression vector set to the content search device in 1st Embodiment of this invention. It is a table | surface which shows the image image impression vector calculated with the content search device in 1st Embodiment of this invention. It is a table | surface which shows the recorded content impression vector set to the content search device in 1st Embodiment of this invention. It is a table | surface which shows the bibliographic information of the recording content set to the content search device in 1st Embodiment of this invention. It is a block diagram which shows the content search device in 2nd Embodiment of this invention. It is a flowchart which shows operation | movement of the content search device in 2nd Embodiment of this invention. It is a table | surface which shows the part value of the representative content calculated with the content search apparatus in 2nd Embodiment of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Content search device 2 ... Content search device 10 ... Input means 20 ... Control means 21 ... Image image generation means 22 ... Template selection means 23 ... Image image impression vector calculation means 24 ... Content search means 25 ... Parts value calculation means 30 ... Storage means 31 ... Parts value storage means 32 ... Template storage means 33 ... Template impression vector storage means 33 ... Impression vector storage means 34 ... Metadata storage means 35 ... Bibliographic information storage means 40 ... Display means 44 ... Parts designation cursor 50 ... Cross Cursor 51, 61 ... Parts value increase / decrease button 52, 62 ... Parts designation button 53, 63 ... Enter button 60 ... Click wheel

Claims (2)

In a content search apparatus for searching for desired content from a recording medium on which a plurality of contents are recorded,
A part value storage means for storing a part value representing the shape of each part as a numerical value when the impression of each content is associated with each part constituting a human face;
A template storage means for storing a plurality of representative facial expression templates each of which has a preset part value range for each part;
Template impression vector storage means for storing an impression value obtained by quantifying the impression degree of each impression word representing a facial expression preset for each template as a template impression vector;
Metadata storage means for storing a recorded content impression vector, which is an impression value for each of a plurality of contents recorded in the recording medium, as metadata;
Image image generation for generating an image representing an impression of the desired content of the user by combining the shapes of the parts corresponding to the selected part value from the part values stored in the part value storage unit Means,
A template selection unit that selects a template that matches a combination of part values of the image image generated by the image image generation unit from the templates stored in the template storage unit;
A template impression vector of the template selected by the template selection unit is acquired from the template impression vector storage unit, and an image image impression vector that is an impression value of the generated image image is calculated based on the acquired template impression vector. Image image impression vector calculation means,
Content search means for calculating a distance between the calculated image image impression vector and each recorded content impression vector, and outputting a candidate for a desired content from content corresponding to the recorded content impression vector in ascending order of the distance; ,
A content search apparatus comprising: A part value calculating means for calculating a part value of each part from the recorded content impression vector of the content selected as a representative from the recording medium;
The image image generating means generates an image image representing an impression of desired content by combining the shapes of the parts corresponding to the part values calculated by the part value calculating means. The content search device according to 1.

Images