JP2001067364A - Complex media retrievalmethod and medium recording program for complex media retrieval - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a data base management system by which retrieval across plural media is easily performed. SOLUTION: This complex medium retrieval method inputs an inquiry sentence 10 in which conditions across plural media are specified by an inquiry analysis part 20 in a data base management system 2 managing data regarding the plural media such as a text, an image and music, and analyzes the inputted inquiry sentence 10. The degree of similarity to a key specified per medium is calculated by a retrieval execution par 30, retrieval is executed, the comprehensive degree of similarity of each retrieval result is calculated from the calculated degree of similarity per medium and the retrieval result is outputted on the basis of the calculated comprehensive degree of similarity.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

TECHNICAL FIELD The present invention relates to text, images,
In a database management system capable of managing data related to a plurality of media such as music, a composite media search method that enables a condition to be specified not only for a single medium but also for a plurality of media, and performs a search based on the condition for a plurality of media. .

[0002]

2. Description of the Related Art FIG. 6 is a diagram for explaining an example of searching a multimedia database according to the prior art.

Conventionally, a multimedia database (D
B) When performing a search across a plurality of media with respect to 53, as shown in FIG. 6A, the search application program (AP) 51 issues a query for a text (Text), a query for an image (Img), Music
A search request that specifies a closed condition such as an inquiry of a single medium is sent to a database management system (DBMS) 5.
2, and the query had to be repeated as many times as the number of media to be searched.

A conventional database management system 5
In No. 2, it was impossible to perform a search in which a relative ratio was added to the condition of each medium as shown in FIG. For this reason, the result of the search (composite media search) in consideration of the specified condition over a plurality of media cannot be returned from the database management system 52 to the search application program 51, and the search within the search application program 51 is performed. The results had to be recalculated.

[0005]

For example, when searching a music information database that manages text information such as a song title, a composer name, lyrics and the like and music information such as a melody for various songs. In addition to searching for the melody from the song title or searching for the song name from a part of the melody, there is also a demand that the user wants to search for the song name from a part of the lyrics that are memorized and a part of the melody. There is.

At this time, the memory of the lyrics is certain, but the memory of the melody is unclear, or conversely, the memory of the lyrics is uncertain, but the memory of the melody is clear. The searcher inputs the key to be searched by voice input of the melody with lyrics or text input from the keyboard and voice input such as humming, but can also specify the importance of the text and music media keys. If possible, it will be useful for improving search accuracy.

SUMMARY OF THE INVENTION The present invention solves the above problems and interprets a condition across multiple media and executes a search in order to realize a composite media search required by a multimedia search application. It is intended to be realized in the management system.

[0008]

According to the present invention, there is provided a database management system for managing data relating to a plurality of media such as text, images, music, etc., in which a query over a plurality of media is specified. Input a sentence, analyze the input query sentence, calculate the similarity with the key specified for each media, execute a search, and execute the search based on the similarity calculated for each media. The main feature is to calculate the overall similarity and to output a search result based on the calculated overall similarity.

[0009] In addition, in the query, it is possible to specify a condition over a plurality of media and a relative ratio of the condition of each medium. In the calculation of the overall similarity, the similarity of each medium is inquired. It is characterized in that weighting is performed in accordance with a relative ratio specified by a sentence to calculate an overall similarity.

A program for realizing the above processing of the database management system by a computer can be stored in a computer-readable recording medium such as a portable medium memory, a semiconductor memory, and a hard disk.

[0011]

FIG. 1 shows an example of a system configuration of a database management system for executing the present invention. In the figure, 1
Is a search application program (AP) for processing a search request from a user, and 10 is a query sentence described in a database language such as SQL.

Database management system (DBMS) 2
Is a query analysis unit 20 having a language processing function for interpreting a query to which a condition and a relative ratio are added for a plurality of media;
Search execution unit 3 that realizes a search function for executing the query
It has 0. The database (DB) 3 stores data on a plurality of media such as texts, images, and music in a data format according to the media.

A query analysis unit 20 of the database management system 2 inputs a query sentence 10 from the search application program 1 and extracts a condition for each medium, and a condition extraction unit 21 for each medium, and a condition ratio (relative ratio) for each medium. ) To extract the condition ratio extracting unit 22.

The search execution unit 30 performs a search on the database 3 corresponding to each medium.
1, an image search unit 33, and a music search unit 35. From the similarities of the individual search results calculated by these similarity calculation units 32, 34, and 36, the relative ratio extracted by the condition ratio extraction unit 22 is calculated. It has an overall similarity calculating unit 37 that calculates the overall similarity according to the following.

The search execution unit 30 determines the degree of similarity with the key specified by the text search unit 31, image search unit 33, and music search unit 35 for each medium without considering the relative ratio. Database 3 by calculating
Perform a search. Next, the overall similarity calculating unit 37 calculates
The similarity is recalculated by adding the relative ratio to the search result for each medium, and a comprehensive search result is created.

FIG. 2 is a diagram for explaining the outline of the processing of the database management system 2 shown in FIG.

As shown in FIG. 2A, the user sets the ratio of search conditions such as the degree of similarity to a key given to each medium, for example, 3 for text, 2 for image, and 1 for music. Weighting can be performed to give a search request instruction to the search application program 1. The search application program 1 satisfies (Text) :( Img) :( Music) = 3: 2:
A query sentence for a query covering a plurality of media including information of a relative ratio such as 1 is created and issued to the database management system 2.

In the database management system 2, FIG.
As shown in (B), the similarity with the designated key is calculated for each medium, and then each similarity is normalized as necessary, and a linear sum or a non-linear sum is calculated from each similarity and the relative ratio. Calculate and calculate the overall similarity. The calculated values are sorted, a search result is created, and the search result is returned to the search application program 1. The search application program 1 presents search results to the user in ascending order of overall similarity.

FIG. 3 is a processing flowchart of the database management system 2. A detailed description of known parts similar to those in the related art is omitted. First, a query request is received from the search application program 1 by a query statement 10 in which a condition, a key, and a relative ratio are specified (S
1). The query analysis unit 20 analyzes the syntax of the query sentence 10 (S2), and notifies the search application program 1 of a syntax error, if any. If the syntax is correct, a detection condition for each medium is extracted (S3). Also,
The condition ratio designated as the relative ratio is extracted (S4).

Next, search processing is distributed for each medium (S
5), the search of the database 3 is performed by calculating the similarity with the key in each of the text search unit 31, the image calculation unit 33, the music search unit 35,.
8, ...).

When the search for each medium is completed, the search results for each medium are sorted by similarity, and then the similarity is normalized (S9). This normalization is performed to match the scale because the expression of the similarity differs depending on the media. For example, the normalization is calculated by the following formula from the highest and lowest possible similarities for each medium.

Similarity after normalization = | Similarity before normalization−
The lowest value | ÷ | the highest value-the lowest value | With this normalization, for example, the closer the music is to 0, the higher the similarity is, and the larger the value of the text is, the higher the similarity is, the same comparison is made. Will be possible.

Next, a linear sum is calculated from each of the normalized similarities and the designated condition ratio (relative ratio) to calculate an overall similarity (S10). Similarity T _i of text for a single search result i, the image similarity is I _i, similarity music and was M _i, the relative ratio, "Text: Image: Music = w _T: w _I : w _M ”, the overall similarity has the following value.

Total similarity of search result i = w _T × T _i + w
_I × I _i + w _M × M _i Next, the entire search results are sorted by the calculated overall similarity (S11), and a list of search results having a total similarity equal to or more than a predetermined value or a predetermined number from the top is listed. Then, it is returned to the search application program 1 of the request source (S12).

[0025]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment in which the present invention is applied to a music search system will be described below as an embodiment of the present invention. FIG. 4 shows an example of a music search database, and FIG. 5 shows an example of an inquiry to the database.

The database 3 shown in FIG.
Relational database management system (ORDB
(MS). Music class (parent class) for music and lyric information of the whole song
Is stored in The feature amount of the music piece extracted from the whole music is stored in the music piece (child class).

A part of the melody is input by humming from the user to the search application program 1 of the present system, a part of the lyrics is specified by text, and an inquiry "What is a song similar to them?" I do. Some of the lyrics are entered as text from a keyboard, etc.
What is input by voice by the search application program 1 may be converted into text data by voice recognition.

In response to the search request, the search application program 1 inquires the user of the relative ratio between the text and the music condition through conversation processing with the user. Here, it is assumed that “Text: Music = 3: 1” is specified. The relative ratio is not determined by the user, but is automatically determined internally by the search application program 1 in accordance with the search environment such as the data amount of each medium in the database 3 and the key input method. Is also good.

From the above results, the query application program 1 generates a query statement 10 described in an extended SQL statement as shown in FIG. In this query statement 10, similarity is a method and WHERE
The overall similarity of the condition specified in the E clause is calculated. s
imilar is an operator that determines whether or not the feature amounts of the tone information described on the right side and the left side are similar. In this example, the left side shows the attribute stored in the music piece class, and the right side shows the MID input by the user by humming.
This is data of the I standard. This data is 'humming.mid'
Stored in a file named.

"Includes" is an operator for determining whether or not the sentences described on the right and left sides are similar. In this example, the left side shows the attributes stored in the music class, and the right side is the text input by the user.

Includes (3), similar
The description (1) indicates that the relative ratio between the text and music conditions is 3: 1.

When the query 10 is input to the database management system 2, the database management system 2
Performs the following processing. First, as a music piece search, "y.
Calculate the feature value similar'humming.mid '
Create a list of x. song names sorted by rity. Also, calculate “x. Lyrics includes 'Sakura Sakura'” in the WHERE clause and sort by x.similarity.
Create a list of song titles. A weight corresponding to the relative ratio specified by the user is added to the similarity in the list of x. Music titles created above and above, and the similarity as the overall similarity is recalculated. The above results are sorted by similarity and returned to the search application program 1.

As a result, as in the search result 4 shown in FIG. 5, the information on the song title and its similarity is delivered to the search application program 1 in the order of the highest similarity.
The search application program 1 notifies the user of the result through a display screen or a speaker.

[0034]

As described above, according to the present invention,
Since search over multiple media is realized in the database management system, complicated search operations on multimedia data by the search application program are not required, and a search application program with high search efficiency can be easily created. become. Also, by searching across multiple media,
The narrowing-down efficiency is improved compared to the search in which a single medium is specified, and the search accuracy can be improved particularly by specifying the condition ratio of each medium search.

[Brief description of the drawings]

FIG. 1 is a diagram illustrating a system configuration example of a database management system that executes the present invention.

FIG. 2 is a diagram illustrating an outline of processing of a database management system.

FIG. 3 is a processing flowchart of a database management system.

FIG. 4 is a diagram showing an example of a music search database.

FIG. 5 is a diagram showing an example of an inquiry to a music search database.

FIG. 6 is a diagram illustrating a conventional technique and a problem of the present invention.

[Explanation of symbols]

 Reference Signs List 1 search application program 2 database management system 3 database 10 query sentence 20 query analysis unit 21 condition extraction unit for each medium 22 condition ratio extraction unit 30 search execution unit 31 text search unit 32 similarity calculation unit 33 image search unit 34 similarity calculation Unit 35 music search unit 36 similarity calculation unit 37 total similarity calculation unit

Continuation of the front page (72) Inventor Junichi Kuroiwa 2-3-1 Otemachi, Chiyoda-ku, Tokyo Inside Nippon Telegraph and Telephone Corporation (72) Inventor Takashi Yoshinishi 2-3-1 Otemachi, Chiyoda-ku, Tokyo F-term (reference) in Nippon Telegraph and Telephone Corporation 5B075 ND16 PP02 PP12 PP23 PP30 PQ02 PQ46 PQ74 PR06 QM08 QP10

Claims (4)

[Claims] 1. A composite media search method in a database management system for managing data relating to a plurality of media, comprising the steps of: inputting and analyzing a query sentence specifying a condition over a plurality of media; and analyzing the query sentence. Based on the results, the process of calculating the similarity with the specified key for each media and executing the search, and calculating the overall similarity of each search result from the similarity calculated for each media And a step of outputting a search result based on the calculated overall similarity. 2. In the step of analyzing the query sentence, a condition over a plurality of media and a relative ratio of the condition of each medium are extracted, and in the step of calculating the overall similarity, the similarity of each medium is calculated. 2. The composite media search method according to claim 1, wherein a total similarity is calculated by performing weighting according to the relative ratio. 3. A recording medium recording a program for searching for a composite medium in a database management system for managing data relating to a plurality of media,
A process of inputting and analyzing a query sentence specifying conditions over a plurality of media, and performing a search by calculating a similarity with a key specified for each media based on the analysis result of the query sentence Processing, calculating the overall similarity of each search result from the similarity calculated for each medium, and outputting the search result based on the calculated overall similarity. A program recording medium for a composite media search, wherein a program to be executed is recorded. 4. A process for analyzing the query sentence, extracting a condition over a plurality of media and a relative ratio of a condition of each medium, and calculating the overall similarity in the process of analyzing the similarity for each medium. 4. The composite medium search program recording medium according to claim 3, wherein a program for calculating a total similarity by weighting each time according to the relative ratio is recorded.