JP3500097B2 - Composite media search method and composite media search program recording medium - Google Patents

Description

Detailed Description of the Invention

[0001]

TECHNICAL FIELD The present invention relates to text, images,
In a database management system capable of managing data related to multiple media such as music, it is possible to specify not only a single media but also a condition that spans multiple media, and relates to a composite media search method that executes a search based on the condition that spans multiple media. .

[0002]

2. Description of the Related Art FIG. 6 is a diagram for explaining an example of retrieval of a multimedia database according to a conventional technique.

Conventionally, multimedia databases (D
B) 53, when performing a search across a plurality of media, as shown in FIG. 6A, the search application program (AP) 51 makes a text (Text) inquiry, an image (Img) inquiry, Music
Database management system (DBMS) 5
It was necessary to repeat this procedure for 2 and repeat the inquiry for the number of media to be searched.

The conventional database management system 5
In No. 2, it was impossible to perform a search in which a relative ratio was added to the conditions of each medium as shown in FIG. 6 (B). Therefore, it is not possible to return the result of the search (complex media search) considering the specified conditions across a plurality of media from the database management system 52 to the search application program 51, and the search is performed in the search application program 51. The result had to be recalculated.

[0005]

For example, in the case of searching a music information database that manages various pieces of music by associating text information such as song titles, composer names, and lyrics with music information such as melodies. In addition to searching for a melody from a song name, or searching for a song name from a part of a melody, there is also a request to search for a song name from a part of the lyrics and a part of the melody that are vaguely stored. There is.

At this time, the memory of the lyrics is certain, but the memory of the melody is uncertain, or the memory of the lyrics is uncertain, but the memory of the melody is clear. The searcher inputs the key to be searched by voice input of the melody with lyrics, or text input from the keyboard and voice input such as humming, but the importance of the key of the text and music media can also be specified. If possible, it would be useful for improving search accuracy.

In order to solve the above-mentioned problems and to realize a composite media search required by a multimedia search application, the present invention interprets a condition across a plurality of media and executes a search in a database. It is intended to be realized within the management system.

[0008]

In order to solve the above-mentioned problems, the present invention provides an inquiry in a database management system that manages data relating to a plurality of media such as text, images, music, etc., in which a condition across a plurality of media is specified. Enter a sentence, analyze the entered query sentence, calculate the similarity with the specified key for each medium, execute the search, and calculate the similarity of each search result from the similarity calculated for each medium. The main feature is that the overall similarity is calculated and the search result is output based on the calculated overall similarity.

In addition, in the query statement, it is possible to specify the relative ratio of the conditions of each medium as well as the condition that the conditions span a plurality of media, and in the calculation of the overall similarity, inquire the similarity of each medium. The feature is that the overall similarity is calculated by performing weighting according to the relative ratio designated by the sentence.

The program for realizing the above processing of the database management system by a computer can be stored in an appropriate recording medium such as a computer-readable portable medium memory, a semiconductor memory, a hard disk.

[0011]

1 shows an example of the system configuration of a database management system for carrying out the present invention. 1 in the figure
Is a search application program (AP) that processes a search request from a user, and 10 is an inquiry sentence described in a database language such as SQL.

Database management system (DBMS) 2
Is a query analysis unit 20 having a language processing function for interpreting a query in which conditions and relative ratios for a plurality of media are added,
Search execution unit 3 that realizes a search function for executing the inquiry
Has 0. The database (DB) 3 stores data relating to a plurality of media such as texts, images and music in a data format according to the media.

The query analysis unit 20 of the database management system 2 inputs the query sentence 10 from the search application program 1 and extracts a condition for each medium, and a condition extraction unit 21 for each medium and a condition ratio (relative ratio) for each medium. ) And a condition ratio extraction unit 22 for extracting

The search execution unit 30 is a text search unit 3 that searches the database 3 for each medium.
1. The image search unit 33 and the music search unit 35 are provided, and the relative ratio extracted by the condition ratio extraction unit 22 is calculated from the similarity of the individual search results calculated by the similarity calculation units 32, 34, and 36. It has a total similarity calculation unit 37 for calculating the total similarity.

First, the search execution unit 30 does not consider the relative ratio, and determines the similarity with the key designated by the text search unit 31, the image search unit 33, and the music search unit 35 for each medium. Database 3 by calculating
Search for. Next, by the total similarity calculation unit 37,
A relative ratio is added to the search results for each media, the similarity is recalculated, and a comprehensive search result is created.

FIG. 2 is a diagram for explaining the outline of the processing of the database management system 2 shown in FIG.

As shown in FIG. 2A, the user determines the ratio of search conditions such as the degree of similarity with the key given to each medium, for example, text is 3, image is 2, music is 1. The search request instruction can be given to the search application program 1 by weighting. The search application program 1 has (Text) :( Img) :( Music) = 3: 2: with conditions and keys for each medium.
A query statement of a query that covers a plurality of media including information of relative ratio such as 1 is created and issued to the database management system 2.

In the database management system 2, FIG.
As shown in (B), the similarity with a specified key is calculated for each medium, and then each similarity is normalized as necessary, and a linear sum or a nonlinear sum is calculated from each similarity and the relative ratio. Then, the total similarity is calculated. The calculated values are sorted, search results are created, and returned to the search application program 1. The search application program 1 presents the search results to the user in descending order of total similarity.

FIG. 3 is a processing flowchart of the database management system 2. Detailed description of known parts similar to those of the conventional art will be omitted. First, an inquiry request by the inquiry sentence 10 in which a condition, a key, and a relative ratio are specified is received from the search application program 1 (S
1). The query analysis unit 20 analyzes the syntax of the query sentence 10 (S2), and if there is a syntax error, notifies the search application program 1 to that effect. If the syntax is correct, the detection condition for each medium is extracted (S3). Also,
The condition ratio designated as the relative ratio is extracted (S4).

Next, the search process is distributed to each medium (S
5), the text search unit 31, the image calculation unit 33, the music search unit 35, ... Are searched for the database 3 by calculating the similarity with the key (S6, S7, S).
8, ...).

When the search for each medium is completed, the search results for each medium are sorted by the degree of similarity, and then the degree of similarity is normalized (S9). This normalization is performed in order to match the scale because the expression of the degree of similarity differs depending on the medium, and is calculated from the maximum value and the minimum value of the degree of similarity that can be taken for each medium, for example, using the following formula.

Similarity after normalization = | Similarity before normalization−
Minimum value | ÷ | Maximum value-Minimum value | By this normalization, even when music has a higher similarity as it approaches 0 and text has a higher similarity as the value is higher, comparison can be performed using the same criteria. It will be possible.

Next, a linear sum is calculated from each normalized similarity and the designated condition ratio (relative ratio) to calculate a total similarity (S10). It is assumed that the text similarity is T _i , the image similarity is I _i , and the music similarity is M _i with respect to one search result i, and the relative ratio is “text: image: music = w _T : w”. _I : w _M ”, the total similarity becomes the following value.

Total similarity of search result i = w _T × T _i + w
_I × I _i + w _M × M _i Next, the entire search results are sorted according to the calculated total similarity (S11), and the total similarity is a predetermined value or more, or a predetermined number of search results from the top are listed. Then, it is returned to the requesting search application program 1 (S12).

[0025]

EXAMPLE An example in which the present invention is applied to a music search system will be described below as an example of the present invention. FIG. 4 shows an example of a music search database, and FIG. 5 shows an example of an inquiry to the database.

The database 3 shown in FIG.
Relational database management system (ORDB
It is a database searched by MS). Music information (lyrics) and lyrics information for the entire song is in the music class (parent class)
It is stored in. The feature amount of the music piece extracted from the entire song is stored in the music piece (child class).

When the user inputs a part of the melody by humming and a part of the lyrics by text to the search application program 1 of the present system, an inquiry is made asking "What songs are similar to them?" To do. Some of the lyrics are entered in text from the keyboard,
What is input by voice by the search application program 1 may be converted into text data by voice recognition.

In response to this search request, the search application program 1 inquires of the user about the relative ratio of the text and music conditions through conversation processing with the user. Here, it is assumed that “text: music = 3: 1” is designated. It should be noted that this relative ratio is automatically determined internally by the search application program 1 according to the search environment such as the amount of data for each medium held by the database 3 and the key input method, rather than inquiring to the user. Good.

From the above result, the retrieval application program 1 generates the inquiry sentence 10 described in the extended SQL sentence as shown in FIG. In this query sentence 10, simplicity is a method and WHER
Compute the overall similarity of the conditions specified by the E clause. s
The imirar is an operator that determines whether the feature quantities of the musical tone information described on the right side and the left side are similar. In this example, the left side shows the attributes stored in the music piece class, and the right side shows the MID that the user has humming input.
It is data of I standard. This data is'humming.mid '
It is stored in a file named.

Includes is an operator that determines whether the sentences written on the right side and the left side are similar. In this example, the left side shows the attributes stored in the music class, and the right side is the text text input by the user.

Includes (3), similar
The description (1) indicates that the relative ratio between the text and music conditions is 3: 1.

When this inquiry sentence 10 is input to the database management system 2, the database management system 2
Performs the following processing. First, as a music piece search, "y.
Feature quantity similar'humming.mid '”is calculated and x.simila
Create a list of x. song titles sorted by rity. Also, "x. Lyrics includes'Sakura Sakura '" in the WHERE clause is calculated and sorted by x.similarity.
Create a list of song titles. The weight according to the relative ratio specified by the user is added to the similarity in the list of x. Song names created in the above and above, and the similarity as the total similarity is recalculated. The above results are sorted by similarity and returned to the search application program 1.

As a result, as in the search result 4 shown in FIG. 5, the information on the music title and its similarity is passed to the search application program 1 in descending order of similarity.
The search application program 1 notifies the user of the result through the display screen or the speaker.

[0034]

As described above, according to the present invention,
Since a search that spans multiple media is realized in the database management system, a complicated search operation for multimedia data by a search application program is not required, and a search application program with good search efficiency can be easily created. become. Also, by searching across multiple media,
Narrowing efficiency is improved compared to a search that specifies a single media, and it is possible to improve search accuracy by enabling the specification of the condition ratio for each media search.

[Brief description of drawings]

FIG. 1 is a diagram showing an example of a system configuration of a database management system that executes the present invention.

FIG. 2 is a diagram illustrating an outline of processing of a database management system.

FIG. 3 is a processing flowchart of a database management system.

FIG. 4 is a diagram showing an example of a database for music search.

FIG. 5 is a diagram showing an example of an inquiry to a music search database.

FIG. 6 is a diagram for explaining the problems of the related art and the present invention.

[Explanation of symbols]

1 Search application program 2 Database management system 3 database 10 inquiry text 20 Query analysis section 21 Condition extractor for each media 22 Condition ratio extractor 30 Search execution unit 31 Text Search Section 32 similarity calculator 33 Image Search Department 34 Similarity calculation unit 35 Music Search Department 36 similarity calculator 37 Total Similarity Calculation Unit

─────────────────────────────────────────────────── ─── Continuation of front page (72) Inventor Takashi Yoshinishi 2-3-1, Otemachi, Chiyoda-ku, Tokyo Inside Nippon Telegraph and Telephone Corporation (56) References Masayoshi Umeda, for query language in image search processing Request, IPSJ Research Report (98-DBS-114), January 19, 1998, Vol. 98, No. 2, p. 1-5 (58) Fields investigated (Int.Cl. ⁷ , DB name) G06F 17/30 JISST file (JOIS)

Claims (2)

(57) [Claims] 1. At least text, image or music
This includes multiple media with different data types.
The data is associated for multiple media et
A method for searching composite media in a database management system for managing composite media to be searched, comprising inputting and analyzing a query statement in which a condition spanning a plurality of media is specified, and comparing the conditions of each media.
Specifically the process of extracting the ratio of, on the basis of the analysis result of the query statement, the process of executing a search by calculating a similarity between the key specified for each medium, are calculated the each media The relative ratio to the similarity
A composite medium characterized by having a process of calculating the overall similarity of each search result by weighting according to the rate, and a process of outputting the search result based on the calculated overall similarity. retrieval method. 2. At least text, image or music
This includes multiple media with different data types.
The data is associated for multiple media et
In database management system for managing the composite media to search a recording medium for recording a program for searching the composite media, as well as enter a query statement that condition spanning the plurality of media are specified analysis, the Relative media conditions
Specifically the process of extracting the ratio of, on the basis of the analysis result of the query statement, a process of performing a search by calculating a similarity between the key specified for each medium, are calculated the each media The relative ratio to the similarity
Record a program that causes a computer to execute the process of calculating the overall similarity of each search result by weighting according to the rate and the process of outputting the search result based on the calculated overall similarity. A program recording medium for searching a complex media, characterized by