KR20100064136A

KR20100064136A - Multimedia data search method

Info

Publication number: KR20100064136A
Application number: KR1020080122567A
Authority: KR
Inventors: 한도근
Original assignee: 현대자동차주식회사
Priority date: 2008-12-04
Filing date: 2008-12-04
Publication date: 2010-06-14

Abstract

The present invention does not use a separate keyword text input means when searching for a desired music, video, video on demand, etc. can be used during driving to provide a convenient driving environment, delivery of a search query, indexing server 20 ), And the transmission of the search results to the telematics terminal 30 are all performed in a short form, thereby minimizing waste of resources due to network interworking due to unnecessary waiting time.

Description

Multimedia data search method

The present invention relates to a method for retrieving multimedia data, and more particularly, it is possible to provide a convenient driving environment because it can be used while driving without using a text input means for a keyword when searching for music, video, video on demand, and the like. The present invention relates to a method for retrieving multimedia data.

Recently, due to the development of multimedia technology, the usage of multimedia data in each field is increasing.

The multimedia data retrieval method according to the prior art uses a text based indexing method. The text search method is a method of searching text by inputting a desired keyword by indexing features of multimedia data into text. Such a text search method requires a driver to input by using a separate keyword text input means while driving, there is a risk of an accident, a feature is inefficiently extracted, and a considerable amount due to the limitation that does not contain all the contents of the multimedia data. Gets wrong multimedia search results.

In addition, text-based search engines require a considerable amount of time to search and their accuracy decreases as the content increases. When the multimedia service is performed through wireless communication, there is a problem that an excessive communication fee is added due to the increase of search time due to the above problem, which may cause a large burden on the user.

An object of the present invention is to provide a multimedia data retrieval method that can provide a convenient driving environment can be used during driving do not use a separate text input means for keywords when searching for desired music, video, video on demand, etc. do.

In addition, the present invention is a multimedia data that can minimize the waste of resources due to network interworking caused by unnecessary waiting time because the delivery of the search query, the search of the indexing server, the delivery of the search results to the telematics terminal is all performed in a short form. It is an object to provide a search method.

In the multimedia data retrieval method according to the present invention, a feature indexing file is generated by extracting audio characteristic values of multimedia data stored in a data server, and the feature indexing file and link information for accessing the corresponding original multimedia data are stored in an indexing server. Making; Generating a query when the driver inputs a search keyword by humming or general voice; Generating a query indexing file by dividing whether the query is a hum or a general voice and extracting audio characteristic values or lylics through digital signal processing according to a case; Providing the driver with a result of the comparison with the feature indexing file stored in the indexing server based on the query indexing file; And when the driver selects the result value corresponding to the desired multimedia data from the result values, corresponding driver multimedia data stored in the data server is provided to the driver by the link information corresponding to the result value. It includes.

In addition, the query indexing file is generated by implementing a voice-to-text function for storing the general voice as text when the query is the general voice.

The feature indexing file is a lyrics composed by directly extracting lyrics of a video including a feature index extracted through digital signal processing of music itself, a text index composed of a brief description made by operators and users, and voice information. (lylics) has an indexing array of indexes,

Forwarding of the query, searching of the indexing server, and forwarding of the result value are performed in short form,

Extracting feature data for generating a feature indexing file from an audio signal through a feature extraction method for extracting an audio feature value; And formatting and standardizing the feature data into the feature indexing file that can be entered into a search tool,

Retrieving the feature indexing file comprises implementing the feature data in a cluster by size; Extracting a group having a specific size by searching for each size; And performing a rescan using the hourly index in the group.

The present invention does not use a text input means for a keyword when searching for a desired music, video, video on demand, etc., which can be used during driving, thereby providing a convenient driving environment.

In addition, the present invention has the effect that it is possible to minimize the waste of resources due to network interworking caused by unnecessary latency because all of the delivery of the search query, the search of the indexing server, the delivery of the search results to the telematics terminal is performed in short form. have.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. However, the invention is not limited to the embodiments described herein but may be embodied in other forms. Rather, the embodiments introduced herein are provided so that the spirit of the present invention is thoroughly and completely disclosed, and the spirit of the present invention to those skilled in the art will be fully delivered. Also, like reference numerals denote like elements throughout the specification.

1 is a block diagram showing an apparatus for retrieving multimedia data according to the present invention.

The multimedia data retrieval apparatus includes a data server 10, an indexing server 20, and a vehicle 30, each of which is wired or wirelessly connected to each other.

The data server 10 includes a multimedia database 12 that stores multimedia data, and extracts audio characteristic values of all multimedia data stored in the multimedia database 12 using a search engine to generate a feature indexing file. In addition, feature indexing files and link information of the original multimedia data are generated.

The indexing server 20 includes an indexing database 22 that stores feature indexing files and link information. When the search query is transmitted from the vehicle 30, the indexing server 20 uses the search engine to set audio characteristics of the search query. Extract and generate a query indexing file, search the indexing database 22 using the query indexing file, and transmit the search result value to the vehicle 30.

The vehicle 30 includes a telematics terminal 32 that communicates with the data server 10 and the indexing server 20, and generates a search query when the driver inputs a search keyword using general voice, humming, or the like. ), The search result value is received from the indexing server 20 and provided to the driver to select the multimedia data desired by the driver.

Here, multimedia data includes images, images, music, sound, etc., humming refers to a nasal music technique with a closed mouth, and a search query refers to a query that defines data in a database to search a database. do.

2 is a flowchart illustrating a method of retrieving multimedia data according to the present invention.

First, the indexing server 20 generates a feature indexing file by extracting audio feature values of multimedia data stored in the data server 10 using a search engine (S10). That is, feature indexing files are generated by extracting audio characteristic values of all multimedia data stored in the multimedia database 12 of the data server 10.

The feature indexing file and the link information for accessing the corresponding original multimedia data are stored in the indexing database 22 of the indexing server 20 (S20).

Afterwards, when the driver inputs a search keyword by voice, hum, or the like using an input means such as a keyboard or a microphone, a search query generated by the telematics terminal 32 of the vehicle 30 is transmitted to the indexing server 20 using a wireless network. It is delivered (S30).

The indexing server 20 generates a query indexing file by extracting audio characteristic values through digital signal processing on a search query input using a search engine (S40).

The feature indexing file stored in the indexing database 22 is searched based on the query indexing file generated as described above, and the comparison review result is transmitted to the telematics terminal 32 of the vehicle 30 (S50).

The telematics terminal 32 provides the transmitted result to the driver, determines whether there is data on the multimedia data desired by the driver among the result values (S60), and outputs a result value corresponding to the multimedia data desired by the driver from the result values. When selected (S70), the corresponding original multimedia data stored in the multimedia database 12 of the data server 10 by the link information for accessing the corresponding original multimedia data is transmitted to the telematics terminal 30 through the wireless network It is provided to the driver (S80).

In addition, the multimedia data retrieval method according to the present invention may search for multimedia data and provide it to the driver even when the driver inputs a text-based search keyword through the keyword input means. Here, the text input by the driver may search for a lylics index formed by extracting a text index composed of a description made by an operator or users, and a dialogue of a video including a voice information and lyrics of a song.

In general, in the case of a movie or a video including voice, voice information is virtually impossible to index, so if a dialogue is found in a certain part of a video, it cannot be searched using a conventional text-based or audio / video content search method.

Accordingly, the present invention implements a voice-to-text function for storing voice information of multimedia data as text. Here, the voice-to-text function is an application example of speech recognition and an algorithm for extracting text from speech information on media.

When the driver inputs a search text using keyword input means such as a touch screen, the telematics terminal 32 generates a search query and transmits the search query to the indexing server 20 using a wireless network.

A search indexing file is generated by extracting audio characteristic values through digital signal processing on a search query input using a search engine.

The feature indexing file stored in the indexing database 22 is searched based on the generated query indexing file and the comparison review result is transmitted to the telematics terminal 32 of the vehicle 30.

The telematics terminal 32 provides the transmitted result to the driver, determines whether there is data on the multimedia data desired by the driver among the result values (S60), and outputs a result value corresponding to the multimedia data desired by the driver from the result values. When selected, the corresponding original multimedia data stored in the multimedia database 12 of the data server 10 is transmitted to the telematics terminal 30 through the wireless network by the link information for accessing the corresponding original multimedia data. It is provided (S80).

The above-described multimedia data retrieval method of the present invention shows an embodiment of inputting a voice, hum or text, but can be fused to generate and implement a single query indexing file.

At this time, the feature indexing file is composed by directly extracting the feature index extracted through the signal processing of the music itself, a text index consisting of briefly written descriptions by operators and users, dialogue of the video including voice information, song lyrics, and the like. It has an indexing array composed of lylics indices. Thus, the search accuracy can be improved and various search channels can be provided. Here, the feature index may be used when not knowing the lyrics but knowing the pitch or part of the melody, and the lyrics index may be used when not knowing the pitch or melody but knowing the part or lyrics of the part.

In the present invention, both the delivery of the search query and the delivery of the search result to the telematics terminal 30 are performed in a short form, thereby minimizing waste of resources due to unnecessary waiting time. In other words, after transmitting the search query, the communication between the telematics terminal 32 and the indexing server 20 is cut off, and the search result in the indexing server 20 is transmitted to the telematics terminal 32 as if it is a short message so that it can be cooked. It is possible to prevent unnecessary waste of resources and resources due to network interworking.

In the above-described embodiment of the present invention, if feature data is extracted from an audio signal through a feature extraction method that extracts audio feature values using a search engine, this is a raw data level. In other words, the form or data format is unstructured for search engine tools. Therefore, feature data should be formatted and standardized into feature indexing files that can be entered into search engine tools. In other words, since the feature data output through digital signal processing vary only in magnitude, they should be formatted into a feature indexing file having a specific value (for example, a value between 0 and 1).

In this way, after each feature data is formatted into a feature indexing file, feature indexing files should be arranged and assigned to a search engine to shorten the search time to ensure an appropriate level of accuracy and search time.

And since the feature indexing files are truncated by time unit, if they are simply arranged in chronological order, it takes tremendous search time to search if the characteristics of the data entered by the query are located in the latter part of the multimedia data. Therefore, the feature indexing files have their respective indexes over time, but the search time is shortened only if they are grouped into clusters of similar size, not hourly. In other words, after extracting the cluster group having similar size by searching the indexing files by size, the scope of the search result should be narrowed through re-search using the hourly index in the cluster group.

It will be apparent to those skilled in the art that various modifications, additions, and substitutions are possible, and that various modifications, additions and substitutions are possible, within the spirit and scope of the appended claims. As shown in Fig.

10: data server

12: Multimedia Database

20: indexing server

22: Indexing Database

30: telematic unit

Claims

Storing, by the indexing server, the feature indexing file generated by extracting audio characteristic values of the multimedia data stored in the data server using a search engine and link information for accessing the original multimedia data;

Generating, by the indexing server, a search query corresponding to a search keyword of a driver's hum or voice information input from a vehicle;

Generating, by the indexing server, a query indexing file by extracting audio characteristic values of the search query using a search engine;

Providing, by the indexing server, a result value of the comparison with the feature indexing file based on the query indexing file to the vehicle; And

When the indexing server selects the result value corresponding to the desired multimedia data from the result value, providing the vehicle with corresponding original multimedia data stored in the data server by the link information corresponding to the result value. Multimedia data retrieval method comprising a.

The method according to claim 1,

The feature indexing file may further include a lylics index configured by extracting a text index consisting of a description written by an operator or users and lyrics of a video including a voice information and lyrics of a song.

The method according to claim 2,

The search engine includes a voice-to-text function for storing the general voice or the voice information as text.

The method according to claim 1,

And transmitting the search query and the result value in a short form.

The method according to claim 1,

Implementing the feature indexing files into clusters by size;

Extracting a cluster group corresponding to the size of the query indexing file among the clusters for each size; And

And rescanning using the hourly index in the cluster group.