KR20040094252A - Music searching system using personal digital assistance - Google Patents

Abstract

PURPOSE: A system for searching a song of the Karaoke using a PDA(Personal Digital Assistant) is provided to search the song including a melody directly inputted from a user through a microphone without using information such as a title and a singer of the song. CONSTITUTION: A query input melody feature extracting module(31) extracts a query input melody feature, which is a form needed for search, from an audio signal inputted through the microphone of the PDA. A song melody feature database(32) stores melody features extracted from each song file. A similarity calculating module(33) searches the song melody feature database by using the query input melody feature. A user interface(34) operates each function for searching the song, outputting a search result, and operating the Karaoke. An infrared output signal correction module(35) converts the operation of the user interface into a signal form receivable to an infrared signal receiver of the Karaoke.

Description

Song retrieval system of song semi-period using personal mobile device {MUSIC SEARCHING SYSTEM USING PERSONAL DIGITAL ASSISTANCE}

The present invention relates to a system for searching for a song stored in a song repeater, by searching for a song of a song repeater based on a melody using a personal portable terminal.

Searching for multimedia data such as movies, photos, and music is performed by searching for bibliographic information such as creator information, title, creation date, etc., and searching based on multimedia contents. have.

However, the former search method is often unable to fully reflect the user's search intent. For example, if you want to find a picture with a blue sky on a white beach, you can't search if there is no bibliographic information describing the landscape. However, the latter retrieval method describes color information, which is information that can be automatically extracted by the image processing technique from the photo itself, as feature information for content-based retrieval of the photo. The desired picture can be retrieved by comparing the feature information extracted from the query image with the feature information of the search target image.

When a user searches for multimedia in this way, if the user searches using the contents of the multimedia, the user can conveniently search more precisely according to the user's search intention. In fact, users may try to search by using information such as the title and creator of the multimedia, but in the case of a video, movement, color information, color, shape information in a photo, music speed, mood information, etc. You can have a lot of interest. Content information can be directly input by multimedia producers or automatically extracted from the information to build a database for searching.

In the case of content-based multimedia search in the above-described content-based multimedia search, if a content query other than bibliographic information (title, player, composer, publication year, etc.) can be made, a search more suited to the search intention can be performed. In the case of music, after listening to some music, the image, the melody, and the speed of the song are memorized, which is easier to remember and longer than the bibliographic information of the music. Therefore, when searching for music, it is efficient to search for music using this content information. Content-based music search can be divided into four types according to the type of query.

1. Input music search of melody query

In this method, the user inputs the melody of the music through a microphone and finds the music containing the melody.

2. Music Style Search

This method is to find the music that corresponds to the style of music by inputting a sample of music of the desired style or by adjusting several variables that define the music style.

3. Fast search of songs

In this method, a user inputs a desired speed as a numerical value and searches for a corresponding speed music.

4. Sample input search

In this method, a stored music file is inputted or a music being played is input into a microphone to find a music in the database.

A music search method using a melody query input will be described in the music data search method. Among the various elements of music, melody can be easily remembered by the listener. Music heard through audio media is easy to remember the melody of an important part, if not the whole part. The search method using this characteristic is a melody query input music search method. The melody of the user's memory is input to the melody input from the music of the database by inputting the melody of the part memorized through the microphone or through an input device such as a keyboard. The music containing the corresponding part is returned as a search result.

A feature of the existing melody query input music search technology is that it targets a database where the melody characteristics of music data are manually indexed. That is, score information, pitch, length, and the like of the original music are manually indexed in advance. The humming query analyzes the signal through the microphone and automatically extracts and retrieves the pitch, length, and so on. Therefore, such melody matching becomes a problem of string matching such as text matching, and methods for efficiently searching for this problem have been published.

As a method for melody matching, in the case of string matching between a melody query represented by a symbol and music, a method of matching using the semantic information of a string used in a general text search is introduced, and the musical semantic information of the melody symbol string is introduced. There is a way to extract and search.

Alternative methods include string matching based on the MIDI format, a standard form of computer music representation, or calculating similarities between two MIDI files based on melody, melody strings, rhythm strings, and chord strings from MIDI files. There is a method of extracting and string matching. In addition, a method of expressing a MIDI file for melody matching may be introduced.

In addition, for the melody matching, a method of extracting a melody element from a sound input or a matching method used when matching a MIDI file and a humming has been introduced, and a method of expressing a melody as a difference from successive notes has been introduced. A method of expressing and matching is also introduced.

The music retrieval technique based on the melody query described so far can be applied to a song accompaniment system (aka karaoke system). When the melody query input based song retrieval technology is applied to a song accompaniment system, a melody query input based song retrieval system may have a form of being embedded in a song repeater. There is a difficulty in that it requires additional work such as additional memory required when modifying the melody feature database when a new song is added, and it is impossible to modify and supplement the melody input music search device. Can be.

Therefore, it is possible to reduce or eliminate the burden of redesigning the existing song half-cycle system, to secure the ease of modifying the melody feature database according to the addition of new songs, and to easily modify and supplement the melody input music search apparatus. Skill is required.

An object of the present invention is to provide a method and apparatus for retrieving a song stored in a song repeater using a personal portable terminal.

In particular, the present invention is a system for retrieving a song stored in the half cycle of the song, the user directly inputs the melody, which is a musical feature that a person can easily remember without using information such as the title of the song or a singer, by directly Provides a method and apparatus for searching for music containing the melody, inputs the melody and compares the melody characteristics of the song stored in the song repeater based on the melody, selects the song and selects it in the song repeater Disclosure of the Invention It is an object of the present invention to provide a song retrieval method using a PDA device and a device for inputting a song number.

1 is a block diagram of a song repeater according to the present invention

2 is a PDA block diagram according to the present invention

3 is a diagram schematically showing a song searching method according to the present invention;

4 is a diagram schematically illustrating a communication method between a PDA and a song repeater according to the present invention;

A song search system of a song repeater using a personal portable terminal of the present invention includes a song repeater including a song database and song selection and playback, communication with a terminal, and an output device of the song selected and played; A personal portable terminal for searching for a song based on a melody input by a user by voice, outputting a search result, and delivering a search result to the song repeater; Characterized in that comprises a.

Hereinafter, the configuration and operation of the song search system of the present invention made as described above will be described in detail with reference to the accompanying drawings.

The music retrieval technique based on the melody query described so far can be applied to the song half cycle. When applied to the song repeater, the melody query input music search algorithm can be located inside the song accompaniment system as hardware. In this case, when the user's humming melody is input through the microphone attached to the song repeater, A device that performs a melody quality music search function analyzes the input humming melody for search, and displays the result on the display device added to the song repeater, and displays the result using a remote controller added to the song repeater. You can use the method to select the desired song from the search results displayed on the device.

However, as described above, when the melody input music retrieval device is configured as hardware inside the song repeater, it is difficult to redesign the existing song repeater system, and the melody characteristic when a new song is added When modifying the database, additional work such as adding additional memory is required, and the disadvantage of modifying and supplementing the melody input music search device is disadvantageous.

In the present invention, a song accompaniment is performed by processing a series of melody input music searches, such as a melody input through a microphone, a music search algorithm through a melody input, a necessary melody feature database operation, selection of a searched song, and the like in the PDA as software. This disadvantage is solved when the device is burdened with hardware.

That is, the present invention performs the melody query input music search process as a melody query input song search system in a portable computer device such as a PDA. This method is easier in modifying the database and supplementing the song search algorithm itself than in the case where the melody query input music retrieval apparatus is configured as a hardware in the song repeater, and solves the existing problems.

1 shows a configuration of a song repeater in the song search system according to the present invention. The song accompaniment device is a kind of entertainment device in which the accompaniment is played by storing the accompaniment of the song so that the accompaniment is played by playing the song selected by the user's selection. Such a song repeater system includes a song input device (microphone) 11 that accepts a user's song input, musical information of a song, that is, a sheet music information storage device for playing the song according to a user's request. (12), a song reproducing apparatus 13 for reproducing a song according to the score information, a display device 14 representing information such as a title of a song, a singer or a song code, and a song lyrics, and a song code of a desired song. Selecting device 15 to input and select, a remote control 16 that functions as a selection device and can operate at a long distance, an infrared signal receiving device 17 to receive infrared signals from the remote control, the song and song input device to be played It is composed of a song output device (speaker) 18 for outputting the user's song input to the input. The remote controller 16 includes an infrared signal output device 161 for transmitting the selections in the remote controller to the song repeater as an infrared signal.

A user who wants to sing a song using the song repeater finds a song to sing using a book type manual provided separately, and inputs a corresponding song number using the remote controller 16. Although not shown, the remote controller 16 is provided with keys for inputting, selecting, reserving, playing, canceling, and stopping a song number. The song number input through the RICON 16 is converted into an infrared signal by the infrared signal output device 161 and transmitted, and the infrared signal is received by the infrared signal receiver 17 of the song repeater and selected by the user. A song corresponding to the song number is selected by the selection device 15. The song selected by the music selection device 15 outputs the music score corresponding to the song in the music storage device 12, and this leads to the reproduction of the song through the song reproducing device 13, and the reproduced song outputs the song. It is output along with a song that a user sings input to the song input device 11 through the device 18. In addition, various information such as song lyrics and music selection status is output through the display device 14.

The present invention relates to a new song selection method that operates in the existing song repeater configuration. In particular, the present invention relates to a search for a song based on communication with a personal digital assistant (PDA) using a remote control interface added to the song repeater. It is to enable the delivery of the selection information.

Figure 2 shows the configuration of a PDA applied to the present invention. PDA is a kind of portable computer, which is a portable personal information terminal that can move and keep working by inserting document file created by computer at home or office, and can manage personal information or schedule like the electronic notebook. In general, a PDA can directly input handwritten information, and can utilize all the functions of the wireless Internet such as online stock trading or electronic commerce as well as information exchange through a computer. In addition, it can be combined with a mobile phone to know various traffic information, and has a variety of functions such as facsimile function.

As shown in FIG. 2, the PDA basically includes a central processing unit (CPU) 21 that performs calculations, a first memory 22 for storing short-term data, a second memory 23 for storing long-term data, It is composed of a display unit 24 for outputting an image signal, and as an interface with an external device, a communication device unit 25 including an infrared communication device, a near field communication device, a serial / parallel communication device, and the like. Devices such as an input device 26, a microphone 27 for voice input, and a speaker 28 for voice signal output are added. In addition, an operating system (software) is installed, and various applications are operated on the operating system.

The user can input various commands, data, etc. to the device using the input device 26, and the central processing unit 21 opens the first memory 22 and the second memory 23 in response to the user's command. It processes the processing, output, storage, and the like of various data, and outputs the data or the operation state of the device through the display unit 24.

The communication device 25 may perform communication with a peripheral device or wireless internet, process a voice signal input by the user through the microphone 27, and transmit an appropriate voice signal through the speaker 28. It also prints.

In the present invention, the algorithm for storing the melody query input music, the operation of the database, the input of the melody, the display of the search result, and the like are performed on the PDA. To this end, the PDA is equipped with a melody query input song retrieval software as shown in FIG. 3, which is stored in the second memory 23 and driven by the central computing unit 21, for example. The search will be performed.

The melody query-based song retrieval software extracts a query input melody feature, which is a form required for retrieval from an audio signal input as a humming or song through a microphone 27 added to the PDA as shown in FIG. Extraction module 31, song melody feature database 32, which extracts and stores the melody feature of each song from the song files, and similarity calculation module 33 for searching the song melody feature database using a query input melody feature. The user interface 34 provides the user with the function of operating each function, outputting search results, and operating a song repeater to perform a song search through a melody query input. To convert the form of a signal that can be received by the signal receiver 17 And an infrared output signal correction module 35.

The similarity calculation module 33 uses an approximate matching method that considers possible input errors such as insertion or deletion of sounds due to the hum of a user or the characteristics of a song query. The result of the search displays the song in the song melody feature database similar to the query melody feature on the display 24 of the PDA using the user interface 34 such as song title, song code, and the like.

A melody based selection operation will be described with reference to FIGS. 2 and 3. When the user inputs a humming or song to the PDA through the microphone 27 for selection, the query input melody feature extraction module 31 extracts the melody feature of the input query input and inputs it to the similarity calculation module 33. The similarity calculation module 33 compares the feature information of the query input melody extracted by the query input melody feature extraction module 31 with the song melody feature information stored in the database 32 and compares the songs based on the similarity. Print the search results. The search result output from the similarity calculation module 33 is displayed on the display unit 24 through the user interface 34. For example, the search result outputs a title, a song code (song number), and the like of the searched song. As such, information about the song selected by the user from the search result is transferred to the song repeater so that the song can be played (played). FIG. 4 schematically shows a communication method between the PDA and the song repeater.

The user interface 34 of the melody query input song retrieval software running on the above-described PDA has the same function as the selection device 15 of the song repeater or the remote control 16 which enables the function of the selection device at a long distance. Buttons are provided. In addition, the infrared output signal correction module 35, which configures the infrared communication device 251 output signal of the PDA for the specific button operation and the infrared output signal from the remote controller 16 of the song repeater, is identical to the melody query input song search. It is included in software 30. That is, the infrared output signal correction module 35 configures the remote control signal (infrared signal) related to the song repeater of the PDA in the same code and format as the remote control signal (infrared signal) of the remote controller 16 of the song repeater. It becomes a kind of signal converter.

Therefore, when a specific button of the user interface 34 of the melody query input song retrieval software 30 mounted on the PDA 20 is pressed, a button having the same function is pressed on the remote controller 16 of the song repeater 10. Infrared output signal having the same configuration as that of the light source is output from the infrared communication device 251 of the PDA, which is received by the infrared signal receiving device 17 of the song repeater 10, as if the selection key of the remote controller 16, This has the same effect as pressing the reservation key, the play execution key, the cancel reservation key, or the play stop key.

As described above, the song retrieved by the melody query input song retrieval software 30 on the PDA 20 is transmitted to the display unit 24 of the PDA via the user interface 34 of the melody query input song retrieval software. It is displayed in the form of information such as a singer and a song code. In this case, the search result is a list of several or dozen songs due to the nature of searching through similar matching, and they are sorted according to the similarity, and the songs most similar to the query melody feature are sorted at the top. The user selects the song through the input device 26 of the PDA when the user thinks that the desired song is looked up, and the selection information corresponding to the song selected by the user is an infrared output signal of the melody query input search software 30. The correction module 35 converts the signal into an output signal for a song code of a type capable of communicating with the infrared signal receiving device 17 of the song repeater 10, and the signal is converted into the infrared communication device 251 of the PDA 10. The music selection information signal is transmitted through the infrared signal receiving device 17 of the song repeater 10 to the song repeater 10 as if the song selection was made using the remote controller 16.

Therefore, the song repeater 10 performs reproduction and output of the music score of the corresponding song based on the input music selection information of the user. In addition to the song selection, the selected song may be reserved or immediately played using the reservation or play button provided in the user interface 34 of the melody query input song search software 30 according to the intention of the user. That is, by using the PDA 20 it is possible to remote control like the remote control 16 of the song repeater 10.

As described above, the present invention includes a song melody feature database on a PDA, extracts a feature from a query melody input by a user in the form of humming, and retrieves the feature based on the similarity with the feature information of the database. The search result was output, and the user selected the song selected from the search result was transferred from the PDA to the song repeater as if the user operated the remote control of the song repeater to enable the play or reservation of the song. Therefore, in the present invention, a system for searching for a melody-based song in a PDA should be constructed, and in particular, a song melody feature database should be constructed.

That is, the song melody feature database must be pre-configured for the melody query input song search function. The song melody feature database 32 is extracted from the song file, and the extraction process may be extracted from the song file in advance through the song melody feature extractor / module at the provider of the song file, and the song files are loaded on the PDA. If so, it can also be extracted directly from the PDA using a melody feature extractor / module. When a new song is added, a new song melody feature database newly extracted by the provider of the song file through an external interface of the PDA 20, for example, a wired or wireless communication interface device included in the communication device unit 25. Can be received and mounted in the PDA.

In addition, according to the present invention, the search algorithm of the melody query input song search software 30 can be modified or supplemented by an easy upgrade. This is because the melody query input song retrieval software 30 is mounted on the PDA 20 which is portable and secures various communication interfaces.

That is, the search algorithm of the melody query input song search software 30 can be modified and supplemented by the corresponding software provider, which means that the search algorithm independently modulates each part of the melody query input song search software. Etc.), when a new search algorithm is modified or supplemented, the module of the search algorithm is modified and supplemented by receiving the module of the search algorithm modified and supplemented through the external interface-communication unit of the PDA. This is because the search function can be performed.

The present invention provides an apparatus and method for performing selection of a song half-period based on a melody query input using a PDA. According to the present invention, a song search module based on a melody query input including a song melody feature database is included in a PDA, and a result of comparing a melody feature input by the user with a humming or the like using a PDA microphone is compared with a database. By outputting, the user can select a song to sing, and the user can select the song selected on the PDA by using an infrared communication device to transfer the song information about the selected song on the PDA so that the reservation and performance of the song can be made. It was.

In particular, in the present invention, since the PDA is equipped with the song search system, the burden of mounting the song search system in the song repeater can be reduced, and modification and supplementation of the song database or search software can be performed more easily.

Claims (10)

Searching for a song selected by a user based on a melody query by transmitting a song search software based on a melody query input installed in a personal mobile terminal, transmitting the searched song information to a song repeater, and transmitting the And receiving song information and performing song accompaniment for the corresponding song. According to claim 1, In order to transmit the song information retrieved from the personal portable terminal, the song information signal is converted and transmitted in the form of a signal of the same configuration as the remote control infrared signal for remote control of the song repeater Song searching method using a personal mobile terminal to the. The method of claim 1, wherein the melody query input based song search comprises: extracting a feature of a query input melody; calculating a similarity degree between the extracted query input melody feature information and feature information of a song melody feature database; And outputting a search result according to the similarity level, and correcting and outputting the song information signal selected from the search result into a signal form having the same configuration as the remote control signal of the song repeater. How to search for songs using half-cycle period. [4] The song half-period of claim 3, wherein the similarity calculation is performed for similar matching to consider possible input errors such as insertion or deletion of sounds due to user's humming or query characteristics. How to search for a song. Song repeater including a song database and song selection and playback, communication with the terminal and output device of the song selection and playback, and song search software based on the melody query input to display melody input by voice by the user. And a personal portable terminal for searching for a song based on the song, outputting the search result, and transmitting the selected song information signal to the song repeater. The method of claim 5, wherein the personal mobile terminal is a database for storing song melody feature information for searching for a song based on a melody query, and a melody query input feature extract for extracting feature information of a melody input by the user by voice. A module, a similarity calculating module for calculating a similarity between the extracted melody query input feature information and a song melody feature information of a database, a user interface for outputting a search result and song selection according to the calculated similarity, and using the user interface And a communication module for transmitting the selected song information signal to a song repeater. 6. The personal portable terminal of claim 5, wherein the personal portable terminal comprises signal converting means for converting the selected song information signal into a signal having the same configuration as the remote control signal of the song repeater and transmitting the song information signal to the song repeater. Song accompaniment system. 7. The song accompaniment system according to claim 6, wherein the user interface has a key input interface having the same function as a remote controller of the song repeater. The song accompaniment system according to claim 6, wherein each module of the melody query input-based song retrieval software is modified and supplemented for each module through a communication means of a personal portable terminal. 7. The method of claim 6, wherein a new database is received from a provider of a song file and a song melody feature database through a communication means of the personal mobile terminal, and the existing song melody feature database is modified to modify the melody for the new song. Song accompaniment system, characterized in that the query input song search.