KR20070067284A - Method of searching information regarding multimedia data, and communication terminal and searching engine server of enabling the method - Google Patents

Abstract

A method for performing information searching on multimedia data, and a communication terminal and a search engine server employing the same are provided to meet the user demands for searching multimedia data such as image information, voice information and text information. A recording unit is maintained to record search result data corresponding to search word information. A request for reproducing certain multimedia data from a user and the multimedia data is reproduced(202,203). A loading input of a search application with respect to the multimedia data is received from a user(204,205). One or more search word information with respect to the multimedia data are extracted from the search application(206). The recording unit is searched according to a search query including the extracted search word information(207). Search result data corresponding to the search word information is read from the recording unit and displayed on a display unit of a communication terminal(208,209).

Description

METHOD OF SEARCHING INFORMATION REGARDING MULTIMEDIA DATA, AND COMMUNICATION TERMINAL AND SEARCHING ENGINE SERVER OF ENABLING THE METHOD}

1 is a diagram schematically illustrating a network connection relationship of a communication terminal according to an embodiment of the present invention.

2 is a flowchart illustrating a method of operating a communication terminal performing information retrieval on multimedia data according to an embodiment of the present invention.

3 is a diagram illustrating an example of a screen displaying search result data according to an exemplary embodiment of the present invention.

4 is a flowchart illustrating a method of operating a search engine server according to an exemplary embodiment of the present invention.

5 is a block diagram showing an internal configuration of a communication terminal according to an embodiment of the present invention.

6 is a diagram illustrating an internal configuration of a search engine server according to an embodiment of the present invention.

<Explanation of symbols for the main parts of the drawings>

500: communication terminal 501: recording means

502: Recording unit 503: Multimedia data processing unit

504: Search application processing unit 505: Search result provider

506: query data transmission and reception unit 507: control unit

The present invention relates to a method for performing information retrieval on multimedia data, a communication terminal and a search engine server employing the method. More particularly, the present invention relates to extracting a search query for a predetermined multimedia data and according to the search query. A method for performing information retrieval on multimedia data provided to a user by retrieving search result data recorded in predetermined recording means, and a communication terminal and a search engine server employing the method.

Today's remarkable development of information and communication technology has rapidly promoted the popularization of communication terminals (mobile devices), so that most ordinary people always have communication terminals such as mobile phones and PDAs. The services provided by these communication terminals gradually move away from the basic voice call services and text services that have been provided as communication technology and user use have increased. For example, wireless Internet, video playback, Expanded to services such as MP3 music.

Due to the expansion of these various services, communication terminals have been able to use the Internet search service that has been conventionally used in computer terminals. For example, Korean Patent Application No. 10-2003-0066555 relates to a wireless information retrieval system and method thereof of a mobile communication terminal. When the mobile communication terminal transmits a search word of information to be searched by using an SMS message, a search server transmits a search word. Disclosed is a configuration in which an information search result corresponding to the search word is provided in a wireless page form and provided to the mobile communication terminal.

However, a search service of a communication terminal simply inputs a text desired by a user as a search word and provides a search result according to the search word to the user. As the multimedia data used in the communication terminal increases, the user of the communication terminal needs to search for multimedia data, for example, image information, audio information, or text information. There is a need for a search method that can be met.

SUMMARY OF THE INVENTION The present invention has been made to solve the above-mentioned problems of the prior art, which is recorded in recording means of a communication terminal in accordance with a search query including search term information extracted by extracting search term information on multimedia data being played in a communication terminal. It is an object of the present invention to provide a method of searching for search result data corresponding to the search word information and performing information search on multimedia data providing the searched search result data to a user, and a communication terminal and a search engine server employing the method. It is done.

The present invention also provides a predetermined search engine when a search result of searching the recording means according to a search query including search term information extracted from multimedia data in a communication terminal and the search result data according to the search query do not exist. Transmitting a search query including the search word information to a server, receiving search result data for the search query from the search engine server, and displaying the search result data on a display means to provide a search for multimedia data to the user. It is an object of the present invention to provide a method for performing information retrieval, a communication terminal and a search engine server employing the method.

In order to achieve the above object and to solve the above-mentioned problems of the prior art, the operation method of the communication terminal for performing the information search for the multimedia data according to an embodiment of the present invention, the search result data corresponding to the search word information Maintaining recording means for recording the data; Receiving a request to play predetermined multimedia data from a user and playing the multimedia data; Receiving a loading input of a search application for the multimedia data from the user; In the search application, extracting one or more search term information for the multimedia data; Searching the recording means according to a search query including the extracted search term information; And reading the search result data corresponding to the search word information by the recording means, and displaying the read search result data on a predetermined display means of the communication terminal.

In addition, a method of operating a search engine server that provides a search result of a search query for multimedia data received from a communication terminal according to another embodiment of the present invention may include: a search result of recording search result data corresponding to first search term information; Maintaining a database; Receiving a search query including second search term information from a predetermined communication terminal; Searching first search term information corresponding to second search term information included in the received search query by referring to the search result database; Reading the search result data corresponding to the first search term information when the first search term information corresponding to the second search term information exists; And transmitting the read search result data to the communication terminal, wherein the communication terminal plays back predetermined multimedia data, extracts one or more search term information on the multimedia data, and extracts the extracted search term information. And transmitting a search query that includes the search query.

In addition, a communication terminal for providing an information search result for multimedia data according to another embodiment of the present invention, recording means for recording the search result data corresponding to the search word information; A multimedia data processor configured to receive a request to play predetermined multimedia data from a user and to play the multimedia data; An application processor for receiving a loading input of a search application for the multimedia data from the user; A query search management unit for extracting one or more search term information of the multimedia data from the search application and searching the recording means according to a search query including the extracted search term information; And a search result providing unit which reads the search result data corresponding to the search word information from the recording means and displays the read search result data on a predetermined display means of the communication terminal.

In addition, a search engine server providing a search result for a search query for the multimedia data received from the communication terminal according to another embodiment of the present invention, a search result database for recording the search result data corresponding to the first search term information ; A search query receiving unit which receives a search query including second search word information from a predetermined communication terminal; Searching for first search term information corresponding to second search term information included in the received search query by referring to the search result database, and when the first search term information corresponding to the second search term information exists, A search query manager configured to read the search result data corresponding to first search word information; And a search result providing unit configured to transmit the read search result data to the communication terminal, wherein the communication terminal plays back predetermined multimedia data, extracts one or more search term information on the multimedia data, and extracts the search term. The search query including the information is transmitted.

The communication terminal used in the present invention is a computer terminal, a public switched telephone network (PSTN) terminal, VoIP, Session Initiation Protocol (SIP), Megaco, PDA (Personal Digital Assistant), cellular phone, PCS (Personal Communication Service) phone, hand Hand-Held PC, CDMA-2000 (1X, 3X) Phone, Wideband CDMA (WCDMA) Phone, Dual Band / Dual Mode Phone, Global Standard for Mobile (GSM) Phone, MBS (Mobile Broadband System) phones, or satellite / terrestrial digital multimedia broadcasting (DMB) phones.

Hereinafter, a method for performing information retrieval on multimedia data of the present invention, a communication terminal and a search engine server employing the method will be described in detail with reference to the accompanying drawings.

1 is a diagram schematically illustrating a network connection relationship of a communication terminal according to an embodiment of the present invention.

As shown in FIG. 1, when the communication terminal 110 receives a loading input of a search application for multimedia data reproduced in the communication terminal 110 from a user, the communication terminal 110 extracts search term information on the multimedia data to provide a predetermined value. The search result data corresponding to the search word information recorded in the recording means is displayed on the predetermined display means. In addition, when the search result data does not exist in the recording means of the communication terminal 110, the communication terminal 110 transmits a search query including the search term information to the search engine server 120, and the search query. Receive search result data for.

2 is a flowchart illustrating a method of operating a communication terminal performing information retrieval on multimedia data according to an embodiment of the present invention.

First, in step 201, the communication terminal maintains recording means for recording the search result data corresponding to the search word information.

According to an embodiment of the present invention, the recording means may be updated by receiving the search result data corresponding to the search word information from a predetermined search engine server every predetermined period.

In step 202, the user inputs a playback request for predetermined multimedia data in the communication terminal. For example, the user may input a play request for playing the music content “I loved” which is the multimedia data in the communication terminal.

In step 203, the communication terminal plays the multimedia data according to the playback request.

In step 204, the user loads and inputs a search application for the multimedia data. That is, the user may load input the search application to search for the multimedia data while playing the multimedia data in the communication terminal.

In step 205 the communication terminal receives a loading input of the search application from the user.

In step 206, the search application of the communication terminal extracts one or more search term information for the multimedia data.

According to an embodiment of the present invention, the multimedia data is image information, and the communication terminal may extract feature information of the image information through image recognition or face recognition of an image included in the image information. The search query to be described later may include the feature information. The video information may be video information recorded in the communication terminal, video information of a user of a counterpart communication terminal output during a video call, and the like. For example, in the search application of the communication terminal, a still image is obtained from the image information of the second user outputted during the video call with the counterpart communication terminal, and a portion corresponding to the face is detected from the obtained still image. Feature information about the detected face may be extracted.

In addition, according to another embodiment of the present invention, the multimedia data is voice information, and the communication terminal converts the voice information into text through a predetermined voice / text conversion module, and the text according to predetermined logic. One or more search term information may be extracted from the. The voice information may be music content information recorded in the communication terminal or voice information of the other party output during a voice call. In one example, the search application of the communication terminal, the voice information about the "must have loved" that is the music content played on the communication terminal through the voice / text conversion module, the text "parting is really easier than meeting is cold only cold One or more search term information "parting, meeting, easy, person" in the text "parting is cooler than meeting," according to the predetermined logic. Can be extracted.

In addition, according to another embodiment of the present invention, the multimedia data is text information, and the communication terminal may extract one or more search term information from the text information according to predetermined logic.

In step 207, the communication terminal searches for the recording means according to a search query including the extracted search word information. For example, when the extracted search term information is “parting, meeting, or easy, person”, the recording means is searched according to a search query including the search term information.

In step 208, the communication terminal reads out the search result data corresponding to the search word information from the recording means. For example, the communication terminal is a search result data corresponding to the search term information "parting, meeting, easy, people," "Title: I love you, singer: Yoon Do-hyun, lyrics: parting is really easier than meeting cold people" I don't know why my heart was taken …… ”.

In step 209, the communication terminal displays the read search result data on predetermined display means of the communication terminal.

3 is a diagram illustrating an example of a screen displaying search result data according to an exemplary embodiment of the present invention.

As shown in FIG. 3, the communication terminal reads the read result data to the user. "Title: I Love You, Singer: Yoon Do Hyun, Lyrics: It's easy to say goodbye than meeting, why did I take a cold heart?" It is possible to display ... on the display means.

According to an embodiment of the present invention, when the communication terminal searches for the recording means according to a search query including the extracted search term information and the search result data according to the search query does not exist, The search query including the search term information may be transmitted to a search engine server of, and search result data for the search query may be received from the search engine server.

According to another embodiment of the present invention, the search result data received from the search engine server may be processed into a predetermined format and displayed on the display means.

4 is a flowchart illustrating a method of operating a search engine server according to an exemplary embodiment of the present invention.

In step 401, the search engine server maintains a search result database that records search result data corresponding to the first search term information.

In step 402, the communication terminal plays predetermined multimedia data, extracts one or more second search term information for the multimedia data, and transmits a search query including the extracted second search term information.

In step 403, the search engine server receives a search query including the second search term information from the communication terminal. In one example, the search engine server may receive a search query including the feature information "AA" extracted from the image information from the communication terminal.

In operation 404, the search engine server searches first search term information corresponding to second search term information included in the received search query by referring to the search result database. For example, the search engine server may search the first search term information corresponding to the feature information “AA” by referring to the search result database.

In operation 405, when the first search term information corresponding to the second search term information exists, the search engine server reads the search result data corresponding to the first search term information. For example, when the first search term information corresponding to the feature information “AA” is present, the search engine server corresponds to “Name: Kim Young-mi, Age: 23 years old, Hobbies: Reading, Movie” corresponding to the first search term information. I can read impression …… "".

In step 406, the search engine server transmits the read search result data to the communication terminal.

In step 407, the communication terminal may receive the search result data transmitted from the search engine server, process it into a predetermined format, and display it on the display means.

5 is a block diagram showing an internal configuration of a communication terminal according to an embodiment of the present invention.

As shown in FIG. 5, the communication terminal 500 includes a recording unit 501, a recording unit 502, a multimedia data processing unit 503, a search application processing unit 504, a search result providing unit 505, and query data transmission and reception. The unit 506 and the control unit 507 are included.

The recording means 501 records the search result data corresponding to the search word information. In addition, according to an embodiment of the present invention, the recording unit 502 receives the search result data corresponding to the search word information from the search engine server every predetermined period from the search engine server every predetermined period to record the search result ( 501).

The multimedia data processing unit 503 receives a reproduction request of predetermined multimedia data from a user and reproduces the multimedia data.

The search application processor 504 receives a loading input of a search application for the multimedia data from the user, extracts one or more search term information for the multimedia data from the search application, and includes the extracted search term information. The recording means is searched according to a search query.

According to an embodiment of the present invention, the multimedia data is image information, and the search application processor 504 extracts feature information of the image information through image recognition or face recognition on an image included in the image information. The search query may include the extracted feature information.

According to another embodiment of the present invention, the multimedia data is voice information, and the search application processor 504 converts the voice information into text through a predetermined voice / text conversion module, and according to predetermined logic. One or more search term information may be extracted from the text.

According to another embodiment of the present invention, the multimedia data is text information, and the search application processing unit 504 may extract one or more search term information from the text information according to predetermined logic.

The search result providing unit 505 reads the search result data corresponding to the search word information from the recording means, and displays the read search result data on predetermined display means of the communication terminal.

According to an embodiment of the present invention, when the query data transmission / reception unit 506 searches by the search application processing unit 504 and the search result data according to the search query does not exist, the query data transmission / reception unit 506 sends the search data to a predetermined search engine server. The search query including the search term information is transmitted, and search result data for the search query is received from the search engine server.

Further, according to another embodiment of the present invention, the search result providing unit 505 processes the search result data received from the search engine server into a predetermined format and displays it on the display means.

The control unit 507 typically includes a processor function to process all data on the terminal device associated with the modem DSP. The more DSPs for processing internal signals, such as baseband signals, the faster the processing in each mode due to the faster processing speed. The controller 507 may include a mobile station modem (MSM), a digital signal processor (DSP), an open multimedia application platform (OMAP), and the like.

6 is a diagram illustrating an internal configuration of a search engine server according to an embodiment of the present invention.

As shown in FIG. 6, the search engine server 600 includes a search result database 601, a search query receiver 602, a search query manager 603, and a search result provider 604.

The search result database 601 records search result data corresponding to the first search word information.

The search query receiving unit 602 receives a search query including the second search word information from a predetermined communication terminal.

The search query manager 603 searches the first search term information corresponding to the second search term information included in the received search query by referring to the search result database, and the first search term corresponding to the second search term information. If the information exists, the search result data corresponding to the first search word information is read.

The search result providing unit 604 transmits the read search result data to the communication terminal.

The operating method of the communication terminal according to the present invention can be implemented in the form of program instructions that can be executed by various computer means and recorded in a computer readable medium. The computer readable medium may include program instructions, data files, data structures, etc. alone or in combination. Program instructions recorded on the media may be those specially designed and constructed for the purposes of the present invention, or they may be of the kind well-known and available to those having skill in the computer software arts. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tape, optical media such as CD-ROMs, DVDs, and magnetic disks, such as floppy disks. Magneto-optical media, and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. The medium may be a transmission medium such as an optical or metal wire, a waveguide, or the like including a carrier wave for transmitting a signal specifying a program command, a data structure, or the like. Examples of program instructions include machine code, such as produced by a compiler, as well as high-level language code that can be executed by a computer using an interpreter. The hardware device described above may be configured to operate as one or more software modules to perform the operations of the present invention, and vice versa.

As described above, although the present invention has been described with reference to limited embodiments and drawings, the present invention is not limited to the above embodiments, and those skilled in the art to which the present invention pertains various modifications and variations from such descriptions. This is possible.

Therefore, the scope of the present invention should not be limited to the described embodiments, but should be determined not only by the claims below but also by the equivalents of the claims.

Search the search result data corresponding to the search term information recorded in the recording means of the communication terminal according to a search query including the search term information extracted by extracting search term information of the multimedia data being reproduced in the communication terminal according to the present invention; By displaying the retrieved search result data on the display means, it is possible to provide the user with a search for multimedia data.

In addition, according to the present invention, if a search result for searching the recording means in accordance with a search query including the search term information extracted from the multimedia data in the communication terminal, the search result data according to the search query does not exist, Transmitting a search query including the search term information to a search engine server, receiving search result data for the search query from the search engine server, and displaying the search result data on a display means to provide the user with search results for multimedia data. have.

Claims (17)

A method of operating a communication terminal for performing information retrieval on multimedia data Maintaining recording means for recording search result data corresponding to search word information; Receiving a play request for predetermined multimedia data from a user and playing the multimedia data; Receiving a loading input of a search application for the multimedia data from the user; In the search application, extracting one or more search term information for the multimedia data; Searching the recording means according to a search query including the extracted search term information; And Reading out search result data corresponding to the search word information from the recording means, and displaying the read search result data on predetermined display means of the communication terminal; Method of operation of a communication terminal comprising a. The method of claim 1, The step of searching the recording means according to the search query including the extracted search word information, Transmitting a search query including the search term information to a predetermined search engine server when there is no search result data according to the search query; And Receiving search result data for the search query from the search engine server Method of operation of a communication terminal comprising a. The method of claim 2, Processing the search result data received from the search engine server into a predetermined format and displaying on the display means; Method of operation of a communication terminal further comprising. The method of claim 2, The step of holding the recording means, And receiving and recording the search result data corresponding to the search word information from the search engine server at predetermined intervals. The method of claim 1, The multimedia data is image information, Extracting at least one search term information for the multimedia data, Extracting feature information of the image information through image recognition or face recognition with respect to an image included in the image information, And the search query includes the extracted feature information. The method of claim 1, The multimedia data is voice information, Extracting at least one search term information for the multimedia data, And converting the voice information into text through a predetermined voice / text conversion module and extracting one or more search term information from the text according to a predetermined logic. The method of claim 1, The multimedia data is text information, Extracting at least one search term information for the multimedia data, Extracting one or more search term information from the text information according to a predetermined logic. A method of operating a search engine server for providing a search result of a search query for multimedia data received from a communication terminal, Maintaining a search result database for recording search result data corresponding to the first search word information; Receiving a search query including second search term information from a predetermined communication terminal; Searching first search term information corresponding to second search term information included in the received search query by referring to the search result database; Reading the search result data corresponding to the first search term information when the first search term information corresponding to the second search term information exists; And Transmitting the read search result data to the communication terminal. Including, The communication terminal plays a predetermined multimedia data, extracts one or more search term information for the multimedia data, and transmits a search query including the extracted search term information. A computer-readable recording medium in which a program for executing the method of any one of claims 1 to 8 is recorded. A communication terminal for providing an information search result for multimedia data, Recording means for recording search result data corresponding to search word information; A multimedia data processor configured to receive a play request for predetermined multimedia data from a user and to play the multimedia data; Receiving a loading input of a search application for the multimedia data from the user, extracting one or more search term information for the multimedia data from the search application, and according to the search query including the extracted search term information; A search application processing unit for searching for; And A search result providing unit for reading the search result data corresponding to the search term information from the recording means and displaying the read search result data on a predetermined display means of the communication terminal; Communication terminal comprising a. The method of claim 10, The search application processing unit, As a result of the search, if the search result data according to the search query does not exist, the search query including the search term information is transmitted to a predetermined search engine server, and the search result data for the search query is retrieved from the search engine server. Received Query Data Transceiver Communication terminal comprising a. The method of claim 11, The search result providing unit, And the search result data received from the search engine server is processed into a predetermined format and displayed on the display means. The method of claim 11, A recording unit for receiving and recording the search result data corresponding to the search term information from the search engine server at predetermined intervals from the search engine server at predetermined intervals. Communication terminal comprising a. The method of claim 10, The multimedia data is image information, The search application processing unit, And extracting feature information of the image information through image recognition or face recognition of the image included in the image information, and wherein the search query includes the extracted feature information. The method of claim 10, The multimedia data is voice information, The search application processing unit, And converting the voice information into text through a predetermined voice / text conversion module and extracting one or more search term information from the text according to a predetermined logic. The method of claim 10, The multimedia data is text information, The search application processing unit, And extracting one or more search term information from the text information according to a predetermined logic. A search engine server for providing a search result for a search query for multimedia data received from a communication terminal, A search result database that records search result data corresponding to the first search term information; A search query receiving unit which receives a search query including second search word information from a predetermined communication terminal; Searching for first search term information corresponding to second search term information included in the received search query by referring to the search result database, and when the first search term information corresponding to the second search term information exists, A search query manager configured to read the search result data corresponding to first search word information; And A search result provider for transmitting the read search result data to the communication terminal Including, The communication terminal reproduces predetermined multimedia data, extracts one or more search term information on the multimedia data, and transmits a search query including the extracted search term information.

Images