KR20130076346A - Apparatus and method for searching multimedia data according to user input - Google Patents

Abstract

PURPOSE: A multimedia search device according to user input and a method thereof are provided to supply an accurate search result by dividing user input data into a panorama and a background and searching for multimedia data based on the user input data. CONSTITUTION: A touch screen unit (50) outputs user input data according to touch input and a feature information extracting unit (30) extracts feature information about multimedia data. A storage unit (40) matches the multimedia data with the feature information to store the same and an input data recognizing unit (60) recognizes the user input data. A control unit (10) compares the feature information about the user input data with the feature information about the multimedia data to search the storage unit for candidate multimedia data and to provide the candidate multimedia data. [Reference numerals] (10) Control unit; (20) Camera unit; (30) Feature information extracting unit; (31) Image characteristic information extracting unit; (32) Video characteristic information extracting unit; (40) Storage unit; (50) Touch screen unit; (60) Input data recognizing unit; (61) Letter cognition unit; (62) Edge cognition unit; (70) Function detection unit

Description

Apparatus and method for searching multimedia according to user input {APPARATUS AND METHOD FOR SEARCHING MULTIMEDIA DATA ACCORDING TO USER INPUT}

The present invention relates to a multimedia retrieval apparatus and method, and more particularly, to an apparatus and method for retrieving multimedia by recognizing input data input by a user using a hand.

In general, as the amount of multimedia content is increased and diversified, electronic devices have been increasingly difficult to search for desired content, and thus, it is increasingly difficult to provide an optimized content retrieval method to the user.

In order to search for multimedia content stored in the current electronic device, a keyword used for content search is input into an input window for content search, and a candidate content list for a plurality of candidate content corresponding to the input keyword is generated and displayed on the screen. do. The user selects a desired content from the candidate content list displayed on the screen or inputs another keyword when there is no desired content in the candidate content list. The user who inputs another keyword selects the desired content from the searched candidate content list using the re-entered keyword.

As described above, in order to search a plurality of multimedia contents, a keyword is input from a user, and a list of multimedia contents corresponding to the input keyword is searched and displayed on the screen.

However, in this case, since the user searches and displays a plurality of multimedia contents corresponding to the input keyword, there is a problem in that the user cannot search the desired contents at once.

In addition, when there is no content desired by the user, the user may have to re-enter keywords again and again to search for desired content.

In addition, there is a disadvantage in that it is difficult for the user to search for a desired image when the photographed subject is similar to the photographed subject, such as a self shot of the user, and there are many images whose background is changed.

Accordingly, the present invention provides a content retrieval apparatus and method for receiving user input data for multimedia content from a user, and searching and providing a list of candidate multimedia contents corresponding to the input user input data.

In accordance with another aspect of the present invention, there is provided a multimedia search apparatus using a user input device, the touch screen unit outputting user input data according to a touch input, and a feature information extracting unit extracting feature information on at least one multimedia data. And a storage unit for storing the multimedia data and the extracted feature information in correspondence, an input data recognition unit for recognizing the user input data, and user input data input through the touch screen unit through the feature information extraction unit. Extracts feature information of the extracted user input data, compares the feature information of the extracted user input data with the feature information of each multimedia data, and retrieves candidate multimedia data having similar feature information from the storage unit; data It characterized in that it comprises a control unit for providing.

In addition, the present invention provides a method for retrieving multimedia information using a user input device in a multimedia retrieval device, the method comprising: extracting feature information on at least one multimedia data; and storing the multimedia data and the extracted feature information in correspondence with each other. And receiving user input data according to a touch input, extracting feature information on the received user input data, feature information on the extracted user input data, and feature information on each multimedia data. And a process of searching for candidate multimedia data having feature information similar to that of the extracted user input data as a result of the comparison, and providing the retrieved candidate multimedia data.

According to the present invention, the user input data such as a drawing of the user is divided into a foreground and a background, and the user can receive more accurate search results by performing a search for multimedia data using the input user input data. have.

1 is a block diagram of a multimedia search apparatus according to a user input according to an embodiment of the present invention;
2 to 6 are exemplary views for explaining a process of searching for image data corresponding to user input data according to an embodiment of the present invention;
3 is an example for describing a process of a user separately inputting user input data according to a foreground and a background, and searching for image data using user input data according to the input foreground and background according to an embodiment of the present invention. Degree,
7 is an exemplary view for explaining a process of searching for video data using user input data according to an embodiment of the present invention;
8 is an exemplary diagram for describing a process of generating content using multimedia data selected according to an embodiment of the present invention and then processing the terminal according to a terminal function;
9 is a flowchart illustrating a process for searching for image data according to user input data according to an embodiment of the present invention;
10 is a flowchart illustrating a process for searching video data according to user input data according to an embodiment of the present invention;
11 is a flowchart illustrating a process for processing a content generated using multimedia data as a terminal function corresponding to user input data according to an embodiment of the present invention.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the following description and the annexed drawings, detailed descriptions of well-known functions and configurations that may unnecessarily obscure the subject matter of the present invention will be omitted.

The present invention receives the summarized user input data about the multimedia data, such as the drawing data of the user, extracts feature data corresponding to the input user input data, and compares the extracted feature data with the feature data of the preset multimedia data. It is characterized by searching for and providing multimedia data having similar feature data.

1 is a diagram illustrating a configuration of a multimedia search apparatus according to a user input according to an exemplary embodiment of the present invention.

In an embodiment of the present invention, the multimedia search apparatus may include a controller 10, a camera unit 20, a feature information extractor 30, a storage unit 40, a touch screen unit 50, an input data recognition unit 60, And a function detector 70.

First, the controller 10 controls the overall operation of the multimedia retrieval apparatus. In particular, the feature information extractor 30 extracts feature information on at least one multimedia data, and extracts each feature information and at least one multimedia. Corresponding data is stored in the storage unit 40. Here, the feature information includes edge information and color information in the image. For example, the controller 10 extracts edge information from the captured image output through the camera unit 20 or generates keyframe data for video data stored in the storage unit 40, and generates the keyframe data on the generated keyframe data. It is possible to extract the edge information about the. Here, the key frame data refers to the most representative frame among scenes having similar screen configurations in the video data.

When the user input data is received through the touch screen unit 50, the controller 10 recognizes the user input data received through the input data recognition unit 60. In this case, the user input data may be drawing data drawn by the user or text data written by the user. In addition, when the control unit 10 receives drawing data drawn by the user through the touch screen unit 50, the control unit 10 may change the foreground, background, and key of the image data for more accurate retrieval. The foreground and background of the frame data may be distinguished and input. To this end, the controller 10 may extract the foreground and background of the image data and the foreground and the background of the key frame data even when the feature information of the multimedia data is extracted through the feature information extractor 30.

For example, when the user wants to search for a picture taken at the beach, the multimedia search apparatus includes a setting button for inputting the foreground and a setting button for inputting the background so that the user inputs the setting button for inputting the foreground. Press to input the foreground drawing data, and press the setting button for inputting the background to input the background drawing data. In addition to this setting, a setting for inputting the foreground and the background may be performed in other ways.

Thereafter, the controller 10 extracts feature data from the input user input data through the feature information extractor 30, searches for similar feature information by comparing the extracted feature information with the feature information stored in the storage 40. . In this case, the controller 10 calculates a difference value between the edge information of the user input data and the edge information stored in the storage unit 40, and if the calculated difference value is smaller than a preset threshold value, the controller 10 determines that the two edge information are similar. Can be. In addition, one or more such feature information may be retrieved. Methods for retrieving this similar feature information include Binary Coherent Edge Descriptors (BiCE), Scale Invariant Feature Transform (SIFT), and Dynamically Architected Instruction Set from Yorktown (MinHash), or MinHash (or the min-wise independent permutations locality A similar hash can be determined using a sensitive hashing scheme.

The controller 10 generates at least one multimedia data corresponding to the retrieved similar feature information as a candidate multimedia list and displays the generated candidate multimedia list on the screen of the touch screen unit 50.

If there is a selection of desired multimedia data among candidate multimedia lists displayed by the user through the touch screen unit 50, the controller 10 displays the selected multimedia data on the screen of the touch screen unit 50. In this case, the screen of the touch screen unit 50 includes a user input area for receiving user input data and an area for displaying a candidate multimedia list. The user input area may also be a display area for displaying the selected multimedia data. In such a case, the selected multimedia data may be overlaid on the user input area.

For example, in the case of video data, the controller 10 searches for keyframe data corresponding to similar feature information and displays video data corresponding to the found keyframe data on the screen of the touch screen unit 50.

Meanwhile, if there is a request for generating content using the selected multimedia data, the controller 10 performs a content generation mode. Here, the content generation mode refers to an operation mode for generating new content data reflecting user's emotion such as decorating, writing, and drawing by using the multimedia data selected by the user.

If content data is generated and then present in the user input, the controller 10 recognizes the input user input data through the input data recognizer 60, and detects the terminal function information corresponding to the recognized user input data. Detects through In detail, the controller 10 detects the terminal function information corresponding to the recognition data similar to the recognized user input data through the function detector 70. In this case, in the present invention, the terminal function setting list in which the terminal function information is set in correspondence with the recognition data is stored in advance in the storage unit 40.

The controller 10 processes the content data by performing a terminal function corresponding to the detected terminal function information on the generated content data. For example, if the input data is a message-shaped picture and an e-mail address, it is assumed that an e-mail transmission / reception function is set among the terminal functions. When the user input data input through the touch screen unit 50 is recognized as the input data and the picture of the message shape through the input data recognition unit 60, the control unit 10 is an email corresponding to the recognized user input data Information about a transmission / reception function is detected through the function detection unit 70, and the content data generated by performing an email transmission / reception function according to the detected information is transmitted to the recognized email address.

The camera unit 20 receives an optical signal and outputs image data.

The feature information extractor 30 includes an image feature information extractor 31 and a video feature information extractor 32, and the image feature information extractor 31 is configured to perform image data output from the camera unit 20. Extract feature information. In this case, the feature information may be edge information. Also, the video feature information extractor 32 extracts feature information about one or more keyframe data extracted corresponding to the video data.

In addition, the feature information extractor 30 may extract feature information by dividing the multimedia data according to the foreground and the background. For example, in the case of image data where two people are standing at the beach, the feature information extractor 30 extracts feature information about the foreground from the image area of two people in the image data, and extracts the feature information about the foreground from the image area of the beach. Feature information about the user can be extracted.

In addition, the feature information extractor 30 extracts feature information on the user input data recognized by the input data recognizer 60.

The storage unit 40 stores at least one multimedia data and feature information corresponding to each multimedia data. In detail, the storage unit 40 stores feature information corresponding to each image data, and stores at least one keyframe data corresponding to each video data and feature information corresponding to each keyframe data. The storage unit 40 also stores preset recognition data corresponding to each terminal function information with respect to the terminal function information that is different from the plurality of terminal functions. Here, the recognition data may be edge data or character data. For example, the storage unit 40 may store terminal function information such as an email transmission / reception function or a short message transmission / reception function in correspondence with edge data representing a letter shape.

The touch screen unit 50 recognizes one or more input information input from a user as contact points according to a contact, converts the input information into user input data, and outputs the converted input point. The touch screen unit 50 includes an input area for recognizing a user's touch input and a candidate list display area for displaying a list of retrieved candidate multimedia data. In this case, the input area may also be a multimedia display area for displaying the selected multimedia data, or drag and paste the selected multimedia data from the candidate list displayed on the candidate list display area.

The input data recognizer 60 recognizes user input data such as a text or a drawing input from a user. The input data recognizer 60 includes a character recognizer 61 and an edge recognizer 62. Here, the character recognition unit 61 recognizes whether the user input data is a character, and the edge recognition unit 62 recognizes the edge of the user input data. In other words, the character recognition unit 61 may use a character recognition method such as OCR, and the edge recognition unit 62 recognizes an edge with respect to the drawing data drawn by the user. To recognize these edges, various methods such as homogeneity operator technique, difference operator technique, threshold technique, gradient edge detector, and the like are used. Canny edge detector is a typical method.

The function detector 70 detects preset terminal function information in response to recognition data similar to recognized user input data among preset terminal function information corresponding to each recognition data stored in the storage 40.

Through this, the user can more accurately search for the multimedia data desired by the user, and furthermore, the user can process the content data generated using the searched multimedia data through the terminal function.

2 to 6 are exemplary diagrams for describing a process of searching for image data corresponding to user input data according to an exemplary embodiment of the present invention.

2 is an exemplary diagram for describing a process of searching for image data using drawing data according to an exemplary embodiment of the present invention.

When the user input data 200 is input through the touch screen unit 50 as shown in FIG. 2, the controller 10 recognizes the user input data input through the input data recognition unit 60, and the recognized user input data is a character. Determine whether it is data or drawing data. When the recognized user input data is drawing data such as the reference numeral 200, the controller 10 extracts feature information, that is, edge information, from the drawing data through the feature information extractor 30.

The controller 10 searches in the storage unit 40 for multimedia data having edge information similar to the extracted edge information, and displays a candidate multimedia list for the at least one retrieved multimedia data on the screen of the touch screen unit 50. . The candidate multimedia list displayed as described above may be shown as 210, 220, 230, and 240 of FIG. 2.

3 is an example for describing a process of a user separately inputting user input data according to a foreground and a background, and searching for image data using user input data according to the input foreground and background according to an embodiment of the present invention. It is also.

As shown in FIG. 3, when the user inputs the foreground and the background separately through the touch screen unit 50 to retrieve the multimedia data, the controller 10 may convert the multimedia data 320 corresponding to the user input data of the input foreground and the background. Search for.

Specifically, as shown in FIG. 3, the user inputs two human face pictures 300 for the foreground and a beach picture 310 for the background through the touch screen unit 50. In this case, the touch screen unit 50 may provide a picture input screen capable of inputting a color for drawing and a tool input such as a brush, a pencil, and a pen.

The controller 10 extracts feature information from the user input data 300 for the foreground through the feature information extractor 30 and extracts the feature information from the user input data 310 for the background. Image data such as reference numeral 320 may be searched for using the feature information about the. In other words, the controller 10 searches for the feature information on the foreground and the background similar to the feature information on the foreground and the background of the user input data in the storage 40, and at least one candidate corresponding to the retrieved similar feature information. The list of image data is displayed on the screen of the touch screen unit 50. Then, if any one of the lists displayed by the user is selected, the controller 10 displays the selected image data 320 on the display area of the touch screen unit 50.

4 to 6 are exemplary diagrams for at least one multimedia data corresponding to user input data according to an embodiment of the present invention.

As illustrated in FIG. 4, when the drawing data 400 for the sandy beach and the sea wave is input, candidate image data such as reference numerals 410, 420, 430, and 440 may be retrieved.

In addition, as shown by reference numeral 500 of FIG. 5, when the drawing data about the ship is input as the user input data for the foreground and the drawing data about the sandy beach and the sea wave is input as the user input data for the background, reference numerals 510 and 520. Candidate image data, such as 530, 540, may be retrieved.

Meanwhile, as shown by reference numeral 600 of FIG. 6, when the drawing data for the sandy beach and the sea wave is input as the user input data for the foreground, and the image data for the sunset is input as the user input data for the background, reference numerals 610 and 620. Candidate image data such as 630, 640 may be retrieved. An embodiment of the present invention may use image data selected by a user as input data. For example, when user input data as shown in FIG. 6 is input, the controller 10 extracts feature information from drawing data of a foreground, extracts feature information from sunset image data of a background, and features similar to the two feature information. At least one image data 610, 620, 630, and 640 having information may be retrieved.

7 is an exemplary diagram for describing a process of searching video data using user input data according to an exemplary embodiment of the present invention.

As illustrated in FIG. 7, the controller 10 generates one or more keyframe data corresponding to each video data included in the video list 700 stored in the storage 40, and keyframes for the generated one or more keyframe data. The list 710 is generated and stored in the storage unit 40. Here, the key frame data refers to the most representative frame among scenes with similar screen configurations of movies such as movies and dramas.

The controller 10 extracts feature information on each keyframe data through the feature information extractor 30, and stores the extracted feature information in correspondence with the keyframe data in the storage 40.

Then, when drawing data such as 720 is input from the user through the touch screen unit 50, the controller 10 recognizes the drawing data, extracts feature information, and keyframes corresponding to the feature information similar to the extracted feature information. The data is retrieved from the storage unit 40. The controller 10 constructs a list of at least one candidate keyframe data found as shown by reference numeral 730 and displays the list on the screen of the touch screen unit 50. If a user selects specific key frame data through the touch screen unit 50, the controller 10 displays video data corresponding to the selected key frame data on the screen of the touch screen unit 50.

8 is an exemplary diagram for describing a process of generating content using multimedia data selected according to an embodiment of the present invention and then processing the terminal according to a terminal function.

According to the present invention, a processing mode for processing content may be performed by applying a terminal function to content generated using multimedia data according to a user's request.

If at least one candidate image data is displayed in the candidate list display area as shown in FIG. 8, and there is an input for dragging the multimedia data 800 from the user and pasting it to a location of 810 in the input area, the controller 10 inputs the corresponding input. The selected image data is dragged and pasted at the position 810 in the area.

Thereafter, when user input data including a comment 820, an e-mail address 830, and a letter-shaped drawing 840 for image data is input from the user, the controller 10 recognizes the input user input data and recognizes the input user input data. The terminal function information corresponding to the user input data is detected through the function detector 70. For example, when information regarding the e-mail transmission / reception function is stored in the storage unit 40 in a letter-shaped drawing 840, the function detection unit 70 detects the e-mail transmission / reception function information. The controller 10 performs an email transmission / reception function of transmitting content data displayed on the screen of the touch screen unit 50 to the recognized email address 830 according to the detected email transmission / reception function information.

Through this, the user can more accurately search for the multimedia data desired by the user, and can further process the content data generated by using the retrieved multimedia data using the terminal function.

9 is a flowchart illustrating a process for searching for image data according to user input data according to an embodiment of the present invention.

9, in operation 900, the control unit 10 receives the captured image data through the camera unit 20, and from the image data input in operation 901, the feature information about the image data through the feature information extractor 30. Extract In this case, the feature information includes edge information and color information.

In step 902, the controller 10 associates the extracted feature information with image data and stores the same in the storage unit 40.

In step 903, the controller 10 determines whether user input data is received from the touch screen unit 50, and if so, proceeds to step 904. If not, the controller 10 continuously determines whether user input data is received in step 903. do.

When the user input data is input, in step 904, the controller 10 recognizes the user input data through the input data recognition unit 60, and extracts feature information on the user input data through the feature information extractor 30.

In operation 905, the controller 10 searches for at least one candidate image data having feature information similar to that of the extracted user input data from the storage 40.

In operation 906, the controller 10 generates a candidate image list for the retrieved at least one candidate image data, and displays the generated candidate image list on the display area of the touch screen unit 50.

In step 907, the controller 10 determines whether a selection of a candidate image list is input through the touch screen unit 50, and if the selection is input, the controller 10 proceeds to step 908. If not, the candidate image is input in step 907. It is continuously determined whether a selection for any one of the lists is input.

In step 908, the controller 10 displays the selected image data on the screen of the touch screen unit 50 and ends the image data search process.

10 is a flowchart illustrating a process for searching video data according to user input data according to an embodiment of the present invention.

In step 1001, the controller 10 generates one or more keyframe data through the feature information extractor 30 corresponding to each video data stored in the storage 40.

In step 1002, the controller 10 extracts feature information from the generated keyframe data through the feature information extractor 30.

In step 1003, the controller 10 stores the extracted feature information and the generated keyframe data in the storage unit 40.

In step 1004, the controller 10 determines whether user input data is received from the touch screen unit 50, and if so, proceeds to step 1005. If not, the controller 10 continuously determines whether user input data is received in step 1004. do.

When the user input data is input, in step 1005, the controller 10 recognizes the user input data through the input data recognition unit 60, and extracts feature information on the recognized user input data through the feature information extracting unit 30. do.

In operation 1006, the controller 10 retrieves at least one candidate keyframe data having feature information similar to feature information about the extracted user input data from the storage 40.

In step 1007, the controller 10 generates a candidate keyframe list for the retrieved at least one candidate keyframe data, and displays the generated candidate keyframe list on the display area of the touch screen unit 50.

In step 1008, the controller 10 determines whether a selection of a candidate keyframe list is input through the touch screen unit 50, and if the selection is input, the controller 10 proceeds to step 1009, and if not, the candidate in step 1008. It continuously determines whether a selection for any of the keyframe lists is input.

In step 1009, the controller 10 searches for video data corresponding to the selected keyframe data from the storage 40, displays the searched video data on the screen of the touch screen unit 50, and then ends the image data search process. .

FIG. 11 is a flowchart illustrating a process for processing a content generated using multimedia data as a terminal function corresponding to user input data according to an embodiment of the present invention.

In operation 1100, the controller 10 generates content data using multimedia data selected from a user among the multimedia data searched through the multimedia data search. Such content refers to content data including image data 810 and comments 820 displayed on the screen of the touch screen unit 50 as described above with reference to FIG. 8.

In step 1101, the controller 10 determines whether the user input data is received. If the user input data is received, the controller 10 proceeds to step 1102. If not, the controller 10 continuously determines whether the user input data is received. Here, the user input data may be the email address 830 and the drawing data 840 in the form of stationery.

In operation 1102, the controller 10 recognizes user input data through the input data recognition unit 60. For example, the input data recognition unit 60 may recognize the e-mail address through the character recognition unit 61 and the drawing data through the edge recognition unit 62.

In step 1103, the controller 10 detects terminal function information, which is set in advance in response to the recognized user data, through the function detector 70. In detail, the controller 10 detects the terminal function information corresponding to the recognition data similar to the recognized user input data through the function detector 70. For example, when the e-mail function is set corresponding to the letter-shaped drawing data, the function detector 70 detects the e-mail function information corresponding to the recognized letter-shaped drawing data.

In step 1104, the controller 10 processes the content data generated using the terminal function corresponding to the terminal function information corresponding to the detected terminal function information, and then ends the content processing process. In detail, the controller 10 transmits the content data to the recognized email address using the detected email function.

Through this, the user can more accurately search for the multimedia data desired by the user, and furthermore, the user can process the content data generated using the searched multimedia data through the terminal function.

10:
20: camera unit
30: feature information extraction unit
31: image feature extraction unit
32: video feature extraction unit
40:
50: touch screen unit
60: input data recognition unit
61: character recognition unit
62: edge recognition unit
70: function detection unit

Claims (14)

In the multimedia search apparatus using a user input device,
A touch screen unit for outputting user input data according to a touch input;
A feature information extraction unit for extracting feature information about one or more multimedia data;
A storage unit for storing the multimedia data and the extracted feature information in correspondence with each other;
An input data recognition unit recognizing the user input data;
The feature information extractor extracts feature information on the user input data, and compares the feature information on the extracted user input data with the feature information on each multimedia data to obtain candidate multimedia data having similar feature information. And a control unit for providing the searched candidate multimedia data after searching from a storage unit. The method of claim 1, wherein the touch screen unit,
And outputting the user input data for the foreground and the user input data for the background. The method of claim 1, wherein the feature information about the user input data is:
Multimedia search apparatus comprising edge data, color data. The method of claim 1, wherein the feature information extracting unit,
And generating at least one keyframe data for the moving image data and extracting feature information on the generated at least one keyframe data when the user input data is moving image data. The method of claim 4, wherein the storage unit,
And storing at least one keyframe data of the video data and feature information of the generated at least one keyframe data in correspondence with each other. 6. The apparatus of claim 5,
After comparing feature information on the extracted user input data and feature information on the keyframe data, the candidate keyframe data having similar feature information is searched from the storage unit, and then the video data corresponding to the retrieved candidate keyframe data. Multimedia search apparatus characterized in that it provides. The method of claim 1,
The apparatus may further include a function detector configured to detect device function information corresponding to the recognized user input data.
The storage unit further stores device function information corresponding to at least one piece of recognition data.
The controller generates content data using at least one candidate multimedia data selected from the provided candidate multimedia data, and controls the function detector to detect device function information corresponding to the recognized user input data from the storage. And processing content data by a device function corresponding to the detected device function information. In the multimedia retrieval method using a user input device in the multimedia retrieval device,
Extracting feature information about one or more multimedia data;
Storing the multimedia data and the extracted feature information in correspondence with each other;
Receiving user input data according to a touch input;
Extracting feature information on the received user input data;
Comparing feature information on the extracted user input data with feature information on each multimedia data;
Searching for candidate multimedia data having feature information similar to that of the extracted user input data as a result of the comparison;
And providing the searched candidate multimedia data. The method of claim 8, wherein the receiving of the user input data comprises:
And a process for distinguishing and receiving user input data for foreground and user input data for background. The method of claim 8, wherein the characteristic information about the user input data,
Multimedia search method comprising edge data, color data. 9. The method of claim 8,
Generating at least one keyframe data for the video data when the user input data is video data;
And extracting feature information on the generated at least one keyframe data. The method of claim 11,
And storing the at least one keyframe data of the video data and the feature information of the generated at least one keyframe data in correspondence with each other. The method of claim 12,
Comparing feature information on the extracted user input data with feature information on the keyframe data;
Retrieving candidate keyframe data having feature information similar to that of the extracted user input data;
And providing video data corresponding to the previously found candidate keyframe data. 9. The method of claim 8,
Generating content data using at least one candidate multimedia data selected from the provided candidate multimedia data;
Detecting device function information corresponding to the recognized user input data among device function information corresponding to at least one piece of recognition data;
And processing content data with a device function corresponding to the detected device function information.

Images