JPH0981590A - Multimedia information retrieval device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide the multimedia information retrieval device which easily retrieves desired video information from multimedia information where video information and character information are integrated. SOLUTION: Matching between the contents meaning expression which is generated by analyzing a natural language as character information stored in a video/character information storage part 108 by a character information analysis part 106 and the retrieval condition meaning expression which is generated by analyzing a natural language inputted as a retrieval condition by a retrieval condition analysis part 102 is discriminated; and when it is judged that matching is successful, a corresponding natural language in the video/ character information storage part 108 is retrieved based on position information stored in a contents meaning expression storage part 103 together with contents meaning expressions, and the correspondence between the retrieved natural language and video information is retrieved to obtain the position where desired video information is stored.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a multimedia information retrieval apparatus and a multimedia information retrieval method for retrieving multimedia information in which video information and character information including natural language are integrated.

[0002]

2. Description of the Related Art Conventionally, according to a video information reproducing apparatus for reproducing video information such as a moving image, a search for video information of a specific scene in a series of video information specifies a position of a frame where reproduction should be started. It was done by.

[0003]

Therefore, according to the conventional video information reproducing apparatus, when the operator tries to retrieve a specific video information, the video information cannot be easily designated.

For example, if a player wants to reproduce a scene in which a touchdown is decided from the video information recorded in a football game, the conventional video information reproducing apparatus can reproduce the video information from the beginning. It is said that the operation is complicated because it is necessary to search for the desired video information by using, or to record the position information where the video information is recorded separately and to search for the desired video information by specifying the position. There was a problem.

On the other hand, in the case of multimedia software typified by the hypertext format, the video information is divided and treated as fragmentary data, and structured together with the character information, such as a chapter or column in a book. That is, the chapter or column having the structure is recorded with a heading such as "First Quarter First Down", and the video information is searched based on such a heading. Therefore, there is a problem that the search cannot be performed unless the heading or the index "XX player touchdown" is attached to the scene in which a certain player decides to touchdown.

In the case of such multimedia software, a method of searching for a character string such as "XX player" or "touchdown" in the character information portion recorded together with the video information can be considered as an explanation. However, in this case, there is a problem in that a number of wrong candidates may be raised and the process for specifying the desired video information is complicated.

The present invention has been made in order to solve the above-mentioned problems, and it is possible to easily retrieve desired video information from multimedia information in which video information and character information are integrated. An object is to provide a search device and a multimedia information search method.

[0008]

In order to achieve the above object, a multimedia information retrieving apparatus according to a first aspect of the present invention comprises a recording means for recording multimedia information in which video information and character information including natural language are integrated. , First creating means for analyzing a natural language recorded in the recording means to create a first semantic expression, input means for inputting a natural language as a search condition, and analyzing the input natural language Second creating means for creating a second meaning expression, matching judging means for judging matching between the first meaning expression and the second meaning expression, and a judgment result of the matching judging means. And a search means for searching the image information recorded in the recording means for the image information corresponding to the input natural language.

According to a second aspect of the multimedia information search apparatus of the present invention, the input means in the first aspect of the multimedia information search apparatus is configured to input a natural language by voice.

According to a third aspect of the present invention, there is provided a multimedia information retrieving apparatus which is provided with replay display means for replaying and displaying retrieved video information in addition to the multimedia information retrieving apparatus according to the first or second aspect. is there.

According to a fourth aspect of the present invention, there is provided the multimedia information retrieval apparatus according to any one of the first to third aspects, wherein the reproduction / display means in the multimedia information retrieval apparatus has a plurality of video information retrieved by the retrieval means. In this case, a plurality of pieces of video information are displayed in a menu format.

According to a fifth aspect of the present invention, there is provided a multimedia information retrieval method, wherein natural language in multimedia information obtained by integrating video information and character information including natural language is analyzed to create a first semantic expression, which is used as a retrieval condition. Second by analyzing the natural language of
Of the first semantic expression and the second semantic expression are determined, and the natural language as the search condition is selected from the video information in the multimedia according to the determination result. It is characterized by retrieving video information corresponding to.

[0013]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be described below with reference to the drawings.

FIG. 1 is a block diagram showing the configuration of a multimedia information retrieval apparatus according to an embodiment of the present invention. In the present embodiment, multimedia information obtained by adding character information in natural language to video information, for example, drama and scenario, sports and commentary, documentary and narration etc. A case will be described in which specific video information is searched for in the multimedia information associated with each other so as to be synchronized.

In FIG. 1, reference numeral 101 denotes a search condition input unit as an input means for the operator to input search conditions for searching multimedia information and instructions for ending the search. The search condition input unit 101 is composed of, for example, an input device such as a keyboard, and the search condition is input in natural language.

The search condition input unit 101 is connected to a search condition analysis unit 102 as a second creating means for analyzing the search condition, and the search condition analysis unit 102 is connected to the matching unit 1.
It is connected via 03 to the content meaning expression storage unit 104 which stores the content meaning expression generated from the character information stored as the multimedia information by the method described later. The matching unit 103 matches the analysis result of the search condition analysis unit 102 and the content meaning expression stored in the content meaning expression storage unit 104.

The content-meaning-expression storage unit 104 is connected to a video / character information storage unit 108 as a recording unit via a character information analysis unit 106 as a first creating unit. The character information stored in the video / character information storage unit 108 is analyzed by the character information analysis unit 106, represented as a content meaning expression, and stored in the content meaning expression storage unit 104. Search condition analysis unit 102 and character information analysis unit 106
Has an analysis dictionary 105 for analysis processing.

The matching unit 103 is connected to a search result display unit 107 as a search unit and a reproduction display unit, and the video information retrieved from the video / character information storage unit 108 based on the matching determination result by the matching unit 103 is , Are displayed on the search result display unit 107.

FIG. 2 is a flow chart showing a search procedure by the multimedia information search apparatus described above.

First, the video / character information storage unit 108
The character information stored in the character information is analyzed by the character information analysis unit 106, and a content meaning expression indicating the content of the character information is created. As mentioned above, the character information consists of natural language,
A natural language processing technique is adopted for the analysis of the natural language. That is, a sentence composed of natural language is divided into “sentences”, then divided into “words” by morphological analysis, and further, the dependency of “phrase” is determined by dependency analysis, and semantic analysis is performed. As a result, a content semantic expression that describes the subject and target of the operation is created (step 201). The created content meaning expression is stored in the content meaning expression storage unit 104. At this time, the position information in the video / character information storage unit 108 in which the natural language corresponding to the stored content meaning expression is stored is also stored in the content meaning expression storage unit 104.

For example, as shown in FIG. 3, video information 30
The multimedia information consisting of 1 and the character information 302 "Smith has decided on the touchdown splendidly" is the content meaning expression 3 as a result of the analysis by the character information analysis unit 106.
It is stored in the content meaning expression storage unit 104 in the format indicated by 03.

In this state, the multimedia information retrieval device waits for an input from the retrieval condition input unit 101 (step S202).

When the input from the search condition input unit 101 is detected, it is judged whether or not the input content is the input of natural language as the search condition (step S203), and the answer is negative (NO). If it is, that is, if an instruction to end the search is input, this procedure is ended. If the answer to step S203 is affirmative (YES), the input natural language is analyzed (step S).
204). That is, a sentence composed of a natural language is divided into "sentences", then divided into "words" by morphological analysis, and further, dependency of "phrases" is determined by dependency analysis and semantic analysis is performed. As a result, a search condition meaning expression indicating the search condition is created. For example, as shown in FIG. 3, when a natural language “Smith has decided to touch down” is input, the result is analyzed by the search condition analysis unit 102 and is expressed in the form shown in the search condition meaning expression 305.

After the analysis of the natural language as the search condition, the search condition meaning expression 305 is associated with the content meaning expression 303 stored in the content meaning expression storage unit 103 in step S201 (step S205). Matching between the content meaning expression 303 and the search condition meaning expression 305 is determined (step S206). Specifically, step S
In the correspondence in 205, search condition meaning expression 3
When 05 is a subset of the content meaning expression 303, that is, when the search condition meaning expression 305 matches a part of the content meaning expression 303, it is determined that the matching has succeeded.

For example, since the search condition meaning expression 305 "Smith has decided the touchdown" is a subset of the content meaning expression "Smith has decided the touchdown splendidly", the matching has succeeded in this case. Is determined. On the other hand, if the search condition meaning expression is "Smith somehow decided to touch down", the matching fails because "however" in the search condition meaning expression and "brilliantly" in the content meaning expression do not correspond. It is judged that it did.

If it is determined in step S206 that the matching has succeeded, the content meaning expression storage unit 103
The natural language corresponding to the content meaning expression 303 is searched on the basis of the position information accumulated together with the content meaning expression 301, and the correspondence between the searched natural language and the video information is searched for to obtain the video / character information. In the storage unit 108,
The position where the desired video information is stored is obtained (step S207). The video information stored in the obtained position is reproduced and displayed as video information 306 by the search result display unit 107 as shown in FIG. 3 (step S).
208). When the reproduction and display of the video information in step S208 is completed, the process returns to the input waiting state in step S202 described above.

If it is determined in step S206 that the matching has failed, a message to that effect is displayed on the search result display section 107 in step S208, and the result is displayed in step S208.
The process returns to the input waiting state of 202.

As described above, according to this embodiment, the content meaning expression generated from the natural language as the character information associated with the video information and the natural language which is the search condition are generated. By determining matching with the search condition meaning expression, it is possible to search for desired video information. Further, since the search condition can be input in natural language, the search can be easily executed.

In the present embodiment, the search condition is input from an input device such as a keyboard or a mouse, but a voice recognition unit is provided so that voice input from a microphone can be used as the search condition. You may comprise.
This makes it possible to input search conditions more easily and accurately.

Further, when performing matching of semantic expressions,
A thesaurus describing the hierarchical relationship of concepts may be used to determine that the matching has succeeded even when the semantic expression of the search condition corresponds to a superordinate concept of the content semantic expression. For example, if the thesaurus has a description of "scoring" as a superordinate concept of "determining touchdown,""Smith brilliantly determined touchdown" for the search condition meaning expression "Smith scored." It can be determined that the content meaning expression "" matches.

Furthermore, when a plurality of candidates are listed as the video information displayed as the search result, the video information is not immediately reproduced, but a plurality of candidates are displayed in a menu format, for example.
The video information to be reproduced may be selected by the operator.

[0032]

As described above, according to the multimedia information search device of the first aspect, the first meaning created by analyzing the natural language recorded in the recording means by the matching determination means. By determining the matching between the expression and the second semantic expression created by analyzing the natural language as the search condition, it is possible to obtain the effect that desired video information can be searched. Moreover, since the natural language is input as the search condition, the operator can easily search for desired video information.

According to the multimedia information search device of the second aspect, since the natural language as the search condition is inputted by voice, the search condition can be inputted more easily and accurately.

According to the multimedia information search device of claim 4, when there are a plurality of video information searched by the search means, the plurality of video information are displayed in a menu format, so that the desired video information can be surely obtained. The effect that it can be reproduced is obtained.

According to the multimedia information retrieval method of claim 5, the first semantic expression created by analyzing the natural language recorded in the recording means and the natural language as the retrieval condition are analyzed. By determining the matching with the created second semantic expression, it is possible to obtain the effect that desired video information can be searched.

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration of a multimedia information search device according to an embodiment of the present invention.

FIG. 2 is a flowchart showing a search procedure by the multimedia information search device shown in FIG.

FIG. 3 is an explanatory diagram showing a procedure for searching video information.

[Explanation of symbols]

 101 search condition input unit 102 search condition analysis unit 103 matching unit 106 character information analysis unit 107 search result display unit 108 video / character information storage unit

Claims (5)

[Claims] 1. A recording means for recording multimedia information in which video information and character information including natural language are integrated, and a natural language recorded in the recording means are analyzed to create a first meaning expression. First creating means, input means for entering a natural language as a search condition; second creating means for analyzing the entered natural language to create a second semantic expression; and the first semantic expression Corresponding to the input natural language from among the video information recorded in the recording means according to the judgment result of the matching judgment means. A multimedia information search device, comprising: a search means for searching video information. 2. The multimedia information retrieval apparatus according to claim 1, wherein the input means is configured to input the natural language by voice. 3. The multimedia information retrieving apparatus according to claim 1, further comprising replay display means for replaying and displaying the retrieved video information. 4. The reproduction / display means is configured to display the plurality of pieces of video information in a menu format when there are a plurality of pieces of video information retrieved by the retrieval means. The multimedia information retrieval apparatus according to item 1. 5. A first semantic expression is created by analyzing a natural language in multimedia information in which video information and character information including a natural language are integrated, and a second natural language is analyzed as a search condition. Of the first semantic expression and the second semantic expression are determined, and the natural language as the search condition is selected from the video information in the multimedia according to the determination result. A method for retrieving multimedia information, characterized by retrieving video information corresponding to.