JPH0728687A - Multimedia knowledge base device and information offering device - Google Patents

Abstract

PURPOSE:To reduce the generation cost for link relation and to effectively utilize respective medium groups just like a dictionary by describing multimedia information at relatively low cost and linking between meaningful medium groups as to the knowledge base device or information offering device which utilizes multimedia such as still pictures, moving pictures, sounds, and texts in the unit of information integrated into structured data that are meaningful to a human being. CONSTITUTION:Multimedia information of still pictures, moving pictures, sounds, texts, etc., is constituted as a certain meaningful gathering (structured data), the structured data are given meaningful description (3) that a computer can perform processing to define as the unit of management of information called an 'article', and an inference engine retrieves information management units (4) on the basis of a prescribed inference rule to offer information which is necessary for a user. Further, the information management unit is utilized as dictionaries and more complicated information is constituted with the meaningful description (3) granted to the structured data and offered to the user.

Description

Detailed Description of the Invention

[0001]

The present invention relates to still images, moving images, audio,
The present invention relates to a knowledge base system configured by integrating multimedia information such as text as a unit (structured data) having a certain meaning, and an information providing device using such a knowledge base system.

In recent data processing, particularly in the field of knowledge-based systems, human knowledge is input and stored in a data processing device in any form and provided to an unspecified number of users. That research is being conducted.

On the other hand, in recent data processing systems, multimedia information such as still images, moving images, voices, texts, etc. has been dealt with, and hypertext systems, multimedia database systems, etc. have been used in the industry. It is becoming popular.

However, in conventional hypertext and multimedia database systems, still images, moving images,
Since multimedia information such as voice and text is managed individually by name, attribute, and hyperlink, search for it with conventional database technology or follow the hyperlink to obtain the desired multimedia information. There was no way but to get.

That is, hypertext is a means for expressing the complex structure of knowledge information held by human memory in a complicated manner, and is excellent for naturally expressing human memory including multimedia data. However, in order to provide the stored information to other human beings, it is necessary to organize and reconstruct the complicated knowledge structure possessed by human beings.

[0006] Further, the human perception of information is originally multimedia, and for example, in the case of using a kitchen knife, it is expressed only by texts and symbols as in the conventional hypertext or multimedia database system. Difficult to do, for example requiring visual information.

In view of such a situation, it is considered that multimedia information such as a still image, a moving image, a voice, and a text, which has difficulty in information retrieval and provision of information like conventional hypertext, is regarded as independent data. If the integrated information of these multimedia information can be handled as a group of information meaningful to human beings, it is possible to solve the above-mentioned difficulty of hypertext information retrieval and information provision.

[0008]

2. Description of the Related Art FIG. 5 is a diagram for explaining a conventional multimedia database device. As shown in the figure, the conventional multimedia database device manages non-language information such as still images, moving images, and audio from a plurality of texts, etc., by their names, attributes, hyperlinks, and the like.

Therefore, in order to retrieve multimedia information such as a desired still image, moving image, voice, etc., there has been no way but to retrieve by a conventional database technology or to follow hyperlinks in order.

For example, it is only possible to first search the text and then search the text for an image or the like by the above-mentioned hyperlink, but it is not possible to semantically search for information to be known.

[0011]

However, as described above, the human perception of information is originally multimedia, and, for example, when using a kitchen knife, the conventional hypertext or multimedia database is used. It is difficult to express only with texts and symbols like a system, and for example, visual knowledge information is required.

It is easier to construct the knowledge information and to make it easier for the user to comprehend such knowledge information in the form of a moving image rather than writing it in text or a knowledge expression language. Become.

On the other hand, on the other hand, when teaching how to tie a string, it is important to know the name of the knot, the purpose of use, and points to be aware of as words. Therefore, it is important to describe the knowledge symbolically.

However, in the conventional multimedia database system or the like, as described above, the non-verbal information such as moving images, still images and sounds is managed individually by its name, attribute, hyperlink, etc. Therefore, the text is searched first, the image or the like is searched by the hyperlink from the text, and if the description of the predetermined portion is desired from the searched image, the predetermined portion is instructed by the button, There is a problem in that it is only possible to search voice for explanation, and it is not possible to semantically search for information to be known.

For information such as a moving image, the user should look at the edited moving image data in advance, or
There was no way but to look at the fixed link relationship. Also, in the method of effectively using multimedia data for various purposes, it is necessary to define the link relationship individually for the above-mentioned circumstances, and it is an effective search for searching the information for the various purposes. The current situation is that there is no means.

In view of the above-mentioned conventional drawbacks, the present invention does not regard multimedia information such as moving images, still images, and audio as independent data, but combines them to provide a certain meaning to humans. It is the basis of the information that it has, and by adding a semantic representation that can be inferred by a computer to this, it is added as a unit (article) of multimedia information, and the semantic It is intended to provide a method for semantically searching by an inference engine using expressions and descriptions.

[0017]

1 and 2 are explanatory views of the principle of the present invention, and FIG. 1 shows an example of the entire structure.
Shows a configuration example of the "article" of the present invention.
The above problems can be solved by a multimedia knowledge base device and an information providing device configured as follows.

(1) Multimedia information such as a still image, a moving image, a voice, and a text is constructed as a unit (structured data) having a certain meaning, and the structured data is
A semantic description that can be processed by a computer is added as an information management unit called an "article". An article database is constructed based on the information management unit, and inference is performed based on the inference rules described in advance. engine
4 is configured to search the above information management unit and provide necessary information to the user.

(2) The management unit of the multimedia information described in the above item (1) is used as the dictionary 6, and the provided information configuration is made by using the semantic description added to the management unit of the multimedia information. Part 3 configures more complex information and provides multimedia information to the user.

(3) The management unit of the multimedia information described in the above item (1) is used as a constituent unit of information, and the semantic description added to the above constituent unit is used to manage another unit of information. Configure to form a hyperlink between and.

(4) The management unit of the multimedia information described in the above item (1) is provided to an unspecified large number of users using a predetermined communication medium such as a CD-ROM or a communication line. To do.

[0022]

That is, in the present invention, in a multimedia knowledge base system, multimedia information such as moving images, still images, voices, and texts is integrated to construct structured data having a certain meaning to humans. In addition,
A semantic description that can be processed by a computer is added to the data, multimedia information is configured as a unit of information called "article", and based on this unit of information,
An article database is constructed from the information of the above-mentioned semantic description, and the article database is provided with, for example, a knowledge base constituting a classification knowledge system. By creating a link from the category item to each article, the inference engine identifies each article or a set of articles corresponding to each category based on the inference rules described in advance. Then, a pair of information meaningful to the human being, that is, an article, specifically, a composite of a moving image, a still image, a voice, and a text, and a semantic description recognizable by a computer is searched, and the semantic Through the description,
The above-mentioned moving image, still image, audio, text composite, and a group of composites are searched.

Therefore, a composite of a moving image, a still image, a sound, and a text is searched for by semantically described information, and more complex composite information is obtained based on the contents of each field of the semantic description. Can be configured.

According to the above method, like the conventional knowledge base, it is impossible to describe in a form such as a logical formula or a mathematical formula, or it is difficult to describe in various ways that requires a great cost. Knowledge can be described at a relatively low cost by using the above-mentioned multimedia information (that is, article). Moreover, instead of linking the relationship between each media such as video, still image, audio, text like the conventional multimedia database, it is a composite of each video, still image, audio and text. Since the link is created using the relation of the semantic description added to,
The link relationship creation cost can be reduced.

Further, using each field of the above-mentioned semantic description added to each article, a hyperlink is set to another article based on the value of each field and the mutual relation. Then, by means of searching for concrete examples of articles that satisfy the constraint conditions of each field of the semantic description, more complicated and dynamic multimedia information can be constructed,
Can provide valuable information.

An article, which is a unit of information composed of a complex of the moving image, still image, voice, and text, and a semantic description recognizable by a computer, is used as a dictionary, and the unit of the information is By using the added semantic description, more complicated information can be configured and useful multimedia information can be provided to the user.

Further, the above multimedia information is stored in a CD-ROM.
By connecting to an unspecified number of users via a communication medium such as a communication line, an information providing service can be provided to a large number of unspecified users.

[0028]

Embodiments of the present invention will be described in detail below with reference to the drawings. 1 and 2 described above are explanatory views of the principle of the present invention.
3 and 4 are views showing an embodiment of the present invention,
FIG. 3 shows a configuration example of the multimedia knowledge base device, and FIG. 4 shows a flow chart of the search process.

In the present invention, still images, moving images, audio,
A management unit of information called "article", in which multimedia information such as text is organized as a unit (structured data) with a certain meaning, and a semantic description that can be processed by a computer is added to the structured data. age,
Article database based on the information management unit
In order to carry out the present invention, a means for constructing 6 and for inferring engine 4 to search the above information management unit and provide necessary information to the user based on the inference rule described in advance is necessary. It is a means. The same reference numerals indicate the same objects throughout the drawings.

First, in the multimedia knowledge base of the present invention, as shown in FIG. 2, multimedia information such as a still image, a moving image, a voice, and a text has a certain meaning (structured structure). Data), and the structured data is provided with a semantic description that can be processed by a computer. The unit of information consisting of a pair of a moving image, a still image, a voice, and a text complex and a semantic description that can be recognized by a computer is called an "article", and this article is a multimedia knowledge-based information. Use as a unit.

In the above complex, for example, even if the required information is "chopped", there are "chopped" of "onion" and "chopped" of "burdock root".
Divide each media into the smallest units that humans can understand,
In the above example, "chopped onion", "chopped beef", etc. are put together so as to be a unit of the above complex, or "chopped", "onion", "article" of "beef" are used as units. The articles may be generated in advance and provided in combination with the above articles according to the request content from the user.

FIG. 1 schematically shows the overall structure of the multimedia knowledge base device of the present invention, and schematically shows the relationship between the knowledge system and articles. The knowledge system 5 is for each purpose of use of knowledge, for example, in the illustrated example, for each purpose of use such as "fishing", "cooking", and "living creatures".
You can have multiple schemes. This knowledge system 5 is, for example, classified into a tree shape and, for example, links the items classified into the lowest rank and the content of each field of the semantic expression of FIG. 2 added to each article. The knowledge system 5 can be traced from the condition required by the user, the knowledge item most suitable for the condition can be extracted, and the relevant article can be taken out and provided to the user.

The semantic description added to each article may be the same as the method of semantic expression described in the conventional knowledge base. For example, as shown in FIG. Type (= trout), knowledge feature, structure (= classification item, field of semantic expression), feature (= anatomical feature, movement feature, feeding behavior feature, etc.), etc. Corresponding to the constraints specified by the content,
By configuring so as to point to other articles, it becomes possible to provide a more complex knowledge information by extracting a complex article group in response to a request from the user.

The creation of a link based on the above-described constraint condition may be performed dynamically, unlike the conventional fixed hyperlink. For example, if knowledge of in-house procedures requires the information of the section manager to approve the in-house procedures, the section manager of the applicant will vary depending on the actual applicant, but the constraint condition of the field of approver is If the link is defined as a constraint condition that the belonging attribute with the applicant is satisfied with the classification attribute of section manager, appropriate section manager information including multimedia information such as a facial photograph will be displayed according to the actual applicant. The article you have will be dynamically linked to the above-mentioned in-house procedure article.

Next, the configuration and operation of the multimedia knowledge base device of the present invention will be described with reference to FIGS. 1) First, what is a user's knowledge search request, such as "trout ecology", "cooking, for example, how to make curry", or "curry"? "Ecology,""method,""definition,""procedure," etc. are issued. {Refer to processing step 100 in FIG. 4} 2) Based on the contents of the knowledge search request in 1), the search request analysis unit 2 extracts the essential condition that the information requested by the user should satisfy. In the example of FIG. 1 and FIG. 2, it is “trout life”, and as another example, the above “how to make curry” or “chopped onion” may be used. {Refer to processing step 101 in FIG. 4} 3) In the provided information structure unit 3, the structure of information actually provided to the user who has made a request, for example, the classification item of the knowledge system and the article corresponding to the classification item. By recognizing a predetermined field of the semantic expression, a constraint condition link {hyperlink etc.} of information to be searched is determined from the knowledge base (article database) 6. {Refer to processing step 102 in FIG. 4} 4) The inference engine 4 uses the knowledge system based on the above constraint conditions.
Search article database 6 using 5,
By using each field of the semantic expression of the article, the article satisfying the above-mentioned constraint is extracted. {Refer to processing step 103 in Fig. 4} 5) The extracted article is provided information provision unit 3
In the above, it is reconstructed as a composite article based on the semantic expression determined in the above item 3) and provided to the user who made the request. {Processing step 10 of FIG. 4
4, 105} The individual information of the article database 6 is, as described above, an article, that is, a still image, a moving image, an audio,
It is a complex of texts and a pair of the above semantic expressions. As shown in FIG. 1, the article may have another article as an element of the knowledge description via each field of the above-mentioned semantic expression.

When the above article is used as a description component of knowledge, for example, the knowledge of cooking described above,
Knowledge of "chopped onion" is shared with reference to various cooking procedure knowledge. As described above, in the multimedia knowledge base device according to the present invention, more complicated knowledge can be provided by combining the articles according to the requirements from the user.

Although not shown, the article is provided with a service for providing multimedia information in response to requests from a large number of unspecified users through a communication medium such as a CD-ROM or a communication line. be able to.

As described above, the multimedia knowledge base apparatus according to the present invention configures multimedia information such as a still image, a moving image, a voice, and a text as a unit (structured data) having a certain meaning, and Structured data with a semantic description that can be processed by a computer is used as an information management unit called an "article". Based on the information management unit, the inference engine uses the above information based on a predetermined inference rule. Search the management unit and provide the necessary information to the user. Another feature is that the information management unit is used as a dictionary, and more complicated information is constructed by the semantic description given to the structured data and provided to the user.

[0039]

As described above in detail, according to the multimedia knowledge base device of the present invention, a complex of multimedia information such as still images, moving images, voices and texts is added. It is possible to search with the semantic information or to further configure the complex information by using the complex as an element based on the semantic content. In addition, like the conventional knowledge base, various kinds of knowledge that could not be described in a format such as a logical expression or that required a great cost for the description are referred to as multimedia articles referred to above. By using a group of information, it is possible to describe at a comparatively low cost, and like conventional multimedia databases, each media such as still images, moving images, voices, and texts can be described. It is possible to reduce the cost of creating a link relationship by dynamically linking with a semantic relationship instead of hyperlinking. Furthermore, by linking using the semantic expressions added to each article, each article can be used in a composite manner, so each article can be used to store various multimedia information just like a dictionary. It can be configured to make effective use.

[Brief description of drawings]

FIG. 1 is an explanatory diagram (1) of the principle of the present invention.

FIG. 2 is an explanatory diagram of the principle of the present invention (No. 2)

FIG. 3 is a diagram showing an embodiment of the present invention (No. 1).

FIG. 4 is a diagram showing an embodiment of the present invention (part 2).

FIG. 5 is a diagram for explaining a conventional multimedia database device.

[Explanation of symbols]

1 Search request 2 Search request analysis unit 3 Providing information structure unit 4 Inference engine 5 Knowledge system 6 Article database 100 to 105 processing steps Still image, video, audio, text and other multimedia information Structured data, complex Meaning Description, semantic expression article, information management unit, group

Claims (4)

[Claims] 1. Structured multimedia data () such as still images, moving images, voices, and texts is structured as structured data () that is a unit having a certain meaning, and the structured data is structured.
A semantic description () that can be processed by a computer is added to () to form an information management unit () called an "article". Based on this information management unit (), an article database is constructed and described in advance. A multimedia knowledge base device characterized in that the inference engine (4) searches the information management unit () to provide necessary information to the user based on the inference rule that has been established. 2. The multimedia information management unit () according to claim 1 is used as a dictionary (6), and the semantic description () added to the multimedia information management unit () is used. An information providing apparatus, characterized in that the provided information constructing unit 3 constructs more complicated information and provides multimedia information to the user. 3. The multimedia information management unit () according to claim 1 is used as an information unit, and the semantic description () added to the unit is used to store other information. A multimedia knowledge base device characterized by forming a hyperlink () with a management unit (). 4. An information providing apparatus, characterized in that the management unit () for multimedia information according to claim 1 is provided to an unspecified number of users using a predetermined communication medium.