JP2000076104A - Multimedia information converting and displaying method, server and client, system and device for multimedia information conversion and display, and storage medium storing multimedia information converting and display program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To browse necessary information using link transition based upon a user's choice by displaying multimedia information while judging an optimum display style using structure information on a script of the layer and order relationship, and set, and properties of the multimedia information. SOLUTION: When some link is specified by a user, a structure information acquiring means 130 acquires structure information corresponding to the link script from a structure information storage means 110 storing structure information to which a meaning including a script of the layer and order relation, set, and properties of multimedia information is added. Then a type decision means 140 decides the style of the whole corresponding data by using the property for the set of structure information, the property of a set which can be traced from the data specified by the link, and the property of the link. Then, a converting means 150 converts the whole corresponding data to data representing the decided style and then a data display means 250 displays the converted data. Consequently, necessary information can be browsed over a bird's eye look at the structure of the information.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a multimedia information conversion and display method, a server and a client, a multimedia information conversion and display system and apparatus, and a storage medium storing a multimedia information conversion and display program. In a system for storing, searching, and browsing, when the structure of multimedia information is held by using a link between the information, the structure of the multimedia information is stored by using the meaning added to the link. The present invention relates to a multimedia information conversion display method, a server and a client, a multimedia information conversion display system and apparatus, and a storage medium storing a multimedia information conversion display program for performing semantic expression and display.

[0002]

2. Description of the Related Art Conventionally, in order to express and display the structure of multimedia information, for example, WWW
(World Wide Web) HTML (Hyper Text Mar
kup Language),
Individually designed expressions and displays are performed using a method called an image map that describes a set of images and link destinations at each point thereof.

[0003] By using such expressions / displays, the structure of multimedia information is visualized, and it becomes possible to browse necessary information while bird's-eye view of the information structure by link transition by user selection. ing.

[0004]

However, in the case where the above-mentioned conventional technology is used, it is necessary to individually design according to the structure of the multimedia information in order to express the structure of the multimedia information in a meaningful manner. For this reason, it is necessary to perform similar operations on similar multimedia information structures, which is wasteful.

[0005] Further, by using a fixed description method such as HTML or image map, interactive description with the user such as rearranging according to the user's preference or organizing according to the situation is performed. Processing is difficult. SUMMARY OF THE INVENTION The present invention has been made in view of the above points, and uses structure information such as a description of a hierarchy / order relationship, a set, and an attribute of multimedia information to determine and use an expression / display method most suitable for the structure. By displaying the information on a device that can perform interactive processing with the user, the structure of the multimedia information is automatically visualized, and the display / expression can be changed by interactive processing with the user. An object of the present invention is to provide a media information conversion display method, a server and a client, a multimedia information conversion display system and apparatus, and a storage medium storing a multimedia information conversion display program.

[0006]

FIG. 1 is a diagram for explaining the principle of the present invention. The present invention (claim 1) provides a multimedia information conversion and display method for performing a semantic expression and display of a structure of multimedia information, wherein when a certain link is designated (step 1), the multimedia information is displayed. The structure information such as the set and the attribute corresponding to the description of the link is acquired from the structure information storage unit storing the structure information to which the meaning including the description of the hierarchy / order relation, the set, and the attribute of the link is stored (step 2). ), The type of the entire data is determined using the attribute of the set of structural information, the attribute of the set traceable from the data designated by the link, and the attribute of the link (step 3), and the entire data is determined. The data is converted to data representing the converted model (step 4), and the converted data is displayed (step 5).

According to the present invention (claim 2), when the entire data is converted into data representing the determined type and displayed,
The names and attributes of the converted data are determined, and selectively obtained by using the names and attributes of the converted data from the personal data in the personal data storage means storing the data stored by the user. , Any of the personal data and the converted data is displayed with priority.

[0008] The present invention (claim 3) is a server in a multimedia information conversion and display system for performing semantic expression and display of the structure of multimedia information. A structure information storage unit storing structure information to which a meaning including a description of a set and an attribute is stored; and, when a certain link is designated, a set or attribute corresponding to the description of the link from the structure information storage unit. The type of the entire data is determined by using the structure information acquisition means for acquiring the structure information, the attribute of the set of the structure information, the attribute of the set traceable from the data designated by the link, and the attribute of the link. There is provided a type determining means, and a converting means for converting the entire corresponding data into data representing the type determined by the type determining means.

The present invention (claim 4) is a client in a multimedia information conversion and display system for performing a semantic expression and display of the structure of multimedia information, and is provided from a server in the multimedia information conversion and display system. There is provided data display means for displaying the transferred converted data. The present invention (claim 5) provides personal data storage means for storing data stored by an individual user, and name determination means for determining the name and attribute of data converted by the server of the multimedia information conversion and display system. A data acquisition unit for selectively acquiring the personal data stored in the personal data storage unit using the name and attribute of the converted data; and an arrangement of any one of the personal data and the converted data. Display control means for giving priority to display on the display means.

[0010] The present invention (claim 6) is a multimedia information conversion and display system for performing a semantic expression and display of the structure of the multimedia information. A structure information storage unit storing structure information to which a meaning including a description of an attribute is added, and a structure such as a set or an attribute corresponding to the description of the link from the structure information storage unit when a link is specified. Using a structure information acquisition unit for acquiring information, an attribute for a set of structure information, an attribute of a set that can be traced from data specified by a link, and an attribute of the link,
A server having type determining means for determining the type of the entire data, a converting means for converting the entire data to data representing the type determined by the type determining means, and displaying the converted data transferred from the server. And a client having a data display unit.

According to the present invention (claim 7), in the client, personal data storage means for storing data stored by an individual user, and the name of the converted data transferred from the server of the multimedia information conversion and display system. Name determining means for determining the personal data stored in the personal data storage means and selectively obtaining data using the name and attribute of the data converted from the personal data stored in the personal data storing means; Display control means for giving priority to one of the data arrangements on the display means.

FIG. 2 is a diagram showing the principle of the present invention. The present invention (claim 8) is a multimedia information conversion and display device for performing a semantic expression and display of the structure of multimedia information, and describes a hierarchy / order relationship, a set, and an attribute of the multimedia information. From a structure information storage unit 110 that stores structure information to which a meaning including “” is added, and from a structure information storage unit 110 when a certain link is designated, such as a set or an attribute corresponding to the description of the link. Using a structure information acquisition unit 130 that acquires the attribute of the set of structure information, an attribute of a set that can be traced from data specified by the link, and an attribute of the link.
Type determining means 140 for determining the type of the entire data
And conversion means 150 for converting the entirety of the corresponding data into data representing the type determined by the type determination means 140, and data display means 250 for displaying the data converted by the conversion means 150.

[0013] The present invention (claim 9) provides personal data storage means for storing data stored by an individual user, name determination means for determining the name and attribute of converted data, and personal data storage means. Data acquisition means for selectively acquiring the name and attribute of data converted from the personal data in the storage means, and display control means for preferentially displaying an arrangement of the personal data and the converted data And further comprising:

According to a tenth aspect of the present invention, there is provided a storage medium storing a multimedia information conversion display program for performing a semantic expression and display of a structure of multimedia information. When a certain link is designated, the structure information storage means storing structure information to which a meaning including a description of a hierarchy / order relation, a set, and an attribute of the multimedia information is stored, A structure information acquisition process for acquiring structure information such as a set or an attribute corresponding to the description of the structure, an attribute for the set of the structure information, an attribute of the set that can be traced from the data specified by the link, and an attribute of the link,
A type determination process for determining the type of the entire data;
There is a conversion process for converting the entirety of the corresponding data into data representing the type determined by the type determination process, and a data display process for displaying the data converted in the conversion process.

The present invention (claim 11) provides a name determination process for determining the name and attribute of the converted data, and a method for determining the name and attribute of the converted data from the personal data in the personal data storage means storing the data stored by the user. A data acquisition process for selectively acquiring using the name and attributes of the converted data,
A display control process for preferentially displaying an arrangement of the personal data and the converted data.

As described above, according to the present invention, in the process of acquiring structural information such as a set and an attribute corresponding to a description of a link, the hierarchical / order relation and the set of multimedia information stored in the semantic structure storage means are determined. Acquisition of structure information to which a meaning such as a description of an attribute is added, and using the structure information,
By extracting the attributes of the set that can be traced from the data specified by the link and the attributes of the link, and determining the type of the entire data, the type of the set of data specified by the link is determined. I have.

On the other hand, when converting the entire data to data representing the determined type, the set of data specified by the link is determined using the type determined by the type determination process for the entire data. By converting the background image to be a semantic expression and a description representing each piece of information on the background image and displaying the converted data on the data display device, the data is displayed on the data display device. It is possible to express semantic expressions without individually designing according to the structure of media information, and browse necessary information while bird's eye view of information structure by link transition by user selection Is possible.

Therefore, when a certain link is specified,
By automatically acquiring and visualizing the structure of the multimedia information, it becomes possible for the user to make a link transition while viewing the information structure from a bird's-eye view. As described above, according to the present invention, it is possible to automatically perform a design according to the structure of multimedia information, which is impossible with the conventional technology, and to perform a semantic expression thereof. In addition, it is possible to browse necessary information while bird's-eye view of the information structure is provided by the link transition by the user's selection.

Further, in the present invention, in the process of determining the name and attribute of the converted data, in the above-mentioned multimedia information conversion and display method, the whole data is converted into the determined type of data. From the converted data, by parsing the data, the name and attribute are determined, and the personal data stored in the personal data storage means are selectively used by using the name and attribute of the converted data. get. At this time, from the personal data stored by the individual user stored in the personal data storage means using those names and attributes, only data suitable for displaying this time is selectively obtained, and The arrangement of any of the data for use and the converted data is preferentially displayed on the data display device. When displaying the personal data and the converted data on the data display device, if the expression emphasizes the change of the user, the personal data is
In the case of an expression that emphasizes general classification, the converted data is arranged with priority, so that the data is mixed with the converted data in the arrangement closest to the state where the user saved the data. It can be displayed on a display device.

Therefore, when a certain link is specified,
By automatically acquiring and visualizing the structure of multimedia information in a form that the user has mixed the previously saved state and newly acquired data, the display / expression by interactive processing with the user can be realized. The link transition can be performed while making the change. In this way, after performing a semantic expression of the structure possessed by multimedia information, sorting according to the user's preference is performed, and it is saved or displayed in an organizational manner according to the situation. It is possible to perform interactive processing with the user.

Further, according to the present invention, the process of acquiring structure information such as a set or an attribute corresponding to the description of the link is performed by describing the hierarchy / order relation, set, or attribute of the multimedia information stored in the meaningful structure storage means. Acquisition of structure information to which a meaning such as is added, and using the structure information, extract the attribute of the set that can be traced from the data specified by the link and the attribute of the link, and determine the model of the entire data. By the determination process, the type of the set of data specified by the link is determined.

On the other hand, in the process of converting the entirety of the relevant data into data representing the determined type, the set of data specified by the link is used by using the type determined in the process of determining the type of the entirety of the relevant data. , And convert it into a description that represents each piece of information on the background image that becomes a semantic expression,
By displaying the converted data on the data display device by displaying it on the data display device, it is possible to express their meaning without having to individually design according to the structure of the multimedia information The user can browse necessary information while viewing the structure of the information by link transition by the user's selection.

Therefore, when a certain link is designated,
The structure of the multimedia information is automatically acquired and visualized, and the user can make a link transition while viewing the information structure from a bird's-eye view. As a result, it is possible to perform semantic expression and display without individually designing according to the structure of the multimedia information, and to provide a bird's-eye view of the information structure by link transition by user selection. However, it is possible to browse necessary information.

Further, according to the present invention, in the process of determining the name and attribute of the converted data, the data is subjected to syntax analysis from the data converted by the process of converting the entire data to the determined type of data. Thus, the name and the attribute are determined, and selectively obtained from the personal data stored in the personal data storage unit using the name and the attribute of the converted data. From the personal data stored by the individual user stored in the personal data storage means using those names and attributes, only data suitable for displaying this time is selectively obtained, and the personal data and One of the converted data is preferentially displayed on the data display device. When displaying the personal data and the converted data on the data display device, the personal data is converted in the case of the expression emphasizing the change of the individual user, and the conversion is performed in the case of the expression emphasizing the general classification. By preferentially arranging the data, the data can be mixed with the converted data and displayed on the data display device in an arrangement closest to the state where the user has stored the data.

Therefore, when a certain link is specified,
By automatically acquiring and visualizing the structure of multimedia information in a form that the user has mixed the previously saved state and newly acquired data, the display / expression by interactive processing with the user can be realized. The link transition can be performed while making the change. In this way, after performing a semantic expression and display of the structure of the multimedia information, rearrangement according to the user's preference is performed, and the result is stored or displayed in an arrangement method according to the situation. It is possible to perform interactive processing with the user such as.

The present invention also provides a multimedia information conversion and display system comprising a server having a meaningful structure storage means, a structure information obtaining means, a type determination means, and a client having a data display means, a meaningful structure storage means, and a structure. It is possible to realize two construction patterns as a multimedia information conversion display device in which the information acquisition unit, the type determination unit, and the data display unit are one device.

[0027]

FIG. 3 is a block diagram of a multimedia information conversion and display system according to the present invention. The system shown in FIG. 1 includes a server 100 and a client 200. Although FIG. 1 shows an example in which the multimedia information conversion and display device is realized in a client / server type, communication between a client and a server is performed by communication between processes in a computer or message communication between objects. It can also be realized as a single system (multimedia information conversion and display device) by recalling a function call.

The server 100 has a semantic structure storage 11
0, a server-side communication control unit 120, a meaningful structure acquisition control unit 130, a type determination unit 140, and a description conversion unit 150. The semantic structure storage unit 110 of the server 100 stores structure information to which a meaning such as a description of a hierarchy / order relation, a set, or an attribute of multimedia information is added.

The server-side communication control section 120 communicates with the client 200. Meaning structure acquisition control unit 130
Acquires structure information such as a set and an attribute corresponding to the description of the link from the meaningful structure storage unit 110. The type determining unit 140 determines the type of the entire data by using the attribute corresponding to the set of the structure information, the attribute of the set traceable from the data specified by the link, and the attribute of the link.

The description converter 150 converts the entire data into data representing the determined model. Client 2
Reference numeral 00 denotes a starter 210, a display controller 220, a data acquisition controller 230, a client-side communication controller 240, and a data display device 250.

The start unit 210 starts the display control unit 220. The display control unit 220 displays the data converted by the server 100 on the data display device 250. The data acquisition control unit 230 acquires the data converted by the server 100 and transfers it to the display control unit 220.

The client-side communication control section 240 performs communication with the server 100. The data display device 250 displays the data passed from the display control unit 220. next,
The operation of the above configuration will be described. When a certain link is specified, the semantic structure acquisition control unit 130 of the server 100 acquires, from the meaningful structure storage unit 110, structure information such as a set and an attribute corresponding to the description of the link.

Next, the type determining unit 140 determines the type of the entire data by using the attribute for the set of structural information, the attribute of the set that can be traced from the data specified by the link, and the attribute of the link. . Description converter 15
0 indicates that the entire data is converted into data representing the model determined by the model determining unit 140, and the server-side communication control unit 120
Transfer to

Thus, the server-side communication control unit 120
Transmits data to the client 200 via the network. In the client 200, the data acquisition corrector 230 passes the data transferred from the server 100 to the display controller 220 and displays the data on the data display device 250.
As a result, the structure of the multimedia information can be automatically acquired and visualized, and the user can perform the link transition.

[0035]

Embodiments of the present invention will be described below with reference to the drawings. The present invention is based on the international standard SGML (Standard Gene
This is effective for a general conversion system of multimedia information structured by a structured description language represented by a ralized markup language).

In this embodiment, a description will be given using HTML (Hyper Text Markup Language) as an example of a structured description language. [First Embodiment] FIG. 4 shows a configuration of a multimedia information conversion and display system according to a first embodiment of the present invention. The system shown in the figure realizes a multimedia information conversion and display system in a client / server type. Communication between a client and a server is performed by communication between processes in a computer, message communication between objects, function call, and the like. By reading this, it is also possible to realize a single system (multimedia information conversion and display device).

The system shown in FIG. 1 includes a server 100 and a client 200. The server 100 includes a meaningful structure storage unit 110, a server-side communication control unit 120, a meaningful structure acquisition control unit 130, a type determination unit 140, and a description conversion unit 150. The client 200 includes an activation unit 210, a display control unit 220, a bookshelf-type data acquisition control unit 230, a client-side communication control unit 240, and a bookshelf-type data display device 250.

The meaningful structure storage unit 110 stores structure information to which a meaning such as a description of a hierarchy / order relation, a set, or an attribute of multimedia information is added as shown in FIG. In the example of FIG. 5, a structural link L from data "root" to two groups G1 and G2.
1, L7 is out, and both groups G1 and G2 are b
It has an attribute of "book".

As a data format for storing the structure information added with the meaning as shown in FIG. 5, for example, it can be stored in a table format as shown in FIGS. FIG.
Is an example of a link management table for storing structure information to which a meaning is added as shown in FIG. 5 according to the first embodiment of the present invention. URL (Uniform Resource Locato) corresponding to the link
r) the address of information on the Internet called U
The ID of the data corresponding to the RL, the hierarchy LV of the data, a list of links coming from the data, and the attributes of the links are registered. The link number root is always registered in this table.

FIG. 6 shows the data root shown in FIG.
Corresponds to the URL “http: //server/index.html”, which is data of the 0th hierarchy, and indicates that links L1 and L7 are set up from there. Similarly, data D1 is registered as link L1, and "http: // server /
It corresponds to the URL "book1 / chap1.html", which is the data of the first hierarchy, and indicates that links L2, L3, and L4 are provided from there.
The link 1 is a link representing “structure” according to the link attribute, and after the link to the data “one layer lower”, the link numbers L2 to L11 are described in the same format as L1. Here, the description of “= L3” and “= L4” in the link destination columns in L5 and L6 means that L3, L4, etc., which have already been expanded, are referred to. Only link attributes are significant. The link attribute “reference” indicates that the link has no structural meaning.

FIG. 7 shows an example of a group management table for storing structure information added with a meaning as shown in FIG. 5 according to the first embodiment of the present invention.
For D, the configuration data that constitutes the group and the group attribute that represents the meaning of the group are registered.
In addition to this table, there is always a group root that includes all data and has no group attribute.

In FIG. 7, groups G1 to G4 in FIG. 5 are registered. In the group G1, the data D1 to D4 in the link management table of FIG. 6 are D1 → D2 → D3 →
It is shown that they belong in the order of D4. Further, having the attribute “book” indicates that this group has a fundamental structure. Similarly, in the group G2, the data D7 to D11 are changed from D7 to D7.
D8 → D9 → D10 → D11, and have a basic structure. Group G3 includes only data D8. Group G4 indicates that data D9 to D11 correspond to D9 → D10 → D11. It is shown that they belong in order. The overlap of data belonging to groups G2 to G4 indicates that groups G3 and G4 are included in group G2.

Next, the operation of the multimedia information conversion and display system shown in FIG. 4 will be described step by step. FIG.
As a first embodiment of the present invention, a bookshelf-type expression / display is described for a list-like structure in which semantic structure information to be expressed / displayed is obtained by collecting a plurality of pieces of information. It describes the case of performing.

The user gives an instruction to the system by the starting unit 210 shown in FIG. However, when the system is in the initial state, the following processing is performed assuming that the root link is specified as an instruction by the user. When the designation of the link by the user is transmitted to the activation unit 210, the designation of the link is transmitted from the activation unit 210 to the display control unit 220.

When the display control unit 220 receives the designation of the link, the designation of the link is transmitted to the bookshelf-type data acquisition control unit 230. When the link specification is received, the bookshelf-type data acquisition control unit 230 determines whether the link specification is a specification of a (local) link existing in the own computer. Then, the designation of the link is transmitted to the client-side communication control unit 240.

The client-side communication control unit 240 uses the communication path to transmit the information required for the designation of the transmitted link and the communication between the client and the server, using the communication path.
, And waits for a response from the server 100. When implementing a cache mechanism, the client side communication control unit 240 can implement the cache mechanism. In the server-side communication control unit 120, the client-type communication control unit 24
Waiting for a request from 0. When there is a request from the client side communication control unit 240, the link specification is extracted from the communication information of the request, and the meaningful structure acquisition control unit 1
Send to 30.

The meaningful structure acquisition control unit 130 searches the meaningful structure storage unit 110 using the sent link designation, and acquires structure information corresponding to the link designation. The structure information as shown in FIG. 5 is stored in the meaningful structure storage unit 110. First, the group specified by the designation of the link is acquired, and the type determination unit 14
Send to 0.

If there is an attribute of the group specified by the specification of the link, the type determining unit 140 sets the attribute as the type of the entire data. Conversely, if there is no group attribute, the type of the entire data is determined using the first group attribute and link attribute that can be traced from the data specified by the link specification. In this embodiment, since the link of the root is specified, the attribute of the group root specified by the specification of the link cannot be used. Therefore, it is determined that the model is of the “bookshelf type” using the attribute “book” of the groups G1 and G2 which can be traced from the data root and the attribute “structure (one layer below)” of the links L1 and L7. As the bookshelf-type structure information, it is not necessary to have more detailed structure information than the group having the “book” attribute, so the structure information in the form as shown in FIG. 8 is extracted. Here, for example, when the link L1 is given as a link specification, the group G1 specified by the link L1 has the attribute “book”, and the structure information in the same group is sent to the meaningful structure acquisition control unit 130. Is transmitted, the structure information as shown in FIG. 9 is extracted.

If the type cannot be determined by the type determining unit 140, it is regarded as a "network type" type, and structural information constituted only by the specification of the requested link is extracted. Also, by including the type of information requested from the client-side communication control unit 240 as one of the information necessary for the communication between the client and the server, the processing of the type determination unit 140 is not performed, and the meaningful structure acquisition control unit is not executed. At 130, it is also possible to obtain structural information that has been forcibly deformed to match the type of the requested information.

The structure information for which the processing in the type determination unit 140 has been completed is sent to the description conversion unit 150 and converted into a description format as shown in FIG. Since the corresponding structure information is determined to be “bookshelf type” by the type determination unit 140, the bookcase type is described. In the example of FIG.
A section sandwiched between <booklist> to <booklist> represents one bookshelf, and <index> to </ in
dex> represents the data on which the bookshelf is based, and is used to determine attributes such as the title of the bookshelf. Where <
include src = “URL”> means that the part is replaced with the data specified by the URL. <Book src = “URL”> represents one book. If the attribute of each book can be determined, “attrib” as an option of <book>
ute name = “attribute” ”.

For example, <book src = “http: // s
erver / book1 / chap1, html ”title =“ Example book 1 ”
When psize = "100">, the U specified by src
The title of the book type information indicated by the RL “http: //server/book1/chap1.html” is “example book 1”, indicating that the total number of pages is 100 pages. On the other hand, when the type determination unit 140 determines that the type is the “main type”, the <incl> sandwiched between <book> to </ book>
ude src = “URL”> represents one book, and the order of <include src = “URL”> describes the arrangement order inside the book.

The description of the structure information converted by the description conversion unit 150 is sent back to the server-side communication control unit 120. The server-side communication control unit 120 adds information necessary for client / server-side communication to the description of the structure information,
A response is returned to the client-side communication control unit 240 using the communication path. In response to the response from the server 100, the client-side communication control unit 240 receives the description of the structure information. The client-side communication control unit 240 determines whether the description of the structural information is the information requested by itself using the received information necessary for the client-server communication,
If the information is not the information requested by the user, an error is sent to the bookshelf-type data acquisition control unit 230. If it is the information requested by the user, the description of the structure information is stored in the bookshelf-type data acquisition control unit 2.
Send to 30.

If the bookshelf-type data acquisition control unit 230 originally determines that the local link is designated,
The local structure information description is read. If the local structure information is not local, the description of the structure information sent from the client side communication control unit 240 is received. The description of the structure information is
Since it is of the "bookshelf type", the type of the bookshelf is specified based on the attribute of the bookshelf in the description of the structure information, and an image representing the bookshelf and data on the book arrangement are created. At this time, if the attribute of the book in the description of the structure information can be used, a semantic arrangement can be realized by adding a process of arranging according to the information. Normally, when book attributes are not available, books are arranged in the order in which they can be arranged.

The display control unit 220 acquires an image of the bookshelf based on the data sent from the bookshelf-type data acquisition control unit 230, converts the arrangement of the books into coordinates in the image, and displays a spine cover representing the book. Of the book, arrange the title of the book,
It is sent to the bookshelf-type data display device 250 to obtain a display as shown in FIG. FIG. 11 is a diagram schematically showing an example of a bookshelf type expression / display of the first embodiment of the present invention. With the image representing the bookshelf as a background image, two books representing books based on the description of the structural information acquired from the client-side communication control unit 240 are arranged on the left desk. These expressions correspond to the URLs shown in the figure, and when these are selected by the user, this URL can be requested to the server as a link, and this type of information can be obtained. FIG.
Here, for the sake of explanation, a simplified image is used, but in reality, by displaying a more realistic display, it is an expression that can evoke a judgment based on the experience of the user. further,
FIG. 11 can be said to be an expression obtained by visualizing the structure of multimedia information as a book and a bookshelf for organizing the book.

As the bookshelf type data display device 250, GU
By using a display having an I (Graphical User Interface), by specifying the expression of a book on the GUI by the pointing device, an instruction that the corresponding link has been selected can be transmitted to the activation unit 210.
Next, the operation of the type determination unit 140 will be described.

FIG. 12 is a flowchart of the type determining section according to the first embodiment of the present invention. Step 101) This step is performed by the type determination unit 140.
And receives the link specifying the content requested from the client 200 sent from the meaningful structure acquisition control unit 130, and starts the process. Step 102) Thereafter, a group specified by the link is specified, and it is determined whether or not the group has an attribute.

Step 103) If there is an attribute,
The attribute is adopted as the model of the group and the process of the model determining unit 140 ends. Step 104) If there is no attribute in step 102, BT and BST are set to 0. BT and BS
T indicates that the group is “book type” or “book shelf type” later.
It is used as an index for determining whether or not.

Step 105) Then, Step 10
The data specified by the link passed to 1 is analyzed, and all links L that can be traced from the data are analyzed.
For (i), the following steps 106 to 1
09 is repeated. Step 106) Branching is performed according to the attribute of the link L (i) and the attribute of the group to which the data specified by the link L (i) belongs. First, "boo" is added to the attribute of the group to which the data specified by the link L (i) belongs.
If "k" is included, it is understood that the group specified by the link passed to step 101 is a bookshelf-type structure that manages book information.

Step 107) For this reason, after the group is determined to be of the “bookshelf” type, step 101, that is, the processing of the type determining unit 140 is ended. Step 108) If the attribute of the link L (i) is “structure” to the lower or the same rank, there is a possibility that a hierarchical “book type” information structure is described. The BT is incremented by 1 to indicate the contribution to the “book type” information structure, and the loop of step 105 is continued.

Step 109) Similarly, link L (i)
Attribute is a higher "structure" or just "reference"
In this case, the information structure of the “bookshelf type” that summarizes various pieces of information having no hierarchical relationship with each other may be described. Represent the contribution to the information structure and continue the loop of step 105. Otherwise, the loop of step 105 is continued without doing anything.

Step 110) Branching is performed using the values of BT and BST obtained as a result of step 105.
In this embodiment, the simplest determination method is adopted, and BT is set to B
If it is smaller than ST, the process proceeds to step 111; otherwise, the process proceeds to step 112. Step 111) If BT is smaller than BST,
The group is determined to be a “bookshelf” type, and step 10
1, that is, the process of the type determination unit 140 ends.

Step 112) If the BT is equal to or greater than the BST, the group is determined to be of the “book” type, and
01, that is, the process of the type determination unit 140 ends. If (BT + BST) is not equal to or larger than a certain threshold value in step 110, the group may be determined to be a “network” type. By the above-described processing, the type of the content (group) specified by the link passed to the type determination unit 140 can be determined, and thereafter, information necessary for configuring the data of the type is added. It can be acquired through the structure acquisition control unit 130.

As described above, by using the multimedia information conversion and display system according to the embodiment of the present invention, the structure of the information specified by the link is determined, and the data expression that is optimal for the determined information structure, and The structure of the multimedia information is automatically visualized using an easy-to-understand display method using metaphor or the like. In this example, the user recognizes that, in this example, the bookshelf is expressed here in a metaphoric way, but that each piece of information, such as a book, is a collection of a certain amount of information. That is, in other words, it is possible to select necessary information, that is, a link to be transited, while viewing the structure of the multimedia information from a bird's-eye view.

[Second Embodiment] Hereinafter, a second embodiment of the present invention will be described. FIG. 13 shows the configuration of a multimedia information conversion and display system according to a second embodiment of the present invention. In the figure, the bookshelf type data acquisition control unit 230 is modified in the multimedia information conversion and display system according to the first embodiment shown in FIG. 13, the same components as those in FIG. 4 are denoted by the same reference numerals, and description thereof will be omitted.

Since the configuration of the server 100 is the same as that of FIG. 4, the description is omitted. The client 200 includes an activation unit 210, a display control unit 220, and a bookshelf-type data acquisition control unit 2.
30, a client-side communication control unit 240, a bookshelf type data display device 250, a name determination unit 260, a personal bookshelf data storage unit 270, and an arrangement unit 280.

The description of the structure information received from the server 100 by the client-side communication control unit 240 is sent to the name determination unit 260 if there is no error. The name determination unit 260 receives the description of the structure information sent from the client-side communication control unit 240, and analyzes the syntax to determine the name and the attribute. The parsing is performed, for example, in a document such as HTML specified by <include src = “URL”> between <index> and <index>.
This is realized by performing processing such as acquiring a character string marked up by <title> in <head>. The acquired name and attribute are stored in the client-side communication control unit 2
The information is sent to the bookshelf-type data acquisition control unit 230 together with the description of the structure information sent from the server 40.

The bookshelf type data acquisition control unit 230, if it is originally determined that the local link is designated,
The local structure information description is read. If the local structure information is not local, the name and attribute sent from the name determination unit 260 and the structure information description sent from the client-side communication control unit 240 are received. Since the description of the structure information is of the “bookshelf type”, data is acquired from the personal bookshelf data storage unit 270 when performing expression / display of the bookshelf type stored locally in the client 200.

The personal bookshelf data storage section 270 basically stores data stored locally for each user as shown in FIG. 14 in a table format. Of course, if a book attribute exists, the attribute is stored for the corresponding ID. The URL in FIG. 14 corresponds to <booksrc = “URL”> written in the description of the structure information sent from the client-side communication control unit 240. Also, in the “Bookshelf Attributes” and “Position” columns, the type of bookshelves that are stored as a result of the interactive processing by the user are displayed. Is displayed. The book with the ID “B1” is displayed on all bookshelves since the attribute column of the bookshelf is “<all>” and the position column is blank. It can be understood that the book of "" is a book displayed in "books" and "middle" on the bookshelf entitled "NTT". The bookshelf attribute field may be not only the bookshelf name but also a character string such as a server name included in the URL specifying the bookshelf, or the bookshelf attribute itself such as “bookshelf of search result”. Similarly, the position column may be a fine specification such as “third from the middle right of the book” or a rough specification such as “desktop”.

The bookshelf-type data acquisition control unit 230 selectively acquires only data matching the name and attribute sent from the name determination unit 260 from the data stored in the personal bookshelf data storage unit 270. Then, the local information is sent to the arrangement unit 280 together with the description of the structure information sent from the client-side communication control unit 240. Arrangement part 280
In this example, data sent from the bookshelf-type data acquisition control unit 230 is analyzed / mixed and arranged as an expression of a bookshelf. When the characteristics of the bookshelf, such as expressions and expressions that emphasize general classification, are obtained, the arrangement is performed using the characteristics. When the characteristics of the bookshelf cannot be obtained, an expression that emphasizes the change of the user is used.

In the process of the arrangement unit 280, first, an image representing a bookshelf as a background image and data relating to the arrangement of books are generated from the names and attributes sent from the name determination unit 260. After that, the personal data acquired from the personal bookshelf data storage unit 270 is transmitted from the client-side communication control unit 240 in the case of the expression that emphasizes the change of the individual user, and the personal data acquired from the personal bookshelf data storage unit 270 in the case of the expression that emphasizes the general classification. By arranging the described data of the structural information with priority, the converted data is mixed with the converted data in the arrangement closest to the state in which the user has stored the data. At this time, if the attribute of the book in the description of the structure information can be used, a semantic arrangement can be realized by adding a process of arranging according to the information. Normally, when book attributes are not available, books are arranged in the order in which they can be arranged. When the arrangement of all books is completed, the generated data is sent to the display control unit 220.

The display control unit 220 acquires an image of the bookshelf based on the data sent from the arrangement unit 280, converts the arrangement of the books into coordinates in the image, and acquires an image such as a spine representing the book. , Arrange the title of the book, and the bookshelf type data display device 2
50 to obtain a display as shown in FIG. FIG.
It is the figure which showed typically the example of expression / display of the bookshelf of the 2nd Example of this invention. With the character string of the title of the book shelf and the title as the background image, two books representing the book by the description of the structural information acquired from the client-side communication control unit 240 and the personal bookshelf data storage unit 270 are placed on the left desk. B3 and B1 according to the data acquired from the personal bookshelf data storage unit 270 are placed in the bookshelf on the right side.
Has been arranged. In FIG. 15, the image is simplified for the sake of explanation. However, in reality, by displaying a more realistic display, it is an expression that can evoke a judgment based on the experience of the user. Further, FIG. 15 can be said to be an expression of the result of visualizing the structure of the multimedia information as a book and a bookshelf for organizing the book.

As the bookshelf type data display device 250, GU
By using a display with I (Graphical User Interface), a GUI with a pointing device
The instruction of the above book can be transmitted to the activation unit 210.
Further, by dragging and dropping the displayed book, the position and the like are changed, and transmitted to the bookshelf-type data acquisition control unit 230 from the display control unit 220 and reflected in the personal bookshelf data storage unit 270, thereby allowing the user to communicate with the user. The display / expression can be changed by the interactive processing of.

Next, the description of the structural information acquired from the local or client-side communication control unit 240 in the bookshelf type data acquisition control unit 230, the name determination unit 260, and the arrangement unit 280, and from the personal bookshelf data storage unit 270 Processing for mixing the acquired data will be described. FIG.
9 is a flowchart showing the operation of the second embodiment of the present invention.

In the figure, a description will be given of a case of an expression in which a change of a user is emphasized. Step 201) In this embodiment, since the processing for bookshelf-type data is described, it is determined whether or not the structure information passed from the local or client-side communication control unit 240 is the “bookshelves” model. If not, the process is terminated. Actually, similar processing is performed for various types. If the structure information is of the “bookshelf” type, the following steps 202 and thereafter are executed.

Step 202) The structure information as shown in FIG. 10 passed from the client side communication control unit 240 is analyzed, and the name of the bookshelf is obtained from the portion between <index> to </ index>. The name is <include
src = “URL”> in a document such as HTML. Step 203) Create an empty bookshelf and an empty registration list. Here, the bookshelf refers to a designation of an image representing the appearance of the bookshelf specified by the name of the bookshelf, and a set of book numbers and coordinate information that can be stored on a desk, a bookshelf, or the like. The registration list indicates an ordered list for registering a book whose position to be registered on the bookshelf later is not determined.

Step 204) The following steps 205 to 208 are repeated for all the elements B (i) of the personal bookshelf data obtained from the personal bookshelf data storage unit 270 as shown in FIG. Step 205) If the attribute of the element B (i) of the personal bookshelf data is acquired, and if "<all>" matching all the attributes or the name of the bookshelf obtained in step 202 is included, the following is performed. The processing from step 206 to step 208 is executed. Otherwise, the loop of step 204 is continued.

Step 206) Branching is performed according to the arrangement information of the element B (i) of the personal bookshelf data. Step 207) When B (i) does not have the layout information, it can be considered that the book type information may be placed at any position. Add to the list.

Step 208) If B (i) has arrangement information, it is arranged on the bookshelf created in step 203 using the arrangement information of B (i). At this time, if the arrangement information is duplicated, it is added to the beginning of the registration list created in step 203. As described above, of the book type information not stored in the personal bookshelf data storage unit 270, only necessary information can be registered in the bookshelf being created.

Step 209) All the descriptions such as <book src = “URL”> registered in the structure information as shown in FIG. 10 passed from the client side communication control unit 240 are registered in the registration created in step 203. Add to the list. Step 210) The process of step 211 is repeated for all the elements B (i) registered in the registration list.

Step 211) B (i) is arranged in an empty portion of the bookshelf sparsely filled in step 208. At this time, the <book described in the structure information as shown in FIG.
Using attributes such as src = “URL” arrangement = “bookshelf”> as a hint, in this example, it may be arranged on a bookshelf.
Variations, such as rearranging the titles of the book type information and then arranging them, can be provided.

As described above, by using the multimedia information conversion and display system according to the present embodiment, the structure of the information specified by the link is determined, and the data expression, metaphor, etc., optimal for the determined information structure are determined. By using an easy-to-understand display method that is used together with, the structure of multimedia information is automatically visualized, and interactive processing with the user can be performed. This allows the user to
In this example, the metaphorical expression of the bookshelf, which is expressed here, can be recognized as being a collection of information to a certain extent, such as a book. Display by interactive processing with the user /
By realizing the change and arrangement of expressions, it becomes possible to perform the arrangement as we do for bookshelves in the real world.

Further, the bookshelf-type data acquisition control unit 230, the name determination unit 260, and the arrangement unit 280 can be automatically arranged by book or bookshelf attributes, or by using an organizing / expression method other than bookshelves. You can also build a convenient system. Further, the present invention constructs each component of the server 100 and the client 200 shown in FIGS. 4 and 13 as a program, and a disk device connected to a computer used as a server or a client, a floppy disk or a CD-ROM. The present invention can be easily realized by storing it in a portable storage medium such as the above and installing it when implementing the present invention.

The present invention is not limited to the above embodiments, but can be variously modified and applied within the scope of the claims.

[0084]

As described above, according to the present invention, in a system for storing, retrieving, and browsing multimedia information, when a structure between information is retained by a hyperlink, the information is added to the hyperlink. When performing the semantic expression of the multimedia information using the given meaning, it is possible to express those semantic expressions without individually designing according to the structure of the multimedia information. Thus, there is an effect that necessary information can be browsed while a bird's-eye view of the information structure is provided by link transition by the user's selection.

Further, according to the present invention, after performing a semantic expression of the structure of the multimedia information, sorting according to the user's preference, organizing according to the situation, etc. are performed. This has the effect that interactive / dynamic processing with the user can be performed. Furthermore, according to the present invention, when performing the semantic expression of the multimedia information using the meaning added to the hyperlink, the design according to the structure of the multimedia information is not individually performed. In addition, it is possible to express these meanings, and it is possible to browse necessary information while bird's-eye view of the information structure by link transition by the user's selection.

Further, according to the present invention, after performing a semantic expression of the structure of the multimedia information, sorting according to the user's preference, organizing according to the situation, etc. are performed. Thus, there is an effect that interactive processing with the user can be performed.

[Brief description of the drawings]

FIG. 1 is a diagram for explaining the principle of the present invention.

FIG. 2 is a principle configuration diagram of the present invention.

FIG. 3 is a configuration diagram of a multimedia information conversion and display system according to the present invention.

FIG. 4 is a configuration diagram of a multimedia information conversion and display system according to a first embodiment of the present invention.

FIG. 5 is an example of structure information to which a meaning such as a description of a hierarchy / order relation, a set, or an attribute of multimedia information according to the first embodiment of the present invention is added.

FIG. 6 is an example of a link management table for storing structure information added with a meaning as shown in FIG. 5 according to the first embodiment of this invention.

FIG. 7 is an example of a group management table for storing structure information added with a meaning as shown in FIG. 5 according to the first embodiment of this invention.

FIG. 8 is an example (No. 1) of bookshelf type structure information extracted by the type determination unit of the first embodiment of the present invention.

FIG. 9 is an example (part 2) of bookshelf-type structure information extracted by the type determination unit according to the first embodiment of the present invention.

FIG. 10 is an example of a bookshelf-type description output from the description conversion unit according to the first embodiment of this invention.

FIG. 11 is a diagram schematically showing an example of a bookshelf type expression / display of the first embodiment of the present invention.

FIG. 12 is a flowchart of a type determining unit according to the first embodiment of the present invention.

FIG. 13 is a configuration diagram of a multimedia information conversion and display system according to a second embodiment of the present invention.

FIG. 14 is an example of personal bookshelf data stored in a table format in a personal bookshelf data storage unit according to the second embodiment of this invention.

FIG. 15 is a diagram schematically illustrating an example of a bookshelf type expression / display according to the second embodiment of the present invention.

FIG. 16 is a flowchart showing the operation of the second embodiment of the present invention.

[Explanation of symbols]

Reference Signs List 100 server 110 meaningful structure storage unit, meaningful structure storage unit 120 server side communication control unit 130 structure information acquisition unit, meaningful structure acquisition control unit 140 type determination unit, type determination unit 150 conversion unit, description conversion unit 200 client 210 Activation unit 220 Display control unit 230 Data acquisition control unit, bookshelf type data acquisition control unit 240 Client side communication control unit 250 Data display means, data display device, bookshelf type data display device 260 Name determination unit 270 Personal bookshelf data storage unit 280 Arrangement section

 ──────────────────────────────────────────────────続 き Continuing on the front page (72) Inventor Masaru Miyamoto 3-19-2 Nishi-Shinjuku, Shinjuku-ku, Tokyo Inside Nippon Telegraph and Telephone Corporation (72) Inventor Hideaki Ozawa 3-192-1, Nishishinjuku, Shinjuku-ku, Tokyo No. Nippon Telegraph and Telephone Corporation F-term (reference) 5B075 ND16 ND35 ND36 PQ02 PQ13 QT06 5B082 GC02

Claims (11)

[Claims] 1. A multimedia information conversion and display method for performing a semantic expression and display of a structure of multimedia information, wherein when a certain link is designated, a hierarchical / order relationship or a set of multimedia information, From the structure information storage unit in which the structure information to which the meaning including the description of the attribute is added is stored,
Acquires structural information such as a set or an attribute corresponding to the description of the link, and uses the attribute for the set of the structural information, the attribute of the set that can be traced from the data specified by the link, and the attribute of the link. A multimedia information conversion and display method, comprising: determining the type of the entire data; converting the entirety of the data into data representing the determined type; and displaying the converted data. 2. When converting the entire data to data representing a determined model and displaying the converted data, a name and an attribute of the converted data are determined, and data stored by an individual user is stored. And selectively acquiring from the personal data in the personal data storage means using the name and attribute of the converted data, and displaying the arrangement of the personal data and the converted data with priority. 2. The multimedia information conversion and display method according to 1. 3. A server in a multimedia information conversion and display system for performing a semantic expression and display of a structure of the multimedia information, the server including a description of a hierarchical / order relationship, a set, and an attribute of the multimedia information. A structure information storage unit in which structure information to which meaning is added is stored; and when a certain link is specified, structure information such as a set or an attribute corresponding to the description of the link is acquired from the structure information storage unit. Structure information obtaining means, and type determination means for determining the type of the entire data by using the attribute for the set of the structure information, the attribute of the set that can be traced from the data specified by the link, and the attribute of the link. Converting means for converting the entirety of the corresponding data into data representing the type determined by the type determining means. Over server. 4. A client in a multimedia information conversion and display system for performing a semantic expression and display of a structure of the multimedia information, wherein the client is a client in the multimedia information conversion and display system. A client having data display means for displaying data. 5. A personal data storage unit for storing data stored by an individual user, and a name determination unit for determining a name and an attribute of converted data transferred from a server of the multimedia information conversion and display system. Data acquisition means for selectively acquiring from the personal data stored in the personal data storage means using the name and attribute of the converted data; and any one of the personal data and the converted data 5. The client according to claim 4, further comprising: display control means for displaying on the display means with priority given to the arrangement of the client. 6. A multimedia information conversion and display system for performing a semantic expression and display of a structure of multimedia information, wherein a meaning including a description of a hierarchy / order relation, a set, and an attribute of the multimedia information is provided. Structure information storage means in which the added structure information is stored; and structure information for acquiring structure information such as a set and an attribute corresponding to the description of the link from the structure information storage means when a certain link is designated. Acquiring means; type determining means for determining the type of the entire data by using an attribute for the set of the structure information, an attribute of a set that can be traced from the data specified by the link, and an attribute of the link; A server having conversion means for converting the entirety of the data into data representing the model determined by the model determination means; Multimedia information conversion display system characterized by having a client having a data display means for displaying the converted data. 7. The client according to claim 1, wherein the client is a personal data storage unit for storing data stored by the user, and determines a name and an attribute of the converted data transferred from the server of the multimedia information conversion and display system. Name determination means; data acquisition means for selectively acquiring from the personal data stored in the personal data storage means using the name and attribute of the converted data; and the personal data and the converted 7. The multimedia information conversion and display system according to claim 6, further comprising: display control means for displaying any of the data on the display means with priority. 8. A multimedia information conversion and display device for performing a semantic expression and display of a structure of multimedia information, wherein a meaning including a description of a hierarchy / order relation, a set, and an attribute of the multimedia information is provided. Structure information storage means in which the added structure information is stored; and structure information for acquiring structure information such as a set and an attribute corresponding to the description of the link from the structure information storage means when a certain link is designated. Acquiring means; type determining means for determining the type of the entire data by using an attribute for the set of the structure information, an attribute of a set that can be traced from the data specified by the link, and an attribute of the link; Converting means for converting the entirety of the corresponding data into data representing the type determined by the type determining means; and displaying the data converted by the converting means. Multimedia information conversion display device characterized by having over data display means. 9. A personal data storage means for storing data stored by an individual user, a name determination means for determining a name and an attribute of the converted data, and a personal data storage means for the personal data storage means. Data acquisition means for selectively acquiring from the data using the name and attribute of the converted data; and display control means for displaying any of the personal data and the converted data preferentially. The multimedia information conversion display device according to claim 8, further comprising: 10. A storage medium storing a multimedia information conversion and display program for performing a semantic expression and display of the structure of multimedia information, wherein the link is mounted on the multimedia information conversion and display device. Is specified, from the structure information storage means in which the structure information to which the meaning including the description of the hierarchy / order relation, set, and attribute of the multimedia information is added,
A structure information acquisition process for acquiring structure information such as a set or an attribute corresponding to the description of the link; an attribute for the set of the structure information; an attribute of a set that can be traced from data specified by the link; and an attribute of the link Using, a type determination process of determining the type of the entire data, a conversion process of converting the entirety of the relevant data into data representing the type determined by the type determination process, and the data converted by the conversion process. A storage medium storing a multimedia information conversion display program having a data display process for displaying. 11. A name judging process for judging a name and an attribute of the converted data, and a process of converting the converted data from personal data in a personal data storage unit storing data stored by an individual user. The multimedia information according to claim 10, further comprising: a data acquisition process for selectively acquiring using the name and the attribute; and a display control process for preferentially displaying an arrangement of the personal data and the converted data. A storage medium storing a conversion display program.