JP3417444B2 - Multimedia client server system - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a client / server system for storing and searching multimedia information, and more specifically, to display a structure of multimedia information from a server system storing multimedia information. , A multimedia client / server system that reduces the amount of information transferred to the client system for processing.

[0002]

2. Description of the Related Art Conventionally, multimedia information of a server is
When used while being transferred to a client in response to a request from a user, all the information necessary for display and processing is transmitted by the client. As such a system, for example, a system such as a video on demand that transfers video information selected by a request from a user until a stop request from the user is given,
As a hypermedia typified by WWW (World Wide Web), there is a system that transfers and displays information associated with a link in response to a user's request to refer to a hyperlink. .

[0003]

In the conventional multimedia system, since a large amount of multimedia information is transferred and displayed as a whole, it is very difficult to browse the information via a network or the like. I needed a long wait or a powerful system. However, like viewing a physical book, the information can be grasped as a whole, but at the same time, only one or two pages of information, which is extremely limited, is presented. There is also a law. When such a representation method is realized on a computer system, it is inefficient to transfer all information for the purpose of grasping the entire information, such as thickness information.

An object of the present invention is to reduce the transfer amount of data for grasping the entire information in a client server system which provides multimedia information by a representation method of information such as a book.

[0005]

A client-server system of the present invention three-dimensionally displays, on a flat display, multimedia information such as a text, an image, a voice, and a moving picture in which a logical structure of a document is described by a text. It is a system that aims to list as the above multidimensional information, is a system that uses an expression that imitates a book as a method of displaying multidimensional information of three dimensions or more, and opens a physical book. As shown in, the presentation of some of the information represented by the shape of the book and the appearance of the information such as headlines is provided to the user at the same time. System.

[0006] Specifically, in the invention of claim 1, in a client system for presenting information to a user, a means for requesting transfer of information to a server system storing a large amount of multimedia information, and a request for the information. When the page displaying the content of the information does not correspond to the page represented by the shape of the book, the means for generating a transfer request of only the attribute information of the data having a large transfer amount of information, and the server. Displaying information on each page from means for interpreting the logical structure of information obtained from the system from text, means for presenting information to the user according to the logical structure, and description of the attribute information and text description And a means for calculating the size of the entire information from the calculated area, and the server system searches only the attribute information and By providing a means for converting into a description of the logical structure of text, even for large data such as image data, it is possible to display a part of the information related to the user's request and an appearance display of the entire information. To be able to do simultaneously.

According to a second aspect of the present invention, in a server system for storing information, when attribute information such as an information display area of each page in a client system for presenting information is obtained, the server system is provided. By providing a means for calculating the area necessary for displaying an image and a means for calculating the area necessary for displaying the text, the external information is data such as text and image. By transferring only the size of the required area to the client system regardless of the type of data, a small amount of data can be transferred and some of the information related to the user's request can be displayed and the overall appearance of the information can be improved. So that it can be displayed at the same time.

[0008]

BEST MODE FOR CARRYING OUT THE INVENTION An embodiment of the present invention will be described in detail below with reference to the drawings. INDUSTRIAL APPLICABILITY The present invention is effective with respect to an overall system for browsing a partial portion of information structured by a structured description language or the like as necessary.
HTML, which is a kind of structured description language, such as a document sent from a WWW server to a WWW client.
A case of browsing information described using (Hyper Text markup Language) will be described as an example.

FIG. 2 shows a specific display example of the present invention.
This is an example in which information is three-dimensionally displayed by simulating a physical book on a flat display. In FIG. 2, 1 is a main frame, 2 is a nail heading, 3 is a sticky note, and 4 is the thickness of the book.

In FIG. 2, as an example of the representation method of a book in the real world on a computer display, by displaying the book frame 1, the area where the contents of the book are displayed and the contents such as sticky notes and nail captions are displayed. Separated from the area where the structure for searching is displayed. In the area where the structure for searching the content is displayed, for example, as the nail heading 2, large items provided by the author and marks for important items are displayed in the vertical direction on the left and right of the main frame 1. On the contrary, like the sticky note 3, by displaying the mark required by the user in the horizontal direction above the main frame 1, the structure of the mark created by the author and the structure of the mark created by the user are separated. . Furthermore, by the information 4 of the thickness of the books displayed on the left and right sides and the bottom of the book frame 1, by pasting the current position information in the information expressed as one book and the nail heading at the position of the corresponding thickness, It is possible to display the amount of each information classified by the nail heading from the outside. In this way, by combining the separation of the area for displaying information and the display for expressing the structure of information, it is possible to express the structure of a book in the real world on a flat display.

The present invention is one of the methods of expressing structured information on a computer display. In each of the following embodiments, the case of expressing the structure of the book shown in FIG. 2 will be described as an example. However, the present invention is not limited to books, and the same effect can be obtained when a system such as a binder that provides information in a structured representation in the real world is represented on a computer display.

<First Embodiment> FIG. 1 shows a system configuration diagram of a first embodiment of the present invention. In the figure, 10 is a client system for presenting multimedia information to the user, 2
0 is a server system that stores a large amount of multimedia information.

The client system 10 is composed of a display, a keyboard and a mouse, receives a request for searching browsing information from a user, and serves as a user interface for a user to browse the information in a book format as shown in FIG. The information interface unit 11, the search condition input by the user, the condition for acquiring a plurality of information in order to make the information described in the structured description language into the main structure, and the searched information A protocol conversion unit 12 that determines the type of access to the server system 20 in order to obtain data described in a structured description language from the value of the location where The client communication unit 13 that transmits to the system 20 and receives the information from the server system 20 and the server communication from the server system 20 The document structure analysis unit 14 that analyzes the received data and determines the information that should be newly acquired from the server system 20 to form the main structure, the information that is displayed on the screen, and the information that is newly acquired from the server system 20. The display information control unit 15 that separates the conditions for acquiring the information and the main information expression unit 16 that changes the information to be displayed on the screen into the expression of this format.

The server system 20 receives the search condition from the client system 10 and transfers the search result to the client system 10 by the server communication unit 21.
And a protocol analysis unit 22 that analyzes the protocol of the search condition sent to the server communication unit 21 and searches the file storage unit 23 for necessary information, and a large number of structured description languages in HTML. A file storage unit 23 for storing multimedia data, and a protocol analysis unit 2
The data type classification unit 2 for selecting the data to be transferred to the client system 10 from the type of the search condition and the search result of the file storage unit 23 sent from
4 and the selected data in the client system 10
And a transfer data generation unit 25 for changing the data to be sent to.

The major flow of the process in the embodiment of the present invention is the URL (Uniform Reform) given by the user.
(Source Locator) as an input of the book information interface unit 11, requesting acquisition of data related to the URL given to the server system 20, and using the obtained related data as information in the book format as shown in FIG. To present to the person. FIG. 3 shows a flowchart of detailed processing steps in the first embodiment.

When the user wants to browse the information in a book form, the user inputs a URL indicating the information to be browsed to the book information interface section 11, and step 10
1, the URL is the book information interface unit 11
Stored in the buffer.

In the embodiment of the present invention, when one identifier called URL, which is the information to be browsed, is given, the information of the identifier to the related information included in the information to be browsed is also acquired and stored in the book. The feature is that it is displayed in the shape of a book as shown in Fig. 2 by ordering like chapters and sections.
The moment a user browses information, the moment needs all the information contained in the page that is currently open and some information about the page that is not currently open, not necessarily a book. It does not need all of the information expressed as.

Therefore, a server is used to determine whether to acquire the information to be actually displayed on the screen or to acquire the information necessary for expressing the entire information in this format from the information of the pages not opened. In order to instruct the system 20, in step 102, the protocol conversion unit 12 makes U
As for the type of access in the RL, a URL is given with a value of "http" when acquiring all information, and a value of "template" when acquiring only information of the skeleton for expressing the entire book. . This created URL
Is sent to the client communication unit 13, and step 103
At, the request is transferred to the server communication unit 21 of the server system 20 as a search request.

The URL sent to the server system 20 is
The protocol analysis unit 22 separates the access type and the identifier indicating the storage location of the information. In step 104, the information indicated by the identifier is acquired from the file storage unit 23, and the access type and the data type classification unit are acquired. Sent to 24.

In the data classification unit 24, in order to determine whether to return the skeletal information to the client or the data as it is, the step 1
In 05, the access type added to the sent data is "http" or "template".
To find out. When the value is "template", when the amount of data when transferring information is large, one information has a large amount of data in order to return only the size of the required area to the client. In step 106, it is determined whether or not the image data has the image data. Then, when the determination result of step 106 is image data, a value of only the size of the image added to the image data is extracted and sent to the transfer data generation unit 25 in step 107. On the other hand, when the access type value given to the URL is “http” or text data having a relatively small data amount that is not image data, the data obtained from the file storage unit 23 is directly transferred to generate transfer data. Send to section 25.

In the transfer data generator 25, step 10
In step 8, the obtained data is used as a description in HTML format and is transferred to the client. If the data in the 16-file storage unit 23 is described in the HTML format, the data other than the image data used as the skeleton may be directly used as the transfer data. The data in the file storage 23 is HT
If not described in ML format, an HTML tag is added according to the structure of data information. Step 10
9, the data described in HTML format is 14
The data is transferred from the server communication unit 21 to the client communication unit 13.

In the client system 10, the document structure analysis unit 14 analyzes the content of the HTML data sent from the server system 20 with reference to the tag in step 110 to obtain one book-type information. URLs that have a reference relationship to the information required for are selected. The selection is performed as follows, for example. In the WWW, information described in HTML is often described between "data" by describing a URL from a file of one HTML data called "HomePage" to a reference to other information called an anchor. Represents a reference relationship. Since the data producer usually stores information on the same server system as a set of data formed by a chain of URLs, the URL to the same server system exists in the acquired HTML data. If so, the information referred to by this URL is selected as the URL, because it should be developed in the same book when the information is expressed as one book.

Next, in step 111, the display information control section 15 displays the contents of the HTML data on the book information interface section 11 with reference to the tags.
The size of the area required for pasting data on each page area is calculated and allocated to the page. One URL
The information described in HTML acquired by does not always fit in the area of one page, so calculate the size of the area required for the characters and images included in the data,
Data is sequentially allocated to the next page area. At this time, every time the page to which the data is attached changes, the value of the page counter in the display information control unit 15 is incremented by one.

Of the information assigned to each page, the information displayed by a nail heading on the surface of the book, such as a chapter title, and the information on the currently opened page are displayed on the 11-book information interface section 11. In order to obtain image data that can be generated, in step 112, the book information interface unit 11 generates an image of the nail index, the frame of the book, and the like in FIG. indicate.

The processing from step 102 onward in FIG. 3 is repeated until there is no information to be expanded. When all the information that should be made into one book has been developed, the processing is stopped. At this time, the book information interface unit 11 displays information in the book format as shown in FIG.

URL for pointing information in the HTML server
As shown in FIG. 4, in addition to the standard access type shown in FIG. 4A, the access type “template” for returning only the skeletal information shown in FIG. Add a URL with.

As an example of the information described in HTML,
As shown in FIG. 5 (A), the structure and data of a document are described by text tags, and for images and the like, only information that can refer to entities such as file names is described in text. When the information shown in FIG. 5A is displayed on the book type screen according to the present embodiment, information such as the thickness of the book is added to the physical book so that the physical book is displayed as shown in FIG. 5B. Same expression.

<Embodiment 2> In the present embodiment 2, in order to reduce the amount of data to be transferred as much as possible with respect to the information necessary only for displaying the thickness and displaying the nail heading, when accessing the server, The size of the frame of the book to be displayed is sent to the server side, the size of the area required when the book is displayed in the book is calculated on the server side, and returned to the client.

FIG. 6 shows a system configuration diagram of the second embodiment. 6 has almost the same configuration as that of FIG. 1, but a display area calculation unit 26 is added to the server system 20 side. In the display area calculation unit 26, the data type classification unit 2 is used by using information on the size of the display frame of the book given in advance.
For the information given from 6, the size of the area to be displayed on the book is calculated and sent to the transfer data generation unit 25.

FIG. 7 is a flow chart showing the process steps in the second embodiment. Client system 10
Receives a request for information retrieval from the user, the URL is sent from the client communication unit 13 to the server communication unit 21, and the protocol analysis unit 2 in the server system 20.
2, the necessary information is acquired from the file storage unit 23 and sent to the data type classification unit 24, Steps 101 to 10
The processing of No. 4 is the same as that of the first embodiment and can be easily inferred, so the description thereof is omitted.

The data classification unit 24 of the server system 20 determines whether to return the skeleton information to the client or the data as it is according to the value of the access type. Access type is "http" or "te
Check if it is "plate". Value is "template"
If so, in order to return information about only the size of the required area to the client, in step 106,
It is determined whether it is image data. This is because in the case of image data, it may be necessary to first calculate only the size of the image and then reduce the area so that the area fits within a predetermined book frame range.

The display area calculation unit 26 is required when arranging the image data, which is data of only the size of the necessary area, and the character data described in text on each page of the book to be used. In step 121, the area information is calculated by referring to the book information storage unit 260 existing in the display area calculation unit 26 based on the book type data added to the access type. Get information about the book you need. The book information necessary for area calculation includes, for example, the width of each page, the height of the area where the information on each page can be displayed, the size of the text in the text, the size of the text in the headline, and the font. .

Regarding the information expressed on the screen by text, in step 122, the value of the information of the structure described in HTML, such as H1 and H2 as shown in FIG. The area required for display is calculated with reference to the value of the character size for the structure of information when displaying the information in the obtained book format. As an example of the calculation of the size of the area, when displaying text, the page width is determined by the width of the character added to the font information given in step 121 and the width of the page of the book that can be displayed. The number of characters that can be displayed in one line on the screen book is determined by dividing the width of the character by the width of the character, and the number of characters of the text to be displayed is divided by the number of characters that can be displayed in one line. Calculate the number of rows. By multiplying the required number of lines and the height of the character, it is possible to calculate the area required on the screen.

Similarly, in the case of image data, the vertical and horizontal size information given to the image data itself, and the horizontal size and vertical length of the page of the book that can be displayed given in step 121 are displayed. If either the vertical or horizontal direction of the image data exceeds the size of the page area by comparing with the information, the value that exceeds the horizontal or vertical direction of the page is applied. The value is made the same, and the other is contracted so as to keep the same ratio. If both the vertical and horizontal values exceed the area of the page, the values that are largely exceeded are adjusted to the values of the corresponding page. If neither the vertical nor horizontal value of the image data exceeds the area,
The vertical and horizontal values of the image data and the size of the area displayed on the book are used. The simplest method of contracting an image can be achieved by thinning out image data at regular intervals.

After calculating the size of the area required for displaying each data of the text and the image, the display area calculating unit 26 calculates the size of the nail heading like the data having the tag H1. In the case of such special text data, the text information is added to the size information in step 123.

Next, in the transfer data generation unit 25, in step 108, as in the first embodiment, the data of only the size of the sent area and the data to be actually displayed are set to H.
The data is in TML format. Steps 109 and 110
The process of is similar to that of the first embodiment and can be easily inferred, so the description thereof is omitted.

In step 111, the display information display unit 15 of the client system 10 displays the content of the HTML data in which the size of the area required for placing on the book is calculated in the book information interface unit 11. In each case, allocate to each page. Information written in HTML acquired by one URL does not always fit in one page area, so if the size of the area required for characters or images contained in the data exceeds one page. , In the case of text, the area is divided into lines, and the remaining area is sequentially allocated to the next page area. In the case of an image, it became clear that the page could not fit in the page. At that point, the pages will be allocated from the beginning of the next page. At this time, the value of the page counter in the display information control unit 15 is incremented by one each time the page to which the data is attached changes sequentially. The process of step 112 is the same as that of the first embodiment, and can be easily inferred, so the description thereof is omitted.

The description of the URL in the second embodiment is as shown in FIG.
As shown in, when only the skeleton of information is returned by the access type, the type of book is input after the symbol of template. The name of the book type may be any data as long as the server system and the client are consistent with each other. For example, if there is a large-sized book and a small-sized book, FIG. As shown in (B), "Lar
An identifier such as "ge" or "Small" may be added.

In the second embodiment, the server system 20
As an example of the data regarding the book included in the book information storage unit 260 in the display area calculation unit 26, the width and height of the displayable area corresponding to at least the size of each book, as shown in FIG. Information and font information for each information type are stored.

FIG. 10 is an example comparing the case of the normal format of the HTML format data transferred from the server system to the client system with the first and second embodiments of the present invention. As shown in FIG. 10, in the case of the first embodiment, the amount of data transferred in the case of the second embodiment is smaller than that in the case of normal HTML data. Specifically, it is as follows.

In the normal HTML data of FIG. 10A, the text data is about 280 Bytes, and the image data is about 1 dot due to compression, and about 0.4600 Bytes is about 73600 Bytes, and the total data is about 74 KB Bytes. It needs to be transferred for this one page.

On the other hand, when the invention of the first embodiment shown in FIG. 10 (B) is carried out, the image data occupying a large amount of the transfer data has a value of only the size of the image, and therefore the image The data transfer amount is about 30 Bytes, and only a total of about 300 Bytes of data is transferred to the client as skeletal data. Therefore, it is possible to compress the data transfer amount to about 0.5, as compared with the case of transferring the normal HTML description data for the representation as a book.

Further, when the invention of the second embodiment shown in FIG. 10C is implemented, the data transfer amount can be reduced also for the text data. In the example shown in FIG. The data is 80 Bytes and the total is about 100 Bytes.
es, which is 0 when transferring normal HTML data.
It is about 1%. Therefore, for example, for a 100-page book, when transferring the information 2 pages actually displayed on the screen and the information expressed as other books, the normal H
When transferring TML data, it is necessary to transfer information for 100 pages, but when implementing the second embodiment, it is only necessary to transfer data for 2 + 100 * 0.001 = 2.1 pages. Good.

<Embodiment 3> In this embodiment, as shown in FIG. 11, the server system 20 side directly gives a URL from the 18 transfer data generation unit 25 to the protocol analysis unit 22 in some cases. This makes it possible to reduce the transfer data. The other configuration is the same as that of FIG.

An example of sending data from the transfer data generating unit 25 to the protocol analyzing unit 22 is a case where text data including a tag for referring to image information is sent. FIG. 12 shows an example of this case. That is, FIG.
As shown in (A), when the text data sent to the transfer data generation unit 25 in the second embodiment returns the skeleton data, the image data indicated by the “IMG” tag is changed to the image shown in FIG. ) On the server system as shown in
The number of data transfers with the client can be reduced by substituting the data of the tag indicated by “GDATASIZE”.

Furthermore, when the HTML description data whose structure is expressed by the text is always described sequentially from the beginning of a page, the size of both the text data area and the image data area, Since the height of each page of the frame of the book is stored in the book information storage unit 260 of the display area calculation unit 26, the height values of each text area and image data area are sequentially calculated as the height of each page area. The value is subtracted from the value of, and when it is no longer allocated on the page, it is sequentially subtracted from the height of the next page.
It is also possible to calculate the number of pages required for the data of the HTML description to be displayed, or it is possible to use only the number of pages as the HTML description.

In the case of the skeleton information, with regard to the size of the area, it is possible to further reduce the amount of transfer data by returning only the required number of pages to the client.

[0048]

As described above, according to the present invention, information whose information structure is described in text is
When only the attribute information is required when presenting the information from the information of the book structure, reduce the transfer amount of the information from the server system that stores the information to the client system that presents the information to the user. Is possible.

According to some actually existing HTML description data, it was possible to reduce the data as shown in Table 1 when the data for the appearance of the information was evaluated.

[0050]

[Table 1]

The data format is as follows:
Each procedure consists of a picture and a brief description, and is in a bulleted form in which both images and text are mixed. About one page of tourist information is attached to each page of image data, and textual guide information is attached to each page. The information on the government office building is basically described in text, and images are embedded in places.

On the average of Table 1, according to the present invention, the information showing the appearance of the same information is transferred to the user only by transferring about 3% of the data as compared with the case of transferring all the data. It became possible to provide the data, and it was possible to greatly reduce the amount of data to be transferred.

Further, according to the invention of the second embodiment, as for the information expressed by the text, by transferring only the data of the size of the area, less information is provided to the user from the server system storing the information. It becomes possible to provide the information of the appearance of the entire information of the same book shape only by transferring the information to the client system.

When the same data as in Example 1 was used for evaluation, the results shown in Table 2 were obtained.

[0055]

[Table 2]

According to Table 2, the effect is very high when the text is the main subject, and the data amount can be reduced to about 30% as compared with the case of the first embodiment. Even in the case where the area of each text is small, such as a bullet point, only by transferring a data amount of about 80% as compared with the first embodiment,
The same effect as in Example 1 could be obtained.

[Brief description of drawings]

FIG. 1 is a system configuration diagram of a first embodiment of a multimedia client / server system according to the present invention.

FIG. 2 is an example in which information is three-dimensionally displayed imitating a physical book on a flat display according to an embodiment of the present invention.

FIG. 3 is a flowchart showing a process of processing according to the first embodiment.

FIG. 4 is a diagram showing a structure of an identifier when a search request is given to the server system in the first embodiment.

FIG. 5 is a diagram showing a relationship when information of contents to be transmitted described in the HTML description in the first embodiment is converted into information of this format.

FIG. 6 is a system configuration diagram of a second embodiment of a multimedia client server system according to the present invention.

FIG. 7 is a flowchart showing a process of processing according to the second embodiment.

FIG. 8 is a diagram showing a structure of an identifier when a search request is given to the server system in the second embodiment.

FIG. 9 is an example of a data structure of a main information storage unit according to the second embodiment.

FIG. 10 is a diagram comparing data returned from the server system in the normal cases of the first and second embodiments.

FIG. 11 is a system configuration diagram of a multimedia client server system according to a third embodiment of the present invention.

FIG. 12 is a diagram showing an example of data returned from the server system according to the third embodiment.

[Explanation of symbols]

10 client system 11 Book information interface section 12 Protocol converter 13 Client Communication Department 14 Document structure analysis unit 15 Display information controller 16 Information Representation Department 20 server system 21 Server Communication Department 22 Protocol converter 23 File storage 24 Data type classification section 25 Transfer data generator 26 Display area calculator 260 book information storage

─────────────────────────────────────────────────── ─── Continuation of the front page (51) Int.Cl. ⁷ Identification code FI G06F 17/30 360 G06F 17/30 360Z 419 419B (72) Inventor Hiroshi Hamada 3-19-2 Nishishinjuku, Shinjuku-ku, Tokyo Within this telegraph and telephone company (56) Reference JP-A-9-265477 (JP, A) JP-A-9-218773 (JP, A) Uri Kaori and 3 others, browsing and searching HTML information by book metaphor , Information Processing Society of Japan, Research Report (96-IM-24), Japan, Information Processing Society of Japan, January 12, 1996, Volume 96, No. 2, p. 37-44 Kyoichi Arai and 4 others, Window interface with page turning function: Book Window, Information Processing Society of Japan Research Report (91-HI-36), Japan, Information Processing Society of Japan, May 9, 1991, Vol. 91, No. 34, 36-3 (58) Fields investigated (Int.Cl. ⁷ , DB name) G06F 12/00 G06F 13/00 G06F 17/30

Claims (2)

(57) [Claims] 1. A multimedia client / server system comprising a server system for storing multimedia information and a client system for providing multimedia information to a user, wherein the multimedia information whose logical structure is described by text is recorded. , List as multidimensional information of three dimensions or more on a flat display, use a representation that imitates a book as a display method at that time, and represent by the shape of the book so that a physical book is opened and read A multimedia client-server system for simultaneously presenting, to a user, a part of the above-mentioned information among the whole information and information about a nail heading and a sticky note, wherein the client system is a server. A means for requesting the transfer of information to the system, and If the page displaying is not the page represented by the shape of the book, a means for generating a request for transfer of only attribute information of data having a large amount of information transfer, and a logic of the information obtained from the server system. The area for displaying the information on each page is calculated from the means for interpreting the physical structure from the text, the means for presenting information to the user according to the logical structure, and the description of the attribute information and the text description. And a means for calculating the size of the entire information from the calculated area, and the server system comprises means for retrieving only the attribute information and converting the result into a description of the logical structure of the text. Characteristic multimedia client server system. 2. When attribute information such as a display area of information of each page in the client system is obtained in the server system, means for calculating an area necessary for displaying an image in the server system. The multimedia client / server system according to claim 1, further comprising: and means for calculating an area required for displaying text.