JP2007066315A - Shared white board history reproduction method, shared white board system, program and recording medium - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To reproduce operations performed on a white board and also to attain synchronous reproduction with a video image during a conference or in class. <P>SOLUTION: The respective operations performed on the white board are stored in history as a file with elapsed time since the history has been started to store on a client 2 or on the server 1 which access the white board. When time difference between elapsed time corresponding to the respective operations and elapsed time corresponding to the previous operations is longer than preset time t<SB>lim</SB>, the respective operation recorded in the file as the history are reproduced at a point of time when the time t<SB>lim</SB>elapses from time when the previous operations of the operations are reproduced and when the time difference is not longer than the preset time t<SB>lim</SB>, the respective operations are reproduced at a point when the time difference elapses from the time when the previous operations of the operations are reproduced. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a shared whiteboard system using a server and a client connected to each other via a network.

SMIL (Synchronized Multimedia Integration Language, Non-Patent Document 1) is a standard for describing a method for displaying media such as text, moving images, and audio while synchronizing them. SMIL uses a system (Non-Patent Document 2) developed by "Real Networks (registered trademark)", and a system developed by "Macromedia (registered trademark)" (Non-patent document 3). It is used as a presentation tool that supports sightseeing guides and company guides by combining image files and image files.
http://www.w3.org/TR/smil20/ http://www.jp.real.com/ http://www.macromedia.com/jp/software/flash/

  The above-described conventional technology is effective in fields such as routine work such as sightseeing guidance and company guidance, and education by auditing. On the other hand, it is not possible to synchronize and replay the video written in the meeting or class when the meeting is made via the network or in the remote class.

  In order to perform synchronized playback, it is necessary to rework the whiteboard drawing results used in actual meetings and classes into a format that can realize animation.

  An object of the present invention is to provide a whiteboard history reproducing method and a shared whiteboard system that reproduce operations performed on a whiteboard during a meeting or in a class and also realize synchronized reproduction with video.

  There are several methods for conferencing using video distribution technology via a network, such as using video conferencing software operating on a personal computer or using a commercially available TV conference system. A whiteboard that can be written / edited by a person connected via a network is realized on a computer and used for the purpose of proceeding with discussion of the conference in combination with the TV conference.

The present inventors have developed whiteboards (JP 2001-166756 A, JP 2001-228842 A, JP 2001-228843 A). In these whiteboards, figures and character strings (hereinafter abbreviated as objects) written there have the following information, for example.
・ Object identifier ・ Object type ・ Coordinate value ・ Color information ・ Identifier of the user who created the object ・ Date and time when the object was created ・ Identifier of the user who modified the object ・ Date and time when the object was modified XML Save using a markup language such as (Extensible Markup Language, http://www.w3.org/XML/). This is equivalent to saving as a history because the objects are recorded in the order they are written on the whiteboard. In addition, by using the time when the writing of the whiteboard is started, it is possible to save the relative time when the object is created or corrected as a history. Further, the use time of the whiteboard can be saved as a history by using the time when the writing is completed.

  A snapshot refers to a file that stores the state that was written on the whiteboard when the user saved the history. When playing back, first read the snapshot and display it on the client. Next, the time when the object was created or the time when the object was modified is read from the file saved as the history, and the difference from the recording start time is calculated. Alternatively, if the relative time is read, the calculation of the difference becomes unnecessary. When the elapsed time is reached using the client timer function, the snapshot is rewritten.

  In addition, the time when an operation on the whiteboard occurs from a file saved as a history is stored. If the snapshot is rewritten when the corresponding time comes while playing the movie, it can be synchronized with the movie.

As described above, according to the present invention, the work on the whiteboard is reproduced in a meeting or a remote lesson using a client or server on which a snapshot and a history of what is written on the whiteboard are saved as a file. Functions can be realized. Files shared on the whiteboard can be displayed by history playback based on the history files accumulated on the client or server by connecting to the network, or stored on the client terminal to the network. Even when not connected, it is possible to display by history reproduction. Also, synchronized playback with video can be realized by executing object operations using time information stored in the history file in accordance with time information when playing back video. By using the user information, it is possible to highlight an object written by a specific user when reproducing the history. Furthermore, it is possible to make the playback speed variable by using the time interval in which the object is written.

  Next, embodiments of the present invention will be described with reference to the drawings.

  FIG. 1 is a configuration diagram of a shared whiteboard system according to an embodiment of the present invention.

  The shared whiteboard system according to this embodiment includes a server 1, a client 2, and a network 3.

  The server 1 includes a data management unit 11, a client information management unit 12, a distribution management unit 13, and a communication management unit 14. The data management unit 11 manages data. The client information management unit 12 manages from which client the user is connected to the server 1. The distribution management unit 13 manages whether data is distributed from the server 1 to connected clients. The communication management unit 14 manages communication with the client 2.

  The client 2 includes a communication management unit 21, a data management unit 22, a data display control unit 23, and a data display unit 24. The communication management unit 21 manages communication with the server 1. The data management unit 22 performs processing for storing information to be written on the whiteboard as a history. The data display unit 24 displays the object data sent from the server 1. The data display control unit 23 controls display of data on the data display unit 24.

  The network 3 is a public communication network, the Internet, or the like that connects the server 1 and the client 2.

  FIG. 2 is a flowchart showing history storage processing in the data management unit 22 of the client. When it is desired to save the information to be written on the whiteboard as a history, first, the snapshot of the corresponding whiteboard is saved in a file (step 101). You can then create / edit / move / delete objects on the whiteboard, group multiple objects, ungroup objects that are grouped, change the top / bottom order of objects, An operation is performed such as enlarging or reducing a certain area, or sharing a file to clients sharing the same whiteboard by drag and drop (step 102).

  When a new object is created, the type of object, coordinate value string for line segments, center coordinate value and radius length for circles, information necessary for drawing the object, line thickness and color , Attribute information such as the degree of filling is stored. The coordinate value can be converted into the coordinate value of the movement destination by combining rotation / parallel movement / enlargement / reduction. Thereby, when the object is edited / moved, the corresponding object ID, coordinate value conversion information, and attribute information are stored. When an object is deleted, the corresponding object ID and deletion information are stored. When a certain area on the whiteboard is enlarged / reduced, the coordinate value of the center of the area to be enlarged / reduced and its magnification are stored. In the case of a file shared by drag and drop, the object ID, coordinate value, storage information, and attribute information are saved.

  When the writing of the whiteboard is finished and the history saving is finished (step 103), the operation history is saved in a file together with the operation time information (step 104), and the process is finished. Since the file shared by drag and drop is stored in the database in the server 1, information on the storage destination is saved as a history. For this reason, when the history is reproduced, the stored information is displayed as a clue for the dragged and dropped file, so the client 2 needs to be connected to the server 1. In the server 1, the history is stored in the data management unit 11. The only difference is that the file to be stored is in the server 1, and the processing procedure is not different from the processing procedure of the client 2.

  FIG. 3 is a flowchart showing a process for enabling the client 2 to reproduce the history without being connected to the server 1. Until the history is saved, the same processing as in FIG. 2 is performed (steps 201 to 203). All files shared by drag and drop are downloaded to the client 2 at the time of saving (step 204), and the entity is stored in a specific directory. Save the operation history to a file along with the operation time information. At that time, the file shared by drag and drop is rewritten from the storage location information of the server 1 to the information of the storage directory of the client 2 (step 205), and the process is terminated.

FIG. 4 is a diagram showing an example of the contents of a file stored as a history. Since the history is saved as XML, what is first declared as XML is written, and then what defines the structure of the document is described. A page refers to a plurality of spaces displayed on a shared whiteboard using a computer and a network. A whiteboard is generated from one or more pages. Each page is independent, and the page is shared by all users. The layer indicates a shared data space used when information to be shared throughout and information necessary for each individual or group are used together and displayed on the screen. Since a page includes a plurality of layers, a layer superposed layer is displayed on the page. Information about the history is described thereafter. The terms shown in FIG. 4 are not fixed. It is used here as an example only.
A description sandwiched between <WBRECORD> to </ WBRECORD> represents all information related to history storage. Declare the file name of the snapshot to be used first as information to be saved as history.
The description sandwiched between <operation> to </ operation> represents the operation content of the object on the whiteboard. The outline of the contents is shown below.
"elapsed_time": Elapsed time since the history was first saved
“whiteboard id”: Information about the whiteboard on which the object is operated
"page id": Information about the page on the whiteboard where the object was operated
"layer id": Information about the layer in the whiteboard page where the object is operated
“object id”: information about the actually operated object, and time = “hh ₁ : mm ₁ : ss ₁ ” indicates the operation time.
"properties": When specified by "layer id", the coordinate value of the center of the area to be enlarged / reduced and the information of the magnification are specified by "object id". Various information such as the state of the attribute information object in time, color and line thickness are stored.
Every time an object operation is performed, <operation> to </ oper
ation> has been added to the file.
When the history saving ends, </ WBRECORD> is written to the file and the history file is saved.

When the whiteboard has one page and has a layer, when only the layer is switched, <layer id> is set to a value different from the previous one, and <object id =. . . > To </ object> are stored in the history file. By using this information, the state written on the whiteboard using two files (snapshot file and history file) can be reproduced including layer switching. If the whiteboard has a plurality of pages and a plurality of layers, when only the layers are switched, <layer id> is set to a value different from the previous one, and <object id =. . . > To </ object> are stored in the history file. When the page is switched, <page id> is set to a value different from the previous one, and <object id =. . . > To </ object> are stored in the history file. By using these pieces of information, the state written on the whiteboard using two files is reproduced including the page and layer switching.

  FIG. 5 is a flowchart showing preprocessing for reproducing the history. When reproducing, a history file and a snapshot are first read (steps 301 and 302). The IDs and attribute information of the objects written in all pages and layers are read from the snapshot (step 303). Also, information on the total time taken for the history and information on the whiteboard, page, and layer are extracted from the history file and set as variables.

  FIG. 6 is a flowchart of a process for reproducing a history. Based on the time information written in <operation>, the relative time when the operation was performed on the object from the history storage start is obtained. Next, information on the actually operated object is set from <object id>. The attribute of this object is read from <properties>. If there is an object with the same object ID before <operation> is displayed, an object in which change information is added to what is currently displayed based on information specified between <operation> and </ operation> Draw. By repeating this, it is possible to reproduce the contents of the operation of the object on the whiteboard.

  It is also possible to set the double speed for playback. When n times is set, the object can be achieved if the object operation is executed when the playback time reaches the value obtained by dividing the elapsed time (elapsed_time) written in the history file by n. Similarly, it can be realized when 1 / n times is set. The user can also stop playback on the way. In that case, the playback time is set to 0, and what is displayed on the screen may be returned to the state in which the snapshot is read. In the case of a pause, the playback time is kept at the time when the pause is instructed. When the reproduction is executed, the process is continued by adding the execution start time to the held value. When the stop is executed, the playback time is set to 0 and the initial state is restored.

  Also, when displaying a video that is being used for a meeting or distance learning using a whiteboard, the information is read from the whiteboard history file and the object is operated according to the video playback time. By displaying the result on the screen, synchronized playback can be realized.

  FIG. 7 shows a history information reproduction image, and FIG. 8 shows a history information reproduction image when a specific person is selected in the participant list. The object is given information about the user who created the object and information about the user who modified the object. When playing back the history, it is possible to create a list of participants from the user information given to the object. When a specific participant is checked, a corresponding object can be selected from the user information, and the display method can be changed from an object created or modified by an unselected participant. As a display method, for example, as shown in FIG. 8, a method of displaying an object in an outline, a method of blinking, or a method of changing a color can be considered.

FIG. 9 shows a history of history storage. An example of an implementation method will be described. o _i indicates a written object. t _i indicates the time taken from the object o _i-1 to the writing of the object o _i . Let T _all be the time taken to save the history. If t _lim is a certain time and t _i is larger than t _lim , it is determined that o _i−1 and o _i are objects that are not drawn continuously. In FIG. 9, it is assumed that t ₂ and t _k-1 exceed t _lim . At this time, the range from o ₃ to o _k-1 is treated as a group, and is O _{[3, k-1]} . Time is represented by T _{[3, k-2]} . This is shown in FIG. At this time, T _[0,1] + T _{[2, k-2]} +. . . When + T _{[#, n-1]} is represented by S, S <T _all . replaced with all t _lim a range exceeding the t _lim, and that the T _{[1, m]} is replaced with _{TS [1, m] = T} all × T [1, m] / S is 11. By applying this, it is possible to reproduce the history at a variable speed. It is also possible to reproduce the history by combining all the above functions. The playback speed may be selected between the variable mode and the double speed mode when the history is played back.

  In addition to the functions of the client 2 realized by dedicated hardware, a program for realizing the functions is recorded on a computer-readable recording medium, and the program recorded on the recording medium is stored in the computer. It may be read by the system and executed. The computer-readable recording medium refers to a recording medium such as a flexible disk, a magneto-optical disk, and a CD-ROM, and a storage device such as a hard disk device built in a computer system. Furthermore, a computer-readable recording medium is a server that dynamically holds a program (transmission medium or transmission wave) for a short period of time as in the case of transmitting a program via the Internet, and a server in that case Some of them hold programs for a certain period of time, such as volatile memory inside computer systems.

It is a block diagram of the shared whiteboard system of one Embodiment of this invention. 7 is a flowchart of history storage processing in the data management unit 22 of the client 2. 4 is a flowchart of processing for enabling a history to be reproduced in a state where the client 2 is not connected to the server 1. It is the figure which showed the content of the history file. It is a flowchart of the pre-processing for reproducing | regenerating a log | history. It is a flowchart of the process for reproducing | regenerating a log | history. It is the figure which showed the history information reproduction image. It is the figure which showed the history information reproduction image at the time of selecting a specific person in a participant list. It is the figure which showed time passage of history preservation. It is the figure which showed what gathered time progress of log | history preservation | save. It is the figure which showed the result which made time passage of history preservation variable.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Server 2 Client 3 Network 11 Data management part 12 Client information management part 13 Distribution management part 14 Communication management part 21 Communication management part 22 Data management part 23 Data display control part 24 Data display part 101-104, 201-205, 301- 313 steps

Claims (6)

In a shared whiteboard system that uses servers and clients connected to each other via a network,
An operation saving step of saving each operation performed on the whiteboard on the client or server accessing the whiteboard as a file in the history together with an elapsed time from the start of saving the history;
For each operation recorded as a history in the file, if the time difference between the elapsed time corresponding to the operation and the elapsed time corresponding to the previous operation is longer than a preset time, Reproduced when the preset time has elapsed since the time when the previous operation was reproduced, otherwise reproduced when the time difference has elapsed since the time when the previous operation was reproduced A reproduction step to
A method for reproducing a shared whiteboard history, comprising:   When the reproduction step is set to reproduce at n times in advance, each operation recorded in the file corresponds to the elapsed time corresponding to the operation and the operation immediately before the operation. If the time difference from the elapsed time is longer than the preset time, it is reproduced when the preset time divided by n has elapsed since the time when the previous operation was reproduced. 2. The method of reproducing a shared whiteboard history according to claim 1, wherein the reproduction is performed at a time when a time obtained by dividing the time difference by n has elapsed from a time when the operation immediately before the operation is reproduced. In a shared whiteboard system that uses servers and clients connected to each other on the network,
Operation saving means for saving each operation performed on the whiteboard on the client or server accessing the whiteboard as a file in the history together with the elapsed time since the history was started,
For each operation recorded as a history in the file, if the time difference between the elapsed time corresponding to the operation and the elapsed time corresponding to the previous operation is longer than a preset time, Reproduced when the preset time has elapsed since the time when the previous operation was reproduced, otherwise reproduced when the time difference has elapsed since the time when the previous operation was reproduced Reproducible means to
A shared whiteboard system characterized by comprising:   When the reproduction unit is set to reproduce at a speed of n times in advance, each operation recorded in the file corresponds to the elapsed time corresponding to the operation and the operation immediately before the operation. If the time difference from the elapsed time is longer than the preset time, it is reproduced when the preset time divided by n has elapsed since the time when the previous operation was reproduced. 4. The shared whiteboard system according to claim 3, wherein the shared whiteboard system is reproduced when a time obtained by dividing the time difference by n has elapsed from a time when the operation immediately before the operation is reproduced. A program for causing a computer to execute the shared whiteboard history reproduction method according to claim 1. A computer-readable recording medium on which the program according to claim 5 is recorded.

Images