JP4555365B2 - Distance education system and multimedia content display control method - Google Patents

Description

  The present invention relates to an education system that can be used for both personal learning and real-time remote lessons, and in particular, can be used as a content creation tool for creating and distributing multimedia content including audio and video in addition to still images. The present invention relates to a remote education system and a multimedia content display control method.

  Conventionally, many of the teaching material creation tools for creating and distributing distance learning system teaching materials are based on recorded seminars and videos of attendance scenes, or a combination of videos and explanatory sheets as the basis of teaching materials. .

For example, Patent Document 1 proposes a multimedia display device that can display a still image or a corresponding position on a document in synchronization with the content of a reproduced moving image. This is because the page and cursor position to be displayed for each time zone of the video playback section and the display section are stored as a scenario, and the page switching and the cursor are moved with the passage of time during video playback. Is to go.
JP 2001-209361 A

  However, the technique described in Patent Document 1 records the motion of the lecturer during the lecture as a moving image, and adds the text and cursor information of the still image to this, so editing such as partial replacement is not easy, There is a problem that special equipment and the work of specialists are required to create teaching materials.

  In addition, if you try to use it for purposes other than personal learning, such as internet conferencing or distance learning, the display timing may vary depending on the line environment such as the viewer's communication speed, which may make it unclear what part of the material you are explaining. There is. For this reason, there is a problem that the use of the teaching material is limited and the degree of freedom of education is limited.

  The present invention has been made in view of the above-described circumstances, and can display a moving image, text, a cursor, and the like in synchronization, can be easily edited, and can be used for independent education or online education. An object of the present invention is to provide a remote education system and a multimedia content display control method.

  In order to achieve the above object, a distance learning system according to the present invention is a distance learning system that displays a cursor on a still image in synchronization with a moving image or sound of a learning material content. Or, the display order of still image data consisting of two or more pages, the video data or audio data to be displayed or played on each page, the elapsed time from the start of page display, the cursor position, and the cursor movement information Control table to be saved, page control means for switching pages and notifying page switching timing, timer means for measuring the time from the start of page display, and receiving the timing notification, referring to the control table, Output or display a video at a predetermined position and a page Reproducing means for displaying and moving the cursor in accordance with the elapsed time from the shown starting, characterized by comprising a.

  In addition to adding moving image or audio data to each page, it has an elapsed time from the start of page display, a cursor position, and cursor movement information to facilitate editing in page units.

  Preferably, a switching condition for each page is stored in the control table, and the page control unit determines whether or not the switching condition is satisfied, and switches to the next page when the switching condition is satisfied. It is good to make it. By providing a page switching condition, it is possible to correct a display timing shift when performing real-time distance learning for a plurality of persons.

  The distance learning system is constructed by a management server connected to a plurality of user terminals via a communication network, and when accessed from the user terminal, transmits a browsing program (viewer) that realizes each means to the user terminal. It is preferable that the learning material content selected by the user and the control table associated with the learning material content are transmitted and the browsing program is executed on the user terminal so that the learning material content is reproduced based on the control table.

  Although there is a method of delivering a stream to the user terminal, it is possible to avoid the influence of the speed and load of the network by downloading the browsing program and the teaching material content to the user terminal and playing the contents, and the content and viewer. Can be saved and viewed offline.

  In the distance learning system according to the present invention, the user terminal converts a general-purpose document file such as PowerPoint (PPT) (registered trademark) or a pdf file existing on the terminal into a still image for each page and loads it. Is converted so that it can be handled as a page for each still image, a still image is displayed for each page, video or audio data is saved in association with the page, and cursor position information and video on the still image of the page are stored. Or an editing processing means for generating a control table by associating and storing audio data recording or time from the recording start timing, and uploading the generated teaching material contents and the control table to a management server. .

In particular, the management server recognizes the access from the user terminal, generates an ad hoc virtual conference room composed of one or more accesses, and transmits the data such as annotations to other users' terminals. A means for registering or changing one user terminal having a certain right to speak, and a page control means executed on the terminal of the user having the right to speak, are connected to another user's terminal via the server at the time of page switching. The page to be displayed should be designated to notify the page switching timing.
If a plurality of users are synchronized to display the same screen, remote lectures can be performed in real time. However, as described above, when the teaching material contents are sent to the user terminal in advance for reproduction, the synchronization control is It becomes difficult. In the present invention, the page switching timing is transmitted to other user terminals with reference to the reproduction timing on the user terminal having the floor to synchronize each page.

  In the distance education system according to the present invention, the terminal of the user having the right to speak accepts writing to the displayed still image, records the locus of the pointer at the time of writing, and does not change for a certain period of time. Detecting and providing means for transmitting the trajectory information from the reference point of the pointer to the other user's terminal via the management server, and when the other user's terminal receives the trajectory information sent from the management server, The trajectory information is synthesized and displayed on the currently displayed still image based on the reference point of the still image.

  When annotations are written on a still image online, not all data including still image data is sent, but pointer trajectory information based on the reference point of the still image is transmitted to reduce the network load. Since synchronization is performed for each page, even if only such trajectory information is sent, deviation on other user terminals can be suppressed to a small level.

  Each user terminal transmits the end timing of the page to the management server when the display of the page is completed, and the management server transmits the end timing sent from the user terminal having the right to speak. The difference from the end timing sent from other users is calculated, and the correction value at each user terminal is calculated based on this difference to correct the start timing and measurement time. good. Thereby, the precision of synchronous control can be raised.

  A multimedia content display control method according to the present invention is a method for performing display control of multimedia content in which still images and moving images or sounds are mixed using a computer system, and includes one or more pages. Still image display order, switching conditions for each page, still image data to be displayed on the page for each page, video or audio data to be played back on the page, and elapsed time from the start of display of the page Edit processing for generating a control table that stores time and cursor movement information in association with each other, and switching conditions for each page, when used in plural via a communication network, have a right to speak or not to have a right to speak Different conditions can be set, and if you have the right to speak, at least the end of the previous page Switch to the next page and notify other users of the timing of page switching. If you do not have the right to speak, switch the page upon receiving the timing notification, and have the right to speak for personal use. Measured by the timer means while receiving the page switching process for switching the page with the same setting as the case and the timing notification, referring to the control table, displaying the audio data output or moving image at the predetermined position And a reproduction process for displaying and moving the cursor in accordance with the elapsed time from the start of the page display.

  In the present invention, the multimedia data is stored for each page, while the page switching condition is stored and the switching condition is set according to the application, so that the multimedia contents can be used in a plurality of usage forms such as personal use and distance learning. And make it usable.

  As described above, according to the present invention, moving images and audio information are recorded in units of pages, and the cursor position is moved with the lapse of time from the start of page display. By setting the start condition of each page, it is possible to suppress a display timing shift between viewers.

  Embodiments of the present invention will be described below. FIG. 1 is a schematic explanatory diagram of a usage pattern of the distance learning system according to the first embodiment.

  Here, the distance learning system is constructed as a management server connected to a communication network such as the Internet. The management server accesses the management server from a terminal, creates teaching materials, and uploads and saves them to the management server. This educational content downloads a viewer, which is a viewing tool, to the student's terminal.

  Next, the contents of the teaching material stored in the server are downloaded and displayed on the student's computer screen through this viewer.

  This procedure will be described with reference to FIG. First, a server is accessed from a user terminal with a browser. As a result, the editing program is downloaded and activated. The user creates, for example, a slide in the ping format from the document file used by the user, registers voice and cursor movement, and creates teaching materials. Upload this to the server in compressed form.

  At the time of personal learning, access is made with a browser, the program of the playback means is automatically downloaded and started, and then the teaching material is downloaded. This teaching material is sent in a compressed format, decompressed and played on the terminal side. Log out when viewing ends.

  FIG. 3 is an explanatory diagram of the data structure of teaching material contents, and FIG. 4 is a setting example for each data type.

  The data structure of the slide Si in the server stores an index (page number and data position on the memory) and data, information such as a data type and a data size in association with each course ID. The data type is a type of data for each display target such as a slide (still image), audio, cursor, moving image, lecturer video. The data file for each data type is compressed and transmitted in units of content for speeding up when uploaded to the server.

Hereinafter, the editing processing procedure in the case of individual learning will be described with reference to FIG.
The video attribute information includes display condition data such as the number of pixels of the original data, the X and Y coordinates of the reference point, the width, and the height. As the reference point, for example, the upper left corner (X = 0, Y = 0) of the slide (still image) is set as the entire reference point.
When one slide has N data types, N data units are generated. FIG. 4 shows an example of the data structure when audio, cursor, and video are combined with the slide 2.

Slide NO. Are set in numerical order, and the playback algorithm attached to each data type is executed during playback. Playback is slide no. The slides are skipped and slide no. The designation is prioritized.
In continuous mode, slides are automatically played back continuously.

  Regarding the control of the cursor and embedded video, when the embedded video or audio recording start button is pressed, the time counter of the slide is started, and if the cursor is clicked during recording, the (x, y) coordinates are read and A unit of (t, x, y) is generated at time t, and a list of additional data of the slide as shown in FIG. 5 is generated. In the case of video control, X (X coordinate data position) in the cursor display is a control signal. Then, negative values are assigned such as start “−1”, stop “−2”, pause “−3”, restart “−4”, jump “−5”, and the like. The time counter is started by pressing the video start button instead of the audio start button. The time counter ends at the later of the video end or audio recording stop button.

  In this way, as shown in FIG. 5, the data structure of the cursor display and the video display can be made common as illustrated as data of the slide i (i-th slide).

Next, an editing processing algorithm will be described with reference to FIG.
First, by adding a start button displayed on the editing screen or a start button of the inserted video, the voice addition is started (S101). Next, it is determined whether or not voice addition is being executed (S102), and if it is being executed, the voice is recorded (S103). When the presence / absence of mouse operation is detected (“YES” in S104), cursor information is generated (S105). If there is a video operation (“YES” in S106), video control information is generated and entered in the list (S107), and the process returns to step S102 again. On the other hand, when execution of voice addition or insertion video (whichever is later) is completed in step S102 ("NO" in S102), voice data and cursor data are saved (S108, S109).

Next, the processing procedure of the reproducing means will be described with reference to FIG.
First, slide (still image) data is read (S201), and a slide is displayed (S202). If there is video data (“YES” in S203), it is played back by the video playback function (S204). If there is audio data (“YES” in S205), it is reproduced by the audio retransmission function (S206). Also, it is determined whether there is cursor data (S207), and if there is cursor data, it is next determined whether voice playback is in progress (S208). If the sound is being played back, the playback elapsed time t is acquired (S209), and the data is searched in the time range 0.1 seconds after that time, that is, (t, t + 0.1) (S210). As a result, if data exists in this time range (“YES” in S211), if the value of the x-coordinate is “0” or more (“YES” in S212), it is assumed that the cursor data is (x, y) points. A cursor is displayed on the screen (S214). On the other hand, if the value of the x coordinate is negative in step S212, the video data is played back as video data. If playback is in progress, playback continues (S213). Then, after the end of steps S213 and S214, or if “NO” in step S211, the process sleeps for 0.1 second and returns to step S208 (S215).
The reproduction process is executed by the above procedure.

  According to the present embodiment, moving images and audio information are recorded in units of slide pages, and the cursor position is moved with the passage of time from the start of page display, so that it is easy to change each page.

Next, a second embodiment of the present invention will be described.
FIG. 8 is a functional block diagram of the distance learning system according to the present embodiment. The distance learning system 1 includes a management server 10 and a user terminal 50 that downloads and executes an editing program or a browsing program from the management server.

  Here, the management server 10 includes a transmission unit 11 that transmits and receives information to and from the communication network 2, a storage unit 13 that stores contents and tables, and a calculation unit 12 that executes calculation processing.

  The storage unit 13 includes an editing program 31 that is transmitted by access from the user terminal 50, a browsing program 32, a speech right management table 33 for managing a sender of information (so-called right holder) between a plurality of user terminals, A teaching material content 35 and a control table 34 associated therewith are stored. Although not shown, the management server 10 includes classroom management data, user management data, and the like.

  In addition, when the computing unit 12 is accessed from a transmission / reception processing unit 21 that exchanges data with the transmission unit 11, an information registration unit 22 that registers teaching material content, a control table, and the like, and a user terminal 50, an editing program Or it has the information transmission means 23 which transmits a browsing program, and the relay means 24 which relays the timing information etc. which are mentioned later. Each means 21-24 of the calculating part 12 is realizable with the program which CPU execution is possible.

  The function of the user terminal 50 varies depending on the type of program to be downloaded. In the user terminal 50 for editors, the transmission / reception processing means 71 for transferring data to and from the transmission unit 51, the editing program, etc. Information receiving means 72 for receiving the above information, editing processing means 73 for creating teaching material contents and control tables, and information transmitting means 74 for uploading the created teaching material contents 35 and the control table 34 to the management server. The functions of the editing processing unit 73 and the information transmitting unit 74 are incorporated by the downloaded editing program 31.

  On the other hand, the calculation unit 55 of the user terminal 50 of the user who uses the teaching material content (referred to as a viewer for convenience) receives information such as the transmission / reception processing means 71 that exchanges data with the transmission unit 51, the program, the teaching material content, and the like. The information receiving means 72 to be acquired, the reproducing means 81 for reproducing the learning material content, the page control means 80 for switching the learning material content page, the speaking right requesting means 75 for requesting the right to speak to the management server 10, and the management server 10 are accessed. The right-of-speech determining means 76 for determining the presence or absence of the right to speak, the start timing transmitting means 77 for transmitting the start timing of the page, the start timing receiving means 78 for receiving the start timing of the page, and the figure on the still image of each page Annotation writing means 79 that accepts writing of characters and characters and transmits the written information to other user terminals It is.

  In addition, the storage unit 54 includes a teaching material content 35 downloaded from the management server 10 and a control table 34 associated therewith.

  In this embodiment, while viewing a teaching material program by a plurality of people, an annotation or voice is transmitted in real time from a terminal of a user who has the right to speak and displayed or output on the terminal screen of another user to lecture And proceeding with meetings.

  The outline of the operation of the distance learning system according to this embodiment will be described below with reference to FIG. Since the contents of the editing process are the same as those in the first embodiment, the description thereof will be omitted, and the description will be focused on the browsing / comment transmission process during the distance learning.

<1. Outline of the transfer procedure for speaking rights during distance learning>
(1) When a client requests a meeting (class) participation, user management information is generated in the server and a connection is established. When the user i has the lecturer authority (speaking right), the authority is requested through the instructor terminal 50. When the server receives this request, the server sets a status indicating the lecturer authority for the user i (in FIG. Right “1”). For other users, a status indicating the questioner is set (speaking right “0” in FIG. 11).
(2) The questioner requests the question authority (speaking right).
(3) An inquiry is made to the instructor, and (4) when the instructor accepts, (5) the management server transfers the speaking authority to the user 1 of the questioner. Thereafter, (6) the questioner switches slides and asks a question with the cursor. The cursor information is distributed to the instructor and other users' terminals via the management server 10 and displayed superimposed on the slides on the terminals. (7) The instructor responds by voice. (8) Thereafter, the questioner is changed as necessary, and the same processing is repeated.
(9) When the conference ends, each user disconnects the connection.
As user information, a user ID given by a random number and the IP address of the user terminal are used. Since the user ID is given ad hoc (temporary or provisional) with a random number, this conference system has a feature that pre-registration is not required.

<2. Page switching process>
First, in addition to the data structure of the control table 34 based on each slide unit shown in FIGS. 3 and 5, the data structure in the present embodiment has a switching condition for each slide page as shown in FIG. It is that.

  In this figure, the page switching conditions differ depending on whether or not there is a floor, and if there is a floor, slide 1 (first page) is started by pressing the start button on the viewing screen. Subsequent slides start with the end of the previous page. On the other hand, when there is no right to speak, the page is switched by receiving the start timing sent via the management server 10.

The above-described page switching process will be specifically described below.
The instructor terminal or the user terminal requesting the speech requests the right to speak to the management server 10 from the speech right requesting means 75 of the user terminal 50.

  When the management server 10 receives this request by the information registration means 22, the management server 10 notifies the current speaker's terminal 50 and receives the permission of the current speaker, thereby renewing the right to speak in the speech right management table 33 shown in FIG. 11. Move to speaker. This right to speak is initially given to the terminal although it is set as a lecturer.

Then, the floor permission notification is transmitted to the terminal 50 of the person who has requested the floor.
Upon receiving the floor-right permission notification, the floor-right determining means 76 of the terminal 50 of the person who has requested the floor-right sets the switching condition to the “with floor-right” mode, while setting the start timing transmitting section 77 in the active state. When each page is switched, the next start page and page start timing signal are transmitted to the management server 10. When the management server 10 receives the start timing signal or the like, the information relay means 24 transfers the start timing signal to the terminal 50 of another user (a user who does not have the right to speak).

  On the other hand, when the other user's terminal does not receive the right to speak from the management server 10 by the right determining unit 76, the terminal is set to the “no right to speak” mode and is managed by the start timing receiving unit 78. When the next page number and start timing transferred from the server 10 are received, they are passed to the page control means 80. The page control unit 80 displays the slide page, and executes a moving image or cursor movement process by the reproducing unit 81 as time elapses from the page display.

  As described above, the feature of the present embodiment is that a user accessing the same content configures a virtual conference room / classroom and automatically controls pages. In addition, it is possible to automatically determine whether the content is accompanied only by a slide or audio / cursor video by the reproducing means 81, and to perform reproduction, a meeting or a lecture corresponding to the content.

<3. Data structure>
The distance learning system according to the present embodiment operates like a virtual classroom.
This is done by the classroom management table of virtual classroom management. FIG. 12 shows an example of the data structure of the classroom management table that the management server 10 has.

  The virtual classroom identifies the classroom with a course ID (content name). The presenter's user ID is registered for each course ID. This presenter is the organizer of the classroom. Note that the initial value of this user ID is set to “0” (state without teacher). The classroom status includes active “0”, pause “1”, and closed “2”.

  In addition, the number of participants in the classroom, the user ID list of the classroom, the annotation board currently displayed (pointer to the annotation file), the image shared in the classroom (current virtual annotation board), for example, A bitmap of 800 × 600 pixels (each pixel has a value of rng), a currently displayed slide NO (pointer to a slide file), and the like are stored in association with each other. Although details of the virtual annotation board will be described later, when an annotation is entered or drawn on a still image (slide) on each page, the information is registered as position information from the reference point of the still image. It is a memory area. When multiple participants can be presenters (who have the right to speak), an annotation board is virtually assigned to each participant (virtual annotation board), and when the participant is a presenter, the participant is assigned to that participant. The number of the selected virtual annotation board is set as the currently displayed annotation board NO.

  One virtual classroom is created for content. Further, the virtual annotation board has a bitmap of 800 × 600 pixels (pixel value is rng), and is configured only by an array of pixels having data values for efficient memory size and processing time.

Participants are performed using the user management table illustrated in FIG.
The user management table is generated for each participant in the virtual classroom and is given a random user ID, the login time of the user, the status (active “0”, pause “1”, closed “2”, the IP address of the user The connection socket between the user and the server is stored.

Next, the data structure of the communication signal will be described.
In the distance education system, two types of message communication are performed. Message communication between the presenter and the server, and between the server and each listener (participant), the drawing element message and the command message. These are identified by the Type variable in FIG.

  As shown in FIG. 14, the message from the user includes a unique name of the message, a unique ID of the user, a message type: drawing element (Shape) “0”, a command “1”, and data: storage “0” of the drawing element object. ”, Command object storage“ 1 ”, etc., and when the message is transmitted, the message transmission time is added and registered.

Next, the structure of the drawing control data illustrated in FIG. 15 will be described.
For each drawing element: pixel “0”: free curve “1”: straight line “2”: circle “3”: rectangle “4”: filled circle “5”: filled rectangle “6”: eraser “7”, etc. The type is identified. Of these, the free curve and eraser are the main functions. Also, the color of the drawing element, the thickness of the line, the number of pixels of the drawing element, the array of pixels of the drawing element (x0, y0), (x1, y1)… (xi, yi)… (xn, yn), etc. Is saved. This pixel array is drawn as a connection of points.

  An important feature of this system is that a free curve can be used to freely draw (write) sentences and expressions with a curve of arbitrary thickness. Moreover, for each set, it is sent to the server in real time, written to the virtual annotation board, sent to each participant (listener), written to each annotation board, and downloaded in advance. It is displayed so as to be overwritten on the slide.

Control such as slide switching is performed by command data illustrated in FIG.
This command type is: update participate number (update the number of participants) “0”, slide switching “1”, presenter switching “2”, presenter application (application to become a presenter) “3”, presenter application Response “4”, play / stop voice / cursor attached to slide (additional voice / cursor can be played) “5”. As command data, type 0 corresponds to the above command type, type 0 is the number of participants, type 1 is the number of the slide to be changed, type 2 is the presenter UserID, type 3 is the presenter applicant's UserID, type Data such as 4 (deny “0”, OK “1”), type 5 (playback execution “0”, stop “1”), etc. are set.

<4. Virtual classroom management>
FIG. 17 is a flowchart showing a virtual classroom management procedure. Here, FIG. 17A is a processing procedure on the management server 10 side, and the user is specified and managed by the connection of the user terminal and the user ID given at the time of class participation. FIG. 17B is a processing procedure on the client, that is, the user terminal 50 side, and the user enters the classroom (S403) by connection permission from the management server (S402). This is linked with the entrance examination management process in step S305 on the management server side. Thereafter, the user terminal passes a message to other user terminals through the management server 10 (S404, S405).

  Note that a virtual classroom is created when a certain content is accessed, is established if there is one or more participants, can participate / leave in the middle, and disappears when the number of participants becomes zero. Also, anyone can be a lecturer (presenter), and the participant who first presses the “Click for Presenter” button is given the lecturer authority (speaking right) from the system. After that, when there is an application, it will be switched with the consent of the current lecturer authority.

FIG. 18 is a flowchart showing a procedure of presenter authority assignment processing.
When the management server 10 receives the presenter application request transmitted from the speaking right request means 75 of the user terminal 50 (S501), the management server 10 obtains permission from the current presenter according to the procedure described in this flowchart (S503, S505). As a result, a response value 0 or 1 is set and returned to the applicant (S509). The user terminal 50 of the applicant receives the response value by the speaking right determination unit 76 and sets the page switching timing.
The authority of the presenter (instructor) is as follows.
1) Annotation board can be written 2) Annotation board (drawing image) can be saved in the server database, and the saved drawing image can be retrieved. The process is performed on the entire slide in a lump. When the image is saved, the original saved image is erased and stored. When the image is taken out, the current drawing image is replaced.
3) When there is a declaration to become a presenter, it can be accepted or rejected.

<5. Online annotation board image transmission / expression / entry method>
The distance learning system is composed of two parts: a content image and a drawing image. A content image is an image of learning content (images or slides) created by an editing program downloaded from a server and managed in the server's content database, and generally consists of a plurality of slides. This is the background image.

A drawn image is an image drawn online in a presentation and is shared by participants. Therefore, as shown in FIG.
Final display image = content image (background) + drawing image.

  A normal online whiteboard does not have a content image, but the system has a function of overwriting a drawing image as an overwriting (annotation) with the learning content as a background. The drawn image has a one-to-one correspondence with the background image (slide) in units of slides, and is saved as an independent file. Therefore, the base slide is not affected at all. At the time of display, a background image is first written and a drawing image is written thereon. From the user's perspective, the annotation appears to be "added" on the background. To erase part of the drawing image, use a square “eraser”, but only the drawing disappears and the background remains the original. Therefore, it has the same function as writing a note on a printed material with a pencil and erasing a part of it with an eraser.

  The drawn image is represented by an array of pixels called a so-called bitmap, but since it has the same pixel array (800x600) as the background content, each pixel is completely overlapped. When a drawing image is superimposed on the background, the background data value (pixel) is replaced with the drawing pixel.

  Note that the content image and the rendered image are represented by the exact same image size of 800 x 600 pixels, and the display position is exactly the same. It appears that the technology has overwritten the image on the background.

Next, the processing algorithm of the online annotation board will be described.
20A shows processing on the management server side, and FIG. 20B shows processing on the client (user terminal) side.
(1) Initialization of content image When the presenter designates a slide number, the number is sent to the server, distributed to all members of the virtual classroom, and the image of the designated slide of the content downloaded in advance is displayed in the user window. Is done. This is used as background. At this time, if there is a drawing image corresponding to the slide, it is also switched at the same time, so that it is seen from the user that the slide in which the annotation is written has been switched. Actually, the display function performs processing to superimpose two images.
(2) Online synchronization of drawing between server and client One drawing buffer memory (virtual annotation board) is provided for each virtual classroom. The server that has received the drawing information from the client first fills in the buffer and distributes it to all members (listeners) of the classroom.
When each member first enters the classroom, the buffer contents are compressed in zip format and sent to the member's client computer.
On the client side, the received data is decompressed and stored as a drawing image. Thereafter, the received drawing data is written thereon.
(3) Final image on the client side When the image is updated on the client side, it is superimposed on the content image and displayed on the display screen.

<6. Virtual classroom management concept>
In the distance education system, it can be properly used as two operation modes. They are a presentation / discussion mode and a synchronous autoplay mode.

(1) Presentation / Discussion mode In each classroom, only one person can be the presenter at the same time. The presenter can switch slides and change modes. All operations of the presenter are reflected to all members in synchronization. All members can apply for presenter authority to the current presenter (see FIG. 18).

  Also, in this state, the presenter can add drawing elements on the content image, which are written on the virtual annotation board on the server, and further synchronized online and distributed to each member.

  The presenter's authority is applied by the listener, and if the current presenter grants permission, it is transferred to the applicant. A member who receives presenter authority becomes a new presenter and takes over all authority. (See Figure 21)

(2) Synchronous automatic playback mode When audio / cursor / video is added to the content, the presenter can start or stop synchronous automatic playback on the viewing screens of all members. This can be used especially for synchronized playback in distance learning.

(Drawing method)
The drawing elements drawn by the presenter are sent to the server, written to the virtual annotation board, sent to each listener, written to each annotation board, and then combined with the slide image for display. . (See FIG. 22).

  The virtual annotation board is a simple set of pixels (bitmap-like) and cannot be recognized as a character or a figure. This is unsuitable for structural handling, but is rather suitable when a person overwrites a printed material with a pencil or the like, erases it with an eraser, and further overwrites it. In particular, this technique is suitable for free description of formulas, free graphics, and various texts.

  As described above, according to the present embodiment, when a real-time remote lesson is performed between a plurality of users, data such as still images (slides), moving images, cursor movement information, and the like are stored in association with each other. Since page switching is performed by transmitting and receiving the switching timing, it is possible to prevent accumulation of misalignment between terminals when reproducing teaching material content downloaded in advance. For this reason, it is possible to display the annotation board more accurately when the annotation board is superimposed on the slide. In the case of distance learning, since the teaching material contents are downloaded and used in advance, the influence of the user's network environment can be reduced. When the teaching material content is used for personal learning, it can be used for both personal learning and real-time distance learning by setting the page switching condition of the control table to the “with speaking right” mode.

  Furthermore, if the canvas is white (plain), it becomes an online whiteboard, if it is a slide, it becomes an overwrite (annotation) board, and if it is a slide with audio / cursor attached, it can be controlled whether to play it. It can be used as a multi-function presentation board.

  The above embodiment is described as a remote education system that can be used for individual learning by downloading multimedia content including voice and moving images, still images, and cursor movement information on still images linked to the sounds and moving images. However, it is obvious that the present invention is not limited to an education system, and can be used as a communication tool including, for example, a conference system.

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a schematic explanatory diagram of a usage pattern of a distance learning system according to a first embodiment of the present invention. It is explanatory drawing of the procedure between the server terminals at the time of editing of teaching material content by the 1st Embodiment of this invention, and reproduction | regeneration. It is explanatory drawing of the data structure of the teaching material content by the 1st Embodiment of this invention. 4 is a setting example for each data type in FIG. 3. FIG. 4 is a data example of the cursor video control of FIG. 3. FIG. It is a flowchart which shows the procedure of the edit process by the 1st Embodiment of this invention. It is a flowchart which shows the procedure of the reproduction | regeneration processing by the 1st Embodiment of this invention. It is a functional block diagram of the distance education system by the 2nd Embodiment of this invention. It is an operation | movement outline | summary of the distance education system by the 2nd Embodiment of this invention. FIG. 9 is a data example of page switching conditions that constitute a part of the control table of FIG. 8. FIG. It is a data structure figure of the floor management table of FIG. It is a data structure figure of the classroom management table which the management server of FIG. 8 has. It is a data structure figure of the user management table which the management server of FIG. 8 has. It is a data structure figure of the communication signal by the 2nd Embodiment of this invention. FIG. 15 shows an example of the data structure when the message type is a drawing element in FIG. It is an example of a data structure in case a message type is a command in FIG. It is a flowchart which shows the management procedure of the virtual classroom by the 2nd Embodiment of this invention. It is a flowchart which shows the procedure of the assignment process of the presenter authority by the 2nd Embodiment of this invention. It is explanatory drawing of the superimposition process of the content image (background slide) and drawing image (annotation; annotation image) by the reproduction | regeneration means of FIG. It is a flowchart which shows the process sequence of the online annotation board by the 2nd Embodiment of this invention. It is a flowchart which shows the outline | summary of the process by the side of the user terminal by the 2nd Embodiment of this invention. It is explanatory drawing of control of the thickness of the line in the drawing process by the 2nd Embodiment of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Distance education system 2 Communication network 10 Management server 11,51 Transmission part 12,55 Operation part 13,54 Storage part 21,71 Transmission / reception processing means 22 Information registration means 23 Information transmission means 24 Information relay means 31 Editing program 32 Viewing program 33 Speaking right management table 34 Control table 35 Teaching material content 50 User terminal 52 Input unit 53 Display unit 72 Information receiving unit 73 Editing processing unit 74 Information transmitting unit 75 Speaking right requesting unit 76 Speaking right determining unit 77 Start timing transmitting unit 78 Start timing receiving Means 79 Annotation writing means 80 Page control means 81 Reproduction means

Claims (4)

A remote education system comprising a plurality of user terminals and a management server connected to each user terminal via a communication network, and displaying a cursor on a slide in synchronization with a video or audio,
The management server
Teaching material content including slides and video or audio consisting of multiple pages,
Set for each page, in the mode with speaking right, the page is switched by pressing the start button or the end of the previous page, and in the mode without speaking right, the switching condition for switching the page by receiving the start timing for each page, and the page Data consisting of a combination of the elapsed time from the start of display and coordinates representing the cursor position with reference to the corner of the slide, and the same data structure as the data, and the coordinates of the coordinates differ from the coordinates A control table including data to which a control signal for video or audio reproduction processing is assigned according to a code value;
A storage unit for storing
When there is access from the user terminal, information transmitting means for transmitting the teaching material content and the control table to the user terminal;
An information registration means for receiving a request for a right to speak sent from the user terminal and transmitting a right to speak notification to the user terminal;
Information relay means for transferring the next start page and start timing sent from the user terminal to other user terminals,
The user terminal is
A storage unit for storing teaching material contents and a control table sent from the management server;
A floor request means for requesting the floor to the management server;
By receiving the floor grant notice sent from the management server, and voice judging means for setting a Switching Operation exchange example condition of the control table to say there mode,
Based on the switching condition of the control table, when the terminal is in the right-to-speak mode, the page is switched by pressing the start button or the previous page ends, while transmitting the next start page and start timing to the management server, When the own terminal is in the right-to-speak mode, page control means for switching pages by receiving the next start page and start timing sent from the management server;
Timer means for measuring the time from the start of page display;
By receiving the start timing, the control table data is referred to every certain time, and the cursor is displayed and moved according to the elapsed time from the start of page display based on the data, and the data is coordinated. If there is a value of a code different from the code of, reproduction means for performing moving image display or audio reproduction processing based on the value,
Distance education system characterized by having The user terminal displays a slide for each page, stores video data or audio data in association with the page, and records or records the video data or audio data with coordinates indicating the cursor position on the slide of the page. Editing processing means for generating the control table by the data structure in association with the time from the start timing;
The distance learning system according to claim 1, wherein the generated teaching material content and the control table are uploaded to the management server. The user end the end of the right to speak there mode, it accepts the writing of the slide that is displayed, and recorded the trajectory of the pointer at the time of writing, by the management server via the trajectory information from the reference point of the pointer, remarks to send to the end of the other of the user end of the no mode rights,
Another of the user end the end of the no right to speak mode, the management upon receipt of the locus information is come sent from the server, the reference point of the slide synthesized and to slide in the current display the locus information to the original display The distance education system according to claim 1 or 2, wherein A method for performing display control of multimedia contents including slides and moving images or sounds using a computer system comprising a plurality of user terminals and a management server connected to each user terminal via a communication network. ,
The management server
Multi-page slides and multimedia content including video or audio,
Set for each page, in the mode with speaking right, the page is switched by pressing the start button or the end of the previous page, and in the mode without speaking right, the switching condition for switching the page by receiving the start timing for each page, and the page Data consisting of a combination of the elapsed time from the start of display and coordinates representing the cursor position with reference to the corner of the slide, and the same data structure as the data, and the coordinates of the coordinates differ from the coordinates A multimedia data stored in the storage unit when accessed from a user terminal. Sending the content and the control table to the user terminal;
The user terminal receives the multimedia content and the control table transmitted from the management server, stores them in a storage unit of the user terminal,
User terminal that has been set in the floor ON mode by the incoming floor grant notice sent from the management server, based on the Switching Operation exchange example condition of the control table, switch pages by pressing or previous page end of the start button Meanwhile, the next start page and start timing are sent to the management server,
The management server transfers the next start page and start timing sent from the user terminal to another user terminal,
The other user terminal performs the page switching process by receiving the next start page and start timing sent from the management server, while being measured by the timer means based on the data of the control table. When the cursor is displayed and moved according to the elapsed time from the start of the page display, and the data has a sign value different from the sign of the coordinates, a moving image display or audio reproduction process is performed based on the value. A display control method for multimedia contents, characterized in that: