JP4102928B2 - Data communication system, data transmission terminal and program - Google Patents

Description

  The present invention relates to a data communication system, a data transmission terminal, and a program suitable for listening between remote locations.

  A method using an electronic blackboard system is known as a method for attending lectures held at remote locations. The electronic blackboard system transmits characters and figures written on a blackboard (or whiteboard) in a lecture room as electronic data to a remote classroom or home. In a remote classroom that has received the electronic data, the board is reproduced on a display device such as a PDP (Plasma Display Panel) based on the transmitted electronic data. The student can listen to the lecture held at a remote place by listening to the audio of the lecture while watching the board written on the display device.

Here, in order to listen to the lecture in real time, it was necessary to synchronize and transmit the lecture voice and the electronic data of the board. As a technique for that purpose, for example, an electronic blackboard system described in Patent Documents 1 to 3 has been proposed.
Japanese Patent Laid-Open No. 2002-77858 JP 2001-292138 A JP 11-338339 A

  However, in the technique described in Patent Document 1, lecture image / sound information is transmitted to the information management center through the ISDN line, and the lecture listener exchanges information with the information management center. There was a problem that it was not possible to exchange questions. Further, in the technologies described in Patent Documents 2 and 3, when a remote conference is performed with a network connection, only the data on the blackboard written on the blackboard (whiteboard) is transmitted, so the atmosphere in which characters are written However, there was a problem that the enthusiasm of the speaker (instructor) was not transmitted.

  The present invention has been made in view of the above circumstances, and an object of the present invention is to provide a data communication system, a data transmission terminal, and a program capable of transmitting the presence of a transmission source.

In order to solve the above problems, the present invention provides a data communication system having a data transmission terminal and a data reception terminal, wherein the data transmission terminal collects voice and generates voice information, and a writing tool. A drawable board, position detecting means for detecting the position of a writing instrument moving on the surface of the board, and image drawing information generating means for generating image control information for drawing an image according to the position detected by the position detecting means; A sound effect control information generating means for generating sound effect control information for instructing control of the sound effect based on a change in position detected by the position detecting means, sound information, image control information, and sound effect control information. A data receiving terminal for receiving audio information, image control information, and sound effect control information; and an image control received by the receiving means. An image display means for displaying an image based on the information, a sound effect generating means for generating a sound effect based on the sound effect control information received by the receiving means, and a sound output based on the sound information received by the receiving means There is provided a data communication system comprising a voice output means.
According to this data communication system, when a pressure is applied on the panel of the data transmission terminal, an image corresponding to the trajectory to which the pressure is applied and a sound effect corresponding to the trajectory to which the pressure is applied are displayed remotely. Can be played back by the data receiving terminal.

The present invention also provides a voice information generating means for collecting voice and generating voice information, a board that can be drawn with a writing instrument, a position detecting means for detecting the position of the writing instrument moving on the board surface, and a position detecting means. Image drawing information generating means for generating image control information for drawing an image according to the detected position, and sound effect control for instructing sound effect control based on the change in position detected by the position detecting means There is provided a data transmission terminal having sound effect control information generating means for generating information and transmission means for transmitting sound information, image control information, and sound effect control information.
According to this data transmission terminal, when pressure is applied on the panel of the data transmission terminal, an image corresponding to the locus on which the pressure is applied and information on the sound effect corresponding to the locus on which the pressure is applied are transmitted. it can.

  Here, the data transmission terminal further includes pressure information generating means for generating pressure information indicating the pressure applied to the board by the writing instrument, and the sound effect generating means includes the position change and pressure information detected by the position detecting means. Sound effect control information may be generated based on the above.

  In the data transmission terminal, the voice information may be transmitted based on the VoIP protocol.

  Furthermore, the present invention is a computer having a board that can be drawn with a writing instrument, collecting voice and generating voice information, voice information generation processing, position detection processing to detect the position of the writing instrument moving on the board surface, Instructing the control of the sound effect based on the image drawing information generation process for generating image control information for drawing an image according to the position detected by the position detection process, and the change in the position detected by the position detection process Provided is a program for executing sound effect control information generation processing for generating sound effect control information and transmission processing for transmitting sound information, image control information, and sound effect control information.

  According to the present invention, when pressure is applied on the panel of the data transmission terminal, the image corresponding to the trajectory to which the pressure is applied and the sound effect corresponding to the trajectory to which the pressure is applied are displayed as remote data. Since it can be played back by the receiving terminal, it is possible to convey the realism of the transmission source to a remote person.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
[1 Configuration of the electronic blackboard system]
[1-1 Overall system configuration]
First, the overall configuration of the electronic blackboard system 1 in the present embodiment will be described. FIG. 1 is a block diagram showing a configuration of an electronic blackboard system 1 in the present embodiment. As shown in FIG. 1, the electronic blackboard system 1 according to the present embodiment includes an apparatus provided at University A 10, an apparatus provided at University B 20, and a SIP server 40. Each device is connected to each other via the Internet 50. Note that the communication network connecting each device is not limited to the Internet, and may be another communication network capable of IP (Internet Protocol) communication, such as a dedicated line or a wireless communication network.

In the present embodiment, University A 10 has a lecture room 10A that is a transmission source for lectures and a laboratory 10B that is a separate room from the lecture room 10A. An electronic blackboard 100 is installed in the lecture room 10A. There is a lecturer in the lecture room 10 </ b> A, and the lecturer lectures using the electronic blackboard 100. The electronic blackboard 100 is connected to the Internet 50 via the campus LAN (Local Area Network) 11 and the router 14, and the lecturer can distribute the lecture to other classrooms via the Internet 50.
In the lecture room 10A, a connection port (not shown) for connecting a personal computer (hereinafter abbreviated as PC) to the campus LAN 11 is installed. One connection port is installed in each seat of the lecture room 10A. There are a plurality of students in the lecture room 10A, and the students take a lecture by sitting on the seat where the connection port is installed. Students can connect their PCs 12 to the campus LAN 11 via connection ports. As will be described later, the student can use the PC 12 to ask the instructor a question by text or voice. In this embodiment, a student A in the lecture room 10A connects his / her PC 12 to the campus LAN 11.
Further, at the University A 10, an IP telephone 13 is connected to the campus LAN 11. The IP telephone 13 is installed in a laboratory 10B, which is a separate room from the lecture room 10A. There is a student B in the laboratory 10B, and as will be described later, the student B can know the state of the lecture being held in the lecture room 10A using the IP telephone set 13.

  University B 20 is a university different from University A. In the classroom of B University 20, an electronic blackboard 100B is installed. The electronic blackboard 100B has the same device configuration as the electronic blackboard 100. In order to distinguish between the electronic blackboard installed at University A and the electronic blackboard installed at University B, the electronic blackboard installed at University B and its components are distinguished by adding a B at the end of the reference number. It shall be. The electronic blackboard 100B is connected to the Internet 50 via the router 24 and can receive lecture data distributed from the University A 10. There are a plurality of students in the classroom of B University 20, and students can take a lecture by watching and listening to the blackboard displayed on the electronic blackboard 100B and the audio of the lecture output from the electronic blackboard 100B. In addition, the student can ask a question to the lecturer in the lecture room 10A of the university A 10 via the electronic blackboard 100B.

  In the present embodiment, University A 10 and University B 20 perform VoIP (Voice over Internet Protocol) communication based on the SIP protocol via the Internet 50. Here, SIP (Session Initiation Protocol) is an application layer signaling protocol for starting / changing / ending a multimedia session on an IP network. The SIP itself provides only basic functions such as “start / change / end of session”, and handling of multimedia data such as voice calls uses a conventionally used RTP (Real Time Protocol). SIP is a text-based protocol developed for the HyperText Transfer Protocol (HTTP), has a feature that it is simple and light in processing, and has excellent compatibility with the Internet. It is suitable as an application layer protocol.

  The SIP server 40 is a server for providing VoIP communication compliant with the SIP protocol. As shown in FIG. 2, the SIP server 40 includes a telephone number registration processing unit 41, a SIP protocol interpretation unit 42, a telephone number-IP address database 43, and a system supervision unit 44. In this embodiment, each terminal is registered in the SIP server 40 in advance. For example, registration is performed as follows. Each terminal transmits a registration request to the SIP server 40 together with its own IP address and telephone number. The SIP server 40 performs terminal registration processing in the telephone number registration processing unit 41. The telephone number registration processing unit 41 reads out the telephone number and IP address transmitted together with the registration request, and registers them in the telephone number-IP address database 43 in association with each other. In the telephone number-IP address database 43, a plurality of pairs of telephone numbers and IP addresses are registered in this way. The SIP protocol interpretation unit 42 has a function of interpreting a header of a packet conforming to the SIP protocol. The system control unit 44 has a function of controlling the above components.

  In the present embodiment, telephone numbers for establishing a communication line for VoIP communication are assigned in advance to the electronic blackboard 100 of University A 10 and the electronic blackboard 100B of University B 20. Although the meaning of each number will be described later, the electronic blackboard 100 is assigned three types of telephone numbers for lectures, questions, and monitors.

[1-2 Electronic blackboard configuration]
FIG. 3 is an external view of the electronic blackboard 100, and FIG. 4 is a block diagram showing the configuration of the electronic blackboard 100. The display control unit 102 is a device for driving the PDP 101, and transmits / receives data or signals relating to an image to be displayed to / from the central control unit 110. A touch panel 103 is installed on the surface of the PDP 101. When the user touches the touch panel 103 using, for example, a stylus pen, the touch panel control unit 104 of the touch panel 103 calculates the coordinates of the touched position and the touched pressure by a method described later. The central control unit 110 can draw the locus of the contact position on the PDP 101 by changing the color of the pixel of the PDP 101 immediately below the position, for example. The display device is not limited to the PDP, and other display devices such as an LCD, an LED display, or a cathode ray tube display may be used.

  In the present embodiment, the touch panel 103 is a known resistive film type touch panel. The touch panel 103 includes a touch panel control unit 104. The touch panel control unit 104 has a function of calculating the xy coordinates of the position touched by the user, calculating the touched pressure, and transmitting the calculated data to the central control unit 110. The touch panel 103 is not limited to the resistive touch panel, and may be a touch panel of another method such as an infrared method, an analog capacitive coupling method, or an ultrasonic surface acoustic wave method.

In the present embodiment, a plurality of strain gauges are densely arranged around the periphery of the touch panel 103. The touch panel control unit 104 calculates the displacement of the elastic film in the y-axis direction from strain gauges (two in total, one above and one below) located at a position closest to the calculated x coordinate. Similarly, the displacement of the elastic film in the x-axis direction is calculated from the strain gauges (two in total on the left and right ones) located at the position closest to the calculated y coordinate. The touch panel control unit 104 stores in advance a displacement-pressure table in which an elastic film displacement value and a pressure value causing the displacement are associated with each other. The touch panel control unit 104 calculates a pressure value associated with the displacement value closest to the calculated displacement from the displacement-pressure table and sets it as the touched pressure. The touch panel control unit 104 transmits the calculated pressure to the central control unit 110.
The method for calculating the pressure is not limited to the above method, and a method may be used in which a function indicating the relationship between displacement and pressure is obtained in advance, and the pressure value is obtained by inputting the displacement into the function. Alternatively, the touch panel 103 may not be provided with a means for measuring pressure, but may be provided with a pressure sensor in the stylus pen and transmitted to the central control unit 110 wirelessly or by wire.

The microphone 105 collects sound and converts it into a sound signal. The audio signal is transmitted to an audio CODEC (CODEr DEcoder) 107 and converted into audio data. The voice data is transmitted to the central control unit 110 via the voice data generation unit 108 and transmitted to the network via the VoIP protocol control unit 111 / network protocol control unit 112. The audio data generation unit 108 can generate audio data based on the data transmitted from the central control unit 110. The audio data generation unit 108 transmits the generated audio data to the audio CODEC 107. The audio CODEC 107 converts audio data into an audio signal and outputs it as audio via the speaker 106.
The numeric keypad 113 is a device for a user to mainly input a telephone number. The numeric keypad 113 outputs a signal corresponding to a user operation input to the central control unit 110.
The sound effect generating unit 109 generates sound effect data based on the data transmitted from the central control unit 110 and instructing the generation of the sound effect, and transmits the sound effect data to the sound data generating unit 108. The sound effect generator 109 stores a waveform database, a pressure-volume table, and a speed-pitch table, which will be described later, for generating sound effects. The sound data generation unit 108 superimposes the sound effect data and the sound data, and transmits the sound data to the sound CODEC 107 as one sound data. From the speaker 106, sound and sound effects are superimposed and output.

  FIG. 5 is a block diagram showing the configuration of the VoIP software module in the VoIP protocol control unit 111 / network protocol control unit 112. As shown in FIG. 5, the VoIP software module includes IP that is a dual layer stack of IPv4 and ICMP (Internet Control Message Protocol) v4 and IPv6 and ICMPv6, and TCP that is a transport layer protocol. (Transmission Control Protocol) and UDP (User Datagram Protocol), HTTPPD higher than application layer, DNS (Domain Name System), DHCP (Dynamic Host Configuration), DHCP (Dynamic Host Protocol), and DHCP (Dynamic Host Protocol). nsfer Protocol), and NTP (Network Time Protocol), and SIP stack and RTP / RTCP is VoIP module consists of user application.

  The central control unit 110 is a computer device having a CPU (Central Processing Unit), a memory, and an external storage device, and has a function of exchanging data with each of the above components and performing various controls and arithmetic processes.

[2 Operation of the electronic blackboard system]
Hereinafter, the operation of the electronic blackboard system 1 in the present embodiment will be described. First, referring to FIG. 6, an operation for establishing a communication line between the electronic blackboard 100B in the classroom of B University and the electronic blackboard 100 of A University who is going to take a lecture in the lecture room 10A of A University at a remote place. explain.
First, someone in the classroom of B University operates the numeric keypad 113B of the electronic blackboard 100B and inputs the lecture telephone number of the lecture room 10A. Alternatively, the electronic blackboard 100B stores a timetable table in which lecture start / end times are associated with lecture transmission sources, and when the lecture start time is reached, the telephone number of the lecture transmission source is automatically input. It is good also as a structure to be. When the telephone number is input, the central control unit 110B of the electronic blackboard 100B generates an INVITE request including the telephone number input from the numeric keypad 113B and transmits it to the SIP server 40.
The SIP server 40 that has received the INVITE request searches the telephone number-IP address database 43 and extracts an IP address associated with the telephone number transmitted together with the INVITE request. In the present embodiment, the IP address of the lecture room 10A associated with the lecture telephone number of the lecture room 10A is extracted. The SIP server 40 transfers the INVITE request to the node specified by the extracted IP address of the lecture room 10A.
Upon receiving the INVITE request, the electronic blackboard 100 in the lecture room 10A displays a pop-up window or the like indicating that there is a connection request from University B on the PDP 101, and a status Ringing indicating the state is displayed on the electronic blackboard 100B via the SIP server 40. Send. The connection request notification method is not limited to the pop-up window display, and other methods such as ringing a sound from the speaker 106 may be used.

  When the instructor of University A who notices the connection request from University B presses the OFFHOOK button on the electronic blackboard 100 and responds to the connection request, a status OK indicating that is transmitted to the electronic blackboard 100B together with the IP address of the electronic blackboard 100. Is done. The electronic blackboard 100B that has received the status OK transmits a status ACK to the electronic blackboard 100 via the SIP server 40. Through the above procedure, a communication line is established between the electronic blackboard 100 and the electronic blackboard 100B. From the header of the INVITE request, the electronic blackboard 100 recognizes that the request source is the blackboard 100B of University B, and switches to a mode for transmitting and receiving voice data and blackboard data (image data). As described above, the SIP server 40 is involved only in establishment of a communication line, is not involved in transmission / reception of voice data and board writing data, and uses the RTP protocol for transmission / reception of voice data and board writing data. That is, the voice data and the board writing data are transmitted and received between the electronic blackboard 100 and the electronic blackboard 100B without using the SIP server 40.

[2-1 Various operations of the electronic blackboard]
Next, the operation of the electronic blackboard 100 will be described. The electronic blackboard 100 according to the present embodiment has (1) a function of performing voice communication via a network, (2) a function of performing communication of a board writing via the network, and (3) generating handwriting sound in synchronization with the board writing. It has the function to make it.

(1) Audio Communication Next, audio data transmission / reception will be described. The voice spoken by the lecturer at University A is collected by the microphone 105 of the electronic blackboard 100 and transmitted to the voice CODEC 107 as a voice signal. The audio CODEC 107 encodes an audio signal using, for example, a PCM (Pulse Code Modulation) system of 8 kbps to generate audio data. Note that the encoding band is not limited to 8 kbps, and may be another band such as 64 kbps or 6.3 kbps. The audio data is transmitted to the central control unit 110 via the audio data generation unit 108. The central control unit 110 stores the voice data in the memory. For example, when the voice data for 20 milliseconds is stored, the central control unit 110 generates a packet including the voice data for 20 milliseconds (hereinafter referred to as voice packet), and the VoIP protocol control unit 111. / Send to the network via the network protocol control unit 112.

The voice packet is transmitted by the VoIP protocol control unit 111 / network protocol control unit 112 with the RTP header, UDP header, IP header, etc. added thereto. Since transmission / reception of voice packets requires real-time characteristics, UDP is used instead of TCP used for transmission / reception of Web access and electronic mail. Even when packets are sent using UDP, it is necessary to control the timing for reproducing the packets. For this purpose, time stamp information is recorded in the RTP header so as to inform the receiving terminal at which timing data should be reproduced.
The electronic blackboard 100B that has received the voice packet stores the voice data in the memory of the central control unit 110 in an appropriate order based on the time stamp information added to the RTP header of the voice packet. The central control unit 110 reads the audio data from the memory, and transmits the read audio data to the audio CODEC 107 via the audio data generation unit 108 at a timing based on the time stamp information. The audio CODEC 107 decodes the audio data into an audio signal and outputs it to the speaker 106B. The speaker 106B outputs the input audio signal as audio. In this way, students in the classroom of B University can listen to the audio of the lecture at A University.

(2) Postcard Communication Next, transmission / reception of an image drawn on the electronic blackboard 100 will be described. As described above, the touch panel 103 is installed on the surface of the PDP 101. When the lecturer touches the touch panel 103 using a stylus pen or the like, as described above, the touch panel control unit 104 outputs xy coordinate data, which is position information indicating the position touched by the lecturer, and the pressure of the touch to the central control unit 110. The central control unit 110 outputs drawing data instructing to change the color of the pixel corresponding to the xy coordinates to the display control unit 102 as image control information.
The display control unit 102 that has received the drawing data changes the color of the pixel corresponding to the xy coordinate on the PDP 101. For example, a pixel that was white in the non-input state is changed to black. As a result, a trajectory as if the instructor traced on the touch panel 103 is displayed on the PDP 101. That is, the instructor can write on the electronic blackboard 100 in the same manner as a conventional blackboard and chalk. It should be noted that means such as a switch for switching between a mode for writing characters / figures on the electronic blackboard and a mode for erasing written characters / figures (blackboard erase mode) are provided. In the blackboard erase mode, the image displayed on the PDP 101 is erased. May be.

The central control unit 110 transmits drawing data to the display control unit 102 and also transmits it to the electronic blackboard 100B. That is, the central control unit 110 generates a packet including the drawing data (hereinafter referred to as a drawing packet), and transmits the packet to the network via the VoIP protocol control unit 111 / network protocol control unit 112. The drawing packet is transmitted with the RTP header, the UDP header, the IP header, etc. added thereto in the VoIP protocol control unit 111 / network protocol control unit 112 in the same manner as the voice packet.
The electronic blackboard 100B that has received the drawing packet stores the audio data in the memory of the central control unit 110 in an appropriate order based on the time stamp information added to the RTP header of the drawing packet. The central control unit 110 reads the drawing data from the memory, and transmits the read audio data to the audio CODEC 107 via the audio data generation unit 108 at a timing based on the time stamp information. The audio CODEC 107 decodes the audio data into an audio signal and outputs it to the speaker 106B. The speaker 106B outputs the input audio signal as audio. The display control unit 102B that has received the drawing data changes the color of the pixel corresponding to the xy coordinate on the PDP 101B. That is, the board written on the electronic blackboard 100 installed in the lecture room 10A of University A is reproduced on the electronic blackboard 100B of the classroom of University B. B University students can listen to the lecture while viewing the blackboard on the electronic blackboard 100B.
In the above description, the aspect in which the xy coordinates and pressure data are converted into drawing data for instructing drawing of an image and then transmitted as image control information has been described. However, the positional information of the xy coordinates is transmitted as image control information as it is. Then, the electronic blackboard 100B that has received it may perform drawing based on the position information. Alternatively, bitmap data of an image displayed based on the drawing data may be transmitted.

(3) Generation of handwriting sound Next, the operation of generating handwriting sound will be described. The handwriting sound is broadly divided into attack sounds and friction sounds. Here, the attack sound refers to a sound of “ton” at the moment when the chalk touches the blackboard, in a conventional blackboard and chalk. In the present embodiment, this attack sound is generated at the moment when the stylus pen touches the touch panel 103. On the other hand, the frictional sound refers to the sound of “cue” when drawing a line on the blackboard with chalk. In the present embodiment, this frictional sound is generated when the stylus pen touching the touch panel 103 is moved while being touched.

  First, an operation for generating an attack sound will be described. The central control unit 110 of the electronic blackboard 100 monitors whether the touch panel control unit 104 of the touch panel 103 outputs xy coordinates. When the xy coordinates are not output for a predetermined time or longer (that is, when the touch panel 103 is not touched for a certain time), the central control unit 110 determines that the electronic blackboard 100 is in a non-writing state. On the other hand, when the xy coordinates are output, it is determined that the writing is in the board writing state. At the moment of transition from the non-paper writing state to the paper writing state, that is, the moment when the instructor touches the touch panel 103 with the stylus pen, the central control unit 110 transmits the pressure value data of the touch to the sound effect generating unit 109 and the attack sound. Instruct the generation of.

  The sound effect generator 109 instructed to generate an attack sound generates MIDI (Musical Instrument Digital Interface) data that is musical sound control information instructing the generation of the sound effect. Specifically, it is determined based on the length of the sound to be generated, the note-on / note-off message for designating the pitch, the program change message for designating the tone of the attack sound “ton”, and the pressure value of the touch. MIDI data including a control message for specifying a volume is generated. The generated MIDI data is transmitted to the audio data generation unit 108. The audio data generation unit 108 that has received the MIDI data extracts the attack sound waveform specified by the MIDI data from its own waveform database, and is processed so as to have the length and pitch specified by the MIDI data. The voice data of the attack sound is generated. The sound data of the generated attack sound is transmitted to the sound CODEC 107. The audio CODEC 107 converts audio data into an audio signal and outputs it to the speaker 106. The speaker 106 outputs an audio signal as audio. That is, an attack sound “ton” is output from the speaker 106 in synchronization with the operation of the instructor touching the touch panel 103. Note that the musical tone control data is not limited to MIDI data, and may be data according to other standards.

The sound effect generating unit 109 also transmits the generated attack sound MIDI data to the central control unit 110. The central control unit 110 packetizes MIDI data (hereinafter referred to as MIDI packets) in the same manner as the above-described audio data and drawing data, and transmits them to the network. The MIDI packet is transmitted by the VoIP protocol control unit 111 / network protocol control unit 112 with an RTP header, a UDP header, an IP header, etc. added thereto.
The electronic blackboard 100B of University B that has received the MIDI packet restores the MIDI data from the MIDI packet in the same manner as in the case of the voice packet and the drawing packet described above. The MIDI data is transmitted to the audio data generation unit 108B, and an attack sound “ton” is output from the speaker 106B in the same manner as described in the electronic blackboard 100 below.

Next, the operation for generating a frictional sound will be described. When it is determined that the electronic blackboard 100 is in the board writing state, the central control unit 110 calculates the moving speed of the pen tip of the stylus pen from the xy coordinates before and after the elapse of a predetermined time of, for example, 20 milliseconds. The central controller 110 further calculates an average nib pressure for 20 milliseconds. The central control unit 110 instructs the sound effect generating unit 109 to generate a frictional sound using the pen tip moving speed and pressure as parameters.
The sound effect generation unit 109 stores in advance a pressure-volume table in which the pressure of the pen tip and the volume of the friction sound are associated, and a speed-pitch table in which the speed of the pen tip and the pitch of the friction sound are associated. The sound effect generation unit 109 searches the pressure-volume table for the pressure value closest to the pressure value given as a parameter, and sets it as the volume of the frictional sound that generates the volume associated with the pressure value. The sound effect generating unit 109 extracts the pitch closest to the speed given as a parameter from the speed-pitch table and sets it as the pitch of the frictional sound. The sound effect generation unit 109 generates MIDI data including a note-on / note-off message that generates a sound of that pitch for 20 milliseconds, a program change message that indicates the tone of the friction sound, and a control change message that indicates the volume of the friction sound. . The generated MIDI data is transmitted to the audio data generation unit 108. The voice data generation unit 108 that has received the MIDI data extracts the waveform of the friction sound specified by the MIDI data from the waveform database held by itself, and is processed so as to have the length and pitch specified by the MIDI data. Generate sound data of attack sound. Hereinafter, the frictional sound is output from the speaker 106 as in the case of the attack sound.

  By the operation of generating the board writing and the handwriting sound described above, in synchronization with the operation of the instructor moving the stylus pen on the touch panel 103, graphics and characters (board writing) are displayed on the PDP 101 of the electronic blackboard 100. The handwriting sound "Ton, Kyu, Kyu" is played in sync with the board. On the other hand, at University B, the board written on University A's electronic blackboard 100 is reproduced on the PDP 101 of the electronic blackboard 100B, and the handwriting sound and the lecturer's lecture sound are reproduced from the speaker 106B in synchronization with the board. This makes it possible for B University to listen to lectures with a sense of reality.

(4) Additional operation of a blackboard from the student side blackboard The University B electronic blackboard 100B is a student side blackboard attending a lecture at University A, and has the same function as the electronic blackboard 100 from which the lecture was sent. Yes. Therefore, it is possible to ask a question by voice through the microphone 105B of the electronic blackboard 100B, or to add a board written using the touch panel 103B. That is, a lecturer at University A can appoint a student at University B to answer a question by voice or answer a mathematical problem written on the electronic blackboard 100 by writing an answer on the electronic blackboard 100B. . The method for transmitting and receiving voice and board is the same as that for the electronic blackboard 100.

  In the above description, the handwriting sound is transmitted to the electronic blackboard 100B of University B as MIDI data. However, the handwriting sound data (PCM data) generated by the electronic blackboard 100 is transmitted to the electronic blackboard 100B. Also good. In this case, the electronic blackboard 100B that has received the handwriting sound data reproduces the handwriting sound in the same manner as the speech data of the lecture. Alternatively, the electronic blackboard 100 may transmit the xy coordinates and the pressure value as data to the electronic blackboard 100B, and the electronic blackboard 100B may generate handwriting sound based on the data of the xy coordinates and the pressure value.

[2-2 Question operation]
As described above, the student A in the lecture room 10 </ b> A of University A connects the PC 12 owned by the student A to the campus LAN 11. Student A may raise his / her hand when he / she does not understand the lecture, but he / she will hesitate to ask a question because this will interrupt the lecture. However, in this embodiment, the student A asks a question via the campus LAN 11 from the PC 12 by the following method.

  Student A activates instant messaging software preinstalled on PC 12. The PC 12 is a computer capable of VoIP communication conforming to the SIP protocol. The student A inputs the question telephone number of the electronic blackboard 100 to the instant messaging software, and establishes a communication line between the electronic blackboard 100 and the PC 12. The operation for establishing a communication line is substantially the same as the operation for establishing a communication line between the electronic blackboard 100 and the electronic blackboard 100B. However, when a telephone number for a question is called, an operation for notifying the instructor that a connection request has been made is not performed, and the terminal is connected to the terminal that has made the question by an automatic response. After confirming that the communication line has been established, Student A enters a question into the instant messaging software. The PC 12 generates a packet (hereinafter referred to as a text packet) corresponding to the text data of the question and transmits it to the electronic blackboard 100.

When the central control unit 110 receives a text packet via a question communication line and obtains text data, the central control unit 110 causes the PDP 101 to display a message notifying that there is a question. When the lecture is over, the instructor notices the message and touches the part where the message is displayed. The message is displayed at a predetermined position on the PDP 101, and a signal indicating that is output to the touch panel control unit 104 while the message is displayed. When the xy coordinates belonging to the area corresponding to the message display position of the PDP 101 are output on the touch panel 103 during the message display, the central control unit 110 instructs the PDP 101 to display the characters indicated by the text data. Thus, on the PDP 101, the question entered by the student A from the PC 12 is displayed as characters. The text data of the question is transmitted from the electronic blackboard 100 to the electronic blackboard 100B of University B, and the question is also displayed on the electronic blackboard 100B.
The lecturer at University A reads out the displayed question and answers this question. In this way, students taking lectures at University A and University B can reliably share the contents of the questions and listen to the answers in real time.

  In addition, although the aspect which asks a question by text input was demonstrated above, the student A can also ask a question by voice via PC12. In this case, after the communication line is established, the student A asks a question by voice through the microphone provided in the PC 12. The question voice is reproduced from the speaker 106 in the same manner as the lecture voice. That is, both the students of University A and University B can listen to the question voice. Thus, for example, even when the lecture room 10A is a large classroom that accommodates several hundred students, and the student A is sitting in the back seat, the student A can ask questions.

[2-3 Lecture room monitoring by ringing tone]
The electronic blackboard system of the present invention provides a function of monitoring the state of a lecture room by using an IP telephone ringing tone (Ring Back Tone, hereinafter referred to as RBT). As described above, the IP telephone 13 is installed in the laboratory 10B, which is a room different from the lecture room 10A, and the IP telephone 13 is connected to the campus LAN 11. The lecture has already begun in the lecture room 10A, but it is assumed that student B is still in the laboratory 10B.

  FIG. 7 is a flow for explaining a procedure for transmitting lecture audio as RBT. Student B operates the push button of the IP telephone 13 and inputs the monitor telephone number of the electronic blackboard 100. The IP telephone 13 transmits an INVITE request including the monitor telephone number of the electronic blackboard 100 to the SIP server 40. The SIP server 40 extracts the IP address corresponding to the monitoring telephone number received from the telephone number-IP address database 43 held by itself. The SIP server 40 transfers the INVITE request to the node specified by the extracted IP address of the electronic blackboard 100.

  The electronic blackboard 100 that has received the INVITE request including the monitor telephone number transmits a status Ringing indicating that the call is in progress to the SIP server 40. The SIP server 40 transfers the received status Ringing to the IP telephone 13. The IP telephone set 13 that has received the normal status Ringing sounds an RBT “Pull Lulu ...”, but when the destination of the INVITE request is the electronic blackboard 100, the RBT “Plu Lu Lu ...” is not ringed. The electronic blackboard 100 transmits the status Ringing to the SIP server 40, specifies the source of the INVITE request from the INVITE request, and transmits the lecture voice packet to the IP telephone 13 that is the source of the INVITE request. The IP telephone set 13 that has received the lecture voice packet outputs the lecture voice reproduced from the lecture voice packet as an RBT from the speaker instead of the normal RBT.

  By listening to the lecture voice output as RBT from the IP telephone 13 as described above, the student B can know the state of the lecture while in the laboratory 10B. For example, even if the experiment in the laboratory 10B is not in time for the start of the extended lecture, it is possible to know that the lecture has not yet advanced to the important part, and Student B decides to attend the lecture even from the middle. Can do.

[3 Variation]
In the above-described embodiment, a mode has been described in which voice + board writing data is communicated between two devices, the electronic blackboard 100 and the electronic blackboard 100B. However, communication lines are connected between the electronic blackboard 100 and a plurality of devices. The lecture may be received by a plurality of apparatuses. For example, the board writing data may be received by the PC 12 connected to the campus LAN 11. According to such an aspect, since the board writing data can be stored in the PC 12, the student does not need to take notes. In addition, it is also possible to connect to the electronic blackboard 100 from the PC or the like at home or the like via the Internet 50 to receive the lecture voice + paper writing data at home or the like. According to such an aspect, the student can take a lecture without going to the university.

  Further, instead of the electronic blackboard 100, a so-called white board may be used, and the white board may be directly written with a marker pen or the like. In this case, a device for detecting the locus of a writing instrument such as a marker pen may be provided instead of the touch panel mechanism.

It is a figure which shows the structure of the electronic blackboard system in embodiment of this invention. It is a block diagram which shows the structure of the SIP server in the embodiment. It is an external view of the electronic blackboard in the embodiment. It is a block diagram which shows the structure of the electronic blackboard in the embodiment. It is a block diagram which shows the structure of the VoIP software module in the VoIP protocol control part / network protocol control part of the electronic blackboard in the embodiment. It is a flowchart which shows the procedure which establishes the electronic blackboard and the communication line for lectures in the embodiment. It is a flowchart which shows the procedure in which the electronic blackboard in the embodiment transmits the audio | voice of a lecture as RBT.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Electronic blackboard system, 10 ... University A, 11 ... Campus LAN, 12 ... PC, 13 ... IP telephone, 20 ... University B, 40 ... SIP server, 50 ... Internet, 100 ... Electronic blackboard, 101 ... PDP, 103 ... Touch panel, 105 ... Microphone, 106 ... Speaker, 108 ... Audio data generation unit, 109 ... Sound effect generation unit, 110 ... Central control unit, 113 ... Numeric keypad

Claims (6)

In a data communication system having a data transmission terminal and a data reception terminal,
The data transmission terminal is
Voice information generating means for collecting voice and generating voice information;
A board that can be drawn with writing instruments,
Wherein detecting the locus of the writing instrument on board surface, and detecting means currently detect whether blackboard state or non blackboard state,
And image drawing data generating means for generating image control information for drawing an image in accordance with prior trajectory danger detection means detects,
Before dangerous out timing that switched from a non blackboard state in blackboard condition is detected by means generates musical tone control data including a message specifying the tone of attack sound is continued in blackboard state by said detecting means Sound effect control information generating means for generating musical tone control data including a message for designating the tone color of the frictional sound,
Transmission means for transmitting the audio information, the image control information, and the sound effect control information,
The data receiving terminal is
Receiving means for receiving the audio information, the image control information, and the sound effect control information;
Image display means for displaying an image based on the image control information received by the receiving means;
Sound effect generating means for generating the attack sound and the friction sound based on the sound effect control information received by the receiving means;
And a voice output means for outputting voice based on the voice information received by the receiving means. Voice information generating means for collecting voice and generating voice information;
A board that can be drawn with writing instruments,
Wherein detecting the locus of the writing instrument on board surface, and detecting means currently detect whether blackboard state or non blackboard state,
And image drawing data generating means for generating image control information for drawing an image in accordance with prior trajectory danger detection means detects,
Before dangerous out timing that switched from a non blackboard state in blackboard condition is detected by means generates musical tone control data including a message specifying the tone of attack sound is continued in blackboard state by said detecting means Sound effect control information generating means for generating musical tone control data including a message for designating the tone color of the frictional sound,
A data transmission terminal comprising: transmission means for transmitting the audio information, the image control information, and the sound effect control information. Pressure information generating means for generating pressure information indicating pressure applied to the board by the writing instrument ;
Storage means for storing a table in which the pressure applied to the board and the volume of the frictional sound are associated with each other ;
The sound effect control information generating means is a sound volume associated with a pressure closest to the pressure indicated by the pressure information generated by the pressure information generating means among the sound volumes included in the table stored in the storage means. The data transmission terminal according to claim 2, wherein musical tone control data including a message designating the sound is generated. Speed information generating means for generating speed information indicating a speed at which the writing instrument moves on the board;
Storage means for storing a table in which the speed at which the writing instrument moves on the board and the pitch of the frictional sound are associated with each other;
Further comprising
The sound effect control information generating means is a pitch associated with a speed closest to the speed indicated by the speed information generated by the speed information generating means among the pitches included in the table stored in the storage means. Generate musical tone control data including a message specifying
The data transmission terminal according to claim 2. The data transmission terminal according to any one of claims 2 to 4, wherein the voice information is transmitted based on a VoIP protocol. To a computer having a board that can be drawn with a writing instrument,
A voice information generation process for collecting voice and generating voice information;
Detecting a locus of the writing instrument in the board surface, and detection processing currently you detect whether blackboard state or non blackboard state,
Image drawing information generation processing for generating image control information for drawing an image according to the position detected by the position detection processing;
Before dangerous out timing that switched from a non blackboard state in blackboard state is detected by the processing, it generates musical tone control data including a message specifying the tone of attack sound is continued in blackboard state by the detection processing If it is detected, sound effect control information generation processing for generating musical sound control data including a message for specifying the tone of the frictional sound,
A transmission program for transmitting the audio information, the image control information, and the sound effect control information.

Images