JP2003224836A - Video conference system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a video conference system which acquires a higher added value by transmitting/receiving other pieces of information in parallel with transmission/reception of video and audio data. <P>SOLUTION: A lecturer station 10 and a terminal station 40 use audio processing sections 11, 41 and video processing sections 12, 42 to transmit/receive audio and video data. In parallel with the processing above, an information processing section 13 of the lecturer station 10 transmits command data. Upon the receipt of the command data, the information processing section 43 of the terminal station 40 acquires corresponding event data from an event database 45 and executes the data. The event data are video data, audio data and HTML data or the like, the system reproduces and displays various contents in addition to a video phone function to realize a system with a high added value and provide a user interface rich in expression. <P>COPYRIGHT: (C)2003,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a video conference system for transmitting and receiving video and audio using the Internet or the like.

[0002]

BACKGROUND OF THE INVENTION There is a so-called video conference system that uses a network such as the Internet, a leased line, and a telephone line to perform video and audio communication at multiple points. There are various usage forms of the video conference system, for example,
It is also used in a remote multipoint education system.

The terminal equipment used by the user is a TV
Telephone device or PC with video / audio input / output devices such as a camera, microphone, speaker, TV with STB (set top box), or PDA
(Personal digital assistant) etc. exist.

[0004]

In a multipoint conferencing system, bidirectional exchange of video and audio is generally the main subject.

Other data in addition to video and audio (other data means data other than video and audio)
There is a video conferencing system for exchanging information, but these other data control data for setting the participating members of the conference, data for authenticating the participating members, and control of video, audio, or bandwidth. Data for video conferencing systems or multipoint education systems, and no attempt has been made to add value to the conferencing system or education system by effectively using data that is not related to video and audio communication. .

In view of the above problems, it is an object of the present invention to construct a system with higher added value by transmitting and receiving other information in addition to video and audio in a video conference system.

[0007]

In order to solve the above-mentioned problems, in the present invention, a conference system consisting of video and audio is used together with data communication, and a terminal side has a database (or rule base). Data is freely issued between terminals in a command (instruction / command) format, and data that is not directly related to video and audio communication is mutually handled.

That is, the invention according to claim 1 is a video conference system for transmitting and receiving video and audio between a first terminal station and a second terminal station connected via a network, wherein the first terminal The station has a-1) command storage means for storing command data corresponding to a predetermined event, and a-
2) means for transmitting command data corresponding to an event to be executed in the second terminal station in parallel with transmission of video data and audio data, wherein the second terminal station is b-1) the command A terminal database storing event data for executing an event corresponding to each command data stored in the storage means, and b-2) in parallel with the reception of video data and audio data, from the first terminal station. In the second terminal station, means for receiving the transmitted command data, and b-3) means for acquiring event data corresponding to the received command data from the terminal database and executing the corresponding event, Is characterized in that an event corresponding to the received command data is executed in parallel with the reproduction of video and audio.

According to a second aspect of the present invention, in the video conference system according to the first aspect, the event data is
In the second terminal station, an image corresponding to the received command data is displayed on the monitor in parallel with the reproduction of the video and the audio.

According to a third aspect of the present invention, in the video conference system according to the first or second aspect, the event data includes voice data, and in the second terminal station, the first terminal station. The audio data as event data corresponding to the received command data is reproduced in parallel with the reproduction of the received video and audio.

The invention according to claim 4 is the video conference system according to any one of claims 1 to 3, further comprising: c) a data server connected to the second terminal station via a network. The data server includes c-1) a center database that stores event data for executing an event corresponding to each command data stored in the command storage means;
The terminal station further comprises b-4) means for acquiring event data corresponding to the command data received from the first terminal station from the center database via the network and executing the corresponding event. Characterize.

According to a fifth aspect of the present invention, in the video conference system according to the fourth aspect, the data server is also connected to the first terminal station via a network, and the event data stored in the center database is , Web contents, and by transmitting command data designating the Web contents currently accessed by the first terminal station itself to the second terminal station, the first terminal station and the second terminal station are connected. Is performing a video conference while accessing the same Web content.

According to a sixth aspect of the present invention, in the video conference system according to any of the first to fifth aspects, a plurality of event data corresponding to one command data is stored in the terminal database. The second terminal station further includes b-5) event determining means for determining which event data to read in accordance with a predetermined rule for the received command data.

According to a seventh aspect of the present invention, there is provided a video conference system for transmitting and receiving video and audio between a first terminal station, a base station, and a plurality of second terminal stations connected via a network. The first terminal station executes a-1) command storage means for storing command data corresponding to a predetermined event, and a-2) transmission of video data and audio data in parallel with the second terminal station. Means for transmitting command data corresponding to an event to the base station, the base station b-1) command data received from the first terminal station, according to a predetermined rule, any of the second terminal Means for relaying and transmitting command data to the determined second terminal station, and each second terminal station is stored in the command storing means c-1). Each command day A terminal database storing event data for executing an event corresponding to the data, and c-2) means for receiving command data transmitted from the base station in parallel with reception of video data and audio data. , C-3) means for acquiring event data corresponding to the received command data from the terminal database and executing the corresponding event, and in the second terminal station receiving the command data, It is characterized in that an event corresponding to the received command data is executed in parallel with the reproduction.

According to an eighth aspect of the present invention, in the video conference system according to any one of the first to seventh aspects, when the communication data amount of the video conference system exceeds a predetermined band, the first In addition, the bandwidth is limited by limiting the transfer amount of the command data.

According to a ninth aspect of the present invention, in the video conference system according to any one of the first to eighth aspects, when the communication data amount of the video conference system exceeds a predetermined band, the first In the command data, second
Bandwidth limitation is performed by limiting the transfer amount in the order of audio data, and thirdly video data.

[0017]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of a video conference system according to the present invention will be described below with reference to the drawings by taking a multipoint remote education system as an example.

{1. System Configuration} FIG. 1 is an overall system diagram of a multipoint remote education system according to an embodiment of the present invention.

The remote education system will be described by using the lecturer station 10 used by the lecturer T and the terminal stations 40A, 40B, 40C used by the students A, B, and C (hereinafter collectively referred to as the terminal station 40, as appropriate). ) And a base station 2 connected to the lecturer station 10 and each terminal station 40 via a network (transmission path)
0, and a Web station 30 connected to the network.

However, the instructor station 10 and the terminal station 40 can be distinguished from the viewpoint of a distance education system, but the functions and roles of the terminals are the same. Therefore, when widely used as a video conference system, the lecturer station 10 also functions as one of the terminal stations.

The lecturer station 10 and the terminal station 40 are composed of a personal computer (PC), a video telephone, a PDA (personal digital assistant), a combination of a television and an STB (set top box), and the like. , Audio processing units 11 and 41, video processing units 12 and 42, and information processing units 13 and 43.

The voice processing units 11 and 41 are provided with a microphone and a speaker, have a function of transmitting voice input from the microphone to a network, receive voice through the network, and receive the received voice through a speaker. Including the function to output from. In addition, the voice processing unit 11,
Reference numeral 14 includes a function of reproducing audio data stored in a hard disk or the like and transmitting the reproduced audio to the network.

The image processing units 12 and 42 include a camera and a monitor, a function of transmitting an image input from the camera to the network, and an image reception via the network, and a received image from the monitor. Including the function to output. In addition, the video processing unit includes a function of reproducing the video data stored in the hard disk or the like and transmitting the reproduced video to the network.

The information processing section 13 accepts a command instruction command from the instructor T, acquires corresponding command data from the command database 15, and transmits the acquired command data to the terminal station 40. The command data is a designated command for specifying event data for executing an event described later. The designated command is text data indicating a file name, a program name, an index, or the like, or text data indicating a URL. Therefore, in any case, the command data has a very small capacity.

The information processing section 43 receives the command data transmitted from the lecturer station 10, acquires event data 71 corresponding to the received command data from the event database 45, and acquires the acquired event data 7
A functional unit that executes 1 is provided.

The above-mentioned processing units are realized by using peripheral equipment, hardware and software of the lecturer station 10 or the terminal station 40.

As shown in FIG. 2, the event database 45 included in the terminal station 40 stores event data 71 and a rule definition file 72.

A plurality of event data 71 are prepared corresponding to each command data registered in the command database 15 of the lecturer station 10. Therefore, the terminal station 40
It is possible to specify and execute the event data 71 by receiving the command data.

Here, as shown in FIG. 3, in this specification, the event data 71 is still image data,
It includes moving image data, audio data, HTML data, and other programs. That is, the event data 71 includes all content data that can be displayed and reproduced on the terminal station 40, and all application data that can be effectively executed on the terminal station 40. Therefore, when the event data designated by the command data is still image data, the information processing unit 43
Displays a still image on the monitor. If the event data is a moving image, the moving image is displayed on the monitor. If the event data designated by the command data is HTML data, the information processing section 43
Display HTML content on the monitor. If it is another program, the program is executed on the CPU of the terminal station 40 and various events are executed.

Further, depending on the type of command data,
Multiple event data 7 for one command data
1 is prepared. In this case, when the command data is received, the event data 71 according to the rule definition file 72 stored in the event database 45.
Is selected.

An example of a method of selecting the event data 71 using the rule definition file 72 will be shown. One method is to add a parameter to the command data transmitted from the lecturer station 10. Then, the rule definition file 72 stores information that specifies the event data 71 from the command data and the parameters. With this, it is possible to make one event data correspond to a plurality of event data 71.

As another method, the rule definition file 7
In 2, the predetermined judgment standard and the determination rule of the event data 71 corresponding to the judgment are recorded. Then, the information processing unit 43 performs a predetermined determination process and compares the determination result with a predetermined determination standard. By this, the corresponding event data 71 is determined.

As described above, in the distance learning system according to the embodiment, the instructor station 10 transmits the command data to the terminal station 40, so that the terminal station 40 transmits the voice and the image as the videophone function. In addition to playback, it is possible to execute various contents and applications.

Further, as described above, since the lecturer station 10 and the terminal station 40 have an equal relationship, the pattern opposite to that described above, that is, the terminal station 40 used by the students is used.
By sending command data from the instructor station 10 to the instructor station 10, various contents and applications can be executed in the instructor station 10 in addition to the reproduction of audio and video as a videophone function. However,
For the sake of simplicity, etc., etc., a functional unit for realizing one-way processing (processing for transmitting command data from the lecturer station to the terminal station) is shown.

The base station 20 has, as a first function, a connection control unit 21 for smoothly performing video and audio communication between the lecturer station 10 and the terminal station 40. The connection control unit 21 has a function of relaying video and audio, and controls the access right of students to each course. The base station 20 also includes an information processing unit 22 and an event database 25.

The event database 25 is stored in the terminal station 40.
The event database 71 is substantially the same as the event database 45 included in, and holds event data 71 and a rule definition file 72. However, it is significant that the event data 71 that the terminal station 40 does not possess is possessed. Further, in the rule definition file 72, rules to be judged by the base station 20 are recorded.

In order for the terminal station 40 to execute the event data 71 held by the base station 20, an event data acquisition instruction is sent from the information processing section 43, and the information processing section 22 specifies in response to this instruction. Event data 71
Is acquired from the event database 25 and transmitted to the terminal station 40.

The Web station 30 includes a Web server application 31 and Web contents 35.
As described above, the event data 71 also includes HTML data, but the command data transmitted from the instructor station 10 to the terminal station 40 specifies the network address (the network address 35 that the web station 30 holds). For example, if the URL is included, the terminal station 40
The information processing unit 43 can execute an event by accessing the Web server 31.

As described above, each terminal station 40 has the event data 71 held by its own terminal and the base station 4
0 holds event data 71 and Web station 30
It is possible to execute an event by using the Web contents 31 owned by.

{2. Usage Example} Next, a usage example of the above-described distance education system will be described with reference to FIGS. 4 and 5.

<First Example> FIG. 4 shows a first example. Distance learning using the above system is performed between the instructor T and the students A, B, and C. Here, only the student A is shown.

When the instructor T raises a question by using voice and gestures, the video and voice are transmitted to the terminal station 40A of the student A.
Played in. When the student A utters an answer to the question, the video and audio are the terminal station 1 of the instructor T.
Played at 0.

When the instructor T utters "Mr. A's correct answer" to this answer, this image and sound are reproduced on the student A side. In addition to saying "Mr. A's correct answer", or , Instead of saying "correct answer", teacher T
When the user issues a command instruction indicating "Correct answer by Mr. A", the information processing section 13 transmits the corresponding command data to the terminal station 40A. In the terminal station 40A, when the information processing section 43 receives this command data, the event database 45
The corresponding event data 71 is acquired and executed.
Here, since the event data 71 is image data in which the character of “correct answer” is decorated, the character of “correct answer” pops up on the monitor of the terminal station 40A.
Also, if the event data is a video showing "correct answer",
Furthermore, another effect can be obtained. Also, music such as fanfare may be played at the same time.

<Second Example> The second example is substantially the same as the first example, but the storage location of the event data 71 is different.
In the example shown in FIG. 4, the terminal station 40A itself holds the image that is the event data 71 indicating the character “correct answer”. However, when the terminal station 40A does not hold the event data 71, May be acquired from the base station 20.

As shown in FIG. 1, the base station 20 includes an information processing unit 22 and an event database 25, as well as the functions and database of the terminal station 40. Therefore, the terminal station 40 can acquire the event data 71 not present in the local terminal from the base station 20 and reproduce and execute the event data 71. Specifically, the information processing unit 43
Requests the base station 20 to acquire the event data 71. In response to the acquisition request, the information processing unit 22 of the base station 20 acquires the corresponding event data 71 from the event database 25 and transmits it to the terminal station 40.

As another example of the second example, as described above, the information processing section 43 of the terminal station 40 is
By accessing the web server 31, it is possible to acquire the web content 35 and execute the event. In this case, web collaboration is possible in which the instructor and the student proceed with the lesson while referring to the same web content.

The method of using the event data stored in the center in this way is effective, for example, when the terminal station 40 is a mobile terminal or the like and has a small storage capacity.

<Third Example> In the third example, the base station 20 distributes command data according to a predetermined rule.

Of the data flow shown in FIG. 1, # 1 indicating the flow of command data is first sent from the instructor station 10
It is transmitted to the information processing unit 22 of the base station 20. Then, the information processing unit 22 determines the transfer destination of the command data according to the rule definition file 72 included in the event database 25, and determines the determined terminal station (terminal station 4 in the figure).
0B), the command data is transmitted.

For example, the instructor T issues a question, and the student A,
Suppose B and C raise their hands. Looking at this video, the instructor T wants to give the answer right to the student B who raised his hand first.
Then, the instructor T gives an instruction to the "student B to give a reply right". This command data is transmitted from the information processing unit 13 to the information processing unit 22. The information processing unit 22 determines to transmit the command data 71 received according to the rule definition file 72 to the terminal station 40B. That is, in the rule definition file 71, a rule is defined in which the command data for giving the answer right to a certain student is transmitted only to the student.

In this way, when the command data 71 is transmitted to the terminal station 40B, the terminal station 40 executes various events. For example, an image in which a lamp is turned on is displayed along with a sound effect for which a right to answer is obtained.

<Fourth Example> The fourth example is a modification of the third example. FIG. 5 shows the flow of processing of the fourth example.

First, the instructor T issues a problem. The video and audio are transmitted to the terminal stations A and B. Students A and B who hear this question promptly obtain an answer right (specifically, they select a fast-press button displayed on the monitor with the mouse or press a dedicated button). ). Then, the information processing unit 43 transmits the request data for the instruction to acquire the response right to the base station 20.

Information processing unit 20 that has received the quick-press command
Determines a student to whom the answer right is given according to the rule definition file 72. If it is defined in the rule definition file 72 that "the answer right is given to the student who receives the quickest push request data earliest", the information processing unit 22 gives the answer right to the terminal station 40B. To do. That is, the data that gives the answer right is transmitted to the terminal station 40B as command data. This allows the terminal station 40B to execute various events indicating that the reply right has been obtained. In the figure, students A and B
Among the two lamps corresponding to, the lamp of the student B is turned on, indicating that the student B has been given the answer right.

{3. Connection control} As shown in FIG. 1, the base station 20 has a management database 24. The management database 24 stores curriculum data, course data, lecturer data, student data, student attendance data, etc. of the distance learning system as a database. Thereby, the connection control unit 21 can control the access right of each student to a specific course. It is also possible to perform authentication processing for each student.

Also, by limiting the number of people who can access one course, it is possible to limit the bandwidth occupied by one course.

{4. Bandwidth control} In the distance learning system of the present embodiment, command data is transmitted and received in addition to video and audio transmissions and receptions as described above. Here, except when there is a sufficient margin in the band of the transmission path to be used, in many cases, the band is limited. In particular, in a video conference system that transmits and receives video and audio, it is a problem because the bandwidth used is wide.

Therefore, the connection control unit 21 monitors the transmitted / received data amount of the lecturer station 10 and the terminal station 40, and limits the transmitted / received data when the transfer amount exceeds a predetermined band. Specifically, the priority order is defined as 1) image data 2) audio data 3) command data, and the restriction is applied from the data with the lowest priority order. That is, when the amount of transmitted / received data exceeds the predetermined band, first, the transfer amount of command data is limited. As a result, the event cannot be executed, but the transmission and reception of the video and audio can be continued. By doing so, it becomes possible to secure the minimum necessary exchange of information while adjusting the total amount of transmitted / received data.

[0059]

As described above, according to the first aspect of the invention, since the command data is transmitted / received and the event is executed in parallel with the transmission of the video and audio, a system having a higher added value is provided. It is possible. Also, by reproducing various contents at the second terminal station, it is possible to provide an expressive user interface. Since the event data itself is not transmitted / received but only the command data which is the instruction command thereof is transmitted / received, the data transfer amount can be reduced.

According to the second aspect of the invention, since the image is displayed on the second terminal station as the event data, it is possible to enjoy various video effects while making a videophone call.

According to the third aspect of the invention, since the sound is reproduced at the second terminal station as the event data, it is possible to enjoy various acoustic effects while making a videophone call.

In the invention according to claim 4, the second terminal station is
Since the event data stored in the center server is executed, the system configuration is flexible and highly expandable.

According to the invention of claim 5, since the first terminal station and the second terminal station simultaneously access the Web content of the center, it is possible to advance the same Web content as a topic in the video conference. is there.

According to the sixth aspect of the present invention, since the event is executed according to the defined rule, the system configuration is highly expandable.

In the invention according to claim 7, the base station determines the terminal station to which the command data should be relayed according to the defined rule, and transmits the command data only to the corresponding terminal station. It becomes possible to perform a filtering process that causes the specified terminal station to execute the event.

In the invention described in claim 8 or claim 9, since the bandwidth is limited by limiting the command data, the minimum necessary information exchange is ensured while adjusting the total amount of transmitted / received data. It becomes possible.

[Brief description of drawings]

FIG. 1 is an overall system diagram of a multipoint remote education system according to the present embodiment.

FIG. 2 is a diagram showing the contents of an event database.

FIG. 3 is a diagram showing the contents of event data.

FIG. 4 is a diagram showing an operation example of command transfer and event execution.

FIG. 5 is a diagram showing an operation example of event execution using a rule base.

[Explanation of symbols]

10 Lecturer Bureau 11,41 Voice processing unit 12,42 Video processing unit 13,43 Information processing unit 15 Command database 25,45 event database 20 base stations 30 Web stations 40 (40A, 40B, 40C) terminal station

Claims (9)

[Claims] 1. A video conference system for transmitting and receiving video and audio between a first terminal station and a second terminal station connected via a network, wherein the first terminal station has a-1) a predetermined value. Command storing means for storing command data corresponding to the event, and a-2) means for transmitting command data corresponding to the event to be executed in the second terminal station in parallel with transmission of video data and audio data. And b-2) a terminal database storing event data for executing an event corresponding to each command data stored in the command storage means, and b-2) a video image. Means for receiving command data transmitted from the first terminal station in parallel with the reception of data and voice data, and b-3) event data corresponding to the received command data. And a means for executing a corresponding event, the second terminal station executes an event corresponding to the received command data in parallel with the reproduction of the video and audio. A featured video conference system. 2. The video conference system according to claim 1, wherein the event data includes image data, and in the second terminal station, in the command data received in parallel with reproduction of video and audio. A video conferencing system in which corresponding images are displayed on a monitor. 3. The video conference system according to claim 1, wherein the event data includes audio data, and in the second terminal station, video and audio received from the first terminal station. A video conference system characterized in that audio data as event data corresponding to the received command data is reproduced in parallel with the reproduction of. 4. The video conference system according to claim 1, further comprising: c) a data server connected to the second terminal station via a network, wherein the data server is And c-1) a center database that stores event data for executing an event corresponding to each command data stored in the command storage means, and the second terminal station further includes b-4). A video conference system comprising: means for acquiring event data corresponding to command data received from the first terminal station from the center database via the network and executing a corresponding event. 5. The video conference system according to claim 4, wherein the data server is also connected to the first terminal station via a network, and the event data included in the center database includes Web content. By transmitting command data designating the Web content which the first terminal station itself is currently accessing to the second terminal station, the first terminal station and the second terminal station form the same Web content. A video conferencing system characterized by advancing a video conference while accessing. 6. The video conference system according to claim 1, wherein the terminal database stores a plurality of event data corresponding to one command data, and the second terminal. The station further comprises: b-5) event determining means for deciding which event data should be read according to a predetermined rule for the received command data. 7. A video conference system for transmitting and receiving video and audio between a first terminal station, a base station and a plurality of second terminal stations connected via a network, wherein the first terminal station comprises: a-1) Command storage means for storing command data corresponding to a predetermined event, and a-2) Command data corresponding to an event to be executed in the second terminal station in parallel with transmission of video data and audio data. And a means for transmitting the command data received from the first terminal station to the second terminal station according to a predetermined rule. Means for relaying and transmitting command data to the determined second terminal station, each second terminal station corresponding to each command data stored in the c-1) command storage means. Lee Terminal database for storing event data for executing the event, c-2) means for receiving command data transmitted from the base station in parallel with reception of video data and audio data, c-3) Means for acquiring event data corresponding to the received command data from the terminal database and executing the corresponding event, in the second terminal station receiving the command data, in parallel with reproduction of video and audio. , A video conference system in which an event corresponding to the received command data is executed. 8. The video conference system according to claim 1, wherein if the communication data amount of the video conference system exceeds a predetermined band, firstly, the command data is transferred. A video conferencing system characterized by limiting the bandwidth by limiting the amount. 9. The video conference system according to claim 1, wherein when the communication data amount of the video conference system exceeds a predetermined band, first, the command data, A video conference system characterized by performing band limitation by limiting the transfer amount in the order of 2 for audio data and 3 for video data.