JP4741325B2 - Multipoint conference method and multipoint conference system - Google Patents

Description

The present invention, among the installed in the multi-point terminal, to transmit and receive video and audio via the server, about the multipoint conference method, and a multipoint conference system.

  By improving the performance of a personal computer (PC), a microphone, a speaker, and a camera are provided in the PC, and the PC is used for bidirectional video / audio communication with other PCs via a network. Is possible. In addition, a terminal specialized for video / audio communication has been realized using these technologies. These terminals are called “clients”. In addition, by using a server having a function of distributing video information / audio information and a call control function for distributing the information to each terminal, a plurality of clients can be connected via the server and It is also possible to provide video / audio communication.

A client that realizes a multipoint conference encodes video information captured by the camera of its own client, transmits it to the communication network, and receives video information transmitted from other clients via the communication network. Encodes and displays it on the monitor. Here, when there are three or more clients participating in the system, there are two major server processing methods for delivering video information transmitted from two or more other clients to a certain client. It can be roughly divided into two methods. First, the first is a method in which a plurality of video information distributed from other clients is selected by a server, or a plurality of video information is synthesized and distributed as a single stream, which is received by the distribution destination client. . In this method, the client of the distribution destination only needs to always receive the video information by one stream and perform the reproduction process. Therefore, even a client having a low processing capability can participate in the system. In the second method, the server distributes video information from other clients to each of the distribution destination clients using separate streams, and the distribution destination clients receive the video information separately from the plurality of streams. Thus, each video information is reproduced separately. In the second method, since a plurality of reception and reproduction processes are simultaneously performed in the distribution destination client, an ability to withstand a high load is required. A technology related to a conference system that transmits and receives video and audio information between clients via a server is disclosed (see Patent Document 1).
“Arcstar IP-VPN Network Image”, “online”, “Search July 20, 2005”, Internet <URL: http://www.ntt-vpn.com/ip-vpn/push/tv/push_tv_net. html>

  However, in the conventional multipoint conference system, in realizing the system, the multipoint conference system using only the first method as described above or the multipoint conference using only the second method is used. It was either a conference system.

Accordingly, an object of the present invention is to provide a multipoint conference method and a multipoint conference system that can participate in the system regardless of the level of processing capability of the client.

To achieve the above object, the present invention provides a first type terminal that can reproduce and process a plurality of received distribution videos, a second type terminal that can reproduce only one received distribution video, and the first type A multipoint conference processing method in a multipoint conference system comprising: a multipoint conference processing device that distributes video information and audio information transmitted from a terminal or the second type terminal to another terminal, wherein the multipoint conference processing device When the first type server in synthesizes the voice information received from each of the first type terminal and the second type terminal and distributes the synthesized voice information to the first type terminal , the synthesized voice delivered by deleting the first kind audio information terminal destination from information, the synthesized voice information when delivered to the second type terminal, voice information from the synthesized speech information second type terminal Delete Delivered, the first type terminal and the second type terminal video information received from each when distributing the first-type terminal, excluding the first type terminal of the destination, the one terminal or the other and distributing video information of the two terminals, and each of the first type terminal and the second type terminal, for delivering or receiving the video information and the audio information, IP address, and at least including communication identifier a port number , Exchanging distribution capability information used for transmission / reception of the video information and the audio information, and the second type server in the multipoint conference processing device destined for the second type terminal distributed by the audio distribution means When the video information received from the first type terminal or the video information received from the second type terminal via the first type server is distributed to the second type terminal, these first type terminals More The video information received from the second type terminal other than the distribution destination video information or the second type terminal is combined into one distribution video, and the combined video information is distributed to the second type terminal of the distribution destination, or the first type server A plurality of video information received from the first type terminal and the video information received from the second type terminal other than the distribution destination are selected and combined into one distribution video, and the combined video information is It is a multipoint conference method characterized by delivering to the second type terminal of the delivery destination .

The present invention also provides a first type terminal capable of reproducing each of a plurality of received distribution videos, a second type terminal capable of reproducing only one received distribution video, the first type terminals and the second type terminals. A multipoint conference processing apparatus that distributes the video information and audio information transmitted by the other terminal to the other terminal, wherein the multipoint conference processing apparatus includes the first type terminal and the second type terminal. with synthesized speech information received from each of the combined when delivering voice information to the first type terminals that delivery to remove the synthesized voice information of the first kind terminal of destination from the voice information and, the to the synthesized speech information when delivering to the second type terminal includes a voice delivery means for delivering from the synthesized speech information by deleting the voice information of the second type terminal, the first type terminal and the second type terminal each The video information received Ri, when delivering the first type terminal, the distribution destination other than the first type terminal and the other of the first kind terminal or the first video distribution means for distributing the video information of the two terminal A communication identifier including at least an IP address and a port number for distributing or receiving the video information and the audio information with each of the first type terminal and the second type terminal, and the video information and the audio information A connection control means for exchanging delivery capability information used for transmission / reception of the first type server, a voice transfer means for transferring voice information destined for a second type terminal delivered by the voice delivery means, Video information received from the first type terminal when the video information received from the first type terminal or the video information received from the second type terminal via the first type server is distributed to the second type terminal. Or arrangement The video information received from the second type terminal other than the destination is synthesized into one delivery video, and the synthesized video information is delivered to the second type terminal of the delivery destination or the first type server via the first type server. A plurality of video information received from a type 1 terminal and video information received from a type 2 terminal other than the distribution destination are selected and combined into one distribution video, and the combined video information is added to the distribution destination No. A multipoint conference system comprising: a second type server having second video distribution means for distributing to two types of terminals .

  According to the present invention, when a client capable of receiving and playing back a plurality of videos separately and a client capable of receiving and playing back only one video are mixed in a multipoint conference system, all of the plurality of videos or By providing a server having a processing function for combining a plurality of images into one, it is possible to configure a system in which even a client that can receive and reproduce only one video can participate in the multipoint conference system. That is, it is possible to provide a multipoint conference system in which all clients can participate in the system regardless of the level of processing capability of the clients.

  Hereinafter, a multipoint conference system according to an embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing the configuration of the multipoint conference system according to the embodiment. In this figure, reference numeral 10 denotes a server group. Reference numerals 20 and 30 denote clients. The clients 20 and 30 and the server group 10 are connected via a communication network. The client 20 is a type A client, and the type A is a client that can simultaneously receive and reproduce different video information distributed by a plurality of streams. Therefore, the type A client 20 has a processing function capable of withstanding a high load. The client 30 is a type B client, and the type B is a client that can receive and play back only one piece of video information distributed by one stream. Therefore, the client 30 is a terminal having a processing function inferior to that of the client 20. Incidentally, the difference between the processing capabilities of the client 20 and client 30, and the processing speed of CPU mounted, and memory capacity, such as by software, may be provided either in advance which type of terminal is determined, also, The type may be determined by the user determining and registering in the server group.

FIG. 2 is a diagram showing functional blocks of the client.
In this figure, functional blocks of type A and type B clients are shown. The type A client 20 includes functions of one audio processing unit 21, one video encoder 22, a plurality of video decoders 23, a camera 24, a microphone 25, and a speaker 26. The video decoder 23 of the client 20 performs a process of decoding each of a plurality of pieces of video information received by separate streams and displaying them on a display or the like. The video encoder 22 encodes the video shot by the camera 24 and transmits it to the server group 10 connected to the communication network. The audio processing unit 21 encodes the audio collected by the microphone 35 and transmits the encoded audio to the server group 10 connected to the communication network, and decodes the audio information received via the communication network and outputs it to the speaker 26. Perform the process.

  The type B client 30 includes functions of one audio processing unit 31, one video encoder 32, one video decoder 33, a camera 34, a microphone 35, and a speaker 36. The video decoder 33 of the client 30 performs a process of decoding only the video information received by one stream and displaying it on a display or the like. Similarly to the video encoder 22, the video encoder 32 encodes video captured by the camera 24 and transmits the encoded video to the server group 10 connected to the communication network. Similarly to the audio processing unit 21, the audio processing unit 31 encodes audio collected by the microphone 35 and transmits it to the server group 10 connected to the communication network, and decodes audio information received via the communication network. Then, a process of outputting to the speaker 36 is performed.

FIG. 3 is a diagram showing functional blocks in each server of the server group.
As shown in the figure, in the multipoint conference system according to the present embodiment, a conference server 11 and a video conversion server 12 exist as a server group. In the conference server 11, the conference management function unit 111 manages the conference system, such as the start of processing of the conference system, the request for participation of the client in the conference system, and the registration processing of the conference system start time. Process. In addition, the connection control function unit 112 transmits an IP address, a port number, and audio information and video information used when the video conversion server 12 and the clients 20 and 30 transmit and receive video information and audio information. Information required for distribution information <video: codec type (MPEG-4 SP3, etc.), bit rate, frame rate, image size, etc., audio: codec type (G.711, etc.), bit rate, etc.> A process of requesting the conversion server 12 and the clients 20 and 30 and notifying other apparatuses and clients is performed.

  The voice synthesis function unit 113 synthesizes the voice information received from the clients 20 and 30, and distributes the synthesized voice information to the client 20 from the synthesized voice information. Process to delete and distribute. Also, the audio information received from the plurality of clients 20 is synthesized and transmitted to the client 30 via the video conversion server 12. Further, when the video transfer function unit 114 distributes the video information transmitted by the clients 20 and 30 to the client 20 that is another distribution destination, the video transfer function unit 114 of the other clients 20 and clients 30 other than the client 20 of the distribution destination. Processing to select and distribute the transmitted video information. The video information received from the plurality of clients 20 is transmitted to the client 30 via the video conversion server 12.

  In the video conversion server 12, the connection control function unit 121 includes an IP address, a port number, audio information, and the like used when the server, the conference server 11, and the client 30 transmit or receive video information and audio information. Delivery capability information required for sending video information <Video: Codec type (MPEG-4 SP3, etc.), bit rate, frame rate, image size, etc. Audio: Codec type (G.711, etc.), bit rate Etc.> is requested and distributed to the conference server 11 and the client 30. The voice processing function unit 122 performs a process of transmitting the synthesized voice information transmitted from the conference server 11 to the client 30 and a process of transferring the voice information transmitted from the client 30 to the conference server 11. The video transcoding function unit 123 also selects or combines a plurality of video information received from the conference server 11 and distributes the video information transmitted from the client 30 to the conference server 11 to the conference server 11. The process to transfer is performed.

FIG. 4 is a diagram showing a connection establishment state.
FIG. 5 is a first diagram showing connection establishment processing.
FIG. 6 is a second diagram showing the connection establishment process.
FIG. 4 shows 15 connection establishment states from a to o when the clients 20-1 and 20-2 as type A and the client 30 as type B are used in the multipoint conference system. Show.
Here, a is a plurality of synthesized voice information (downlink) transmitted from the conference server 11 to the client 20-1 and voice information collected by the client 20-1 transmitted from the client 20-1 to the conference server 11 ( (Uplink) connection.
In addition, b indicates a connection (up) of video information transmitted from the client 20-1 to the conference server 11.
In addition, c indicates a connection for transmitting a video imaged by the client 20-2 among connections (downstream) of video information transmitted from the conference server 11 to the client 20-1.
In addition, d indicates a connection for transmitting a video imaged by the client 30 among connections (downstream) of video information transmitted from the conference server 11 to the client 20-1.

Also, e is a plurality of synthesized voice information (downlink) transmitted from the conference server 11 to the client 20-2 and voice information (uplink) collected by the client 20-2 transmitted from the client 20-2 to the conference server 11. ) Connection.
F indicates a connection (up) of video information transmitted from the client 20-2 to the conference server 11.
Further, g indicates a connection for transmitting a video captured by the client 20-1 among connections (downstream) of video information transmitted from the conference server 11 to the client 20-2.
In addition, h indicates a connection for transmitting video captured by the client 30 among video information connections (downstream) transmitted from the conference server 11 to the client 20-2.

I represents a connection between a plurality of synthesized audio information (downlink) transmitted from the conference server 11 to the video conversion server 12 and a voice information (uplink) of the client 30 transmitted from the video conversion server 12 to the conference server 11. Show.
Further, j indicates a connection (downward) through which the conference server 11 transmits the video information received from the client 20-1 to the video conversion server 12.
Further, k indicates a connection (downlink) through which the conference server 11 transmits the video information received from the client 20-2 to the video conversion server 12.
Further, l indicates a connection (uplink) through which the video conversion server 12 transmits the video information received from the client 30 to the conference server 11.

M represents a connection between a plurality of synthesized audio information (downstream) transmitted from the video conversion server 12 to the client 30 and a collected audio information (upstream) of the client 30 transmitted from the client 30 to the video conversion server 12. Show.
Further, n indicates a connection (downlink) of the converted video information transmitted from the video conversion server 12 to the client 30.
In addition, o indicates a connection (up) of video information transmitted from the client 30 to the video conversion server 12.

Next, a process for establishing a connection between a server group and a client will be described with reference to FIGS.
First, the conference server 11 accepts access from the clients 20-1, 20-2, and 30 in advance, and the participation registration of the clients in the conference system is performed. For example, the conference server 11 includes a processing unit of a web server, and in the web page distributed by the web server processing unit, the SIP address of each client, the conference identification number, Information such as the start time and information indicating whether each client is a type A or type B client are registered. These registration processes may be input to the conference server 11 by the administrator of the multipoint conference system. Thereby, for each conference identification number, each registration information such as the SIP address, conference start time, and type type of the clients 20 and 30 participating in the conference represented by the conference identification number is registered in the database.

  Next, the conference management function unit 111 of the conference server 11 compares the counted time with the conference start time recorded in the database, and detects the start of the process of the conference system when they match. Then, the conference management function unit 111 transmits information on the conference holding request to the connection control function unit 112 (step S1). In addition, the conference management function unit 111 transmits information on a conference participation request to the registered client to the connection control function unit 112 (step S2). Then, the connection control function unit 112 starts connection establishment processing between each registered client and the conference server 11 and the video conversion server 12. Hereinafter, the connection establishment process flow will be described in order.

First, it is assumed that the client 20-1 has already established a connection. Next, it is assumed that the conference management function unit 111 transmits the conference participation request information for the client 30 to the connection control function unit 112 in the above-described step S2. Then
(Step S3) The connection control function unit 112, IP address information used when the video transfer function unit 114 distributes video to the video conversion server 12, and distribution capability information <video: codec type (MPEG-4 SP3, etc.) , Bit rate, frame rate, image size, etc., audio: codec type (G.711, etc.), bit rate, etc.> are notified to the video conversion server 12. As a response, the connection control function unit 121 of the video conversion server 12 receives an IP address for transmitting / receiving video and audio information to / from the client 30 and each reception port for receiving the video and audio information from the client 30. The number and the distribution capability information for transmitting and receiving video and audio information to and from the client 30 are returned to the conference server 11.

  (Step S4) Next, the connection control function unit 112 transfers the information received from the video conversion server 12 in step S3 to the client 30. As a result, each information (IP address, reception port, and distribution capability information) used when the video conversion server 12 transmits and receives video and audio information to and from the client 30 is notified to the client 30. Then, the client 30 transmits / receives video / audio information to / from the video conversion server 12 and a reception port number used when receiving video / audio information from the video conversion server 12 to the conference server 11. Returns the delivery capability information to be used.

(Step S <b> 5) Next, the connection control function unit 112 notifies the video conversion server 12 of the reception port number used when receiving audio information from the video conversion server 12. Further, as a reply, the connection control function unit 121 of the video conversion server 12 notifies the conference server 11 of a transmission port number used when transmitting audio information to the conference server 11.
(Step S <b> 6) Next, the connection control function unit 112 notifies the video conversion server 12 of a transmission port number used when transmitting audio information to the video conversion server 12. As a reply, the connection control function unit 121 of the video conversion server 12 notifies the conference server 11 of a reception port number used when receiving audio information from the conference server 11.

  (Step S7) Then, the connection control function unit 112 notifies the information (the reception port number of the voice information) received from the video conversion server 12 in step S6 to the voice synthesis function unit 113. As a response, the speech synthesis function unit 113 transmits OK or NG information to the connection control function unit 112.

(Step S <b> 8) Next, the connection control function unit 112 notifies the video conversion server 12 of an IP address and a reception port number used when video information is received from the video conversion server 12. Then, as a reply, the connection control function unit 121 of the video conversion server 12 transmits to the conference server 11 the IP address used for transmitting the video information to the conference server 11 and its transmission port number.
(Step S9) Next, the connection control function unit 112 sends the information received from the client 30 in Step S4 (reception port numbers used when receiving video and audio and distribution capability information) to the video conversion server 12. Send. Thereby, information used when the client 30 transmits and receives video and audio information to and from the video conversion server 12 is notified to the video conversion server 12 via the conference server 11. In addition, as a reply, the video conversion server 12 transmits to the conference server 11 a transmission port number used when transmitting audio information to the client 30 and a transmission port number used when transmitting video information to the client 30. .

  (Step S10) Next, the connection control function unit 112 receives the information received from the video conversion server 12 in Step S8 (the IP address used when the video conversion server 12 transmits the video information to the conference server 11 and its transmission port number). ) To the video transfer function unit 114. The video transfer function unit 114 returns OK or NG information to the connection control function unit 112 as a reply.

(Step S11) Next, the connection control function unit 112 sends distribution capability information <video: codec type (MPEG-4 SP3, etc.), bit rate, frame rate, image size, etc.> as information when transmitting video information. Transmit to client 20-1. As a response, the client 20-1 transmits to the conference server 11 the reception port number when receiving video information from the conference server 1 and the distribution capability information used when transmitting the video information.
(Step S12) Further, the connection control function unit 112 notifies the video transfer function unit 114 of the information (reception port number and distribution capability information) received from the client 20-1 in step S11. Then, the video transfer function unit 114 returns OK or NG information as the reply.

  (Step S13) The connection control function unit 112 also uses the IP address and the transmission port number used when transmitting the video information to the video conversion server 12, and the distribution capability information used when transmitting the video <video: codec. Type (MPEG-4 SP3, etc.), bit rate, frame rate, image size, etc.> are transmitted to the video conversion server 12. Also, as a response, the video conversion server 12 receives the video information from the conference server 11 and its receiving port number, distribution capability information used for receiving the video information <video: codec type (MPEG-4 SP3, etc.), bit rate, frame rate, image size, etc.> are transmitted to the conference server 11.

  (Step S14) In addition, the connection control function unit 112 notifies the video transfer function unit 114 of the information (IP address, reception port number, distribution capability information) received from the video conversion server 12 in Step S13. Thereby, various information used when transmitting the video information to the video conversion server 12 is notified to the video transfer function unit 114. In response, the video transfer function unit 114 notifies the connection control function unit 112 of OK or NG information. As described above, the process for establishing a connection between the client 30 and each server in the server group 10 is completed by the processes in steps S2 to S14.

Next, the conference management function unit 111 of the conference server 11 notifies the connection control function unit 112 of a conference participation request to the client 20-2, and starts a connection establishment process with the client 20-2.
(Step S15) The connection control function unit 112 transmits an IP address for receiving video information and audio information and each reception port number to the client 20-2. Then, as a reply, the client 20-2 transmits to the conference server 11 a reception IP address and each reception port number for receiving audio information and video information.
(Step S16) Next, the connection control function unit 112 notifies the voice synthesis function unit 113 of the IP address received in step S15 and the reception port number for receiving the voice information. Also, as a response, the speech synthesis function unit 113 transmits OK or NG information to the connection control function unit 112.
(Step S17) The connection control function unit 112 notifies the video transfer function unit 114 of the IP address received in step S15 and the reception port number for receiving the video information. In addition, as a response, the video transfer function unit 114 transmits OK or NG information to the connection control function unit 112.

(Step S18) Next, the connection control function unit 112 uses an IP address used when transmitting / receiving video information to / from the client 20-1, and distribution capability information <video: codec type as information used for transmitting / receiving the video information. (MPEG-4 SP3, etc.), bit rate, frame rate, image size, etc.> are transmitted to the client 20-1. In response, the client 20-1 transmits the IP address used when transmitting / receiving the video information, the reception port number of the video information, and the distribution capability information as information used for transmitting / receiving the video information to the conference server 11.
(Step S19) The connection control function unit 112 notifies the video transfer function unit 114 of the reception port number and distribution capability information of the video information received from the client 20-1. In response, the video transfer function unit 114 notifies the connection control function unit 112 of OK or NG information.

(Step S20) The connection control function unit 112 also obtains the IP address used when transmitting the video information to the video conversion server 12, the transmission port number, and the distribution capability information used for transmission / reception of the video information. Send to. Then, as a reply, the connection control function unit 121 of the video conversion server 12 transmits to the conference server 11 an IP address for receiving the video information, its reception port, and distribution capability information.
(Step S21) Then, the connection control function unit 112 uses the information received from the video conversion server 12 in Step S20 (IP address, reception port, distribution capability information used when the video conversion server 12 receives the video information) as a video transfer function. Notification to the unit 114. In response, the video transfer function unit 114 transmits OK or NG information to the connection control function unit 112.

(Step S22) The connection control function unit 112 also transmits to the client 20-2 an IP address and distribution capability information used when transmitting the video information to the client 20-2. As a result, the client 20-2 returns the IP address, the reception port number, and the distribution capability information used when receiving the video information from the conference server 11 to the conference server 11.
(Step S23) The connection control function unit 112 converts the IP address received from the client 20-2, the reception port number, and distribution capability information into the first video information (for example, video information distributed from the client 20-1). ) Is sent to the video transfer function unit 114 as information used when transmitting to the client 20-2. Then, the video transfer function unit 114 returns OK or NG information.

(Step S24) The connection control function unit 112 also transmits to the client 20-2 an IP address and distribution capability information used when transmitting the video information to the client 20-2. Thereby, the client 20-2 returns the IP address, the reception port number, and the distribution capability information used when receiving the video information from the conference server 11 to the conference server 11.
(Step S25) The connection control function unit 112 receives the IP address received from the client 20-2, the reception port number, and distribution capability information, and second video information (for example, video information distributed from the client 30). The video transfer function unit 114 is notified as information used when transmitting to the client 20-2. Then, the video transfer function unit 114 returns OK or NG information.
With the above processing, the connection establishment processing (a to o shown in FIG. 4) between the client 20-2 and each server in the server group 10 is completed. The connection establishment process between the client 20-1 and the server group 10 is performed in the same manner.

  Then, when the connection establishment process is completed with all the clients participating in the conference, the conference server 11 notifies each client of a distribution start instruction. Then, the type A clients 20-1 and 20-2 sequentially transmit the video information captured by the camera 24 and the audio information collected by the microphone 25 to the conference server 11. In addition, the type B client 30 sequentially transmits the video information captured by the camera 34 and the audio information collected by the microphone 35 to the video conversion server 12. Then, distribution processing of video information and audio information is performed by processing of the conference server 11 and the video conversion server 12 described below.

<Processing in the conference server>
The voice synthesis function unit 113 of the conference server 11 receives and synthesizes voice information (voice information collected by the client 30) transmitted from the client 20-1, the client 20-2, and the video conversion server 12. Then, the synthesized audio information is transmitted to the client 20-1, the client 20-2, and the video conversion server 12. At that time, when transmitting the synthesized voice information to the client 20-1, the voice synthesis function unit 113 transmits the synthesized voice information by omitting the voice information received from the client 20-1 from the synthesized voice information. Similarly, when the synthesized voice information is transmitted to the client 20-2, the voice synthesis function unit 113 transmits the synthesized voice information by omitting the synthesized voice information from the synthesized voice information. When transmitting information to the video conversion server 12, the audio information received from the video conversion server 12 is omitted from the synthesized audio information.

  The video transfer function unit 114 of the conference server 11 transfers the video information transmitted from the client 20-1, the client 20-2, and the video conversion server 12. For example, the video transfer function unit 114 transmits the video information received from the video conversion server 12 and the client 20-2 to the client 20-1 as separate sessions. In addition, the video transfer function unit 114 transmits the video information respectively received from the video conversion server 12 and the client 20-1 to the client 20-2 as separate sessions. The video transfer function unit 114 transmits the video information received from the client 20-1 and the client 20-2 to the video conversion server 12 as separate sessions.

<Processing in video conversion server>
The audio processing function unit 122 of the video conversion server transfers the audio information received from the client 30 to the conference server 11 and transmits the synthesized audio information received from the conference server 11 to the client 30. The video transcoding function unit 123 combines video information received from the conference server 11 through separate sessions. At this time, all received video information may be synthesized, or only pre-selected video information may be synthesized. Then, the video transcoding function unit 123 transmits the synthesized video information to the client 30.

  In the case where a type A client 20 capable of receiving and playing back a plurality of videos separately and a type B client 30 capable of receiving and playing back only one video are mixed in the multipoint conference system by the above processing. By providing the video conversion server 12 having a processing function for synthesizing videos into one, it is possible to configure a system in which even a type B client can participate in a multipoint conference system. That is, it is possible to provide a multipoint conference system in which all clients can participate in the system regardless of the level of processing capability of the clients.

  The conference server, video conversion server, and client described above have a computer system inside. The process described above is stored in a computer-readable recording medium in the form of a program, and the above process is performed by the computer reading and executing this program. Here, the computer-readable recording medium means a magnetic disk, a magneto-optical disk, a CD-ROM, a DVD-ROM, a semiconductor memory, or the like. Alternatively, the computer program may be distributed to the computer via a communication line, and the computer that has received the distribution may execute the program.

  The program may be for realizing a part of the functions described above. Furthermore, what can implement | achieve the function mentioned above in combination with the program already recorded on the computer system, and what is called a difference file (difference program) may be sufficient.

It is a block diagram which shows the structure of a multipoint conference system. It is a figure which shows the functional block of a client. It is a figure which shows the functional block in each server of a server group. It is a figure which shows a connection establishment state. It is a 1st figure which shows a connection establishment process. It is a 2nd figure which shows a connection establishment process.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 ... Server group 11 ... Conference server 12 ... Video conversion server 20, 30 ... Client 111 ... Conference management function part 112, 121 ... Connection control function part 113 ... Speech synthesis Function unit 114 ... Video transfer function unit 122 ... Audio processing function unit 123 ... Video transcoding function unit

Claims (2)

A first type terminal capable of reproducing each of a plurality of received distribution videos;
A second type terminal that can process only one received distribution video;
A multipoint conference processing device that distributes video information and audio information transmitted by the first type terminal and the second type terminal to other terminals;
A multipoint conference method in a multipoint conference system comprising:
The first type server in the multipoint conference processing apparatus is
When synthesizing the voice information received from each of the first type terminal and the second type terminal and distributing the synthesized voice information to the first type terminal , the first of the distribution destinations is determined from the synthesized voice information . delivered by deleting the voice information of the seed terminal, when delivering the synthesized voice information to the second type terminals, and distributed from the synthesized speech information by deleting the voice information of the second type terminal,
The video information received from each of the first type terminal and the second type terminal, when delivering the first type terminal, except for the one terminal of the destination, the one terminal of the other or the two terminal Of video information ,
A communication identifier including at least an IP address and a port number for delivering or receiving the video information and the audio information with each of the first type terminal and the second type terminal, and transmission / reception of the video information and the audio information Exchange the delivery capability information used for
The second type server in the multipoint conference processing device is:
Transferring voice information destined for the second type terminal delivered by the voice delivery means;
Video information received from the first type terminal when distributing the video information received from the first type terminal or the video information received from the second type terminal to the second type terminal via the first type server The video information received from the second type terminal other than the information and the distribution destination is combined into one distribution video, and the combined video information is distributed to the second type terminal as the distribution destination, or via the first type server. The video information received from the first type terminal and the video information received from the second type terminal other than the distribution destination are selected and combined into one distribution video, and the combined video information is distributed to the distribution unit. A multipoint conference method characterized by delivering to a second type terminal . A first type terminal capable of reproducing each of a plurality of received distribution videos;
A second type terminal that can process only one received distribution video;
A multipoint conference processing device that distributes video information and audio information transmitted by the first type terminal and the second type terminal to other terminals;
A multipoint conference system comprising:
The multipoint conference processing apparatus includes:
When synthesizing the voice information received from each of the first type terminal and the second type terminal and distributing the synthesized voice information to the first type terminal , the first of the distribution destinations is determined from the synthesized voice information . delivered by deleting the voice information of the seed terminal, when delivering the synthesized voice information to the second type terminal delivers remove the synthesized voice information of the second type terminal from the audio information audio Delivery means;
The video information received from each of the first type terminal and the second type terminal, when delivering the first type terminal, except for the one terminal of the destination, the one terminal of the other or the two terminal First video distribution means for distributing the video information;
A communication identifier including at least an IP address and a port number for delivering or receiving the video information and the audio information with each of the first type terminal and the second type terminal, and transmission / reception of the video information and the audio information Connection control means for exchanging distribution capability information used for
A first type server having
A voice transfer means for transferring voice information destined for the second type terminal delivered by the voice delivery means;
Video information received from the first type terminal when distributing the video information received from the first type terminal or the video information received from the second type terminal to the second type terminal via the first type server The video information received from the second type terminal other than the information and the distribution destination is combined into one distribution video, and the combined video information is distributed to the second type terminal as the distribution destination, or via the first type server. The video information received from the first type terminal and the video information received from the second type terminal other than the distribution destination are selected and combined into one distribution video, and the combined video information is distributed to the distribution unit. A second video distribution means for distributing to the second type terminal ;
A second type server having
Multipoint conferencing system which is characterized in that it consists of.

Images