JP2005341324A - Multi-point video conference system, multi-point video conference control method, server apparatus, multi-point video conference control program, and program recording medium thereof - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a multi-point video conference system, capable of relieving a load imposed on a server apparatus for controlling a communication conference and gathering the number of downstream connections of into one. <P>SOLUTION: The server apparatus 1 provides macroblock number information (header update information), in response to a display position of an image of each client in a composited image to each client terminal 2. In each client terminal 2, when a coding section 23 attaches a header to a coded stream in the unit of VP, an MB number determining section 28 determines a macroblock number, denoting the position of the coded stream and stores the macro block number to a VP header. A VOP header is attached to the coded stream at the top of the composite image. The server apparatus 1 uses a stream selection / connection 12 to connect the coded streams received from each client terminal 2 in units of VP and distributes the connected coded stream to each client terminal 2. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a multiplexing method of video encoded data, and in particular, a distribution server receives an encoded stream encoded in a plurality of client terminals having encoding and decoding functions, and the encoded stream is transmitted to each client terminal. It is related with the multipoint video conference system of the form which distributes.

  In general, in a multipoint video conference system, a distribution server receives encoded streams of video shot by a plurality of client terminals from each client terminal, and the distribution server decodes the encoded streams received from each client terminal. The decoded image of each client terminal is synthesized into one image, and the re-encoded image is distributed to each client terminal (for example, see Patent Document 1 and Patent Document 2). .

  FIG. 11 is a diagram showing an example of a conventional multipoint video conference system. In FIG. 11, reference numeral 100 denotes a server device that controls a multipoint video conference and distributes video to each client, and 110a to 110z are client terminals of clients participating in the multipoint video conference. The server device 100 includes reception buffers 101a to 101z corresponding to the client terminals 110a to 110z, decoding units 102a to 102z, an image synthesis unit 103, a re-encoding unit 104, and a transmission buffer 105.

  The server device 100 is connected to each client terminal 110a to 110z via a network, receives an encoded stream from each client terminal 110a to 110z, and stores it in reception buffers 101a to 101z corresponding to each client terminal. The decoding units 102a to 102z decode the encoded streams stored in the reception buffers 101a to 101z, respectively.

  The image composition unit 103 arranges the images decoded by the decoding units 102a to 102z at predetermined positions and composes them into one image, and the re-encoding unit 104 re-encodes the synthesized images. The re-encoded composite image stream is stored in the transmission buffer 105 and distributed to each of the client terminals 110a to 110z via the network.

  FIG. 12 is a diagram showing an example of another conventional multipoint video conference system. In FIG. 12, 120 is a server device that controls a multi-point video conference and distributes video to each client, and 130a to 130z are client terminals of clients participating in the multi-point video conference. The server device 120 includes reception buffers 121a to 121z and transmission buffers 122a to 122z corresponding to the client terminals 130a to 130z.

  Each of the client terminals 130a to 130z includes cameras 131a to 131z that capture the client, displays 132a to 132z, an encoding unit 133 that encodes the captured video, a transmission buffer 134 for transmitting the encoded stream, A plurality of reception buffers 135 for receiving the encoded streams distributed by the server apparatus 120, encoded stream decoding units 136 corresponding to the respective reception buffers 135, decoded images decoded by the respective decoding units 136, and the own terminal And an image composition unit 137 that composes an image captured by the camera.

  In the example of FIG. 11 described above, after the encoded stream of the client terminal is once decoded in the server apparatus 100, the entire image is synthesized and re-encoded. However, in the system of FIG. The encoded streams received from the client terminals 130a to 130z are distributed to the client terminals 130a to 130z via the reception buffers 121a to 121z and the transmission buffers 122a to 122z.

In each of the client terminals 130a to 130z, when the encoded stream distributed from the server device 120 is received by the reception buffer 135, the received encoded stream is decoded by the decoding unit 136, and the decoded image is sent to the image synthesis unit 137. In the image composition unit 137, these images and images taken by the terminal are synthesized and displayed on the displays 132a to 132z. Thereby, the video of the client participating in the video conference is displayed on the displays 132a to 132z of the client terminals 130a to 130z.
Japanese Patent No. 3097736 JP-A-11-187372

  In the conventional multipoint video conference system as shown in FIG. 11, after the encoded stream received from each of the client terminals 110a to 110z is once decoded in the server device 100, the entire image is synthesized and then re-encoded. There is a problem in that the server apparatus 100 needs to perform decoding, image synthesis, and re-encoding processing, which increases the load on the server apparatus 100.

  In the case of the conventional multipoint video conference system as shown in FIG. 12, the load on the server device 120 may be small, but connection is made between the server device 120 and each of the client terminals 130a to 130z by the downstream amount. Therefore, there is a problem that the necessary communication ports increase as the number of clients participating in the video conference increases. There is also a problem that it is not preferable from the viewpoint of network resources and security.

  SUMMARY OF THE INVENTION An object of the present invention is to solve the above problems, and to realize a multipoint video conference system capable of reducing the load on the server device and integrating the number of downstream connections into one.

  In order to solve the above problems, the present invention provides a multipoint video conference system in which a server device receives encoded streams encoded by a plurality of client terminals and distributes the encoded streams to the respective client terminals. The server device notifies each client terminal of header update information for determining a macroblock (hereinafter referred to as MB) number corresponding to the image display position of each client. This header update information includes, for example, the MB number (offset) indicating the position of the first MB of each client image to be encoded in the decoded image related to the composite stream, and the number of MBs in one line of the decoded image related to the composite stream. (MB update parameter).

  Each client terminal determines the MB number of the first MB in each VP (Video Packet) generated in the encoding process based on the received header update information, stores the MB number in the VP header, An encoded stream in VP units is transmitted to the server device.

  The server device concatenates the encoded streams received from the client terminals, generates a composite stream, and distributes it to each client terminal. Each client terminal that has received the encoded stream from the server device sequentially decodes in units of VP, and displays the decoded image at the position indicated by the MB number in the VP header.

  That is, the present invention provides a multipoint video conference system comprising a plurality of client terminals and a server device connected to the client terminals via a network and controlling a communication conference between the client terminals. Means for notifying each client terminal of header update information for determining which position of the composite image when the encoded data of the images of each client corresponds to the composite image of a plurality of clients; Means for receiving an encoded stream from the terminal in units of video packets, and means for sequentially distributing the encoded stream received from each of the client terminals in units of video packets to each of the client terminals; , Means for encoding the input video at its own terminal, and code Means for adding header information to data in units of video packets, and setting information indicating the position of encoded data in a composite image determined based on header update information notified from the server device in the header; Means for transmitting an encoded stream of a video packet to which information is added to the server apparatus; means for receiving the encoded stream from the server apparatus in units of video packets; means for decoding the received encoded stream; Means for displaying the processed image data at a position set in the header.

  The present invention relates to a multipoint video conference system in which a server device receives encoded streams encoded by a plurality of client terminals having encoding and decoding functions and distributes the encoded streams to the respective client terminals. The client device stores the MB number determined based on the header update information received from the server device in the VP header, and transmits the encoded stream in VP units to the server device. Since a plurality of transmitted encoded streams are concatenated to generate and distribute a composite stream, a process for decoding the encoded stream received from each client terminal in the server device, a process for combining a plurality of decoded images, and a composite image No need to re-encode the server, reducing the load on the server Door can be.

  In addition, since the client terminal sequentially decodes the composite stream received from the server device in units of VP and displays the decoded image at the position corresponding to the MB number in the VP header, encoding synchronization processing in each client terminal is unnecessary. Therefore, it is possible to simply and efficiently decode and display the encoded stream when displaying an image.

  Also, since the encoded video data of each client terminal is synthesized and distributed from the server device, it is not necessary to establish connections as many as the number of clients participating in the video conference, and the number of downstream connections to the client terminals is integrated into one. And increase in resources necessary for communication ports and other communications can be suppressed.

  Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 shows a configuration example of a multipoint video conference system according to the present invention. In FIG. 1, 1 is a server device that controls a multipoint video conference and distributes video to each client, and 2 is a client terminal of a client that participates in the multipoint video conference. In the embodiment of the present invention, for example, four clients A to D participate in a multipoint video conference using the client terminal 2. Of course, the number of participants in the communication conference according to the present invention is not limited to four, and a plurality of participants may be used.

  In the server device 1, 11 is a reception buffer for storing the encoded stream transmitted from each client terminal 2, and 12 is sequentially received from each reception buffer 11 and connected to each VP-unit encoded stream to generate a combined stream. A stream selection / concatenation unit, 13 is a transmission buffer for storing the synthesized encoded stream, 14 is a header update information generation unit for generating header update information, and 15 is for transmitting the generated header update information to the client terminal 2. It is a header update information transmission unit.

  In the client terminal 2, reference numeral 23 denotes an encoding unit that encodes video captured by the camera 21, reference numeral 24 denotes a transmission buffer that stores the encoded stream, and reference numeral 25 denotes reception that stores the combined encoded stream from the server apparatus 1. A buffer, 26 is a decoding unit for decoding the encoded stream stored in the reception buffer 25, 27 is a display control unit for controlling the display of decoded data on the display 22, and 28 is a VP header in the encoding process by the encoding unit 23 It is an MB number determination part which determines the MB number stored in the inside.

  FIG. 2 is a diagram illustrating details of the encoding unit of the client terminal. In this example, an MPEG-4 encoding method is used. The encoding unit 23 is basically the same as a conventionally used encoder, but the variable length encoding unit 240 is different from the conventional encoder in generating header information.

  The subtraction unit 231 calculates a difference signal between the input image signal and the predicted image signal that is the output of the motion compensation unit 239 for each macroblock. The DCT unit 232 performs a discrete cosine transform (DCT) on the difference signal. The quantization unit 233 quantizes the DCT coefficient according to the quantization parameter determined by the code amount control unit 241. The variable length coding unit 240 performs variable length coding on the quantized DCT coefficient and outputs the result to the transmission buffer 24.

  The quantized DCT coefficient is also output to the inverse quantization unit 234, and the inverse quantization unit 234 performs inverse quantization. The inverse DCT unit 235 performs inverse discrete cosine transform on the signal after inverse quantization. The adder 236 adds the output signal of the motion compensation unit 239 to the signal after inverse DCT. The frame memory 237 stores the added signal as a reference image. The motion prediction unit 238 performs motion prediction based on the reference image stored in the frame memory 237 and the input image signal. The motion compensation unit 239 generates a predicted image signal from the reference image stored in the frame memory 237 based on the motion vector detected by the motion prediction unit 238. The motion vector output from the motion prediction unit 238 is variable length encoded by the variable length encoding unit 240 and output to the transmission buffer 24.

  Further, the MB number determination unit 28 determines the first MB number of each VP based on the header update information received from the server device 1. The variable length encoding unit 240 generates a VOP (Video Object Plane) header for each frame and assigns it before the MB encoded information, and generates a VP header for each VP and assigns it before the MB encoded information. To do. Information on the MB number determined by the MB number determination unit 28 is stored in the VP header. That is, in the VP header, not the original MB number indicating the position in the client image but the MB number indicating the MB position in the composite image obtained by combining the images of the plurality of clients is stored. These encoded streams are transmitted from the transmission buffer 24 to the server device 1 in units of VP.

  In this embodiment, the variable length encoding unit 240 sets the MB number determined by the MB number determining unit 28 in the VP header. However, the variable length encoding unit 240 is similar to the conventional one. The MB number is set in the VP header, and the VP header is rewritten based on the output of the MB number determining unit 28 when the encoded stream is stored in the transmission buffer 24 or when the encoded stream is transmitted to the server device 1. Implementation is also possible.

  The operation of the multipoint video conference system adopting the configuration shown in FIG. 1 is as follows. For example, when a session between each client terminal 2 and the server apparatus 1 is established, information on the image size transmitted from each client terminal 2 to the server apparatus 1 and the image size of the decoded image of the combined stream, and information on the layout of the combined stream And decide. Then, the header update information generation unit 14 of the server device 1 performs header update information for each client based on the image size and the image size information of the decoded image of the composite stream transmitted from each client terminal 2 to the server device 1. Is generated. The header update information transmission unit 15 transmits the generated header update information to the client terminal 2 of the corresponding client.

  The MB number determination unit 28 of each client terminal 2 determines the MB number to be stored in the VP header based on the header update information received from the server apparatus 1, and the encoding unit 23 outputs the MB number determination unit 28. Based on the signal, the video captured by the camera 21 is encoded to generate an encoded stream in VP units and stored in the transmission buffer 24. In the encoding process by the encoding unit 23, the MB number determined by the MB number determining unit 28 is stored in the VP header.

  The server device 1 receives the encoded stream transmitted from the transmission buffer 24 of each client terminal 2 by the reception buffer 11. The stream selection / concatenation unit 12 receives encoded streams in VP units from the reception buffer 11, stores them sequentially in the transmission buffer 13, and transmits them to the client terminal 2 via the network. A combined stream obtained by concatenating the encoded streams is distributed to each client terminal 2.

  In the client terminal 2, the composite stream distributed from the server device 1 is stored in the reception buffer 25. The decoding unit 26 sequentially decodes the composite stream stored in the reception buffer 25 in units of VP, and the display control unit 27 is located at a position corresponding to the MB number stored in the VP header in the display screen of the display 22. Display the decoded image.

  FIG. 3 is a diagram showing an example of a decoded image of the composite stream displayed on each client terminal 2, and FIG. 4 is a diagram showing an MB number of each MB in the decoded image shown in FIG. In the embodiment of the present invention, for example, as shown in FIG. 3, with respect to an image composed of N × M MBs of each of client A to client D, the image of client A is displayed at the upper left of the entire decoded image, and , An image of client C is arranged at the lower left, and an image of client D is arranged at the lower right.

  FIG. 5 is a diagram showing header update information for each client transmitted from the server apparatus 1 to the client terminal 2 in order to form a decoded image with the layout shown in FIGS. 3 and 4. The header update information shown in FIG. 5 includes the MB number (offset) in the decoded image related to the combined stream of the first MB of each client image to be encoded, and the number of MBs in one line of the decoded image related to the combined stream ( MB update parameter).

  In the embodiment of the present invention, as shown in FIGS. 3 and 4, the MB update parameter is “2N” because the number of MB in one line of the decoded image related to the composite stream is 2N.

  Further, for example, since the image of the client A is arranged at the upper left in the decoded image, referring to the MB number shown in FIG. 4, the offset for the client A (the MB number in the entire decoded image of the first MB of the encoding target image) ) Becomes “0”. Since the image of client B is arranged at the upper right in the decoded image, referring to the MB number shown in FIG. 4, the offset for client B is “N”. Similarly, the offset for client C is “2MN” and the offset for client D is “2MN + N”.

  The processing of the MB number determination unit 28 of each client terminal 2 will be described with reference to FIGS. FIG. 6 is a diagram showing an example of a processing flow of MB number determination by the MB number determination unit 28 of each client terminal and header generation in the variable length encoding unit 240. FIG. As a result, it is a figure which shows the header information (VOP header or VP header) provided to the encoded stream of each VP unit of the 1st column-M-th column on each client side. Note that the numbers in parentheses in the VP header in FIG. 7 indicate MB numbers stored in the VP header.

  In the processing flow shown in FIG. 6, the MB number determination unit 28 receives header update information from the server device 1 (step S1). For example, in the header update information shown in FIG. 5, the offset and MB update parameter corresponding to the client related to the terminal itself are received.

  Next, the variable-length encoding unit 240 determines whether the MB currently being encoded is the MB in the first column (line) in the client image associated with the terminal (step S2). If the MB is in the first column, it is determined whether the image of the client related to the terminal is an image arranged at the upper left in the decoded image related to the composite stream (step S3). A VOP header is created (step S4) and attached to the encoded stream (step S5).

  In step S2, if it is not the MB in the first column, the value of “offset + MB update parameter × (current column−1)” received in step S1 is set as the MB number, and the VP header storing the MB number is displayed. It is created (step S6) and given to the encoded stream (step S5).

  For example, the client terminal 2 related to the client A receives the offset “0” and the MB update parameter “2N” shown in FIG. 5 in step S1. Since the image of client A is arranged at the upper left in the decoded image, if the MB to be encoded is the MB in the first column, a VOP header is created (step S4) and attached to the encoded stream (Step S5). As a result, as shown in FIG. 7A, a VOP header is added to the VP in the first column.

  When the MB to be encoded is the MB in the second column, “offset + MB update parameter × (current column−1)” is “0 + 2N × (2-1) = 2N”, so the MB number “ 2N "is created (step S6), and the created VP header is added to the encoded stream. As a result, as shown in FIG. 7A, a VP header storing the MB number “2N” is added to the VP in the second column. For the third and subsequent columns, the processing according to the processing flow shown in FIG. 6 is performed. As a result, on the client A side, for example, a header (VOP header or VP header) as shown in FIG. 7A is added to the encoded stream of each VP unit in the first column to the Mth column.

  Further, for example, the client device 2 related to the client B receives the offset “N” and the MB update parameter “2N” shown in FIG. 5 in step S1. If the MB to be encoded is the MB in the first column, the process proceeds from step S2 to step S3. Since the image of client B is arranged at the upper right in the decoded image, the process proceeds from step S3 to step S6. In step S6, “offset + MB update parameter × (current column−1)” is “N + 2N × (1-1) = N”, so a VP header storing the MB number “N” is created (step S6). S6). As a result, as shown in FIG. 7B, a VP header storing the MB number “N” is added to the VP in the first column.

  When the encoding target MB is the MB in the second column, “offset + MB update parameter × (current column−1)” is “N + 2N × (2-1) = 3N” in step S6. VP header storing MB number “3N” is created. As a result, as shown in FIG. 7B, a VP header storing the MB number “3N” is assigned to the VP in the second column. For the third and subsequent columns, the processing according to the processing flow shown in FIG. 6 is performed. As a result, on the client B side, for example, a VP header as shown in FIG. 7B is added to the encoded stream of each VP unit in the first column to the Mth column.

  Also in the client device 2 related to the client C 2 and the client device 2 related to the client D, for example, as shown in FIGS. 7C and 7D, respectively, by performing processing according to the processing flow shown in FIG. Such a VP header is added to the encoded stream of each VP unit in the first column to the Mth column.

  FIG. 8 is a diagram illustrating an encoded stream generated in the client terminal 2. Here, an encoded stream composed of M VPs generated at a client terminal related to client A will be described as an example. Each VP is composed of N MB encoded information and a header (VOP header or VP header) added before the MB encoded information.

  As shown in the header information for client A in FIG. 7A, a VOP header is added to the VP in the first column. Therefore, in the VP in the first column shown in FIG. 8, the VOP header is added before the encoded information of the Nth MBs from the 0th to the (N-1) th.

  As shown in the header information in FIG. 7A, a VP header storing the MB number “2N” is assigned to the VP in the second column. Therefore, in the VP in the second column shown in FIG. 8, VP (2N) is added before the encoded information of the N MBs from the 2Nth to the 3N-1th. In the same manner, in the VP of the Mth column, VP (2 (M−1) before the encoded information of the N MBs from the 2nd (M−1) N to the 2nd MN−N−1th. ) N) VP header is added.

  FIG. 9A is a diagram showing an example of an encoded stream composed of a plurality of VPs transmitted from each client terminal 2 to the server apparatus 1, and FIG. 9B is a stream selection / concatenation of the server apparatus 1. FIG. 6 is a diagram illustrating an example of a composite stream generated in the unit 12. The characters in the VP shown in FIG. 9 indicate the VOP header or the MB number in the VP header.

  When the encoded stream shown in FIG. 9A is transmitted from each client to the reception buffer 11 of the server apparatus 1, the stream selection / concatenation unit 12 of the server apparatus 1 encodes the encoded stream from each reception buffer 11 in units of VPs. Are sequentially extracted and concatenated to generate a composite stream as shown in FIG.

  FIG. 10 is a diagram for explaining the decoded image update processing in each client terminal 2. FIG. 10A shows an example of a composite stream composed of a plurality of VPs received by each client terminal 2 from the server device 1, and FIG. 10B shows the arrangement of the decoded image of each VP in the entire display screen. FIG.

  Information of the header set by each client terminal 2 is given to each VP constituting the composite stream shown in FIG. In the example shown in FIG. 10A, for example, VOP header information is assigned to the first VP, the MB number “2N” is assigned to the second VP, and the MB is assigned to the third VP. The information of the VP header of the number “N”, the MB number “2M” is assigned to the fourth VP, and the MB number “3N” is assigned to the fifth VP.

  Each client terminal 2 that has received the composite stream shown in FIG. 10A from the server device 1 sequentially decodes in units of VP, and the decoded image in units of VP is displayed at the display position indicated by the information of the header of each VP. indicate. For example, a decoded image of a VP to which header information called a VOP header is assigned is arranged at the upper left position (position (1) in the figure) in the display screen shown in FIG. The decoded image of the VP to which the VP header information of “2N” is assigned is arranged at a position (position (3) in the figure) where the leading MB becomes the 2N-th MB through all MBs.

  Similarly, the decoded image of the VP to which the VP header information with the MB number “N” is assigned is arranged at the position (2) in the figure, and the VP header information with the MB number “3N” is given. The decoded image of the VP is arranged at the position (4) in the figure. As a result, an image obtained by combining a plurality of client images is displayed on the display 22 of the client terminal 2.

  When the frame rate of the encoded stream uploaded from each client terminal 2 to the server device 1 is different, an area that is frequently updated and an area that is less frequently updated in the display of the decoded image may occur. There is no problem because the display as a whole is not disturbed.

  The processes performed by the server device 1 and the client terminal 2 can be realized not only by hardware and firmware but also by a computer and a software program, and the program can be stored on a computer-readable recording medium. It can be recorded and provided through a network.

It is a figure which shows the structural example of the multipoint video conference system which concerns on this invention. It is a figure which shows the detail of the encoding part of a client terminal. It is a figure which shows the example of the decoded image of a synthetic | combination stream. It is a figure which shows MB number of each MB in a decoded image. It is a figure which shows the header update information for every client. It is a figure which shows the example of the processing flow of MB number determination and a header production | generation. It is a figure which shows the header information provided to the encoding stream of each VP unit. It is a figure which shows the encoding stream produced | generated in a client terminal. It is a figure which shows the example of the encoding stream transmitted from each client to a server apparatus, or the synthetic | combination stream produced | generated in a server apparatus. It is a figure explaining the decoding image update process in each client terminal. It is a figure which shows the example of the conventional multipoint video conference system. It is a figure which shows the example of the other conventional multipoint video conference system.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1,100,120 Server apparatus 2,110a-110z, 130a-130z Client terminal 11,25,101a-101z, 121a-121z, 135 Reception buffer 12 Stream selection / concatenation part 13,24,105,122a-122z, 134 Transmission buffer 14 Header update information generation unit 15 Header update information transmission unit 21, 131a to 131z Camera 22, 132a to 132z Display 23, 133 Encoding unit 26, 102a to 102z, 136 Decoding unit 27 Display control unit 28 MB number determination unit 103, 137 Image synthesizing unit 104 Re-encoding unit 231 Subtraction unit 232 DCT unit 233 Quantization unit 234 Inverse quantization unit 235 Inverse DCT unit 236 Addition unit 237 Frame memory 238 Motion prediction unit 239 Motion compensation unit 240 Variable length encoding unit 2 41 Code amount control unit

Claims (5)

In a multipoint video conference system comprising a plurality of client terminals and a server device connected to the client terminals via a network and controlling a communication conference between the client terminals,
The server device
Means for notifying each client terminal of header update information for determining which position of the composite image when the encoded data of each client image corresponds to the composite image of a plurality of clients;
Means for receiving an encoded stream from each client terminal in units of video packets;
Means for sequentially delivering the encoded stream received from each client terminal to each client terminal in units of video packets;
Each of the client terminals is
Means for encoding the input video at the terminal;
Means for adding header information to the encoded data in units of video packets, and setting information indicating the position of the encoded data in the composite image determined based on the header update information notified from the server device in the header;
Means for transmitting an encoded stream of a video packet to which the header information is added to the server device;
Means for receiving an encoded stream from the server device in units of video packets;
Means for decoding the received encoded stream;
Means for displaying the decoded image data at a position set in the header. A multipoint video conference system, comprising: In a multipoint video conference control method comprising a plurality of client terminals and a server device connected to the client terminals via a network and controlling a communication conference between the client terminals,
A process in which the server device notifies each client terminal of header update information for determining which position of the synthesized image when the encoded data of each client image is composed of a plurality of client images; ,
A process in which each of the client terminals encodes an input video in the terminal;
Information indicating the position of the encoded data in the composite image determined by each client terminal by adding header information to the encoded data in units of video packets and based on the header update information notified from the server device in the header The process of setting
A process in which each client terminal transmits an encoded stream of a video packet to which the header information is added to the server device;
A process in which the server device receives an encoded stream from each client terminal in units of video packets;
A process in which the server device sequentially delivers the encoded stream received from each client terminal to each client terminal in units of video packets;
Each client terminal receives a coded stream from the server device in units of video packets;
Each client terminal decoding a received encoded stream;
The multi-point video conference control method, wherein each client terminal has a process of displaying the decoded image data at a position set in the header. A server device in a multipoint video conference system that is connected to a plurality of client terminals via a network and controls a communication conference between the client terminals,
Means for notifying each client terminal of header update information for determining which position of the composite image when the encoded data of each client image corresponds to the composite image of a plurality of clients;
Means for receiving an encoded stream from each client terminal in units of video packets;
Means for delivering the encoded stream received from each client terminal sequentially to each client terminal in units of video packets. A multipoint video conference control program for causing a computer of a server device in a multipoint video conference system connected to a plurality of client terminals via a network to control a communication conference between the client terminals,
Means for notifying each client terminal of header update information for determining which position of the composite image when the encoded data of each client image corresponds to the composite image of a plurality of clients;
Means for receiving an encoded stream from each client terminal in units of video packets;
As a means for sequentially delivering the encoded stream received from each client terminal to each client terminal in units of video packets,
A multipoint video conference control program for causing the computer to function. A computer-readable recording medium having a multipoint video conference control program connected to a plurality of client terminals via a network and recorded by a server computer in a multipoint video conference system for controlling a communication conference between the client terminals. A recording medium,
Means for notifying each client terminal of header update information for determining which position of the composite image when the encoded data of each client image corresponds to the composite image of a plurality of clients;
Means for receiving an encoded stream from each client terminal in units of video packets;
As a means for sequentially delivering the encoded stream received from each client terminal to each client terminal in units of video packets,
A multipoint video conference control program recording medium, wherein a program for causing the computer to function is recorded.

Images