JP2005341325A - Multi-point video conference system, multi-point video conference control method, server apparatus, multi-point video conference control program, and program recording medium thereof - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a multi-point video conference system, capable of relieving a load imposed on a server apparatus for controlling a communication conference and gathering the number of downstream connections of into one. <P>SOLUTION: When the server apparatus 1 receives a coded stream in the unit of VP from each client terminal 2, a header update processing section 12 carries out processing of rewriting a VOP header into a VP header and processing of revising a macroblock number in the VP header on the basis of related information between a predetermined client and a display position of the image and generates a coded stream resulting from compositing coded streams from a plurality of clients. Each client terminal receives the composite coded stream from the server apparatus 1 and decodes and displays the coded stream to display the composited image of the plurality of clients. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a multiplexing method of video encoded data, and in particular, a distribution server receives an encoded stream encoded in a plurality of client terminals having encoding and decoding functions, and the encoded stream is transmitted to each client terminal. It is related with the multipoint video conference system of the form which distributes.

  In general, in a multipoint video conference system, a distribution server receives encoded streams of video shot by a plurality of client terminals from each client terminal, and the distribution server decodes the encoded streams received from each client terminal. The decoded image of each client terminal is synthesized into one image, and the re-encoded image is distributed to each client terminal (for example, see Patent Document 1 and Patent Document 2). .

  FIG. 11 is a diagram showing an example of a conventional multipoint video conference system. In FIG. 11, reference numeral 100 denotes a server device that controls a multipoint video conference and distributes video to each client, and 110a to 110z are client terminals of clients participating in the multipoint video conference. The server device 100 includes reception buffers 101a to 101z corresponding to the client terminals 110a to 110z, decoding units 102a to 102z, an image synthesis unit 103, a re-encoding unit 104, and a transmission buffer 105.

  The server device 100 is connected to each client terminal 110a to 110z via a network, receives an encoded stream from each client terminal 110a to 110z, and stores it in reception buffers 101a to 101z corresponding to each client terminal. The decoding units 102a to 102z decode the encoded streams stored in the reception buffers 101a to 101z, respectively.

  The image composition unit 103 arranges the images decoded by the decoding units 102a to 102z at predetermined positions and composes them into one image, and the re-encoding unit 104 re-encodes the synthesized images. The re-encoded composite image stream is stored in the transmission buffer 105 and distributed to each of the client terminals 110a to 110z via the network.

  FIG. 12 is a diagram showing an example of another conventional multipoint video conference system. In FIG. 12, 120 is a server device that controls a multi-point video conference and distributes video to each client, and 130a to 130z are client terminals of clients participating in the multi-point video conference. The server device 120 includes reception buffers 121a to 121z and transmission buffers 122a to 122z corresponding to the client terminals 130a to 130z.

  Each of the client terminals 130a to 130z includes cameras 131a to 131z that capture the client, displays 132a to 132z, an encoding unit 133 that encodes the captured video, a transmission buffer 134 for transmitting the encoded stream, A plurality of reception buffers 135 for receiving the encoded streams distributed by the server apparatus 120, encoded stream decoding units 136 corresponding to the respective reception buffers 135, decoded images decoded by the respective decoding units 136, and the own terminal And an image composition unit 137 that composes an image captured by the camera.

  In the example of FIG. 11 described above, after the encoded stream of the client terminal is once decoded in the server apparatus 100, the entire image is synthesized and re-encoded. However, in the system of FIG. The encoded streams received from the client terminals 130a to 130z are distributed to the client terminals 130a to 130z via the reception buffers 121a to 121z and the transmission buffers 122a to 122z.

In each of the client terminals 130a to 130z, when the encoded stream distributed from the server device 120 is received by the reception buffer 135, the received encoded stream is decoded by the decoding unit 136, and the decoded image is sent to the image synthesis unit 137. In the image composition unit 137, these images and images taken by the terminal are synthesized and displayed on the displays 132a to 132z. Thereby, the video of the client participating in the video conference is displayed on the displays 132a to 132z of the client terminals 130a to 130z.
Japanese Patent No. 3097736 JP-A-11-187372

  In the conventional multipoint video conference system as shown in FIG. 11, after the encoded stream received from each of the client terminals 110a to 110z is once decoded in the server device 100, the entire image is synthesized and then re-encoded. There is a problem in that the server apparatus 100 needs to perform decoding, image synthesis, and re-encoding processing, which increases the load on the server apparatus 100.

  On the other hand, in the case of the conventional multi-point video conference system as shown in FIG. Therefore, there is a problem that the necessary communication ports increase as the number of clients participating in the video conference increases. There is also a problem that it is not preferable from the viewpoint of network resources and security.

  In the conventional technology as described above, each client terminal receives and decodes a composite image stream from the server device, and receives and decodes only an encoded stream of a specific client image from the server device. Thus, the same decoding unit could not be used for decoding.

  SUMMARY OF THE INVENTION An object of the present invention is to solve the above problems, and to realize a multipoint video conference system capable of reducing the load on the server device and integrating the number of downstream connections into one.

  In order to solve the above problems, the present invention provides a multipoint video conference system in which a server device receives encoded streams encoded by a plurality of client terminals and distributes the encoded streams to the respective client terminals. For the encoded stream encoded by the client terminal and uploaded to the server apparatus, the server apparatus obtains the macroblock number stored in the VP header of the encoded stream of each VP (Video Packet) unit from each client terminal. It changes so that it may become a corresponding display position of the synthesized image when it synthesize | combines an image, It distributes to each client terminal, It is characterized by the above-mentioned.

  In addition, the VOP (Video Object Plane) header of the encoded stream of the client other than the head display position in the composite image is rewritten to the VP header. The stream with the rewritten header is output for each VP unit (for example, one line of a macro block).

  Each client terminal that has received the encoded stream from the server device sequentially decodes in units of VP, and displays the decoded image at the display position indicated by the changed macroblock number in the VP header.

  When the client terminal selects one image area (stream), the server apparatus passes through the header update process and outputs the encoded stream from the client terminal corresponding to the designated image area as it is.

  The present invention relates to a multipoint video conference system in which a server device receives encoded streams encoded by a plurality of client terminals having encoding and decoding functions and distributes the encoded streams to the respective client terminals. Since the composite stream is generated and distributed by updating the header of each encoded stream uploaded to the server device and changing the macroblock number in the VP header, the code received from each client terminal in the server device The process for decoding the encrypted stream, the process for synthesizing a plurality of decoded images, and the process for re-encoding the synthesized image are not required, and the load on the server device can be reduced.

  Further, since the client terminal sequentially decodes the composite stream received from the server device in units of VP and displays the decoded image at the position corresponding to the macroblock number in the VP header, the encoding synchronization processing in each client terminal is performed. It is unnecessary, and an encoded stream can be generated, decoded, and displayed easily and efficiently.

  Also, since the encoded video data of each client terminal is synthesized and distributed from the server device, it is not necessary to establish connections as many as the number of clients participating in the video conference, and the number of downstream connections to the client terminals is integrated into one. And increase in resources necessary for communication ports and other communications can be suppressed.

  In addition, when the client terminal selects and receives one of the encoded streams of a specific client instead of the composite stream, it can be decoded by the same decoding unit as that for the composite stream, thus simplifying the decoding process. Can be

  Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 shows a configuration example of a multipoint video conference system according to the present invention. In FIG. 1, 1 is a server device that controls a multipoint video conference and distributes video to each client, and 2 is a client terminal of a client that participates in the multipoint video conference. In the embodiment of the present invention, for example, four clients A to D participate in a multipoint video conference using the client terminal 2. Of course, the number of participants in the communication conference according to the present invention is not limited to four, and a plurality of participants may be used.

  In the server apparatus 1, 11 is a reception buffer for receiving and storing the encoded stream transmitted from each client terminal 2, and 12 is sequentially receiving the encoded stream in units of VP from each reception buffer 11 and updating the contents of the VP header. In addition, a header update processing unit 13 to be combined is a transmission buffer in which the combined encoded stream is stored.

  In the header update processing unit 12, 14 is a stream input unit for reading the encoded stream stored in the reception buffer 11 in VP units, and 15 is a macro block (hereinafter referred to as MB) number after change in the VP header. MB number determination unit, 16 rewrites a specific VOP header to a VP header, and changes the MB number in the VP header to the MB number after the change, 17 is a VP unit in which the MB number is changed This is a stream output unit for transferring the encoded stream to the transmission buffer 13.

  In the client terminal 2, reference numeral 23 denotes an encoding unit that encodes video captured by the camera 21, 24 denotes a transmission buffer that stores an encoded stream to be transmitted to the server apparatus 1, and 25 denotes a synthesized code from the server apparatus 1. A reception buffer for storing the encoded stream, 26 is a decoding unit for decoding the encoded stream stored in the reception buffer 25, and 27 is a display control unit for controlling the display of the decoded data on the display 22.

  FIG. 2 is a diagram illustrating details of the encoding unit of the client terminal. In this example, an MPEG-4 encoding method is used. The encoding unit 23 is the same as an encoder generally used conventionally. The subtraction unit 231 calculates a difference signal between the input image signal and the predicted image signal that is the output of the motion compensation unit 239 for each macroblock. The DCT unit 232 performs a discrete cosine transform (DCT) on the difference signal. The quantization unit 233 quantizes the DCT coefficient according to the quantization parameter determined by the code amount control unit 241. The variable length coding unit 240 performs variable length coding on the quantized DCT coefficient and outputs the result to the transmission buffer 24.

  The quantized DCT coefficient is also output to the inverse quantization unit 234, and the inverse quantization unit 234 performs inverse quantization. The inverse DCT unit 235 performs inverse discrete cosine transform on the signal after inverse quantization. The adder 236 adds the output signal of the motion compensation unit 239 to the signal after inverse DCT. The frame memory 237 stores the added signal as a reference image. The motion prediction unit 238 performs motion prediction based on the reference image stored in the frame memory 237 and the input image signal. The motion compensation unit 239 generates a predicted image signal from the reference image stored in the frame memory 237 based on the motion vector detected by the motion prediction unit 238. The motion vector output from the motion prediction unit 238 is variable length encoded by the variable length encoding unit 240 and output to the transmission buffer 24.

  Also, the variable length coding unit 240 generates a VOP (Video Object Plane) header for each frame and adds it to the front of the MB encoded information, generates a VP header for each VP, To grant. The MB number information at the head of the MB encoded information is stored in the VP header.

  In the multipoint video conference system having the configuration shown in FIG. 1, first, the encoding unit 23 of each client terminal 2 encodes the video captured by the camera 21 and stores it in the transmission buffer 24.

  The reception buffer 11 of the server device 1 stores the code stream transmitted from the transmission buffer 24 of each client terminal 2 via the network in association with the client ID of the transmission source client. The stream input unit 14 of the header update processing unit 12 receives the client ID and the encoded stream in VP units from the reception buffer 11. The MB number determination unit 15 determines the changed MB number in the VP header based on the client ID.

  In addition, when the header is a VOP header, the header / MB number changing unit 16 of the header update processing unit 12 rewrites it into a VP header according to the client ID, and changes the MB number in the VP header to the MB after the change Change to a number. The stream output unit 17 sequentially transfers the encoded stream of VP units whose MB numbers have been changed to the transmission buffer 13.

  As a result, the encoded stream of each VP unit of a plurality of clients is distributed as a composite stream from the transmission buffer 13 to each client terminal 2 via the network.

  In the client terminal 2, the composite stream transmitted from the server device 1 is stored in the reception buffer 25. The decoding unit 26 sequentially decodes the composite stream stored in the reception buffer 25 in units of VP, and the display control unit 27 is located at a position corresponding to the MB number stored in the VP header in the display screen of the display 22. Display the decoded image.

  FIG. 3 is a diagram showing a VOP header and a VP header set by each client terminal. In general, VP is a unit obtained by dividing a VOP (Video Object Plane) into an arbitrary number of MBs. In the embodiment of the present invention, for example, encoding information of N MBs as shown in FIG. Data with a VP header (VOP header for the first VP) attached to one line is used as a VP unit, and each client terminal 2 transmits a VOP consisting of M VPs as an encoded stream. Shall.

  As shown in FIG. 3, in the first VP, information for decoding the entire VOP is stored before the encoded information of the Nth MBs from the 0th to the (N-1) th. A VOP header is added. In the second VP, a VP header of VP (N) is added before the encoded information of N MBs from the Nth to the 2N−1th.

  The number in parentheses of the VP header indicates the sequence number (MB number) of the first MB of the N MBs in the VP. Since the first MB in the second VP is the Nth MB, the MB number having a value of “N” is stored in the VP header of the second VP. Similarly, an MB number having a value of “(M−1) N” is stored in the VP header of the Mth VP.

  FIG. 4 is a diagram illustrating an example of a decoded image of a composite stream displayed on each client terminal, and FIG. 5 is a diagram illustrating an MB number of each MB in the decoded image illustrated in FIG. FIG. 6 is a diagram showing update information of the header of the encoded stream in VP units transmitted from each client for constituting a decoded image with the layouts shown in FIGS. 4 and 5.

  An example in which the decoded image of the encoded stream including the encoded information of N × M MBs transmitted from the client A to the client D is arranged in the decoded image of the composite stream with the layout shown in FIGS. I will explain to you. Here, the image size transmitted from each client and the image size of the decoded image of the composite stream are determined when a session between each client terminal 2 and the server apparatus 1 is established.

  Further, the server device 1 generates and holds arrangement position information that is correspondence information between a client ID for uniquely identifying each client and the arrangement position of the image of each client. This arrangement position information includes data items such as a client name, a client ID, an arrangement position, and an image size (number of MBs).

  As shown in FIG. 4, the image of client A is arranged at the upper left of the entire display screen, the image of client B is arranged at the upper right of the entire display screen, and the image of client C is arranged at the lower left of the entire display screen. In the example in which the image D is arranged at the lower right of the entire display screen, the server apparatus 1 holds arrangement position information as shown in FIG. The arrangement position may be coordinate information.

  Further, the header update processing unit 12 of the server device 1 generates and holds, for example, header update information shown in FIG. 6 based on the arrangement position information. The header update information is, for example, correspondence information between pre-change header information and post-change header information for each client ID. The header information before change is the type of the header of the encoded stream transmitted from each client (in the case of the VOP header) or the MB number information stored in the VP header. As the item, MB number information after the change is stored. Instead of having the header update information in the form of a table, it may be obtained by a predetermined calculation formula each time.

  The header update information shown in FIG. 6 will be described in detail. Since the image of client A with client ID = 1 is arranged at the upper left of the entire display screen, if the header of the encoded stream transmitted from client A is a VOP header, the header information after change is changed to “ “As is”.

  When the MB number stored in the VP header of the VP transmitted from the client A is “N”, the MB number in the decoded image of the first MB of this VP is “2N” as shown in FIG. Therefore, the corresponding post-change header information is set to “2N”.

  Further, for example, since the image of the client B is arranged at the upper right of the entire display screen, when the header of the encoded stream transmitted from the client B is a VOP header, the header information after change is set to “VP (N ) ”. The reason why the MB number stored in this VP header is “N” is that the MB number in the decoded image of the first MB of this VP is “N” as shown in FIG.

  In the same way, the header update processing unit 12 of the server apparatus 1 generates and holds update information of the header of the encoded stream transmitted from each client.

  FIG. 8 is a diagram illustrating an example of a processing flow of the header update processing unit 12. When the header update processing unit 12 receives the client ID and the encoded stream of VP units from the reception buffer 11 (step S1), the header update processing unit 12 refers to the update information of the header shown in FIG. Post header information is determined (step S2). Specifically, the MB number to be stored in the changed VP header is determined.

  Next, the VOP header / VP header is searched. Whether the header is a VOP header or a VP header can be determined by a difference in bit pattern (step S3). It is determined whether the header of the encoded stream is a VOP header or a VP header (step S4). If the header is a VOP header, whether the decoded image is an encoded stream transmitted by a client displayed at the upper left of the entire display screen. Is determined (step S5). For example, it is determined whether or not the client A with client ID = 1 whose decoded image is displayed on the upper left is the transmission source.

  If the client whose decoded image is displayed at the upper left of the entire display screen is the transmission source, it is transferred to the transmission buffer 13 in units of VP (step S7). For example, when the transmission source is the client A with the client ID = 1, the header information after the change corresponding to the header information “VOP header” before the change is “as is” as shown in FIG. Instead, the data is passed to the transmission buffer 13 in units of VP.

  In step S5, if the client whose decoded image is displayed at the upper left of the entire display screen is not the transmission source, the VOP header is changed to the VP header, and the changed MB number determined in step S2 is included in the VP header. Store (step S6) and proceed to step S7. For example, when the transmission source is the client C with the client ID = 3, the post-change header information corresponding to the pre-change header information “VOP header” in the header update information illustrated in FIG. 6 is “VP (2MN)”. Therefore, the MB number “2MN” is stored in the VP header.

  In step S4, if the header of the encoded stream is not a VOP header but a VP header, the MB number in the VP header is changed to the changed MB number determined in step S2 (step S8). Proceed to S7.

  For example, if the source client is client A with client ID = 1 and the MB number in the VP header of the received VP is 2N, the corresponding change is made as shown in the header update information shown in FIG. The rear header information is “4N”. Therefore, the MB number in the VP VP header is changed to “4N”.

  FIG. 9 is a diagram for explaining decoded image update processing in each client terminal. FIG. 9A shows an example of a composite stream composed of a plurality of VPs received by each client terminal 2 from the server device 1, and FIG. 9B shows a decoded image of each VP in the entire display screen. It is a figure which shows arrangement | positioning.

  Information of the header changed by the server device 1 is given to each VP constituting the composite stream shown in FIG. In the example shown in FIG. 9A, for example, VOP header information is assigned to the first VP, the MB number “2N” is assigned to the second VP, and the MB is assigned to the third VP. The information of the VP header with the number “N” is given.

  Each client terminal 2 that has received the composite stream shown in FIG. 9A from the server device 1 sequentially decodes in units of VP, and the decoded image in units of VP is displayed at the display position indicated by the information of the header of each VP. indicate. For example, a decoded image of a VP to which header information called a VOP header is assigned is arranged at the upper left position (position (1) in the figure) in the display screen shown in FIG. The decoded image of the VP to which the VP header information of “2N” is assigned is arranged at a position (position (3) in the figure) where the leading MB becomes the 2N-th MB through all MBs.

  Similarly, the decoded image of the VP to which the VP header information with the MB number “N” is assigned is arranged at the position (2) in the figure, and the VP header information with the MB number “3N” is given. The decoded image of the VP is arranged at the position (4) in the figure.

  When the frame rate of the encoded stream uploaded from each client terminal 2 to the server device 1 is different, an area that is frequently updated and an area that is less frequently updated in the display of the decoded image may occur. There is no problem because the display as a whole is not disturbed.

  FIG. 10 is a diagram illustrating a configuration example of a multipoint video conference system according to another embodiment of the present invention. In addition to the configuration shown in FIG. 1, each client terminal 2 has an image selection instruction unit 28. The server device 1 also includes an image selection information receiving unit 18 and a stream selection / switching unit 19.

  The image selection instruction unit 28 of the client terminal 2 determines whether to receive a combined stream obtained by combining images of a plurality of clients or to receive a single image of a specific client by inputting instruction information from the user, The image selection information is sent to the image selection information receiving unit 18 of the server device 1. The default is, for example, a composite stream. When the user clicks a specific client image with a pointing device such as a mouse in an image displayed by decoding the composite stream, the selection information of the client image is displayed in the image selection instruction unit 28. To the server device 1.

  Further, when the user clicks in a state where an image of a specific client is displayed, image selection information of a composite stream (composite image) is sent from the image selection instruction unit 28 to the server device 1.

  When the composite stream is selected, the stream selection / switching unit 19 in the server device 1 transmits the composite stream generated from the encoded stream of each client by the header update processing unit 12 via the transmission buffer 13 and the network. Delivered to the client terminal 2.

  Further, when the image selection information receiving unit 18 receives image selection information of a specific client, the stream selection / switching unit 19 is controlled by the image selection information, and the stream selection / switching unit 19 The encoded stream stored in the reception buffer 11 corresponding to the client ID is output to the transmission buffer 13 as it is. As a result, only the image of the selected specific client is displayed on the client terminal 2.

  In the server device 1 shown in FIG. 10, if a plurality of sets of the image selection information receiving unit 18, the stream selection / switching unit 19, and the transmission buffer 13 are provided for each client participating in the conference, each client terminal is provided. It is possible to individually select whether to display a composite image or a specific client image every two.

  The processes performed by the server device 1 and the client terminal 2 can be realized not only by hardware and firmware but also by a computer and a software program, and the program can be stored on a computer-readable recording medium. It can be recorded and provided through a network.

It is a figure which shows the structural example of the multipoint video conference system which concerns on this invention. It is a figure which shows the detail of the encoding part of a client terminal. It is a figure which shows the VOP header and VP header which are set by each client terminal. It is a figure which shows the example of the decoded image of a synthetic | combination stream. It is a figure which shows MB number of each MB in a decoded image. It is a figure which shows the update information of a header. It is a figure which shows arrangement position information. It is a figure which shows the example of the processing flow of a header update process part. It is a figure explaining the decoded image generation process in each client terminal. It is a figure which shows the structural example of other embodiment of this invention. It is a figure which shows the example of the conventional multipoint video conference system. It is a figure which shows the example of the other conventional multipoint video conference system.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1,100,120 Server apparatus 2,110a-110z, 130a-130z Client terminal 11,25,101a-101z, 121a-121z, 135 Reception buffer 12 Header update process part 13,24,105,122a-122z, 134 Transmission Buffer 14 Stream input unit 15 MB number determination unit 16 Header / MB number change unit 17 Stream output unit 18 Image selection information reception unit 19 Stream selection / switching unit 21, 131a to 131z Camera 22, 132a to 132z Display 23, 133 Coding Unit 26, 102a to 102z, 136 decoding unit 27 display control unit 28 image selection instruction unit 103, 137 image synthesis unit 104 re-encoding unit 231 subtraction unit 232 DCT unit 233 quantization unit 234 inverse quantization unit 235 inverse DCT unit 23 6 Adder 237 Frame memory 238 Motion predictor 239 Motion compensator 240 Variable length encoder 241 Code amount controller

Claims (6)

In a multipoint video conference system comprising a plurality of client terminals and a server device connected to the client terminals via a network and controlling a communication conference between the client terminals,
Each of the client terminals is
A frame of an input video in its own terminal is encoded in units of blocks, and a first header is added to the beginning of the frame for an encoded data group of a predetermined number of blocks, and an encoded data group other than the head of the frame is assigned to the encoded data group. On the other hand, means for generating an encoded stream provided with a second header including information indicating the position of the block;
Means for transmitting the generated encoded stream to the server device;
Means for receiving an encoded stream from the server device;
Means for decoding the received encoded stream;
Means for displaying the decoded image,
The server device
Means for receiving an encoded stream from each of the client terminals;
The encoded stream received from each client terminal is analyzed in units to which the first header or the second header is added, and based on relationship information between a predetermined client and the display position of the image. , By executing a process of rewriting the first header with the second header or a process of changing the information indicating the position of the macroblock in the second header to the corresponding display position. Header update processing means for generating an encoded stream obtained by synthesizing the received encoded stream;
Means for transmitting the encoded stream synthesized by the header update processing means to each of the client terminals. In the multipoint video conference system according to claim 1,
The client terminal is
A means for selecting whether to display an image composed of a plurality of client images or a specific client image, and to transmit the image selection information to the server device;
The server device
Means for receiving the image selection information;
Means for selecting either the encoded stream synthesized by the header update processing means or the encoded stream received from a specific client terminal based on the received image selection information, and transmitting the selected stream to the client terminal. A featured multipoint video conference system. In a multipoint video conference control method in a system comprising a plurality of client terminals and a server device connected to the client terminals via a network and controlling a communication conference between the client terminals,
The client terminal encodes an input video frame in its own terminal in units of blocks, and assigns a first header to the beginning of the frame for an encoded data group of a predetermined number of blocks. A process of generating an encoded stream with a second header including information indicating the position of the block for the encoded data group;
Each client terminal transmits the generated encoded stream to the server device;
A process in which the server device receives an encoded stream from each of the client terminals;
The server device analyzes the encoded stream received from each of the client terminals in a unit to which the first header or the second header is added, and determines a predetermined client and its image display position. Based on the relationship information, by executing a process of rewriting the first header to the second header or a process of changing information indicating the position of the macroblock in the second header to a corresponding display position, A header update process for generating an encoded stream obtained by combining the encoded streams received from the client terminals;
A process in which the server device transmits an encoded stream synthesized by the header update process to each client terminal;
A process in which the client terminal receives an encoded stream from the server device;
A process in which the client terminal decodes the received encoded stream;
The multi-point video conference control method characterized in that the client terminal includes a process of displaying a decoded image. A server device in a multipoint video conference system that is connected to a plurality of client terminals via a network and controls a communication conference between the client terminals,
Each client terminal encodes the frame of the input video in its own terminal in units of blocks, and adds a first header to the head of the frame for the encoded data group of a predetermined number of blocks, except for the head of the frame. Means for receiving an encoded stream with a second header including information indicating the position of the block for the encoded data group;
The encoded stream received from each client terminal is analyzed in units to which the first header or the second header is added, and based on relationship information between a predetermined client and the display position of the image. , By executing a process of rewriting the first header with the second header or a process of changing the information indicating the position of the macroblock in the second header to the corresponding display position. Header update processing means for generating an encoded stream obtained by synthesizing the received encoded stream;
Means for transmitting the encoded stream synthesized by the header update processing means to each of the client terminals. A multipoint video conference control program for causing a computer of a server device in a multipoint video conference system connected to a plurality of client terminals via a network to control a communication conference between the client terminals,
Each client terminal encodes the frame of the input video in its own terminal in units of blocks, and adds a first header to the head of the frame for the encoded data group of a predetermined number of blocks, except for the head of the frame. Means for receiving an encoded stream with a second header including information indicating the position of the block for the encoded data group;
The encoded stream received from each client terminal is analyzed in units to which the first header or the second header is added, and based on relationship information between a predetermined client and the display position of the image. , By executing a process of rewriting the first header with the second header or a process of changing information indicating the position of the macroblock in the second header to a corresponding display position. Header update processing means for generating an encoded stream obtained by synthesizing the received encoded stream;
As means for transmitting the encoded stream synthesized by the header update processing means to each client terminal,
A multipoint video conference control program for causing the computer to function. A computer-readable recording medium having a multipoint video conference control program connected to a plurality of client terminals via a network and recorded by a server computer in a multipoint video conference system for controlling a communication conference between the client terminals. A recording medium,
Each client terminal encodes the frame of the input video in its own terminal in units of blocks, and adds a first header to the head of the frame for the encoded data group of a predetermined number of blocks, except for the head of the frame. Means for receiving an encoded stream with a second header including information indicating the position of the block for the encoded data group;
The encoded stream received from each client terminal is analyzed in units to which the first header or the second header is added, and based on relationship information between a predetermined client and the display position of the image. , By executing a process of rewriting the first header with the second header or a process of changing information indicating the position of the macroblock in the second header to a corresponding display position. Header update processing means for generating an encoded stream obtained by synthesizing the received encoded stream;
As means for transmitting the encoded stream synthesized by the header update processing means to each client terminal,
A multipoint video conference control program recording medium, wherein a program for causing the computer to function is recorded.

Images