JPH07222131A - System and method for combining screen for multi-spot conference - Google Patents

Abstract

PURPOSE:To provide a combining system for screens for multi-spot conference which can shorten delay time cause by screen combination and can reduce the storage capacity of buffer memories. CONSTITUTION:One screen for each of terminals A-D is divided into three GOB by a QCIF system, compressively encoded and respectively stored in buffer memories 12a-12d. A GOB detection circuit 13 detects the GOB storage positions of the buffer memories 12a-12, detects the storage of screen data for one screen in one buffer memory or more and transmits a control signal to a control circuit 14. By the control of the control circuit 14, a combining multiplexer circuit 15 inserts a GOB '0' with no screen data into the GOB of the buffer memory for which one screen is not arranged, for example, and four screens are combined into a data sequence based on a CIF system and are outputted in order. On the other hand, a bit rate adjusting part 18 of the control circuit 14 adjusts a transmission line bit rate so as not to disturb a transmission line even when the data of GOB '0' are inserted.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a multipoint conference screen compositing system for compositing the screens of respective TV screens of a plurality of videoconference terminals with coded data which is data-compressed and sending the composited coded data to the terminals. In particular, the present invention relates to a multi-point conference screen composition system and method in which a coded data storage circuit is simplified and a delay time until an image is reproduced by a decoder is shortened.

[0002]

2. Description of the Related Art Conventionally, in this kind of multipoint conference screen compositing system, a multipoint control unit (MCU) 26, for example, as shown in FIG. In the case of receiving the encoded screen data from the terminal A, combining and multiplexing, and transmitting to the respective video conference terminals A to D, the terminal A
When one screen of screen data from D is collected, GOB (Group Of Blo), which is the minimum unit of encoded data.
image data for one screen is created.

Each of the video conference terminals A to D has encoders 21a to 21a.
21d, the encoders 21a to 21d receive continuous image signals of the television, that is, camera signals A to D, and the received image signals are H.264 standardized by ITU-T.
It is encoded by an encoding procedure based on the H.261 system, multiplexed with voice, control signal, etc., and output.

The structure of screen data will be described below. CIF adopted as the format of screen data
(Common intermediate format) system is NTSC system and PA
It is defined as a format common to the L system,
With one screen (picture) as one frame, a hierarchical structure is adopted in the order of GOB, macroblock, and block.
One frame is a collection of 6 × 2 (horizontal × horizontal) GOBs, each GOB is composed of 3 × 11 macroblock data, and each macroblock is composed of 2 × 2 blocks. Is composed of 8 × 8 pixels.

The frame and GOB are added with necessary information such as an identification code composed of a unique word and a mode code,
It is formed as encoded data. Addresses are added to macroblocks and blocks, and they are composed of only necessary bit data. As described above, since GOB has the identification code of the unique word, it constitutes the minimum unit capable of identifying the block delimiter of the encoded image data when decoding. Also, the vertical and horizontal resolutions reduced to 1/2 are defined as the QCIF (Quaternary Common Intermediate Format) system, and in this QCIF system, one screen is composed of three GOBs.

Here, it is assumed that the encoders 21a to 21d of the terminals A to D shown in the figure send three GOBs for one screen to the multipoint control unit (MCU) 26 by the QCIF method. That is, as shown in FIG. 8, GOBa1 to a3 are sent from the encoder 21a, and GOBb1, b2, b3 from the other encoders 21b to 21d.
~ D1, d2, d3 are respectively sent.

The multipoint control unit (MCU) 26 shown in FIG.
It has buffer memories 22a to 22d, a screen storage circuit 23, a control circuit 24, and a synthesizing / multiplexing circuit 25.
The buffer memories 22a to 22d are provided in the encoders 21a to 2d, respectively.
The encoded data of the image signal output from 1d is GOBa1 to a3, GOBb1 to b for each of three screens.
3, GOBc1 to c3, GOBd1 to d3 are received and temporarily stored and stored.

The screen storage circuit 23 detects that the buffer memories 22a to 22d store screen data of 12 GOBs for three screens and four terminals in total, and notifies the control circuit 24 of the screen data. To do. Based on the control signal from the control circuit 24 which received this notification, the synthesizing / multiplexing circuit 2
5 reads coded data from the buffer memories 22a to 22d in units of three GOBs, which are the minimum units, and, as shown in FIG.
The two GOBs are rearranged into the format of one screen of the CIF system to form and output the screen data. In this case, the coded data sent from each of the terminals A to D has a bit rate that is ¼ of the bit rate output from the synthesizing / multiplexing circuit 25.

[0009]

As described above, in the conventional multipoint conference screen compositing system, the multipoint control unit (MCU) receives all four screens of the QCIF system received from the terminals A to D, respectively. After being accumulated in the buffer memory,
It is output after being combined and multiplexed on one screen by the CIF method. In this configuration, the screen data sent from each of the terminals A to D is not synchronized, so that it is temporarily stored in each buffer memory and then stored in all buffer memories.
After the screen data for the frames are prepared, it is necessary to read them sequentially in the order of forming the composite screen.

For this reason, during the time when one screen data from the terminal which is the most delayed in transmission is stored in the buffer memory,
There is a problem in that the storage capacity of the buffer memory needs to be large enough so that screen data from other terminals does not overflow from the buffer memory.

In addition, when the buffer memory has a large storage capacity and it takes a long time to put the encoded data in and out of the buffer memory, the screen is sent from the sending terminal and then the receiving side decodes four screens. The delay time until the signal is reproduced becomes long. In such cases,
In the image transmission in the form of a conference performed while looking at the screen, there is a problem that the delay of the screen becomes large as a result of the delay of the screen becoming noticeable, the sense of discomfort is felt, and the smoothness of the conference is hindered.

An object of the present invention is to limit the storage capacity of a buffer memory for receiving screen data from each terminal and temporarily storing the screen data, and to reduce the delay time due to screen synthesis and transmission of screen data. It is to provide a screen synthesis system for a conference.

[0013]

A multipoint conference screen compositing system according to the present invention composes screens of respective TV screens of a plurality of videoconference terminals with encoded data which is data compressed,
In a multipoint video conference system for sending combined encoded data to a terminal, a buffer memory means for accumulating an image signal generated at each terminal in a minimum unit (GOB) of encoded data, and the GOB as a screen data at a predetermined location. The GOB detecting means for determining and outputting the same and the encoded data GOB for reading and reading the GOBs from the buffer memory means are arranged in the order necessary for performing the screen combination by the determination output. If there is no coded data, the coded data GOB0 indicating that there is no coded data is inserted into the vacant part, and the coded data GOB0 is combined and multiplexed, and the coded data GOB.
In order that the bit rate of the coded data of the combined screen does not exceed the bit rate of the transmission line even if 0 is inserted, the total of the coded data of the partial screen corresponding to each terminal is set smaller than the bit rate of the transmission line. It comprises a bit rate adjusting means and a decoding means for receiving and decoding the encoded data including the encoded data GOB0.

Further, as another means, a line control means for connecting and receiving the image signal generated at each terminal by connecting to the terminal by a digital line, and a connection control means for controlling connection of the conference line, and receiving from the terminal via the line control means. The screen data is extracted from the digital signal, and the minimum unit of encoded data (GO
Image data separation means for sending to the buffer memory means according to B) is additionally provided to the means, and the decoding means is provided in the terminal.

Further, the decoding means receives the encoded data, separates the screen data to be an image signal from the received encoded data and outputs the separation circuit, and the quantized data of the transform coefficient from the received screen data. And send it separately,
A variable length decoder for transmitting signals such as GOB number and motion vector, and the GOB number received from the variable length decoder,
Dequantize the quantized data received from the variable length decoder and a controller that controls the decoding process by sending a signal such as a motion vector, and decompress it to the original quantized level signal and send it. An inverse quantizer, an IDCT device that performs inverse discrete cosine transform of the signal received from the inverse quantizer to obtain a quantized prediction error signal and sends the quantized prediction error signal, and a prediction error signal from the IDCT device are separately generated. An adder that adds a prediction signal and obtains and outputs a decoded signal; and a motion adaptive predictor that adaptively obtains a prediction signal according to the motion vector received from the controller and predictively outputs the next prediction signal. , Delaying the decoded signal received from the adder by a delay circuit having a period of about 1 frame,
A frame memory for sending the delayed signal to the motion adaptive predictor and a D / A converter for digital / analog converting the decoded signal received from the adder and outputting the digital / analog converted signal.

[0016]

DESCRIPTION OF THE PREFERRED EMBODIMENTS Next, the present invention will be described with reference to the drawings.

FIG. 1 is a functional block diagram showing an embodiment of the present invention. In the multipoint conference screen compositing system shown in FIG. 1, the image signal compositing function is shown and described as in the conventional example, and the other functions are omitted. The multipoint control unit (MCU) 16 shown in FIG. 1 includes buffer memories 12a to 12d corresponding to the terminals A to D and a GOB detection circuit 1.
3, a control circuit 14, and a synthetic multiplexing circuit 15,
The control circuit 14 has a bit rate adjusting unit 18. The MCU 16 of FIG. 1 differs from the MCU of FIG. 7 in that
The GOB detection circuit 13 is provided instead of the screen detection circuit, and the bit rate adjustment unit 18 is provided in the control circuit 14.

In the data terminals A to D for handling images, the camera signals A to D are converted into screen data by the respective encoders 11a to 11d, and are compressed and encoded in GOB units by the QCIF method, and are transmitted. Sent to.

1 sent from each of the encoders 11a to 11d
The screen of the frame is divided into three GOBs and encoded as described in the related art. That is, as shown in FIG. 2, GOBa1 to a3 are sent from the encoder 11a, and GOBb1 and GOBb from the encoders 11b to 11d.
2, b3 to d1, d2, d3 are respectively sent.
The GOBa1 to a3 to be sent are temporarily stored and accumulated in the buffer memory 12a of the MCU 16. Similarly, GOBb
1, b2, b3 to GOBd1, d2 and d3 are temporarily stored and stored in the buffer memories 12b to 12d, respectively.

The GOB detection circuit 13 of the MCU 16 detects which position on the screen the GOB has been sent to each of the buffer memories 12a to 12d. When screen data for one screen by three GOBs is accumulated in one or more of the buffer memories 12a to 12d, the GOB detection circuit 13
Sends a GOB accumulation detection signal to the control circuit 14. The control circuit 14 sets the frequency at a predetermined cycle based on the detection result.
A control signal for combining one screen data into one combined screen data is output to the combining / multiplexing circuit 15. The combining / multiplexing circuit 15 combines the four screens from the buffer memories 12a to 12d in accordance with the control signal, performs a process of multiplexing, and outputs the combined / multiplexed signal.

For example, as shown in FIG. 2, data for one screen is stored in each of the three buffer memories 12a, 12c, and 12d, that is, three GOBa1 to a3 and cGO to c, respectively.
3 and d1 to d3 are accumulated, the combining and multiplexing circuit 15
Reads the data accumulated in the buffer memories 12a, 12c, 12d in GOB units in the order corresponding to the screen of one frame of the CIF method under the control of the control circuit 14.

In this case, the synthesizing / multiplexing circuit 15 determines that the terminal B
The missing screen data from is judged to be "no screen data", that is, "no change from previous screen", which is the same as the previous screen, and G is replaced by three GOBb1 to GOBb3 instead of G.
Data of OB0 is forcibly created and output. As a result,
As shown in FIG. 2, a format composed of 12 GOBs is combined as a CIF data string and output by the combining / multiplexing circuit 15.

As described above, GOB0 indicating that the difference is "0" is inserted into GOB0 at a position where one screen is not stored and sent out. For this reason,
The sum of the data rates of the encoders 11a to 11d needs to be smaller than the output rate of the synthesizing / multiplexing circuit 15 by the amount of data added at the time of insertion. That is, when the encoders 11a to 11d are output at the same transmission rate, the output of each encoder is set to a bit rate smaller than 1/4 of the transmission rate of the output of the MCU 16. These bit rates are controlled by the control circuit 14
A predetermined value may be set in advance in the bit rate adjusting unit 18.

The control circuit 14 of the MCU 16 for controlling the multipoint conference uses the bit rate adjusting unit 1 when the screen synthesis is performed.
8 to obtain adjustment information, and send a control signal to the terminals A to D as necessary to set the transmission bit rate of each of the encoders 11a to 11d.

When the bit rate sent from each terminal can be finely set and the screens can be combined, a small bit rate can be set for a terminal with a little motion and a large bit rate for a terminal with a lot of motion. It can also be controlled to be distributed.

A header is attached to each GOB. For this reason, a new header is also added to the data of each GOB that has been screen-combined as one screen data of the CIF method in accordance with the transmission.

The coded data is sent in a hierarchical structure for each frame. The data of each frame is
GO divided into frame headers and subsequent GOBs
It is composed of B data. As mentioned above, GOB
This corresponds to 1/12 of the CIF frame and 1/3 of the QCIF frame.

The decoder 17 receives the output frame of the MCU 16 and processes the part of GOB0 as the difference signal zero. Therefore, the monitor signal output from the decoder 17 does not change the screen of the terminal B, but changes only the screens of the other terminals A, C, and D. The number of GOBs of the screen data sent to the decoder 17 is four encoders 1.
Compared with the number of GOBs generated in 1a to 11d, GOB0
Is increased by the amount including. Therefore, the decoder 17 needs to perform decoding in a faster process than the encoders 11a to 11d. That is, a measure for shortening the decoding processing time for each screen of one frame or skipping the processing of the block of the difference signal zero by GOB0 is adopted, and the processing speed is equivalently increased. For details of the decoder 17,
An example will be described later with reference to FIG.

Next, with reference to FIG. 1 together with FIG. 3, another specific example of the combined multiplexed output will be described. In this example,
In each of the QCIF frames of the screens of the terminals A, B, C and D, 2 / of the screen is displayed for each of the buffer memories 12a to 12d.
Coded data of GOBs of 3, 1/3, 1/3, and 3/3 are accumulated, and only one terminal D stores three GOBs d1 to d3 for the entire screen in the buffer memory 11d. , GOB detection circuit 13 is detected.

As shown in FIG. 3, G being accumulated in frames other than GOBs d1 to d3 in the QCIF frame.
OB is GOBa1, GOBa2, GOBb1, GOB
It is assumed that they are four of c2. Further, in the CIF frame on the screen combination using the GOB data, GOB0 is inserted in the place of the GOB which is not accumulated. That is, when the GOB data is stored in each buffer memory when the screen synthesis is started and the GOB data is read in the arrangement order of the CIF frames, the stored GOB is read and the GOB data is not stored. A code GOB0 indicating that there is no difference signal is inserted as the GOB data.

In this way, the screen synthesis is performed for each GOB block as it is with the encoded data. Further, the header is also replaced according to the CIF frame. With this configuration, the number of GOBs accumulated in the buffer memories 12a to 12d of the MCU 16 and the retention time are minimized, and as a result, the storage capacity of the buffer memories 12a to 12d and the transmission delay time can be shortened. .

In this case, the decoder 17 carries out the decoding process with GOB0 as the difference signal zero. When the decoded image signal is displayed on the monitor, the difference signal is forcibly set to zero, so that the partial screen of the combined screen, which is decoded in the middle of the screen, is displayed. When it is visually unpleasant, an output frame memory may be provided separately from the decoding process frame memory.

With this configuration, in each of the four partial screens in the composite screen, when the third GOB is sent and decoded and three GOBs are gathered, the partial screen is output to the frame memory for output. Is temporarily written to. If the decoding device is configured to temporarily display the partial screen in the output frame memory and then output the display, the display of each partial screen is improved so as to be switched for each frame.

Even if the screens of the terminals A, B, C, and D are slightly different from the original frame rates in the combined screens, the difference can be regarded as an error range. To determine that three GOBs are aligned on each partial screen, the number indicating which GOB is included in one frame added to the screen data of the GOB is detected, and the third GOB of each partial screen is detected. This can be done by checking whether or not it is a GOB number.

Next, a specific example of the decoder 17 shown in FIG. 1 will be described with reference to FIG. As shown,
The decoder 17 includes a separation circuit 61, a variable length decoder 62, a controller 63, an inverse quantizer 64, an IDCT (inverse discrete cosine transform) 65, an adder 66, a motion adaptive predictor 67, a frame memory 68, and , D / A (digital / analog)
It has a converter 69.

In the decoder 17, the separation circuit 61 receives the encoded data, separates the screen data to be an image signal from the received encoded data, and outputs it to the variable length decoder 62. The variable length decoder 62 separates the quantized data of the transform coefficient from the received screen data and sends it to the inverse quantizer 64, and G
Signals such as the OB number, motion vector, and mode signal are sent to the controller 63. The controller 63 controls the motion vector signal, G
A signal indicating the presence / absence of OB0 is sent to the motion adaptive predictor 67 and the frame memory 68 to control the decoding process.

The dequantizer 64 dequantizes the quantized data received from the variable length decoder 62 to decompress it into a signal of the original quantized level, and sends it to the IDCT unit 65. The IDCT device 64 obtains a quantized prediction error signal by inverse discrete cosine transform of the received signal, and sends it to the adder 66. The adder 66 adds the prediction error signal from the IDCT unit 64 and the prediction signal from the motion adaptive predictor 67 to obtain a decoded signal, and sends this decoded signal to the frame memory 68 and the D / A converter 69. Send out.

The motion adaptive predictor 67 adaptively obtains a prediction signal according to the motion vector sent from the transmitting side,
Outputs the next prediction signal. The frame memory 68 delays the received decoded signal with a delay circuit having a period of approximately one frame, and sends the delayed signal to the motion adaptive predictor 67.

The decoder 17 having this configuration receives the GOB0
The block GO data is not decoded, and the next GOB is decoded. The frame memory 68 is configured to be able to read a decoded image signal from an arbitrary address in units of GOB blocks, skip the GOB0 block, and read the decoded image signal from the beginning of the next GOB block. The decoded image signal around the block is necessary for the motion adaptive predictor 67 to output a prediction signal according to the motion vector, and this decoded image signal is also read. For this purpose, the frame memory
In order to be able to separately read only the range at the head of the GOB, a dual memory is provided so that reading can be performed in parallel, or it is possible to perform reading at twice the frame period.

In another decoder, the decoding frame period is shortened to shorten the decoding processing time, and the GOB0 block is directly decoded as the differential signal "0", so that the GOB0 block can be dealt with.

Next, FIG. 5 is a block diagram showing a second embodiment of the present invention. The conference terminals 711 to 714 have the function of the decoder included in the MCU in FIG. 1 in addition to the conventional function of outputting the encoded data obtained by encoding and multiplexing the image signal and the audio signal, respectively. The multipoint conference control unit (MCU) 79 has a configuration in which the decoder is removed from the MCU of FIG. 1 and a line control circuit 72 for connecting digital lines from the conference terminals 711 to 714 is additionally provided. ing.

On the other hand, the line control device 72 is connected to the conference terminal 71.
1 to 714 through digital lines, and on the other hand, the transmission lines to the image data separation circuits 731 to 734 are connected to the output of the combining / multiplexing circuit 75 to control the connection of the conference participation line. Image data separation circuits 731-734
Outputs the screen data separated from the voice data from the encoded data received from the conference terminals 711 to 714.

The separated screen data is supplied to the buffer memories 741 to 744 in GOB units. The buffer memories 741 to 744, the composite multiplexing circuit 75, the GOB detection circuit 76, and the control circuit 77 have the same functions as those having the same names shown in FIG. That is, the GOB detection circuit 76 detects the GOB in the buffer memories 741 to 744.
Perform accumulation detection. The control circuit 77 outputs a screen synthesizing control signal to the synthesizing / multiplexing circuit 75, and the synthesizing / multiplexing circuit 75
Inserts a GOB0 signal indicating "no screen data for that block" into a block in which GOB data has not reached the buffer memory, and creates a composite screen from the screen data of four screens. An image signal based on the screen data synthesized in step S1 is obtained.

The control unit 78 for controlling the MCU 79 controls the line control circuit 72 for transmitting and receiving the coded data in which the signal such as the voice signal through the conventional path (not shown) is multiplexed by the synthesizing / multiplexing circuit 75. . The encoded data given to the line control circuit 72 through the line control circuit 72 is controlled by the control unit 78, and the conference terminals 711 to 71 are connected.
4 are sent to the decoding devices provided in the respective No. 4 and are decoded by the decoding devices. As a result, signals obtained by synthesizing four screens into one screen are output to the conference terminals 711 to 71.
It is output to the screen in each of 4

Next, FIG. 6 shows an embodiment in which the screens of the six terminals are combined. In this embodiment, the delay time can be shortened as will be apparent from the following description.

FIG. 6 shows the QC sent from six terminals.
Three GOBs (18 in total) of QCIF are extracted from each of the 6 screens by the IF frame. In this example,
Three GOBs included in the QCIF frame from each terminal
, Valid screen data is placed in two GOBs,
A QCIF frame in which a signal of GOB0 that is differentially encoded data "0" is arranged in the remaining one GOB is sent from each terminal to the MCU.

In the MCU, only valid two out of three GOBs are taken out, and a total of 12 GOs from six terminals are taken out.
B are arranged in order, formed into one CIF frame, and transmitted. MCU detects by GOB unit,
This GOB can only be found at the location where GOB data exists for each GOB.
Data is placed, and the data of the difference "0" is placed in other places.
By inserting GOB0, the screen of the CIF frame is synthesized. With this configuration, the delay time can be shortened.

In the above description, four QCIF frames are used.
One screen (Figs. 1-3) or six screens (Fig. 6)
Is formed on one composite screen of CIF frames. However, when sending one screen of one terminal to another terminal as it is, instead of a composite screen, the MCU controls the switching in response to the request from each terminal and selects and switches the encoded data from the desired terminal as it is. May be sent out. In this case, the image signal from the terminal is transmitted by the screen data switched to the CIF frame mode.

In addition, two of the four-screen composite display and the one-screen display
If there are two requests to the MCU, the MCU receives screen data from all terminals in the QCIF frame mode, and
The screen data of the QCIF frame may be sent to the terminal requesting the screen display. In this case, the MCU sends the composite screen data of four QCIF frames instead of the one screen of the QCIF frame to the terminal, and the receiving terminal requesting the one screen selects and displays the desired one screen. It has a function.

Furthermore, the terminal requested to display one screen is CI
The configuration may be such that the F frame is output in the mode and the MCU synthesizes the screen. In this case, a configuration is adopted in which the coded data corresponding to a black bar or the like is forcibly generated and inserted in the GOB portion of the screen of the mode based on the CIF frame, or the immediately preceding QCIF data is saved. Good. The resolution of the image displayed on a single screen can be increased by the configuration in which the data of the immediately preceding QCIF is stored.

Further, in this configuration, the CI is
It is not necessary to convert the F frame to the QCIF frame, and the multiplexing process of the dummy data is performed by the MCU. If only the signal that can distinguish the area where the dummy data is contained is multiplexed by the MCU, it is possible to multiplex the desired composite image signal in addition to the three screens decoded on the receiving side and display the screen of four-screen composition. It is possible.

The audio signal is not shown, and
Although not described, the audio signal is processed by the audio signal processing unit by a conventional method. When the image data separation circuit has a sound data separation function, the separated sound data is sent to the sound signal processing unit. The voice signal processing unit adds and synthesizes voice signals obtained from voice data from respective points. If the added and synthesized audio signal is distributed to each terminal as it is, the audio signal sent by itself returns as an echo at each terminal, and the generated audio becomes annoying. In order to prevent this, the audio signal processing unit
Each voice signal sent from each corresponding terminal is subtracted from the added and synthesized voice signal to obtain a synthesized voice signal, and this synthesized voice signal is converted to synthesized voice data and sent to each corresponding terminal. is doing.

Each synthesized voice data is multiplexed with the synthesized screen data and sent as encoded data to each terminal. Therefore, the function of multiplexing the synthetic voice data and the synthetic screen data can be included in the synthetic multiplexing circuit.

Although omitted in the embodiment, in the case of data that has been subjected to error correction coding, since the GOB structure cannot be seen, the buffer memory creates it in the error-correction-decoded data string, and CIF is used for screen multiplexing. It is necessary to perform error correction coding after multiplexing the GOB data sequence of If the audio data is also multiplexed, the audio data is separated and the necessary processing is performed.

As described above, the buffer memories 12a to 12d of the embodiment shown in FIG. 1 have different memory configurations for each terminal, but if the writing and reading speeds are increased, one memory is used. In the device, a memory area corresponding to each terminal can be separately defined, and a multiplexed memory configuration can be provided.

Further, since each GOB unit is detected, combined, and output, it is sufficient if any one of the partial screens is aligned with one frame.
Furthermore, if there is even one GOB, it will be combined and sent out.
It is not necessary to store all the GOBs until one frame is completed, and the delay time becomes the unit of GOB, and further, the delay time of the best frame rate of the encoder of the terminal can be shortened. Also, the buffer memory capacity can be reduced because the delay time can be reduced, and the hardware scale can be reduced.

In the above description, only the case of synthesizing in GOB units has been described, but the present invention can also be applied to the case of synthesizing in data units instead of GOB.

[0058]

As described above, according to the present invention, the screen synthesis is performed on a GOB basis, and the data of the difference "0" is forcibly inserted on a GOB basis at a place where there is no screen data. With this configuration, the delay time due to the screen synthesis in the MCU can be shortened and the buffer memory storage capacity can be reduced compared to the conventional configuration in which the screen synthesis is performed after waiting for the screens to be aligned. A screen synthesis system can be obtained.

[Brief description of drawings]

FIG. 1 is a functional block diagram showing an embodiment of the present invention.

FIG. 2 is an explanatory view of a combination screen showing an embodiment combined by FIG.

FIG. 3 is an explanatory view of a combining screen showing another embodiment combined by FIG.

4 is a functional block diagram showing an embodiment of the decoder of FIG. 1. FIG.

FIG. 5 is a functional block diagram showing another embodiment of the present invention.

FIG. 6 is an explanatory view of a composition screen showing an embodiment different from FIGS. 2 and 3;

FIG. 7 is a functional block diagram showing a conventional example.

FIG. 8 is an explanatory view of a combining screen showing an example of combining by FIG.

[Explanation of symbols]

 A to D terminals 11a to 11d Encoders 12a to 12d, 741 to 744 Buffer memory 13, 76 GOB detection circuit 14, 77 Control circuit 15, 75 Combined multiplexing circuit 16 Multipoint control unit (MCU) 17 Decoder 61 Separation circuit 62 variable length decoder 63 controller 64 inverse quantizer 65 IDCT (Inverse Discrete Cosine Transform) device 66 adder 67 motion adaptive predictor 68 frame memory 69 D / A converter 72 line control circuit 78 control unit 79 multipoint conference Control device 711-714 Conference terminal 731-734 Image data separation circuit

─────────────────────────────────────────────────── ─── Continuation of the front page (51) Int.Cl. ⁶ Identification code Internal reference number FI Technical indication H04N 7/18 U

Claims (8)

[Claims] 1. An image generated at each terminal in a multipoint video conference system in which each television screen of a plurality of video conference terminals is screen-combined with encoded data that is data-compressed and the combined encoded data is sent to the terminal. GOB (Group Of Block), which is the minimum unit of encoded data
buffer memory means for accumulating according to
The GOB detection means for determining and outputting that the OBs are aligned at predetermined locations as screen data, and the determination output,
Coded data GO for reading and reading the GOBs from the buffer memory means in the order necessary for screen composition
If B does not exist, coded data GOB0 indicating that there is no coded data is inserted in a vacant portion, and the coded data GOB0 for the coded data of the combined screen is combined and multiplexed. Bits that keep the total of the encoded data of the partial screen corresponding to each terminal smaller than the transmission line bit rate so that the bit rate of the encoded screen encoded data does not exceed the transmission line bit rate even if A multipoint conference screen synthesizing system comprising rate adjusting means and decoding means for receiving and decoding encoded data including the encoded data GOB0. 2. A multipoint video conferencing system for synthesizing screens of respective television screens of a plurality of video conference terminals with coded data obtained by data compression, and sending the synthesized coded data to the terminal. The minimum unit GOB of coded data is the line control means for connecting and receiving the image signal generated at each terminal and for controlling the connection of the conference line, and the screen data taken out from the digital signal received from the terminal via the line control means. Image data separating means to be transmitted by the GOB, buffer memory means for accumulating the coded data received from the image data separating means by the GOB, and GOB detection for judging and outputting that the GOB is aligned as a screen data at a predetermined position. Means, and the buffer memory means in the order necessary to perform screen synthesis based on this determination output. If the encoded data GOB to be read out is not complete, the encoded data GOB0 indicating that there is no encoded data is inserted in the empty portion to synthesize and multiplex the encoded data of the combined screen. The combining / multiplexing means for outputting and the coded data of the partial screen corresponding to each terminal so that the bit rate of the coded data of the combined screen does not exceed the bit rate of the transmission line even if the coded data GOB0 is inserted. A bit rate adjusting means for keeping the sum of the above values smaller than the transmission path bit rate, and a decoding means provided in the terminal for receiving and decoding the encoded data including the encoded data GOB0. A multi-point conference screen compositing system characterized in that 3. The separation circuit according to claim 1, wherein the decoding means receives the encoded data, separates the screen data to be an image signal from the received encoded data, and outputs the separated screen data. A quantized data of the transform coefficient is separated from the screen data and transmitted, and a variable length decoder for transmitting signals such as GOB number and motion vector, and the GOB number and motion vector received from the variable length decoder. A controller that sends out a signal and controls the decoding process, and an inverse quantizer that dequantizes the quantized data received from the variable length decoder to expand it to a signal of the original quantization level and sends it out. And an IDCT (Inverse Discrete Cosine Transform) device that performs inverse discrete cosine transform of the signal received from the inverse quantizer to obtain a quantized prediction error signal and sends it, and a prediction error signal from the IDCT device separately. Generate An adder that adds a prediction signal and obtains and outputs a decoded signal; and a motion adaptive predictor that adaptively obtains a prediction signal according to the motion vector received from the controller and predictively outputs the next prediction signal. , A frame memory 68 for delaying the decoded signal received from the adder by a delay circuit having a period of approximately one frame, and sending the delayed signal to the motion adaptive predictor; and a digital signal for decoding the decoded signal received from the adder. D / A (digital / analog converted and output
An analog) converter, which is a screen synthesis system for multipoint conference. 4. The multipoint conference screen compositing system according to claim 3, wherein the frame memory is provided with a dual memory capable of separately reading only the range at the head of the GOB. 5. The multi-point conference screen compositing system according to claim 3, wherein the frame memory is read out at a rate twice as fast as a frame period. 6. The multi-point conference screen compositing system according to claim 3, wherein the frame memory accelerates a decoding frame period and decodes a GOB0 block as a difference signal “0” as it is. . 7. The terminal according to claim 1, wherein each terminal has QCIF
The three GOBs of the (4-minute common intermediate format) system
The last one of the two is transmitted as GOB0, and the combining / multiplexing circuit receiving the encoded data forms one screen by the CIF method for six terminals. A multi-point conference screen composition system. 8. An image generated at each terminal in a multipoint video conference method in which each television screen of a plurality of video conference terminals is screen-combined with encoded data obtained by data compression and the combined encoded data is sent to the terminal. A buffer memory means for accumulating a signal for each required unit of encoded data is prepared, and the encoded data of the unit read from the buffer memory means is not available in a predetermined number. A method for synthesizing a screen for a multipoint conference, characterized in that coded data indicating that there is no coded data is inserted into, and coded data of a synthesized screen is synthesized and multiplexed.