JP3586484B2 - Screen synthesis system and method for multipoint conference - Google Patents

Description

[0001]
[Industrial applications]
The present invention relates to a multipoint conference screen synthesis system and method for synthesizing screens of coded data obtained by compressing respective TV screens of a plurality of video conference terminals with data and transmitting the synthesized coded data to the terminal. The present invention relates to a multipoint conference screen synthesizing system and method in which a storage circuit for coded data is simplified and a delay time until an image is reproduced by a decoder is reduced.
[0002]
[Prior art]
Conventionally, in this type of multipoint conference screen synthesizing system, as shown in FIG. 7, a multipoint control unit (MCU) 26 is coded from each of the video conference terminals A to D at four points, for example. When screen data is received, combined and multiplexed and transmitted to each of the video conference terminals A to D, when the screen data from the terminals A to D is completed for one screen, the minimum unit of the encoded dataDefined asGOB (Group Of Blocks)Unit ofAnd creates image data for one screen.
[0003]
Each of the video conference terminals A to D has encoders 21a to 21d. The encoders 21a to 21d receive continuous image signals of the television, that is, camera signals A to D, and convert the received image signals into ITU-D signals. H. standardized by H. H.261, and multiplexed with audio, control signals, etc., and output.
[0004]
Here, the structure of the screen data will be described. The CIF (Common Intermediate Format) system adopted as the format of screen data is defined as a format common to the NTSC system and the PAL system, and one screen (picture) is defined as one frame and GOB, macro block, and block are used. Hierarchical structure is adopted in order. One frame is a collection of 6 × 2 (vertical × horizontal) GOBs. The GOB is composed of 3 × 11 macroblock data, and the macroblock is composed of 2 × 2 blocks. Is composed of 8 × 8 pixels.
[0005]
The frame and the GOB are formed as encoded data by adding necessary information such as an identification code and a mode code made of a unique word. The macro block and the block have addresses added thereto and are composed of only necessary bit data. Since the GOB has the unique word identification code as described above, it constitutes a minimum unit that can identify a block segment of encoded image data when decoding. In addition, one in which the resolution in both the vertical and horizontal directions is reduced to 各 々 is defined as a QCIF (4-minute common intermediate format) system. In this QCIF system, one screen is composed of three GOBs.
[0006]
Here, it is assumed that the encoders 21a to 21d of the illustrated terminals A to D transmit three GOBs for one screen to the multipoint control unit (MCU) 26 by the QCIF method. That is, as shown in FIG. 8, GOBa1 to GO3 are transmitted from the encoder 21a, and GOBb1, b2, b3, to d1, d2, and d3 are transmitted from the other encoders 21b to 21d, respectively. I have.
[0007]
The multipoint control unit (MCU) 26 in FIG. 7 includes buffer memories 22 a to 22 d, a screen storage circuit 23, a control circuit 24, and a combining and multiplexing circuit 25. The buffer memories 22a to 22d receive the coded data of the image signals output from the encoders 21a to 21d for three GOBa1 to GO3, GOBb1 to b3, GOBc1 to c3, and GOBd1 to d3 for one screen, Temporarily memorize and store.
[0008]
The screen storage circuit 23 detects that the buffer memories 22a to 22d have stored the screen data of a total of 12 GOBs for three screens and four terminals for one screen, and notifies the control circuit 24 of the detection. Based on the control signal from the control circuit 24 receiving this notification, the combining and multiplexing circuit 25 reads out the coded data from the buffer memories 22a to 22d three by the minimum unit GOB, and as shown in FIG. Next, the screen data is formed and output by rearranging the data of four screens of the QCIF system, that is, 12 GOBs into the format of one screen of the CIF system. In this case, the coded data sent from each of the terminals A to D has a bit rate of １ ／ of the bit rate output from the combining and multiplexing circuit 25.
[0009]
[Problems to be solved by the invention]
As described above, in the conventional multipoint conference screen synthesizing system, the multipoint control unit (MCU) stores all four screens of the QCIF system received from the terminals A to D in the buffer memory, The data is synthesized and multiplexed on one screen according to the system and output. In this configuration, since the screen data sent from each of the terminals A to D is not synchronized, the screen data is temporarily stored in each buffer memory, stored in all the buffer memories, and the screen data for one frame is prepared. Thereafter, it is necessary to sequentially read out the images in the order of forming the composite screen.
[0010]
For this reason, the storage capacity of the buffer memory is set large enough so that screen data from other terminals does not overflow from the buffer memory until one screen data from the terminal whose transmission is delayed most is stored in the buffer memory. Therefore, there is a problem that the storage capacity of the buffer memory becomes large.
[0011]
Further, in the case where the storage capacity of the buffer memory is large and it takes a long time to take in and out the coded data in the buffer memory, etc., the decoded signal obtained by combining the four screens on the receiving side after the screen is transmitted by the transmitting terminal is reproduced. The delay time before the operation becomes longer. In such a case, in the image transmission in the form of a conference in which the user looks at the screen, the delay of the screen becomes large, and the mutual response becomes noticeable. There are points.
[0012]
An object of the present invention is to provide a screen for a multipoint conference in which the size of the storage capacity of a buffer memory for temporarily storing screen data received from each terminal can be limited, and a delay time due to screen synthesis and transmission of screen data can be reduced. It is to provide a synthesis system.
[0013]
[Means for Solving the Problems]
The screen synthesizing system for a multipoint conference according to the present invention synthesizes each TV screen received from a plurality of video conference terminals with encoded data obtained by data compression, and synthesizes the encoded data.Each terminalIs displayed on the screenScreen synthesis system for multipoint conferenceAt
SaidOn each terminalGOB(Group Of Blocks, The smallest unit of encoded data of the image signal), and receives the received image signal for each GOB.Buffer memory means for storing;
For at least one terminal, the GOB constituting the television screen of the terminal is equivalent to one screen.As screen dataOf the buffer memoryGOB detection means for judging and outputting that it is aligned with a predetermined location;
The GOB is read out from the buffer memory means in the order necessary for performing the screen composition based on the determination output,GOB to readEncoded dataAccumulatedIf notIn the part of GOB that is not storedA multiplexing means for inserting coded data GOB0 indicating that there is no coded data and for multiplexing the coded data of the combined screen, and encoding the combined screen even if the coded data GOB0 is inserted; Ensure that the data bit rate does not exceed the transmission path bit rateSum of the output bit rates of the encoders of each terminalAnd a multi-point control device having bit rate adjusting means for keeping the bit rate smaller than the transmission line bit rate, and in each of the terminals, when receiving the encoded data including the encoded data GOB0, Decoding means for decoding in a processing time in which an increase due to the encoded data GOB0 is compensated.
[0014]
As described above, the multipoint control apparatus inserts the coded data GOB0 into the unreceived GOB portion of the coded data on the screen. However, even when the coded data GOB0 is inserted, it does not exceed the bit rate of the transmission path. By reducing or skipping the processing time for deleting the encoded data GOB0 received by the decoding means of each terminal receiving this, the size of the storage capacity of the buffer memory for receiving and temporarily storing the screen data from each terminal Is limited, and the delay time due to screen composition and transmission of screen data is reduced.
[0015]
Ie, The decoding meansIsA separating circuit for receiving encoded data, separating and outputting screen data that becomes an image signal from the received encoded data, separating and transmitting quantized data of transform coefficients from the received screen data, and A variable-length decoder for transmitting a signal such as a motion vector, a controller for transmitting a signal such as the GOB number and the motion vector received from the variable-length decoder and controlling decoding processing, A dequantizer that dequantizes the quantized data received from the decoder to expand the signal to the original quantization level and sends the signal, and an inverse discrete cosine transform of the signal received from the dequantizer to perform quantization. An IDCT (Inverse Discrete Cosine Transform) unit for obtaining and transmitting a coded prediction error signal, an adder for adding a prediction signal generated separately from the prediction error signal from the IDCT unit and obtaining and transmitting a decoded signal; , The system A motion adaptive predictor that adaptively obtains a prediction signal according to the motion vector received from the adder and predicts and outputs the next prediction signal, and delays a decoded signal received from the adder by a delay circuit having a period of substantially one frame. A frame memory for sending a delayed signal to the motion adaptive predictor, and a D / A (digital / analog) converter for digital / analog converting and outputting a decoded signal received from the adder.Is characterized bying.
[0016]
【Example】
Next, the present invention will be described with reference to the drawings.
[0017]
FIG. 1 is a functional block diagram showing one embodiment of the present invention. In the screen synthesizing system for a multipoint conference shown in FIG. 1, the image signal synthesizing function is illustrated and described as in the conventional example, and other functions are omitted. A multipoint control unit (MCU) 16 shown in FIG. 1 includes buffer memories 12a to 12d corresponding to terminals A to D, a GOB detection circuit 13, a control circuit 14, and a multiplexing / multiplexing circuit 15. A bit rate adjusting unit 18 is provided therein. The MCU 16 in FIG. 1 differs from the MCU in FIG. 7 in that a GOB detection circuit 13 is provided instead of a screen detection circuit, and a bit rate adjustment unit 18 is provided in the control circuit 14.
[0018]
In the data terminals A to D which handle images, the camera signals A to D are converted into screen data by the respective encoders 11a to 11d, and are also subjected to data compression encoding in GOB units by the QCIF method and transmitted to the transmission path. You.
[0019]
A screen of one frame transmitted from each of the encoders 11a to 11d is divided into three GOBs and encoded as described in the related art. That is, as shown in FIG. 2, GOBa1 to aBa are transmitted from the encoder 11a, and GOBb1, b2, b3, to d1, d2, and d3 are transmitted from the encoders 11b to 11d, respectively. The transmitted GOBa1-a3 are temporarily stored in the buffer memory 12a of the MCU 16. Similarly, GOBb1, b2, b3 to GOBd1, d2, d3 are temporarily stored and stored in the buffer memories 12b to 12d, respectively.
[0020]
The GOB detection circuit 13 of the MCU 16 detects where on the screen the GOB has been sent to each of the buffer memories 12a to 12d. When screen data for one screen by three GOBs is accumulated in one or more of the buffer memories 12a to 12d, the GOB detection circuit 13 sends a GOB accumulation detection signal to the control circuit 14. The control circuit 14 outputs a control signal for synthesizing the four screen data to one synthesized screen data at a predetermined cycle based on the detection result to the synthesis multiplexing circuit 15. The combining and multiplexing circuit 15 combines the four screens from the buffer memories 12a to 12d according to the control signal, performs a multiplexing process, and outputs a combined and multiplexed signal.
[0021]
For example, as shown in FIG. 2, when data for one screen, that is, three GOBa1 to a3, c1 to c3, and d1 to d3, respectively, are stored in three buffer memories 12a, 12c, and 12d, respectively. The multiplexing circuit 15 reads the data stored in the buffer memories 12a, 12c, and 12d in GOB units in an order corresponding to one frame of the CIF system under the control of the control circuit 14.
[0022]
In this case, the combining and multiplexing circuit 15 determines the missing screen data from the terminal B as "no screen data", that is, "there is no change from the previous screen," which is the same as the previous screen. The data of GOB0 is forcibly created and output in place of the two GOBb1 to GOBb3. As a result, as shown in FIG. 2, a format composed of 12 GOBs is combined as a data stream of the CIF system, and output by the combining and multiplexing circuit 15.
[0023]
As described above, GOB data indicating that the difference is “0” is inserted into GOB0 at a place where one screen is not accumulated, and sent out. Therefore, the sum of the data rates of the encoders 11a to 11d needs to be lower than the output rate of the combining and multiplexing circuit 15 by the amount of data added at the time of insertion. That is, when the encoders 11a to 11d output at the same transmission rate, the output of each encoder is set to a bit rate smaller than 1/4 of the transmission rate of the output of the MCU 16. These bit rates may be set to predetermined values in the bit rate adjusting unit 18 of the control circuit 14 in advance.
[0024]
The control circuit 14 of the MCU 16, which controls the multipoint conference, drives the bit rate adjustment unit 18 to obtain adjustment information when performing screen synthesis, and sends control signals to the terminals A to D as necessary to transmit each encoder. The transmission bit rates of 11a to 11d are set.
[0025]
If the bit rate transmitted from each terminal can be set finely and screen composition can be performed, allocate a small bit rate to terminals with a screen with little motion and allocate a high bit rate to a terminal with a screen with lots of motion. Can also be controlled.
[0026]
Each GOB is provided with a header. For this reason, a new header is also added to the data of each GOB screen-synthesized as one screen data of the CIF system in accordance with the transmission.
[0027]
The encoded data is sent in a hierarchical structure for each frame. The data of each frame includes a frame header and GOB data that is divided for each GOB. As described above, the GOB is equivalent to 1/12 of the CIF frame and 1/3 of the QCIF frame.
[0028]
The decoder 17 receives the output frame of the MCU 16 and processes the location of GOB0 as the difference signal zero. Therefore, the monitor signal output from the decoder 17 does not change the screen of the terminal B, but changes only the screens related to the other terminals A, C, and D. The number of GOBs of the screen data sent to the decoder 17 is larger than the number of GOBs generated by the four encoders 11a to 11d by the amount including GOB0. For this reason, it is necessary for the decoder 17 to perform the decoding with a faster process than the encoders 11a to 11d. That is, measures are taken to shorten the processing time of decoding for each frame of one frame, or to skip the processing of the block with zero difference signal in GOB0, thereby equivalently increasing the processing speed. The details of the decoder 17 will be described with reference to FIG.
[0029]
Next, another specific example of the combined multiplexed output will be described with reference to FIGS. In this example, in the QCIF frames of the screens of the terminals A, B, C, and D, GOB encoding of 2/3, 1/3, 1/3, and 3/3 of the screen is performed for each of the buffer memories 12a to 12d. A case is shown in which the GOB detection circuit 13 detects that data has been accumulated and that only one terminal D has accumulated three GOBs d1 to d3 for the entire screen in the buffer memory 11d.
[0030]
As shown in FIG. 3, it is assumed that the GOBs accumulated in frames other than GOBd1 to GOBd3 in the QCIF frame are GOBa1, GOBa2, GOBb1, and GOBc2. In addition, GOB0 is inserted at a location of a GOB that is not stored in the CIF frame on the screen synthesis using the GOB data. That is, when screen synthesis is started and GOB data is read out in the order of arrangement of CIF frames, if GOB data is stored in each buffer memory, the stored GOB is read out and if GOB data is not stored Is inserted with a code GOB0 indicating that there is no difference signal as the GOB data.
[0031]
In this way, screen synthesis is performed for each GOB block without changing the encoded data. Also, the header is replaced according to the CIF frame. In this configuration, the number of GOBs stored in the buffer memories 12a to 12d of the MCU 16 and the residence time are minimized, and as a result, the storage capacities of the buffer memories 12a to 12d and the transmission delay time can be reduced. .
[0032]
In this case, the decoder 17 performs the decoding process by setting the location of GOB0 as the difference signal zero. When the decoded image signal is displayed on the monitor, the difference signal is forcibly set to zero, so that the decoded partial image of the combined screen is displayed in the middle of the screen. If the image is visually unsightly, a frame memory for output may be provided separately from the frame memory for decoding.
[0033]
In this configuration, in each of the four partial screens in the composite screen, the third GOB is transmitted and decoded, and when the three GOBs are completed, the partial screen is temporarily written to the frame memory for output. Is done. If the decoding device is configured to temporarily write the partial screen in the output frame memory and then output and display, the display of each partial screen is improved so as to be switched for each frame.
[0034]
Even if the screens of the terminals A, B, C, and D are slightly different from the original frame rates in the combined screens, the difference can be regarded as an error range. The determination that three GOBs are completed in each partial screen is performed by detecting the number indicating which GOB in one frame added to the screen data of the GOB, and determining the third GOB of each partial screen. This can be implemented by checking whether the number is a GOB number.
[0035]
Next, a specific example of the decoder 17 shown in FIG. 1 will be described with reference to FIG. As illustrated, the decoder 17 includes a separation circuit 61, a variable length decoder 62, a controller 63, an inverse quantizer 64, an IDCT (inverse discrete cosine transform) unit 65, an adder 66, and a motion adaptive predictor 67. , A frame memory 68, and a D / A (digital / analog) converter 69.
[0036]
In the decoder 17, the separation circuit 61 receives the encoded data, separates screen data that becomes an image signal from the received encoded data, and outputs the screen data to the variable length decoder 62. The variable length decoder 62 separates the quantized data of the transform coefficient from the received screen data and sends out the same to the inverse quantizer 64, and sends out signals such as a GOB number, a motion vector, and a mode signal to the controller 63. The controller 63 sends a motion vector signal, a signal indicating the presence or absence of GOB0, and the like to the motion adaptive predictor 67 and the frame memory 68 to control decoding processing.
[0037]
The inverse quantizer 64 inversely quantizes the quantized data received from the variable length decoder 62, decompresses the quantized data to a signal of the original quantization level, and sends the signal to the IDCT unit 65. The IDCT unit 64 obtains a quantized prediction error signal by performing an inverse discrete cosine transform on the received signal, and sends it to the adder 66. The adder 66 adds the prediction error signal from the IDCT unit 64 and the prediction signal from the motion adaptive predictor 67 to obtain a decoded signal, and sends the decoded signal to the frame memory 68 and the D / A converter 69. Send out.
[0038]
The motion adaptive predictor 67 adaptively obtains a prediction signal according to the motion vector transmitted from the transmission side, and outputs the next prediction signal. The frame memory 68 delays the received decoded signal by a delay circuit having a period of about one frame, and sends the delayed signal to the motion adaptive predictor 67.
[0039]
The decoder 17 having this configuration does not perform decoding processing on the received data of the GOB0 block, and proceeds to the decoding operation of the next GOB. The frame memory 68 is configured to be able to read the decoded image signal from an arbitrary address in GOB block units, and skips the GOB0 block and reads the decoded image signal from the beginning of the next GOB block. Note that the decoded image signal around the block is necessary for the motion adaptive predictor 67 to output a predicted signal corresponding to the motion vector, and the decoded image signal is also read out. For this purpose, the frame memory is provided with a double memory so that data can be read out separately only in a range at the head of the GOB so that data can be read out in parallel, or read out at twice the frame period. Configure as possible.
[0040]
In another decoder, the countermeasure for the GOB0 block can be taken by shortening the decoding processing time by increasing the decoding frame period and decoding the GOB0 block as the difference signal “0” as it is.
[0041]
Next, FIG. 5 is a block diagram showing a second embodiment of the present invention. The conference terminals 711 to 714 have a decoder function included in the MCU in FIG. 1 in addition to the conventional function of encoding and multiplexing the image signal and the audio signal and outputting coded data. The multipoint conference control unit (MCU) 79 has a configuration in which a decoder is deleted from the MCU of FIG. 1 and a line control circuit 72 for connecting digital lines from the conference terminals 711 to 714 is additionally provided. ing.
[0042]
The line control device 72 is connected to the conference terminals 711 to 714 via a digital line on the one hand, and connects a transmission line to the image data separation circuits 731 to 734 and the output of the combining and multiplexing circuit 75 on the other hand, Perform connection control of the conference participation line. The image data separation circuits 731 to 734 separate the encoded data received from the conference terminals 711 to 714 from audio data, and output respective screen data.
[0043]
The separated screen data is supplied to the buffer memories 741 to 744 in GOB units. The buffer memories 741 to 744, the combining and multiplexing circuit 75, the GOB detection circuit 76, and the control circuit 77 have the same functions as those having the same names shown in FIG. That is, the GOB detection circuit 76 detects GOB accumulation in the buffer memories 741 to 744. The control circuit 77 outputs a control signal for screen synthesis to the synthesis multiplexing circuit 75, and the synthesis multiplexing circuit 75 indicates "there is no screen data for that block" for a block whose GOB data has not reached the buffer memory. While inserting the GOB0 signal indicating that the image signal is obtained, an image signal based on screen data synthesized from the screen data of the four screens into one synthesized screen is obtained.
[0044]
A control unit 78 that controls the MCU 79 controls a line control circuit 72 that transmits and receives coded data obtained by multiplexing a signal such as an audio signal through a conventional path (not shown) by a combining and multiplexing circuit 75. The encoded data provided to the line control circuit 72 through the line control circuit 72 is sent to decoding devices provided in the conference terminals 711 to 714 under the control of the control unit 78, and is decoded by the decoding device. Be converted to As a result, a signal obtained by combining the four screens into one screen is output to each of the conference terminals 711 to 714.
[0045]
Next, FIG. 6 shows an embodiment in which the screens of the six terminals are combined. In this embodiment, the delay time can be reduced as will be apparent from the following description.
[0046]
In FIG. 6, three GOBs (18 in total) of QCIF are extracted from each of six screens based on QCIF frames transmitted from six terminals. In this example, of the three GOBs included in the QCIF frame from each terminal, valid screen data is arranged in two GOBs and GOB0 of differentially encoded data “0” is assigned to the remaining one GOB. The QCIF frame in which the signal is arranged is transmitted from each terminal to the MCU.
[0047]
In the MCU, only two valid GOBs are taken out of the three GOBs, and a total of 12 GOBs from the six terminals are arranged in order and transmitted in one CIF frame. The MCU performs detection in GOB units, arranges this GOB data only in a location where GOB data exists in GOB units, and inserts data “0” with a difference “0” and GOB0 in other locations to thereby obtain the CIF frame. Combine screens. With this configuration, the delay time can be reduced.
[0048]
In the above description, four screens (FIGS. 1 to 3) or six screens (FIG. 6) based on the QCIF frame are formed on one composite screen of the CIF frame. However, when one screen of a certain terminal is sent to another terminal as it is, instead of a composite screen, the MCU performs switching control according to a request from each terminal, and switches and selects encoded data from a desired terminal as it is. May be sent. In this case, the image signal from the terminal is transmitted by the screen data switched to the CIF frame mode.
[0049]
When two requests, that is, four-screen combined display and one-screen display, are received by the MCU, the MCU receives screen data from all terminals in the QCIF frame mode, and sends the QCIF frame to the terminal that has requested one-screen display. May be transmitted. In this case, the MCU sends the synthesized screen data of the QCIF frame of four screens to the terminal instead of the one screen of the QCIF frame, and the receiving terminal that requests the one screen selects and displays the desired one screen. Has a function.
[0050]
Further, a configuration may be adopted in which the terminal requested to display one screen outputs the image in the mode based on the CIF frame, and the MCU synthesizes the screen. In this case, a configuration is adopted in which coded data corresponding to a black bar or the like is forcibly generated and inserted into the GOB portion of the screen of the mode in the CIF frame mode, or the data of the immediately preceding QCIF is stored. Is also good. With this configuration for storing the data of the immediately preceding QCIF, the resolution of the image displayed on one screen can be increased.
[0051]
In this configuration, the terminal does not need to convert the CIF frame to the QCIF frame, and the multiplexing process of the dummy data is performed by the MCU. If only a signal that can be distinguished as an area where dummy data is contained is multiplexed by the MCU, a desired composite image signal can be multiplexed in addition to the three screens decoded on the receiving side to display a four-screen composite screen. It is possible.
[0052]
Although the audio signal is not shown or described, the audio signal is processed by the audio signal processing unit by a conventional method. When the image data separation circuit has a sound data separation function, the separated sound data is sent to the sound signal processing unit. The audio signal processing unit adds and synthesizes audio signals obtained from audio data from each point. If the added and synthesized audio signal is distributed to each terminal as it is, each terminal returns the transmitted audio signal as an echo, so that the generated sound is annoying. To prevent this, the audio signal processing section obtains a synthesized audio signal by subtracting each of the audio signals transmitted from the corresponding terminals from the added and synthesized audio signal, and converts the synthesized audio signal into synthesized audio data. And sends it to the corresponding terminal.
[0053]
Each synthesized voice data is multiplexed with synthesized screen data and sent to each terminal as encoded data. Therefore, the function of multiplexing the synthesized voice data and the synthesized screen data can be included in the synthesis multiplexing circuit.
[0054]
Although omitted in the embodiment, in the case of data subjected to error correction coding or the like, since the structure of the GOB is not visible, the buffer memory creates an error correction decoded data sequence, and performs GOF data of CIF by screen multiplexing. It is necessary to perform error correction coding after multiplexing into columns. If the audio data is also multiplexed, the audio data is separated and the necessary processing is performed.
[0055]
As described above, in the buffer memories 12a to 12d of the embodiment of FIG. 1, different memory configurations are shown for each terminal. However, if the writing and reading speeds are increased, one memory device can be used. A memory area corresponding to each terminal can be separately determined to have a multiplexed memory configuration.
[0056]
In addition, since detection and synthesis in GOB units are performed and output, any one of the partial screens need only be aligned with one frame. Further, if even one GOB is synthesized and transmitted, all GOBs are output until one frame is aligned. Does not need to be stored, the delay time is in the unit of GOB, and can be further reduced to the delay time of the best frame rate of the encoder of the terminal. Also, the buffer memory capacity can be reduced to the extent that the delay time can be reduced, and the hardware scale can be reduced.
[0057]
In the above description, only the case of combining in GOB units has been described, but the present invention can be applied to the case of combining in data units instead of GOB.
[0058]
【The invention's effect】
As described above, according to the present invention, screen composition is performed in units of GOB, and data having a difference “0” is forcibly inserted in units of GOB where there is no screen data. With this configuration, the delay time due to the screen synthesis by the MCU can be reduced and the storage capacity of the buffer memory can be reduced, as compared with the conventional configuration in which the screen synthesis is performed after the screens are aligned. A screen composition system can be obtained.
[Brief description of the drawings]
FIG. 1 is a functional block diagram showing one embodiment of the present invention.
FIG. 2 is an explanatory view of a combined screen showing an embodiment combined with FIG. 1;
FIG. 3 is an explanatory view of a combined screen showing another embodiment combined with FIG. 1;
FIG. 4 is a functional block diagram showing one embodiment of the decoder of FIG. 1;
FIG. 5 is a functional block diagram showing another embodiment of the present invention.
FIG. 6 is an explanatory view of a composite screen showing another embodiment different from FIGS. 2 and 3;
FIG. 7 is a functional block diagram showing an example of the related art.
FIG. 8 is an explanatory diagram of a composite screen showing an example of the composition performed by FIG. 7;
[Explanation of symbols]
AD terminal
11a to 11d encoder
12a-12d, 741-744 Buffer memory
13,76 GOB detection circuit
14,77 control circuit
15, 75 synthesis multiplexing circuit
16 Multipoint control unit (MCU)
17 Decoder
61 Separation circuit
62 Variable Length Decoder
63 controller
64 inverse quantizer
65 IDCT (Inverse Discrete Cosine Transformer)
66 adder
67 Motion Adaptive Predictor
68 frame memory
69 D / A converter
72 Line control circuit
78 Control unit
79 Multipoint conference controller
711-714 Conference terminal
731 to 734 Image data separation circuit

Claims (10)

In a multipoint conference screen synthesis system in which each TV screen received from a plurality of video conference terminals is screen-synthesized with coded data obtained by data compression, and the synthesized coded data is received by each terminal and displayed on a screen,
The GOB in each terminal and a buffer memory means for storing received image signals generated for each (Group Of Blocks, the smallest unit of the encoded data of the image signal), the received image signal to each of the GOB,
GOB detection means for judging and outputting that at least one terminal has GOBs constituting a television screen of the terminal as screen data for one screen at a predetermined location in the buffer memory ,
None encoded data portion of the GOB which a sequentially reads the GOB from said buffer memory means, the encoded data of the GOB to be read is not stored if it is not accumulated required to perform screen synthesized by the judgment output A multiplexing means for inserting coded data GOB0 indicating that the coded data GOB0 is inserted, and multiplexing the coded data of the synthesized screen, and a bit rate of the coded data of the synthesized screen even when the coded data GOB0 is inserted. A multi-point control device having bit rate adjustment means for keeping the sum of the output bit rates of the encoders of each terminal smaller than the transmission line bit rate so as not to exceed the transmission line bit rate, and The terminal compensates for an increase due to the encoded data GOB0 received when the encoded data including the encoded data GOB0 is received. Multipoint conference screen synthesis system, characterized in that it comprises a decoding means for performing decoding processing time. The multipoint control device according to claim 1, wherein the multipoint control device is connected to the terminal via a digital line, transmits / receives a digital signal to / from each terminal, and controls connection of a conference participation line , and the line control unit. Image data separating means for extracting image data which becomes an image signal from the digital signal received from the terminal and transmitting the image data which becomes the image signal for each GOB , wherein the buffer memory means encodes for each GOB. A screen synthesizing system for a multipoint conference, wherein data is received from the image data separating means and stored. 2. A separating circuit according to claim 1, wherein said decoding means receives the encoded data, separates and outputs screen data which becomes an image signal from the received encoded data, and a quantized data of a transform coefficient from the received screen data. And a variable-length decoder for transmitting a signal such as a GOB number and a motion vector, and a signal for the GOB number and the motion vector received from the variable-length decoder. A controller that performs control, an inverse quantizer that inversely quantizes the quantized data received from the variable-length decoder, expands the signal to the original quantization level, and sends out the signal. IDCT (inverse discrete cosine transform) unit for obtaining and sending a quantized prediction error signal by performing an inverse discrete cosine transform on the resulting signal, and a prediction signal generated separately from the prediction error signal from the IDCT unit, Decryption And a motion adaptive predictor that adaptively obtains a prediction signal according to the motion vector received from the controller, and predictively outputs the next prediction signal, and a decoding device that receives the signal from the adder. A frame memory for delaying the signal by a delay circuit having a period of substantially one frame, and transmitting the delayed signal to the motion adaptive predictor; And a (digital / analog) converter. 4. The multipoint conference screen synthesizing system according to claim 3, wherein the frame memory is provided with a dual memory that can be separately read out only in a range at the head of a GOB. 4. The method according to claim 3, wherein the frame memory reads out the block of the received encoded data GOB0 by reading the frame at twice as fast as the frame period, and reads out only the range at the head of the next GOB separately. Screen synthesis system for multipoint meetings. 4. The frame memory according to claim 3, wherein the frame memory compensates for an increase due to the received encoded data GOB0 by increasing the decoding frame cycle and decoding the GOB0 block as a differential signal "0" as it is. Screen synthesis system for multi-point meetings. 2. The composite multiplex according to claim 1, wherein each terminal transmits encoded data in which the last one of the three GOBs of the QCIF (Quarterly Common Intermediate Format) system is formed as the GOB0, and receives the encoded data. A multi-point conference screen synthesizing system, wherein the conversion circuit forms one screen by the CIF method for six terminals. In a multipoint conference screen synthesizing method of synthesizing each TV screen of a plurality of video conference terminals with coded data obtained by compressing data and sending out the synthesized coded data to each terminal ,
A buffer memory that receives an image signal generated for each GOB (Group Of Blocks, minimum unit of coded data of the image signal) in each terminal, and stores the received image signal for each GOB; It is determined that GOBs constituting the television screen of the terminal are aligned in a predetermined location of the buffer memory as screen data for one television screen, and the GOBs are output in the order necessary for performing the screen synthesis by this determination output. The GOB is read from the buffer memory, and if the encoded data of the GOB to be read is not accumulated, the encoded data GOB0 indicating that there is no encoded data is inserted into the portion of the GOB which is not accumulated, and By synthesizing and multiplexing the coded data and transmitting it, the coded data of one screen Kept as small fall within transmission bit rate, in each terminal, when receiving the encoded data the encoded data GOB0 is included, the decoding processing time that compensates the increase in the encoded data GOB0 received A method for synthesizing a screen for a multipoint conference, characterized in that: 9. The multipoint conference screen synthesizing method according to claim 8, wherein the increment of the encoded data GOB0 is compensated by faster decoding processing. 9. The multipoint conference screen synthesizing method according to claim 8, wherein an increase in the encoded data GOB0 is compensated by skipping the encoded data GOB0.

Images