JP2001045495A - Image compositing device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image composite device for a multiple place video conference system that obtains a composite image with a simple configuration in order to decode an encoded signal transmitted from many places to prepare a composite image and to transmit to each terminal. SOLUTION: An encoding system such as H. 261 can easily construct the composite quasi-moving image between small-scale devices by fetching only intra-frame encoded data, smoothing intra-frame encoded data from each video conference terminal 6 in a buffer memory 3 to be subjected to time division multiplexing and decoding an image compositing with the intra-frame encoded data as one signal string because the encoding system has a mode in which encoding is performed by combining intra-frame encoding and intra-frame prediction coding and intra-frame encoding is transmitted in the ratio of once in tens of frames.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a screen synthesizing apparatus, in particular, a multipoint control unit (Multipoint Control Unit).
The present invention relates to a screen synthesizing apparatus suitable for obtaining a multipoint synthesized screen in an MCU.

[0002]

2. Description of the Related Art In the information age, time is an important factor, and a multipoint video conference system in which a multipoint is connected via a television line to conduct a conference is rapidly spreading. With the introduction of this system, it is possible to significantly reduce or reduce time and expenses.

[0003] When a multi-point video conference is performed using an image coding transmission apparatus terminal, each terminal is connected to a multi-point control unit (MCU).
However, if the screens of all the participating points can be viewed on the terminal, it is convenient to grasp the whole situation, and a smooth video conference can be realized. In order to view all points, it is necessary for the MCU to decode the encoded transmission signal from each terminal, create a composite screen from the decoded image signal, encode this composite screen, and encode and transmit it to each terminal. is there. In order to perform the decoding process and the synthesizing process, a decoding device and a screen reduction circuit for reducing the screen are required in accordance with the number of synthesized screens, so that the device scale tends to be large.

For this reason, it is difficult for the screen combining function to balance the disadvantage of viewing the image signal with the combined screen with the disadvantage of increasing the cost due to an increase in the scale of the apparatus. When conducting a video conference with multipoint participation, the speaker (sender) / chairperson /
Selecting each image, such as a desired point, is used substantially more. In the multi-point composite screen, the screen at each point becomes smaller as the number of points increases. Although the composite screen is effective for knowing the general outline, it is too small to view a specific screen in detail, and it is necessary to select and display one required screen.

[0005] As an example, in order to create a 16-point composite screen using an MCU, 16 decoding devices and 16 screen reduction circuits are required to decode signals from each terminal. Very large. In order to use the composite screen as a display function of the general outline, these 16-point images may be quasi-still images, but for this purpose, it is necessary to add an additional encoding function for transmitting different image information. Yes, ITU-T H3
It cannot be realized by a system configured based on standard specifications such as 20.

Therefore, several techniques have been proposed as screen composition techniques for MCUs. For example, JP-A-10-2
Japanese Patent Application Publication No. 00872 discloses a multipoint video conference control device (MCU) that suppresses deterioration of the image quality of a composite screen in “Multipoint Video Conference Control Device and Screen Composition Coding Method”. As technical means therefor, decoding information from each decoding device is compared, the inconspicuousness of image deterioration is divided into relative stages, and this is supplied to a coding unit as a combined coding control signal. . The encoding unit changes the quantization characteristic according to the combined encoding control signal when encoding is performed according to the occupation amount of the buffer memory.

FIG. 8 shows a block diagram of this prior art.
It comprises a plurality of video conference terminals 63-1 to 63-n, a network 62 and an MCU 69. The MCU 69 includes a data exchange unit 64, a plurality of decoding units 65-1 to 65-n, an image combining unit 66, a combined encoding control unit 68, and an encoding unit 67. Network 6 from each video conference terminal 63
The signal input to the data exchange unit 64 of the MCU 69 through the second unit 2 is sent to the selected decoding unit 65, and is decoded to output an image signal. In the image synthesizing unit 66, screen synthesis is performed on the image selected by the supplied selection signal. The combined image signal is supplied to the coding unit 67, and the quantization screen is adaptively changed in accordance with the control signal from the combining and coding control unit 68 to code and output the combined screen. This output transmits a composite screen to the video conference terminal 63 via the data exchange unit 64 and the network 62.

Japanese Patent Laid-Open Publication No. Hei 10-313453 discloses a "method and system for connecting a multipoint video conference between different speeds" in a multipoint videoconference using multi-screen synthesis. Relatively inexpensive with a simple configuration,
A technology that eliminates the need for setting a transfer rate of each terminal is disclosed. For this purpose, in the MCU, video from each terminal having a different transfer rate is decoded by a decoder, decimated and synthesized, and then encoded at a plurality of encoders at each transfer rate and transmitted to each terminal. The transfer rate of the received video is automatically detected and encoded and decoded. In the screen synthesis, an image decoded by a decoder that decodes a signal from each terminal is synthesized by an image synthesizer.

Further, the "video signal multiplex decoding apparatus" disclosed in Japanese Patent Application Laid-Open No. Hei 4-317285 combines video coded data sent from multiple points into one screen using a screen synthesis circuit, and then changes the data. It discloses that a single variable-length decoding circuit reproduces video signals from multiple points by performing long decoding. FIG. 9 shows a block diagram of the configuration. That is, this device is a digital / analog converter (D / A) 9
7, a frame memory (FM) 95, an adder circuit 94, a variable length decoding circuit (VLDEC) 93, a screen synthesizing circuit 98, a plurality of receiving buffer memories (BM) 92a to 92d, and a plurality of receiving circuits 91a to 91d. You.

[0010] A plurality of encoded data receiving circuits 91a to 91d
The video coded data received at BM92a ~
92d is temporarily stored. The screen synthesizing circuit 98
The coded video data supplied from a plurality of points is read out from 92a to 92d with the same frame phase, and a plurality of screens are synthesized and converted into coded video data for one screen. At this time, if the total number of blocks sent from a plurality of points is larger than the number of blocks on the output screen, the extra block data is deleted. On the other hand, if the number is small, dummy block data is inserted, and the number of blocks in the composite screen matches the number of blocks in the output screen. The synthesized screen coded data from the screen synthesis circuit 98 is converted into difference information by a variable length decoding circuit 93, added to the reproduced video data of the previous frame by an addition circuit 94, and the video of the current frame is reproduced.
/ A 97 to be converted into an analog signal. The reproduced video data is stored in the frame memory 95 in order to reproduce the image of the next frame.

Furthermore, Japanese Patent Application Laid-Open No. 9-149332 discloses a "multi-screen creation device" which discloses a multi-screen creation device that is small in size and inexpensive. FIG. 10 is a block diagram showing the configuration. That is, the input terminal 121 to which the multiplex digital signal is input, the separation circuit 122,
Picture detection circuit 123, decode circuit 124, multi-screen creation circuit 125, control circuit 126, and display screen 12
7. Here, the separation circuit 122 multiplexes a plurality of compressed digital video signals and sequentially separates N digital video signals from a multiplexed digital signal transmitted. The detection circuit 123 detects only an intra-frame coded portion from the N digital video signals separated by the separation circuit 122. The decoding circuit 124 sequentially decodes the coded portions in the N frames detected by the detection circuit 123. This decoding circuit 124
The control circuit 126 controls the N video signals decoded by the control circuit 126 to compress and display the N video signals into the respective areas A to D on the display screen 127 where the display area is divided into N.

[0012]

The above-mentioned prior arts 1 to 4 have several problems. First, in Conventional Examples 1 and 2, in order to see the state of all points from each terminal of the multipoint video conference system, the MCU decodes the decoded data of the image at each point, synthesizes the screens, And transmits the encoded signal to each point. Therefore,
In order to decode an image at each point by the MCU, decoding apparatuses for the number of points are required, and the apparatus becomes large-scale and expensive.

In the above-mentioned prior art 3, the signal from the reception buffer memory 92 is decoded by the signal synthesized by the screen synthesizing circuit 98, so that the variable length decoding circuit 9 is decoded.
3. The addition circuit 94 and the frame memory 95 can be constituted by one processing circuit. However, the screen combining method reads video coded data from the n point and combines n screens to obtain 1
It is converted into video encoded data for the screen. At this time, it is necessary to reduce the size of the screen on the transmitting side in advance so that the total number of blocks sent from the n point matches the number of blocks on the output screen. For this purpose, for example, each of the four screens sent in the normal size is reduced to half in length and width, and
There is a problem that the screens cannot be combined such that the screens are combined into one screen, and the screens are cut off to half the size in the vertical and horizontal directions, so that the screen is chipped. Also, the method of adjusting the screen by increasing or decreasing the block data is effective only when the encoding method is the inter-frame method, and the prediction method such as H.261 which is a general standard method of the video conference terminal is used in the prediction method. Since the motion compensation prediction method is used, if the block is deleted, decoding cannot be performed correctly.

Furthermore, in order to apply the above-mentioned prior art 4 to an MCU control device, a compressed digital signal transmitted from each terminal to the MCU is once multiplexed and applied as a multiplexed digital signal. . However, in this conventional technique, it takes a long time to refresh the N screens once because the N digital video signals are decoded in order from the multiplexed digital signal. I can't see it. For example, in a transmission terminal of a video conference apparatus, at least one frame is required to perform intra-frame encoding as a refresh for error recovery in order to increase transmission efficiency.
周期 2 second period. Then, when a conference is held at N = 16 multipoints, if one frame is detected at an average of 0.5 to 1 second, the average period at which a synthesized screen is obtained is 8 to 16 seconds. Since it takes time, the composite screen is close to a still image, and is unsuitable as an MCU composite screen.

An object of the present invention is to decode a coded signal sent from multiple points to form a composite screen and transmit it to each terminal. It is an object of the present invention to provide a screen synthesizing device for a point teleconference system.

[0016]

In order to solve the above-mentioned problems, a screen synthesizing apparatus according to the present invention employs the following characteristic configuration.

(1) Having a plurality of video conference terminals, each of the video conference terminals receiving a signal obtained by coding an image signal by an encoding method having both intra-frame coding and inter-frame coding modes In a screen synthesizing apparatus for creating a synthesized screen, a data separation unit for separating only intra-frame encoded data from each of the video conference terminals, a buffer memory for smoothing the data separated by the separation unit, and a buffer memory A screen synthesizing apparatus, comprising: an intra-frame decoding unit for intra-frame decoding of the output of the above; and a screen synthesizing unit for creating a synthesized screen from the intra-frame decoding unit.

(2) The screen synthesizing apparatus according to the above (1), further comprising a control unit for performing intra-frame encoding at a desired cycle in the video conference terminal.

(3) A decoding unit for performing time-division multiplexing on the intra-frame coded data from the video conference terminal and performing intra-frame decoding by a single intra-frame decoding unit; The screen synthesizing apparatus according to the above (1), further comprising a single screen reducing unit for performing band limitation and resampling.

(4) A data separation unit comprising a single data separation circuit for separating intra-frame coded data from the coded data from each of the video conference terminals by time-division multiplexing processing, and the buffer memory is divided into address spaces. (1) having a buffer memory composed of one memory
Screen synthesis device.

(5) The picture synthesizing apparatus according to the above (1), further comprising a control section for assigning a priority to the order in which the intraframe encoded data of one frame is stored in the buffer memory and performing decoding.

(6) An identification signal generating section for multiplexing the identification signal encoded data with the intra-frame encoded data, and detecting each of the identification signals from a decoded image signal obtained by decoding the multiplexed signal, for each frame. The screen synthesizing apparatus according to the above (5), further comprising a screen synthesizing unit for writing the reproduced image in the corresponding screen synthesizing position.

(7) A video conference terminal having means for adding an identification signal to a part of an image signal on the transmitting side and performing intra-frame encoding, and a decoded image signal obtained by decoding intra-frame encoded data on the receiving side. The screen synthesizing apparatus according to the above (5), further comprising: a screen synthesizing unit that detects an identification signal and writes a reproduced image of each frame at a synthesizing position of the corresponding screen.

[0024]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS A preferred embodiment of a screen synthesizing apparatus according to the present invention will be described below in detail with reference to the configuration and operation thereof.

First, FIG. 1 shows a first embodiment of the screen synthesizing apparatus according to the present invention.
It is a block diagram of an example of an embodiment. This screen composition device
Plurality of video conference terminals 6a to 6d, network 7 and MC
It is composed of U8. The MCU 8 includes a data exchange unit 1,
A plurality of data separation units 2a to 2d, a plurality of buffer memories 3
a to 3d, an encoding unit 5, an intra-frame decoding unit 32, and an image synthesizing unit 33.

A plurality of (four in the example of FIG. 1) video conference terminals 6 a to 6 d are connected to the MCU 8 via the network 7. Each video conference terminal 6 is, for example, ITU-T H.3.
20. For example, an encoding system and a decoding system based on ITU-T H.261 are adopted. Each video conference terminal 6 encodes (compresses and encodes) the image signal.
Then, the data is transmitted to the network as terminal data. H.26
The coding 1 has coding modes of intra-frame coding (I) and inter-frame prediction coding (P). Normally, this is performed by inter-frame predictive coding. However, intra-frame coding is required to initialize decoding when terminal switching is selected and decoding start when a transmission path error occurs. Intra-frame encoding is performed every one second) and encoded data is transmitted.

The multipoint video conference controller (MCU) 8
The terminal image data from each video conference terminal 6 transmitted to the network 7 is received by the data exchange unit 1 and supplied to the data separation unit 2 corresponding to each video conference terminal 6.
When the screen synthesis is not performed, the terminal image data is not decoded, but is exchanged as it is and sent out to the network 7 as transmission data to the desired video conference terminal 6.

The data separation section 2 has a function of detecting the head of switching of the encoding mode from the terminal image data sequence,
The intra-frame coded data and the inter-frame prediction coded data are separated from the terminal image data, and only the intra-frame coded data is supplied to the buffer memory 3. The buffer memory 3 corresponding to each terminal supplies the accumulated intra-frame encoded data to the intra-frame decoding unit 32 in a predetermined order. The intra-frame coded compressed data detected by the data separation unit 2 is supplied to a buffer memory 3, and each buffer memory 3 has a capacity to store compressed data obtained by intra-frame coding of one frame screen. In the n (for example, n = 16) buffer memories 3, the intra-frame coded data for each frame of the image of the corresponding terminal is sequentially written at a cycle of about 1 second. Reading of the buffer memory 3 depends on the processing control request of the intra-frame decoding unit 32.
sequentially selected and read from the n buffer memories 3;
It is supplied to the intra-frame decoding unit 32. If the intra-frame coded compressed data is not stored in the buffer memory 3 after being read by a certain buffer memory 3 until the next reading, the data is skipped and output without a signal.

The intra-frame decoding unit 32 has a function of decoding only the mode of the intra-frame encoding system (I) among the encoding systems of the H.261 video conference terminal 6,
The intra-frame coded compressed data supplied from the buffer memory 3 is decoded to obtain a decoded image signal.
3.

The standard NTSC television signal (image signal) is 1
It is composed of 30 frames of signal per second. The signal processing speed of a normal decoding device is usually 30 frames / second or more, and the processing speed of the intra-frame encoding unit is equal to or more than that. For this reason, when the encoding unit of the video conference terminal 6 is set to perform the intra-frame encoding once per second, if the intra-frame encoded data from 30 points is multiplexed, it takes 1 second. In this case, a signal corresponding to 30 frames is obtained, but with this degree, the intra-frame decoding unit 32 can sufficiently process the signal.

The image synthesizing unit 33 selects the desired number of screens, reduces the size of the image signal captured by the intra-frame encoding sent from each video conference terminal 6 to a desired size, and The images are arranged in order and temporarily stored in a synthesized frame memory (not shown) so as to form a synthesized screen, and then supplied to the encoding unit 5. The encoding unit 5 has the same encoding function as the encoding unit of the video conference terminal 6, and adaptively switches the encoding mode between intra-frame encoding and inter-frame predictive encoding, and The composite image signal supplied from 33 is encoded, and the encoded data is supplied to the data exchange unit 1.

The data exchange unit 1 selects coded data corresponding to an image desired by each video conference terminal 6 based on a control signal sent from each video conference terminal 6 and sends the coded data to the network 7. When a composite screen is requested for each of the video conference terminals 6, encoded data of a screen composite signal is selected and transmitted. When an image of one screen of a certain video conference terminal 6 is requested, the coded data from the corresponding video conference terminal 6 is selected and transmitted.

The video conference terminal 6 receives the encoded data via the network 7 and decodes it to obtain a decoded signal. When the video conference terminal 6 wants to know the whole situation, it sends a selection signal to select the reception of the composite screen to the MCU 8. Each screen in the composite screen is a quasi-moving image that changes by one frame per second, but it is enough to know the entire movement. After the overall appearance is determined, a screen for speaker selection is selected. In the speaker selection, the data exchange unit 1 of the MCU 8 estimates, based on the voice signal, a terminal which is considered to be speaking among the video conference terminals 6 participating in the video conference, and this estimated one The screen of the video conference terminal 6 is automatically selected, and the encoded data is transmitted to the video conference terminal 6.

FIG. 3 shows an intra-frame decoding unit 32 shown in FIG.
It is a specific block diagram of. In-frame decoding unit 32
Comprises a variable-length decoding unit 41 and an intra-frame decoder 42, and performs H.261 intra-frame decoding. It has a processing speed of decoding an image of 30 frames per second, which is the same as the number of frames of a normal television signal. The multiplexed signal of the time-division multiplexed intra-frame encoded data supplied from the buffer memory 3 in FIG. 1 is sequentially decoded, and a decoded image for each frame is output. The decoded image is 1
The screen of each video conference terminal 6 is decoded and output in the order of time-division multiplexing for each frame.

Next, FIG. 4 shows a specific block diagram of the image synthesizing section 33 in FIG. The image synthesizing unit 33 includes a band limiting filter 51, a resampling circuit (resampler) 5
2 and a composite frame memory 53. The band limiting filter (decimating filter) 51 performs band limiting in advance so as to have a frequency of 1/4 in the vertical and horizontal directions so that aliasing distortion does not occur when the number of samples of the image signal is reduced by resampling. The re-sampling circuit 52 vertically and horizontally converts the band-limited signal to a desired reduction ratio by one each.
The output is thinned out to / 4. A screen reduction circuit 55 composed of a band limiting filter 51 and a resampling circuit 52 generates an image signal thinned to a desired size. The composite frame memory 53 writes the thinned image signal so that the screen of each video conference terminal 6 is arranged at a desired position on the composite screen by controlling the write address. Therefore, the screen of each video conference terminal 6 in the composite screen is sequentially updated for each frame in the composite screen. When n = 16, the screens from all the video conference terminals 6 are updated after one second.
The combined screen is read from the combined frame memory 53 at a rate of 30 frames per second.

The signal of the composite screen read from the image compositing unit 33 is supplied to the encoding unit 5, and the encoded data encoded by the H.261 encoding method is supplied to the data exchange unit 1. . The data exchange unit 1 selects encoded data corresponding to an image desired by each video conference terminal 6 based on a control signal sent from each video conference terminal 6 and sends the encoded data to the network 7.

Next, FIG. 5 shows how the coded data in the frame is multiplexed. Of the encoded data signals supplied to each data separation unit 2, the signals supplied to the first four data separation units 2 are indicated by signals a to d in FIG. "I"
Indicates intra-frame encoded data, and “P” indicates inter-frame encoded data. “O” indicates that there is no data. The intra-frame coded data is separated by each data separation unit 2, and each buffer memory 3 stores the intra-frame coded data "I". The multiplexed signal y obtains a multiplexed multiplexed signal by sequentially selecting intra-frame encoded data “I” in the order of the signals a, b, c, and d. In FIG. 5, m + 2
At the timing of the second frame cycle, since the data of the next intra-frame encoding is not stored in the buffer memory 3, for example, a signal of "O" is multiplexed as a signal indicating no data. Intraframe encoded data Ib of signal b
* Is multiplexed after waiting for the next cycle. Multiplexed signal y
Are supplied to the intra-frame decoding unit 32 to perform intra-frame decoding.

The amount of actual coded data varies from frame to frame due to differences between the TV conference terminals 6 and temporal variations. In addition, the amount of information is generally larger in the case of intra-frame encoding "I" than in the case of inter-frame prediction encoding "P". Therefore, in the schematic display shown in FIG. 5, all the data blocks have the same length for simplicity, but actually differ from each other.

Next, a second embodiment of the screen synthesizing apparatus of the present invention will be described with reference to the block diagram of FIG. Note that the same reference numerals are used for the same components as those of the screen composition device of FIG. This MCU 34 includes a data exchange unit 1, a plurality of data separation units 2a to 2d, an intra-frame decoding unit 32, an image synthesizing unit 33, and an encoding unit 5 as shown in FIG.
MCU8. However, it is characterized in that the outputs of the plurality of data separation units 2 are collectively supplied to a single buffer memory 31. The buffer memory 31 processes data of a plurality of channels in parallel. If the data rate is about 1.5 Mb / s, the buffer memory for 16 channels can be processed in parallel by time division with one high-speed memory.

Next, as a method of multiplexing the intra-frame coded compressed data signal, as described above, in the method of sequentially multiplexing for each of the video conference terminals 6 and decoding by the intra-frame decoding unit 32, the shift of the acquisition timing Will delay decoding by up to about 1 second. FIG. 6 shows a block diagram of a third embodiment for improving this. The MCU 85 includes a data exchange unit 1 and a data separation unit 8 having a plurality of data separation units 2.
01, a buffer memory 802 having a plurality of buffer memories 81, a control unit 82, an intra-frame decoding unit 83, an image synthesizing unit 84, and an encoding unit 5.

The data exchange section 1, data separation section 2 and encoding section 5 operate in the same manner as the corresponding elements in FIG. When one frame of the intra-frame coded data separated from the data separation unit 2 is accumulated, the buffer memory 81 controls the control unit 8.
2 is supplied with a number for distinguishing the buffer memory 81 and a status signal indicating that the buffer memory 81 has been stored.

The control unit 82 assigns priorities to the order in which one frame data is accumulated from the status signals supplied from the respective buffer memories 81, and sequentially reads out from the buffer memory 81 by time division multiplexing for each frame. To be supplied to the intra-frame decoding unit 83. If none of the buffer memories 81 has accumulated the intra-frame encoded data for one frame, data "0" is output as no data. Since the decoding is performed immediately after the data of one frame is accumulated in the buffer memory 81, the delay time until the image signal is reproduced can be reduced.

The intra-frame decoding section 83 sequentially decodes the multiplexed signal for each frame, and supplies a decoded image signal to the image synthesizing section 84. If there is no data,
No decryption is performed. In the image synthesizing unit 84, after band-limiting the image signal, the image signal is resampled and reduced to a desired size. The control unit 82 knows which buffer memory 81 number the reduced screen image is (ie, which video conference terminal image), and therefore specifies the memory address of the combined screen position corresponding to the number, and reduces the reduced image signal. Write data to the composite frame memory.

In this embodiment, since the decoding is performed preferentially from the point where the intra-frame coded compressed data has been accumulated for one frame, the delay time in the buffer memory 81 can be reduced, and Signal transmission delay can be reduced.

When the data rate is 384 kb / s, the data rate for 16 video conference terminals is 384 × 16 = 6.1.
44 Mb / s. At such a speed, the data separation unit 801 and the buffer memory 802 operate satisfactorily. Therefore, each of the data separation unit 801 and the buffer memory 802 can be stored in one unit without providing each processing unit in parallel.
When they are put together, the size of the apparatus can be further reduced by performing parallel processing in a time-division manner. In this case, the data from each video conference terminal is separated by time division multiplexing, and the buffer memory is composed of a read / write independent memory, and each buffer memory area is divided and used in the address space. In addition, the cycle sent in intra-frame encoding does not need to be limited to about 1 second, and can be set freely. When the period is short or when the number of combined screens is large, the number of frames of multiplexed intra-frame data per second may exceed 30 frames. In this case, if the processing speed of the intra-frame decoding unit is sufficient, the decoding is performed as it is. If the processing speed is not enough, decoding is performed by thinning out data of an appropriate frame.

The intra-frame decoding unit 83 is a general-purpose H.261
If the input interface is configured using the decoder of
In the case of a transmission line interface having a frame configuration of H221 and a buffer memory for smoothing data at the input side, if the input data is multiplexed with intra-frame encoded data of each terminal for each frame, the decoder It is difficult to determine which video conference terminal is the reproduced image signal output after decoding. Therefore, in the reproduced image signal, an identification signal for identifying which video conference terminal is the image signal is included in the image signal, and the identification signal is detected from the reproduced image signal output from the decoder,
It is conceivable to determine which video conference terminal is the image signal. As a method for inserting the identification signal, it is conceivable that the video conference terminal on the transmitting side inserts the identification signal into the image signal, or inserts the data of the identification screen during the time for multiplexing the intra-frame encoded data.

A fourth embodiment of the screen synthesizing apparatus according to the present invention to which such a function is added will be described with reference to the block diagram of FIG. The MCU 90 of this embodiment includes a data exchange unit 1, a data separation unit 8 including a plurality of data separation units 2a to 2d.
01, a buffer memory 802 including a plurality of buffer memories 81a to 81d, a control unit 82, a multiplexing unit 86, a decoding unit 88, an identification signal generation unit 87, an identification signal detection unit 89, an image synthesis unit 84, and an encoding unit 5. It is composed of

The data exchange unit 1 receives a signal from each video conference terminal and supplies data to the corresponding data separation unit 2. The data separation unit 2 separates only the intra-frame coded data, supplies it to the buffer memory 81, and stores it.
The identification signal generating section 87 generates, based on the control signal from the control section 82, compressed data in which an image signal of one frame capable of identifying the number of the designated buffer memory 81 is intra-frame coded, and outputs the compressed data to the multiplexing section 86. Supply. In the image signal of one frame for identification, a number identification signal is inserted at the head, and the rest are fixed values.

The multiplexing section 86 stores 1 in the buffer memory 81.
The frames are selected and taken in the order in which the intra-frame coded data are accumulated. Before this fetching, compressed data obtained by encoding the number of the corresponding buffer memory 81 is first fetched from the identification signal generating section 87, and the data are alternately multiplexed in the order of the identification number data of the buffer memory 81 and the data of the image signal of the corresponding number To be Also, the multiplexed data is reconstructed into a frame of desired coded data so that the multiplexed data can be directly decoded by the decoding unit 88 at the subsequent stage. Decoding section 88
Is composed of a general-purpose H.261 decoding LSI (or CPU), and decodes encoded data to reproduce and output an image signal.

The identification signal detecting section 89 detects an identification number from the reproduced image signal and supplies it to the image synthesizing section 84. The image synthesizing unit 84 limits the band of the image of the next one frame every time the identification signal is detected from the reproduced image signal, reduces the image to a desired size, and then outputs the identification signal supplied from the identification signal detection unit 89. Specify the memory address corresponding to the position on the composite screen of the number and write.

The encoding section 5 encodes the composite screen supplied from the image compositing section 84 and supplies it to the data exchange section 1.
In order to obtain a composite screen, an existing decoder can be used to decode a multi-point screen to form a composite image, thereby reducing the device scale. As described above, the data separator 2 can perform processing by time-division multiplexing and can be configured by one data separator 801. Further, the buffer memory 81 can be made into one buffer memory 802 by dividing the memory into a plurality of areas and using them.

The configuration and operation of the preferred embodiment of the screen synthesizing apparatus according to the present invention have been described above. However, the present invention should not be limited to only such specific examples, and it can be easily understood that various modifications can be made depending on the application.

[0053]

As will be understood from the above description, according to the picture synthesizing apparatus of the present invention, only intra-frame coded data is separated and smoothed by the buffer memory, and intra-frame coded compressed data is multiplexed and decoded. Thus, the decoding can be performed by one decoding unit without dropping the intra-frame encoded data, and the apparatus scale is small, and a synthesized screen signal of the quasi-moving image can be transmitted to each terminal. Also, by observing the accumulation state of the intra-frame compressed data in the buffer memory and performing preferential decoding processing on the accumulated frames, it is possible to perform decoding without dropping the screen in the frame. Has a remarkable effect.

[Brief description of the drawings]

FIG. 1 is a block diagram of a first embodiment of a screen synthesizing apparatus according to the present invention.

FIG. 2 is a block diagram of a screen synthesizing apparatus according to a second embodiment of the present invention;

FIG. 3 is a specific configuration diagram of an intra-frame decoding unit in FIGS. 1 and 2;

FIG. 4 is a specific configuration diagram of a screen composition unit in FIGS. 1 and 2;

FIG. 5 is a diagram showing a schematic example of multiplexing in the present invention.

FIG. 6 is a block diagram of a third embodiment of the screen synthesizing apparatus according to the present invention.

FIG. 7 is a block diagram of a fourth embodiment of a screen synthesizing apparatus according to the present invention.

FIG. 8 is a block diagram of an example of a conventional screen synthesizing apparatus.

FIG. 9 is a block diagram of another example of the conventional screen synthesizing apparatus.

FIG. 10 is a block diagram of still another example of the conventional screen synthesizing apparatus.

[Description of Signs] 1 Data exchange unit 2 Data separation unit 3, 31, 81 Buffer memory 5 Encoding unit 6 Video conference terminal 7 Network 8, 34, 85, 90 Multipoint control unit (MCU) 32, 83 In-frame decoding Multiplexing sections 33 and 84 image synthesis section 51 band limiting filter 52 resampling circuit 82 control section 86 multiplexing section 87 identification signal generation section 89 identification signal detection section

Claims (7)

[Claims] 1. A video conference terminal comprising a plurality of video conference terminals, wherein each of the video conference terminals receives and synthesizes a signal obtained by encoding an image signal by an encoding method having both intra-frame coding and inter-frame coding modes. In a screen synthesizing device for creating a screen, a data separation unit that separates only intra-frame encoded data from each of the video conference terminals, a buffer memory that smoothes data separated by the separation unit, An intra-frame decoding unit for intra-frame decoding the output;
A screen combining unit for creating a combined screen from the intra-frame decoding unit. 2. The screen synthesizing apparatus according to claim 1, further comprising a control unit for performing intra-frame encoding at a desired cycle in said video conference terminal. 3. A decoding section for performing time-division multiplexing of intra-frame coded data from the video conference terminal and performing intra-frame decoding by a single intra-frame decoding section, and providing a multiplexed reproduced image signal with a band. 2. The screen synthesizing apparatus according to claim 1, further comprising: a single screen reduction unit for limiting and resampling. 4. A data separation unit comprising a single data separation circuit for separating intra-frame coded data from coded data from each of the video conference terminals by time division multiplexing processing, and the buffer memory is divided into address spaces. 2. The screen synthesizing apparatus according to claim 1, further comprising: a buffer memory configured by one memory. 5. A control unit for performing decoding by assigning priorities to the order in which one frame of intra-frame encoded data is stored in the buffer memory.
Screen synthesizing apparatus according to 1. 6. An identification signal generator for multiplexing identification signal encoded data with the intra-frame encoded data, detecting the identification signal from a decoded image signal obtained by decoding the multiplexed signal, and 6. The screen synthesizing apparatus according to claim 5, further comprising a screen synthesizing unit that writes a reproduced image at a synthesizing position of the corresponding screen. 7. A video conference terminal having means for adding an identification signal to a part of an image signal on a transmission side and performing intra-frame encoding, and a method for performing the identification based on a decoded image signal obtained by decoding intra-frame encoded data on a receiving side. 6. The screen synthesizing device according to claim 5, further comprising: a screen synthesizing unit that detects a signal and writes a reproduced image of each frame at a synthesizing position of the corresponding screen.