JPH1066088A - Multi-point video conference control device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To almost accurately calculate moving vector information to be added to an encoded composite moving image to be transmitted to each video conference terminal equipment by a simple operation by calculating the moving vector information based on moving vector information obtained at the time of decoding each original moving image, a composite position and a reduction ratio for reduction compositing. SOLUTION: A system control part 22 reads out a moving vector value for four macro blocks(MBs) of a position corresponding to the position of an MB read out from a sound/moving picture multi-plexing part 27 by a moving picture encoding/decoding part 26 from a moving vector table. Moving vector information calculated based on moving vector information obtained at the time of decoding respective original moving images constituting a composite moving image, a composite position and a reduction ratio for reduction compositing is added to each encoded composite moving image. Thereby the moving vector information to be added to the encoded composite moving image can be almost accurately calculated without considering the height of correlation between the moving vector information of respective original moving images and the moving vector information of the composite moving image.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method for decoding coded moving images, each of which has been motion-compensated interframe predictive coded with motion vector information and received from video conference terminals installed at a plurality of points. A synthesized moving image is created by reducing and synthesizing the obtained moving images from each of the video conference terminal devices, and the synthesized moving image is subjected to motion-compensated inter-frame predictive coding again with motion vector information, thereby obtaining an encoded synthesized moving image. The present invention relates to a multipoint video conference control device having at least a function of generating an image and transmitting the generated image to each of the video conference terminal devices.

[0002]

2. Description of the Related Art The communication system in most current video conference systems is based on ITU-T Recommendation H.264. In accordance with ITU-T Recommendation H.320, which is a series of the encoding / decoding system for moving images. 261.

[0003] The ITU-T Recommendation H. 261 is an inter-frame predictive encoding method for encoding a difference between frames, and detects a motion of an object by comparing preceding and succeeding frames, and detects a motion of a previous frame. The motion-compensated inter-frame predictive coding method, which can reduce the prediction error by predicting the next frame after shifting the object by the amount of movement, is also defined as an option, but most video conferencing The terminal device or the multipoint video conference control device supports the inter-frame predictive coding method with motion compensation.

In the inter-frame predictive coding system with motion compensation, a large number of small areas (blocks) constituting a frame are used.
A motion vector is detected by calculation every time.

[0005] As a method of detecting a motion vector, a block matching method, a gradient method, and the like are known.
ITU-T Recommendation H. In H.261, a block matching method is adopted, and a block closest to the focused macro block in the current frame within a search range of ± 15 pixels for each macro block (MB: 16 × 16 pixels). From the previous frame, and the difference between the pixel positions is used as a motion vector for the macroblock of interest in the current frame.

[0006] The encoding process in the inter-frame predictive encoding system with motion compensation includes, in addition to the above-described motion vector detection process,
It is composed of a number of data processing such as DCT processing, inverse DCT processing, quantization processing, inverse quantization processing, variable length encoding processing, etc. Among them, the motion vector detection processing is performed for each frame generated about 30 times per second. Many (if the frame format is FCIF (352 pixels x 288 lines), 3
For each of the 96 (22 × 18) macroblocks, a search range of ± 15 pixels (31 × 31 = 961)
961 comparison operations are required, and the processing amount for that occupies more than half of the total encoding processing amount.

[0007] On the other hand, the multipoint video conference control device transmits a moving image received from each video conference terminal device to ITU-T Recommendation T.40. H.120, which is appropriately mixed according to various mixing schemes indicated in the draft of the Recommendation 120 and transmitted to each video conference terminal apparatus. Among them, the moving picture processing in the multipoint TV conference control apparatus in the case of the mixing scheme using a transcoder is described with reference to FIG. This will be described schematically with reference to FIG. In addition,
As a premise of FIG. 17, the multipoint video conference control device includes:
Video conference terminal device (hereinafter simply abbreviated as terminal) A to E
It is assumed that the five terminals are connected.

In FIG. 1, a multipoint TV conference control device decodes an encoded moving image (encoded moving image) received from terminals A to E, respectively.
Each moving image is obtained, and each of the moving images is reduced by half in the vertical and horizontal directions to create a reduced moving image, and the reduced moving images are combined for each terminal. That is, for the terminal A, the reduced moving images of the moving images received from the terminals B to E except for the moving image received from the terminal A are combined to create a combined moving image, and for the other terminals, Similarly, a composite moving image is created. Then, the combined moving image for each terminal is re-encoded to create an encoded combined moving image, and transmitted to each terminal.

[0009]

As described above, in the mixing method using the transcoder, the coded moving picture received from each terminal is decoded once, returned to the moving picture, and then arbitrarily reduced or synthesized. Since the processing can be performed, there is an advantage that various moving image processing can be performed in the multipoint video conference control device, while it is necessary to simultaneously re-encode each moving image for each terminal for each terminal, It is necessary to provide encoders for the number of communication channels connectable to the terminal.

[0010] The encoder for each communication channel must have the same function as that provided in one video conference terminal, and these encoders support inter-frame predictive coding with motion compensation. If so, the multipoint video conference control device needs to include encoders having motion vector detection functions for the number of communication channels. On the other hand, the motion vector detection function occupies more than half of the entire processing amount of the encoding processing as described above. Therefore, the multipoint video conference control device needs to include a high-performance arithmetic processing device in order to provide a motion vector detection function for the number of communication channels, and the device cost is increased accordingly.

SUMMARY OF THE INVENTION The present invention has been made in view of the above circumstances, and provides a multipoint video conference control apparatus capable of calculating motion vector information to be added to an encoded combined moving image to be transmitted to each video conference terminal device by a simple calculation. The purpose is to provide.

[0012]

According to a first aspect of the present invention, there is provided a multi-point video conference controller connected to video conference terminals installed at a plurality of locations and connected to the respective video conference terminals. Receiving means for receiving a coded moving image subjected to motion compensation inter-frame predictive coding together with motion vector information added to each small area constituting a frame from the device, and each of the received video conference terminal devices A decoding means for decoding the encoded moving image from each, a reduced synthesizing means for creating a synthesized moving image by reducing and synthesizing each decoded moving image, and the synthesized moving image, Re-encoding means for generating an encoded synthesized moving image by performing motion-compensated inter-frame prediction encoding with motion vector information added for each small region constituting a frame; And a transmitting means for transmitting the coded composite moving image to each of the video conference terminal devices, wherein the decoding means receives the coded original moving image of the current frame received from each of the video conference terminal devices. Current vector motion vector storage means for storing the motion vector information obtained when decoding is performed for each current frame, and for each small area constituting the current frame of the coded composite moving image in the re-encoding means. The motion vector information stored in the current frame motion vector storage means for the current frame of each of the moving images reduced and synthesized as the synthesized moving image, and the synthesized moving image of each of the moving images. A motion vector calculated based on a synthesis position on an image and a reduction ratio of the moving images in the reduction synthesis unit. Characterized by comprising a Le calculating means.

According to a second aspect of the present invention, there is provided a multi-point video conference control device which connects video conference terminal devices installed at a plurality of locations, and outputs a video signal from each of the video conference terminal devices for each small area constituting a frame. Receiving means for receiving an encoded moving image subjected to motion compensation inter-frame predictive encoding with added motion vector information, and decoding for respectively decoding the received encoded moving image from each video conference terminal device Means for reducing and combining the decoded original moving images to form a combined moving image; and a method for adding the combined moving image to each small area constituting a frame. Re-encoding means for generating an encoded combined moving image by performing motion-compensated inter-frame prediction encoding with vector information, and each of the video conference terminal devices A multi-point video conference control device having at least a transmission unit for transmitting, the motion vector information obtained when the decoding unit decodes the coded moving image of the current frame received from each of the video conference terminal devices. A current frame motion vector storage unit for storing each current frame, and a center of a search range of motion vector information to be added to each of the small regions constituting the current frame of the encoded combined moving image in the re-encoding unit. ,
The motion vector information stored in the current frame motion vector storage means for the current frame of each of the moving images reduced and synthesized as the synthesized moving image, the synthesized position of each of the moving images on the synthesized moving image, A search range center calculating means for calculating for each small area based on a reduction ratio of the original moving image in the reduction synthesizing means, wherein the re-encoding means comprises a current frame of the encoded synthetic moving image. Motion vector information to be added for each small area is searched for within a predetermined range centered on the search range center calculated for each small area by the search range center calculating means.

According to a third aspect of the present invention, there is provided the multipoint video conference control device according to the first aspect, wherein the decoding means receives one frame before the current frame from each video conference terminal device. Further comprising a previous frame motion vector storage means for storing the motion vector information obtained when decoding the coded moving image of the frame of the above, wherein the motion vector calculation means, the re-encoding means in the re-encoding means When a frame skip occurs at the time of encoding, the re-encoding means reduces and synthesizes the motion vector information to be added to each of the small regions constituting the current frame of the encoded combined moving image as the combined moving image. The motion vector for each small area stored in the current frame motion vector storage means for the current frame of And a small area indicated by a motion vector for each small area of the current frame stored in the current frame motion vector storage means, one frame from the current frame of each original moving image reduced and synthesized as the synthesized moving image. The motion vector information obtained by adding the motion vector stored in the previous frame motion vector storage means for the previous frame for each of the small regions constituting the current frame of each of the moving images, and the synthesized moving image of each of the moving images. The calculation is performed based on the above combination position and the reduction ratio of the moving image in the reduction combination means.

According to a fourth aspect of the present invention, there is provided the multipoint video conference control apparatus according to the second aspect, wherein the decoding means is one frame before the current frame received from each video conference terminal device by the decoding means. The image processing apparatus further includes a previous frame motion vector storage unit that stores motion vector information obtained when decoding the encoded moving image of the frame, wherein the search range center calculation unit calculates the composite moving image of the combined moving image in the re-encoding unit. If a frame skip occurs during encoding, the re-encoding means sets the center of the search range of the motion vector information to be added to each small area constituting the current frame of the encoded composite moving image, and Each current frame of the original moving image reduced and synthesized as a moving image is stored in the current frame motion vector storage unit. With respect to the motion vector for each area and the small area indicated by the motion vector for each small area of the current frame stored in the current frame motion vector storage means, the current size of each original moving image reduced and synthesized as the synthesized moving image is obtained. The motion vector information obtained by adding the motion vector stored in the previous frame motion vector storage means for each of the small regions forming the current frame of each of the moving images for the frame one frame before the frame, and the motion vector information of each of the moving images. The calculation is performed based on a combination position on the combined moving image and a reduction ratio of the original moving image in the reduction combining unit.

[0016]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, a multipoint video conference control apparatus according to an embodiment of the present invention will be described in detail with reference to the accompanying drawings.

FIG. 1 shows a configuration of a video conference system including a multipoint video conference control device according to the present embodiment.

In FIG. 1, reference numerals 1, 19, and 20 denote video conference terminals having the same configuration, which are related to the present invention.
An SDN line 18 connects to an ISDN network. Although not shown, the video conference terminal devices related to the present invention are not limited to the three devices 1, 19 and 20. Reference numeral 21 denotes a multipoint video conference control device according to the present invention, which is connected to an ISDN network via an ISDN line 28.

FIG. 2 shows a block configuration of the video conference terminal device 1 of the video conference terminal devices related to the present invention.

In FIG. 1, reference numeral 2 denotes a system control unit including a CPU, a memory, a timer, etc., 3 a magnetic disk device for storing various programs and data, and 4 an ISDN layer 1 of the ISDN. The ISDN interface unit 5 that performs signal processing and signal processing of the layer 2 of the D channel is provided in the ITU-T Recommendation H.5. 221 is a multimedia multiplexing / demultiplexing unit that multiplexes / demultiplexes data of a plurality of media, 6 is a microphone for audio input, 7 is an A / A after amplifying an input signal from the microphone 6. An audio input processing unit that performs D conversion, 8 is an audio encoding / decoding unit that performs encoding / decoding / echo cancellation of the audio signal, and 9 is an audio signal decoding unit that converts the audio signal decoded by the audio encoding / decoding unit 8 into a D / Audio output processing unit that amplifies after A conversion,
Reference numeral 10 denotes a speaker for outputting audio from the audio output processing unit 9, reference numeral 11 denotes a video camera for inputting video, and reference numeral 12 denotes NTSC decoding of a video signal from the video camera 11.
A video input processing unit 13 for performing signal processing such as A / D conversion is provided in accordance with ITU-T Recommendation H.264. Encoding of moving images conforming to H.261
A moving image encoding / decoding unit 14 for decoding is a video output for performing signal processing such as D / A conversion, NTSC encoding, and graphics synthesis on the video signal decoded by the moving image encoding / decoding device 13. A processing unit, 15 is a monitor for displaying the received moving image and graphics information, 16 is a user interface control unit for controlling a console, 1
7 is a console comprising operation keys and a display unit, 18 is I
SDN line.

FIG. 3 shows a block configuration of the multipoint video conference controller 21. In FIG. 1, reference numeral 22 denotes a system control unit that controls the entire system and includes a CPU, a memory, a timer, and the like.

Reference numeral 23 denotes an ISDN interface unit;
Is a multimedia multiplexing / demultiplexing unit, 25 is a voice coding / decoding unit for coding / decoding a voice signal, and 26 is an ITU-
Recommendation T. H. This is a moving image encoding / decoding unit that encodes / decodes a moving image in compliance with H.261, and a communication channel 1 is configured by 23 to 26 components.

The above configuration is the configuration of the communication channel 1, but as shown in the figure, the multipoint video conference controller 21
Comprises 1 to n communication channels, communication channel 1
Although not shown, the other communication channels have the same configuration as the communication channel 1 and are connected to ISDN lines.

Reference numeral 27 denotes an audio / video multiplex unit for synthesizing audio and video data decoded by each communication channel between channels and delivering the data to each channel; and 28, an ISDN line connected to the communication channel. It is. In the figure, the connection from the audio coding / decoding unit 25 to the audio / video multiplexing unit 27, the connection from the video coding / decoding unit 26 to the audio / video multiplexing unit 27, and the connection between the audio / video multiplexing The connection from the plexing unit 27 to the moving image encoding / decoding unit 26 is shown as a single connection to avoid complexity in the figure, but these are actually connected separately for each of the communication channels 1 to n. ing.

Next, the basic operation of the video conference system will be described with reference to FIG. In the figure, when starting a video conference, it is necessary to first connect a line between each video conference terminal device and the multipoint video conference control device 21 (the partner terminal). This follows the normal calling procedure performed through LAPD. The SETUP (call setup message) has a digital transmission capability (BC) and an H.264 lower layer consistency (LLC). 221, send high layer consistency (HLC) as a conference.

When the partner terminal analyzes the SETUP and the communication possibility is approved, the partner terminal returns CONN (response) and the call is established. Here, in lower layer consistency, H.264 is used. Reference numeral 221 denotes the multimedia multiplexing / multiplexing in FIG.
The ITU-T Recommendation H.264 implemented in the separation unit 5 221 has been implemented.

When a call is established, the system control unit 2 controls the multimedia multiplexing / demultiplexing unit 5 to transmit a multi-frame synchronization signal and establish multi-frame synchronization.
Further, the system control unit 2 conforms to ITU-T Recommendation H.264. In accordance with H.242, the multimedia multiplexing / demultiplexing unit 5 is controlled to notify the capability and establish a communication mode. This is described in H. This is performed on the BAS signal on the H.221, and the necessary channel setting and bit rate allocation are performed with the common capability. In the present embodiment, three channels of audio, video, and data (LSD) are assigned. When the communication mode is determined, each channel can be handled as independent data,
Start operation as a video conference.

The above procedure is performed between each video conference terminal and each communication channel of the multipoint video conference control device 21, so that a multipoint video conference can be performed via the multipoint video conference control device 21. Becomes

When the video conference is started, the system control unit 2 starts the audio encoding / decoding unit 8 and the moving image encoding / decoding unit 13 to enable two-way communication of audio, moving image, and data. .

At the end of the video conference, the system control unit 2
Are the audio encoding / decoding unit 8 and the video encoding / decoding unit 13
And controls the ISDN interface 4 to release the call according to the procedure shown in FIG.

The user operates each of the operations described above (calling,
The start of “end of meeting” is performed by operating the console 17.
The input operation data is notified to the system controller 2 via the user interface controller 16. The system control unit 2 analyzes the operation data, starts or stops an operation according to the operation content, creates display data of guidance to a user, and sends the guidance display data to the console 17 via the user interface control unit 16. Display.

The multipoint video conference controller 21 is connected to one video conference terminal for each communication channel and operates a multipoint video conference in the above procedure.
In the example described above, an example is described in which connection is made by calling from the video conference terminal device side, but a call is made from the multipoint video conference control device 21 to the video conference terminal device set at a predetermined time. And can be connected.

In the video conference control device 21, voice
The moving image multiplexing unit 27 synthesizes the sound decoded in each communication channel and allocates it to each communication channel, and also reduces and synthesizes the moving image decoded in each communication channel and allocates it to each communication channel. I do.

FIG. 5 shows a connection form of each video conference terminal (terminal) and a multipoint video conference controller 21 (not shown).
3 shows an example of an image transmitted to each of the video conference terminal devices and displayed.

In the figure, terminals A to E are each a video conference terminal device (1, 19, FIG. 1 in FIG. 1) participating in a conference.
20 and the like), and shows a display image on each terminal of a moving image reduced and combined by the multipoint video conference control device. Each alphabet A to E in the square indicates a moving image transmitted from each of the video conference terminal devices A to E to the multipoint video conference control device. As can be seen from the figure, in each video conference terminal device, a video image obtained by reducing and synthesizing the video images from the video conference terminal devices at four points other than the own terminal by the multipoint video conference control device is displayed. . Thereby, the conference participant in each video conference terminal device has a video conference while watching the conference scene in the other terminal.

In order to create such a composite moving image,
In the audio / video multiplexing unit 27 of the multipoint video conference control device 21, after reducing the decoded moving image from the video encoding / decoding unit 26 of each communication channel, the reduced video for each communication channel is reduced. Each of the images is stored in an image memory arranged in the audio / video multiplexing unit 27, sequentially read out from the image memory in accordance with a synthesis mode for each communication channel and synthesized, and the synthesized moving image is transferred to each communication channel. I do. The transfer is performed in parallel on each communication channel. The synthesized moving image transferred to each communication channel is encoded by the moving image encoding / decoding unit 26 of each communication channel and transmitted to each video conference terminal device. In each video conference terminal device, as shown in FIG. As described above, the received encoded combined moving image is decoded by the moving image encoding / decoding unit 13 and displayed on the monitor 15. Note that the combination mode of moving images from each communication channel in the audio / video multiplexing unit 27 of the multipoint video conference control device 21 is variable depending on the situation during the video conference (movement of the floor) and the system control unit. It can be changed at any time according to the settings from 22.

Next, in the multi-point video conference control device according to the present invention, the decoding process of the moving image received from each video conference terminal device and the encoding process of the synthesized video transmitted to each video conference terminal device The first to third embodiments will be described separately.

First, the decoding processing of the moving image received from each video conference terminal device shown in FIG. 6 according to the first embodiment, and the code of the synthesized moving image transmitted to each video conference terminal device shown in FIG. The conversion process will be described.

These two processes are performed so that writing and reading to the motion vector table shown in FIG. 8, which will be described later, do not conflict with each other, and the above-described processing in the image memory in the audio / moving image multiplex unit 27 is performed. It is performed in parallel while controlling so that the correspondence of the motion vector table shown in FIG. 8 is not impaired. In addition, audio / video multiplex unit 2
The update of the motion vector table shown in FIG.
7 is executed prior to updating the image memory. During that time, the correspondence between the motion vector table shown in FIG. 8 and the image in the image memory is lost, so that the encoding process at the corresponding position (in units of macroblocks) is prohibited. These processes are executed in parallel for each communication channel.

The decoding processing procedure according to the first embodiment shown in FIG. 6 will be described below. Note that the format of the moving image to be decoded in this decoding process is FCIF (3
52 pixels × 288 lines). Therefore, one frame of the moving image includes the first to twelfth GOBs. Further, one GOB includes 33 macroblocks (MB). Therefore, one frame is composed of 12 × 33 = 396 macro blocks. Note that the arrangement order in a GOB frame and the arrangement of macroblocks in a GOB are determined by ITU.
-T Recommendation H. 261 and are well known per se, and therefore detailed description is omitted.

First, the moving picture code of each communication channel
The decoding unit 26 receives from each video conference terminal device,
By adding the motion vector information in units of macroblocks, decoding of 1 MB of image data in the current frame of the coded moving image that has been subjected to the inter-frame predictive coding with motion compensation is executed (process 101).

Then, the motion vector value of the MB obtained by the decoding process is referred to as the GOB containing the MB.
(See ITU-T Recommendation H.261) and the MB address for specifying the location of the MB in the GOB to which the MB belongs, to the system control unit 22 (process 102). If a motion vector value is not attached to the MB, the motion vector value is set to an X component,
The Y component is notified as a value of 0.

The system control unit 22 includes the system control unit 2
A motion vector table as shown in FIG. 8 is secured in the temporary storage memory arranged in the memory 2 and the MB position in the current frame determined by the address and the GOB number of the MB notified in the process 102 in the motion vector table. Is stored with the motion vector value of the MB.
3).

Here, the motion vector table shown in FIG. 8 will be described. In the figure, (a) shows the current frame (22 in the X direction) × (18 in the Y direction) = 396.
A table for storing the X component of the motion vector V for each of the MBs, and (b) is a table for storing the Y component. In each of these component storage tables, Px
Is M in the X direction of the current frame before being decoded and reduced.
Py indicates the position in B units, and Py indicates the position in MB units in the Y direction. Such a motion vector table is provided for an image (five in this embodiment) serving as a source of moving image synthesis, and stores the motion vector value notified from the moving image coding / decoding unit 26 in each communication channel. . If there is no MB data and there are skipped MBs, the vector values corresponding to those positions are rewritten to 0 at the same time. The position of the skipped MB is the ITU
-T Recommendation H. 261 is determined according to the transmission order of the MB data.

The moving picture coding / decoding section 26 transfers 1 MB of image data obtained by decoding to the audio / moving picture multiplexing section 27 (process 104). The above processing is repeated for all MBs constituting the current frame (No loop of the determination 105). In the audio / video multiplexing unit 27, as described above, the video of the current frame decoded from the video encoding / decoding unit 26 of each communication channel is reduced, and then, Audio / video multiplexing unit 27
The image data is stored in an image memory arranged therein, sequentially read out from the image memory in accordance with a synthesis mode for each communication channel, and synthesized, and the synthesized moving image is transferred in parallel to each communication channel. As a result, the motion vector information of the current frame for each communication channel is stored in the motion vector table for each communication channel of the system control unit 22 and the reduced moving image for each communication channel corresponding to the motion vector table is stored. It is stored in the audio / video multiplex unit 27.

The procedure for performing the inter-frame predictive coding with motion compensation on the reduced synthesized moving image for each communication channel based on the motion vector table for each communication channel obtained by the above-described decoding processing will be described. This will be described with reference to the encoding processing procedure according to the first embodiment shown in FIG.

In the figure, the moving picture coding / decoding section 26 for each communication channel synthesizes image data from each reduced picture data stored for each communication channel in the audio / moving picture multiplexing section 27 described above. 1 MB of image data corresponding to the position and MB position is read (process 201).

Further, the system control unit 22 executes the processing 201
, The MB read from the audio / video multiplex unit 27 by the video encoding / decoding unit 26 of each communication channel.
4 MB at a position corresponding to the position of (MB currently being processed)
8 is read from the motion vector table shown in FIG. 8 (process 202), and the read 4M
The MB currently being processed based on the motion vector value for B
Is calculated by the following calculation (process 203).

Now, the motion vector value at each MB position stored in the motion vector table of the audio / video multiplexing unit 27 for each communication channel is set to V = (Vx, Vy). The motion vector value of the MB is MV = (MVx, MVy), the X component MVx of the MV is obtained by the following (Equation 1), and the Y component MVy of the MV is obtained by the following (Equation 2).

[0050]

(Equation 1)

Qualitatively regarding the meanings of the above (Equation 1) and (Equation 2), the motion vector value of the MB currently being processed is determined by the value of the MB constituting the moving picture received by each communication channel. Among them, a plurality of MBs which are substantially included in the MB currently being processed by the reduction synthesis process
The motion vector of each of the four MBs (in the present embodiment) is summed and divided by the total number (4 in the present embodiment) to obtain a value averaged by the reduction ratio (1 in the present embodiment). / 2).

FIG. 9 shows the motion vector MV (MP) of the MB currently being processed in the reduced composite image that is being encoded.
x, MPy) and the original image that has been substantially included in the MB currently being processed by the reduction synthesis processing.
× 2 = each motion vector V for 4 MBs
(Px, Py), V (Px + 1, Py), V (Px, P
y + 1) and V (Px + 1, Py + 1). Also, as can be seen from the figure, the reduced composite image currently being encoded includes four images each of which has been reduced in length and width by half in the original image. Note that the arrow of each motion vector in the figure indicates that an image at a position corresponding to the previous frame of the current frame is referenced.

2 × 2 = 4 MBs of original image before reduction synthesis
Corresponds to one MB in the reduced composite image. Therefore, the motion vector value of one MB in the reduced composite image has a large correlation with the motion vector values of four MBs in the original image corresponding to the MB. Therefore, the motion vector value obtained by averaging the motion vector values of the four MBs of the original image can be considered to be substantially the same as the motion vector value of the corresponding one MB in the reduced composite image.
However, since the original image is actually reduced at a reduction ratio of に in the X direction and the Y direction in the reduced composite image, the motion of each pixel becomes １ ／. A half of the motion vector value obtained by averaging the motion vector values of the four MBs of the image becomes the motion vector value of one corresponding MB in the reduced composite image.

As described above, the system control unit 22
The motion vector of the MB currently being processed is calculated. The amount of calculation for the calculation is much smaller than that of the conventional block matching method and the like. Calculation amount can be greatly reduced. Note that the system control unit 22 further calculates the motion vector value for each MB constituting the reduced composite image shown in FIG. , ITU-
Recommendation T. H. According to ITU-T Recommendation H.261, it is not possible to refer to the outside of an image according to regulations. The motion vector value is clipped in the range defined by H.261.

Then, the system control unit 22 sets the calculated motion vector MV of the MB currently being processed in the moving image encoding / decoding unit 26 (process 204).

The motion vector MV of the MB currently processed set in the moving picture coding / decoding section 26 is responsible for the motion compensated inter-frame predictive coding of the input moving picture in the moving picture coding / decoding section 26. , Of the moving image encoder shown in FIG. 10 having the variable delay function for motion compensation. The moving image encoder shown in FIG.
-T Recommendation H. 261 is different from that of the first embodiment in that the motion vector of the MB currently being processed and detected by the image memory 100 having the variable delay function for motion compensation is This is the point given as MV. Accordingly, it becomes unnecessary to detect a motion vector by an enormous amount of calculation by the image memory 100 having the variable delay function for motion compensation.

The moving picture coder shown in FIG. 10 in the moving picture coding / decoding section 26 reads from the audio / moving picture multiplexing section 27 in the processing 201 according to the motion vector MV set by the system control section 22. 1M processing
The reduced synthesized image for B is subjected to inter-frame predictive coding with motion compensation (process 205). Thus, the moving image encoder shown in FIG. 10 outputs the motion vector v together with other encoding parameters to a video signal multiplexing unit (not shown) in the moving image encoding / decoding unit 26 as in the related art. The motion vector output as the motion vector v is the motion vector MV itself set by the system control unit 22. In the encoding in the moving picture encoder shown in FIG. 10, INTER other than detection of a motion vector is used.
The determination of / INTRA, ON / OFF of the loop filter, and the like are left to the adaptive control of the moving picture encoder shown in FIG.

The above processing is repeated for all the MBs constituting the current frame of the reduced composite moving image for each communication channel (No loop of decision 206).

Next, a description will be given of a process of decoding a moving image received from each TV conference terminal device and a process of encoding a composite moving image transmitted to each TV conference terminal device according to the second embodiment.

The decoding processing of the moving image received from each video conference terminal device is the same as the decoding processing procedure according to the first embodiment shown in FIG. As a result of the decoding processing shown in (1), the motion vector of the current frame for each communication channel is stored in the motion vector table for each communication channel of the system control unit 22, and the reduction for each communication channel corresponding to the motion vector table Moving image is audio / video multiplexing unit 27
Is stored.

A procedure for the case where the reduced synthesized moving image for each communication channel is subjected to inter-frame predictive coding with motion compensation based on the motion vector table for each communication channel obtained by the decoding process shown in FIG. About FIG. 1
This will be described with reference to the encoding processing procedure according to the second embodiment shown in FIG.

In the figure, the moving picture coding / decoding section 26 of each communication channel synthesizes image data from the reduced image data stored for each communication channel in the audio / moving picture multiplexing section 27 described above. 1 MB of image data corresponding to the position and MB position is read (process 301).

Further, the system control unit 22 performs processing 201
, The MB read from the audio / video multiplex unit 27 by the video encoding / decoding unit 26 of each communication channel.
4 MB at a position corresponding to the position of (MB currently being processed)
The motion vector value of the minute is read from the motion vector table shown in FIG.
2) Based on the read 4 MB motion vector values, the motion vector MV of the MB currently being processed is set to the first
Similarly to the embodiment, the motion vector search range SA is calculated based on the calculations shown in (Equation 1) and (Equation 2), and the motion vector search range SA is determined based on the calculated motion vector MV (Process 30).
3).

A method of determining the motion vector search range SA will be described with reference to FIG. In the figure,
(P) is the center point of the MB currently being processed to determine the motion vector search range SA, and (S) is the conventional ITU
The search range (± 15 pixels) of a motion vector in an encoder conforming to −T Recommendation H.261, (a), (b) and (c) show some specific examples of the motion vector MV obtained by calculation, (A (B) and (C) show the motion vector search range SA corresponding to the motion vectors (a), (b) and (c), respectively.

The system control unit 22 basically indicates the calculated motion vector MV ((a) (b) (c), etc.) with the center point (p) of the MB currently being processed as the origin. A motion vector search range SA is defined as a range of ± 3 pixels around a pixel.
To be determined. However, the search range is set by performing clipping so as not to exceed the conventional search range (± 15 pixels: this is the upper limit of the search range in ITU-T Recommendation H.261). That is, for example, in FIG.
The ranges of (A) and (B) show no clipping and the (C) shows the clipped range. As in the case of the first embodiment, when processing the MB at the position indicated by the diagonal lines of the reduced composite image in FIG. 2
In 61, since it is not possible to refer to the outside of the image,
TU-T Recommendation H. The search range is clipped to the range defined by H.261. By clipping the search range in this manner, a moving image encoder described below can be used in the ITU-T Recommendation H.264. A motion vector v that does not deviate from the definition of H.261 can be obtained.

In the first embodiment, the motion vector MV calculated in the process 303 is used as it is as the motion vector of the MB currently being processed. However, in the second embodiment, the motion vector search range is determined. Is not used as the motion vector of the MB currently being processed.

The system control section 2 sets the determined motion vector search range SA in the moving picture coding / decoding section 26 (step 304).

The motion vector search range SA for the currently processed MB set in the video encoding / decoding unit 26
Is provided to the image memory 101 having the variable delay function for motion compensation of the moving image encoder shown in FIG. 13 which performs the motion-compensated inter-frame predictive coding of the input moving image in the moving image encoding / decoding unit 26. .

The moving picture encoder shown in FIG.
ITU-T Recommendation H. H.261, which is different from the conventional one in that the image memory 101 having the variable delay function for motion compensation is itself used by the ITU-T Recommendation H.264. 261. In the range of ± 15 pixels, clipping is performed as needed so that the pixel to be referred is within the frame to be encoded, and a motion vector search of the MB currently being processed is performed. While the range was set, the MB currently being processed
Is given by the system control unit 22 as the motion vector search range SA.

The moving picture coder shown in FIG. 13 in the moving picture coding / decoding section 26 has the size of the reduced composite image of the MB currently being processed within the motion vector search range SA set by the system control section 22. Search for a motion vector (Process 30
5). This search method itself is already known per se, such as a block matching method.

Here, comparing the processing amount of the conventional motion vector search processing with the processing amount of the motion vector search processing in the second embodiment, the processing amounts are almost proportional to the width of the motion vector search range. Therefore, assuming that there is no clipping of the motion vector search range, conventionally, 31 (± 15 pixels) × 31 (± 15 pixels) = 961
In the second embodiment, 7 (± 3 pixels) × 7 (± 3 pixels) = 49 comparison operations are required, and the processing amount is about 1/20. .

As described above, in the second embodiment,
The reason that the search range of the motion vector can be narrowed is
Motion vector search range S for MB currently being processed
As described in the first embodiment, the motion vector MV that is the base for determining A is included in the MB currently being processed by the reduction synthesis process. Is obtained by multiplying the average value of the motion vectors of a plurality of MBs of the original image having a large correlation by the reduction ratio, and to take a value substantially close to the true motion vector value of the MB currently being processed. That is, even if the motion vector search range is not widened as in the related art, if the motion vector is searched for a minute range around the pixel indicated by the motion vector MV, the true value of the MB currently being processed is found therein. Can be said to exist.

As a result, it is possible to detect a motion vector with a processing amount much smaller than that of the related art and more accurately than in the first embodiment.

The moving picture encoder shown in FIG. 13 in the moving picture coding / decoding section 26 uses the motion vector of the MB currently being processed obtained in the processing 305, and accordingly, in the processing 301, the audio / moving picture multiplexing section 27 in the processing 301. Then, the reduced composite image of the MB currently being processed, which has been read from, is subjected to inter-frame predictive coding with motion compensation (process 306). As a result, FIG.
3 outputs a motion vector v together with other coding parameters to a video signal multiplexing unit (not shown) in the moving image encoding / decoding unit 26 as in the related art. The motion vector output as v is detected within the motion vector search range SA set by the system control unit 22. In addition,
In the encoding in the moving picture encoder shown in FIG. 13, the determination of INTER / INTRA, ON / OFF of the loop filter, etc. other than the detection of the motion vector is performed by adapting the moving picture encoder shown in FIG. Committed to control.

The above process is repeated for all the MBs constituting the current frame of the reduced composite moving image for each communication channel (No loop in decision 307).

Next, a description will be given of a process of decoding a moving image received from each TV conference terminal device and a process of encoding a composite moving image transmitted to each TV conference terminal device according to the third embodiment.

The third embodiment is a modification of the first embodiment. The decoding process of the moving image received from each video conference terminal device is performed according to the decoding process of the first embodiment shown in FIG. The processing is basically the same. However, in the third embodiment, the system control unit 22 stores, for each communication channel, a motion vector table as shown in FIG. 8 for the current frame and two frames for the frame (previous frame) one frame before the current frame. It has one point.

That is, when the current frame is being decoded, one of the two motion vector tables is used to store the motion vector value obtained when the current frame is decoded, and When decoding the frame, the motion vector value obtained when decoding the next frame is stored in a table other than the table storing the motion vector value of the current frame, The two motion vector tables are switched and used for each frame.

As a result, when decoding the current frame, the motion vector value for the previous frame obtained when decoding the previous frame is the motion vector value for storing the motion vector value for the current frame. It is always stored in a motion vector table different from the vector table.

As described above, the motion-compensated inter-frame predictive code of the reduced composite image according to the third embodiment is performed after the respective motion vector values of the previous frame of the current frame are stored in the respective motion vector tables. The procedure of the conversion process will be described with reference to FIG.

In the figure, the moving picture coding / decoding section 26 for each communication channel synthesizes image data from each reduced picture data stored for each communication channel in the audio / moving picture multiplexing section 27 described above. 1 MB of image data corresponding to the position and MB position is read (process 401).

Further, the system control unit 22 executes a process 401
, The MB read from the audio / video multiplex unit 27 by the video encoding / decoding unit 26 of each communication channel.
4 MB at a position corresponding to the position of (MB currently being processed)
The minute motion vector value is read from the motion vector table for the current frame shown in FIG. 8 (process 402).

Then, whether a frame skip has occurred,
That is, it is checked whether or not two or more decoded images have been rewritten between the immediately preceding frame and the current frame (decision 403). If no frame skip has occurred (No in decision 403), the same decoding as in the first embodiment can be performed using the motion vector values stored in the motion vector table for the current frame. The processing after the processing 406 is performed in the same manner as in the first embodiment. Actually, Steps 406 and 40
7, the processing 408, and the judgment 409 are respectively performed in FIG.
20 in the decoding process according to the first embodiment shown in FIG.
3, Steps 204, 205, and 206. In other words, the feature of the decoding process according to the third embodiment is the procedure for calculating the motion vector MV of the MB currently being processed in the current frame when a frame skip occurs.

That is, when a frame skip has occurred (Yes in decision 403), the 4 MB motion vector of the current frame read in the process 402 shows the MB of the previous frame of the position (corresponding position) indicated by the motion vector. Are read (process 404), and the motion vectors of the 4 MB of the current frame and the motion vector of the MB of the previous frame indicated by the vectors are respectively added, and the motion vectors of the current frame are added. Each new motion vector of 4 MB is set (process 405).

A specific example of the addition processing of the 1 MB motion vector of the current frame and the motion vector of the MB of the previous frame indicated by the motion vector will be described with reference to FIG. FIG. 11A shows the motion vector of the MB of the current frame read out in the process 402, and FIG.
MB of current frame read in process 402 (dotted frame)
, And the motion vector (solid line) of the MB (solid frame) of the previous frame read by the process 404 indicated by the motion vector.

The MB of the current frame actually read in the process 402 is a 2 × 2 4 MB, but the figure shows two patterns for one MB.
In the same figure (a), the vector (a) is the vector MB
, The lower left MB is pointed out, and the vector read out in processing 404 accordingly becomes the lower left MB vector (b), and the vector (b) is added to the vector (a). On the other hand, the vector (c) is
B, and the vector read out in step 404 in accordance therewith becomes the vector (d) of the MB at the same position, and the vector (d) is added to the vector (c).

That is, x of the motion vector of the MB (16 pixels × 16 pixels) of the current frame read out in the process 402
And if the magnitude (absolute value) of each component in the y direction is 8 or more, the motion vector of any of the eight surrounding MBs in the previous frame is determined. The motion vector of the MB at the same position is
At step 405, the data is read out and added at step 405.

The 4M of the current frame in the process 405
B and the motion vector of the MB of the previous frame indicated by the addition are performed by adding the vector (v1) and the vector (v
2) is performed in the same manner as that can be combined with the vector (v), and specifically, can be performed by adding each motion vector for each component in the x and y directions.

As described above, when a frame skip occurs, the frame skip occurs when the frame skip occurs.
By adding the motion vector for each MB of the previous frame stored in the motion vector table as shown in FIG. 8 at the time of decoding to the motion vector of the MB of the current frame indicating those MBs, skipping is performed. Since the motion amount of the image of the extracted frame can be interpolated, even if a frame skip occurs, it is possible to perform the motion-compensated inter-frame predictive encoding process on the reduced composite image as in the first embodiment.

The encoding process according to the third embodiment described above is the same as the encoding process according to the first embodiment shown in FIG.
03, processes 404 and 405), but the process when a frame skip occurs can be added to the encoding process according to the second embodiment shown in FIG. . That is, between the process 302 and the process 303 of the encoding procedure according to the second embodiment shown in FIG. 11, the judgment 403 in the encoding procedure according to the third embodiment shown in FIG. By adding the processing 404 and the processing 405, even when a frame skip occurs, the same inter-frame predictive coding processing with reduced motion of the reduced composite image as in the second embodiment can be performed.

In each of the embodiments described above, the present invention is applied to a reduced moving image obtained by reducing four original moving images at a reduction ratio of 縦 each in the vertical and horizontal directions as an encoded reduced combined moving image for each communication channel. Has been described as an example in which a reduced composite image obtained by combining two images in both the vertical and horizontal directions is encoded. However, the present invention provides a method for calculating the degree of correlation between the motion vector information of the original moving image and the motion vector information of the reduced composite moving image. Since it is used, the present invention is not limited to this, and can correspond to various reduction ratios (including cases where the vertical and horizontal reduction ratios are different), the number of synthesized moving images, and the synthesis position of the reduced synthesized moving images.

Further, in each of the embodiments described above, the inter-frame predictive coding method with motion compensation of the reduced synthesized moving image according to the present invention is applied to the multipoint video conference control device. In a moving image recording / reproducing / editing device, the present invention is applicable to a case where a plurality of dynamic images subjected to inter-frame prediction coding with motion compensation are decoded, reduced and synthesized, and inter-frame prediction coding with motion compensation is performed again. Needless to say. However, like a multipoint video conference control device, a plurality of motion-compensated inter-frame predictive-encoded dynamic images are decoded, reduced and synthesized, and the motion-compensated inter-frame predictive-encoding process is performed by a plurality of communications. Since it is necessary to perform the processing in real time for the number of channels, the detection of the motion vector must be performed simultaneously for a plurality of reduced composite images. The effect of the inter-frame predictive coding method with motion compensation for the reduced synthesized moving image is great.

[0093]

According to the first aspect of the present invention, the motion vector information added to each of the encoded synthesized moving images is the same as the motion vector information at the time of decoding each of the moving images constituting the synthesized moving images. , Is calculated based on the synthesis position and the reduction ratio at the time of the reduction synthesis, so that it is necessary to consider the high correlation between the motion vector information of the original moving image and the motion vector information of the synthesized moving image as in the related art. In addition, it is not necessary to perform the motion vector detection process again on the combined moving image, and the motion vector information to be added to the encoded combined moving image transmitted to each video conference terminal device can be almost accurately calculated by a simple calculation. Therefore, it is possible to reduce the calculation processing load for detecting the motion vector, and accordingly, it is possible to reduce the cost of the calculation processing device at a low speed, thereby reducing the device cost.

According to the second aspect of the present invention, the center of the search range in the re-encoding means for the motion vector information to be added to each of the encoded combined moving images is the original moving image constituting the combined moving image. Is calculated based on the motion vector information at the time of decoding, the synthesis position, and the reduction ratio at the time of reduction synthesis, and the re-encoding means performs motion only within a predetermined range centered on the calculated search range center. In order to search for the vector, the motion vector information of the moving image and the
Without considering the high correlation of the motion vector information of the synthesized moving image, the synthesized moving image does not need to perform the motion vector detection process again in a relatively wide search range,
Similarly to the first aspect, the motion vector information to be added to the encoded combined moving image transmitted to the video conference terminal device can be calculated by a simple calculation. Therefore, the processing load for detecting a motion vector can be much smaller than in the past. Furthermore, in consideration of the high correlation between the motion vector information of the original moving image and the motion vector information of the synthesized moving image,
The invention according to claim 1, wherein an approximate value of the motion vector of the synthesized moving image is obtained as the center of the search range, and the re-encoding means detects a true motion vector that should be within a predetermined range in the vicinity thereof. It is possible to more accurately calculate the motion vector information of the synthesized moving image.

According to the third aspect of the present invention, when a frame skip occurs when the re-encoding unit encodes the synthesized moving image, the motion vector calculation unit determines whether each small area of the current frame has a small area. The motion vector information of the previous frame is added to each of the motion vector information of the first frame and the second frame to interpolate the motion information of the skipped frame. Can effectively be realized, and the versatility of the invention according to claim 1 can be enhanced.

According to the fourth aspect of the present invention, when a frame skip occurs when the re-encoding unit encodes the synthesized moving image, the search range center calculation unit includes:
For each of the motion vector information for each small area of the current frame, the motion vector information of the previous frame is added to interpolate the motion information of the skipped frame. Even in this case, the invention according to claim 2 can be effectively realized, and the versatility of the invention according to claim 2 can be enhanced.

[Brief description of the drawings]

FIG. 1 is a diagram showing a configuration of a video conference system including a multipoint video conference control device according to an embodiment of the present invention.

FIG. 2 is a diagram showing a block configuration of a video conference terminal device according to the embodiment of the present invention.

FIG. 3 is a diagram showing a block configuration of a multipoint video conference control device according to an embodiment of the present invention.

FIG. 4 is a diagram showing a basic operation of the video conference system according to the embodiment of the present invention.

FIG. 5 is a schematic diagram showing a reduced composite image transmitted from the multipoint video conference control device to each video conference terminal device and displayed according to the embodiment of the present invention;

FIG. 6 is a flowchart showing a decoding processing procedure according to the first embodiment of the present invention.

FIG. 7 is a flowchart illustrating an encoding processing procedure according to the first embodiment of the present invention.

FIG. 8 is a diagram showing a specific example of a motion vector table stored in a system control unit of the multipoint video conference control device.

FIG. 9 is a diagram illustrating a correspondence relationship between four macroblocks in an original image and one MB in a reduced composite image.

FIG. 10 is a diagram showing a block configuration of a moving image encoder according to the first embodiment of the present invention.

FIG. 11 is a flowchart illustrating an encoding processing procedure according to the second embodiment of the present invention.

FIG. 12 is a diagram showing a specific example of a motion vector search range set in an encoding processing procedure according to the second embodiment of the present invention.

FIG. 13 is a diagram illustrating a block configuration of a moving image encoder according to a second embodiment of the present invention.

FIG. 14 is a flowchart illustrating an encoding processing procedure according to the third embodiment of the present invention.

FIG. 15 is a diagram illustrating a relationship between a motion vector of a current frame and a motion vector of a previous frame added in an encoding processing procedure according to the third embodiment of the present invention.

FIG. 16 is a diagram showing a method of adding motion vectors.

FIG. 17 illustrates a conventional multipoint video conference control device that decodes, reduces, combines, re-encodes, and transmits coded moving images received from each video conference terminal device to each video conference terminal device. It is the figure which showed the process typically.

[Explanation of symbols]

 1, 19, 20 Video conference terminal device 2 System control unit 3 Magnetic disk device 4 ISDN interface unit 5 Multimedia multiplexing / demultiplexing unit 6 Microphone 7 Audio input processing unit 8 Audio encoding / decoding unit 9 Audio output processing unit 10 Speaker 11 Video camera 12 Video input processing unit 13 Video encoding / decoding unit 14 Video output processing unit 15 Monitor 16 User interface control unit 17 Console 18, 28 ISDN line 22 System control unit 23 ISDN interface unit 24 Multimedia multiplexing / demultiplexing unit 25 Audio encoding / decoding unit 26 Video encoding / decoding unit 27 Audio / video multiplexing unit

Claims (4)

[Claims] 1. A video conference terminal device installed at a plurality of points is connected, and each of the video conference terminal devices receives
Receiving means for receiving a coded moving image subjected to motion-compensated inter-frame predictive coding with motion vector information added for each small region constituting a frame, and encoding from the received video conference terminal device A decoding unit for decoding the original moving images, a reduced synthesizing unit for creating a synthesized moving image by reducing and synthesizing the decoded original moving images, and forming a frame of the synthesized moving image Re-encoding means for generating an encoded combined moving image by performing motion-compensated inter-frame predictive encoding with motion vector information added for each small area; A multi-point video conference control device comprising at least a transmission unit for transmitting the current frame to the device. A current frame motion vector storage means for storing motion vector information obtained when decoding the coded moving image for each current frame, and a current frame of the coded synthesized moving image in the re-encoding means. The motion vector information stored in the current frame motion vector storage means for the current frame of each of the moving images reduced and synthesized as the synthesized moving image, A multi-point video conference control device, comprising: a motion vector calculating means for calculating based on a synthesized position on the synthesized moving image and a reduction ratio of the original moving images in the reduction synthesizing means. 2. A video conference terminal device installed at a plurality of points is connected, and each of the video conference terminal devices receives
Receiving means for receiving a coded moving image subjected to motion-compensated inter-frame predictive coding with motion vector information added for each small region constituting a frame, and encoding from the received video conference terminal device A decoding unit for decoding the original moving images, a reduced synthesizing unit for creating a synthesized moving image by reducing and synthesizing the decoded original moving images, and forming a frame of the synthesized moving image Re-encoding means for generating an encoded combined moving image by performing motion-compensated inter-frame predictive encoding with motion vector information added for each small area; A multi-point video conference control device comprising at least a transmission unit for transmitting the current frame to the device. A current frame motion vector storage means for storing motion vector information obtained when decoding the coded moving image for each current frame, and a current frame of the coded synthesized moving image in the re-encoding means. The center of the search range of the motion vector information to be added to each small area is defined as the motion vector information stored in the current frame motion vector storage unit for the current frame of each of the original moving images reduced and synthesized as the synthesized moving image. A search range center calculating unit that calculates, for each small area, a synthesis position of each of the moving images on the synthesized moving image and a reduction ratio of the moving images in the reduction and synthesis unit. The searching means calculates motion vector information to be added to each small area constituting the current frame of the encoded combined moving image by the search range center calculating means. Each subregion multipoint video conference control apparatus characterized by searching within a predetermined range around the search range center is calculated for. 3. A preceding frame for storing motion vector information obtained when the decoding unit decodes an encoded moving image of a frame one frame before the current frame, which is received from each of the video conference terminals. Further comprising a motion vector storage means, wherein the motion vector calculation means, when a frame skip occurs at the time of encoding the synthetic moving image in the re-encoding means, the re-encoding means The motion vector information to be added to each small area constituting the current frame of the image is stored in the current frame motion vector storage means for the current frame of each original moving image reduced and synthesized as the synthesized moving image. The motion vector for each area and the motion vector for each small area of the current frame stored in the current frame motion vector storage means. For the small area pointed to by Tor,
The motion vector stored in the previous frame motion vector storage means for the frame one frame before the current frame of each of the moving images reduced and synthesized as the synthesized moving image is used as each of the current frames of each of the moving images. 2. The method according to claim 1, wherein the calculation is performed based on the motion vector information added for the small area, a synthesis position of each of the moving images on the synthesized moving image, and a reduction ratio of the moving images in the reduction synthesizing unit. The multipoint video conference controller according to the above. 4. A previous frame motion which stores motion vector information obtained when the decoding means decodes an encoded moving image of a frame one frame before the current frame received from each of the video conference terminals. Further comprising a vector storage means, wherein the search range center calculation means,
When a frame skip occurs during the encoding of the synthesized moving image in the re-encoding unit, the motion to be added by the re-encoding unit for each small area constituting the current frame of the encoded synthesized moving image. The center of the search range of the vector information is set as the motion vector for each small area stored in the current frame motion vector storage means for the current frame of each original moving image reduced and synthesized as the synthesized moving image, and the current frame. For the small area indicated by the motion vector for each small area of the current frame stored in the motion vector storage means, the previous frame of the current frame of each original moving image reduced and synthesized as the synthetic moving image is set to the previous frame. The motion vector stored in the frame motion vector storage means is added to each of the small areas constituting the current frame of each of the moving images. 3. The method according to claim 2, wherein the motion vector information is calculated based on the added motion vector information, a synthesized position of each of the moving images on the synthesized moving image, and a reduction ratio of the moving images in the reduction synthesizing unit. Multipoint video conference controller.