JPH10271508A - Moving image interpolation device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To realize a moving image interpolation device at a receiver side by which a missing frame is accurately recovered. SOLUTION: A moving image interpolation device is provided with a transmitter 1 that consists of a frame interleave section 11 conducting frame interleaving in matching with a transmission rate and a frame interpolation capability of a receiver, of a motion detection section 12 obtaining a motion vector from interleaved image data, of a coder 13 applying compression coding to moving image information, and of a buffer 14 storing image data and moving vector data, and with a receiver 2 that consists of a buffer 21, of a decoder 22 decoding compressed image data, and of a frame interpolation section 23 interpolating an interpolation frame with a received motion vector. Then the frame interpolation section 23 re-configures an proper interpolation frame between adjacent transmission frames from a motion vector to conduct interpolation processing to the frame.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a moving picture interpolating apparatus used for moving picture compression processing and the like. More specifically, the present invention relates to a method for transmitting motion information for a thinned-out frame on a transmitting side and moving a lost frame on a receiving side. The present invention relates to a moving image interpolation device that can accurately reproduce information based on information.

[0002]

2. Description of the Related Art A moving image interpolation apparatus is considered to be mainly used in a video conference or the like. In a conventional moving image processing apparatus, a moving image interpolation apparatus that reproduces a frame that has been lost on the receiving side is used. In a device using the conventional frame interpolation method, motion is detected for each received frame, and an interpolation frame to be interpolated is reconstructed based on a ratio of a distance between the frame to be interpolated and a frame before and after the received frame. Was. As a known example, there is a moving image compression apparatus disclosed in Japanese Patent Application Laid-Open No. Hei 7-177514 which detects motion between frames, nonlinearly approximates the motion information, and predictively interpolates to reconstruct an interpolated frame.

[0003]

However, in such a conventional moving image processing apparatus, since the detection of a motion vector is performed by an interpolation method using a straight line or a non-linear approximation, the moving object has a mechanical motion. And does not move irregularly. This is due to the approximation. The present invention
It is an object of the present invention to provide a moving image interpolation device capable of accurately reproducing a missing frame.

[0004]

According to the present invention, in order to achieve the above object, a frame image between each frame is obtained from moving image information including a plurality of pieces of frame image information and motion information for frames thinned out between the frames. In a video interpolating apparatus that performs predictive interpolation, approximation processing is performed by having as many motion vectors as the number of interpolated frames for thinned frames to be interpolated according to the transmission rate and the frame interpolation processing capability of the receiving side as motion information. Not to do.

[0005] For example, in low-rate moving image compression in a video conference or the like, the cause of the low rate is often the communication capability, and the moving image processing capability generally exceeds the communication capability. In such a case, in the present invention, by configuring the motion information on the decimated frame in accordance with the transmission rate and the frame interpolation processing capability of the receiving side on the transmission side,
On the receiving side, the thinned frames are reconstructed based on the motion information. Therefore, since the approximation process for the motion is not performed, natural motion can be accurately reproduced.

[0006]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will be described below with reference to the drawings.

FIGS. 1 to 3 show an embodiment of a moving picture interpolation apparatus according to the present invention.

FIG. 1 is a block diagram showing the configuration of a video interpolator according to the present invention. In FIG. 1, the video interpolator comprises a transmitter 1 and a receiver 2.

In FIG. 1, a transmitter 1 of a moving picture interpolation apparatus is shown.
Is composed of a frame thinning unit 11, a motion detecting unit 12, an encoder 13, a buffer 14, and the like.

[0010] The frame thinning section 11 performs frame thinning according to the transmission rate and the frame interpolation capability of the receiver 2, and outputs image data of the transmission frame to the encoder 13. On the other hand, among the input image data, the image data of the frame to be reproduced by the receiver 2 is
2 is output.

The motion detecting section 12 detects a motion vector for a frame to be reproduced and outputs the motion vector to an encoder. Similarly, a plurality of frames can be thinned out, and the motion vectors are detected by the number of interpolation frames. The motion information is output to a buffer.

[0012] The encoder 13 encodes the image data based on the image data of the transmission frame and the motion vector output by the motion detection unit 12, and outputs the image data to the buffer 14.

The buffer 14 transmits the image data and the motion information data output from the encoder 13 and the motion detector 12 in accordance with the transmission rate.

[0014] The receiver 2 of the video interpolator includes a buffer 2
1, a decoder 22, a frame interpolation unit 23 and the like.

The transmitted data is stored in a buffer 21, and the encoded image data is output to a decoder 22.

The decoder 22 decodes the encoded image data and outputs it to the frame interpolator 23.

The frame interpolation unit 23 reconstructs an appropriate interpolation frame between adjacent transmission frames based on the motion vector obtained as the motion information from the buffer 21 and the image data output from the decoder 22. Performs interpolation.

FIG. 2 shows the interpolation method described in this embodiment. The dotted line part in the figure indicates a part reconstructed by interpolation. In the image data, the circle moves irregularly. After the frame I1 is transmitted, the motion vectors V11 and V12 and the frame P1 are transmitted.
Motion vector V11, frame I1 and frame P1
To reconstruct the interpolation frame B11. Similarly, using the motion vectors V11 and V12, the interpolation frame B12
Reconfigure. Further, the motion vectors V21 and V22 and the frame 12 are transmitted. An interpolated frame B21 is reconstructed from the motion vector V21 and the frames P1 and I1. Similarly, the interpolation frame B22 is reconstructed using the motion vectors V21 and V22. As described above, it is possible to accurately reproduce an irregularly moving circle.

FIG. 3 shows the contents described in FIG. 2 in a frame. Since the transmission rate is low, frame data of B11, B12, B21, and B22 in the middle cannot be transmitted. Frame P1
Are transmitted, the decimated frames B11, B1
2 are transmitted together with the image data of the frame P1 as motion information. The frames B11 and B12 are reconstructed and interpolated based on the respective motion vectors. Similarly, when transmitting frame I1,
The respective motion vectors for the decimated frames B21 and B22 are transmitted together with the image data of the frame 12 as motion information. The frames B21 and B22 are reconstructed and interpolated based on the respective motion vectors.

FIG. 4 shows Japanese Patent Application Laid-Open No. Hei 7-1 as a known example.
Using the technology of the 77514 video compression device, FIG.
3 illustrates a case in which a moving image having the motion indicated by. B11, B12, B21, B indicated by dotted lines
Reference numeral 22 denotes a frame that is nonlinearly approximated and interpolated using the motion vectors V1 and V2. Since the approximation process is performed on the motion, the actual irregular motion cannot be expressed, and the motion becomes a smooth curved motion.

The example used in FIGS. 2 and 3 is MPEG
It is very similar to video information using compression technology. MPE
In the G technology, generally, an I-frame having all image data, a P-frame constituting an image using information of the immediately preceding I-frame, one preceding I-frame and one succeeding P-frame, or It consists of a B frame that constitutes an image using information of the immediately preceding P frame and the immediately succeeding I frame. Normally, when image data filed using the MPEG technology is reproduced at a low rate, B frames are often thinned out. That is, as described with reference to FIG. 3, the image data of the frames I1, P1, and I2 is transmitted and reproduced. However, suppose that the thinned B frame is reconstructed by using the moving image interpolation processing.
The B frame and the P frame have motion vectors as information in order to use preceding and succeeding frames. In the present invention, the motion vectors of the B frame and the P frame are used when reconstructing the interpolation frame. That is, when the frames B11 and B12 are thinned out after the transmission of the frame I1, the information of the motion vector of the frames P11 and B12 is added to the information of the frame P1 and transmitted. Similarly, when thinning out the frames B21 and B22, the information of the frame I1 is added with the information of the motion vectors of the frames B21 and B22 and transmitted. As described above, the present invention can be used even when the transmission rate of image data filed by the MPEG technology is increased.

[0022]

According to the present invention, it is possible to add a motion vector for a frame decimated at the time of transmission to a transmission frame and predictively reconstruct an interpolation frame based on the motion vector for the frame decimated at the time of frame interpolation. Therefore, it is possible to accurately reproduce even irregular movements.

[Brief description of the drawings]

FIG. 1 is a functional block diagram of an embodiment according to the present invention.

FIG. 2 is an explanatory diagram of frame interpolation according to the embodiment.

FIG. 3 is an explanatory diagram of frame interpolation according to the embodiment;

FIG. 4 is an explanatory diagram of frame interpolation in a conventional technique.

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 1 ... Transmitter, 2 ... Receiver, 11 ... Frame thinning part, 12 ... Motion detection part, 13 ... Encoder, 14, 21 ...
Buffer, 22: decoder, 23: frame interpolation unit.

Claims (2)

[Claims] 1. A moving image interpolating apparatus for predictively interpolating a frame between frames from moving image information comprising motion information between frames together with a plurality of frame image information. A video interpolating apparatus for performing predictive interpolation according to motion information. 2. The motion information between frames has motion vectors for the frames to be interpolated, which are thinned out, in accordance with the transmission rate and the frame interpolation processing capability of the receiver. The moving image interpolation device according to claim 1, wherein