JPH07255056A - Moving picture reproducing method and device therefor - Google Patents

Abstract

PURPOSE:To provide a moving image reproducing method which can obtain a reproduced smooth image excellent in picture quality even when a moving picture includes quick motion. CONSTITUTION:When a moving picture signal with frame structure and motion vectors corresponding to it are received and the moving picture signal is reproduced by using the motion vectors, not only a received image frame is reproduced, but also a new image frame is generated by motion compensation using the motion vectors. The new image frame is generated by using motion vectors which are closest to the frame in terms of time and compensating the motion by varying its coefficient, and it is generated at a time position corresponding to a coefficient value between received image frames. Consequently, images frames (I21 I2', B3, B3', B4, B4', P5...) which are more than received frames (I2, B3, B4, P5...) can be outputted.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to decoding a coded moving picture, and more particularly to a method and apparatus for reproducing a moving picture from a moving picture signal using a motion vector.

[0002]

2. Description of the Related Art Today, transmission of images in broadcasting, videophones, videoconferences, etc., transmission of moving image data in computer networks, etc. have become widespread with the development of new media. However, moving image data requires a larger amount of information than other signals, and the data transmission speed becomes a problem when transmitting a moving image signal. For example,
When the moving image signal is composed of 25 to 30 frames per second, transmission processing of about 80 to 100 Mbits per second is required. Therefore, some kind of information reduction method is required to transmit a moving image signal at high speed. In order to deal with this problem, various types of moving image signal transmission methods have been conventionally studied.

In the so-called "frame dropping" method, for example, instead of sending all moving image signals composed of 25 to 30 frames per second to the receiving side, frames are thinned out at a predetermined interval to reduce the total amount of moving image data. It reduces the transmission speed and improves the transmission speed. However, the dropped signal must be returned to the standard moving image signal on the receiving side. As a method for this, as shown in FIG. 5, a method in which a frame transmitted immediately before is repeated as it is, and as shown in FIG. 6, a method in which a frame missing from the transmitted and received frames by linear interpolation is reproduced. And have been adopted.

In the iterative method shown in FIG. 5, for example, the transmitted frame 101 is used as it is and the frame 101 is used.
A frame a is generated between
Is used as it is to generate a frame b between the frames 102 and 103. Further, in the linear interpolation method shown in FIG. 6, both transmitted frames 101 and 102 are used to generate an interpolated frame a between them, and the frame 102
And 103 are used to generate an interpolated frame b therebetween.

Furthermore, recently, MPEG (moving pic)
(ture experts group) has also been proposed a video compression encoding system. This method divides an image into blocks of a certain size on the transmission side, prepares a motion vector for each block, transmits the image data to the reception side together with the image data, and attempts to reproduce the image using the motion vector on the reception side. To do.

FIG. 7 is a schematic diagram showing a missing frame creating method in the moving picture compression coding method. It is assumed that there is a missing frame a between the frames 101 and 102 transmitted from the transmitting side, and a case in which an arbitrary block αa in this frame a is created. First, block αa
The motion vector between the frames 101 and 102 passing through the
And the block α1 and α2 of 102 are used to predict the block αa by a method such as linear interpolation. This process is performed on other blocks as well to create the entire frame a.

[0007]

However, in any of the above methods, there is a problem that the image quality is deteriorated when the moving image has a large motion.

In the repetitive method, the "frame-dropped" signal is not substantially reproduced, and the immediately preceding frame is repeated as it is, so that a smooth moving image cannot be reproduced. Further, in the linear interpolation method, image blur occurs when the image has a strong motion. Even in the missing frame creating method in the compression encoding method, if there is a strong motion in the image, a smooth moving image cannot be obtained.

Further, in order to reproduce a moving image having a large amount of motion with good image quality, a method has been proposed in which a transmitting side detects and transmits a motion vector and the receiving side uses the motion vector to reproduce an image. (JP-A-62-213493, JP-A-2-296479).

However, these methods similarly only reproduce the missing frame, and do not provide sufficient image quality for an image having a vigorous motion.

An object of the present invention is to provide a moving image reproducing method and apparatus which can obtain a good image quality even in a moving image which moves rapidly.

[0012]

A moving image reproducing method according to the present invention reproduces a received image frame by performing motion compensation using a motion vector, and changes the coefficient of the motion vector to perform motion compensation. It is characterized in that a new image frame is generated at a time point separated from the received image frame by a time corresponding to the coefficient value.

The new image frame is generated by using the motion vector closest to the image frame on the time axis. The varying coefficient is the temporal distance ratio between the received image frame and the new image frame. It should be noted that the same reproduction can be performed when a field structure is used instead of a frame.

The moving picture reproducing apparatus of the present invention is a memory for storing an already reproduced image frame and a decoding for reproducing the received image frame by performing motion compensation by shifting the motion vector from the reproduced image frame. And a coefficient control means for changing the coefficient of the motion vector at the time of performing the motion compensation in order to generate a new image frame at a time away from the received image frame.

[0015]

The method and apparatus for reproducing a moving picture according to the present invention receives a moving picture signal having a frame or field structure and a motion vector corresponding thereto and reproduces the moving picture signal using the motion vector. Not only is the received image frame reproduced by motion compensation using a vector, but a new image frame is generated by performing motion compensation by changing the coefficient of the motion vector. Therefore, as a result, more image frames than the number of received image frames can be output.

For example, referring to FIG. 3, when a new image frame (I2 ') is generated following the reproduced received image frame (I2), the previously reproduced received image frame (I2 and P5) is generated. The motion vector (b and c) closest to the frame (I2 ′) to be newly created is stored in the frame memory, and
The coefficients (weighting) of the motion vectors b and c are changed so as to correspond to the position of the frame (I2 ′), and a new frame (I2 ′) is created between the received image frames I2 and B3.

[0017]

Embodiments of the present invention will now be described in detail with reference to the drawings.

FIG. 1 is a functional block diagram showing an embodiment of a moving image reproducing apparatus according to the present invention. In this embodiment, a receiving buffer 1, an inverse VLC (Variable Length Code) circuit 2, an inverse quantization circuit 3, and an inverse DCT (Discrete Cosine Tran) are provided.
sform) circuit 4, frame memories 6a and 6b, motion compensation circuits 8a and 8b, an averaging circuit 9, and a selection switch 10, and a coefficient creation unit 11 and a coefficient variable control unit 12 for realizing the operation according to the present invention. It is mainly composed of. However, the functions other than the frame memory may be realized by an integrated circuit.

The received image contains a motion vector, and once stored in the buffer 1, the inverse VLC circuit 10
The screen type, macroblock type, motion vector, and quantized DCT coefficient are separated into various data. Here, the macroblock is 6 blocks (8 × 8
Pixels), of which 4 blocks are luminance signals and 2 blocks are color difference signals.

The screen types are I picture, P picture,
And B picture. The I picture is an intra-frame encoded screen, and is a screen type in which all macroblocks are intra-frame encoded. The P picture is an inter-frame predictive coding screen, and is a screen type in which intra-frame coding and inter-frame coding can be selected for each macroblock. The B picture is a bidirectional predictive coding screen, and is a screen type that allows past and future I and P pictures to be used for prediction. Since the bidirectional prediction matches the nature of the moving image, the B picture improves the prediction accuracy. The screen type data is output to the switch 7, and the output is switched depending on the screen type.

The macroblock type represents four types, and the selection switch 10 operates by this data. That is, the terminal a is used in the forward / backward prediction type predicted from the future, and the terminal b is used in the interpolative prediction predicted from both the future and the past bidirectionally as in the B picture, and the terminal / backward predicted from the past. The terminal c is selected in the forward prediction, and the terminal d is selected when the forward and backward predictions are not involved.

Quantized DCT separated by inverse VCL
Each coefficient is transformed through the inverse quantization circuit 3 and the inverse DCT circuit 4. Then, if it is an I picture, the data is stored in either of the frame memories 6a and 6b via the adder circuit 5 (here, it is used alternately).

In the case of a P picture, the data is added by the addition circuit 5 with the pixel value of which the corresponding position on the screen of one frame memory is motion-compensated by the motion compensation circuit for each macroblock, and the other frame memory. Stored in. Then, during the decoding process of the I picture and the P picture, the output of the frame memory 6a or 6b which is not currently being written is output through the b terminal or the c terminal of the switch 7.

In the case of a B picture, the outputs of the two frame memories are motion-compensated by the motion compensation circuits 8a and 8b using motion vectors, and their predicted pixel values or bidirectional predicted pixels averaged by the averaging circuit 9 are calculated. The value is selected by the selection switch 10 and added by the addition circuit 5 to obtain a reproduced image. In this case, the image currently undergoing the decoding process is directly output through the terminal a of the switch 7 without being stored in the frame memory.

The motion compensation circuits 8a and 8b perform motion compensation on the frame data stored in the frame memories 6a and 6b using the motion vector. In addition to this, when a new frame is generated according to the present invention, the coefficient variable control unit adjusts the coefficient (weighting) of the motion vector when the motion compensation circuits 8a and 8b perform motion compensation so as to correspond to the time point of the new frame. 12 to change. This variable coefficient is created by the coefficient creating unit 11 at the time of creating a new frame, and is multiplied by the motion vectors of the motion compensation circuits 8a and 8b by the coefficient variable control unit 12. As will be described later, this variable coefficient represents the temporal distance ratio between the transmitted frame and the newly created frame.

The image signal output through the switch 7 is input to the post-processing circuit 13, where, for example, NTSC.
It is converted into a format and output.

[0027] The operation of the operation moving picture reproducing apparatus having such a configuration will be described with reference to FIGS. FIG. 2 is a flowchart showing a series of processes in which a transmission source image is encoded and transmitted, and the received image is decoded to reproduce and generate a moving image. The original transmission images (B0, B
1, I2, etc.) is image data after compression processing of image frames by thinning on the transmitting side. In addition, I, P, and B in the figure indicate an I picture, a P picture, and a B picture, respectively.

First, on the transmitting side, B0, B1, I
The original image data is output in the order of 2, B3, B4, P5, B6, B7, P8, ..., The transmission order is changed by the encoding process, and the first B0 and B1 B pictures are skipped. The I2 picture is transmitted first. this is,
This is because the B picture is a bidirectional predictive coding screen created based on past and future I pictures and P pictures.

The same applies to the next B pictures B3 and B4. The transmission processing for P5, which is a P picture, is performed first, and then the transmission processing for B3, B4 is performed. Therefore, the order of the screens supplied to the receiving side through the media is I2, B0, B1, P5, B3, B4, P8, ...
・ It becomes. These received images are input to the device shown in FIG.
Decoding, frame playback and generation, and
The screen sequence is restored by the switch 7, and a double speed image is obtained. This point will be described in more detail.

Frame Reproduction FIG. 3 is a block diagram showing a coefficient creating unit 11, a coefficient variable control unit 12, motion compensation circuits 8a and 8b, an averaging circuit 9,
5 is a schematic diagram showing a specific operation example of the selection switch 10 and the addition circuit 5. FIG. In the figure, vector a, vector b, and vector d indicate forward motion vectors, and vector c and vector e indicate backward motion vectors. However, since I2 and P5 are necessary for reproduction of B3 and B4, only the portion surrounded by the broken line in FIG. 2 is illustrated in FIG.

In the case of a B picture which is a bidirectional predictive coding screen, its data is obtained by a weighted average of two motion vectors. For example, block B3 (X3, Y
3) is obtained by the following equation using the motion vector b = (bx, by) and the motion vector c = (cx, cy).

B3 block: X3 = (1/2) {(X
2 + bx) + (X5 + cx)} Y3 = (1/2) {(Y2 + by) + (Y5 + cy)} where (X2, Y2) is the corresponding block of the I2 picture, and (X5, Y5) is the corresponding block of the P5 picture. . That is, the motion compensation circuits 8a and 8b use the I2 and P5 block data stored in the frame memories 6a and 6b, respectively, to set the motion vector b to the corresponding block of the I2 picture and the motion vector c to the corresponding block of the P5 picture. By adding them respectively, and averaging them by the averaging circuit 9 to obtain the predicted value of the corresponding block of the B3 picture, which is added in the adding circuit 5 through the terminal b of the selection switch 10 to obtain the reproduced image in the switch 7 Is output through the terminal a.

The B4 picture block is also obtained in the same manner. That is, the motion vector d = (dx, dy) and the motion vector e = (ex, ey) are used to obtain the weighted average by the following equation.

B4 block: X4 = (1/2) {(X
2 + dx) + (X5 + ex)} Y4 = (1/2) {(Y2 + dy) + (Y5 + ey)} Since the P picture, which is the inter-frame predictive coding screen, is obtained only by the one-way vector, for example, the block of the P5 picture moves. Using only the vector a = (ax, ay), P5 block: X5 = (X2 + ax) Y5 = (Y2 + ay). That is, if I2 frame data is stored in the frame memory 6b, the motion compensation circuit 8b outputs I2
The motion vector a is added to the picture corresponding block, and the predicted value is added through the terminal c of the selection switch 10 to the addition circuit 5.
Is added in. Then, after it is stored in the other frame memory 6a, it is output through the terminal c of the switch 7.

Since the I2 picture is a macroblock type picture which has been intra-frame coded, the terminal d, ie, Intra, is selected by the selection switch 10 and the inverse D
The output of the CT circuit 4 is the same as that of the frame memory 6a or 6
Store in b. Then, it is output from the stored frame memory through the terminal b or c of the switch 7.

Frame Generation In FIG. 3, newly generated I2 ', B3', B
Each 4'frame will be described.

First, I2 'is a screen following the I2 picture on the time axis, and the motion vectors closest to this I2' on the time axis are the vector b and the vector c. In order to express I2 'using these motion vectors b and c, the coefficient of the motion vector b may be changed from 1 to 1/2 and the coefficient of the motion vector c may be changed from 1 to 5/4. This variable coefficient represents the temporal distance ratio between the transmitted frame and the newly generated frame. This is because it is considered that the temporal distance ratio is almost equal to the ratio of the magnitude of image movement. In this way, when the coefficient of the motion vector is changed by the coefficient variable control unit 12 and the terminal b is selected by the selection switch 10, the block of the I2 ′ picture is obtained by the following formula using the I2 corresponding block and the P5 corresponding block. .

X2 '= (1/2) {X2 + (1/2) b
x + X5 + (5/4) cx} Y2 '= (1/2) {Y2 + (1/2) by + Y5 +
(5/4) cy} B3 'is a screen following the B3 picture, and if the vector d and the vector c are used as the closest motion vectors on the time axis, the weighting is 3 for the motion vector d.
/ 4, which is also 3/4 for the motion vector c.
Therefore, the block of the B3 'picture is obtained by the following equation.

X3 '= (1/2) {X2 + (3/4) d
x + X5 + (3/4) cx} Y3 '= (1/2) {Y2 + (3/4) dy + Y5 +
(3/4) cy} Furthermore, B4 'is a screen following the B4 picture, and if the vectors a and e are used as the motion vectors closest to the time axis, the B4' picture block is obtained by the following equation.

X4 '= (1/2) {X2 + (5/6) a
x + X5 + (1/2) ex} Y4 '= (1/2) {Y2 + (5/6) ay + Y5 +
(1/2) ey} Here, a new interpolation frame is generated using bidirectional motion vectors, but of course, even when only one unidirectional vector is present, the closest one is also the closest on the time axis. A new frame can be generated using one or more motion vectors.

By changing the coefficient of the motion vector of the motion compensation circuit by the coefficient variable control unit 12 in this way, a new frame can be generated, and a moving image having a larger number of frames than the transmitted image (in the present embodiment). You can play back double-speed images. As a result, it is possible to obtain a good image quality even for a moving image that moves rapidly.

FIG. 4 is a schematic block diagram of a broadcasting system to which the moving picture reproducing apparatus of this embodiment is applied. In the figure, the transmitting side is composed of a data compressing section 41, a priority encoder 42, a transmission encoder 43, and a modem FAC encoder 44, and the receiving side is, contrary to the above, a modem FAC decoder 45, a transmission decoder 46,
It is composed of a priority decoder 47 and a decompression unit 48. The data processing flow of this system is shown in FIG. The decompression unit 48 corresponds to the interpolation frame generation function in this embodiment shown in FIG.

[0043]

As described in detail above, the moving picture reproducing method and apparatus according to the present invention receives a moving picture signal having a frame or field structure and a corresponding motion vector, and uses the motion vector. When reproducing a moving image signal, not only is a received image frame reproduced by motion compensation using a motion vector, but also a new image frame is generated by performing motion compensation by changing the coefficient of the motion vector.

As a result, it is possible to output a larger number of image frames than the number of received image frames, obtain a smooth reproduced image even for a moving image with a lot of movement, and realize a good image quality.

[Brief description of drawings]

FIG. 1 is a functional block diagram showing an embodiment of a moving image reproducing apparatus according to the present invention.

FIG. 2 is a flowchart showing a series of processes in the transmission / reception system using the present embodiment that encodes and transmits a transmission source image and decodes the received image to reproduce and generate a moving image.

FIG. 3 is a schematic diagram showing a specific operation example of new frame generation in the present embodiment.

FIG. 4 is a schematic configuration diagram of a broadcasting system to which the present invention is applied.

FIG. 5 is a schematic diagram showing a conventional method of reproducing a frame by repetition.

FIG. 6 is a schematic diagram showing a conventional method of reproducing a frame by linear interpolation.

FIG. 7 is a schematic diagram showing a conventional method of detecting a motion vector from a transmitted moving image signal and reproducing a frame using the motion vector.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 buffer 2 inverse LVC circuit 3 inverse quantization circuit 4 inverse DCT circuit 5 addition circuit 6a frame memory 6b frame memory 7 switch 8a motion compensation circuit 8b motion compensation circuit 9 averaging circuit 10 selection switch 11 coefficient creation unit 12 coefficient variable control unit 13 post-processing unit 41 compression unit 42 priority encoder 43 transmission encoder 44 modem FAC encoder 45 modem FAC decoder 46 transmission decoder 47 priority decoder 48 decompression unit

Claims (20)

[Claims] 1. A moving picture reproducing method for receiving a moving picture signal having a frame / field structure and a motion vector thereof and reproducing the moving picture using the motion vector, wherein motion compensation is performed using the motion vector. By reproducing the received image frame / field, and by performing the motion compensation by changing the coefficient of the motion vector, a new image is obtained when the received image frame / field is separated by a time corresponding to the coefficient value. A moving image reproducing method characterized by generating a frame. 2. The moving image reproducing method according to claim 1, wherein the new image frame / field is generated using a motion vector closest to the image frame / field on the time axis. 3. The moving image reproducing method according to claim 1, wherein the coefficient is a temporal distance ratio between the received image frame / field and the new image frame / field. . 4. A moving image in which each frame receives a moving image signal having a frame structure composed of a plurality of pixel blocks and a motion vector added to each of the pixel blocks, and reproduces the moving image using the motion vector. In the image reproducing method, the received image frame is reproduced by performing motion compensation using the motion vector, and the motion compensation is performed by changing the coefficient of the motion vector, so that only the time corresponding to the coefficient value is reached. A moving image reproducing method, characterized in that a new image frame is generated at a point of time away from the received image frame. 5. The moving image reproducing method according to claim 4, wherein the new image frame is generated by using one or more motion vectors closest to the image frame on the time axis. 6. In the new image frame, a value generated by motion compensation in which each coefficient of the one or more motion vectors is changed is weighted averaged by the number of the motion vectors for each pixel block. The moving image reproducing method according to claim 4 or 5, wherein the moving image reproducing method is generated. 7. A method for receiving a moving picture signal with a reduced number of frames and a motion vector, and reproducing a moving picture using the motion vector, wherein the reception is performed by performing motion compensation using the motion vector. By reproducing the image frame and the missing frame in the moving image signal and performing the motion compensation by changing the coefficient of the motion vector, at a time point when the received image frame is separated from the received image frame for a time corresponding to the coefficient value. A moving image reproducing method characterized in that a new image frame is generated. 8. A method of receiving a moving image signal in which the number of frames is reduced by thinning out a frame on the transmitting side and a motion vector thereof, and reproducing the moving image using the motion vector on the receiving side, Corresponding to the coefficient value by reproducing the received image frame and the missing frame in the moving image signal by performing motion compensation using a vector, and performing the motion compensation by changing the coefficient of the motion vector. A new image frame is generated at a time point distant from the received image frame for a predetermined time. 9. The moving image reproducing method according to claim 7, wherein the new image frame is generated by using a motion vector closest to the image frame on the time axis. 10. The moving image reproducing method according to claim 7, wherein the coefficient is a temporal distance ratio between the received image frame and the new image frame. . 11. A moving picture reproducing method for receiving a moving picture signal having a frame / field structure and a motion vector thereof, and reproducing the moving picture using the motion vector, wherein motion compensation is performed using the motion vector. By decoding the received image frame / field, and by performing the motion compensation by changing the coefficient of the motion vector, a new image is obtained when the received image frame / field is separated from the received image frame / field for a time corresponding to the coefficient value. A moving image reproducing method characterized by generating a frame. 12. The moving image reproducing method according to claim 11, wherein the new image frame / field is generated by using a motion vector closest to the image frame / field on the time axis. 13. The moving image reproducing method according to claim 11, wherein the coefficient is a temporal distance ratio between the received image frame and the new image frame. 14. A moving picture reproducing apparatus which receives a moving picture signal having a frame / field structure and a motion vector thereof and reproduces a moving picture using the motion vector, and stores already reproduced picture frames / fields. Memory for decoding, a decoding means for reproducing the received image frame / field by performing motion compensation by shifting only the motion vector from the reproduced image frame / field, and a decoding unit newly added at a time point apart from the received image frame / field. And a coefficient control unit that changes the coefficient of the motion vector when performing the motion compensation in order to generate a different image frame / field. 15. The moving image reproducing apparatus according to claim 14, wherein the new image frame / field is generated using a motion vector closest to the image frame / field on the time axis. 16. The coefficient is the received image frame /
15. A temporal distance ratio between a field and the new image frame / field.
Alternatively, the moving image reproducing apparatus according to Item 15. 17. A device for receiving a moving image signal with a reduced number of frames and a motion vector, and reproducing a moving image using the motion vector, a memory for storing an already reproduced image frame, Decoding means for performing motion compensation by shifting only the motion vector from the reproduced image frame to reproduce a missing frame in the received image frame and the moving image signal, and a new decoding unit at a point apart from the received image frame. And a coefficient control unit that changes a coefficient of the motion vector at the time of performing the motion compensation to generate an image frame. 18. An apparatus for receiving a moving image signal in which the number of frames is reduced by thinning out a frame on the transmitting side and a motion vector thereof, and reproducing the moving image using the motion vector on the receiving side, which has already been reproduced. A memory for storing the reproduced image frame, and a decoding means for reproducing the received image frame and a missing frame in the moving image signal by performing motion compensation by shifting the motion vector from the reproduced image frame. And a coefficient control unit that changes a coefficient of the motion vector at the time of performing the motion compensation to generate a new image frame at a time point apart from the received image frame. . 19. The moving image reproducing apparatus according to claim 17, wherein the new image frame is generated by using a motion vector closest to the image frame on the time axis. 20. The moving image reproducing apparatus according to claim 17, wherein the coefficient is a temporal distance ratio between the received image frame and the new image frame. .