JP2003284075A - Method and apparatus for coding moving image, and method and apparatus for decoding - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a moving image coding method which enables a small increase in the quantity of operation and in overhead of coded data, and increases an estimation efficiency, when a fading image which is hard to be dealt with by conventional moving image coding methods such as an MPEG method, etc., is coded. <P>SOLUTION: When a moving image is coded, coding is performed by performing appropriate changeover of following ways; a plurality of decoded moving image signals are used as reference frames, and an estimated macroblock image is formed from one of the reference frames for each macroblock, or reference macroblocks are cut out of the reference frames, and their average value is used as an estimated macroblock image, or reference macroblocks are cut out of the reference frames, and an estimated macroblock image is formed by linear external insertion or linear internal insertion according to the distance between a reference frame and a coding frame. <P>COPYRIGHT: (C)2004,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a motion compensation prediction interframe coding method and apparatus and a motion compensation prediction interframe decoding method and apparatus using a plurality of reference frames.

[0002]

2. Description of the Related Art As a motion compensation predictive interframe coding method for moving images, MPEG1 (ISO / IEC11172-
2), MPEG2 (ISO / IEC13818-2),
MPEG4 (ISO / IEC14496-2) and the like have been widely put into practical use. In these coding methods, coding is performed by a combination of an intra-frame coded picture (I picture), a forward prediction inter-frame coded picture (P picture), and a bidirectionally predicted inter-frame coded picture (B picture).

A P picture is coded with the immediately preceding P or I picture as a reference image, and a B picture is coded with the immediately preceding and immediately following P or I pictures as a reference image. In MPEG, it is possible to selectively generate a prediction image for each macroblock from one video frame or a plurality of video frames. In the case of a P picture, a predicted image is normally generated from one reference frame in macroblock units, and in the case of a B picture, a predicted image is generated from any one of the forward and backward reference images, and In some cases, reference macroblocks are cut out from the reference images in the rear, and a predicted image is generated from the average value thereof. Information on these prediction modes is embedded in the encoded data for each macroblock.

However, in any of the predictive coding methods, prediction is effective when the same video image is temporally translated in parallel in a macroblock size or a larger area. Scaling and rotation of
Alternatively, it cannot be said that the prediction efficiency is necessarily good with respect to the time variation of the amplitude of the signal such as fade-in / fade-out. In particular, in encoding at a fixed bit rate, if these videos with poor prediction efficiency are input, the image quality may be significantly deteriorated. In addition, variable bit rate encoding suppresses image quality deterioration, so a large amount of code is consumed for these videos with poor prediction efficiency.
This leads to an increase in the total code amount.

On the other hand, the temporal enlargement / reduction or rotation of the image,
Alternatively, since fade-in / fade-out and the like can be approximated by affine transformation of a moving image signal, if the prediction using the affine transformation is performed, the prediction efficiency for these videos is significantly improved. However, in order to estimate the parameters of the affine transformation, a huge amount of parameter estimation calculation is required at the time of encoding.

Specifically, it is necessary to convert the reference image with a plurality of conversion parameters to determine the parameter that minimizes the prediction residual, which results in an enormous amount of conversion operations. As a result, the cost such as the encoding calculation amount or the hardware scale becomes enormous. Also, it is necessary to encode not only the residual signal but also the conversion parameter itself,
The overhead of encoded data becomes enormous. Further, at the time of decoding, the inverse affine transformation is required, and the cost of the decoding calculation or the hardware scale becomes enormous.

[0007]

As described above, in the conventional moving picture coding method such as MPEG, there is a problem that sufficient prediction efficiency cannot be obtained with respect to time change of moving pictures other than parallel movement. there were. Further, in the moving picture coding and decoding method using the affine transformation, although the prediction efficiency itself is improved, the overhead of coded data increases and
There is a problem in that encoding and decoding costs are significantly increased.

According to the present invention, particularly for a faded image, which is not good at the conventional moving image encoding method such as MPEG, the increase in the amount of calculation and the overhead of encoded data is small, and the prediction efficiency is significantly improved. It is an object of the present invention to provide a moving image coding method and device and a decoding method and device that are capable of decoding.

[0009]

In order to solve the above-mentioned problems, according to the present invention, in motion-compensated prediction interframe coding in which a plurality of moving picture frames are referred to for each macroblock, a plurality of reference frames are extracted from the plurality of reference frames. A reference macroblock is generated and one of the plurality of reference macroblocks is generated.
Or an average value of the plurality of reference macroblocks, or either linear extrapolation prediction or linear interpolation prediction by the plurality of reference macroblocks is selected as a prediction macroblock, and the selected prediction macroblock and the code are selected. The first feature is that the prediction error signal with the encoded macroblock, the prediction mode information, and the motion vector are encoded.

In decoding motion-compensated inter-frame coded data that refers to a plurality of moving picture frames for each macroblock, coded motion vector data,
Prediction mode information and a prediction error signal are received, and a prediction macroblock is generated from a specific one frame of the plurality of reference frames according to the motion vector and the prediction mode, or a plurality of reference frames are generated. Of the plurality of reference macroblocks as a prediction macroblock, or a prediction macro from either linear extrapolation prediction or linear interpolation prediction by the plurality of reference macroblocks. The second feature is to select whether to generate a block and add the generated prediction macroblock and the prediction error signal.

In the conventional moving picture coding system such as MPEG, when a predictive macroblock image is generated from a plurality of reference frames, the reference macroblock is cut out from each reference frame and the average value of the signal is used. Therefore,
When the amplitude of the video signal changes with time due to a fade or the like, the prediction efficiency may decrease. However, the first aspect of the present invention
Alternatively, according to the second feature, when the predicted image is generated by extrapolating or interpolating by linear prediction from a plurality of frames, and the amplitude of the video signal monotonously changes with time,
It is possible to greatly improve the prediction efficiency, and it is possible to perform high-quality coding with high image quality.

In the inter-frame predictive coding, it is general that the coding side uses the decoded image of the already coded image as the reference frame, and the decoding side uses the already decoded image as the reference frame. Target. Therefore, the influence of the coding noise in the reference frame contributes to the decrease in the prediction efficiency. It is known that averaging reference macroblocks cut out from a plurality of reference frames has a noise removal effect and contributes to improvement of coding efficiency. This is an operation equivalent to the technique known as a loop filter in predictive coding.

According to the first or second aspect of the present invention, an averaging process from a plurality of reference frames having a high loop filter effect, or an input of linear interpolation or linear interpolation effective for a fade image or the like is performed. It is possible to select the optimum prediction mode according to the image, and it is possible to improve the coding efficiency for an arbitrary input image.

According to the present invention, in the motion compensation predictive interframe coding for referring to a plurality of moving picture frames for each macroblock, the plurality of reference frames are two frames coded immediately before the target frame to be coded. In the linear extrapolation prediction using the plurality of reference macroblocks, a reference macroblock signal generated from a reference frame that is one frame before the reference macroblock signal that is generated by doubling the amplitude of the reference macroblock signal generated from the immediately preceding reference frame. The third feature is that the prediction macroblock is generated by subtracting

In the decoding of the motion-compensated prediction interframe coded data that refers to a plurality of moving picture frames for each macroblock, the plurality of reference frames are two frames decoded immediately before the target frame to be coded. And
In the linear extrapolation prediction using the plurality of reference macroblocks, a reference macroblock signal generated from a reference frame one frame before is obtained from a signal obtained by doubling the amplitude of the reference macroblock signal generated from the immediately preceding reference frame. A fourth feature is that the prediction macroblock is generated by subtracting the prediction macroblock.

As described above, the conventional moving picture coding system such as MPEG has a problem that the prediction efficiency is poor when the amplitude of the video signal varies with time due to a fade or the like. For example, if V (t) is the video frame at time t and V '(t) is the video frame at time t after the fade processing, fade-in and fade-out are realized by equations (1) and (2), respectively. You can do it. In Expression (1), (a) indicates the fade period, the feed-in starts at time t = 0, and the fade-in ends at time T. Further, in Equation (2) shows that (b) is shows the fade period, the fade-out process from time T ₀ is, fade-out initiated time T ₀ + T is completed.

[0017]

[Equation 1]

Here, the frame Y ′ (t) at time t, which has been subjected to the fade processing, is the frame to be coded, and is at time t−1.
And the same fade-processed two frames Y ′ at time t-2
(t-1) and Y '(t-2) are assumed to be reference frames.

First, consider the case where a predicted image P (t) is generated from the average value of these two frames as shown in equation (3). P (t) = {Y '(t-1) + Y' (t-2)} / 2 (3) Here, the fade period of (a) of Formula (1) and (b) of Formula (2) is Considering that, each of the prediction images of Equation (3) is
It is shown by Formula (4) and Formula (5). Now, it is assumed that there is no time variation of the original signal Y (t) before fading, that is, Y (t) is constant regardless of t, and Y (t) = C
Assuming (constant), Equations (4) and (5) become Equations (6) and (7), respectively. P (t) = C × (2t−3) / 2T (6) P (t) = C × (2T−2t + 3 + 2T ₀ ) / 2T (7) On the other hand, the signal Y ′ (t) to be encoded Is expressed by the equations (8) and (9). Y '(t) = C × t / T (8) Y' (t) = C × (T-t + T 0) / T (9) Equation (8) and Equation (9) Y '(t) From, formula (6)
And the prediction error signal D (t) obtained by subtracting the prediction image P (t) of Expression (7) are Expression (10) and Expression (11), respectively. D (t) = C × 3 / 2T (10) D (t) = − C × 3 / 2T (11) On the other hand, according to the third and fourth features of the invention,
The predicted image P (t) shown in 2) is generated. P (t) = 2 × Y ′ (t-1) −Y ′ (t-2) (12) Assuming that Y (t) = C (constant) as above, the fade-in and Predicted images at the time of fading out of Expression (2) are Expression (13) and Expression (14), respectively.
Indicated by. P (t) = C × t / T (13) P (t) = C × (T−t + T ₀ ) / T (14) Formulas (13) and (14) are represented by Formula (8) and Formula (8). The prediction error signal D (t) obtained by subtracting the predicted image from the coded image matches with the image to be coded shown in (9) and is 0 in any case. As described above, in a faded image, a residual signal is generated by conventional motion compensation such as MPEG, but according to the third and fourth characteristics of the present invention, the residual signal is eliminated and the prediction efficiency is improved. It can be seen that it will be greatly improved.

1 / T in the equations (1) and (2) indicates the speed of change over time of fade-in and fade-out, and from the equations (10) and (11), in the conventional motion compensation, It can be seen that the prediction residual increases and the coding efficiency decreases as the fade changing speed increases. On the other hand, according to the third and fourth characteristics of the present invention, it is possible to obtain a high prediction efficiency regardless of the fade changing speed.

In the present invention, in addition to the first and third features of the present invention, the encoded motion vector is a motion vector relating to a specific one reference frame among the plurality of reference frames. Is the fifth feature.

In addition to the second and fourth characteristics of the present invention, the received motion vector data is a motion vector relating to a specific one reference frame among the plurality of reference frames, and the motion vector A sixth feature is that the data is scale-converted according to the inter-frame distance between the decoding target frame and the reference frame to generate a motion vector for another reference frame.

According to the first to fourth inventions of the present invention, it is possible to obtain a higher prediction efficiency for a faded image or the like by using a plurality of reference images. However, for each encoded macroblock, if the motion vectors for a plurality of reference images are individually multiplexed into the encoded data,
Coding overhead becomes large. ITU-TH.
In an encoding method such as H.263, the motion vector of the B picture is not sent, and the motion vector of the P picture straddling the B picture is scaled according to the inter-frame distance between the reference image and the image to be coded to obtain the B picture of the B picture. There is an encoding method called a direct mode that uses a motion vector. This is a model that approximates that the moving image to be encoded has a substantially constant or stationary motion speed when viewed in a short time of several frames. In many cases, it is effective to reduce the motion vector code amount. is there.

According to the fifth and sixth characteristics of the present invention, B
Similar to the picture direct mode, in the P picture, only one motion vector among the motion vectors for a plurality of reference frames is coded, and the decoding side receives the received motion according to the inter-frame distance from the reference image. The vector can be scaled and used, and the improvement of the coding efficiency according to the first to fourth aspects of the present invention can be realized without increasing the coding overhead.

In the present invention, in addition to the fifth feature of the present invention, the motion vector relating to the specific one reference frame is normalized according to the inter-frame distance between the reference frame and the frame to be encoded. The seventh feature is that these are motion vectors.

In addition to the sixth feature of the present invention, the motion vector relating to the one specific received reference frame is normalized according to the inter-frame distance between the reference frame and the frame to be encoded. The eighth feature is that it is a motion vector.

According to the seventh and eighth characteristics of the present invention, the reference scale of the motion vector to be coded becomes constant even if the inter-frame distance changes, and the motion vector scaling processing for each of the plurality of reference frames is performed by the reference frame. It is possible to perform the calculation only with the information on the inter-frame distance between the frame and the frame to be encoded. Also, division is required to perform arbitrary scaling, but since the motion vector to be encoded is normalized by the inter-frame distance, scaling processing can be realized only by multiplication. It is possible to reduce the cost of encoding and encoding.

In the present invention, in addition to the first and third features of the present invention, the encoded motion vector is a first motion vector related to a specific one reference frame among the plurality of reference frames. And a plurality of motion vectors for other plurality of reference frames, wherein the plurality of motion vectors represent the first motion vector according to the inter-frame distance between the encoding target frame and the plurality of reference frames, The ninth feature is that the encoded motion vector is encoded as a difference vector between the motion vector and the plurality of motion vectors.

Further, in addition to the second and fourth characteristics of the present invention, the received motion vector data includes a motion vector related to a specific one reference frame among the plurality of reference frames and another reference frame. Of the plurality of reference frames by performing scale conversion of the motion vector data according to the inter-frame distance between the decoding target frame and the reference frame and adding the difference vector. The tenth feature is that a motion vector related to a reference frame other than one frame is generated.

According to the fifth and sixth features of the present invention, a plurality of reference frames can be displayed without increasing the coding overhead of the motion vector information in the case of a still image or a video having a constant motion speed. It is possible to improve the prediction efficiency by using this. However, if the speed of motion is not constant, simple prediction of the motion vector may not provide sufficient prediction efficiency.

On the other hand, in the dual-prime prediction, which is one prediction mode of the MPEG2 moving picture coding, the motion prediction using two consecutive fields
The configuration is such that a difference vector between a motion vector for one field, a motion vector obtained by scaling the motion vector according to an inter-field distance, and a motion vector for the other field is encoded. The motion vector is represented with 1/2 pixel precision, which gives 2
The averaging of the reference macroblocks in the field brings about a loop filter effect by an adaptive spatiotemporal filter and makes it possible to suppress an increase in coding overhead, which greatly contributes to improvement in coding efficiency.

According to the ninth and tenth features of the present invention,
In addition to the loop filter effect by the adaptive spatiotemporal filter that suppresses the increase in coding overhead similar to dual-prime prediction, it is possible to further improve the prediction efficiency for faded images, etc. It is possible to obtain high coding efficiency.

In the present invention, the first, the third, the fifth, the
In addition to the seventh and ninth characteristics, the prediction mode includes a first flag indicating whether the prediction uses one specific reference frame or a plurality of reference frames, and the plurality of prediction modes. Of the reference frame is composed of a second flag indicating whether the prediction is based on the average value of the plurality of reference macroblocks or the linear extrapolation or linear interpolation of the plurality of reference macroblocks. The eleventh feature is that the flag is included in the header data of the encoded frame or the header data for a plurality of encoded frames.

In addition to the second, fourth, sixth, eighth and tenth characteristics of the present invention, the received prediction mode is
A first indicating either prediction using one specific reference frame or prediction using a plurality of reference frames
Flag and the prediction using the plurality of reference frames is a prediction based on an average value of a plurality of reference macroblocks,
Alternatively, it is composed of a second flag indicating whether the prediction is performed by linear extrapolation or linear interpolation of a plurality of reference macroblocks, and the second flag is a part of header data of a coded frame or header data for a plurality of coded frame groups. The twelfth feature is that it is received as.

As described above, according to the present invention, a predicted macroblock signal is generated from only a specific reference frame for each macroblock of a coded frame among a plurality of reference frames, or a plurality of reference images are generated. By adaptively switching between generating the prediction macroblock signal from the average value or generating the prediction macroblock signal by linear extrapolation or linear interpolation of multiple reference images, the prediction efficiency is improved and high efficiency is achieved. High quality encoding is possible.

For example, in a video portion in which the background appears and disappears in time in the same frame, it is effective to predict only a specific reference frame among a plurality of reference frames (here, the prediction mode 1). The portion that has little temporal variation is predicted from the average value of a plurality of reference images (here, the prediction mode 2) to obtain a loop filter effect of removing coding distortion in the reference image. When the amplitude of a video signal such as a fade image changes with time, the prediction efficiency can be improved by linear extrapolation or linear interpolation of a plurality of reference images (here, the prediction mode 3). Becomes

Generally, in the conventional coding method, when the optimum prediction mode is selectively switched for each macroblock in this way, a flag indicating the prediction mode is set for each macroblock and header data for each macroblock is set. Are encoded by including. However, when many prediction modes are switched and used, there is a problem that the coding overhead of the flag indicating the prediction mode increases.

According to the eleventh and twelfth features of the present invention, each coded frame is limited to the combination of the prediction mode 1 and the prediction mode 2 or the combination of the prediction mode 1 and the prediction mode 3. A second flag indicating which of the above combinations and a first flag indicating the prediction mode 1 or the prediction mode 2 or the prediction mode 3 are prepared, and the second flag indicating the combination of the prediction modes is The first flag, which is included in the header data of the encoded frame and indicates the prediction mode, can be changed for each macroblock. By including the first flag in the header data of the macroblock, it is possible to reduce the overhead related to the prediction mode in the encoded data. Becomes

When the amplitude of the video signal changes with time as in a faded image, since the amplitude changes uniformly within the frame, it is not necessary to switch between prediction mode 2 and prediction mode 3 for each macroblock, It is fixed for each time and does not cause any decrease in prediction efficiency.

On the other hand, the temporal appearance and obscuration of the background occurs within a frame regardless of the temporal change of the amplitude of the video signal, and if fixed for each frame, the prediction efficiency is lowered. Therefore, it is necessary to switch the optimum prediction mode for each macroblock by the first flag.
Therefore, by separating the flag indicating the prediction mode into the frame header and the macroblock header as described above, it is possible to reduce the coding overhead without lowering the prediction efficiency.

Further, according to the present invention, in motion-compensated prediction interframe coding in which a plurality of moving picture frames are referred to for each macroblock, a prediction macroblock is generated by linear prediction from the plurality of reference frames, and the prediction macroblock is generated. A thirteenth feature is that a prediction error signal and a motion vector of a block and a coding macroblock are coded for each macroblock, and a set of prediction coefficients for the linear prediction is coded for each frame.

In addition to the thirteenth feature of the present invention, the fourteenth feature of the present invention is that the plurality of reference frames are frames temporally past the frame to be coded.

Further, in decoding motion-compensated prediction interframe coded data that refers to a plurality of moving picture frames for each macroblock, the motion vector data and prediction error signal coded for each macroblock, and for each frame. Receives a set of encoded prediction coefficients, generates a prediction macroblock from the plurality of reference frames according to the motion vector and the prediction coefficient, and adds the generated prediction macroblock and the prediction error signal. This is the fifteenth feature of the present invention.

In addition to the 15th feature of the present invention, the 16th feature of the present invention is that the plurality of reference frames are frames temporally past the frame to be coded.

According to the thirteenth to sixteenth features of the present invention, the prediction coefficient in the arbitrary time direction can be set, so that not only when the temporal change of the video signal amplitude in a faded image is constant, but also in the video signal amplitude. For any time fluctuation of, by using an optimal prediction coefficient set on the encoding side, it is possible to improve the prediction efficiency, and by multiplexing and transmitting the prediction coefficient to encoded data, The same linear prediction as that at the time of encoding can be performed at the time of decoding, and highly efficient predictive encoding can be performed.

According to the present invention, it is possible to improve the coding efficiency by predicting from a plurality of reference frames, but the reference frame is derived from a frame preceding and succeeding in time like a B picture in MPEG. The prediction may be configured to refer to a plurality of past and future frames temporally. Further, as in the case of only the I picture and P picture of MPEG, a configuration in which only a temporally past frame is referred to may be a configuration in which a plurality of past P pictures and I pictures are used as reference images.

With such a structure, the conventional MP
It is possible to realize higher quality encoding than the EG encoding. In particular, also in the coding of P pictures using only past images, it is possible to significantly improve the coding efficiency as compared to the conventional case by using a plurality of past reference frames unlike the conventional case. In the coding that does not use the B picture, the delay for rearranging the coded frames is not necessary, and the coding with low delay is possible. Therefore, according to the present invention, even in the coding with low delay, it is better than the conventional one. A large improvement in coding efficiency can be obtained.

Furthermore, the present invention provides a moving picture coding method for performing motion compensation predictive interframe coding using at least one reference frame for each coding target block included in a coding target frame of an input moving picture, A first prediction block generation mode for generating a prediction block signal from a single reference frame, and a second prediction block generation mode for generating a prediction block signal by linear sum prediction of a plurality of reference blocks cut out from a plurality of reference frames For each of the coding blocks, a step of coding the difference signal between the selected prediction block signal and the signal of the block to be coded, linear sum prediction the average value prediction of a plurality of reference blocks Or linear interpolation prediction based on the display times of a plurality of reference frames and encoding target frames For each of a plurality of pixel blocks in the encoding target frame or for each encoding target frame, and a first indicating whether the first or second prediction block generation mode is selected when generating the prediction block signal. Encoding the encoding mode information for each encoding target block or for each of a plurality of encoding target blocks,
Encoding the second encoding mode information indicating which of the mean value prediction and the linear interpolation prediction is selected for the linear sum prediction for each of a plurality of pixel blocks of the encoding target frame or for each encoding target frame. It is characterized by having.

On the other hand, in the moving picture decoding method for performing motion compensation inter prediction frame decoding using at least one reference frame for each decoding target block included in the decoding target frame of the moving picture, the prediction block signal A step of decoding a prediction error signal of a signal of a block to be decoded, a first prediction block generation mode for generating a prediction block signal from a single reference frame when generating a prediction block signal on the encoding side, and a plurality of The first coding mode information indicating which of the second prediction block generation modes for generating a prediction block signal by linear sum prediction of a plurality of reference blocks cut out from the reference frame Multiple steps for receiving and decoding for each of multiple blocks to be decoded, and multiple references for linear sum prediction The second coding mode information indicating which of the lock average value prediction or the linear interpolation prediction based on the display times of the plurality of reference frames and the coding target frame is selected is set for each of the plurality of pixel blocks of the decoding target frame. Alternatively, the step of receiving and decoding for each frame to be decoded, the step of generating a prediction block signal according to the decoded first coding mode information and the second coding mode information, and the generated prediction block Generating a reproduced video signal using the signal and the decoded prediction error signal.

[0050]

1 is a block diagram of a moving picture coding method according to an embodiment of the present invention. For the input moving image signal 100, the frame stored in the first reference frame memory 117 and the second reference frame memory 118
The predicted macroblock generation means 119 generates a predicted image from the frame stored in the prediction macroblock selection means 119, and the prediction macroblock selection means 120 selects the optimum prediction macroblock to obtain the prediction error signal 101 between the input signal and the prediction signal. On the other hand, DCT transform (discrete cosine transform) 112, quantization 1
13 and variable length coding 114 are performed, and coded data 1
02 is output. The encoded data 102 to be output includes
The motion vector information, which will be described later, is also encoded and output together with the prediction mode information. The signal quantized by the quantizing means 113 is inversely quantized by the inverse quantizing means 115, added with the prediction signal 106 to generate the local decoded image 103, and written into the reference frame memory 117.

In this embodiment, the prediction error signal 101 is D
Although it is encoded by CT transform, quantization, and variable length coding, for example, the DCT transform may be replaced by a wavelet transform, or the variable length coding may be replaced by an arithmetic coding.

In this embodiment, the first reference frame memory 117 stores the locally decoded image of the frame encoded immediately before, and the second reference frame memory 1 is stored.
In 18, the locally decoded image of the previously encoded frame is stored. In the prediction macroblock generation means 119, a prediction macroblock signal 130 generated only from the image of the first reference frame memory 117, and a prediction macroblock signal 131 generated only from the image of the second reference frame memory 118,
From the predicted macroblock signal 132 obtained by averaging the reference macroblock signals cut out from the first and second reference frame memories, and the signal obtained by doubling the amplitude of the reference macroblock signals cut out from the first reference frame memory 117. , A prediction macroblock signal 133 is generated by subtracting the reference macroblock signal cut out from the second reference frame memory 118. These predicted macroblock signals are cut out from multiple positions in the frame,
Generate a plurality of predicted macroblock signals.

In the predictive macroblock selecting means 120,
For the plurality of prediction macroblock signals generated by the prediction macroblock generation means 119, the input moving image signal 100 is input.
The difference from the encoding target macroblock signal cut out from is calculated, and the prediction macroblock having the minimum error is selected for each encoding macroblock. The relative position of the selected prediction macroblock viewed from the encoding target macroblock is used as a motion vector, and the selected prediction macroblock generation method (from 130 to 1 in FIG. 1).
Any one of 33) is coded for each macroblock as a prediction mode.

Here, when the moving image signal is composed of luminance and color difference signals, the same motion vector and prediction mode are applied to the respective signal components of each macroblock to generate a prediction error signal. To do.

FIG. 2 is a block diagram of a moving picture decoding method according to the embodiment of the present invention. The moving picture decoding method of FIG. 2 is a method of inputting and decoding the coded data coded by the moving picture coding method according to the embodiment of the present invention shown in FIG.

The variable length code of the input coded data 200 is decoded by the variable length code decoding means 214, and the prediction error signal 201, motion vector information and prediction mode information 202 are extracted. The prediction error signal 201 is subjected to inverse quantization 215 and inverse DCT 216 and added with the prediction signal 206 to generate a decoded image 203.

The decoded image 203 is written in the first reference frame memory 217. The prediction signal 206 is the second reference frame memory 218 in which the immediately preceding decoded image signal recorded in the first reference frame memory 217 and the moving image signal decoded earlier than that are recorded.
Prediction macroblock generation means 219 and prediction macroblock selection means 220 according to the motion vector extracted from the encoded data 200 and the prediction mode
By this, the same prediction signal as the prediction macroblock signal used at the time of encoding is generated.

FIG. 3 is an example schematically showing the relationship of inter-frame prediction according to the embodiment of the present invention. In the figure, 302 is a frame to be encoded, 301 is the frame immediately before that, and 300 is the frame before that. When encoding or decoding frame 302,
The decoded image of the frame 301 is stored in the first reference frame memory 117 of FIG. 1 or 217 of FIG. 2 and the second reference frame memory of 118 of FIG. 1 or 218 of FIG.
The frame 300 is stored in the reference frame memory of.

Reference numeral 305 in FIG. 3 denotes a macroblock to be encoded, and the reference macroblock 303 of the reference frame 300 and the reference macroblock 3 of the reference frame 301.
A prediction macroblock is generated by using either or both of 04. In the figure, 306 and 307 are motion vectors indicating the positions of the reference macroblocks 303 and 304, respectively. At the time of encoding, the encoding macroblock 30
The optimum motion vector and prediction mode for 5 are searched. In addition, at the time of decoding, a prediction macroblock signal is generated using the motion vector and the prediction mode included in the encoded data.

FIG. 4 and FIG. 5 relate to the embodiment of the present invention and show examples of inter-frame prediction relationships using three or more reference frames. 4 shows an example using a plurality of past reference frames, and FIG. 5 shows an example using a plurality of past and future reference frames.

In FIG. 4, reference numeral 404 denotes a frame to be encoded, and 400 to 403 are reference frames for the frame 404. Reference numeral 413 denotes a coded macroblock. In coding, for each coded macroblock, from each reference frame, in accordance with the motion vector (405 to 408 in the figure) for each reference frame, the reference macroblock (see FIG. Cut out 409 to 412),
A prediction macroblock is generated by linear prediction from a plurality of reference macroblocks. Next, a set of a motion vector and a prediction mode that minimizes a prediction error is selected in one of the plurality of reference macroblocks or a prediction mode of a prediction macroblock by linear prediction. One set of linear prediction coefficients is determined for each coded frame, for example, from changes in average luminance between frames over time. The determined set of prediction coefficients is coded as header data of a coded frame, and the motion vector, prediction mode, and prediction error signal of each macroblock are coded for each macroblock.

Further, at the time of decoding, a prediction macroblock is generated from a plurality of reference frames from the motion vector and prediction mode information for each macroblock, using a set of linear prediction coefficients received for each frame, Decoding is performed by adding the prediction error signal.

In FIG. 5, reference numeral 502 indicates an encoding target frame, and reference numerals 500, 501, 503 and 504 indicate reference frames. In the case of FIG. 5, the frames are rearranged in the order of 500, 501, 503, 504, 502 at the time of encoding and decoding, and in the case of encoding, a plurality of locally decoded frames and decoding are performed. In the case of 1, a plurality of already decoded frames are used as reference frames. Encoding target macroblock 511
On the other hand, similarly to the example of FIG.
One of 9, 510, 512, and 513, or any one of prediction signals by linear prediction from them, is selected and encoded for each macroblock.

FIG. 6 is a diagram showing an encoding method and a decoding method of motion vector information according to the embodiment of the present invention.
As in the example of FIG. 3, in inter-frame coding using a plurality of reference frames, when a prediction macroblock signal is generated using a plurality of reference macroblock signals for each coding macroblock, a plurality of macroblocks are generated for each macroblock. It is necessary to encode the motion vector information of Therefore, as the number of macroblocks to be referred to increases, the overhead of motion vector information to be coded increases, which causes a drop in coding efficiency. In the example of FIG. 6, reference macroblock signals are cut out from two reference frames,
It shows an example in which one motion vector and a vector obtained by scaling the motion vector according to the inter-frame distance are used when generating a prediction macroblock signal.

In the figure, reference numeral 602 is a frame to be encoded,
Reference frames 601 and 600 are reference frames. Also, 611
And 610 indicate the motion vector. The points shown in black indicate the pixel positions in the vertical direction, and the points shown in white indicate the interpolation points with 1/4 pixel accuracy. FIG. 6 shows an example in which motion compensation prediction is performed with 1/4 pixel accuracy. The pixel accuracy of motion compensation is 1 pixel, 1/2 pixel, 1/8 pixel,
It is defined for each encoding method. Generally, a motion vector is expressed with a precision of motion compensation, and a reference image is generally generated by interpolation from image data of a reference frame.

In FIG. 6, focusing on the pixel 605 to be encoded, it is assumed that a point 603 that is 2.5 pixels vertically away from the reference frame 600 is referred to, and a motion vector indicating a shift of 2.5 pixels is displayed. 610 is encoded. on the other hand,
The prediction from the reference frame 601 for the same pixel 605 is generated by scaling the above-described encoded motion vector 610 according to the inter-frame distance. Here, the motion vector for the frame 601 is 2.5 / 2 = 1.25 pixels in consideration of the inter-frame distance, and the pixel 604 in the reference frame 601 is used as the reference pixel of the pixel 605 in the encoded frame 602.

The motion vector to be coded for each macroblock is 1 even when the macroblock to be coded refers to a plurality of frames by scaling the motion vector with the same accuracy at the time of coding and decoding. Only one motion vector is needed, and it is possible to prevent an increase in coding overhead. Here, if the scaling result of the motion vector is not on the sampling point of the motion compensation accuracy,
The scaled motion vector shall be rounded by rounding off fractions.

FIG. 7 is a diagram showing an encoding method and a decoding method of motion vector information different from FIG. 6 according to the embodiment of the present invention. In the example of FIG. 6, when the temporal motion speed of the moving image is constant, the overhead of the motion vector in the encoded data can be efficiently reduced. On the other hand, when the temporal motion of a moving image is monotonous but the motion speed is not constant, using a simply scaled motion vector causes a decrease in prediction efficiency and causes a decrease in coding efficiency. May be In FIG. 7, FIG.
Similarly, as the reference pixel of the pixel 506, a prediction pixel is generated from the reference pixels of the two frames of the reference frames 700 and 701. Here, the pixel 703 of the frame 700
Then, the pixel 705 of the frame 701 is referred to.

Similar to the example of FIG. 6, the motion vector 710 for the frame 700 is encoded, and in addition, the motion vector 711 for the frame 701 is encoded as a difference vector 720 with respect to the scaled vector of the motion vector 710. It Set motion vector 710 to 1
By scaling to / 2, the position of the pixel 704 in the frame 701 is shown, and the predicted pixel 70
5 and the pixel 704, the difference vector 720 which shows the difference amount is encoded. Normally, the magnitude of the above-mentioned difference vector becomes smaller for a temporally monotonous motion, so even if the motion speed is not constant, the prediction efficiency is not reduced and the increase in motion vector overhead is suppressed. As a result, efficient encoding can be performed.

FIG. 8 is a diagram showing still another motion vector information encoding method and decoding method according to the embodiment of the present invention. In the example of FIG. 8, the frame 803 is an encoding target frame, the frame 602 is skipped, and the frames 801 and 800 are reference frames. Further, with respect to the pixel 806, the pixel 804 of the reference frame 800 and the pixel 8 of the reference frame 801
Reference numeral 05 is a reference pixel for generating a predicted pixel.

Similar to the example of FIG. 6 or 7, it is also possible to generate the motion vector for the reference frame 801 by encoding the motion vector 811 for the reference frame 800 and using the motion vector obtained by scaling the motion vector 811. However, in the case of FIG. 8, scaling of 2/3 times is required for the motion vector 811 due to the relationship between the inter-frame distances of the reference frame and the encoded frame. Not limited to the example of FIG. 8, in order to perform arbitrary scaling, the denominator becomes an arbitrary integer that is not a power of 2, and division is necessary. Motion vector scaling is necessary both at the time of encoding and decoding, and especially division is
In both hardware and software, the cost and the calculation time are long, resulting in an increase in the cost of encoding and decoding.

On the other hand, in FIG. 8, the motion vector 811 to be encoded is normalized by the inter-frame distance.
10 is encoded, and the normalized motion vector 810 is encoded according to the inter-frame distance between the encoded frame and each reference frame, and the difference vector between the scaled motion vector and the original motion vector is encoded. . That is, the reference pixel 804 is the normalized motion vector 81.
It is generated from a motion vector obtained by multiplying 0 by 3 and a difference vector 820, and the reference pixel 805 is a motion vector obtained by doubling the normalized motion vector 810 and a difference vector 82.
It is generated from 1. The configuration of FIG. 8 prevents an increase in the coding overhead of the motion vector without lowering the prediction efficiency, and further, the scaling of the motion vector can be realized only by multiplication, so that the calculation cost of the coding and the decoding is increased. It becomes possible to suppress.

FIG. 9 is a block diagram of a moving picture coding method according to the second embodiment of the present invention. The second embodiment of the present invention is an embodiment using the eleventh and twelfth features of the present invention described above, and is applied to the input image 900 with respect to the moving picture coding embodiment shown in FIG. This is a configuration in which the fade detection means 900 is added. The fade detection unit 900 calculates the average luminance value of each frame of the input moving image signal, and if the temporal change in luminance has a certain slope, determines that the image is a faded image, and the result 701 is the prediction mode. Notify the selection means 120.

If the fade detection means 900 determines that the image is a fade image, the prediction mode is limited to either prediction from one reference frame or linear extrapolation or linear interpolation of a plurality of reference frames. Then, the optimum motion vector and prediction mode are determined for each macroblock. The determined motion vector and the first flag indicating the prediction mode are written in the header of the macroblock, and the prediction error signal is encoded. A second flag indicating a set of possible prediction modes is written in the header data of the frame and output.

If the fade detection means 900 determines that the image is not a faded image, the prediction mode is limited to either prediction from one reference frame or prediction based on the average value of a plurality of reference frames, and similarly the optimum Different motion vectors and prediction modes are determined, and motion vectors, prediction modes, and prediction signals are similarly encoded.

When the coded data coded with the configuration of FIG. 9 is received and decoded, the first mode indicating the prediction mode is used.
And the second flag to determine the prediction mode for each macroblock, generate a prediction macroblock signal from the motion vector sent for each macroblock and the determined prediction mode, and decode the encoded prediction error signal. Then, decoding is performed by adding the prediction signal. With such a configuration, as described in the eleventh and twelfth aspects of the present invention, it becomes possible to reduce the coding overhead of the prediction mode information.

Next, the procedure of moving picture coding according to the third embodiment of the present invention will be described with reference to FIG. Video frames to be encoded are input one frame at a time,
A fade image is detected for the entire frame or for each slice formed of a plurality of pixel blocks in the frame based on the temporal change of the average value of the luminance values within the frame (step S1). For each pixel block in the frame,
One optimum reference frame is selected from a plurality of reference frames to generate a prediction pixel block signal (single frame prediction), or a prediction pixel block is generated by prediction using a linear sum of two reference pixel block signals. Yes (linear sum prediction) is selected.

In the linear sum prediction, when the input moving image is detected to be a faded image, temporal linear interpolation (interpolation or extrapolation based on the time distance between frames) prediction is performed, and when it is not a faded image, 2 is used. A prediction pixel block is generated by the average value of two reference pixel block signals. The second coding mode information indicating whether the linear sum prediction using a plurality of frames is the average value prediction or the temporal linear interpolation prediction is coded as frame (picture) or slice header data (step S2). ).

Next, the detection result of step S1 is examined in step S3 to determine whether or not the input moving image is a fade image. Here, when the input moving image is determined to be a fade image, an encoding mode (step S5) for selecting a single prediction block from a plurality of reference frames for each pixel block and an encoding mode by temporal linear interpolation prediction Of (step S4), the coding mode having the higher coding efficiency, that is, the one having the smaller generated code amount is determined (step S8).

After this, the first coding mode information indicating single-frame prediction or linear sum prediction, and information on other selected coding modes (identification information of reference frame used for prediction, motion vector information) Etc.) are encoded in the macroblock header area (step S10).
Finally, the difference signal (prediction error signal) between the selected prediction block signal and the signal of the encoding target block is encoded (step S11), and each encoded data is output.

On the other hand, if the result of determination in step S3 is that the input moving image is not a faded image, the single-frame prediction mode (step S6) or average value prediction mode (step S7) is the most suitable encoding mode. Is selected (step S9). Hereinafter, similarly, the encoding of the information regarding the encoding mode (step S10) and the encoding of the difference signal (step S11) are performed.

According to the result of the fade detection in step S1, each block in the frame or slice is encoded as described above, and one frame (picture) or one block is encoded.
When the coding of all pixel blocks in the slice is completed (step S12), the fade of the frame or slice to be coded next is detected (step S1), and the coding is similarly advanced. In the above description, an example is shown in which one frame is coded as one picture, but one field may be coded as one picture.

11 and 12 are views showing the data structure of the moving image coded data according to the present embodiment. Figure 11
Shows a part of the data structure including the header data of the picture or slice, and FIG. 12 shows a part of the macroblock data. In the header area of the picture or slice, the second encoding indicating information “time_info_to_be_displayed” regarding the display time of the encoding target frame and whether the above-described linear sum prediction is temporal linear interpolation prediction or average value prediction. Flag “linear_w” that is mode information
The eighted_prediction_flag ”is encoded.
When the “weighted_prediction_flag” is 0, it means the average value prediction, and when it is 1, it means the time linear interpolation prediction.

A plurality of macroblock coded data are included in the coded data of a picture or slice, and each macroblock data has a structure as shown in FIG. In the header area of the macroblock data, together with reference frame selection information, motion vector information, and the like, information indicating whether single frame prediction from a selected single frame or prediction by a linear sum from a plurality of frames (first (1 encoding mode information) is "macroblo
ck_type ”.

FIG. 13 schematically shows the structure of the entire time series of moving image coded data including the data structure shown in FIGS. 11 and 12. At the beginning of the encoded data,
Information of a plurality of coding parameters, such as image size, which is invariable in one coding sequence as a whole, is used as a sequence header (S
H).

Next, each image frame or field is coded as a picture. Each picture has a picture header (PH) and picture data (Picture
data) are sequentially encoded as a set. In the picture header (PH), information “time_info_to_be_displayed” related to the display time of the encoding target frame shown in FIG.
And the second coding mode information “linear_weighted_predic
tion_flag "is encoded as DTI and LWP, respectively. The picture data is divided into one or a plurality of slices (SLC), and is sequentially encoded for each slice.

In each slice, the SLC first encodes the encoding parameters for each pixel block in the slice as a slice header (SH), and follows the slice header SH, one or more macroblock data (M).
B) is sequentially encoded. Macroblock data M
In B, the first coding mode information “macroblock_type” shown in FIG. 12 is coded as MBT, and information about coding of each pixel in a macroblock, such as motion vector information (MV). Is encoded, and finally, an orthogonal transform coefficient (DCT) encoded by performing orthogonal transform (for example, discrete cosine transform) on the pixel signal or the prediction error signal to be encoded is included.

Here, the second coding mode information "linear_weighted_predicti" included in the picture header PH is used.
The on_flag ”may be configured to be encoded for each slice using the slice header SH.

Next, the procedure of moving picture decoding in this embodiment will be described with reference to FIG. In the present embodiment, the encoded data encoded by the moving image encoding method shown in FIG. 10 and having the data structure shown in FIGS. 11 and 12 is input and decoded. From the input encoded data, the header information of the picture or slice is decoded, and information about the display time of the encoding target frame "time_info_to_be_displayed",
Second coding mode information "linear_weighted_prediction
_flag "is decrypted (step S30).

Further, the header information of the macroblock is decoded for each macroblock in the picture or slice, and "macroblock_typ" including the identification information of the reference frame, the motion vector information, and the first coding mode information.
Decoding such as "" is performed (step S31).

When the decoded first coding mode information indicates single frame prediction, a prediction block signal is generated according to the prediction mode information such as reference frame identification information and motion vector information (step S34). ).
When the first coding mode information indicates prediction by a linear sum from a plurality of frames, the average prediction (step 35) or the time linear interpolation prediction is performed according to the decoded second coding mode information (step S33). (Step S3
A prediction signal is generated by any of the methods of 6).

Next, the coded prediction error signal is decoded and added with the generated prediction signal to generate a decoded image (step S37). For each macroblock in the picture or slice, decoding is sequentially performed from each macroblock header, and when decoding of all macroblocks in the picture or slice is completed (step S38), the decoding is continued from the next picture or slice header. Proceed with decryption.

As described above, in the present embodiment, the first coding mode information indicating whether the information on the coding mode is the single frame prediction or the prediction by the linear sum from a plurality of frames, and the prediction by the linear sum are time linear. Separation into second coding mode information indicating interpolation prediction or average prediction, and coding the first coding mode information for each macroblock and the second coding mode information for each picture or slice. Thus, it is possible to reduce the overhead of encoding the encoding mode information while maintaining the encoding efficiency.

That is, since the second coding mode information indicates a wide area characteristic in a frame such as a fade image, the second coding mode information is coded for each slice or frame, and the macro As compared with the case of coding for each block, the code amount for coding the coding mode information itself can be suppressed, and the coding efficiency does not significantly decrease.

On the other hand, the first coding mode information is coded for each macro block, so that the first coding mode information can be processed according to the individual characteristics of each pixel block (for example, a video image that is partially visible or hidden in time). It is possible to determine an appropriate mode, and it is possible to further improve coding efficiency.

As described above, in the present embodiment, the coding frequencies of the first and second coding modes are determined in consideration of the characteristics of the moving image, so that high-quality and high-quality coding is performed. It becomes possible.

Next, the temporal linear interpolation prediction in this embodiment will be described in detail with reference to FIGS. F0, F1, F2 in FIG. 15 and F0, F2, F1 in FIG. 16 indicate frames that are temporally continuous. In FIGS. 15 and 16, F2 indicates a frame to be encoded or decoded, and F0 and F1 indicate reference frames. Here, with respect to the examples of FIGS. 15 and 16, let us consider a case where a certain pixel block in a frame to be encoded or decoded performs prediction by a linear sum of two reference frames.

When the linear sum prediction is the average value prediction, the prediction pixel block is generated by the simple average of the reference blocks cut out from each reference frame. The reference pixel block signals cut out from the frames F0 and F1 are respectively ref
When 0 and ref1 are set, the respective predicted pixel block signals pred2 in FIGS. 15 and 16 are calculated according to the following mathematical expression (15). pred2 = (ref0 + ref1) / 2 (15) On the other hand, in the case of temporal linear interpolation prediction, a linear sum is calculated according to the time distance between the encoding or decoding target frame and each reference frame. As shown in FIG. 11, the information “time_info_to_be_displayed” regarding the display time in the picture or slice header area for each encoding target frame
Is encoded, and the display time of each frame is calculated based on this information at the time of decoding. Here, the display times of the frames F0, F1, and F2 are Dt0, Dt1, and
Let it be Dt2.

In the example of FIG. 15, since the current frame is predicted from the past two frames, linear extrapolation prediction is performed.
In the example of 6, the linear interpolation prediction is performed from the future and past frames. 15 and 16, where Rr is the time distance between two reference frames and Rc is the time distance from the reference frame that is the earliest in time with respect to the encoding target frame to the encoding target frame, Like Rr = Dt1-Dt0, Rc = Dt2-Dt0 (16) The linear extrapolation prediction and the linear interpolation prediction based on the time distance are calculated by the following formula (17) in both cases of FIG. 15 and FIG. To be done. pred2 = {(Rr-Rc) * ref0 + Rc * ref1} / Rr (17) Further, the equation (17) can be transformed into the equation (18). Pred2 = ref0 + (ref1-ref0) * Rc / Rr (18) For an image in which the signal amplitude monotonously changes between frames, such as a fade image or cross-fade image, for a very short time (for example, three consecutive frames). In (), it is possible to make a first-order approximation of the time variation of the signal amplitude. Therefore, by performing the time linear interpolation (linear extrapolation or linear interpolation) according to the time distance between the frame to be encoded and the two reference frames as in the present embodiment, a more accurate predicted image can be obtained. It becomes possible to generate. as a result,
The efficiency of inter-frame prediction is improved, without degrading the image quality,
It is possible to further reduce the amount of generated code or perform higher quality coding at the same bit rate.

The above-described encoding and decoding processing of the present invention may be realized by hardware, or part or all of the processing may be executed by software using a computer. Therefore, according to the present invention, for example, the following program can be provided.

(1) In a program for causing a computer to perform a motion-compensated prediction interframe coding process for referring to a plurality of moving picture frames for each macroblock, a plurality of reference macroblocks are generated from the plurality of reference frames. Processing and one of the plurality of reference macroblocks
A process of selecting an average value of the plurality of reference macroblocks or a linear extrapolation prediction or a linear interpolation prediction by the plurality of reference macroblocks as a prediction macroblock; and the selected prediction macroblock, A program for causing the computer to perform a process of encoding a prediction error signal with a coding macroblock, prediction mode information, and a motion vector.

(2) In a program for causing a computer to perform decoding processing of motion-compensated prediction interframe coded data that refers to a plurality of moving image frames for each macroblock, coded motion vector data and prediction mode A process of receiving information and a prediction error signal, and depending on the received motion vector and the prediction mode,
(A) generate a prediction macroblock from a specific one frame of the plurality of reference frames, or (b) generate a plurality of reference macroblocks from the plurality of reference frames and calculate an average value of the plurality of reference macroblocks. Or (c) a process of selecting whether to generate a prediction macroblock from linear extrapolation prediction or linear interpolation prediction using the plurality of reference macroblocks, and (c) the generated prediction A program for causing the computer to perform a process of adding a macroblock and the prediction error signal.

(3) In a program for causing a computer to perform a motion-compensated prediction interframe coding process for referring to a plurality of moving picture frames for each macroblock, a prediction macroblock is linearly predicted from the plurality of reference frames. A process of generating, a process of coding the prediction error signal and the motion vector of the prediction macroblock and the coding macroblock for each macroblock, and a process of coding the set of prediction coefficients of the linear prediction for each frame. And a program for causing the computer to perform.

(4) In a program for causing a computer to perform decoding processing of motion-compensated prediction interframe coded data that refers to a plurality of moving picture frames for each macroblock, a motion vector coded for each macroblock A process of receiving a set of data and a prediction error signal and a prediction coefficient coded for each frame, and a process of generating a prediction macroblock from the plurality of reference frames according to the received motion vector and prediction coefficient. And a program for causing the computer to perform the process of adding the generated prediction macroblock and the prediction error signal.

[0105]

As described above, according to the present invention,
For video such as fade-in / fade-out, which was not good at conventional moving picture coding methods such as MPEG, prediction efficiency can be improved without requiring a large increase in the amount of calculation and cost of coding and decoding. It is possible to greatly improve it, and also to provide a moving image encoding and decoding system with high image quality and high efficiency, which has a small overhead of encoded data.

[Brief description of drawings]

FIG. 1 is a block diagram of a moving picture coding method according to an embodiment of the present invention.

FIG. 2 is a block diagram of a moving picture decoding method according to an embodiment of the present invention.

FIG. 3 is a diagram showing a relationship of inter-frame prediction according to the embodiment of the present invention.

FIG. 4 is a diagram showing a relationship of inter-frame prediction according to the embodiment of the present invention.

FIG. 5 is a diagram showing a relationship of inter-frame prediction according to the embodiment of the present invention.

FIG. 6 is a diagram showing an example of an encoding method and a decoding method of motion vector information according to the embodiment of the present invention.

FIG. 7 is a diagram showing an example of an encoding method and a decoding method of motion vector information according to the embodiment of the present invention.

FIG. 8 is a diagram showing an example of an encoding method and a decoding method of motion vector information according to the embodiment of the present invention.

FIG. 9 is a block diagram of a moving picture coding method according to an embodiment of the present invention.

FIG. 10 is a flowchart showing a procedure of a moving picture coding method according to the embodiment of the present invention.

FIG. 11 is a diagram showing an example of a data structure of a picture header or slice header of moving image encoded data according to the first embodiment.

FIG. 12 is a diagram showing an example of a data structure of a macroblock of moving image encoded data according to the first embodiment.

FIG. 13 is a diagram showing an example of the data structure of the entire moving image encoded data according to the embodiment of the present invention.

FIG. 14 is a flowchart showing a procedure of a moving picture decoding method according to the same embodiment.

FIG. 15 is a diagram illustrating time linear interpolation in the same embodiment.

FIG. 16 is a diagram illustrating time linear interpolation in the same embodiment.

[Explanation of symbols]

112 ... DCT means 113 ... Quantization means 115, 215 ... Inverse quantization means 116, 216 ... Inverse DCT means 117, 217 ... First reference frame memory 118, 218 ... Second reference frame memory 119, 219 ... Prediction macroblock generation means 120, 220 ... Prediction macroblock selection means 900 ... Fade detecting means

   ─────────────────────────────────────────────────── ─── Continued front page    (72) Inventor Tsuyoshi Nagai             1st Komukai Toshiba-cho, Sachi-ku, Kawasaki-shi, Kanagawa             Inside the Toshiba Research and Development Center (72) Inventor Yoshihiro Kikuchi             1st Komukai Toshiba-cho, Sachi-ku, Kawasaki-shi, Kanagawa             Inside the Toshiba Research and Development Center (72) Inventor Wataru Asano             1st Komukai Toshiba-cho, Sachi-ku, Kawasaki-shi, Kanagawa             Inside the Toshiba Research and Development Center F-term (reference) 5C059 LB11 MA05 MA19 MA23 MC11                       ME01 NN01 NN28 PP04 TA08                       TA29 TA50 TB07 TC00 TC03                       TC12 TD03 UA02 UA05 UA33                 5J064 AA01 BA13 BB03 BC01 BC08                       BC16 BC21 BC23 BC25 BD01

Claims (26)

[Claims] 1. A motion compensation predictive interframe coding method for referring to a plurality of moving picture frames for each macroblock, wherein a plurality of reference macroblocks are generated from the plurality of reference frames and one of the plurality of reference macroblocks is generated. One of the average values of the plurality of reference macroblocks or the linear extrapolation prediction or the linear interpolation prediction by the plurality of reference macroblocks is selected as a prediction macroblock, and the selected prediction macroblock is encoded. A moving picture coding method characterized by coding a prediction error signal with a macroblock, prediction mode information, and a motion vector. 2. The plurality of reference frames are two frames coded immediately before a frame to be coded, and in the linear extrapolation prediction by the plurality of reference macroblocks, the reference macroblock generated from the immediately preceding reference frame. 2. The moving picture according to claim 1, wherein the prediction macroblock is generated by subtracting a reference macroblock signal generated from a reference frame that is one frame before the signal whose signal amplitude is doubled. Encoding method. 3. The moving picture coding method according to claim 1, wherein the motion vector to be coded is a motion vector relating to a specific one reference frame among the plurality of reference frames. . 4. The motion vector for the specific one reference frame is a motion vector normalized according to an inter-frame distance between the reference frame and a frame to be encoded. The described moving picture coding method. 5. The encoded motion vector is a first motion vector related to a specific one reference frame among the plurality of reference frames and a plurality of motion vectors for other plurality of reference frames, The plurality of motion vectors is a difference vector between the motion vector obtained by scaling the first motion vector according to the inter-frame distance between the encoding target frame and the plurality of reference frames, and the plurality of motion vectors. The moving picture coding method according to claim 1, wherein the moving picture coding method is coded. 6. The prediction mode uses a plurality of reference frames, and a first flag indicating either prediction using one specific reference frame or prediction using a plurality of reference frames. The prediction is composed of a second flag indicating whether the prediction is based on an average value of a plurality of reference macroblocks or a prediction based on linear extrapolation or linear interpolation of the plurality of reference macroblocks, and the second flag is a coded frame. 6. The moving picture coding method according to claim 1, wherein the moving picture coding method is included in the header data of 1. or the header data for a plurality of coding frame groups. 7. A method of decoding motion-compensated prediction interframe coded data, which refers to a plurality of moving picture frames for each macroblock, receives coded motion vector data, prediction mode information, and a prediction error signal, Depending on the motion vector and the prediction mode, (a) a prediction macroblock is generated from a specific one frame of the plurality of reference frames, or (b) a plurality of reference macroblocks is generated from a plurality of reference frames. And generate an average value of the plurality of reference macroblocks as a prediction macroblock, or (c) generate a prediction macroblock from either linear extrapolation prediction or linear interpolation prediction by the plurality of reference macroblocks. Whether or not to add the generated prediction macroblock and the prediction error signal. Image decoding method. 8. The plurality of reference frames are two frames decoded immediately before a frame to be coded, and in linear extrapolation prediction using the plurality of reference macroblocks,
It is possible to generate the prediction macroblock by subtracting the reference macroblock signal generated from the reference frame one frame before from the signal obtained by doubling the amplitude of the reference macroblock signal generated from the immediately previous reference frame. The moving picture decoding method according to claim 7, characterized in that 9. The received motion vector data is a motion vector related to a specific one reference frame among the plurality of reference frames, and the motion vector data is an inter-frame distance between a decoding target frame and a reference frame. 9. The moving picture decoding method according to claim 7, wherein the scale conversion is performed according to the above to generate a motion vector for another reference frame. 10. The received motion vector for one specific reference frame is a motion vector normalized according to an inter-frame distance between the reference frame and a frame to be encoded. Item 10. The moving picture decoding method according to Item 9. 11. The received motion vector data is a motion vector related to a specific one reference frame among the plurality of reference frames and a difference vector related to another reference frame, and the motion vector data is to be decoded. Scale conversion is performed according to an inter-frame distance between a frame and a reference frame, and the difference vector is added to generate a motion vector related to a reference frame other than the specific one frame among the plurality of reference frames. The moving picture decoding method according to claim 7 or 8. 12. The received prediction mode includes a first flag indicating either prediction using one specific reference frame or prediction using a plurality of reference frames, and the plurality of reference frames. The used prediction is composed of a second flag indicating whether the prediction is an average value of a plurality of reference macroblocks or a prediction by linear extrapolation or linear interpolation of a plurality of reference macroblocks, and the second flag is encoded. 12. The moving picture decoding method according to claim 7, wherein the moving picture decoding method is received as a part of header data of a frame or header data for a plurality of encoded frame groups. 13. A motion compensation predictive interframe coding method for referring to a plurality of moving picture frames for each macroblock, wherein a predictive macroblock is generated by linear prediction from the plurality of reference frames, and the predictive macroblock and the code are generated. A motion picture coding method, wherein a prediction error signal and a motion vector with a coded macroblock are coded for each macroblock, and the set of prediction coefficients for linear prediction is coded for each frame. 14. The moving picture coding method according to claim 13, wherein the plurality of reference frames are frames temporally past the coding target frame. 15. A method of decoding motion-compensated inter-frame coded data that refers to a plurality of moving picture frames for each macroblock, comprising: motion vector data and prediction error signals coded for each macroblock; A prediction macroblock is generated from the plurality of reference frames according to the motion vector and the prediction coefficient, and the generated prediction macroblock and the prediction error signal are added. A moving picture decoding method characterized by the above. 16. The moving picture decoding method according to claim 15, wherein the plurality of reference frames are frames temporally past the frame to be coded. 17. A motion compensation predictive interframe coding apparatus for referring to a plurality of moving picture frames for each macroblock, a unit for generating a plurality of reference macroblocks from the plurality of reference frames, and the plurality of reference macroblocks. One of the plurality of reference macroblocks, or means for selecting either linear extrapolation prediction or linear interpolation prediction by the plurality of reference macroblocks as a prediction macroblock, and the selected prediction macro A moving picture coding apparatus, comprising: a prediction error signal between a block and a coding macroblock; prediction mode information; and means for coding a motion vector. 18. A device for decoding motion-compensated inter-frame coded data, which refers to a plurality of moving picture frames for each macroblock, means for receiving coded motion vector data, prediction mode information and prediction error signals. And (a) generating a prediction macroblock from a specific one frame of the plurality of reference frames, or (b) making a plurality of references from a plurality of reference frames, according to the received motion vector and the prediction mode. A macroblock is generated and an average value of the plurality of reference macroblocks is generated as a prediction macroblock, or (c) a prediction macro is performed from either linear extrapolation prediction or linear interpolation prediction by the plurality of reference macroblocks. Means for selecting whether to generate a block, and the generated prediction macroblock and the prediction error signal Video decoding apparatus characterized by comprising a means for calculation. 19. A motion compensation predictive interframe coding apparatus for referring to a plurality of moving picture frames for each macroblock, a means for generating a predictive macroblock by linear prediction from the plurality of reference frames, and the predictive macroblock. A moving image comprising means for encoding a prediction error signal and a motion vector of a macroblock and an encoded macroblock for each macroblock, and means for encoding a set of prediction coefficients for the linear prediction for each frame. Image coding device. 20. A motion compensation predictive inter-frame coded data decoding apparatus that refers to a plurality of moving picture frames for each macroblock, wherein motion vector data and prediction error signals coded for each macroblock, and each frame. Means for receiving a set of prediction coefficients coded into, a means for generating a prediction macroblock from the plurality of reference frames according to the received motion vector and prediction coefficient, and the generated prediction macroblock And a means for adding the prediction error signal, the moving picture decoding apparatus. 21. A program for causing a computer to perform a motion-compensated inter-frame coding process for referring to a plurality of moving picture frames for each macro block, the process of generating a plurality of reference macro blocks from the plurality of reference frames. And a process of selecting one of the plurality of reference macroblocks, an average value of the plurality of reference macroblocks, or either linear extrapolation prediction or linear interpolation prediction by the plurality of reference macroblocks as a prediction macroblock. And a program for causing the computer to perform a process of encoding a prediction error signal between the selected prediction macroblock and the coding macroblock, prediction mode information, and a motion vector. 22. A program for causing a computer to perform a decoding process of motion-compensated prediction interframe coded data, which refers to a plurality of moving picture frames for each macroblock, wherein coded motion vector data and prediction mode information are included. And a process of receiving a prediction error signal, and (a) generating a prediction macroblock from a specific one frame of the plurality of reference frames according to the received motion vector and the prediction mode, or (b) A plurality of reference macroblocks are generated from a plurality of reference frames and an average value of the plurality of reference macroblocks is generated as a prediction macroblock, or (c) linear extrapolation prediction or linear intra prediction by the plurality of reference macroblocks. A process of selecting whether to generate a prediction macroblock from any of the insertion predictions; Program for causing a processing for adding the prediction error signal and the prediction macro block to the computer. 23. A program for causing a computer to perform a motion-compensated prediction interframe coding process for referring to a plurality of moving picture frames for each macroblock, wherein a prediction macroblock is generated by linear prediction from the plurality of reference frames. Processing, a process of coding the prediction error signal and the motion vector of the prediction macroblock and the coding macroblock for each macroblock, and a process of coding the set of prediction coefficients of the linear prediction for each frame. A program to be executed by the computer. 24. A program for causing a computer to perform a decoding process of motion-compensated prediction interframe coded data that refers to a plurality of moving picture frames for each macroblock, wherein motion vector data coded for each macroblock. And a process of receiving a prediction error signal and a set of prediction coefficients coded for each frame, and a process of generating a prediction macroblock from the plurality of reference frames according to the received motion vector and prediction coefficient. A program for causing the computer to perform the process of adding the generated prediction macroblock and the prediction error signal. 25. A moving picture coding method for performing motion compensation prediction interframe coding using at least one reference frame for each coding target block included in a coding target frame of an input moving picture, One of a first prediction block generation mode for generating a prediction block signal from a reference frame and a second prediction block generation mode for generating the prediction block signal by linear sum prediction of a plurality of reference blocks cut out from a plurality of reference frames Selecting for each coding block, coding a differential signal between the selected prediction block signal and the signal of the coding target block, and performing the linear sum prediction on the average of the plurality of reference blocks. A line based on the display time of the plurality of reference frames and the encoding target frame, which is used as the value prediction Selecting for each of a plurality of pixel blocks in the encoding target frame or for each of the encoding target frames, whether to perform interpolation prediction; and in the first and second prediction block generation modes when generating the prediction block signal. Encoding first encoding mode information indicating which is selected for each of the encoding target blocks or for each of the plurality of encoding target blocks; and the average value prediction and the linear interpolation in the linear sum prediction. And a second encoding mode information indicating which one of the predictions is selected, for each of a plurality of pixel blocks of the encoding target frame or for each of the encoding target frames. 26. A moving picture decoding method for performing motion compensation prediction inter-frame decoding using at least one reference frame for each decoding target block included in a decoding target frame of a moving picture, Decoding a prediction error signal of the signal of the block to be decoded, a first prediction block generation mode for generating a prediction block signal from a single reference frame when generating a prediction block signal on the encoding side, and The first coding mode information indicating which of the second prediction block generation modes for generating the prediction block signal is selected by the linear sum prediction of the plurality of reference blocks cut out from the plurality of reference frames is the decoding target. Receiving and decoding for each block or for each of the plurality of decoding target blocks; The second coding mode information indicating which of the average value prediction of the plurality of reference blocks or the linear interpolation prediction based on the display times of the plurality of reference frames and the encoding target frame is selected for prediction is decoded. Receiving and decoding for each of a plurality of pixel blocks of the target frame for decoding or for each of the target frame for decoding, the prediction block according to the decoded first coding mode information and second coding mode information A moving image decoding method comprising: a signal generating step; and a reproduced moving image signal generating step using the generated prediction block signal and the decoded prediction error signal.