JP2006086684A - Motion picture encoder and encoding program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To realize a motion picture encoder producing an encoded signal arranged at a proper interval of reference image with high encoding efficiency immediately after input of an image having a violent motion vector. <P>SOLUTION: The motion picture encoder comprises a first frame memory 11 for storing an image of unknown encoding type, a second frame memory 26 for storing a reference image outputted immediately before, an M value determining means 14 for determining the motion vector of an input image based on a reference image stored in the second frame memory 26 by a motion vector detecting means 12, determining the M value indicative of the interval to a reference image following to the reference image outputted immediately before when the motion vector is smaller than a predetermined value as an optimal value, and decreasing the M value as an optimal value when the motion vector of the input image is larger than the predetermined value and different therefrom, and a means 16 for determining encoding of the input image by using that M value. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to motion compensation inter-frame predictive coding, and in particular, optimally sets an interval between a coded reference image (I picture and P picture) and a reference image arranged next to the reference image, The present invention relates to a moving picture coding apparatus and a moving picture coding program for coding a picture between past and future reference pictures as a motion compensated picture (B picture).

  In addition to transform coding, inter-frame prediction coding technology that reduces temporal redundancy is used as a compression coding method for video signals. For inter-frame prediction, it detects the motion between the past frame and the current frame and generates the current frame signal from the past frame, and detects the motion between the future frame and the current frame. There is backward prediction that generates the current frame from the future frame. In an encoding method defined by MPEG (moving picture experts group), three types of pictures are used: I (Intra-coded) picture, P (Predictive-coded), and B (Bidirectionally predictive-coded). The pictures to be encoded are usually arranged in the order of I, B, B, P, B, B, P,. Here, the B picture is encoded by bi-directional prediction using the I picture and / or P picture arranged before and after the B picture as a reference picture. Here, the interval between the reference image and the reference image arranged next is called an M value by a number in units of frames. The normal M value is set to 3, but in the case of an image with intense motion, higher encoding efficiency can be obtained by using an M value smaller than 3.

Japanese Patent Application Laid-Open No. H10-228561 has a memory for holding an input video signal for several frames, and the P signal interval is determined using the P picture interval determination / control unit stored in the memory. Based on the determined interval information of the P picture, the visual characteristic of the fast moving image is detected, and an instruction is given to encode the fast moving frame with the P or I picture. That is, a configuration of an image encoding device is disclosed in which the encoding efficiency is improved and the image quality is improved by using an M value smaller than a normal M value.
JP 2002-10270 A

  However, in normal coding, when the above picture arrangement is arranged as I1, B2, B3, P4, B5, B6, P7, B8,..., P4 one-way motion prediction is performed using I1. An interval (M value) to the next P picture is set based on a motion vector that is sometimes obtained. The M value is changed from B5. Therefore, even if a severe motion vector is detected after I1, it is only arranged between B5 and P7 in correspondence with the change of the M value, and a severe motion vector is detected at B2. However, it was impossible to obtain a high-quality encoded signal in which the M value was changed immediately after I1.

  In the image encoding device disclosed in Patent Document 1, since a memory of several frames is provided in the input circuit of the encoding device, an encoded signal in which the M value is changed immediately after a severe motion vector is detected is obtained. However, the encoded signal can be obtained only after a delay of several frames due to the delay time by the memory of several frames arranged in the input circuit of the encoding device. Therefore, it is impossible to obtain a high-quality encoded signal in which the M value is changed immediately after the intense motion vector is detected.

  Therefore, the present invention has been made to solve the above-described problems, and a high-quality encoded signal in which the M value (interval of reference images) is changed even immediately after a severe motion vector is detected. It is an object of the present invention to provide a moving picture coding apparatus and a moving picture coding program capable of obtaining the above with a minimum delay time.

According to a first aspect of the present invention, a reference image obtained by performing one-way predictive encoding from a past image of an input image, or a reference image encoded in the past and a reference image encoded in the future, When encoding using a predetermined rule using any one of the motion prediction encoded images obtained by performing bi-directional predictive encoding from the above, encoding efficiency decreases when encoding using the predetermined rule Is a moving picture coding apparatus that performs the one-way predictive coding when the predetermined rule is interrupted and the two-way predictive coding of the inputted image is to be performed. A first frame memory for storing an image, a second frame memory for storing a reference image output immediately before, and a reference frame stored in the second frame memory based on the reference image. The The motion vector detection means for obtaining the motion vector of the input image, and the motion vector of the input image obtained by the motion vector detection means is smaller than a predetermined value and in the same direction The M value indicating the interval from the reference image output immediately before to the next reference image is set as an optimal value, and the M value is reduced when the motion vector of the input image is larger than a predetermined value and different. An M value calculation determination unit that is an optimum value, and a code that determines that the input image stored in the first frame memory is to be encoded using the M value output from the M value calculation determination unit There is provided a moving image encoding apparatus comprising: a converted image determining means.
The second invention is a reference image obtained by performing one-way predictive coding from a past image of an input image, or two from a reference image coded in the past and a reference image coded in the future. When encoding according to a predetermined rule using any one of motion prediction encoded images obtained by performing direction prediction encoding, when encoding using the predetermined rule reduces the encoding efficiency, A moving picture coding program for performing the one-way predictive coding when the predetermined rule is interrupted and the bi-directional predictive coding of the inputted image is to be performed, and stores the inputted image Based on the first frame memory, the second frame memory for storing the reference image output immediately before, and the reference image stored in the second frame memory, before being stored in the first frame memory A motion vector detection means for obtaining a motion vector of an input image and an output immediately before the motion vector of the input image obtained by the motion vector detection means is smaller than a predetermined value and in the same direction. An M value indicating an interval from the reference image to the next reference image is set as an optimal value, and when the motion vector of the input image is larger than a predetermined value and is different, the M value is decreased to obtain an optimal value. M-value calculation determination means for performing encoding, and determination of an encoded image for determining that the input image stored in the first frame memory is to be encoded using the M value output from the M-value calculation determination means And a moving picture encoding program for causing the control means to function using the control means.

  According to the present invention, based on the first frame memory for storing the unknown image, the second frame memory for storing the reference image output immediately before, and the reference image stored in the second frame memory. A motion vector detecting means for obtaining a motion vector of the inputted image stored in the first frame memory, and a motion vector of the inputted image obtained by the motion vector detecting means is smaller than a predetermined value. And an M value indicating an interval from the immediately preceding reference image to the next reference image in the same direction as an optimal value, and a motion vector of the input image being larger than a predetermined value, and M value calculation determining means for reducing the M value to an optimum value when different, and using the M value output from the M value calculation determining means, the first frame memory Since there is a special configuration comprising an encoded image determination means for determining that the stored input image is to be encoded, high-quality encoding with the M value changed even immediately after a violent motion vector is detected A moving picture coding apparatus and a moving picture coding program capable of obtaining a signal with a minimum delay time can be realized.

A video encoding apparatus according to an embodiment of the present invention will be described below with reference to FIGS.
FIG. 1 is a block diagram illustrating a configuration example of a moving image encoding apparatus according to an embodiment of the present invention.
FIG. 2 is a flowchart showing an operation related to the determination of the optimum M value according to the embodiment of the present invention.
FIG. 3 is a diagram showing the relationship between the motion vector and the M value according to the embodiment of the present invention.
FIG. 4 is a diagram for explaining a method of determining the M value according to the embodiment of the present invention.
FIG. 5 is a flowchart showing an encoding operation using the optimum M value according to the embodiment of the present invention.

  The moving picture coding apparatus is capable of obtaining a high-quality coded signal in which the M value is changed even immediately after a severe motion vector is detected with a minimum delay time. The first frame memory for storing an unknown image, the second frame memory for storing a reference image output immediately before, and the reference image stored in the second frame memory And a motion vector detecting means for obtaining a motion vector of the inputted image stored in the first frame memory, and a motion vector of the inputted image obtained by the motion vector detecting means is a predetermined value. If the value is smaller than the reference value, the M value indicating the interval from the immediately preceding reference image to the next reference image is taken as the optimum value, and the motion vector of the input image is M value calculation deciding means for reducing the M value to an optimum value when the value is larger than a predetermined value and different, and using the M value outputted from the M value calculation deciding means, the first frame The present invention is realized by using an encoded image determining unit that determines to encode an input image stored in a memory.

The configuration of the moving picture coding apparatus will be described.
1 includes a frame memory 11, a motion prediction unit (ME) 12, a motion prediction result holding unit 13, an M value candidate calculation unit 14, an M value determination unit 15, an encoded image determination unit 16, A subtracting unit 18, a DCT (discrete cosine transform) unit 21, a quantizing unit 22, an inverse quantizing unit 23, an inverse DCT unit 24, an adding unit 25, a frame memory 26, a motion compensating unit (MC) 27, an encoding unit 28, And an MV (motion vector) code amount counting unit 29.
A video signal is input to the frame memory 11. The encoding unit 28 outputs a compressed encoded signal.

The operation of the moving picture coding apparatus will be described.
First, the video signal input to the frame memory 11 is temporarily stored therein. The encoded image determination unit 16 determines to process the picture (image) of the video signal input first as an I (Intra-coded) picture that is a reference image. The image is input to the DCT unit 21 where it is DCT transformed and quantized by the quantization unit 22. A part of the signal obtained by quantization is subjected to variable length coding by the encoding unit 28 and is output, and the other part is inversely quantized by the inverse quantization unit 23, and the inverse DCT unit 24 performs inverse quantization. The decoded image obtained by the conversion is input to the frame memory 26 via the adding unit 25 and stored therein.

  Next, in the frame memory 11, a picture (image) of a video signal that is temporarily set as a motion compensated image by the encoded image determination unit 16 is temporarily stored. A temporarily stored image is input to the motion prediction unit 12, and a motion vector is obtained between the input image and a reference image stored in the frame memory 26. The obtained motion vector is input to the motion compensation unit 27 and also input to the motion prediction result holding unit 13 and held (stored) therein. A motion vector stored in the motion prediction result holding unit 13 is input to the M value candidate calculation unit 14. The M-value candidate calculation unit 14 detects whether the motion amount of the input motion vector is small, large, or large, and whether the vector directions are the same or different from each other. Is done. When the detected motion vector is small and when it is large but facing the same direction, an M value candidate that does not change the M value is output. The M value candidates calculated by the M value candidate calculation unit 14 are supplied to the encoded image determination unit 16 via the M value determination unit 15.

  The picture stored in the frame memory 11 by the encoded image determination unit 16 is acquired as an image to be processed as a motion prediction image, and is input to one of the subtraction units 18. The other of the subtracting unit 18 receives a motion compensated image generated by the motion compensating unit 27 based on the motion vector obtained by the motion predicting unit 12 and the reference image stored in the frame memory 26. The image signal component of the difference from the reference image obtained by subtraction by the subtracting unit 18 is DCT, quantized and variable-length encoded in the same manner as described above to generate an encoded output signal and the motion used for encoding Output with vector. The MV code amount counting unit 29 calculates the code amount of the motion vector output together with the encoded signal.

  Here, encoding performed by a general moving picture experts group (MPEG) is performed with an M value of 3. Therefore, the picture (picture) of the video signal that is temporarily stored in the frame memory 11 next is provisionally determined by the encoded picture determination section 16 as a motion compensated picture (B picture = Bidirectionally predictive-coded picture), and the above code Is started. If the motion vector calculated by the M value candidate calculation unit 14 is detected as having a large amount of motion and pointing in different directions, the M value should be changed to a small value. Output M value candidates. In the M value determination unit 15, when an M value candidate that the M value should be changed to a small value is input, and when the code amount of the motion vector counted by the MV code amount count unit 29 is large, the coding efficiency Is determined not to be processed as a motion prediction image, but to be processed as a reference image. In other words, the M value is determined to be changed small. When it is determined by the encoded image determination unit 16 to be processed as a reference image, the image temporarily stored in the frame memory 11 is acquired as a reference image, and then encoded as a reference image. The encoded signal is output from the moving image encoding device.

Next, this will be described in detail.
FIG. 2 is a flowchart showing an operation for determining a new optimum M value.
First, when encoding an input video signal, the occupancy information of the motion vector code included in the encoded signal obtained by encoding one frame before in S (step) 71 is an MV code amount count unit. 29. If the occupancy information exceeds a threshold value, encoding is more efficient when encoded as an I picture or P picture as a reference image than as a B picture as a motion prediction encoded image. It is determined that Proceeding to S75, the M value determination unit 15 determines a new optimum M value by reducing the number of initial candidate M values, for example, M = 3, to M = 2 or M = 1 from the encoded image determination unit 16. Then, encoding is performed using the new optimum M value.

  In S71, the occupation rate of the MV code amount is smaller than the threshold, and when it is detected that normal encoding has been performed so far, the motion vector of the input image is obtained in S72. The motion vector is also performed when the encoded image determination unit 16 plans to encode with a B picture. The magnitude of the motion vector being obtained in S73 is detected. When a large motion vector is detected in S74, the motion vector direction is the same as the panned image, or a random motion such as a sea wave with different motion vector directions is applied. Is detected. In the case of an image having a random motion, encoding efficiency becomes higher when encoded as a reference image than when encoded as a motion compensated image. In S75, even if the M value determination unit 15 is scheduled to encode the motion compensation image with respect to the encoded image determination unit 16, the initial candidate M value is set so as to perform encoding as a reference image. The type of the encoded image is determined so that encoding with a reduced size is performed. When there is no large motion vector, and when there is a large motion vector and the motion vector is directed in the same direction, the encoding scheduled for S77 from the beginning, for example, with an M value of 3, is continued. In S76, it is detected whether there is a next input image. If there is an input image, the operation from S71 is repeated. When there are no more input images, the operation for determining a new optimum M value is terminated.

FIG. 3 shows the relationship between the motion vector and the M value.
In the figure, the horizontal direction is the distance between the pixels between the two frames of the image in the motion vector search range. In this example, the M value is 4 when most of the motion vectors are 0 to 15 pixels / frame. Similarly, when the motion vector is 16 to 31, the M value is 3, when 32 to 47 is 2, and when 48 or more. Set to 1. (1) is 18 pixels / frame, so the M value is 3, (2) is 60, so the M value is 1, and (3) is 35, so the M value is 2.
For the motion vector, the largest motion vector value is obtained from the motion amounts of the vector sequentially obtained from the upper left block, and finally the motion vector value is obtained with reference to the dispersion value (randomness in the vector direction) of the obtained motion vector value. Find the correct M value. The optimum M value is determined by the M value determination unit 15 when the motion prediction unit 12 detects a motion vector for most of the images of one frame stored in the frame memory 11. The encoded image determination unit 16 acquires a reference image to be encoded from the frame memory 11 based on the determination. In the frame memory 26, a decoded reference image of the acquired reference image is obtained. The reference image is used to encode a motion prediction image between the reference image in the foreground.

FIG. 4 shows a temporal relationship between the reference image and the motion prediction image.
In the figure, the horizontal direction is time, and N is current. A reference picture (I or P picture) is encoded in N-3, which is three frames past the present. In N-2 and N-1, motion prediction pictures (B pictures) that are non-reference pictures are encoded. N + 1 and later are future pictures and the picture type is unknown. The motion prediction unit 12 obtains a motion vector between the current encoded reference picture stored in the frame memory 26 and the picture type unknown picture stored in the frame memory 11.
When the obtained motion vector value is 32 to 47, the M value is changed from 3 to 2, and a picture type unknown picture of N + 2 is determined as a reference picture. In the encoded image determination unit 16, the N + 2 picture is encoded as a reference picture. Thereafter, a picture whose N + 1 picture type is unknown is set as a motion prediction picture, and both the reference picture N and the reference picture N + 2 are referred to and subjected to motion prediction encoding.
If the value of the motion vector between the current encoded reference picture and the picture type unknown picture N + 1 is 48 or more, the picture type unknown picture N + 1 is determined as a reference picture. The encoded image determination unit 16 acquires the picture type unknown picture N + 1 and encodes it as a reference picture.

  In normal MPEG coding, when the motion vector between the N-3 reference picture and the N reference picture is small, the motion vector obtained when the picture type unknown picture N + 3 is coded as a reference picture Since the subsequent picture type is determined based on, the change of the M value is performed after N + 4. In the case of the present embodiment, the M value can be changed from N + 1 picture type unknown pictures. Therefore, when a picture changes from an image with little motion to an image with much motion, efficient coding is performed from the first image detected as having a large motion vector.

Here, there is a large amount of calculation for obtaining an image with good matching for obtaining a motion vector by searching. Therefore, the result of the motion vector calculation performed to determine the picture type is also used for motion prediction encoding, so that the processing amount for the motion vector calculation does not increase.
The motion vector reuse operation will be described with reference to FIG.
First, in S81, the encoded picture determination unit 16 acquires the next picture. In S82, it is detected whether the acquired picture is a reference picture. If it is not a reference picture, whether it is a picture immediately after the reference picture is detected in S91. If it is immediately after, the previously obtained motion vector can be reused. The motion vector stored in the motion prediction result holding unit 13 is used, and the motion prediction picture is encoded such that a motion prediction image is created from the reference image immediately before being stored in the frame memory 26. If it is detected in S91 that the picture is not immediately after the reference picture, encoding is performed in S93 by the same method as in the prior art.
If it is detected in S82 that it is a reference picture, a motion vector is detected in S83, and a motion vector detection result is stored in S84. The stored motion vector detection result is used when the motion prediction picture immediately after the next reference picture is encoded.
In S85, M value candidates are calculated based on the stored motion vectors. In S86, the M value is determined based on the candidate. Encoding is performed based on the M value determined in S87. In S88, it is detected whether there is a picture to be encoded next. If there is a picture to be encoded, the operation from S81 is repeated. When it is detected that there is no next picture, the encoding operation is terminated.

  In the above-described moving image encoding apparatus, the input video signal is encoded with the initial value of M being 3; however, when the encoded signal is a reference image and when it is a motion compensated image In either case, motion vector activity is detected. When it is determined that the encoding efficiency is improved when the motion compensation encoding process is changed to the reference image encoding process, an encoding method that allows the processing method to be changed immediately can be realized. In this encoding method, although the memory capacity of the frame memory 11 is the same as that used in normal MPEG, the image signal that has been stored in the memory is immediately transferred to the motion prediction unit. Since the motion vector is calculated in 12 and it is detected whether or not encoding with an appropriate M value is performed, the change of the M value can also be executed during the calculation of one frame image.

  As described above, according to the video encoding apparatus shown in the embodiment, the first frame memory 11 that stores the input image, the second frame memory 26 that stores the reference image output immediately before, Based on the reference image stored in the second frame memory 26, motion vector detecting means 12 for obtaining a motion vector of the input image stored in the first frame memory 11, and the motion vector detection When the motion vector of the input image obtained by the means 12 is smaller than a predetermined value, the M value indicating the interval to the reference image next to the reference image output immediately before is set as the optimum value, and the input M value calculation determination means 14 for reducing the M value to an optimum value when the motion vector of the image to be processed is larger than a predetermined value and different, and the M value calculation determination means 14 The coded image determining means 16 for determining that the input image stored in the first frame memory is to be encoded using the M value output from the first frame memory. It is possible to realize a moving picture coding apparatus and a moving picture coding program capable of obtaining a high-quality coded signal in which the M value is changed immediately after detection with a minimum delay time.

  Video signal encoding apparatus for performing encoding by optimally setting an interval between an encoded reference picture (I picture and P picture) and an encoded reference picture (B picture) arranged next to the reference picture Applicable to.

It is a block diagram which shows the structural example of the moving image encoder which concerns on implementation of this invention. It is a figure of the flowchart which shows the operation | movement which concerns on determination of the optimal M value based on implementation of this invention. It is a figure which shows the relationship between the motion vector and M value which concern on implementation of this invention. It is a figure for demonstrating the determination method of M value based on implementation of this invention. It is a figure of the flowchart which shows the operation | movement of the encoding using the optimal M value based on implementation of this invention.

Explanation of symbols

11 Frame memory 12 Motion prediction unit 13 Motion prediction result holding unit 14 M value candidate calculation unit 15 M value determination unit 16 Encoded image determination unit 18 Subtraction unit 21 DCT unit 22 Quantization unit 23 Inverse quantization unit 24 Inverse DCT unit 25 Adder 26 Frame memory 27 Motion compensator 28 Encoder 29 MV code amount count unit

Claims (2)

Perform bi-directional predictive coding from a reference image obtained by performing one-way predictive coding from a past image of an input image or from a reference image coded in the past and a reference image coded in the future When encoding according to a predetermined rule using any of the motion prediction encoded images obtained in the above case, if the encoding efficiency decreases when encoding using the predetermined rule, the predetermined rule is interrupted. A video encoding device that performs the one-way predictive encoding when the input image is to be subjected to the two-way predictive encoding,
A first frame memory for storing the input image;
A second frame memory for storing the reference image output immediately before;
Motion vector detection means for determining a motion vector of the input image stored in the first frame memory based on the reference image stored in the second frame memory;
When the motion vector of the input image obtained by the motion vector detection means is smaller than a predetermined value and in the same direction, the interval to the reference image next to the reference image output immediately before is indicated. M value calculation determination means for setting an M value as an optimal value and reducing the M value to an optimal value when a motion vector of the input image is larger than a predetermined value and different,
Using the M value output from the M value calculation determining means, the encoded image determining means for determining to encode the input image stored in the first frame memory;
A moving picture encoding apparatus comprising: Perform bi-directional predictive coding from a reference image obtained by performing one-way predictive coding from a past image of an input image or from a reference image coded in the past and a reference image coded in the future When encoding according to a predetermined rule using any of the motion prediction encoded images obtained in the above case, if the encoding efficiency decreases when encoding using the predetermined rule, the predetermined rule is interrupted. A video encoding program for performing the one-way predictive encoding when the bi-directional predictive encoding of the input image is to be performed,
A first frame memory for storing the input image;
A second frame memory for storing the reference image output immediately before;
Motion vector detection means for determining a motion vector of the input image stored in the first frame memory based on the reference image stored in the second frame memory;
When the motion vector of the input image obtained by the motion vector detection means is smaller than a predetermined value and in the same direction, the interval to the reference image next to the reference image output immediately before is indicated. M value calculation determination means for setting an M value as an optimal value and reducing the M value to an optimal value when a motion vector of the input image is larger than a predetermined value and different,
Using the M value output from the M value calculation determining means, the encoded image determining means for determining to encode the input image stored in the first frame memory;
A moving picture encoding program for causing a program to function using a control means.

Images