JP3599942B2 - Moving picture coding method and moving picture coding apparatus - Google Patents

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to a moving image encoding method and apparatus for compressing and encoding a moving image video signal such as a television signal using an inter-frame predictive encoding method. In particular, at the time of a scene change (scene change), a moving image that achieves image quality stabilization by preventing an adverse effect such as an increase in the code amount of a non-reference frame that is not referred to in the inter-frame predictive coding method. The present invention relates to an image encoding method and apparatus.
[0002]
[Prior art]
As a method for compressing and encoding a moving image signal, the MPEG method is standardized as MPEG1-Video (ISO / IEC 11172-2) and MPEG2-Video (ISO / IEC-13818-2). Instead, it is also widely used in broadcasting and communication fields such as digital broadcasting and digital transmission.
[0003]
In the MPEG system, a GOP (Group of Picture), which is a set of a plurality of frame units (one field, one frame, or the like), is configured in a frame sequence. In a GOP, at least one frame is compression-coded by intra-frame compression, and other frames are compression-coded with reference to other frames (inter-frame compression = inter-frame predictive coding). You. Images compressed and encoded by intra-frame compression are I pictures, and images compressed and encoded using inter-frame compression are P pictures and B pictures.
[0004]
In intra-frame compression, techniques such as DCT (Discrete Cosine Transform), quantization, and variable length coding (VLC) are employed.
First, image data in a frame is divided into small regions of 16 × 16 pixels called macroblocks, and DCT transform is performed for each 8 × 8 pixel block in the small region to obtain a frequency region.
[0005]
Since human visual characteristics are insensitive to high frequencies, reducing the amount of codes assigned to high-frequency regions in the above-mentioned frequency region reduces the amount of data without causing significant visual deterioration to humans. Can be. This assignment is performed by quantization. Quantization is realized by dividing 8 × 8 frequency domain data (coefficient matrix) obtained by DCT transform of a block of 8 × 8 pixels by a certain matrix. On the decoder side, the original 8 × 8 frequency domain data (coefficient matrix) can be obtained by multiplying (inverse quantizing) the data obtained by the variable length decoding by the above matrix. Become. That is, it is possible to obtain a matrix for performing the inverse DCT operation.
[0006]
In the above, by setting a large division value for high-order data, that is, data in a high-frequency region, it is possible to reduce the encoding accuracy in the high-frequency region. That is, the data amount corresponding to the high frequency region can be significantly reduced. The quantized data thus obtained has many zero values. Therefore, in this data, a variable length encoding table is formed by a set of the number of zero values and the subsequent real values. Variable-length coding is a type of entropy coding, in which a short code is assigned to data with a high frequency of occurrence, and a long code is assigned to data with a low frequency of occurrence, resulting in highly efficient compression. It is a method to perform.
[0007]
In the inter-frame compression, a technique called motion compensation using a motion vector is employed in addition to the techniques of DCT, quantization, and variable-length coding described above.
First, image data in a frame is divided into small areas of 16 × 16 pixels called macroblocks, as in the case of intra-frame compression. The inter-frame compression requires not only image data to be encoded (current image data) but also image data of another frame to be referred to by the current image data (reference image data). As the reference image data, a frame existing at a position temporally close to the current image data is used. For each macroblock, an area (macroblock) that is close in data to the current encoding-target macroblock (current macroblock) is searched from the image data of the reference frame. The “region that is close (approximate) in terms of data” usually refers to a region in which only the luminance of pixel data is compared, and the average of the square errors thereof shows the smallest value. In MPEG, this search is performed with an accuracy of a half pixel unit. The search range is a region within a predetermined range from the position in the reference frame corresponding to the position of the current macroblock in the current frame. This search is called “motion vector” detection. Further, in the inter-frame predictive coding, correcting a positional shift between a reference macroblock and a current macroblock by using a "motion vector" is referred to as "motion compensation". The value of this “motion vector” is also variable-length coded.
[0008]
If there is image data to be referred to, it is possible to cut out the above “data-wise close” region on the decoder side. However, the image data of the extracted reference macroblock and the image data of the current macroblock to be encoded are not completely the same. Therefore, it is necessary to compensate for this difference. In MPEG, in order to compensate for this difference, a difference macroblock is constructed by taking a difference for each pixel (luminance, chrominance), and for each region of 8 × 8 pixels, compression in a frame unit is performed. DCT operation and quantization are performed in the same manner as in. However, in this case, the quantization is generally performed with reduced accuracy because the target is differential data. For this reason, a quantization table that does not distinguish between the high-frequency region and the low-frequency region is usually prepared as a default for the difference data. Of course, it is possible to freely change this quantization table on the encoder side.
[0009]
The compression code thus obtained is subjected to inverse quantization and inverse DCT. When the image data of the referenced frame exists inside the decoder, the value after the inverse DCT described above is added to the frame area specified by the motion amount, so that the decoding of the macroblock can be performed. Will be
[0010]
In MPEG, there are two types of frames that are compression-encoded using inter-frame compression. A frame referred to only from a past frame is called a P picture, and a frame referred to from both the past and the future is called a B picture. A P picture is predicted from an I or P picture that is located temporally (= in display order) in the past. B pictures are predicted from I or P pictures located in the past and / or future in time (= in display order). That is, the B picture is a non-reference frame that is not referred to for motion prediction (= a non-reference frame that is not referred to for calculating difference data), and the I and P pictures are reference frames that can be referred to for motion prediction. == frame that can be referred to for calculating difference data).
[0011]
The code amount of the differential compression code compressed by combining these compression methods is several-hundredths to one-hundredths smaller than the data amount of the original image data. Holding. However, since a method of referring to another frame is used, a configuration for holding image data of the reference frame on the decoder side is required. On the decoder side, the decoded image data is used as the image data of the reference frame. Therefore, the encoder must perform encoding in consideration of this fact. That is, it is necessary to encode the I-frame and the P-picture which are the frames to be referred to, decode them under the same conditions as those of the decoder, and hold the decoded image data of the reference frame. Then, the P picture and the B picture must be encoded with reference to the decoded image data.
[0012]
FIG. 4 shows an example of a frame sequence of an MPEG moving image. (A) in the figure shows the display order of each frame. The order of the frame type most commonly used in MPEG is shown, but is not limited to this. The arrow indicates that the start frame is referred to when encoding / decoding the end frame. For example, since the B pictures 51 and 52 are frames that are encoded / decoded with reference to the I picture 53, the I picture 53 is already decoded when the B pictures 51 and 52 are encoded / decoded. Must be stored in the image memory. Similarly, since the B pictures 54 and 55 are frames that are encoded / decoded with reference to the I picture 53 and the P picture 56, when the B pictures 54 and 55 are encoded / decoded, The I picture 53 and the P picture 56 need to be decoded and stored in the image memory. Similarly, since the P picture 56 is a frame that is encoded / decoded with reference to the I picture 53, the I picture 53 has already been decoded and encoded when the P picture 56 is encoded / decoded. Must be held in
[0013]
For this reason, the recording order of each frame described above is as shown in FIG. Decoding is performed in the recording order, and the decoded image data of the I picture and the P picture are sequentially stored, so that the above-described B picture and the P picture can be encoded / decoded. In order to make such a reference, the I-picture itself needs to maintain its image quality. Also, since the P picture is used for referring to another P picture or B picture, it is necessary to maintain high image quality. For this reason, the image quality as a whole is generally maintained by allocating the largest amount of code to the I picture and then allocating the code amount in the order of the P picture and the B picture.
[0014]
[Problems to be solved by the invention]
However, in the related art, when a scene change (scene change) occurs in a B picture with the smallest data amount allocation, a so-called bidirectional prediction code for performing coding with reference to past and future bidirectional frames. Can not be performed. For this reason, a high compression rate cannot be maintained in the B picture, and the code amount of the B picture increases.
[0015]
Also, in the video encoding device, since the amount of code output from the device is monitored and the quantization scale is controlled to control the overall amount of code, a rate control is performed as described above. When the code amount of the changed B picture increases, the quantization of the other frames is coarsely controlled, resulting in a problem that the image quality of the other frames deteriorates.
[0016]
It is an object of the present invention to suppress an increase in the code amount of a B picture without deteriorating the image quality of a B picture in which a scene change has occurred, thereby preventing the deterioration of the image quality of another frame.
[0017]
[Means for Solving the Problems]
According to the first aspect of the present invention, a series of frame moving image signals including an I frame, a P frame, and a B frame are used as input signals to perform intra-frame encoding for an I frame, and perform encoding for a P frame and a B frame. A moving image encoding method for performing inter-frame encoding in which prediction data indicating a reference frame is combined with encoded data obtained by encoding difference data from a preset reference frame and encoded data is output. When a scene change is detected in the frame, the difference data for the B frame is replaced with zero, and the prediction direction information of the B frame is changed from the one indicating the set reference frame to the B frame. A moving image characterized in that an I or P frame whose display order is close is replaced with a frame designating a reference frame and output. It is an encoding method.
[0018]
According to a second aspect of the present invention, in the first aspect, when a scene change is detected in the B frame, the difference data is reduced to zero even for a B frame existing from the B frame to the reference frame after the replacement. And outputting the prediction direction information by replacing the one indicating the set reference destination frame with the one indicating the replacement reference frame. It is.
[0019]
According to a third aspect of the present invention, a series of frame moving image signals including an I frame, a P frame, and a B frame are input, an intra-frame encoding is performed on the I frame, and a P frame and a B frame are encoded. In a moving image encoding apparatus for performing inter-frame encoding in which prediction direction information indicating the reference destination frame is combined with encoded data obtained by encoding difference data with a preset reference destination frame and output, A scene change detecting unit that detects a frame where a scene change is performed among frames constituting a series of frame moving image signals; and, when the scene change detected by the scene change detecting unit is the B frame, The difference data for the B frame is replaced with zero, and prediction direction information of the B frame is replaced. A moving unit that replaces the frame indicating the set reference frame with a frame indicating the I or P frame whose display order is closer to the B frame as the frame to be referenced. Device.
[0020]
According to a fourth aspect of the present invention, in the third aspect, when the frame in which the scene change is detected by the scene change detecting unit is the B frame, the replacing unit performs a process from the B frame to the reference frame after the replacement. Also for the B frame existing between them, the difference data is replaced with zero, and the prediction direction information is replaced from the one indicating the set reference frame to the one indicating the replaced reference frame. And outputting the moving image.
The scene change can be detected by, for example, a change in a high-frequency component of a moving image, a movement of an edge component of a frame, a change in a low-frequency component, and the like. The method of detecting a scene change is known as disclosed in JP-A-6-153146, JP-A-7-38842, and JP-A-7-79431.
[0026]
BEST MODE FOR CARRYING OUT THE INVENTION
1. Typical MPEG encoder (FIG. 3).
First, a typical MPEG encoder will be described with reference to FIG. Although the description has been made by taking the MPEG encoder as an example, a moving picture encoding device or method that does not conform to the MPEG standard may have the configuration of the present invention.
[0027]
11 is an input means, 12 is a frame reorderer, 13 is a scan conversion processor for converting a raster scan into a macroblock, 14 is a subtractor, 16 is a DCT processor, 18 is a quantizer for performing weighting, and 20 is variable length coding. 22, a read buffer from the encoder, 23 output means, 24 a rate controller, 26 an inverse quantizer, 28 an inverse DCT processor, 30 an adder, 32 a frame memory, 32a and 32b a frame memory 32 , A motion compensation processor, 36 a mode determiner, and 38 a motion detector.
[0028]
First, a frame signal is input from the input unit 11 to the frame reordering unit 12 in the order of FIG. The frame reordering unit 12 changes the frame order from the order of FIG. 4A to the order of FIG. That is, the frame that is compression-encoded as an I picture is set as the head, and the order of the frames is rearranged so that the referenced frame precedes the referenced frame.
[0029]
Next, data is input to the scan conversion / macroblock processor 13. Usually, image data is recorded in a raster scan format. This is a format in which data is output one pixel at a time from the upper left of the screen, one line is output, and then the data of the next line is output. In MPEG, data is processed in units of small areas called macroblocks of 16 × 16 pixels, so it is necessary to convert a raster scan format into a macroblock format. In MPEG1, a macroblock is composed of 16 × 16 pixels of luminance data and 8 × 8 pixels of chrominance data. In MPEG2, the ratio of the color difference is changed depending on the frame format. However, a range corresponding to 16 × 16 pixels in luminance is processed as a macroblock, and a search for a motion vector is also performed using this as a unit.
[0030]
Hereinafter, processing of data output from the scan conversion / macroblock processor 13 will be described for each compression method.
[0031]
1-1. For I-picture.
I-picture data is usually not inter-frame predictive coded and is not searched for a motion vector (however, the introduction of a motion vector into an I-picture is exceptionally allowed in MPEG for error recovery). Therefore, it is directly input to the DCT processor 16. After the DCT processor 16 converts the image data into the spatial frequency, the quantizer 18 weights each frequency domain. The quantized coefficient data is input to the variable-length encoder 20 and subjected to variable-length encoding.
[0032]
The code after the variable length coding cannot output data at a fixed transfer rate. Therefore, the read buffer 22 buffers the transfer rate, and the output means 23 outputs the compressed code string at a constant transfer rate.
[0033]
The code amount output from the variable length encoder 22 is monitored by the rate controller 24. If the difference from the code amount set for each image type (I, P, B picture) is large, The scale is changed. At the time of quantization, the above-described quantization matrix is multiplied by a quantization scale, and this quantization scale is controlled by the rate controller 24 as described above. When the quantization scale is increased, the quantization is coarsely performed. As a result, the generated code amount decreases, but the image quality deteriorates. When the quantization scale is reduced, the quantization is performed finely, and as a result, the generated code amount increases, but the image quality is improved.
[0034]
The output of the quantizer 18 is also input to the inverse quantizer 26. This is for decoding the aforementioned reference image data. That is, after the inverse quantization, the inverse DCT is performed by the inverse DCT processor 28, and the decoded image data is stored in the image memory 32. In this way, the reference I picture is held in the image memory.
[0035]
1-2. For a P picture.
When encoding a P picture, a motion vector is searched for each macroblock. This search is performed with reference to the I picture or the P picture stored in the image memory 32. However, motion vectors are not searched for all macroblocks. For example, in the case of a fast-moving frame, in the case of a frame that pans to the left and the like, the region on the right end is a region that first appears in the frame. . Therefore, in such a case, a mode in which a motion vector is not searched is applied as in the case of a macroblock of an I picture. In addition, there is a mode for cutting out block data at the same position as it is in order to easily perform background processing. Which mode is effective for encoding the macroblock is determined by the mode determiner 36.
[0036]
When the mode determiner 36 determines to perform motion compensation, the command is input to the motion compensator 34. The motion vector obtained by the motion detector 38 is also input to the motion compensator 34. Based on these data, the motion compensator 34 reads the data of the corresponding area from the image memory 32 and outputs the data to the subtractor 14. The subtracter 14 calculates the difference from the current macroblock, and the difference data is encoded by the DCT 16 and the quantizer 18 in the same manner as in the case of the I-picture described above. In the case of a P picture, it is necessary to store the P picture in the image memory 32 because it is referred to by other P pictures located in the future in time and B pictures located in the past and the future. Therefore, the data is processed by the inverse quantizer 26 and the inverse DCT 28, but as it is, it is difference data and not original image data. For this reason, the image data of the corresponding area is output from the image memory 32 to the adder 30 by using the motion vector used for encoding the macroblock again, and is added to the decoded difference data. The data is decrypted. This image data is stored in the image memory 32 and is provided for the above-mentioned reference.
[0037]
1-3. For B picture.
The coding method for a B picture is basically the same as that for a P picture. However, in the case of a B picture, since it is compression-encoded with reference to past and future I and P pictures, it is necessary to search for a motion vector twice that of a P picture. Further, since the B picture is not used for reference, it is not necessary to store the decoded image data in the image memory 32. In the case of a B picture, the encoding mode of the macroblock is selected from four types: prediction from the past and the future, prediction from the past, prediction from the future, and intra-unit encoding.
Thus, a bit stream of the compressed image code of the MPEG system is generated.
[0038]
2. First embodiment (FIG. 1).
FIG. 1 shows an example of an embodiment of the present invention. 1, blocks having the same functions as those in FIG. 3 are denoted by the same reference numerals, and description thereof will be omitted.
[0039]
102 is a scene change detector, 103 is a replacement determiner, 104 is a switch, 106 is a switch, and 108 is a switch. Each of the switches 104, 106, and 108 is switched by a control signal from the replacement determination unit 103.
[0040]
The scene change detecting unit 102 determines whether or not there is a scene change based on the image data input from the input unit 11. For example, the presence or absence of a scene change (scene change) can be determined based on a change in the high-frequency component of the image data, a shift in the edge component of the frame, a change in the low-frequency component, and the like. In the present invention, a technique for detecting a scene change is not limited. Further, as a frame that is used as a reference for determining a scene change of the current frame, a frame that is located immediately before the current frame on the time axis, or a frame that the current frame refers to at the time of encoding can be used.
[0041]
When a scene change is detected by the scene change detector 102, the replacement determiner 103 determines whether the scene change has been performed in a B picture, an I picture, or a P picture. In other words, it is determined whether the processing has been performed on a B-picture which is a non-reference frame not referred to for calculating the difference data, or whether the processing has been performed on an I-picture or P-picture which can be referred to for calculating the difference data. I do.
[0042]
As a result, when it is determined that a scene change has been detected in an I picture or a P picture, the replacement determiner 103 controls the switches 104, 106, and 108 to maintain the normal position (terminal a). Command. In this case, the same operation as in FIG. 1 is performed. On the other hand, when it is determined that a scene change has been detected in a B picture, the replacement determiner 103 sends to each of the switches 104, 106, and 108 the macroblock of the B picture from the normal position (terminal a). Command switching.
[0043]
As a result, the switching unit 104 switches the macroblock of the B picture to the terminal b so that zero data is input to the DCT processor 16 instead of the input from the subtractor 14 (input of difference data). Can be Therefore, the differential compression code output from the quantizer 18 becomes zero data.
[0044]
In addition, the switch 106 switches to the terminal b side so that zero data indicating no motion is sent to the variable length encoder 20 in place of the detected motion vector for each macroblock of the B picture. Can be Thereby, the motion vector combined with the compression code of the macroblock (in this case, zero data as described above) becomes zero data indicating no motion.
[0045]
In addition, the switch 108 replaces the coding mode determined for each macroblock of the B picture with a frame (I picture or P picture) that is close to the display order of the B picture and can be referred to for calculating difference data. ) Is switched to the terminal b side (forward prediction) or the terminal c side (backward prediction) so that the macroblock type information having the direction of ()) as the prediction direction is sent to the variable-length encoder 20. As a result, the macroblock type information combined with the compression code of the macroblock (in this case, zero data as described above) becomes information indicating the prediction direction. Note that whether to select the terminal b side (forward prediction) or the terminal c side (backward prediction) can be easily determined by, for example, selecting a direction of a reference frame that is close in time. Note that a configuration in which only one of the terminal b side (forward prediction) and the terminal c side (backward prediction) is prepared is also possible.
[0046]
As a result of the switching of the switches 104, 106, and 108, the variable-length encoder 20 shows a compression code of all "0" and indicates that there is no motion over the period of the B picture. The motion vector “0” information and macroblock type information indicating an I picture or a P picture whose display order is close to the B picture are input. These are variable-length coded and then combined and output. In the MPEG system, when data in a macroblock satisfies a specific condition, there is no need to encode data for the macroblock. The above-described input to the variable length encoding means 20 satisfies this condition. Therefore, encoding is performed only for a small number of macroblocks that must be encoded. For this reason, the code amount of the B picture is suppressed to an extremely small amount.
[0047]
In this way, for a B picture in which a scene change is detected, information indicating that the B picture is replaced with an I picture or a P picture whose display order is close to that of the B picture is output. For this reason, the decoder (not shown) reproduces and outputs the I picture or the P picture indicated by the information to be replaced, instead of the B picture.
[0048]
In FIG. 1, the motion vector information switched to the terminal b by the switch 106 and the macroblock type information switched to the terminal b or the terminal c by the switch 108 are input to the motion compensator 34. However, as inputs to the motion compensator 34, normal motion vector information output from the motion detector 38 and normal macroblock type information output from the mode determiner 36 are adopted. Is also good.
[0049]
In the above description, an example is described in which control is performed so as to output information indicating that a B picture in which a scene change has occurred is replaced with an I picture or a P picture whose display order is closer to the B picture. If there is another B picture between the picture and the I picture or P picture whose display order is close to the B picture, for each macroblock in the other B picture, It can also be configured to process in the same way as a picture.
[0050]
For example, when a scene change is detected in the B picture 58 of FIG. 4, as described above, control is performed so that the B picture 58 is replaced with the P picture 59, and the other B pictures 57 are replaced with the P pictures 56 (P (It may be a picture 59). This configuration corresponds to claims 10 and 11.
[0051]
3. Second embodiment (FIG. 2).
FIG. 2 shows an example of another embodiment of the present invention. 2, blocks having the same functions as those in FIGS. 1 and 3 are denoted by the same reference numerals, and description thereof will be omitted.
[0052]
In FIG. 2, the switch 104 is disposed between the scan conversion macroblock generator 13 and the subtractor 14, and data read from the image memory 32 is input to the terminal b of the switch 104. That is, the same image data read from the image memory 32 is input to both input terminals of the subtractor 14 when the switch 104 is switched to the terminal b. Therefore, when the switch 104 is switched to the terminal b, zero data is input to the DCT processor 16, and the same effect as in FIG. 1 can be obtained.
[0053]
【The invention's effect】
According to the present invention, when a scene change occurs in a non-reference frame, an increase in the code amount of the non-reference frame can be suppressed. For this reason, the coding efficiency other than the scene change can be improved. As a result, stable compression encoding with high image quality can be realized.
[Brief description of the drawings]
FIG. 1 is a block diagram showing an example of an embodiment of a moving image encoding device according to the present invention.
FIG. 2 is a block diagram showing an example of another embodiment of the video encoding device of the present invention.
FIG. 3 is a block diagram showing a typical moving picture encoding device.
FIG. 4 is an explanatory diagram of a reference relationship between frames in inter-frame predictive coding.
[Explanation of symbols]
102 Scene Change Detector
103 Replacement decision unit
104 switch
106 switch
108 switch

Claims (4)

Using a series of frame video signals including an I frame, a P frame and a B frame as an input signal, intra-frame encoding is performed for the I frame, and a preset reference frame is set for the P frame and the B frame. A moving image encoding method for performing inter-frame encoding in which prediction direction information indicating the reference destination frame is combined with encoded data obtained by encoding difference data with
When a scene change is detected in the B frame, the difference data for the B frame is replaced with zero, and the prediction direction information of the B frame is changed from the one indicating the set reference frame to the B frame. Is output by replacing the I or P frame whose display order is close to the one indicating the reference frame.
A moving picture coding method characterized by the above-mentioned. In claim 1,
When a scene change is detected in the B frame, for the B frame existing between the B frame and the reference frame after the replacement, the difference data is replaced with zero, and the prediction direction information is replaced by the prediction direction information. Output by replacing with the one indicating the reference frame after the replacement from the one indicating the set reference frame,
A moving picture coding method characterized by the above-mentioned. A series of frame moving image signals including an I frame, a P frame, and a B frame are input, an intra-frame encoding is performed for the I frame, and a preset reference destination is assigned to the P frame and the B frame. A moving image encoding apparatus that performs inter-frame encoding in which prediction direction information indicating the reference destination frame is combined with encoded data obtained by encoding difference data from a frame and output.
Scene change detection means for detecting a frame in which a scene change is performed among frames constituting the series of frame moving image signals,
When the frame in which a scene change is detected by the scene change detecting means is the B frame, the difference data for the B frame is replaced with zero, and the prediction direction information of the B frame is set to the set reference destination. Replacement means for replacing an I or P frame whose display order is closer to the B frame from a frame indicating the frame with a frame indicating the I or P frame as a reference frame;
A moving picture coding apparatus comprising: In claim 3,
When the frame in which the scene change is detected by the scene change detecting unit is the B frame, the replacing unit also stores the difference data for the B frame existing between the B frame and the reference frame after the replacement. Is replaced with zero, and the prediction direction information is output by replacing the one indicating the set reference frame with the one indicating the reference frame after the replacement,
A moving picture coding apparatus characterized by the above-mentioned.

Images