JP2015033105A - Motion picture encoding device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a motion picture encoding device capable of effectively reducing a data bus access amount without significantly deteriorating image quality.SOLUTION: A search range buffer 102 stores reference image data within a search range for motion vector detection from a reference frame buffer 211. A decimal pixel position prediction image creation part 103 creates a decimal pixel position prediction image by means of a six-tap filter. A decimal pixel position prediction image creation part 104 creates a decimal pixel position prediction image by means of a two-tap filter. A selection part 105 selects, as a prediction image of a decimal pixel position, output of the creation part 103 when encoding I and P pictures and selects output of the creation part 104 when encoding B pictures.

Description

  The present invention relates to a moving image encoding apparatus that compresses and encodes moving image data, and more specifically to a moving image encoding apparatus that performs motion vector detection processing in units of encoded blocks.

  In recent years, higher resolution and higher frame rate of moving images have been advanced. For example, in a digital video camera or the like, a product that handles 60 frames / second progressive video called 60P with a HD (High Definition) image having 1920 × 1080 pixels has already been put on the market.

  Higher resolution and higher frame rate are expected to continue. In the future, it is expected to handle an image of 4096 pixels × 2160 pixels called a 4k image, an image of 7680 pixels × 4320 pixels called a super high vision, and a high frame rate image such as 120 frames / second.

  When such high resolution and / or high frame rate image data is encoded, the access amount and the processing amount to the image memory increase with an increase in the number of pixels and / or the number of frame rates.

  Various methods have been proposed as compression encoding methods for moving image data, but motion compensated prediction methods have become mainstream in both encoding efficiency and image quality. As one of the typical moving image compression coding systems, H.264 H.264 encoding method. H. The H.264 encoding method is adopted for AVCHD, which is a high-definition recording method for video cameras, and one-segment broadcasting for terrestrial digital broadcasting, and is widely spread.

  H. In the H.264 encoding method, an image is divided into 16 pixels × 16 pixels called a macroblock, and encoding is performed in units of the macroblock. That is, main processing of encoding, for example, motion vector detection, frequency conversion, quantization processing, variable length encoding processing, and the like are executed on a macroblock basis.

  Among these processes, the motion vector detection process has the largest processing amount and the largest access amount to the image memory. The motion vector detection process is a process in which block matching is performed between a reference image and an encoding target image with respect to a macroblock to be encoded, and a position where the two match most is detected as a motion vector. Encoding efficiency can be increased by performing motion compensation processing using the detected motion vector.

  H. In the H.264 encoding method, 1/2 pixel accuracy and 1/4 pixel accuracy are used as motion vector accuracy in addition to integer pixel accuracy. These pixels at the decimal pixel positions are generated by interpolating the pixels at the integer pixel positions. In the MPEG-2 encoding method or the like, two pixels at integer pixel positions are linearly interpolated as an interpolation process for generating pixels at 1/2 pixel positions. In the H.264 encoding method, a 6-tap filter is applied to 6 pixels at integer pixel positions around a 1/2 pixel position. In the 6-tap filter process, the reference image must be read from the image memory in a range wider than the motion search range, and the amount of data access increases compared to the MPEG-2 encoding method.

  The amount of data flowing on the data bus becomes very large due to an increase in the data access amount. For example, in a high resolution and high frame rate image, a desired performance cannot be satisfied due to a bus bottleneck. In order to avoid the bus bottleneck, it is necessary to reduce the access amount to the data bus as much as possible.

  As a method for reducing the access amount to the data bus, Patent Document 1 describes a method for reducing the read amount of a reference image at the time of motion vector detection. Specifically, interpolation processing for creating a predicted image at the time of motion vector detection is simply performed using a filter with a small number of taps at the end of the search range.

JP 2007-124238 A

  In the conventional motion compensated prediction encoding method, a reference image is generated by a prediction image generation unit that calculates a difference between a prediction image and an encoding target image, separately from the generation of a prediction image for motion vector detection. In addition, the difference image generation unit generates the predicted image by the 6-tap filter process according to the standard. Therefore, even if the motion vector detection means employs filter processing with a small number of taps for creating a predicted image, a situation may arise in which a pixel outside the search range is required in order to create a predicted image in the predicted image creating unit. That is, the effect of reducing the data bus access amount is limited.

  An object of the present invention is to provide a moving picture coding apparatus that can effectively reduce the amount of data bus access.

  The moving picture encoding apparatus according to the present invention is a motion prediction unit that predicts a moving picture with a decimal pixel accuracy based on a reference picture, and a motion between the encoding target picture in the moving picture and the reference picture. A motion prediction means for outputting a vector, a prediction image obtained from the reference image at the position of the motion vector, a difference image of the encoding target image with respect to the prediction image, and an encoding for encoding the difference image Means, decoding means for decoding the output of the encoding means, addition means for adding the predicted image to the output of the decoding means, and a reference frame for storing the output image of the addition means as the reference image A motion picture encoding apparatus comprising: a buffer, wherein the motion prediction means is configured such that when the encoding target image is a non-reference image, the encoding target image is reference image data. Characterized by including a generation means for generating a predicted image of the sub-pel position from the reference image small number of taps in than that.

  According to the present invention, it is possible to reduce the read amount of a reference image for creating a predicted image without significantly degrading the image quality, and to reduce the data access amount.

It is a schematic block diagram of the motion estimation part of Example 1 of this invention. 1 is a block diagram of a schematic configuration of a moving image encoding apparatus according to Embodiment 1 of the present invention. It is explanatory drawing of the prediction image creation method of the decimal pixel position by 6 tap filter. It is explanatory drawing of the prediction image creation method of the decimal pixel position by 2 tap filter. It is explanatory drawing of a picture type and a reference relationship. It is explanatory drawing of the range required for prediction image preparation. It is a schematic block diagram of the motion estimation part of Example 2 of this invention. It is a flowchart of the prediction image selection process based on the bus congestion degree and picture type of Example 2 of this invention.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 1 shows a schematic configuration block diagram of an embodiment of a moving picture encoding apparatus according to the present invention applied to an H.264 encoding system, and FIG. 1 shows a schematic configuration block diagram of a motion prediction unit which is a characteristic part of this embodiment. . In the present embodiment, the moving image data is divided into encoded blocks having a predetermined number of pixels such as macroblocks, and then compressed and encoded by the motion compensated prediction encoding method in units of encoded blocks.

  In the frame buffer 201, input image data to be encoded (encoding target image data) is written from an image input unit (for example, an imaging unit) (not shown). Although the details will be described later with reference to FIG. 2, the motion prediction unit 202 detects a motion vector of an encoding block of the encoding target image data and outputs difference image data indicating a difference from the prediction image data. To do. The motion prediction unit 202 supplies the calculated difference image data to the orthogonal transform unit 203, the prediction image data used for the difference image calculation to the adder 209, and the motion vector to the entropy encoding unit 206.

  The orthogonal transform unit 203 performs discrete cosine transform on the difference image data from the motion prediction unit 202 and supplies transform coefficients to the quantization unit 205. The quantization unit 205 quantizes the transform coefficient from the orthogonal transform unit 203 with a quantization step size specified by the quantization control unit 204. The quantization unit 205 supplies the quantized transform coefficient obtained by the quantization to the entropy coding unit 206 and supplies it to the inverse quantization unit 207 of the local decoder.

  The entropy encoding unit 206 performs entropy encoding by performing a zigzag scan, an alternate scan, or the like on the quantized transform coefficient from the quantization unit 205. The entropy encoding unit 206 adds encoding method information such as a motion vector, a quantization step size, and encoding block division information from the motion prediction unit 202 to code data generated by entropy encoding, and encodes the encoded data. Create a stream. In addition, the entropy encoding unit 206 calculates a generated code amount for each encoding block and outputs it to the quantization control unit 204.

  The quantization control unit 204 determines the quantization step size so that the subsequent encoded block is encoded with the target code amount according to the generated code amount information from the entropy encoding unit 206, and outputs the quantization step size to the quantization unit 205. To do.

  The inverse quantization unit 207 inversely quantizes the transform coefficient quantized by the quantization unit 205 and supplies the transform coefficient (representative value) to the inverse orthogonal transform unit 208. The inverse orthogonal transform unit 208 performs inverse discrete cosine transform on the output of the inverse quantization unit 207 to restore difference image data, and supplies the difference image data to the adder 209.

  The adder 209 adds the predicted image data from the motion prediction unit 202 to the restored difference image data from the inverse orthogonal transform unit 208 to restore the image data. The restored image data is deblocked by the deblocking filter 210 and stored in the reference frame buffer 211 as a local decoded image. The inverse quantization unit 207, the inverse orthogonal transform unit 208, the adder 209, and the deblocking filter 210 constitute a local decoder.

  In this embodiment, an I picture for intra coding, a P picture for forward prediction coding, and a B picture for bidirectional prediction coding can be selected. Each frame image of the input image is encoded in a predetermined order as an I, P, B picture with a predetermined number of frames as a unit (so-called GOP (Group Of Pictures)). The encoding control unit 212 designates the picture type of the frame image to the motion prediction unit 202.

  With reference to FIG. 1, the configuration and operation of the motion prediction unit 202 will be described. In the encoded image buffer 101, an encoded block of image data to be encoded is read from the frame buffer 201 and stored. Although the details will be described later, local search image data in a slightly wider range including the range necessary for motion vector detection is read from the reference frame buffer 211 and stored in the search range buffer 102.

  The 6-tap decimal pixel position predicted image creation unit 103 generates predicted image data by applying a 6-tap filter to the reference image data in the search range buffer 102. Further, the 2-tap decimal pixel position predicted image creation unit 104 generates predicted image data by applying a 2-tap filter to the reference image data in the search range buffer 102. When detecting a motion vector with integer pixel accuracy, the selection unit 105 reads out a predetermined number of pixel data serving as predicted image data from the search range buffer 102 and supplies the pixel data to the motion vector detection unit 106. The selection unit 105 also supplies predicted image data created by one of the decimal pixel position predicted image creation units 103 and 104 to the motion vector detection unit 106 when detecting a motion vector with decimal pixel accuracy.

  The motion vector detection unit 106 matches the predicted image data from the selection unit 105 with the encoded block of the encoded image buffer 101 and calculates a difference. By calculating the difference value while sweeping the range necessary for the predicted image from the search range of the search range buffer 102, the motion vector detection unit 106 is the motion that indicates the position on the reference image that is most similar to the coding block. The vector can be determined.

As described above, the decimal pixel position predicted image creation unit 103 reads pixel values within a predetermined range of the reference image from the search range buffer 102 and applies a 6-tap filter (number of taps = 6) to 1/2. A predicted image with pixel accuracy is generated. Using a 6-tap filter is It is defined by the H.264 encoding method. That is, as shown in FIG. 3, the decimal pixel position predicted image creation unit 103 uses 6 pixels a to f at the peripheral integer positions on the same line in creating the pixel X at the 1/2 pixel position. The value of the pixel X is the following formula (1):
X = (a-5 * b + 20 * c + 20 * d-5 * e + f + 16) >> 5 (1)
It is shown. “>> 5” indicates a 5-bit shift operation on the right.

On the other hand, the decimal pixel position predicted image creation unit 104 reads a predetermined range of pixel values from the search range buffer 102 and creates a predicted image with 1/2 pixel accuracy by applying a 2-tap filter (number of taps = 2). That is, as shown in FIG. 4, the decimal pixel position predicted image creation unit 104 uses two pixels c and d at adjacent integer positions on the same line in creating the pixel X at the ½ pixel position. The value of the pixel X is the following formula (2):
X = (c + d + 1) >> 1 (2)
It is shown. “>> 1” indicates a 1-bit shift operation on the right.

  The decimal pixel position predicted image creation unit 103 creates a predicted image with 1/2 pixel accuracy using 6 pixels at integer pixel positions, whereas the decimal pixel position predicted image creation unit 104 has two pixels at integer pixel positions. Is used to generate a reference image with 1/2 pixel accuracy. Therefore, the decimal pixel position predicted image creation unit 103 requires reference image data in a wide range outside the range required by the decimal pixel position predicted image creation unit 104 by two pixels. In other words, in the predicted image creation by the decimal pixel position predicted image creation unit 104, the read amount of the pixel value from the search range buffer 102 is significantly smaller than that of the decimal pixel position predicted image creation unit 103. However, since the predicted image data created by the decimal pixel position predicted image creation unit 104 does not comply with the encoding standard, an error occurs between the decoded image on the receiving side and the local decoded image.

  When a motion vector is to be detected at an integer pixel position, the selection unit 105 selects predicted image data with integer pixel accuracy read from the search range buffer 102 and supplies the predicted image data to the motion vector detection unit 106. When the motion vector is to be detected at the decimal pixel position, the selection unit 105 selects the predicted image data at the decimal pixel position from the decimal pixel position predicted image creation unit 103 or 104 according to the picture type information from the encoding control unit 212. To the motion vector detecting unit 106.

  H. In the H.264 encoding method, there are a picture that is referred to when a subsequent picture is encoded, that is, a picture that is used to generate a predicted image for predictive encoding, and a picture that is not referred to. Normally, the I picture and the P picture are referred to when the subsequent picture is encoded, while the B picture is not referred to when the subsequent picture is encoded.

  FIG. 5 shows an example of the picture type and the reference relationship. In FIG. 5, pictures 501 to 507 are arranged in order from the left in the display order. “I”, “P”, and “B” indicate picture types, and the numbers added after these indicate the order of encoding. For example, the picture 501 is an I picture and is encoded first.

  The P picture 504 is encoded with reference to the I picture 501. B picture 502 and B picture 503 are encoded with reference to I picture 501 for forward prediction and P picture 504 for backward prediction. The P picture 507 is encoded with reference to the P picture 504. The B picture 505 and the B picture 506 are encoded with reference to the P picture 504 for forward prediction and the P picture 507 for backward prediction. As described above, the I picture and the P picture are referred to when the subsequent picture is encoded, but the B picture is not referred to.

  If an error occurs between a local decoded image of a picture referenced at the time of encoding a subsequent picture and a decoded image on the receiving side, the error accumulates due to repeated reference, resulting in deterioration in image quality. On the other hand, for a picture that is not referred to when the subsequent picture is encoded, even if an error occurs between the local decoded image and the decoded image on the receiving side, the error only affects the picture. Little impact on image quality. Therefore, in this embodiment, a simple decimal pixel position predicted image creation method is adopted for a picture that is not referred to when a subsequent picture is encoded. Hereinafter, a picture (image) that can be referred to when a subsequent picture (subsequent image) is encoded is referred to as a reference picture (reference image), and a non-reference picture (image) is referred to as a non-reference picture (non-reference image). ).

  Therefore, in this embodiment, when detecting a motion vector at a decimal pixel position, the selection unit 105 outputs the outputs of the decimal pixel position predicted image creation units 103 and 104 in the following manner in consideration of reduction of the memory access amount. To choose.

  For the reference picture, the selection unit 105 selects the output image of the decimal pixel position predicted image creation unit 103 when detecting the motion vector at the decimal pixel position. That is, when encoding an I picture and a P picture, a 6-tap decimal pixel position predicted image creation unit 103 that is guaranteed to be the same as the predicted image creation method on the receiving side is employed.

  On the other hand, for the non-reference picture, the selection unit 105 selects the output image of the 2-tap decimal pixel position predicted image creation unit 104 when detecting the motion vector at the decimal pixel position. That is, when encoding B, a 2-tap decimal pixel position predicted image creation unit 104 that can reduce the memory access amount is employed, although it is not the same as the predicted image creation method on the receiving side.

  The search range buffer 102 reads, from the reference frame buffer 211, reference image data in the search range necessary for the motion vector detection unit to detect motion for the encoded block. When a predicted image at a decimal pixel position is created with a 6-tap filter, pixel data for two pixels is further required in the vertical and horizontal directions. For example, when the size of the coding block is 16 pixels × 16 pixels and the search range is three pixels, top, bottom, left, and right, image data of (16 + 6) × (16 + 6) pixel range as the search range is stored in the search range buffer 102. Necessary. However, when a predicted image is generated using a 6-tap filter, a reference image of 2 pixels is required in the upper, lower, left, and right sides outside this search range, so that the search range buffer 102 eventually has (16 + 6 + 4) × (16 + 6 + 4). ) Image data in the pixel range is required. On the other hand, when the 2-tap filter is used, reference image data outside the search range is not necessary, and the search range buffer 102 only needs to have image data in the (16 + 6) × (16 + 6) pixel range.

  FIG. 6 is a schematic diagram illustrating a coding block, a search range for motion detection, and a range necessary for creating a predicted image by the creation units 103 and 104. In FIG. 6, the search range is ± 3 pixels both vertically and horizontally at integer pixel positions. 6A shows a range required by the fractional pixel position predicted image creation unit 103 of the 6-tap filter, and FIG. 6B shows a range required by the fractional pixel position predicted image creation unit 104 of the 2-tap filter. Indicates the range.

  In the example illustrated in FIG. 6, the 6-tap filter decimal pixel position prediction image creation unit 103 requires 26 pixels × 26 pixels (= 676 pixels), whereas the 2-tap filter decimal pixel position prediction image creation unit 104 may be 22 pixels × 22 pixels (= 484 pixels). Therefore, in the non-reference picture (B picture), data access corresponding to 192 pixels (= 676 pixels-484 pixels) can be reduced compared to the reference pictures (I, P pictures). This is a reduction rate of 28% (= (192/676) × 100%).

  The motion vector detection unit 106 detects a position closest to the encoded block within the search range as a motion vector with decimal precision by block matching. After this motion vector detection, the motion vector detection unit 106 generates a predicted image at the motion vector position by the decimal pixel position predicted image creation units 103 and 104 and the selection unit 105, and generates a difference image from the encoded block. Finally, the motion vector detection unit 106 supplies the detected motion vector to the entropy encoding unit 206, the difference image data to the orthogonal transformation unit 203, and the prediction image data of the motion vector position to the adder 209.

  Depending on whether the picture to be encoded is a reference picture or a non-reference picture, the use of a 6-tap filter in the former case and the use of a 2-tap filter in the latter case significantly impairs image quality. Therefore, the data access amount can be reduced.

  As an image compression coding method, H.264 is used. Although the embodiment using the H.264 encoding method has been described, the above embodiment is applicable to a motion compensation prediction encoding method in general.

  Moreover, although the method using a 2-tap filter has been described as a predicted image creation method that does not comply with the standard, this is an example for explanation. Other prediction image creation methods can be applied as long as the data access amount can be reduced as compared with the prediction image creation method compliant with the standard.

  An embodiment will be described which is modified so as to take into consideration the degree of congestion of the data bus. FIG. 7 shows a schematic block diagram of such a modified embodiment. The same components as those in FIG. 1 are denoted by the same reference numerals.

  A motion prediction unit 701 shown in FIG. 7 is arranged in place of the motion prediction unit 202 in the encoding device shown in FIG. The motion prediction unit 701 includes a bus congestion degree determination unit 702. In addition to the function of the encoded image buffer 101, the encoded image buffer 703 supplies information on the number of processing cycles for reading encoded image data from the frame buffer 201 to the bus congestion degree determination unit 702. In addition to the function of the search range buffer 102, the search range buffer 704 supplies information on the number of processing cycles for reading reference image data from the reference frame buffer 211 to the bus congestion degree determination unit 702.

  Since the number of processing cycles for reading the encoded image data and the reference image data varies depending on the congestion degree of the data bus, the bus congestion degree can be determined by referring to these. The bus congestion degree determination unit 702 determines that the data bus congestion degree is low when the number of read processing cycles of the encoded image data and the reference image data is smaller than a predetermined threshold value. On the other hand, when either or both of the read processing cycle numbers of the encoded image data and the reference image data are equal to or greater than a predetermined threshold value, the bus congestion degree determination unit 702 determines that the data bus congestion degree is high.

  The determination by the bus congestion degree determination unit 702 may be performed based on the number of processing cycles for each coding block. For example, the determination may be made on the basis of the number of processing cycles in units of a plurality of encoded blocks, such as the number of processing cycles at the time when the processing of 10 encoded blocks is completed. Although the bus congestion degree is determined using the number of read processing cycles of the encoded image data and the reference image, other methods that can determine the data bus congestion degree may be used.

  The selection unit 705 selects the output of the creation unit 103 or 104 as predicted image data at the decimal pixel position according to the picture type and the determination result of the bus congestion degree determination unit 702 and supplies the selected output to the motion vector detection unit 106.

  When the data bus congestion is low, the processing can be performed in time without reducing the data access amount, and therefore it is preferable to select a predicted image created by a method compliant with the coding standard. From this viewpoint, when the bus congestion degree determination unit 702 determines that the data bus congestion degree is low, the selection unit 705 selects the output of the decimal pixel position predicted image creation unit 103 regardless of the picture type.

  On the other hand, when the degree of data bus congestion is high, processing may not be in time within a predetermined time unless the data access amount is reduced. Accordingly, when the bus congestion degree determination unit 702 determines that the data bus congestion level is high, the selection unit 705 determines whether the sub-pixel position predicted image creation unit 103 or the The output of the same 104 is selected. That is, the selection unit 705 selects the output of the decimal pixel position predicted image creation unit 103 for the I and P pictures that are reference pictures, and creates the decimal pixel position predicted image for the B picture that is a non-reference picture. The output of the unit 104 is selected.

  FIG. 8 shows a flowchart of selection of a decimal pixel position predicted image based on the bus congestion level and the reference picture / non-reference picture.

  The bus congestion degree determination unit 702 determines the data bus congestion degree from the number of processing cycles required for reading the encoded image data and the reference image data (S801). When the bus congestion degree determination unit 702 determines that the data bus congestion degree is low (S802), the selection unit 705 outputs the output from the fractional pixel position prediction image creation unit 103 of the 6-tap filter which is a method compliant with the encoding standard. Is selected (S803).

  When the bus congestion degree determination unit 702 determines that the data bus congestion degree is high (S802), the selection unit 705 determines whether or not the encoding target picture is a reference picture (S804). When the encoding target picture is a reference picture (S804), the selection unit 705 selects the output of the fractional pixel position predicted image creation unit 103 of the 6-tap filter that is a method compliant with the encoding standard (S805).

  When the encoding target picture is a non-reference picture, the selection unit 705 selects the output of the fractional pixel position predicted image creation unit 104 of the 2-tap filter (S806).

  As shown in FIG. 8, when the data bus is not congested, it is possible to encode and decode without impairing the image quality by adopting a predicted image using a predicted image creation method compliant with the encoding standard. Further, when the data bus is congested, it is possible to reduce the data access amount while suppressing the deterioration of the image quality, thereby making it easy to satisfy the desired processing performance. That is, according to the degree of congestion of the data bus, the data access amount is reduced without losing the image quality as much as possible, and the encoding operation that satisfies the desired processing performance is realized.

  Obviously, various modifications may be made without departing from the scope of the invention as defined in the claims.

Claims (5)

A motion prediction means for predicting a motion image with decimal pixel accuracy based on a reference image, the motion vector between the reference image of the encoding target image in the motion image, and the reference at the position of the motion vector Motion prediction means for outputting a predicted image obtained from an image and a difference image of the encoding target image with respect to the predicted image;
Encoding means for encoding the difference image;
Decoding means for decoding the output of the encoding means;
Adding means for adding the predicted image to the output of the decoding means;
A video encoding device comprising a reference frame buffer for storing an output image of the adding means as the reference image,
When the encoding target image is a non-reference image, the motion prediction unit calculates a predicted image at a decimal pixel position from the reference image with a smaller number of taps than when the encoding target image is a reference image. A moving picture coding apparatus comprising a creating means for creating. Furthermore, it has a judging means for judging the degree of congestion of the data bus,
When the data bus has a low degree of congestion, the creating means counts the number of taps when the encoding target image is a reference image regardless of whether the encoding target image is a non-reference image or a reference image. The video encoding apparatus according to claim 1, wherein a predicted image at a decimal pixel position is created from the reference image.   The said creating means creates the prediction image of the said decimal pixel position with the number of taps based on an encoding standard, when the said encoding object image is a reference image. Video encoding device.   The moving picture encoding apparatus according to claim 3, wherein the number of taps conforming to the encoding standard is six.   When the encoding target image is a non-reference image, the generation unit generates a predicted image at the decimal pixel position from the reference image by a 2-tap filter, and the encoding target image is a reference image. 5. The moving picture encoding apparatus according to claim 1, wherein a predicted image at a decimal pixel position is created from the reference image by a 6-tap filter.

Images