JP2011176503A - Image encoding device, method, program, and integrated circuit - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image encoding device which suppresses the increase in capacity of an adjacent MB information memory in the case that an input image size becomes larger. <P>SOLUTION: The image encoding device 100 includes: a storage part 160a which stores therein adjacent macro block information relating to an already encoded macro block; an encoding part 161a which encodes a target macro block to be encoded, on the basis of the adjacent macro block information; and an encoding control part 200 which, in accordance with an image size, changes an encoding system in the encoding part and changes the number of pieces of information capable of being stored per prescribed area in the storage part. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to an image encoding device, an image encoding method, an image encoding program, and an image encoding integrated circuit that encode data relating to an image, and to an image encoding technique that encodes images of a plurality of sizes.

  In recent years, with the development of digital video technology, the amount of video data to be handled, particularly the amount of moving image data, has been increasing. For example, HD (High Definition) video that has recently been put into practical use has a data amount that is about six times that of conventional SD (Standard Definition) video.

  On the other hand, with the improvement of information processing capabilities of computers and other devices, moving image compression using complex operations is possible, and the compression rate of video data has been greatly increased. H. has been attracting attention in recent years. H.264 / AVC is a standard that realizes a compression rate about twice that of MPEG-2. H. In H.264 / AVC, a high compression rate is realized by combining many compression techniques. For this reason, compared with the conventional compression system, the amount of calculation is also increasing significantly (refer nonpatent literatures 1 and 2 and patent documents 1).

FIG. 1 is a block diagram showing a functional configuration of a conventional image encoding device 500.
The image encoding apparatus 500 shown in FIG. 1 is an example of an image encoding device that performs encoding by H.264 / AVC. An outline of the operation of the image coding apparatus 500 will be described below.

  First, the image data to be encoded is input to the inter-screen prediction unit 501 and the intra-screen prediction unit 502 for each block called MB (MacroBlock).

FIG. 2 is a diagram illustrating an inter-screen prediction method.
As illustrated in FIG. 2, the inter-screen prediction unit 501 detects an image block having a high correlation with the encoding target MB from the already encoded pictures stored in the reference image storage unit 600. Further, a difference between the encoding target MB and the detected image block is calculated to obtain a prediction residual, and the prediction residual and a motion vector representing the motion amount of the block between pictures are sent to the mode selection unit 503. Output. A maximum of two pictures used for inter-screen prediction can be selected, and predictions using the two pictures are distinguished as L0 prediction and L1 prediction, respectively.

FIG. 3 is a diagram illustrating a prediction unit for inter-screen prediction.
Further, as shown in FIG. 3, inter-screen prediction can be performed not in units of 16 × 16 pixels, which is the size of MB, but in units of 8 × 16 pixels, 16 × 8 pixels, 8 × 8 pixels, etc., in which MB is divided into smaller blocks. is there.

FIG. 4 is a diagram illustrating adjacent MB pixels used for intra prediction.
The in-screen prediction unit 502 predicts the image of the encoding target MB using the pixels of the adjacent MBs on the left, upper left, upper, and upper right of the encoding target MB that are already encoded as shown in FIG. To do. Further, the difference between the image of the encoding target MB and the generated predicted image is calculated to obtain a prediction residual, and the prediction residual and the intra-screen prediction mode information are output to the mode selection unit 503.

  The mode selection unit 503 selects the encoding target MB based on the data output from each of the inter-screen prediction unit 501 and the intra-screen prediction unit 502 by any prediction method of inter-screen prediction and intra-screen prediction. Select whether to encode. Then, the prediction residual obtained by the selected prediction method is output to the orthogonal transform unit 504, and when the inter-screen prediction is selected, the motion vector is selected. When the intra-screen prediction is selected, the prediction residual is calculated. Each prediction mode information is output to the prediction information encoding unit 520.

  The orthogonal transform unit 504 performs orthogonal transform on the input prediction residual and calculates an orthogonal transform coefficient obtained by orthogonally transforming the prediction residual. The orthogonal transform coefficient is output to the quantization unit 505.

  The quantization unit 505 first quantizes the input orthogonal transform coefficient, and outputs the quantized orthogonal transform coefficient to the binarization unit 506 and the inverse quantization unit 510.

  When inter-mode prediction is selected by the mode selection unit 503, the prediction information encoding unit 520 first moves in the adjacent MBs on the left, upper left, upper, and upper right of the encoding target MB that have already been encoded. Based on the vector information, an estimated motion vector is calculated. Then, a difference value between the input motion vector and the calculated estimated motion vector is obtained, and the obtained difference value is output to the binarization unit 506.

  On the other hand, when the intra-screen prediction is selected by the mode selection unit 503, the prediction information encoding unit 520 firstly predicts the intra-screen prediction in the left and upper neighboring MBs that have already been encoded. Based on the mode information, the estimated intra prediction mode is calculated. Then, the intra prediction mode used for encoding and information on whether or not the prediction modes match are output to the binarization unit 506. Here, the output information on whether or not the prediction modes match is information on whether or not the in-screen prediction mode used for encoding matches the calculated estimated in-screen prediction mode. .

  Also, the motion vector information or the intra prediction mode information input from the mode selection unit 503 is at least one code among the right, lower left, lower, and lower right neighboring MBs of the encoding target MB. Stored in the memory inside the prediction information encoding unit 520.

  The inverse quantization unit 510 performs inverse quantization on the input quantized orthogonal transform coefficient, and outputs the orthogonal transform coefficient obtained by the inverse quantization to the inverse orthogonal transform unit 511.

  The inverse orthogonal transform unit 511 performs inverse orthogonal transform on the input orthogonal transform coefficient, and outputs the prediction residual generated by the inverse orthogonal transform to the reconstructed image generation unit 512.

  The reconstructed image generation unit 512 includes a prediction image generated by the method selected by the mode selection unit 503 among the inter-screen prediction and the intra-screen prediction, and the prediction residual input by the inverse orthogonal transform unit 511. Are added to generate a reconstructed image of the input image. Then, the reconstructed image generation unit 512 outputs the generated reconstructed image to the intra-screen prediction unit 502 and the deblock processing unit 514, respectively.

  The intra-screen prediction unit 502 is an encoding target that is necessary for the intra-screen prediction process of encoding each of the adjacent MBs on the right, lower left, lower, and lower right of the encoding target MB in the input reconstructed image. The pixels in the rightmost column in the MB and the pixels in the lowermost line are held in the memory inside the intra prediction unit 502.

FIG. 5 is a diagram illustrating pixels of adjacent MBs used for the deblocking process.
The deblocking processing unit 514 uses the pixels of the adjacent MBs above and to the left of the encoding target MB shown in FIG. 5 and the input reconstructed image, and the MB boundary and the unit of 4 × 4 pixels in the MB. The deblocking process is performed on the block boundary, and the image data after the deblocking process is output to the reference image storage unit 600 as decoded image data. In addition, the pixels on the rightmost 4 columns and the pixels on the lowermost 4 lines in the encoding target MB, which are used when encoding the adjacent MBs on the right side and the lower side of the encoding target MB, are stored in the memory inside the deblock processing unit 514. Hold. The decoded image data output to the reference image storage unit 600 is referred to by the inter-screen prediction unit 501 when encoding subsequent pictures.

  On the other hand, the binarizing unit 506 converts the quantized orthogonal transform coefficient input from the quantizing unit 505 and the encoded prediction information input from the prediction information encoding unit 520 into binary data. And the converted binary data is output to the arithmetic encoding unit 507.

  The arithmetic encoding unit 507 encodes the input binary data, and outputs a stream obtained by encoding the binary data from the image encoding apparatus 500 as an encoded stream.

FIG. 6 is a diagram illustrating a configuration of the arithmetic encoding unit 507.
The arithmetic coding unit 507 includes a context index calculation unit 551, a symbol occurrence probability information holding unit 552, and a binary arithmetic coding unit 553.

  The context index calculation unit 551 calculates a context index (hereinafter referred to as ctxIdx) indicating symbol occurrence probability information used for binary arithmetic coding according to the input binary symbol syntax, and the calculated ctxIdx is used as a symbol. Output to the occurrence probability information holding unit 552.

  The symbol occurrence probability information holding unit 552 holds symbol occurrence probability information corresponding to each ctxIdx, and outputs the symbol occurrence probability information corresponding to the inputted ctxIdx to the binary arithmetic encoding unit 553.

  The binary arithmetic encoding unit 553 performs a binary arithmetic encoding process based on the input binary symbol and the symbol occurrence probability information, and outputs the stream subjected to the binary arithmetic encoding process as a code stream. To do.

Here, in the process of calculating ctxIdx in the context index calculation unit 551, in the calculation of ctxIdx for a part of the syntax, the same syntax in the left and upper neighboring MBs of the encoding target MB Information is required. Such syntax includes
Mb_skip_flag (information indicating whether MB is skip-encoded)
Mb_field_decoding_flag (information indicating whether MB is field-encoded or frame-encoded)
Mb_type (MB coding type information)
Coded_block_pattern (appearance pattern information of coefficient after quantization)
Ref_idx_10 (information for specifying a picture to be referenced in L0 inter-screen prediction)
Ref_idx_l1 (information for specifying a picture to be referred to in inter-L1 prediction)
Mvd_l0 (motion vector information in inter-L0 screen prediction)
Mvd_l1 (motion vector information in L1 inter-frame prediction)
Intra_chroma_pred_mode (color difference in-screen prediction mode information)
Coded_block_flag (information indicating presence / absence of coefficient after quantization),
There is. When binary data of these syntaxes is input to the context index calculation unit 551, ctxIdx is determined by referring to information on the same syntax in the left and upper adjacent MBs of the encoding target MB. The syntax information is stored in the memory inside the context index calculation unit 551 because the syntax information is used for encoding the right and lower adjacent MBs of the encoding target MB.

JP 2007-251865 A

ITU-T Recommendation H.264, Advanced Video Coding for generic audiovisual services. 2003. ISO / IEC 14496-10: Coding of audio-visual objects-Part 10: Advanced Video Coding. 2003.

FIG. 7 is a diagram illustrating an MB that holds information as adjacent MB information.
In the conventional image encoding apparatus 500, the intra prediction process in the intra prediction unit 502, the prediction mode encoding process in the prediction information encoding unit 520, the deblocking process in the deblock processing unit 514, the context index In the ctxIdx calculation process in the calculation unit 551, information on at least one MB among the left, upper left, upper, and upper right adjacent MBs of the encoding target MB is required. For this reason, as shown in FIG. 7, it is necessary to store information on (number of MBs per MB line + 1) number of encoded MBs in a memory. For this reason, there has been a problem that the capacity of a memory for holding adjacent MB information used for encoding must be increased in proportion to the width of the encoded image size (the width 200a in FIG. 7).

  The present invention solves such a conventional problem, and limits the encoding method according to the image size of an input image, and in conjunction therewith, can be stored per predetermined area of the memory. An object of the present invention is to provide an image encoding device that suppresses an increase in the capacity of an adjacent MB information memory when the input image size increases by changing the number of adjacent MB information.

  In order to achieve the above object, an image encoding device of the present invention is an image encoding device that encodes an input image, and is an already encoded macro among information included in the input image. An adjacent macroblock information related to a block, the storage unit storing adjacent macroblock information referred to when encoding the encoding target macroblock, and the encoding target macro based on the adjacent macroblock information The encoding unit that encodes a block, and the encoding method in the encoding unit is changed according to the image size of the input image, and the adjacent unit that can store per predetermined area in the storage unit And an encoding control unit that changes the number of macroblock information.

  The image encoding method of the present invention is an image encoding method for encoding an input image, and among information included in the input image, adjacent macroblock information relating to an already encoded macroblock. A storage unit that stores adjacent macroblock information referred to when encoding the macroblock to be encoded, and the macroblock to be encoded based on the adjacent macroblock information. According to the encoding step encoded by the encoding unit and the image size of the input image, the encoding method in the encoding unit is changed, and the storage unit can store per predetermined area. An image encoding method including an encoding control step of changing the number of adjacent macroblock information may be used.

  An image encoding program for causing a computer to execute the above encoding method may be constructed.

  Further, an image encoding integrated circuit having a configuration for executing the above encoding method may be constructed.

  According to the image encoding device, the image encoding method, the image encoding program, and the image encoding integrated circuit of the present invention, the encoding method is limited in accordance with the image size of the input image, and in conjunction therewith. Thus, by changing the number of adjacent MB information that can be accumulated per predetermined area of the memory, it is possible to suppress an increase in the capacity of the adjacent MB information memory when the input image size increases.

FIG. 1 is a block diagram showing a functional configuration of a conventional image coding apparatus. FIG. 2 is a diagram illustrating an inter-screen prediction method. FIG. 3 is a diagram illustrating a prediction unit for inter-screen prediction. FIG. 4 is a diagram illustrating adjacent MB pixels used for intra prediction. FIG. 5 is a diagram illustrating pixels of adjacent MBs used for the deblocking process. FIG. 6 is a block diagram showing a functional configuration of an arithmetic coding unit in a conventional image coding apparatus. FIG. 7 is a diagram illustrating an MB that holds information as adjacent MB information. FIG. 8 is a block diagram showing a functional configuration of the image coding apparatus according to Embodiment 1. FIG. 9 is a block diagram illustrating a functional configuration of the arithmetic encoding unit of the image encoding device according to the first embodiment. FIG. 10 is a block diagram illustrating a functional configuration of the image encoding device according to the second embodiment. FIG. 11 is a block diagram illustrating a functional configuration of the arithmetic encoding unit of the image encoding device according to the second embodiment. FIG. 12 is a diagram illustrating an MBAFF encoding method. FIG. 13 is a diagram showing the upper adjacent MB at the time of MBAFF encoding. FIG. 14 is a diagram illustrating upper adjacent pixels used for intra-screen prediction at the time of MBAFF encoding. FIG. 15 is a diagram illustrating upper adjacent pixels used in the deblocking process at the time of MBAFF encoding. FIG. 16 is a diagram illustrating an encoding control unit, an inter-screen prediction unit, and a context index calculation unit. FIG. 17 is a flowchart of the image encoding device.

The best mode for carrying out the present invention will be described below with reference to the drawings.
The image encoding device (the image encoding device 100, the image encoding device 300) of the embodiment is an image encoding device that encodes an input image, and is already included in the information included in the input image. An accumulating unit (accumulating unit 160a (adjacent MB information memory 160) that accumulates adjacent macroblock information that is adjacent macroblock information related to an encoded macroblock and is referred to when encoding a macroblock to be encoded. ), An encoding unit (encoding processing unit 161a) that encodes the encoding target macroblock based on the adjacent macroblock information, and the encoding according to the image size of the input image Change the encoding method (whether or not to limit L1 prediction in the inter-screen prediction unit 201) in the storage unit (encoding processing unit 161a), and Kicking, it is an image encoding apparatus and an encoding control unit (coding control unit 200) to change the number of storable neighboring macroblock information per predetermined area.

(Embodiment 1)
FIG. 8 is a diagram illustrating a hardware configuration of the image encoding device 100 according to the first embodiment.

  The image encoding apparatus 100 includes an encoding control unit 200, an inter-screen prediction unit 201, an intra-screen prediction unit 502, a mode selection unit 503, an orthogonal transform unit 504, a quantization unit 505, a binarization unit 506, and an inverse quantization unit. 510, an inverse orthogonal transform unit 511, a reconstructed image generation unit 512, a deblock processing unit 513, a prediction information encoding unit 220, and an arithmetic encoding unit 207.

FIG. 9 is a block diagram showing a functional configuration of the arithmetic encoding unit 207 of FIG.
In addition, the arithmetic encoding unit 207 includes a context index calculation unit 251, a symbol occurrence probability information holding unit 552, and a binary arithmetic encoding unit 553 as illustrated in FIG. 9.

  Note that the intra-screen prediction unit 502, mode selection unit 503, orthogonal transform unit 504, quantization unit 505, binarization unit 506, inverse quantization unit 510, inverse orthogonal transform unit 511, reconstructed image generation unit 512, deblocking The processing unit 513, the symbol occurrence probability information holding unit 552, the binary arithmetic encoding unit 553, and the reference image storage unit 600 are each an image encoding device shown in FIG. 1 (and FIG. 2) shown as a conventional example. 500 and the arithmetic encoding unit 507 are the same as the components having the same reference numerals and names, and detailed description thereof will be omitted as appropriate.

Hereinafter, an image encoding method in the image encoding apparatus 100 will be described.
The encoding control unit 200 outputs, to the inter-screen prediction unit 201, control information related to the restriction of the inter-screen prediction method, based on the input image size information given from the outside, the prediction information encoding unit 220, and the arithmetic Control information for changing the number of adjacent macroblock information that can be accumulated per predetermined area of the memory is output to the encoding unit 207. As means for limiting the inter-screen prediction method, there are a limitation on L1 prediction (see the above description) and a block type.

First, a case where the encoding control unit 200 restricts L1 prediction will be described.
When restricting L1 prediction, the inter-screen prediction unit 201 implements encoding only by L0 prediction. Therefore, the prediction information encoding unit 220 does not need to store the motion vector information related to the L1 prediction in the memory, and can halve the memory area required to store the motion vector information. For this reason, by restricting L1 prediction, it is possible to hold motion vector information for 2 MB in a memory area that conventionally holds motion vector information for adjacent MBs for 1 MB.

  Further, when the L1 prediction is limited, the binary data input to the arithmetic coding unit 207 does not include the syntaxes ref_idx_l1 and mvd_l1 related to the L1 prediction. Therefore, the context index calculation unit 251 of the arithmetic coding unit 207 can halve the memory area necessary for holding the ref_idx and mvd information, compared to the case where the L1 prediction is not limited. For this reason, by limiting L1 prediction, it is possible to hold ref_idx and mvd information for 2 MB in a memory area that conventionally holds ref_idx and mvd information for 1 MB of adjacent MBs.

  Next, when the encoding control unit 200 restricts the block type of inter-screen prediction to only 16 × 16 pixel units, inter-screen using three types of block types of 16 × 16 pixel units, 16 × 8 pixel units, and 8 × 16 pixel units. This will be described in comparison with the case of making a prediction.

  When inter-screen prediction is performed in units of 16 × 8 pixels and 8 × 16 pixels, motion vector information for each region is calculated for each divided region in the MB. However, when the block type of inter-screen prediction is limited to only 16 × 16 pixel units, the prediction information encoding unit 220 does not need to store the motion vector information for each divided area in the MB, and the motion vector information The memory area required for holding can be halved. For this reason, by limiting the block type of inter prediction to only 16 × 16 pixel units, the motion vector information for 2 MB is held in the memory area that conventionally holds the motion vector information for the adjacent MB for 1 MB. It becomes possible.

  In addition, when performing inter-screen prediction in units of 16 × 8 pixels and 8 × 16 pixels, the syntax of ref_idx_10 / 1 and mvd_10 / 1 appears for each divided area in the MB. However, if the block type for inter-screen prediction is limited to only 16 × 16 pixel units, the information of ref_idx_10 / 1 and mvd_10 / 1 is one per MB. Therefore, the adjacent MB information holding unit of the arithmetic coding unit 207 can halve the memory area necessary for holding the ref_idx and mvd information, compared to the case of performing inter-screen prediction in units of 16 × 8 pixels and 8 × 16 pixels. For this reason, by restricting the block type of inter prediction to only 16 × 16 pixel units, 2 MB of ref_idx and mvd information are conventionally stored in a memory area that holds ref_idx and mvd information in adjacent MBs of 1 MB. It becomes possible to hold.

  Furthermore, the restriction of L1 prediction and the restriction of block type can be performed simultaneously. In this case, the prediction information encoding unit 220 can reduce the memory area necessary for holding the motion vector information to a quarter compared with the case where the intra prediction is not limited. For this reason, it is possible to hold motion vector information for 4 MB in a memory area that conventionally holds motion vector information for adjacent MBs for 1 MB by simultaneously performing L1 prediction restriction and block type restriction. Is possible.

  Further, the context index calculation unit 251 of the arithmetic encoding unit 207 can reduce the memory area necessary for holding the ref_idx and mvd information to a quarter compared with the case where the intra prediction is not limited. For this reason, by performing restriction of L1 prediction and restriction of block type at the same time, ref_idx and mvd information for 4 MB are conventionally stored in a memory area that holds ref_idx and mvd information in an adjacent MB for 1 MB. It becomes possible to hold.

  As described above, by utilizing the fact that the memory capacity required by the prediction information encoding unit 220 can be reduced due to the limitation of the inter-screen prediction method, the encoding control unit 200 can predict inter-screen prediction when the input image size is large. Notifying the unit 201 to limit the inter-screen prediction method, and notifying the prediction information encoding unit 220 to change the number of adjacent macroblock information that can be accumulated per predetermined area of the memory . Accordingly, it is possible to suppress an increase in the capacity of the adjacent MB information memory in the prediction information encoding unit 520 when the input image size increases.

  Further, by utilizing the fact that the memory capacity required by the context index calculation unit 251 can be reduced due to the limitation of the inter-screen prediction method, the encoding control unit 200 uses the inter-screen prediction unit 201 when the input image size is large. In addition, a notification is made to limit the inter-screen prediction method, and the context index calculation unit 251 is notified to change the number of adjacent macroblock information that can be accumulated per predetermined area of the memory.

  Specifically, the syntax information of the adjacent MB, which is increased because the image size is increased, is stored in a free memory area due to the restriction of the inter-screen prediction method. This makes it possible to suppress an increase in the capacity of the adjacent MB information memory (for example, the adjacent MB information memory 160 in FIG. 16) inside the context index calculation unit 251 when the input image size increases.

  The image coding apparatus 100 according to the first embodiment limits a coding method in accordance with the image size of an input image, and is a memory that is referred to when performing arithmetic coding and prediction information coding in conjunction with the coding method. The number of adjacent macroblock information that can be stored per predetermined area in (storage unit 160a (adjacent MB information memory 160)) is changed. Thereby, even when the input image size becomes large, an increase in memory capacity necessary for arithmetic coding and prediction information coding can be suppressed.

(Embodiment 2)
FIG. 10 is a diagram illustrating a hardware configuration of the image encoding device 300 according to the first embodiment. The image encoding apparatus 300 includes an encoding control unit 400, an inter-screen prediction unit 501, an intra-screen prediction unit 402, a mode selection unit 503, an orthogonal transform unit 504, a quantization unit 505, a binarization unit 506, and an inverse quantization unit. 510, an inverse orthogonal transform unit 511, a reconstructed image generation unit 512, a deblock processing unit 413, a prediction information coding unit 420, and an arithmetic coding unit 407.

FIG. 11 is a block diagram illustrating a functional configuration of the arithmetic encoding unit 407.
The arithmetic coding unit 407 includes a context index calculation unit 451, a symbol occurrence probability information holding unit 552, and a binary arithmetic coding unit 553 as shown in FIG.

  Note that the inter-screen prediction unit 501, mode selection unit 503, orthogonal transform unit 504, quantization unit 505, binarization unit 506, inverse quantization unit 510, inverse orthogonal transform unit 511, reconstructed image generation unit 512, symbol generation The probability information holding unit 552, the binary arithmetic encoding unit 553, and the reference image storage unit 600 are respectively the image encoding device 500 and the arithmetic encoding shown in FIG. In the part 507, it is the same as that of the component which has the same code | symbol and a name, The detailed description is abbreviate | omitted suitably.

Hereinafter, an image encoding method in the image encoding apparatus 300 will be described.
The encoding control unit 400 switches execution of MBAFF (Macro Block Adaptive Frame / Field) encoding based on information of an input image size given from the outside. In addition to performing this switching, the encoding control unit 400 can store in the in-screen prediction unit 402, the deblock processing unit 413, the prediction information encoding unit 420, and the arithmetic encoding unit 407 per predetermined area of the memory. Control information for changing the number of adjacent macroblock information is output.

  FIG. 12 is a diagram illustrating an MBAFF (Macro Block Adaptive Frame / Field) encoding method.

  First, MBAFF encoding will be described. As shown in FIG. 12, MBAFF encoding is an encoding method that switches field or frame encoding in units of MB pairs composed of two adjacent upper and lower MBs. When MBAFF is used, adjacent MBs used in context index calculation in arithmetic coding differ depending on whether the MB is field-coded or frame-coded.

FIG. 13 is a diagram illustrating an example when the upper adjacent MB pair is field-encoded.
In this case, depending on whether the encoding target MB pair is field-encoded or frame-encoded, the arithmetic encoding unit 407 differs in which field information of the upper adjacent MB pair is used as the upper adjacent MB. Therefore, the context index calculation unit 451 needs to hold any MB information of the MB pair, and needs to secure twice as many memory areas as compared with the case of performing normal encoding.

  Therefore, the encoding control unit 400 restricts MBAFF encoding when the input image size is large, and the number of adjacent MB information that can be accumulated per predetermined area of the memory with respect to the context index calculation unit 451. Notify me to change Specifically, the syntax information of the adjacent MB that is increased due to the increase in the image size is stored in a memory area that is free due to the restriction of MBAFF. This makes it possible to suppress an increase in the capacity of the adjacent MB information memory when the input image size increases.

  Similarly, for the upper adjacent MB information used by the prediction information encoding unit 420, as shown in FIG. 13, which of the upper adjacent MB pairs depends on whether the encoding target MB pair is field-encoded or frame-encoded. The field information is used as the upper adjacent MB. Therefore, the prediction information encoding unit 420 also needs to hold motion vector information and intra prediction mode information of both MBs in the MB pair, and has twice as much memory as the case of performing normal encoding. It is necessary to secure an area.

  Therefore, when the input image size is large, the encoding control unit 400 restricts MBAFF encoding, and also notifies the prediction information encoding unit 420 of adjacent MB information that can be accumulated per predetermined area of the memory. Notify you to change the number. Specifically, the adjacent MB information increased due to the increase in the image size is stored in a memory area that is free due to the restriction of MBAFF. Thereby, it is possible to suppress an increase in the capacity of the adjacent MB information memory when the input image size becomes large.

FIG. 14 is a diagram illustrating upper adjacent pixels used for intra-screen prediction at the time of MBAFF encoding.
Similarly, for the upper adjacent pixels used by the intra prediction unit 402, as shown in FIG. 14, depending on whether the encoding target MB pair is field-encoded or frame-encoded, It differs in which pixel of the adjacent MB pair is used for prediction. Therefore, the intra-screen prediction unit 402 needs to hold the pixels of any line, and needs to secure twice as much memory area as compared with the case of performing normal encoding.

  Therefore, the encoding control unit 400 restricts MBAFF encoding when the input image size is large, and the number of adjacent MB information that can be stored per predetermined area of the memory with respect to the intra prediction unit 402. Notify me to change Specifically, the pixel information of the adjacent MB that has increased due to the increase in the image size is stored in a memory area that is free due to the limitation of MBAFF. Thereby, it is possible to suppress an increase in the capacity of the adjacent MB information memory when the input image size becomes large.

FIG. 15 is a diagram illustrating upper adjacent pixels used in the deblocking process at the time of MBAFF encoding.
Further, as shown in FIG. 15, the upper adjacent pixels used by the deblocking processing unit 413 are, when the encoding target MB pair is field-encoded, the number of upper adjacent pixels twice as many as that of non-MBAFF encoding. Pixels are used for processing. Therefore, the deblocking processing unit 413 needs to secure twice as much memory area as compared with the case of performing normal encoding.

  Therefore, the encoding control unit 400 restricts MBAFF encoding when the input image size is large, and also allows the deblocking processing unit 413 to store the number of adjacent MB information that can be accumulated per predetermined area of the memory. Notify me to change Specifically, the pixel information of the adjacent MB that has increased due to the increase in the image size is stored in a memory area that is free due to the limitation of MBAFF. This makes it possible to suppress an increase in the capacity of the adjacent MB information memory when the input image size increases.

  The image encoding device 300 according to the second embodiment limits MBAFF encoding according to the image size of the input image. In addition to performing this restriction, the image coding apparatus 300 is linked to the memory (arithmetic coding unit 407) that is referred to during arithmetic coding, prediction information coding, intra-frame prediction coding, and deblocking processing. Adjacent MB information memory (adjacent MB information memory 160), adjacent MB information memory of the prediction information encoding unit 420, adjacent MB information memory of the intra prediction unit 402, and adjacent MB information memory of the deblock processing unit 413) The number of adjacent macroblock information that can be accumulated per area is changed. Thereby, even when the input image size becomes large, an increase in memory capacity necessary for arithmetic coding, prediction information coding, intra prediction coding, and deblocking processing can be suppressed.

  Specifically, in the image encoding device according to the embodiment, for example, the following processing may be performed.

  FIG. 16 is a diagram illustrating control of the inter-screen prediction unit and the context index calculation unit by the encoding control unit.

  The encoding processing unit 161a includes an inter-screen prediction unit 201 (first processing unit) and an adjacent MB information processing unit 161 (second processing unit). The storage unit 160a includes an adjacent MB information memory 160.

  The encoding control unit 200 (S1 in FIG. 17) performs inter-screen prediction when the image size of the image input to the image encoding apparatus (see the horizontal width 200a in FIG. 7) is smaller than a predetermined size. A predetermined notification (previously described) is not performed to the unit 201 and the context index calculation unit 251, but a notification is performed when it is large.

  The inter-screen prediction unit 201 (S2 (Sx)) performs L1 prediction (predetermined method) when performing the inter-screen prediction (first process) only when the above-described notification is not received from the encoding control unit 200. When used and notified, L1 prediction is not used.

  Also, the adjacent MB information memory 160 (accumulation unit 160a, S3) accumulates the first number of adjacent macroblocks when the above-mentioned notification is not sent from the encoding control unit 200 to the context index calculation unit 251. When the number of adjacent MBs is increased, the second number of adjacent MB information larger than the first number is stored.

  Here, the adjacent MB information memory 160 stores the first adjacent MB information (ref_idx_l1 and mvd_l1) when the first number of adjacent MB information is accumulated and the above notification is used in which L1 prediction is used. Including).

  On the other hand, in the case where the above-mentioned notification is made, in which the second number of adjacent MB information is accumulated and L1 prediction is not used, the adjacent MB information memory 160 includes a predetermined number included in the first adjacent MB information. Second neighboring MB information that does not include the received information (ref_idx_l1 and mvd_l1) is stored.

  At this time, the number of adjacent MB information that can be stored per predetermined area in the adjacent MB information memory 160 is larger than that in the case of storing the first adjacent MB information. This is because the second adjacent MB information does not include the predetermined information and can be stored with a smaller capacity than the other first adjacent MB information.

  Here, the predetermined information is used in the context index calculation (second process) only when L1 prediction is used, and is not used in the calculation when L1 prediction is not used. (Ref_idx_l1 etc.).

  If the L1 prediction is used in the inter-screen prediction unit 201 when the context index calculation unit 251 calculates the context index (second processing), the adjacent MB information processing of the context index calculation unit 251 is performed. The unit 161 (S4 (Sx)) uses the above-described predetermined information (ref_idx_l1, etc.) and does not use the information unless L1 prediction is used.

As a result, AVC / H. H.264 encoding processing is performed.
However, as described above, when the image size of the input image (for example, the number of horizontal widths 200a in FIG. 7 + 1) is large, the number of adjacent MB information accumulated per predetermined area is large.

  That is, when the image size is large, the above notification is given, and the processing method is limited by using only the L0 prediction without using the L1 prediction. When this restriction is applied, predetermined information (such as ref_idx_l1) is not used in the calculation of the context index. Therefore, when notified, the second adjacent MB information that does not include this information and can be stored with a relatively small capacity is stored in the storage unit, and as described above, a relatively large amount per predetermined area. Are accumulated. As a result, even if the capacity of the storage unit is small, a large number of adjacent MB information corresponding to a large image size is stored, an increase in the capacity of the storage unit is avoided, and the capacity of the storage unit can be reduced.

  Here, when L1 prediction (predetermined method processing) is used, appropriate processing is performed only when the context index is calculated based on the first adjacent MB information larger than the second adjacent MB information. If the calculation is performed based on the second adjacent MB information smaller than the first adjacent MB information, an inappropriate process is performed. On the other hand, when L1 prediction is not used, appropriate processing is performed even if the context index is calculated based on the second adjacent MB information smaller than the first adjacent MB information.

  The L1 prediction (predetermined method processing) and the second adjacent MB information are processing and information having this relationship with each other. The second adjacent MB information is specifically information that does not include ref_idx_l1 and the like (predetermined information) included in the first adjacent MB information as described above. May be.

  As described in the second embodiment, for example, only when a predetermined notification is not given from the encoding control unit (encoding control unit 400 in FIG. 10), the MBAFF encoding is performed and the notification is received. In this case, MBAFF encoding may not be performed and MBAFF encoding may be limited.

  As described above, the notification destination of the notification by the encoding control unit 400 includes the arithmetic encoding unit 407, the prediction information encoding unit 420, the intra-screen prediction unit 402, and the deblock processing unit 413. The capacity of each adjacent MB information memory (such as the adjacent MB information memory 160 of the arithmetic encoding unit 407) having the plurality of configurations may be reduced.

  That is, the storage unit 160a of the image encoding device 300 may further include not only the adjacent MB information memory 160 of the context index calculation unit 451 but also an adjacent MB information memory having another configuration such as the prediction information encoding unit 420.

  And the encoding process part 161a may further contain not only the adjacent MB information processing part 161 of the context index calculation part 451 but the adjacent MB information processing part of another structure.

  Storage unit 160a includes only one of the plurality of adjacent MB information memories described above, and encoding processing unit 161a includes only one adjacent MB information processing unit corresponding to the adjacent MB information memory. But you can.

  The encoding processing unit 161a may further include other configurations in addition to the inter-screen prediction unit 201 and one or more adjacent MB information processing units (adjacent MB information processing unit 161 and the like). For example, the context index calculation unit 251 may be included in the encoding processing unit 161a or may be a part of the encoding processing unit 161a. Further, for example, the encoding processing unit 161a includes the encoding control unit 200 and the adjacent MB information memories (adjacent MB information memory 160a and the like) among the functional blocks (FIG. 8 and the like) of the image encoding device 100. Others (all) other than.

  As described above, when a predetermined notification is received from the encoding control unit, only inter-screen prediction is performed in units of 16 × 16 pixels, and units smaller than 16 × 16 pixels (16 × 8 pixel units, 8 × 16 pixels). Inter-frame coding in units) may not be performed and may be limited (see Embodiment 1). Specifically, as described above, the first adjacent macroblock information includes a plurality of pieces of information each corresponding to one of a plurality of small regions (regions in units of 16 × 8 pixels). The two adjacent macroblock information includes only one piece of information corresponding to one large region (region of 16 × 16 pixel unit or the like), and may be information that can be stored with a relatively small capacity.

  Note that an image encoding device (image encoding device 100, 300) according to the present invention includes a CPU (Central Processing Unit), a system LSI (Large Scale Integration), a RAM (Random Access Memory), a ROM (Read Only Memory), An HDD (Hard Disk Drive), a network interface, or the like may be provided. Furthermore, a drive device capable of reading and writing with respect to a portable recording medium such as a DVD-RAM, a Blu-ray disk, and an SD (Secure Digital) memory card may be provided.

  Note that the image encoding device may be an embedded system such as a digital video camera, a digital recorder, a digital television, a game machine, or a mobile phone.

  Further, a program for controlling the image encoding device (hereinafter referred to as an encoding program) is installed in the HDD or ROM, and each function of the image encoding device is executed by executing the encoding program. May be realized.

  The image encoding program may be recorded on a recording medium readable by a hardware system such as a computer system or an embedded system. Furthermore, the program may be read and executed by another hardware system via the recording medium. Thereby, each function of the image coding apparatus can be realized in another hardware system. Here, as a computer system-readable recording medium, an optical recording medium (for example, a CD-ROM), a magnetic recording medium (for example, a hard disk), a magneto-optical recording medium (for example, an MO), or a semiconductor memory. (For example, a memory card).

  The image encoding program may be held in a hardware system connected to a network such as the Internet or a local area network. Furthermore, it may be downloaded to another hardware system via a network and executed. Thereby, each function of the image coding apparatus can be realized in another hardware system. Here, as a network, a terrestrial broadcast network, a satellite broadcast network, a PLC (Power Line Communication), a mobile telephone network, a wired communication network (for example, IEEE802.3), and a wireless communication network (for example, IEEE802.11). There is.

  Alternatively, each function of the recording / reproducing apparatus may be realized by a recording / reproducing circuit mounted on the recording / reproducing apparatus.

  The image coding circuit is a semi-custom LSI such as a full custom LSI (Large Scale Integration), ASIC (Application Specific Integrated Circuit), or a FPGA (Field Programmable Gate Array) or CPLD (Complex Gate Gate Array). Such a programmable logic device or a dynamically reconfigurable device whose circuit configuration can be dynamically rewritten may be used.

  Furthermore, the design data for forming each function of the image encoding device in the image encoding circuit may be a program described in a hardware description language (hereinafter referred to as an HDL program). Further, it may be a gate level netlist obtained by logical synthesis of an HDL program. Alternatively, macro cell information in which arrangement information, process conditions, and the like are added to a gate level netlist may be used. Further, it may be mask data in which dimensions, timing, and the like are defined. Here, as hardware description languages, there are VHDL (Very high speed integrated circuit Hardware Description Language), Verilog-HDL, and SystemC.

  Furthermore, the design data may be recorded on a readable recording medium in a hardware system such as a computer system or an embedded system. Further, the program may be read and executed by another hardware system via the recording medium. Then, the design data read by the other hardware system via these recording media may be downloaded to the programmable logic device via the download cable.

  Alternatively, the design data may be held in a hardware system connected to a network such as the Internet or a local area network. Furthermore, it may be downloaded to another hardware system via a network and executed. Then, the design data acquired in another hardware system via these networks may be downloaded to the programmable logic device via a download cable.

  Alternatively, the design data may be recorded in a serial ROM so that it can be transferred to the FPGA when energized. The design data recorded in the serial ROM may be downloaded directly to the FPGA when energized.

  Alternatively, the design data may be generated by a microprocessor and downloaded to the FPGA when energized.

  Although the present invention has been described based on the above embodiment, the present invention is not limited to the above embodiment, and various improvements and modifications can be made without departing from the gist of the present invention. is there. For example, the form which combined the component of Embodiment 1 and the component of Embodiment 2 may be taken.

  The present invention relates to AVC / H. An image encoding device compliant with H.264 is provided. In particular, this is useful when an image encoding apparatus encodes images of a plurality of sizes.

DESCRIPTION OF SYMBOLS 100 Image coding apparatus 200 Coding control part 201 Inter-screen prediction part 207 Arithmetic coding part 220 Prediction information coding part 502 Intra-screen prediction part 503 Mode selection part 504 Orthogonal transformation part 505 Quantization part 506 Binarization part 510 Reverse Quantization unit 511 Inverse orthogonal transform unit 512 Reconstructed image generation unit 513 Deblock processing unit

Claims (14)

An image encoding device that encodes an input image,
Among the information included in the input image, storage is information about adjacent macroblock information related to a macroblock that has already been encoded, and stores adjacent macroblock information that is referred to when encoding a macroblock to be encoded. And
An encoding unit that encodes the encoding target macroblock based on the adjacent macroblock information;
Coding that changes the coding method in the coding unit and changes the number of adjacent macroblock information that can be accumulated per predetermined area in the storage unit according to the image size of the input image An image encoding device comprising a control unit.   The image encoding device according to claim 1, wherein the encoding control unit limits the number of pictures to be referred to in inter-screen prediction in the encoding unit according to the image size of the input image.   The image encoding device according to claim 1, wherein the encoding control unit limits the number of macroblock divisions in inter-screen prediction in the encoding unit according to the image size of the input image.   2. The encoding control unit according to claim 1, wherein the encoding control unit restricts switching between frame encoding and field encoding in units of macroblocks in the encoding unit according to the image size of the input image. Image encoding device.   The image coding apparatus according to claim 1, wherein the adjacent macroblock information is information used for arithmetic coding processing in the coding unit.   The image coding apparatus according to claim 1, wherein the adjacent macroblock information is information used to calculate motion vector information in an encoded stream.   The image coding apparatus according to claim 1, wherein the adjacent macroblock information is image information used for intra prediction processing in the coding unit.   The image coding apparatus according to claim 1, wherein the adjacent macroblock information is information used for a deblocking process in the coding unit.   The image coding apparatus according to claim 1, wherein the adjacent macroblock information is information used for calculation of intra prediction mode information in an encoded stream.   The image encoding apparatus applies an AVC / H. The image encoding apparatus according to claim 1, wherein encoding according to H.264 is performed. The encoding unit includes a first processing unit and a second processing unit,
The encoding control unit does not perform a predetermined notification when the image size of the input image is smaller than a predetermined size, and performs the notification when it is larger,
The first processing unit performs inter-screen prediction, and when performing the inter-screen prediction, the L1 prediction is used only when the notification is not made from the encoding control unit, and when the notification is made , Without using the L1 prediction,
The storage unit
When the notification is not made from the encoding control unit, the first number of adjacent macroblock information is accumulated, and when the notification is made, a second number larger than the first number is stored. Storing the neighboring macroblock information;
The first number of neighboring macroblock information is accumulated and the L1 prediction is used, and in the case where the notification is not made, the first neighboring macroblock information is accumulated;
In the case where the relatively large number of the second number of adjacent macroblock information is accumulated and the L1 prediction is not used or the notification is made, the predetermined number included in the first adjacent macroblock information is predetermined. Storing the second adjacent macroblock information that does not include the information, storing a number greater than the number of storing the first adjacent macroblock information per predetermined area per predetermined area,
The predetermined information is information that is used in calculating a context index only when the L1 prediction is used, and is not used in the calculation when the L1 prediction is not used.
When calculating the context index, the second processing unit uses the predetermined information when the L1 prediction is used, and when the L1 prediction is not used. The image encoding apparatus according to claim 1, wherein the information is not used. An image encoding method for encoding an input image,
Among the information included in the input image, the storage unit stores adjacent macroblock information related to a macroblock that has already been encoded, and is referred to when encoding the macroblock to be encoded. An accumulation step to accumulate;
Based on the adjacent macroblock information, an encoding step in which an encoding unit encodes the encoding target macroblock;
Coding that changes the coding method in the coding unit and changes the number of adjacent macroblock information that can be accumulated per predetermined area in the storage unit according to the image size of the input image An image encoding method including a control step. An image encoding program for a computer to encode an input image,
Among the information included in the input image, adjacent macroblock information related to a macroblock that has already been encoded, and adjacent macroblock information referred to when encoding the macroblock to be encoded is stored by the storage unit. An accumulation step to accumulate;
An encoding step of encoding the encoding target macroblock based on the adjacent macroblock information;
Encoding for changing the encoding method in the encoding step and changing the number of adjacent macroblock information that can be stored per predetermined area in the storage unit according to the image size of the input image An image encoding program for causing a computer to execute a control step. An image encoding integrated circuit for encoding an input image,
Among the information included in the input image, storage is information about adjacent macroblock information related to a macroblock that has already been encoded, and stores adjacent macroblock information that is referred to when encoding a macroblock to be encoded. And
An encoding unit that encodes the encoding target macroblock based on the adjacent macroblock information;
Coding that changes the coding method in the coding unit and changes the number of adjacent macroblock information that can be accumulated per predetermined area in the storage unit according to the image size of the input image An image coding integrated circuit comprising a control unit.

Images