JP6360591B2 - Image encoding apparatus and method - Google Patents

Description

Embodiments described herein relate generally to an image encoding apparatus used for encoding an image and a method thereof.

In order to perform coding with a desired code amount in image coding using transform / quantization, there has been a rate control method for adaptively switching the quantization scale from a past code amount. The conventional rate control method is intended to converge to a desired code amount. For this reason, it is not always guaranteed to converge within the target code amount.

In 1-pass encoding, it is conceivable that the target code amount is exceeded as a result of encoding. When the size of the bit stream is defined in advance when the encoder is configured by hardware, there is a problem that the bit stream breaks if the target code amount is exceeded.

An encoding scheme that avoids stream buffer corruption has been disclosed. The minimum and maximum code amounts are determined in units of macro blocks, and a method is described in which encoding is performed with the minimum code amount if the stream buffer breaks if the maximum code amount occurs in a block to be encoded. .

In the above method, the encoding condition is switched depending on whether or not the stream buffer is broken even if the maximum code amount occurs in the block. In general, it is desirable not to select the encoding with the minimum code amount because the image quality greatly deteriorates. On the other hand, since the above method is equivalent to having a margin for the maximum code amount, there is a problem that encoding with the minimum code amount is easily selected.

JP 2008-271213 A

“MPEG-2 Test Model 5 (TM5)”, Doc. ISO / IEC-JTC1 / SC29 / WG11 / N0400, Apr. 1993. Chapter 10, “Rate Control and Quantization”

The problem to be solved by the present invention is to prevent degradation of the image quality while preventing the stream buffer from being broken even in one-pass encoding.

The present invention has been devised to solve the above problems, and includes a first encoding unit capable of encoding each of a plurality of pixel blocks included in a first segment included in an input image , and the plurality of pixels. A second encoding unit capable of encoding each block with a predetermined first code amount; and the first encoding for an encoding target pixel block that is one of the plurality of pixel blocks. And a determination unit that determines which one of the second encoding units to use, and the determination unit starts from the first pixel block encoded first among the plurality of pixel blocks. The sum of the cumulative code amount up to the second pixel block encoded immediately before the pixel block and the code amount of the encoding target pixel block encoded by the first encoding unit is the first code block. Segment target code When the difference between the amount and the code amount generated by encoding the pixel block encoded after the encoding target pixel block by the second encoding unit among the plurality of pixel blocks is smaller than If it is determined that the first encoding unit is used and it is determined that the second encoding unit is used, pixels to be encoded after the encoding target pixel block among the plurality of pixel blocks At least a part of the encoding target pixel block used for prediction in the block encodes a difference between a pixel value of the local decoded image generated by the first encoding unit and a pixel value of the encoding target pixel block. An image encoding device is provided.
In addition, a first encoding unit capable of encoding each of the plurality of pixel blocks included in the first segment included in the input image, and each of the plurality of pixel blocks can be encoded with a predetermined first code amount. Determining whether to use the first encoding unit or the second encoding unit for a second encoding unit and an encoding target pixel block that is one of the plurality of pixel blocks A determination unit that performs the determination from the first pixel block encoded first among the plurality of pixel blocks to the third pixel block encoded before the pixel block to be encoded. And the maximum code amount when the first encoding unit encodes the third pixel block up to the encoding target pixel block is the target code of the first segment. Quantity and said multiple pictures When the pixel block encoded after the encoding target pixel block among the blocks is smaller than the difference from the code amount generated by encoding by the second encoding unit, the first code If it is determined that the encoding unit is to be used and it is determined that the second encoding unit is to be used, among the plurality of pixel blocks, the pixel block encoded after the encoding target pixel block is used for prediction. Image coding in which at least a part of the encoding target pixel block is additionally encoded with a difference between a pixel value of a local decoded image generated by the first encoding unit and a pixel value of the encoding target pixel block Providing equipment.
Similarly, a first encoding unit capable of encoding each of the plurality of pixel blocks included in the first segment of the input image, and encoding each of the plurality of pixel blocks with a predetermined first code amount. A possible second encoding unit, a third encoding unit capable of encoding each of the plurality of pixel blocks in advance with a second code amount larger than the first code amount, and the plurality of pixel blocks A determination unit that determines which one of the first encoding unit, the second encoding unit, and the third encoding unit is used for an encoding target pixel block that is one of The determination unit includes a third unit when a code amount of the encoding target pixel block by the first encoding unit exceeds a code amount of the encoding target pixel block by the third encoding unit. It is determined that the encoding unit is used, and the previous When it is determined that the third encoding unit is to be used, at least one of the encoding target pixel blocks used for prediction in a pixel block encoded after the encoding target pixel block among the plurality of pixel blocks. The unit provides an image encoding device that additionally encodes a difference between a pixel value of a local decoded image generated by the first encoding unit and a pixel value of the encoding target pixel block.

Even in the 1-pass encoding, it is possible to suppress degradation of the image quality while preventing the stream buffer from being broken.

The figure which shows an encoding / decoding process unit. The figure which shows an image coding apparatus. The figure which shows the 1st encoding part 103. FIG. The figure which shows a local decoding production | generation part. The timing chart which shows the operation example of an image coding apparatus. The flowchart which shows the operation example of an image coding apparatus. The figure which shows the example of the syntax of the coding data in a unit. The figure which shows the relationship of transition of accumulation code amount, and encoding mode determination. The figure which shows another example of an image coding apparatus. The timing chart which shows the operation example of another example of an image coding apparatus. The flowchart which shows the operation example of another example of an image coding apparatus. The figure which shows the example of the syntax of the coding data in a unit. The figure which shows the example of the syntax of the coding data in a unit. The figure which shows another example of an image coding apparatus. The figure which shows the example of the syntax of the coding data in a unit. The figure which shows the example of the syntax of the coding data in a unit. The flowchart which shows the operation example of a mode determination part. The figure which shows another example of an image coding apparatus. The flowchart which shows the operation example of another example of an image coding apparatus.

Hereinafter, the image encoding device of the present embodiment will be described in detail with reference to the drawings. Note that in the following embodiments, the same reference numerals are assigned to the same operations, and duplicate descriptions are omitted as appropriate.

(First embodiment)
In the following embodiment, as shown in FIG. 1, a unit for performing an encoding process on a processing target image (input image) is referred to as a unit, and a unit whose code amount should be guaranteed is referred to as a segment. In the example of FIG. 1, the unit is a pixel block, and the segment is one unit line of the image.
The encoding apparatus according to the present embodiment does not limit the code amount generated in each unit, but the code amount of the input image is set so that the total code amount generated in the segment is equal to or less than a predetermined desired code amount. Do.

  FIG. 2 is a diagram illustrating the image encoding device 100 according to the present embodiment.

The image encoding device 100 includes a switch 101, a switch 102, a first encoding unit 103, a second encoding unit 104, an entropy encoding unit 105, an encoding mode determination unit 106, a local decode generation unit 107, and an encoding control unit. 108.

The switch 101 receives the input image data of the unit, and receives an encoding mode determination unit 106.
The encoding mode information is received. The switch 101 sends an input image to the first encoding unit 103 when the encoding mode information indicates the first encoding mode. The switch 101 sends an input image to the second encoding unit 104 when the encoding mode information indicates the second encoding mode.

The first encoding unit 103 performs encoding processing by transform / quantization on the received encoding target unit, and generates first encoded data. Details of the encoding process will be described later.

The second encoding unit 104 performs an encoding process on the received encoding target unit so as to obtain a predetermined code amount, and generates second encoded data.

The switch 102 receives encoding mode information from the encoding mode determination unit 106. When the encoding mode information indicates the first encoding mode, the first encoded data is received from the first encoding unit 103. When the encoding mode information indicates the second encoding mode, the second encoded data is received from the second encoding unit 104. Thereafter, the received encoded data is sent to the entropy encoding unit 105 and the local decode generation unit 107.

The entropy encoding unit 105 performs entropy encoding processing on the received encoded data to generate a bitstream. Code amount information indicating the code amount generated at that time is sent to the encoding mode determination unit 106.

The encoding mode determination unit 106 receives code amount information from the entropy encoding unit 105. The encoding mode determination unit 106 sets the encoding mode information for the subsequent unit based on the code amount information and sends it to the switch 101 and the switch 102.

The local decode generation unit 107 receives encoded data from the switch 102 and performs a decoding process corresponding to the encoding process to generate a local decoded image. When the first encoded data is received, a decoding process corresponding to the encoding process performed by the first encoding unit 103 is performed. When the second encoded data is performed, a decoding process corresponding to the decoding process performed by the second decoding unit 104 is performed. The generated local decoded image is sent to the first encoding unit 103 and the second encoding unit 104. The local decoded image is used to generate a predicted image when the subsequent unit is encoded.

The encoding control unit 108 controls the first encoding unit 103 and the second encoding unit 104 by performing feedback control and quantization control of the generated code amount, control of the prediction direction, and the like.

  The first encoding unit 103 will be described in detail with reference to FIG.

FIG. 3 is a diagram showing details of the first encoding unit 103. The first encoding unit 103 includes a predicted image generation unit 109, a subtraction unit 110, and a transform / quantization unit 111.

The predicted image generation unit 109 receives the local decoded image from the local decode generation unit 107, performs a predetermined prediction process, and generates a predicted image corresponding to the processing target unit. The predicted image generation unit 109 sends the predicted image to the subtracting unit 110. Although any method may be used as the prediction process, an example in which spatial prediction from adjacent units is used will be described in the present embodiment. For example, H.M. Similar to existing codecs such as H.264 / AVC, spatial prediction is performed using pixels of adjacent units. The prediction direction of the spatial prediction at that time is the encoding control unit 10.
8 is set.

The subtracting unit 110 receives the processing target unit of the input image from the switch 101 and receives the corresponding predicted image from the predicted image generating unit 109. The subtraction unit 110 generates a prediction error image by subtracting the pixel value of the prediction image from the pixel value of the input image. The generated prediction error image is sent to the transform / quantization unit 111.

The transformation / quantization unit 111 performs transformation processing on the prediction error image received from the subtraction unit 110. As the transformation process, for example, orthogonal transformation using DCT (discrete cosine transformation) or the like is performed to generate a transformation coefficient. Note that the transform coefficient may be generated using a method such as wavelet transform or independent component analysis. Next, the transform / quantization unit 111 performs a quantization process on the transform coefficient based on the quantization parameter set by the encoding control unit 108 to generate encoded data.

As described above, the first encoding unit 103 performs an encoding process by a general encoding method that performs transformation and quantization on a prediction error image.

  The local decode generation unit 107 will be described in detail with reference to FIG.

FIG. 4 is a diagram showing details of the local decode generation unit 107. The local decode generation unit 107 includes a prediction error image generation unit 109, an inverse quantization / inverse conversion unit 112, and an addition unit 113.

The inverse quantization / inverse transform unit 112 receives the encoded data from the switch 102. The inverse quantization / inverse transform unit 112 performs inverse quantization on the orthogonal transform coefficient after quantization included in the encoded data according to the quantization parameter set by the encoding control unit 108. The inverse quantization / inverse transform unit 112 performs an inverse transform corresponding to the orthogonal transform performed by the transform / quantization unit 111 on the transform coefficient obtained by the inverse quantization, and generates a prediction error image. The obtained prediction error image is sent to the adding unit 113.

The addition unit 113 receives the prediction image from the prediction image generation unit 109, adds the prediction image received from the inverse quantization / inverse conversion unit 112, and generates a local decoded image. The obtained local decoded image is output to the first encoding unit 103 and the second encoding unit 104 and also output to the predicted image generation unit 109 in the local decode generation unit 107, and a prediction process is performed in the subsequent unit. Used when.

The predicted image generation unit 109 performs the same processing as the predicted image generation unit 109 of the first encoding unit 103 illustrated in FIG. Thereby, the same prediction image can be used in the first encoding unit 103 and the local decode generation unit 107. Note that the code generation unit 107 does not have the prediction image generation unit 109 locally, and sends the prediction image generated by the first encoding unit 103 or the second encoding unit 104 to the local decoding generation unit 107 as it is. It doesn't matter.

Next, the operation of the image coding apparatus 100 according to the present embodiment will be described with reference to FIGS.

  FIG. 5 shows an operation timing chart of the image encoding device 100.

The image coding apparatus 100 according to the present embodiment receives an input image and performs coding processing in a pipeline for each unit according to the timing chart shown in FIG. The horizontal axis in FIG. 5 indicates time. Encoding processing is sequentially performed in the order of units # 0 to # 3 indicated on the vertical axis. The image encoding apparatus 100 performs three operations of determining the prediction direction and determining the encoding mode, encoding processing and local decoding, and entropy encoding for each unit in N cycles. The subsequent processing procedure will be described with reference to the flowchart of FIG.

FIG. 6 illustrates an example of a processing flow of the image encoding device 100.
First, when the image coding apparatus 100 receives pixel data included in the processing target unit in the input image, the coding control unit 108 determines the prediction direction of the first coding mode (S601).
. As described above, the first encoding unit 103 performs general encoding by spatial prediction and transform / quantization processing. The prediction direction of the spatial prediction may be determined by an arbitrary method.
In the present embodiment, an example will be described in which the prediction direction that minimizes the error between the prediction image and the input image is selected from all prediction directions.

Next, the encoding mode determination unit 106 determines whether the encoding is performed in the first encoding mode or the second encoding mode (S602). Coding mode information indicating whether encoding is performed in the first encoding mode or the second encoding mode is sent to the switch 101. Details of the processing in the encoding mode determination unit 106 will be described later.

  S601 and S602 are executed in the first N cycles of each unit.

When the coding mode information indicates the first coding mode, the switch 101 (S603,
yes), the input image is sent to the first encoding unit 103.

The predicted image generation unit 109 of the first encoding unit 103 generates a predicted image according to the prediction direction determined by the encoding control unit 108 from the pixel values of adjacent units included in the local decoded image (S604). The subtraction unit 110 generates a prediction error image by subtracting the prediction image from the input image (S605). The transform / quantization unit 111 performs transform and quantization processing on the prediction error image to generate a transform coefficient after quantization (S606). Transformer / quantizer 11
1 generates first encoded data (S607). The first encoded data includes information such as a transform coefficient after quantization, encoding mode information indicating that encoding has been performed in the first encoding mode, a prediction direction, and a quantization parameter. The first encoded data is the local decode generation unit 107.
Sent to.

The local decode generation unit 107 generates a local decoded image by performing a decoding process corresponding to the encoding process performed by the first encoding unit 103 on the first encoded data (S
608). The generated local decoded image is a first encoding unit 103 and a second encoding unit 10.
4 is sent.

On the other hand, when the encoding mode information indicates the second encoding mode (S603, no), the second
The encoding unit 104 generates a prediction image in the second encoding mode (S609). The operation of the second encoding unit 104 of this embodiment will be described in detail. The second encoding unit 104 performs encoding with a predetermined code amount. When there is a possibility that a desired code amount may be exceeded by encoding by the first encoding unit 103, switching to encoding by the second encoding unit 104 can be performed. Although any method may be used as the encoding method in the second encoding unit 104, encoding with a specific code amount is required as described above. Furthermore, it is desirable that encoding can be performed with a sufficiently small code amount as compared with the code amount that can be generated by the first encoding unit 103. In the present embodiment, for example, encoding with a fixed length is realized by encoding a prediction image as it is as a local decoded image without encoding the transform coefficient after quantization.

The second encoding unit 104 generates second encoded data (S610). The second encoded data includes encoding mode information indicating that encoding is performed in the second encoding mode. In addition,
Details of the processing in the second encoding unit 104 and the second encoded data will be described later. The local decode generation unit 107 generates a local decoded image by performing a decoding process corresponding to the encoding process performed by the second encoding unit 104 on the second encoded data (S611).
. The generated local decoded image is sent to the first encoding unit 103 and the second encoding unit 104. Note that the local decoded image generated in S608 and S611 is used as a reference image in prediction processing in the subsequent unit.

The above-mentioned S604 to S608 or S609 to S611 is the second N of each unit.
Executed in a cycle.

The encoded data is sent to the entropy encoding unit 105. Entropy encoding unit 105
Performs entropy encoding on the first encoded data or the second encoded data to generate a bitstream (S612). Output the generated bitstream (S
613). When encoding of all the units included in the segment to be processed has not been completed (S614, no), the entropy encoding unit 105 generates code amount information indicating the generated code amount, and an encoding mode determination unit The data is output to 106 (S615). At this time, although details will be described later, the encoding mode determination unit 106 determines the encoding mode in the next unit from the relationship between the received code amount information and the desired code amount set for the entire segment.
The code amount information is also fed back to the encoding control unit 108 and used when setting quantization parameters.

When encoding of all units included in the segment to be processed has been completed (S614)
yes), the segment encoding process ends.

The above S612 to S615 are executed in the last N cycles. The unit encoding process in this embodiment is composed of the above three stages. As shown in FIG. 5, processing is performed by shifting the timing one stage at a time from the units in the order of processing, and a plurality of units are processed in a pipeline. The processing included in each cycle may be processing other than the example described above. Also, the number of stages may be other than three.

FIG. 7 shows an example of syntax in the encoded data of a unit. In the syntax of FIG. 7, codec_mode indicating the encoding mode is first encoded at the beginning of the encoded data. For example, codec_mode can be expressed by a 1-bit flag.

When the encoding mode information indicates encoding by the first encoding unit 103, that is, codec_
When the mode is MODE_1, delta_qp that is the difference between the quantization parameter in the immediately preceding unit and the quantization parameter of the target unit is encoded. Subsequently, pred_mode indicating the prediction direction of spatial prediction is encoded. delta_qp and pred
Since the range of values that can be taken in advance is known, _mode may be encoded with a fixed length, for example. Encoding at a fixed length simplifies the decoding process for the bitstream, and improves encoding efficiency when compared to variable length encoding when there is no bias in the syntax value. Finally, coeff which is a transform coefficient after quantization is variable-length encoded. NUM_
COEFF indicates the number of conversion coefficients.

On the other hand, when the encoding mode information indicates encoding by the second encoding unit 104, that is, code.
When c_mode is MODE_2, pr is the same as the encoding by the first encoding unit 103.
ed_mode is encoded. However, the transform coefficient after quantization is not encoded.
Therefore, it is not necessary to encode delta_qp indicating the quantization parameter.

With such a syntax configuration, the encoding in the second encoding unit 104 is accompanied by a large image quality degradation, but the encoding can always be performed with a sufficiently small code amount as compared with the first encoding unit 103. it can. The above syntax is an example, and syntax may be added or a part of the syntax may be deleted in the same manner as a general codec. The syntax may be changed depending on the component in the image.

Next, the operation of the encoding mode determination unit 106 will be described in detail. The encoding mode determination unit 106 receives the code amount generated from the entropy encoding unit 105 and accumulates and stores the code amount generated in the segment. From the accumulated code amount held, it is determined whether or not there is a possibility of exceeding the code amount to be guaranteed in the segment when the subsequent unit is encoded in the first encoding mode. If there is a possibility of exceeding, the second encoding mode is selected. Otherwise, the first encoding mode is selected to switch the encoding mode.

FIG. 8 is a diagram showing the relationship between the transition of the code amount and the determination of the encoding mode. A specific operation example of the coding mode determination unit 106 will be described with reference to FIG. The horizontal axis in FIG. 8 indicates the unit number in the segment, and the vertical axis indicates the accumulated code amount. The solid line indicates the upper limit value of the accumulated code amount that the stream buffer does not break when it is assumed that all the remaining units in the segment are encoded in the second encoding mode. A thin broken line indicates the target code amount. In general, when encoding with CBR (constant bit rate), the target is to generate an equal amount of code in all units, and by controlling the quantization parameter, encoding with the target amount of code is performed. Plan However, the target code amount cannot always be achieved in advance by 1-pass encoding.

For this reason, the code amount varies from unit to unit as shown by the thick broken line or the two-dot chain line in FIG. The two-dot chain line is a diagram illustrating an example of the accumulated code amount when all the remaining units in the segment can be encoded in the first encoding mode. If the actually generated accumulated code amount is below the solid line, the code amount can be guaranteed. A thick broken line is a diagram illustrating an example of the accumulated code amount when the second encoding mode is required. When the accumulated code amount actually generated exceeds the solid line, the code amount cannot be guaranteed even if the second encoding mode is switched.
In the example indicated by the two-dot chain line, the code amount greatly exceeding the target code amount is generated by the encoding in the first encoding mode. Since the remaining code amount limit is reached in the remaining units, the subsequent units are all encoded in the second encoding mode to guarantee the upper limit of the code amount.

In order to perform the above operation, the encoding mode determination unit 106 determines the encoding mode according to the equations (1) and (2).

First, equation (1) will be described. Btarget is the target code amount in the segment. Available is the available code amount in the remaining units excluding the unit that performs encoding mode determination (hereinafter referred to as the target unit) among the units to be encoded in the segment. Btotal indicates the accumulated code amount fed back when determining the encoding mode of the target unit. Bavailable can be calculated by subtracting Btotal from Btarget.

As shown in FIG. 5, since the encoding apparatus 100 according to the present embodiment performs pipeline processing, a delay occurs in the feedback of the code amount. That is, in the example of FIG. 5, at the time of determining the encoding mode of Unit # 3, only the code amount of Unit # 0 is fed back.
The generated code amounts of it # 1 and Unit # 2 are unknown. Therefore, in equation (1), the maximum number of bits Bmax that can theoretically be generated in the first encoding mode is defined, and the maximum number of bits is generated for a unit whose generated code amount is unknown, including the target unit. The amount of code that can be used by the remaining units is calculated. In the example shown in FIG. 5, the number of delay units Udelay in the feedback of the code amount is 2. The number of delay units is changed as appropriate according to the pipeline configuration and processing contents.

Next, equation (2) will be described. Bmin indicates the amount of code generated in the second encoding mode. Uleft indicates the number of remaining units excluding the target unit and the delay unit. That is, even if the maximum code amount is generated in the code amount feedback delay unit and the target unit, encoding is performed in the first encoding mode if the code amount falls below the available code amount limit. Otherwise, the code amount is guaranteed from the target unit by performing encoding in the second encoding mode. The above is the processing related to the image encoding device 100.

In the present embodiment, in addition to the first encoding mode in which normal conversion / quantization is performed, the code amount can be uniquely determined, and encoding is performed with a smaller code amount than in the first encoding mode. A second encoding mode for performing the above is prepared, and it is ensured that the desired code amount is not exceeded by switching between the two encoding modes. For example, each encoding processing unit (unit) is normally encoded in the first encoding mode, and in the unit (segment) that guarantees the code amount, all the remaining units are all in the second encoding mode. By switching to encoding in the second encoding mode when it is found that the encoding amount cannot be guaranteed unless encoding is performed, the encoding amount can be reliably ensured. Also, by determining the encoding mode as described above, encoding in the normal first encoding mode is performed as much as possible, and encoding in the second encoding mode is performed only when the code amount is insufficient. By doing so, it is possible to guarantee the upper limit of the accumulated code amount in the segment while maintaining the image quality by the normal encoding until immediately before the code amount is insufficient.

(Second Embodiment)
In the image coding apparatus 100 according to the first embodiment, since a delay has occurred in the feedback of the code amount, the maximum code amount that can be generated in the first encoding mode is added for a plurality of units as shown in Expression (1). There was a need. This is equivalent to controlling with a margin, and depending on the compression ratio and segment size, the margin becomes a large overhead,
There is a possibility that sufficient image quality may not be obtained by far below the desired code amount. Therefore, the present embodiment is different in that processing is performed so as not to cause a feedback delay of the code amount.

FIG. 9 is a diagram illustrating an image encoding device 200 according to the present embodiment. The image encoding device 200 includes a first encoding unit 103, a second encoding unit 104, an entropy encoding unit 105A, an entropy encoding unit 105B, a local decode generation unit 107A, and a local decode generation unit 10.
7B, encoding control unit 108, switch 114, switch 115, and encoding mode determination unit 1
16

The first encoding unit 103, the second encoding unit 104, and the encoding control unit 108 perform the same operations as those of the image encoding device 100, and thus the description thereof is omitted. Further, the entropy encoding unit 105A
And 105B perform the same operation as the entropy encoding unit 105, and the local decode generation unit 107A and the local decode generation unit 107B perform the same operation as the local decode generation unit 107, respectively, and thus description thereof will be omitted.

The switch 114 includes a local decode generator 107A and a local decode generator 107.
B receives local decoded images of the first encoding mode and the second encoding mode from B, selects any encoded data according to the encoding mode determined by the encoding mode determination unit 116, and selects the first encoding unit 103. And sent to the second encoding unit 104. The sent local decoded image is used for prediction as a reference image when a subsequent unit is encoded.

The switch 115 receives the encoding mode information from the encoding mode determination unit 116. When the coding mode information indicates the first coding mode, the switch 115 receives the bit stream of the first coding mode from the entropy coding unit 105A. When the encoding mode information indicates the second encoding mode, the bit stream of the second encoding mode is received from the entropy encoding unit 105B. Thereafter, the received bit stream is output.

The encoding mode determination unit 116 receives the code amount generated when encoding is performed in the first encoding mode from the entropy encoding unit 105A, sets the encoding mode information for the target unit, and outputs it to the switch 114 and the switch 115 To do. The encoding mode determination unit 116 is different from the encoding mode determination unit 106 in that the encoding mode for the current encoding target unit is set instead of the encoding mode for the subsequent unit.

Next, the encoding process of the image encoding device 200 according to the present embodiment will be described with reference to FIGS.
1 will be described in detail.

  FIG. 10 is an operation timing chart of the image encoding device 200.

The image encoding device 200 receives an input image and performs an encoding process in a pipeline for each unit. The encoding process follows the timing chart shown in FIG.
Unlike the above, encoding in both the first encoding mode and the second encoding mode is executed to generate a bitstream, and finally the encoding mode is determined.

  FIG. 11 is a diagram illustrating an example of a flowchart of processing.

When the image encoding device 200 receives a pixel corresponding to the processing target unit in the input image,
The encoding control unit 108 determines the prediction direction of the first encoding mode (S1101).

The predicted image generation unit 109 of the first encoding unit 103 generates a predicted image from the pixel values of adjacent units included in the local decoded image according to the prediction direction determined in S1101. The subtraction unit 110 generates a prediction error image by subtracting the prediction image from the input image (S11).
03).

The transform / quantization unit 111 performs transform and quantization processing on the prediction error image to generate a transform coefficient after quantization (S1104).

The transform / quantization unit 111 generates first encoded data (S1105). The first encoded data includes a transform coefficient after quantization, encoding mode information indicating that encoding has been performed in the first encoding mode, a prediction direction, and a quantization parameter.

The local decode generation unit 107A generates a local decoded image by performing a decoding process corresponding to the encoding process performed on the first encoded data (S1106). The generated local decoded image is sent to the switch 114. Entropy encoding unit 105
A performs entropy encoding on the first encoded data to generate a bitstream (S1107).

In S1101 to 1107, encoding in the first encoding mode is performed, but the input image is also input to the second encoding unit 104, and encoding is also performed in the second encoding mode.
FIG. 11 describes an example in which the second encoding unit 104 does not perform transformation / quantization as in the first embodiment. However, the encoding method of the second encoding unit 104 is not limited to this.

The second encoding unit 104 generates a predicted image and generates second encoded data (S1108).
). The second encoded data is sent to the local decode generation unit 107B.

The local decode generation unit 107B generates a local decoded image by performing a decoding process corresponding to the encoding process performed on the second encoded data (S1110). In this example, since the predicted image becomes a local decoded image as it is, the predicted image may be received from the second encoding unit 104 and used as it is as a local decoded image, or the second encoding unit 10
4 may be performed. The generated local decoded image is displayed on the switch 114.
Sent to.

The second encoded data is sent to the entropy encoding unit 105B. The entropy encoding unit 105B generates a bit stream by entropy encoding the second encoded data (S1111).

The encoding mode determination unit 116 receives the generated code amount from the entropy encoding unit 105A, and determines the encoding mode for the target unit (S1112). The encoding mode information is sent to the switch 114 and the switch 115. Details of the processing of the encoding mode determination unit 116 will be described later.

When the encoding mode information indicates the first encoding mode (S1113, yes), the switch 114 converts the local decoding image in the first encoding mode received from the local decoding generation unit 107A into the first encoding unit 103. And sent to the second encoding unit 104 (S1114). The switch 115 outputs the bit stream of the first encoding mode received from the entropy encoding unit 105A (S1115).

When the encoding mode information indicates the second encoding mode (S1113, no), the switch 114 converts the local decoding image in the second encoding mode received from the local decoding generation unit 107B into the first encoding unit 103. And sent to the second encoding unit 104 (S1116). The switch 115 outputs the bit stream of the second encoding mode received from the entropy encoding unit 105B (S1117).

When encoding of all units included in the segment to be processed has not been completed (S1
118, no), the entropy encoding unit 105A and the entropy encoding unit 105B,
The generated code amount is sent to the encoding mode determination unit 116 (S1119). However, since the second encoding mode performs encoding with a fixed code amount, it is not always necessary to send the generated code amount from the entropy encoding unit 105B.

When encoding of all units included in the segment to be processed is completed (S1118)
, Yes), the segment encoding process ends.

The operation of the encoding mode determination unit 116 will be described in detail. Coding mode determination unit 116
Receives the code amount generated in the first encoding mode from the entropy encoding unit 105A,
The encoding mode in the target unit is determined. The image coding apparatus 200 differs from the image coding apparatus 100 in that the coding mode is determined after all processing stages for each unit are completed as shown in FIG. Therefore, even when pipeline processing is performed, there is no delay in code amount feedback.

The encoding mode determination unit 116 calculates the available code amount B in the remaining unit according to Equation (3).
The availability is calculated.

Here, Bcurrent indicates a code amount generated when encoding is performed in the first encoding mode in the target unit. The encoding mode determination unit 116 includes an entropy encoding unit 105A.
Bcurrent is received. Btotal indicates the accumulated code amount fed back at the time of encoding the target unit as in the equation (1). As shown in FIG. 11, the point that the amount of code generated in the immediately preceding unit can be immediately fed back is different from the image coding apparatus 100.
The coding mode is determined by Equation (2) using the available value calculated by Equation (3). In this embodiment, by determining the encoding mode as described above, it is possible to determine the encoding mode without having a margin corresponding to the maximum code amount for several units. Therefore, encoding with less image quality deterioration can be realized. The operation of the second encoding unit 104 will be described in detail. Similar to the image encoding device 100, the second encoding unit 104 of the present embodiment may use any method as long as encoding is performed with a predetermined code amount or less. However, the image encoding device 200 has restrictions on the generation of a local decoded image as compared with the image encoding device 100. By performing the processing at the timing shown in FIG. 10, the encoding processing in the first encoding mode and the second encoding mode is executed simultaneously. Since this is pipeline processing, a part of the local decoded image must be determined in the second processing stage of each unit. For example, when Unit # 0 executes the third processing stage, Unit # 1 executes encoding in the first encoding mode at the second processing stage, so Unit # 1
It is necessary to determine a local decoded image to be used for the prediction. Since the rightmost pixel in each unit is used in prediction, there is a restriction that the local decoded image must not differ between the first encoding process and the second encoding process.

FIG. 12 is a diagram illustrating an example of syntax of encoded data of units in the present embodiment. The above problem can be solved by the syntax shown in FIG. In FIG. 12, when codec_mode is MODE_1, the syntax in the first encoding mode is the same as that in FIG. On the other hand, when codec_mode is MODE_2,
The syntax in the second encoding mode is different from that in FIG. In the syntax in FIG.
Since only the information related to the prediction direction is encoded, the local decoded image becomes equal to the predicted image. Therefore, the local decoded image generated in the first encoding mode of the present embodiment may be different. Therefore, the pixel value of the local decoded image generated in the first encoding mode may be encoded by pixel code (value) as pixel_value for the pixel used for prediction of the immediately adjacent unit. PCM in this case refers to processing for outputting pixel values as they are. NUM_PCM
_PIXEL indicates the number of pixels encoded by pixel_value. In this case, the code amount in the second encoding mode is larger than that in FIG. 7, but the problem relating to the prediction of subsequent units can be solved.
FIG. 13 is a diagram illustrating another example of the syntax of the encoded data of the unit in the present embodiment. In FIG. 13, when the second encoding mode is selected, no additional information is encoded,
There are restrictions on the prediction direction. From equations (3) and (2), once the second encoding mode is selected, the first encoding mode is not selected in the same segment.
Therefore, for example, by prohibiting prediction from the left unit, it is possible to solve the problem related to prediction of the subsequent unit. In this embodiment, an example in which only prediction from above is always performed in the second encoding mode is described. The prediction from above is an example, and any direction may be used as long as it does not affect the prediction of subsequent units. In the encoding by the second encoding mode, the image quality is greatly deteriorated. In the present embodiment, by performing the coding mode determination as described above, the second coding mode is controlled so as not to be selected as compared with the image coding device 100, and the upper limit of the cumulative code amount generated in the segment is controlled. Can be guaranteed.

(Third embodiment)
The image encoding device 200 includes a first encoding unit 1 in order to guarantee the code amount in segment units.
03 and the second encoding unit 104 are adaptively switched. However, the image coding apparatus 200 can guarantee the code amount in the segment unit, but cannot guarantee the code amount in the unit unit. For example, depending on the encoding parameter, there is a possibility that encoded data having a code amount larger than the data amount of the input image is generated by the first encoding unit 103 performing encoding. Therefore, the encoding apparatus 300 of the present embodiment includes the first encoding unit 1
03 and the 3rd encoding part 117 which encodes by the method different from the 2nd encoding part 104 are provided.

  FIG. 14 is a diagram illustrating the image encoding device 300.

The image encoding device 300 includes a first encoding unit 103, a second encoding unit 104, and a third encoding unit 1
17, entropy encoding units 105A to 105C, mode determination unit 120, switch 118, switch 119, local code generation units 107A to 107C, and encoding control unit 108. A description of the configuration that performs the same operation as that of the image encoding devices 100 and 200 will be omitted. The third encoding unit 117 receives the pixels of the encoding target unit, performs an encoding process so as to obtain a predetermined code amount, and generates third encoded data.

The switch 118 includes a local decode generation unit 107A and a local decode generation unit 107.
The local decoding images of the first encoding mode, the second encoding mode, and the third encoding mode are received from the B and local decode generation unit 107C. The switch 118 selects any local decoded image according to the encoding mode determined by the encoding mode determination unit 120.
The data is sent to the first encoding unit 103, the second encoding unit 104, and the third encoding unit 117. The local decoded image is used for prediction as a reference image when a subsequent unit is encoded.

The switch 119 receives bit streams of the first encoding mode, the second encoding mode, and the third encoding mode from the entropy encoding unit 105A, the entropy encoding unit 105B, and the entropy encoding unit 105C. The switch 119 outputs any bit stream according to the encoding mode determined by the encoding mode determination unit 120.

The encoding mode determination unit 120 receives the amount of code generated when encoding in the first encoding mode from the entropy encoding unit 105A, sets the encoding mode information for the target unit, and outputs it to the switch 118 and the switch 119 To do. The overall processing flow of the image encoding device 300 will not be described. For example, S1101 to S11 in FIG.
The third encoding unit 117 may perform encoding at the same time as 07 and S1108 to S1111 and output a bit stream according to the selected encoding mode. In the following description, an example in which the image encoding device 300 performs pipeline processing according to the timing chart of FIG. As another modification, as shown in the first embodiment, when the determined encoding mode indicates the third encoding mode, the third encoding unit 117 includes the third encoded data and It may be configured to generate a local decoded image. The operation of the third encoding unit 117 will be described in detail.

FIG. 17 is a diagram illustrating details of an operation in which the third encoding unit 117 determines the encoding mode.
As described above, the encoding amount by the first encoding unit 103 cannot always guarantee the unit-unit code amount. On the other hand, since the second encoding unit 104 performs encoding with a predetermined code amount, the code amount can be guaranteed even in units. However, since the encoding by the second encoding unit 104 is assumed to exceed the target code amount for each segment, the image quality is significantly deteriorated.

The third encoding unit 117 performs encoding with a predetermined code amount or less. It is assumed that the code amount of the third encoding unit 117 is sufficiently larger than the code amount of the second encoding unit 104. When encoding is performed by the first encoding unit 103 and a code amount larger than the code amount of the third encoding unit 117 is generated, switching to encoding by the third encoding unit 117 is performed. It is possible to guarantee the maximum generated code amount in units.

Although any method may be used as the encoding method in the third encoding unit 117, encoding with a specific code amount or less is required as described above. For example, the pixel value of the input image may be output as it is in the PCM mode. As described above, the encoding by the first encoding unit 103 may generate encoded data that exceeds the data amount of the input image. Therefore, by setting the third encoding mode to PCM, when the first encoding unit 103 exceeds the data amount of the input image, encoding by PCM can be performed. An example of syntax in this case is shown in FIG.
The case where codec_mode indicates MODE_1 and MODE_2 is the same as that in FIG. 13, but in the case of MODE_3, that is, in the third encoding mode, NUM_PIXEL_
As many pixel values as the number of pixels in the unit indicated by UNIT are input_pixel_val.
Encoded as ue.

Although the case of PCM has been described here, quantization is performed on the pixel value of the input image,
The number of bits per pixel may be reduced. In addition, a prediction / conversion process similar to that of the first encoding unit 103, or a general DPCM (differentiated) for PCM encoding a difference between adjacent pixels
It is also possible to perform quantization with a predetermined parameter for the transform coefficient and the error image by performing a process based on (alpulse-code modulation).

Further, when it is necessary to generate a local decoded image generated not in the input image but in another encoding mode for some or all pixels in the unit, difference data may be added. Similar to the second encoding mode, there are restrictions on the local decoded image in the third encoding mode. That is, the pixel existing at the right end in the unit needs to be the same as that generated by the first encoding unit 103. For example, when encoding by PCM, the pixel value of the input image is encoded by PCM, and then the pixel value of the local decoded image generated by the first encoding unit 103 and the input image for the rightmost pixel of the unit. The difference value with the pixel value of is calculated, and the difference value is encoded. Thereby, the above problem can be avoided. The corresponding local decode generation unit can obtain the rightmost pixel in the unit by adding the pixel value of the input image obtained by decoding the stream generated by PCM and the decoded difference value. Therefore, the encoding apparatus 300 may perform the prediction process using the pixel of the local decoded image generated by the first encoding unit 103 as a reference pixel. At this time, the third encoding unit 117 may receive and process the pixel value of the local decoded image generated by the first encoding unit 103, or may be generated by the third encoding unit 117.

FIG. 16 is a diagram illustrating an example of syntax. In the example of FIG. 16, the difference value between the pixel value of the local decoded image generated by the first encoding unit 103 and the pixel value of the input image is expressed as diff_pixel.
It differs from the example of FIG. 15 in that it is encoded as l_value. NUM_REF_PIXE
L indicates the number of pixels used as reference pixels. In this embodiment, only the rightmost pixel of the unit is targeted. Therefore, NUM_REF_PIXEL has the same value as the unit height. Note that the pixel at the lower end of the unit may be a target in order to avoid complicated processing.

The above processing is an example, and any method may be used as long as it satisfies the restrictions on the rightmost pixel in the unit.

  The operation of the encoding mode determination unit 120 will be described in detail.

The encoding mode determination unit 120 receives the code amount generated in the first encoding mode from the entropy encoding unit 105A, and determines the encoding mode in the target unit. As shown in Expression (4), the mode determination unit 120 sets the smaller value of the code amount BMODE_1 in the first encoding mode and the code amount BMODE_3 in the third encoding mode as Bcurrent.

The encoding mode determination unit 120 uses the obtained Bcurrent to calculate B according to Equation (3).
Available is calculated. The encoding mode is determined by equation (5).

  FIG. 17 is a diagram illustrating the operation of the encoding mode determination unit 120.

First, when BMODE_1 becomes BMODE_3 or more (S1701, yes),
codec_type is MODE_3, and Bcurrent is BMODE_3 (
S1702). Otherwise (S1701, no), codec_type is set to MODE.
_1 and Bcurrent is set to BMODE_1 (S1703).

Available is calculated according to equation (3) (S1704). Whether the code amount in the segment can be guaranteed even if encoding is performed in the first encoding mode or the third encoding mode that has already been selected is determined in the same manner as in Expression (2). If it is determined that the code amount is not guaranteed at this time (
S1705, no), the second encoding mode is selected (S1706). Otherwise (S
1705, yes), the encoding mode selected in S1702 or S1703 is used.

With the above processing, while guaranteeing the maximum code amount in units by the third encoding mode,
The code amount in segment units can be guaranteed by the second encoding mode.

Here, the third coding mode is used for the purpose of guaranteeing the code amount in units. Third
Since the encoding mode assumes a sufficiently large code amount compared to the second encoding mode,
For example, high-quality encoding can be performed as compared with the first encoding mode and the second encoding mode, such as when using PCM. Therefore, for example, as shown in Equation (6), when it is found that even if all the remaining units in the segment are encoded in the third encoding mode, the target code amount is not exceeded, encoding in the third encoding mode is performed. You may switch to

According to the image encoding device 300 of the present embodiment, it is possible to perform an encoding process that guarantees a code amount in units.

(Fourth embodiment)
In general, it is possible to converge to a desired code amount by adaptively setting a quantization scale by rate control. Therefore, the image encoding device 400 according to the present embodiment performs rate control.

  FIG. 18 is a diagram illustrating the image encoding device 400.

The image encoding device 400 further includes a rate control unit 121. Other components perform the same operations as those of the image encoding device 300, and thus description thereof is omitted.

The rate control unit 121 receives the code amount of the first coding mode from the entropy coding unit 105 </ b> A and the coding mode information from the coding mode determination unit 120. The rate control unit 121 performs rate control by adaptively setting the encoding parameters of the first encoding unit 103 through the encoding control unit 108. For rate control, a general method such as that used in MPEG-2 TM5 may be used.

  The operation of the rate control unit 121 will be described.

  FIG. 19 is a diagram illustrating an operation example of the image encoding device 400.

When an image to be encoded is input to the image encoding device 400, the rate control unit 121
Sets parameters for rate control (S1901). For example, when the rate control unit 121 performs the above-described TM5 rate control, an initial value of the rate buffer and a feedback strength for the generated code amount are set. A predetermined fixed value may be used for the rate control parameter. Further, it may be set according to information such as the size and format of the input image.

The rate control unit 121 sets a coding parameter such as a quantization parameter required in the first coding mode based on the set rate control parameter (S1902). The first encoding unit 103 performs encoding in the first encoding mode on the unit of the input image according to the encoding parameter set in S1902 (S1903). Second encoding unit 10
4 and the third encoding unit 117 perform encoding in the second encoding mode and the third encoding mode, respectively (S1904, S1905). For these encoding processes, the image encoding device 3
Since the same operation as 00 is performed, the description is omitted.

Based on these encoding results, the encoding mode determination unit 120 selects an encoding mode (
S1906). The switch 118 sends a local decoded image corresponding to the selected encoding mode, and the switch 119 outputs a bitstream corresponding to the selected encoding mode (S1907). When encoding of all units in the segment is not completed (S1908, no), the entropy encoding unit 105A sends the code amount of the first encoding mode to the encoding mode determination unit 120 and the rate control unit 121, The encoding mode determination unit 120 sends the encoding mode information to the rate control unit 121 (S1909). In the second encoding mode and the third encoding mode, encoding is performed with a predetermined code amount, so that it is possible to determine the code amount using only the encoding mode information without sending the generated code amount.

When encoding of all units in the segment is completed (S1908, yes) and the second encoding mode is selected (S1910, yes), the rate control unit 121 changes the parameter of the next segment. (S1911). As a parameter changing method, for example, the feedback strength of the code amount is increased. The fact that the second coding mode has been selected means that the normal rate control cannot be converged because the amount of code is excessive. By changing the rate control parameters adaptively, stable control can be performed so that the second coding mode is not selected in the next segment.

In the present embodiment, the processing when the second coding mode is selected has been described. However, when the third coding mode is selected due to the remaining code amount, a sufficient code amount cannot be used. It shows that. In such a case, it is possible to obtain a stable image quality by setting parameters such as weakening the feedback intensity.

Although the case where the rate control parameter is changed in units of segments has been described here, a unit in which a plurality of units are combined or a frame unit may be used.

Thus, the rate control unit 121 feeds back the generated code amount in units and controls the rate. For example, when a code amount larger than the target code amount is generated, the generated code amount is suppressed by increasing the quantization parameter, and when a code amount smaller than the target code amount is generated, the quantum amount is increased. It is possible to achieve convergence to a target code amount by performing encoding so that the generated code amount is increased by reducing the encoding parameter.

In the present embodiment, the rate control unit 121 has been explicitly described for the sake of clarity.
These rate control processes may be performed by the encoding control unit 108.

Although several embodiments of the present invention have been described, these embodiments are presented by way of example and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the scope of the invention. These embodiments and modifications thereof are included in the scope and gist of the invention, and are included in the invention described in the claims and the equivalents thereof.

Image coding apparatus 100, 200
Switches 101, 102,
First encoding unit 103,
Second encoding unit 104,
Entropy encoding unit 105, 105A, 105B
Coding mode determination unit 106, 116
Local decode generators 107, 107A, 107B
Prediction error image generation unit 109, inverse quantization / inverse transformation unit 112, addition unit 113
Encoding control unit 108
Rate control unit 121

Claims (10)

A first encoding unit capable of encoding each of a plurality of pixel blocks included in the first segment of the input image;
A second encoding unit capable of encoding each of the plurality of pixel blocks with a predetermined first code amount;
A determination unit that determines which of the first encoding unit and the second encoding unit is used for an encoding target pixel block that is one of the plurality of pixel blocks;
With
The determination unit includes a cumulative code amount from a first pixel block encoded first among the plurality of pixel blocks to a second pixel block encoded immediately before the pixel block to be encoded, and the first The sum of the code amount of the encoding target pixel block encoded by one encoding unit is encoded after the target pixel amount of the first segment and the encoding target pixel block among the plurality of pixel blocks. If less than the difference between the code amount generated by encoding by the pixel block to be of the second encoding unit, and determine the constant with the first encoding unit,
When it is determined that the second encoding unit is used, at least of the encoding target pixel block used for prediction in a pixel block encoded after the encoding target pixel block among the plurality of pixel blocks. A part of the image encoding device encodes a difference between a pixel value of a local decoded image generated by the first encoding unit and a pixel value of the encoding target pixel block .   The determination unit encodes the encoding target pixel block after the first encoding unit encodes the encoding target pixel block and the second encoding unit encodes the encoding target pixel block. The image encoding device according to claim 1, wherein it is determined which of the first encoding unit and the second encoding unit is used. A first encoding unit capable of encoding each of a plurality of pixel blocks included in the first segment of the input image;
A second encoding unit capable of encoding each of the plurality of pixel blocks with a predetermined first code amount;
A determination unit that determines which of the first encoding unit and the second encoding unit is used for an encoding target pixel block that is one of the plurality of pixel blocks;
With
The determination unit includes a cumulative code amount from a first pixel block encoded first among the plurality of pixel blocks to a third pixel block encoded before the pixel block to be encoded, The sum of the maximum code amount when the first encoding unit encodes the pixel block after 3 pixel blocks to the encoding target pixel block is the target code amount of the first segment and the plurality of pixel blocks. When the difference between the pixel block encoded after the encoding target pixel block and the code amount generated by encoding the pixel block by the second encoding unit is smaller than the first encoding unit, Is determined to be used ,
When it is determined that the second encoding unit is used, at least of the encoding target pixel block used for prediction in a pixel block encoded after the encoding target pixel block among the plurality of pixel blocks. A part of the image encoding device additionally encodes a difference between a pixel value of a local decoded image generated by the first encoding unit and a pixel value of the encoding target pixel block . The second encoding unit does not perform prediction using a pixel of a pixel block encoded immediately before the encoding target pixel block;
The image encoding device according to any one of claims 1 to 3.   A first encoding unit capable of encoding each of a plurality of pixel blocks included in the first segment of the input image;
  A second encoding unit capable of encoding each of the plurality of pixel blocks with a predetermined first code amount;
  A third encoding unit capable of encoding each of the plurality of pixel blocks with a second code amount that is predetermined and larger than the first code amount;
  It is determined which of the first encoding unit, the second encoding unit, and the third encoding unit is used for an encoding target pixel block that is one of the plurality of pixel blocks. A determination unit;
With
  The determination unit performs third encoding when the code amount of the encoding target pixel block by the first encoding unit exceeds the code amount of the encoding target pixel block by the third encoding unit. Part to use,
  If it is determined that the third encoding unit is to be used, at least of the encoding target pixel block used for prediction in a pixel block encoded after the encoding target pixel block among the plurality of pixel blocks. A part of the image encoding device additionally encodes a difference between a pixel value of a local decoded image generated by the first encoding unit and a pixel value of the encoding target pixel block. When the second encoding unit encodes the encoding target pixel block, the code in the encoding for the second segment included in the input image rather than the feedback strength of the code amount in the encoding for the first segment. as towards the amount of feedback intensity increases, the image coding apparatus according to any one of claims 1 to 4, further comprising a rate control unit for rate controlling the first encoding unit. It said second encoding unit performs only predictions from above, do not encode the transform and quantization coefficients, the image coding apparatus according to any one of claims 1 to 4. A first encoding step capable of encoding each of the plurality of pixel blocks included in the first segment of the input image;
A second encoding step capable of encoding each of the plurality of pixel blocks with a predetermined first code amount;
A determination step of determining which of the first encoding step and the second encoding step is used for an encoding target pixel block that is one of the plurality of pixel blocks;
With
In the determination step, a cumulative code amount from a first pixel block encoded first among the plurality of pixel blocks to a second pixel block encoded immediately before the encoding target pixel block, and the first The sum of the code amount of the encoding target pixel block encoded by one encoding step is encoded after the target code amount of the first segment and the encoding target pixel block among the plurality of pixel blocks. If less than the difference between the code amount generated by encoding by the pixel block to be of the second coding step, it determines that use of the first encoding step,
When it is determined that the second encoding unit is used, at least of the encoding target pixel block used for prediction in a pixel block encoded after the encoding target pixel block among the plurality of pixel blocks. A part of the method is an image encoding method for encoding a difference between a pixel value of a local decoded image generated by the first encoding unit and a pixel value of the encoding target pixel block. A first encoding step capable of encoding each of the plurality of pixel blocks included in the first segment of the input image;
A second encoding step capable of encoding each of the plurality of pixel blocks with a predetermined first code amount;
A determination step of determining which of the first encoding step and the second encoding step is used for an encoding target pixel block that is one of the plurality of pixel blocks;
With
In the determination step, a cumulative code amount from a first pixel block encoded first among the plurality of pixel blocks to a third pixel block encoded before the encoding target pixel block, and the first The sum of the maximum code amount when the first encoding unit encodes the pixel block after 3 pixel blocks to the encoding target pixel block is the target code amount of the first segment and the plurality of pixel blocks. When the pixel block to be encoded after the pixel block to be encoded is smaller than the difference in code amount generated by encoding by the second encoding unit, the first encoding step is performed. It is determined that it will be used,
When it is determined that the second encoding unit is used, at least of the encoding target pixel block used for prediction in a pixel block encoded after the encoding target pixel block among the plurality of pixel blocks. A part of the method is an image encoding method for encoding a difference between a pixel value of a local decoded image generated by the first encoding unit and a pixel value of the encoding target pixel block .   A first encoding step capable of encoding each of the plurality of pixel blocks included in the first segment of the input image;
  A second encoding step capable of encoding each of the plurality of pixel blocks with a predetermined first code amount;
  A third encoding step in which each of the plurality of pixel blocks is predetermined and can be encoded with a second code amount larger than the first code amount;
  It is determined which of the first encoding step, the second encoding step, and the third encoding step is used for an encoding target pixel block that is one of the plurality of pixel blocks. A determination step;
With
  In the determination step, if the code amount of the pixel block to be encoded by the first encoding step exceeds the code amount of the pixel block to be encoded by the third encoding step, a third encoding is performed. Decide to use a step,
  If it is determined that the third encoding step is used, at least of the encoding target pixel block used for prediction in a pixel block encoded after the encoding target pixel block among the plurality of pixel blocks. A part of the image encoding method, wherein a difference between a pixel value of a local decoded image generated by the first encoding unit and a pixel value of the encoding target pixel block is additionally encoded.

Images