JP2007267340A - Image encoder - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To reduce an operation amount and to evade extreme deviation of a code amount by executing encoding and rate control so that the code amount becomes an upper limit value or less and a lower limit value or more at specified parts in the compression encoding of frames. <P>SOLUTION: The image encoder comprises: an entropy encoding means 103 for outputting code data for each encoding path on the basis of control information, a rate control information extraction means 105 for judging the continuation and end of the encoding path on the basis of the calculation of distortion and the code amount needed for the rate control, and a code data extraction means 106 for extracting the code data to an encoding end path on the basis of the control information. The rate control information extraction means 105 comprises: calculation means 111 and 112 for calculating the distortion and the code amount generated by the encoding of the respective encoding paths; a calculation means 114 for calculating an inclination based on the relation of the distortion and the code amount; and an encoding path deriving means 115 for outputting the control information so that the total code amount of entire components is settled to be a target code amount or less, and the total code amount of the specified component is settled to be a prescribed maximum value or less on the basis of the inclination, the total code amount of the entire components, and the total code amount of the specified component. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an image coding apparatus capable of rate control for limiting the code amount distribution to a specific component in a frame to a specified lower limit value or more or an upper limit value or less.

  Currently, the still image coding algorithm JPEG is widespread mainly on the Internet, but on the other hand, the JPEG2000 project has been newly launched since 1997 against the background of further performance improvement and function addition as the next generation coding method. And started by a joint organization of ITU. In December 2000, the main technical content of Part 1 that defines the basic method of the JPEG2000 algorithm was finalized. The outline of the basic scheme of the JPEG2000 encoding algorithm will be described below according to the recommendation (ISO / IEC 154444-1: 2000).

  First, an input image signal is subjected to two-dimensional wavelet transform by a wavelet transform unit, and is divided into a plurality of subbands. Here, the two-dimensional wavelet transform is realized as a combination of the one-dimensional wavelet transform. That is, a process of sequentially performing vertical one-dimensional wavelet transform for each column and a process of sequentially performing horizontal one-dimensional wavelet transform for each line. Here, the one-dimensional wavelet transform is composed of a low-pass filter, a high-pass filter, and a down sampler having predetermined characteristics.

  The two-dimensional wavelet transform coefficient generated in this way is expressed by expressing the low-frequency component as L, the high-frequency component as H, the horizontal conversion by the first character, and the vertical sub-scanning conversion by the second character. Expressed as LL, HL, LH, HH. These band-divided components are called subbands. Here, the horizontal and vertical low-frequency components (LL components) are recursively subjected to wavelet transform. The number of wavelet transforms recursively applied is called a decomposition level, and the level is described before LL, HL, LH, and HH. In other words, when the number of wavelet transform decompositions is 2, the decomposition level of the lowest resolution component is 2, whereas the decomposition level of HL, LH, and HH of the highest resolution component is 1.

  Next, the wavelet transform coefficient in each subband is quantized with a quantization step size set for each subband.

  Next, after the wavelet transform coefficients after quantization of each subband are divided into fixed-size areas called code blocks, the code block consisting of multi-value data is converted into a binary bit plane representation, and each bit plane is converted to It is divided into three encoding passes (Significant Propagation Decoding Pass, Magnitude Refinement Pass, and Cleanup Pass).

  The binary signal output from the three coding passes is subjected to context modeling for each coding pass and subjected to entropy coding.

  In parallel with the entropy encoding process, the code amount and distortion information for each encoding pass are calculated in each code block.

  Finally, rate control is performed to adjust the code amount to a target code size or less while minimizing image quality degradation (distortion) using the Lagrange multiplier method.

  The rate control method is not standardized, and any method can be used according to the application, but the following is recommended by the recommendation (ISO / ITU 15444-1: 2000) An outline of the mechanism of the rate control unit described as reference information in 14.3 will be described.

In this method, when the truncation point in each code block i is ni, the code amount up to each truncation point is R (i, ni), and the distortion is D (i, ni), the Lagrange multiplier method is used. The variable λ is adjusted until the total code amount R in the entire screen generated by the truncation point that maximizes the expression (1) is within the target code amount Rmax.
Σ (R (i, ni) −λD (i, ni)) (1)

  Here, the distortion indicates how much the mean square error of the reproduced image is reduced compared to when the code data is not transmitted when a code up to a certain coding pass is sent. Strictly speaking, This is the amount of distortion reduction. Therefore, before encoding, when distortion D is 0 and encoding is performed up to the final bit plane, distortion D becomes equal to the mean square error.

Finding the truncation point that maximizes the expression (1) is to find the truncation point at which the slope of the tangent line becomes λ ⁻¹ when the code amount R and distortion D of each code block are represented as an RD curve. Is equivalent to In the two code blocks c1 and c2, the truncation points where the slope of the tangent is λ ⁻¹ are nc1 and nc2, and the code amounts up to the truncation points are R (c1, nc1) and R (c2, nc2) Such R is added to all code blocks and compared with Rmax. When this is seen for each code block, it is necessary to find a truncation point ni that maximizes (R (i, ni) −λD (i, ni)) as follows.

Set ni = 0
For k = 1, 2, 3, ...
Set ΔR (i, k) = R (i, k) −R (i, ni)
and ΔD (i, k) = D (i, k) −D (i, ni)
If (ΔD (i, k) / ΔR (i, k))> λ ⁻¹ then set ni = k

  However, with this algorithm, the truncation point ni cannot be obtained unless the above processing is executed for a large number of λs. Therefore, the inclination S (i, k) = ΔD (i, k) / ΔR (i, k) is corrected in advance so as to monotonically decrease with respect to k. Specifically, the process is performed as follows.

(1) set Ni = {n} (ie the set of all truncation point)
(2) Set p = 0
(3) For k = 1, 2, 3,..., Kmax
If k belongs to Ni
Set ΔR (i, k) = R (i, k) −R (i, p),
and ΔD (i, k) = D (i, k) −D (i, p)
Set S (i, k) = ΔD (i, k) / ΔR (i, k)
If p ≠ 0 and S (i, k)> S (i, p),
then remove p from Ni, and go to step (2)
Otherwise, set p = k

This process optimization truncation points for lambda given, may be the largest k in Ni satisfying S (i, k)> λ -1.

  When the derivation of such information is completed for all the code blocks, code data is generated so that the target code amount Rmax is obtained. Specifically, λ that gives the maximum Rsum that satisfies Rsum ≦ Rmax with respect to the total code amount Rsum for a certain λ is found. Here, the total code amount for a certain λ can be found only after the cut point is uniquely determined in each code block and the sum of the code data up to the cut point is calculated. Therefore, in order to find λ that gives the maximum Rsum satisfying Rsum ≦ Rmax, generally, the total code amount for a plurality of candidates of λ is calculated, and λ that gives the total code amount close to a desired value is calculated by a convergence operation. calculate. When λ is obtained, the code data up to the cut-off point corresponding to the λ is collected from all the code blocks, and the number of coding passes in each code block is added as additional information to form the final code data. In this way, code data that minimizes distortion under the target code amount Rmax can be generated.

  The JPEG2000 international standard as described above can be obtained through a standardization organization such as ISO or ITU-T. The latest information on JPEG2000 can be obtained by referring to http://www.jpeg.org.

  Further, as this type of image encoding device, there is one in which the code amount distribution to each block is made uniform even when the total code amount exceeds a certain amount (see, for example, Patent Document 1).

  In addition, by considering additional code amount distribution that does not depend on the quantization width, there is a method that solves the shortage of local code distribution amount and optimizes the code amount for each frame (for example, Patent Documents). 2).

  In addition, there is one in which code amount distribution is determined using frequency characteristics (see, for example, Patent Document 3).

  Also, the target information amount of each encoding method is calculated and distributed so that the amount of information generated by encoding with I, P, and B pictures is optimized in accordance with the visual characteristics of the image. There are some (see, for example, Patent Document 4).

  Furthermore, there is a technique that enables high coding processing by improving the efficiency of division processing for calculating the slope of code amount versus distortion (for example, see Patent Document 5).

JP 09-252477 A JP-A-10-108179 JP 2003-230162 A Japanese Patent Laying-Open No. 2005-303362 JP 2005-109917 A

  In the rate control method described above, code allocation that minimizes coding distortion over the entire screen with a certain total code amount is performed, so that most codes are concentrated on specific components depending on the image, and some decoders fail. May also occur. On the other hand, depending on the image, the code amount of the specific component is very small, and even if it is optimal from the viewpoint of minimizing coding distortion in the entire screen, there is a possibility that the subjective image quality will partially deteriorate. .

  In addition, when the code amount of the specific component is limited by the above method, after the code amount distribution is determined by the convergence calculation, when the code amount of the specific component is out of the minimum value range from the specified maximum value, Again, it is necessary to determine the code amount distribution again by the convergence calculation excluding the specific component. When there are a plurality of specific components, there is a problem that the number of code amount allocations is further increased and the amount of calculation increases.

The present invention has been made to solve the above-described problems. In each code block, the process of encoding only the coding pass whose slope is larger than the parameter λ ⁻¹ is sequentially performed by changing the value of λ. In this process, an image encoding device capable of performing encoding and rate control in one pass by monitoring the code amount of a specific component, reducing the amount of calculation, and avoiding an extreme code amount bias. The purpose is to obtain.

  The image encoding device according to the present invention divides a two-dimensional signal into a plurality of code blocks, decomposes the code block into bit planes, divides each bit plane into one or more encoding passes, and based on control information An entropy encoding means for encoding and outputting code data for each encoding pass, a code memory for storing code data from the entropy encoding means, and calculation of distortion and code amount necessary for rate control Rate control information extraction means for determining the continuation and termination of the coding pass, and code data up to the coding end pass are extracted from the code memory based on the control information from the rate control information extraction means and output as a code stream A rate calculation information extracting unit that calculates a distortion caused by encoding of each encoding pass. A code amount calculating means for calculating the code amount of each coding pass, and a slope based on the relationship between the distortion and the code amount in each code block from the distortion from the distortion calculating means and the code amount from the code amount calculating means. Inclination calculation means for calculating, based on the inclination, the total code amount of all components, and the total code amount of specific components, the total code amount of all components is equal to or less than the target code amount, and the total code amount of specific components is a predetermined maximum value The encoding path deriving unit outputs control information for controlling each encoding to the entropy encoding unit and the code data extraction unit so as to be within the following range.

According to the present invention, the code amount of the specific component is monitored while the process of encoding only the coding pass whose slope is larger than the parameter λ ⁻¹ in each code block is sequentially performed by changing the value of λ. It is possible to perform encoding and rate control in one pass, reduce the amount of calculation, and avoid an extreme bias in code amount.

Embodiment 1 FIG.
The embodiment of the present invention shows a method for rate control so that the code amount of a specific component is within a maximum value specified in advance when encoding is performed. Here, the specific component means that only the R component, the lowest resolution component, etc. are specified among the three color components of RGB.

  FIG. 1 is a block diagram showing a configuration of an image coding apparatus according to Embodiment 1 of the present invention. 1 includes a wavelet transform unit 101 that recursively performs two-dimensional wavelet transform on an input image signal, and a quantization step in which wavelet transform coefficients generated by the wavelet transform unit 101 are set in advance. Quantization means 102 that performs quantization processing by size, entropy encoding means 103 that decomposes the quantized wavelet transform coefficients into bit planes and performs binary arithmetic encoding, and temporarily stores the entropy encoded code data A code memory 104, a rate control information extraction unit 105 that calculates distortion and code amount necessary for rate control, determines to which path encoding is to be performed, and which path is to be encoded, rate control Code data up to the encoding end pass according to the encoding end pass output from the information extraction means 105. Extracting from the code memory 104, and a code data extracting means 106 for output as a code stream.

  FIG. 2 is a block diagram showing the configuration of the rate control information extracting means 105 shown in FIG. As shown in FIG. 2, the rate control information extraction unit 105 is based on the distortion calculation unit 111 that calculates distortion for each coding pass based on the output from the quantization unit 102 and the output from the entropy encoding unit 103. A code amount calculation unit 112 that counts the code amount for each coding pass, and a rate at which the distortion data from the distortion calculation unit 111, the code amount data from the code amount calculation unit 112, the inclination thereof, and the like are stored for each coding pass. The distortion memory 113, the inclination calculation means 114 for calculating the inclination of the code amount versus distortion, and whether or not to continue the encoding in the encoding pass, the encoding pass to be encoded is derived and the result is A coding path deriving unit 115 that outputs the control information to the entropy coding unit 103 and the code data extraction unit 106;

  Next, the operation will be described. First, in FIG. 1, for example, an image signal from an image input device (not shown) such as an image scanner, a digital camera, or a network or a storage medium is converted into a vertical and horizontal one-dimensional wavelet transform by a wavelet transform unit 101. It is applied two-dimensionally in both directions, and is divided into subbands. Here, the one-dimensional wavelet transform is realized by a filter bank of a low-pass filter and a high-pass filter. FIG. 3 shows an example in which two-dimensional wavelet transformation is recursively performed twice by the wavelet transformation means 101 shown in FIG. In the figure, the first number represents the decomposition level, and the following two alphabetic characters L or H represent the types of filters in the horizontal and vertical directions. L represents a low-pass filter, and H represents the result of applying a high-pass filter. In addition, “recursively” performing wavelet transform twice means that when 1LL, 1HL, 1LH, and 1HH are first generated by the first wavelet transform, the second wavelet transform is performed on the 1LL. This means that 2LL, 2HL, 2LH, and 2HH are generated by conversion.

  The quantization processing means 102 quantizes the wavelet transform coefficient with the quantization step size set for each subband.

  The entropy encoding unit 103 divides the wavelet transform coefficients of each subband into fixed-size rectangular areas called code blocks, and then converts each code block made up of multi-value data into a binary bit plane. Usually, the size of the code block is set to a size of 64 × 64, 32 × 32, or the like.

  Here, the decomposition of the bit plane will be described in detail with reference to FIG. FIG. 4A shows an example of a 4 × 4 code block. FIG. 4B shows the result of converting these data into a 1-bit signal representing the positive and negative and the representation of the absolute value, and arranging the data in binary representation in the vertical direction for each row. . Next, FIG. 4C shows a collection of bits having the same bit numbers as in FIG. 4B. Here, when the least significant bit (LSB) is the 0th bit and the most significant bit (MSB) is the 3rd bit, the 0th bit plane is a collection of the 0th bit. A collection of 1 bits is a first bit plane, a collection of 2 bits is a second bit plane, and a collection of 3 bits is a third bit plane. In addition to this, a sign bit plane is created as a collection of bits representing positive and negative.

  In the entropy encoding means 103, each bit in the bit plane is divided into three types of encoding pass, Significant Propagation Decoding Pass, and Magnitude Refinement Pass according to the context. And divide it into Cleanup Pass.

  Next, the entropy encoding means 103 performs context modeling for arithmetic encoding for each encoding pass. However, bit planes that are all 0 from the MSB plane are not subjected to context modeling and encoding, and only the number of all 0 bit planes is written in the header. For the bit plane in which 1 appears first, all bits are classified as a cleanup pass, while the other bit planes are classified into three types of encoding passes as described above.

  FIG. 5 shows an example in which the number of bit planes in the code block is 6 and the number of effective bit planes is 4. When the context modeling is completed, entropy coding using arithmetic codes is performed.

  In parallel with the entropy encoding process, the distortion calculation unit 111 in the rate control information extraction unit 105 performs the encoding pass and the previous previous encoding pass every time encoding of a certain path in each code block is completed. The difference ΔD of the distortion D at, and its accumulated value D = D + ΔD are calculated. Here, the distortion indicates how much the mean square error of the reproduced image is reduced when a code up to a certain coding pass is transmitted compared to when the code data is not transmitted. That is, the amount of reduction in deformation. Therefore, accumulating the difference ΔD up to the last bit plane is equal to the mean square error. As a calculation method of the difference ΔD, a method introduced in Annex J of JPEG2000 recommendation (ISO / ITU 15444-1: 2000) can be used.

  Simultaneously with the processing of the distortion calculation unit 111, the code amount calculation unit 112 causes the number of output bytes ΔR in the coding pass and the accumulated value R = R + ΔR each time encoding of a certain path is completed in each code block. Is calculated. These distortion D and code amount R are stored in the rate distortion memory 113 after being assigned indexes such as tile numbers, decomposition levels, subbands, code blocks, and encoding passes.

  In addition, the slope calculation unit 114 calculates the slope S from the code amount data R and distortion data D, and the rate distortion memory 113 has the same coding path slope as the distortion data and code amount data. Store it in a location where it can be seen. As a result, data is stored in the rate distortion memory 113 as shown in FIG.

  Further, the coding path deriving unit 115 determines whether or not to continue the encoding in the code block from the slope S and the parameter λ (hereinafter, rate control parameter) to a further path, and the determination result is an entropy code. Sent to the digitizing means 103. Also, the encoding pass deriving unit 115 skips the encoding of the code block if the encoding target code block is included in the specific component and the total code amount of the specific component has reached the specified maximum value. Instructs the entropy encoding means 103.

  If the encoding is continued, the entropy encoding unit 103 encodes the next pass, and the rate control information extraction unit 105 calculates the slope from the output byte number ΔR and distortion D difference ΔD in the encoding pass. The means 114 calculates the slope S and again determines whether or not to continue the encoding until a further pass. If the encoding is not continued, the encoding end information is sent to the entropy encoding means 103 and the encoding end pass is sent to the code data extraction means 106. The encoding control related to the rate control will be described in detail later using a flowchart.

  The code data extracting means 106 reads code data from the code memory 104 from the encoding end pass to the determined path from the code memory 104, adds the number of encoding passes included in each code block as additional information, and then adds them. Arrange in the specified order, add predetermined header information, and output as a code stream.

Here, details of processing in the rate control information extraction unit 105 and the entropy encoding unit 103 will be described. In this processing method, a plurality of rate control parameter λ candidates are prepared in advance, and encoding is performed for all code blocks up to an encoding pass that satisfies a certain parameter λ. At that time, it is determined whether or not the code amount has reached the target amount. If it has reached, the encoding is completed. If not, the next parameter λ candidate is set, and the parameter λ is set for all code blocks. Encode again until satisfied. Here, not only the total code amount of all code blocks but also the total code amount of code blocks of a specific component designated in advance is monitored. If the code amount of the specific component reaches the maximum value designated in advance, the code allocation to the specific component is determined as the maximum value, and thereafter, the code amount to the specific component is not increased. In this way, the process of encoding by gradually changing the parameter λ is performed until the total code amount reaches the target amount. Whether or not a certain rate control parameter λ is satisfied is determined by calculating the slope S of the RD curve at the end of each path and determining whether or not the slope S has reached λ ⁻¹ .

Hereinafter, a method for determining a path to be encoded will be described with reference to a flowchart shown in FIG. 7 which is mainly an operation content of the encoded path deriving unit 115 in the rate control information extracting unit 105. The rate control parameter candidate λ (t) is set in advance as follows.
λ (t) = {λ (0), λ (1), λ (2),..., λ (tmax)}
(Λ (t) <λ (t + 1))

  The initial value of the index t of the rate control parameter λ is t = 0 (t = 0 to tmax), the code block index i = 0 (i = 0 to imax), the total code amount counter Rsum = 0, the specific component The total code amount counter Psum = 0, the variable k (i) for storing the coding pass in each code block is all −1 for the code block (the index of the next pass skipping the zero bit plane is 0, k (i ) = − 1 to kmax, and the initial value is k (i) = − 1 for the convenience of the counter.

  Although the memory is not shown in the block diagram, k (i) is a variable stored for each code block. The index t of the rate control parameter, the index i of the code block, and the counters Rsum and Psum of the total code amount are Variables common to all code blocks.

  In addition, an update flag indicating whether or not the encoding is further advanced in a certain code block is Cflag (i) = 1 (i = 0 to imax). Initially, the index t of the rate control parameter λ is incremented for all coding blocks, and when the total code amount of the specific component exceeds the specified maximum value during coding, the coding block belonging to the specific component The update flag is set to 0, and encoding is not performed thereafter. It is assumed that the target total code amount Rmax and the maximum value Pmax of the specific component code amount are specified in advance and stored in the coding path deriving unit 115.

  First, in step 701, it is determined whether or not the next encoding pass is to be encoded with the code block i using the update flag Cflag (i). Initially, encoding is performed for all code blocks. After the code amount of the specific component exceeds the specified code amount, the update flag of the code block belonging to the specific component is set to 0, and the rate control parameter is set further. Do not update. In step 702, when the update flag Cflag (i) is 1, the encoding pass to be encoded is set to k (i), and in step 703, the encoding pass k (i) of the code block i is set. Encode.

  After encoding, the difference ΔD between the encoding pass calculated by the distortion calculation unit 111 and the previous previous encoding pass ΔD, the code amount ΔR in the encoding pass calculated by the code amount calculation unit 112, etc. Using the relationship between the rate and distortion stored in the rate distortion memory 113, the slope S (i, k (i)) of the path k (i) is calculated in step 704.

  In step 705, the total code amount Psum of the specific component is calculated. Although not shown in the figure, information for determining whether or not each code block belongs to a specific component is included in the coding path deriving means 115, and each time a code block of a specific component is encoded The total code amount of the specific component is calculated by adding the code amount ΔR of the encoding pass to be encoded to the total code amount Psum of the specific component by the conversion path deriving unit 115. In step 706, the coding pass deriving unit 115 adds the code amount ΔR of the coding pass to be coded to the total code amount Rsum, thereby calculating the total code amount.

  Next, in step 707, the code amount Psum of the specific component is compared with the maximum value Pmax of the specific component code amount. If Psum> Pmax, in step 708, the update flag is set to Cflag = 0 for all code blocks belonging to the specific component, and thereafter, the code block of the specific component is not encoded. Since the maximum value Pmax is exceeded in pass k (i), in order to exclude the final coding pass of the current code block, in step 709, the index of the final coding pass is set to k (i) -1, and the total code The code amount ΔR of the final coding pass is subtracted from the amount counter.

  If Psum ≦ Pmax, encoding is continued without changing the update flag. In step 710, the total code amount Rsum is compared with the target total code amount Rmax. If the total code amount Rsum has reached the target total code amount Rmax, the encoding ends at this point, and in each code block, Information k (i) indicating to which path is encoded is sent to the code data extraction means 106. In step 712, since the maximum value Pmax is exceeded in pass k (i), the index of the final coding pass is set to k (i) -1 in order to exclude the final coding pass of the current code block.

If the total code amount Rsum does not reach the target total code amount Rmax, in step 711, the slope S (i, k (i)) in the current coding pass and the slope threshold value λ (t) ⁻¹ are calculated. If the slope S is large, the next pass is encoded (step 711 → step 702). If the slope S is less than λ (t) ⁻¹ , the encoded path code data is temporarily stored in the memory, the encoding of this code block is interrupted, and the code block index i is incremented. Then, the processing shifts to encoding of the next code block (step 711 → step 713 → step 714 → step 701). Although not shown in the flowchart, it is assumed here that even when the encoding pass is the final encoding pass of the code block, the processing of the next code block is performed.

In the next code block as well, encoding is performed until the slope becomes less than λ (t) ⁻¹ . After this is performed for all the code blocks, the index t of the parameter λ is incremented to decrease the slope threshold λ (t) ⁻¹ , λ is set as the next candidate, and the codes of all the code blocks are again set. Until the slope becomes less than λ (t) ⁻¹ (step 711 → step 713 → step 715 → step 701). In step 715, even if λ (t) is updated, S (i, k (i)) <λ (t) ⁻¹ , that is, the updated λ is already encoded. May be greater than the slope at. In this case, although not shown in the flowchart, the process jumps to step 713 to pass the encoding of the block, and proceeds to the encoding of the next code block.

  Next, how the rate distribution is determined by changing λ will be described using the example of FIG. In this example, image data composed of RGB three components is encoded with the total code amount target value Rmax = 100. The total code amounts of the R, G, and B components are R0, R1, and R2, respectively, and the maximum value Pmax of the R component code amount is 40.

Initially starting from lambda ^-1 = 100, continued coding until the slope becomes S <λ ^-1 = 100 in each code block. When this was executed for all code blocks, the total code amount Rsum = 55. Since the total amount of code Rsum is less than the target total code amount Rmax, the lambda as a step down lambda ^-1 = 90, again, it continued coding until the slope becomes S <λ ^-1 = 90 in each code block. When this was executed for all code blocks, the total code amount Rsum = 62.

Since the total code amount Rsum is still less than the target total code amount Rmax, when λ is lowered by one step and λ ⁻¹ = 80, the code amount of the R component has exceeded its maximum value 40 on the way. The code block belonging to the R component is not encoded thereafter. The maximum code amount that does not exceed the maximum value 40 for the specific component was 39. Encoding is continued for the G component and B component, and is continued until the slope S satisfies S <λ ⁻¹ = 80 in all code blocks excluding the R component. λ ⁻¹ = 80 When the encoding was completed, the total code amount Rsum = 69. In the table, the values that do not change because the update is stopped are shaded.

Since the total code amount Rsum is still less than the target total code amount Rmax, encoding is performed with λ as the next value 70. In this case as well, no further encoding is performed on the R component. Encoding for the G component and B component is continued until the slope S satisfies S <70 in all code blocks. λ ⁻¹ = 70 When the encoding was completed, the total code amount Rsum = 79.

  Even in this case, since Rsum is less than the target total code amount Rmax, encoding is performed with λ as the next value 60. During this encoding, since the total code amount has reached Rmax = 100, the encoding ends at that time. The maximum code amount not exceeding the target total code amount Rmax was R1 = 30 and R2 = 30. Through the above processing, the code amount allocation to the R, G, and B components is 39, 30, and 30, respectively, and the total code amount is 99.

  In the first embodiment, whether or not the code amount of the specific component is the maximum code amount is monitored and the code amount of the specific component is limited. However, the code amount is not necessarily limited. It is also possible to limit the coding distortion of a specific component to a specified value or less by monitoring the coding distortion. In the first embodiment, one type of specific component is used. However, it is also possible to set a plurality of specific components and set their maximum values. For example, it is also possible to limit the total code amount of each color component by monitoring the total code amount of each component in image data composed of RGB three components.

  As described above, by setting the code amount of the specific component to be equal to or less than the maximum value, decoding can be performed even with a small decoder having few resources, leading to cost reduction of the decoder. In addition, when the code amount of a specific component is limited by a method of performing a convergence operation that searches for a gradient λ that satisfies the target total code amount by performing a convergence operation after encoding all the encoding passes shown as the conventional technology, If the code amount of the specific component is less than the minimum value, a convergence operation for searching for an appropriate λ is required again. On the other hand, in the first embodiment, it is only necessary to monitor the code amount of a specific component and determine whether or not to continue the encoding of each encoding pass. It can be said that it is effective.

Embodiment 2. FIG.
In the first embodiment described above, the method of limiting the code amount of the specific component to be equal to or less than the specified maximum value has been described. However, in the second embodiment, the minimum value in which the code amount of the specific component is specified. A method of restricting to the above will be described.

  Since the configuration of the image encoding apparatus according to Embodiment 2 is the same as that according to Embodiment 1 shown in FIG. 1, the description thereof is omitted, and rate control information extraction means 105 and entropy encoding means 103 in FIG. Details of the process will be described. In this method, a plurality of rate control parameter λ candidates are prepared in advance, and encoding is performed for all code blocks up to an encoding pass that satisfies a certain λ. At that time, it is determined whether or not the code amount has reached the target amount. If it has reached, the encoding is completed. If not, the next candidate for λ is set and all code blocks satisfy that λ. Encoding is performed again until

  Here, after the total code amount reaches the target value, it is determined whether or not the total code amount of the specific component exceeds a predetermined minimum value. If the total code amount exceeds the minimum value, the encoding ends. If it is less than the minimum value, only the code block belonging to the specific component is executed until the code amount exceeds the minimum value. The code amount allocation to the specific component ends at that stage.

Next, for the code block other than the specific component, the value of λ is set in the reverse order to the previous one (so that λ becomes small), and the coding pass having a slope exceeding λ ⁻¹ is encoded. The code data of the block. The process of sequentially lowering λ is performed until the total code amount falls below the target total code amount.

Hereinafter, a method for determining a path to be encoded will be described with reference to the flowcharts shown in FIGS. 9 to 11 which are mainly the operation contents of the encoded path deriving unit 115 in the rate control information extracting unit 105. The rate control parameter candidate λ (t) is set in advance as follows.
λ (t) = {λ (0), λ (1), λ (2),..., λ (tmax)}
(Λ (t) <λ (t + 1))

  The initial value of the rate control parameter index t is t = 0 (t = 0 to tmax), the code block index i = 0 (i = 0 to imax), the total code amount counter Rsum = 0, the specific component The total code amount counter Psum = 0, the variable k (i) for storing the coding pass in each code block is all −1 for the code block (the index of the next pass skipping the zero bit plane is 0, k (i) = −1 to kmax, and the initial value is k (i) = − 1 for the convenience of the counter).

  Although the memory is not shown in the block diagram, k (i) is a variable stored for each code block. The index t of the rate control parameter, the index i of the code block, and the counters Rsum and Psum of the total code amount are Variables common to all code blocks.

  In addition, an update flag indicating whether or not the encoding is further advanced in a certain code block is Cflag (i) = 1 (i = 0 to imax). It is assumed that the target total code amount Rmax and the specific component code amount minimum value Pmin are specified in advance and stored in the coding path deriving unit 115.

  First, in step 902, the encoding pass to be encoded is set to k (i), and in step 903, the encoding pass k (i) is encoded. After encoding, the difference ΔD between the encoding pass calculated by the distortion calculation unit 111 and the previous previous encoding pass ΔD, the code amount ΔR in the encoding pass calculated by the code amount calculation unit 112, etc. Using the relationship between the rate and distortion stored in the rate distortion memory 113, the slope S (i, k (i)) of the path k (i) is calculated in step 904.

  In step 905, the total code amount Psum of the specific component is calculated. Although not shown in the figure, information for determining whether or not each code block belongs to a specific component is included in the coding path deriving means 115, and each time a code block of a specific component is encoded, The total code amount of the specific component is calculated by adding the code amount ΔR of the encoding pass to be encoded to the total code amount Psum of the specific component by the conversion path deriving unit 115. In step 906, the coding pass deriving unit 115 adds the code amount ΔR of the coding pass to be coded to the total code amount Rsum, thereby calculating the total code amount of all components.

Next, in step 907, the total code amount Rsum is compared with the total code amount target value Rmax. If the total code amount does not reach the target total code amount, in step 911, the magnitude of the slope S (i, k) and λ (t) ⁻¹ in the current coding pass is determined, and the slope S is large. For example, the next pass is encoded (step 911 → step 902). If the slope S is less than λ (t) ⁻¹ , the encoded pass code data is temporarily stored in the memory, the encoding of this code block is interrupted, and the code block index i is incremented. The processing shifts to encoding of the next code block (step 911 → step 912 → step 913 → step 902). Although not shown in the flowchart, even when the coding pass is the final coding pass of the code block, the process proceeds to the next code block.

In the next code block as well, encoding is performed until the slope becomes less than λ (t) ⁻¹ . After this is performed for all code blocks, λ index t is incremented, λ is set as the next candidate, and encoding of all code blocks is performed again until the slope becomes less than λ (t) ^−1. Perform (Step 911 → Step 912 → Step 914 → Step 902). In step 914, even if λ (t) is updated, S (i, k (i)) <λ (t) ⁻¹ , that is, the updated λ is already encoded. It may be larger than the slope. In that case, although not shown in the flowchart, the process jumps to step 912 to pass the encoding of the block, and proceeds to the encoding of the next code block.

  Next, if it is determined in step 907 that Rsum> Rmax, in order to exclude the final coding pass of the current code block in step 908, the index of the final coding pass is set to k (i) -1, and the total code The code amount ΔR of the final coding pass is subtracted from the amount counter. In step 909, the total code amount Psum of the specific component is compared with the minimum value Pmin, and if the total code amount Psum of the specific component is equal to or greater than the minimum value Pmin, the encoding ends.

  If the total code amount Psum of the specific component is less than the minimum value Pmin, encoding is continued until the total code amount Psum exceeds the minimum value Pmin only for code blocks belonging to the specific component. In step 910, the index i of the current code block and the index t of the rate control parameter λ are temporarily stored in isstop and tstop, then the process proceeds to step 1001 in FIG. 10, the update flag of the code block of the specific component is set to 1, and others The update flag of the code block is set to 0. The following processing from step 1002 to step 1012 is substantially the same as the processing from step 902 to step 914 except that only the code block of the specific component is the object to be encoded, so detailed description will be omitted. If it is determined in step 1008 that the total code amount Psum of the specific component has exceeded the minimum value Pmin during encoding, the loop consisting of step 1002 to step 1012 is exited. As a result, for the specific component, the coding pass output as the code stream is determined.

  In the next stage, the coding pass is truncated until the total code amount reaches the target value in the code blocks other than the specific component. In order to return the rate control parameter and the index of the code block to be encoded to the stage where the encoding of the code block of the specific component is stopped, the index of the rate control parameter λ and the index of the code block stored in step 1013 are set to t, 11, the process proceeds to FIG. 11. In step 1101, the update flag of the code block of the specific component is set to 0, and other than the specific component is set to 1. From here, the coding pass is rounded down in the scanning order opposite to when the coding is advanced until the total code amount Rsum of all components becomes equal to or less than the target value Rmax.

  First, in step 1102, an update flag indicating whether or not encoding is further advanced in a certain code block is set to Cflag (i) = 1 (i = 0 to imax). In step 1103, one coding pass is deleted, and in step 1104, the gradient is read from the rate distortion memory 113 (encoding is already completed, so it is not necessary). In step 1105, the total code amount Rsum of all components at that time is calculated. In step 1106, the total code amount Rsum is compared with the target value Rmax. If the total code amount Rsum is equal to or less than the target value Rmax, the code is encoded at this time. The process of determining the conversion path ends. Information k (i) indicating which path is output as a code stream in each code block is sent to the code data extraction means 106.

If the total code amount Rsum is larger than the target value Rmax, in step 1107, the magnitudes of the slopes S (i, k) and λ (t) ⁻¹ in the current coding pass are compared. Returning to 1103, the next pass is further encoded. If the slope S becomes equal to or ^larger than λ (t) ⁻¹ , the code block index i is decremented through steps 1108 and 1109, the process returns to step 1102, and the processing is moved to the next code block. Although not shown in the flowchart, even when the coding pass is the first coding pass of the code block, the processing of the next code block is assumed. Similarly, the next code block is processed until the inclination becomes λ (t) ⁻¹ or more. After this is performed for all code blocks, in step 1110, the index t of the rate control parameter λ is decremented to increase the slope threshold, λ is set as the next candidate, and the code other than the specific component is again set. Scan the block.

  Next, how the rate distribution is determined by changing λ will be described using the example of FIG. In this example, image data composed of RGB three components is encoded with the total code amount target value Rmax = 100. Assume that the total code amounts of the R, G, and B components are RO, R1, and R2, respectively, and the minimum value Pmin of the R component is 30.

Initially starting from lambda ^-1 = 100, continued coding until the slope becomes S <λ ^-1 = 100 in each code block. When this was executed for all code blocks, the total code amount Rsum = 85. Yet, because Rsum is less than the target total code amount, the lambda as a step down lambda ^-1 = 90, again, it continued coding until the slope becomes S <λ ^-1 = 90 in each code block. When this was executed for all code blocks, the total code amount Rsum = 92.

Since Rsum is still less than the target total code amount, when λ is reduced by one step and λ ⁻¹ = 80, the total code amount becomes the maximum when the target value is 100 or less at the stage where a certain encoding pass is encoded 99. It became. However, since the code amount of the specific component is less than the minimum value, encoding is continued only for code blocks belonging to the specific component. As λ ^-1 = 80, the code amount of the specific component at the stage of encoding the entire coding blocks of a particular component became 22. In the table, the values that do not change because the update is stopped are shaded.

Since the code amount of the specific component is less than the minimum value of 30, the encoding is continued with λ lowered by one step and λ ⁻¹ = 70. When this was executed for all code blocks of the specific component, the total code amount Psum of the specific component was 25. Since Psum is less than the minimum value 30, when λ is further lowered by one step and λ ⁻¹ = 80, the total code amount of the specific component exceeds the minimum value 31 on the way. Exit.

Thereafter, a truncation encoding pass other than the specific component is determined. When a coding block other than the specific component with a slope of less than λ ⁻¹ = 80 was discarded, the total code amount was 108. Since it is larger than the total code amount target value 100, when λ is increased by one step and λ ⁻¹ = 90 and the encoding pass is truncated, the total code amount is 104. Since it is still larger than the total code amount target value 100, when λ is increased by one step and λ ⁻¹ = 100 and the encoding pass is discarded, the total code amount falls below the target value 100 on the way. Exit. With the above processing, the encoding pass to be output as code data in all code blocks is determined, the code amount allocation to each of the R, G, and B components is 31, 34, and 34, respectively, and the total code amount is 99. .

  In the second embodiment, the code amount of the specific component is limited so that the specific component code amount is equal to or larger than the minimum code amount. However, the code amount is not necessarily limited, and for example, the coding distortion of the specific component is monitored. Thus, it is possible to limit the coding distortion of a specific component to a specified value or more. In the second embodiment, one type of specific component is used, but it is also possible to set a plurality of specific components and set their minimum values. For example, it is also possible to limit the total code amount of each color component by monitoring the total code amount of each component in image data composed of RGB three components.

  In the second embodiment, the code amount of the specific component is limited so that the specific component code amount is equal to or greater than the minimum code amount. However, the specific component code amount described in the first embodiment is less than or equal to the maximum code amount. It is also possible to control so that the code amount of the specific component is not less than the minimum code amount and not more than the maximum code amount in combination with the above method. For this purpose, it is determined whether or not the code amount of the specific component exceeds the maximum value between steps 906 and 907 in FIG. 9 as in the process of step 707 in FIG. Processing similar to that in step 708 and step 709 in FIG. 7 is performed, and thereafter, the code block belonging to the specific component may not be encoded.

  In the method of simply minimizing the mean square error of the entire screen shown as the conventional technique, the code amount distribution of the specific component may become extremely small in some cases, and the subjective image quality may be deteriorated. As described above, if it is ensured that the code amount of the specific component is equal to or larger than the minimum value, a situation in which the image quality of the specific component is extremely deteriorated can be avoided, and the image quality is improved.

  In addition, when the coding amount of a specific component is limited by the method of performing the convergence calculation that searches for the gradient λ that satisfies the target total code amount by performing the convergence calculation after encoding all the encoding passes shown as the conventional technology If the code amount of the specific component is less than the minimum value, a convergence operation for searching for an appropriate λ is required again. On the other hand, in the second embodiment, it is only necessary to monitor the code amount of a specific component and determine whether or not to continue the encoding of each encoding pass. It can be said that it is effective.

It is a block diagram which shows the structure of the image coding apparatus which concerns on Embodiment 1 of this invention. It is a block diagram which shows the structure of the rate control information extraction means 105 shown by FIG. It is a figure which shows a subband when wavelet transform is carried out to the decomposition level 2 by the wavelet transform means 101 shown in FIG. It is a figure explaining decomposition | disassembly of the bit plane by the entropy encoding means 103 shown by FIG. 1, (a) represents an example of a 4x4 code block, (b) represents the data shown to (a). , A 1-bit signal representing positive and negative values and an absolute value representation, and the result of binary representation of the data in the vertical direction is arranged in units of rows, and (c) represents the data shown in (b). Examples of collecting the same bit number are shown below. It is a figure explaining decomposition | disassembly from the bit plane to an encoding pass by the entropy encoding means 103 shown by FIG. It is a figure which shows the data stored in the rate distortion memory 113 shown by FIG. It is a flowchart which shows the process of the image coding apparatus in Embodiment 1 of this invention. It is a figure explaining the order of the encoding in Embodiment 1 of this invention. It is a flowchart which shows the process of the image coding apparatus in Embodiment 2 of this invention. 10 is a flowchart illustrating processing subsequent to FIG. 9. It is a flowchart which shows the process following FIG. It is a figure explaining the order of the encoding in Embodiment 2 of this invention.

Explanation of symbols

  101 wavelet transform means, 102 quantization means, 103 entropy coding means, 104 code memory, 105 rate control information extraction means, 106 code data extraction means, 111 distortion calculation means, 112 code amount calculation means, 113 rate distortion memory, 114 Inclination calculation means, 115 Coding path derivation means.

Claims (4)

A two-dimensional signal is divided into a plurality of code blocks, the code block is decomposed into bit planes, each bit plane is divided into one or more coding passes, and coding is performed for each coding pass based on control information. Entropy encoding means for outputting data;
A code memory for storing code data from the entropy encoding means;
Rate control information extraction means for determining the continuation and termination of the coding pass based on the calculation of distortion and code amount necessary for rate control;
Code data extraction means for extracting code data up to an encoding end pass from the code memory based on control information from the rate control information extraction means and outputting it as a code stream;
The rate control information extraction means includes:
Distortion calculating means for calculating distortion caused by encoding of each encoding pass;
Code amount calculation means for calculating the code amount of each coding pass;
An inclination calculating means for calculating an inclination based on the relationship between the distortion and the code amount in each code block from the distortion from the distortion calculating means and the code amount from the code amount calculating means;
Based on the slope, the total code amount of all the components, and the total code amount of the specific component, each encoding is performed so that the total code amount of all the components is less than the target code amount and the total code amount of the specific component is less than a predetermined maximum value. An image encoding apparatus comprising: an encoding path deriving unit that outputs control information for controlling the entropy encoding unit and the code data extracting unit. The image encoding device according to claim 1,
The encoding path derivation means includes:
Encoding a coding pass in which the slope exceeds a predetermined slope threshold;
When the total code amount of all the components does not reach the target code amount, the inclination threshold is decreased, and a process of further encoding a coding path that has not been encoded before is performed. Control repeatedly until the total code amount reaches the target code amount,
When the total code amount of the specific component specified in the middle of encoding exceeds the maximum value, the encoding is controlled so that the code block belonging to the specific component is not encoded thereafter. An image encoding device. The image encoding device according to claim 1,
The encoding path derivation means includes:
Encoding a coding pass in which the slope exceeds a predetermined slope threshold;
When the total code amount of all the components does not reach the target code amount, the inclination threshold is decreased, and a process of further encoding a coding path that has not been encoded before is performed. Control repeatedly until the total code amount reaches the target code amount,
When the total code amount of all the components reaches the target code amount and the total code amount of the specific component is less than a predetermined minimum value, encoding of only the code block belonging to the specific component is performed, Control the encoding to continue until the minimum value is exceeded,
After that, only for code blocks other than the specific component, the process of increasing the gradient threshold and deleting the coding pass that does not satisfy the gradient threshold from the code stream is such that the total code amount of all the components is less than the target code amount And an encoding pass when the total code amount of all the components falls below the target code amount is determined as a final output pass. The image encoding device according to claim 1,
The encoding path derivation means includes:
Encoding a coding pass in which the slope exceeds a predetermined slope threshold;
When the total code amount of all the components does not reach the target code amount, the inclination threshold is decreased, and a process of further encoding a coding path that has not been encoded before is performed. Control repeatedly until the total code amount reaches the target code amount,
When the total code amount of the specific component specified during the encoding exceeds the maximum value, the encoding is controlled so as not to code the code block belonging to the specific component thereafter.
When the total code amount of all the components reaches the target code amount and the total code amount of the specific component is less than a predetermined minimum value, encoding of only the code block belonging to the specific component is performed, Control the encoding to continue until the minimum value is exceeded,
After that, only for code blocks other than the specific component, the process of increasing the gradient threshold and deleting the coding pass that does not satisfy the gradient threshold from the code stream is such that the total code amount of all the components is less than the target code amount And an encoding pass when the total code amount of all the components falls below the target code amount is determined as a final output pass.

Images