JP2006303670A - Image encoder and encoding program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image encoder and an image encoding program in which degradation in decoding image quality of ROI can be prevented by creating a code stream of image encoding data where the decoding image quality of non-ROI can be sustained at a constant level, and the profile of ROI can be recognized accurately on the decoding side. <P>SOLUTION: When the code block of processing object belongs to low frequency component or a subband close to low frequency component, information of encoding path concerning the coefficients (corresponding to non-ROI) in that code block other than the coefficients corresponding to ROI is stored partially or entirely in an upper layer set to store the information of ROI. Consequently, a code stream of image encoding data in which the decoding image quality of non-ROI can be sustained at a constant level, and the profile of ROI can be recognized accurately on the decoding side can be created. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention can maintain the decoding image quality of a non-region of interest (non-ROI) at a certain level, and accurately recognizes the shape of the region of interest (ROI) on the decoding side. The present invention relates to an image encoding apparatus and an image encoding program that generate a code stream of encoded image data based on the JPEG2000 system and prevent deterioration of decoded image quality of ROI.

  An image contains a large amount of information, and when storing and transmitting the image information, the enormous amount of data that the image information has becomes a problem. Therefore, when storing and transmitting image information, a high-efficiency code that reduces (compresses) the data of the image information by adding an operation that excludes signal redundancy and visual redundancy of the image information. Is used.

  As a method for performing such encoding, there is a JPEG2000 method standardized by ISO (International Organization for Standardization) that performs orthogonal transformation on an input image and entropy-encodes the orthogonally transformed coefficient for each bit plane.

  The JPEG2000 system has an ROI function that preferentially encodes a region of interest (ROI) set in an image. When the ROI function is used, the ROI is preferentially encoded. Therefore, depending on the set code amount, the decoding image quality of a non-region of interest (non-ROI: Non-Region Of Interest) is extremely deteriorated. there were.

On the other hand, in the invention described in Japanese Patent Application Laid-Open No. 2003-174645, the coefficient corresponding to the non-ROI is designated as the ROI in the coefficient of the low frequency component or the component close to the low frequency component of the wavelet transformed coefficient. A method for maintaining the image quality at a certain level is disclosed.
JP 2003-174645 A

  However, designating all coefficients of low frequency components or components close to low frequency components as ROIs means that the decoding side cannot correctly recognize the shape of the ROI designated at the time of encoding, and depending on the set code amount Causes degradation of the image quality of the ROI.

  An object of the present invention is to generate a codestream of image encoded data based on the JPEG2000 method, which can maintain the non-ROI decoded image quality at a certain level and can accurately recognize the ROI shape on the decoding side. It is another object of the present invention to provide an image encoding device and an image encoding program that prevent deterioration of the decoded image quality of ROI.

In order to solve the above problems, the present invention provides the following apparatus and program.
(1) Orthogonal transformation means for performing orthogonal transformation that decomposes image data in which a region of interest is designated into sub-bands of low-frequency components and high-frequency components and converts them into coefficients;
Shift-up means for inputting the coefficient from the orthogonal transform means, and performing shift-up only for a coefficient corresponding to the region of interest among the coefficients;
Using the coefficient bit modeling process for each code block that is a coding unit, the coefficient output from the shift-up means is binarized by a predetermined coding pass, entropy-coded, and coded Entropy encoding means for obtaining encoded data for each block;
When decoding sequentially from encoded data stored in the upper layer, the layer structure is set so that the image quality of the decoded image is improved in stages as the encoded data stored in the lower layer is decoded. Code stream generation means for storing encoded data for each code block generated by the entropy encoding means in each layer to obtain a code stream having the layer structure;
With
The code stream generation means includes
The encoded data related to the coefficient corresponding to the region of interest among the encoded data of the code block including the coefficient corresponding to the region of interest is stored in an upper layer designated in advance as a layer for the region of interest,
A code block including a coefficient corresponding to the region of interest and a coefficient other than a coefficient corresponding to the region of interest among encoded data in a code block included in the subband having a low frequency component equal to or lower than a predetermined frequency. Storing at least a part of the encoded data in an upper layer designated in advance as a layer for the region of interest;
The code block is a code block that does not include a coefficient corresponding to the region of interest, and at least a part of the encoded data of the code block included in the subband of the low frequency component equal to or lower than the predetermined frequency Is stored in the upper layer designated in advance as
Coded data of a code block that is a code block that does not include a coefficient corresponding to the region of interest and that is included in the subband having a frequency component higher than the predetermined frequency is designated in advance as a layer for the region of interest. Means for obtaining the codestream by storing in a lower layer than the upper layer.
An image encoding apparatus characterized by that.
(2) Orthogonal transformation means for performing orthogonal transformation that decomposes image data in which a region of interest is specified into sub-bands of low-frequency components and high-frequency components and converts them into coefficients.
Shift-up means for inputting the coefficient from the orthogonal transform means, and performing shift-up only for a coefficient corresponding to the region of interest among the coefficients;
Using the coefficient bit modeling process for each code block that is a coding unit, the coefficient output from the shift-up means is binarized by a predetermined coding pass, entropy-coded, and coded Entropy encoding means for obtaining encoded data for each block;
When decoding sequentially from encoded data stored in the upper layer, the layer structure is set so that the image quality of the decoded image is improved in stages as the encoded data stored in the lower layer is decoded. Code stream generation means for storing encoded data for each code block generated by the entropy encoding means in each layer to obtain a code stream having the layer structure;
Is an image encoding program for causing a computer to realize
The code stream generation means includes
The encoded data related to the coefficient corresponding to the region of interest among the encoded data of the code block including the coefficient corresponding to the region of interest is stored in an upper layer designated in advance as a layer for the region of interest,
A code block including a coefficient corresponding to the region of interest and a coefficient other than a coefficient corresponding to the region of interest among encoded data in a code block included in the subband having a low frequency component equal to or lower than a predetermined frequency. Storing at least a part of the encoded data in an upper layer designated in advance as a layer for the region of interest;
The code block is a code block that does not include a coefficient corresponding to the region of interest, and at least part of the encoded data of the code block included in the subband of the low frequency component equal to or lower than the predetermined frequency is a layer for the region of interest. Is stored in the upper layer designated in advance as
Coded data of a code block that is a code block that does not include a coefficient corresponding to the region of interest and that is included in the subband having a frequency component higher than the predetermined frequency is designated in advance as a layer for the region of interest. Means for obtaining the codestream by storing in a lower layer than the upper layer,
As an image encoding program realized on a computer.

  According to the present invention, it is possible to generate a code stream of image encoded data based on the JPEG2000 method that can maintain the non-ROI decoded image quality at a certain level and can accurately recognize the ROI shape on the decoding side. Further, it is possible to prevent degradation of the decoded image quality of ROI.

[Outline of JPEG2000 algorithm]
First, an outline of the JPEG2000 encoding algorithm, which is a prerequisite technology of the embodiment of the present invention, will be described.

  FIG. 1 is a diagram showing an overall configuration of an embodiment of an image encoding apparatus according to the present invention. A basic JPEG2000 encoding process will be described with reference to FIG. The JPEG2000 algorithm includes a DC level shift unit 101, a wavelet transform unit (orthogonal transform processing unit) 102, a quantization unit 103, an entropy encoding unit 104, and a stream generation unit 105. However, the DC level shift unit 101 and the quantization 103 are optional and are not necessarily required processing units.

  The input image is DC level shifted by the DC level shift unit 101. The DC level shift is a process of subtracting a central value from a pixel value in a possible range of the pixel value in the input image. For example, when the possible range of pixel values is 0 to 255, a process of subtracting 128, which is the central value, from all pixel values is performed.

  The wavelet transform unit (orthogonal transform processing unit) 102 performs wavelet transform on the image subjected to the DC level shift. The wavelet transform performs a process of dividing an input image into low frequency components and high frequency component subbands, and JPEG 2000 employs octave decomposition that recursively performs wavelet transform on low frequency components.

  FIG. 2 shows an example of wavelet transform in the JPEG2000 system. When the original image 201 is wavelet transformed, a coefficient 202 of decomposition level 1 (decomposition level 1) is obtained. The wavelet transform can be further performed on the 1LL subband which is the low frequency component subband in the coefficient 202 of the decomposition level 1. When the 1LL subband is wavelet transformed, a coefficient 203 of decomposition level 2 (decomposition level 2) is obtained. Further, when the wavelet transform is performed on the 2LL subband which is a low frequency component subband, a coefficient 204 at the decomposition level 3 (decomposition level 3) is obtained.

  The quantization unit 103 performs quantization for each subband in the wavelet-transformed coefficient. Here, when the ROI is specified for the input image, the coefficient corresponding to the ROI is shifted up by the MAX SHIFT method.

  The quantized coefficients are subjected to coefficient bit modeling processing for each code block by the entropy encoding unit 104. As shown in FIG. 3, the code block is a block of a unit for performing entropy coding in each subband. The coefficient bit modeling process is a process of performing a binarization process by an encoding pass and arithmetically encoding a binarized bit string to obtain encoded data. In the encoding pass, three encoding passes (Significant Propagation Pass, Magnitude Refinement Pass, and Cleanup Pass) are defined in JPEG2000, and a coefficient is scanned and binarized according to a predetermined rule.

  The encoded data of each code block subjected to entropy encoding is rate-controlled by the code stream generation unit 105, and further, a header is added and output as a code stream.

Here, the layer structure of the code stream will be described. As shown in FIG. 4, the layer structure determines how the code block information in each code block is stored. In FIG. 4, the higher the layer from layer 5 to layer 1, the higher the layer. When the information amount of the code block 1 stored in the layer 1 is compared with the information amount of the code block 2, it can be seen that the information amount of the code block 2 is larger. In addition, it can be seen that information on the code block 4 is not stored in the layer 1. In this way, the coding apparatus can arbitrarily determine the layer structure. Here, the decoding side sequentially decodes the encoded data stored in the upper layer, and as the encoded data stored in the lower layer is decoded, the image quality of the decoded image as a whole improves stepwise. The layer structure is set. In general, as the number of layers to be set increases, the layer structure can control the image quality of the decoded image in more stages (the image quality of the decoded image can be improved sequentially over more stages). . (Which lower layer is decoded depends on the set code amount.)
[Example 1]
FIG. It is a block diagram which shows the internal structure of the code stream production | generation part of one Example of the image coding apparatus of this invention. FIG. 6 shows in detail the codestream generation unit in FIG.

  The number of coding passes and coded data output from the entropy coding unit 104 shown in FIG. 1, information indicating whether the code block is a ROI target and a MAX SHIFT coefficient (ROI information), and a code block The sub-band information (sub-band information) is input to the ROI determination unit 601.

  Based on the ROI information and the subband information, the ROI determination unit 601 determines in which layer the coding path information of the code block that is the target of the code stream generation processing is stored. Here, the coding pass information of the code block indicates the number of coding passes processed by the coding pass of coefficient bit modeling and the corresponding coded data.

  Based on the ROI information and the subband information, the ROI determination unit 601 does not include a coefficient corresponding to the ROI based on the ROI information and the subband information, and the code block is low frequency component or close to low frequency component Assume that it is determined that it belongs to a subband (that is, the code block belongs to a subband having a low frequency component equal to or lower than a predetermined frequency). In this case, at least a part of the coding pass information of the code block is stored in any one of the upper layers set so as to store the coding pass information regarding the coefficient corresponding to the ROI. Information is created and output to the layer structure creation unit 602. The predetermined frequency is set according to the size of the input image.

  Based on the storage layer designation information obtained from the ROI determination unit 601, the layer structure creation unit 602 stores part or all of the coding path information of the processing target code block in the designated upper layer. Depending on the amount of information that can be stored in the upper layer, it is determined whether to store part or all of the information of the coding path.

  Assume that the ROI determination unit 601 determines that the code block to be processed is a code block including a coefficient corresponding to the ROI based on the ROI information. In this case, the layer structure creation unit 602 is configured to store the information on the coding pass regarding the coefficient corresponding to the ROI in the processing target code block and the information on the coding pass regarding the coefficient corresponding to the ROI. Stored in a higher layer.

  Based on the ROI information and the subband information, the ROI determination unit 601 includes a code block to be processed that includes a coefficient corresponding to the ROI, and the code block is a low frequency component or a sub frequency component close to a low frequency component. Assume that it is determined that it belongs to a band (that is, the code block belongs to a subband of a low frequency component equal to or lower than a predetermined frequency). In this case, in the layer structure creation unit 602, at least a part of the coding pass information related to the coefficient other than the coefficient corresponding to the ROI in the code block to be processed (coefficient corresponding to the non-ROI) is related to the coefficient corresponding to the ROI. The information is stored in an upper layer that is set to store information on the coding pass. The coding pass information related to the coefficient corresponding to the ROI in the processing target code block is naturally stored in an upper layer that is set to store the coding pass information related to the coefficient corresponding to the ROI.

  Based on the ROI information and the subband information, the ROI determination unit 601 does not include a coefficient corresponding to the ROI based on the ROI information and the subband information, and the code block has a frequency component sub-frequency higher than the predetermined frequency. Suppose that it is determined that the band belongs. In this case, in the layer structure creation unit 602, the coding path information of the processing target code block is lower than the upper layer set so that the coding path information related to the coefficient corresponding to the ROI is stored. Store in the layer. This is the same as the processing for the code block not including the coefficient corresponding to the conventional ROI.

Specific examples are shown below.
FIG. 7 shows the layer structure when the ROI is designated. In FIG. 7, code blocks 1 and 2 are code blocks including only coefficients corresponding to ROI, and other code blocks are not including coefficients corresponding to ROI. In addition, layers 1 and 2 are assumed to be higher layers that are set so as to store information on coding passes relating to coefficients corresponding to ROI.

  It can be seen that the information of the code blocks 1 and 2 having only the coefficient corresponding to the ROI is stored in the upper layers 1 and 2 and the other code blocks are in the lower order of the information. FIG. 7 is configured such that the image quality of the decoded image can be controlled in more stages as the number of layers to be set increases (the image quality of the decoded image can be sequentially improved over more stages). That is, it is a layer structure created by a conventional JPEG2000 encoding apparatus.

  FIG. 8 shows a layer structure when this embodiment is applied. As in FIG. 7, code blocks 1 and 2 are coat blocks having only coefficients corresponding to ROI, and the other code blocks have no coefficient corresponding to ROI. Further, the code block 3 is a code block belonging to a subband of a low frequency component, and the code block 4 is a code block belonging to a subband close to the low frequency component (that is, the code blocks 3 and 4 are low frequency components having a predetermined frequency or less). Belongs to the subband). Also, let layers 1 and 2 be higher layers that are set to store information on coding passes related to coefficients corresponding to ROI.

  Since each of the code blocks 3 and 4 does not have a coefficient corresponding to the ROI and is a low-frequency component or a sub-band code block close to the low-frequency component, the ROI determination unit 601 generates information to be stored in the upper layer. Yes. Therefore, a part of the information on the coding pass of the code blocks 3 and 4 is stored in the upper layers of the layers 1 and 2. In addition, a part of the information of the coding pass of the code blocks 1 and 2 has moved to the layer 3, which means that if the information amount up to the layer 2 is fixed, the code block 3 Since the information of 4 is added, the information does not fit in the information amount up to the layer 2. However, when the amount of information up to layer 2 is not fixed, the information of the coding passes of code blocks 1 and 2 may be stored up to layer 2. Further, the coding path information of the code blocks 3 and 4 may be stored in the layer 2 and lower layers without being stored in the layer 1. You may store in the layer below from layer 1 and layer 3. In short, at least part of the coding path information of the code blocks 3 and 4 is stored in at least one of the upper layers 1 and 2 that are set to store the coding path information related to the coefficient corresponding to the ROI. It is sufficient that it is stored as a minimum.

  Next, the layer structure creation processing of this embodiment will be described with reference to FIG. The entropy encoding unit 104 performs coefficient bit modeling, inputs information on the encoding pass of the processed code block (the number of encoding passes and the encoded data), and sequentially performs processing for each code block.

  Information on the target code block (encoding pass information, information on whether the code block includes encoded data of coefficients corresponding to ROI (ROI information), information on which subband the code block belongs to (ROI information) Subband information)) is extracted (step 501).

  Whether the target code block includes a coefficient corresponding to ROI, or whether the target code block belongs to a low frequency component or a subband close to a low frequency component (that is, the code block has a predetermined frequency) (Belonging to the following low frequency component subbands) (step 502). If the target code block includes a coefficient corresponding to the ROI, or if the target code block belongs to a low-frequency component or a subband close to the low-frequency component, part of the information on the coding path of the target code block Alternatively, all of them are stored in an upper layer in which ROI information is stored (step 503).

  That is, in a code block that includes a coefficient corresponding to the ROI or is determined to belong to a low frequency component or a subband close to the low frequency component, a coding pass related to a coefficient corresponding to the ROI in the code block. Is stored in an upper layer that is set to store information on the coding pass related to the coefficient corresponding to the ROI. Store at least a part of coding path information related to coefficients other than the coefficient corresponding to the ROI in the code block in an upper layer that is set to store coding path information related to the coefficient corresponding to the ROI. To do.

  On the other hand, when it is determined No in step 502, that is, based on the ROI information and the subband information, the target code block does not include a coefficient corresponding to the ROI, and the code block has the predetermined frequency. Assume that it is determined that a higher frequency component subband belongs. In this case, as in a general encoding method, information on the encoding pass of the target code block is stored in each designated layer (information is stored in a layer depending on image quality: step 504). Here, the coding pass information of the target code block is stored in a lower layer than the upper layer that is set to store the coding pass information regarding the coefficient corresponding to the ROI. This is the same as the processing for the code block not including the coefficient corresponding to the conventional ROI.

  It is determined whether or not all code blocks have been processed (step 505). If all code blocks have been processed, the layer structure creation processing is terminated, and if not, the process returns to step 501.

  The present invention includes a program for causing a computer to realize the functions of the above-described image encoding apparatus. This program may be read from a recording medium and loaded into a computer, or may be transmitted via a communication network and loaded into a computer.

It is a block diagram which shows the whole structure of one Example of the image coding apparatus of this invention. It is explanatory drawing which shows the subband in each decomposition level when a decomposition level is 3. FIG. It is explanatory drawing which shows the relationship between a subband and a code block. It is explanatory drawing which shows a layer structure. It is a flowchart of layer structure creation in one Example. It is a block diagram which shows the internal structure of the code stream production | generation part in one Example. It is explanatory drawing which shows a layer structure when using a ROI function. It is explanatory drawing which shows a layer structure when one Example is applied using a ROI function.

Explanation of symbols

DESCRIPTION OF SYMBOLS 101 DC level shift part 102 Wavelet transformation part 103 Quantization part 104 Entropy encoding part 105 Code stream generation part

Claims (2)

Orthogonal transform means for performing orthogonal transform that decomposes image data in which a region of interest is specified into sub-bands of low-frequency components and high-frequency components and converts them into coefficients,
Shift-up means for inputting the coefficient from the orthogonal transform means, and performing shift-up only for a coefficient corresponding to the region of interest among the coefficients;
Using the coefficient bit modeling process for each code block that is a coding unit, the coefficient output from the shift-up means is binarized by a predetermined coding pass, entropy-coded, and coded Entropy encoding means for obtaining encoded data for each block;
When decoding sequentially from encoded data stored in the upper layer, the layer structure is set so that the image quality of the decoded image is improved step by step as the encoded data stored in the lower layer is decoded. Code stream generation means for storing the encoded data for each code block generated by the entropy encoding means in each layer to obtain a code stream having the layer structure;
With
The code stream generation means includes
The encoded data related to the coefficient corresponding to the region of interest among the encoded data of the code block including the coefficient corresponding to the region of interest is stored in an upper layer designated in advance as a layer for the region of interest,
A code block including a coefficient corresponding to the region of interest and a coefficient other than a coefficient corresponding to the region of interest among encoded data in a code block included in the subband having a low frequency component equal to or lower than a predetermined frequency. Storing at least a part of the encoded data in an upper layer preliminarily designated as a layer for the region of interest;
The code block is a code block that does not include a coefficient corresponding to the region of interest, and at least part of the encoded data of the code block included in the subband of the low frequency component equal to or lower than the predetermined frequency is a layer for the region of interest. Is stored in the upper layer designated in advance as
Coded data of a code block that is a code block that does not include a coefficient corresponding to the region of interest and that is included in the subband having a frequency component higher than the predetermined frequency is designated in advance as a layer for the region of interest. Means for obtaining the codestream by storing in a lower layer than the upper layer.
An image encoding apparatus characterized by that. Orthogonal transformation means for performing orthogonal transformation that decomposes the image data in which the region of interest is designated into sub-bands of low frequency components and high frequency components and converts them into coefficients,
Shift-up means that receives the coefficient from the orthogonal transform means, and shifts up only the coefficient corresponding to the region of interest among the coefficients;
Using the coefficient bit modeling process for each code block that is a coding unit, the coefficient output from the shift-up means is binarized by a predetermined coding pass, entropy-coded, and coded Entropy encoding means for obtaining encoded data for each block;
When decoding sequentially from encoded data stored in the upper layer, the layer structure is set so that the image quality of the decoded image is improved step by step as the encoded data stored in the lower layer is decoded. Code stream generation means for storing encoded data for each code block generated by the entropy encoding means in each layer to obtain a code stream having the layer structure;
Is an image encoding program for causing a computer to realize
The code stream generation means includes
The encoded data related to the coefficient corresponding to the region of interest among the encoded data of the code block including the coefficient corresponding to the region of interest is stored in an upper layer designated in advance as a layer for the region of interest,
A code block including a coefficient corresponding to the region of interest and a coefficient other than a coefficient corresponding to the region of interest among encoded data in a code block included in the subband having a low frequency component equal to or lower than a predetermined frequency. Storing at least a part of the encoded data in an upper layer designated in advance as a layer for the region of interest;
The code block is a code block that does not include a coefficient corresponding to the region of interest, and at least a part of the encoded data of the code block included in the subband of the low frequency component equal to or lower than the predetermined frequency Is stored in the upper layer designated in advance as
Coded data of a code block that is a code block that does not include a coefficient corresponding to the region of interest and that is included in the subband having a frequency component higher than the predetermined frequency is designated in advance as a layer for the region of interest. Means for obtaining the codestream by storing in a lower layer than the upper layer,
As an image encoding program realized on a computer.

Images