JP4013721B2 - Image processing device - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an image processing apparatus that handles image data, such as a copying machine, a printer, and a scanner.
[0002]
[Prior art]
Currently, as a still image compression encoding method, the JPEG method for compressing image data by discrete cosine transform is widely used. However, the image data is compressed by wavelet transform for further improvement in compression performance and function expansion. Development and popularization of the JPEG2000 system is underway. One of the features of the JPEG2000 system is that a specific area in the image data is encoded with priority over other areas so that the outline can be recognized at the initial transmission stage, or a higher image quality than other areas. It is known that a region of interest (hereinafter referred to as ROI) can be arbitrarily set by encoding into
In relation to this feature, when printing out a JPEG2000 file in which the ROI is set, an uncomfortable image may be generated due to a difference in image quality between the ROI and the surrounding area. In response to this, conventionally, it is known that the restored image is subjected to smoothing processing or the like in the vicinity of the region boundary to alleviate the discontinuity at the boundary (see, for example, Patent Document 1). .
[0004]
[Patent Document 1]
Japanese Patent Laid-Open No. 9-117447 (page 4, FIG. 2)
[0005]
[Problems to be solved by the invention]
However, in such a method, in order to grasp the position of the region boundary on the image data, it is necessary to generate mask information about the ROI from the quantized value of the wavelet transform coefficient and to develop the bitmap. Requires a very large memory capacity to store mask information about the ROI.
[0006]
The present invention has been made in view of the above technical problem, and can suppress image discontinuity between the ROI and the surrounding area without reducing the memory capacity, and can reduce an uncomfortable image. An object is to provide a processing apparatus.
[0007]
[Means for Solving the Problems]
The invention according to claim 1 of the present application relates to a tile for each tile in the image processing apparatus for decoding compressed image data divided into a plurality of tiles each serving as a processing unit in encoding / decoding processing. Based on the frequency component conversion coefficient, detection means for detecting the presence state of the region of interest set in the compressed image data, and each tile is determined from only the region of interest based on the presence state of the region of interest detected by the detection unit. A region of interest tile, a non-region of interest tile consisting only of a non-region of interest, a region of interest boundary tile in which the region of interest and the region of non-interest are mixed, and the interest determined by the determination unit A setting unit that sets a weighting amount that reflects a distance from the region-of-interest boundary tile for the region tile and the region-of-interest boundary tile; Processing means for performing a predetermined process on the frequency component transform coefficients corresponding to the region of interest and the non-region of interest set in each region of interest tile and the region of interest boundary tile based on the weighted amount. It is characterized by that.
[0008]
The invention according to claim 2 of the present application is the invention according to claim 1, wherein the processing means applies frequency component conversion coefficients corresponding to the regions of interest in the region of interest tile and the region of interest boundary tile. The process of substituting with 0 from the lower order to the predetermined range is performed, and the range to be replaced with 0 is reduced as the distance from the region of interest boundary tile increases.
[0009]
Furthermore, the invention according to claim 3 of the present application is the invention according to claim 1 or 2, wherein the processing means is a high frequency signal in a subband obtained by decomposing compressed image data by frequency component conversion at the time of encoding. The processing is performed only on the component side sub-band.
[0010]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings.
Embodiment 1 FIG.
FIG. 1 is a block diagram showing an internal configuration of an image processing apparatus according to Embodiment 1 of the present invention. This figure shows an example in which a so-called direct print is performed in which a JPEG2000 file is directly supplied to the image processing apparatus 1 from the Internet or another computer via a network, processed, and printed out. Yes. The supply source of the JPEG2000 file in direct printing is not limited to this. For example, an external device such as a digital camera or a scanner, or a recording medium such as a compact flash (TM) or smart media (TM) is used. May be.
[0011]
The image processing apparatus 1 includes a CPU 2, a memory block 3, a codec 4 for a JPEG2000 file, a printer I / F 5, a printer unit 6, and a network I / F 7 connected to a common bus 8. In this apparatus 1, data transfer between each configuration excluding the CPU 2 and the memory block 3 is performed by direct memory access (DMA), and the CPU 2 performs DMA activation control so that print data can be transferred from file input. Control up to output.
[0012]
In the image processing apparatus 1, when a JPEG2000 file is input from the outside via a network, the file is first transferred from the network I / F 7 to the memory block 3 through the path a. In the memory block 3, code data constituting the JPEG2000 file is stored, and this code data is sequentially transferred to the codec 4 through the path b.
[0013]
In the codec 4, the code data is decoded. As will be described in detail later, this codec 4 can execute processing for suppressing image quality discontinuity between the ROI and the surrounding area as needed when handling a JPEG2000 file including the ROI. The decrypted data is transferred to the memory block 3 through the path c.
[0014]
In the memory block 3, bitmap data is created in a predetermined area based on the transferred decoded data. This bitmap data is transferred to the printer I / F 5 via the route d when it is created for one page, and then output to the printer unit 6 as print data.
[0015]
FIG. 2 is a diagram showing a flow of a JPEG2000 file decoding process by the codec 4 in the image processing apparatus 1. Here, the image data is divided into a plurality of rectangular tiles each serving as a basic processing unit of wavelet transform, and for each tile, a tile consisting of only ROI (hereinafter referred to as ROI tile), a non-interest region (hereinafter referred to as non-ROI). Processing is performed while determining whether the tile is composed of only (notation) (hereinafter referred to as a non-ROI tile) or a tile that exists at the boundary of the ROI and has a mixture of ROI and non-ROI (hereinafter referred to as an ROI boundary tile). The case of performing will be described.
[0016]
The JPEG2000 file input to the codec 4 is first unformatted. In this formatting cancellation, the encoded sequence of the JPEG2000 file is analyzed, and the ROI information described in the header included in the encoded sequence is read. As will be described later with reference to FIG. 5, the JPEG2000 file including the ROI usually has the most significant bit (MSB) side compared to the non-ROI quantized value of the frequency component transform coefficient (wavelet transform coefficient) corresponding to the ROI. Are encoded in a state shifted by S bits. The above ROI information is information representing the bit shift amount S to which the quantized value of the transform coefficient corresponding to the ROI is shifted.
[0017]
The code data after the formatting cancellation is entropy decoded. The decoded data is divided into a plurality of bit planes and sub-bit planes that are parallel to each other. From this state, the coefficient bit modeling is canceled, and thereby the transform coefficient is obtained.
[0018]
The ROI is analyzed based on the conversion coefficient obtained in this way. Here, first, the presence status of the ROI in the currently processed tile is detected from the acquired conversion coefficient, and further, based on the detection result, the tile is composed of only the ROI (hereinafter referred to as ROI tile). , A tile composed of only non-ROI (hereinafter referred to as non-ROI tile), or a tile in which ROI and non-ROI are mixed (hereinafter referred to as ROI boundary tile).
[0019]
Thereafter, a weighting process is performed on tiles determined as ROI tiles and ROI boundary tiles. The weighting process will be described later with reference to FIG. Then, based on the result of this weighting process and the ROI information read at the time of file input, a conversion coefficient obtained by shifting the conversion coefficient after canceling the coefficient bit modeling by a predetermined bit shift amount (for example, bit shift amount S) is obtained. Is done.
[0020]
If the JPEG2000 file has been previously quantized, the transform coefficient is then inversely quantized. Thereafter, each color component is generated by performing inverse wavelet transform (IDWT) on the data.
[0021]
FIG. 3 shows an example of image data including the ROI 10. In this figure, a region excluding the ROI 10 set in an arbitrary shape is indicated by reference numeral 12 as a non-ROI.
[0022]
FIG. 4 is an enlarged view of the frame X surrounding the region including the ROI 10 and the non-ROI 12 in FIG. As can be seen from FIG. 4, the image data is divided into a plurality of tiles each serving as a basic processing unit of wavelet transform. Although this tile size differs depending on the processing system, when an MFP (Multifunction Peripherals) is adopted as the image processing apparatus 1, for example, approximately 128 × 128 is appropriate as this size due to memory capacity restrictions.
[0023]
In FIG. 4, any one of symbols A, B, and C indicating the type of tile is attached to the upper left of each tile. Specifically, the tile A is an ROI boundary tile including the ROI 10 and the non-ROI 12, the tile B is an ROI tile including only the ROI 10, and the tile C is a non-ROI tile including only the non-ROI 12.
[0024]
As described with reference to FIG. 2, in this embodiment, when decoding a JPEG2000 file, the tiles are discriminated, and then the tiles determined as ROI boundary tile A and ROI tile B are weighted. To do. In this weighting process, information reflecting the distance from the ROI boundary tile A is set for the ROI boundary tile A and the ROI tile B. In FIG. 4, “1” is set for the ROI boundary tile A. On the other hand, for the ROI tile B, an example is shown in which “2”, “3”, and “4” are set in accordance with the distance from the ROI boundary tile A.
[0025]
By the way, as described above, the JPEG2000 file including the ROI is encoded in a state where the quantized value of the wavelet transform coefficient corresponding to the ROI is shifted to the MSB side by S bits compared to that of the non-ROI. This is a result of adopting a typical max-shift method as a method for distinguishing between ROI and non-ROI in a JPEG2000 file. FIG. 5 is a conceptual diagram of the max-shift method.
[0026]
Normally, when the ROI is set in the JPEG2000 file, first, mask information indicating the position of the pixel specified in the ROI is generated, then the bit shift amount S is determined, and the wavelet transform coefficient for all the pixels is determined. Is quantized by S bits to the MSB side. Thereafter, based on the mask information generated in advance, the quantized value of the wavelet transform coefficient is shifted to the LSB side by S bits for the pixels excluding the pixels designated as the ROI. As a result, as shown on the left side of FIG. 5, the quantized value of the wavelet transform coefficient is shifted to the MSB side by S bits only for the pixel designated as ROI. Then, encoding is performed in this state.
[0027]
During the decoding of such JPEG2000 file, quantized values of the wavelet transform coefficients of more than 2 ^S is recognized as corresponding to ROI, these are as shown on the right side of FIG. 5, the least significant bit (LSB) side It is shifted by S bits, and the shift by the max-shift method is released.
[0028]
In this embodiment, the normal bit shift processing shown in FIG. 5 is performed on the non-ROI tile C as necessary, while the ROI boundary tile A and the ROI tile B are processed in FIGS. As shown in FIG. 5, by performing the bit shift processing based on the above-described weighting amount, the image quality discontinuity of the ROI 10 and the surrounding non-ROI 12 can be reduced.
[0029]
Hereinafter, with reference to FIGS. 6 to 9, the bit shift process performed based on the weighting amounts “1”, “2”, “3”, and “4” will be described.
[0030]
FIG. 6 is a diagram conceptually illustrating a bit shift process performed on the ROI boundary tile A to which the weighting amount “1” is attached. For the ROI boundary tile A for which the weighting amount “1” is set, as the quantized value of the wavelet transform coefficient of 2 ^S or more is recognized as corresponding to the ROI, the lower S bits of the quantized value of the transform coefficient The 0th, (S + 1) th and (S + 2) th bits are replaced with 0. Thereafter, the quantized value of the wavelet transform coefficient corresponding to the ROI is shifted to the LSB side by S bits.
[0031]
FIG. 7 is a diagram conceptually illustrating a bit shift process performed on the ROI tile B in which the weighting amount “2” is set. For the ROI tile B for which the weighting amount “2” is set, as the quantized value of the wavelet transform coefficient of 2 ^S or more is recognized as corresponding to the ROI, the lower S of the quantized value of the transform coefficient The bit and the (S + 1) -th bit are replaced with 0. Thereafter, the quantized value of the wavelet transform coefficient corresponding to the ROI is shifted to the LSB side by S bits.
[0032]
FIG. 8 is a diagram conceptually illustrating a bit shift process performed on the ROI tile B in which the weighting amount “3” is set. For the ROI tile B in which the weighting amount “3” is set, as the quantized value of the wavelet transform coefficient of 2 ^S or more is recognized as corresponding to the ROI, the lower S of the quantized value of the transform coefficient The bit is replaced with 0. Thereafter, the quantized value of the wavelet transform coefficient corresponding to the ROI is shifted to the LSB side by S bits.
[0033]
FIG. 9 is a diagram conceptually illustrating a bit shift process performed on the ROI tile B in which the weighting amount “4” is set. For the ROI tile B in which the weighting amount “4” is set, the quantized values of the wavelet transform coefficients of 2 ^S or more are recognized as corresponding to the ROI, and the wavelet transform coefficients corresponding to these ROIs are quantized. The value is shifted to the LSB side by S bits. This bit shift processing is the same as the normal bit shift processing shown in FIG.
[0034]
As described above, by performing specific bit shift processing based on the weighting amount for the ROI boundary tile A and the ROI tile B, the information amount in the ROI boundary tile and the neighboring tiles is reduced. The difference in image quality of the non-ROI 12 can be suppressed, and the uncomfortable feeling of the image can be reduced.
[0035]
Note that it is conceivable to suppress the deterioration in image quality by performing the above processing only on the high frequency component of the wavelet transform coefficient. For example, when the image data is encoded at wavelet decomposition level 2, the data to be decoded is in a state of being decomposed into subbands as shown in FIG. 10 by frequency component conversion. In this figure, LL represents a horizontal / vertical low frequency component, LH represents a horizontal low frequency component / vertical high frequency component, HL represents a horizontal high frequency component / vertical low frequency component, and HH represents a horizontal / vertical high frequency component. That is, image quality degradation can be suppressed by performing the above-described processing only on the high frequency component side subbands (HH2, HL2, LH2, and HH1 indicated by hatching) of the subbands. it can.
[0036]
FIG. 11 is a flowchart of ROI boundary determination processing for each tile. In this process, first, in S21, a frequency component transform coefficient (wavelet transform coefficient) from which coefficient bit modeling has been canceled is generated. Next, in S22, the dynamic range (lower limit to upper limit) of the generated wavelet transform coefficient is analyzed. That is, how many bits each wavelet transform coefficient is detected. Subsequently, in S23, it is determined whether or not the detected dynamic range is equal to or larger than the bit shift amount S according to the above-described max-shift method read from the header included in the encoded sequence of the JPEG2000 file.
[0037]
As a result of S23, when it is determined that the dynamic range is not equal to or greater than S, the process proceeds to S27, and the tile currently being processed is determined as a non-ROI tile C. Thereafter, the process proceeds to S28.
[0038]
On the other hand, if it is determined as a result of S23 that the dynamic range is greater than or equal to S, the process proceeds to S24, and it is subsequently determined whether or not there are wavelet transform coefficients of less than S bits in the tile. As a result, if it is determined that there are no wavelet transform coefficients of less than S bits, the process proceeds to S26, where the currently processed tile is determined to be ROI tile B, and the position information in the image data of the tile is further determined. Saved. Thereafter, the process proceeds to S28.
[0039]
On the other hand, if it is determined in S24 that there is a wavelet transform coefficient of less than S bits, the process proceeds to S25, where the tile currently being processed is determined to be the ROI boundary tile A, and further in the image data of that tile. Is stored. Thereafter, the process proceeds to S28.
[0040]
In S28, it is determined whether or not processing of all tiles has been completed. If it is determined that the process has not ended, the process returns to S21, and the subsequent processes are repeated for the next tile. On the other hand, if it is determined that the process has been completed, the process proceeds to S29.
[0041]
In S29, a weighting process is performed on the ROI boundary tile A and the ROI tile B based on the stored position information. Further, in S30, a predetermined bit shift process is performed on the wavelet transform coefficient for each tile based on the weighting amount set for each tile by the weighting process. This is the end of the process.
[0042]
FIG. 12 is a flowchart showing details of the weighting process (S29 in FIG. 11) for the ROI tile A and the ROI tile B. In this process, first, in S31, it is determined whether or not the tile being processed is the ROI boundary tile A. As a result, if it is determined that the tile is the ROI boundary tile A, the process proceeds to S32, the weighting amount “1” is set, and the process ends.
[0043]
On the other hand, if it is determined that the tile being processed is not the ROI boundary tile A, the process proceeds to S33, and it is subsequently determined whether or not the tile being processed is the ROI tile B. As a result, when it is determined that the tile is not ROI tile B (that is, non-ROI tile), the process proceeds to S39, and the process ends without setting the weighting amount.
[0044]
On the other hand, if it is determined as a ROI tile B as a result of S33, the process proceeds to S34, and subsequently, the tile being processed becomes the ROI boundary tile A based on the position information stored in S26 of FIG. It is determined whether the tile is adjacent. As a result, if it is determined that the tile is adjacent to the ROI boundary tile A, the process proceeds to S35, the weighting amount “2” is set, and the process ends.
[0045]
On the other hand, if it is determined as a result of S34 that the tile is not adjacent to the ROI boundary tile A, the process proceeds to S36, and subsequently, the ROI of the tile being processed based on the position information stored in S26 of FIG. It is determined whether the distance from the boundary tile A is less than or equal to the value X. As the reference ROI boundary tile A, the one closest to the tile being processed is used. As a result, when it is determined that the distance from the ROI boundary tile A is equal to or less than the value X, the process proceeds to S37, the weighting amount “3” is set, and the process ends.
[0046]
On the other hand, if it is determined as a result of S36 that the distance from the ROI boundary tile A is greater than the value X, the process proceeds to S38, the weighting amount “4” is set, and the process ends.
[0047]
FIG. 13 is a flowchart showing details of the conversion coefficient processing (S30 in FIG. 11) for each tile. In this process, first, in S51, the weighting amount for the tile being processed is detected.
When the weighting amount is “1”, the process proceeds to S54, and the lower S bit, (S + 1) bit, and (S + 2) bit of the wavelet transform coefficient are set to 0 as shown in FIG. , Go to S55.
[0048]
When the weighting amount is “2”, the process proceeds to S53, and as shown in FIG. 7, the lower S bit and (S + 1) bit of the wavelet transform coefficient are set to 0, and then the process proceeds to S55. .
[0049]
Further, when the weighting amount is “3”, the process proceeds to S52, and as shown in FIG. 8, the lower S bit of the wavelet transform coefficient is set to 0 and then the process proceeds to S55.
[0050]
Furthermore, if the weighting amount is “4” or the weighting amount is not set, the process proceeds to S55 as it is.
[0051]
In S55, the wavelet transform coefficient recognized as corresponding to the ROI is shifted to the LSB side by S bits. This is the end of the process.
[0052]
Note that the present invention is not limited to the illustrated embodiments, and it goes without saying that various improvements and design changes are possible without departing from the scope of the present invention.
[0053]
【The invention's effect】
As is apparent from the above description, according to the invention of claim 1 of the present application, an image processing device that decodes compressed image data divided into a plurality of tiles each serving as a processing unit in encoding / decoding processing. For each tile, based on the frequency component conversion coefficient for the tile, the presence state of the region of interest set in the compressed image data is detected, and based on the detected state of interest region, each tile is A region of interest tile consisting only of a region of interest, a non-region of interest tile consisting of only a non-region of interest, a region-of-interest border tile in which a region of interest and a region of non-interest are mixed, A weighting amount that reflects the distance from the region-of-interest boundary tile is set for the region-of-interest boundary tile, and each region of interest is based on the set weighting amount. The frequency component conversion coefficients corresponding to the region of interest and the region of interest in the tile and the region of interest boundary tile are subjected to predetermined processing, so that for each tile, the image quality discontinuity between the region of interest and the region of non-interest is reduced. Processing that suppresses and reduces the uncomfortable feeling of the image is possible.
[0054]
Further, according to the invention according to claim 2 of the present application, as the predetermined processing, a predetermined frequency component conversion coefficient corresponding to the region of interest in each region of interest tile and the region of interest boundary tile is predetermined from the lower order. The range of replacement is reduced to 0 as the distance from the region-of-interest boundary tile increases, so that the amount of information in the region-of-interest boundary tile and its neighboring tiles is reduced. The difference in image quality between the region and the non-interest region can be suppressed, and the uncomfortable feeling of the image can be reduced.
[0055]
Furthermore, according to the invention of claim 3 of the present application, in the predetermined processing, only the subbands on the high frequency component side among the subbands in which the compressed image data is decomposed by the frequency component conversion at the time of encoding. Thus, the amount of information in the region-of-interest boundary tile and its neighboring tiles is reduced, while the image quality deterioration can be suppressed.
[Brief description of the drawings]
FIG. 1 is a block diagram showing an internal configuration of an image processing apparatus according to Embodiment 1 of the present invention.
FIG. 2 is a diagram showing a flow of a JPEG2000 file decoding process by a codec in the image processing apparatus.
FIG. 3 is a diagram illustrating image data including an ROI.
FIG. 4 is an enlarged view inside a frame X in FIG. 4;
FIG. 5 is a conceptual diagram of a typical max-shift method as a method for distinguishing between ROI and non-ROI in a JPEG2000 file.
FIG. 6 is a diagram conceptually illustrating a bit shift process performed on a tile for which a weighting amount “1” is set.
FIG. 7 is a diagram conceptually illustrating a bit shift process performed on a tile for which a weighting amount “2” is set.
FIG. 8 is a diagram conceptually illustrating a bit shift process performed on a tile for which a weighting amount “3” is set.
FIG. 9 is a diagram conceptually illustrating a bit shift process performed on a tile for which a weighting amount “4” is set.
FIG. 10 is a diagram illustrating an aspect in which image data encoded at wavelet decomposition level 2 is decomposed into subbands.
FIG. 11 is a flowchart of ROI boundary determination processing for each tile.
FIG. 12 is a flowchart for weighting processing (S29 in FIG. 11) for each tile.
FIG. 13 is a flowchart of conversion coefficient processing (S30 in FIG. 11).
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 1 ... Image processing apparatus 2 ... CPU
3 ... Memory block 4 ... Codec 5 ... Printer I / F
6 ... Printer unit 7 ... Network I / F
8 ... Bus 10 ... ROI
12 ... Non-ROI

Claims (2)

In an image processing apparatus for decoding compressed image data divided into a plurality of tiles each serving as a processing unit in encoding / decoding processing,
Detecting means for detecting the presence of a region of interest set in the compressed image data for each tile from the frequency component conversion coefficient for the tile;
Based on the presence state of the region of interest detected by the detection means, each tile is a region of interest tile consisting only of the region of interest, a non-region of interest tile consisting of only the non-region of interest, and a region of interest where a region of interest and a region of interest are mixed A discriminating means for discriminating whether the tile is one of the boundary tiles;
Setting means for setting a weighting amount reflecting a distance from the region-of-interest boundary tile for the region-of-interest tile and the region-of-interest boundary tile determined by the determining unit;
Processing means for performing predetermined processing on the frequency component transform coefficients corresponding to the regions of interest and non-regions of interest in each region of interest tile and region of interest boundary tile based on the weighting amount set by the setting unit; Prepared ,
The predetermined processing performed by the processing means performs processing for replacing the frequency component conversion coefficients corresponding to the regions of interest in the region-of-interest tiles and the region-of-interest boundary tiles with 0 from the lower order to a predetermined range, Including increasing the distance from the region-of-interest boundary tile, reducing the range to be replaced with 0,
An image processing apparatus. 2. The processing unit according to claim 1, wherein the processing means performs processing only on a subband on a high frequency component side among subbands obtained by decomposing compressed image data by frequency component conversion at the time of encoding. Image processing device.

Images