JP7390788B2 - Image encoding device and its control method and program - Google Patents

Description

The present invention relates to a compression encoding technique for image data.

Conventionally, regions extracted by face recognition, etc., and character/line drawing regions separated by image region separation are identified as regions of interest (ROI), and the amount of code in regions other than the ROI is reduced and compressed. Various techniques are known to do this. As a simpler ROI compression technique, a method is known in which an image is divided into rectangular regions such as tiles, and the image quality is changed between the rectangular regions that include the ROI region and the other rectangular regions. A method for realizing such ROI encoding using JPEG2000, which is an international standard encoding method, is described in Patent Document 1.

Furthermore, in JPEG2000, there is a method known for identifying and encoding the ROI region pixel by pixel using a method called the max shift method, which bit-shifts the DWT transform coefficients included in the ROI using discrete wavelet transform (DWT). It is being A method for realizing ROI encoding using this mechanism is described in Patent Document 2.

Japanese Patent Application Publication No. 2003-339047 Japanese Patent Application Publication No. 2004-235935

Although it is referred to as ROI encoding, there are cases in which a certain degree of image quality deterioration can be tolerated in the ROI region, and cases in which it is desirable to perform lossless compression on the ROI region.

For example, text/line drawing areas may become unrecognizable due to deterioration, so it is desirable to perform lossless compression. Alternatively, when synthesizing a free-viewpoint image from multi-viewpoint videos of a person shot by multiple cameras, it is necessary to take corresponding points for details of the subject, and the target of free-viewpoint image synthesis in the images shot by each camera It is desirable to use lossless compression for the area.

However, assuming lossless compression of the ROI region, it is difficult to reduce encoded data using conventional techniques that perform ROI encoding in units of predetermined rectangular regions. This is because if the important region is small relative to the rectangular region size, the rectangular region determined to be losslessly compressed becomes large, and data outside the ROI region is also losslessly compressed, making it difficult to reduce the amount of code. Conversely, if the rectangular area size is determined according to the minimum area size that can be set as the ROI area, the header data accompanying each rectangular area will occupy too much of the total code amount, so it is still necessary to reduce the code amount. becomes difficult. It is also possible to limit the rectangular area size to a certain size and control the quantization step on a line-by-line basis. However, in that case, control is performed only in the height direction, and if the height of the rectangular area and the height of the important area become the same, the rectangular area will be losslessly compressed, making it impossible to reduce the amount of code.

On the other hand, if the ROI region is determined pixel by pixel using the JPEG2000 max shift method, no code data is wasted. However, bit-shifting the entire coefficient and deleting lower bits outside the ROI region corresponds to quantization of 2 to the nth power, and there is a problem in that the image quality of the lossy region cannot be precisely controlled. Furthermore, when decoding a tile to which an ROI has been set, it is necessary to perform processing to return the shifted amount, so there is a problem in that the decoding process is different between tiles with an ROI and tiles without such a setting.

The present invention has been made in view of the above problems, and it is possible to maintain high image quality of image data including a region of interest while controlling the amount of code outside the region of interest. Rather, the aim is to provide a technique for generating coded data that can be efficiently decoded using the same algorithm for both the region of interest and the outside of the region of interest.

In order to solve this problem, for example, the image encoding device of the present invention has the following configuration. That is,
An image encoding device,
a setting means for setting a region of interest in image data to be encoded;
Transforming means for obtaining a plurality of subbands by performing wavelet transformation on the encoding target image data using a reversible filter;
quantization means for quantizing the coefficients included in each subband obtained by the conversion means on a line-by-line basis according to a quantization parameter;
encoding means for entropy encoding the quantized coefficients of each subband obtained by quantization on a line-by-line basis to generate encoded data to which information specifying the quantization parameter is added;
and a control means for controlling the quantization means and the encoding means,
The control means, for each subband,
determining means for determining whether a line of interest made up of coefficients includes a coefficient generated with reference to pixels in the region of interest;
correction means for correcting the quantized coefficients obtained by the quantization means,
The control means includes:
If the determining means determines that the line of interest does not include a coefficient generated by referring to pixels within the region of interest, the determining means causes the quantizing means to calculate each coefficient of the line of interest using the quantization parameter. controlling the quantization and causing the encoding means to generate encoded data including information specifying the quantization parameter;
If the determining means determines that the line of interest includes a coefficient generated by referring to the pixel within the region of interest, the determining means instructs the quantizing means to determine which of the coefficients of the line of interest includes the coefficient generated by referring to the pixel within the region of interest. The coefficients generated by referring to the pixel in the region of interest are controlled to maintain their values, and the coefficients generated without referring to the pixels in the region of interest are quantized using the quantization parameter, and , the correction means corrects the quantized coefficients obtained by the quantization means by inversely quantizing them using the quantization parameter, and the encoding means quantizes the line of interest. The method is characterized in that encoded data including information indicating that the step width is "1" is generated.

According to the present invention, image data including a region of interest can be processed while maintaining high image quality for the region of interest, as well as controlling the amount of code outside the region of interest. It is possible to generate encoded data that can be efficiently decoded using the same algorithm.

FIG. 1 is a block diagram showing the configuration of an encoding device in a first embodiment. 5 is a flowchart showing image encoding processing in the first embodiment. FIG. 3 is a diagram for explaining the ROI and DWT in the encoding target image in the first embodiment. FIG. 3 is a diagram showing the quantization step of each subband and the structure of encoded data. 5 is a flowchart showing encoding processing for ROI in the first embodiment. 7 is a flowchart illustrating ROI non-reference transformation coefficient correction processing in the first embodiment. 5 is a flowchart showing encoding processing of subband DWT transform coefficients in the first embodiment. FIG. 3 is a diagram showing a positional relationship between a transformation coefficient of interest and surrounding encoded transformation coefficients. 7 is a flowchart illustrating ROI non-reference transformation coefficient correction processing in the second embodiment.

DESCRIPTION OF EMBODIMENTS Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the present invention, and not all combinations of features described in the present embodiments are essential to the solution of the present invention. Note that the same configurations will be described using the same reference numerals.

[First embodiment]
FIG. 1A is a block configuration diagram of an image encoding apparatus that encodes an image including a region of interest (ROI) in this embodiment. This image encoding device includes a CPU 101 , an input section 102 , an ROI setting section 103 , a code data generation section 104 , a display section 105 , a memory 106 , and a storage section 107 .

The CPU 101 is involved in the processing of each component and controls the entire device. The input unit 102 inputs instructions from the user, image data to be encoded, and the like. Therefore, the input unit 102 includes a pointing device such as a keyboard and a mouse, and includes an interface for inputting image data. The ROI setting unit 103 sets a region of interest ROI in the image data to be encoded. The code data generation unit 104 performs discrete wavelet transform (DWT) on pixel values of image data to be encoded, and performs run-length encoding and predictive encoding on the obtained DWT transform coefficients. The code is encoded by switching between the two as appropriate (details will be described later). As the display unit 105, a liquid crystal display or the like is normally used. The memory 106 is composed of a ROM and a RAM, and is used to store programs executed by the CPU 101 and various parameters, and as a work area for the CPU 101. The storage unit 107 is a part that stores image data, programs, etc., and usually uses a hard disk or the like. Further, it is assumed that a control program necessary for the processing of the flowchart described later is stored in the storage unit 107 or in the ROM of the memory 106. If the program is stored in the storage unit 107, the CPU 101 once reads the program into the RAM in the memory 106 and executes it. Regarding the system configuration, there are various components other than those described above, but since they are not the main focus of the present invention, their explanation will be omitted.

The encoded data generation unit 104 of the embodiment performs DWT on the image data to be encoded, quantizes the obtained DWT transform coefficients, and encodes them. Then, pixels within the ROI in the image data to be encoded are losslessly (reversibly) encoded, and pixels outside the ROI are lossy (irreversibly) encoded. For this reason, a reversible 5-3 tap DWT filter is used in the wavelet transform, but the type and size of the filter is not limited to this. In order to losslessly encode pixels within the ROI, DWT transform coefficients generated with reference to pixels within the ROI may be losslessly encoded. In other words, when at least one of the five inputs of the filter receives pixel data of the ROI, the DWT transform coefficients calculated by that filter are losslessly encoded. Lossy encoding is allowed for DWT transform coefficients generated without referring to pixels within the ROI.

FIG. 1(b) is a block diagram of the code data generation section 104. Code data generation section 104 includes a DWT section 110, a quantization section 111, an encoding method selection section 112, a run-length encoding section 113, a predictive encoding section 114, a code string generation section 116, and switches 109 and 110. .

The DWT unit 110 inputs image data to be encoded line by line, performs DWT on each pixel included in the input line (hereinafter referred to as the line of interest) using a DWT filter, and performs DWT on each pixel included in the input line (hereinafter referred to as the line of interest). Generate a band.

The quantization unit 111 refers to the ROI area information supplied from the ROI area setting unit 103 and determines whether the line of interest in the subband of interest includes a DWT transform coefficient generated by referring to pixels in the ROI. Determine. Note that, hereinafter, the DWT transform coefficients will be simply referred to as transform coefficients. The DWT transformation coefficients generated by referring to pixels within the ROI (within the region of interest) are referred to as ROI reference transformation coefficients, and pixels within the ROI are not referred to, that is, all pixels input to the DWT filter are outside the ROI. The DWT transform coefficients generated when the pixel is the ROI non-reference transform coefficient are called ROI non-reference transform coefficients.

If the line of interest does not include ROI reference transform coefficients, the quantization unit 111 quantizes the transform coefficients of all subbands obtained from the line of interest according to preset quantization parameters, and converts the quantized transform coefficients into switches. 109 to the encoding method selection unit 112.

On the other hand, when the line of interest includes ROI reference transform coefficients, the quantization unit 111 performs quantization step "1" for the ROI reference transform coefficients among the transform coefficients of all subbands obtained from the line of interest according to the set quantization parameter. ” (equivalent to not quantizing), and ROI non-reference transform coefficients are quantized according to preset quantization parameters. Then, the quantization section 111 supplies the obtained quantized transform coefficients to the coefficient modification section 120 via the switch 109.

The coefficient modification unit 120 modifies ROI non-reference transformation coefficients among the transformation coefficients of the line of interest supplied from the quantization unit 111 based on the ROI information. Specifically, the coefficient correction unit 120 maintains the values of the ROI reference transform coefficients supplied from the quantizer 111, and dequantizes the ROI non-reference transform coefficients. As a result, among the transform coefficients of each subband obtained from the line of interest, the ROI reference transform coefficient remains the value immediately after DWT, and reversibility is maintained. In addition, although ROI non-reference transform coefficients are irreversible because they include errors that depend on the quantization step value, they are values that have been restored to the level of the coefficients obtained by DWT, and can be data that contributes to encoding efficiency. .

Note that when generating encoded data of transform coefficients of a line of interest, information indicating a quantization parameter for each line is also stored in the header of the encoded data of that line. Therefore, information indicating that the quantization step width is "1" is stored in the header of the encoded data of the transform coefficients that has passed through the coefficient correction unit 120. Furthermore, information indicating the set quantization parameter is stored in the header of the encoded data of the transform coefficients that has not been passed through the coefficient correction unit 120.

Coding method selection section 112 outputs each transform coefficient to either run-length encoding section 113 or predictive encoding section 114. The run-length encoding section 113 and the predictive encoding section 114 perform respective encoding processes and store the generated encoded data in a buffer 115a in the encoded data output section 115. When the coded data of the transform coefficients of all the subbands are stored in the buffer 115a, the coded stream generation unit 116 arranges the coded data in a predetermined order, shapes it as a coded stream (coded stream), and outputs it.

In this embodiment, in order to simplify the explanation, the image data to be encoded will be explained as monochrome multivalued image data expressed by a single component (for example, luminance) and expressed by 8 bits per pixel. However, it can also be applied to image data in which the brightness value is expressed using a bit number other than 8 bits, such as 10 bits, 12 bits, 14 bits, etc., for each pixel. Furthermore, when one pixel is composed of multiple color components such as RGB and CMYK, the image data separated for each component can be encoded, so the encoding target is monochrome multi-valued image data. It is not limited. It should be understood that the monochrome multivalued image data with 8 bits per pixel is merely an example.

Next, image encoding processing in the encoded data generation unit 104 in the embodiment will be described with reference to the flowchart of FIG. 2.

In S201, the code data generation unit 104 obtains the number W of pixels in the horizontal direction and the number H of pixels in the vertical direction of the encoding target image input from the input unit 102. If the image data to be encoded is image data obtained from an image sensor, the CPU 101 may set information indicating the resolution (number of horizontal and vertical pixels) of the image sensor in the code data generation unit 104. . Furthermore, if the image data to be encoded is saved in a file format, the CPU 101 may acquire information on W and H from the file header and set it in the encoded data generation unit 104.

In S202, the code data generation unit 104 acquires ROI region information (information indicating the shape and position of the ROI) from the ROI setting unit 103. The method for specifying the ROI region is not the main focus of this application, and any method may be used. For example, results of automatic detection of text/line drawing areas, faces, specific objects, etc. from the image input from the input unit 102 may be automatically acquired as the ROI area. Alternatively, any closed region specified by the user using a mouse or the like may be acquired as the ROI region. In this embodiment, it is assumed that a pre-learned automatic detection result of a specific object is acquired as an ROI region, and the region 301 in FIG. 3(a) is acquired as the ROI region.

In S203, the code data generation unit 104 acquires the number of executions of DWT and a quantization parameter that specifies a quantization step value when quantizing each subband obtained by DWT. It is assumed that these values are set in advance by the user using the input unit 102. The number of subbands can be determined by "number of times DWT is executed x 3 + 1". In this embodiment, the number of times the DWT is executed is "2". Therefore, the number of generated subbands is 2LL, 2HL, 2LH, 2HH, 1HL, 1LH, and 1HH, a total of seven, as shown in FIG. 3(b). Further, the quantization step values shown in FIG. 4A are determined for each of these subbands, and the code data generation unit 104 acquires these values. Further, the code data generation unit 104 determines whether the transform coefficient is an ROI-referenced transform coefficient or an ROI-non-referenced transform coefficient based on the number of executions of DWT and the DWT filter size.

In S204, the encoded data generation unit 104 initializes the counter y by assigning "0" to the counter y indicating the vertical position of the image data to be encoded. In the embodiment, since the number of pixels in the vertical direction of the image to be encoded is H, the value of the counter y can range from 0 to H-1.

In S205, the code data generation unit 104 acquires the image data for one line of the y-th row into a line buffer (not shown) in the DWT unit 110.

Then, in S206, the code data generation unit 104 controls the DWT unit 110 to apply a 5-3 tap filter, which is an integer type filter adopted in JPEG2000, to the image data stored in the line buffer. , execute DWT twice. Generally, two-dimensional DWT can be realized by applying one-dimensional DWT to the horizontal and vertical directions of the image to be transformed. Note that horizontal DWT can be performed on one line of image data, but when vertical DWT is performed, image data whose number of lines depends on the filter size and the number of times DWT is performed is required. In other words, in reality, the DWT unit 110 has a buffer memory of that capacity, executes DWT with a certain delay with respect to the input line, and outputs the result. Here, for the sake of simplicity, the explanation will be continued on the assumption that seven subband transformation coefficients are obtained from each line of image data each time that line is input.

In S207, the code data generation unit 104 determines whether or not there is an ROI reference transformation coefficient among the transformation coefficients obtained from the image data of the y row, that is, the line of interest. If it is determined that the ROI reference transformation coefficient exists in the transformation coefficients of the line of interest, the encoded data generation unit 104 advances the process to S215 and executes ROI encoding processing. Proceeding to S215 corresponds to the case in which the transform coefficients obtained by quantization by the quantizer 111 are supplied to the coefficient correction unit 120 via the switch 109 in FIG. 1(b).

On the other hand, if it is determined that there is no ROI-referenced transformation coefficient among the transformation coefficients obtained from the line of interest, that is, if all the transformation coefficients are ROI-non-referenced transformation coefficients, the code data generation unit 104 advances the process to S209. In the case of this embodiment, as can be seen from FIG. 3(a), the first few lines are outside the ROI region, so the encoding of the first few lines branches from S207 to S209. The process proceeds to S209 because the transform coefficients obtained by quantization by the quantizer 111 in FIG. This corresponds to the case where

In S209, the code data generation unit 104 initializes the subband counter s by assigning "0" to it. In this embodiment, it is assumed that counter s values "0" to "6" are assigned to seven subbands 2LL to 1HH, respectively. Further, the subband indicated by the counter s is simply referred to as subband s.

In S210, code data generation unit 104 controls quantization unit 111 to quantize the transform coefficient of subband s using the corresponding quantization step obtained in S203.

In S211, the code data generation unit 104 adds information indicating the quantization step width of the subband s to the header of the code data of the subband s. The information is stored in the temporary storage buffer for s.

In S212, the code data generation unit 104 determines whether to use the run-length encoding unit 113 or the predictive encoding unit 114 for the quantized transform coefficient of the subband s generated in S210, The encoding unit selected based on the determination result is used to perform encoding. The selected encoding section stores the generated encoded data so as to follow the header of the subband s in the encoded data output section 115.

Here, as shown in FIG. 4(b), the code data temporary storage buffer for each subband secured in the code data output unit 115 stores information indicating the quantization step width of the line. The code data of the transform coefficients is saved. Then, combinations of quantization step width and code data are stored in order from the first line to the last line. Details of the encoding process in S212 will be described later with reference to FIG.

In S213, the code data generation unit 104 adds "1" to the subband counter s to update it. In S214, the code data generation unit 104 compares the counter s with the total number of subbands "1+(DWT number×3)". If s is the same as the total number of subbands, this means that encoding of all subbands for the line of interest has been completed, and the process advances to S216. Furthermore, if s and the total number of subbands are different, code data generation section 104 returns the process to S210 in order to encode the remaining subbands.

In S216, the code data generation unit 104 adds "1" to the line counter y to update it. Then, in S217, the code data generation unit 104 compares the line counter y and the height H of the image. If y and H match, this means that the encoding process has been completed for all lines, and the process advances to S218. If y and H do not match, the code data generation unit 104 returns the process to S205 and processes the next line.

In S218, the code data generation unit 104 controls the code string generation unit 116, arranges the code data of each subband in a predetermined order, forms and outputs a coded data string (stream), and End the flow. In this embodiment, as shown in FIG. 4(b), encoded data strings arranged in order from the low frequency subband are generated and output.

Next, the process of S215 will be explained with reference to the flowchart of FIG.

In S501, the code data generation unit 104 generates lossless coefficient position information P(s, x) for identifying the transformation coefficient generated with reference to pixels within the ROI. In this lossless coefficient position information P(s,x), s represents a subband and x represents a position in the horizontal direction. The code data generation unit 104 sets a value of "1" if the transform coefficient located in the horizontal direction x in the subband s is an ROI-referenced transform coefficient, and a value of "0" if it is an ROI-non-referenced transform coefficient. Generate P(s, x) with .

In S502, the code data generation unit 104 obtains a predetermined minimum value N of subband numbers to be quantized. This N is set in the code data generation unit 104 by the CPU 101, and in this embodiment, it is assumed that N=4. That is, the DWT transform coefficients of high frequency subbands 1HL (subband number 4), 1LH (subband number 5), and 1HH (subband number 6) at decomposition level 1 (maximum resolution) are quantized.

In S503, the code data generation unit 104 assigns zero to the counter s to initialize it. Note that the subband indicated by the counter s is also simply referred to as subband s. In S504, code data generation section 104 compares counters s and N to determine whether the transform coefficient of subband s is to be quantized. If the counter s is equal to or greater than N acquired in S502, the code data generation unit 104 determines that the data is to be quantized, and advances the process to S505. Otherwise, the code data generation unit 104 advances the process to S506.

In S505, the code data generation unit 104 controls the coefficient modification unit 120 to modify the ROI non-reference transformation coefficients among the transformation coefficients of the subband s obtained from the line of interest. The method for modifying the ROI non-reference transformation coefficients will be described in detail later using the flowchart of FIG. 6.

In S506, the code data generation unit 104 stores the quantization step width “1” used when quantizing the transform coefficient of the line of interest in the subband s in its header. Output to band s code data temporary storage buffer. In the next step S507, code data generation section 104 encodes the transform coefficients of subband s. Details of this processing will be described later using FIG. 7. In S508, the code data generation unit 104 adds "1" to the subband counter s to update it. Then, in S509, the code data generation unit 104 compares the subband number s and the total number of subbands "1+(number of times DWT is executed x 3)". If the two match, it means that encoding has been completed for all subbands, and the encoded data generation unit 104 ends this flow. On the other hand, if there is a mismatch, the code data generation unit 104 returns the process to S504 and encodes another subband.

Next, with reference to the flowchart in FIG. 6, the correction processing of ROI non-coefficients to be non-losslessly encoded in S505 in FIG. 5 will be described.

In S601, the code data generation unit 104 obtains the number sW of DWT transform coefficients in one line of the subband of interest. The number of DWT transform coefficients differs for each subband number, and is 1/2 of the number X of pixels divided by DWT or DWT transform coefficients. However, when the number X of pixels or DWT transform coefficients to which DWT is applied is an odd number, subbands LH and LL are (X+1)/2, and subbands HL and HH are (X-1)/2.

In S602, the code data generation unit 104 initializes the counter sX by assigning "0" to it. In S603, the lossless coefficient position information P(s, sX) generated in S501 is referred to, and it is determined whether the value is "1". If it is “1”, the sX-th transformation coefficient [sX] is the ROI reference transformation coefficient and is not modified. Therefore, the code data generation unit 104 advances the process to S605.

On the other hand, when P(s, sX) is “0”, the sX-th transformation coefficient [sX] is an ROI non-reference transformation coefficient. Therefore, the code data generation unit 104 advances the process to S604. In S604, the code data generation unit 104 rounds the sX-th transform coefficient [sX] according to the quantization step width obtained in S204. Specifically, the code data generation unit 104 quantizes the transform coefficient [sX] with the quantization step width, and inversely quantizes the value obtained with the same quantization step width. Then, the code data generation unit 104 outputs the transform coefficient obtained by the inverse quantization as the sX-th new transform coefficient [sX]. Note that the larger the quantization step width used, the larger the rounding error of the transform coefficient obtained through S604, but the values that can be taken are limited, and the encoding efficiency can be increased. Particularly, when the subband of interest is a high frequency subband, most of the transform coefficients after modification by the process of S604 become "0", which means that encoding efficiency can be improved.

In S605, the code data generation unit 104 adds "1" to the counter sX and updates it. Then, in S606, the code data generation unit 104 compares the counter sX with the number sW of transform coefficients for one line obtained in S601. If sX=sW, the code data generation unit 104 determines that the modification of the ROI non-reference transform coefficients in the line of interest of the subband of interest has been completed, and ends this flow. If sX<sW, it means that there is a possibility that unmodified ROI non-reference transformation coefficients exist. Therefore, the code data generation unit 104 returns the process to S603.

Next, the encoding process of subband transform coefficients will be explained with reference to the flowchart of FIG. 7. This process is also the process of S212 in FIG. 2 and S507 in FIG.

In S701, the code data generation unit 104 obtains the number sW of transform coefficients for one line of the subband of interest. Then, in S702, the code data generation unit 104 assigns "0" to a counter sX for specifying one transform coefficient, and initializes the counter sX. The value of the counter sX indicates the position from the left end of the conversion coefficient.

The subsequent processes of S703 to S705 are the processes of the encoding method selection unit 112 in the coded data generation unit 104.

As shown in FIG. 8, the encoding method determination process is performed by referring to the values of the surrounding transform coefficients a, b, and c, where x is the position of the transform coefficient of interest. Then, referring to these three peripheral transform coefficients a, b, and c, the encoding method selection unit 112 determines the encoding method of the target transform coefficient It is supplied to one of the converting sections 114. The selection criteria for the encoding method in the embodiment is different between subband LL, which is a low frequency component, and subbands which are other high frequency components. Therefore, in S703, the encoding method selection unit 112 determines whether or not the transform coefficients of the subband LL are to be encoded based on the counter s.

If it is determined that the transform coefficients of subband LL are to be encoded, the encoding method selection unit 112 advances the process to S705. On the other hand, if it is determined that the subband is other than LL, the encoding method selection unit 112 advances the process to S704.

In S704, the encoding method selection unit 112 determines whether the transform coefficient a to the left of the focused transform coefficient x is "0". If the transform coefficient a is "0", the encoding method selection unit 112 advances the process to S705. On the other hand, if the transform coefficient a is other than "0", the encoding method selection unit 112 advances the process to S708.

In S705, the encoding method selection unit 112 determines whether all surrounding transform coefficients a, b, and c have the same value. If all of the transform coefficients a, b, and c have the same value, the encoding method selection unit 112 advances the process to S706, and determines that at least one of the transform coefficients a, b, and c has a value different from the others. If so, the encoding method selection unit 112 advances the process to S708.

Note that when the determination result in S704 is Yes and the determination result in S705 is Yes, the relationship among the surrounding transformation coefficients a, b, and c is a=b=c=0. Further, when the transform coefficient x to be encoded is at the left end of the subband, the transform coefficient a does not exist. Furthermore, when transform coefficient x is within the first line of the subband, transform coefficients b and c do not exist. A conversion coefficient that does not exist in this way is assumed to be a preset value (for example, "0") and processed (as long as it is shared with the decoding side, the value does not matter). .

The determination processing in S703 to S705 above will be explained in an easy-to-understand manner as follows.

In general, run-length encoding is superior to predictive encoding in terms of encoding efficiency under conditions where the same value continues. All the transform coefficients included in the subband LL can be viewed as a reduced image that is 1/4 the size of the original image both horizontally and vertically. Therefore, in the case of subband LL, whether or not there is a high possibility that the same value will continue can be estimated based on whether or not a plurality of transformation coefficients around the transformation coefficient of interest have the same value. Therefore, if the surrounding transform coefficients a, b, and c have the same value, run-length encoding is started with the transform coefficient of interest as the starting point of the run. On the other hand, if continuity cannot be expected, predictive encoding is performed on the transformation coefficient of interest based on encoded surrounding transformation coefficients.

On the other hand, due to the nature of the transform coefficients of high frequency components other than subband LL, although "0" may be continuous, it is rare that the same value other than "0" is continuous. Therefore, for high-frequency components, run-length encoding is started with the target transform coefficient as the start point of the run only when the transform coefficients a, b, and c around the target transform coefficient are all "0". If even one of the surrounding transform coefficients a, b, and c is not "0", continuity cannot be expected, so predictive encoding is performed on the target transform coefficient based on the encoded surrounding transform coefficients.

As described above, when the process proceeds to S706, the code data generation unit 104 controls the run length encoding unit 113 to count the run length starting from the transformation coefficient of interest [sX]. The run length encoding unit 113 outputs a code indicating the counted run when the run is interrupted. Then, in S707, the run length counted in S706 is added to the counter sX, and then the position of the coefficient of interest to be input to the encoding method selection section is updated. Thereafter, the code data generation unit 104 advances the process to S708 in order to predictively encode the coefficients at the end of the run.

In S708, the code data generation unit 104 controls the predictive encoding unit 114 to perform predictive encoding on the transformation coefficient x of interest. The predictive encoding unit 114 obtains a predicted value p (simply p=a) from the surrounding encoded transform coefficients a, b, and c, and calculates the difference d (= xa) is entropy encoded.

After that, in S709, the code data generation unit 104 adds "1" to the counter sX in order to encode the next transform coefficient. Then, in S710, the code data generation unit 104 compares the counter sX with the number sW of transform coefficients for one line. If sX=sW, this means that all the transform coefficients of the line of interest have been encoded, so the code data generation unit 104 ends this flow. In other cases, the code data generation unit 104 returns the process to S703 in order to encode the next transform coefficient.

The above is the processing content of the code data generation unit 104 in the embodiment. When the above processing is performed, the higher the number of zero coefficients in a high frequency subband, the easier it is to be run-length coded. Therefore, the DWT transform coefficients that are not included in the ROI are quantized and are more likely to become "0". As a result, it becomes easier to shift to run-length encoding, and a high compression rate can be expected. In other words, the amount of code in areas other than the ROI can be reduced.

When the method of this embodiment is used, the quantization step width corresponding to each subband is written in the header of the encoded data in the lines of areas 302 and 303 in FIG. 3(c). Then, the quantization step width "1" is stored in the header of the encoded data of the line including the regions 304 and 305 and the ROI. Further, since most of the regions 304 and 305 are DWT transform coefficients obtained without referring to pixels in the ROI region, they are encoded data that has undergone quantization→inverse quantization processing. Therefore, even if the quantization step width is described as "1", it is possible to decode the DWT transform coefficients of the ROI region and other non-ROI regions without distinguishing them during decoding, which simplifies the processing. You can also expect high-speed decoding.

Furthermore, in this embodiment, among the lines of DWT transform coefficients that include the ROI region (coefficient lines with a quantization step width of "1"), for subbands 1HL, 1LH, and 1HH, the DWT transform coefficients outside the ROI region are Although quantization is used as a correction target, the number of subbands may be dynamically changed. For example, if the ROI to be subjected to lossless compression occupies a small proportion of the entire image, the total amount of code will not become very large. Therefore, only subbands 1HL, 1LH, and 1HH in this embodiment are targeted for quantization, and image quality deterioration outside the ROI region is prevented as much as possible. On the other hand, if the area of interest to be losslessly compressed occupies a large proportion of the entire image, the overall code amount tends to increase, so all high-frequency subbands other than the LL subband are quantized and the code amount is You can make it smaller. Therefore, a threshold value may be provided to determine whether the ROI is large, and the quantization target may be determined according to the threshold value.

Furthermore, the quantization step width of DWT transform coefficients that do not include the ROI can be controlled more finely than a power of 2, which not only makes it easier to control the code amount of the entire image, but also allows finer control of the image quality outside the ROI region. Can be done.

Furthermore, since DWT transform coefficients can be processed line by line, the amount of memory used during encoding can be reduced.

In the above embodiment, the encoded data generation unit 104 has been described as having the configuration shown in FIG. I do not care.

[Second embodiment]
In the first embodiment described above, details of the correction process (S505 in FIG. 5) of ROI non-reference conversion coefficients in a line including ROI reference conversion coefficients have been described with reference to the flowchart in FIG. 6. At that time, it has also been explained that quantization→inverse quantization is performed for ROI non-reference transform coefficients.

In this embodiment, an example will be described in which, among ROI non-reference coefficients, conversion coefficients having values within a preset range are set to zero.

Details of the process of S505 in FIG. 5 in the second embodiment will be described below with reference to the flowchart in FIG. 9. In the figure, the same processes as in the first embodiment are given the same reference numerals as in FIG. 6, and detailed explanation thereof will be omitted.

S601 and S602 are the same as those described in the first embodiment. In S901, the code data generation unit 104 obtains a positive threshold value B that defines a coefficient range for replacing with zero. The reason why the threshold value B is positive is to enable the possible range of the DWT conversion coefficient to be replaced with zero to be determined based on the absolute value. Note that this threshold value B is determined by the CPU 101 according to the user's instruction input, and is set in the code data generation unit 104. Further, the threshold value B is set to a value larger than the quantization step width used for that subband.

In S603, the code data generation unit 104 refers to the lossless coefficient position information P(s, sX) and determines whether the value is "1" or not. When the lossless coefficient position information P (s, sX) is “1”, the transformation coefficient of interest [sX] is the ROI reference transformation coefficient and must be losslessly encoded. Therefore, the code data generation unit 104 does not modify the transformation coefficient of interest and advances the process to S605.

When the lossless coefficient position information P (s, sX) is not "1" but "0", the transformation coefficient of interest [sX] is an ROI non-reference transformation coefficient and is subject to correction. Therefore, the code data generation unit 104 branches the process from S603 to S902. In S902, the code data generation unit 104 compares the absolute value of the conversion coefficient of interest [sX] and the value B. If the absolute value of the conversion coefficient of interest [sX] is less than or equal to the value B, the code data generation unit 104 advances the process to S903. In S903, the code data generation unit 104 replaces the conversion coefficient of interest [sX] with zero.

The processing in S604, S605, and S606 is the same as in the first embodiment.

Although S902 has been described as comparing the absolute value of the conversion coefficient of interest [sX] with the threshold B, this is equivalent to determining whether the following conditions are satisfied.
Condition: -B≦Attention conversion coefficient [sX]≦B

As a result of the above, the DWT transform coefficients obtained from the line of interest of the image data to be encoded include the ROI reference transform coefficients calculated with reference to pixels in the ROI, and the absolute value of the ROI reference transform coefficients If it is determined that is less than the threshold value, it is replaced with "0". As a result, the probability of "0" occurring is higher than in the first embodiment, and encoding efficiency can be further improved.

[Other embodiments]
In the second embodiment, the DWT transform coefficients in a certain range are uniformly corrected to zero depending on the subband, but the first and second embodiments may be combined. That is, the coefficients may be modified for subbands 2HL, 2LH, and 2HH using the method of the first embodiment, and the coefficients for higher frequency subbands 1HL, 1LH, and 1HH may be modified using the method of the second embodiment.

Furthermore, in the second embodiment, the value B that defines the range of DWT transform coefficients to be replaced with zero may be dynamically changed. For example, this value B may be changed depending on the type of subband. That is, for example, for subband 1HH, a larger value B is set, a wide range of DWT transform coefficients are set to zero, and the value B is decreased for subbands close to low frequencies to prevent excessive deterioration of image quality. Good too. Alternatively, the value B may be set based on the ratio of the area indicated by the ROI to the image data to be encoded. When the ratio of the area of the ROI to the entire image is large, the value of B is increased to suppress the amount of code. On the other hand, if the ratio is small, the value B may be reduced to reduce the difference in image quality between the ROI region and other pixels.

In this embodiment, in order to simplify the explanation, the number of times the DWT is executed is "2", but there is no limit to the number of times the DWT is executed.

Further, in the embodiment, a method has been described in which image data to be encoded is encoded without being divided into tiles, but the image data may be divided into a plurality of tiles. In that case, each tile may be encoded using the flow shown in FIG. 2. Another feature of the embodiment is that tiles that include an ROI and tiles that do not include an ROI can be decoded using exactly the same method.

In this embodiment, the quantization step width is written at the beginning of the code data of the DWT transform coefficient of each line, but they may be written in one place for each subband, each tile, or each image. Furthermore, when dividing tiles, the quantization step width is determined to be one for subbands of tiles outside the ROI. In that case, the quantization step width may be written only once for each subband of each tile. Alternatively, the quantization step width may be compressed and stored in code data.

Further, in the above embodiment, an example has been described in which image data is input in units of one line and encoded, but it may also be input in units of blocks of a predetermined size set in advance and encoded in units of blocks.

The present invention provides a system or device with a program that implements one or more of the functions of the embodiments described above via a network or a storage medium, and one or more processors in the computer of the system or device reads and executes the program. This can also be achieved by processing. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

101... CPU, 102... Input unit, 103... ROI setting unit, 104... Code data generation unit, 105... Display unit, 106... Memory, 107... Storage unit, 110... DWT unit, 111... Quantization unit, 112... Code encoding scheme selection unit, 113... run length encoding unit, 114... predictive encoding unit, 115... code data output unit, 116... code string generation unit

Claims (6)

An image encoding device,
a setting means for setting a region of interest in image data to be encoded;
Transforming means for obtaining a plurality of subbands by performing wavelet transformation on the encoding target image data using a reversible filter;
quantization means for quantizing the coefficients included in each subband obtained by the conversion means on a line-by-line basis according to a quantization parameter;
encoding means for entropy encoding the quantized coefficients of each subband obtained by quantization on a line-by-line basis to generate encoded data to which information specifying the quantization parameter is added;
and a control means for controlling the quantization means and the encoding means,
The control means, for each subband,
determining means for determining whether a line of interest made up of coefficients includes a coefficient generated with reference to pixels in the region of interest;
correction means for correcting the quantized coefficients obtained by the quantization means,
The control means includes:
If the determining means determines that the line of interest does not include a coefficient generated by referring to pixels within the region of interest, the determining means causes the quantizing means to calculate each coefficient of the line of interest using the quantization parameter. controlling the quantization, and causing the encoding means to generate encoded data including information specifying the quantization parameter;
If the determining means determines that the line of interest includes coefficients generated by referring to pixels within the region of interest, the determining means instructs the quantization means to determine whether the line of interest includes coefficients generated by referring to pixels within the region of interest, among the coefficients of the line of interest. The coefficients generated by referring to the pixel in the region of interest are controlled to maintain their values, and the coefficients generated without referring to the pixels in the region of interest are quantized using the quantization parameter, and , the correction means corrects the quantized coefficients obtained by the quantization means by inversely quantizing them using the quantization parameter, and the encoding means quantizes the line of interest. An image encoding device that generates encoded data that includes information indicating that a step width is "1". The correction means includes:
The image encoding device according to claim 1, characterized in that when a condition that the absolute value of the value indicated by the coefficient obtained by the quantization means is equal to or less than a preset threshold is satisfied, the coefficient is changed to zero. . 3. The image encoding apparatus according to claim 1, wherein the reversible filter is an integer type filter, and is a 5-3 tap filter defined in JPEG2000. The encoding means includes:
run-length encoding means for encoding a run of transform coefficients;
Predictive encoding means for predictively encoding the transform coefficients;
The target transform coefficient is set as the starting point of the run based on which subband the target transform coefficient to be entropy-encoded belongs to and the relationship among the plurality of encoded transform coefficients surrounding the target transform coefficient. and selecting means for selecting whether to encode the target transform coefficient using the run-length encoding means or to encode the target transform coefficient using the predictive encoding means. The image encoding device according to item 1. A method for controlling an image encoding device, the method comprising:
a setting step of setting a region of interest in the image data to be encoded;
a transformation step of obtaining a plurality of subbands by performing wavelet transformation on the encoding target image data using a reversible filter;
a quantization step of quantizing the coefficients included in each subband obtained in the conversion step in accordance with a quantization parameter on a line-by-line basis;
an encoding step of entropy encoding the quantized coefficients of each subband obtained by quantization line by line to generate encoded data to which information specifying the quantization parameter is added;
a control step for controlling the quantization step and the encoding step,
The control process includes, for each subband,
a determination step of determining whether a line of interest made up of coefficients includes a coefficient generated with reference to pixels in the region of interest;
a correction step of correcting the quantized coefficients obtained in the quantization step,
The control step includes:
If the determination step determines that the line of interest does not include a coefficient generated by referring to pixels within the region of interest, the quantization step is performed to calculate each coefficient of the line of interest using the quantization parameter. controlling the quantization, and causing the encoding step to generate encoded data including information specifying the quantization parameter;
If the determination step determines that the line of interest includes a coefficient generated by referring to the pixel in the region of interest, the quantization step includes the coefficients of the line of interest that are generated by referring to the pixels in the region of interest. control to maintain the value of the coefficient generated with reference to the pixel in the region of interest, and quantize the coefficient generated without reference to the pixel in the region of interest using the quantization parameter; For the modification step, the quantized coefficients obtained in the quantization step are modified by inverse quantization using the quantization parameter, and for the encoding step, a quantization step is performed for the line of interest. A method for controlling an image encoding device, the method comprising: generating encoded data including information indicating that the width is "1". A computer program for causing the computer to execute each step of the control method according to claim 5 by being read and executed by a computer.

Images