JP4701448B2 - Region of interest encoding method - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention assigns a coding bit rate to a specific region of interest such as a person or an object in the entire image, thereby maintaining the image quality of the region of interest higher than that of other regions. It relates to the conversion method.
[0002]
[Prior art]
As a method of efficiently encoding an image, the image quality of the region of interest is compared with other regions by preferentially allocating the encoding bit rate to a specific region of interest such as a person or object in the entire image. There is a method of dividing the ROI (Region of Interest) and the non-ROI of unimportant information such as the background.
[0003]
Typical examples of this method include a Maxshift method (first conventional example) and an EBCOT (Embedded Block Coding with Optimized Truncation) as a bit plane method.
[0004]
The Maxshift method (first conventional example) designates an ROI part in an arbitrary form, and reversibly compresses the part while irreversibly compresses the non-ROI part. Specifically, first, a known wavelet transform is performed on the original image to obtain the distribution of wavelet coefficients as shown in FIG. 13, and among these distributions, the largest coefficient distribution is conceived for the non-ROI portion. The value Vm of the wavelet coefficient is obtained in advance. Then, the number of bits S such that S> = max (Vm) is obtained, and only the wavelet coefficient of the ROI part Ar1 is shifted by S bits in the increasing direction as shown in FIG. For example, if the value of Vm is “255” in decimal (ie, “11111111” in binary), S = 8 bits, and the value of Vm is “128” in decimal (ie, binary). Similarly, in the case of “10000000”), S = 8 bits. In this case, the wavelet coefficient of the ROI part Ar1 is shifted by S = 8 bits in the increasing direction as shown in FIG. Become. As a result, the ROI portion Ar1 can be set to a lower compression rate than the non-ROI portion, and thus it is possible to obtain lossless compressed data for the ROI portion Ar1. According to this method, there is no need to obtain definition information such as the shape of the ROI portion in advance on the decoding side, and it is convenient because reversible decoding of the ROI portion can be performed simply by decoding as it is. .
[0005]
The bit plane EBCOT (second conventional example) divides an image into a plurality of rectangular blocks B as shown in FIG. 15, and compresses each rectangular block B with a priority of the bit rate. . Therefore, according to EBCOT, image compression with a bit rate higher than that of other rectangular blocks B can be performed for some rectangular blocks B.
[0006]
[Problems to be solved by the invention]
In the Maxshift method (first conventional example), the lossless compression is performed in the ROI part, so there is a certain limit in the compression rate of the part, and the size of the compressed image file must be relatively large as a whole. I didn't get it. Conversely, in order to reduce the size of the image file compressed by increasing the compression rate in the Maxshift method, the bit rate of the non-ROI portion must be greatly reduced, and therefore the image quality of the non-ROI portion is the same as that of the ROI portion. It will be extremely poor compared to the image quality.
[0007]
On the other hand, in EBCOT, since the bit rate is changed for each rectangular block B, definition information such as the coordinates, size, and boundary of the block is required in advance on the decoding side.
[0008]
Also, in EBCOT, the ROI part was specified only as the rectangular block B, so it is not possible to define the ROI part of an arbitrary shape. For example, when it is desired to increase the bit rate of only the human face in the background In such a case, the processing becomes extremely difficult.
[0009]
Therefore, an object of the present invention is to efficiently encode an image by irreversibly compressing an ROI portion having an arbitrary shape, and defining the shape and size of the ROI portion in advance on the decoding side. It is an object of the present invention to provide a region-of-interest encoding method that does not require information.
[0010]
[Means for Solving the Problems]
In order to solve the above problems, a first aspect of the present invention, a first step of the wavelet transform of the original image, a plurality of mask areas of any shape specified for the original image, the wavelet transform A second step of generating a mask signal by developing on a corresponding wavelet plane, and a third step of assigning a code amount for compression to the plurality of mask regions and a non-mask region other than the plurality of mask regions, respectively A fourth step of performing quantization and encoding according to the code amount allocated in the third step, and when the second step overlaps portions related to different mask regions developed in the wavelet plane , the overlapped part to generate a masked signal as a part related to high priority mask area, the fourth step, the plurality of mask areas and the unmasked region Performed individually the quantization and the encoding, and generates a respective compressed data.
[0012]
The invention according to claim 2 is the region-of-interest coding method according to claim 1, wherein the fourth step corresponds to a wavelet coefficient of the original image wavelet transformed in the first step to the mask signal. Quantization and encoding are performed on each wavelet coefficient related to the plurality of mask regions taken out and the wavelet coefficient related to the non-mask region .
[0014]
DETAILED DESCRIPTION OF THE INVENTION
FIG. 1 is a flowchart showing a region of interest encoding method according to one embodiment of the present invention. This region-of-interest encoding method will be described along this flowchart.
[0015]
<Compression method>
The image compression is executed according to the flow on the compression device Co side in FIG.
[0016]
As shown in FIG. 1, the compression apparatus Co first performs wavelet transform on digital image data 1 that is an original image. The wavelet transform is a conversion method that performs coding compression by dividing an image into a low-pass side and a high-pass side using a two-part filter bank. Here, as a typical method of the wavelet transform, due to the difference in the form of the wavelet packet tree, the mallat type shown in FIG. 2 and FIG. 3, the scale type shown in FIG. 4 and FIG. A plurality of types such as the packet type shown in FIG. Here, one of these is selected and wavelet transformation is performed. 2, 4, and 6 illustrate one-dimensional wavelet transform modes, and FIGS. 3, 5, and 7 illustrate two-dimensional wavelet transform modes. However, the development hierarchy of the wavelet transform is not limited to the number of hierarchies shown in each figure and is arbitrary.
[0017]
Here, the mallat type shown in FIG. 2 and FIG. 3 has a low frequency component (low-pass component: refer to an image shown by a relatively small block in FIG. 2 and a path shown by “L” in FIG. 3). Low pass under the assumption that it contains more information than high frequency components (high pass component: see image shown in relatively large blocks in FIG. 2 and path shown as “H” in FIG. 3) This is a system that repeats only the filter. On the other hand, since the wavelet packet base can correspond to an arbitrary binary tree structure, in the scale type shown in FIGS. 4 and 5, a low frequency component (low-pass component: indicated by a relatively small block in FIG. 4). In addition to the image and the path indicated by “L” in FIG. 5, high frequency components (high pass component: the image indicated by the largest block in FIG. 4 and “H” in the first branch in FIG. 5) Also in the path shown), only one stage is developed for a lower frequency component (low-pass component: “L”) and a higher frequency component (high-pass component: “H”). Further, in the packet type shown in FIGS. 6 and 7, the frequency component (low-pass component: “L”) and the high frequency component (high-pass component: “H”) are developed in all branches as in the short-time Fourier transform. And adopts a complete tree structure. Here, which of these types is selected is based on a rate determined based on cost and capacity, and a type that minimizes the restoration distortion is selected.
[0018]
Next, in the compression apparatus Co, a mask signal 3 for designating an important part as an ROI part is given to the wavelet-transformed image data 2. For example, in the human image data (original image) 1 as shown in FIG. 8, when only the face portion below the forehead is set as the ROI portion, a single mask area 4 is set as shown in the white portion in FIG. This is designated as the ROI part. The mask area 4 can be specified corresponding to the original image 1 using a so-called pointing input device such as a mouse while viewing the original image 1 on the screen of the display device.
[0019]
Although FIG. 9 shows an example in which a single ROI portion is specified for the original image 1, a plurality of regions may be specified as the ROI portion. These are defined by different mask signals 3. Note that the remaining portion obtained by removing all mask regions 4 for all ROI portions becomes a non-ROI portion 5 (FIG. 9).
[0020]
Next, when a plurality of mask signals 3 are given as shown in FIG. 1, priority is given to the plurality of mask signals 3. The higher the priority, the higher the amount of information, for example the bit rate, and the less the loss during expansion. As priorities at this time, designation is performed in ascending order of numbers such as “1”, “2”.
[0021]
Then, corresponding to the wavelet transform of the type selected as described above (mallat type / scale type / packet type), the mask region 4 is developed on the wavelet plane to generate the mask signal 3.
[0022]
Here, the method of converting the mask signal into a portion corresponding to the wavelet plane depends on the number of taps of the wavelet transform filter.
[0023]
For example, as shown in FIG. 11, a reversible 5 × 3 filter (a filter in which the number of taps on the decomposition-side low-pass filter is 5 taps and the number of taps on the decomposition-side high-pass filter is 3 taps) in the wavelet transform arithmetic processing is used. Assuming that the even-numbered (2n-th) pixel data of the original image 1 is designated as the ROI portion, the n-th data of the low-pass filter (low-frequency side) 7 and the high-pass filter (high Assuming that the (n-1) th and nth data of (region side) 8 are ROI portions, the mask signal is developed on the wavelet plane. When the odd-numbered (2n + 1) -th pixel data of the original image 1 is designated as the ROI portion, the n-th and (n + 1) -th data of the low-pass filter (low-frequency side) 7 and the high-pass filter ( Assuming that the (n−1) th, nth and (n + 1) th data of the (high frequency side) 8 are ROI portions, the mask signal is developed on the wavelet plane. FIG. 11 shows only the correspondence between the original image 1 and the wavelet plane of the first layer, but the same recursive expansion is performed for the expansion of deeper layers.
[0024]
Or, for example, as shown in FIG. 12, in the wavelet transform calculation process, a Deubechies 9 × 7 filter (the number of taps of the low-pass filter on the decomposition side is 9 taps and the number of taps of the high-pass filter on the decomposition side is 7 taps) ) Is applied, when the even-numbered (2n-th) pixel data of the original image 1 is designated as the ROI portion, the (n−1) -th, n of the low-pass filter (low-frequency side) 7 The (n + 1) th data and the (n-2) th, (n-1) th, nth and (n + 1) th data of the high-pass filter (high frequency side) 8 are assumed to be ROI portions. The mask signal is developed on the wavelet plane. When the odd-numbered (2n + 1) -th pixel data of the original image 1 is designated as the ROI portion, the (n−1) -th, n-th, (n + 1) -th of the low-pass filter (low-frequency side) 7 And the (n + 2) th data and the (n-2) th, (n-1) th, nth, (n + 1) th and (n + 2) th data of the high-pass filter (high frequency side) 8 are ROI portions. The mask signal is developed on the wavelet plane. FIG. 12 shows only the correspondence between the original image 1 and the wavelet plane of the first layer, but the same recursive expansion is performed for the expansion of deeper layers.
[0025]
In the low-pass filter (low-frequency side) 7 and the high-pass filter (high-frequency side) 8, the correspondence relationship in FIG. 11 and FIG. The part where the ROI part overlaps with the other pixel data of the image 1 is assumed to be the ROI part, and the mask signal is developed on the wavelet plane.
[0026]
A white portion 4a in FIG. 10 is a region (hereinafter referred to as a “development mask region”) 4 in which the mask region (ROI portion) 4 is developed on a mallat wavelet plane as described above. A mask signal 3 corresponding to 4a is generated and applied to the wavelet transformed image data 2. Reference numeral 5 a in FIG. 10 indicates a region where the non-ROI portion 5 is developed on the wavelet plane (hereinafter referred to as “development non-mask region”). In a portion where the mask regions (ROI portion) 4 overlap each other and a portion where the mask region (ROI portion) 4 overlaps with the non-ROI portion 5, whichever is higher in priority is set as the mask region (ROI portion) 4. The lower one is processed as the non-ROI portion 5.
[0027]
In FIG. 1, the wavelet coefficients of the image data 2 subjected to wavelet transformation are extracted in association with the respective mask signals 3.
[0028]
Next, priorities are assigned to the mask regions 4a developed on the wavelet plane as shown in FIG. As described above, when a plurality of ROI portions 4 are set for the original image 1, the number of development mask regions 4a developed on the wavelet plane for each of the plurality of ROI portions 4 is set. The same priority order is assigned to the mask areas 4a corresponding to each other, and finally the priority order is set for all the development mask areas 4a.
[0029]
Here, when a plurality of ROI portions 4 are set for the original image 1, a plurality of development mask regions 4 a overlap each other in a portion that has passed the low-pass filter (low frequency side) 7 on the wavelet plane. There can be. In this case, the priority order of the overlapping portions is determined as the development mask region 4a having the higher priority among the plurality of overlapping development mask regions 4a.
[0030]
Then, in the image data 2 subjected to wavelet transformation, the wavelet coefficients of the unmasked unmasked area 5a that are not applied to any mask signal are extracted. In this case, the priority of the development non-mask area 5a is lower than that of all the development mask areas 4a (that is, a larger number is given in ascending order of the numbers), and the wavelet coefficient has a value of “0”, for example. Adopted.
[0031]
In this way, the wavelet coefficients of the plurality of ROI parts 6a and 6b are output. The output data of the wavelet coefficients of the ROI portions 6a, 6b... And the unmasked unmasked area 5a will be referred to as “ROI signal” below.
[0032]
Next, according to the priorities set in advance, bit amounts are allocated to the ROI portions 6a and 6b and the expanded non-mask area 5a, respectively.
[0033]
As a bit amount allocation method at this time, the information amount of the ROI portions 6a and 6b, for example, the bit amount is set to a value that does not satisfy the bit amount necessary for lossless compression. Specifically, for example, first, the compression rate bit amount of the entire image is determined, and among them, a predetermined amount of bit amount is sequentially assigned in descending order of the priority of each ROI portion 6a, 6b. A first method for allocating the bit amount to the expanded non-mask area 5a, and a second method for directly determining the bit amount at a predetermined ratio according to the priority of each ROI portion 6a, 6b,. However, any method is selected in advance, and the bit amount of each ROI portion 6a, 6b... And the unmasked area 5a is determined according to the selected method. At this time, as described above, the bit amount of the ROI portions 6a and 6b is set to a value less than the bit amount necessary for the lossless compression. It should be noted that a mode in which the bit amount of the ROI portions 6a and 6b is assigned to be greater than or equal to the bit amount necessary for the lossless compression and a mode in which the bit amount necessary for the lossless compression is set to a value less than the bit amount may be selected.
[0034]
Then, the ROI signals in the ROI portions 6a, 6b,... And the unmasked area 5a are quantized to generate binarized data 10a, 10b,. As a method of this quantization processing, binarization may be performed by a method similar to a bit plane encoding method such as EBCOT or SPIHT (Image Compression with Set Partitioning in Hierarchical Trees).
[0035]
Then, entropy coding is performed on each of the binarized data 10a, 10b... 10z using a predetermined method such as arithmetic coding such as MQ coder or QM coder or Huffman coding.
[0036]
In this way, compressed data 11a, 11b,... 11z for each ROI portion 6a, 6b,.
[0037]
Then, the compressed data 11a, 11b,... 11z are arranged in order according to the priority order, and are sent to a predetermined path, for example, or recorded on a predetermined recording medium such as a hard disk drive.
[0038]
<Decoding method>
The decoding of the image is executed in the flow on the decompression device Ex side in FIG.
[0039]
In the decompression device Ex, as shown in FIG. 1, the compressed data 11a, 11b,... 11z are entropy decoded according to a predetermined method, and binarized in the order in which they are sequentially arranged (that is, the order of priority of the bit rate). Generated data 21a, 21b... 21z.
[0040]
Next, the binarized data 21a, 21b,... 21z are inversely quantized (multi-valued) to generate a plurality of ROI signals 22a, 22b,. At this time, inverse quantization (multi-value quantization) is performed in the order sequentially arranged (that is, the order of higher bit rate priority), and a plurality of ROI signals 22a, 22b,. Generate in high order.
[0041]
Then, on the wavelet plane, wavelet-transformed wavelet coefficients are synthesized for each ROI signal 22a, 22b,. At this time, since the priority of each ROI signal 22a, 22b ... 22z is the order in which the data is sent (that is, the order in which the data is arranged), the wavelet coefficients are sequentially synthesized in accordance with the given order, Wavelet transformed image data 23 is formed. If overlapping portions are generated between the plurality of ROI signals 22a, 22b,... 22z, the higher priority order is selected. If the wavelet coefficients of the non-ROI portion 5 are set to “0”, the values of the ROI signals 22a, 22b,... 22z corresponding to the ROI portions 6a, 6b. Just do it.
[0042]
Finally, the inverse wavelet transform is performed to restore the image data 24 in accordance with the wavelet transform type (mallat type / scale type / packet type).
[0043]
As described above, since the irreversible compression is performed on the ROI portions 6a, 6b... Having an arbitrary shape converted into the portion corresponding to the wavelet plane, the Maxshift method (first conventional example) in which the lossless compression has been performed. ), The size of the compressed image file as a whole can be reduced. .. Can be designated as compared with EBCOT (second conventional example). For example, when it is desired to increase the bit rate of only the face of a person in the background, the processing is not performed. It becomes easy.
[0044]
【The invention's effect】
According to the onset bright, as compared to the EBCOT (Second conventional example), to specify an ROI portion of any shape, such as in order to enhance the bit rate of only the face of the person in the background, the process Becomes easier.
[0045]
Furthermore, according to the present invention, even when a plurality of mask regions having an arbitrary shape are designated, it is possible to easily perform irreversible compression on each mask region. In this case, there is an effect that it is possible to freely set the assigned code amount by individually assigning the code amount to each mask region.
[Brief description of the drawings]
FIG. 1 is a flowchart showing a region-of-interest encoding method according to an embodiment of the present invention.
FIG. 2 is a diagram showing a mallat-type wavelet packet tree;
FIG. 3 is a diagram showing a mallat-type wavelet plane;
FIG. 4 is a diagram illustrating a space type wavelet packet tree;
FIG. 5 is a diagram showing a space-type wavelet plane;
FIG. 6 is a diagram illustrating a packet-type wavelet packet tree.
FIG. 7 is a diagram showing a packet type wavelet plane;
FIG. 8 is a diagram illustrating an example of an original image.
9 is a diagram showing a single mask area set for the original image of FIG. 8. FIG.
10 is a diagram showing a state in which the mask region of FIG. 9 is developed on a mallat wavelet plane. FIG.
FIG. 11 is a diagram illustrating a correspondence relationship between a low-frequency side and a high-frequency side in the inverse wavelet 5 × 3 filter and a mask region between the input side and the input side.
FIG. 12 is a diagram illustrating a correspondence relationship of a mask region between a low frequency side and a high frequency side and an input side in an inverse wavelet 9 × 7 filter.
FIG. 13 is a diagram illustrating a distribution of wavelet coefficients after a known wavelet transform is performed on an original image.
FIG. 14 is a diagram showing a distribution of wavelet coefficients in the first conventional example.
FIG. 15 is a diagram for explaining the concept of EBCOT of the second conventional example.
[Explanation of symbols]
1 Image data (original image)
10a to 10z Binary data 11a to 11z Compressed data 2 Wavelet transformed image data 21a to 21z Binary data 22a to 22z ROI signal 3 Mask signal 4, 4a Mask area 5, 5a Non-ROI portion 6a , 6b ... ROI partial Co compression device Ex expansion device

Claims (2)

A first step of wavelet transforming the original image;
A second step of generating a mask signal to deploy multiple mask areas of any shape specified for the original image, the wavelet plane corresponding to the wavelet transform,
A third step of assigning a code amount when compressing each of the plurality of mask regions and a non-mask region other than the plurality of mask regions;
A fourth step of performing quantization and encoding according to the code amount allocated in the third step;
With
In the second step, when the portions related to different mask regions developed in the wavelet plane overlap each other, a mask signal is generated with the overlapping portion as a portion related to the mask region having a high priority,
The region-of-interest encoding method, wherein the fourth step performs the quantization and the encoding individually on the plurality of mask regions and the non-mask region, and individually generates compressed data.   The region of interest encoding method according to claim 1,
  In the fourth step, the wavelet coefficients of the original image wavelet transformed in the first step are quantized into wavelet coefficients related to a plurality of mask areas extracted in association with the mask signal and wavelet coefficients related to a non-mask area. And a region of interest encoding method, wherein encoding is performed.

Images