JPH10294851A - Image data processing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To apply direct image processing to image data that are compressed. SOLUTION: In the case of extracting directly a desired image area from a Wavelet conversion image obtained by Wavelet conversion and compression image data having a zero tree generated based on coefficients of the Wavelet conversion image, an image LL3 with a minimum resolution is decoded and displayed, and the edge of the extracted area is designated, and as to images HL3 , LH3 , HH3 of the same resolution, a pixel whose coordinate corresponds to that of a pixel in an extracted area of the LL3 is decided to be an extracted area. Then the image LL2 with higher resolution by one step is decoded and displayed, a pixel being a descendent of the edge pixel designated by the LL3 is used for an edge object so as to designate an extracted area in the LL2 . As to images HL2 , LH2 , HH2 , the pixel corresponding to the pixel whose coordinate corresponds to that of the pixel designated by the LL2 . Then the images LL2 HL2 , LH2 , HH2 are used to decode and display the LL1 , and similarly the extracted area is designated. Then a zero tree structure of the designated area pixel is extracted and stored.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to digital image processing, and more particularly to a method for directly processing compressed image data obtained by data compression.

[0002]

2. Description of the Related Art Image data has an enormous amount of data, and when storing image data, it is widely performed to reduce the amount of data by performing data compression by an appropriate method. .

In order to perform some processing on a desired area of the data image data which has been compressed, the compressed image data is once expanded and restored to the original image, and the compressed image data is restored. A method has been adopted in which a desired area is extracted and subjected to desired processing, and then data compression is performed again.

However, when compression / expansion is repeated,
There has been a problem that the image deteriorates each time.

An object of the present invention is to solve the above-mentioned problem, and to provide an image data processing method capable of directly extracting a desired area and performing a desired process without expanding data-compressed image data. The purpose is to provide.

[0006]

According to a first aspect of the present invention, there is provided an image data processing method comprising extracting compressed data of a desired region of image data which has been compressed. . Here, it is preferable that the data that has been subjected to the data compression has a wavelet transform image obtained by the wavelet transform and a zero tree created based on the coefficients of the wavelet transform image.

According to a third aspect of the present invention, there is provided an image data processing method comprising two or more compressed image data having a wavelet transformed image obtained by a wavelet transform and a zero tree created based on coefficients of the wavelet transformed image. , Compressed data of a desired area is extracted from each of them, and each extracted image data is replaced with compressed data of a background at a desired position on the composite image.

[0008]

Embodiments of the present invention will be described below. Note that, in the following, multi-resolution analysis is performed on the compressed image data by wavelet transform, and an importance map is created based on the wavelet transform image obtained by the multi-resolution analysis. It is assumed that the data is obtained by subjecting (significant map) to predetermined encoding.

For details of such a data compression method, see, for example, “Jeromu M. Shapiro,” “Embedded Image Codin.
g Using Zerotree of Wavelet Coefficients "IEEE Tran
saction on signal processing, Vol.41, No.12, pp.3445-
3462, December 1993], is roughly as follows.

First, a wavelet transform is performed on the original image data by, for example, the Shapiro method to obtain a wavelet transformed image. This wavelet transform image is composed of several subband images. FIG.
Is a diagram showing an example of the arrangement of sub-band images of a wavelet transform image, wherein an original image is represented by LL ₃ , HL ₃ , LH ₃ ,
HH ₃ , HL ₂ , LH ₂ , HH ₂ , HL ₁ , LH ₁ , H
Shows an example of dividing into sub-band images 10 of H _1.

In order to obtain such a sub-band image, a filter for passing low-frequency components of the original image, a filter for passing high-frequency components in the horizontal direction, a filter for passing high-frequency components in the vertical direction, Four filters, which pass high-frequency components, are operated. Thereby, a sub-band image LL ₁ composed of low-frequency components of the original image (not shown in FIG. 1), a sub-band image LH ₁ composed of high-frequency components in the horizontal direction of the original image, and a vertical height of the original image Four sub-band images are obtained: a sub-band image HL ₁ composed of band components and a sub-band image HH ₁ composed of high-band components oblique to the original image.

[0012] Next, subjected to the same processing for the sub-band image LL _1, (not shown in FIG. 1) subband image LL ₂ formed of a low-frequency component of the sub band images LL _1, subband image LL _The sub-band image LH ₂ composed of the horizontal high-frequency component _{1 and} the sub-band image HL ₂ composed of the vertical high-frequency component of the sub-band image LL ₁ and the sub-band image LL ₁
Obtain four sub-band images of the sub-band images HH ₂ consisting of oblique high-frequency component. Then, further, performs the same processing for the sub-band image LL _2, subband image LL ₃ made of a low-frequency component of the sub band image LL _2, consisting transversely of the high-frequency component of the sub band image LL ₂ The sub-band image HL ₃ , the sub-band image LL composed of the vertical high-frequency components of the sub-band image LH ₃ and the sub-band image LL _2
2 sub-band image HH ₃ composed of high-frequency components in the oblique direction
Are obtained. This series of processing, the original image is as shown in FIG. 1, except for the two sub-band image LL ₁ and LL _2, it will have been divided into sub-band images of a total of 10.

In the example shown in FIG. 1, the sub-band images LL ₁ and LH obtained by the first-stage filtering are
₁ , HL ₁ , and HH ₁ each have half the resolution of the original image, and the sub-band images LL ₂ , LH ₂ , HL ₂ , and HH ₂ obtained by the second-stage filtering have the first resolution. Sub-band images LL ₃ , LH ₃ , and HL which are １ ／ of the sub-band images obtained by the third-stage filtering.
₃ and HH ₃ are half the resolution of the sub-band image obtained by the second-stage filtering.

As is clear from the above description, the sub-band image at each stage is obtained by vertically shifting the sub-band image at the previous stage.
Since it is processing to divide into frequency components in the horizontal and oblique directions, 4
When the two sub-band images are combined by filter processing, the original image can be restored. Specifically, in FIG. 1, four sub-band images LL ₃ and LH
₃ , HL ₃ , and HH ₃ are combined to produce a subband image LL ₂
Can be restored, and the four subband images LL ₂ ,
When LH ₂ , HL ₂ , and HH ₂ are combined, the subband image L
L ₁ can be restored, and the original image can be restored by combining the four sub-band images LL ₁ , LH ₁ , HL ₁ , and HH ₁ .

Although the value of a pixel in each subband image is generally called a wavelet coefficient, it is simply referred to as a coefficient in the following.

This wavelet transform method is also referred to as multi-resolution analysis, which can simultaneously represent image data of a plurality of sub-band images having different resolutions, and compresses each sub-band image to achieve efficient data compression. Is attracting attention as a method that can perform.

Also, since multi-resolution images can be stored and transmitted, they have the progressive nature of the JPEG method. In other words, image data with a small amount of information is sequentially transmitted at a low resolution, and a rough image can be displayed quickly. Therefore, the contents (contents) of the image can be grasped, the transmission can be continued as needed, and the transmission can be performed if unnecessary. Can be stopped.

Further, the DC used in the JPEG system
T (Discrete Cosine Transform)
Then, as you increase the compression ratio, block-like distortion appears,
Although it is unsightly, the wavelet transform method causes less image degradation than the JPEG method even at the same compression ratio. It does not give a strange impression.

As described above, the sub-band image is obtained by performing the wavelet transform. After that, the importance map is created by expressing the coefficients of each sub-band image in a tree. This will be described below.

Incidentally, the coefficients at the same position in each subband image correspond to each other. That is, now, x-coordinate for each sub-band in Fig. 1, assuming that taken in the direction shown the y coordinates, the coefficients of the position coordinates of the sub-band image _{LL 3 (x i, y i} ) is other The coordinates (x _i ,
y _i ). However, the number of pixels at the corresponding coordinate position differs depending on the resolution of the subband image. More specifically, as shown by the small rectangle in FIG. 1, one pixel in the lower-right corner of the LL ₃ is the same resolution H
L _3, but with respect to the LH _3, HH ₃ subband image corresponds to one pixel of the lower right corner, high 1-step resolution HL
₂ , LH ₂ , and HH ₂ correspond to 4 pixels of 2 × 2 in the lower right corner, and have HL ₁ ,
LH ₁ and HH ₁ correspond to 4 × 4 16 pixels at the lower right corner.

From this, in the case shown in FIG. 1, FIG.
As shown in FIG. _Three The coefficient of one pixel of
You can think of a tree structure as a starting point. Toes
LL_Three HL for the coefficient of one pixel with_Three , LH
_Three , HH_Three Of one pixel at the same position in each subband image
Number corresponds, HL_Three To the coefficient of one pixel at that position
For the sub-band image HL_Two 2x2 pixels at the same position
HL correspond_Two 2 × 2 pixels at the position
For the coefficient of subband image HL₁ 4 at the same position
The coefficients of × 4 pixels correspond. Similarly, LH_Three Said
For the coefficient of one pixel at the position, the subband image LH_Two of
The coefficient of 2 × 2 pixels at the same position corresponds to LH_Two of
Subband image for the 2 × 2 pixel coefficient at the position
LH₁ 4 × 4 pixels at the same position.
Furthermore, HH_Three For the coefficient of one pixel at that position
Band image HH_Two 2x2 pixel coefficient at the same position
HH_Two Of the 2 × 2 pixel coefficient at that position
Sub-band image HH₁ Of 4 × 4 pixels at the same position
The numbers correspond.

[0022] In this manner, so that the tree structure by the number of pixels subband image LL ₃ is obtained. This means that the coefficients of the subband image are represented in a tree. In the following, in the tree shown in FIG. 2, the direction of the arrow in the figure is referred to as the descendant side, and the direction opposite to the arrow is referred to as the parent side.

Then, referring to this tree, all the coefficients of the wavelet transform image are converted to 4 by the processing shown in FIG.
Classify by type and assign each code.

First, one coefficient is fetched, and it is determined whether or not this coefficient is important (step S1). This decision
The comparison is performed by comparing the absolute value of the coefficient with a predetermined threshold. If the coefficient is equal to or larger than the threshold, it is determined to be important. The threshold may be any value that can remove the noise component.

If it is determined in step S1 that the coefficient is important, the sign of the coefficient is determined. If the coefficient is positive, a positive code is assigned to the coefficient. If the coefficient is negative, a negative code is assigned. (Step S2).

If the coefficient is not important in step S1, it is determined whether or not the coefficient is a descendant of the zero tree root, that is, whether or not there is a zero tree root when tracing the tree from the coefficient to the parent side. Judgment (Step S
3) If it is a descendant of the zero tree root, the coefficient is not subject to encoding. However, if the coefficient is not a descendant of the zero tree root, it is next determined whether or not the coefficient has significant descendants, that is, whether or not there is a significant coefficient on the descendant side of the coefficient (step S4) If there is an important coefficient on the descendant side, an isolated zero point code is assigned to the coefficient, and if there is no important coefficient on the descendant side, a zero tree root code is assigned to the coefficient. . In addition, as these four kinds of codes, actually, one different alphabetic character is used.

This will be described with an example. For example, in FIG. 2, it is assumed that the coefficient of LL ₃ on the most parent side is important.
It is assumed that attention is paid to the coefficient of HH ₃ which is one of the descendants.
At this time, if the coefficient of HH ₃ is important, a positive or negative code is assigned according to the sign. If the coefficient is insignificant, the parent coefficient L
L ₃ is important, because it is not a zerotree root, NO is determined at decision step S3. If all the coefficients of HH ₂ and HH ₃ which are descendants of the coefficient are not significant, the coefficient is assigned the code of the zero tree root. Otherwise, the coefficient of the isolated zero point is assigned to the coefficient. A code will be assigned. When the code zerotree root is assigned to the coefficient now HH _3, HH ₂ and HH its descendants
All coefficients of ₁ are not significant and will all be discarded. That is, the zero tree root is the end point of one line in the tree.

As described above, it is determined whether or not the coefficient is important by comparing with the threshold value. Further, it is determined whether or not the unimportant coefficient is a zero tree root, and the descendant coefficients of the zero tree root are discarded. This can reduce the amount of data.

By applying the above processing to all the coefficients of the sub-band image, the coefficients discarded when judged as YES in the judgment of step S3, that is, all the coefficients except for the coefficients of the descendants of the zero tree root, , One of the above four types of codes is assigned, and a tree can be created using these codes. This is called a zero tree, and by using this zero tree, image data can be efficiently expressed with a small amount of data. Then, the codes thus obtained are arranged in a predetermined order to create a table. This table is called an importance map.

Data compression can be performed by performing appropriate compression encoding such as entropy encoding only on the coefficient to which the positive code is assigned and the coefficient to which the negative code is assigned. That is, although data compression is not achieved only by creating a subband image by wavelet transform, a zero tree is created based on the subband image, and coefficients to which positive codes are assigned and negative codes are assigned. The data is compressed by performing the compression encoding on the coefficient thus set.

As described above, according to this data compression method, the coefficient to which the positive code of the subband image is assigned and the coefficient to which the negative code is assigned are compression-coded, Is obtained. It should be noted that compression encoding can be performed on the importance map created from the zero tree.

To restore an image from these data, the compression-encoded coefficients are decompressed, the subband image to which each decompressed coefficient belongs and its position are determined based on the zero tree, and the subband image of the subband image is decompressed. The value may be written at the position, and a predetermined value, for example, 0 may be written for the other coefficients. This is because coefficients other than the expanded coefficients are isolated zero points or descendants of the zero tree root or the zero tree root, and are not important coefficients.

The data compression method has been described above.
Next, the present invention relates to a method for directly processing image data compressed by the above-described method. First, a case where a process of extracting a desired region in the compressed image data is performed will be described.

As is apparent from the above description, since the coordinate values of each subband image coincide with the coordinate values of the original image, all coefficients located in the image area to be extracted are cut out with reference to the zero tree. Thus, each sub-band image can be configured for a desired image portion to be extracted.

Now, assuming that a part of the image data composed of the sub-band images as shown in FIG. 1 is to be extracted, a process for extracting only a necessary part of the zero tree by designating an extraction region by appropriate means is performed. Good.

The specification of the extraction area can be performed, for example, by the following method. First, displayed on the display to restore the sub-band image LL ₃ of lowest resolution, the edge of the to be extracted area by specifying a pointing device such as a mouse, performing area specification. Next, for three subband images HL ₃ , LH ₃ , and HH _{3 having} the same resolution, LL
₃ pixels and the coordinates of the extracted area of the stipulated extraction area corresponding pixels. Next, using LL ₃ , HL ₃ , LH ₃ , and HH ₃ , the sub-band image LL ₂ having a high resolution of one step is restored and displayed, and the descendant pixels of the edge pixel designated by LL ₃ are edged. used as a candidate, that specifies the extracted area in the LL _2. Here, as for HL ₂ , LH ₂ , and HH _2, it is assumed that a pixel whose coordinates correspond to the pixel specified by LL ₂ is specified. Then, next, LL ₂ , HL ₂ , L
The sub-band image LL ₁ is restored and displayed using H ₂ and HH _2, and the descendant of the edge pixel designated by LL ₂ is designated as an extraction area edge candidate as an extraction area. even here,
As for HL ₁ , LH ₁ , and HH _1, it is assumed that a pixel whose coordinates correspond to the pixel specified by LL ₁ is specified. When the designation of extraction is completed for all the subband images in this way, the zero tree structure of the designated area pixels is extracted and stored. As a result, all information for reproducing the image of the designated extraction area can be stored.

The extraction processing has been described above. Next, the composition will be described. Here, a case will be described in which a desired portion is extracted from each of two pieces of image data compressed by the above-described method and combined.

First, image data to be a base of a composite image is created. The structure of the underlying image data must be the same as the structure of the two compressed image data to be combined. That is, two pieces of compressed image data having an image to be combined are shown in FIG.
As shown in the following, it is assumed that the image has sub-band images up to the stages of LL ₃ , HL ₃ , LH ₃ , and HH ₃ .
LL ₃ , HL ₃ , LH
₃ , a sub-band image and a zero tree up to the stage of HH ₃ are created.

Next, a desired area is extracted from each compressed image data by the above-described method. Then, each extracted zero tree is replaced with the zero tree at the pasting position in the composite image. As a result, a zero tree of a composite image in which the extracted zero tree of each image is arranged at a desired position can be obtained, and an importance map may be created based on the zero tree.

Although the embodiments of the present invention have been described above, the present invention is not limited to the above embodiments, and various modifications are possible. For example, in the above description, the compressed image data has a wavelet transform image obtained by the wavelet transform, and a zero tree created based on the coefficients of the wavelet transform image. The present invention can be applied not only to compressed image data but also to subband encoded image data. In the above description, the processes of extraction and synthesis have been described, but other image processing can be performed in the same manner.

[Brief description of the drawings]

FIG. 1 is a diagram for explaining a subband image obtained by a wavelet transform.

FIG. 2 is a diagram illustrating a zero tree.

FIG. 3 is a diagram for describing an encoding algorithm of an importance map.

Claims (3)

[Claims] 1. An image data processing method, comprising extracting compressed data of a desired area of data image data which has been compressed. 2. The data compressed image data includes a wavelet transform image obtained by a wavelet transform and a zero tree created based on coefficients of the wavelet transform image. Item 2. The image data processing method according to Item 1. 3. Compressed data of a desired area is respectively selected from two or more compressed image data having a wavelet transformed image obtained by the wavelet transform and a zero tree created based on coefficients of the wavelet transformed image. An image data processing method comprising extracting, and replacing each extracted image data with background compressed data at a desired position on a composite image.