KR20070101632A - Method for encoding stereo residual image and apparatus thereof - Google Patents

Abstract

A method for encoding a stereo residual image and an apparatus thereof are provided to change an information storage order according to the characteristic of a stereo residual image, thereby restoring a stereo image with high quality in a lesser bit rate. A method for encoding a stereo residual image comprises the following steps of: performing the wavelet conversion of the residual image of a stereo image and computing the edge tendency of a wavelet subband of the converted image(300,310); setting a threshold value from the computed edge tendency(320); performing the dominant pass of the wavelet subband when the set threshold value is higher than the absolute value of the wavelet subband(330); and applying subordinate pass to the dominant passed subband until a difference between the dominant passed subband coefficient and the coefficient of the stereo image is smaller than a difference between the coefficient of a subband not dominant passed and the coefficient of the stereo image(340).

Description

Stereo residual image encoding method and apparatus therefor {Method for encoding stereo residual image and Apparatus approx}

1 illustrates subbands according to wavelet transform.

2 illustrates a stereo residual image encoding apparatus according to the present invention.

3 is a flowchart of a stereo residual image encoding method according to the present invention.

4 is a detailed flowchart of FIG. 3.

5A shows the original image.

5B illustrates a stereo image according to binocular difference estimation.

5c illustrates a stereo image to which the conventional embedded wavelet encoding method is applied.

5d illustrates a stereo image to which the embedded wavelet encoding method according to the present invention is applied.

The present invention relates to video encoding, and more particularly, to a stereo residual video encoding method.

With the recent development of display technology, it becomes possible to realistically view three-dimensional images and three-dimensional videos. Among the images with three-dimensional information, the stereo imaging system is widely used because it can express three-dimensional with only two images on the left and the right. Of course, stereo images do not represent all spatial information of objects in three-dimensional space. However, in the case of humans, three-dimensional images can be recognized by the difference between the left eye and the right eye, and the stereo image system is composed of the left and right images, so that the human can feel the three-dimensional image.

In order to display one 3D image, a stereo image of left and right is required. However, if two images are compressed independently of each other, twice as much storage space as a conventional two-dimensional image is compressed. Similarly, the transmission requires twice as much communication bandwidth as the conventional two-dimensional image. However, since stereo images have similar information, the compression ratio of images can be improved by using them.

Since stereo images were taken at different locations at the same time, a lot of information overlaps between the two images on the left and right. Therefore, it is possible to increase the compression efficiency by removing this redundancy. In the case of stereo images, the redundancy between the left image and the right image has many characteristics similar to the time redundancy in the conventional two-dimensional video, and such redundancy can be eliminated by using a disparity estimation method. .

In the related art, a binocular difference estimation method is widely used as a method for encoding a conventional general image for encoding a residual image.

 Disparity estimation is performed by independently encoding a left image as a reference image and then predicting the right image in block units from the coded left image. However, this method does not seem to reflect the characteristics of the stereo residual image. That is, the stereo residual image has a value close to zero, but includes a relatively large number of vertical edge components due to the left and right position difference of the stereo camera. In addition, the degree of distribution is different depending on the performance of the stereo image encoder and the characteristics of the input image. Nevertheless, even though there is a difference in the performance of the video encoder and the characteristics of the input video, it does not properly reflect the characteristics of the stereo residual video because the video is uniformly encoded.

Accordingly, the conventional invention has a problem in that proper coding according to the characteristics of an image is not possible using a uniform coding sequence even though the distribution of the stereo image encoder is different depending on the performance of the stereo image encoder and the characteristics of the input image.

Therefore, the first technical problem to be achieved by the present invention is to provide a stereo residual image encoding method for assigning an appropriate encoding order according to each stereo residual image by reflecting the characteristics of the stereo image.

Another object of the present invention is to provide a stereo residual image encoding apparatus using the stereo residual image encoding method.

The present invention to achieve the first technical problem,

Wavelet transforming the residual image of the stereo image, and calculating edge tendency of the wavelet subband of the converted image; Setting a threshold from the computed edge trend diagram; Main-coding the wavelet subband when the absolute value of the coefficient of the wavelet subband is greater than the set threshold value; And the main coded until the difference between the main coded subband coefficients and the coefficients of the stereo picture is smaller than the difference between the uncoded subband coefficients corresponding to the main coded subband coefficients and the coefficient of the stereo picture. Provided is a stereo residual image encoding method comprising applying dependent encoding to a subband.

The present invention to achieve the second technical problem,

A tendency calculator configured to wavelet transform a residual image of the stereoscopic image and calculate edge tendencies of the wavelet subbands of the converted image; A threshold setting unit for setting a threshold value from the calculated edge tendency; A main encoder for main encoding the wavelet subband when the coefficient size of the wavelet subband is greater than the set threshold value; And the main coded until the difference between the main coded subband coefficients and the coefficients of the stereo picture is smaller than the difference between the uncoded subband coefficients corresponding to the main coded subband coefficients and the coefficient of the stereo picture. Provided is a stereo residual image encoding apparatus including a dependent encoding unit applying dependent encoding to a subband.

The combined method of wavelet transform, zero tree encoding, and embedded encoding is called embedded zero tree wavelet coding (EZW). When discrete residual wavelet transform (DWT) of the stereo residual image is performed, each subband is highly correlated in the horizontal, vertical, and diagonal directions. Digital data can be divided into significant bits and non-critical bits. The most significant bit is toward the most significant bit (MSB) and the least significant bit is towards the least significant bit (LSB). There are various examples of this. In the case of binary numbers in which a sign bit exists, determining the sign becomes the most significant bit. If you encode some values from the most significant bit, it will be an efficient encoding scheme. This coding scheme is called bit plane coding.

The embedded zero-tree wavelet coding (EZW) is an encoding method capable of encoding up to a desired bit rate, and receiving an encoded bitstream from a decoder and enabling optimal decoding up to that state even if it is interrupted at any moment. This feature enables progressive transmission of the image. The embedded zerotree wavelet coding (EZW) first performs zerotree coding to determine the importance of each coefficient and then performs embedded encoding on the coefficients determined as the significant coefficient. However, due to the correlation of ZeroTree's parental descendant coefficients, all bands are shuffled in the process of encoding one bit plane of wavelet transformed coefficients, so that all bands are shuffled in the encoded bit stream. do. Therefore, the existing embedded zero-tree wavelet coding (EZW) using ZeroTree cannot effectively restore multiple resolutions.

Therefore, the present invention transforms and applies an embedded zero-tree wavelet coding (EZW) method, which is widely used for general video encoding. Modified embedded zero-tree wavelet coding (EZW) for stereo residual images transformed by discrete-time wavelet transform (DWT), which orders the information contained in the subordinate list according to the direction and importance of edge components. It rearranges and increases the number of times the information is encoded so that important information is stored first.

Hereinafter, the present invention will be described in detail with reference to the accompanying drawings.

1 illustrates subbands according to wavelet transform.

If the coefficients of the low resolution subbands in the wavelet tree structure of the image are called parents, the coefficients in the higher resolution subbands are descendants. 2, one coefficient in HL3 is associated with a coefficient of 2x2 in HL2 and a coefficient of 4x4 in HL1. That is, the coefficients of each subband represent the same position in an image. However, as the level goes up, the number of coefficients decreases because down sampling is performed. Therefore, if the coefficient value is small at the upper level, the value can be expected to be low at the lower level.

의 순서대로 검색을 한다. 하지만 이는 영상의 특성을 반영하여 부호화 한 것이 아니라 영상의 특성과는 상관없이 일률적으로 부호화 한것이다.In general, since the energy is concentrated in the low frequency part, efficient encoding is possible using the conventional embedded zero-tree wavelet coding (EZW) method. In the conventional main encoding process, the wavelet transform coefficients are searched in order from the lowest to the highest and sequentially from left to right in the band. In other words,

Search in order. However, it is not encoded by reflecting the characteristics of the image, but uniformly encoded regardless of the characteristics of the image.

However, in the present invention, the edge tendency of the subbands of the wavelet tree is calculated, the threshold value is calculated, and the subbands having the relatively strong edge components are preferentially encoded according to the characteristics of the image. There is a characteristic.

2 illustrates a stereo residual image encoding apparatus according to the present invention.

First, the wavelet converter 200 wavelet converts the input stereo residual image. The input image is decomposed into a plurality of subbands through a multi-resolution wavelet transform in the wavelet transform unit.

The edge tendency calculator 210 calculates an edge tendency of each wavelet subband of the wavelet transformed image. Most stereo residual images have a value close to zero, but due to the difference in the left and right positions of the stereo camera, the stereo image includes a relatively large number of edge components in the vertical direction. In addition, since the distribution degree varies depending on the performance of the stereo video encoder and the characteristics of the input video, the edge tendency of each wavelet subband is encoded to be encoded according to the characteristics of the video rather than using a conventional conventional coding sequence. compute edge tendency.

The threshold setting unit 220 sets a threshold value for determining whether the main coder 230 has an important coefficient by applying a value to the edge calculation degree of the wavelet subband calculated by the edge tendency calculator.

The main encoder 230 determines whether or not to encode each wavelet subband by applying the threshold set by the threshold setting unit 220. If the coefficient of each wavelet subband is greater than or equal to the threshold, it becomes a significant coefficient and is major coded. In the main encoder 230, values having relatively strong edge components are first encoded.

Compare the symbols of the subbands that had the main coded strong edge tendency and the symbols of the corresponding subbands that did not have the edge tendency, The dependent encoding unit 240 performs main encoding until the difference between the coefficient value and the coefficient value of the subband of the original image is smaller than the difference between the corresponding coefficient value of the unencoded subband and the coefficient value of the original image. Subordinate pass is repeated only for subbands with strong edge components. The dependent encoder 240 performs the dependent encoding only on the coefficient determined as the significant coefficient by the main encoder 230. That is, the coefficient determined by the main encoding unit 230 as an important coefficient is stored in the dependent encoding list. Significant coefficients stored in the dependent coded list already know the sign information through main coding. The dependent encoder 240 encodes a difference value between the original coefficient value and the quantized value by using the already known sign information.

The feed unit 250 determines whether a target bit rate, that is, a target bit rate, of the image passed through the main encoder 230 and the dependent encoder 240 is reached. Coding stops when the target bit rate is reached, otherwise the scaling factor value that determines the threshold is halved. After that, the stereo residual image is repeatedly passed through the main encoder 230 and the dependent encoder 240, and when the predetermined target bit string is reached, the feed unit finishes encoding and outputs the bit string of the encoded image. With this feature, the receiving side can reconstruct the image even if the reception of the bit string stops in the middle.

3 is a flowchart of a stereo residual image encoding method according to the present invention.

의 여러 개의 부대역으로 분해된다.First, the stereo residual image is input, and the image is wavelet transformed (step 300). In other words, the input image undergoes multi-resolution wavelet transformation

It is broken down into several subbands.

The edge tendency of the wavelet subbands of the converted image is calculated (step 310). The process of calculating the edge tendency is a process of determining which band contains important information among corresponding subbands.

With this value, a threshold value for determining whether a significant coefficient of the dominant pass is determined is set (step 320).

Through this calculation process, the value of the wavelet subband having the strong edge component is major coded (step 330).

If the value of the wavelet subband is greater than or equal to the threshold, a symbol of POS (positive) or NEG (negative) is generated according to the sign of the coefficient, and the coefficient becomes a significant coefficient and added to the dependent encoding list to improve encoding accuracy. . Through this process, values with strong edge components are first encoded in the dominant pass.

Subordinate pass is applied only to the subband having the main coded strong edge component (step 340). The dependent encoding process is performed only on coefficients determined as significant coefficients after the main encoding process is completed. Coefficients determined as significant coefficients during the main encoding process are stored in the dependent encoding list. The dependent encoding process is a process of encoding a difference value between the original coefficient value and the quantized value by using known code information. The magnitude encoding process compares the magnitude of the original coefficient with the magnitude of the reconstructed coefficient.

In order to compensate for the deterioration of the quality of the image, the symbols of the subbands having the main coded strong edge tendency are compared with the subbands having the edge tendency that do not. Strongly encoded edge component until the difference between the coefficient value of the band and the coefficient value of the subband of the stereo picture is smaller than the difference between the coefficient value of the non-main coded subband and the coefficient value of the subband of the original picture. Only the subband having a repeating pass (subordinate pass) process is repeated.

4 is a detailed flowchart of FIG. 3.

의 여러 개의 부대역으로 분해된다.First, the stereo residual image is input and wavelet transform is performed on the image (step 400). In other words, the input image undergoes multi-resolution wavelet transformation

It is broken down into several subbands.

The edge tendency of the wavelet subbands of the converted image is calculated (step 410).

는 에지 경향도(edge tendency),

는 대응하는 서브밴드의 에너지,

은 서브밴드의 레벨,

는 해당하는 레벨

의 임의의 서브밴드를 나타내며, (1)(2)(3)은 각각

의 서브밴드를 나타낸다. 즉, 서브밴드

는 수직 에지를 나타내며,

은 수직 에지를 나타내고,

는 웨이브릿 변환된 영상의 우측하단에 위치한다.here

Is the edge tendency,

Is the energy of the corresponding subband,

Is the level of the subband,

Is the corresponding level

Denotes any subband of (1) (2) (3), respectively

Represents a subband of. That is, subband

Represents the vertical edge,

Represents a vertical edge,

Is located at the lower right of the wavelet transformed image.

With this value, a threshold value for determining whether a significant coefficient of the dominant pass is set is set (step 420).

는 임계값,

은 웨이브릿의 레벨,

는 웨이브릿의 방향을 나타내며

는 임계값을 설정하는 스케일링 팩터(scaling factor)를 나타내며 본 발명에서는 1로 설정한다. 즉,

은

레벨에서의

방향 웨이브릿 밴 드의 임계값

를 의미한다.In Equation 2 above

Is the threshold,

Is the level of the wavelet,

Indicates the direction of the wavelet

Denotes a scaling factor for setting a threshold and is set to 1 in the present invention. In other words,

silver

At the level

Threshold for Directional Wavelet Band

Means.

Through this calculation process, the value of the wavelet subband having the strong edge component is coded (step 430).

If the value of the wavelet subband is greater than or equal to the threshold, a symbol of POS (positive) or NEG (negative) is generated according to the sign of the coefficient, and the coefficient becomes a significant coefficient and added to the dependent encoding list to improve encoding accuracy. . However, if the value of the wavelet subband is less than the threshold, the existence of coefficients whose magnitude is greater than or equal to the threshold among the children remaining in the main list is examined. If present, it generates IZ (isolated zero), and if not, it generates zero-tree root (ZTR) symbol. This is the same as the conventional embedded zerotree wavelet coding (EZW). However, a significant difference between the conventional embedded zerotree wavelet coding (EZW) and the present invention is that coefficients of subbands determined as important coefficients in the dominant pass are preferentially encoded in the bit string of the residual image.

That is, through this process, values having a strong edge component are first encoded in the main encoding process (430). However, the difference between the values stored in the dependent encoding list composed of the important coefficients and the original image tends to be larger for the important coefficients having relatively strong edge components.

Dependent encoding is performed only on coefficients determined as significant coefficients after the main encoding process (step 430) (step 440). Coefficients determined as significant coefficients in the main encoding process are stored in the dependent encoding list. Significant coefficients stored in the dependent encoding list are already known through the main encoding process. The dependent encoding process 440 is a process of encoding a difference value between a value of an original coefficient and a quantized value using known code information.

On the other hand, some of the significant coefficients stored in the dependent encoding list consequently degrade the image quality compared to conventional embedded zerotree wavelet (EZW) coding. Therefore, in order to compensate for the deterioration of the image quality, the symbols of subbands having the main coded strong edge tendency are compared with those of the subbands having the edge tendency that do not. Main coded until the difference value between the coefficient value of the coded subband and the subband coefficient value of the original picture is smaller than the difference between the coefficient value of the non-main coded subband and the subband coefficient value of the original picture Only the subband having a strong edge component is repeated a subordinate pass (step 450).

가 상대적으로 강한 에지 경향도(edge tendency)를 가지는 서브밴드라 할 경우,

밴드값과 원영상의 이에 해당하는 밴드값간의 차가 대응되는

밴드값과 원영상의 이에 해당되는 밴드값간의 차보다 작아질 때까지 종속 부호화 과정을 반복하는 것이다.E.g

Is a subband having a relatively strong edge tendency,

The difference between the band value and the corresponding band value of the original image

The dependent encoding process is repeated until the difference between the band value and the band value corresponding to the original image is smaller.

After the primary main encoding process 430 and the dependent encoding process 440, the encoding is stopped when the target bit rate, which is a predetermined target bit rate, is stopped. Otherwise, the scaling factor value is halved. In short, the main encoding process (step 430) and the dependent encoding process (step 440) are repeatedly executed (step 460). In this way, when the predetermined target bit rate is reached, the encoding of the residual image is terminated. With this feature, the image can be restored even if the receiver stops in the middle of receiving the bit string.

5A shows the original image. Comparing this image with the stereo image 5b according to the binocular difference estimation, it can be seen that the image 5b is seriously deteriorated in image quality. That is, referring to FIG. 5B, it can be seen that the column of the gray portion on the left side of the original image is not reconstructed in the stereo image according to the binocular difference estimation. 5C illustrates a stereo image to which a conventional embedded wavelet encoding method is applied, and FIG. 5D illustrates a stereo image to which the embedded wavelet encoding method according to the present invention is applied. Comparing the original image and the stereo image using the conventional embedded wavelet encoding method and the stereo image using the embedded wavelet encoding method according to the present invention, it can be seen that the performance of the image quality is superior to the stereo image according to the binocular difference estimation. . However, unlike the original image, the stereo image to which the conventional embedded wavelet encoding method is applied can be seen that the distortion of a predetermined image appears in the pillar portion of the left side of the image, but the stereo image to which the embedded wavelet encoding method according to the present invention is applied. Is very similar to the original image.

Table 1 compares the conventional method and the Peak Signal to Noise Ratio (PSNR) improvement according to the present invention by applying various wavelet basis functions to famous stereo images. PSNR is a measure to determine the degree of deterioration of the quality of the original image and the image after some image processing. The higher the PSNR improvement, the better the performance.

여기서 PSNR은로 정의된다.

Where PSNR is defined as

Referring to Table 1, the results of the test of the image quality performance in stereo residual image encoding of four images of Pentagon, Parking meter, corridor and Sawtooth are shown. In general, it can be seen that the application of the stereo residual image encoding method according to the present invention is higher than the conventional method, and the improvement of the image quality of the image is higher at a lower bit rate.

The present invention can be embodied as code that can be read by a computer (including all devices having an information processing function) in a computer-readable recording medium.

The computer-readable recording medium includes all kinds of recording devices in which data is stored which can be read by a computer system. Examples of computer-readable recording devices include ROM, RAM, CD-ROM, magnetic tape, floppy disks, optical data storage devices, and the like. The computer readable recording medium can also be distributed over network coupled computer devices so that the computer readable code is stored and executed in a distributed fashion.

Although the present invention has been described with reference to the embodiments shown in the drawings, this is merely exemplary, and it will be understood by those skilled in the art that various modifications and equivalent other embodiments are possible. Therefore, the true technical protection scope of the present invention will be defined by the technical details of the appended claims.

According to the efficient encoding method of the stereo residual image according to the present invention, by changing the information storage order according to the characteristics of the stereo residual image, it is possible to restore a high quality stereo image at a lower bit rate.

Claims (11)

Wavelet transforming the residual image of the stereo image, and calculating edge tendency of the wavelet subband of the converted image; Setting a threshold from the computed edge trend diagram; Main-coding the wavelet subband when the absolute value of the coefficient of the wavelet subband is greater than the set threshold value; And The main coded subband until the difference between the main coded subband coefficients and the coefficient of the stereo picture is smaller than the difference between the uncoded subband coefficient corresponding to the main coded subband coefficient and the coefficient of the stereo picture. And applying dependent coding to the band. The method of claim 1, The main encoding step, And encoding the wavelet subband having a relatively strong edge tendency preferentially. The method of claim 1, And storing the coefficients of the main coded wavelet subband in a dependent encoding list. The method of claim 1, Comparing the dependent coded bit rate with a target bit rate; And And repetitively applying the main encoding step and the dependent encoding step by lowering a threshold value when the target bit rate does not reach the target bit rate. The method of claim 4, wherein Repeated steps apply, And repetitively applying the main encoding step and the dependent encoding step by lowering a threshold value to half when the target bit rate does not reach the target bit rate. A non-transitory computer-readable recording medium having recorded thereon a program for executing the method of claim 1. A tendency calculator configured to wavelet transform a residual image of the stereoscopic image and calculate edge tendencies of the wavelet subbands of the converted image; A threshold setting unit for setting a threshold value from the calculated edge tendency; A main encoder for main encoding the wavelet subband when the coefficient size of the wavelet subband is greater than the set threshold value; And The main coded subband until the difference between the main coded subband coefficients and the coefficient of the stereo picture is smaller than the difference between the uncoded subband coefficient corresponding to the main coded subband coefficient and the coefficient of the stereo picture. And a dependent encoder which applies the dependent encoding to the band. The method of claim 7, wherein The main coding unit, And stereo wavelet subband having a relatively strong edge tendency is preferentially encoded. The method of claim 7, wherein The main coding unit, And a dependent encoding list unit which stores coefficients of the main coded wavelet subband in a dependent encoding list. The method of claim 7, wherein A bit rate comparison unit for comparing the dependent coded bit rate with a target bit rate; And And a feed unit for repeatedly applying the main encoding and the dependent encoding by lowering a threshold when the target bit rate does not reach the target bit rate. The method of claim 10, The feed unit, And a feed unit for repeatedly applying the main encoding and the dependent encoding by lowering a threshold value to half when the target bit rate does not reach the target bit rate.

Images