JP4522951B2 - Moving picture encoding method and apparatus, decoding method and apparatus, moving picture processing program, and computer-readable recording medium - Google Patents

Description

  The present invention relates to a moving image encoding method and apparatus, a decoding method and apparatus, a moving image processing program, and a computer-readable recording medium, and in particular, a moving image encoding method for efficiently transmitting and decoding a moving image. The present invention relates to an apparatus, a decoding method and apparatus, a moving image processing program, and a computer-readable recording medium.

  Lossless encoding that can reproduce an original image without distortion is particularly necessary in the field of still images that require high-definition images such as medical, art, and printing, and various methods have been proposed (for example, non-coding). (See Patent Documents 1, 2, and 3).

  On the other hand, research on lossy coding has been actively conducted on moving images, but in recent years, lossless coding of moving images is required in digital broadcasting and digital cinema archiving and editing.

  As the lossless encoding method, various methods have been proposed for still images. As international standard encoding, standards such as JPEG-LS (see Non-Patent Document 1), JPEG2000 lossless mode (see Non-Patent Document 2), and the like. is there.

  JPEG-LS is a method based on non-linear predictive coding and context modeling that can predict changes following local changes in the image, and is faster and more efficient in coding than JPEG2000 lossless mode (for example, Non-patent document 4).

  On the other hand, JPEG2000 has various advanced functions that JPEG-LS does not have, such as resolution scalability and SNR scalability, and ROI (Region Of Interest) function that can change the compression rate for each image area.

  On the other hand, from the viewpoint of coding efficiency, it is desirable to use inter-frame correlation, and it is expected that a higher compression rate can be obtained. The lossless video coding method using inter-frame correlation is a technique that improves the coding efficiency by using motion compensation to remove inter-frame correlation, although it does not have a resolution scalability function (for example, Non-Patent Document 5, 6).

  In addition, the lossless video encoding method using inter-frame correlation has a resolution scalability function and a function that is excellent in encoding efficiency (for example, see Non-Patent Document 7). It is a hybrid coding that combines wavelet transform and predictive coding, and adaptively removes intra-frame correlation and inter-frame correlation in the lowest frequency band of the wavelet, thereby efficiently reducing entropy with a small amount of computation. This utilizes the property that a natural image has a strong inter-frame correlation between the lowest frequency bands and is weak in other bands. This method is called multiresolution lossless video coding (MLVC).

  On the other hand, although the inter-frame correlation between the frequency bands tends to be weak in the high frequency band, the magnitude is not constant and varies depending on the image. A method has been proposed in which the encoding efficiency is improved by extending MLVC and adaptively changing the band for removing intra-frame correlation and inter-frame correlation according to the statistical properties of the target image (for example, non-patent literature) 8).

This method is called a multi-resolution variable lossless video coding method Ex-MLVC (Extended Multiresolution Lossless Video Coding).
ISO / ITC JTC1 / SC29 FCD14495, Lossless and near-lossless coding of continuous tone still images (JPEG-LS), ISO / IEC JTC1 / SC29 / WG1, July 1997. ISO / IEC JTC1 / SC29 WG1 N1646R, JPEG2000 Part I Final Committee Draft Ver, 1.0, ISO / IEC JTC1 / SC29 WG1, March 2000. T. Nakachi, T, Fujii and J. Suzuki, “Pel adaptive predictive coding based on image segmentation for lossless compression,” IEICE Trans. On Fundamentals, vol. E82-A, no.6, June 1999. D. Santa-Cruz and T. Ebrahimi, “A study of JPEG2000 still image coding versus other standards,” vol. 2, pp.673-676, Tampere, Finland, EUSIPCO-2000, Sep. 2000. Naoki Ono and Yoshiyuki Yashima, “Performance Evaluation of Lossless Coding Using Motion Compensation”, IEICE Technical, IE95-18, pp.21-27, 1995. Junichi Nakajima, Yoshiyuki Yashima, Naoki Kobayashi, “Examination of hierarchical lossless coding based on MPEG-2 coding parameters” IEICE-Technology, IE99-124, pp.25-30, 2000. T. Nakachi, T. Sawabe, T. Fujii and T. Fujii, “Multiresolution lossless video coding using inter / intra frame adaptive prediction,” IEICE Trans. On Fundamentals, vol. E85-A, no.8, pp.1822- 1830, Aug, 2002. T. Nakachi, T. Sawabe and T. Fujii, “A Study on Multiresolution Lossless Video Codiong Using Inter / Intra Frame Adaptive Prediction,” VCIP 2003, vol.5150, pp.1685-1696, June 2003.

  However, although both MLVC and Ex-MLVC methods have spatial resolution scalability, they do not define an efficient transmission order of encoded data. Note that having spatial resolution scalability indicates that images of different spatial resolutions can be decoded step by step from one encoded bit stream.

  The present invention has been made in view of the above points, and a moving image encoding method and apparatus capable of efficiently transmitting encoded data in moving image lossless encoding / decoding having spatial resolution scalability, and It is an object to provide a decoding method and apparatus, a moving image processing program, and a computer-readable recording medium.

  FIG. 1 is a diagram for explaining the principle of the present invention.

The present invention (Claim 1) is a moving picture encoding method in which an input original image is band-divided by wavelet transform (step 1), and intra-frame and inter-frame prediction is performed for each pixel for each divided band. And
When playing back at the same resolution as the original image on the decoding side,
In the encoding means, inter-frame correlation of the band coded pixel neighborhood signal divided by the band dividing means, if greater than Jo Tokoro threshold performed (Step 2, Yes), the inter-frame prediction (Step 3 ), When the inter-frame correlation is smaller than a predetermined threshold (No in Step 2), an intra-frame prediction (Step 4) is performed, and an encoding step is performed.
For each encoded frame, where N is a positive integer, the number of wavelet division levels, and k is an integer between 1 and N,
A first transmission step (step 5) for transmitting encoded data of the lowest frequency band LLN first;
Second transmission step of transmitting encoded data of frequency bands HLk, LHk, and HHk higher than the lowest frequency band in order of increasing the ratio of the number of processing times to intra-frame prediction of inter-frame prediction of each band. (Step 6) .

The present invention (Claim 2) is a moving picture decoding method for decoding data encoded by the moving picture encoding method according to claim 1 in order to reproduce it at the same resolution as the original image ,
With N being a positive integer and the number of wavelet division levels, and k being an integer of 1 to N, the decoding means encodes the encoded data for each encoded frame,
A first decoding step (step 7) for first decoding the encoded data of the lowest frequency band LLN;
Second decoding step of decoding encoded data of frequency bands HLk, LHk, and HHk higher than the lowest frequency band in order from the second to the largest in the ratio of the number of processing times to intra-frame prediction of inter-frame prediction of each band (Step 8) .

The present invention (Claim 3) is a moving image encoding method for dividing an input original image by wavelet transform and performing intra-frame and inter-frame prediction for each divided band for each pixel,
When playing back in stages for each resolution of 1/2 ⁿ times the original image on the decoding side,
In the encoding means, when the inter-frame correlation of the encoding target pixel neighboring signal in the band divided by the band dividing means is larger than a predetermined threshold, inter-frame prediction is performed, and the inter-frame correlation is smaller than the predetermined threshold. A coding step of performing intra-frame prediction and coding;
For each encoded frame, where N is a positive integer and the number of wavelet division levels, and k is an integer between 1 and (N−1),
A first transmission step of transmitting encoded data of the lowest frequency band LLN first;
The encoded data of the three frequency bands HLN, LHN, and HHN of the Nth level are transmitted from the second to the fourth, in the order of the ratio of the number of processing times with respect to the intraframe prediction of the interframe prediction of each band. A transmission step;
Three frequency bands HLN of the N levels, LHN, HHN higher frequency band HLk, LHk, the fifth and subsequent encoded data for HHK, for each level in order of the high frequency band from a low frequency band, and the same level Includes a third transmission step for transmitting in the order of HLk, LHk, and HHk.

The present invention (Claim 4) is a moving picture decoding for decoding data encoded by the moving picture encoding method according to claim 3 in order to reproduce it step by step at a resolution of 1/2 ⁿ times the original image. A method of
With N being a positive integer and the number of wavelet division levels, and k being an integer not less than 1 and not more than (N−1), the decoding means encodes the encoded data for each encoded frame.
A first decoding step of first decoding encoded data of the lowest frequency band LLN;
Three frequency bands HLN of the N levels, LHN, 4th 2 to the encoded data HHN, the Gosuru restored sequentially ratio of the number of processing times is larger for the intra prediction of the prediction between bands of the frame 2 Decryption step of
Three frequency bands HLN of the N levels, LHN, HHN higher frequency band HLk, LHk, the fifth and subsequent encoded data for HHK, for each level in order of the high frequency band from a low frequency band, and the same level Includes a third decoding step for decoding in the order of HLk, LHk, and HHk.

  FIG. 2 is a principle configuration diagram of the present invention.

The present invention (Claim 5) has a band dividing means 20 for dividing a band of an input original image by wavelet transform, and performs moving picture coding for performing intra-frame and inter-frame prediction for each divided band for each pixel. A device,
When playing back at the same resolution as the original image on the decoding side,
If the inter-frame correlation of the divided band coded pixel sensor signal of the band dividing means 20 is constant greater than threshold Tokoro performs interframe prediction, if between the frame correlation is smaller than the predetermined threshold Encoding means 30 for performing intra-frame prediction and encoding;
For each encoded frame, where N is a positive integer and the number of wavelet division levels, and k is an integer between 1 and N,
A first transmission means 51 for first transmitting encoded data of the lowest frequency band LLN;
Second transmission means for transmitting encoded data of frequency bands HLk, LHk, and HHk higher than the lowest frequency band in order from the second to the largest in the ratio of the number of processing times to intra-frame prediction of inter-frame prediction of each band 52 .

The present invention (Claim 6) is a moving picture decoding apparatus for decoding data encoded by the moving picture encoding apparatus according to claim 5 in order to reproduce it at the same resolution as the original image ,
For each frame in which the encoded data is encoded, where N is a positive integer and the number of wavelet division levels, k is an integer between 1 and N ,
First decoding means 61 for first decoding encoded data of the lowest frequency band LLN;
Second decoding means for decoding the encoded data of the frequency bands HLk, LHk, and HHk higher than the lowest frequency band in the descending order of the ratio of the number of processing times to intra-frame prediction of inter-frame prediction of each band. 62 .

The present invention (Claim 7) has a band dividing means for dividing an input original image into bands by wavelet transform, and a moving picture coding apparatus that performs intra-frame and inter-frame prediction for each divided band for each pixel. Because
When playing back in stages for each resolution of 1/2 ⁿ times the original image on the decoding side,
When the inter-frame correlation of the encoding target pixel neighborhood signal of the band divided by the band dividing unit is larger than a predetermined threshold, inter-frame prediction is performed, and when the inter-frame correlation is smaller than the predetermined threshold, Encoding means for performing intra prediction and encoding;
For each encoded frame, where N is a positive integer and the number of wavelet division levels, and k is an integer between 1 and (N−1),
First transmission means for transmitting encoded data of the lowest frequency band LLN first;
The encoded data of the three frequency bands HLN, LHN, and HHN of the Nth level are transmitted from the second to the fourth, in the order of the ratio of the number of processing times with respect to the intraframe prediction of the interframe prediction of each band. Transmission means;
Three frequency bands HLN of the N levels, LHN, HHN higher frequency band HLk, LHk, the fifth and subsequent encoded data for HHK, for each level in order of the high frequency band from a low frequency band, and the same level Includes third transmission means for transmitting in the order of HLk, LHk, and HHk.

According to the present invention (Claim 8), video decoding is performed so that the data encoded by the video encoding apparatus according to Claim 7 is decoded so as to be reproduced stepwise to a resolution of 1/2 ⁿ times the original image. A device,
For each frame in which the encoded data is encoded, where N is a positive integer and the number of wavelet division levels, k is an integer of 1 to (N-1),
First decoding means for first decoding the encoded data of the lowest frequency band LLN;
Decode second to fourth encoded data of the three frequency bands HLN, LHN, and HHN of the Nth level in descending order of the ratio of the number of processing times to intraframe prediction of interframe prediction of each band Decryption means;
The encoded data of the frequency bands HLk, LHk, and HHk higher than the three frequency bands HLN, LHN, and HHN of the Nth level are the same for each level in the order from the fifth frequency band to the high frequency band. Within the level , there is a third decoding means for decoding in the order of HLk, LHk, and HHk.

The present invention (Claim 9) is a moving image processing program that causes a computer to function as the moving image processing apparatus according to any one of Claims 5 to 8.

The present invention (Claim 10) is a computer-readable recording medium storing a program that causes a computer to function as the moving image processing apparatus according to any one of Claims 5 to 8.

  As described above, according to the present invention, efficient stepwise transmission of encoded data can be realized in a moving image lossless encoding method / decoding method having spatial resolution scalability.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[First embodiment]
In the present embodiment, an efficient stepwise transmission method of encoded data in Ex-MLVC will be described. In the following, description will be made using the MLVC lossless encoder which is the basis of Ex-MLVC.

  FIG. 3 shows the configuration of the MLVC lossless encoder according to the first embodiment of the present invention.

  The MLVC lossless encoder shown in the figure includes a lossless color transform unit 10, a lossless wavelet transform unit 20, a space-time adaptive prediction coding unit 30, an entropy coding unit 40, and a multiplexing unit 50.

  The reversible color transform unit 10 reduces the correlation between the color components of the input original image and outputs the signal to the reversible wavelet transform unit 20.

  The reversible wavelet transform unit 20 divides the band of the input moving image signal using a reversible filter, realizes resolution scalability, reduces intra-frame correlation, and uses the lowest frequency band as a space-time applied predictive encoding unit. 30 and outputs to the entropy encoding unit 40 other than the lowest frequency band. For signals other than the lowest frequency band, spatio-temporal adaptive predictive coding is not performed considering that the inter-frame correlation is weak and the improvement rate of the coding efficiency with respect to the increase in time calculation is small.

  The space-time adaptive prediction encoding unit 30 performs inter-frame prediction when the inter-frame correlation of the encoding target pixel neighborhood signal is larger than a predetermined value, and performs intra-frame prediction when the inter-frame launch is smaller than the predetermined value. Encoding is performed and output to the multiplexing unit 50.

  The entropy encoding unit 40 performs entropy encoding on a signal other than the lowest frequency band and outputs the signal to the multiplexing unit 50.

  The multiplexing unit 50 multiplexes signals from the space-time adaptive prediction encoding unit 30 and the entropy encoding unit 40 and outputs an encoded bit stream to the decoder.

  In this embodiment, in order to realize a random access function, intra-frame coding (Inter-coded picture: I frame) is performed at an appropriate interval. A collection of images between intra-frame coding is called GOP (Group Of Pictures), and in this embodiment, two types of intra-frame coding and inter-frame prediction coded frames (predictive-coded pictures: P frames) are used. Consists of frames.

In this embodiment, a wavelet is used as the conversion method of the reversible wavelet transform unit 20 because of the ease of realizing resolution scalability. In order to realize lossless encoding, (5, 3) lossless filters (for example, AR Calderbank et. Al., “Wavelet transforms that map integers to integers,” vol. E85-A, no.8, pp.1822- 1830, Aug. 2002.), and use the same image in the vertical and horizontal directions. The wavelet division method divides an image into four frequency bands by performing a one-dimensional wavelet transform independently in the vertical and horizontal directions of the image, and recursively divides the frequency band carrying the lowest frequency into four frequency bands. Splitting (eg, SG Mallat, “A Theory for multiresolution signal decomposition: The wavelet representation,” IEEE Trans. Pattern Analysis & Machine Intelligence, vol. 11, pp. 674-693, July 1989) is used.

  FIG. 4 shows an example of two-level Mallat division. The reversible wavelet transform unit 20 outputs a signal in the lowest frequency band (LL) to the spatiotemporal adaptive prediction encoding unit 30, and outputs signals in other bands to the entropy encoding unit 40.

  JPEG-LS, the international standard for still images related to lossless encoding, is capable of prediction that follows changes in local properties of an image using a method based on nonlinear predictive encoding and context modeling. Compared with JPEG2000 lossless mode, It is known that the coding efficiency is excellent at high speed. MLVC achieves high coding efficiency by extending the nonlinear predictor used in JPEG-LS to inter-dimensional prediction.

  The MLVC spatio-temporal adaptive prediction encoding unit 30 has a configuration as shown in FIG. The space-time adaptive prediction encoding unit 30 includes a two-dimensional predictor 31, a three-dimensional predictor 32, a motion estimation unit 33, a correlation coefficient calculation unit 34, a shift operator 35, and an entropy encoding unit 36. Apply the method to the signal of the lowest frequency band. In the I frame, a two-dimensional predictor 31 (removal of intra-frame correlation) is applied. Also, in order to increase the coding efficiency, in the P frame, the two-dimensional predictor 31 and the three-dimensional predictor 32 (removing intra-frame correlation and inter-frame correlation simultaneously) are adaptively switched to generate a residual signal. In MLVC, this method is called “space-time adaptive predictive coding”.

Switching between the two-dimensional predictor 31 and the three-dimensional predictor 32 in the spatio-temporal adaptive predictive encoding unit 30 is performed by the correlation coefficient calculation unit 34 in the vicinity signal values x _{1 to} x ₄ in the current frame a shown in FIG. And the correlation coefficient R of the neighborhood signal values y _{1 to} y ₄ in the reference frame b is calculated and determined. Here, for y ₁ to y ₄ , a signal sequence shifted by the shift operator 35 based on the result of motion estimation by the motion estimation unit 33 is used. The correlation coefficient in the correlation coefficient calculation unit 34, not less than the threshold T _h a correlation coefficient, determines that correlation between frames is large, using a three-dimensional predictor 32. Otherwise, a two-dimensional predictor 31 is used.

That means
When the correlation coefficient R < _Th , the two-dimensional predictor 31 is used.
• For the correlation coefficient R ≧ T _h is used three-dimensional predictor 32.

  However,

２次元及び３次元予測は、以下のように行う。

Two-dimensional and three-dimensional prediction is performed as follows.

1) Two-dimensional prediction:
In the present invention, a non-linear predictor that adaptively switches among a plurality of predictors prepared in advance is used in order to perform prediction following a local property change of an image. Here, the same nonlinear predictor as JPEG-LS is used. In JPEG-LS, three types of predictors are switched according to the state of the encoding target pixel neighborhood signal to generate a residual signal. When it is determined that there are edges in the vertical direction and the horizontal direction, prediction is performed using one pixel adjacent to each edge direction, and when it is determined that the luminance change is smooth, prediction is performed using three adjacent pixels. To do.

ここで、ｘ₁、x₂、ｘ₃は符号化対象画素近傍の信号値であり、「４」で定義される。

Here, x ₁ , x ₂ , and x ₃ are signal values in the vicinity of the encoding target pixel, and are defined by “4”.

2) 3D prediction:
In the case of three-dimensional prediction, similarly to the two-dimensional prediction, non-linear prediction is performed in which the predictor is switched depending on the state of the neighborhood signal value of the pixel to be encoded. When it is determined that there is an edge in the vertical direction or the horizontal direction, prediction is performed using signals adjacent to each edge direction of the current frame a and the reference frame b. As for the edge direction, the absolute difference value | y ₀ −y ₃ | in the vertical direction of the reference frame b is compared with the absolute difference value | y ₀ −y ₁ | in the horizontal direction. to decide. When it is determined that the edge is in the vertical direction (in the case of the following condition i), the vertical signals x ₃ , y ₀ , and y ₃ with the current frame a and the reference frame b are processed in the same manner as in the two-dimensional prediction. Select the prediction signal. In the case of a horizontal edge (in the case of condition ii below), a predictor is selected by the same method. If it is determined not to be an edge, the average value of three neighboring pixels (x ₁ , x ₃ , y ₀ ) is used as the predicted value.

Condition i: When │y ₀ -y ₁ │-│y ₀ -y ₃ │> T:

条件ii：│ｙ₀−ｙ₁│−│ｙ₀−ｙ₃│＜−Ｔの場合：

Condition ii: When | y ₀ −y ₁ | − | y ₀ −y ₃ | <−T:

条件iii：条件ｉ,ii以外の場合

Condition iii: Other than conditions i and ii

MLVCでは、最低周波数帯域以外の信号については、フレーム間相関が弱く、計算時間の増加に対する符号化効率の向上率が小さいことを考慮し、時空間適応予測符号化部３０による、時空間適応予測符号化を行わない。

In MLVC, the spatio-temporal adaptive prediction encoding unit 30 performs spatio-temporal adaptive prediction in consideration of the fact that the inter-frame correlation is weak for signals other than the lowest frequency band and the improvement rate of the encoding efficiency with respect to the increase in calculation time is small. Does not encode.

  Ex-MLVC provides a model that adaptively changes the adaptive band of spatio-temporal adaptive predictive coding according to the statistical properties of the image for the purpose of further improving the coding efficiency.

In Ex-MLVC, for the lowest frequency band, the spatio-temporal adaptive prediction encoding unit 30 performs spatio-temporal adaptive prediction processing in the same manner as MLVC. An adaptive prediction process is performed, and in the weak band, the wavelet coefficients are directly encoded by the entropy encoding unit 30. However,
1) Keep the amount of calculation as low as possible,
2) Eliminate the amount of additional information
Therefore, the inter-frame correlation is not directly calculated in each band.

  As shown in FIG. 7, using the fact that wavelet coefficients have a correlation between bands in the same direction with reference to the lowest frequency band, the number of processing times of three-dimensional prediction and two-dimensional prediction in a frequency band one level lower Switching is performed according to the ratio.

1) When R _inter ≧ TH in the 1-level low frequency band, space-time adaptive prediction is performed.

2) When R _inter <TH in the 1-level low frequency band, wavelet coefficient coding is performed.
However,

ここで、Inter、Intraは、それぞれ３次元予測器３２及び２次元予測器３１の処理回数を表す。なお、これらの処理回数は、それぞれ時空間適応予測符号化部３０内のメモリ（図示せず）に格納されており、Ｒ_interの計算を行う際に読み出される。

Here, Inter and Intra represent the number of processings of the three-dimensional predictor 32 and the two-dimensional predictor 31, respectively. These processing times are stored in a memory (not shown) in the spatio-temporal adaptive predictive coding unit 30 and are read when R _inter is calculated.

When the value of R _inter is large, it indicates that the number of processing times of the three-dimensional predictor 32 is large. That is, it means that the correlation between frames is strong. Further, TH in the above 1) and 2) represents a predetermined threshold value.

  Next, the bandwidth unit processing procedure will be described.

  The case where the number of wavelet division levels in the reversible wavelet transform unit 20 is “2” will be described as an example.

  FIG. 8 is a diagram (band division = 2 levels) for explaining band unit processing of space-time adaptive prediction in the first embodiment of the present invention. It is determined whether or not the spatiotemporal adaptive prediction processing is performed in the direction of the arrow shown in FIG.

  FIG. 9 shows a processing procedure for each band in the first embodiment of the present invention.

First, in Ex-MLVC, in the lowest frequency band LL2, the space-time adaptive prediction encoding unit 30 performs space-time adaptive prediction processing in the same manner as MLVC. After all processing is completed, R _inter is calculated (step 101).

When R _Inter ≧ TH (step 102, Yes), spatiotemporal adaptive prediction processing is performed in the HL2, LH2, and HH2 bands (steps 103, 106, and 109). If R _Inter <TH (step 102, No), the process ends.

When the spatiotemporal adaptive prediction process is performed, the correlation coefficient calculation unit 34 of the spatiotemporal application prediction encoding unit 30 calculates R _Inter for each of the HL2, LH2, and HH2 bands. It is determined whether or not the condition of R _Inter ≧ TH is satisfied in each band (steps 104, 107, and 110), and it is determined whether or not to perform the spatio-temporal application prediction process in the HL1, LH1, and HH1 bands.

  Next, the small block unit processing procedure will be described.

  In the wavelet transform region, it is known that components corresponding to the same spatial position in each frequency band have a strong correlation. Using this property, it is determined whether to perform space-time application prediction in small blocks.

  FIG. 10 is a diagram for explaining processing in units of space-time adaptive prediction blocks in the first embodiment of the present invention. In the figure, the processing direction when the number of division levels is “2” is shown.

  FIG. 11 shows a processing procedure for each block in the first embodiment of the present invention.

  For the processing procedure, the method shown in FIG. 9 is performed for each block and is repeated until the total number of blocks (N) is reached.

  Thereby, it becomes possible to follow the local property of the image, and the coding efficiency is improved.

  The residual signal or wavelet coefficient generated by the above band unit processing or small block unit processing is output as encoded data by the entropy encoding unit 40.

  In Ex-MLVC, the time priority at the time of transmission and decoding can be freely set by the scalability function. Two scalability units are defined: C: color and R: spatial resolution.

FIG. 12 shows an example of an RC structure in which the spatial resolutions are preferentially arranged in the first embodiment of the present invention. However, this is an example, and it is up to the user to determine the order in which the components such as LL and HH are transmitted by the encoder or decoded by the decoder. Not done. Here, the order in which each component (LL2, HL2, LH2,...) Of spatial resolution is efficiently transmitted and decoded will be described. In the present invention, when realizing spatial resolution scalability,
1) When playing back at the same resolution as the original image;
2) When reproducing in stages at resolutions of 1/2 ⁿ times the original image;
The following two technologies are provided.

  First, in the present embodiment, the above case 1) will be described.

  In the case of an I picture, the transmission (encoder) or decoding (decoder) order is fixed, and the pictures are arranged so as to satisfy the following two conditions.

・ Low frequency band to high frequency band;
・ In frequency band within the same level , HL → LH → HH;
FIG. 12A shows an RC data structure when the wavelet level = 2. FIG. 13A shows the order in the wavelet region corresponding to FIG.

  In the case of a P picture, rearrangement is performed so as to satisfy the following conditions.

-The first is the lowest frequency band LLN (where N is a positive integer and the number of wavelet division levels);
・ The second and subsequent frequency bands are in descending order of R _Inter ;
That is, in transmitting from the multiplexing unit 50 of the encoder (output), with reference to pre-stored in a memory such as the conditions, but the lowest frequency band LLN is larger frequency band R _Inter ahead Transmit to. Similarly, in the decoder, with reference to the previously it said conditions are stored memory or the like, but the lowest frequency band LLN decodes earlier larger frequency band R _Inter.

Here, R _Inter represents the ratio of the number of processing times of the three-dimensional predictor 32 and the two-dimensional predictor 31, and means that the larger the number of R _Inter , the larger the number of processing times of the three-dimensional predictor 32. That is, the above condition is based on the premise that the three-dimensional predictor 32 contributes more greatly to the coding efficiency than the two-dimensional predictor 31.

Number wavelet division level is "2", with respect to the size of the R _Inter,
・ HL2 → LH2 → HL1 → HH2 → LH1 → HH1
In this case, the order of frequency band transmission or decoding is as shown in FIG. 12B and FIG. 13B.

[Second Embodiment]
As the second embodiment, a case of “2) stepwise reproduction for each 1/2 ⁿ resolution of the original image when realizing the spatial resolution scalability” will be described.

  The I pictures are arranged in a fixed order based on the same criteria as in “1) When reproducing at the same resolution as the original image” described above. In the case of a P picture, rearrangement is performed so as to satisfy the following conditions.

・ Low frequency band to high frequency band;
-The first is the lowest frequency band LLN (where N is a positive integer and the number of wavelet division levels);
・ 2-4th is descending order of R _Inter for HLN, LHN, HHN;
- 5 th and subsequent order, the frequency band HL in each level, LH, and HH, corresponding, Relais bell HL, LH, and HH sequence;
According to

Regarding the size of R _{Inter when} the number of wavelet division levels is “2”,
・ HL2 → LH2 → HH2
In this case, the order of transmission in the encoder of each frequency band or decoding in the decoder is as shown in FIG. 12 (c) and FIG. 13 (c). That is, the multiplexing unit 50 transmits the encoded bit stream with reference to a memory or the like in which the above conditions are stored in advance. Also in the decoder, the input data is decoded with reference to a memory or the like in which the above conditions are stored in advance.

  In addition, the operations of the encoder and decoder described above can be constructed as a program, installed in a computer used as an encoder and decoder, executed, or distributed via a network. is there.

  Further, the constructed program can be stored in a portable storage medium such as a hard disk device or a flexible disk / CD-ROM, and installed in a computer or distributed.

  The present invention is not limited to the above-described embodiment, and various modifications and applications can be made within the scope of the claims.

  The present invention is applicable to a moving image encoding / decoding technique.

It is a figure for demonstrating the principle of this invention. It is a principle block diagram of this invention. It is a block diagram of the MLVC encoder in the 1st Embodiment of this invention. It is an example of Mallet division | segmentation (2 levels) in the 1st Embodiment of this invention. It is a block diagram of the space-time adaptive prediction encoding part in the 1st Embodiment of this invention. It is a figure which shows the encoding target signal x and the vicinity signal in the 1st Embodiment of this invention. It is a figure which shows the correlation between the bands in the 1st Embodiment of this invention. It is a figure (band division = 2 levels) for demonstrating the process of the band unit of the spatiotemporal adaptive prediction in the 1st Embodiment of this invention. It is the processing procedure (band division | segmentation = 2 level) of the band unit in the 1st Embodiment of this invention. This is block-by-block processing (band division = 2 levels) for spatiotemporal adaptive prediction in the first embodiment of the present invention. It is the processing procedure (band division | segmentation = 2 level) of the block unit in the 1st Embodiment of this invention. It is an example (in the case of wavelet division | segmentation level = 2) of the RC structure put in order with the spatial resolution in the 1st, 2nd embodiment of this invention. It is an example (wavelet domain expression) of the transmission order and decoding order of the spatial resolution component in the 1st, 2nd embodiment of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Reversible color transformation part 20 Reversible wavelet transformation part, Band-division means 30 Space-time adaptive prediction encoding part, Encoding means 40 Entropy encoding part 50 Multiplexing part 51 First transmission means 52 Second transmission means 61 1st Decoding means 62 second decoding means 31 two-dimensional predictor 32 three-dimensional predictor 33 motion estimator 34 shift operator

Claims (10)

A moving image encoding method in which an input original image is band-divided by wavelet transform, and intra-frame and inter-frame prediction is performed for each pixel for each divided band,
When playing back at the same resolution as the original image on the decoding side,
In the encoding means, when the inter-frame correlation of the encoding target pixel neighboring signal in the band divided by the band dividing means is larger than a predetermined threshold, inter-frame prediction is performed, and the inter-frame correlation is smaller than the predetermined threshold. A coding step of performing intra-frame prediction and coding;
For each encoded frame, where N is a positive integer and the number of wavelet division levels, and k is an integer between 1 and N,
A first transmission step of transmitting encoded data of the lowest frequency band LLN first;
Second transmission for transmitting encoded data of frequency bands HLk, LHk, and HHk higher than the lowest frequency band in order from the second to the largest in the ratio of the number of processing times to intra-frame prediction of inter-frame prediction of each band. Steps,
A moving image encoding method comprising: A video decoding method for decoding data encoded by the video encoding method according to claim 1 in order to reproduce the data at the same resolution as an original image,
In the decoding means, N is a positive integer and the number of wavelet division levels, k is an integer of 1 to N, and the encoded data is encoded for each encoded frame.
A first decoding step of first decoding encoded data of the lowest frequency band LLN;
Second decoding for decoding the encoded data of the frequency bands HLk, LHk, and HHk higher than the lowest frequency band in the order of the ratio of the number of processing times with respect to the intra-frame prediction of the inter-frame prediction of each band. Steps,
A moving picture decoding method comprising: A moving image encoding method in which an input original image is band-divided by wavelet transform, and intra-frame and inter-frame prediction is performed for each pixel for each divided band,
When playing back in stages for each resolution of 1/2 ⁿ times the original image on the decoding side,
In the encoding means, when the inter-frame correlation of the encoding target pixel neighboring signal in the band divided by the band dividing means is larger than a predetermined threshold, inter-frame prediction is performed, and the inter-frame correlation is smaller than the predetermined threshold. A coding step of performing intra-frame prediction and coding;
For each encoded frame, where N is a positive integer and the number of wavelet division levels, and k is an integer between 1 and (N−1),
A first transmission step of transmitting encoded data of the lowest frequency band LLN first;
The encoded data of the three frequency bands HLN, LHN, and HHN of the Nth level are transmitted from the second to the fourth, in the order of the ratio of the number of processing times with respect to the intraframe prediction of the interframe prediction of each band. A transmission step;
The encoded data of the frequency bands HLk, LHk, and HHk higher than the three frequency bands HLN, LHN, and HHN of the Nth level are the same for each level in order from the fifth frequency to the high frequency band. Within the level , a third transmission step for transmitting in the order of HLk, LHk, HHk;
A moving image encoding method comprising: A video decoding method for decoding data encoded by the video encoding method according to claim 3 in order to reproduce it step by step at a resolution of 1/2 ⁿ times the original image,
In the decoding unit, N is a positive integer and the number of wavelet division levels, k is an integer of 1 to (N-1), and the encoded data is encoded for each frame.
A first decoding step of first decoding encoded data of the lowest frequency band LLN;
Three frequency bands HLN of the N levels, LHN, 4th 2 to the encoded data HHN, the Gosuru restored sequentially ratio of the number of processing times is larger for the intra prediction of the prediction between bands of the frame 2 Decryption step of
The encoded data of the frequency bands HLk, LHk, and HHk higher than the three frequency bands HLN, LHN, and HHN of the Nth level are the same for each level in order from the fifth frequency to the high frequency band. A third decoding step for decoding in the order of HLk, LHk, HHk within the level ;
A moving picture decoding method comprising: A moving image encoding apparatus having band dividing means for dividing an input original image by wavelet transform, and performing intra-frame and inter-frame prediction for each divided band for each pixel,
When playing back at the same resolution as the original image on the decoding side,
When the inter-frame correlation of the encoding target pixel neighboring signal of the band divided by the band dividing unit is larger than a predetermined threshold, inter-frame prediction is performed, and when the inter-frame correlation is smaller than the predetermined threshold Encoding means for performing intra-frame prediction and encoding;
For each encoded frame, where N is a positive integer and the number of wavelet division levels, and k is an integer between 1 and N,
First transmission means for transmitting encoded data of the lowest frequency band LLN first;
Second transmission for transmitting encoded data of frequency bands HLk, LHk, and HHk higher than the lowest frequency band in order of increasing the ratio of the number of processing times to intra-frame prediction of inter-frame prediction of each band. Means,
A moving picture encoding apparatus comprising: A video decoding device that decodes data encoded by the video encoding device according to claim 5 in order to reproduce the data at the same resolution as the original image,
For each frame in which the encoded data is encoded, where N is a positive integer and the number of wavelet division levels, k is an integer between 1 and N,
First decoding means for first decoding the encoded data of the lowest frequency band LLN;
Second decoding for decoding the encoded data of the frequency bands HLk, LHk, and HHk higher than the lowest frequency band in the order of the ratio of the number of processing times with respect to the intra-frame prediction of the inter-frame prediction of each band. Means,
A moving picture decoding apparatus comprising: A moving image encoding apparatus having band dividing means for dividing an input original image by wavelet transform, and performing intra-frame and inter-frame prediction for each divided band for each pixel,
When playing back in stages for each resolution of 1/2 ⁿ times the original image on the decoding side,
When the inter-frame correlation of the encoding target pixel neighboring signal of the band divided by the band dividing unit is larger than a predetermined threshold, inter-frame prediction is performed, and when the inter-frame correlation is smaller than the predetermined threshold, Encoding means for performing intra-frame prediction and encoding;
For each encoded frame, where N is a positive integer and the number of wavelet division levels, and k is an integer between 1 and (N−1),
First transmission means for transmitting encoded data of the lowest frequency band LLN first;
The encoded data of the three frequency bands HLN, LHN, and HHN of the Nth level are transmitted from the second to the fourth, in the order of the ratio of the number of processing times with respect to the intraframe prediction of the interframe prediction of each band. Transmission means;
The encoded data of the frequency bands HLk, LHk, and HHk higher than the three frequency bands HLN, LHN, and HHN of the Nth level are the same for each level in order from the fifth frequency to the high frequency band. Within the level , a third transmission means for transmitting in the order of HLk, LHk, HHk;
A moving picture encoding apparatus comprising: A moving picture decoding apparatus that decodes data encoded by the moving picture encoding apparatus according to claim 7 in order to reproduce it step by step to a resolution of 1/2 ⁿ times the original image,
For each frame in which the encoded data is encoded, where N is a positive integer and the number of wavelet division levels, k is an integer of 1 to (N-1),
First decoding means for first decoding the encoded data of the lowest frequency band LLN;
Decode second to fourth encoded data of the three frequency bands HLN, LHN, and HHN of the Nth level in descending order of the ratio of the number of processing times to intraframe prediction of interframe prediction of each band Decryption means;
The encoded data of the frequency bands HLk, LHk, and HHk higher than the three frequency bands HLN, LHN, and HHN of the Nth level for the fifth and subsequent levels, from the low frequency band to the high frequency band for each level , and Third decoding means for decoding in the order of HLk, LHk, HHk within the same level ;
A moving picture decoding apparatus comprising: Computer
9. A moving image processing program that causes the moving image processing apparatus according to claim 5 to function. Computer
A computer-readable recording medium storing a program that functions as the moving image processing apparatus according to any one of claims 5 to 8.

Images