JP2008211810A - Moving image reversible encoding method and decoding method thereof, and program therefor - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To perform moving image reversible encoding with excellent encoding efficiency and decoding thereof, with space resolution scalability. <P>SOLUTION: A band dividing section 10 divides a source signal into a plurality of space resolution bands. A space-time adaptive prediction encoding dividing section 11 is provided for each band and estimates a motion of a target object for the unit of a small block to determine a motion vector. The motion vector defines a motion vector of a bottom frequency band as a reference. An inter-frame correlation coefficient R is calculated from a current frame and the reference frame shifted by the motion vector, the R is compared with a threshold value. In the case that the inter-frame correlation is strong, three-dimensional prediction is performed but in the case that the inter-frame correlation is weak, two-dimensional prediction is performed to produce a predictive residual signal. An entropy encoding section 12 performs entropy encoding upon the predictive residual signal produced by the space-time adaptive prediction encoding section 11 and the motion vector used by the space-time adaptive prediction encoding section 11 and outputs an encoded bit stream. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to encoding and decoding techniques for efficiently transmitting and storing moving images.

  As a lossless image encoding method, JPEG-LS or JPEG2000 lossless encoding modes are known. JPEG-LS encodes a difference between a prediction signal that is an output of a predictor and an original signal by using an intra-frame predictor. By using the intra-frame predictor, the coding efficiency is improved by utilizing the correlation of signals in the frame. In JPEG2000, lossless Wavelet conversion is performed, and Wavelet coefficients are encoded. Since the image signal is divided into spatial frequency bands by using Wavelet transform and encoding is performed for each band, spatial resolution scalability is provided.

  However, since both of the conventional systems are intended for still images, signal correlation between frames cannot be used when adapted to moving images. As lossy encoding of moving images, there is a method that performs spatial compensation in the subband region as a method that has spatial resolution scalability and uses inter-frame correlation, but motion compensation performs prediction in units of blocks. Encoding efficiency is not always good. Having spatial resolution scalability indicates that images of different spatial resolutions can be directly decoded from one encoded bitstream.

  An object of the present invention is to propose a moving image lossless encoding method, a decoding method thereof, and a program thereof having spatial resolution scalability and excellent encoding efficiency.

In order to solve the above problem, the invention according to claim 1 is a lossless encoding method for moving images.
The original image is divided into bands,
A motion vector is obtained by performing SAD calculation for each lowest frequency band among the divided bands to obtain a motion vector. Pixels in a block that give an SAD value larger than a predetermined threshold T are subjected to two-dimensional prediction, and are equal to or lower than the predetermined threshold T. Pixels in the block giving the SAD value perform space-time adaptive prediction to obtain a prediction residual signal, encode the prediction residual signal and the motion vector used in the space-time adaptive prediction,
In the second lowest frequency band coding, the pixels of the spatial block corresponding to the block for which the two-dimensional prediction is performed in the lowest frequency band are directly coded, and the other blocks are subjected to SAD calculation to obtain a motion vector. The pixels in the block that give a SAD value greater than a predetermined threshold T2 are directly encoded, and the pixels in the block that give a SAD value less than or equal to the threshold T2 are subjected to spatiotemporal adaptive prediction to obtain a prediction residual signal. Encode the residual signal and the motion vector used in the space-time adaptive prediction,
In the subsequent high frequency band encoding, the pixels of the spatial block corresponding to the block directly encoded in the frequency band encoded immediately before are directly encoded, and the other blocks perform the SAD calculation to obtain the motion vector. The pixels in the block that give a SAD value larger than the threshold value Tx determined for each frequency band are directly encoded, and the pixels in the block that give a SAD value less than or equal to the predetermined threshold value Tx are predicted by performing space-time adaptive prediction. When obtaining a residual signal and encoding the prediction residual signal and the motion vector used in the space-time adaptive prediction,
In the spatio-temporal adaptive prediction, the correlation coefficient between the current pixel and the reference frame pixel value to be encoded between the current frame and the reference frame shifted by the shift means is calculated using the motion vector obtained by the SAD calculation. If the correlation coefficient is small, switch to three-dimensional prediction that performs prediction using the encoding target pixel neighboring signal values of the current frame and the reference frame. If the correlation coefficient is small, the encoding target pixel neighboring signal of the current frame Switch to two-dimensional prediction that uses only values to make predictions,
In the case of switching to the two-dimensional prediction, a prediction residual signal is calculated by switching a plurality of two-dimensional predictors for each pixel from the encoding target pixel vicinity signal value of the current frame,
When switching to the three-dimensional prediction, the prediction residual signal is calculated by switching a plurality of three-dimensional predictors for each pixel from the encoding target pixel neighboring signal values of the current frame and the reference frame. This is a lossless video encoding method.

  The invention according to claim 2 is characterized in that, in claim 1, a code indicating that the spatio-temporal adaptive prediction processing has not been performed is added to an encoded signal of a block in which an SAD value larger than a threshold value is obtained. This is a lossless video encoding method.

According to a third aspect of the present invention, in the lossless decoding method for outputting a moving image,
Receiving a direct coding block for each band, a prediction residual signal and a motion vector encoded by the video lossless encoding method according to claim 1;
In the order of the lowest frequency band to the highest frequency band, for each band, the direct coding block, the prediction residual signal and the motion vector are decoded, and the space-time adaptive prediction decoding is further performed from the prediction residual signal and the motion vector,
When decoding a moving image by combining the decoded bands,
In the spatio-temporal adaptive prediction decoding, a correlation coefficient between a decoding target pixel neighboring signal value of a reference frame shifted using the motion vector and a decoding target pixel neighboring signal value composed of a decoded signal of a signal in the current frame When the correlation coefficient is large, switch to 3D prediction, and when the correlation coefficient is small, switch to 2D prediction.
In the case of switching to the two-dimensional prediction, a prediction signal is calculated by switching a plurality of two-dimensional predictors for each pixel from the decoding target pixel neighboring signal value including the decoded signal of the signal in the current frame,
In the case of switching to the three-dimensional prediction, from the decoding target pixel neighboring signal value composed of the decoded signal of the signal in the current frame and the decoding target pixel neighboring signal value of the reference frame shifted using the motion vector, Calculate prediction signals by switching multiple 3D predictors for each pixel,
A moving image lossless decoding method comprising decoding a target signal by adding the prediction residual signal to the prediction signal.

According to a fourth aspect of the present invention, there is provided a lossless encoding apparatus for moving images.
The procedure for dividing the original image into bands,
A motion vector is obtained by performing SAD calculation for each lowest frequency band among the divided bands to obtain a motion vector. Pixels in a block that give an SAD value larger than a predetermined threshold T are subjected to two-dimensional prediction, and are equal to or lower than the predetermined threshold T. A pixel in the block that gives the SAD value performs a spatiotemporal adaptive prediction to obtain a prediction residual signal, and encodes the prediction residual signal and the motion vector used in the spatiotemporal adaptive prediction.
In the second lowest frequency band coding, the pixels of the spatial block corresponding to the block for which the two-dimensional prediction is performed in the lowest frequency band are directly coded, and the other blocks are subjected to SAD calculation to obtain a motion vector. The pixels in the block that give a SAD value greater than a predetermined threshold T2 are directly encoded, and the pixels in the block that give a SAD value less than or equal to the threshold T2 are subjected to spatiotemporal adaptive prediction to obtain a prediction residual signal. A procedure for encoding a residual signal and a motion vector used in the space-time adaptive prediction;
In the subsequent high frequency band encoding, the pixels of the spatial block corresponding to the block directly encoded in the frequency band encoded immediately before are directly encoded, and the other blocks perform the SAD calculation to obtain the motion vector. The pixels in the block that give a SAD value larger than the threshold value Tx determined for each frequency band are directly encoded, and the pixels in the block that give a SAD value less than or equal to the predetermined threshold value Tx are predicted by performing space-time adaptive prediction. Obtaining a residual signal, and executing a procedure for encoding the prediction residual signal and the motion vector used in the space-time adaptive prediction,
In the spatio-temporal adaptive prediction, the correlation coefficient between the current pixel and the reference frame pixel value to be encoded between the current frame and the reference frame shifted by the shift means is calculated using the motion vector obtained by the SAD calculation. If the correlation coefficient is small, switch to three-dimensional prediction that performs prediction using the encoding target pixel neighboring signal values of the current frame and the reference frame. If the correlation coefficient is small, the encoding target pixel neighboring signal of the current frame Switch to two-dimensional prediction that uses only values to make predictions,
In the case of switching to the two-dimensional prediction, a prediction residual signal is calculated by switching a plurality of two-dimensional predictors for each pixel from the encoding target pixel vicinity signal value of the current frame,
When switching to the three-dimensional prediction, a procedure for calculating a prediction residual signal by switching a plurality of three-dimensional predictors for each pixel from the signal to be encoded pixel vicinity signal values of the current frame and the reference frame is executed. It is a program for.

The invention according to claim 5 provides a lossless decoding apparatus that outputs a moving image,
A procedure for receiving a direct coding block for each band, a prediction residual signal, and a motion vector encoded by the moving image lossless encoding method according to claim 1;
A procedure for decoding a direct coding block, a prediction residual signal, and a motion vector for each band in order from the lowest frequency band to the highest frequency band, and further performing space-time adaptive prediction decoding from the prediction residual signal and the motion vector,
A procedure for decoding the moving image by performing band synthesis on the decoded band,
In the spatio-temporal adaptive prediction decoding, a correlation coefficient between a decoding target pixel neighboring signal value of a reference frame shifted using the motion vector and a decoding target pixel neighboring signal value composed of a decoded signal of a signal in the current frame When the correlation coefficient is large, switch to 3D prediction, and when the correlation coefficient is small, switch to 2D prediction.
In the case of switching to the two-dimensional prediction, a prediction signal is calculated by switching a plurality of two-dimensional predictors for each pixel from the decoding target pixel neighboring signal value including the decoded signal of the signal in the current frame,
In the case of switching to the three-dimensional prediction, from the decoding target pixel neighboring signal value composed of the decoded signal of the signal in the current frame and the decoding target pixel neighboring signal value of the reference frame shifted using the motion vector, Calculate prediction signals by switching multiple 3D predictors for each pixel,
It is a program for executing a procedure for decoding a target signal by adding the prediction residual signal to the prediction signal.

  In the present invention, in order to realize spatial resolution scalability, an image is divided into spatial resolution bands, and encoding is performed for each band. For encoding, spatiotemporal adaptive prediction encoding is used to increase the encoding efficiency by switching between intra-frame prediction (two-dimensional prediction) and inter-frame prediction (three-dimensional prediction) for each pixel.

  According to the present invention, it is possible to efficiently perform lossless encoding of moving images and to store them with a small disk capacity. Furthermore, since it has spatial resolution scalability, it is possible to decode an image with a spatial resolution according to the performance and application of the image display device. When decoding from a low band to an arbitrary band, an image having a lower spatial resolution than the original image can be reproduced, and when all data is decoded, an image having the same resolution as the original image is reproduced. If you want to reproduce an image with a resolution lower than the original image depending on the accuracy and application of the image display device, you only need to decode up to the required bandwidth. If you perform decoding, the image is directly decoded from the encoded data. The processing time is shorter than when the resolution conversion is performed after the image having the same resolution as the original image is reproduced. Also, when transmitting an encoded bit stream, only the necessary data is transmitted, so the transmission rate is also reduced.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

[First Embodiment]
FIG. 1 shows a basic configuration for realizing a moving picture coding method having spatial resolution scalability. In FIG. 1, 10 is a band dividing unit, 11 is a spatio-temporal adaptive prediction encoding unit provided for each divided band, and 12 is an entropy encoding unit.

  A moving picture encoding method as an operation example of the configuration of FIG. 1 will be described below. First, the input image signal is divided into a plurality of spatial resolution bands by the band dividing unit 10. Next, the residual signal generated by the space-time adaptive prediction encoding unit 11 is encoded by the entropy encoding unit 12 for each divided band.

  The band dividing unit 10 applies the octave division shown in FIG. 2 to the horizontal direction and the vertical direction of the image, respectively. In the octave division, the input signal can be divided into a plurality of bands by dividing the band one after another using a two-division filter, and finally the band is divided as shown in FIG.

  In FIG. 3, L represents a low frequency component, and H represents a high frequency component. The two-division filter used here and the band synthesis filter used on the decoding side are completely reconstructed filters in order to maintain reversibility.

  The space-time adaptive prediction encoding unit 11 performs prediction using the following equation.

f (a0, a1, a2, a3, ..., b0, b1, b2, b3, ...)
However,
f: Prediction function a0, a1, a2, a3, ...: Pixel value of the encoding target frame b0, b1, b2, b3, ...: Reference frame pixel value The reference frame is temporally forward Frame, the rear frame, and both the front and rear frames, the reference frame needs to be decoded first on the decoder side. The position of the pixel used for prediction is determined under the following conditions. Let the position of the pixel to be encoded be (x, y) (x is the position in the horizontal direction and y is the position in the vertical direction). In the case of a frame to be encoded, it is a pixel that is decoded first in the vicinity and on the decoding side. For example, when decoding from the leftmost pixel of the first row to the right and then decoding the second row from left to right for each row, the position (xa, ya) of the pixel used for prediction is as follows: It is necessary to satisfy the conditions.

y−y0 <ya <y and x−x0 <xa <x + x1
Or
ya = y and x−x2 <xa <x
However,
y0: Appropriate integer that determines the range in the vicinity in the vertical direction x0, x1, x2: Appropriate integer that determines the range in the vicinity in the horizontal direction The position (xb, yb) of the pixel used for prediction of the reference frame satisfies the following conditions There is a need.

y + vy-y3 <ya <y + vy + y4 and x + vx-x3 <xa <x + vx + x4
However,
y3, y4: Appropriate integers that define the vertical neighborhood range x3, x4: Appropriate integers that define the horizontal neighborhood range V (vx, vy): Motion vector to the reference frame Motion vector V (vx, vy) Is obtained by performing motion estimation on the reference frame.

  The entropy encoding unit 12 entropy-encodes the residual signal of each band and the motion vector used in the space-time adaptive prediction encoding unit 11 to create an encoded bitstream.

  FIG. 5 shows a basic configuration of a space-time adaptive predictive coding unit that realizes the space-time adaptive predictive coding method. In FIG. 5, 31 is a block unit motion estimation means, 32 is a shift means, 33 is a two-dimensional predictor, 34 is a motion estimation three-dimensional predictor, 35 is a correlation coefficient R calculation means, and 36 is a first decision branching means. , 37 is addition / subtraction means, and 38 is addition / subtraction means.

  As predictors, three types of two-dimensional predictors and seven types of three-dimensional predictors are prepared, and the predictors are switched by inter-frame correlation.

  The three-dimensional predictor is effective when the inter-frame correlation is strong, but there is a possibility that the residual signal becomes large when the inter-frame correlation is weak. Therefore, a method of switching to a two-dimensional predictor is adopted when the inter-frame correlation is weak. In order to switch between the two-dimensional predictor and the three-dimensional predictor, the correlation coefficient of the decoded signal in the vicinity of the encoding target pixel in the current frame and the reference frame is calculated. When the correlation coefficient is large, that is, when the waveform of the signal in the current frame and the signal in the reference frame are similar, the prediction accuracy is considered to improve, so that three-dimensional prediction is performed. In other cases, two-dimensional prediction is performed.

  A prediction method for each predictor and a specific switching method for the predictor will be described below. The prediction method of each predictor is as follows.

Two-dimensional predictor 0 Predicted signal y = min (a, b) (1)
Two-dimensional predictor 1 Predicted signal y = max (a, b) (2)
Two-dimensional predictor 2 Predicted signal y = a + bc (3)
Three-dimensional predictor 3 Predicted signal y = min (a, x ′) (4)
Three-dimensional predictor 4 Prediction signal y = max (a, x ′) (5)
Three-dimensional predictor 5 Predicted signal y = a + x′−a (6)
Three-dimensional predictor 6 Predicted signal y = min (b, x ′) (7)
Three-dimensional predictor 7 Prediction signal y = max (b, x ′) (8)
Three-dimensional predictor 8 Predicted signal y = b + x′−b (9)
Three-dimensional predictor 9 Predicted signal y = (a + b + x ′) / 3 (10)
As shown in FIG. 7A, a, b, and c used here are decoded values of the upper, left, and upper right pixels adjacent to the pixel x to be encoded. a ′, b ′, x ′ are the decoded values of the pixels of the reference frame, and are based on the motion vector (k, l) between the frame to be encoded and the reference frame (k: horizontal direction, l: vertical direction) Define the pixel position. The motion vector is calculated in advance by a block matching method or the like in units of small blocks of L × L pixels and transmitted as additional information. If the position of the pixel x to be encoded is (i, j), the position of x ′ is (i + k, j + l). Further, as shown in FIG. 7B, a ′ and b ′ are the pixels on the left side adjacent to x ′. As the reference frame, any one of a temporally forward frame, a backward frame, and a forward and backward frame is used. However, on the decoder side, the reference frame needs to be decoded first.

If R ≦ T0 and c ≧ max (a, b), select 2D predictor 0 Select R ≦ T0 and c ≦ min (a, b) Select 2D predictor 1 R ≦ T0 and min (a, b) b) If <c <max (a, b) Select 2D predictor 2 If R> T0 and S> T1 and a ′ ≧ max (a, x ′) Select 3D predictor 3 R> T0 When S> T1 and a ′ ≦ min (a, x ′), the three-dimensional predictor 4 is selected. R> T0, S> T1, and min (a, x ′) <a ′ <max (a, x ′) In the case of 3D predictor 5 is selected R> T0 and S <−T1 and b ′ ≧ max (b, x ′) 3D predictor 6 is selected R> T0 and S <−T1 and b ′ ≦ min In the case of (b, x ′), the three-dimensional predictor 7 is selected. In the case of R> T0, S <−T1, and min (b, x ′) <b ′ <max (b, x ′), the three-dimensional predictor. 8 is selected When R> T0 and -T1 <S <T1, the three-dimensional predictor 9 is selected.
R = aa ′ + bb ′ + cc ′ + dd ′ − (a + b + c + d) (a ′ + b ′ + c ′ + d ′) (11)
S = | x′−b ′ | − | x′−a ′ | (12)
T0, T1: Thresholds a, b, c, d, a ′, b ′, c ′, d ′, x ′: Decoded values of the pixels shown in FIG.

  Here, R is a correlation coefficient, and when R is larger than the threshold value T0, a three-dimensional predictor is selected. In other cases, a two-dimensional predictor is selected. This selection is actually performed in two stages. That is, the first decision branch means 36 in FIG. 5 performs the first decision branch based on the R value, and first switches between the two-dimensional predictor or the three-dimensional predictor to switch the two-dimensional predictor 33 or the three-dimensional predictor 34. The remaining second decision branch is performed to select one of the predictors 1 to 9.

  The switching method of the three predictors of the two-dimensional predictor has been proposed and is the same as the method adopted in the international standard JPEG-LS for still image lossless compression. When it is determined that there is an edge in the vertical direction and the horizontal method, prediction is performed using one pixel adjacent to each edge direction, and prediction is performed using three adjacent pixels in other cases.

  In the case of three-dimensional prediction as well as the two-dimensional predictor, adaptive prediction is performed in which the predictor is switched depending on the state of the neighborhood signal value of the encoding target pixel. When it is determined that there is an edge in the vertical direction or the horizontal direction, prediction is performed using signals adjacent to each edge direction of the current frame and the reference frame. As for the edge direction, the difference absolute value | x′−a ′ | in the vertical direction of the reference frame is compared with the absolute difference value | x′−b ′ | in the horizontal direction, and a direction larger than the threshold T1 is determined as an edge. To do. S in the equation (12) is a difference value between them and is a parameter for determining the edge direction. When it is determined that the edge is a vertical edge for determining the edge direction, a prediction signal is selected in the same manner as the two-dimensional predictor with respect to the vertical signals of the current frame and the reference frame. The predictor is selected in the same manner for the edge in the horizontal direction. When it is determined that it is not an edge, the average value of three neighboring pixels is set as the predicted value.

  The addition / subtraction means 37 or 38 outputs a difference between the prediction signal output from the predictor and the original frame signal, that is, a prediction residual signal. The prediction residual signal is quantized by a quantizer (not shown). The quantized residual signal and the motion vector used in the three-dimensional predictor are input to the entropy encoding unit 12 in FIG. 1, and an encoded bit stream is output. When the quantization step of the quantizer is set to 1, the present encoding method is a lossless encoding method.

[Second Embodiment]
FIG. 4 shows a basic configuration for realizing the decoding method for decoding the data encoded in the first embodiment. In FIG. 4, 20 is an entropy decoding unit, 21 is a space-time adaptive predictive decoding unit provided for each band, and 22 is a band synthesizing unit.

  A decoding method as an operation example of the configuration of FIG. 4 is shown below. First, the entropy decoding unit 20 obtains a motion vector and a prediction residual signal used for prediction from the encoded bit stream. Next, the encoded signal is decoded using the image signal and residual signal that have already been decoded by the space-time adaptive predictive encoding unit 21. Next, the band synthesizing unit 22 synthesizes the outputs of the spatiotemporal adaptive prediction encoding units 21 to decode the image.

  FIG. 6 shows a basic configuration of the space-time adaptive predictive decoding unit. In FIG. 6, 41 is a shift means, 42 is a two-dimensional predictor, 43 is an adder, 44 is a motion estimation three-dimensional predictor, 45 is an adder, 46 is a correlation coefficient R calculator, and 47 is a first determination. It is a branching means.

  The space-time adaptive predictive decoding method as the operation example of FIG. 6 is as follows. First, in order to determine whether to use the two-dimensional predictor 42 or the three-dimensional predictor 44, from the decoded signal of the reference intra-frame signal and the current intra-frame signal shifted by the shift means 41 using the motion vector, The correlation coefficient R is calculated by the correlation coefficient R calculation means 46. When the correlation coefficient R is larger than the threshold value T0, the first decision branching unit 47 switches to the motion estimation three-dimensional predictor 44 side to perform three-dimensional prediction, and otherwise, the two-dimensional predictor 42. Switch to the side and perform 2D prediction. The configuration of the two-dimensional predictor 42 and the motion estimation three-dimensional predictor 44 and the switching of a plurality of predictors prepared therein are the same as those in the first embodiment. The motion estimation three-dimensional predictor 44 generates a prediction signal using the decoded signal of the current intraframe signal and the reference intraframe signal shifted using the motion vector, and the adder 45 adds the residual signal to the prediction signal. Is added to restore the target signal of the current frame. The two-dimensional predictor 42 generates a prediction signal using the decoded signal of the signal in the current frame, and the adding unit 43 adds the residual signal to the prediction signal, thereby restoring the target signal of the current frame.

[Third Embodiment]
In this embodiment, the spatio-temporal adaptive predictive coding unit 11 of the first embodiment estimates the motion of the target object in each band by the block matching method. In the block matching method, the following SAD (Sum of Absolute Difference) value is calculated in order to obtain a motion vector.

SAD (k, l) = Σi _{= 1} ^L Σj _{= 1} ^L | x (i, j) −y (i + k, j + l) | (13)
Here, SAD (k, l) is calculated in the range of (k, l) -w <k, 1 <w (w is the window size), and the vector (k, l) giving the minimum SAD value moves. Selected as a vector. In order to improve the prediction accuracy of the three-dimensional predictor, SAD is usually calculated for each band. The calculation of SAD takes a considerable amount of time. Considering this, the following three simple encoding methods (1), (2), and (3) are proposed to reduce the calculation time while minimizing the decrease in encoding efficiency.

  (1) The inter-frame correlation and intra-frame correlation of the band-divided signal are stronger as the frequency is lower and weaker as the frequency is higher. Since signals other than the lowest frequency band have weak inter-frame correlation and intra-frame correlation, the effect of space-time adaptive prediction coding does not appear so significantly. Using this property, space-time adaptive predictive coding is applied only to the lowest frequency band. The calculation amount of SAD is extremely reduced, and the two-dimensional and three-dimensional prediction processing times are also reduced. The motion vector in the lowest frequency band and the prediction residual signal are entropy coded, and the signal other than the lowest frequency band is directly entropy coded.

  FIG. 8 shows a basic configuration diagram for realizing this method. In FIG. 8, 50 is a band dividing unit, 51 is a space-time adaptive prediction encoding unit for the lowest frequency band, and 52 is an entropy encoding unit 52. First, the input image signal is divided into a plurality of spatial resolution bands by the band dividing unit 50. Next, a residual signal is generated by the spatio-temporal adaptive prediction encoding unit 51 only for the lowest frequency band among the divided bands, and entropy encoding unit 52 performs entropy encoding. The entropy encoding unit 52 directly encodes signals in other bands.

(2) A motion vector is obtained by performing motion estimation only in the lowest frequency band. In other bands, space-time adaptive prediction encoding is performed using motion vectors based on the motion vector in the lowest frequency band. Since the movement of the target object does not change for each band, the movement of the high-frequency object and the movement of the low-frequency object should be equal. That is, it is assumed that the small block motion vectors existing in the same direction from the low range to the high range are the same. For example, the movements of the small blocks indicated by the arrows in FIG. However, the number of pixels doubles in the vertical and horizontal directions each time the band increases in one high region, so the motion vector in one high region is
(2k, 2l)
It becomes. The motion vector of the N stage high region is
(N × k, N × l)
It becomes.

  The basic configuration diagram for realizing this method is the same as that in FIG. 1, but the motion vector used in the space-time adaptive prediction encoding unit 11 for each band is a motion vector based on the motion vector in the lowest frequency band. Use. The motion vector in the lowest frequency band and the prediction residual signal in the entire band are entropy encoded by the entropy encoding unit 12.

(3) When calculating SAD, it is performed from low to high. In the example of FIG. 3, the order is LL3->HL3->LH3->HH3->HL2->LH2->HH2->HL1->LH1-> HH1. First, SAD is calculated for each small block in the lowest frequency band. SAD
SAD (k, l)> T (threshold value) (14)
For the signals in the small blocks, it is determined that the inter-frame correlation is weak, and the two-dimensional prediction is performed without performing the spatiotemporal adaptive prediction process. In addition, the entropy coding is directly performed on the corresponding high-frequency small block signals without performing the space-time adaptive prediction coding.

For a signal in a block that does not satisfy Expression (14), space-time adaptive prediction coding is performed as usual. In the subsequent high-band block, SAD
SAD (k, l)> Tx (threshold in high band. Threshold varies depending on band) (15)
In this case, it is determined that the correlation between frames is weak, and direct entropy coding is performed without performing space-time adaptive prediction coding. In addition, when the SAD of the corresponding low-frequency block already satisfies the equation (14), it is determined that the direct entropy encoding is performed, so the calculation of the SAD and the determination of the equation (15) Not performed.

  A small block that satisfies the conditions of equations (14) and (15) transmits a unique code LIMIT instead of a motion vector as additional information, and does not transmit additional information in the corresponding high-frequency small block. The motion vector, the unique code LIMIT, and the prediction residual signal are entropy encoded.

  The basic configuration diagram for realizing this method is the same as that in FIG. 1, but each space-time adaptive predictive encoder 11 has a function of calculating SAD from low to high in block unit motion estimation. And a function for determining whether to perform space-time adaptive predictive coding, a function for transmitting a unique code LIMIT, and the like. The entropy encoder 12 entropy-encodes the motion vector, the unique code LIMIT, a signal not subjected to space-time adaptive prediction encoding, or a prediction residual signal from each space-time adaptive prediction encoder 11.

[Fourth Embodiment]
A simple decoding method of a signal encoded by a simple encoding method according to a third embodiment will be described.

  (1) FIG. 9 shows a basic configuration diagram for decoding a signal encoded by the simple encoding (1). In FIG. 9, 60 is an entropy decoding unit, 61 is a space-time adaptive prediction decoding unit for the lowest frequency band, and 62 is a band synthesizing unit.

  In this method, using the motion vector entropy-decoded by the entropy decoding unit 60 and the prediction residual signal, the spatio-temporal prediction decoding unit 61 decodes the signal in the lowest frequency band. Other bands are directly decoded by the entropy decoding unit 60 by entropy decoding. The band synthesizer 62 synthesizes the output of each band and decodes the image.

(2) The basic configuration diagram for realizing the decoding method for decoding the signal encoded by the simple encoding (2) is the same as that in FIG. 4, but the spatio-temporal adaptive predictive decoding unit for each band. The motion vector used in 21 is a motion vector based on the motion vector in the lowest frequency band. That is, based on the motion vector (k, l) of the lowest frequency band entropy decoded by the entropy decoding unit 20, the motion vector of the N-stage high band is
(N × k, N × l)
Calculate according to Based on the result, the spatio-temporal adaptive predictive decoding unit 21 performs decoding processing for each band, and the band synthesizing unit 22 synthesizes the output of each band to decode the image.

  (3) The basic configuration diagram for realizing the decoding method for decoding the signal encoded by the simple encoding (3) is the same as that in FIG. 4, but the space-time adaptive predictive decoding unit for each band. 21, using the motion vector (k, l) of the lowest frequency band entropy decoded by the entropy decoding unit 20 and the prediction residual signal, the lowest frequency band to the highest frequency band (in the example of FIG. 3). LL3-> HL3-> LH3-> HH3-> HL2-> LH2-> HH2-> HL1-> LH1-> HH1) in order, and output by performing spatio-temporal adaptive predictive decoding processing or by entropy decoding by eigencode LIMIT Direct output of frequency band signals. The band synthesizer 22 synthesizes the output of each band and decodes the image.

It is a figure which shows the basic composition for implement | achieving the moving image encoding method by 1st Embodiment of this invention. It is a figure explaining an octave division. It is a figure explaining the zone | band division | segmentation of an image. It is a figure which shows the basic composition for implement | achieving the moving image decoding method by 2nd Example of this invention. It is a figure which shows the basic composition of the space-time adaptive prediction encoding part in this invention. It is a figure which shows the basic composition of the space-time adaptive prediction decoding part in this invention. (A), (b) is a figure explaining the signal used for prediction. It is a figure which shows the basic composition for implement | achieving simple encoding (1) by 3rd Example of this invention. It is a figure which shows the basic composition for implement | achieving simple decoding (1) by 4th Example of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 ... Band division part 11 ... Spatio-temporal adaptive prediction encoding part 12 ... Entropy encoding part 20 ... Entropy decoding part 21 ... Spatio-temporal adaptive prediction decoding part 22 ... Band synthesis part 31 ... Block unit motion estimation means 32 ... Shift means 33 ... Two-dimensional predictor 34 ... Motion estimation three-dimensional predictor 35 ... Correlation coefficient R calculating means 36 ... First decision branching means 37 ... Addition / subtraction means 38 ... Addition / subtraction means 41 ... Shift means 42 ... Two-dimensional predictor 43 ... Addition means 44 ... Motion estimation three-dimensional predictor 45 ... Addition means 46 ... Correlation coefficient R calculation means 47 ... First decision branch means 50 ... Band division section 51 ... Spatio-temporal adaptive prediction encoding section 52 ... Entropy encoding section 60 ... Entropy decoding unit 61 ... Spatio-temporal adaptive prediction decoding unit 62 ... Band synthesis unit

Claims (5)

In a lossless encoding method for moving images,
The original image is divided into bands,
A motion vector is obtained by performing SAD calculation for each lowest frequency band among the divided bands to obtain a motion vector. Pixels in a block that give an SAD value larger than a predetermined threshold T are subjected to two-dimensional prediction, and are equal to or lower than the predetermined threshold T. Pixels in the block giving the SAD value perform space-time adaptive prediction to obtain a prediction residual signal, encode the prediction residual signal and the motion vector used in the space-time adaptive prediction,
In the second lowest frequency band coding, the pixels of the spatial block corresponding to the block for which the two-dimensional prediction is performed in the lowest frequency band are directly coded, and the other blocks are subjected to SAD calculation to obtain a motion vector. The pixels in the block that give a SAD value greater than a predetermined threshold T2 are directly encoded, and the pixels in the block that give a SAD value less than or equal to the threshold T2 are subjected to spatiotemporal adaptive prediction to obtain a prediction residual signal. Encode the residual signal and the motion vector used in the space-time adaptive prediction,
In the subsequent high frequency band encoding, the pixels of the spatial block corresponding to the block directly encoded in the frequency band encoded immediately before are directly encoded, and the other blocks perform the SAD calculation to obtain the motion vector. The pixels in the block that give a SAD value larger than the threshold value Tx determined for each frequency band are directly encoded, and the pixels in the block that give a SAD value less than or equal to the predetermined threshold value Tx are predicted by performing space-time adaptive prediction. When obtaining a residual signal and encoding the prediction residual signal and the motion vector used in the space-time adaptive prediction,
In the spatio-temporal adaptive prediction, the correlation coefficient between the current pixel and the reference frame pixel value to be encoded between the current frame and the reference frame shifted by the shift means is calculated using the motion vector obtained by the SAD calculation. If the correlation coefficient is small, switch to three-dimensional prediction that performs prediction using the encoding target pixel neighboring signal values of the current frame and the reference frame. If the correlation coefficient is small, the encoding target pixel neighboring signal of the current frame Switch to two-dimensional prediction that uses only values to make predictions,
In the case of switching to the two-dimensional prediction, a prediction residual signal is calculated by switching a plurality of two-dimensional predictors for each pixel from the encoding target pixel vicinity signal value of the current frame,
When switching to the three-dimensional prediction, the prediction residual signal is calculated by switching a plurality of three-dimensional predictors for each pixel from the encoding target pixel neighboring signal values of the current frame and the reference frame. A moving image lossless encoding method.   The video lossless encoding method according to claim 1, wherein a code indicating that the spatio-temporal adaptive prediction processing has not been performed is added to an encoded signal of a block in which an SAD value larger than a threshold value is obtained. In a lossless decoding method for outputting a moving image,
Receiving a direct coding block for each band, a prediction residual signal and a motion vector encoded by the video lossless encoding method according to claim 1;
In the order of the lowest frequency band to the highest frequency band, for each band, the direct coding block, the prediction residual signal and the motion vector are decoded, and the space-time adaptive prediction decoding is further performed from the prediction residual signal and the motion vector,
When decoding a moving image by combining the decoded bands,
In the spatio-temporal adaptive prediction decoding, a correlation coefficient between a decoding target pixel neighboring signal value of a reference frame shifted using the motion vector and a decoding target pixel neighboring signal value composed of a decoded signal of a signal in the current frame When the correlation coefficient is large, switch to 3D prediction, and when the correlation coefficient is small, switch to 2D prediction.
In the case of switching to the two-dimensional prediction, a prediction signal is calculated by switching a plurality of two-dimensional predictors for each pixel from the decoding target pixel neighboring signal value including the decoded signal of the signal in the current frame,
In the case of switching to the three-dimensional prediction, from the decoding target pixel neighboring signal value composed of the decoded signal of the signal in the current frame and the decoding target pixel neighboring signal value of the reference frame shifted using the motion vector, Calculate prediction signals by switching multiple 3D predictors for each pixel,
A moving picture lossless decoding method comprising decoding the target signal by adding the prediction residual signal to the prediction signal. In a lossless encoding device for moving images,
The procedure for dividing the original image into bands,
A motion vector is obtained by performing SAD calculation for each lowest frequency band among the divided bands to obtain a motion vector. Pixels in a block that give an SAD value larger than a predetermined threshold T are subjected to two-dimensional prediction, and are equal to or lower than the predetermined threshold T. A pixel in the block that gives the SAD value performs a spatiotemporal adaptive prediction to obtain a prediction residual signal, and encodes the prediction residual signal and the motion vector used in the spatiotemporal adaptive prediction.
In the second lowest frequency band coding, the pixels of the spatial block corresponding to the block for which the two-dimensional prediction is performed in the lowest frequency band are directly coded, and the other blocks are subjected to SAD calculation to obtain a motion vector. The pixels in the block that give a SAD value greater than a predetermined threshold T2 are directly encoded, and the pixels in the block that give a SAD value less than or equal to the threshold T2 are subjected to spatiotemporal adaptive prediction to obtain a prediction residual signal. A procedure for encoding a residual signal and a motion vector used in the space-time adaptive prediction;
In the subsequent high frequency band encoding, the pixels of the spatial block corresponding to the block directly encoded in the frequency band encoded immediately before are directly encoded, and the other blocks perform the SAD calculation to obtain the motion vector. The pixels in the block that give a SAD value larger than the threshold value Tx determined for each frequency band are directly encoded, and the pixels in the block that give a SAD value less than or equal to the predetermined threshold value Tx are predicted by performing space-time adaptive prediction. Obtaining a residual signal, and executing a procedure for encoding the prediction residual signal and the motion vector used in the space-time adaptive prediction,
In the spatio-temporal adaptive prediction, the correlation coefficient between the current pixel and the reference frame pixel value to be encoded between the current frame and the reference frame shifted by the shift means is calculated using the motion vector obtained by the SAD calculation. If the correlation coefficient is small, switch to three-dimensional prediction that performs prediction using the encoding target pixel neighboring signal values of the current frame and the reference frame. If the correlation coefficient is small, the encoding target pixel neighboring signal of the current frame Switch to two-dimensional prediction that uses only values to make predictions,
In the case of switching to the two-dimensional prediction, a prediction residual signal is calculated by switching a plurality of two-dimensional predictors for each pixel from the encoding target pixel vicinity signal value of the current frame,
When switching to the three-dimensional prediction, a procedure for calculating a prediction residual signal by switching a plurality of three-dimensional predictors for each pixel from the signal to be encoded pixel vicinity signal values of the current frame and the reference frame is executed. Program for. In a lossless decoding device that outputs moving images,
A procedure for receiving a direct coding block for each band, a prediction residual signal, and a motion vector encoded by the moving image lossless encoding method according to claim 1;
A procedure for decoding a direct coding block, a prediction residual signal, and a motion vector for each band in order from the lowest frequency band to the highest frequency band, and further performing space-time adaptive prediction decoding from the prediction residual signal and the motion vector,
A procedure for decoding the moving image by performing band synthesis on the decoded band,
In the spatio-temporal adaptive prediction decoding, a correlation coefficient between a decoding target pixel neighboring signal value of a reference frame shifted using the motion vector and a decoding target pixel neighboring signal value composed of a decoded signal of a signal in the current frame When the correlation coefficient is large, switch to 3D prediction, and when the correlation coefficient is small, switch to 2D prediction.
In the case of switching to the two-dimensional prediction, a prediction signal is calculated by switching a plurality of two-dimensional predictors for each pixel from the decoding target pixel neighboring signal value including the decoded signal of the signal in the current frame,
In the case of switching to the three-dimensional prediction, from the decoding target pixel neighboring signal value composed of the decoded signal of the signal in the current frame and the decoding target pixel neighboring signal value of the reference frame shifted using the motion vector, Calculate prediction signals by switching multiple 3D predictors for each pixel,
A program for executing a procedure for decoding a target signal by adding the prediction residual signal to the prediction signal.

Images