JP2008512023A - Method and apparatus for motion prediction - Google Patents

Abstract

A spatial hierarchy method and apparatus for video streams is disclosed. A reference motion vector is introduced into the compression scheme of the present invention, and according to this reference motion vector, the base layer and the enhancement layer respectively obtain the motion vector of the corresponding frame of the image of the video stream, whereby the base layer and Each enhancement layer is generated. The introduced reference motion vector associates motion prediction for the base layer with motion prediction for the enhancement layer, thereby reducing the overall amount of motion prediction computation for the base layer and enhancement layer. Furthermore, since the reference frame for obtaining the reference motion vector is obtained from the original video sequence and no further strict operation is performed on the original video sequence, the reference motion vector represents the actual motion within the video sequence. Reflect well.

Description

  The present invention relates to a method and apparatus for compressing a video stream, and more particularly, to a method and apparatus for compressing a video stream using a spatial layered compression scheme.

  Due to the large amount of data contained in digital video, transmitting a high resolution video signal is a major problem when producing high definition television programs. In particular, each frame of a digital image is a still picture (also called an image) composed of a group of pixel points (also called pixels). The amount of pixels depends on the display definition of the particular system. Therefore, the amount of original digital information of high definition video is very large. MPEG-2, MPEG-4 and H.264. Many video compression standards, such as H.263, are generated to reduce the necessary data to be transmitted.

  All of the above-mentioned standards support layer technologies, including spatial layers, temporal layers, SNR layers, etc. In hierarchical coding, a bitstream is divided into two types of bitstreams or layers for coding that exceed the bitstream or layer. Thereafter, during decoding, the respective layers are combined as desired to form a high resolution signal. For example, the base layer provides a low resolution video stream, and the enhancement layer provides additional information to enhance the base layer image.

  In addition to adapting the hierarchical compression scheme within the current spatial hierarchical compression scheme, motion prediction is used to obtain a predicted image according to the relationship between the previous frame and the subsequent frame Is done. Before being compressed, the input video stream is processed to form I, P and B frames and form a sequence according to the parameter settings. An I frame is encoded according to its own information only, a P frame is encoded predictively according to the closest I or P frame at the front, and a B frame is encoded predictively according to itself or the frames before and after it. Turn into.

  FIG. 1 is a block diagram of a video coder 100 that supports MPEG-2 / MPEG-4 spatial hierarchical compression. The video encoder 100 includes a base encoder 112 and an enhancement and encoder 114. The base encoder includes a downsampler 120, motion prediction (ME) means 122, motion compensator (MC) 124, right angle conversion circuit (for example, discrete cosine transform (DCT)) 135, quantizer (Q) 132, variable length code. It includes a quantizer (VLC) 134, a bit rate control circuit 135, an inverse quantizer (IQ) 138, an inverse transform circuit (IDCT) 140, switches 128 and 144, and an upsampler 150. The enhancement encoder 114 includes a motion predictor 154, a motion compensator 155, a right angle transform (eg, DCT transform) circuit 158, a quantizer 160, a variable length encoder 162, a bit rate control circuit 164, an inverse quantizer 166, An inverse conversion circuit (IDCT) 168 and switches 170 and 172 are included. All functions of the above means are known in the art and will not be described in detail here.

  Motion prediction is one of the most time-consuming parts in a video compression system, that is, it is known that the coding efficiency of a video compression system decreases as the amount of motion prediction calculations increases. In the hierarchical coding compression scheme described above, while predicting a video image of the same frame, motion prediction is performed for both the base layer and the enhancement layer, and there is no relationship between them. However, when motion prediction is performed for each of the base layer and enhancement layer, the prediction is performed for the same image frame, so a relatively large portion of the search process is repeated, which results in a large amount of computation for motion prediction. The coding efficiency of the compression scheme is low. Therefore, there is a need for a spatial layered video compression scheme with good coding efficiency.

  The present invention is directed to a highly efficient spatial hierarchical compression method in order to overcome the problems of the spatial hierarchical compression scheme described above by introducing a reference motion vector. The base layer motion prediction can be associated with the enhancement layer motion prediction, the inherently iterative search process can be completed at once, and a small amount of search is performed, which Based on this, the computational complexity of motion prediction is reduced and the efficiency of the compressed encoding is improved.

  Embodiments according to the present invention disclose a method and apparatus for spatially hierarchical compression of a video stream. First, for each frame of the image in the video stream, process the original video stream to obtain a reference motion vector, then downsample the reference motion vector and downsample the video stream Secondly, obtaining the motion vector of the corresponding frame of the image of the downsampled video stream according to the downsampled reference motion vector, and then using each motion vector to obtain the downsampled video stream Process the corresponding frame of the image to generate a base layer, and finally obtain the motion vector of the corresponding frame of the image of the video stream while generating the enhancement layer according to the reference motion vector Shi Processing the video stream by using the motion vector and the base layer, thereby generating an enhancement layer.

  An alternative embodiment according to the present invention discloses yet another method and apparatus for spatial hierarchical compression of a video stream. First, the video stream is downsampled to obtain a reference motion vector for each frame of the image of the downsampled video stream, and second, downsampled according to the reference motion vector. Process the motion vector of the corresponding frame of the image in the video stream, then use the motion vector to process the downsampled video stream, thereby generating the base layer, and finally the enhancement layer During the generation, the reference motion vector is upsampled, the motion vector of the corresponding frame of the image of the generated video stream is obtained according to the upsampled reference motion vector, and the motion vector and the base layer are used. Processing the video stream by, thereby generating the enhancement layer.

  Another embodiment according to the invention discloses a further method and apparatus for spatially hierarchical compression of a video stream. First, the video stream is processed, thereby generating a base layer, and then up-sampling the motion vector for each frame of the base layer image, thereby providing a reference to the corresponding frame of the image And finally obtain the motion vector of the corresponding frame of the video stream image according to the reference motion vector, thereby processing the enhancement stream by processing the video stream using the motion vector and the base layer. Generate.

  Other objects and advantages will become apparent upon reading the following detailed description and the accompanying drawings and claims, along with a thorough understanding of the present invention. The present invention will be described in more detail with reference to embodiments and the accompanying drawings. Throughout the drawings, the same reference numerals indicate similar or corresponding features or functions.

  FIG. 2 is a conceptual diagram of an encoding system using a reference motion vector according to an embodiment of the present invention. The encoding system 200 is used for hierarchical compression, the base layer portion is used to provide low resolution base information of the video stream, and the enhancement layer is used to transmit edge enhancement information. Both types of information are recombined at the receiving terminal to form high resolution picture information.

  The encoding system 200 includes an acquisition unit 216, a base layer acquisition unit 212, and an enhancement layer acquisition unit 214.

  Here, the acquisition unit 216 is used to process the original video stream, and thereby acquires a motion vector that is a reference for each frame of the image of the video stream. The acquisition unit 216 includes a motion prediction unit 276 and a frame memory 282. The frame memory 282 is used for storing the original video sequence. The motion prediction means 276 is used to obtain a reference frame (eg, I or P frame) from the frame memory 282 and performs motion prediction on the current frame (eg, P frame) according to the reference frame, thereby calculating the current frame by calculation. A reference motion vector of the frame is derived.

  The base layer acquisition unit 212 processes the video stream using the reference motion vector, thereby generating a base layer. Means 212 has downsamples 120, 286. Downsampler 120 is used to downsample the original video stream. The downsampler 286 is used to downsample the reference motion vector. Of course, those skilled in the art know that it is possible to perform downsampling on the original video stream and the reference motion vector with one downsampler.

  The base layer acquisition unit 212 further includes a motion vector acquisition unit 222. The motion vector acquisition means 222 is used to acquire the motion vector of the corresponding frame of the image of the downsampled video stream based on the downsampled reference motion vector. The process by which the motion vector acquisition unit 222 acquires a motion vector is described as follows.

  Base layer acquisition means 212 further comprises base layer generation means 213, using the motion vectors to process the downsampled video stream and thereby generate the base layer. Except for the downsamplers 120 and 286 and the motion vector acquisition unit 222, all other units in the base layer acquisition unit 212 are basically the same as the base layer encoder in FIG. 1 and belong to the base layer generation unit 213. , Motion compensator 124, DCT transform circuit 130, quantizer 132, variable length encoder 134, bit rate control circuit 135, inverse quantizer 138, inverse transform circuit 140, arithmetic units 125 and 148, switches 128 and 144, and An upsampler 150 is included. The process in which the base layer generation unit 213 generates the base layer based on the motion vector output from the motion vector acquisition unit 222 is substantially the same as that of the prior art, and will be described in detail below.

  Compared to FIG. 1, in the base layer acquisition means 212 described above, the same reference numerals indicate components having the same or similar functions or features. The difference between the motion prediction unit 122 and the motion vector acquisition unit 222 is a method for acquiring a motion vector. The motion prediction means 122 of FIG. 1 directly uses a reference frame in a frame memory (not shown) for searching in a large search window to obtain the motion vector of the corresponding frame of the image of the video stream, and FIG. The motion vector acquisition means 222 performs a search with a small search window based on the reference motion vector in order to acquire the motion vector of the corresponding frame of the video stream image.

  The enhancement layer acquisition unit 214 processes the video stream by using the reference motion vector and the base layer, thereby generating an enhancement layer. The means 214 includes a motion vector acquisition means 254 and an enhancement layer generation means 215.

  The motion vector acquisition means 254 is used to acquire the motion vector of the corresponding frame of the image of the video stream based on the reference motion vector.

  The enhancement layer generation means 215 processes the video stream by using the motion vector and the base layer, thereby generating an enhancement layer. The enhancement layer acquisition unit 214 is substantially the same components as those in the enhancement layer encoder 114 of FIG. 1 except for the motion vector acquisition unit 254, all of which belong to the enhancement layer generation unit 215 and are motion compensators. 155, a DCT circuit 158, a quantizer 160, a variable length encoder 162, a bit rate control circuit 164, an inverse quantizer 166, an inverse DCT circuit 168, and switches 170 and 72. These components are similar in function to the corresponding components of the base layer acquisition means 212. The process of generating the enhancement layer by using the motion vector output from the motion vector acquisition unit 254 by the enhancement layer generation unit 215 is essentially the same as the prior art process, and the detailed description is as follows. Given.

  Compared to FIG. 1, in the base layer acquisition means 214 described above, the same reference numerals indicate components having the same or similar features and functions. The difference between the motion prediction means 154 and the motion vector acquisition means 254 is how they acquire the motion vector. The motion prediction means 154 of FIG. 1 directly uses a reference frame in a frame memory (not shown) to search in a large search window to obtain the motion vector of the corresponding frame of the image of the video stream. The motion vector acquisition means 254 further performs a search in a small search window based on the reference motion vector in order to acquire the motion vector of the corresponding frame of the image of the video stream.

  As shown in FIG. 2, the base layer acquisition unit 212 and the enhancement layer acquisition unit 214 use the reference motion vector output by the acquisition unit 216 to acquire each motion vector, thereby generating the base layer and the enhancement layer. The process to do is described in more detail below.

  The original video stream is input to the acquisition unit 216 and then supplied to the motion prediction unit 276 and the frame memory 282, respectively. Note that, before being supplied to the acquisition means 216, the video stream is processed to form I, P, B frames, and I, B, P, B, P.P. . . , B, P form such a series. The input video sequence is stored in the frame memory 282. The motion predictor 276 is used to obtain a reference frame (eg, I frame) from the frame memory 282, and performs motion prediction on the current frame (eg, P frame) according to the reference frame, thereby Calculate the reference motion vector of the macroblock. A macroblock is a sub-block with 16 * 16 pixels in the current encoded frame, and the block between the current macroblock and the reference frame is matched to calculate the reference motion vector of the current macroblock. This is used to obtain the reference motion vector of the current frame.

  There are four methods used for picture prediction in MPEG, including intra-frame coding, forward prediction coding, backward prediction coding and bi-directional prediction coding. The I frame is an intra frame encoded image, the P frame is an image of intra frame encoding, forward prediction encoding, or backward prediction encoding, and the B frame is intra frame encoding, forward prediction encoding, or bidirectional. It is an image of predictive coding.

  The motion prediction unit 276 performs forward prediction on the P frame and calculates a reference motion vector. Furthermore, the motion prediction means performs forward or bi-directional prediction on the B frame and calculates a reference motion vector. Motion prediction is not required for intra-frame coding.

  Taking forward prediction to a P frame as an example, the process of calculating a reference motion vector is described as follows. The motion prediction means 276 reads the previous reference frame from the frame memory 282 and searches in the search window of the previous reference frame of the macroblock that most closely matches the pixel block of the current frame. There are several algorithms of match searching in the prior art, and generally the state of matching is the square-square error (MAD) or absolute value (MSE) between the pixel of the current input block and the pixel of the corresponding block of the reference frame. ). The corresponding block of the reference frame with the minimum value MAD or MSE is the optimal matching block, and the relative position of the optimal matching block with respect to the current block position is the reference motion vector.

  Through the processing described above, the motion prediction unit 276 in the acquisition unit 216 acquires the reference motion vector of the frame of the image of the video stream. After being downsampled by the downsampler 286, the reference motion vector is supplied to the motion prediction means 222 of the base layer acquisition means 212 so that the motion prediction means 222 performs further prediction on the same frame of the base layer image. The Further, the reference motion vector is supplied to the motion prediction unit 254 and the enhancement layer acquisition unit 214 so that the motion prediction unit 254 performs further motion prediction on the same frame of the image in the enhancement layer.

While the acquisition unit 216 predicts the motion of the input video stream, the base layer acquisition unit 212 and the enhancement layer acquisition unit 214 predictively encode the input video stream, and the predictive encoding is based on the motion based on the base layer and the enhancement layer. There is a slight delay in time due to the need for further motion prediction based on the vector.
The process by which the base layer performs further motion prediction based on previous reference motion vectors is described below.

  The original input video signal is divided by the separator and supplied to each of the base layer acquisition unit 212 and the enhancement layer acquisition unit. In the base layer acquisition unit, the input video stream is supplied to the downsampler 120. A downsampler is a low pass filter that is used to reduce the resolution of an input video stream. The downsampled video stream is supplied to the motion vector acquisition unit 222. The motion vector acquisition means 222 acquires an image of the previous reference frame of the video sequence stored in the frame memory, and based on the down-sampled reference motion vector of the current frame output from the previous downsampler 286. Search for the macroblock that best matches the current frame within the small search window of the previous reference frame, thereby obtaining the video motion vector of the corresponding frame of the image of the downsampled video stream.

  After receiving the above-described prediction mode, reference motion vector and motion vector from the motion vector acquisition means 222, the motion compensator 124 is a frame memory (not shown) that is encoded and partially decoded based on the prediction mode. ) Stored in the previous reference frame image data, reference motion vector and motion vector, and shift the previous frame of the image according to the reference motion vector and then shift again according to the motion vector, thereby Predict frames. Of course, the previous frame of the image can be shifted once by an amount that is the sum of the reference motion vector and the motion vector, and in this case, the reference motion vector and the sum of the motion vectors is the sum of the frame of the image. Used as a motion vector.

  The motion compensator 124 then provides the predicted image to the arithmetic unit 125 and the switch 144. Arithmetic unit 125 also receives the input video stream and calculates the difference between the image of the input video stream and the predicted image coming from motion compensator 124. The difference is supplied to the DCT circuit 130. When the prediction mode received from the motion prediction unit 122 is intra frame prediction, the motion predictor 124 does not output a predicted image. In such a case, the arithmetic unit 125 does not perform the processing described above, but directly inputs the video stream to the DCT circuit 130.

  The DCT circuit 130 performs DCT processing on the signal output from the arithmetic unit to obtain the DCT coefficient supplied to the quantizer 132. The quantizer 132 sets a size for quantization (quantization level) based on the amount of data stored in the buffer, and uses the quantization level to change the DCT coefficient supplied from the DCT circuit 130. Quantize. The quantized DCT coefficient and the set quantization magnitude are supplied to the VLC unit 134.

  According to the magnitude of quantization supplied from the quantizer 132, the VLC unit 134 converts the quantization coefficient from the quantizer into a variable length code such as a Huffman code, thereby generating a base layer.

  Further, the converted quantization coefficient is output to a buffer (not shown). The quantization coefficient and the quantization magnitude are supplied to an inverse quantizer 138 that inversely quantizes the quantization coefficient according to the quantization magnitude so as to convert the quantization coefficient into a DCT coefficient. The DCT coefficients are supplied to an inverse DCT unit 140 that performs an inverse DCT transform to DCT coefficients. The obtained inverse DCT coefficient is supplied to the arithmetic unit 148. Arithmetic unit 148 receives the inverse DCT coefficient from inverse DCT unit 140 and receives data from motion compensator 124 according to the position of switch 144. Arithmetic unit 148 calculates the sum of the signal supplied by inverse DCT unit 140 and the predicted image supplied by motion compensator 124 and partially decodes the original image. However, if the prediction mode consists of intra-frame coding, the output of the inverse DCT unit 140 is sent directly to the frame memory. The decoded image acquired by the arithmetic unit 148 is supplied to the frame memory for future use as a reference frame for intra-frame encoding, forward encoding, backward encoding, or bidirectional encoding. And memorized.

  The output of arithmetic unit 140 is also provided to upsampler 150 to generate a reconstructed stream having substantially the same resolution as that of the high resolution input video stream. However, due to the filter and the damage caused by compression and decompression, the reconstructed stream has some error. Such differences are determined by subtracting the reconstructed high-resolution video stream from the original unchanged high-resolution video stream and input to the enhancement layer to be encoded. Therefore, the enhancement layer encodes and compresses frames having such difference information.

  The process of enhancement layer predictive coding is very similar to that of the base layer. After the acquisition unit 216 acquires the reference motion vector, the reference motion vector is supplied to the motion prediction unit 254 of the enhancement layer acquisition unit 214. As described above, the motion prediction unit 254 performs further motion prediction on the image of the same frame in the enhancement layer based on the reference motion vector, thereby obtaining the motion vector of the corresponding frame of the image of the video stream. Then, according to the prediction mode, the reference motion vector and the motion vector, the motion compensator 155 shifts the reference frame accordingly and predicts the current frame. This motion prediction process is similar to that of the base layer and will not be described in further detail here.

  FIG. 3 is a flowchart of encoding by using a reference motion vector according to an embodiment of the present invention. This flow is an operation flow of the means 200.

First, a specific high-resolution video stream, such as a video stream having a resolution of 1920 * 1080i, is received (step S305).
Next, a reference motion vector is acquired for each frame of the image of the video stream (step S310). Assuming that the current frame is a P frame, the macroblock that best matches the current frame is searched in the reference frame I search window, for example, the search is the value recommended by motion estimation ± This is done in a search window having a size of 15 pixels. After the optimal matching block is found, the shift between the current block and the matching block is the reference motion vector. Since this reference motion vector is obtained by predicting the reference frame in the original video stream without error, it accurately reflects the actual video motion.

Acquisition process of the reference motion vector is expressed by the following equation, where (B _x, B _y) represents a motion vector.

式（１）では、ａｒｇはＳＡＤが最小であるときに現在のマクロブロックに対応する動きベクトルである。

In equation (1), arg is the motion vector corresponding to the current macroblock when SAD is minimum.

式（２）では、ＳＡＤは、２つのマクロブロックの類似性を示し、それぞれの画素間の違いの絶対値であり、ｍ及びｎは、水平及び垂直方向のそれぞれにおけるマッチングブロックの移動する成分であり、Ｐ_c（i,j）及びＲ_p（i,j）は、現在のフレームの画素と前の基準フレームの画素である。添え字“ｃ”及び“ｐ”は、“現在のフレーム”及び“前のフレーム”をそれぞれ表す。

In equation (2), SAD indicates the similarity between two macroblocks and is the absolute value of the difference between the respective pixels, and m and n are the moving components of the matching block in the horizontal and vertical directions, respectively. Yes, P _c (i, j) and R _p (i, j) are the pixel of the current frame and the pixel of the previous reference frame. The subscripts “c” and “p” represent “current frame” and “previous frame”, respectively.

  The reference motion vector is used to re-predict motion in the base layer and enhancement layer of the video stream, respectively, and the base layer and enhancement layer only need motion prediction in a small range based on this reference motion vector This reduces the computational complexity and increases the compressed coding efficiency of the coding system.

Next, in order to obtain (B _x , B _y ′), the reference motion vector (B _x , B _y ) is down-sampled (step S312).
For example, in order to reduce the resolution to 720 * 480i, the video stream is downsampled (step S316).

  According to the downsampled reference motion vector (BX ', By'), the motion vector of the corresponding frame of the image of the downsampled video stream is obtained (step 332). The corresponding frame of the image described here is the same frame as the current frame when the reference motion vector is acquired. This is based on the reference motion vector (Bx ′, By ′) by further searching for macroblocks that are predicted in the same frame and that are optimistically matched to the current block in a small search window of the reference frame. This is because the motion vector (Dx1; Dy1) is acquired. This is evidenced by experience that the search window is a new search window of ± 2 pixels. The search process is more clearly understood with reference to equations (3) and (4).

動き予測が基準の動きベクトル（Ｂx’，Ｂy’）に基づいてサーチすることが式（４）により示される。基準の動きベクトルを計算するときにサーチの大部分は終了しているため、このステップにおいて最適な整合しているブロックを発見するために非常に制限されたサーチのみが必要とされる。±２画素のサーチウィンドウにおけるサーチ量は、±１５画素のサーチウィンドウのそれよりも少ない。

Equation (4) shows that the motion prediction searches based on the reference motion vector (Bx ′, By ′). Since most of the search is done when calculating the reference motion vector, only a very limited search is needed to find the best matching block in this step. The search amount in the search window of ± 2 pixels is smaller than that of the search window of ± 15 pixels.

  Downsampled video stream to generate a base layer. Processing is performed using the motion vector (step S326). The predicted frame of the current frame is obtained simply by shifting the reference frame according to the reference motion vectors and motion vectors described above, and known processing is sufficient to generate the base layer.

  In accordance with the reference motion vector (Bx, By), the motion vector of the corresponding frame of the video stream image is acquired (step S332). When the reference motion vector is acquired, the corresponding frame of the image here is the same frame as the current frame. This is based on the reference motion vector (Bx, By) by further searching for macroblocks that are predicted in the same frame and that are optimistically matched to the current block in a relatively small search window of the reference frame. This is because the motion vector (Dx2; Dy2) is acquired. The method for obtaining the motion vector is similar to the method for obtaining the motion vector by the base layer, and thus detailed description thereof is omitted.

Next, the video stream is processed using the motion vector and the base layer, thereby generating an enhancement layer (step S336).
Thus, in this embodiment, the reference motion vector is used by the base layer and enhancement layer simultaneously to predict motion, thus reducing the computational complexity for searching in both layers and compressing Encoding efficiency is increased.

The computational complexity of the compression scheme for the present invention and the prior art FIG. 1 is analyzed and compared as follows.
The resolution of the high definition (HD) frame and standard definition (SD) is assumed to be 1920 × 1088i and 720 × 480i, respectively, and the search window consists of ± 15 pixels. The computational complexity of the error rate SAD between two microblocks for the Y component is T _SAD .

The total number of macroblocks of HD frames and SD frames (considering only the Y component) is 8160 and 1350, respectively, and when motion prediction is performed for each macroblock in the search window of ± 15 pixels, The largest amount of computation for obtaining a suitable motion vector for a block is (31 * 31 * T _SAD = 961 * T _SAD ). The calculation amount of the HD frame is (8160 * 961 * T _SAD = 7,841,760 * T _SAD ), and the calculation amount of the SD frame (base layer) is (1350 * 961 * T _SAD = 1,297, 350 * T _SAD ).

For the coding system shown in FIG. 1, the largest overall motion vector calculation for each frame is the sum of the HD frame and SD frame calculations, ie (9, 139, 110). * T _SAD ).

For the coding system shown in FIG. 2, the amount of calculation of the reference motion vector is (7,841,760 * T _SAD ). When motion prediction for each macroblock is performed with a relatively small search window (± 2 pixels), the most computational effort to obtain a suitable motion vector is (5 * 5 * T _SAD = 25 * T _SAD ). The calculation amount of the SD frame (base layer) is (1350 * 25 * T _SAD = 33,750 * T _SAD ), and the calculation amount of the HD frame (enhancement layer) is (8160 * 25 * T _SAD = 204, 00 * T _SAD ).

For the coding system shown in FIG. 2, the overall largest amount of motion vector computation for each frame is the sum of the computations of the reference motion vector, and the amount of SD frames in a relatively small search window. And search for the amount of HD frames in a relatively small search window, ie, (7,875,510 * T _SAD ).

In comparison with the encoding system shown in FIG. 1, the encoding system shown in FIG. 2 reduces the computational complexity by the following percentages.
R = | 7,875,510-9,139,110 | / 9,139,110 = 14%
FIG. 4 is a conceptual diagram of an encoding system using a reference motion vector according to another embodiment of the present invention. The encoding system 400 of this embodiment is similar to the system shown in FIG. 2, and the description here concentrates only on the differences between them, and the same parts are omitted. The difference between them is that the acquisition means 410 has a downsampler 120 and a reference motion vector acquisition means 416. The original video stream is first downsampled by the downsampler 120. The down-sampled video stream is then supplied to the reference motion vector acquisition means 416, i.e. to the motion prediction means 476 and the frame memory 282, respectively, so that the reference motion vector of each frame of the image of the video stream. To get. The reference motion vector is then supplied directly to the motion prediction means 422 of the base layer acquisition means 412, and based on the reference motion vector, the means 422 can detect the motion of the corresponding frame of the image of the downsampled video stream. The motion is predicted again with a relatively small search window to obtain the vector, and then the base layer generation means 413 processes the down-sampled video stream by using the motion vector, thereby generating the base layer. To do.

  Further, in the enhancement layer acquisition unit 414, the reference motion vector described above is first up-sampled by the up-sampler 486, and then the motion vector acquisition unit, that is, the motion vector prediction unit 454 performs the correspondence of the image of the video stream. In order to obtain the motion vector of the frame to be reproduced, the motion is predicted again based on the upsampled reference motion vector. The video stream is then processed by the enhancement layer generation means 415 with the reference motion vector and the base layer, thereby generating an enhancement layer.

  From the above description, it can be seen that motion predictions in the base layer and the enhancement layer are related to each other, and the iterative search that needs to be performed by them when predicting images of the same frame is completed at one time, And the enhancement layer predicts again within a relatively small search window based on the same reference motion vector. Since the search process is greatly saved, the overall coding system complexity is reduced.

  FIG. 5 is a conceptual diagram of an encoding system using a reference motion vector according to a further embodiment of the present invention. The encoding system 500 of this embodiment is similar to the system shown in FIG. 2, and the description here concentrates only on the differences between them and omits similar parts. The difference is that the motion prediction means 522 of the base layer acquisition means 512 outputs the motion vector of each frame of the base layer image, and the motion vector corresponds to the image by the reference motion vector acquisition means, that is, the upsampler 586. Up-sampled for use as a frame reference motion vector, the reference motion vector is supplied to the motion prediction means 554 of the enhancement layer acquisition means 514. Based on the reference motion vector, the motion prediction is processed again with a relatively small search window, thereby obtaining the motion vector of the corresponding frame of the video stream image. Then, according to the reference motion vector, the enhancement vector generation means 515 generates the enhancement layer in a manner similar to the embodiment shown in FIG. 2 in the same manner as the output of the base layer.

  From what has been described above, in this embodiment, the enhancement layer obtains the motion obtained at the base layer so as to omit the search portion that is identical to that of the base layer and thus reduce the overall amount of computation of the coding system. Based on the vector, the enhancement layer processes the search again with a relatively small range.

  Although the present invention has been described with particular embodiments, many alternatives, modifications and variations will be apparent to those skilled in the art in light of the above description. Accordingly, it is intended to embrace all such alternatives, modifications and variations that fall within the spirit and scope of the appended claims.

1 is a block diagram of a conventional spatial hierarchical compression video encoder. FIG. It is a conceptual diagram of the encoding system using the reference | standard motion vector which concerns on embodiment of this invention. 6 is a flowchart of encoding by using a reference motion vector according to an embodiment of the present invention. It is a conceptual diagram of the encoding system using the reference | standard motion vector which concerns on another embodiment of this invention. FIG. 6 is a conceptual diagram of an encoding system using a reference motion vector according to a further embodiment of the present invention.

Claims (16)

A spatial hierarchical compression method of a video stream,
a) processing the video stream to obtain a reference motion vector for each frame of an image of the video stream;
b) processing the video stream using the reference motion vector to generate a base layer;
c) processing the video stream using the reference motion vector and the base layer to generate an enhancement layer;
Including methods. Said step a)
Down-sampling the video stream;
Obtaining the reference motion vector for each frame of an image of a downsampled video stream;
The method of claim 1 comprising: Said step b)
Obtaining the motion vector of a corresponding frame of an image of the downsampled video stream according to the reference motion vector;
Processing the downsampled video stream using the motion vector to generate the enhancement layer;
The method of claim 2 comprising: Said step c)
Up-sampling the reference motion vector;
Obtaining the motion vector of a corresponding frame of an image of the video stream according to an upsampled motion vector;
Processing the video stream using the motion vector and the base layer to generate an enhancement layer;
The method of claim 2 comprising: Said step b)
Down-sampling the reference motion vector;
Down-sampling the video stream;
Obtaining the motion vector of a corresponding frame of an image of the downsampled video stream in response to the downsampled reference motion vector;
Processing the downsampled video stream using the motion vector to generate the base layer;
The method of claim 1 comprising: Said step c)
Obtaining a motion vector of a corresponding frame of an image of the video stream according to the reference motion vector;
Processing the video stream using the motion vector and the base layer to generate the enhancement layer;
The method of claim 5 comprising: A spatial hierarchical compression method of a video stream,
a) processing the video stream to generate a base layer;
b) up-sampling the motion vector of each frame of the base layer image to obtain a reference motion vector of the corresponding frame of the image;
c) processing the video stream using the reference motion vector and a base layer to generate an enhancement layer;
Including methods. Said step c)
Obtaining the motion vector of a corresponding frame of an image of the video stream according to the reference motion vector;
Processing a video stream using the motion vector and the base layer to generate the enhancement layer;
The method of claim 7 comprising: A spatial hierarchical compression device for a video stream,
Obtaining means used to process the video stream and obtain a reference motion vector for each frame of an image of the video stream;
Base layer acquisition means for processing the video stream using the reference motion vector and generating a base layer;
Enhancement layer acquisition means for processing the video stream using the reference motion vector and the base layer to generate an enhancement layer;
Having a device. The acquisition means includes
A downsampler used to downsample the video stream;
Reference motion vector acquisition means used to acquire the reference motion vector of each frame of the image of the downsampled video stream;
The apparatus of claim 9 comprising: The base layer acquisition means includes
Motion vector acquisition means used to acquire the motion vector of a corresponding frame of the image of the downsampled video stream based on the reference motion vector;
Base layer generating means for processing the down-sampled video stream using the motion vector and generating the base layer;
The apparatus of claim 10 comprising: The enhancement layer acquisition means includes:
An upsampler used to upsample the reference motion vector;
Motion vector acquisition means for acquiring the motion vector of a corresponding frame of an image of the video stream according to an upsampled motion vector;
Enhancement layer generating means for processing the video stream using the motion vector and the base layer to generate the enhancement layer;
The apparatus of claim 10 comprising: The base layer acquisition means includes
A downsampler used to downsample the reference motion vector and the video stream;
Motion vector acquisition means used to acquire the motion vector of a corresponding frame of the image of the downsampled video stream based on the downsampled reference motion vector;
Base layer generating means for processing the down-sampled video stream using the motion vector and generating the base layer;
The apparatus of claim 9 comprising: The enhancement layer acquisition means includes:
Motion vector acquisition means for acquiring the motion vector of the corresponding frame of the image of the video stream according to the reference motion vector;
Enhancement layer generating means for processing the video stream using the motion vector and the base layer to generate the enhancement layer;
The apparatus of claim 9 comprising: A spatial hierarchical compression device for a video stream,
Base layer acquisition means used to process the video stream to generate a base layer;
Reference motion vector acquisition means used to upsample the motion vector of each frame of the base layer image and acquire a reference motion vector of the corresponding frame of the image;
Enhancement layer acquisition means used to process the video stream using the reference motion vector and a base layer to generate an enhancement layer;
Including the device. The enhancement layer acquisition means includes:
Motion vector acquisition means for acquiring the motion vector of the corresponding frame of the image of the video stream according to the reference motion vector;
Enhancement layer generating means for processing the video stream by using the motion vector and the base layer and generating the enhancement layer;
The apparatus of claim 15 comprising:

Images