KR100828378B1 - Method and apparatus for transferring video frame in telecommunication system - Google Patents

Abstract

The present invention provides a method of encoding video frames in a communication system, comprising: forming a video frame of consecutive still frames, frame reconstruction data of at least one frame as a reference frame, and motion data of previously encoded neighboring blocks; Storing motion data, determining motion data of a block to be encoded by motion data of one or more previously encoded neighboring blocks, wherein the neighboring block is formed by the stored reference frame, and has a predetermined picture quality. And determining frame data and motion data indicating a block to be encoded, which provides a predetermined encoding efficiency.

Description

Method and apparatus for transmitting video frame in communication system {Method and apparatus for transferring video frame in telecommunication system}

The present invention relates to a method and apparatus for encoding and decoding a video frame in a communication system when the video frame consists of a series of continuous still frames, wherein the transmitted frames are typically frame blocks or segments, e.g. The data in each frame block is divided into pixel groups specific for the frame and typically contains information indicative of the position, color and luminance of the frame block.

It is difficult to transmit video frames in communication systems, such as videophones, video conferencing systems or Internet connections. This is because of the large amount of data required to transmit the video frame, since the more bits required to transmit the data, the higher the data rate should be. Various methods have been developed to solve this problem. In order to transmit a frame, a frame is usually divided into frame blocks whose size is selected for the system. Frame block information generally includes information on the position, color, and luminance of the frame block within the frame itself. Frame block data is compressed for each block using a preferred encoding method. Compression is based on removing less important data. Compression methods are mainly divided into three classes: reduced spectral redundancy, reduced spatial redundancy, and reduced temporal redundancy. Typically various combinations of these methods are used for compression.

For example, the YUV color model applies to spectral redundancy reduction. The YUV color model takes advantage of the fact that the human eye is more sensitive to chrominance, ie changes in luminance than in color. The YUV model has one luminance component (Y) and two chrominance components (U, V). The chrominance components are also sub-sampled. For example, according to the H.263 video coding standard, the luminance block is 16 x 16 pixels, and both chrominance blocks covering the same area as the luminance block are 8 x 8 pixels. In this standard, a combination of one luminance block and two chrominance blocks is referred to as a macro block.

In order to reduce spatial redundancy, for example, discrete cosine transformation (DCT) is used. The block to be compressed in the DCT is preferably 8 x 8 pixels. In DCT, the pixel representation of a frame block is converted to a space-frequency representation. Moreover, only signal frequencies within the frame block have high amplitude coefficients, while coefficients of signals not within the frame block are close to zero. DCT is also a lossless transformation and causes interference to the signal only if it is quantized.

Temporal redundancy can be reduced by using the fact that consecutive frames are usually similar to each other. Thus, instead of compressing each individual frame, motion data of the frame blocks is generated. The basic principle is as follows. The best reference code previously encoded is searched for the frame block to be encoded. The motion between the reference block and the frame block to be encoded is modeled and the calculated motion vector coefficients are transmitted to the receiver. The difference between the block to be encoded and the reference block is represented as a prediction error component or frame. The problem is to find a reference block that produces as good coding efficiency as possible (good picture quality with as few bits as possible), and high computational capacity and thus also computation time is needed for comparison.

It is therefore an object of the present invention to provide an improved method of encoding video frames, an improved video encoder and an improved subscriber terminal. The present invention is based on using one or more previously encoded neighboring blocks and frame reconstruction data of at least one or more reference frames or reference blocks to encode frame reconstruction data. Depending on how the desired coding efficiency with sufficient picture quality is obtained, motion data and frame reconstruction data encoded with the method of the present invention or with other known methods are selected for use.

This is a method of encoding video frames in a communication system, comprising: forming a video frame of consecutive still frames, storing frame reconstruction data of at least one frame and motion data of previously encoded neighboring blocks as reference frames; Determining motion data of a block to be encoded by motion data of one or more previously encoded neighboring blocks, wherein the neighboring block is formed by the stored reference frame and determines frame reconstruction data of a frame to be encoded. And selecting to use the frame reconstruction data and motion data representing a block to be encoded, the predetermined image quality being provided with a predetermined picture quality. Is achieved.

This is a method of encoding video frames in a communication system, comprising: forming a video frame of consecutive still frames, storing frame reconstruction data of at least one frame and motion data of previously encoded neighboring blocks as reference frames, Determining motion data of the frame block to be encoded using only neighboring blocks having the same reference frame used to encode the frame reconstruction data of the block to be encoded, and providing a predetermined encoding efficiency with a predetermined picture quality, A video frame encoding method comprising the step of selecting to use the frame reconstruction data and motion data.

The invention also provides means for forming a video frame of consecutive still frames, means for storing frame reconstruction data of at least one frame and motion data of previously encoded neighboring blocks as reference frames, Means for determining motion data of a block to be encoded by motion data, the neighboring block being formed by the stored reference frame, means for determining frame reconstruction data of a frame to be encoded, and predetermined encoding with a predetermined picture quality It relates to a subscriber station comprising motion data representing a block to be encoded and means for selecting to use the frame reconstruction data, which provides efficiency.

The present invention also provides means for forming a video frame of continuous still frames, means for storing frame reconstruction data of at least one frame and motion data of previously encoded neighboring blocks as reference frames, encoding frame reconstruction data of a block to be encoded. Means for determining motion data of a frame block to be encoded using only neighboring blocks having the same reference frame used to make the data, and using the frame reconstruction data and the motion data, which provide a predetermined encoding efficiency with a predetermined picture quality. It relates to a subscriber station comprising means for selecting to.

The invention also provides means for forming a video frame of consecutive still frames, means for storing frame reconstruction data of at least one frame and motion data of previously encoded neighboring blocks as reference frames, Means for determining motion data of a block to be encoded by motion data, the neighboring block being formed by the stored reference frame, means for determining frame reconstruction data of a frame to be encoded, and predetermined encoding with a predetermined picture quality It relates to a video encoder comprising motion data representing a block to be encoded and means for selecting to use the frame reconstruction data, which provides efficiency.

The present invention also provides means for forming a video frame of continuous still frames, means for storing frame reconstruction data of at least one frame and motion data of previously encoded neighboring blocks as reference frames, encoding frame reconstruction data of a block to be encoded. Means for determining motion data of a frame block to be encoded using only neighboring blocks having the same reference frame used to make the data, and using the frame reconstruction data and the motion data, which provide a predetermined encoding efficiency with a predetermined picture quality. Relate to a video encoder comprising means for selecting to do so.

The invention also relates to a computer program that can be read by a computer and that performs a method of encoding video frames in a communication system, the method comprising the steps of forming a video frame of consecutive still frames, the reference frame being of at least one frame; Storing frame reconstruction data and motion data of previously encoded neighboring blocks, and determining motion data of a block to be encoded by motion data of one or more previously encoded neighboring blocks, wherein the neighboring block is the stored reference frame; Using the frame reconstruction data and the motion data representing the block to be encoded, which are formed by the following steps, determining frame reconstruction data of a frame to be encoded, and providing a predetermined encoding efficiency with a predetermined picture quality. It relates to a computer program comprising the step of selecting.

The invention also relates to a computer program that can be read by a computer and that performs a method of encoding video frames in a communication system, the method comprising the steps of forming a video frame of consecutive still frames, the reference frame of at least one frame; Storing frame reconstruction data and motion data of previously encoded neighboring blocks, using only neighboring blocks having the same reference frame used to encode frame reconstruction data of the block to be encoded, to obtain motion data of the frame block to be encoded. Determining, and selecting for use of the frame reconstruction data and motion data having a predetermined picture quality and providing a predetermined encoding efficiency.

The method and apparatus of the present invention provide savings in computation time and capacity while sufficient image quality.

Hereinafter, the present invention will be described by the preferred embodiment with reference to the accompanying drawings.

1 illustrates an example of a communication system.

2A to 2C show one estimation process of vectors using a reference frame.

3 shows a process of estimating a motion vector using neighboring blocks.

4 is a flowchart illustrating method steps for finding a reference block and predicting a motion vector.

5 shows an example of finding a reference block and predicting a motion vector.

6 shows an example of an encoder.

7 shows an example of a decoder.                 

8 shows an example of a subscriber station.

9 shows a second example of finding a reference block and predicting a motion vector.

Figure 1 shows in a simplified manner one digital data transmission system to which the solution of the present invention can be applied. It is part of a cellular wireless system that includes a base station 104 that wirelessly connects 108 and 110 with subscriber stations 100 and 102, which may be fixedly installed or vehicle-mounted terminals or portable terminals. The transceivers in the base station are connected to an antenna unit, with which the radio link to the subscriber stations is implemented. The base station further connects with a base station controller 106 which transmits connections of terminals elsewhere in the network. The base station controller controls the several base stations connected to it in a centralized manner. The control unit in the base station controller performs call control, data traffic control, mobility management, statistics collection, and signaling.

The cellular wireless system may also have a connection to a public telephone network. In that case, a transcoder belonging to the system converts different encoding formats of speech used between the public telephone network and the cellular wireless network to suit each other. For example, from a fixed network's 64 kbit / s format to a cellular wireless network format (eg 13 kbit / s) and vice versa.

It is apparent to those skilled in the art that the present invention can also be applied to fixed communication systems, such as the Internet.

The purpose of encoding a video frame is to reduce the number of bits needed to transmit the frame. 2A-2C illustrate, in a simplified manner, a prior art motion coding method based on predicting a motion vector of a frame block (the term block is also used in the text) using a reference frame previously stored in memory. This type of encoding is referred to as inter-coding, which means using similarities between frames of a series of frames in compression encoding.

In the example of FIGS. 2A-2C, reference frame 206 is divided into blocks or segments and previously encoded. 2B and 2C do not represent frame blocks. The reference frame may be before or after the frame to be encoded of the series of frames. 2A also shows a current frame 204 to be encoded that is divided into blocks. Only one block 200 of the blocks is shown for clarity. The size of the blocks is N × M. The blocks are typically square. That is, N = M. But it can also be rectangular for example in geometry. The current block 200 and the reference block 214 shown in FIG. 2C need not be geometrically similar. This is because, in fact, filmed objects change, such as rotation or twisting. The size and shape of the block is determined appropriately for each situation by considering the following. For example, a small block size provides a better picture. However, it reduces the reliability of the defined motion vector. Because the definition is done only by a few pixels. More motion vector data should also be generated if smaller blocks are used. This is because the frame must be divided into a larger number of blocks. Moreover, faster algorithms for determining motion vectors when the block size is large are more efficient. For example, in some of the current video coding standards, N = M = 16.

In order to estimate the motion, the matching reference block is found as much as possible in the reference frame 206 for the block 200 to be encoded. This is 214 in FIG. 2C. The suitability of the reference blocks depends on the selection criteria. The purpose of the selection criterion is to find one block that is typically capable of minimizing the number of bits needed to transmit the block to be encoded while the picture quality is good enough. Thus, the selection criteria can be changed depending on the purpose of the application.

The search is usually adjacent to and immediately adjacent to the region at the frame location corresponding to the block 200 to be encoded in the current frame 204, shown by the search region 210 in FIGS. 2B and 2C within the reference frame 206. In some areas. By limiting the search area smaller than the entire reference frame, the time used for searching and required calculations is reduced. The size of the search area is determined appropriately for the purpose of the application, for example, by determining whether the motion is slow or fast based on the frames of the previously encoded frame string. If the motion is slow, the search area can be limited to smaller than if the motion is fast. The search is typically performed around the predicted motion vector.

In the example of Figs. 2A to 2C, the position of the frame block is represented by coordinates (x, y) at the upper left corner of the block. The x coordinate represents the position of the pixel or block in the horizontal direction and the y coordinate represents the position of the pixel or block in the vertical direction. The coordinates of the block to be encoded are indicated as 202. In FIG. 2C, the best possible reference frame block 214 has been found in the search region 210. Coordinates (x, y) 208 are the coordinates of the location where the block 200 to be encoded would have moved if it had not moved and is therefore identical to coordinates 202. They are shown for illustrative purposes. Block 212 shows the case where block 200 has not moved and is also shown for purposes of illustration. The motion of the frame block to be encoded may now be indicated as a change from coordinates (x + u, y + v) 216 to coordinates (x, y) 208. In general, the coordinates (x, y) are defined as origins, so the motion vector 218 of the block to be encoded can be represented as the coordinates (u, v). The motion vector can also be represented using the definition of motion vector coefficients and motion models.

3 shows in a simplified manner prior art motion coding methods based on predicting motion vector coefficients of a frame block to be encoded using previously encoded neighboring blocks of the frame to be encoded. The neighboring blocks of the frame block 304 to be encoded, which have already been previously encoded and used to predict the motion vector coefficients of the frame block to be encoded, are referred to herein as prediction candidates. In FIG. 3, the prediction candidates are neighbor blocks 300, 302, 306. Prediction candidates 300, 302, 306 are marked for identification with certain pixel positions 308, 310, 312 in the example of FIG. 3. The prediction candidate 306 is to the left of the block to be coded and is marked with the pixel position 308. The prediction candidate 300 is above the block to be encoded and marked with the pixel location 310. The prediction candidate 302 is in the upper right corner of the block to be encoded and is marked with the pixel position 312. As can be seen in FIG. 3, the pixel locations 308, 310, 312 are in their blocks close to the block 304 to be encoded. It should be noted that the location and number of neighboring blocks with respect to the block to be encoded may be different from those shown in FIG. 3.

In the case shown in FIG. 3, all three prediction candidates are used to predict the motion vector coefficients of block 304 to be encoded. For example, the required motion vector coefficients can be predicted as the median of the motion vector coefficients obtained from these prediction candidates. It is also possible to use only one prediction candidate to predict the motion vector coefficients of block 304. In that case the coefficients are obtained from the motion vector coefficients of the prediction block 306, the motion vector coefficients of the prediction block 300, or the motion vector coefficients of the prediction block 302.

Previously encoded neighboring blocks further away than immediately adjacent may also be used. However, it should be considered that in that case the difference in the movement of the blocks usually increases as the distance increases.

Inter-coding and intra-coding are typically combined block by block in encoding video frames to obtain the best possible result. Intra-coded frames may be sent at appropriate intervals. This is to prevent the accumulation of possible errors caused by the transmission channel per frame, which in turn causes failure of the decoding of the frame.

The methods shown in FIGS. 2A-2C and 3 may be used for both encoding and decoding. The method used by the encoder with the necessary frame data is transmitted to the decoder as a bit stream.

As shown in FIG. 3, it should be noted that blocks may also be segmented. Segmentation is desirable in situations where different parts of the frame move in different directions. For example, the background remains the same and a single object, such as a person, a car, or something else of that kind moves. The frame block may be segmented by dividing it into segments vertically, such as in block 300, but not horizontally, or in both directions, such as in block 306. By segmentation, it is also possible to reduce the number of bits to be transmitted by using a suitable portion of another already encoded frame.

When determining the motion vector coefficients of the frame block to be coded, various known models for motion prediction are used. Different models are suitable for defining different movements, such as representing vertical and horizontal movements or representing rotations. Such models include an afine motion model. The motion of a pixel, frame segment or block in the pseudo motion model is represented by the equation in which Δx and Δy form a motion vector.

Where a ₀ , a ₁ , a ₂ , b ₀ , b _1, and b ₂ are motion vector coefficients, and x and y are vector components. x represents the position of the pixel or block in the horizontal direction and y represents the position of the pixel or block in the vertical direction. Alternatively, the motion of a pixel, frame segment or block in a translational model is represented by the equation in which Δx and Δy form a motion vector.

Where a ₀ and b ₀ are motion vector coefficients, and x and y are vector components. The motion models used are stored in the memory of both the encoder and the decoder. When several motion models are used for encoding, the encoder selects the model according to the motion (for example, for an upward motion, a model representing the upward motion is used for encoding). Information about the model used is sent to the decoder along with the motion vector coefficients.

4 is a flowchart illustrating method steps for video frame encoding. In the following description, a frame block refers to a macro block, block or segment according to the YUV model formed by, for example, pixels. Performance of the method begins at block 400.

In block 402, the video frame is formed of consecutive still frames. To form a video frame, the still frames are preferably transmitted at a rate at which the human visual system can no longer distinguish distinct frames from each other.

In block 404, coded reconstruction data of the frame blocks (e.g., the luminance and color of the prediction error frame and pixels obtained from the calculation of motion vector coefficients) are stored in memory. Frame reconstruction data is encoded according to applicable standards, for example H.263 or MPEG. The definition of motion vector coefficients and prediction error frame has been described above. One or more reference frames are formed of frame reconstruction data stored in memory. In addition, motion data of previously encoded neighboring blocks, ie motion vector coefficients and possibly a used motion model, are stored in memory. It is most desirable to use neighboring blocks that are close to the block to be encoded. However, sometimes it is also possible to use previously coded blocks further away in the frame.

In block 406, motion data of a block to be encoded is determined by previously encoded neighboring blocks, preferably using motion vector coefficients of one or more neighboring blocks.

In block 408, reconstruction data of a block to be encoded is determined using, for example, a reference frame.

The method can be enhanced by calculating motion data of a frame block to be encoded using only neighboring blocks having the same reference frame as used to encode frame reconstruction data of the block to be encoded. Similarities often exist between frames in the same frame string. It is thus possible for a sufficiently similar block to be found in a previously encoded frame in which a suitable reference block has already been found for a previously encoded neighboring block, or a block corresponding to the block to be encoded. For example, motion vector coefficients of one or more neighboring blocks are used and the search region of the reference block of the block to be encoded is limited to an environment defined by the motion vector coefficients of the neighboring block in the reference frame.

When only one inter-coded neighboring block has the same reference frame as the block to be encoded, the motion vector of this neighboring block is selected as one motion vector candidate, and the other candidate is for example several neighboring blocks. It may be a median calculated from motion vectors.

It should be noted that the block used to encode the block is encoded in the same manner (inter) as the manner in which the block to be encoded is to be encoded. That is, when the neighboring block is intra-coded, it does not have a reference frame. If there are no prediction candidates with reference frames, prior art intra-coding may be used or all reference frames stored in memory may be checked.

A reference block is a frame block used for compression encoding by a video encoder, that is, an encoder to which frame reconstruction data of a frame block is to be encoded. The frame block that matches as much as possible with the block to be encoded and is retrieved from one or more reference frames previously encoded and stored in memory is selected as the reference block, whereby the prediction error frame is made as small as possible.

In block 410, an encoding method is selected in which motion data and frame reconstruction data of a frame block to be encoded are most preferably encoded, that is, a predetermined encoding efficiency is reached with a predetermined picture quality. Note that, as an encoding method of a block to be encoded, any prior art method such as encoding or inter-coding described above by the motion data of neighboring blocks is selected, and the block to be encoded is directly encoded by a reference block found from a reference frame. Should be. In the case of selecting an encoding method, the purpose is to make the number of bits as small as possible, for example, while the quality of the picture corresponds to the quality of a normal video picture, or to achieve picture quality depending on the application purpose even though the number of bits is smaller. It is enough to match the number of bits to the number of standards used.

The coding efficiency can be estimated by, for example, a Lagrangian function that defines the most efficient segmentation and coding. The function is as shown in Equation 3.

Where D (S _k ) is an error that occurs in despreading the frame, R (S _k ) is the number of bits needed to transmit the frame, and λ is a Lagrangian constant.

The Lagrangian function represents the relationship between the quality of the frame to be transmitted and the bandwidth required for the transmission path. In general, the higher the picture quality, the greater the bandwidth. The choice of the most preferred compression encoding is always a compromise between the number of bits needed and the picture quality.

Arrow 412 represents the repeatability of the method block by block until the frame is encoded. It should be noted that preferably the method is repeated until all the frames of the series of frames have been encoded.

Execution of the method ends at block 414.

5 shows an example of finding a reference block having the same reference frame as that used to encode frame reconstruction data of a block to be encoded and predicting motion vectors when only neighboring blocks are used. It should be noted that prior art methods are usually also used when coding a block. In the end, an encoding method that provides the desired coding efficiency with sufficient picture quality is selected for use.

The frame page 500 to be encoded is divided into blocks, and only a few of the blocks are shown for clarity. Block 506 is a block to be encoded. Frames 502, 504, 532, and 508 are reference frames stored in memory. The reference frame of blocks 510, 514 of previously coded neighboring blocks, marked as number 1, is frame 502. The block 506 to be encoded and the reference blocks 510 and 514 have not moved in the frame 500 compared to the frame 502. The reference frame of neighboring block 512, marked as number 2 in the figure, is frame 504. The reference frame of neighboring block 516, marked as number 4 in the figure, is frame 508. Blocks within frame 500 have been moved relative to frames 504 and 508. Frame 532 was not used to encode any of the neighboring blocks of block 506 to be encoded. That is, it is not a reference frame of any neighboring block. Thus, frame 532 is not used to find the reference block for block 506 to be encoded in this example.                 

Next, neighboring blocks 510 and 514 are used as prediction candidates of the motion vectors of block 506 to be encoded. A check is performed to see if the encoding type (inter) of the prediction candidates is the same as the encoding type to be used to encode the current block. If so, block 526 is used as a reference block and motion vector coefficients are obtained by comparing the motion vector coefficients obtained by comparing the positions of blocks 510 and 518 and the positions of blocks 514 and 520. Obtained by calculating the mean or median of motion vector coefficients. Motion vector coefficients for neighboring block 510 are obtained by comparing the positions of blocks 510 and 518 and motion vector coefficients for neighboring block 514 are obtained by comparing the positions of blocks 514 and 520. do. In addition, the prediction error frame is determined by comparing block 506 to be encoded with reference block 526. In the example of FIG. 5, the block position in frames 500 and 502 has not changed. Therefore, motion vector coefficients are not determined.

Next, the encoding type of the neighboring block 512 is checked. If the encoding type of the neighboring block 512 is the same as the encoding type to be used for the current block 506, the reference block is retrieved in the frame 504 and the motion vector coefficients obtained from the comparison of the blocks 512 and 522 are block. Used as the motion vector coefficients of 506. The prediction error frame is then determined by reference block 528.

Next, the coding type of the neighboring block 516 is checked. If the encoding type of the neighboring block 512 is the same as the encoding type to be used for the current block 506, the reference block is retrieved in the frame 508 and the motion vector coefficients obtained from the comparison of the blocks 516 and 524 are obtained. Used as the motion vector coefficients of block 506. The prediction error frame is then determined by reference block 530.

The motion vector coefficients for block 506 can also be determined by calculating the mean or median of the motion vector coefficients of blocks 510, 512, and 514.

In addition, when several motion models are used, the used motion model is displayed.

Block 534 is intra-coded and therefore has no reference frame and is not used to encode block 506 if a preferred encoding method is otherwise found.

The location and number of neighboring blocks and the number of stored reference frames may differ from those shown in FIG. 5.

9 shows a second example of predicting motion vectors and finding a reference block. In this example, if only one neighboring block is inter-coded and this neighboring block has the same reference frame as the block to be encoded, the block to be encoded is encoded by only one neighboring block.

The frame 900 to be encoded is divided into blocks, and only a few of the blocks are shown for clarity. Block 912 is a block to be encoded. Frames 902, 904, and 906 are reference frames stored in memory. The reference frame of the previously encoded neighboring block 914 (marked as 1) is frame 902. The reference frame of block 910 (marked 2) is frame 904. Block 908 is intra-coded and thus has no reference frame.

The example of FIG. 9 tests whether suitable reference blocks can be found among all the reference frames stored in memory, ie frames 902, 904, 906 in this example, for block 912 to be encoded. When testing whether frame 902 has a suitable reference block, the motion vector of block 914 is used as a motion vector prediction candidate. Because block 914 is only one of three neighboring blocks, the reference frame is frame 902. When testing whether frame 904 has a suitable reference block, the motion vector of block 910 is used as a motion vector prediction candidate. Because block 910 is only one of three neighboring blocks, the reference frame is frame 904. Frame 906 is not a reference frame to any of the neighboring blocks. Thus in the case described in the example, the motion vector is predicted using the median of the blocks 910, 914 and 908, for example the motion vector of the inter-coded block 908 is set to zero in that case. do.

It should be noted that when predicting motion vectors of a block to be encoded, it is possible to use not only motion data and reference frame data of neighboring blocks, but also segmentation information of, for example, a macro block to which the block to be encoded belongs. The location and number of neighboring blocks and the number of stored reference blocks may differ from those shown in FIG. 9.

6 shows an example of an encoder in which the above-described method of encoding video frames may be performed. The design principle of the video encoders is the prediction error frame E _n (x, y) 600 between the block I _n (x, y) 620 to be coded and the prediction candidate (selected reference block) P _n (x, y) 612. Will be minimized. The prediction error frame is defined as the difference between the block 620 to be encoded and the prediction candidate 612 and is obtained at the subtractor 632.

Prediction block 612 is obtained by the method according to the preferred embodiment of the present invention by using the selected reference frame and motion vectors. The coefficients of the motion vectors Δx (x, y), Δy (x, y) are formed at the motion vector calculation block 630 as the sum of the predicted motion vectors and the found difference coefficients. Predictive motion vector coefficients are obtained using the motion vectors of a neighboring block having the same reference frame as the block to be encoded, according to a preferred embodiment of the present invention. Other methods are also used to predict motion vectors. One of the other methods is the prior art method, in which the block to be encoded is directly encoded by the reference frame. The group of motion vectors of all the pixels in the frame is called the motion vector field. Since there are a plurality of pixels in one frame, one frame is actually divided into blocks when video encoding and motion information is transmitted block by block to a receiver which is a decoder shown in FIG.

At least one motion vector selected from the predicted motion vectors or motion vector fields formed therefrom is encoded in motion vector coding block 628. Motion models known in the art are used for encoding, for example the examples have been described above. The coded motion vectors or formed motion vector fields are sent to the multiplexer 624.                 

In block 618, the frame is formed of frame blocks by segment or by block. The reference block or prediction candidate is shown in equation (5).

(622)은 메모리(610)에 저장되고 x 및 y는 벡터 성분들이다. x는 수평 방향으로 픽셀, 프레임 세그먼트 또는 블록의 위치를 나타내고, y는 수직 방향으로 픽셀, 프레임 세그먼트 또는 블록의 위치를 나타낸다.

및

는 픽셀, 프레임 세그먼트 또는 블록의 움직임 벡터를 형성한다.Where one or more reference frames

622 is stored in memory 610 and x and y are vector components. x indicates the position of the pixel, frame segment or block in the horizontal direction, and y indicates the position of the pixel, frame segment or block in the vertical direction.

And

Form a motion vector of a pixel, frame segment or block.

At block 602, prediction error frame E _n (x, y) 600 is encoded and typically represented by a finite second-order transform, for example by a discrete cosine transform (DCT). The coefficients obtained by the DCT transform are quantized and encoded and then transmitted to the multiplexer 624 and to a receiver, which is the decoder shown in FIG.

(608)은 가산기(634)에서 예측 블록 P_n(x,y)(612)에 추가되고 복호화된 블록

이 결과로서 획득된다. 그 결과로부터 새로운 참조 블록이 획득되거나, 동일한 페이지에 속하는 블록들을 결합함으로써, 참조 프레임(614)이 획득되고 메모리(610)에 저장된다. 따라서, 송신기 비디오 부호기 및 수신기 비디오 부호기는 사용을 위한 동일한 참조 프레임 또는 블록을 갖는다.The encoder also includes a decoder block 606 in which the prediction error frame E _n (x, y) is decoded. This prediction error frame

608 adds and decodes the predicted block P _n (x, y) 612 at the adder 634

This is obtained as a result. A new reference block is obtained from the result, or by combining blocks belonging to the same page, the reference frame 614 is obtained and stored in the memory 610. Thus, the transmitter video encoder and the receiver video encoder have the same reference frame or block for use.

In the multiplexer 624, the information including motion vector information is combined with the prediction error frame and this frame information 616 is sent to the decoder (FIG. 7).

The video encoder functions described above may be implemented in various ways, for example by software running on a processor or by hardware such as logic or application specific semiconductors (ASIC) composed of separate components.

7 shows an example of a decoder in which the above-described method for reducing time redundancy in video frame transmission by the decoder can be performed. The encoded prediction error frame 702 and the encoded motion vector coefficients or motion vector field coefficients 712 of the received video signal 700 are separated from each other in the demultiplexer 710.

(722)이 있다. 동일한 프레임 블록들 및/또는 참조 프레임이 부호기 및 복호기 메모리에 저장된다.The memory 716 of the decoder includes a reference frame divided into at least one previously encoded neighboring block and / or blocks.

There is 722. The same frame blocks and / or reference frames are stored in the encoder and decoder memory.

(706)은 블록(704)에서 형성된다.At block 714, prediction candidate P _n (x, y) 718 is formed in the same manner as at encoder block 618 (FIG. 6). Decoded Prediction Error Frame

706 is formed at block 704.

(706) 및 예측 후보 P_n(x,y)(718)가 함께 더해진다. 이것은 수학식 6의 복호화된 블록

(708)을 생성한다.In adder 720, the decoded prediction error frame

706 and prediction candidate P _n (x, y) 718 are added together. This is the decoded block of equation (6)

Generate 708.

(722)은 메모리(716)에 저장되고, x 및 y는 벡터 성분들이다. x는 수평 방향으로 픽셀 또는 블록의 위치를 나타내고, y는 수직 방향으로 픽셀 또는 블록의 위치를 나타낸다.

및

는 픽셀, 프레임 세그먼트 또는 블록의 움직임 벡터를 형성한다.Here, a reference frame divided into one or more previously encoded neighboring blocks and / or blocks

722 is stored in memory 716, where x and y are vector components. x indicates the position of the pixel or block in the horizontal direction, and y indicates the position of the pixel or block in the vertical direction.

And

Form a motion vector of a pixel, frame segment or block.

The video encoder functions described above may be implemented in various ways, for example by software running on a processor or by hardware such as logic or application specific semiconductors (ASIC) composed of separate components.

8 shows one terminal of a wireless system in which the method of the preferred embodiment of the present invention may be performed. The transmitter of the subscriber station 824 and the transmitter of the wireless system network component partially perform the same task. Terminal 824 may be, for example, a microcomputer comprising a mobile phone or radio units, but the invention is not limited thereto. The terminal described above includes an antenna 800 used to transmit and receive signals through a duplex filter. The terminal may also include several antennas or a multi-antenna system. The terminal also includes a transmitter 802 for amplifying the modulated signal and transmitting it to the antenna. The terminal also includes a modulator 804 that modulates a carrier having a data signal containing the desired information according to the selected modulation scheme. The terminal also includes a receiver 806 that amplifies the signal coming from the antenna and down-converts it to the desired intermediate frequency or directly to baseband. The terminal also includes a demodulator 808 that demodulates the received signal so that the data signal can be separated from the carrier.

The subscriber terminal also controls the operation of the other parts of the terminal, such as camera 820 and video codec 822, and can generate data generated by the user, such as digital signal processing (DSP), digital-to-analog conversion and filtering, or And a control block 816 that takes the necessary actions to process the user's voice. In addition, in spread-spectrum systems, such as WCDMA, the signal spectrum is spread over a wide band by a pseudo random spread code at the transmitter and despread at the receiver. Thus attempts to increase the capacity of the channel. The control block also performs encoding and decoding, such as channel and speech encoding. In addition, the control block adapts the signal and signaling information to be transmitted to match the air interface standard of the radio system used. The subscriber station includes different types of one or more memories (not shown) which may be separate or separate parts of other blocks. In the example of FIG. 8, the control block also includes a program for controlling the operation of the terminal. In the example of FIG. 8, although a video codec 822 is separated that encodes and decodes the reconstruction data of a frame, it may also be part of the control block 816.

When transmitting the frames, the camera 820 writes the digital image to the memory block 818. The camera 820 may also record an image directly to the codec 822. The codec encodes a frame selected by one of the methods or by a combination of several methods. The camera 820, codec 822 and memory 818 may reside within subscriber terminal 824 or may belong to a separate camera unit 826. When belonging to a separate camera unit 826, the camera unit 826 has an interface 828 and is connected to the subscriber terminal 824.

The user of the subscriber station can see the frame on the display 814 as needed. The coded frame reconstruction data is transmitted to the communication system in the same manner as voice or text data and for example to another telephone.

The operational blocks of the terminal described above, such as the codec block 822 and the control block, which are required for frame encoding and decoding, are in various ways, for example, by logic or custom semiconductors composed of discrete components or by software running on a processor. It may be implemented by hardware such as (ASIC). The terminal functions shown in the figures may also be combined in the operation blocks in many different ways from the figures. For example, codec block 822 may be separated into an encoder and a decoder.

The user interface of the terminal includes a speaker or earphone 810, a microphone 812, a display 814 and possibly a keyboard, all of which are connected to the control block 816.                 

Although the present invention has been described above with reference to examples according to the accompanying drawings, it is clear that the present invention is not limited thereto but may be modified in many ways within the scope of the invention disclosed in the appended claims.

The present invention can be applied with some video coding standards such as H.263 or MPEG.

Claims (66)

A method of encoding a video sequence, Performing prediction of a block to be coded using one of a plurality of reference frames, wherein the prediction comprises: performing prediction including motion data and a reference frame number; Comparing the reference frame number of the block to be coded with the reference frame number of a previously coded block; And If at least one of the previously coded blocks has a reference frame number different from the reference frame number of the block to be coded, the movement of the previously coded block with the same reference frame number as the block to be coded Predicting the motion data of the block to be coded using only data. The method of claim 1, If only one of the previously coded blocks has the same reference frame number as the block to be coded, the motion data prediction of the block to be coded is coded with the previously coded block with the same reference frame number as the coded block. And set to be equal to the motion data of the extracted block. The method of claim 1, The motion data prediction of the block to be coded is a median of the motion data of the previously coded block having the same reference frame number as the block to be coded. The method of claim 1, The motion data prediction of the block to be coded is an average of the motion data of the previously coded block having the same reference frame number as the coded block. The method according to claim 1 or 3, And the motion data of the block is represented by a motion vector. The method according to claim 1 or 3, And said previously coded block is a block directly neighboring said block to be coded. The method of claim 6, And said direct neighboring block is located on the left, top and right side of said coded block. 4. A method according to claim 1 or 3, wherein the previously coded block is not directly in proximity to the frame block to be coded, but further away from the block to be coded. The method according to claim 1 or 3, Retrieving a predictive block in the plurality of available reference frames; and Selecting the prediction block for the block to be coded to obtain a predetermined coding efficiency or a predetermined picture quality. The method of claim 9, The search for the predictive block in the plurality of available reference frames is based on a support set of motion data and a reference frame based on reference data and motion data of a previously coded block. How to sequence code. 10. The method of claim 9, wherein the coding efficiency is calculated using a Lagrangian function. A subscriber station comprising a video codec 822 for performing a method of encoding a video sequence, the method comprising: Performing prediction of a block to be coded using one of a plurality of reference frames, wherein the prediction comprises: performing prediction including motion data and a reference frame number; Comparing the reference frame number of the block to be coded with the reference frame number of a previously coded block; And If at least one of the previously coded blocks has a reference frame number different from the reference frame number of the block to be coded, the movement of the previously coded block with the same reference frame number as the block to be coded Predicting the motion data of the block to be coded using only data. The method of claim 12, If only one of the previously coded blocks has the same reference frame number as the block to be coded, the motion data prediction of the block to be coded is coded with the previously coded block with the same reference frame number as the coded block. And is set to be equal to the motion data of the block. The method according to claim 12 or 13, And the motion data of the block is represented by a motion vector. The method according to claim 12 or 13, Retrieving a predictive block in the plurality of available reference frames; and Selecting the prediction block for the block to be coded to obtain a preset coding efficiency or a preset picture quality. 16. The apparatus of claim 15, wherein the search for the predictive block in a plurality of available reference frames is based on a reference set based on a support set of motion data and reference numbers and motion data of previously coded blocks. Subscriber terminal, characterized in that. The method according to claim 12 or 13, And a coding efficiency is calculated using a Lagrangian function. The method according to claim 12 or 13, And the subscriber station (824) is a radio system terminal. 14. A subscriber station (824) according to claim 12 or 13, characterized in that the one selected from the group consists of a hair terminal and a microcomputer comprising a radio part. 14. A subscriber station (824) according to claim 12 or 13, comprising a camera. Means for performing prediction of a block to be coded using one of a plurality of reference frames, the prediction comprising means for performing prediction including motion data and a reference frame number; Means for comparing the reference frame number of the block to be coded with the reference frame number of a previously coded block; And If at least one of the previously coded blocks has a reference frame number different from the reference frame number of the block to be coded, the movement of the previously coded block with the same reference frame number as the block to be coded Means for predicting the motion data of the block to be coded using only data. The method of claim 21, If only one of the previously coded blocks has the same reference frame number as the block to be coded, the motion data prediction of the block to be coded is coded with the previously coded block with the same reference frame number as the coded block. Video encoder, characterized in that it is set equal to the motion data of the extracted block. The method of claim 21, The motion data prediction of the block to be coded is a median of the motion data of the previously coded block having the same reference frame number as the block to be coded. The method of claim 21, The motion data prediction of the block to be coded is an average of the motion data of the previously coded block with the same reference frame number as the coded block. The method of claim 21, And the motion data of the block is represented by a motion vector. The method of claim 21 or 23, The previously coded block is a block directly neighboring the block to be coded. 27. The video encoder of claim 26, wherein the directly neighboring blocks are located on the left, top, and right sides of the coded block. 24. The video encoder of claim 21 or 23, wherein the previously coded block is not directly in proximity to the frame block to be coded, but further away from the block to be coded. 24. The apparatus of claim 21 or 23, wherein the encoder Means for retrieving a predictive block in the plurality of available reference frames; and Means for selecting the prediction block for the block to be coded to obtain a predetermined coding efficiency or a predetermined picture quality. The method of claim 29, The search for the predictive block in the plurality of available reference frames is based on a support set of motion data and a reference frame based on reference data and motion data of a previously coded block. Encoder. 30. The method of claim 29, wherein the coding efficiency is A video encoder, characterized in that it is calculated using a Lagrangian function. A computer readable recording medium storing computer readable code for executing a method of encoding a video frame. Performing prediction of a block to be coded using one of a plurality of reference frames, wherein the prediction comprises: performing prediction including motion data and a reference frame number; Comparing the reference frame number of the block to be coded with the reference frame number of a previously coded block; And If at least one of the previously coded blocks has a reference frame number different from the reference frame number of the block to be coded, the movement of the previously coded block with the same reference frame number as the block to be coded Predicting the motion data of the block to be coded using only data; and storing computer readable code for executing a method of encoding a video frame. The method of claim 32, If only one of the previously coded blocks has the same reference frame number as the block to be coded, the motion data prediction of the block to be coded is coded with the previously coded block with the same reference frame number as the coded block. Computer readable code storing computer readable code for executing a method of encoding a video frame, characterized in that it is set equal to the motion data of the compiled block. The method of claim 32, And wherein said motion data prediction of said block to be coded is a median of said motion data of said previously coded block having the same reference frame number as said block to be coded. Computer-readable recording medium for storing type codes. The method of claim 32, Wherein the motion data prediction of the block to be coded is an average of the motion data of the previously coded block having the same reference frame number as the coded block. Computer-readable recording medium for storing the code. The method of claim 32 or 34, And the computer readable code for executing a method of encoding a video frame, wherein the motion data of the block is represented by a motion vector. The method of claim 32 or 34, And the previously coded block is a block directly neighboring the block to be coded. The method of claim 32 or 34, And said direct neighboring block is located on the left, top, and top of the coded block, the computer readable recording medium storing computer readable code for executing a method of encoding a video frame. The method of claim 32 or 34, And the previously coded block is not directly in proximity to the frame block to be coded, but further away from the block to be coded. 35. The method of claim 32 or 34, wherein the method Retrieving a predictive block in the plurality of available reference frames; and Selecting the prediction block for the block to be coded to obtain a predetermined coding efficiency or a predetermined picture quality; storing computer readable code for executing a method of encoding a video frame Computer-readable recording medium. The method of claim 32 or 34, The search for the predictive block in the plurality of available reference frames is based on a support set of motion data and a reference frame based on reference data and motion data of a previously coded block. A computer readable recording medium storing computer readable code for executing a method of encoding a frame. The method of claim 32 or 34, And a coding efficiency is calculated using a Lagrangian function. A computer readable recording medium storing computer readable code for executing a method of encoding a video frame. A method of decoding encoded video data, the method comprising Receiving motion prediction information for a block to be decoded from the encoded video data, wherein the motion prediction information indicates at least the reference frame number and motion difference data; Comparing the reference frame number of the block to be decoded with the reference frame number of a previously decoded block; And Motion data of the previously decoded block having the same reference frame number as the block to be decoded, if at least one of the previously decoded blocks has a reference frame number different from the reference frame number of the block to be decoded. Performing prediction of the motion data of the block to be decoded using only; And And forming motion data of the block to be decoded based on the motion data prediction and the received motion difference data. The method of claim 43, If only one of the previously coded blocks has the same reference frame number as the block to be decoded, the motion data prediction of the block to be decoded is the previously coded code having the same reference frame number as the block to be coded. And set to be equal to the motion data of the encoded block. The method of claim 43, The motion data prediction of the block to be coded is a median of the motion data of the previously coded block having the same reference frame number as the block to be coded. The method of claim 43, And wherein the motion data prediction of the block to be coded is an average value of the motion data of the previously coded block having the same reference frame number as the block to be coded. The method of claim 43 or 45, And said motion data of said block is represented by a motion vector. The method of claim 43 or 45, Said previously coded block is a block directly neighboring said block to be coded. 49. The method of claim 48 wherein And said direct neighboring blocks are located on the left, top and top of the coded block. 46. The method of claim 43 or 45, wherein the previously coded block is Wherein the previously coded block is not directly in proximity to the block of frames to be coded, but further away from the block to be coded. A video decoder for decoding coded video data, the decoder comprising: Means for receiving motion prediction information for a block to be decoded in the encoded video data, the motion prediction information including at least the reference frame number and motion difference data; Means for comparing the reference frame number of a block to be decoded with the reference frame number of a previously decoded block; Motion data of the previously decoded block having the same reference frame number as the block to be decoded, if at least one of the previously decoded blocks has a reference frame number different from the reference frame number of the block to be decoded. Means for performing prediction of the motion data of the block to be decoded using only; And Means for forming motion data of the block to be decoded based on the motion data prediction and the received motion difference data. The method of claim 51 wherein If only one of the previously coded blocks has the same reference frame number as the block to be decoded, the motion data prediction of the block to be decoded is the previously coded code having the same reference frame number as the block to be coded. And decode coded video data, the coded video data being set equal to the motion data of the encoded block. The method of claim 51 wherein The motion data prediction of the block to be coded is a median of the motion data of the previously coded block having the same reference frame number as the block to be coded. The method of claim 51 wherein The motion data prediction of the block to be coded is an average value of the motion data of the previously coded block having the same reference frame number as the block to be coded. The method of claim 51 or 53, And the motion data of the block is represented by a motion vector. The method of claim 51 or 53, And the previously coded block is a block directly neighboring the block to be coded. The method of claim 56, wherein And said direct neighboring block is located on the left, top and top of the coded block. The method of claim 51 or 53, And the previously coded block is not immediately in close proximity to the frame block to be coded, but further away from the block to be coded. A subscriber station comprising a video codec 822 for performing a method of decoding encoded video data, the method comprising: Receiving motion prediction information for a block to be decoded from the encoded video data, wherein the motion prediction information indicates at least the reference frame number and motion difference data; Comparing the reference frame number of the block to be decoded with the reference frame number of a previously decoded block; And Motion data of the previously decoded block having the same reference frame number as the block to be decoded, if at least one of the previously decoded blocks has a reference frame number different from the reference frame number of the block to be decoded. Performing prediction of the motion data of the block to be decoded using only; And forming motion data of a block to be decoded based on the motion data prediction and the received motion difference data. The video codec 822 performs a method of decoding encoded video data. Subscriber terminal. A subscriber station according to claim 12, If only one of the previously coded blocks has the same reference frame number as the block to be decoded, the motion data prediction of the block to be decoded is the previously coded code having the same reference frame number as the block to be coded. And a video codec (822) for performing a method of decoding coded video data, characterized in that it is set equal to the motion data of a block. 61. The method of claim 59 or 60, And a video codec (822) for performing a method of decoding coded video data, wherein said motion data of said block is represented by a motion vector. 61. The method of claim 59 or 60, And a video codec (822) for performing a method of decoding coded video data, wherein said previously coded block is a block directly neighboring the block to be coded. 63. The method of claim 62, And the direct neighboring block comprises a video codec (822) for performing a method of decoding coded video data, characterized in that it is located on the left, top and top of the coded block. 61. The method of claim 59 or 60, The subscriber terminal comprising a video codec 822 for performing a method of decoding coded video data, characterized in that the previously coded block is not directly in proximity to the frame block to be coded, but further away from the block to be coded. . 61. The method of claim 59 or 60, And a video codec (822) for performing a method of decoding encoded video data, characterized in that the subscriber terminal is a wireless system terminal. 61. The method of claim 59 or 60, And a video codec (822) for performing a method of decoding coded video data, characterized in that it is comprised of a microcomputer comprising a mobile terminal and a wireless portion.

Images