JP2007312397A - Method and apparatus for video frame transfer in communication system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To save calculation time and capability, while maintaining proper image quality. <P>SOLUTION: A method of encoding a video frame in a communication system, includes steps of creating video frames each of which is composed of continuous still frames; storing frame restructure data of at least one frame as a reference frame and movement data of encoded adjacent blocks; determining the movement data of encoded blocks using movement vectors of one or more encoded adjacent blocks from adjacent blocks created by using the stored reference frames; and determining the frame reconstruction data and movement data that indicate encoded blocks. The method has a prescribed image quality and provides prescribed encoding efficiency. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a method and apparatus for coding and decoding video frames in a communication system when the video frames are composed of a series of consecutive still frames. In that frame, the transfer frame is typically divided into a plurality of frame blocks or segments (eg, pixel groups) specific to each frame, and the data in each frame block is typically Contains information representing the luminance, color, and position of the frame block.

  In a communication system (for example, a videophone, a video conferencing system, or an Internet connection), transfer of a video frame is a burdensome task because a large amount of data must be transferred. This is because the more data required, the higher the data transfer rate that is required. At the time of frame transfer, the frame is usually divided into frame blocks, and the size is determined according to the system. Frame block information typically includes information about the luminance, color, and position of the frame block in the frame. The frame block data is compressed for each block using a desired encoding method. The principle of compression is to remove unnecessary data. The compression methods can be roughly classified into three. Spectral redundancy reduction, spatial redundancy reduction, and temporal redundancy reduction. When compressing, these methods are usually used in combination.

  For example, spectral redundancy reduction is applied to the YUV color model. The YUV color model takes advantage of the fact that the human eye is more sensitive to changes in luminance than changes in chrominance (ie, color). The YUV model has one luminance component (Y) and two chrominance components (U, V). The chrominance component is subsampled. For example, a luminance block according to the H.263 video coding standard is 16 × 16 pixels, and two chrominance blocks cover the same area as the luminance block, but 8 × 8 pixels. One luminance block and two chrominance blocks are called macroblocks in this standard.

  In order to reduce spatial redundancy, for example, a discrete cosine transform (DCT) is used, and the compressed block is 8 × 8 pixels. In DCT, the pixel representation of the frame block is converted to a spatial frequency representation. Furthermore, only the signal frequency present in the frame block has a large amplitude coefficient. The coefficient of the signal that does not exist in the frame block is close to zero. DCT is a lossless conversion, and the interference is only when the signal is quantized.

  To reduce temporal redundancy, take advantage of the fact that consecutive frames are usually similar to each other. That is, instead of compressing each individual frame, motion data of the frame block is generated. The principle is as follows. That is, for the frame block to be encoded, search for the previously encoded reference block as much as possible, model the motion difference between the reference block and the block to be encoded, calculate the motion vector coefficient, Send. The difference between the block to be encoded and the reference block is expressed as a prediction error component. The problem is therefore to find a reference block that yields the best possible coding efficiency (with as few bits as possible and sufficiently high image quality), the high computational power required for comparison and thus the short computation time.

  An object of the present invention is to provide an improved method of an encoded video frame, an improvement of a video coder (encoder), and an improvement of a subscriber terminal device. The present invention uses a technique for using frame reconstruction data of one or more reference frames or blocks, and of one or more already coded neighboring blocks when encoding frame reconstruction data. Based on. The selection of frame reconstruction data and motion data to be encoded by the method of the present invention or other known methods depends on how the desired coding efficiency is obtained while maintaining sufficient image quality. Can be decided.

  In a communication system, a video frame encoding method for achieving this includes the steps of creating a video frame of continuous still frames, frame reconstruction data of one or more frames as reference frames, and adjacent blocks that have already been encoded. The step of storing the motion data of the block, and the step of determining the motion data of the block to be encoded using the motion data of the already encoded adjacent block (the adjacent block is generated by the stored reference frame) , Determining the frame reconstruction data of the frame to be encoded, selecting frame reconstruction data and motion data representing a block to be encoded (maintaining predetermined image quality and providing predetermined coding efficiency) Including the step of:

  In a communication system, a video frame coding method for accomplishing this includes the steps of creating a video frame of continuous still frames, frame reconstruction data of one or more frames as reference frames, and motion of already encoded adjacent blocks. Storing data, determining motion data of the block to be encoded using only neighboring blocks having the same reference frame used to encode the frame reconstruction data of the block to be encoded Selecting frame reconstruction data and motion data (maintaining a predetermined image quality and providing a predetermined coding efficiency).

  The present invention comprises means for creating a video frame consisting of continuous still frames, means for storing frame reconstruction data of at least one frame and motion data of encoded adjacent blocks as reference frames, and said storage Means for determining motion data of a block to be encoded using motion vectors of one or more encoded adjacent blocks from adjacent blocks created using the generated reference frame; Means for determining frame reconstruction data of the frame to be encoded, and the frame reconstruction data and motion data representing a block to be encoded so as to have a predetermined image quality and provide a predetermined encoding efficiency. And a means for selecting for use.

  The invention also includes means for creating a video frame consisting of continuous still frames, means for storing frame reconstruction data of at least one frame and motion data of encoded adjacent blocks as reference frames; Means for determining motion data of a frame block being encoded using only adjacent blocks having the same reference frame as used to encode the frame reconstruction data of the block being encoded; And a means for selecting the frame reconstructed data and motion data for use so as to provide a predetermined coding efficiency.

  The invention also includes means for creating a video frame consisting of continuous still frames, means for storing frame reconstruction data of at least one frame and motion data of encoded adjacent blocks as reference frames; Means for determining motion data of a block to be encoded using motion vectors of one or more encoded adjacent blocks from adjacent blocks created using the stored reference frame; Means for determining frame reconstruction data of the frame to be processed, and motion representing the frame reconstruction data and the block to be encoded so as to have a predetermined image quality and provide a predetermined encoding efficiency Means for selecting data for use, and a video coder.

  The invention also includes means for creating a video frame consisting of continuous still frames, means for storing frame reconstruction data of at least one frame and motion data of encoded adjacent blocks as reference frames; Means for determining motion data of a frame block to be encoded using only adjacent blocks having the same reference frame used to encode the frame reconstruction data of the block to be encoded; A video coder comprising said frame reconstruction data and means for selecting motion data for use so as to have a predetermined image quality and to provide a predetermined encoding efficiency.

  The present invention also includes the steps of creating a video frame consisting of continuous still frames, storing frame reconstruction data of at least one frame and motion data of encoded adjacent blocks as reference frames, Determining motion data of a block to be encoded using motion vectors of one or more encoded adjacent blocks from adjacent blocks created using the reference frame; and frame reconstruction data of the encoded frame Selecting the frame reconstruction data and motion data representing the block to be encoded for use so as to have a predetermined image quality and provide a predetermined encoding efficiency; A video frame symbol in a communication system that can be read by a computer. A computer program for executing the method.

  The present invention also includes the steps of creating a video frame composed of continuous still frames, storing frame reconstruction data of at least one frame and motion data of encoded adjacent blocks as reference frames, and trying to encode Determining the motion data of the frame block of the frame to be encoded using only adjacent blocks having the same reference frame as used when encoding the frame reconstruction data of the block to be encoded, and a predetermined image quality. And having the step of selecting the frame reconstruction data and motion data for use to provide a predetermined coding efficiency, the computer-readable video frame in the communication system The present invention relates to a computer program for executing an encoding method.

The method and apparatus of the present invention provides savings in computation time and capacity while maintaining image quality in a proper manner.
Hereinafter, the present invention will be described using preferred embodiments with reference to the drawings.

  FIG. 1 shows a simplified digital data transmission system to which the present invention can be applied. It is part of a cellular radio system and has a base station 104 connected to subscriber terminal devices 100, 102 via radios 108, 110. The terminal device may be fixed, a terminal device installed in a vehicle, or a mobile terminal device. The transceiver of the base station is connected to the antenna unit, and a wireless link is provided to the subscriber terminal device. The base station is further connected to a base station controller 106. The controller connects to any terminal device in the network. The base station controller centrally controls a plurality of base stations connected to the base station controller. The control unit of the base station controller is responsible for call control, data traffic control, mobility management, statistical aggregation and signaling.

  The cellular radio system can be connected to a public telephone network. In that case, the transcoder of the system converts various audio digital code formats used between the public telephone network and the cellular radio network so as to be compatible with each other. An example is the conversion of a fixed network from 64 kbit / s format to a cellular radio network format (eg 13 kbit / s) and vice versa.

  It will be apparent to those skilled in the art that the present invention can also be applied to fixed communication systems such as the Internet.

  The purpose of encoding a video frame is to reduce the number of bits required for frame transfer. 2a to 2c show a conventional motion encoding method. The principle of this method is to predict the motion vector of a frame block (and the term block is used in the text) using a reference frame already stored in memory. This type of encoding is called inter-coding, and utilizes the similarity between a series of frames in compression encoding.

  In the example of FIGS. 2a to 2c, the reference frame 206 is divided into blocks and segments and already encoded. 2b and 2c do not show frame blocks. The reference frame may be before or after a frame to be encoded in a series of frames. The current frame 204 shown in FIG. 2a is encoded and divided into blocks. Only one block 200 is shown in the figure. The block size is N × M. The block is typically square, ie N = M. However, it can also be rectangular, for example. The current block 200 and the reference block 214 in FIG. 2c may not be geometrically similar. This is because filmed objects actually rotate, twist and deform. The size and shape of the block are defined in consideration of the following matters according to each state. For example, a small block size gives a better picture but reduces the reliability of the defined motion vector. This is because the definition is made using a small number of pixels. When using smaller blocks, more motion vector data must be generated. This is because the frame must be divided into a large number of blocks. Furthermore, fast algorithms that define motion vectors are more efficient when the block size is large. For example, in the current video coding standard, N = M = 16.

  To estimate the motion, find the most suitable reference block in the reference frame 206 for the coded block 200 and call this 214 in FIG. The suitability of the reference block depends on the selection criteria. The purpose of the selection is typically to find a block that minimizes the number of bits required to transfer the block being encoded while maintaining good image quality. Thus, the selection criteria depends on the purpose of the application.

  The search range is typically a reference frame 206 at the frame position corresponding to the block 200 that is encoded in the current frame 204 and is within the immediate proximity, represented by the search region 210 in FIGS. 2b and 2c. This is a predetermined area. By limiting the search range to a narrow range within the reference frame, the search time and the required calculation amount are reduced. The size of the search range is determined by determining whether the motion is slow or slow based on the frames of the encoded frame string to suit the purpose of the application. If the motion is slow, the search range can be limited to a smaller range than if the motion is fast. Typically, the search is performed in the environment surrounding the predicted motion vector.

In the example of FIGS. 2a to 2c, the position of the frame block is represented by the coordinates (x, y) of the upper left corner of the block. The x coordinate represents the horizontal position of the pixel or block, and the y coordinate represents the vertical position of the pixel or block. Mark the coordinates of the encoded block as 202. In FIG. 2 c, assume that the best reference frame block 214 is found within the search range 210. Coordinates (x, y) 208 are the coordinates of the position that would be located when the encoding block 200 did not move (ie, the same as the coordinates 202). These are shown for illustrative purposes. Block 212 represents block 200 when not moving, and is shown for explanation. The motion of the encoded frame block can be displayed as a change from coordinates (x + u, y + v) 216 to coordinates (x, y) 208.
In general, the coordinates (x, y) are defined as the origin (origo). Accordingly, the motion vector 218 of the encoded block can be displayed in coordinates (u, v). In addition, the motion vector can be expressed using a motion vector coefficient and a motion model definition.

  FIG. 3 shows an overview of a prior art motion coding method, which is based on the motion vector coefficients of a frame block being coded using already coded neighboring blocks of the frame being coded. Is to make predictions. The neighboring blocks of the frame block 304 being encoded have already been encoded and are used to predict the motion vector coefficients of the encoded frame block, referred to herein as prediction candidates. In FIG. 3, the prediction candidates are adjacent blocks 300, 302, and 306. In the example of FIG. 3, the prediction candidates 300, 302, and 306 are marked at the pixel positions 308, 310, and 312 for identification. The prediction candidate 306 is located on the left side of the coding block and has a mark at the pixel position 308, and the prediction candidate 300 is located above the coding block and is marked at the pixel position 310. The prediction candidate 302 is located in the upper right corner of the encoded block, and a mark is attached to the pixel position 312. Referring to FIG. 3, it can be seen that the pixel positions 308, 310, and 312 are located close to the encoding block 304 in each block. It should be noted that the number and position of blocks adjacent to the block being encoded can differ from that shown in FIG.

  In the case shown in FIG. 3, three prediction candidates are used to predict the motion vector coefficients of the block 304 being encoded. For example, the required motion vector coefficients can be predicted as the median of the motion vector coefficients obtained from these prediction candidates. It is also possible to use only one prediction candidate to predict the motion vector coefficient of block 304, in which case the motion vector coefficient of prediction block 306, the motion vector coefficient of prediction block 300, or the prediction block 302 A coefficient can be obtained from the motion vector coefficient.

  Instead of immediate proximity, it is also possible to use remote already encoded neighboring blocks. However, in that case, the point to be considered is that the motion error of the block usually increases as the distance increases.

  When video frame coding is performed, the best results are obtained by combining inter-frame coding and inter-frame coding for each block. By transmitting intra-frame encoded frames at appropriate intervals, an error occurring in the transfer channel is accumulated in transferring the next frame from one frame to prevent a frame encoding failure from occurring.

  The methods shown in FIGS. 2a to 2c and 3 can be used for coding and decoding. The method used by the coder with the necessary frames is transferred as a bit stream to the decoder.

  As shown in FIG. 3, the block can also be segmented. Segmentation is desirable when each frame portion moves in various ways. As an example, the case where the background does not change while a single object, such as one person, one car, etc., moves can be mentioned. The frame block can be divided vertically and segmented, for example, as block 300. Although not shown, it can be split horizontally or in both directions as in block 306. Segmentation reduces the number of transfer bits and allows the use of a suitable part of an already encoded frame.

When defining the motion vector coefficients of the frame block being encoded, various known models can be used for motion estimation. Each model is suitable for defining various movements such as vertical and horizontal movements and rotations. In the affine motion model included in this model, pixel motion, frame segments or blocks can be represented using the following equations that form a motion vector using Δx and Δy.
Δx (x, y) = a ₀ + a ₁ x + a ₂ y
Δy (x, y) = b ₀ + b ₁ x + b ₂ y (1)

However, a0, a1, a2, b0, b1, b2 are motion vector coefficients, x and y are vector components, x represents a horizontal pixel or block position, and y is Represents a vertical pixel or block location. Alternatively, in a translational model, the motion of a pixel, frame segment or block is represented by the following equation, where Δx and Δy form a motion vector.
Δx (x, y) = a ₀
Δy (x, y) = b ₀ (2)

  However, a0 and b0 are motion vector coefficients, and x and y are vector components. The used model is stored in the memory of a coder (encoder) and a decoder (decoder). If multiple motion models are used for encoding, the coder selects a model according to the motion (eg, for upward motion, a model representing upward motion is used and encoded), Information in the model used is sent to the decoder along with the motion vector coefficients.

FIG. 4 is a flowchart for video frame encoding. In the following description, a frame block means, for example, a block, segment, or macroblock based on a YUV model formed by pixels.
The method starts at block 400.

  In block 402, the video frame is composed of successive still frames. In order to form a video frame, it is desirable for the human visual system to transmit still frames at a speed that does not identify the frames as separate frames.

  At block 404, the encoded reconstruction data of the frame block (eg, prediction error frame calculated from pixel color and luminance, motion vector coefficients) is stored in memory. The frame reconstruction data is encoded according to an applicable standard (for example, H.263 or MPEG). The definition of the motion vector coefficient and the prediction error frame is as described above. One or more reference frames are formed using the frame reconstruction data stored in the memory. Furthermore, the motion data (that is, the motion vector coefficient) of the adjacent block already encoded and the used motion model are stored in the memory. Most preferred is to use the nearest neighbor block of the block being encoded, but sometimes it is possible to use an already encoded block at a distant location in the frame.

  At block 406, motion data for the block being encoded is determined using the adjacent block that has already been encoded. Preferably, it is determined using motion vector coefficients of one or more adjacent blocks.

  In block 408, the reconstructed data of the block being encoded is encoded using, for example, a reference frame.

  In order to enhance the method, motion data of a frame block that is encoded using only neighboring blocks can be calculated. The adjacent block has the same reference frame used for encoding the frame reconstruction data of the encoded block. Since there are similar frames in the same frame string, there is a possibility that a block corresponding to the block being encoded is found out of the encoded frames or a block that is sufficiently similar. An appropriate reference block is found for the encoded adjacent block from the encoded frame. For example, motion vector coefficients of one or more adjacent blocks are used. The search range of the reference block of the block being encoded is limited to the environment defined by the motion vector coefficient of the adjacent block in the reference frame.

  Under the condition that the only inter-frame encoded adjacent block has the same reference frame as the block being encoded, the motion vector of this adjacent block is selected as one motion vector candidate. As another candidate, for example, a median calculated from a plurality of adjacent block motion vectors can be selected.

It should be noted that the block used for block coding is inter-coded in the same way that the block being coded is coded, i.e., adjacent blocks within the frame. When encoding, it does not have a reference frame.
When there is no prediction candidate having a reference frame, the conventional intraframe coding is used, or all reference frames stored in the memory are checked.

  A reference block is a frame block that is used when a video coder (ie, an encoder that encodes frame reconstruction data of a frame block) performs compression encoding. The frame block is searched from one or more reference frames that are already encoded and stored in the memory, and is selected as the reference block, as matching the block that is encoded as much as possible. At that time, the prediction error frame is selected to be as small as possible.

  In block 410, an encoding method is selected, and the frame reconstruction data and motion data of the block being encoded are optimally encoded, ie, a predetermined efficiency is obtained while maintaining a predetermined image quality. Is encoded. It should be noted that, as an encoding method of a block to be encoded, the encoding method using motion data of adjacent blocks, or a conventional method (for example, interframe encoding method, The block to be encoded is directly encoded using the reference block found in the reference frame). The purpose for selecting the encoding method is, for example, to reduce the number of bits while the image quality corresponds to that of a normal video picture, or to adapt the number of bits to the number of bits of the standard used (for example, (Even if fewer bits are needed to achieve image quality depending on the application object).

Coding efficiency can be estimated using, for example, the following Lagrangian function to determine the most efficient segmentation and coding.
L (Sk) = D (Sk) + λR (Sk) (3)

However, D (Sk) is an error generated when despreading a frame,
R (Sk) is the number of bits required to transfer the frame,
λ is a Lagrange constant.

  The Lagrangian function represents the relationship between the quality of the transfer frame and the bandwidth required for the transmission path. Generally speaking, the higher the picture quality of a picture, the greater the bandwidth. Optimal compression coding is always a compromise between the required number of bits and image quality.

Arrow 412 represents repeating the method for each block until the frame is encoded. It should be noted that in this method, it is desirable to repeat until all the frames of a series of frames are encoded.
Execution of the method ends at block 414.

  FIG. 5 shows an example in which a motion vector is predicted to find a reference block. In this example, only adjacent blocks are used, and this adjacent block has the same reference frame used to encode the frame reconstruction data of the block being encoded. It should be noted that the prior art method is usually used for block coding. Finally, an encoding method is selected, and a desired encoding efficiency with sufficient image quality can be obtained.

  The frame page 500 being encoded is divided into blocks. Only a few are shown for clarity. Assume that block 506 is encoded. Frames 502, 504, 532 and 508 are reference frames stored in the memory. Frame 502 is the reference frame of blocks 510 and 514 which are encoded adjacent blocks (marked with the number 1). The encoding block 506 and the reference blocks 510 and 514 are not moved in the frame 500 compared to the frame 502. Frame 504 is a reference frame for adjacent block 512 (marked with numeral 2 in the figure). Frame 508 is a reference frame for adjacent block 516 (marked with numeral 4 in the figure). The frame 532 is a frame that is not used when any adjacent block of the block 506 being encoded is encoded. That is, it does not become the reference frame of any adjacent block. Thus, frame 532 is not used in this example when finding a reference block for encoding block 506.

  Next, adjacent blocks 510 and 514 are used as motion vector prediction candidates for the block 506 being encoded. Check whether the encoding type (inter) of the prediction candidate is the same as that used for encoding the current block. If they are the same, block 526 is used as a reference block. Then, an average or median of motion vector coefficients is calculated to calculate a motion vector coefficient. The motion vector is obtained by comparing the positions of the blocks 510 and 518, and the motion vector coefficient is calculated for the adjacent block 510 from these. The motion vector is obtained by comparing the positions of the positions 514 and 520, and the motion vector coefficient is calculated for the adjacent block 514 from these positions. Further, the encoding block 506 and the reference block 526 are compared to determine a prediction error frame. In the example of FIG. 5, the block positions in the frames 500 and 502 are not changed. Therefore, no motion vector coefficient is determined.

  Next, the coding type of the adjacent block 512 is checked. If it is the same as that used in current block 506, a reference block is searched for frame 504. The motion vector coefficient calculated from the comparison between the blocks 512 and 522 is used as the motion vector coefficient of the block 506. Next, a prediction error frame is determined using reference block 528.

  Next, the encoding type of adjacent block 516 is checked. If it is the same as that used in current block 506, a reference block is searched for frame 508. The motion vector coefficient calculated from the comparison between the blocks 516 and 524 is used as the motion vector coefficient of the block 506. A prediction error frame is then determined by reference block 530.

  The motion vector coefficients for the block 506 can also be determined by calculating the median or average value of the motion vector coefficients of the blocks 510, 512, and 514.

  If a plurality of motion models are used, the motion model to be used is indicated.

  Since block 534 is intra-frame coding, it does not have a reference frame. Even if there are other preferred encoding methods, they are not used to encode block 506.

  The number and location of adjacent blocks, as well as the number of stored reference frames, may be different from that shown in FIG.

  FIG. 9 illustrates a second example in which a motion vector is predicted and a reference block is found. In this example, if only one neighboring block is interframe coded and the neighboring block has the same reference frame as the coding block, the coding block is only one neighboring block. Is encoded using.

  An encoded frame 900 is divided into blocks. Only a few of them are shown for clarity. Assume that block 912 is encoded. Frames 9102, 904, and 906 are reference frames stored in the memory. The reference frame of the encoded adjacent block 914 (marked as number 1) is frame 902, and the reference block of block 910 (marked 2) is frame 904. Block 908 is intraframe encoded and therefore has no reference frame.

What the example of FIG. 9 seeks to find is a matching reference block from all reference frames stored in memory (ie, frames 902, 904, 906 in this example) for the encoding block 912. Whether it can be. When checking whether the frame 902 has a matching reference block, the motion vector of the block 914 is used as a motion vector prediction candidate. This is because block 914 is one of three neighboring blocks and this reference frame is 902. Similarly, when checking whether the frame 904 has a matching reference block, the motion vector of the block 910 is used as a motion vector prediction candidate. This is because block 910 is one of three neighboring blocks and this reference frame is 904.
Frame 906 does not become a reference frame for any adjacent block. Thus, in this example, motion vectors are predicted using, for example, the median of blocks 910, 914, and 908. For example, in that case, the motion vector of the interframe coding block 908 is set to zero.

  It should be noted here that not only the motion data and reference frame data of the adjacent block but also the macro block (for example, the block to be encoded belongs) in predicting the motion vector of the encoded block. It is possible to use segmentation information. The number and location of adjacent blocks, as well as the number of stored reference blocks, can be different from that shown in FIG.

  FIG. 6 illustrates an example of a coder that can implement the above method of encoding a video frame. The design principle of the video coder is that a prediction error frame En (x, y) 600 between the encoding block In (x, y) 620 and the prediction candidate (selected reference block) Pn (x, y) 612 is used. Is to minimize. The prediction error frame is defined as a difference between the encoding block 620 and the prediction candidate 612 and is calculated using an adder 632. That is

          En (x, y) = In (x, y) −Pn (x, y) (4)

  The prediction block 612 is obtained using the selected reference frame and a plurality of motion vectors based on a method according to a preferred embodiment of the present invention. The motion vector (Δx (x, y), Δy (x, y)) coefficients are generated as the sum of the predicted motion vectors in the motion vector calculation block 630, and the coefficients are different. According to a preferred embodiment of the present invention, the predicted motion vector coefficients are calculated using the motion vectors of neighboring blocks. The adjacent block has the same reference frame as the block to be encoded. Other methods are also used for motion vector prediction. One of them is to directly encode a block to be encoded using a reference frame by a conventional method. A group of motion vectors for all pixels in a frame is called a motion vector field. Since there are a large number of pixels in a frame, in reality, when video encoding is performed, the frame is divided into blocks, and the motion information is transferred to the decoder of the receiver shown in FIG. 7 for each block.

  At least one motion vector is selected from a motion vector field composed of a predicted motion vector or a predicted motion vector, and is encoded in a motion vector encoding block 628. For example, the well-known motion model of the above embodiment is used for encoding. The encoded motion vector or the motion vector field formed therefrom is transferred to the multiplexer 624.

  At block 618, a frame is formed from the frame block, segment by segment, or block by block. The reference block or prediction candidate has the following format.

  At block 602, the prediction error frame En (x, y) 600 is encoded and is typically represented as a finite two-dimensional series transform (eg, using a discrete cosine transform (DCT)). The coefficients calculated by the DCT transform are quantized, encoded, and transferred to the multiplexer 624 and the decoder (decoder) of the receiver shown in FIG.

  In the multiplexer 624, information including motion vector information is combined with a prediction error frame. This frame information 616 is transferred to the decoder (FIG. 7).

  The function of the video coder is implemented in various ways such as software executed on a processor, or hardware such as logic composed of individual components or application-specific integrated circuit (ASIC).

  FIG. 7 shows an example of a decoder. Using this decoder, the above method is implemented to reduce temporal redundancy in the video frame. The encoded prediction error frame 702 and the encoded motion vector coefficient or motion vector field coefficient 712 of the received video signal 700 are separated from each other in the demultiplexer 710.

  The functions of the video coder are implemented in various ways, such as software running on a processor, or hardware such as logic composed of discrete components or ASICs.

  One terminal device of the wireless system is shown in FIG. The wireless system is used to implement the method of the preferred embodiment of the present invention. The transmitter of the subscriber terminal device 824 and the transmitter of the component of the radio system network partially execute the same task. The terminal device 824 may be, for example, a mobile phone or a microcomputer (including but not limited to) a wireless part. The terminal device includes an antenna 800, and uses it to transmit and receive signals through a duplex filter. The terminal device also includes a plurality of antennas, that is, a multi-antenna system. Further, the terminal device obtains from the antenna a transmitter 802 that amplifies the modulated signal and transfers it to the antenna, a modulator 804 that modulates the carrier of the data signal including the desired information based on the selected modulation method, and the antenna. A receiver 806 that amplifies the received signal and downconverts it to a desired intermediate frequency or directly to baseband, and a decoder 808 that decodes the received signal and separates the data signal from the carrier wave.

  The subscriber terminal device also controls the operation of other parts of the terminal device, such as the camera 820 and the video codec 822, and the user's voice or digital signal processing (DSP), digital analog conversion processing, filter processing, etc. The control block 816 performs operations necessary for processing data generated by the user. Also, in a spread spectrum system such as WCDMA, the signal spectrum is spread over a wide band using a pseudo-random spreading code at the transmitter (spread) and demodulated at the receiver (despread) to increase the channel capacity. To do. The control block also performs coding and decoding, such as channel / speech coding. In addition, the control block adjusts the transmission signal and signaling information to conform to the air system air interface standard. The subscriber terminal device includes one or more various types of memories (not shown, but are parts of other blocks or individual parts). In the example of FIG. 8, the control block also has a program for controlling the operation of the terminal device. In the example shown in the figure, the video codec 822 encodes and decodes the reconstructed data of the frame. However, the video codec 822 is an individual part, but can also be a part of the control block 816.

  When transferring the frame, the camera 820 records the digital image in the memory block 818. The camera 820 can also record the image directly on the codec 822. The codec encodes the selected frame using one or a combination of methods. The camera 820, the codec 822, and the memory 918 can be built into the subscriber terminal 824 or built into the individual camera unit 816. In the latter case, it is connected to the subscriber terminal device 824 via the interface 828.

  The user of the subscriber terminal device can view the frame on the display device 814 as necessary. The encoded frame reconstruction data is transferred to the communication system in the same way as voice or text data, for example, to other telephones.

  There are many ways to implement the operation blocks of the terminal device (for example, the control block and the codec block 822 required for encoding / decoding). For example, it is implemented by software executed on a processor, or hardware such as logic composed of individual components or ASICs. The functions of the terminal device shown in the figure can be incorporated into the operation block in many ways different from the figure. For example, the codec block 822 can be divided into a coder and a decoder.

  The user interface of the terminal device includes a speaker or earpiece 810, a microphone 812, a display device 814 and possibly a keyboard (all connected to a control block 816).

Although the present invention has been described with reference to the embodiments with reference to the drawings, the present invention is not limited thereto, and various modifications can be made within the scope of the technical idea described in the claims. Obviously it can be done.
The present invention is applicable with multiple video coding standards such as H.263 or MPEG.

1 is a block diagram of a communication system. 2A to 2C are diagrams illustrating a vector estimation process using a reference frame. It is a figure which shows the estimation process of the motion vector using an adjacent block. It is a flowchart which shows the method of finding a reference block and estimating a motion vector. An example showing how to find a reference block and predict a motion vector is shown. It is a block diagram of a coder. It is a block diagram of a decoder. It is a block diagram of a subscriber terminal device. A second example showing how to find a reference block and predict a motion vector is shown.

Claims (73)

Prediction for a block to be encoded using one of a plurality of reference frames, wherein the prediction comprises prediction consisting of motion data and the number of reference frames;
Comparing the reference frame number of the block to be encoded with the reference frame number of the previously encoded block;
If there is at least one previously encoded block having a reference frame number different from the reference frame number of the block to be encoded, it is a previously encoded block and is to be encoded. Using only motion data of the block with the same number of reference frames as the block, predicting motion data of the block to be encoded,
A method for encoding a video sequence comprising:   If only one previously encoded block has the same number of reference frames as the block to be encoded, the motion data prediction of the block to be encoded is the previously encoded block. The method of claim 1, wherein the method is set equal to the motion data of a block having the same number of reference frames as the block to be encoded.   The motion data prediction of a block to be encoded is a median of motion data of a block that has been previously encoded and has the same number of reference frames as the block to be encoded. Method.   The motion data prediction of a block to be encoded is an average of motion data of blocks that have been previously encoded and have the same number of reference frames as the block to be encoded. Method.   The method according to claim 1 or 3, wherein the motion data of the block is represented by a motion vector.   4. A method according to claim 1 or 3, wherein the previously encoded block is the block closest to the block to be encoded.   The method according to claim 6, wherein the nearest block is located on a left side, an upper side, and an upper right side of a block to be encoded.   The method according to claim 1 or 3, wherein the previously encoded block does not exist in the immediate vicinity of the block to be encoded and is spaced from the block to be encoded. Search for a prediction block from multiple available reference frames,
4. A method according to claim 1 or 3, further comprising selecting a prediction block for the current image block to achieve a predetermined code efficiency or a predetermined image quality.   10. The method of claim 9, wherein the search for a predictive block from among a plurality of available reference frames is based on a set of motion data candidates, a reference frame based on motion data, and a reference frame of a previously encoded block. .   The method according to claim 9, wherein the coding efficiency is estimated by a Lagrangian function.   4. The method according to claim 1 or 3, further comprising selecting one motion model from a plurality of motion models and notifying the selected motion model. A terminal (824) having a video codec (822) that performs a method for encoding a video sequence, the method including:
Prediction for a block to be encoded using one of a plurality of reference frames, wherein the prediction comprises prediction consisting of motion data and the number of reference frames;
Comparing the reference frame number of the block to be encoded with the reference frame number of the previously encoded block;
If there is at least one previously encoded block having a reference frame number different from the reference frame number of the block to be encoded, it is a previously encoded block and is to be encoded. Predict motion data of a block to be encoded using only motion data of a block having the same number of reference frames as the block.   If only one previously encoded block has the same number of reference frames as the block to be encoded, the motion data prediction of the block to be encoded is the previously encoded block. The terminal according to claim 13, which is set equal to motion data of a block having the same number of reference frames as a block to be encoded.   The terminal according to claim 13 or 14, wherein the motion data of the block is represented by a motion vector. Search for a prediction block from multiple available reference frames,
The terminal according to claim 13 or 14, further comprising selecting a prediction block for the current image block to achieve a predetermined code efficiency or a predetermined image quality.   The terminal according to claim 16, wherein the search for a prediction block from among a plurality of available reference frames is based on a candidate set of motion data, a reference frame based on motion data, and a reference frame of a previously encoded block. .   The terminal according to claim 16, wherein the coding efficiency is estimated by a Lagrangian function.   15. A terminal (824) according to claim 13 or 14, wherein the subscriber terminal (824) is a terminal of a wireless system.   15. Terminal (824) according to claim 13 or 14, selected from the group consisting of a mobile phone and a microcomputer with radio parts.   15. A terminal (824) according to claim 13 or 14, comprising a camera. Prediction for a block to be encoded using one of a plurality of reference frames, the prediction being a prediction means for performing prediction consisting of motion data and the number of reference frames;
Means for comparing a reference frame number of a block to be encoded with a reference frame number of a previously encoded block;
If there is at least one previously encoded block having a reference frame number different from the reference frame number of the block to be encoded, it is a previously encoded block and is to be encoded. A means for predicting motion data of a block to be encoded using only motion data of a block having the same number of reference frames as the block;
Having video encoder.   If only one previously encoded block has the same number of reference frames as the block to be encoded, the motion data prediction of the block to be encoded is the previously encoded block. The video encoder according to claim 22, which is set equal to the motion data of a block having the same number of reference frames as the block to be encoded.   The motion data prediction of a block to be encoded is a median of motion data of a block that has been previously encoded and has the same number of reference frames as the block to be encoded. Video encoder.   23. The motion data prediction of a block to be encoded is an average of motion data of blocks previously encoded and having the same number of reference frames as the block to be encoded. Video encoder.   The video encoder according to claim 22 or 24, wherein the motion data of the block is represented by a motion vector.   25. A video encoder according to claim 22 or 24, wherein the previously encoded block is the block closest to the block to be encoded.   28. The video encoder according to claim 27, wherein the most recent block is located on the left side, upper side, and upper right side of the block to be encoded.   25. Video encoder according to claim 22 or 24, wherein the previously encoded block is not in the immediate vicinity of the block to be encoded and is spaced from the block to be encoded. Means for searching for a prediction block from a plurality of available reference frames;
25. A video encoder according to claim 22 or 24, further comprising means for selecting a prediction block for the current image block to achieve a predetermined code efficiency or a predetermined image quality.   The video encoder of claim 30, wherein the search for a predictive block from among a plurality of available reference frames is based on a set of motion data candidates, a reference frame based on the motion data, and a reference frame of a previously encoded block. .   The video encoder according to claim 30, wherein the coding efficiency is estimated by a Lagrangian function. Means for selecting one motion model from a plurality of motion models;
25. A video encoder according to claim 22 or 24, further comprising means for informing the selected motion model. A computer readable medium storing computer readable code for performing a video frame encoding method, the method comprising:
A prediction for a block to be encoded using one of a plurality of reference frames, the prediction comprising a prediction consisting of motion data and the number of reference frames;
Comparing the reference frame number of the block to be encoded with the reference frame number of the previously encoded block;
If there is at least one previously encoded block with a reference frame number that is different from the reference frame number of the block to be encoded, it is a previously encoded block and is to be encoded. The motion data of the block to be encoded should be predicted using only the motion data of the block having the same number of reference frames as the target block.   If only one previously encoded block has the same number of reference frames as the block to be encoded, the motion data prediction of the block to be encoded is the previously encoded block. 35. The computer program of claim 34, set equal to the motion data of a block having the same number of reference frames as the block to be encoded.   35. The computer of claim 34, wherein the motion data prediction for a block to be encoded is a median of motion data for a block that has been previously encoded and has the same number of reference frames as the block to be encoded. program.   35. The computer of claim 34, wherein the motion data prediction of a block to be encoded is an average of motion data of blocks previously encoded and having the same number of reference frames as the block to be encoded. program.   37. The computer program according to claim 34 or 36, wherein the motion data of the block is represented by a motion vector.   37. A computer program according to claim 34 or 36, wherein the previously encoded block is the block immediately adjacent to the block to be encoded.   40. The computer program according to claim 39, wherein the most recent block is located on the left side, upper side, and upper right side of the block to be encoded.   37. The computer program according to claim 34 or 36, wherein the previously encoded block does not exist in the immediate vicinity of the block to be encoded and is separated from the block to be encoded.   The method searches for a prediction block from a plurality of available reference frames and selects a prediction block for the current image block to achieve a predetermined code efficiency or a predetermined image quality. 37. The computer program according to claim 34 or 36, further comprising:   43. The computer of claim 42, wherein a search for a predictive block from among a plurality of available reference frames is based on a set of motion data candidates, a reference frame based on motion data, and a reference frame of a previously encoded block. program.   The computer program according to claim 42, wherein the encoding efficiency is estimated by a Lagrangian function.   37. The computer program according to claim 34, further comprising selecting one motion model from a plurality of motion models and notifying the selected motion model. A method for decoding encoded video data, comprising:
Motion prediction information for a block to be decoded in encoded video data, receiving at least the number of reference frames and motion difference data;
Comparing the reference frame number of the block to be encoded with the reference frame number of the previously encoded block;
If the number of reference frames is different from the number of reference frames of the block to be encoded, and there is at least one previously encoded block, it is a previously encoded block and is to be encoded. Predicting the motion data of the block to be encoded using only the motion data of the block having the same number of reference frames as the block; and
Creating motion data of a block to be decoded based on the motion data and the received motion difference data;   If only one previously encoded block has the same number of reference frames as the block to be decoded, the motion data prediction of the block to be decoded is the previously encoded block. 47. The method of claim 46, set equal to the motion data of a block having the same number of reference frames as the block to be encoded.   The motion data prediction of a block to be encoded is a median of motion data of a block that has been previously encoded and has the same number of reference frames as the block to be encoded. Method.   The prediction of motion data of a block to be encoded is an average of motion data of blocks that have been previously encoded and have the same number of reference frames as the block to be encoded. Method.   49. A method according to claim 46 or 48, wherein the motion data of the block is represented by a motion vector.   49. A method according to claim 46 or 48, wherein the previously encoded block is the block closest to the block to be encoded.   52. The method of claim 51, wherein the most recent block is located on the left side, upper side, and upper right side of the block to be encoded.   49. A method according to claim 46 or 48, wherein the previously encoded block is not in the immediate vicinity of the block to be encoded and is spaced from the block to be encoded. Determining one motion model from a plurality of motion models;
49. A method according to claim 46 or 48, further comprising performing prediction on the block to be decoded using the motion model and motion data.   The method of claim 54, wherein the motion model is determined by signaling of encoded video data. A decoder for decoding encoded video data, comprising:
Motion prediction information for a block to be decoded in encoded video data, at least means for receiving a reference frame number and motion difference data;
Means for comparing the reference frame number of the block to be encoded with the reference frame number of the previously encoded block;
If there is at least one previously encoded block with a reference frame number that is different from the reference frame number of the block to be encoded, it is a previously encoded block and is to be encoded. Means for predicting motion data of a block to be encoded using only motion data of a block having the same reference frame number as the block
Means for generating motion data of a block to be decoded based on the motion data and the received motion difference data;   If only one previously encoded block has the same number of reference frames as the block to be decoded, the motion data prediction of the block to be decoded is the previously encoded block. 57. A video decoder according to claim 56, set equal to the motion data of a block having the same number of reference frames as the block to be encoded.   57. The motion data prediction of a block to be encoded is a median of motion data of a block that has been previously encoded and has the same number of reference frames as the block to be encoded. Video decoder.   57. The motion data prediction of a block to be encoded is an average of motion data of blocks previously encoded and having the same number of reference frames as the block to be encoded. Video decoder.   59. A video decoder according to claim 56 or 58, wherein the motion data of the block is represented by a motion vector.   59. A video decoder according to claim 56 or 58, wherein the previously encoded block is the block nearest the block to be encoded.   64. The video decoder according to claim 61, wherein the nearest block is located on the left side, upper side, and upper right side of the block to be encoded.   59. Video decoder according to claim 56 or 58, wherein the previously encoded block is not in the immediate vicinity of the block to be encoded and is spaced from the block to be encoded. Means for determining one motion model from a plurality of motion models;
59. A video decoder according to claim 56 or 58, further comprising means for performing prediction on the block to be decoded using the motion model and motion data.   The video decoder of claim 64, wherein the motion model is determined by signaling of encoded video data. A terminal (824) having a video codec (822) for performing a method for decoding encoded video data, the method comprising:
Motion prediction information for a block to be decoded in encoded video data, receiving at least the number of reference frames and motion difference data;
Comparing the reference frame number of the block to be encoded with the reference frame number of the previously encoded block;
If there is at least one previously encoded block with a reference frame number that is different from the reference frame number of the block to be encoded, it is a previously encoded block and is to be encoded. Predicting motion data of a block to be encoded using only motion data of a block having the same number of reference frames as the block
Creating motion data of a block to be decoded based on the motion data and the received motion difference data;   If only one previously encoded block has the same number of reference frames as the block to be encoded, the motion data prediction of the block to be encoded is the previously encoded block. The terminal according to claim 66, set equal to the motion data of a block having the same number of reference frames as the block to be encoded.   68. The terminal according to claim 66 or 67, wherein the motion data of the block is represented by a motion vector.   68. A terminal according to claim 66 or 67, wherein the previously encoded block is the block nearest the block to be encoded.   The terminal according to claim 69, wherein the nearest block is located on a left side, an upper side, and an upper right side of a block to be encoded.   68. A terminal as claimed in claim 66 or 67, wherein the previously encoded block is not in the immediate vicinity of the block to be encoded and is spaced from the block to be encoded.   68. A terminal according to claim 66 or 67, wherein the subscriber terminal is a terminal of a wireless system.   68. The terminal according to claim 66 or 67, which is selected from the group consisting of a mobile phone and a microcomputer having wireless components.

Images