JP5248632B2 - Techniques for motion estimation - Google Patents

Description

  H. also known as Advanced Video Codec (AVC). H.264 and MPEG-4 Part 10 are ITU-T / ISO video compression standards that are expected to be widely pursued in the industry. H. The H.264 standard is prepared by the Joint Video Team (JVT) and is known as VCEG (Video Coding Expert Group) ITU-T SG16 Q.264. 6 and ISO / IEC JTC1 / SC29 / WG11 known as MPEG (Motion Picture Wxpert Group). H. H.264 is a digital TV Digital TV (DTV) broadcast, Direct Broadcast Satelite (DBS) video, Digital Subscriber Line (DSL) video, Interactive Storage Media (ISM), Multimedia Digital TV (DSM), Multimedia Media Designed for applications in the area of Video Surveillance (RVS).

  Motion estimation in video coding can be used to improve video compression performance by removing or reducing temporal redundancy between video frames.

  To encode the input block, conventional motion estimation can be performed at the encoder in a particular search window of the reference frame. This allows the determination of a motion vector that minimizes the sum of absolute differences (SAD) between the input block and the reference frame in the reference frame. Motion vector (MV) information can then be sent to a decoder for motion compensation. The motion vector can be determined on a fractional pixel basis and an interpolation filter can be used to calculate the fractional pixel value.

  If the original input frame is not valid at the decoder, motion estimation (ME) at the decoder can be performed using the reconstructed reference frame. When encoding a predicted frame (P frame), there can be multiple reference frames in the forward reference buffer. When encoding a bi-predictive frame (B frame), there can be multiple reference frames in the forward buffer and at least one reference frame in the reverse reference buffer. For B-frame coding, mirror motion estimation (ME) or projected motion estimation (ME) can be performed to obtain a motion vector (MV). For predictive frame (P-frame) encoding, projected motion estimation (ME) can be performed to obtain motion vectors.

  In other contexts, block-based motion vectors can be generated at the video decoder by performing motion estimation on previously decoded valid pixels for a block in one or more frames. It is. Valid pixels can be, for example, spatially contiguous blocks in the current frame's series of scan coding orders, blocks in previously decoded frames, or down-sampled in lower layers when hierarchical coding is used It can be a block or the like in a frame that has been recorded. Effective pixels can alternatively be a combination of the above blocks.

  In conventional video coding systems, motion estimation is performed at the encoder side to determine motion vectors for prediction of the current coding block, and these motion vectors are encoded on the binary stream. It needs to be transmitted to the decoder side for motion compensation of the current decoding block. Some modern video coding standards, such as H.264. In H.264 / AVC, a macroblock (MB) can be partitioned into smaller blocks for encoding, and a motion vector can be assigned to each sub-partitioned block. As a result, when a macroblock is partitioned into 4 × 4 blocks, there are up to 16 motion vectors for predictive coding macroblocks and up to 32 motion vectors for bi-predictive coding macroblocks (MB). As a result, substantial bandwidth is used to transmit motion vector information from the encoder to the decoder.

FIG. 6 is a diagram illustrating an example of an embodiment for determining a motion vector for a current block in a bi-predicted frame using mirror motion estimation ME. FIG. 10 is a diagram illustrating an example of projection motion estimation that determines a motion vector for a current block in a prediction frame based on two forward reference frames. It is a figure which shows the extended reference block. It is a figure which shows the block which adjoins the present block spatially. It is a figure which shows the process according to embodiment. FIG. 6 illustrates an embodiment used to determine a motion vector. An exemplary H.264 having a self-motion vector derivation module. 1 illustrates a H.264 video encoder architecture. FIG. H. having self-motion vector derivation module. 2 is a diagram illustrating a H.264 video decoder. FIG.

  A digital video clip has a plurality of consecutive video frames. The motion of an object or background in successive frames can follow a smooth trajectory, and the motion in successive frames can have a relatively strong temporal correlation. Using this correlation, a motion vector can be derived for the current coding block by estimating motion from the reconstructed reference picture. Motion vector determination at the decoder can reduce the transmission bandwidth compared to motion estimation performed at the encoder.

  If the original input pixel information is not valid at the decoder, motion estimation at the decoder can be performed using the reconstructed reference frame and a valid reconstructed block of the current frame. Here, the term “valid” means that the block has been reconstructed prior to the current block. When encoding a predicted frame, there may be multiple reference frames in the forward reference buffer. When encoding a bi-predicted frame, there may be multiple reference frames in the forward reference buffer and at least one reference frame in the backward reference buffer.

  Next, performing motion estimation at the decoder to obtain a motion vector for the current block according to one embodiment will be described. For bi-predictive frame coding, mirror motion estimation or projected motion estimation can be performed to determine a motion vector. For predictive frame coding, projected motion estimation can be performed to determine a motion vector. Note that the terms “frame” and “picture” are used interchangeably herein, as will be appreciated by those skilled in the art.

  Instead of receiving a motion vector from an encoder, various embodiments are provided for a decoder to determine a motion vector for a decoded block. Decoder side motion estimation may be performed based on temporal frame correlation and spatially adjacent blocks of the reference block and spatially adjacent blocks of the current block. For example, the motion vector can be determined by performing a decoder-side motion search between two reconstructed pictures in the reference buffer. Projection motion estimation (ME) can be used for blocks in the predicted picture, and both projection motion estimation and mirror motion estimation can be used for blocks in the bi-predictive picture. Also, motion estimation can be performed on block type subdivisions. Coding efficiency can be improved by applying an adaptive search region for decoder side motion search. For example, a technique for determining a search area is disclosed in US Patent Application Publication No. 12 / 582,061 filed on October 20, 2009.

  FIG. 1 shows an example of an embodiment that uses mirror motion estimation to determine a motion vector for the current block in a bi-predictive frame. In the embodiment of FIG. 1, there may be two bi-prediction frames 110 and 115 between the forward reference frame 120 and the backward reference frame 130. Frame 110 may be the current encoded frame. When encoding the current block 140, mirror motion estimation can be performed to obtain a motion vector by performing a search in each of the search windows 160 and 170 of the reference frames 120 and 130, respectively. . As mentioned above, if the current input block is not valid at the decoder, mirror motion estimation can be performed with these two reference frames.

  FIG. 2 shows the current block in the prediction frame based on two forward reference frames: forward Ref0 (shown as reference frame 220) and forward Ref1 (shown as reference frame 230). 3 shows an example of projection motion estimation that determines a motion vector for. Those reference frames can be used to derive a motion vector for the target block 240 in the current frame 210. The search window 270 can be specified in the reference frame 220 and the search path can be specified in the search window 270. For each motion vector MV 0 in the search path, its projected motion vector MV 1 can be determined in the search window 260 of the reference frame 230. For each pair of motion vectors, ie, MV0 and its associated motion vector MV1, the metric such as the sum of absolute differences is (1) a reference block 280 in which MV0 is oriented in the reference frame 220, and (2) a reference frame. 230 can be calculated with reference block 250 facing MV1. The motion vector MV0 that obtains the optimal value for that metric, eg, the smallest SAD (sum of absolute differences), can then be selected as the motion vector for the target block 240.

  Techniques for determining motion vectors for the scenario described with respect to FIGS. 1 and 2 are shown in FIGS. 2 and 4 of US Patent Application Publication No. 121 / 566,823, filed September 25, 2009. It is shown.

  An exemplary search for motion vectors can be processed as shown in processes 300 and 500 of U.S. Patent Application Publication No. 121 / 566,823. Next, a summary of the process for determining motion vectors for the scenario of FIG. 1 herein is provided. A search window can be specified in the forward reference frame. This search window can be the same in both the encoder and the decoder. The search path can be specified in the forward search window. As long as the encoder and decoder follow the same search path, a full search scheme or any fast search scheme can be used here. For MV0 in the search path, its mirror motion vector MV1 can be obtained in the backward search window. Here, it is assumed that the motion trajectory is a relatively short straight line during the relevant time period. Metrics such as the sum of absolute differences (SAD) are calculated between (i) a reference block facing MV0 in a forward reference frame and (ii) a reference block facing MV1 in a backward reference frame. Can be done. These reference blocks are indicated in FIG. 1 by reference numerals 150 and 180, respectively. A determination can be made as to whether any additional motion vector MV0 is present in the search path. If it is preferred, the process is repeated and more than one MV0 is obtained if each MV0 has an associated MV1. In addition, for each such associated pair, a metric such as, for example, the sum of absolute differences (SAD) is obtained. The MV0 that produces the desired value for that metric, ie, the smallest SAD, can be selected, but is not limited thereto. This MV0 can then be used to predict the motion for the current block.

  Next, a summary of the process for determining the motion vector for the scenario of FIG. 2 herein is provided. A search window can be specified in the first forward reference frame. This window can be the same in both the encoder and the decoder. The search path can be specified in this search window. For example, a full search scheme or a fast search scheme is used here, so that the encoder and decoder can follow the same search path. For the motion vector MV0 in the search path, the projected motion vector MV1 can be obtained in the second search window. Here, it is assumed that the motion trajectory is a straight line in this short path. Metrics such as SAD can be calculated between (i) a reference block that faces MV0 in the first reference frame and (ii) a reference block that faces MV1 in the second reference frame. . A determination can be made as to whether there are any additional motion vectors MV0 that remain in the search path and have not yet been considered. If at least one MV0 remains, the process is repeated, and for the other MV0, the corresponding projected motion vector MV1 can be determined. In this way, a set of MV0 and MV1 pairs can be determined and a metric, eg, SAD, can be calculated for each pair. One of the plurality of MV0s is selected, and the selected MV0 obtains a desired value for the metric, eg, the minimum SAD, but is not limited thereto. Since a SAD metric of 0 is a theoretically optimal value, a minimum effective value for the SAD metric, i.e., a value close to 0, indicates a preferred mode. This MV0 is then used to predict the motion for the current block.

In various embodiments, the sum of absolute differences between two mirror blocks or between two projection blocks in two reference frames is determined to determine a motion vector. The size of the current block is M × N pixels, and the position of the current block is represented by the coordinates of the upper left pixel of the current block. In various embodiments, the motion vector in reference frame R ₀ is MV ₀ = (mv ₀ _x, mv ₀ _y), and the corresponding motion vector in other reference frame R ₁ is MV ₁ = (mv ₁ _x, mv ₁ _y), the motion search metric is determined using the following equation (1).
J = J ₀ + α ₁ J ₁ + α ₂ J ₂ (1)
Here, J ₀ is the reference that the reference block MV0 is facing, the backward reference frame (or the second forward reference frame in the scenario of FIG. 2) is MV1 in facing in (i) a forward reference frame Represents the sum of absolute difference values calculated between the blocks and disclosed in US patent application Ser. No. 12 / 566,823, filed Sep. 25, 2009,
J ₁ is an expanded metric based on spatially adjacent blocks of the reference block, and J ₂ is an expanded metric based on spatially adjacent blocks of the current block, where α ₁ And α ₂ are two weighting factors. The weighting factors α ₁ and α ₂ can be determined by simulation and the default is set to 1.

The motion vector MV0 that obtains the optimal value for the value J, eg, the minimum value SAD, according to equation (1) can then be selected as the motion vector for the current block. The motion vector MV0 is expressed by the following equation MV1 = (d ₁ / d ₀ ) × MV0
Has an associated motion vector MV1 defined according to
Here, when the current block is in a bi-predictive picture, d ₀ represents the distance between the picture of the current frame and the forward reference frame shown in FIG.
When the current block is in the predicted picture, d ₀ represents the distance between the picture of the current frame and the first forward reference frame shown in FIG.
When the current block is in the bi-predictive picture, d ₁ represents the distance between the picture of the current frame and the backward reference frame shown in FIG. 1, and the current block is in the predictive picture At some point, d ₁ represents the distance between the picture of the current frame and the second forward reference frame shown in FIG.

  For the scenario of FIG. 1, given the pair of motion vectors MV0 and MV1 obtained for the current block, the forward prediction P0 (MV0) can be obtained by MV0 and the backward prediction P1 (MV1 ) Can be obtained by MV1 and bi-directional prediction can be obtained by both MV0 and MV1. The bi-directional prediction can be, for example, the average of P0 (MV0) and P1 (MV1), or the weighted average (P0 (MV0) * d1 + P1 (MV1) * d0) / (d0 + d1). Alternative functions can be used to obtain bi-directional prediction. In an embodiment, the encoder and decoder can use the same prediction method. In an embodiment, the selected prediction method can be signaled in a bitstream identified or encoded in the standard specification.

  In the scenario of FIG. 2, the prediction for the current block can be obtained in different ways. These predictions may be, for example, P0 (MV0), P1 (MV1), (P0 (MV0) + P1 (MV1)) / 2 or (P0 (MV0) * d1 + P1 (MV1) * d0 / (d0 + d1)) In other embodiments, other functions can be used, and their predictions can be obtained in the same way in both the encoder and decoder. The prediction method can be identified in the standard specification or signaled in the encoded bitstream.

In various embodiments, J ₀ can be determined using the following equation:

ここで、Ｎ及びＭは現在のブロックのｘ寸法及びｙ寸法のそれぞれであり、
Ｒ_０は第１順方向参照フレームであり、Ｒ_０（ｘ＋ｍｖ_０＿ｘ＋ｉ，ｙ＋ｍｖ_０＿ｙ＋ｊ）は位置（ｘ＋ｍｖ_０＿ｘ＋ｉ，ｙ＋ｍｖ_０＿ｙ＋ｊ）のＲ_０における画素値であり、
Ｒ_１は、ミラー動き推定（ＭＥ）についての第１逆方向参照フレーム又は投影動き推定についての第２順方向参照フレームであり、Ｒ_１（ｘ＋ｍｖ_１＿ｘ＋ｉ，ｙ＋ｍｖ_１＿ｙ＋ｊ）は位置（ｘ＋ｍｖ_１＿ｘ＋ｉ，ｙ＋ｍｖ_１＿ｙ＋ｊ）のＲ_１における画素値であり、
ｍｖ_０＿ｘは、参照フレームＲ_０のｘ方向における現在のブロックについての動きベクトルであり、
ｍｖ_０＿ｙは、参照フレームＲ_０のｙ方向における現在のブロックについての動きベクトルであり、
ｍｖ_１＿ｘは、参照フレームＲ_１のｘ方向における現在のブロックについての動きベクトルであり、
ｍｖ_１＿ｙは、参照フレームＲ_１のｙ方向における現在のブロックについての動きベクトルである。

Where N and M are the x and y dimensions of the current block, respectively.
R ₀ is the first forward reference frame, R ₀ (x + mv ₀ _x + i, y + mv ₀ _y + j) is the pixel value in R _{0 at} position (x + mv ₀ _x + i, y + mv ₀ _y + j),
R ₁ is a first backward reference frame for mirror motion estimation (ME) or a second forward reference frame for projection motion estimation, and R ₁ (x + mv ₁ _x + i, y + mv ₁ _y + j) is a position (x + mv ₁ _x + i). , Y + mv ₁ _y + j) at R ₁ ,
mv _{0 —} x is the motion vector for the current block in the x direction of the reference frame R ₀ ,
mv ₀ _y is a motion vector for the current block in the y direction of reference frame R ₀ ,
mv _{1 —} x is the motion vector for the current block in the x direction of the reference frame R ₁ ,
mv ₁ _y is a motion vector for the current block in the y direction of the reference frame R ₁ .

  When the motion vector is directed toward the fractional pixel position, the pixel value is obtained by interpolation, for example H.264. It is obtained by bilinear interpolation or 6-tap interpolation specified in the H.264 / AVC standard specification.

Description of variables J ₁ is made with reference to FIG. FIG. 3 shows the expanded reference block. M × N reference blocks 302 are extended at four edges with an extended edge size that is W ₀ , W ₁ , H ₀ and H ₁ respectively. Accordingly, each of the reference blocks in reference frames R ₀ and R ₁ used to determine the motion vector in the scenario of FIGS. 1 and 2 is extended according to the embodiment of FIG. In some embodiments, the metric J ₁ is calculated using the following equation:

ここで、Ｍ及びＮは元の参照ブロックの寸法である。拡張された参照ブロックの寸法は（Ｍ＋Ｗ_０＋Ｗ_１）ｘ（Ｎ＋Ｈ_０＋Ｈ_１）であることに留意されたい。

Here, M and N are the dimensions of the original reference block. Note that the dimensions of the expanded reference block are (M + W ₀ + W ₁ ) × (N + H ₀ + H ₁ ).

Description of variables J ₂ is made with reference to FIG. FIG. 4 shows a block that is spatially next to the current block 402. The variable J _2, the reference block is noted that as described with reference to the current block in contrast. The current block is positioned in the new picture. Block 402 is the current block of M × N pixels. Since the decoding of the blocks is in raster scan order, the four decoded spatially neighboring areas, namely the left neighbor area A ₀ , the top neighbor area A ₁ , the top left neighbor area A ₂ and the top right neighbor area it is possible to have a _3. When the current block is at multiple frame boundaries or not at the upper or left boundary of the parent macroblock (MB), some of the spatially adjacent regions are not valid for the current block. Validity flags are defined as γ ₀ , γ ₁ , γ ₂ and γ ₃ for the four regions. If the region flag is equal to 1, the region is valid; if the region flag is equal to 0, the region is not valid. In that case, the valid spatial region is defined as A _avail for the current block as follows:
A _avail = γ ₀ A ₀ + γ ₁ A ₁ + γ ₂ A ₂ + γ ₃ A ₃
Therefore, the metric J ₂ is calculated as follows.

ここで、Ｃ（ｘ，ｙ）は、現在のブロックを境界付ける領域内の現在のフレームにおける画素であり、ω_０及びω_１は、新しいピクチャと、参照フレーム０及び１との間のフレーム距離に従って設定される、若しくは０．５に設定されることが可能である２つの重み係数である。

Where C (x, y) is the pixel in the current frame in the region that bounds the current block, and ω ₀ and ω ₁ are the frame distances between the new picture and the reference frames 0 and 1 Are two weighting factors that can be set according to or can be set to 0.5.

When Rx represents a new picture, equal weight occurs when the distance from R ₀ to R _x is equal to the distance from R ₁ to R _x . If the distance from R ₀ to R _x is different from the distance from R ₁ to R _x , the weighting factor is set accordingly based on the weighting difference.

  In the embodiment, the parameters in FIG. 4 are set as shown in the following equation, but are not limited thereto.

図５は、実施形態に従った処理を表している。ブロック５０２は、現在のブロックが双予測ピクチャ内にあるときの順方向参照フレームにおける、又は現在のブロックが予測ピクチャ内にあるときの第１順方向参照フレームにおける、探索ウィンドウを指定する段階である。この探索ウィンドウは、エンコーダ及びデコーダの両方において同じであることが可能である。

FIG. 5 shows processing according to the embodiment. Block 502 is the step of specifying a search window in the forward reference frame when the current block is in the bi-predictive picture or in the first forward reference frame when the current block is in the predicted picture. . This search window can be the same in both the encoder and the decoder.

  Block 504 includes specifying a search path in the forward search window. Here, a full search scheme or any fast search scheme is used as long as the encoder and decoder follow the same search path.

Block 506 includes, for each MV0 in the search path, (1) a motion vector MV1 in the search window for the second reference frame and (2) a reference block in the first reference frame and a second reference frame to which MV1 is directed. Determining a metric based on the reference block. For MV0 in the search path, when the current block is in the bi-predictive picture, the mirror motion vector MV1 is obtained in the backward search window. For MV0 in the search path, when the current block is in the predicted picture, the projected motion vector MV1 is obtained in the search window for the second forward reference frame. Here, it is possible to assume that the motion trajectory is a relatively short straight line during the relevant time period. MV1 is obtained as a function of the following equation for MV0, where d0 and d1 are the distances between the current frame and each of the respective reference frames.
MV1 = (d ₁ / d ₀ ) × MV0
Block 508 comprises selecting a motion vector MV0 having the most favorable metric. For example, the above metric J is determined, and MV0 associated with the minimum value of the metric J is selected. This MV0 can then be used to predict the motion for the current block.

  FIG. 6 illustrates an embodiment used to determine motion vectors. System 600 can include a body of memory 610 and a processor 620 that can have one or more computer-readable media capable of storing computer program logic 640. The memory 610 can be implemented, for example, as a removable medium such as a hard disk drive, a compact disk drive, or a read only memory (ROM) drive. The memory can be accessed remotely over the network by the processor 620. The processor 620 and the memory 610 can be in communication using any of a number of techniques known to those skilled in the art, such as a bus. The logic contained in memory 610 can be read and executed by processor 620. One or more I / O ports and / or I / O devices, collectively shown as I / O 630, can also be connected to processor 620 and memory 610. The I / O port can have one or more antennas for a wireless communication interface, or can have a wired communication interface.

  Computer program logic 640 may include motion estimation logic 660. When motion estimation logic is executed, it is possible to perform the motion estimation process described above. The motion estimation logic 660 can include, for example, projection motion estimation logic that, when executed, performs the above operations. Logic 660 may be, for example, mirror motion estimation logic, logic that performs motion estimation based on temporal or spatial neighboring blocks of the current block, or logic that performs motion estimation based on underlying blocks corresponding to the current block. , Or as an alternative.

  A search range vector may be generated prior to motion estimation logic 660 that performs the process. This search range vector can be executed as described above by the search range calculation logic 650. The technique executed for the search calculation is described in, for example, US Application Publication No. 12 / 582,061 filed on October 20, 2009. Once the search range vector is generated, this vector can be used to demarcate the search performed by motion estimation logic 660.

  The logic that performs the search range vector determination can be incorporated into a self-motion vector derivation module used in larger codec architectures. FIG. 7 illustrates an exemplary H.264 that may have a self motion vector derivation module 740. H.264 video encoder architecture 700, where H.264 video encoder architecture 700 is shown. H.264ha video codec standard. Current video information can be provided from the current video block 710 in the form of multiple frames. The current video can be passed to the difference unit 711. The difference unit 711 may be part of a Differential Pulse Code Modulation (DPCM) (also called core video coding) loop that may have a motion compensation stage 722 and a motion estimation stage 718. It is. The loop may also have an intra prediction stage 720 and an intra interpolation stage 724. In some cases, an in-loop deblocking filter 726 can also be used in the loop.

  The current video 710 can be provided to the difference unit 711 and the motion estimation stage 718. Motion compensation stage 722 or intra-interpolation stage 724 can then produce an output via switch 723 that can be subtracted from the current video to produce a residual. It is. The remainder is then transformed and quantized in transform / quantization stage 712 and subjected to entropy coding in block 714. As a result, a channel output is obtained at block 716.

  The output of motion compensation stage 722 or internal interpolation stage 724 can be provided to an adder 733 that can also accept inputs from inverse quantization unit 730 and inverse transform unit 732. The latter two units cancel the transformation and quantization of the transformation / quantization stage 712. Inverse transform unit 732 may be dequantized and provide the inverse transformed information back to the loop.

  The self-motion vector deriving module 740 can perform the above processing for deriving a motion vector. Self-motion vector derivation module 740 can accept the output of in-loop deblocking filter 726 and can provide an output to motion compensation stage 722.

  FIG. 8 shows the H.264 with self-motion vector derivation module 810. A H.264 video decoder 800 is shown. Here, the decoder 800 for the encoder 700 of FIG. 7 may have a channel input 838 coupled to the entropy decoding unit 840. The output from the decoding unit 840 can be provided to the inverse quantization unit 842 and the inverse transform unit 844 and to the self motion vector derivation module 810. Self-motion vector derivation module 810 can be coupled to motion compensation unit 848. The output of the entropy decoding unit 840 can also be supplied to an intra interpolation unit 854 that can be supplied to a changeover switch 823. Information from the inverse transform unit 844 and either the motion compensation unit 848 selected by the switch 823 or the intra interpolation unit 854 is then combined and supplied to the in-loop deblocking unit 846 and back to the intra interpolation unit. Can be supplied as follows. The output of the in-loop deblocking unit 846 can then be provided to the self motion vector derivation module 810.

  The self motion vector deriving module is positioned in the video encoder and can be synchronized with the video decoder side. The self-motion vector derivation module can alternatively be applied to a general video codec architecture, It is not limited to the H.264 encoding architecture. Thus, motion vectors are not transmitted from the encoder to the decoder, which can save transmission bandwidth.

  Various embodiments use a combined spatial-temporal motion search for decoder-side motion estimation (ME) of the self-motion vector (MV) derivation module to improve the coding efficiency of the video codec system.

  The graphics and / or processing techniques described herein can be implemented in various hardware architectures. For example, graphics functions and / or video functions can be integrated into the chipset. Alternatively, a separate graphics processor and / or video processor can be used. In other embodiments, the graphics functions and / or video functions can be implemented by a general purpose processor having a multi-core processor. In other embodiments, these functions can be implemented in a consumer electronics device.

Embodiments of the present invention use software stored by a motherboard, hardwired logic, memory device, and by a microprocessor, firmware, ASIC (application specific integrated circuit) and / or FPGA (Field Programmable Gate Array). It can be implemented as any one or more of microchips or integrated circuits that are implemented or a combination thereof. The term "logic (logic)" may be as exemplified, with a combination of software or hardware and / or software and hardware.

  Embodiments of the present invention perform operations according to embodiments of the present invention when executed by one or more machines, such as, for example, a computer, a network of computers, or other electronic devices. It can be provided as a computer program product that can have one or more machine-readable media on which device-executable instructions provided in one or more machines are stored. Machine-readable media include floppy (registered trademark) disk, optical disk, CD-ROM (Compact Disc-Read Only Memories), magneto-optical disk, ROM (read-only memory), RAM (random access memory), EPROM (erasable) Programmable ROM), EEPROM (electrical extinguishable / programmable ROM), magnetic card or optical card, flash memory, or other type of machine-executable medium suitable for storing machine-executable instructions, such as It is not limited to.

  The accompanying drawings and detailed description above provide examples of the invention. Although shown as different functional elements in them, one skilled in the art can appreciate that one or more of those elements can be successfully combined into a single functional element. Alternatively, a particular element can be divided into a plurality of functional elements. Elements in one embodiment can be added to other embodiments. Further, the steps of any flowchart need not be performed in the order shown, and not all of those steps need be performed. In addition, a stage that does not depend on another stage can be executed in parallel with the other stage. However, the scope of the invention is in no way limited to those specific examples. Regardless of whether it is explicitly described in the specification, various modifications such as structure, dimensions, and material use are possible. The scope of the present invention is described in the claims filed on the same day.

Claims (20)

A method performed by a computer comprising:
Designating a search window in a first reference frame in a video decoder;
Designating a search path in the search window of the first reference frame;
Each motion vector MV0 is directed from the current block toward the reference block in the search window. For each motion vector MV0 in the search path, the corresponding first in the second reference frame is directed toward the reference block. Determining a second motion vector MV1, wherein the corresponding second motion vector MV1 is a function of the motion vector MV0;
Determining a metric for each pair of the motion vector MV0 and the second motion vector MV1 in the search path, the metric having a combination of a first metric, a second metric, and a third metric. The first metric is based on temporal frame correlation, the second metric is based on spatially adjacent blocks of the reference block, and the third metric is spatially adjacent blocks of the current block. Based on the stage;
Selecting the motion vector MV0 for which the corresponding value of the motion vector MV0 for the metric is a desired value, wherein the selected motion vector MV0 is used as a motion vector for the current block; And providing a picture for display, wherein the picture for display is based on a portion of the selected motion vector MV0;
Having a method. The method of claim 1, wherein the steps of determining a metric are:
Determining a weighted average of the first metric, the second metric, and the third metric;
Having a method. 3. A method as claimed in claim 1 or 2, wherein the step of determining a metric is:
Next formula

Determining the first metric based on where N and M are the y and x dimensions of the current block, respectively, and R ₀ has a first forward reference frame; R ₀ (x + mv ₀ _x + i, y + mv ₀ _y + j) is the pixel value at R _{0 at} position (x + mv ₀ _x + i, y + mv ₀ _y + j), and R ₁ is the first reverse reference frame or projection motion estimation for mirror motion estimation R ₁ (x + mv ₁ _x + i, y + mv ₁ _y + j) has a pixel value in R _{1 at} position (x + mv ₁ _x + i, y + mv ₁ _y + j), and mv ₀ _x is a reference frame R has a motion vector for the current block in the ₀ in the x-direction, _mv 0 _y is present in the y-direction of the reference frame _{R 0} Has a motion vector for the block, mv 1 _{_x} has a motion vector for the current block in the x direction of the reference frame R _1, mv 1 _{_y} is for the current block in the y direction of the reference frame R ₁ Having a motion vector, stage;
Having a method. 4. A method as claimed in any preceding claim, wherein the steps of determining a metric are:
Next formula

Determining the second metric based on, where W ₀ , W ₁ , H ₀ and H ₁ are expanded edge sizes, and the expanded reference block dimensions are (M + W ₀ + W ₁ ) × (N + H ₀ + H ₁ ), steps;
Having a method. 5. A method as claimed in any preceding claim, wherein the step of determining a metric is:
Next formula

Determining the third metric based on where A _avail has an area around the current block and C (x, y) is in the area bounding the current block Having pixels in the current frame, ω ₀ and ω ₁ are two weighting factors set according to the frame distance between the new picture and the reference frames 0 and 1;
Having a method.   6. The method according to any one of claims 1 to 5, wherein the current block is in a bi-predictive picture, the first forward reference frame comprises a forward reference frame, and the second forward The method, wherein the direction reference frame comprises a reverse reference frame.   7. A method as claimed in any preceding claim, wherein the current block is in a predicted picture, the first forward reference frame comprises a first forward reference frame, and the second The method wherein the forward reference frame comprises a second forward reference frame.   8. A method according to any one of the preceding claims, wherein the metric is a sum of absolute differences and the desired value is a minimum sum of absolute differences. A method according to any one of the preceding claims, wherein:
In the encoder, determining a motion vector for the current block by designating a second search window in a third reference frame and designating a second search path in the second search window of the third reference frame. ;
Each motion vector MV2 is directed from the current block toward the reference block in the second search window, and for each motion vector MV2 in the second search path, is directed toward the reference block in the fourth reference frame. Determining a corresponding second motion vector MV3;
Determining a metric for each pair of motion vector MV2 and second motion vector MV3 in the second search path, the metric comprising a combination of a first metric, a second metric, and a third metric Selecting the motion vector MV2 for which the corresponding value of the motion vector MV2 for the metric is a desired value, the selected motion vector MV2 being a motion vector for the current block Used as a stage;
The method further comprising: A logic means for determining the respective motion vector MV0 in the search path, each MV0 is facing towards the current block of the reference block in the search window, the logic means;
A logic means for determining the second motion vector MV1 corresponding facing towards the reference block in the second reference frame, the corresponding second motion vector MV1 is a function of MV0, logic means;
Logic means for determining a metric for each pair of motion vector MV0 and corresponding second motion vector MV1 in the search path, the metric comprising a first metric, a second metric and a third metric The first metric is based on temporal frame correlation, the second metric is based on spatially adjacent blocks of the reference block, and the third metric is spatially adjacent to the current block based on the block, the logic means; a logic means corresponding value of the motion vector MV0 of and the metric to select the MV0 is the desired value, the motion vector MV0 said selected, said the current block Logical means used as a motion vector of
A video decoder. The video decoder according to claim 10, wherein:
Logic means for designating the search window in the first reference frame;
Logic means for designating a search window in and the second reference frame; logic means for specifying the searched route in the search window of the first reference frame;
A video decoder. 12. A video decoder according to claim 10 or 11, wherein the logic means is to determine a metric:
Next formula

To determine the first metric, where N and M are the y and x dimensions of the current block, respectively, and mv ₀ _x is the x direction of the reference frame R ₀ Mv ₀ _y has a motion vector for the current block in the y direction of reference frame R ₀ , and mv ₁ _x is a current block in the x direction of reference frame R ₁ Mv ₁ _y has a motion vector for the current block in the y direction of reference frame R ₁ ;
Video decoder. 13. A video decoder as claimed in any one of claims 10 to 12, wherein the logic means is for determining a metric:
Next formula

To determine the second metric, where W ₀ , W ₁ , H ₀ and H ₁ are extended edge sizes, and the expanded reference block dimensions are (M + W ₀ + W ₁ ) × (N + H ₀ + H ₁ );
Method. 14. A method as claimed in any one of claims 10 to 13, wherein the logic means is to determine a metric:
Next formula

, Wherein A _avail has an area around the current block and C (x, y) is in the area bounding the current block With pixels in the current frame, ω ₀ and ω ₁ are two weighting factors set according to the frame distance between the new picture and reference frames 0 and 1;
Video decoder.   15. A video decoder according to any one of claims 10 to 14, wherein the current block is in a bi-predictive picture, the first forward reference frame comprises a forward reference frame, and the second A video decoder, wherein the forward reference frame comprises a backward reference frame.   16. The video decoder according to any one of claims 10 to 15, wherein the current block is in a predicted picture, the first forward reference frame comprises a first forward reference frame, The video decoder, wherein the two forward reference frames comprise a second forward reference frame. display;
A processor communicatively coupled to the display, the processor comprising:
Each motion vector MV0 is directed from the current block toward the reference block in the search window, and determines each motion vector MV0 in the search path;
A corresponding second motion vector MV1 facing the reference block in the second reference frame is a function of the motion vector MV0, determining the corresponding second motion vector MV1;
The metric for each pair of motion vector MV0 and corresponding second motion vector MV1 in the search path has a combination of a first metric, a second metric, and a third metric, wherein the first metric is time. The second metric is based on spatially neighboring blocks of the reference block, and the third metric is based on spatially neighboring blocks of the current block, Decide
A processor in which the motion vector MV0 is selected whose corresponding value of the motion vector MV0 for the metric is a desired value, and the selected motion vector MV0 is used as a motion vector for the current block;
Having a system. The system of claim 17, wherein:
A wireless network interface communicatively coupled to the processor;
Further comprising a system. 19. A system as claimed in claim 17 or 18, wherein the processor:
Next formula

Determining the first metric based on:
Next formula

And determining the second metric based on:

Determining the third metric based on:
Where N and M are the y and x dimensions of the current block, respectively, mv ₀ _x has a motion vector for the current block in the x direction of the reference frame R ₀ , and mv ₀ _y is a reference has a motion vector for the current block in the y direction of the frame _{R 0, mv} 1 _x has a motion vector for the current block in the x direction of the reference frame _{R 1, mv} 1 _y the reference frame _{R 1} has a motion vector for the current block in the y direction;
W ₀ , W ₁ , H ₀ and H ₁ are expanded edge sizes, and the dimensions of the expanded reference block are (M + W ₀ + W ₁ ) × (N + H ₀ + H ₁ )
A _avail has an area around the current block, C (x, y) has pixels in the current frame in the area that bounds the current block, and ω ₀ and ω ₁ are the new pictures And two weighting factors set according to the frame distance between and reference frames 0 and 1;
system. 20. A system according to any one of claims 17 to 19, comprising:
When the current block is in a bi-predictive picture, the first forward reference frame has a forward reference frame and the second forward reference frame has a backward reference frame;
When the current block is in a predicted picture, the first forward reference frame has a first forward reference frame and the second forward reference frame has a second forward reference frame;
system.

Images