EP2026584A1 - Estimation et compensation de mouvement avec correction de vecteur de mouvement basée sur des valeurs de composantes verticales - Google Patents

Estimation et compensation de mouvement avec correction de vecteur de mouvement basée sur des valeurs de composantes verticales Download PDF

Info

Publication number
EP2026584A1
EP2026584A1 EP20080167885 EP08167885A EP2026584A1 EP 2026584 A1 EP2026584 A1 EP 2026584A1 EP 20080167885 EP20080167885 EP 20080167885 EP 08167885 A EP08167885 A EP 08167885A EP 2026584 A1 EP2026584 A1 EP 2026584A1
Authority
EP
European Patent Office
Prior art keywords
field
motion
motion vector
chrominance
picture
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Withdrawn
Application number
EP20080167885
Other languages
German (de)
English (en)
Inventor
Tatsushi Otsuka
Takahiko Tahira
Akihiro Yamori
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Fujitsu Semiconductor Ltd
Original Assignee
Fujitsu Semiconductor Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Fujitsu Semiconductor Ltd filed Critical Fujitsu Semiconductor Ltd
Publication of EP2026584A1 publication Critical patent/EP2026584A1/fr
Withdrawn legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/186—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a colour or a chrominance component
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/103—Selection of coding mode or of prediction mode
    • H04N19/112—Selection of coding mode or of prediction mode according to a given display mode, e.g. for interlaced or progressive display mode
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51—Motion estimation or motion compensation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51—Motion estimation or motion compensation
    • H04N19/56—Motion estimation with initialisation of the vector search, e.g. estimating a good candidate to initiate a search

Definitions

  • the present invention relates to a motion estimation and compensation device, and more particularly to a motion estimation and compensation device that estimates motion vectors and performs motion-compensated prediction of an interlaced sequence of chrominance-subsampled video frames.
  • MPEG Moving Picture Experts Group
  • YCbCr color coding scheme
  • the YCbCr scheme allocates a greater bandwidth to luminance information than to chrominance information. In other words, people would notice image degradation in brightness, but they are more tolerant about color degradation.
  • a video coding device can blur away the chromatic information when encoding pictures, without fear of being detected by the human eyes. The process of such color information reduction is called subsampling.
  • FIG. 50 shows 4:2:2 color sampling format.
  • a consecutive run of four picture elements called "pels” or "pixels”
  • the 4:2:2 format only allows Cb and Cr to be placed every two pixels while giving Y to every individual pixel, whereas the original signal contains all of Y, Cb, and Cr in every pixel.
  • two Y samples share a single set of Cb and Cr samples.
  • the average amount of information contained in a 4:2:2 color signal is only 16 bits per pixel (i.e., Y(8)+Cb(8) or Y(8)+Cr(8)), whereas the original signal has 24 bits per pixel. That is, the signal contains chrominance information of one-half the luminance information.
  • FIG. 51 shows 4:2:0 color sampling format.
  • the chrominance components of a picture is subsampled not only in the horizontal direction, but also in the vertical direction by a factor of 2, while the original luminance components are kept intact. That is, the 4:2:0 format assigns one pair of Cb and Cr to a box of four pixels. Accordingly, the average amount of information contained in a color signal is only 12 bits per pixel (i.e., ⁇ Y(8) ⁇ 4+Cb(8)+Cr(8) ⁇ /4). This means that chrominance information contained in a 4:2:0 picture is one quarter of luminance information.
  • the 4:2:2 format is stipulated as ITU-R Recommendation BT.601-5 for studio encoding of digital television signals.
  • Typical video coding equipment accepts 4:2:2 video frames as an input format. The frames are then converted into 4:2:0 format to comply with the MPEG-2 Main Profile. The resulting 4:2:0 signal is then subjected to a series of digital vide coding techniques, including motion vector search, motion-compensated prediction, discrete cosine transform (DCT), and the like.
  • DCT discrete cosine transform
  • the video coder searches given pictures to find a motion vector for each square segment, called macroblock, with a size of 16 pixels by 16 lines. This is achieved by block matching between an incoming original picture (i.e., present frame to be encoded) and a selected reference picture (i.e., frame being searched). More specifically, the coder compares a macroblock in the original picture with a predefined search window in the reference frame in an attempt to find a block in the search window that gives a smallest sum of absolute differences of their elements. If such a best matching block is found in the search window, then the video coder calculates a motion vector representing the displacement of the present macroblock with respect to the position of the best matching block. Based on this motion vector, the coder creates a predicted picture corresponding to the original macroblock.
  • FIG. 52 schematically shows a process of finding a motion vector. Illustrated are: present frame Fr2 as an original picture to be predicted, and previous frame Fr1 as a reference picture to be searched.
  • the present frame Fr2 contains a macroblock mb2 (target macroblock).
  • Block matching against this target macroblock mb2 yields a similar block mb1-1 in the previous frame Fr1, along with a motion vector V representing its horizontal and vertical displacement.
  • the pixels of this block mb1-1 shifted with the calculated motion vector V are used as predicted values of the target macroblock mb2.
  • the block matching process first compares the target macroblock mb2 with a corresponding block mb1 indicated by the broken-line box mb1 in FIG. 52 . If they do not match well with each other, the search algorithm then tries to find a block with a similar picture pattern in the neighborhood of mb1. For each candidate block in the reference picture, the sum of absolute differences is calculated as a cost function to evaluate the average difference between two blocks. One of such candidate blocks that minimizes this metric is regarded as a best match. In the present example, the block matching process finds a block mb1-1 as giving a minimum absolute error with respect to the target macroblock mb2 of interest, thus estimating a motion vector V as depicted in FIG. 52 .
  • FIG. 53 schematically shows how video images are coded with a motion-compensated prediction technique.
  • a motion vector V is found in a reference picture Fr
  • the best matching block mb1-1 in this picture Fr1 is shifted in the direction of, and by the length of the motion vector V, thus creating a predicted picture Pr2 containing a shifted version of the block mb1-1.
  • the coder then compares this predicted picture Pr2 with the present picture Fr2, thus producing a difference picture Er2 representing the prediction error. This process is called a motion-compensated prediction.
  • the example pictures of FIG. 52 show a distant view of an aircraft descending for landing. Since a parallel motion of a rigid-body object like this example does not change the object's appearance in the video, the motion vector V permits an exact prediction, meaning that there will be no difference between the original picture and the shifted picture.
  • the coded data in this case will only be a combination of horizontal and vertical components of the motion vector and a piece of information indicating that there are no prediction errors.
  • the moving object is, for example, a flying bird
  • the moving object is, for example, a flying bird
  • DCT coding to this prediction error, thus yielding non-zero transform coefficients.
  • Coded data is produced through the subsequent steps of quantization and variable-length coding.
  • the present invention provides a motion estimation and compensation device for estimating motion vectors and performing motion-compensated prediction.
  • This motion estimation and compensation device has a motion vector estimator and a motion compensator.
  • the motion vector estimator estimates motion vectors representing motion in given interlace-scanning chrominance-subsampled video signals. The estimation is accomplished by comparing each candidate block in a reference picture with a target block in an original picture by using a sum of absolute differences (SAD) in luminance as similarity metric, choosing a best matching candidate block that minimizes the SAD, and determining displacement of the best matching candidate block relative to the target block.
  • the motion vector estimator gives the SAD of each candidate block an offset determined from the vertical component of a candidate motion vector associated with that candidate block. With this motion vector correction, the estimated motion vectors are less likely to cause discrepancies in chrominance components.
  • the motion compensator produces a predicted picture using such motion vectors and calculates prediction error by subtracting the predicted picture from the original picture.
  • Digital TV broadcasting and other ordinary video applications use interlace scanning and 4:2:0 format to represent color information.
  • Original pictures are compressed and encoded using techniques such as motion vector search, motion-compensation, and discrete cosine transform (DCT) coding.
  • Interlacing is a process of scanning a picture by alternate horizontal lines, i.e., odd-numbered lines and even-numbered lines. In this mode, each video frame is divided into two fields called top and bottom fields.
  • the 4:2:0 color sampling process subsamples chromatic information in both the horizontal and vertical directions.
  • conventional motion vector estimation could cause a quality degradation in chrominance components of motion-containing frames because the detection is based only on the luminance information of those frames.
  • motionless or almost motionless pictures can be predicted with correct colors even if the motion vectors are calculated solely from luminance components, there is an increased possibility of mismatch between a block in the original picture and its corresponding block in the reference picture in their chrominance components if the video frames contain images of a moving object.
  • Such a chrominance discrepancy would raise the level of prediction errors, thus resulting in an increased amount of coded video data, or an increased picture degradation in the case of a bandwidth-limited system.
  • FIG. 1 is a conceptual view of a motion estimation and compensation device according to a first embodiment of the present invention.
  • This motion estimation and compensation device 10 comprises a motion vector estimator 11 and a motion compensator 12.
  • the motion vector estimator 11 finds a motion vector in luminance components of an interlaced sequence of chrominance-subsampled video signals structured in 4:2:0 format by evaluating a sum of absolute differences (SAD) between a target block in an original picture and each candidate block in a reference picture. To suppress the effect of possible chrominance discrepancies in this process, the motion vector estimator 11 performs a motion vector correction that adds different offsets to the SAD values being evaluated, depending on the value that the vertical component of a motion vector can take.
  • the term "block” refers to a macroblock, or a square segment of a picture, with a size of 16 pixels by 16 lines.
  • the motion vector estimator 11 identifies one candidate block in the reference picture that shows a minimum SAD and calculates a motion vector representing the displacement of the target block with respect to the candidate block that is found.
  • the vertical component of a motion vector has a value of 4n+0, 4n+1, 4n+2, or 4n+3, where n is an integer.
  • Those values correspond to four candidate blocks B0, B1, B2, and B3, which are compared with a given target block B in the original picture in terms of SAD between their pixels.
  • the motion vector estimator 11 gives an offset of zero to the SAD between the target block B and the candidate block B0 located at a vertical distance of 4n+0.
  • the motion vector estimator 11 gives offset values that are determined adaptively.
  • the term "adaptively" means here that the motion vector estimator 11 determines offset values in consideration of at least one of transmission bitrate, quantization parameters, chrominance edge information, and prediction error of chrominance components.
  • the quantization parameters include quantization step size, i.e., the resolution of quantized values. Details of this adaptive setting will be described later.
  • FIGS. 2 and 3 show a reference picture and an original picture which contain a rectangular object moving in the direction from upper left to lower right. Specifically, FIG. 2 shows two-dimensional images of a top and bottom fields constituting a single reference picture, and FIG. 3 shows the same for an original picture. Note that both pictures represent only the luminance components of sampled video signals. Since top and bottom fields have opposite parities (i.e., one made up of the even-numbered lines, the other made up of odd-numbered lines), FIGS. 2 and 3 , as well as several subsequent drawings, depict them with an offset of one line.
  • FIG. 2 Compare the reference picture of FIG. 2 with the original picture of FIG. 3 , where the black boxes (pixels) indicate an apparent motion of the object in the direction from upper left to lower right. It should also be noticed that, even within the same reference picture of FIG. 2 , an object motion equivalent to two pixels in the horizontal direction is observed between the top field and bottom field. Likewise, FIG. 3 shows a similar horizontal motion of the object during one field period.
  • FIGS. 4 and 5 show the relationships between 4:2:2 format and 4:2:0 format in the reference picture and original picture of FIGS. 2 and 3 . More specifically, FIG. 4 contrasts 4:2:2 and 4:2:0 pictures representing the same reference picture of FIG. 2 , with a focus on the pixels at a particular horizontal position x1 indicated by the broken lines in FIG. 2 . FIG. 5 compares, in the same manner, 4:2:2 and 4:2:0 pictures corresponding to the original picture of FIG. 3 , focussing on the pixels at another horizontal position x2 indicated by the broken lines in FIG. 3 .
  • FIGS. 4 and 5 The notation used in FIGS. 4 and 5 are as follows: White and black squares represent luminance components, and white and black triangles chrominance components, where white and black indicate the absence and presence of an object image, respectively.
  • the numbers seen at the left end are line numbers. Even-numbered scan lines are represented by broken lines, and each two-line vertical interval is subdivided into eight sections, which are referred to by the fractions "1/8," "2/8,” “3/8,” and so on.
  • the process of converting video sampling formats from 4:2:2 to 4:2:0 actually involves chrominance subsampling operations.
  • the first top-field chrominance component a3 in the 4:2:0 picture is interpolated from chrominance components a1 and a2 in the original 4:2:2 picture. That is, the value of a3 is calculated as a weighted average of the two nearest chrominance components a1 and a2, which is actually (6 ⁇ a1 + 2 ⁇ a2)/8 since a3 is located "2/8" below a1, and "6/8" above a2.
  • the chrominance component a3 is represented as a gray triangle, since it is a component interpolated from a white triangle and a black triangle.
  • the first bottom-field chrominance component b3 in the 4:2:0 reference picture is interpolated from 4:2:2 components b1 and b2 in the same way. Since b3 is located "6/8" below b1, and "2/8" above b2, the chrominance component b3 has a value of (2 ⁇ b1 + 6 ⁇ b2)/8, the weighted average of its nearest chrominance components b1 and b2 in the original 4:2:2 picture. The resulting chrominance component a3 is represented as a white triangle since its source components are both white triangles.
  • Original pictures shown in FIG. 5 are also subjected to a similar process of format conversion and color subsampling.
  • FIGS. 4 and 5 only show a simplified version of color subsampling, actual implementations use more than two components in the neighborhood to calculate a new component, the number depending on the specifications of each coding device.
  • the aforementioned top-field chrominance component a3, for example, may actually be calculated not only from a1 and a2, but also from other surrounding chrominance components. The same is applied to bottom-field chrominance components such as b3.
  • FIGS. 6 to 9 the moving rectangular object discussed in FIGS. 2 to 5 is now drawn in separate luminance and chrominance pictures in 4:2:0 format. More specifically, FIGS. 6 and 7 show luminance components and chrominance components, respectively, of a 4:2:0 reference picture, while FIGS. 8 and 9 show luminance components and chrominance components, respectively, of a 4:2:0 original picture. All frames are divided into top and bottom fields since the video signal is interlaced.
  • the 4:2:0 format provides only one color component for every four luminance components in a block of two horizontal pixels by two vertical pixels.
  • four pixels Y1 to Y4 in the top luminance field ( FIG. 6 ) are supposed to share one chrominance component CbCr (which is actually a pair of color-differences Cb and Cr representing one particular color). Since it corresponds to "white" pixels Y1 and Y2 and "black” pixels Y3 and Y4, CbCr is depicted as a "gray" box in FIG. 7 for explanatory purposes.
  • Area R1 on the left-hand side of FIG. 8 indicates the location of the black rectangle (i.e., moving object) seen in the corresponding top-field reference picture of FIG. 6 .
  • area R2 on the right-hand side of FIG. 8 indicates the location of the black rectangle seen in the corresponding bottom-field reference picture of FIG. 6 .
  • the two arrows are motion vectors in the top and bottom fields. Note that those motion vectors are identical (i.e., the same length and same orientation) in this particular case, and therefore, the present frame prediction yields a motion vector consisting of horizontal and vertical components of +2 pixels and +2 lines, respectively.
  • FIGS. 10 and 11 show motion vectors found in the 4:2:0 reference and original pictures explained. More specifically, FIG. 10 gives luminance motion vectors (called “luminance vectors,” where appropriate) that indicate pixel-to-pixel associations with respect to horizontal positions x1 of the reference picture ( FIG. 6 ) and x2 of the original picture ( FIG. 8 ). In the same way, FIG. 11 gives chrominance motion vectors (or “chrominance vectors,” where appropriate) that indicate pixel-to-pixel associations with respect to horizontal positions x1 of the reference picture ( FIG. 7 ) and x2 of the original picture ( FIG. 9 ).
  • FIGS. 10 and 11 The notation used in FIGS. 10 and 11 are as follows: White squares and white triangles represent luminance and chrominance components, respectively, in such pixels where no object is present. Black squares and black triangles represent luminance and chrominance components, respectively, in such pixels where the moving rectangular object is present. That is, "white” and “black” symbolize the value of each pixel.
  • Va be a luminance vector obtained in the luminance picture of FIG. 8 .
  • the luminance vector Va has a vertical component of +2 lines, and the value of each pixel of the reference picture coincides with that of a corresponding pixel located at a distance of two lines in the original picture.
  • Located two lines down from this pixel y1a is a pixel ylb, to which the arrow of motion vector Va is pointing.
  • every original picture element has a counterpart in the reference picture, and vice versa, no matter what motion vector is calculated. This is because luminance components are not subsampled.
  • Chrominance components have been subsampled during the process of converting formats from 4:2:2 to 4:2:0. For this reason, the motion vector calculated from non-subsampled luminance components alone would not work well with chrominance components of pictures.
  • the motion vector Va is unable to directly associate chrominance components of a reference picture with those of an original picture. Take a chrominance component c1 in the top-field original picture, for example. As its symbol (black triangle) implies, this component c1 is part of a moving image of the rectangular object, and according to the motion vector Va, its corresponding chrominance component in the top-field reference picture has to be found at c2.
  • the motion vector Va suggests that c2 would be the best estimate of c1, but c2 does not exist.
  • the conventional method then uses neighboring c3 as an alternative to c2, although it is in a different field. This replacement causes c1 to be predicted by c3, whose chrominance value is far different from c1 since c1 is part of the moving object image, whereas c3 is not. Such a severe mismatch between original pixels and their estimates leads to a large prediction error.
  • a chrominance component c4 at line #3 of the bottom-field original picture is a chrominance component c4 at line #3 of the bottom-field original picture. While a best estimate of c4 would be located at c5 in the bottom-field reference picture, but there is no chrominance component at that pixel position. Even though c4 is not part of the moving object image, c6 at line #2 of the top-field picture is chosen as an estimate of c4 for use in motion compensation. Since this chrominance component c6 is part of the moving object image, the predicted picture will have a large error.
  • video coding devices estimate motion vectors solely from luminance components of given pictures, and the same set of motion vectors are applied also to prediction of chrominance components.
  • the chrominance components have been subsampled in the preceding 4:2:2 to 4:2:0 format conversion, and in such situations, the use of luminance-based motion vectors leads to incorrect reference to chrominance components in motion-compensated prediction.
  • the motion compensator uses a bottom-field reference picture, when it really needs to use a top-field reference picture.
  • the motion compensator uses a top-field reference picture, when it really needs to use a bottom-field reference picture.
  • Such chrominance discrepancies confuse the process of motion compensation and thus causes additional prediction errors. The consequence is an increased amount of coded data and degradation of picture quality.
  • FIGS. 12A to 16B show several different patterns of luminance motion vectors, assuming different amounts of movement that the aforementioned rectangular object would make.
  • the rectangular object has moved purely in the horizontal direction, and thus the resulting motion vector V0 has no vertical component.
  • the object has moved a distance of four lines in the vertical direction, resulting in a motion vector V4 with a vertical component of +4.
  • the luminance vectors V0 and V4 can work as chrominance vectors without problem.
  • the object has moved vertically a distance of one line, and the resulting motion vector V1 has a vertical component of +1.
  • This luminance vector V1 is unable to serve as a chrominance vector. Since no chrominance components reside in the pixels specified by the motion vector V1, the chrominance of each such pixel is calculated by half-pel interpolation. Take a chrominance component d1, for example. Since the luminance vector V1 fails to designate an existing chrominance component in the reference picture, a new component has to be calculated as a weighted average of neighboring chrominance components d2 and d3. Another example is a chrominance component d4. Since the reference pixel that is supposed to provide an estimate of d4 contains no chrominance component, a new component has to be interpolated from neighboring components d3 and d5.
  • the object has moved vertically a distance of two lines, resulting in a motion vector V2 with a vertical component of +2.
  • This condition produces the same situation as what has been discussed above in FIGS. 10 and 11 .
  • the coder would mistakenly estimate pixels outside the object edge with values of inside pixels.
  • the object has moved vertically a distance of three lines, resulting in a motion vector V3 with a vertical component of +3.
  • This condition produces the same situation as what has been discussed in FIGS. 13A and 13B . That is, no chrominance components reside in the pixels specified by the motion vector V3.
  • Half-pel interpolation is required to produce a predicted picture. Take a chrominance component e1, for example. Since the luminance vector V3 fails to designate an existing chrominance component in the reference picture, a new component has to be calculated as a weighted average of neighboring chrominance components e2 and e3. Another similar example is a chrominance component e4. Since the reference pixel that is supposed to provide an estimate of e1 has no assigned chrominance component, a new component has to be interpolated from neighboring components e3 and e5.
  • the Japanese Patent Application Publication No. 2001-238228 discloses a technique of reducing prediction error by simply rejecting motion vectors with a vertical component of 4n+2. This technique, however, does not help the case of 4n+1 or 4n+3. For better quality of coded pictures, it is therefore necessary to devise a more comprehensive method that copes with all different patterns of vertical motions.
  • a more desirable approach is to deal with candidate vectors having vertical components of 4n+1, 4n+2, and 4n+3 in a more flexible way to suppress the increase of prediction error, rather than simply discarding motion vectors of 4n+2.
  • the present invention thus provides a new motion estimation and compensation device, as well as a video coding device using the same, that can avoid the problem of chrominance discrepancies effectively, without increasing too much the circuit size or processing load.
  • This section provides more details about the motion estimation and compensation device 10 according to a first embodiment of the invention, and particularly about the operation of its motion vector estimator 11.
  • FIG. 17 shows an offset table. This table defines how much offset is to be added to the SAD of candidate blocks, for several different patterns of motion vector components. Specifically, the motion vector estimator 11 gives no particular offset when the vertical component of a motion vector is 4n+0, since no chrominance discrepancy occurs in this case. When the motion vector has a vertical component is 4n+1, 4n+2, or 4n+3, there will be a risk of chrominance discrepancies. Since the severity in the case of 4n+2 is supposed to be much larger than the other two cases, the offset table of FIG. 17 assigns a special offset value OfsB to 4n+2 and a common offset value OfsA to 4n+1 and 4n+3.
  • the motion vector estimator 11 determines those offset values OfsA and OfsB in an adaptive manner, taking into consideration the following factors: transmission bitrates, quantization parameters, chrominance edge condition, and prediction error of chrominance components.
  • the values of OfsA and OfsB are to be adjusted basically in accordance with quantization parameters, or optionally considering transmission bitrates and picture color condition.
  • FIGS. 18A to 19B show how to determine an offset from transmission bitrates or chrominance edge condition. Those diagrams illustrate such situations where the motion vector estimator 11 is searching a reference picture to find a block that gives a best estimate for a target macroblock M1 in a given original picture.
  • candidate blocks M1a and M1b in a reference picture have mean absolute difference (MAD) values of 11 and 10, respectively, with respect to a target macroblock M1 in an original picture.
  • Mean absolute difference (MAD) is equivalent to an SAD divided by the number of pixels in a block, which is 256 in the present example.
  • M1a is located at a vertical distance of 4n+0, and M1b at a vertical distance of 4n+1, both relative to the target macrablock M1.
  • Either of the two candidate blocks M1a and M1b is to be selected as a predicted block of the target macroblock M1, depending on which one has a smaller SAD with respect to M1.
  • a sharp chrominance edge if present, would cause a chrominance discrepancy, and a consequent prediction error could end up with a distorted picture due to the effect of quantization.
  • the motion vector estimator 11 gives an appropriate offset OfsA so that M1a at 4n+0 will be more likely to be chosen as a predicted block even if the SAD between M1 and Mb1 is somewhat smaller than that between M1 and M1a.
  • the first candidate block M1a at 4n+0 is selected as a predicted block, in spite of the fact that SAD of M1b is actually smaller than that of H1a, before they are biased by the offsets.
  • This result is attributed to offset OfsA, which has been added to SAD of M1b beforehand in order to increase the probability of selecting the other block M1a.
  • Blocks at 4n+0 are generally preferable to blocks at 4n+1 under circumstances where the transmission bitrate is low, and where the pictures being coded have a sharp change in chrominance components.
  • the difference between a good candidate block at 4n+0 and an even better block at 4n+1 (or 4n+3) is no more than one in terms of their mean absolute difference values, choosing the second best block would impose no significant degradation in the quality of luminance components.
  • the motion vector estimator 11 therefore sets an offset OfsA so as to choose that block at 4n+0, rather than the best block at 4n+1, which could suffer a chrominance discrepancy.
  • FIGS. 19A and 19B show a similar situation, in which a candidate macroblock M1a has an MAD value of 12, and another candidate block M1c has an MAD value of 10, both with respect to a target block M1 in an original picture.
  • M1a is located at a vertical distance of 4n+0, and M1c at a vertical distance of 4n+2, both relative to the target block M1.
  • Blocks at 4n+0 are generally preferable to blocks at 4n+2 under circumstances where the transmission bitrate is low, and the pictures being coded have a sharp change in chrominance components.
  • the difference between a good candidate block at 4n+0 and an even better block at 4n+2 is no more than two in terms of their mean pixel values, choosing the second best block at 4n+0 would impose no significant degradation in the quality of luminance components.
  • the motion vector estimator 11 therefore sets an offset OfsB so as to choose that block at 4n+0, rather than the best block at 4n+2, which could suffer a chrominance discrepancy.
  • High-bitrate environments unlike the above two examples, permit coded video data containing large prediction error to be delivered intact to the receiving end.
  • FIG. 20 shows an example program code for motion vector estimation, which assumes a video image size of 720 pixels by 480 lines used in ordinary TV broadcasting systems. Pictures are stored in a frame memory in 4:2:0 format, meaning that one frame contains 720x480 luminance samples and 360x240 chrominance samples.
  • Yo[y][x] be individual luminance components of an original picture
  • Yr[y][x] those of a reference picture
  • each such component takes a value in a range of 0 to 255.
  • Vx and Vy be the components of a motion vector found in frame prediction mode as having a minimum SAD value with respect to a particular macroblock at macroblock coordinates (Mx, My) in the given original picture.
  • Vx and Vy are obtained from, for example, a program shown in FIG. 20 , where Mx is 0 to 44, My is 0 to 29, and function abs(v) gives the absolute value of v.
  • the program code of FIG. 20 has the following steps:
  • FIGS. 21A and 21B show a process of searching for pixels in calculating an SAD.
  • Yo[My*16+y][Mx*16+x] represents a pixel in the original picture
  • Yr[Ry+y][Rx+x] a pixel in the reference picture.
  • Step S3 is what is added according to the present invention, while the other steps of the program are also found in conventional motion vector estimation processes.
  • the processing functions proposed in the present invention are realized as a program for setting a different offset depending on the vertical component of a candidate motion vector, along with a circuit designed to support that processing. With such a small additional circuit and program code, the present invention effectively avoids the problem of chrominance discrepancies, which may otherwise be encountered in the process of motion vector estimation.
  • FIGS. 22 to 35 we will discuss again the situation explained earlier in FIGS. 2 and 3 . That is, think of a sequence of video pictures on which a dark, rectangular object image is moving in the direction from top left to bottom right. Each frame of pictures is composed of a top field and a bottom field. It is assumed that the luminance values are 200 for the background and 150 for the object image, in both reference and original pictures. The following will present various patterns of motion vector components and resulting difference pictures.
  • the term "difference picture” refers to a picture representing differences between a given original picture and a predicted picture created by moving pixels in accordance with estimated motion vectors.
  • FIG. 22 shows a reference and original pictures when the motion vector has a vertical component of 4n+2, and FIG. 23 shows a resulting difference picture.
  • FIG. 24 shows a reference and original pictures when the motion vector has a vertical component of 4n+1, and FIG. 25 shows a resulting difference picture.
  • FIG. 26 shows a reference and original pictures when the motion vector has a vertical component of 4n+0, and FIG. 27 shows a resulting difference picture.
  • FIG. 28 shows a reference picture and an original picture when the motion vector has a vertical component of 4n+3, and FIG. 29 shows a resulting difference picture. All those pictures are shown in an interlaced format, i.e., as a combination of a top field and a bottom field.
  • the motion vector agrees with the object motion, which is +2. This allows shifted reference picture elements to coincide well with the original picture.
  • the difference picture of FIG. 23 thus shows nothing but zero-error components, and the resulting SAD value is also zero in this condition. The following cases, however, are not free from prediction errors.
  • FIG. 24 a motion vector with a vertical component of 4n+1 is illustrated.
  • the present invention enables the second best motion vector shown in FIG. 26 to be selected. That is, an offset OfsB of more than 600 makes it possible for the motion vector with a vertical component of 4n+0 ( FIG. 26 ) to be chosen, instead of the minimum-SAD motion vector with a vertical component of 4n+2.
  • FIG. 30 shows a reference picture and an original picture when the motion vector has a vertical component of 4n+1
  • FIG. 31 shows a resulting difference picture
  • FIG. 32 shows a reference picture and an original picture when the motion vector has a vertical component of 4n+0
  • FIG. 33 shows a resulting difference picture
  • FIG. 34 shows a reference picture and an original picture when the motion vector has a vertical component of 4n+2
  • FIG. 35 shows a resulting difference picture.
  • FIG. 30 a motion vector with a vertical component of 4n+1 is shown. Since this vector agrees with the actual object movement, its SAD value becomes zero as shown in FIG. 31 .
  • the SAD value is as high as 2500 in the case of 4n+0.
  • the SAD value is 2300 in the case of 4n+2.
  • the present invention enables the second best motion vector shown in FIG. 32 to be selected. That is, an offset OfsA of more than 2500 makes it possible for the motion vector with a vertical component of +0 ( FIG. 32 ) to be chosen, instead of the minimum-SAD motion vector with a vertical component of +1.
  • FIGS. 36 to 39 the following is yet another set of examples, which the rectangular object has non-uniform luminance patterns.
  • FIG. 36 shows a reference picture and an original picture when the motion vector has a vertical component of 4n+2
  • FIG. 37 shows a resulting difference picture.
  • FIG. 38 shows a reference picture and an original picture when the motion vector has a vertical component of 4n+0
  • FIG. 39 shows a resulting difference picture.
  • FIG. 36 involves a vertical object movement of +2, as in the foregoing example of FIG. 22 , but the rectangular object has non-uniform appearance. Specifically, it has a horizontally striped texture with two different luminance values, 40 and 160.
  • the motion vector with a vertical component of +2 yields a difference picture with no errors.
  • Motion vectors with vertical components of 4n+1, 4n+3, and 4n+2 are prone to produce chrominance discrepancies.
  • OfsA and OfsB are prone to produce chrominance discrepancies.
  • the present invention manages those discrepancy-prone motion vectors by setting adequate OfsA and OfsB to maintain the balance of penalties imposed on the luminance and chrominance.
  • While SAD offsets OfsA and OfsB may be set to appropriate fixed values that are determined from available bitrates or scene contents, the present invention also proposes to determine those offset values from prediction error of chrominance components in an adaptive manner as will be described in this section.
  • the motion compensator 12 has an additional function to calculate a sum of absolute differences in chrominance components.
  • This SAD value referred to by Cdiff, actually includes absolute differences in Cb and those in Cr, which the motion compensator 12 calculates in the course of subtracting a predicted picture from an original picture in the chrominance domain.
  • FIG. 40 shows a program for calculating Cdiff.
  • This program is given a set of difference pictures of chrominance, which are among the outcomes of motion-compensated prediction. Specifically diff_CB[][] and diff_CR[][] represent difference pictures of Cb and Cr, respectively. Note that three underlined statements are new steps added to calculate Cdiff, while the other part of the program of FIG. 40 has existed since its original version to calculate differences between a motion-compensated reference picture and an original picture.
  • the motion compensator 12 also calculates an SAD value of luminance components.
  • Vdiff represent this SAD value in a macroblock. While a macroblock contains 256 samples (16x16) of luminance components, the number of chrominance samples in the same block is only 64 (8x8) because of the 4:2:0 color sampling format. Since each chrominance sample consists of a Cb sample and a Cr sample, Cdiff contains the data of 128 samples of Cb and Cr, meaning that the magnitude of Cdiff is about one-half that of Vdiff.
  • Vdiff luminance SAD value
  • Cdiff corresponding chrominance SAD value
  • the above proposed method still carries a risk of producing too large OfsA or OfsB to allow vertical vector components of 4n+1, 4n+3, and 4n+2 to be taken, the actual implementation requires some appropriate mechanism to ensure the convergence of OfsA and OfsB by, for example, setting an upper limit for them.
  • Other options are to gradually reduce OfsA and OfsB as the process advances, or returning OfsA and OfsB to their initial values when a large scene change is encountered.
  • Vdiff k / Cdiff k m ⁇ Vdiff k / Cdiff k m
  • m the number of such macroblocks
  • k the identifier of a macroblock that satisfies Vdiff(k) ⁇ OfsA and Vdiff(k) ⁇ OfsB.
  • the conditions about Vdiff are to avoid the effect in the case where vectors are restricted to 4n+0 due to OfsA and OfsB.
  • This section describes a second embodiment of the present invention.
  • the first embodiment adds appropriate offsets, e.g., OfsA and OfsB, to SAD values corresponding to candidate motion vectors with a vertical component of 4n+1, 4n+3, or 4n+2, thus reducing the chance for those vectors to be picked up as a best match.
  • the second embodiment takes a different approach to solve the same problem. That is, the second embodiment avoids chrominance discrepancies by adaptively switching between frame prediction mode and field prediction mode, rather than biasing the SAD metric with offsets.
  • FIG. 41 shows a conceptual view of the second embodiment.
  • the illustrated motion detection and compensation device 20 has a motion vector estimator 21 and a motion compensator 22.
  • the motion vector estimator 21 estimates motion vectors using luminance components of an interlaced sequence of chrominance-subsampled video signals. The estimation is done in frame prediction mode, and the best matching motion vector found in this mode is referred to as the "frame vector.”
  • the motion vector estimator 21 selects an appropriate vector(s), depending on the vertical component of this frame vector.
  • the vertical component of the frame vector can take a value of 4n+0, 4n+1, 4n+2, or 4n+3 (n: integer).
  • the motion vector estimator 21 chooses that frame vector itself if its vertical component is 4n+0. In the case that the vertical component is 4n+1, 4n+2, and 4n+3, the motion vector estimator 21 switches its mode and searches again the reference picture for motion vectors in field prediction mode. The motion vectors found in this field prediction mode are called "field vectors." With the frame vectors or field vectors, whichever selected, the motion compensator 22 produces a predicted picture and calculates prediction error by subtracting the predicted picture from the original picture. In this way, the second embodiment avoids chrominance discrepancies by selecting either frame vectors or field vectors.
  • MPEG-2 coders can select either frame prediction or field prediction on a macroblock-by-macroblock basis for finding motion vectors. Normally, the frame prediction is used when top-field and bottom-field motion vectors tend to show a good agreement, and otherwise the field prediction is used.
  • the resulting motion vector data contains a horizontal and vertical components of a vector extending from a reference picture to an original picture.
  • the lower half of FIG. 41 shows a motion vector Vb in frame prediction mode, whose data consists of its horizontal and vertical components.
  • the motion estimation process yields two motion vectors for each frame, and thus the resulting data includes horizontal and vertical components of each vector and field selection bits that indicate which field is the reference field of that vector.
  • the lower half of FIG. 41 shows two example field vectors Vc and Vd.
  • Data of Vc includes its horizontal and vertical components and a field selection bit indicating "top field" as a reference field.
  • Data of Vd includes its horizontal and vertical components and a field selection bit indicating "bottom field” as a reference field.
  • the present embodiment enables field prediction mode when the obtained frame vector has a vertical component of either 4n+1, 4n+2, or 4n+3, and by doing so, it avoids the problem of chrominance discrepancies. The following will provide details of why this is possible.
  • FIG. 42 shows how to avoid the chrominance discrepancy problem in field prediction.
  • a discrepancy in chrominance components is produced when a frame vector is of 4n+2 and thus, for example, a chrominance component c1 of the top-field original picture is supposed to be predicted by a chrominance component at pixel c2 in the top-field reference picture. Since there exists no corresponding chrominance component at that pixel c2, the motion compensator uses another chrominance component c3, which is in the bottom field of the same reference picture (this is what happens in frame prediction mode). The result is a large discrepancy between the original chrominance component c1 and corresponding reference chrominance component c3.
  • the motion compensator operating in field prediction will choose a closest pixel c6 in the same field even if no chrominance component is found in the referenced pixel c2. That is, in field prediction mode, the field selection bit of each motion vector permits the motion compensator to identify which field is selected as a reference field. When, for example, a corresponding top-field chrominance component is missing, the motion compensator 22 can choose an alternative pixel from among those in the same field, without the risk of producing a large error. This is unlike the frame prediction, which could introduce a large error when it mistakenly selects a bottom-field pixel as a closest alternative pixel.
  • the second embodiment first scans luminance components in frame prediction mode, and if the best vector has a vertical component of 4n+2, 4n+1, or 4n+3, it changes its mode from frame prediction to field prediction to avoid a risk of chrominance discrepancies.
  • Field prediction produces a greater amount of vector data to describe a motion than frame prediction does, thus increasing the overhead of vector data in a coded video stream.
  • the present embodiment employs a chrominance edge detector which detects a chrominance edge in each macroblock, so that the field prediction mode will be enabled only when a chrominance discrepancy is likely to cause a significant effect on the prediction efficiency.
  • chrominance edge a high contrast portion in a picture.
  • chrominance edges have nothing to do with luminance components.
  • FIG. 43 is a table showing the relationship between vertical components of a frame vector and those of field vectors.
  • the motion vector estimator 21 first finds a motion vector in frame prediction mode. If its vertical component is either of 4n+2, 4n+1, and 4n+3, and if the chrominance edge detector indicates the presence of a chrominance edge, the motion vector estimator 21 switches itself to field prediction mode, thus estimating field vectors as shown in the table of FIG. 43 .
  • FIG. 44 shows field vectors when the frame vector has a vertical component of 4n+2.
  • the motion vector estimator 21 in field prediction mode produces the following two field vectors in the luminance domain.
  • top-field motion vector points from the top-field reference picture to the top-field original picture, has a vertical component of 2n+1, and is accompanied by a field selection bit indicating "top field.”
  • bottom-field motion vector points from the bottom-field reference picture to the bottom-field original picture, has a vertical component of 2n+1, and is accompanied by a field selection bit indicating "bottom field.”
  • the above (2n+1) vertical component of vectors in the luminance domain translates into a half-sized vertical component of (n+0.5) in the chrominance domain.
  • the intermediate chrominance component corresponding to the half-pel portion of this vector component is predicted by interpolation (or averaging) of two neighboring pixels in the relevant reference field.
  • the estimates of chrominance components f1 and f2 are (Ct(n)+Ct(n+1))/2 and (Cb(n)+Cb(n+1))/2, respectively.
  • the half-pel interpolation performed in field prediction mode has some error
  • the amount of this error is smaller than that in frame prediction mode, which is equivalent to the error introduced by a half-pel interpolation in the case of 4n+1 or 4n+3 (in the first embodiment described earlier).
  • the reason for this difference is as follows: In field prediction mode, the half-pel interpolation takes place in the same picture field; i.e., it calculates an intermediate point from two pixels both residing in either top field or bottom field. In contrast, the half-pel interpolation in frame prediction mode calculates an intermediate point from one in the top field and the other in the bottom field (see FIGS. 13A and 13B ).
  • FIG. 45 shows field vectors when the frame vector has a vertical component of 4n+1.
  • the motion vector estimator 21 in field prediction mode produces the following two field vectors in the luminance domain.
  • One field vector (or top-field motion vector) points from the bottom-field reference picture to the top-field original picture, has a vertical component of 2n, and is accompanied by a field selection bit indicating "bottom field.”
  • the other field vector (or bottom-field motion vector) points from the top-field reference picture to the bottom-field original picture, has a vertical component of 2n+1, and is accompanied by a field selection bit indicating "top field.”
  • FIG. 46 shows field vectors when the frame vector has a vertical component of 4n+3.
  • the motion vector estimator 21 in field prediction mode produces the following two field vectors in the luminance domain.
  • One field vector (or time point motion vector) points from the bottom-field reference picture to the top-field original picture, has a vertical component of 2n+2, and is accompanied by a field selection bit indicating "bottom field.”
  • the other field vector (or bottom-field motion vector) points from the top-field reference picture to the bottom-field original picture, has a vertical component of 2n+1, and is accompanied by a field selection bit indicating "top field.”
  • FIG. 47 shows a process of 2:3 pullup and 3:2 pulldown.
  • a motion picture camera captures images at 24 frames per second. Frame rate conversion is therefore required to play a 24-fps motion picture on 30-fps television system. This is known as "2:3 pullup" or "telecine conversion.”
  • Frame A is converted to three pictures: top field A T , bottom field A B , and top field A T .
  • Frame B is then divided into bottom field B B and top field B T .
  • Frame C is converted to bottom field C B , top field C T , and bottom field C B .
  • Frame D is divided into top field D T and bottom field D B .
  • four 24-fps frames with a duration of one-sixth second ((1/24) ⁇ 4) are converted to ten 60-fps fields with a duration of one-sixth second ((1/60) ⁇ 10).
  • This section describes video coding devices employing a motion estimation and compensation device according to the present invention for use with MPEG-2 or other standard video compression system.
  • FIG. 48 shows a structure of a video coding device employing a motion estimation and compensation device 10 according to the first embodiment of the present invention.
  • the illustrated video coding device 30-1 has the following components: an A/D converter 31, an input picture converter 32, a motion estimator/compensator 10a, a coder 33, a local decoder 34, a frame memory 35, and a system controller 36.
  • the coder 33 is formed from a DCT unit 33a, a quantizer 33b, and a variable-length coder 33c.
  • the local decoder 34 has a dequantizer 34a and an inverse DCT (IDCT) unit 34b.
  • IDCT inverse DCT
  • the A/D converter 31 converts a given analog video signal of TV broadcasting or the like into a digital data stream, with the luminance and chrominance components sampled in 4:2:2 format.
  • the input picture converter 32 converts this 4:2:2 video signal into 4:2:0 form.
  • the resulting 4:2:0 video signal is stored in the frame memory 35.
  • the system controller 36 manages frame images in the frame memory 35, controls interactions between the components in the video coding device 30-1, and performs other miscellaneous tasks.
  • the motion estimator/compensator 10a provides what have been described as the first embodiment.
  • the motion vector estimator 11 reads each macroblock of an original picture from the frame memory 35, as well as a larger region of a reference picture from the same, so as to find a best matching reference block that minimizes the sum of absolute differences of pixels with respect to the given original macroblock, while giving some amount of offset to.
  • the motion vector estimator 11 then calculates the distance between the best matching reference block and the original macroblock of interest, thus obtaining a motion vector.
  • the motion compensator 12 also makes access to the frame memory 35 to retrieve video signals and create therefrom a predicted picture by using the detected motion vectors and subtracting corresponding reference images from the original picture. The resulting prediction error is sent out to the DCT unit 33a.
  • the DCT unit 33a performs DCT transform to convert the prediction error to a set of transform coefficients.
  • the quantizer 33b quantizes the transform coefficients according to quantization parameters specified by the system controller 36.
  • the results are supplied to the dequantizer 34a and variable-length coder 33c.
  • the variable-length coder 33c compresses the quantized transform coefficients with Huffman coding algorithms, thus producing coded data.
  • the dequantizer 34a dequantizes the quantized transform coefficients according to the quantization parameters and supplies the result to the subsequent IDCT unit 34b.
  • the IDCT unit 34b reproduces the prediction error signal through an inverse DCT process. By adding the reproduced prediction error signal to the predicted picture, the motion compensator 12 produces a locally decoded picture and saves it in the frame memory 35 for use as a reference picture in the next coding cycle.
  • FIG. 49 shows a structure of a video coding device employing a motion estimation and compensation device 20 according to the second embodiment of the present invention.
  • the illustrated video coding device 30-2 has basically the same structure as the video coding device 30-1 explained in FIG. 48 , except for its motion estimator/compensator 20a and chrominance edge detector 37.
  • the motion estimator/compensator 20a provides the functions of the second embodiment of the invention.
  • the chrominance edge detector 37 is a new component that detects a chrominance edge in a macroblock when the motion estimator/compensator 20a needs to determine whether to select frame prediction mode or field prediction mode to find motion vectors.
  • the chrominance edge detector 37 examines the video signal supplied from the input picture converter 32 to find a chrominance edge in each macroblock and stores the result in the frame memory 35.
  • the motion vector estimator 21 estimates motion vectors from the original picture, reference picture, and chrominance edge condition read out of the frame memory 35. For further details, see the first half of this section.
  • the present invention circumvents the problem of discrepancies in chrominance components without increasing the circuit size or processing load.
  • the first embodiment adds appropriate offsets to SAD values corresponding to candidate blocks in a reference picture before choosing a best matching block with a minimum SAD value to calculate a motion vector.
  • This approach only requires a small circuit to be added to existing motion vector estimation circuits.
  • the second embodiment provides a chrominance edge detector to detect a sharp color contrast in a picture, which is used determine to whether a chrominance discrepancy would actually lead to an increased prediction error.
  • the second embodiment switches from frame prediction mode to field prediction mode only when the chrominance edge detector suggests to do so; otherwise, no special motion vector correction takes place. In this way, the second embodiment minimizes the increase in the amount of coded video data.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)
  • Color Television Systems (AREA)
  • Image Analysis (AREA)
EP20080167885 2004-07-27 2004-12-16 Estimation et compensation de mouvement avec correction de vecteur de mouvement basée sur des valeurs de composantes verticales Withdrawn EP2026584A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2004219083A JP4145275B2 (ja) 2004-07-27 2004-07-27 動きベクトル検出・補償装置
EP20040257850 EP1622387B1 (fr) 2004-07-27 2004-12-16 Estimation et compensation de mouvement avec correction de vecteur de mouvement basée sur des valeurs de composantes verticales

Related Parent Applications (1)

Application Number Title Priority Date Filing Date
EP20040257850 Division EP1622387B1 (fr) 2004-07-27 2004-12-16 Estimation et compensation de mouvement avec correction de vecteur de mouvement basée sur des valeurs de composantes verticales

Publications (1)

Publication Number Publication Date
EP2026584A1 true EP2026584A1 (fr) 2009-02-18

Family

ID=34930928

Family Applications (2)

Application Number Title Priority Date Filing Date
EP20040257850 Expired - Fee Related EP1622387B1 (fr) 2004-07-27 2004-12-16 Estimation et compensation de mouvement avec correction de vecteur de mouvement basée sur des valeurs de composantes verticales
EP20080167885 Withdrawn EP2026584A1 (fr) 2004-07-27 2004-12-16 Estimation et compensation de mouvement avec correction de vecteur de mouvement basée sur des valeurs de composantes verticales

Family Applications Before (1)

Application Number Title Priority Date Filing Date
EP20040257850 Expired - Fee Related EP1622387B1 (fr) 2004-07-27 2004-12-16 Estimation et compensation de mouvement avec correction de vecteur de mouvement basée sur des valeurs de composantes verticales

Country Status (6)

Country Link
US (1) US20060023788A1 (fr)
EP (2) EP1622387B1 (fr)
JP (1) JP4145275B2 (fr)
KR (1) KR100649463B1 (fr)
CN (1) CN100546395C (fr)
DE (1) DE602004022280D1 (fr)

Families Citing this family (53)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7506267B2 (en) * 2003-12-23 2009-03-17 Intel Corporation Compose rate reduction for displays
US20060120612A1 (en) * 2004-12-08 2006-06-08 Sharath Manjunath Motion estimation techniques for video encoding
US8780957B2 (en) 2005-01-14 2014-07-15 Qualcomm Incorporated Optimal weights for MMSE space-time equalizer of multicode CDMA system
WO2006099082A2 (fr) * 2005-03-10 2006-09-21 Qualcomm Incorporated Classification de contenus pour traitement multimedia
US10555775B2 (en) 2005-05-16 2020-02-11 Intuitive Surgical Operations, Inc. Methods and system for performing 3-D tool tracking by fusion of sensor and/or camera derived data during minimally invasive robotic surgery
JP2007028393A (ja) * 2005-07-20 2007-02-01 Matsushita Electric Ind Co Ltd 双方向予測符号化ピクチャの直接モードブロックの動きベクトルを求める方法
US8761259B2 (en) * 2005-09-22 2014-06-24 Qualcomm Incorporated Multi-dimensional neighboring block prediction for video encoding
US8879857B2 (en) 2005-09-27 2014-11-04 Qualcomm Incorporated Redundant data encoding methods and device
US8654848B2 (en) * 2005-10-17 2014-02-18 Qualcomm Incorporated Method and apparatus for shot detection in video streaming
US8948260B2 (en) * 2005-10-17 2015-02-03 Qualcomm Incorporated Adaptive GOP structure in video streaming
US20070206117A1 (en) * 2005-10-17 2007-09-06 Qualcomm Incorporated Motion and apparatus for spatio-temporal deinterlacing aided by motion compensation for field-based video
US20070171280A1 (en) * 2005-10-24 2007-07-26 Qualcomm Incorporated Inverse telecine algorithm based on state machine
US9131164B2 (en) * 2006-04-04 2015-09-08 Qualcomm Incorporated Preprocessor method and apparatus
US8379723B2 (en) * 2006-06-27 2013-02-19 Intel Corporation Chroma motion vector processing apparatus, system, and method
JP2008042332A (ja) * 2006-08-02 2008-02-21 Toshiba Corp 補間フレーム作成方法及び補間フレーム作成装置
JP4762095B2 (ja) * 2006-09-20 2011-08-31 株式会社東芝 画像復号装置及び画像復号方法
JP2009010492A (ja) * 2007-06-26 2009-01-15 Hitachi Ltd 画像復号化装置及び画像変換回路
JP4678015B2 (ja) * 2007-07-13 2011-04-27 富士通株式会社 動画像符号化装置及び動画像符号化方法
CN101415122B (zh) * 2007-10-15 2011-11-16 华为技术有限公司 一种帧间预测编解码方法及装置
KR101336951B1 (ko) * 2007-11-02 2013-12-04 삼성전자주식회사 휴대 단말기 및 그 파노라마 이미지 촬영 모드 수행 방법
US9456192B2 (en) * 2007-12-14 2016-09-27 Cable Television Laboratories, Inc. Method of coding and transmission of progressive video using differential signal overlay
US8355077B2 (en) * 2008-03-27 2013-01-15 Csr Technology Inc. Adaptive windowing in motion detector for deinterlacer
US8345996B2 (en) * 2008-07-07 2013-01-01 Texas Instruments Incorporated Determination of a field referencing pattern
JP2010187116A (ja) * 2009-02-10 2010-08-26 Toshiba Corp 動画処理装置
JP5115498B2 (ja) * 2009-03-05 2013-01-09 富士通株式会社 画像符号化装置、画像符号化制御方法およびプログラム
GB2469679B (en) * 2009-04-23 2012-05-02 Imagination Tech Ltd Object tracking using momentum and acceleration vectors in a motion estimation system
US8515203B2 (en) * 2009-06-25 2013-08-20 Pixart Imaging Inc. Image processing method and image processing module for a pointing device
CN101883286B (zh) * 2010-06-25 2012-12-05 无锡中星微电子有限公司 运动估计中的校准方法及装置、运动估计方法及装置
MX2013008691A (es) 2011-02-10 2013-08-21 Panasonic Corp Metodo de codificacion de imagenes en movimiento, aparato de codificacion de imagenes en movimiento, metodo de decodificacion de imagenes en movimiento, aparato de decodificacion de imagenes en movimiento y aparato de codificacion y decodificacion de imagenes en movimiento.
US20120207212A1 (en) * 2011-02-11 2012-08-16 Apple Inc. Visually masked metric for pixel block similarity
WO2012118359A2 (fr) 2011-03-03 2012-09-07 한국전자통신연구원 Procédé permettant de déterminer un paramètre de quantification de composant de différence de couleur et dispositif utilisant ledit procédé
US9363509B2 (en) 2011-03-03 2016-06-07 Electronics And Telecommunications Research Institute Method for determining color difference component quantization parameter and device using the method
CN103444181B (zh) 2011-04-12 2018-04-20 太阳专利托管公司 运动图像编码方法、运动图像编码装置、运动图像解码方法、运动图像解码装置及运动图像编码解码装置
US9167247B2 (en) * 2011-05-20 2015-10-20 Panasonic Intellectual Property Corporation Of America Methods and apparatuses for encoding and decoding video using inter-color-plane prediction
US9485518B2 (en) 2011-05-27 2016-11-01 Sun Patent Trust Decoding method and apparatus with candidate motion vectors
ES2911670T3 (es) 2011-05-27 2022-05-20 Sun Patent Trust Aparato, procedimiento y programa para decodificar imágenes en movimiento
JP5937589B2 (ja) 2011-05-31 2016-06-22 パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカＰａｎａｓｏｎｉｃ Ｉｎｔｅｌｌｅｃｔｕａｌ Ｐｒｏｐｅｒｔｙ Ｃｏｒｐｏｒａｔｉｏｎ ｏｆ Ａｍｅｒｉｃａ 動画像復号化方法、および動画像復号化装置
JP5729817B2 (ja) * 2011-06-29 2015-06-03 日本電信電話株式会社 動画像符号化装置、動画像復号装置、動画像符号化方法、動画像復号方法、動画像符号化プログラム及び動画像復号プログラム
CA2836063C (fr) 2011-06-30 2020-06-16 Panasonic Corporation Procede de decodage d'image, procede de codage d'image dispositif de decodage d'image, dispositif de codage d'image et dispositif de codage/decodage d'image
MX341415B (es) 2011-08-03 2016-08-19 Panasonic Ip Corp America Metodo de codificacion de video, aparato de codificacion de video, metodo de decodificacion de video, aparato de decodificacion de video y aparato de codificacion/decodificacion de video.
IN2014CN02602A (fr) 2011-10-19 2015-08-07 Panasonic Corp
JP2013172323A (ja) * 2012-02-21 2013-09-02 Toshiba Corp 動き検出装置、画像処理装置および画像処理システム
WO2013189257A1 (fr) 2012-06-20 2013-12-27 Mediatek Inc. Procédé et appareil de prédiction bidirectionnelle pour codage vidéo évolutif
JP2014143488A (ja) * 2013-01-22 2014-08-07 Nikon Corp 画像圧縮装置、画像復号装置およびプログラム
US20150023436A1 (en) 2013-07-22 2015-01-22 Texas Instruments Incorporated Method and apparatus for noise reduction in video systems
KR101582093B1 (ko) * 2014-02-21 2016-01-04 삼성전자주식회사 단층 촬영 장치 및 그에 따른 단층 영상 복원 방법
US10277921B2 (en) * 2015-11-20 2019-04-30 Nvidia Corporation Hybrid parallel decoder techniques
US10764589B2 (en) * 2018-10-18 2020-09-01 Trisys Co., Ltd. Method and module for processing image data
CN112889284A (zh) 2018-10-22 2021-06-01 北京字节跳动网络技术有限公司 基于子块的解码器侧运动矢量推导
CN109597488B (zh) * 2018-12-12 2019-12-10 海南大学 空间展示平台角度距离主动适应方法
CN113452921A (zh) * 2020-03-26 2021-09-28 华为技术有限公司 图像处理方法和电子设备
CN112632426B (zh) * 2020-12-22 2022-08-30 新华三大数据技术有限公司 网页处理方法及装置
CN114577498B (zh) * 2022-02-28 2024-05-14 北京小米移动软件有限公司 空调转矩补偿参数的测试方法及装置

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP0863675A2 (fr) * 1997-03-07 1998-09-09 General Instrument Corporation Détermination de mouvement et compensation de plaines d'objet de vidéo pour vidéo numérique entrelacée
JP2001238228A (ja) 2000-02-24 2001-08-31 Nippon Hoso Kyokai <Nhk> 動きベクトル検出装置
EP1359768A1 (fr) * 2002-04-10 2003-11-05 Microsoft Corporation Arrondis du vecteur de mouvement de chrominance
EP1406448A2 (fr) * 2002-09-06 2004-04-07 Fujitsu Limited Appareils pour le codage et le décodage d'images en mouvement
EP1411719A2 (fr) * 2002-10-18 2004-04-21 Broadcom Corporation Méthode et système de conversion de signal vidéo entrelacé en signal vidéo progressif au moyen d'une détection de contour en couleurs
JP2004219083A (ja) 2003-01-09 2004-08-05 Akashi Corp 振動発生機

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH04207788A (ja) * 1990-11-30 1992-07-29 Sony Corp 画像信号符号化装置及び方法
JP3764488B2 (ja) * 1997-06-03 2006-04-05 株式会社日立製作所 画像符号化及び復号化方法及び装置
JP3797209B2 (ja) * 2001-11-30 2006-07-12 ソニー株式会社 画像情報符号化方法及び装置、画像情報復号方法及び装置、並びにプログラム
JP4100067B2 (ja) 2002-07-03 2008-06-11 ソニー株式会社 画像情報変換方法及び画像情報変換装置

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP0863675A2 (fr) * 1997-03-07 1998-09-09 General Instrument Corporation Détermination de mouvement et compensation de plaines d'objet de vidéo pour vidéo numérique entrelacée
JP2001238228A (ja) 2000-02-24 2001-08-31 Nippon Hoso Kyokai <Nhk> 動きベクトル検出装置
EP1359768A1 (fr) * 2002-04-10 2003-11-05 Microsoft Corporation Arrondis du vecteur de mouvement de chrominance
EP1406448A2 (fr) * 2002-09-06 2004-04-07 Fujitsu Limited Appareils pour le codage et le décodage d'images en mouvement
EP1411719A2 (fr) * 2002-10-18 2004-04-21 Broadcom Corporation Méthode et système de conversion de signal vidéo entrelacé en signal vidéo progressif au moyen d'une détection de contour en couleurs
JP2004219083A (ja) 2003-01-09 2004-08-05 Akashi Corp 振動発生機

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
"Information technology - Generic coding of moving pictures and associated audio information: Video", ITU-T STANDARD IN FORCE (I), INTERNATIONAL TELECOMMUNICATION UNION, GENEVA,, CH, no. H262 2/, 1 February 2000 (2000-02-01), XP017401436 *

Also Published As

Publication number Publication date
KR20060010689A (ko) 2006-02-02
EP1622387A2 (fr) 2006-02-01
CN1728832A (zh) 2006-02-01
US20060023788A1 (en) 2006-02-02
EP1622387A3 (fr) 2007-10-17
JP4145275B2 (ja) 2008-09-03
DE602004022280D1 (de) 2009-09-10
KR100649463B1 (ko) 2006-11-28
EP1622387B1 (fr) 2009-07-29
JP2006041943A (ja) 2006-02-09
CN100546395C (zh) 2009-09-30

Similar Documents

Publication Publication Date Title
EP1622387B1 (fr) Estimation et compensation de mouvement avec correction de vecteur de mouvement basée sur des valeurs de composantes verticales
US5453799A (en) Unified motion estimation architecture
EP2323406B1 (fr) Codage et décodage pour vidéo entrelacé.
US8711937B2 (en) Low-complexity motion vector prediction systems and methods
US8514939B2 (en) Method and system for motion compensated picture rate up-conversion of digital video using picture boundary processing
US7426308B2 (en) Intraframe and interframe interlace coding and decoding
US6509930B1 (en) Circuit for scan conversion of picture signal using motion compensation
US8780970B2 (en) Motion wake identification and control mechanism
EP0951184A1 (fr) Procede de conversion de signaux numeriques et appareil de conversion de signaux numeriques
US20030202587A1 (en) Video data encoding apparatus and method for removing a continuous repeat field from the video data
US20080084932A1 (en) Controlling loop filtering for interlaced video frames
KR20060047595A (ko) 적응 시간적인 예측을 채용하는 움직임 벡터 추정
US20080137740A1 (en) Method of transcoding data from the MPEG2 standard to an MPED4 standard
US7072399B2 (en) Motion estimation method and system for MPEG video streams
WO1999067952A1 (fr) Procede et dispositif de conversion de nombre de trames de signaux d&#39;images
US6738426B2 (en) Apparatus and method for detecting motion vector in which degradation of image quality can be prevented
KR101038794B1 (ko) 인터레이스형 비디오의 코딩 및 디코딩
US20080192828A1 (en) Method for pixel prediction with low complexity
JPH07147670A (ja) 画像データの補間装置
JP2001309389A (ja) 動きベクトル変換装置及び方法

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

AC Divisional application: reference to earlier application

Ref document number: 1622387

Country of ref document: EP

Kind code of ref document: P

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): DE FR GB

17P Request for examination filed

Effective date: 20090812

AKX Designation fees paid

Designated state(s): DE FR GB

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE APPLICATION IS DEEMED TO BE WITHDRAWN

18D Application deemed to be withdrawn

Effective date: 20110701