US20030043918A1 - Method and apparatus for performing video image decoding - Google Patents

Method and apparatus for performing video image decoding Download PDF

Info

Publication number
US20030043918A1
US20030043918A1 US09470741 US47074199A US2003043918A1 US 20030043918 A1 US20030043918 A1 US 20030043918A1 US 09470741 US09470741 US 09470741 US 47074199 A US47074199 A US 47074199A US 2003043918 A1 US2003043918 A1 US 2003043918A1
Authority
US
Grant status
Application
Patent type
Prior art keywords
image
method
dct
motion compensation
downsampled
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Abandoned
Application number
US09470741
Inventor
Hong Jiang
Allen Simon
Val Cook
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Intel Corp
Original Assignee
Intel Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51Motion estimation or motion compensation
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/59Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving spatial sub-sampling or interpolation, e.g. alteration of picture size or resolution

Abstract

In one embodiment, a method of performing video image decoding includes the following. A compressed video image is downsampled in the frequency domain. The downsampled video image is inverse transformed. Motion compensation for the downsampled image is performed in the spatial domain.

Description

    BACKGROUND
  • 1. Field [0001]
  • This disclosure is related to video processing and, more particularly, to decoding video images. [0002]
  • 2. Background [0003]
  • Due to implementation constraints, motion compensation hardware employed in video decoding is typically designed for a given video image resolution. For example, without limitation, the MPEG2 specification, ISO/IEC 13818-2 MPEG-2 Video Coding Standard, “Information technology Generic coding of moving pictures and associated audio information: Video,” March, 1995, hereinafter referred to as “MPEG2,” may impose a video resolution of 720 pixels times 480 pixels at 30 frames per second. In a conventional design, the engine that performs the decoding will typically only generate images at the video resolution that the compressed video bit stream specifies. As a result of the amount of memory employed to hold decoded images, higher resolution compressed video bit streams, such as MPEG2 bit streams for digital television (DTV) content, for example, will not run on such a system. If sufficient memory is available to decode at the full specified resolution, and a user chooses to view the video on a smaller window on a computer platform, for example, downscaling is performed on the full size decoded image at display time and, therefore, full resolution decoding is still employed. A need, therefore, exists for a method or technique for a system to operate on or produce video resolutions other than the resolutions specified by the compressed video bit stream providing the video or image data.[0004]
  • BRIEF DESCRIPTION OF THE DRAWINGS
  • The subject matter regarded as the invention is particularly pointed out and distinctly claimed in the concluding portion of the specification. The invention, however, both as to organization and method of operation, together with objects, features and advantages thereof, may best be understood by reference to the following detailed description when read with the accompanying drawings in which: [0005]
  • FIG. 1 is a block diagram illustrating an embodiment of a conventional pipeline for decoding compressed digital video images; [0006]
  • FIG. 2 is a block diagram illustrating an embodiment of an apparatus for performing video image decoding in accordance with the present invention; [0007]
  • FIG. 3 is a schematic diagram illustrating the result on a macroblock of applying an embodiment of a method for performing video decoding in accordance with the present invention; [0008]
  • FIG. 4 is a schematic diagram illustrating the result on a macroblock of applying another embodiment of a method for performing video decoding in accordance with the present invention; [0009]
  • FIG. 5 is a schematic diagram illustrating the result on a macroblock of applying yet another embodiment of a method for performing video decoding in accordance with the present invention; [0010]
  • FIG. 6 is a schematic diagram illustrating the result on a macroblock of applying still another embodiment of a method for performing video decoding in accordance with the present invention; [0011]
  • FIG. 7 is a block diagram illustrating an embodiment of hardware that may be employed to implement an embodiment of a method for performing video decoding in accordance with the present invention; [0012]
  • FIG. 8 illustrates FIG. 7 with an overlay to schematically illustrate correspondence with an embodiment of a three-dimensional (3D) rendering pipeline; [0013]
  • FIG. 9 is a schematic diagram illustrating the pixel operation of a bilinear interpolation, such as may be employed in an embodiment of a method for performing video decoding in accordance with the present invention; [0014]
  • FIG. 10 is a block diagram illustrating an embodiment of a bilinear interpolator such as may be employed in an embodiment of a method for performing video decoding in accordance with the present invention; [0015]
  • FIG. 11 is a block diagram illustrating an embodiment of a video decoder subsystem coupled with a video display subsystem, such as may employ an embodiment of a method for performing video decoding in accordance with the present invention; [0016]
  • FIG. 12 is a schematic diagram illustrating a scenario of spatial positions of regularly interlaced video data with uniformly positioned fields, where lines in the bottom field are positioned half-way between two lines in the top field, such as may be employed in an embodiment in accordance with the invention; [0017]
  • FIG. 13 is a schematic diagram illustrating a scenario of spatial position of regularly interlaced video data with non-uniformly positioned fields, where lines in the bottom field are positioned {fraction (1/4)} of the way between two adjacent lines in the top field, such as may be employed in an embodiment in accordance with the invention; [0018]
  • FIG. 14 is a schematic diagram illustrating a scenario of spatial position of regularly interlaced video data with non-uniformly positioned fields, where lines in the bottom field are positioned {fraction (1/8)} of the way between two adjacent lines in the top field, such as may be employed in an embodiment in accordance with the invention; [0019]
  • FIG. 15 is a schematic diagram illustrating the results of a DDA-based vertical scaling operation for uniformly-positioned interlaced video source, such as may be employed in an embodiment in accordance with the invention; and [0020]
  • FIG. 16 is a schematic diagram illustrating the results of a DDA-based vertical scaling operation for non-uniformly-positioned interlaced video source, such as may be employed in an embodiment in accordance with the invention.[0021]
  • DETAILED DESCRIPTION
  • In the following detailed description, numerous specific details are set forth in order to provide a thorough understanding of the invention. However, it will be understood by those skilled in the art that the present invention may be practiced without these specific details. In other instances, well-known methods, procedures, components and circuits have not been described in details so as not to obscure the present invention. [0022]
  • As previously indicated, conventionally, a video decode and display system normally designed for a given maximum resolution will typically not operate on bit streams that specify higher video resolutions. Likewise, if a user chooses to view the video in a smaller window, downscaling the bit stream is conventionally achieved at display time and, therefore, full resolution decoding still occurs. Since full resolution decoding followed by downscaling adds cost in the form of additional computation, additional memory, additional memory bandwidth, and complex downscaling at display time, it would be desirable if downscaling of a bit stream could be accomplished without full resolution decoding. [0023]
  • Although the invention is not limited in scope in this respect, FIG. 7 is a schematic diagram illustrating an embodiment of a hardware motion compensation engine that may be employed to implement an embodiment of a method of performing video decoding in accordance with the invention. For example, as shall be described in greater detail hereinafter, a three-dimensional (3D) pipeline may be employed to efficiently perform motion compensation, as illustrated in FIG. 8, although other hardware platforms other than a 3D pipeline may be employed to implement embodiments of a method of performing video decoding in accordance with an embodiment of the invention. FIGS. 7 and 8 are described in more detail hereinafter. [0024]
  • FIG. 1 is a block diagram illustrating an embodiment of conventional pipeline for performing video image decoding. FIG. 2 is a block diagram illustrating an embodiment of an apparatus for performing video decoding in accordance with the present invention. In one embodiment in accordance with the invention, as shall be described in greater detail hereinafter, a compressed video image in the frequency domain is downsampled at [0025] 230 and then inverse transformed at 220. Motion compensation is performed on the downsampled image in the spatial domain at 210. Alternatively, as shall also be described in greater detail hereinafter, the compressed image in the frequency domain may be inverse transformed at 240 and then downsampled in the spatial domain and motion compensated. Although the invention is not limited in scope in this respect, one example of a compressed video image in the frequency domain comprises a discrete cosine transform (DCT) image. Likewise, although the invention is not limited in scope in this respect, such a DCT image may comply with the MPEG2 specification, as shall be described in greater detail hereinafter. In this context, although MPEG2 is referred to, including the aspect that divides an image into 16×16 macroblocks, the invention is not limited in scope to employing MPEG, including MPEG2, to employing macroblocks of this particular size and shape or even to employing macroblocks at all.
  • As illustrated in FIG. 2 and as shall be described in greater detail, the DCT image may be downsampled before being delivered to the motion compensation engine. Likewise, as indicated above, downsampling may be applied in this embodiment either before the inverse DCT, such as at [0026] 230, or after the inverse DCT, such as at 240, depending upon a variety of factors. In this particular embodiment, although the invention is not limited in scope in this respect, the blocks illustrated in FIG. 2 prior to the vertical line are implemented in software and the blocks after the vertical line are implement in hardware. Conventionally, such video processing to accomplish downsampling would be performed in hardware; however, an advantage of an embodiment of a method of performing video decoding in accordance with the present invention is that it provides the capability to perform the processing in software due at least in part to greater processing efficiency in comparison with conventional approaches. Therefore, one advantage of this approach is that it provides greater flexibility. In such an embodiment, the decoder software may transfer the downsampled prediction error to the motion compensation hardware and the motion vectors may be adjusted substantially in accordance with the downsampling ratio, as explained hereinafter. In this particular embodiment, although, again the invention is not limited in scope in this respect, downsampling ratios of 1:1, 2:1, 4:1, and 8:1, along either of the horizontal, vertical or both directions, may be supported. In this particular embodiment, where MPEG2 is employed, the downsampling ratio is limited to no more than 8:1 due to the native eight-by-eight MPEG2 block size. However, this limitation may not apply in alternative embodiments. Furthermore, in alternative embodiments, even for MPEG2, downsampling ratios other than a power of two may be implemented, such as, for example, 3:1.
  • As illustrated in FIG. 2, the motion compensation hardware may operate directly on the downsampled bit stream. In this particular embodiment, where MPEG2 is employed, the downsampling ratio may be n, where n equals 1, 2, 4, and 8. In a motion compensation process, a motion vector of a processed macroblock specifies the relative distance of reference data from the processed macroblock. Let (V[0027] x, Vy)=(vector[r][0], vector[r][1]) be the original motion vector for a macroblock, where Vx and Vy, the horizontal and vertical components of the motion vector, are in the form of 16-bit signed value, although the invention is not limited in scope in this respect. According to the MPEG2 standard, the least significant bit (LSB) of Vx and Vy is used to indicate the half-pixel resolution reference. Denote the whole pixel motion displacement for the luminance component by (Dx Y, Dy Y)and the fractional offset for the luminance component by (Fx Y, Fy Y). Due to limited subpixel precision, the fractional offset (Fx Y, Fy Y may also be called as the half-pixel offset flag. When there is no downsampling to the bitstream, these may be calculated from the motion vector as follows: { D x Y = V x >> 1 , D y Y = V y >> 1 , and [ 1 ] { F x Y = V x & 1 , F y Y = V y & 1. [ 2 ]
    Figure US20030043918A1-20030306-M00001
  • where “>>” indicates a right shift operation and “&” indicates a “logic AND” operation. If F[0028] x Y is set, or precisely is non-zero, horizontal interpolation, such as computing an average, may be applied to the reference pixels. If Fy Y is set, vertical interpolation, such as computing an average, may be applied to the reference pixels. If both are set, interpolations along both directions may be applied.
  • The chrominance motion displacement may also derived from the same set of motion vector signal information. For YUV 4:2:0 color space format, for example since the dimension of chrominance (Cb, Cr) pictures is half of that of the luminance component picture along both horizontal and vertical directions, the whole pixel displacement (D[0029] x C, Dy C) and fractional offset (Fx C, Fy C) for the chrominance components of the processed macroblock may be determined as follows: { D x C = ( V x / 2 ) >> 1 , D y C = ( V y / 2 ) >> 1 , and [ 3 ] { F x C = ( V x / 2 ) & 1 , F y C = ( V y / 2 ) & 1 , [ 4 ]
    Figure US20030043918A1-20030306-M00002
  • where symbol ‘/’ denotes regular integer division with truncation of the result toward zero. Notice that in this example the chrominance fractional offset is also in half chrominance pixel resolution. [0030]
  • Ignoring the luminance and chrominance superscripts of terms (D[0031] x, Dy) and (Fx, Fy), the motion prediction operation may be, in one embodiment, implemented with simple adders and shifters as the following pseudo-code illustrates.
    MC_Prediction(p, q)
    {
    if( Fx !=1 && Fy !=1) /* full-pel prediction in both directions */
    P[q][p] = R[n][m];
    if( Fx ==1 && Fy !=1) /* full-pel vertical, half-pel horizontal */
    P[q][p] =(R[n][m] + R[n][m+1]) //2;
    if( Fx !=1 && Fy ==1) /* half-pel vertical, full pel horizontal */
    P[q][p] = (R[n][m] + R[n+1][m]) //2;
    if( Fx ==1 && Fy ==1) /* half-pel prediction in both directions */
    P[q][p] = (R[n][m] +R[n][m+1] + R[n+1][m] + R[n+1][m+1]) //4;
    }// end Mc_Prediction(p, q)
  • In this example, the division symbol “//” denotes rounding up to the next larger integer (rounding away from zero). Symbols p and q represent integer indices in the destination image along horizontal and vertical directions, respectively. Symbols m and n represent integer indices in the reference image along horizontal and vertical directions, respectively. The reference pixel location (m, n) may be derived from the motion vector displacement term (D[0032] x, Dy).
  • In this particular embodiment of the invention, the motion-compensated prediction applied to the downsampled bit stream is performed directly using the downsampled reference images and the original motion vectors decoded from the compressed bitstream. The motion vectors used in the prediction may also be specified by the motion displacement (D[0033] x Y, Dy Y), (Dx C, Dy C) and motion fractional offset (Fx Y, Fy Y), (Fx C, Fy C) with reference to the downsampled image. Contrary to conventional motion fractional offset that is only a one bit value in MPEG2, as previously described, more precision is preserved for (Fx Y, Fy Y), (Fx C, Fy C) in a downsampling operation in this particular embodiment in accordance with the invention. Consequently, the simple averaging operation described above may be replaced by more accurate interpolation operations. In one embodiment, for example, a bilinear interpolation unit may be used in the motion prediction calculation of motion compensation, although the invention is not limited in scope in this respect. The video or image reconstruction quality may also be improved by using a higher order interpolation unit. A bilinear interpolator typically employs more hardware than an averaging based interpolator. However, it is a common feature that may be provided as part of a state-of-the-art graphics controller hardware. For example, it may be found in the texture pipeline of a three-dimensional (3D) rendering engine or an image processor for image scaling or filtering. In one embodiment, therefore, as illustrated in FIG. 8, a 3D pipeline may include a bilinear interpolator, designated 820 and 830, such as one having a 6-bit interpolation phase value, as shown by FIG. 10. In such an embodiment, although the invention is not limited in scope in this respect, motion displacement and motion fractional offset may be calculated from motion vectors, decoded from the compressed bitstream as follows: { D x Y = V x >> SubD x D y Y = V y >> SubD y and { D x C = ( V x / 2 ) >> SubD x D y C = ( V y / 2 ) >> SubD y and [ 5 ] { F x Y = ( V x & FMaskD x ) << SubR x F y Y = ( V y & FMaskD y ) << SubR y and { F x C = ( ( V x / 2 ) & FMaskD x ) << SubR x F y C = ( ( V y / 2 ) & FMaskD y ) << SubR y . [ 6 ]
    Figure US20030043918A1-20030306-M00003
  • For these relationships and this embodiment, values for the subsampled displacement shifts SubD[0034] x and SubDy, the subsampled fractional masks FMaskDx, FMaskDy, and the subsampled bilinear interpolation phase shifters SubRx, SubRy, based at least in part on the downsampling ratio, may be determined. These are provided in Table 1, below, for a system with 6-bit interpolation phase value range. It will be appreciated that the values for a system with a different interpolation precision may, likewise, be derived as desired. It will also be appreciated the corresponding interpolation parameters for a system with a different interpolation filter other than a bilinear interpolation filter may also be derived as desired.
    TABLE 1
    Variables that are used to set the bilinear
    interpolation parameters for downsampling.
    Downsampling Ratio
    1:1 2:1 4:1 8:1
    SubDx or SubDy 1 2 3 4
    FMaskDx or FMaskDy 0 × 01 0 × 03 0 × 07 0 × 0F
    SubRx or SubRy 5 4 3 2
  • With the above equations defining the motion displacement and motion fractional values, the motion-compensated prediction may be described for this embodiment by the following bilinear interpolation relation: [0035]
  • P[q][p]={(0x40−F y)·[(0x40−F xR[n][m]+F x ·R[n][m+1]]+F y·[(0x40−F xR[n+1][m]+F x ·R[n+1][m+1]]}//0x80
  • ={(0x40−F y)·[(0x40−F xR[q+D y ][p+D x ]+F x ·R[q+D y ][p+D x+1]]+F y·[(0x40−F xR[q+D y+1][p+D x ]+F x ·R[q+D y+1][p+D x+1]]}//0x80.  [7]
  • where, the reference pixel location (m, n) is derived from the motion vector displacement term (D[0036] x, Dy).
  • FIG. 9 illustrates the spatial relationship of the four reference pixels contributing to the prediction of the reconstructed pixel for this embodiment. As previously indicated, one embodiment of a bilinear interpolator is illustrated in FIG. 10. In this embodiment, bilinear interpolator [0037] 1001 is formed by two linear interpolators, 1020 and 1030, that operate along one spatial direction followed by another linear interpolator, 1040, that operates along the orthogonal direction. The output signal from linear interpolator 1040 passes through a rounding and saturation unit 1050 that converts the output signal to a specified finite precision form, although the invention is not limited in scope in this respect, and this is just one example of a bilinear interpolator embodiment. Furthermore, the invention is not limited in scope to employing bilinear interpolation.
  • For this embodiment, the above mentioned motion compensation operation may be implemented with a hardware motion compensation system, such as the one, [0038] 701, illustrated in FIG. 7, although, again, the invention is not limited in scope in this respect. Here, the operation is applied on a macroblock basis, however, as previously indicated, this is merely a feature of MPEG2 and alternative embodiments are possible. The operation of this particular embodiment shall now be described in detail. Command parser and address generator unit 810 receives motion compensation instructions for a given macroblock and generates destination addresses and transmits the destination addresses to correction data memory interface unit 815 and destination data memory interface unit 825. Correction data memory interface unit 815 uses this(these) destination address(es) to load correction data from a correction data buffer(not shown). Destination data memory interface unit 825 uses this(these) destination address(es) to send the final output data from the motion compensation engine to the destination buffer(not shown). Command parser and address generator unit 810 also generates a prediction address (or addresses) in the reference picture or image using information about the current macroblock and its motion vectors and sends this to reference data memory interface unit 835. The reference data memory interface unit uses this to load data from a forward reference buffer, or from a backward reference buffer, or from both a forward reference buffer and a backward reference buffer(not shown).
  • The command parser and address generator unit also generates subpixel fractional information to be applied to the bilinear interpolation units, [0039] 820 and 830. Of these two bilinear interpolation units, one performs forward prediction and one performs backward prediction. Here, each bilinear interpolation unit uses the fractional information to interpolate data from the reference buffer. It is conceivable that these two bilinear interpolation units may be implemented as a single hardware unit. In the case of a single hardware bilinear interpolation unit is implemented, this bilinear interpolation unit may be used sequentially if forward and backward bi-directional prediction is desired.
  • The output signals from the forward bilinear interpolation unit and the backward bilinear interpolation unit are added together in combine predictions unit [0040] 850. The combine predictions unit performs proper scaling and saturation to the data, such as according to a compression standard, such as, for example, MPEG2. The output signal from the combine predictions unit is then sent to prediction correction unit 860 and the correction data are added to the motion prediction data and final output data, for this embodiment, are generated. The output data from the prediction corrections unit is then sent to memory by the destination data memory interface.
  • As illustrated in FIG. 8, the above mentioned embodiment of motion compensation implementation may be implemented using existing 3D rendering hardware that is currently a common feature in graphics controller hardware. The boxes in dotted lines map the motion compensation aspects of this embodiment just described into some 3D hardware units, for illustration purposes. Of course, other hardware mapping and hardware reusing are also possible and may now be implemented by one of ordinary skill in the art. In this particular embodiment, the reference buffers are mapped as texture buffers. Therefore, the texture memory and texture cache may be used to obtain the reference data load from memory. After that, the 3D texture pipeline that typically contains bilinear interpolators or even tri-linear interpolators may be used to perform the bilinear interpolation and prediction combination operations in motion compensation. Then, the 3D texture blend unit may be used to perform the prediction correction operation. The 3D color and destination memory interface unit may be used to write the output signals of motion compensation to memory. [0041]
  • Several embodiments where MPEG2 coding has been employed shall be described. As previously explained, the invention is not limited in scope to these particular embodiments. Any one of a number of other video or image coding specifications and/or storage formats may be employed. Nonetheless, these embodiments are provided as examples of implementations of a method of performing video image decoding in accordance with the present invention. In this context, three main categories of MPEG2 coding types shall be described. One coding type comprises a frame image with frame prediction or frame motion compensation employed. In this context, the term frame image or frame type refers to a progressive sequence display of data signals for an image, such as is commonly employed on computer platforms having monitors. The term frame prediction or frame motion compensation refers to a particular format for the prediction error and for the motion vectors that have been coded or produced by an encoder. It is desirable to know the format in which this signal information is encoded in the bitstream in order to perform decoding to reconstruct the image that produced this signal information. Therefore, if frame prediction or frame decoding is employed, then the prediction error is stored in a frame format, analogous to the format employed for a frame image. A second coding type comprises a field image with field motion compensation or field prediction. The term field image or field type generally refers to a technique commonly employed for television sets or television set displays in which half of the image is displayed separately at a rate that allows the human eye to merge the images. In this format, field data lines, that is, lines of signal data from a field image, are stored in an interlaced format. Therefore, top field and bottom field lines are alternated or interlaced within a frame of signal data. The term field motion compensation or field prediction refers to the format in which the prediction error and motion vectors are stored in which prediction may be predicated upon the so-called top fields or bottom fields independently. In a field encoded image, the top and bottom fields are each encoded as separate images, and then displayed in an interlaced format. The motion prediction data for the top and bottom fields in this case is based in part on recently decoded fields. A third MPEG2 coding type employed in this context comprises a frame image with field motion compensation or field prediction. In this format, both fields are encoded as a single image, but the motion compensation data for each of its two fields is based in part on previously decoded fields. In MPEG2, this third format has two variations. In one variation, such as illustrated in FIG. 5, the luminance DCT data is encoded on a frame basis, while in the other variation, such as illustrated in FIG. 6, the luminance DCT data is stored on a field basis. This coding type is between the two coding types mentioned above in that both formats may be interspersed on a macroblock basis. More specifically, on a macroblock basis, data signals may be stored as a frame image with either field or frame prediction. [0042]
  • Because these particular embodiments relate to a DCT domain downsampling implementation for MPEG2 coding types, downsampling and motion compensation that is applied to the vertical direction will be employed. The horizontal direction in a video frame is handled similarly for the MPEG2 coding types described above, and therefore, in this embodiment, the horizontal direction is handled in a similar manner as the approach described below for a frame image with frame prediction, although, in a particular implementation of video image decoding in accordance with the present invention, this aspect may vary. Further, the illustrations given herein illustrate the technique for luminance component only. Nevertheless, an extension of this technique, once described, to handle the chrominance component of MPEG is within the ability of one of ordinary skill in the art. Further, in other applications with multiple components encoded in the bitstream, such as, but not limited to, RGB encoded JPEG images, the extension of the technique described herein to each of the components is within the ability of one of ordinary skill in the art. [0043]
  • FIG. 3 is a schematic diagram of an embodiment of a method for performing video image decoding in accordance with the present invention in which a DCT image that complies with the MPEG2 specification is employed. In this particular embodiment, a frame image with frame motion compensation, as described above, is the MPEG2 coding type employed. FIG. 3 illustrates two 8×8 luminance blocks in a macroblock in which downsampling in the DCT domain occurs. Column 1 illustrates spatial positioning for data lines of the two blocks prior to downsampling. Column 2 illustrates spatial locations for the data lines after downsampling. Therefore, column 2 illustrates the effect on the data positioning of downsampling in the DCT domain and then performing the inverse DCT. Likewise, as FIG. 3 illustrates, column 3 illustrates downsampling for a ratio of 4:1, as opposed to a ratio of 2:1 for column 2. As shown in FIG. 3, the downsampled lines are uniformly distributed in space after downsampling and inverse transforming. This would occur in this embodiment in a similar way for the downsampled pixels in the horizontal direction. Therefore, the downsampled frame image and frame motion vectors may be in a manner similar to the approach applied to the original image. The result of downsampling is to convert the 16×16 macroblocks and the 8×8 blocks they contain to smaller blocks. For example, after 2:1 horizontal subsampling and 4:1 vertical subsampling, each 8×8 block is decoded into a 4×2 block, and each 16×16 macroblock is decoded into an 8×4 macroblock. Thus, motion compensation for any given downsampled block, such as blocks with size 4×4, 4×2, 2×4, 2×2, or 1×1 in this embodiment, may be directly conducted on the downsampled references using scaled motion vectors employing, in this particular embodiment, the technique described previously. Therefore, although the invention is not limited in scope in this respect, the previously described motion compensation hardware may be efficiently employed to perform this signal processing operation. [0044]
  • FIG. 4 is a diagram illustrating an embodiment of a method for performing video image decoding in accordance with the present invention in which another MPEG2 coding type is employed. In this particular embodiment, a field image with field motion compensation is employed, as described above. Considering the nature of the two temporally separated fields for one frame, field based downsampling may introduce spatial aliasing and/or a non-uniform positioning of the lines from the two fields. The non-uniform positioning that may result is illustrated in FIG. 4 in which, again, downsampling has been applied, and then the inverse DCT, to illustrate the effect on this coding type. However, the non-uniform line spacing does not affect motion vectors. Likewise, adjustments to the line positioning illustrated in FIG. 4, such as for the prediction error, may be accomplished using interpolation techniques such as bilinear interpolation. Again, the 3D hardware pipeline previously described may be employed to implement these interpolations. Therefore, in this particular embodiment, motion compensation, as well as the spatial positioning of the downsampled blocks, should include the exact line positioning for each field. [0045]
  • In another embodiment, instead of employing the approach illustrated in FIG. 4, which produced non-uniform vertical line spacing, selected taps may be employed for the top field and bottom field lines to produce a downsampled image that is uniformly spaced in the vertical direction. For example, although the invention is not limited in scope in this respect, two spatial filters, one respectively for each of the bottom and top fields, may be employed. In addition, a similar approach may alternatively be employed in the frequency domain, such as the DCT domain. Where it is employed in the frequency domain, the transformed data signals may be phase shifted, rather than spatially shifted. The relation of a spatial shift and its corresponding transform domain operation may be derived using convolution property of the particular transform. [0046]
  • FIGS. 5 and 6 each illustrate portions of embodiments of a method for performing video image decoding in accordance with the present invention for an MPEG2 coding type described as a frame image with field motion compensation. FIG. 5 illustrates application of a portion of an embodiment of a method for performing video decoding in accordance with the invention to a macroblock stored in this format as a frame type with field prediction and frame DCT. In contrast, FIG. 6 illustrates application of a portion of an embodiment of a method for performing video decoding in accordance with the present invention to a macroblock stored in this format as a frame type with field prediction and field DCT. It may be convenient to convert the image data and prediction or motion compensation data to one format, either frame or field. Likewise, conversion to a frame format may generally involve temporal filtering, which might involve a modification of the previously illustrated 3D pipeline hardware. However, of course, the invention is not limited in scope in this respect and this approach may be employed with a hardware pipeline, for example, that includes this feature. In this particular embodiment, however, operations are performed to place the frame data in a field format, and to place the frame motion compensation data into a field motion compensation format. Each field is then processed separately in the spatial domain to accomplish motion compensation, in this particular embodiment. [0047]
  • One modification, then, for this particular embodiment is to convert a frame downscaled macroblock into a field downscaled macroblock. In this particular embodiment, as illustrated in FIG. 5, this is accomplished by reconstruction of the blocks in a macroblock at full vertical resolution in the spatial domain by inverse transformation from the DCT domain, interlacing the block into two fields and downscaling it vertically in the spatial domain. Therefore, for this embodiment, the vertical downscaling is effectively moved to after performing the inverse DCT, as illustrated in FIG. 1. Likewise, motion compensation is performed on each field separately, as mentioned above. If the motion compensation were frame based, then, in this embodiment, the prediction error could be converted to field based using the technique illustrated. To convert frame motion vectors to field based, the frame motion vector may be employed for each of the top and bottom field motion vectors. A difference between the embodiments illustrated in FIG. 5 and FIG. 6 is whether the macroblock is stored as a frame macroblock or a field macroblock. As previously discussed, if it is stored as a frame macroblock, then interleaving is performed as illustrated in FIG. 5. In contrast, as illustrated in FIG. 6, if the macroblock is stored as a field macroblock, then interleaving is performed, as illustrated, and the data lines may be processed as previously described for an interleaved field format. [0048]
  • An aspect of an embodiment in accordance with the invention is the downscaling of a video image in the frequency domain, such as an MPEG2 image in the DCT domain, although the invention is not limited in scope in this respect. This may be discussed by referring to one-dimensional (1D) signals. The results for 2D signals would be an extension of this approach due to the separability of operations. Likewise, the case of 2:1 downscaling will be discussed as representative of other downscaling ratios. In general, implementing downscaling in the frequency domain is well-known and there are many well-known ways to accomplish it. The invention is not restricted in scope to a particular approach and this discussion is provided as only one example. [0049]
  • The filtering of finite digital signals in the sample domain is performed using convolution. A well-known circular convolution may be obtained, for example, by a periodic extension of the signal and filter. It may be efficiently performed in the discrete Fourier transform (DFT) domain by simple multiplication of the discrete Fourier transforms of the signal and filter and then applying the inverse DFT to the result. For the DCT, a convolution may be applied that is related to, but different from the DFT convolution. This is described, for example, in “Symmetric Convolution and the Discrete Sine and Cosine Transforms,” by S. Martucci, IEEE Transactions on Signal Processing, Vol. 42, No. 5, May 1994, and includes a symmetric extension of the signal and filter, linear convolution, and applying a window to the result. For example, assuming that the signal is represented as s(n), n=0, . . . , N−1, and its corresponding transform (DCT domain) coefficients is represented as S(u), u=0, . . . , N−1, and the filter is represented as h(m), m=0, . . . , M−1, then the DCT may be represented in matrix form as S=C*s, with s, S being column vector form of the signal and its DCT coefficients and C being the DCT matrix, as follows: [0050]
  • C u,n=(2/N)1/2 k(u)cos[π(u(2n+1)/2N)], where u,n=0, . . . , N−1  [8]
  • where [0051]
  • k(u)=
  • 1/{square root}{square root over (2)}, where u=0
  • 1, u=1, . . . , N−1  [9]
  • Assume a symmetric low pass even length filter h(m) with filter length M, where M=2*N, the DCT coefficients H(u) for the filter may be obtained by applying the convolutional form described above to the right half of the filter, which is equivalent to multiplication of the right half coefficients by the transform matrix: [0052]
  • Du,m=2 cos[πu(2m+1)/2n], where u,m=1, . . . , N−1  [10]
  • The filtering is then performed by element-by-element multiplication of the signal DCT coefficients and the filter DCT coefficients and taking the appropriate inverse DCT transform of the DCT-domain multiplication results: [0053]
  • Y(u)=S(u)*H(u), where u=0 . . . , N−1  [11]
  • Not only filtering, but also downsampling, may be performed in the DCT domain. For downsampling by two, the result of the element-by-element multiplication is folded across the middle half point and subtracted and after that scaled by 1/12. Mathematically, this is illustrated as: [0054]
  • [Y(u)−Y(N−u)]/{square root}{square root over (12)}, where u=0, . . . (N/2)−1  [12]
  • The decimated signal is then obtained by applying the inverse DCT transform of the length N/2. There are several special cases that might be usefully applied in this embodiment, although the invention is not limited in scope in this respect. For example, a brickwall filter with coefficients [11110000] in the DCT domain may be implemented that can further simplify the DCT domain downsampling by two operation. Specifically, the special filter shape avoids folding and addition. Another filter with coefficients [1 1 1 1 0.5 0 0 0] provides a transform function of an antialising filter for the downsampling by two operation. Other filters may also be employed, of course. [0055]
  • Likewise, it will be appreciated that in this particular embodiment, a low pass, linear interpolation filter has been implemented to perform the downsampling; nonetheless, the invention is not limited in scope in this respect. For example, linear filters other than low pass filters or, alternatively, non-linear filters, such as, for example, a median filter, an adaptive edge-enhancement filter may be employed. It will, of course, be appreciated that some linear filters may effectively be implemented using motion compensation hardware and bilinear interpolation, although the invention is not limited in scope in this respect. [0056]
  • Filtering may also be applied after motion compensation or downsampling. More specifically, variations in clarity of the resulting images may become apparent to the human eye, particularly as the images are viewed in sequence. In some embodiments, it may be desirable to smooth these variations or, alternatively, enhance the images having less clarity. Therefore, any one of a number of filters, linear or non-linear, may be applied. For example, an edge enhancement image may be applied, although the invention is not limited in scope in this respect. Again, it will be appreciated that some linear filters may be effectively implemented using a 3D hardware pipeline and bilinear interpolation. [0057]
  • Of course, as previously indicated, the invention is not restricted in scope to the embodiments previously described. For example, in an alternative embodiment, where a 3D hardware pipeline is employed to implement a bilinear interpolation operation, a 3×3, 4×4, or greater interpolation operation may be implemented in place of a 2×2 bilinear interpolation operation. Likewise, in another alternative embodiment, as greater computational resources are demanded by the decoder in order to keep up with the video bit stream being provided or received, the decoder may be adapted to downsample at higher ratios in order to allow graceful degradation in the quality of the images provided. Likewise, the decoder may be adapted to perform the reverse as well. [0058]
  • In another embodiment, instead of downsampling all video images, the decoder may be adapted to downsample only some of the video images. For example, specific images may be selected for downsampling, such as by transmitting a signal indication, or the decoder may be adapted to downsample a subset of the received video images based at least in part on a predetermined criteria, such as, as one example, decoding I and P frames at full resolution while subsampling B frames. Therefore, any one of a number of approaches may be employed and the invention is not restricted in scope to any particular approach. [0059]
  • Another aspect of an embodiment in accordance with the invention is the display of the decoded video images that are downsampled in the frequency domain, such as an MPEG2 image in the DCT domain, although the invention is not limited in scope in this respect. In this particular embodiment, the video decoder subsystem discussed above is coupled to a video display subsystem, as illustrated in FIG. 10. Both the video decoder subsystem and the video display subsystem may couple with the memory subsystem, where decoded video images may reside. As illustrated in FIG. 10, in the memory subsystem, the decoded video images are labeled as video buffer [0060] 1, video buffer 2 and so on. The number n of decoded video images may be chosen according to the video decoder and video display subsystems. In such an embodiment, besides typical information, such as the decoded image size (X, Y), the video decoder subsystem may couples with the video display subsystem with additional signals, such as the Picture Type (PICT) and the vertical subsampling factor (VSFF), that relate to the transform-domain downsampling operation. Signals such as PICT and VSFF may be used to adjust the video display subsystem to properly display the decoded video images that are downsampled in the transform domain using an embodiment in accordance with the invention.
  • The video display subsystem handles displaying the decoded video images on the screen. The size of the desired display video window may not be the same as the source video image. In this case, the source video may be scaled up or down to match the display window size, corresponding to the process of interpolation and decimation, respectively. Quality scaling involves proper filtering of the source video data to reduce aliasing artifacts. In one approach, a finite impulse response (FIR) filter, where only finite number of input pixels contributes to a particular output pixel, is an example, of a scaling filter implemented in the video display subsystem. A filter for spatial scaling of a video signal is normally a 2-dimensional (2D) function. In practice, separable filters may be used to reduce the hardware complexity and cost. In other words, the scaling of a video signal is applied to the vertical and horizontal directions independently. In the following, the vertical scaling operation is addressed since it is relevant to the uniform and non-uniform field scan line distribution that the proposed video decoder generates. [0061]
  • For a given source size N[0062] src and a destination size Ndest, the forward scaling factor (in contrary to the backward scaling factor that we will define later) is defined as the ratio of the source size over destination size: S f = N src N dest . [ 13 ]
    Figure US20030043918A1-20030306-M00004
  • Denoting the source sampling step as unity, we can define a DDA (Digital Differential Analyzer) value for a given output line as the relative position to the source line vertical positions. Normally, a DDA accumulator contains a fixed-point value. The integer portion of the DDA value, denoted by int(DDA), indicates the closest source line number, while the fractional portion of the DDA value, denoted by fract(DDA), corresponds to the relative distance from that source line. The initial phase of a scaling operation is defined as the initial value of the DDA accumulator (DDA[0063] 0=DDA(0)) that is associated with the first output line from the scaling filter. Then the sample position of a succeeding output line may be described by the DDA value accumulated by the scaling factor.
  • DDA(n)=DDA(n−1)+S f, for n=1, Ndest−1,  [14]
  • where n is the index to the output video lines. [0064]
  • For a source video image that is created by the above mentioned video decoder subsystem and is in a frame type with the transform domain downsampling as illustrated in FIG. 3, its display is similar to the non-downsampled video image, although the scaling factor is different. [0065]
  • For a source video image that is created by the above mentioned video decoder subsystem and is in a field type with the transform domain downsampling but with uniformly distributed scan lines as illustrated in FIG. 12, again, its display method is similar to the non-downsampled field video image, although the scaling factor is different. However, for a source video image that is created by the above mentioned video decoder subsystem and is in a field type with the transform domain downsampling but with non-uniformly distributed scan lines, as illustrated in FIG. 13 and FIG. 14, the conventional field video display method cannot be applied to this kind of video images. Instead, proper vertical position adjustment is employed to display the top and bottom fields of the transform-domain downsampled video images correctly. [0066]
  • Let the distance between two adjacent lines in a field to be 1 unit. As illustrated in FIG. 12, for the non-downsampled field-type video image, the first line in the bottom field (line 1) is 0.5 unit below the first line in the top field (line 0). This is also true for the subsequent lines in the top and bottom fields. The results of a DDA-based vertical scaling operation for uniformly-positioned interlaced video source are illustrated in FIG. 15. The example shows the upscaling factor of 3:8. FIG. 15(a) is the case of scaling from the top field with an initial phase of DDA[0]=0.0, and FIG. 15(b) is the case of scaling from the bottom field with an initial phase of DDA[0]=−0.5. [0067]
  • When vertical downsampling by two is performed in the transform domain, the first line in bottom field (line 1) is 0.25 units below the first line in the top field (line 0) as illustrated in FIG. 13. FIG. 16 illustrates the results of aDDA-based vertical scaling operation for non-uniformly-positioned interlaced video source. The example shows the upscaling factor of 3:8. FIG. 16(a) is the case of scaling from the top field with an initial phase of DDA[0]=0.0, and FIG. 16(b) is the case of scaling from the bottom field with an initial phase of DDA[0]=−0.25. [0068]
  • Similarly, FIG. 14 illustrates that the first line in the bottom field is 0.125 units below the fist line in the top field when a vertical downsampling by four is performed in the transform domain. [0069]
  • It will, of course, be understood that, although a particular embodiment has just been described, the invention is not limited in scope to a particular embodiment or implementation. For example, one embodiment may be in hardware, whereas another embodiment may be in software. Likewise, an embodiment may be in firmware, or any combination of hardware, software, or firmware, for example. Likewise, although the invention is not limited in scope in this respect, one embodiment may comprise an article, such as a storage medium. Such a storage medium, such as, for example, a CD-ROM, or a disk, may have stored thereon instructions, which when executed by a system, such as a computer system or platform, or an imaging system, may result in a method of performing video image decoding in accordance with the invention, such as, for example, one of the embodiments previously described. [0070]
  • While certain features of the invention have been illustrated as described herein, many modifications, substitutions, changes and equivalents will now occur to those skilled in the art. It is, therefore, to be understood that the appended claims are intended to cover all such embodiments and changes as fall within the true spirit of the invention. [0071]

Claims (34)

  1. 1. A method of performing video image decoding comprising:
    downsampling a compressed video image in the frequency domain;
    inverse transforming the downsampled video image; and
    performing motion compensation for the downsampled image in the spatial domain.
  2. 2. The method of claim 1, wherein the compressed video image in the frequency domain comprises a discrete cosine transform (DCT) image.
  3. 3. The method of claim 2, wherein the DCT image is stored as a DCT image that complies with an MPEG specification.
  4. 4. The method of claim 3, wherein the DCT image is stored as a frame type image.
  5. 5. The method of claim 4, wherein the motion compensation data signals are stored as frame prediction type motion compensation.
  6. 6. The method of claim 3, wherein the DCT image is stored as a field type image.
  7. 7. The method of claim 6, wherein the motion compensation data signals are stored as field prediction type motion compensation.
  8. 8. The method of claim 1, and further comprising displaying the downsampled spatial image so that resulting non-uniform vertical spacing of data signal lines in the downsampled spatial image appear substantially uniform on a computer monitor.
  9. 9. The method of claim 1, wherein the downsampling is performed using an integer ratio.
  10. 10. The method of claim 1, wherein performing motion compensation comprises scaling motion vectors in accordance with the downsamping ratio.
  11. 11. The method of claim 10, wherein motion vector scaling comprises implementing an interpolation operation.
  12. 12. The method of claim 11, wherein motion vector scaling comprises implementing a bilinear interpolation operation.
  13. 13. The method of claim 12, wherein the bilinear interpolation operation is implemented on 3D pipeline hardware.
  14. 14. The method of claim 1, wherein downsampling comprises implementing a linear filter as a bilinear interpolation operation.
  15. 15. The method of claim 14, wherein the bilinear interpolation operation is implemented on 3D pipeline hardware.
  16. 16. A method of performing video image decoding comprising:
    inverse transforming a compressed video image;
    downsamping the inverse transformed image in the spatial domain; and
    performing motion compensation for the downsampled image in the spatial domain.
  17. 17. The method of claim 16, wherein the compressed video image comprises a discrete cosine transform (DCT) image.
  18. 18. The method of claim 17, wherein the DCT image is stored as a DCT image that complies with an MPEG specification.
  19. 19. The method of claim 18, wherein the DCT image comprises macroblocks stored as frame macroblocks and macroblocks stored as field macroblocks.
  20. 20 The method of claim 19, and further comprising: converting the frame macroblocks to field macroblocks prior to downsampling in the spatial domain.
  21. 21. The method of claim 19, wherein the motion compensation data signals are stored as field prediction type motion compensation.
  22. 22. The method of claim 16, wherein performing motion compensation comprises scaling motion vectors in accordance with a downscaling ratio.
  23. 23. The method of claim 22, wherein motion vector scaling comprises implementing an interpolation operation.
  24. 24. The method of claim 23, wherein motion vector scaling comprises implementing a bilinear interpolation operation.
  25. 25. The method of claim 24, wherein the bilinear interpolation operation is implemented on 3D pipeline hardware.
  26. 26. The method of claim 16, wherein downsampling comprises implementing a linear filter as a bilinear interpolation operation.
  27. 27. The method of claim 26, wherein the bilinear interpolation operation is implemented on 3D pipeline hardware.
  28. 28. An article comprising: a storage medium, having stored thereon instructions, that when executed by a platform, result in the following:
    downsampling a compressed video image in the frequency domain;
    inverse transforming the downsampled video image; and
    performing motion compensation for the downsampled image in the spatial domain.
  29. 29. The article of claim 28, wherein the instructions, when executed further result in the compressed video image in the frequency domain comprising a discrete cosine transform (DCT) image.
  30. 30. The article of claim 29, wherein the instructions, when executed, further result in the DCT image being stored as a DCT image that complies with an MPEG specification.
  31. 31. The article of claim 28, wherein the instructions, when executed, further result in: displaying the downsampled spatial image so that resulting non-uniform vertical spacing of data signal lines in the downsampled spatial image appear substantially uniform on a computer monitor.
  32. 32. An article comprising: a storage medium, having stored thereon instructions, that when executed by a platform, result in the following:
    inverse transforming a compressed video image;
    downsamping the inverse transformed image in the spatial domain; and
    performing motion compensation for the downsampled image in the spatial domain.
  33. 33. The article of claim 32, wherein the instructions, when executed further result in the compressed video image in the frequency domain comprising a discrete cosine transform (DCT) image.
  34. 34. The article of claim 33, wherein the instructions, when executed, further result in the DCT image being stored as a DCT image that complies with an MPEG specification.
US09470741 1999-12-20 1999-12-20 Method and apparatus for performing video image decoding Abandoned US20030043918A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US09470741 US20030043918A1 (en) 1999-12-20 1999-12-20 Method and apparatus for performing video image decoding

Applications Claiming Priority (7)

Application Number Priority Date Filing Date Title
US09470741 US20030043918A1 (en) 1999-12-20 1999-12-20 Method and apparatus for performing video image decoding
AU2619001A AU2619001A (en) 1999-12-20 2000-10-25 Method and apparatus for performing video image decoding
CN 00819034 CN1236619C (en) 1999-12-20 2000-10-25 Method and apparatus for performing video image decoding
EP20000989714 EP1240785A1 (en) 1999-12-20 2000-10-25 Method and apparatus for performing video image decoding
PCT/US2000/041603 WO2001047280A1 (en) 1999-12-20 2000-10-25 Method and apparatus for performing video image decoding
TW89125882A TW502541B (en) 1999-12-20 2000-12-05 Method and apparatus for performing video image decoding
US10379187 US7263231B2 (en) 1999-12-20 2003-03-03 Method and apparatus for performing video image decoding

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US10379187 Continuation US7263231B2 (en) 1999-12-20 2003-03-03 Method and apparatus for performing video image decoding

Publications (1)

Publication Number Publication Date
US20030043918A1 true true US20030043918A1 (en) 2003-03-06

Family

ID=23868846

Family Applications (2)

Application Number Title Priority Date Filing Date
US09470741 Abandoned US20030043918A1 (en) 1999-12-20 1999-12-20 Method and apparatus for performing video image decoding
US10379187 Active 2021-08-05 US7263231B2 (en) 1999-12-20 2003-03-03 Method and apparatus for performing video image decoding

Family Applications After (1)

Application Number Title Priority Date Filing Date
US10379187 Active 2021-08-05 US7263231B2 (en) 1999-12-20 2003-03-03 Method and apparatus for performing video image decoding

Country Status (4)

Country Link
US (2) US20030043918A1 (en)
EP (1) EP1240785A1 (en)
CN (1) CN1236619C (en)
WO (1) WO2001047280A1 (en)

Cited By (21)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20010022817A1 (en) * 2000-03-08 2001-09-20 Toru Yamada Image data processing apparatus and motion compensation processing method used therefor, and recording medium
US20030185301A1 (en) * 2002-04-02 2003-10-02 Abrams Thomas Algie Video appliance
US20040101056A1 (en) * 2001-02-05 2004-05-27 Wong Daniel W. Programmable shader-based motion compensation apparatus and method
US20070160142A1 (en) * 2002-04-02 2007-07-12 Microsoft Corporation Camera and/or Camera Converter
US20090049112A1 (en) * 2006-02-22 2009-02-19 The University Of Akron Interleaved method for parallel implementation of the fast fourier transform
US20090323825A1 (en) * 2006-11-10 2009-12-31 Arthur Mitchell Processing of Sub-Sampled Images
US20100142924A1 (en) * 2008-11-18 2010-06-10 Panasonic Corporation Playback apparatus, playback method, and program for performing stereoscopic playback
US20110211815A1 (en) * 2008-11-18 2011-09-01 Panasonic Corporation Reproduction device, reproduction method, and program for steroscopic reproduction
US20120082205A1 (en) * 2010-09-30 2012-04-05 Alcatel-Lucent Usa Inc. Apparatus and method for generating compressive measurements of video using spatial and temporal integration
US20120082208A1 (en) * 2010-09-30 2012-04-05 Alcatel-Lucent Usa Inc. Method and apparatus for arbitrary resolution video coding using compressive sampling measurements
US20130129326A1 (en) * 2010-08-04 2013-05-23 Nxp B.V. Video player
US20140269895A1 (en) * 2013-03-13 2014-09-18 Qualcomm Incorporated Integrated downscale in video core
US8929456B2 (en) 2010-09-30 2015-01-06 Alcatel Lucent Video coding using compressive measurements
US20150312592A1 (en) * 2013-01-10 2015-10-29 Huawei Technologies Co., Ltd. Methods and apparatuses for coding and decoding depth map
US9319578B2 (en) 2012-10-24 2016-04-19 Alcatel Lucent Resolution and focus enhancement
US9344736B2 (en) 2010-09-30 2016-05-17 Alcatel Lucent Systems and methods for compressive sense imaging
US9398310B2 (en) 2011-07-14 2016-07-19 Alcatel Lucent Method and apparatus for super-resolution video coding using compressive sampling measurements
US9563806B2 (en) 2013-12-20 2017-02-07 Alcatel Lucent Methods and apparatuses for detecting anomalies using transform based compressed sensing matrices
US9600899B2 (en) 2013-12-20 2017-03-21 Alcatel Lucent Methods and apparatuses for detecting anomalies in the compressed sensing domain
US9894324B2 (en) 2014-07-15 2018-02-13 Alcatel-Lucent Usa Inc. Method and system for modifying compressive sensing block sizes for video monitoring using distance information
US10063883B2 (en) 2013-01-10 2018-08-28 Huawei Technologies Co., Ltd. Methods and apparatuses for coding and decoding depth map

Families Citing this family (19)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7215708B2 (en) * 2001-05-22 2007-05-08 Koninklijke Philips Electronics N.V. Resolution downscaling of video images
US6868124B2 (en) * 2001-06-18 2005-03-15 Webtv Networks Inc. (Microsoft) Method and systems for compressing a video stream with minimal loss after subsampled decoding
US7088776B2 (en) 2002-07-15 2006-08-08 Apple Computer, Inc. Method and apparatus for variable accuracy inter-picture timing specification for digital video encoding
US6728315B2 (en) 2002-07-24 2004-04-27 Apple Computer, Inc. Method and apparatus for variable accuracy inter-picture timing specification for digital video encoding with reduced requirements for division operations
CN1947146B (en) 2004-04-27 2010-06-16 Nxp股份有限公司 Method of down-sampling data values
US20060062311A1 (en) * 2004-09-20 2006-03-23 Sharp Laboratories Of America, Inc. Graceful degradation of loop filter for real-time video decoder
KR100722972B1 (en) 2004-10-26 2007-05-30 삼성전자주식회사 Device and method for processing image signal in digital broadcasting receiver
US7986846B2 (en) 2004-10-26 2011-07-26 Samsung Electronics Co., Ltd Apparatus and method for processing an image signal in a digital broadcast receiver
JP2006174415A (en) * 2004-11-19 2006-06-29 Ntt Docomo Inc Image decoding apparatus, image decoding program, image decoding method, image encoding apparatus, image encoding program, and image encoding method
US20060285590A1 (en) * 2005-06-21 2006-12-21 Docomo Communications Laboratories Usa, Inc. Nonlinear, prediction filter for hybrid video compression
US8848793B2 (en) * 2007-10-31 2014-09-30 Broadcom Corporation Method and system for video compression with integrated picture rate up-conversion
US8542732B1 (en) * 2008-12-23 2013-09-24 Elemental Technologies, Inc. Video encoder using GPU
CN101848383A (en) * 2009-03-24 2010-09-29 虹软(上海)科技有限公司 Downsampling decoding method for MPEG2-format video
US8264611B2 (en) * 2010-03-27 2012-09-11 Newport Media, Inc. System and method for video downscaling
CN103098464A (en) * 2010-08-04 2013-05-08 Nxp股份有限公司 Video decoder with down-sampler in the frequency domain
US8472736B2 (en) 2010-09-30 2013-06-25 The Charles Stark Draper Laboratory, Inc. Attitude estimation by reducing noise with dragback
US8472737B2 (en) 2010-09-30 2013-06-25 The Charles Stark Draper Laboratory, Inc. Attitude estimation in compressed domain
US8472735B2 (en) * 2010-09-30 2013-06-25 The Charles Stark Draper Laboratory, Inc. Attitude estimation with compressive sampling of starfield data
CN103108175A (en) * 2011-11-09 2013-05-15 深圳市德赛微电子技术有限公司 Motion compensation structure in multimode video decoder

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5262854A (en) * 1992-02-21 1993-11-16 Rca Thomson Licensing Corporation Lower resolution HDTV receivers
US5614957A (en) * 1994-10-11 1997-03-25 Hitachi America, Ltd. Digital picture-in-picture decoder
US5673358A (en) * 1993-01-13 1997-09-30 Hitachi America, Ltd. Method for generating a reduced rate digital bitstream by selecting data as a function of the position within an image to which the data corresponds
US5684539A (en) * 1994-09-26 1997-11-04 Hitachi America, Ltd. Method and apparatus for processing encoded video data to reduce the amount of data used to represent a video image

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP3022713B2 (en) * 1993-09-30 2000-03-21 日本ビクター株式会社 Image signal processing method
US5835160A (en) * 1995-09-13 1998-11-10 Oak Technology, Inc. Sampling rate conversion using digital differential analyzers
US6175592B1 (en) * 1997-03-12 2001-01-16 Matsushita Electric Industrial Co., Ltd. Frequency domain filtering for down conversion of a DCT encoded picture
US6215822B1 (en) * 1997-12-30 2001-04-10 Sony Corporation Motion compensated digital video decoding and buffer memory addressing therefor
US6222550B1 (en) * 1998-12-17 2001-04-24 Neomagic Corp. Multiple triangle pixel-pipelines with span-range pixel interlock for processing separate non-overlapping triangles for superscalar 3D graphics engine

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5262854A (en) * 1992-02-21 1993-11-16 Rca Thomson Licensing Corporation Lower resolution HDTV receivers
US5673358A (en) * 1993-01-13 1997-09-30 Hitachi America, Ltd. Method for generating a reduced rate digital bitstream by selecting data as a function of the position within an image to which the data corresponds
US5684539A (en) * 1994-09-26 1997-11-04 Hitachi America, Ltd. Method and apparatus for processing encoded video data to reduce the amount of data used to represent a video image
US5614957A (en) * 1994-10-11 1997-03-25 Hitachi America, Ltd. Digital picture-in-picture decoder
US5614952A (en) * 1994-10-11 1997-03-25 Hitachi America, Ltd. Digital video decoder for decoding digital high definition and/or digital standard definition television signals
US5635985A (en) * 1994-10-11 1997-06-03 Hitachi America, Ltd. Low cost joint HD/SD television decoder methods and apparatus

Cited By (30)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20010022817A1 (en) * 2000-03-08 2001-09-20 Toru Yamada Image data processing apparatus and motion compensation processing method used therefor, and recording medium
US6831951B2 (en) * 2000-03-08 2004-12-14 Nec Corporation Image data processing apparatus and motion compensation processing method used therefor, and recording medium
US20040101056A1 (en) * 2001-02-05 2004-05-27 Wong Daniel W. Programmable shader-based motion compensation apparatus and method
US7885336B2 (en) * 2001-02-05 2011-02-08 Ati Technologies Ulc Programmable shader-based motion compensation apparatus and method
US20030185301A1 (en) * 2002-04-02 2003-10-02 Abrams Thomas Algie Video appliance
US20070160142A1 (en) * 2002-04-02 2007-07-12 Microsoft Corporation Camera and/or Camera Converter
US20090049112A1 (en) * 2006-02-22 2009-02-19 The University Of Akron Interleaved method for parallel implementation of the fast fourier transform
US8606839B2 (en) * 2006-02-22 2013-12-10 The University Of Akron Interleaved method for parallel implementation of the fast fourier transform
US20090323825A1 (en) * 2006-11-10 2009-12-31 Arthur Mitchell Processing of Sub-Sampled Images
US20110211815A1 (en) * 2008-11-18 2011-09-01 Panasonic Corporation Reproduction device, reproduction method, and program for steroscopic reproduction
US20100142924A1 (en) * 2008-11-18 2010-06-10 Panasonic Corporation Playback apparatus, playback method, and program for performing stereoscopic playback
US8301013B2 (en) 2008-11-18 2012-10-30 Panasonic Corporation Reproduction device, reproduction method, and program for stereoscopic reproduction
US8335425B2 (en) 2008-11-18 2012-12-18 Panasonic Corporation Playback apparatus, playback method, and program for performing stereoscopic playback
US20130129326A1 (en) * 2010-08-04 2013-05-23 Nxp B.V. Video player
US9634690B2 (en) * 2010-09-30 2017-04-25 Alcatel Lucent Method and apparatus for arbitrary resolution video coding using compressive sampling measurements
US20120082208A1 (en) * 2010-09-30 2012-04-05 Alcatel-Lucent Usa Inc. Method and apparatus for arbitrary resolution video coding using compressive sampling measurements
US8644376B2 (en) * 2010-09-30 2014-02-04 Alcatel Lucent Apparatus and method for generating compressive measurements of video using spatial and temporal integration
US9344736B2 (en) 2010-09-30 2016-05-17 Alcatel Lucent Systems and methods for compressive sense imaging
US8929456B2 (en) 2010-09-30 2015-01-06 Alcatel Lucent Video coding using compressive measurements
US20120082205A1 (en) * 2010-09-30 2012-04-05 Alcatel-Lucent Usa Inc. Apparatus and method for generating compressive measurements of video using spatial and temporal integration
US9398310B2 (en) 2011-07-14 2016-07-19 Alcatel Lucent Method and apparatus for super-resolution video coding using compressive sampling measurements
US9319578B2 (en) 2012-10-24 2016-04-19 Alcatel Lucent Resolution and focus enhancement
US10034021B2 (en) * 2013-01-10 2018-07-24 Huawei Technologies Co., Ltd. Methods and apparatuses for coding and decoding depth map
US20150312592A1 (en) * 2013-01-10 2015-10-29 Huawei Technologies Co., Ltd. Methods and apparatuses for coding and decoding depth map
US10063883B2 (en) 2013-01-10 2018-08-28 Huawei Technologies Co., Ltd. Methods and apparatuses for coding and decoding depth map
US9432614B2 (en) * 2013-03-13 2016-08-30 Qualcomm Incorporated Integrated downscale in video core
US20140269895A1 (en) * 2013-03-13 2014-09-18 Qualcomm Incorporated Integrated downscale in video core
US9563806B2 (en) 2013-12-20 2017-02-07 Alcatel Lucent Methods and apparatuses for detecting anomalies using transform based compressed sensing matrices
US9600899B2 (en) 2013-12-20 2017-03-21 Alcatel Lucent Methods and apparatuses for detecting anomalies in the compressed sensing domain
US9894324B2 (en) 2014-07-15 2018-02-13 Alcatel-Lucent Usa Inc. Method and system for modifying compressive sensing block sizes for video monitoring using distance information

Also Published As

Publication number Publication date Type
CN1435054A (en) 2003-08-06 application
CN1236619C (en) 2006-01-11 grant
US20030198392A1 (en) 2003-10-23 application
EP1240785A1 (en) 2002-09-18 application
US7263231B2 (en) 2007-08-28 grant
WO2001047280A1 (en) 2001-06-28 application

Similar Documents

Publication Publication Date Title
US5784114A (en) Motion compensated video processing
US6850571B2 (en) Systems and methods for MPEG subsample decoding
US6263119B1 (en) Method and apparatus for resizing images using the discrete trigonometric transform
US6445828B1 (en) Transform domain resizing of an image compressed with field encoded blocks
US6650781B2 (en) Image decoder
US7110459B2 (en) Approximate bicubic filter
US6243421B1 (en) Apparatus for decoding coded video data with reduced memory size
Strobach Tree-structured scene adaptive coder
Park et al. L/M-fold image resizing in block-DCT domain using symmetric convolution
US6385248B1 (en) Methods and apparatus for processing luminance and chrominance image data
US6690427B2 (en) Method and system for de-interlacing/re-interlacing video on a display device on a computer system during operation thereof
US7075993B2 (en) Correction system and method for enhancing digital video
US6539058B1 (en) Methods and apparatus for reducing drift due to averaging in reduced resolution video decoders
RU2302707C2 (en) Dynamic encoding filters
US6650708B1 (en) Video signal encoding apparatus
US20040120401A1 (en) Motion estimation engine with parallel interpolation and search hardware
US20090238535A1 (en) Merging video with time-decimated high-resolution imagery to form high-resolution video frames
EP0632657A1 (en) Method of prediction of a video image
US6690838B2 (en) Image processing circuit and method for reducing a difference between pixel values across an image boundary
US20030202607A1 (en) Sub-pixel interpolation in motion estimation and compensation
US20060146935A1 (en) Method and apparatus for sub-pixel motion compensation
US6628714B1 (en) Down converting MPEG encoded high definition sequences to lower resolution with reduced memory in decoder loop
US6823014B2 (en) Video decoder with down conversion function and method for decoding video signal
US5973755A (en) Video encoder and decoder using bilinear motion compensation and lapped orthogonal transforms
US6289132B1 (en) Apparatus and method for optimized compression of interlaced motion images

Legal Events

Date Code Title Description
AS Assignment

Owner name: INTEL CORPORATION, CALIFORNIA

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:JIANG, HONG H.;SIMON, ALLEN H.;COOK, VAL G.;REEL/FRAME:010682/0453;SIGNING DATES FROM 20000214 TO 20000222