JP2008125078A - Motion compensation method and apparatus supporting multi-codec - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a motion compensation method and apparatus supporting multi-codec. <P>SOLUTION: The motion compensation method comprises the steps of: using at least one of a plurality of methods for interpolating pixels of a reference image corresponding to a current image, namely, vertical linear filtering, horizontal linear filtering and cubic filtering in accordance with codec information representing any one of a plurality of codecs such as MPEG4, H.264/AVC and VC1, to calculate a value of a pixel positioned between pixels of the reference image from values of the pixels of the reference image; and restoring the current image by adding motion compensation data including the calculated pixel value to a difference between the reference image and the current image. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a method and apparatus for decoding a moving image, and more particularly, to a method and apparatus for compensating for motion.

  Decoding a moving image goes through various processes. Such a process includes entropy decoding, inverse transform, inverse quantization, and motion compensation. Of these, motion compensation takes the longest processing time. Motion compensation refers to a motion vector that represents a displacement between a block of a current image and a corresponding block of a reference image, extracts a portion shared between the reference image and the current image from the reference image, Means the process of restoring the current image by adding the extracted data (hereinafter referred to as “motion compensation data”) and the difference data between the reference image and the current image (hereinafter referred to as “residue data”). To do. In this way, the motion compensation process requires a large amount of memory access and computation because a lot of image data must be processed. Since the amount of processing for such motion compensation is proportional to the size of the image, if the size of the image increases, in the worst case, the entire image is not processed within the desired time, resulting in natural video. Reproduction may be impossible.

  Most video codec engineers prefer to implement video codecs in software form. Compared to the case where the video codec is implemented in hardware, the development period is shortened, the cost is reduced, and the video codec can be easily reused. In particular, the video codec itself has excellent flexibility. This is because when an error occurs in the video codec, this can be easily corrected. However, the heart of video codecs is real-time processing. Even if the implementation of the software has many advantages, it will be useless if the video codec cannot process the encoding and decoding of images in real time. On the other hand, if the video codec is implemented in hardware, the development period will be long, the cost will increase, the flexibility will be low, and the video codec's overall structure will change if the video codec needs to be modified It can be dangerous. However, above all, when the video codec is implemented in hardware, it can be optimized for the target, so the video encoding and decoding speed is dramatically faster than when processing with software. There are advantages.

  Due to the conflicting advantages and disadvantages of the hardware implementation method and the software implementation method as described above, most video codec engineers mix and use these two methods. That is, the modules constituting the moving image codec are implemented by dividing each part into a part to be processed by S / W and a part to be processed by H / W. In this case, motion compensation is an essential part that must be implemented in hardware. The amount of data computation required for motion compensation is large, but above all, memory access is much larger than other parts, so software processing is incompatible.

  Recently, an important issue for implementing a video codec is to enable a variety of video formats to be played back by a single video player. Video formats that are mainly used are MPEG4, VC1, and H.264 / AVC. However, when implementing these three codecs on a single device, if the motion compensation for each codec is implemented, the amount of data computation and memory access will greatly increase when processed by software. Even when implemented in hardware, there is a problem in that the size and manufacturing cost of hardware increase.

  The technical problem to be solved by the present invention is that the playback of moving images can be processed in real time while supporting multi-codecs to reduce the amount of data calculation and memory access, and reduce the size and production cost of hardware It is another object of the present invention to provide an apparatus and method for reducing the chip size and reducing the chip manufacturing cost for motion compensation. Another object of the present invention is to provide a computer-readable recording medium on which a program for causing the computer to execute the method is recorded.

  The technical problem to be solved by the present invention is not limited to the technical problem as described above, and other technical problems may exist. This can be clearly understood by those skilled in the art from the following description.

  The motion compensation method according to the present invention for solving the above-described problem is: (a) Among a plurality of methods for interpolating pixels of a reference image corresponding to a current image by codec information representing any one of a plurality of codecs. Calculating a value of a pixel located between the pixels from a value of a pixel of the reference image using at least one method; and (b) motion compensation data including the calculated pixel value; Restoring the current image by adding a difference between a reference image and the current image.

  In the step (a), (a1) a value between pixels positioned on any one vertical line of any one block of the reference image is selectively determined based on the codec information. Calculating a value of a pixel located at a position between (a2) a pixel value located on any one horizontal line of the block and a pixel located on the horizontal line according to the codec information. A step of calculating a value of a pixel located; and (a3) a pixel located between pixels located in the cubic from a value of a pixel located in any one of the blocks selectively according to the codec information Calculating a value of.

  In order to solve the above-mentioned problems, the present invention provides a computer-readable recording medium on which a program for causing a computer to execute the motion compensation method is recorded.

  A motion compensation apparatus according to the present invention for solving the above-described problem is provided by at least one of a plurality of methods for interpolating pixels of a reference image corresponding to a current image by codec information representing any one of a plurality of codecs. A motion compensation engine that calculates a value of a pixel located between the pixels from a value of a pixel of the reference image using a method; motion compensation data including the calculated pixel value; the reference image; An adder for restoring the current image by adding the difference with the current image.

  The motion compensation engine may be configured to select a value of a pixel located between a value of a pixel located on any one vertical line of any one block of the reference image and a pixel located on the vertical line according to the codec information. And a horizontal filter that calculates a value of a pixel located between pixels located on the horizontal line from a value of a pixel located on any one horizontal line of the block selectively according to the codec information And a cubic filter that calculates a value of a pixel located between pixels located in the cubic from a value of a pixel located in any one of the blocks selectively according to the codec information.

  According to the present invention, a common codec among a plurality of codecs such as vertical filtering, horizontal filtering, cubic filtering, etc., according to codec information representing any one of a plurality of codecs such as MPEG4, H.264 / AVC, and VC1. By selectively performing this part, the chip size can be minimized by implementing the motion compensation supporting the multi-codec in hardware. In particular, as described above, by modularizing so that common parts can be selectively performed, implementation in hardware is facilitated, and at the same time, the number of parts required for the chip is reduced, and the manufacturing cost of the chip is reduced. Decreases.

  In addition, according to the present invention, vertical filtering, horizontal filtering, and cubic filtering are performed in a pipeline form during a period of waiting to receive data from the external memory and a period of receiving data from the external memory. The compensation processing time is minimized, so that a DMB playback device supporting multi-codec can be easily manufactured by encoding and decoding images in real time while supporting multi-codec.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 1 is a table showing characteristics of a filter for motion compensation used in each of various video compression codecs. In order to restore a higher resolution image, the current video compression codec has a motion vector that does not indicate only pixels in the reference image, i.e., integer pixels, but 1/2 pixels located in 1/2 between integer pixels. Alternatively, a ¼ pixel located at ¼ between integer pixels is also indicated. However, since the ½ pixel and the ¼ pixel are pixels that do not originally exist in the reference image, the value of this pixel is generated using a pixel interpolation method. FIG. 1 shows the characteristics of a filter used for such pixel interpolation for each moving image compression codec.

  Referring to FIG. 1, the chrominance components of MPEG4, H.264 / AVC, and VC1 all use a cubic filter, and VC1 has both a linear filter and a linear filter. use. Regarding luminance components, MPEG4 uses a cubic filter, H.264 / AVC uses a linear filter, and VC1 uses both a cubic filter and a linear filter. I understand. Hereinafter, the form of pixel interpolation by MPEG4, H.264 / AVC, and VC1 will be described in detail with reference to FIGS. 2A to 2E.

  2A is a diagram illustrating a form of pixel interpolation of luminance components according to H.264 / AVC. The uppercase alphabet shown in FIG. 2A represents an integer pixel, and the lowercase alphabet represents a subpixel. The integer pixel means an original pixel used for pixel interpolation, and the sub-pixel means a pixel having a value generated by pixel interpolation using the integer pixel, that is, 1/2 pixel or 1/4 pixel. . Referring to FIG. 2A, pixel interpolation for luminance components of H.264 / AVC is performed up to 1/4 pixel unit. First, the 1/2 pixel shown in FIG. 2A, that is, the pixel at the b, h, m, and s positions is calculated using a 6-tap filter having 6 coefficients according to the following equation (1). Is done.

b1 = (E-5 * F + 20 * G + 20 * H-5 * I + J)
h1 = (A-5 * C + 20 * G + 20 * M-5 * R + T)
b = Clip1Y ((b1 + 16) >> 5)
h = Clip1Y ((h1 + 16) >> 5) (1)
j1 = cc-5 * dd + 20 * h1 + 20 * m1-5 * ee + ff, or j1 = aa-5 * bb + 20 * b1 + 20 * s1-5 * gg + hh
j = Clip1Y ((j1 + 512) >> 10)
s = Clip1Y ((s1 + 16) >> 5)
m = Clip1Y ((m1 + 16) >> 5)
In the above formula, “*” represents a multiplication operation. The same applies to the following.

  Next, the ¼ pixel shown in FIG. 2A, that is, the remaining pixels are calculated using an average filter according to the following equation (2). This average filter is a kind of 2-tap filter having two coefficients, and the same applies to the following.

a = (G + b + 1) >> 1
c = (H + b + 1) >> 1
d = (G + h + 1) >> 1
n = (M + h + 1) >> 1
f = (b + j + 1) >> 1
i = (h + j + 1) >> 1
k = (j + m + 1) >> 1 (2)
q = (j + s + 1) >> 1
e = (b + h + 1) >> 1
g = (b + m + 1) >> 1
p = (h + s + 1) >> 1
r = (m + s + 1) >> 1
In the above formulas (1) and (2), the notation “>>” means a right shift operation, and the number after “>>” represents how many bits are shifted. For example, “>> 1” means 1-bit shift, which is equivalent to bisection. The same applies to the following.

  2B is a diagram illustrating a state of pixel interpolation of color difference components by H.264 / AVC. P1, P2, P3, and P4 shown in FIG. 2B represent integer pixels, and the convergence points of the four arrows represent sub-pixels. Referring to FIG. 2B, pixel interpolation for the color difference component of H.264 / AVC is performed at an arbitrary position, not a fixed position. That is, a pixel at an arbitrary position shown in FIG. 2B is calculated using a cubic filter according to the following equation (3). This cubic filter is a kind of 4-tap filter having four coefficients, and the same applies to the following.

P = ((8−xFrc) * (8−yFrc) * P1 + xFrc * (8−yFrc) * P2 + (8−xFrc) * yFrc * P3 + xFrc * yFrc * P4 + 32) >> 6 (3)
FIG. 2C is a diagram illustrating a form of pixel interpolation using a linear filter by VC1. P1, P2, P3, and P4 shown in FIG. 2C represent integer pixels, and points located between the integer pixels P2 and P3 represent sub-pixels. Referring to FIG. 2C, pixel interpolation using a linear filter by VC1 is performed up to 1/4 pixel unit. That is, the ½ pixel and ¼ pixel corresponding to the black dots shown in FIG. 2C are calculated using a cubic filter according to the following equation (4).

P = (a * P1 + b * P2 + c * P3 + d * P4 + rc) >> s (4)
In the above equation (4), the notations “a”, “b”, “c”, and “d” are the coefficients of the four taps of the cubic filter, and their values depend on the position of the sub-pixel. It is determined. Also, the value of the notation “s” is determined by the position of the sub-pixel.

  FIG. 2D is a diagram showing a state of pixel interpolation by MPEG4. P1, P2, P3, and P4 shown in FIG. 2D represent integer pixels, and black dots positioned between the integer pixels represent sub-pixels. Referring to FIG. 2D, pixel interpolation for MPEG4 luminance and chrominance components is performed up to ½ pixel unit. For example, if the position of P1 is dx = 0 and dy = 0, the ½ pixel corresponding to the black point shown in FIG. 2C is calculated using a cubic filter according to the following equation (5). Is done.

dx = 1, dy = 0: P = (P1 + P2 + 1−rc) >> 1
dx = 0, dy = 1: P = (P1 + P3 + 1−rc) >> 1 (5)
dx = 1, dy = 1: P = (P1 + P2 + P3 + P4-rc) >> 1
In the above formula (5), the notation “rc” means rounding control and has 0 or 1. The same applies to the following.

  FIG. 2E is a diagram illustrating a state of pixel interpolation using a cubic filter by VC1. P1, P2, P3, and P4 shown in FIG. 2E represent integer pixels, and the convergence points of the four arrows represent sub-pixels. Referring to FIG. 2E, pixel interpolation for the color difference component of H.264 / AVC is performed at an arbitrary position that is not a fixed position. That is, a pixel at an arbitrary position shown in FIG. 2B is calculated using a cubic filter according to the following equation (6).

P = (a * P1 + b * P2 + c * P3 + d * P4 + rc) >> s (6)
In the above equation (6), the notations “a”, “b”, “c”, and “d” are the coefficients of the four taps of the cubic filter, and the values depend on the position of the sub-pixel. It is determined. Also, the value of the notation “s” is determined by the position of the sub-pixel.

  As described above, the number of taps and the coefficient values of the filters used in MPEG4, H.264 / AVC, and VC1 are different from each other, but such coefficient values can be adjusted. By setting it to 0, the number of taps can be adjusted. For example, the 6-tap filter is used as a 4-tap filter by setting the coefficient value of 2 taps out of 6 taps constituting the 6-tap filter to 0. Further, by setting the coefficient value of two taps out of four taps constituting the 4-tap filter to 0, this 4-tap filter is used as a 2-tap filter, that is, an average filter. Therefore, it can be seen that there is a part that can be mutually shared between the filters used in MPEG4, H.264 / AVC, and VC1, and if this is used, any of the above three codecs (CODEC) can be used. However, motion compensation can be efficiently processed using a smaller number of filters.

  Among the above three codecs, H.264 / AVC interpolates pixels in the most complicated manner. However, since H.264 / AVC uses a linear filtering 6-tap filter for the luminance component and a cubic filtering 4-tap filter for the chrominance component, these are MPEG4 and VC1. This is used for the cubic filtering type 4-tap filter and the VC1 linear filtering type 4-tap filter. Therefore, the pixel interpolation method of MPEG4 and VC1 can be said to be a subset of the pixel interpolation method of H.264 / AVC.

  In other words, one linear filter having six taps can be selected from the MPEG4 linear filter, the H.264 / AVC linear filter, and the VC1 linear filter by adjusting the coefficient value of each of the six taps. Also used for. In addition, a cubic filter having four taps is adjusted to each of the MPEG4 cubic filter, the H.264 / AVC cubic filter, and the VC1 cubic filter by adjusting the coefficient values of the four taps. used. However, the luminance component pixel interpolation process according to H.264 / AVC requires an average filtering process according to Equation (2).

  However, each of the ¼ pixels calculated by the equation (2) differs in surrounding pixels used at the time of pixel interpolation depending on the position. That is, the values of the pixels at the e, g, p, and r positions are calculated using the pixel values at the diamond positions centered on the respective pixels, and the values of the pixels at the f, i, k, and g positions are It is calculated using the pixel value at the cross position centered on each pixel. As described above, the 1/4 pixel interpolation process of H.264 / AVC is a complicated filtering process according to the position of the pixel, so that it was not easy to be implemented as hardware.

  FIG. 3 is a diagram illustrating a cubic filter according to an embodiment of the present invention. Referring to FIG. 3, in the present embodiment, the H.264 / AVC 1/4 pixel interpolation process as described above is applied to a cubic filter as shown in FIG. If the cubic filter shown in FIG. 3 is used, the H.264 / AVC 1/4 pixel interpolation process is performed through a common module called a cubic filter.

  In general, a cubic filter uses the value of an integer pixel located at each of the cubic vertices, and uses the value of a half pixel located at the center of each of the cubic sides and the center of the cubic. The value of the located half pixel (Half pel) is calculated. In the present embodiment, the calculation corresponding to the equation (2) is processed using such a characteristic of the cubic filter. For example, the value of the quarter pixel at position a (Quater pel) is calculated by applying the value of the integer pixel at position G and the value of the half pixel at position b to the cubic filter shown in FIG. Is done. Those skilled in the art will understand that the values of ¼ pixels at other positions are also calculated as shown in FIG.

  FIG. 4 is a block diagram of a multi-codec assisted motion compensation apparatus according to an embodiment of the present invention. Referring to FIG. 4, the multi-codec support motion compensation apparatus according to the present embodiment includes a slave interface (slave I / F) 10, a master interface (master I / F) 20, a DMA (Direct Memory Access) I / F 30, The motion compensation engine 40, a first memory 50, a second memory 60, and an adder 70 are included.

  The slave I / F 10 is transmitted from an external video encoding device (not shown) via a system bus of a video decoding device (not shown) including the multi-codec support motion compensation device according to the present embodiment. The residue data is fetched, and the fetched residue data is output to the second memory 60. Also, the slave I / F 10 fetches an MC (Motion Compensation) parameter, which is a parameter necessary for motion compensation, from the external video encoding device via the system bus, and uses the fetched MC parameter as a master I / F Output to 20.

  The master I / F 20 interprets the MC parameters input from the slave I / F 10, selects the MC parameters based on the interpretation results, and outputs them to the DMA I / F 30 and the motion compensation engine 40. , The DMA module 31 constituting the DMA I / F 30 and the first-in first-out (FIFO) module 32 are controlled, and the vertical filter 41, the first horizontal filter 42, the first horizontal filter 42 constituting the motion compensation engine 40 are controlled. The respective operations of the two horizontal filter 43, the cubic alignment unit 44, and the cubic filter 45 are controlled.

  The first memory 50 stores motion compensation data generated by the motion compensation engine 40, and the second memory 60 stores residue data input from the slave I / F 10. In general, the first memory 50 and the second memory 60 are implemented by SRAM (Static Random Access Memory). The adder 70 restores the current image by adding the motion compensation data stored in the first memory 50 and the residue data stored in the second memory 60, and the restored current image data is DMA I / F Output to 30.

  The DMA I / F 30 includes a DMA module 31 and a FIFO module 32. The DMA module 31 refers to the motion vector representing the displacement between the block of the current image and the corresponding block of the reference image among the MC parameters input from the master I / F 20, and externally passes through the system bus. A 32-bit unit data of the reference image shared between the reference image and the current image is fetched from a memory (not shown), and the current image restored by the adder 70 is stored in the external memory.

  The FIFO module 32 extracts 32-bit unit data fetched by the DMA module 31 in 8-bit units and sequentially outputs them to the motion compensation engine 40. In particular, the FIFO module 32, when there is no data indicated by a motion vector in the reference image, other than 8-bit unpacking that outputs only data actually required for motion compensation to the motion compensation engine 40, this motion vector. Also perform padding with.

  The motion compensation engine 40 performs motion compensation on 8-bit data input from the DMA I / F 30. More specifically, the motion compensation engine 40 outputs codec information representing one of a plurality of codecs in the MC parameter input from the master I / F 20 and one of the plurality of color components. A plurality of methods for interpolating pixels of a reference image corresponding to the current image by filter information representing any one of a plurality of filters and color information representing at least one of vertical linear filtering, horizontal linear filtering, and cubic filtering Using one method, the value of the pixel located between the pixels is calculated from the value of the pixel of the reference image.

  Referring to FIG. 4, the motion compensation engine 40 further includes a vertical filter 41, a first horizontal filter 42, a second horizontal filter 43, a cubic alignment unit 44, and a cubic filter 45.

  The vertical filter 41 includes codec information representing one of a plurality of codecs in the MC parameter input from the master I / F 20, color information representing one of a plurality of color components, and a plurality of Based on the filter information representing one of the filters, the vertical pixel value is calculated from the value of the integer pixel located on any one vertical line of any block of the reference image selectively input from the DMA I / F 30. The value of 1/2 pixel located between the integer pixels located on the line is calculated. In particular, the vertical filter 41 is implemented as a linear filter having six taps, and the value of the integer pixel located on the vertical line is applied to the six taps, thereby being input from the master I / F 20. Of the MC parameters, the value of ½ pixel indicated by the motion vector of the block is calculated.

  The first horizontal filter 42 is a block of any one of reference images selectively input from the DMA I / F 30 according to codec information, color information, and filter information among the MC parameters input from the master I / F 20. From the value of the integer pixel located on any one of the horizontal lines, the value of 1/2 pixel located between the integer pixels located on the horizontal line is calculated. In particular, the first horizontal filter 42 is implemented as a linear filter having six taps, and is applied from the master I / F 20 by applying the value of the pixel located on the horizontal line to the six taps. Among the MC parameters, the value of 1/2 pixel indicated by the motion vector of the block is calculated.

  The second horizontal filter 43 is another horizontal line that is not a horizontal line that is selectively processed by the first horizontal filter 42 according to codec information, color information, and filter information among the MC parameters input from the master I / F 20. From the value of the integer pixel or 1/2 pixel located on the line, the value of 1/2 pixel located between the integer pixels located on the horizontal line is calculated. Here, 1/2 pixel is calculated from surrounding integer pixels. In particular, the second horizontal filter 43 is implemented as a linear filter having six taps, and the value of an integer pixel or ½ pixel located on the horizontal line is applied to the six taps, thereby obtaining a master I / F Calculates the value of 1/2 pixel indicated by the motion vector of the block among the MC parameters input from 20.

  The cubic alignment unit 44 selectively selects 1/2 pixel, 1 / C, which is indicated by the motion vector of the block according to codec information, color information, and filter information, among MC parameters input from the master I / F 20. The value of the integer pixel or the half pixel input from the DMA I / F 30 is aligned with the cubic position required to generate the value of the pixel of 4 pixels or an arbitrary position.

  The cubic filter 45 is selectively aligned by the cubic alignment unit 44 according to the codec information, color information, and filter information among the MC parameters input from the master I / F 20 as shown in FIG. From the values of the four integer pixels, the values of the ½ pixel, the ¼ pixel, or the pixel at an arbitrary position located between the aligned pixels are calculated. The cubic filter 45 is selectively input from the first horizontal filter 42 and the second horizontal filter 43 according to codec information, color information, and filter information among the MC parameters input from the master I / F 20. From the value of two integer pixels or 1/2 pixel located in any one of the luminance component blocks, 1/2 pixel or 1/4 pixel located between the two pixels located in this cubic Is calculated. In particular, the cubic filter 45 is implemented by a cubic filter having four taps, and the value of two or four pixels at the cubic position is converted into two taps or four taps corresponding to the cubic position. By applying, the values of 1/2 pixel, 1/4 pixel, and a pixel at an arbitrary position indicated by the motion vector of the block among the MC parameters input from the master I / F 20 are calculated.

  Each of the vertical filter 41, the first horizontal filter 42, the second horizontal filter 43, the cubic alignment unit 44, and the cubic filter 45 does not operate on all the pixels, but operates on the value of the motion vector. That is, when the motion vector indicates the position of an integer pixel among the MC parameters input from the master I / F 20, the motion between the reference screen and the prediction screen accurately corresponds to a multiple of the integer pixel interval. Therefore, since the value of the integer pixel constituting the reference screen becomes the value of the pixel constituting the prediction screen, the vertical filter 41, the first horizontal filter 42, the second horizontal filter 43, the cubic alignment unit 44, and the cubic Pixel value interpolation by the filter 45 is unnecessary. However, when the motion vector indicates the position of 1/2 pixel or 1/4 pixel among the MC parameters input from the master I / F 20, the motion between the reference screen and the prediction screen is exactly an integer. Since this is a case that does not correspond to a multiple of the pixel interval, the value interpolated using the values of the integer pixels that constitute the reference screen becomes the value of the pixels that constitute the prediction screen. 42, pixel value interpolation by the second horizontal filter 43, the cubic alignment unit 44, and the cubic filter 45 is necessary.

  If pixel value interpolation by the vertical filter 41, the first horizontal filter 42, the second horizontal filter 43, the cubic alignment unit 44, and the cubic filter 45 is performed on the entire reference image, the vertical filter 41, the first horizontal filter 42, the first 2 The value of the integer pixel that has been passed through without being processed by the horizontal filter 43 and the cubic filter 45, the value of 1/2 pixel, the value of 1/4 pixel, and the value of the pixel at an arbitrary position calculated by these filters Is completed as motion compensation data of the current image corresponding to the reference image.

  5 to 7 are diagrams illustrating operation scenarios for each codec in the motion compensation apparatus illustrated in FIG. 5 to 7, the luminance component of H.264 / AVC and VC1 are subjected to horizontal filtering and vertical filtering to perform linear filtering, and in other cases, cubic filtering through cubic alignment. I do.

  5 is a diagram illustrating an operation scenario of the H.264 / AVC linear filter and the cubic filter in the motion compensation apparatus illustrated in FIG. Referring to FIG. 5, the motion compensator shown in FIG. 4 performs H.264 / AVC luminance component filtering according to the following operation scenario.

  The DMA I / F 30 refers to the motion vector representing the displacement between the block of the current image and the corresponding block of the reference image among the MC parameters input from the master I / F 20, and passes through the system bus. Then, the luminance component of the reference image is fetched from the external memory in units of blocks of a predetermined size, and the fetched luminance component block is output to the vertical filter 41.

  Of the MC parameters input from the master I / F 20, the vertical filter 41 indicates that the codec information represents H.264 / AVC, the color information represents a luminance component, and the filter information represents a linear filter. From the value of the integer pixel located on any one vertical line among the integer pixels constituting the luminance component block input from F30, the half pixel located between the integer pixels located on this vertical line Calculate the value. In particular, the vertical filter 41 applies the value of the integer pixel located on the vertical line to six taps, so that the motion vector of the block among the MC parameters input from the master I / F 20 is The value of the designated 1/2 pixel is calculated, and the calculated value of 1/2 pixel is output to the first horizontal filter 42 or the second horizontal filter 43. If the motion vector of the block indicates an integer pixel, the vertical filter 41 directly uses the value of the integer pixel constituting the luminance component block input from the DMA I / F 30 as it is. Output to the filter 43.

  For example, if the position of the pixel indicated by the motion vector is h shown in FIG. 2A, the vertical filter 41 uses six taps of integer pixel values at positions A, C, G, M, R, and T. To calculate the half pixel value at the h position by calculating h1 = (A-5 * C + 20 * G + 20 * M-5 * R + T), h = Clip1Y ((h1 + 16) >> 5) To do.

  Of the MC parameters input from the master I / F 20, the first horizontal filter 42 is vertical if codec information represents H.264 / AVC, color information represents a luminance component, and filter information represents a linear filter. From the value of the integer pixel positioned on any one horizontal line among the integer pixels constituting the luminance component block input from the DMA I / F 30 via the filter 41, the integer pixel positioned on this horizontal line is calculated. The value of 1/2 pixel located between them is calculated. In particular, the first horizontal filter 42 applies the value of the integer pixel located on the horizontal line to six taps, so that the motion of the block among the MC parameters input from the master I / F 20 is changed. The half pixel value indicated by the vector is calculated, and the calculated half pixel value is output to the cubic filter 45. Further, if the motion vector of the block indicates an integer pixel, the first horizontal filter 42 outputs the value of the integer pixel constituting the luminance component block input from the DMA I / F 30 to the cubic filter 45 as it is. To do. Further, the first horizontal filter 42 outputs the half-pixel value input from the vertical filter 41 to the cubic filter 45 as it is.

  For example, if the position of the pixel indicated by the motion vector is b shown in FIG. 2, the first horizontal filter 42 has six integer pixel values at positions E, F, G, H, I, and J. By applying b1 = (E−5 * F + 20 * G + 20 * H−5 * I + J) and b = Clip1Y ((b1 + 16) >> 5) calculate.

  Of the MC parameters input from the master I / F 20, the second horizontal filter 43 is vertical if codec information represents H.264 / AVC, color information represents a luminance component, and filter information represents a linear filter. Among the integer pixels constituting the luminance component block input from the DMA I / F 30 via the filter 41, this value is obtained from the values of the integer pixels or 1/2 pixels located on the other horizontal lines that are not the horizontal lines. A value of 1/2 pixel located between an integer pixel or 1/2 pixel located on the horizontal line is calculated. In particular, the second horizontal filter 43 applies the value of the integer pixel or ½ pixel located on the horizontal line to six taps, so that among the MC parameters input from the master I / F 20, The value of ½ pixel indicated by the motion vector of this block is calculated, and the calculated value of ½ pixel is output to the cubic filter 45. In addition, if the motion vector of the block indicates an integer pixel, the second horizontal filter 43 outputs the value of the integer pixel constituting the luminance component block input from the DMA I / F 30 to the cubic filter 45 as it is. To do. In addition, the second horizontal filter 43 outputs the value of ½ pixel input from the vertical filter 41 to the cubic filter 45 as it is.

  For example, if the position of the pixel indicated by the motion vector is j shown in FIG. 2, the second horizontal filter 43 sets the half pixel value at the cc, dd, h1, m1, ee, and ff positions. Apply to 6 taps and calculate j1 = cc-5 * dd + 20 * h1 + 20 * m1-5 * ee + ff, j = Clip1Y ((j1 + 512) >> 10), the value of 1/2 pixel at j position Is calculated. In the above equation (2), in order to obtain the value of a certain ¼ pixel as in f = (b + j + 1) >> 1, the value of two ½ pixels may be required at the same time. . In such a case, the first horizontal filter 42 and the second horizontal filter are used to simultaneously process the process of calculating the value of 1/2 pixel at the b position and the process of calculating the value of 1/2 pixel at the j position. 43 exists separately.

  In the cubic filter 45, if the codec information represents H.264 / AVC, the color information represents the luminance component, and the filter information represents the linear filter among the MC parameters input from the master I / F 20, the first horizontal It is located between two pixels located in this cubic from the value of an integer pixel or a half pixel located in any one of the luminance component blocks input from the filter 42 and the second horizontal filter 43. A 1/4 pixel value is calculated. In particular, the cubic filter 45 is input from the master I / F 20 by applying the values of the two pixels at the cubic position to two taps corresponding to the cubic position among the four taps. A value of ¼ pixel indicated by the motion vector of the block is calculated from the MC parameters thus calculated, and the calculated value of ¼ pixel is output to the first memory 50. Further, the cubic filter 45 outputs the value of the integer pixel or ½ pixel input from the first horizontal filter 42 or the second horizontal filter to the first memory 50 as it is.

  For example, if the position of the pixel indicated by the motion vector is a shown in FIG. 2, the cubic filter 45 uses four taps for the integer pixel value at the G position and the ½ pixel value at the b position. Is applied to the two taps corresponding to the cubic position, and a = (G + b + 1) >> 1 is calculated to calculate the value of the ¼ pixel at the a position.

  Referring to FIG. 5, the motion compensator shown in FIG. 4 performs H.264 / AVC color difference component Cb filtering according to the following operation scenario.

  The DMA I / F 30 refers to the motion vector representing the displacement between the block of the current image and the corresponding block of the reference image among the MC parameters input from the master I / F 20, and passes through the system bus. Then, the color difference component Cb of the reference image is fetched from the external memory in units of blocks of a predetermined size, and the fetched block of the color difference component Cb is output to the cubic alignment unit 44.

  Of the MC parameters input from the master I / F 20, the cubic alignment unit 44, if the codec information represents H.264 / AVC, the color information represents the color difference component Cb, and the filter information represents a cubic filter, An integer constituting a block of the color difference component Cb input from the DMA I / F 30 at a cubic position required to generate a pixel value at an arbitrary position indicated by the motion vector input from the I / F 20 Four of the pixel values are aligned, and the four aligned integer pixel values are output to the cubic filter 45.

  In the cubic filter 45, among the MC parameters input from the master I / F 20, if the codec information represents H.264 / AVC, the color information represents the color difference component Cb, and the filter information represents a cubic filter, FIG. As described above, from the values of the four integer pixels aligned by the cubic alignment unit 44, the values of the pixels at arbitrary positions located between the aligned pixels are calculated. In particular, the cubic filter 45 applies the values of the four pixels at the cubic position to four taps corresponding to the cubic position, so that the block of the MC parameters input from the master I / F 20 is the block. The value of ¼ pixel indicated by the motion vector is calculated, and the calculated value of ¼ pixel is output to the first memory 50.

  For example, if the position of the pixel indicated by the motion vector is the convergence point of the four arrows shown in FIG. 2B, the cubic filter 45 converts the integer pixel values at the positions P1, P2, P3, and P4 to four values. Apply to tap and calculate P = ((8−xFrc) * (8−yFrc) * P1 + xFrc * (8−yFrc) * P2 + (8−xFrc) * yFrc * P3 + xFrc * yFrc * P4 + 32) >> 6 Thus, the value of the pixel at an arbitrary position is calculated.

  Referring to FIG. 5, each of the DMA I / F 30, the cubic alignment unit 44, and the cubic filter 45 operates in the same manner as the operation scenario of the color difference component Cb with respect to the color difference component Cr. Therefore, detailed description of the operation scenario of the color difference component Cb filtering of H.264 / AVC is omitted.

  FIG. 6 is a diagram illustrating an operation scenario of the linear filter of VC1 in the motion compensation apparatus illustrated in FIG. Referring to FIG. 6, the motion compensation apparatus shown in FIG. 4 performs luminance component filtering of VC1 according to the following operation scenario.

  The DMA I / F 30 refers to the motion vector representing the displacement between the block of the current image and the corresponding block of the reference image among the MC parameters input from the master I / F 20, and passes through the system bus. Then, the luminance component of the reference image is fetched from the external memory in units of blocks of a predetermined size, and the fetched luminance component block is output to the vertical filter 41.

  Of the MC parameters input from the master I / F 20, the vertical filter 41 is input from the DMA I / F 30 if the codec information indicates V1, the color information indicates a luminance component, and the filter information indicates a linear filter. From the value of the integer pixel located on any one vertical line among the integer pixels constituting the luminance component block, 1/2 pixel or 1/4 pixel located between the integer pixels located on this vertical line Is calculated. In particular, the vertical filter 41 applies the value of the integer pixel positioned on the vertical line to four taps, so that the motion vector of the block is designated among the MC parameters input from the master I / F 20. The value of 1/2 pixel or 1/4 pixel to be calculated is calculated, and the calculated value of 1/2 pixel or 1/4 pixel is output to the first horizontal filter 42. The vertical filter 41 can calculate the value of 1/2 pixel or 1/4 pixel by adjusting the coefficient value of the 4-tap filter. If the motion vector of the block indicates an integer pixel, the vertical filter 41 outputs the value of the integer pixel constituting the luminance component block input from the DMA I / F 30 to the first horizontal filter 42 as it is. .

  In the embodiment shown in FIG. 5, a 6-tap filter is used as the vertical filter 41 and the first horizontal filter 42. The 6-tap filter is used as a 4-tap filter without changing the hardware structure without using two of the 6 taps. Therefore, in the embodiment shown in FIG. The one horizontal filter 42 can be a 4-tap filter.

  Of the MC parameters input from the master I / F 20, the first horizontal filter 42 passes through the vertical filter 41 if the codec information represents V 1, the color information represents a luminance component, and the filter information represents a linear filter. Then, from the integer pixel value positioned on any one horizontal line among the integer pixels constituting the luminance component block input from the DMA I / F 30, 1 positioned between the integer pixels positioned on this horizontal line The value of / 2 pixel or 1/4 pixel is calculated. In particular, the first horizontal filter 42 applies the integer pixel value positioned on the horizontal line to four taps, so that the motion vector of the block is designated among the MC parameters input from the master I / F 20. The value of 1/2 pixel or 1/4 pixel is calculated, and the calculated value of 1/2 pixel or 1/4 pixel is output to the cubic filter 45. Further, the first horizontal filter 42, if the motion vector of the block indicates an integer pixel, the value of the integer pixel constituting the luminance component block input from the DMA I / F 30 via the vertical filter 41. The data is output to the first memory 50 as it is. Further, the first horizontal filter 42 outputs the value of 1/2 pixel or 1/4 pixel input from the vertical filter 41 to the first memory 50 as it is.

  For example, if the position of the pixel indicated by the motion vector is the black dot shown in FIG. 2C, the first horizontal filter 42 applies the integer pixel values at positions P1, P2, P3, and P4 to four taps. Then, by calculating P = (a * P1 + b * P2 + c * P3 + d * P4 + rc) >> s, the value of 1/2 pixel or 1/4 pixel corresponding to the black point shown in FIG. 2C is calculated. A specific example of the vertical filter 41 is not shown, but is the same as described above except that the horizontal filtering is changed to the vertical filtering. In addition, referring to FIG. 6, each of the DMA I / F 30, the vertical filter 41, and the second horizontal filter 43 operates in the same manner as the operation scenario of the luminance component with respect to the color difference components Cb and Cr.

  In the process of compensating for the luminance component of the reference image as a VC1 linear filter, there is no case where two horizontal filters have to be used at the same time. Therefore, only one of the horizontal filters may be used. In particular, in the present embodiment, the first horizontal filter 42 is used out of the first horizontal filter 42 and the second horizontal filter 43. However, those skilled in the art may use the second horizontal filter 43. I understand.

  FIG. 7 is a diagram showing an operation scenario of the cubic filter of MPEG4 and VC1 in the motion compensation apparatus shown in FIG. Referring to FIG. 7, the motion compensation apparatus shown in FIG. 4 performs luminance component filtering of MPEG4 and VC1 according to the following operation scenario.

  The DMA I / F 30 refers to the motion vector representing the displacement between the block of the current image and the corresponding block of the reference image among the MC parameters input from the master I / F 20, and passes through the system bus. Then, the color difference component Cb of the reference image is fetched from the external memory in units of blocks of a predetermined size, and the fetched luminance component block is output to the cubic alignment unit 44.

  If the codec information represents MPEG4 or VC1, the color information represents the luminance component, and the filter information represents the cubic filter among the MC parameters input from the master I / F 20, the cubic alignment unit 44 The block of the luminance component input from the DMA I / F 30 is configured at a cubic position required for generating a value of a half pixel indicated by the motion vector input from 20 or a pixel at an arbitrary position. Four of the integer pixel values are aligned, and the aligned four integer pixel values are output to the cubic filter 45.

  The cubic filter 45 is shown in FIG. 3 when codec information represents MPEG4 or VC1, color information represents a luminance component, and filter information represents a cubic filter among MC parameters input from the master I / F 20. As a result, from the values of the four integer pixels aligned by the cubic alignment unit 44, the values of the ½ pixel positioned between the aligned pixels or the pixel at an arbitrary position are calculated. In particular, the cubic filter 45 applies the values of the four pixels at the cubic position to the four taps corresponding to the cubic position, so that among the MC parameters input from the master I / F 20, the block filter 45 The value of the ½ pixel indicated by the motion vector or a pixel at an arbitrary position is calculated, and the calculated value of the ½ pixel or the pixel at an arbitrary position is output to the first memory 50.

  For example, if the pixel position indicated by the motion vector is the black dot shown in FIG. 2D, the cubic filter 45 applies the integer pixel values at the positions P1, P2, P3, and P4 to four taps. , P = (P1 + P2 + 1−rc) >> 1, P = (P1 + P3 + 1−rc) >> 1, P = (P1 + P2 + P3 + P4-rc) >> 1 and corresponding to the black point shown in FIG. 2D The pixel value is calculated. If the position of the pixel indicated by the motion vector is the convergence point of the four arrows shown in FIG. 2E, the cubic filter 45 converts the integer pixel values at the positions P1, P2, P3, and P4 to four values. The value of the pixel at an arbitrary position is calculated by calculating P = (a * P1 + b * P2 + c * P3 + d * P4 + rc) >> s by applying to the tap.

  Referring to FIG. 7, each of the DMA I / F 30, the cubic alignment unit 44, and the cubic filter 45 operates in the same manner as the operation scenario of the luminance component with respect to the color difference components Cb and Cr.

  In particular, this embodiment adopts a register array system. When performing both horizontal filtering by the first horizontal filter 42 or the second horizontal filter 43 and vertical filtering by the vertical filter 41, the result value of horizontal filtering is used in vertical filtering, or the result value of vertical filtering is set horizontally. Used for filtering.

  FIG. 8 is a diagram showing a register array chain inside the vertical filter 41 shown in FIG. Referring to FIG. 8, the register array chain in the vertical filter 41 shown in FIG. 4 is the number of pixels constituting any one line of any one block in the case of H.264 / AVC. Is composed of 21 8-bit registers. In the case of a codec other than H.264 / AVC, the number of pixels constituting any one line of any one block is less than 21. In particular, since H.264 / AVC uses a 6-tap filter, five such register array chains are required and are connected in series. The output value of any one of the register array chains becomes the input value of another register array chain connected thereto.

  The register array chain inside the vertical filter 41 shown in FIG. 4 receives the value of one pixel corresponding to 8 bits per clock from the DMA I / F 30, and stores the input pixel value in the R0 register. At the same time, the pixel values currently stored in each of the 21 registers are shifted one by one to the right, thereby sending out the pixel values stored in the R20 register.

In particular, the register array chain inside the vertical filter 41 shown in FIG. 4 is 8 × 1 for each clock when the position of the pixel indicated by each motion vector of H.264 / AVC and VC1 is an integer pixel.
Depending on the block size, one of the pixel value stored in the R3 register, the pixel value stored in the R7 register, and the pixel value stored in the R15 register is output through the MUX. For example, when the motion compensation engine 40 performs motion compensation in units of 4 × 4 blocks, the register array chain inside the vertical filter 41 shown in FIG. 4 outputs the pixel value stored in the R3 register, and 8 When motion compensation is performed in units of × 8 blocks, the pixel value stored in the R7 register is output. When motion compensation is performed in units of 16 × 16 blocks, the pixel value stored in the R15 register is output. Output.

  If the position of the pixel indicated by the H.264 / AVC motion vector is not an integer pixel, the register array chain in the vertical filter 41 shown in FIG. 4 is stored in the R8 register according to the block size. One of the value, the pixel value stored in the R12 register, and the pixel value stored in the R20 register is output. For example, the register array chain inside the vertical filter 41 shown in FIG. 4 outputs the pixel value stored in the R8 register when the motion compensation engine 40 performs motion compensation in units of 4 × 4 blocks. When the motion compensation is performed in units of 8 × 8 blocks, the pixel value stored in the R12 register is output. When the motion compensation is performed in units of 16 × 16 blocks, the pixel stored in the R20 register is output. Output the value.

  If the position of the pixel indicated by the motion vector in the VC1 codec is not an integer pixel, the register array chain in the vertical filter 41 shown in FIG. 4 has an output value of the R10 register and an R18 register depending on the block size. Output one of the output values. For example, the register array chain in the vertical filter 41 shown in FIG. 4 outputs the pixel value stored in the R10 register when motion compensation is performed in units of 8 × 8 blocks by the motion compensation engine 40. When the motion compensation is performed in units of 16 × 16 blocks, the pixel value stored in the R20 register is output. In the case of VC1, 4 × 4 block mode is not supported.

  FIG. 9 is a configuration diagram of a 6-tap filter inside the vertical filter 41 shown in FIG. Referring to FIG. 9, the 6-tap filter inside the vertical filter 41 shown in FIG. 4 receives the output value of the register array chain shown in FIG. That is, the 6-tap filter inside the vertical filter 41 shown in FIG. 4 outputs six pixel values output and accumulated during the six clocks from the register array chain shown in FIG. From the five integer pixel values stored in the registers 1 to 5 and one pixel value that is currently input, the value of the ½ pixel located between the pixels is calculated. Since an output value from one register array chain is stored in one register of a 6-tap filter, five register array chains are required corresponding to five registers of the 6-tap filter. That is, the 6-tap filter shown in FIG. 9 has nine operations to calculate the operation shown in the equation (1), for example, b1 = (E-5 * F + 20 * G + 20 * H-5 * I + J). Register, four multipliers, and five adders. The two input terminals of the leftmost adder shown in FIG. 9 and the input terminals of the four multipliers correspond to six taps.

  FIG. 10 is a timing diagram for data processing inside the motion compensation apparatus shown in FIG. Referring to FIG. 10, after requesting the external memory to transmit 32-bit unit data corresponding to the reference image, a data latency period for waiting to receive this data and the 32-bit unit from the external memory. The DMA I / F 30, the vertical filter 41, the first horizontal filter 42, the second horizontal filter 43, and the cubic filter 45 perform their respective operations in a pipeline form during the data transmission period in which the data is received. . In general, since the data latency period and the data transmission period are very long, the filtering operations of the vertical filter 41, the first horizontal filter 42, the second horizontal filter 43, and the cubic filter 45 are performed hidden in this period. Thereby, motion compensation can be processed at higher speed.

  FIG. 11 is a flowchart illustrating a multi-codec assisted motion compensation method according to an exemplary embodiment of the present invention. Referring to FIG. 11, the multi-codec support motion compensation method according to the present embodiment includes steps processed in a time series by the multi-codec support motion compensation device shown in FIG. Therefore, even if the contents are omitted below, the contents described above for the multi-codec support motion compensation apparatus shown in FIG. 11 are also applied to the multi-codec support motion compensation method according to the present embodiment.

  In step 100, the multi-codec assisted motion compensation device receives from the external encoding device via the system bus the residue data corresponding to the difference between the reference image and the current image and the MC parameters necessary for motion compensation. Fetch.

  In step 200, the multi-codec assisted motion compensation apparatus stores the residue data fetched in step 100.

  In step 300, the multi-codec assisted motion compensator refers to a motion vector representing a displacement between the block of the current image and the corresponding block of the reference image among the MC parameters fetched in step 100, and determines the system bus. Via the external memory, the 32-bit unit data of the reference image shared between the reference image and the current image is fetched.

  In step 400, the multi-codec support motion compensation apparatus performs 8-bit unpacking and padding using a motion vector on the 32-bit unit data fetched in step 300.

  In step 500, the multi-codec assisted motion compensation device selects codec information representing one of a plurality of codecs and a color representing one of a plurality of color components from among the MC parameters fetched in step 100. Integer pixels located in any one vertical line of any one block of the reference image selectively corresponding to the processing result in step 400 according to the information and filter information representing any one of the plurality of filters From the value of ½, the value of ½ pixel located between the integer pixels located on this vertical line is calculated.

  More specifically, in step 500, the multi-codec assisted motion compensation apparatus determines that the codec information represents H.264 / AVC, the color information represents a luminance component, and the filter information represents a linear filter, or the codec information. Represents VC1, color information represents a luminance component or color difference component, and filter information represents a linear filter, any one vertical line of any one block of the reference image corresponding to the processing result in step 400 From the value of the integer pixel located at, the value of ½ pixel located between the integer pixels located on this vertical line is calculated. For the remaining cases, the vertical filtering in step 500 as described above is not performed.

  In step 600, the multi-codec support motion compensation apparatus selectively selects one of the reference images corresponding to the processing result in step 400 according to the codec information, color information, and filter information among the MC parameters fetched in step 100. From the value of the integer pixel located on any one horizontal line of the one block, the value of 1/2 pixel or 1/4 pixel located between the integer pixels located on this horizontal line is calculated.

  More specifically, in step 600, the multi-codec assisted motion compensation apparatus determines that the codec information represents H.264 / AVC, the color information represents a luminance component, and the filter information represents a linear filter, or the codec information. Represents VC1, color information represents a luminance component or color difference component, and filter information represents a linear filter, any one vertical line of any one block of the reference image corresponding to the processing result in step 400 From the value of the integer pixel located at, the value of 1/2 pixel or 1/4 pixel located between the integer pixels located on this vertical line is calculated. For the remaining cases, the vertical filtering in step 600 as described above is not performed.

  In step 700, the multi-codec assisted motion compensation apparatus selects other horizontal lines that are not processed in step 600 according to codec information, color information, and filter information from among the MC parameters fetched in step 100. From the value of the integer pixel or 1/2 pixel located at, the value of the 1/2 pixel located between the integer pixel or 1/2 pixel located on this horizontal line is calculated.

  More specifically, in step 700, the multi-codec assisted motion compensation apparatus determines in step 600 that the codec information represents H.264 / AVC, the color information represents a luminance component, and the filter information represents a linear filter. From the value of the integer pixel or 1/2 pixel located on another horizontal line that is not the horizontal line to be processed, the value of 1/2 pixel located between the integer pixel or 1/2 pixel located on this horizontal line is calculated To do. For the remaining cases, the vertical filtering in step 700 as described above is not performed.

  In step 800, the multi-codec assisted motion compensation apparatus is selectively located in any one of the luminance component blocks according to the codec information, color information, and filter information among the MC parameters fetched in step 100. From the integer pixel or the value of ½ pixel calculated in step 600 and step 700, the value of ¼ pixel located between the two pixels located in the cubic is calculated.

  More specifically, in step 800, the multi-codec assisted motion compensation apparatus determines that the luminance component of the luminance component when the codec information represents H.264 / AVC, the color information represents a luminance component, and the filter information represents a linear filter. From the integer pixel located in any one cubic block of the block or the value of ½ pixel calculated in step 600 and step 700, the value of ¼ pixel located between the two pixels located in this cubic Is calculated. For the remaining cases, the cubic filtering in step 800 as described above is not performed.

  In step 900, the multi-codec assisted motion compensation apparatus selectively selects a ½ pixel indicated by the motion vector of the block according to codec information, color information, and filter information from among the MC parameters fetched in step 100. The value of the integer pixel corresponding to the processing result in step 400 is aligned with the cubic position required to generate the value of 1/4 pixel or a pixel at an arbitrary position.

  More specifically, in step 900, the multi-codec assisted motion compensation apparatus determines that the codec information is MPEG4 if the codec information represents H.264 / AVC, the color information represents a color difference component, and the filter information represents a cubic filter. When color information represents a luminance component or color difference component and filter information represents a cubic filter, or codec information represents VC1, color information represents a luminance component or color difference component, and filter information represents a cubic filter Further, an integer corresponding to the processing result in step 400 at a cubic position required to generate a value of 1/2 pixel, 1/4 pixel, or a pixel at an arbitrary position indicated by the motion vector of the block Align pixel values. For the remaining cases, the cubic alignment in step 900 as described above is not performed.

  In step 1000, the multi-codec assisted motion compensator, from the values of the four integer pixels aligned in step 900, ½ pixel located between the aligned pixels, as shown in FIG. , 1/4 pixel, or the value of a pixel at an arbitrary position is calculated.

  In step 1100, the multi-codec support motion compensation apparatus checks whether the process from step 300 to step 1000 is completed for the entire reference image. If the process is completed, the process proceeds to step 1200. If not completed, the process returns to step 300.

  In step 1200, the multi-codec assisted motion compensation apparatus stores motion compensation data generated through steps 300 to 1000.

  In step 1300, the multi-codec assisted motion compensation apparatus restores the current image by adding the motion compensated data stored in step 1200 and the residue data stored in step 200.

  On the other hand, the embodiment of the present invention described above can be created by a program that can be executed by a computer, and is embodied by a general-purpose digital computer that operates the program using a computer-readable recording medium. Further, the data structure used in the above-described embodiment of the present invention is recorded through various means on a computer-readable recording medium. The computer-readable recording medium is a magnetic storage medium (for example, a ROM (Read Only Memory), a floppy (registered trademark) disk, a hard disk, etc.), or an optically readable medium (for example, a CD-ROM, DVD, etc.). Including such storage media.

  In the above, this invention was demonstrated centering on the desirable embodiment. Those skilled in the art will appreciate that the present invention may be embodied in variations that do not depart from the essential characteristics of the invention. Accordingly, the disclosed embodiments should be considered in an illustrative, not a limiting sense. The scope of the present invention is shown not in the above description but in the claims, and all differences within the equivalent scope should be construed as being included in the present invention.

  The present invention can be applied to a device for decoding a moving image, such as a digital TV, a computer, a mobile phone, and the like.

It is a figure which shows the characteristic of the filter for the motion compensation used with each of various moving image compression codecs in a table | surface. It is a figure which shows the mode of the pixel interpolation of the luminance component by H.264 / AVC. It is a figure which shows the mode of the pixel interpolation of the color difference component by H.264 / AVC. It is a figure which shows the mode of the pixel interpolation which uses a linear filter by VC1. It is a figure which shows the mode of the pixel interpolation by MPEG4. It is a figure which shows the mode of the pixel interpolation which uses a cubic filter by VC1. It is a figure which shows the cubic filter by one Embodiment of this invention. 1 is a configuration diagram of a multi-codec support motion compensation apparatus according to an embodiment of the present invention. FIG. 5 is a diagram illustrating an operation scenario of an H.264 / AVC linear filter and a cubic filter in the motion compensation apparatus shown in FIG. It is a figure which shows the operation | movement scenario of the linear filter of VC1 in the motion compensation apparatus shown by FIG. FIG. 5 is a diagram showing an operation scenario of an MPEG4 and VC1 cubic filter in the motion compensation apparatus shown in FIG. 4. FIG. 5 is a diagram showing a register array chain inside the vertical filter shown in FIG. 4. FIG. 5 is a configuration diagram of a 6-tap filter inside the vertical filter shown in FIG. 4. FIG. 5 is a timing chart regarding data processing inside the motion compensation device shown in FIG. 4. 3 is a flowchart of a multi-codec assisted motion compensation method according to an exemplary embodiment of the present invention.

Explanation of symbols

10 Slave I / F
20 Master I / F
30 DMA I / F
31 DMA module 32 FIFO module 40 motion compensation engine 41 vertical filter 42 first horizontal filter 43 second horizontal filter 44 cubic alignment unit 45 cubic filter 50 first memory 60 second memory 70 adder

Claims (19)

(A) By using at least one method among a plurality of methods for interpolating pixels of a reference image corresponding to the current image by codec information representing any one of a plurality of codecs, Calculating a value of a pixel located between the pixels from the value;
(B) motion compensation data including the calculated pixel value, and restoring the current image by adding a difference between the reference image and the current image. . The step (a) includes:
(A1) Based on the codec information, a value of a pixel positioned between pixels positioned on the vertical line is calculated from a value of a pixel positioned on any one vertical line of any one block of the reference image. And steps to
(A2) calculating a value of a pixel positioned between pixels positioned on the horizontal line from a value of a pixel positioned selectively on any one horizontal line of the block according to the codec information;
(A3) calculating a value of a pixel located between the pixels located in the cubic from a value of a pixel located in any one of the blocks selectively according to the codec information. The motion compensation method according to claim 1, wherein:   In each of the (a1) step, the (a2) step, and the (a3) step, a pixel value is selectively calculated based on the codec information and color information representing one of a plurality of color components. The motion compensation method according to claim 2, wherein:   If the codec information represents H.264 / AVC and the color information represents a luminance component, each of the (a1) step, the (a2) step, and the (a3) step calculates a pixel value. The motion compensation method according to claim 3, wherein: The step (a1) calculates values of pixels positioned between pixels positioned on the vertical line by applying the values of pixels positioned on the vertical line to six taps.
The step (a2) calculates the values of pixels positioned between the pixels positioned on the horizontal line by applying the values of the pixels positioned on the horizontal line to six taps.
In the step (a3), the value of the pixel located between the pixels located in the cubic is obtained by applying the value of the two pixels located in the cubic to the two taps among the four taps. The motion compensation method according to claim 4, wherein the motion compensation method is calculated.   If the codec information represents H.264 / AVC and the color information represents a color difference component, the steps (a1) and (a2) do not calculate pixel values, and the step (c) The motion compensation method according to claim 3, wherein a pixel value is calculated.   Each of the (a1) step, the (a2) step, and the (a3) step is selectively performed on the pixel according to filter information that represents one of the codec information, the color information, and a plurality of filters. 4. The motion compensation method according to claim 3, wherein a value is calculated.   If the codec information represents VC1, the color information represents a luminance component or a color difference component, and the filter information represents a linear filter, the steps (a1) and (a2) calculate pixel values, The motion compensation method according to claim 7, wherein the step (a3) does not calculate a pixel value.   If the codec information represents VC1, the color information represents a luminance component or a color difference component, and the filter information represents a cubic filter, the step (a1) and the step (a2) do not calculate a pixel value. The motion compensation method according to claim 7, wherein the step (a3) calculates a pixel value.   The motion compensation method according to claim 1, wherein the step (a) calculates a value of a pixel indicated by a motion vector of any one block of the reference image.   The method further includes a step of calculating a value of a pixel positioned between pixels positioned on the other horizontal line from a value of a pixel positioned on another horizontal line that is not the horizontal line selectively according to the codec information. The motion compensation method according to claim 2. Aligning the pixel values to the cubic positions required to selectively generate pixel values indicated by the block motion vector according to the codec information;
The motion compensation of claim 2, further comprising: calculating values of pixels positioned between the aligned pixels from the values of the aligned pixels selectively according to the codec information. Method. The step (a1) calculates a value of a half pixel located between pixels located on the vertical line from a value of an integer pixel located on the vertical line,
The step (a2) calculates a value of a half pixel located between pixels located on the horizontal line from a value of an integer pixel located on the horizontal line,
The step (a3) includes any one integer pixel constituting the block, a value of 1/2 pixel calculated in the step (a), and a value of 1/2 pixel calculated in the step (b). The motion compensation method according to claim 2, further comprising: calculating a value of a ¼ pixel located between the pixels located in the cubic. A step of selectively outputting a pixel value stored in a register at a predetermined position among a plurality of registers according to a position of a pixel indicated by each motion vector of the codec;
The motion compensation according to claim 1, wherein the step (a) calculates a value of a pixel located between the pixels from a value of the pixels output and accumulated during a predetermined number of clocks. Method.   The motion compensation method according to claim 14, wherein any one of the registers at the predetermined position is output according to a size of the block.   Each of the (a1) step, the (a2) step, and the (a3) step during a period of waiting to receive data corresponding to the image from the external memory and a period of receiving the data from the external memory. The motion compensation method according to claim 2, wherein the motion compensation is performed in a pipeline form. Based on codec information representing any one of a plurality of codecs, using at least one of a plurality of methods for interpolating a pixel of a reference image corresponding to a current image, the pixel value of the reference image Calculating a value of a pixel located between the pixels;
A motion compensation method comprising: calculating motion compensation data including the calculated pixel value; and restoring the current image by adding a difference between the reference image and the current image. A computer-readable recording medium on which a program to be executed on the computer is recorded. Based on codec information representing any one of a plurality of codecs, using at least one of a plurality of methods for interpolating a pixel of a reference image corresponding to a current image, the pixel value of the reference image A motion compensation engine that calculates the values of pixels located between the pixels;
A motion compensation apparatus comprising: motion compensation data including the calculated pixel value; and an adder that restores the current image by adding a difference between the reference image and the current image. The motion compensation engine is:
Vertically calculating the value of pixels located between pixels located on the vertical line from the value of pixels located on any one vertical line of any one block of the reference image according to the codec information Filters,
A horizontal filter for calculating a value of a pixel located between pixels located on the horizontal line from a value of a pixel located on any one horizontal line of the block selectively according to the codec information;
A cubic filter that calculates a value of a pixel located between the pixels located in the cubic from a value of a pixel located in any one of the blocks selectively according to the codec information. The motion compensation apparatus according to claim 18.

Images