JP4847460B2 - Image interpolation method and pixel interpolation device - Google Patents

Description

  The present invention relates to a tap filter process for obtaining an interpolated pixel value of a digital image, and more particularly to a pixel interpolation method used for a moving picture encoding process / decoding process by inter-picture predictive encoding.

  In high-efficiency compression coding of moving images, inter-picture predictive coding using correlation between pictures is used, and MPEG-2, MPEG-4 (Part 2), H.264, and H.264 are used. H.264 | MPEG-4 (Part 10) AVC (hereinafter simply referred to as H.264). In these encoding standards, the current picture is divided into rectangular pixel blocks made up of a plurality of pixels, and processing is performed for each block. In inter-picture predictive coding, a motion vector indicating motion between a block on a current picture and a block on a reference picture to be referred to and a prediction block that has been motion-compensated based on the block on the reference picture. Generate and encode the difference in pixel values between the prediction block and the block on the current picture.

  In order to increase the encoding efficiency by increasing the accuracy of the predicted picture generated by the above-described motion compensation, each component of the motion vector is a unit (1/2 accuracy, 1/4 accuracy, etc.) subdivided from integer accuracy pixels. ). In such motion compensation, when a predicted picture is generated from a reference picture, pixel interpolation processing is required, and the calculation method is defined by each coding standard.

  H. is one of the encoding standards. In H.264, as a method of the above-described pixel interpolation process, the 6-tap filter process is referred to as Reference 1 (Joint Video Team (JVT) of ISO / IEC MPEG & ITU-T VCEG; “Draft ITU-T Recommendation and Final Draft Int. of Joint Video Specification (see ITU-T Rec. H.264 | ISO / IEC 14496-10 AVC) ”, section 8.4.2.2).

  Briefly describing this, this 6-tap filter has tap coefficients {1, -5, 20, 20, -5, 1}, and for pixels lined up on a horizontal or vertical line on the reference picture, When the pixel value of the pixel at the i-th (i: integer) position is expressed as a [i], the pixel value of the interpolation pixel at the (k + 1/2) -th (k: integer) position is as follows: Ask for. First, the intermediate pixel value “b” of the interpolated pixel is obtained according to (Equation 1), and further, normalization and saturation processing are performed according to (Equation 2), and the pixel value “ c ".

  FIG. 8 is a diagram showing the positional relationship between reference picture pixels, that is, integer precision pixels, half precision interpolation pixels, and quarter precision interpolation pixels (Reference 1: FIG. 8-4). ). In the same figure, pixels A to U surrounded by circles are pixels of integer precision, and pixels b, h, j, m, s, and aa to hh surrounded by squares are pixels of 1/2 precision. The pixels a, c, d, e, f, g, i, k, n, p, q, and r surrounded by the remaining squares are 1/4 precision pixels.

  At this time, the pixel values of the pixels b and s are obtained by performing the above-described 6-tap filter processing on the pixel values of the pixels arranged on the horizontal line where the respective pixels are located. The pixel values of the pixels h and m are obtained by performing the above-described 6-tap filter processing on the pixel values of the pixels arranged on the vertical line where the respective pixels are located. The pixel value of the pixel j is obtained by subjecting the pixel values of the six pixels aa, bb, b, s, gg, and hh obtained by the horizontal six-tap filter processing to vertical six-tap filter processing. Required by doing. Alternatively, the pixel value of the pixel j is reversed with respect to the pixel values of the six pixels cc, dd, h, m, ee, and ff obtained by the 6-tap filter process in the vertical direction by reversing the order of the filter direction. It can also be obtained by performing a horizontal 6-tap filter.

  The pixel value of a ¼ precision pixel is determined from the average of two pixel values of adjacent ½ precision pixels or integer precision pixels. As an example, the pixel value of the pixel a is obtained by averaging the pixel value of the integer precision pixel G and the pixel value of the half precision pixel b, and the pixel value of the pixel e is the half precision pixel b. The pixel value of the pixel f is calculated by averaging the pixel value of the pixel h and the pixel value of the pixel h of the half precision. The average is obtained.

  When a predicted picture is generated by interpolation processing using the 6-tap filter as described above, a considerable amount of processing is required to obtain the pixel value of one interpolation pixel. For example, if the interpolated pixel is one of the pixels f, i, j, k, and q, the calculation is performed on the pixel values of 36 integer precision pixels. Such interpolation processing is required for the number of pixels included in the prediction block, and all the blocks that perform inter-picture prediction are targets. Therefore, this motion compensation processing is performed by moving image encoding processing or In the decoding process, the calculation amount is enormous.

  FIG. 9 is a flowchart illustrating an example of a conventional pixel interpolation method for obtaining a pixel value of an interpolation pixel using the above-described 6-tap filter. Hereinafter, the conventional pixel interpolation method shown in FIG. 9 will be described.

  When the pixel interpolation process is started in step S0, initial values are given to the variable “m” and the variable “k” in step S1. Here, the variable “k” is an index for reading the pixel value of the symmetric reference pixel from the array variable a that stores the pixel value of the pixel constituting the reference picture. In correspondence with FIG. 8 described above, for example, in the case of obtaining the pixel value of the interpolated pixel b, the variable “k” indicates the pixel J with integer precision. The variable “m” is an index indicating the position in the prediction block of the interpolated pixel to be obtained. The variable “i” is an index indicating a reference point for reading the pixel value of the reference pixel. And it shows the integer pixel located in the upper left of the interpolation pixel calculated | required initially, and shows the pixel G on FIG.

  In step S2, pixel values of five reference pixels are read from the end of the reference range. This process is a preparation stage for obtaining the pixel value of the first interpolated pixel.

  In step S3, the pixel value of the reference pixel located at the variable “k” is read.

  In step S4, the calculation of (Equation 1) is performed to obtain the intermediate pixel value “b” of the interpolated pixel.

In step S5, normalization and saturation processing of the intermediate pixel value “b” is performed according to (Equation 2) to obtain the pixel value “c” of the interpolated pixel. The normalization is the intermediate pixel value “b” obtained by (Equation 1).
Is divided by the value “32”. The function Clip (x) is a function that suppresses (clips) a range of 0 to 255 when the variable “x” exceeds the range of 0 to 255.

  In step S6, the pixel value “b” of the interpolated pixel obtained in step S5 is output.

  In step S7, the variable “m” and the variable “k” are respectively incremented by the value “1” in preparation for the next loop.

  In step S8, it is determined whether the number of generated interpolation pixels has reached a prescribed number “N”. If the determination result is “No” (the specified number “N” has not been reached), the control is returned to step S3, and steps S3 to S8 are repeated. If the determination result is “Yes” (the predetermined number “N” has been reached), the control is moved to step S9, and the series of pixel interpolation processing is terminated.

  As described above, the pixel value of one interpolated pixel is obtained every time the loop process including steps S3 to S8 in FIG. 9 is performed once.

  FIG. 10 shows a block diagram of a conventional pixel interpolation device using a 6-tap filter. FIG. 10 shows a 6-tap filter used for the interpolation processing described above, based on the configuration of the tap filter disclosed in Document 2 (Japanese Patent Laid-Open No. 7-15734). As shown in FIG. 10, the conventional pixel interpolation device includes registers (abbreviated as “R” in the drawing) 1 to 6, multiplication units (also abbreviated as “M”) 7 to 10, and addition units (also the same). , Abbreviated as “+”) 11, 12 and a clipper 13. The registers 1 to 6 constitute a shift register that receives the pixel value “a [k]” of the input reference pixel.

  The operation of the conventional pixel interpolation device shown in FIG. 10 will be described below.

  The shift register composed of the registers 1 to 6 is shifted by a shift control signal not shown in the drawing, and the pixel value “a [k]” of the input reference pixel is sequentially transferred to each register. Pixel values of six consecutive reference pixels are held. The multipliers 7 to 10 are the pixel values of the respective reference pixels stored in the registers 2 to 5 and the predetermined coefficient shown in the figure (the multiplier 7 and the multiplier 10 are multiplied by the coefficient “−5”. The unit 8 and the multiplication unit 9 perform multiplication with the coefficient “20”) to obtain respective products. The adder 11 adds the value of the register 1, the value of the register 6, and the product of the multipliers 7 to 10 to obtain a sum. The adder 12 adds a constant “16” to the sum obtained by the adder 11, and the clipper 13 performs a saturation process on the addition result and outputs a pixel value “c” of the interpolated pixel. As a result, the pixel value of the reference pixel is sequentially input in accordance with the shift control signal, so that the pixel value “c” of the consecutive interpolated pixels is expressed by the above-described (Equation 1) and (Equation 2). It is obtained by tap filter processing.

  Here, the multipliers 7 to 10 can be configured by bit shift, addition, and sign inversion because the multiplier is a predetermined constant, and the circuit scale is smaller than that of a normal multiplier.

  In the pixel interpolation device to which the conventional pixel interpolation method described above is applied, the calculation amount is constant regardless of the pixel value of the reference pixel, and the calculation amount is not reduced considering the pixel value. There are certain limitations on the speeding up of software processing and the reduction of power consumption in pixel interpolation devices.

Further, as a method of speeding, as Document 3 (Japanese Patent 200 2 -190 948 JP) is disclosed, by using a look-up table for holding a multiplication result, even made an attempt to reduce the computational load However, in this method, there is a problem that the circuit scale increases by providing a lookup table.

  As described above, the conventional pixel interpolation method and pixel interpolation device cannot sufficiently cope with the following problems.

  In other words, in order to respond to the demand for higher definition and increased frame rate for higher quality of moving images, the number of pixels processed per unit time must be increased. There is a certain limit and it is difficult to speed up motion compensation.

In addition, when such a moving image encoding / decoding technique involving pixel interpolation processing is used in a mobile device such as a mobile phone, power is supplied from a battery with limited capacity. Reduction is also an important issue.
JP 7-15734 A JP 2002-190948 A Joint Video Team (JVT) of ISO / IEC MPEG & ITU-T VCEG; ”, 8.4.2.2

  SUMMARY OF THE INVENTION An object of the present invention is to provide an image interpolation method and a pixel interpolation device that enable high-speed pixel interpolation processing and low power consumption.

The pixel interpolation method according to the first aspect of the present invention is a pixel interpolation method by tap filter processing, in which an addition step for obtaining a sum of pixel values of an adjacent pixel pair composed of two reference pixels and a pixel value of the adjacent pixel pair A subtraction step for obtaining a difference between the pixel values, a determination step for determining whether the absolute value of the difference between the pixel values is equal to or less than a specific value, and a first product for multiplying the sum of the pixel values by a first coefficient to obtain a first product. 1 multiplication step. In the determination step, when it is determined that the absolute value of the pixel value difference exceeds the specific value, the second value is obtained by multiplying the difference value of the pixel value by a plurality of second coefficients to obtain a plurality of second products. A first step and a plurality of second products are added to or subtracted from each value of a plurality of registers provided corresponding to each of a plurality of consecutive interpolation pixels. The method further includes a first addition / subtraction step of accumulatively adding values to be the interpolation pixel values of the plurality of interpolation pixels. In the determination step, when it is determined that the absolute value of the difference between the pixel values is equal to or less than the specific value, the first product is added to or subtracted from the corresponding register value of the plurality of registers, It further includes a second addition / subtraction step of accumulating and adding values that are the interpolation pixel values of the plurality of interpolation pixels.

  According to this configuration, when the absolute value of the difference between the pixel values of the adjacent pixel pair is equal to or smaller than a specific value, the processing of the second multiplication step related to the difference between the pixel values of these adjacent pixels and the register It is possible to provide a pixel interpolation method that omits the addition or subtraction process of the second product. Accordingly, the pixel interpolation process can be executed at high speed by the omitted process.

In the pixel interpolation method according to the second invention, an addition step, a subtraction step, a determination step, a first multiplication step, a second multiplication step, a first addition / subtraction step, and a second addition / subtraction step Are repeatedly executed for a plurality of adjacent pixel pairs different from the adjacent pixel pair to obtain respective interpolation pixel values of the plurality of interpolation pixels.

  According to this configuration, each pixel value can be obtained by a series of processes for a plurality of interpolated pixels in the block.

In the pixel interpolation method according to the third aspect of the invention , the specific value is less than or equal to the value indicated by the least significant bit when each pixel value of the adjacent pixel pair is displayed in binary.

  According to this configuration, when the absolute value of the difference between the pixel values of the adjacent pixel pair is equal to or less than the value “0” or the value indicated by the least significant bit when the pixel value is displayed in binary, the pixels of these adjacent pixels It is possible to provide a pixel interpolation method that omits the processing of the second multiplication step related to the value difference and the processing of adding or subtracting the second product to the register.

In the pixel interpolation method according to the fourth aspect of the present invention , the first coefficient and the plurality of second coefficients are powers of 2. In the first multiplication step, the sum of the pixel values is set to the first value. In the second multiplication step, the difference value of the pixel values is bit-shifted by the power exponent of each of the plurality of second coefficients.

  According to this configuration, the multiplication operation can be executed faster than a normal multiplier.

In the pixel interpolation method according to the fifth aspect of the present invention, the tap coefficient in the tap filter processing is given by {1, -5, 20, 20, -5, 1},
b [k] = ((a [k-2] -a [k-1])-4 * (a [k-1] -a [k]) + 16 * (a [k] + a [k + 1]) + 4 * (A [k + 1] -a [k + 2])-(a [k + 2] -a [k + 3])) / 32
Based on the formula of

  According to this configuration, the H.264 H.264 | MPEG-4 AVC compliant pixel interpolation processing can be executed.

In the pixel interpolation method according to the sixth aspect of the present invention, the first step of reading the pixel value a [k], the pixel value a [k] and the pixel value a [k] of the adjacent pixel stored in the first register −1], the difference value is stored in the second register, the sum value is shifted left by 4 bits and added to the third register, and the absolute value of the difference A third step of determining whether or not is less than or equal to a specific value, and in the third step, if it is determined that the absolute value of the difference exceeds the specific value, the second register is changed from the value of the fourth register A fifth step of subtracting the value of the second register, a value obtained by shifting the value of the second register left by 2 bits to the fifth register, a fifth step of subtracting from the sixth register, and a pixel of the kth position As the interpolated pixel value, the value of the fourth register is read, and the pixel value a [k] is set to the first value. Loops through further comprising a sixth step of storing the register, in a third step, when the absolute value of the difference is determined to be less than a specific value, it performs a loop process including a sixth step of.

  According to this configuration, the H.264 In accordance with the H.264 | MPEG-4 AVC, it is possible to efficiently and easily execute the pixel interpolation process in which the tap coefficients are given by {1, -5, 20, 20, -5, 1}.

According to a seventh aspect of the present invention, there is provided a pixel interpolation device, an adder that calculates a sum of pixel values of an adjacent pixel pair composed of two pixels, a subtractor that calculates a difference between pixel values of adjacent pixel pairs, A determination unit that determines whether the absolute value is equal to or less than a specific value, a plurality of registers, a first multiplication unit that multiplies a sum of pixel values by a predetermined coefficient, and a predetermined coefficient for a difference value of the pixel values A difference value between pixel values obtained by the second multiplication unit and the subtraction unit or multiplication results obtained by the first multiplication unit and the second multiplication unit with respect to the respective register values of the plurality of registers. In addition, a plurality of addition / subtraction units that add or subtract and pass to each subsequent register of each register are provided. When the determination unit determines that the absolute value of the pixel value difference is equal to or less than the specific value, the determination unit performs an operation related to the adjacent pixel pair that is determined to have the absolute value of the pixel value difference equal to or less than the specific value. The second multiplication unit and the plurality of addition / subtraction units are controlled so as not to execute.

  According to this configuration, when the absolute value of the difference between the pixel values of the adjacent pixel pair is equal to or less than a specific value, the multiplication process of the second multiplication unit related to the difference between the pixel values of these adjacent pixels, and It is possible to provide a pixel interpolation device that omits addition / subtraction processing of a plurality of addition / subtraction units. Therefore, by using the pixel interpolation device having this configuration, it is possible to increase the speed of generation of a predicted picture in motion compensation while reducing power consumption as the amount of calculation is reduced.

  According to the present invention, it is possible to realize a pixel interpolation method and a pixel interpolation device that improve the processing speed while reducing power consumption. This pixel interpolation device can perform motion compensation in predictive coding of moving images. The present invention can also be applied to applications such as pixel generation during image enlargement or reduction.

  Next, embodiments of the present invention will be described with reference to the drawings.

(Embodiment 1)
FIG. 1 is a flowchart of a pixel interpolation method according to Embodiment 1 of the present invention.
The flowchart shown in FIG. 1 is an example of the pixel interpolation method of this embodiment. FIG. 2 is an explanatory diagram for explaining a pixel interpolation method according to Embodiment 1 of the present invention.

  First, the basic matters that led the inventors to devise the pixel interpolation method of this embodiment will be described.

  The pixel value of the interpolation pixel obtained by the pixel interpolation method of the present embodiment is 6 taps having tap coefficients {1, -5, 20, 20, -5, 1} with respect to the pixel value of the pixel of the reference picture. It is calculated | required by performing the filter process by a filter. This is because, with respect to pixels on one line in the horizontal direction or vertical direction of the reference picture, when the pixel value of an integer-precision pixel at the kth position is a [k], The pixel value “c” of the half-precision interpolated pixel located at the position (k + 1/2) between the (k + 1) -th pixels is obtained as an intermediate pixel value “b” by (Equation 1), It shows that it is obtained by normalization and saturation processing according to (Equation 2).

  As shown in (Expression 3), the right side of (Expression 1) is the difference between the pixel values of adjacent reference pixels (a [k−2] −a [k−1]), (a [k− 1] −a [k]), (a [k + 1] −a [k + 2]), (a [k + 2] −a [k + 3]) and the sum of the pixel values of adjacent reference pixels (a [k] + A [k + 1]) can be replaced with the sum form of terms.

  Furthermore, by replacing with the form of (Equation 3), the coefficients of multiplication with respect to the difference between the pixel values of adjacent pixels are coefficients “1”, “−4”, “4”, and “−1”, respectively. The coefficient of multiplication for the sum of the pixel values of the pixels is the coefficient “16”. As a result, the multiplications by the coefficients “4” and “16” are 2 2 and 2 4, respectively, and can be replaced with 2-bit and 4-bit left bit shift operations, respectively. Therefore, (Equation 3) is rewritten to (Equation 4).

  Here, the operator “<<” indicates a left bit shift operation.

  Further, the division by the value “32” in the equation (Equation 2) can be replaced with a 5-bit right bit shift operation since the value “32” is 2 to the fifth power. Therefore, (Equation 2) is rewritten as (Equation 5).

  Here, the operator “>>” indicates a right bit shift operation.

  As a result, the pixel value “c” of the interpolated pixel is obtained by (Equation 5) using the intermediate pixel value “b” obtained by (Equation 4).

  In this way, by converting (Equation 1) and (Equation 2) to obtain the pixel value of the interpolated pixel into the form of (Equation 4) and (Equation 5), the multiplication part and the division part by the coefficient are respectively Since only one bit shift operation is required, the operation can be lightened.

  As a result of the above considerations, in the pixel interpolation method of the present embodiment, the pixel interpolation process proceeds based on the form of (Expression 4) in which the calculation term is composed of the difference and sum of the pixel values of adjacent pixels.

  Hereinafter, the pixel interpolation method of this embodiment will be described with reference to FIGS. 1 and 2.

  In the table shown in FIG. 2, the values in the table represent the coefficients of the respective terms on the right side of (Equation 3), the rows represent combinations of pixel values of adjacent pixels to be referenced, and the columns represent intermediate pixel values of the interpolated pixels to be obtained. Represents.

  The calculation of the pixel value of the interpolated pixel is performed along the column to which the pixel value of the interpolated pixel to be obtained belongs. First, in the case of the calculation of the coefficient “16” in the table, the product of the coefficient and the sum of two pixel values of adjacent pixels belonging to the same row is obtained. In the case of calculation of coefficients “1”, “−4”, “4”, and “−1” other than the coefficient “16”, the product of each coefficient and the difference between two pixel values of adjacent pixels belonging to the same row Ask for. Subsequently, an intermediate pixel value “b” of the interpolated pixel is obtained by integrating these products in the column direction.

As an example, for the intermediate pixel value b [m + 3] of the interpolated pixel, the column is viewed in order from the top, and in the first calculation of the coefficient “1”, the value “a [k−2] −a [k −1] ”, the value“ −4 * (a [k−1] −a [k]) ”is added in the next calculation of the coefficient“ −4 ”, and in the calculation of the coefficient“ 16 ”, The value “16 * (a [k] + a [k + 1])” is added to the coefficient “4”.
In the calculation of “4 * (a [k + 1] −a [k + 2])”, the value “− (a [k + 2] −a [k + 3])” is added in the calculation of the coefficient “−1”. Is done. The result obtained from the above calculation is the same as (Equation 3).

  In the flowchart of the pixel interpolation method of the present embodiment shown in FIG. 1, the flow has a loop structure in which a series of processes from step S13 to step S24 is performed, and this one-time process is shown in the table of FIG. This corresponds to one line of processing.

  For example, in the table of FIG. 2, the pixel value a [k−1] of the pixel at the position “k−1” and the pixel value a [k] of the pixel at the position “k” are formed as two pixels forming an adjacent pixel pair. An example of the processing of a row to which the symbol belongs (a row surrounded by a broken line in the table) is as follows. That is, the pixel value a [k] of the pixel at the position “k” is read out, and the difference (a [k−1] −) from the pixel value a [k−1] of the pixel at the position “k−1” that has already been read out. a [k]), the difference is subtracted from the intermediate pixel value b [m], and a value four times the difference is added to the intermediate pixel value b [m + 1] to obtain the intermediate pixel value b [m + 2]. 16 times the sum (a [k−1] + a [k]) of both pixel values of the pixel at the position “k−1” and the pixel at the position “k” is added to the intermediate pixel value b [m + 3 ] Is subtracted by four times the above difference, and the above difference is added to the intermediate pixel value b [m + 4].

  The processing described above will be described in more detail with reference to FIG.

  In step S10 of FIG. 1, pixel interpolation processing starts.

  In step S11, initial values are given to the variable “k” and the variable “m”.

  In step S12, processing before the start of loop processing is performed as prolog processing.

  In step S13, the pixel value a [k] of the pixel at the position “k” is read out. Here, the pixel value a [k−1] of the pixel at the position “k−1” adjacent to the pixel at the position “k” has already been read out by the processing of the previous loop, and the value is retained. . Alternatively, when this processing is the first loop processing, the pixel value a [k−1] has already been read out by the prologue processing in step S12, and the value is held.

  In step S14, a value obtained by shifting the sum of the pixel value a [k−1] and the pixel value a [k] by 4 bits to the left (that is, a value multiplied by 16) is added to the intermediate pixel value b [m + 2]. This corresponds to the processing related to the coefficient “16” in the row surrounded by the broken line in the table shown in FIG.

  In step S15, a difference (a [k-1] -a [k]) between the pixel value a [k-1] and the pixel value a [k] is obtained, and the intermediate pixel value b [m + 4] is replaced. This corresponds to the processing related to the coefficient “1” in the row surrounded by the broken line in the table shown in FIG.

  Whether or not the absolute value (| a [k−1] −a [k] |) (that is, | b [m + 4] |) of the difference between the pixel values of adjacent pixels is equal to or less than the specific value “b0” in step S16 Determine.

  If the determination result in step S16 is “Yes” (difference is equal to or less than a specific value), the process from step S17 to step S20 is skipped, and the control is transferred to step S21. If the determination result is “No” (the difference exceeds a specific value), the control is moved to step S17.

  In step S17, the difference (a [k-1] -a [k]) (that is, b [m + 4]) of the pixel values of adjacent pixels is subtracted from the intermediate pixel value b [m], thereby the intermediate pixel value is interpolated. Pixel value b [m] is determined. This corresponds to the processing related to the coefficient “−1” in the row surrounded by the broken line in the table shown in FIG.

  In step S18, a value obtained by shifting the difference between the pixel values of adjacent pixels by 2 bits to the left, that is, a value multiplied by 4 is stored in the register r.

  In step S19, the value of the register r is added to the intermediate pixel value b [m + 1]. This corresponds to the processing related to the coefficient “4” in the row surrounded by the broken line in the table shown in FIG.

  In step S20, the value of the register r is subtracted from the intermediate pixel value b [m + 3]. This corresponds to the processing related to the coefficient “−4” in the row surrounded by the broken line in the table shown in FIG.

  Through the processes from step S13 to step S20, the process belonging to the line surrounded by the broken line in the table shown in FIG. 2 is performed.

  In step S21, normalization processing and saturation processing of the intermediate pixel value b [m] are performed to obtain a pixel value c [m].

  In step S22, the pixel value c [m] is output.

  In step S23, the variable “m” and the variable “k” are respectively incremented by the value “1” in preparation for the next loop.

  In step S24, it is determined whether a series of loop processing has been completed. If the determination result is “No” (loop processing is not completed), the control is returned to step S13, and the loop processing from step S13 to step S24 is repeated. If the determination result is “Yes” (the loop process ends), the control proceeds to step 25.

  In step S25, the remaining process is performed as the epilogue process, the process proceeds to step S26, and the series of pixel interpolation processes is terminated.

  As described above, in the pixel interpolation method of the present embodiment, when the absolute value of the difference between the pixel values of adjacent pixels is equal to or smaller than a specific value, the calculation of the term relating to this difference is not performed. Steps S17 to S20 are skipped. As a result, the amount of calculation necessary for the pixel interpolation process is reduced, and the pixel interpolation process can be speeded up. Furthermore, an effect of reducing the power required for calculation by reducing the amount of calculation is also expected.

  The specific value “b0” is, for example, a value indicated by the least significant bit when the pixel value of the pixel to be processed is displayed in binary. Alternatively, a value close to zero, that is, a noise level value may be set as the specific value “b0”.

  Alternatively, the specific value b0 = 0 may be simply set, and in step S16, it may be determined whether the difference (a [k−1] −a [k]) is zero.

  When the pixel interpolation method of this embodiment is applied to moving image compression coding, the difference between the pixel values of adjacent pixels is zero because the high frequency components of the image are removed at low bit rates due to the nature of moving image compression coding. It can be expected that there are many combinations of adjacent pixels, and the effect of the skip processing described above becomes more remarkable.

Furthermore, in the pixel interpolation method of this embodiment, when obtaining the pixel value of the interpolated pixel, the pixel value of one pixel is read out by a single loop process, and the difference and sum of the pixel values of one adjacent pixel are calculated. Find the difference and sum. Then, by repeating the loop processing, the pixel values of consecutive interpolated pixels are efficiently obtained without performing overlapping calculations. Therefore, the pixel interpolation method according to the present embodiment avoids redundant calculation as compared with the conventional technique in which (Equation 3) or (Equation 4) is calculated every time in order to obtain the pixel value of the interpolation pixel. Therefore, it is extremely efficient, and as a result, the pixel interpolation process can be speeded up.

  Here, what we want to call attention to is that the effect of the above-described pixel interpolation method of the present embodiment can be obtained by simply converting (Equation 1) into (Equation 3) or (Equation 4). Instead, it is obtained for the first time by the configuration of the pixel interpolation processing according to the pixel interpolation method of the present embodiment, which is performed according to the flowchart of FIG.

  In addition, the coefficient of each term of (Equation 3) is a power of 2, and each multiplication can be processed by replacing with a bit shift operation as shown in (Equation 4). Contributes to realizing high-speed processing.

  The flow chart shown in FIG. 1 is an example in the implementation of the present invention, and the flow control structure and the order of each processing step are not limited to those in FIG. ) May be changed as appropriate according to the processing based on.

(Embodiment 2)
FIG. 3 is a block diagram of a pixel interpolation apparatus according to Embodiment 2 of the present invention. The pixel interpolation device of this embodiment specifically executes the pixel interpolation method of the present invention described in the first embodiment of the present invention.

  As shown in FIG. 3, the pixel interpolation device of this embodiment includes registers 11 to 16, adders 21 to 27, bit shifters 31 to 33, selectors 41 to 43, a comparator 51, a clipper 61, and constant units 71 to 71. 72. In FIG. 3, the register is abbreviated as “R”, the adder as “+”, the bit shifter as “BS”, the selector as “SEL”, and the comparator as “C”.

  The operation of the pixel interpolation device of this embodiment will be described below.

3, registers 11 to 16, the control signal not shown in the figure, the uptake of the data that are controlled. The pixel value a [k] of the pixel at the position “k” among the plurality of reference pixels is also input to the register 11 in accordance with the control signal. That is, the register 11 stores pixel values a [k−5], a [k−4], and a [k] for each pixel of the control signal according to the reference pixel arrangement order. -3], a [k-2]... Accordingly, the pixel values of the adjacent pixels are stored in the register 11 and the register 12, respectively, the difference between the pixel values of the adjacent pixels is obtained by the adder 21, and the pixel value of the adjacent pixel is obtained by the adder 22. The sum of is required. (The adder 21 functions as a subtractor. The same applies to the adder 23 and the adder 26.)
Hereinafter, a case where the pixel value a [k + 3] is held in the register 11 in a certain cycle of the control signal will be described.

  At this time, pixel values a [k + 3] and a [k + 2] are stored in the registers 11 and 12, respectively. A difference (a [k + 2] −a [k + 3]) between adjacent pixel values is output from the adder 21, and a sum (a [k + 2] + a [k + 3]) of adjacent pixel values is output from the adder 22. Is output.

  The comparator 51 compares the absolute value (| a [k + 2] −a [k + 3] |) of the difference between the pixel values output from the adder 21 with a constant “b0” set in the constant unit 71. A signal indicating whether or not the absolute value of the difference is equal to or smaller than a constant “b0” is output. When the signal output from the comparator 51 indicates that the absolute value of the difference exceeds the constant “b0”, the selectors 41, 42, and 43 are connected to the adders 23, 25, and 26, respectively. Output is selected.

  The bit shifter 31 shifts the difference between the pixel values output from the adder 21 by 2 bits to the left (that is, 4 times), and the bit shifter 32 shifts the sum of the pixel values output from the adder 22 by 4 bits to the left (that is, 16). Output).

  The register 13 stores the pixel value difference (a [k + 1] −a [k + 2]) of the adjacent pixels one cycle before, and the adder 23 subtracts the output of the bit shifter 31 from this value. (Expression 6) is obtained and output from the selector 41.

  The register 14 stores the output (formula 7) of the adder 23 one cycle before,

By adding this value and the output of the bit shifter 32 by the adder 24, (Equation 8) is obtained.

  The register 15 stores the output (Equation 9) of the adder 24 one cycle before,

This value and by adding in the adder 25 and the output of the bit shifter 31, are prompted the (number 10), is output from the selector 42.

  The register 16 stores the output (Equation 11) of the selector 42 one cycle before,

By subtracting the difference between the pixel values output from the adder 21 from this value by the adder 26, (Equation 1) is obtained. This coincides with the right side of (Equation 3), and is output from the selector 43 as the intermediate pixel value b [m] of the interpolated pixel.

  A constant “16” set in the constant unit 72 is added to the intermediate pixel value b [m] output from the selector 43 by the adder 27, and the result is right-shifted by 5 bits by the bit shifter 33 (that is, the constant “ Then, the result is saturated by the clipper 61, and the pixel value c [m] (value of (Equation 5)) of the interpolated pixel is obtained.

  In the series of processes described above, when the signal output from the comparator 51 indicates that the absolute value of the difference between adjacent pixel values is equal to or less than the constant “b0”, the selectors 41, 42, 43 Select the outputs of the registers 13, 15 and 16, respectively, and the adders 23, 25 and 26 and the bit shifter 31 do not perform an operation.

  As described above, the pixel interpolation device according to the present embodiment performs operations in the adders 23, 25, and 26 and the bit shifter 31 when the absolute value of the difference between the pixel values of adjacent pixels is equal to or less than the specific value “b0”. Therefore, it is possible to reduce the power consumption associated with the calculation in these calculators.

  Further, by converting the 6-tap filter processing for interpolation of pixel values into the form of (Equation 3), the coefficients “4” and “16” in which the coefficients of the terms of (Equation 3) are powers of 2 Thus, the multiplication operation can be executed by one bit shifter of 2 bits left and 4 bits left respectively. Further, in the standardization of the intermediate pixel value of the interpolated pixel, since the division denominator is a coefficient “32” that is a power of 2, it is possible to execute the division operation with one right 5 bit bit shifter. Become. As described above, in the pixel interpolation device according to this embodiment, the multiplication operation and the division operation can be performed by the bit shifter, so that the circuit scale of the pixel interpolation device can be reduced.

(Embodiment 3)
FIG. 4 is a block diagram of a pixel interpolation apparatus according to Embodiment 3 of the present invention. The pixel interpolation device of this embodiment is a pixel interpolation device that processes 16-bit configuration image data, in which two 8-bit configuration pixel values are packed.

  The pixel interpolation device according to the present embodiment basically has a configuration in which the pixel interpolation device according to the second embodiment of the present invention shown in FIG. 3 is arranged in parallel and common portions are omitted. That is, the pixel interpolation device of the present embodiment includes an upper part for obtaining the pixel value of the odd-numbered interpolation pixel, a lower part for obtaining the pixel value of the even-numbered interpolation pixel, and other common parts. The upper part includes registers 11, 14a and 16a, adders 21a to 27a, bit shifters 31a to 33a, selectors 41a to 43a, a comparator 51a, a clipper 61a, and a constant part 72a, and the lower part includes registers 13b and 15b and an addition. 21b to 27b, bit shifters 31b to 33b, selectors 41b to 43b, a comparator 51b, a clipper 61b, and a constant part 72b, and a common part includes a constant part 71.

  In FIG. 4, the register is abbreviated as “R”, the adder as “+”, the bit shifter as “BS”, the selector as “SEL”, and the comparator as “C”.

Figure 5 is a Ru layout der two pixel values that are packed in a third embodiment of the present invention. Image data of the illustrated 16-bit configuration in FIG. 5, stores the pixel values a_even the even-numbered pixels in the upper 8 bits and the pixel value a_odd the odd-numbered pixels in the lower 8 bits store. Note that the arrangement of the two packed pixel values may be different from that in FIG.

  With reference to FIG. 4, an outline of the operation of the pixel interpolation device of this embodiment will be described below.

  The pixel value a [2k], which is the upper pixel data of the image data shown in FIG. 5, is input to the input terminal 91a, and the pixel value a [2k + 1], which is the lower pixel data of the image data, is input to the input terminal 91b. Is done. The pixel value c [2m + 1] of the odd-numbered interpolated pixel is output to the output terminal 99a, and the pixel value c [2m] of the even-numbered interpolated pixel is output to the output terminal 99b.

  In the pixel interpolation device of this embodiment, 16-bit image data in which two pixel values are packed is input for each control signal cycle. That is, for each cycle, odd-numbered pixel values are input to the input terminal 91a in the order of pixel values a [2k-3], a [2k-1], a [2k + 1], a [2k + 3]. The even-numbered pixel values are input to the input terminal 91b in the order of pixel values a [2k−2], a [2k], a [2k + 2], a [2k + 4].

  Hereinafter, in a certain cycle of the control signal, the pixel value a [2k + 2] is input to the input terminal 91a, the pixel value a [2k + 3] is input to the input terminal 91b, and the pixel value a [2k + 1] one cycle before is input to the register 11. The case where is held will be described.

  At this time, the difference (a [2k + 1] −a [2k + 2]) between adjacent pixel values is obtained by the adder 21a, and the sum (a [2k + 1] + a [2k + 2] of adjacent pixel values is obtained by the adder 22a. ]) Is obtained, and the adder 21b obtains the difference (a [2k + 2] −a [2k + 3]) of the pixel values of the adjacent pixels, and the adder 22b obtains the sum of the pixel values of the adjacent pixels (a [2k + 2] + A [2k + 3]).

  The bit shifters, adders, registers, and clippers located after the adders 21a, 21b, 22a, and 22b operate in substantially the same manner as the pixel interpolation device according to the second embodiment of the present invention. Therefore, description of those operations is omitted.

  As a result, the selector 43a outputs the value of (Equation 13) as the intermediate pixel value b [2m + 1] of the odd-numbered interpolation pixel, and the selector 43b outputs the intermediate pixel value b [2m of the even-numbered interpolation pixel. ], The value of (Expression 14) is output.

  The intermediate pixel value of (Equation 13) output from the selector 43a and the intermediate pixel value of (Equation 14) output from the selector 43b are normalized and saturated in accordance with (Equation 5) in the subsequent stage. Processing is performed. Finally, the pixel value c [2m + 1] of the odd-numbered interpolation pixel is output to the output terminal 99a, and the pixel value c [2m] of the even-numbered interpolation pixel is output to the output terminal 99b. .

  In the series of processes described above, when the signal output from the comparator 51a indicates that the absolute value of the difference between the pixel values of adjacent pixels exceeds the constant “b0”, the selector 41b adds the adder 23b. The selector 42b selects the output of the adder 25b, and the selector 43a selects the output of the adder 26a. When the signal output from the comparator 51b indicates that the absolute value of the difference between the pixel values of adjacent pixels exceeds a constant “b0”, the selector 41a selects the output of the adder 23a. The selector 42a selects the output of the adder 25a, and the selector 43b selects the output of the adder 26b.

  On the other hand, when the signal output from the comparator 51a indicates that the absolute value of the difference between the pixel values of adjacent pixels is equal to or less than the constant “b0”, the selector 41b selects the output of the register 13b, The selector 42b selects the output of the register 15b, and the selector 43a selects the output of the register 16a. The bit shifter 31a, the adder 23b, the adder 25b, and the adder 26a do not perform calculations.

  When the signal output from the comparator 51b indicates that the absolute value of the difference between adjacent pixel values is equal to or less than the constant “b0”, the selector 41a selects the output of the adder 21a. The selector 42a selects the output of the adder 24a, and the selector 43b selects the output of the selector 42b. The bit shifter 31b, the adder 23a, the adder 25a, and the adder 26b do not perform calculations.

  As described above, when the absolute value of the difference between the pixel values of adjacent pixels is equal to or less than the specific value “b0”, the pixel interpolation device according to the present embodiment performs the operation of the arithmetic unit related to the difference calculation. Therefore, power consumption associated with these calculations can be reduced.

  It will be apparent from the above description that the pixel interpolation device of the present embodiment can equally enjoy the features that the pixel interpolation device of Embodiment 2 of the present invention enjoys.

  Also, if the pixel interpolation device of this embodiment is expanded from a 2-parallel structure to a 4-parallel structure, 4 bits of 8-bit pixel values packed into 32-bit image data are input. It is possible to easily realize a pixel interpolation device that can perform pixel interpolation processing on an interpolation pixel at a time.

  In the first to third embodiments of the present invention described above, the H.264 standard. The description has been given along with the generation of the interpolation pixel by the 6-tap filter of the H.264 encoding standard. The present invention is not limited to the generation of interpolated pixels according to the H.264 coding standard, but can also be applied to the generation of interpolated pixels by other tap filter processing and motion compensation, and has the same effects as described above.

(Embodiment 4)
FIG. 6 is a block diagram of a moving picture coding apparatus according to Embodiment 4 of the present invention. The moving picture encoding apparatus 100 of this embodiment shown in FIG. 6 includes a subtractor 110, an orthogonal transform / quantization unit 120, a variable length encoding unit 130, an inverse quantization / inverse orthogonal transform unit 140, an adder 150, an in-loop A filter 160, a frame buffer 170, and a motion compensator 180 are provided, and the motion compensator 180 includes a prediction image generator 181 and a pixel interpolation unit 182.

  However, in FIG. 6, only processing blocks related to inter-picture prediction encoding of the moving picture encoding apparatus 100 of the present embodiment are shown, and processing blocks related to intra-picture prediction encoding are not shown.

  The pixel interpolation unit 182 included in the video encoding device 100 of the present embodiment is the pixel interpolation device of the second embodiment or the pixel interpolation device of the third embodiment of the present invention.

  Below, operation | movement of the moving image encoder 100 of this form is demonstrated.

  The difference between the original image input from the input terminal 101 and the predicted image output from the motion compensator 180 is calculated by the subtractor 110, and the residual is output to the orthogonal transform / quantization unit 120.

  The orthogonal transform / quantization unit 120 performs orthogonal transform (for example, discrete cosine transform) on the residual output from the subtractor 110, quantizes the obtained coefficient, and outputs a quantized transform coefficient.

  The variable length encoding unit 130 performs variable length encoding on the quantized transform coefficient output from the orthogonal transform / quantization unit 120 and outputs the result to the output terminal 109 as encoded image data.

  At the same time, the inverse quantization / inverse orthogonal transform unit 140 performs inverse quantization / inverse orthogonal transform on the quantized transform coefficient to obtain a restored residual. The adder 150 adds the restored residual to the predicted image output from the motion compensator 180 to restore the reconstructed image.

  The reconstructed image is stored in the frame buffer 170 as a decoded image after the block noise is removed by the in-loop filter 160.

  The motion compensator 180 reads out a decoded image as a reference image from the frame buffer 170, performs a calculation in the pixel interpolation unit 182 based on motion vector information (not shown), obtains an interpolation pixel, and performs prediction. The image generator 181 generates a motion-compensated predicted image.

  In the moving picture coding apparatus 100 according to the present embodiment, the process in the motion compensator 180, in particular, the pixel in the pixel interpolation unit 182 in the process until the input original image is variable-length coded and the coded image data is output. The ratio of interpolation processing to the whole is large. Therefore, by using the pixel interpolation device according to the second embodiment or the pixel interpolation device according to the third embodiment of the present invention as the pixel interpolation unit 182, high-speed pixel interpolation processing can be performed, and the motion compensator 180. Motion compensation processing can be performed at high speed. As a result, the moving picture coding apparatus 100 capable of high-speed processing can be realized. Furthermore, the moving image encoding apparatus 100 is configured such that the pixel interpolation unit 182 does not perform useless calculations, and thus has a feature of low power consumption.

  The moving image encoding apparatus 100 according to the present embodiment converts an input original image into an H.264 format. In accordance with H.264, variable length encoded image data can be efficiently encoded.

  Note that the moving picture encoding apparatus 100 according to the present embodiment is an H.264 format. The present invention is not limited to the variable length coding of the H.264 coding standard, and can be applied to variable length coding based on other coding standards by using other tap filter processing.

(Embodiment 5)
FIG. 7 is a block diagram of a moving picture decoding apparatus according to Embodiment 5 of the present invention. In FIG. 7, the same components as those in FIG. 6 are denoted by the same reference numerals, and the description thereof is omitted.

  7 includes a variable length decoding unit 210, an inverse quantization / inverse orthogonal transform unit 140, an adder 150, an in-loop filter 160, a frame buffer 170, and a motion compensator 180. The motion compensator 180 includes a prediction image generator 181 and a pixel interpolation unit 182.

However, FIG. 7 shows only processing blocks related to decoding of inter-picture predictive-coded image data among the processing blocks of the moving picture decoding apparatus 200 of the present embodiment, and decoding of intra-picture predictive-coded image data. The processing block concerning is not shown.

  The pixel interpolation unit 182 included in the moving picture decoding apparatus 200 according to the present embodiment is the pixel interpolation apparatus according to the second embodiment or the pixel interpolation apparatus according to the third embodiment of the present invention.

  Hereinafter, the operation of the moving picture decoding apparatus 200 according to this embodiment will be described.

  The variable length encoded image data input to the input terminal 201 is decoded into a quantized transform coefficient by the variable length decoding unit 210 and output to the inverse quantization / inverse orthogonal transform unit 140.

  The inverse quantization / inverse orthogonal transform unit 140 performs an inverse quantization / inverse orthogonal transform process on the quantized transform coefficient decoded by the variable length decoding unit 210 to obtain a residual.

  The adder 150 adds the residual and the predicted image output from the motion compensator 180 to obtain a reconstructed image.

  The reconstructed image is subjected to block noise removal by the in-loop filter 160, temporarily stored in the frame buffer 170 as a decoded image, and output to the output terminal 209.

  The motion compensator 180 reads a decoded image as a reference image from the frame buffer 170, performs a calculation in the pixel interpolation unit 182 based on motion vector information not shown, obtains an interpolation pixel, and obtains a predicted image A generator 181 generates a motion-compensated predicted image.

  In the moving picture decoding apparatus 200 according to the present embodiment, in the process from decoding the input variable length encoded image data to outputting the decoded image, the process in the motion compensator 180, in particular, the pixel interpolation in the pixel interpolation unit 182 is performed. The ratio of the insertion process to the whole is large. Therefore, by using the pixel interpolation device according to the second embodiment or the pixel interpolation device according to the third embodiment of the present invention as the pixel interpolation unit 182, high-speed pixel interpolation processing can be performed, and the motion compensator 180. Motion compensation processing can be performed at high speed. As a result, the moving picture decoding apparatus 200 capable of high-speed processing can be realized. Furthermore, the moving picture decoding apparatus 200 is configured such that the pixel interpolation unit 182 does not perform useless calculations, and thus has a feature of low power consumption.

  The moving picture decoding apparatus 200 of the present embodiment is an H.264 standard. It is possible to efficiently decode variable-length encoded image data encoded according to H.264.

  Note that the moving picture decoding apparatus 200 according to the present embodiment is an H.264 format. The present invention is not limited to decoding variable-length encoded image data of the H.264 encoding standard, but by using other tap filter processing, decoding of variable-length encoded image data in accordance with other encoding standards Even adaptation is possible.

  The pixel interpolation method according to the present invention can be used in a moving image processing apparatus that requires high-speed processing of moving images, such as a mobile phone with a camera, and its application field.

FIG. 1 is a flowchart of a pixel interpolation method according to Embodiment 1 of the present invention. It is explanatory drawing explaining the pixel interpolation method in Embodiment 1 of this invention. It is a block diagram of the pixel interpolation apparatus in Embodiment 2 of this invention. It is a block diagram of the pixel interpolation apparatus in Embodiment 3 of this invention. It is an arrangement diagram of two packed pixel values in the third embodiment of the present invention. It is a block diagram of the moving image encoder in Embodiment 4 of this invention. It is a block diagram of the moving image decoding apparatus in Embodiment 5 of this invention. It is a figure which shows the positional relationship of the pixel of integer precision, the interpolation pixel of 1/2 precision, and the interpolation pixel of 1/4 precision. It is a flowchart which shows an example of the conventional pixel interpolation method which calculates | requires the pixel value of an interpolation pixel by 6 tap filter. It is a block diagram of the conventional pixel interpolation apparatus by a 6 tap filter.

Claims (5)

A pixel interpolation method by tap filter processing,
An adding step for obtaining a sum of pixel values of an adjacent pixel pair composed of two reference pixels;
A subtraction step for obtaining a difference between pixel values of the adjacent pixel pairs;
A determination step of determining whether an absolute value of the difference between the pixel values is equal to or less than a specific value;
A first multiplication step of multiplying the sum of the pixel values by a first coefficient corresponding to a first interpolated pixel to obtain a first product,
In the determination step, when it is determined that the absolute value of the difference between the pixel values exceeds a specific value,
A second multiplication step of multiplying the difference value of the pixel values by a second coefficient corresponding to a second interpolation pixel to obtain a second product;
The first product is accumulated and added to a first register corresponding to the first interpolation pixel, and the second product is accumulated to a second register corresponding to the second interpolation pixel. Performing a first cumulative addition step of adding,
In the determination step, when it is determined that the absolute value of the difference between the pixel values is not more than a specific value,
A pixel interpolation method for executing a second cumulative addition step of cumulatively adding the first product to the first register. The addition step, the subtraction step, the determination step, the first multiplication step, the second multiplication step, the first cumulative addition step, and the second cumulative addition step, The pixel interpolation method according to claim 1, wherein the pixel interpolation method is repeatedly executed for a plurality of adjacent pixel pairs different from the adjacent pixel pairs. The pixel interpolation method according to claim 1, wherein the specific value is equal to or less than a value indicated by a least significant bit when each pixel value of the adjacent pixel pair is displayed in binary. The first coefficient and the second coefficient are powers of 2, and
In the first multiplication step, the sum of the pixel values is bit-shifted by a power exponent of the first coefficient,
2. The pixel interpolation method according to claim 1, wherein, in the second multiplication step, the difference value of the pixel values is bit-shifted by an exponent of the second coefficient . An adder for calculating the sum of pixel values of adjacent pixel pairs composed of two pixels;
A subtraction unit for obtaining a difference between pixel values of the adjacent pixel pair;
A determination unit for determining whether an absolute value of the difference between the pixel values is equal to or less than a specific value;
A first register corresponding to the first interpolated pixel;
A second register corresponding to the second interpolated pixel;
A first multiplier that multiplies the sum of the pixel values by a first coefficient corresponding to the first interpolated pixel;
A second multiplier that multiplies the difference value of the pixel values by a second coefficient corresponding to the second interpolation pixel;
When the determination unit determines that the absolute value of the difference between the pixel values exceeds the specific value, the first product is cumulatively added to the first register, and the second register is A first cumulative adder that cumulatively adds a second product;
When the determination unit determines that the absolute value of the difference between the pixel values is equal to or less than the specific value, the first register includes a second cumulative addition unit that cumulatively adds the first product. , Pixel interpolation device.