JP4693649B2 - Image processing apparatus and image data interpolation method - Google Patents

Description

  The present invention relates to an image data interpolation method when an image is enlarged or reduced, and an image processing apparatus for performing image data interpolation processing.

  In an image processing apparatus, when an image based on image data input from the outside is displayed after being enlarged, reduced, rotated, deformed, or the like, geometric conversion is performed on the image data. In such a case, if the image data is vector data, it is only necessary to perform coordinate conversion for each vector data. On the other hand, if the image data is raster data, the following three processes are required for geometric conversion. First, “coordinate conversion processing” is required. The coordinate conversion process is a process of converting the coordinates (of the image data) of the input image into the coordinates of the output image based on a predetermined conversion formula, or a process of converting the coordinates of the output image into the coordinates of the input image. That is. Second, “image rearrangement processing” is required. The image rearrangement process is to rearrange the image data of the input image so that the coordinates of the output image after the coordinate conversion process are in a grid arrangement. Third, “image data interpolation processing” is required. Image data interpolation processing is processing that inserts image data of coordinates necessary for image rearrangement processing because the coordinates of the image data of the output image obtained by coordinate conversion processing are generally not arranged in a grid pattern. It is.

  In recent years, with the diversification of image processing apparatuses, image enlargement display or reduction display is performed in many apparatuses. For example, in a digital still camera, a color image is enlarged or reduced by digital signal processing by a digital zoom function. Regarding the enlargement / reduction of the image, the method of calculating the pixel value (hereinafter referred to as “interpolation value”) of the point to be interpolated (hereinafter referred to as “interpolation point”) has a great influence on the image quality and processing speed. give.

  Conventionally, the nearest neighbor method (nearest neighbor interpolation method), the bilinear method (bilinear interpolation method), and the bicubic method are used as calculation methods for obtaining the interpolation value (hereinafter referred to as “interpolation value calculation method”). (Cubic convolution interpolation method) is known. The nearest neighbor method and the bilinear method have a high processing speed because the arithmetic processing is simple, but they are inferior to the bicubic method in terms of image quality. On the other hand, the bicubic method is excellent in terms of image quality because of the high accuracy of interpolation, but requires a long time for processing because the calculation processing is complicated. For this reason, one of the interpolation value calculation methods is employed according to the specifications required for the apparatus.

  Incidentally, in recent years, there has been proposed an image processing apparatus that calculates an interpolation value of image data by a different interpolation value calculation method depending on the type of signal component of the image data. FIG. 14 is a block diagram illustrating a configuration of a main part of a conventional image processing apparatus that realizes an interpolation method using a plurality of interpolation value calculation methods. In this image processing apparatus, the interpolation value is calculated by the interpolation calculation unit 910. The number of interpolation processing units 910 is equal to the number of interpolation value calculation methods to be employed. In the example illustrated in FIG. 14, a first interpolation processing unit 911 and a second interpolation processing unit 912 are provided, and interpolation values are calculated by different interpolation value calculation methods in these interpolation processing units. . The input image memory 900 stores input image data for each type of signal component. The input image data is given to either the first interpolation processing unit 911 or the second interpolation processing unit 912 depending on the type of the signal component or the like. When the interpolation value is calculated by the first interpolation processing unit 911 or the second interpolation processing unit 912, the interpolation value is stored in the output image memory 920 for each type as output image data.

  Japanese Patent Application Laid-Open No. 9-298600 discloses an image processing apparatus that includes an interpolation processing unit (pixel density conversion unit) for each color signal component and calculates an interpolation value by a different interpolation value calculation method depending on the color signal component. Is disclosed. FIG. 15 is a block diagram showing a configuration of the image processing apparatus. As shown in FIG. 15, the signal processing unit 931 takes input image data from the image memory 930 and divides it into a luminance signal Y and color difference signals U and V. For the luminance signal Y, the first pixel density conversion unit 932 calculates an interpolation value by the bicubic method. On the other hand, for the color difference signals U and V, the second pixel density conversion unit 940 calculates an interpolation value by the bilinear method. The reason why the interpolation value of the luminance signal Y is calculated by the bicubic method is that the human eye is generally more sensitive to the luminance signal than the color difference signal, and high-precision interpolation is required. is there.

Japanese Patent Application Laid-Open No. 2001-14454 discloses an image processing apparatus that implements calculation of interpolation values by a plurality of interpolation value calculation methods using software. FIG. 16 is a block diagram showing the configuration of this image processing apparatus. This image processing apparatus includes a luminance signal processing unit 970 for processing a luminance signal and a color difference signal processing unit 980 for processing a color difference signal. Then, the program used by the processor 954 is switched according to the type of image data for which the interpolation value is calculated.
JP-A-8-251400 Japanese Patent Laid-Open No. 9-298660 Japanese Patent Laid-Open No. 11-75060 Japanese Patent Laid-Open No. 2001-14454 JP 2005-223490 A

  However, in the conventional image processing apparatus, if a plurality of interpolation value calculation methods are to be employed for geometric conversion of image data, an interpolation processing unit is required for each interpolation value calculation method. As a result, the circuit scale increases and the power consumption increases. However, in recent years, downsizing of image processing apparatuses and reduction of power consumption are required. Also, the processing by software is not sufficient in terms of processing speed.

  SUMMARY An advantage of some aspects of the invention is that it provides an image processing apparatus that realizes an interpolation method capable of obtaining good image quality without increasing the circuit scale.

A first invention is an image processing apparatus that performs an interpolation calculation of image data based on a predetermined interpolation calculation expression,
According to the type of image data that is the subject of the interpolation calculation, an interpolation calculation input data generation unit that generates interpolation calculation input data to be substituted into the interpolation calculation formula;
An interpolation calculation unit that performs the interpolation calculation by substituting the interpolation coefficient calculated based on a predetermined interpolation coefficient calculation expression according to the type of the image data and the input data for interpolation calculation into the interpolation calculation expression; Prepared ,
The image data includes a first image component and a second image component,
The interpolation calculation input data generation unit includes:
When the image data to be subjected to the interpolation calculation is the first image component, the data is 16 points around the interpolation point whose value is to be obtained by the interpolation calculation, and is a third-order convolution interpolation. 16 points of data necessary for the interpolation calculation by the method are generated as the input data for the interpolation calculation,
When the image data to be subjected to the interpolation calculation is the second image component, four points around the interpolation point and four points necessary for the interpolation calculation by the bilinear interpolation method And data having data other than the four points surrounding the interpolation point out of the 16 points surrounding the interpolation point as zero value are generated as the input data for the interpolation calculation,
The interpolation calculation unit is configured to perform the cubic convolution as the predetermined interpolation calculation formula when the image data to be subjected to the interpolation calculation is either the first image component or the second image component. The interpolation calculation is performed based on an interpolation calculation expression based on an interpolation method .

According to a second invention, in the first invention,
An interpolation coefficient holding unit that holds an interpolation coefficient corresponding to the type of the image data in advance, and applies the interpolation coefficient held in advance to the interpolation calculation unit according to the type of the image data that is an object of the interpolation calculation; Furthermore, it is characterized by providing.

According to a third invention, in the first invention,
The image data is YUV format data including a luminance component as the first image component and a color difference component as the second image component.

According to a fourth invention, in the first invention,
The interpolation calculation unit includes four calculation units having the same configuration,
Each arithmetic unit obtains an intermediate value for calculating an interpolation value based on five data as the interpolation coefficient and four data as the input data for interpolation calculation by the following formula: .
P = (P1 · f (x1) + P2 · f (x2) + P3 · f (x3) + P4 · f (x4)) · f (y)
Here, P is the intermediate value, f (x1), f (x2), f (x3), f (x4), and f (y) are five data as the interpolation coefficients, P1, P2, P3, And P4 are four data as the input data for the interpolation calculation.

A fifth invention is the fourth invention,
Each arithmetic unit includes five multipliers that perform a multiplication process for obtaining the intermediate value and three adders that perform an addition process for obtaining the intermediate value.

A sixth invention is an image data interpolation method for performing an interpolation calculation based on a predetermined interpolation calculation expression,
Interpolation calculation input data generation step for generating interpolation calculation input data to be substituted into the interpolation calculation formula according to the type of image data to be subjected to the interpolation calculation;
An interpolation coefficient calculation step for calculating an interpolation coefficient to be substituted into the interpolation calculation formula according to the type of image data to be subjected to the interpolation calculation;
An interpolation calculation step for performing the interpolation calculation by substituting the interpolation calculation input data and the interpolation coefficient into the interpolation calculation formula ,
The image data includes a first image component and a second image component,
In the interpolation calculation input data generation step,
When the image data to be subjected to the interpolation calculation is the first image component, the data is 16 points around the interpolation point whose value is to be obtained by the interpolation calculation, and is a third-order convolution interpolation. 16 points of data necessary for the interpolation calculation by the method are generated as the input data for the interpolation calculation,
When the input image data to be subjected to the interpolation calculation is the second image component, it is data of four points around the interpolation point and is necessary for the interpolation calculation by the bilinear interpolation method. Point data and data in which data other than the four points around the interpolation point out of the 16 points around the interpolation point are generated as the input data for the interpolation calculation,
In the interpolation calculation step, the cubic convolution is used as the predetermined interpolation calculation formula regardless of whether the image data to be subjected to the interpolation calculation is the first image component or the second image component. The interpolation calculation is performed based on an interpolation calculation expression based on an interpolation method .

A seventh invention is the sixth invention, wherein
Instead of the interpolation coefficient calculation step, an interpolation coefficient acquisition step of acquiring a previously held interpolation coefficient according to the type of image data to be subjected to the interpolation calculation is provided.

In an eighth aspect based on the sixth aspect,
The image data is YUV format data including a luminance component as the first image component and a color difference component as the second image component.

According to a ninth invention, in the sixth invention,
The interpolation calculation step includes four calculation steps for performing the same calculation,
In each calculation step, an intermediate value for calculating an interpolation value is obtained by the following equation based on the five data as the interpolation coefficient and the four data as the input data for interpolation calculation. To do.
P = (P1 · f (x1) + P2 · f (x2) + P3 · f (x3) + P4 · f (x4)) · f (y)
Here, P is the intermediate value, f (x1), f (x2), f (x3), f (x4), and f (y) are five data as the interpolation coefficients, P1, P2, P3, And P4 are four data as the input data for the interpolation calculation.

A tenth invention is the ninth invention,
Each calculation step includes five multiplication steps for performing multiplication processing for obtaining the intermediate value and three addition steps for performing addition processing for obtaining the intermediate value.

According to the first aspect, the interpolation calculation unit substitutes the interpolation calculation input data and the interpolation coefficient corresponding to the type of image data to be subjected to the interpolation calculation into the predetermined interpolation calculation formula. Interpolation calculation is performed. For this reason, the interpolation value corresponding to the type of the image data is calculated by one interpolation calculation unit. Thus, suitable interpolation processing is performed for each type of image data without increasing the circuit scale and power consumption.
When the object of the interpolation calculation is the first image component, 16 points of data necessary for the interpolation calculation by the third-order convolution interpolation method are generated as the input data for the interpolation calculation, and the object of the interpolation calculation is In the case of the second image component, four points of data necessary for the interpolation calculation by bilinear interpolation and data having 12 points around it as zero values are generated as input data for interpolation calculation. On the other hand, the interpolation calculation unit performs the interpolation calculation based on the interpolation calculation formula based on the third-order convolution interpolation method when the interpolation calculation target is either the first image component or the second image component. Is called. As a result, it is possible to switch between the interpolation calculation giving priority to the image quality and the interpolation calculation giving priority to the processing speed according to the type of image data by one interpolation calculation unit.

  According to the second aspect, the interpolation coefficient corresponding to the type of image data is held in the interpolation coefficient holding unit in advance, and the interpolation coefficient is given to the interpolation calculation unit according to the type of image data to be subjected to the interpolation calculation. . For this reason, it is not necessary to calculate the interpolation coefficient when executing the interpolation calculation, and the processing time is shortened.

According to the third aspect , when the interpolation calculation for the luminance component is performed, the data necessary for the interpolation processing by the third-order convolution interpolation method becomes the input data, and the interpolation calculation for the color difference component is performed. The data necessary for the interpolation processing by the bilinear interpolation is input data for the interpolation calculation. For this reason, the interpolation processing is performed for the luminance component without reducing the image quality, and the interpolation processing is performed for the color difference component without reducing the processing speed.

<1. Review>
Before describing the embodiment of the present invention, a method for calculating an interpolation value of image data will be described. Here, the bicubic method, the bilinear method, and the nearest neighbor method that are generally known in the past will be described. In the following description, as shown in FIG. 1, the coordinates of the interpolation point of the input image are (u, v). In FIG. 1, a point (observation point) having a pixel value as image data of the input image is indicated by reference numeral 2. Further, a square area having four observation points around the interpolation point as vertices is referred to as a “first pixel block 101”, and a square area having the 16 observation points around the interpolation point as vertices is designated as “first pixel block 101”. This is referred to as a “second pixel block 102”.

<1.1 Bicubic method>
First, an interpolation value calculation method by the bicubic method will be described with reference to FIG. FIG. 2 is a diagram illustrating observation points in the second pixel block 102. In the bicubic method, pixel values of 16 observation points in the second pixel block 102 are P11 to P44.

ここで、式（３−１）〜（３−３）のｆ（ｔ）については厳密には式（４）によって算出されるべきものであるが、デジタル演算によるｓｉｎ関数の処理が容易ではないことから、式（３−１）〜（３−３）によって近似計算がなされている。

なお、式（３−１）および式（３−２）の「ａ」は実数の定数であって、「ａ」の値を変えることにより補間の特性を変えることができる。また、式（２−１）〜（２−８）は、補間点から第２の画素ブロック１０２内の１６個の観測点までの水平方向および垂直方向の距離を算出するためのものである。そして、式（２−１）〜（２−８）で算出された値を式（３−１）〜（３−３）の「ｔ」に代入することによって、補間点からの距離に応じて各観測点の画素値に係る係数（重み係数）が算出される。 The interpolated value P is calculated by 16 pixel values P11 to P44, formulas (1), formulas (2-1) to (2-8), and formulas (3-1) to (3-3). The

Here, strictly speaking, f (t) in the expressions (3-1) to (3-3) should be calculated according to the expression (4), but it is not easy to process the sin function by digital calculation. Therefore, the approximate calculation is performed by the equations (3-1) to (3-3).

Note that “a” in the expressions (3-1) and (3-2) is a real constant, and the characteristic of interpolation can be changed by changing the value of “a”. Expressions (2-1) to (2-8) are used to calculate the horizontal and vertical distances from the interpolation point to the 16 observation points in the second pixel block 102. And according to the distance from an interpolation point by substituting the value calculated by Formula (2-1)-(2-8) to "t" of Formula (3-1)-(3-3). A coefficient (weight coefficient) related to the pixel value at each observation point is calculated.

<1.2 Bilinear method>
Next, an interpolation value calculation method using the bilinear method will be described with reference to FIG. FIG. 3 is a diagram illustrating observation points in the first pixel block 101. In the bilinear method, pixel values of four observation points in the first pixel block 101 are Pi j to Pi + 1 j + 1.

なお、バイリニア法による補間値計算の式は一般的には式（８）で表される。しかし、計算を容易にするために各観測点の重み係数を式（５−１）、式（５−２）、および式（６−１）〜（６−４）で算出することにより、式（７）で補間値計算の式が表される。

The interpolation value P includes four pixel values Pi j to Pi + 1 j + 1, Equation (5-1), Equation (5-2), Equation (6-1) to (6-4), and Equation (7). Is calculated by

Note that the equation for calculating the interpolation value by the bilinear method is generally expressed by equation (8). However, in order to facilitate the calculation, the weighting coefficient of each observation point is calculated by the equations (5-1), (5-2), and (6-1) to (6-4). (7) represents the formula for calculating the interpolation value.

<1.3 Nearest neighbor method>
Next, an interpolation value calculation method by the nearest neighbor method will be described with reference to FIG. As shown in FIG. 4, in the interpolation value calculation method using the nearest neighbor method, the pixel value at the observation point closest to the interpolation point is set as the interpolation value P. That is, the interpolation value P is calculated by Expression (9), Expression (10-1), and Expression (10-2).

<Comparison of computational complexity>
Here, the difference between the calculation amount of the interpolation value calculation by the bicubic method and the calculation amount of the interpolation value calculation by the bilinear method will be described. The bicubic method interpolation value calculation expression represented by Expression (1) can be modified into Expressions (11-1) to (11-4) and Expression (12). The weighting factors f (t) in the equations (11-1) to (11-4) are calculated by the equations (3-1) to (3-3). P1 to P4 are sums of values at four observation points (values obtained by multiplying pixel values by weighting factors), and are values calculated in the process of calculating the interpolation value P. The interpolation value P is calculated by the equation (12) using these P1 to P4.

  As described above, the weighting coefficient in the interpolation value calculation by the bicubic method is calculated for the 16 observation points in the second pixel block 102 by the equations (3-1) to (3-3). On the other hand, the weighting coefficient in the calculation of the interpolation value by the bilinear method is calculated by the equations (6-1) to (6-4) for only four observation points in the first pixel block 101. Thereby, it is recognized that the amount of calculation for calculating the weighting coefficient is larger in the bicubic method than in the bilinear method. Further, even when the weighting factor is in the form of a data table, it is recognized that the amount of calculation for calculating the interpolation value P is four times the amount of calculation by the bicubic method. Is done.

  Thus, the amount of calculation of the bicubic method is larger than the amount of calculation of the bilinear method. Therefore, the processing time required to calculate the interpolation value is shorter in the bilinear method than in the bicubic method.

<1.5 Interpolation Method in the Present Invention>
In the present invention, the interpolation value calculation formula of the bilinear method represented by Expression (7) and the interpolation value calculation expression (after deformation) of the bicubic method represented by Expressions (11-1) to (11-4) As for the image data that had previously been calculated by the bilinear method as described below, the interpolation value can be calculated by the bicubic interpolation value calculation formula without complicating the processing. Is calculated. In the following description, as shown in FIG. 5, the pixel values at 16 observation points in the second pixel block 102 are P11 to P44, and four observations in the first pixel block 101 are included. The pixel values of the points are P22, P32, P23, and P33.

  The calculation of the interpolation value in the present invention for the image data for which the interpolation value is conventionally calculated by the bilinear method will be described. When calculating the interpolation value, for the image data of the four observation points in the first pixel block 101, the pixel value is substituted into Equation (1). On the other hand, for 12 pieces of image data excluding the observation points in the first pixel block 101 among the observation points in the second pixel block, a zero value is substituted into Expression (1). That is, when the pixel value of each observation point to be substituted into the equation (1) is schematically represented, the value is as shown in FIG. Here, the weighting coefficient is calculated by the equations (6-1) to (6-4) which are equations of the bilinear method.

When the interpolation value P is calculated as described above, Equation (13), Equations (14-1) to (14-4), and Equation (15) are derived. Here, the sum of Expression (14-3) and Expression (14-4) is the sum of the values obtained by multiplying the pixel values of the four observation points in the first pixel block 101 by the weighting coefficient. Show. Therefore, what combined Formula (14-3) and Formula (14-4) has shown the same content as Formula (7). Thereby, the pixel value P calculated by Formula (7) and the pixel value P calculated by Formula (15) become the same value. In this way, even for image data for which interpolation values have been calculated by the conventional bilinear method, the interpolation values can be calculated by the interpolation value calculation formula by the bicubic method without complicating the processing.

  The interpolation method described above can be applied to, for example, an image processing apparatus. FIG. 7 is a block diagram illustrating a configuration of a main part of an image processing apparatus to which the above-described interpolation method is applied. In the configuration of the conventional image processing apparatus shown in FIG. 14, a first interpolation processing unit 911 and a second interpolation processing unit 912 that perform interpolation value calculation using different interpolation value calculation methods are provided. On the other hand, in the configuration of the present invention, only one interpolation processing unit 812 is provided. In the interpolation processing unit 812, the interpolation value is calculated by the interpolation value calculation formula of the bicubic method. In other words, both the image data for which the interpolation value was calculated by the conventional bilinear method and the image data for which the interpolation value was calculated by the conventional bicubic method are interpolated by the interpolation value calculation formula by the bicubic method. Calculated.

  Hereinafter, an embodiment of the present invention will be described with reference to the accompanying drawings.

<2. Overall Configuration and Operation of Image Processing Apparatus>
FIG. 8 is a block diagram showing the overall configuration of the image processing apparatus according to an embodiment of the present invention. The image processing apparatus includes a size conversion processing unit 100, an input device 200, an image memory 300, an output device 400, and a control processor 500. The size conversion processing unit 100 includes a control unit 10, an image input unit 20, an input data control unit 30, an interpolation processing unit 40, a coefficient unit 50, an output data control unit 60, and an image output unit 70. The control unit 10 includes a control register 11. The image input unit 20 includes an input buffer 21. The image output unit 70 includes an output buffer 71. In this embodiment, the image input unit 20 and the input data control unit 30 realize an interpolation calculation input data generation unit, the interpolation processing unit 40 realizes an interpolation calculation unit, and the coefficient unit 50 holds an interpolation coefficient. Is realized.

  In this image processing apparatus, the YUV method is adopted as the format of image data. For the luminance component Y in the image data, an interpolation value is calculated from image data for 16 points around the interpolation point. On the other hand, for the color difference components Cb and Cr in the image data, an interpolation value is calculated from the image data for four points around the interpolation point.

  A control processor 500 controls the entire image processing apparatus. The control unit 10 controls the image input unit 20, the input data control unit 30, the interpolation processing unit 40, the coefficient unit 50, the output data control unit 60, and the image output unit 70 based on the instruction signal given from the control processor 500. In order to do so, an image input unit control signal S1, an input data control unit control signal S2, an interpolation processing unit control signal S3, a coefficient unit control signal S4, an output data control unit control signal S5, and an image output unit control signal S6 are output. . The control register 11 stores various parameters for interpolation processing described later.

  The input device 200 outputs YUV format image data (input image data) Di. That is, the input device 200 outputs the luminance signal Y and the color difference signals Cb and Cr as the input image data Di. The image memory 300 holds input image data Di output from the input device 200 and output image data Do output from the image output unit 70. The image input unit 20 acquires input image data Di (Y, Cb, Cr) for calculating the interpolation value P from the image memory 300 based on the image input unit control signal S1 given from the control unit 10, and these Is stored in the input buffer 21. Then, the image input unit 20 supplies the input image data Di (Y, Cb, Cr) to the input data control unit 30.

  The input data control unit 30 receives the input image data Di (Y, Cb, Cr) supplied from the image input unit 20 based on the input data control unit control signal S2 given from the control unit 10, and receives the interpolation processing unit 40. Data (hereinafter referred to as “input data for interpolation calculation”) Dh is generated. The interpolation calculation input data Dh is supplied from the input data control unit 30 to the interpolation processing unit 40. The coefficient unit 50 includes an interpolation coefficient necessary for calculation based on the bicubic method interpolation value calculation formula (hereinafter referred to as “bicubic method interpolation coefficient”) and an interpolation coefficient necessary for calculation based on the bilinear method interpolation value calculation formula. (Hereinafter referred to as “bilinear interpolation coefficient”) is stored in advance as a data table, and the interpolation coefficient Hk is supplied to the interpolation processing unit 40 based on the coefficient control signal S4 given from the control unit 10. The bicubic interpolation coefficients are calculated by the equations (3-1) to (3-3), and the bilinear interpolation coefficients are calculated by the equations (6-1) to (6-4).

  The interpolation processing unit 40 receives the interpolation calculation input data Dh supplied from the input data control unit 30 and the interpolation coefficient Hk supplied from the coefficient unit 50 based on the interpolation processing unit control signal S3 given from the control unit 10. The interpolation value P is calculated. The interpolation processing unit 40 supplies the calculated interpolation value P to the output data control unit 40 as output data P1 to P4.

  The output data control unit 60 receives the output data P1 to P4 supplied from the interpolation processing unit 40 based on the output data control unit control signal S5 given from the control unit 10, and outputs the output image data Do as the interpolation value P. Generate. The output data control unit 60 supplies the output image data Do to the image output unit 70. The image output unit 70 receives the output image data Do supplied from the output data control unit 60, and stores the output image data Do in the output buffer 71 based on the output data control unit control signal S 6 supplied from the control unit 10. . Then, the image output unit 70 transfers the output image data Do stored in the output buffer 71 to the image memory 300 in units of blocks.

  The output device 400 takes out the output image data Do from the image memory 300 and outputs (displays) the same to the outside.

<3. Configuration of Interpolation Processing Unit>
FIG. 9 is a block diagram illustrating a configuration of the interpolation processing unit 40. As shown in FIG. 9, the interpolation processing unit 40 includes first to fourth arithmetic units 41 to 44. In the first arithmetic unit 41, the value of “P1” in Expression (11-1) is calculated. In the second arithmetic unit 42, the value of “P2” in Expression (11-2) is calculated. In the third arithmetic unit 43, the value of “P3” in Expression (11-3) is calculated. In the fourth arithmetic unit 44, the value of “P4” in the equation (11-4) is calculated. Thus, in the first to fourth arithmetic units 41 to 44, the values (intermediate results) P1 to P4 for calculating the interpolation value P are calculated using the bicubic interpolation value calculation formula.

FIG. 10 is a diagram for explaining data input to the first to fourth arithmetic units 41 to 44 and data output from the first to fourth arithmetic units 41 to 44. Expressions (11-1) to (11-4) can be transformed into Expressions (16-1) to (16-4), respectively. Data corresponding to the right side of the equations (16-1) to (16-4) is input to the first to fourth arithmetic units 41 to 44. Then, data corresponding to the left side of the equations (16-1) to (16-4) is output from the first to fourth arithmetic units 41 to 44. For example, as shown in FIG. 10, data corresponding to the right side of Expression (16-1) is input to the first arithmetic unit 41. In addition, data corresponding to the left side of Expression (16-1) is output from the first arithmetic unit 41.

  FIG. 11 is a circuit diagram showing a configuration of the first arithmetic unit 41. As shown in FIG. 11, the first arithmetic unit 41 includes five multipliers 411 to 414 and 418 and three adders 415 to 417. By configuring these multipliers and adders as shown in FIG. 11, the arithmetic processing of Expression (16-1) is performed. The configurations of the second to fourth arithmetic units 42 to 44 are the same as the configuration of the first arithmetic unit 41.

<4. Size conversion processing>
Next, the procedure of image size conversion processing (hereinafter referred to as “size conversion processing”) in the present embodiment will be described. FIG. 12 is a flowchart showing the procedure of the size conversion process. When the size conversion processing starts, the control processor 500 causes the control register 11 in the control unit 10 to process component designation information indicating which of the luminance component Y or the color difference components Cb and Cr is to be processed, the input image size, Then, the enlargement / reduction ratio is set as a parameter (step S100). Here, the input image size and the enlargement / reduction ratio are given to the control processor 500 from the outside via an input interface (not shown), for example. When all the parameters necessary for the size conversion process are set in the control register 11, the process proceeds to step S110.

  In step S110, the control unit 10 calculates the size of the output image and the coordinates of the output image data based on the parameters set in the control register 11. These calculation results are stored in the control register 11. When the calculation result is stored in the control register 11, the control unit 10 provides the image input unit control signal S1 to the image input unit 20, and provides the input data control unit control signal S2 to the input data control unit 30, and the interpolation processing unit. The control signal S3 is supplied to the interpolation processing unit 40, the coefficient unit control signal S4 is supplied to the coefficient unit 50, the output data control unit control signal S5 is supplied to the output data control unit 60, and the image output unit control signal S6 is supplied to the image output unit 70. To give. Thereafter, the process proceeds to step S120.

  In step S <b> 120, the image input unit 20 acquires input image data Di necessary for calculating an interpolation value from the image memory 200 based on the image input unit control signal S <b> 1 given from the control unit 10. At that time, the image input unit 20 refers to the parameters set in the control register 11. Specifically, the processing component designation information is referred to and it is determined which of the luminance component Y or the color difference components Cb and Cr is the processing target. As a result, when the image data to be processed is the luminance component Y, the pixel value at the observation point in the second pixel block 102 is acquired from the image memory 200 as the input image data Di. On the other hand, when the image data to be processed is the color difference components Cb and Cr, the pixel value at the observation point in the first pixel block 101 is acquired from the image memory 200 as the input image data Di. That is, when the processing target is the luminance component Y, input image data Di for 16 points around the interpolation point is acquired from the image memory 200, and when the processing target is the color difference components Cb and Cr, the interpolation point The input image data Di for four points around is obtained from the image memory 200. The input image data Di acquired from the image memory 200 is stored in the input buffer 21 of the image input unit 20.

  When the processing target is the luminance component Y, the image input unit 20 supplies the input image data Di for 16 points to the input data control unit 30. On the other hand, when the processing target is the color difference components Cb and Cr, the image input unit 20 observes the first pixel block 101 among the four points of input image data and the observation points in the second pixel block. Input image data having pixel values of 12 points excluding the points as zero values is supplied to the input data control unit 30. As described above, the content of the input image data Di supplied from the image input unit 20 to the input data control unit 30 differs depending on the processing target of the image data. When the input image data Di is supplied to the input data control unit 30 by the image input unit 20, the process proceeds to step S130.

  In step S130, the interpolation value P is calculated (interpolation calculation process). Specifically, the input data control unit 30 receives the input image data Di supplied from the image input unit 20 based on the input data control unit control signal S2 provided from the control unit 10, and receives the input data Dh for interpolation calculation. Is generated. The input data control unit 30 supplies the interpolation calculation input data Dh to the interpolation processing unit 40. The coefficient unit 50 supplies an interpolation coefficient Hk for calculating the interpolation value P to the interpolation processing unit 40 based on the coefficient unit control signal S4 given from the control unit 10. Here, when the processing target is the luminance component Y, the bicubic interpolation coefficient is supplied to the interpolation processing unit 40, and when the processing target is the luminance components Cb and Cr, the bilinear interpolation coefficient is the interpolation processing unit 40. To be supplied. The interpolation processing unit 40 performs calculation processing for calculating the interpolation value P based on the interpolation calculation input data supplied from the input data control unit 30 and the interpolation coefficient Hk supplied from the coefficient unit 50. The results obtained by the arithmetic processing are supplied from the interpolation processing unit 40 to the output data control unit 60 as output data P1 to P4. The output data control unit 60 generates output image data Do (interpolation value P) based on the output data P1 to P4 supplied from the interpolation processing unit 40. The output data control unit 60 supplies the output image data Do to the image output unit 70. Hereinafter, a detailed processing procedure of step S130 will be described.

  FIG. 13 is a flowchart showing a detailed procedure of the interpolation calculation process. When the interpolation calculation process is started, it is first determined whether or not the image data to be processed is the luminance component Y (step S200). As a result, when it is determined that the processing target is the luminance component Y, the process proceeds to step S210. On the other hand, when it is determined that the processing target is not the luminance component Y, that is, the processing target is the color difference components Cb and Cr, the process proceeds to step S220. This determination is made based on the processing component designation information held as a parameter in the control register 11.

In step S210, the interpolation value P for the luminance component Y is calculated. The input data control unit 30 supplies the input image data Di supplied from the image input unit 20 to the interpolation processing unit 40 as interpolation calculation input data Dh. Here, the image data for 16 points in the second pixel block 102 is supplied from the image input unit 20 to the input data control unit 30. In this step S210, the image data for these 16 points are supplied as they are to the interpolation processing unit 40 as input data for interpolation calculation. The coefficient unit 50 supplies the bicubic interpolation coefficient to the interpolation processing unit 40 based on the coefficient unit control signal S4 given from the control unit 10. The interpolation processing unit 40 receives the interpolation calculation input data Dh and the bicubic interpolation coefficient, and performs calculation processing based on the equations (16-1) to (16-4). The interpolation processing unit 40 supplies the result of the arithmetic processing to the output data control unit 60 as output data P1 to P4 . The output data control unit 60 performs a calculation process based on Expression (12) based on the output data P1 to P4 supplied from the interpolation processing unit 40. Thereby, the interpolation value P is calculated. The output data control unit 60 supplies the interpolation value P to the image output unit 70 as output image data Do. When step S210 ends, the process proceeds to step S140 shown in FIG.

  As described above, for the luminance component Y of the input image data, the interpolation value P is calculated based on the interpolation value calculation method by the bicubic method.

  In step S220, the interpolation calculation input data Dh is generated. The input data control unit 30 receives the input image data Di supplied from the image input unit 20. Here, the image data for four points in the first pixel block 101 is supplied from the image input unit 20 to the input data control unit 30. The input data control unit 30 sets the pixel values for the four points of the image data and the observation points in the second pixel block 102 excluding the observation points in the first pixel block 101 to zero values. The interpolation calculation input data Dh is generated based on the processed image data. That is, image data as shown in FIG. 6 is generated as the interpolation calculation input data Dh. When the interpolation calculation input data Dh is generated, the process proceeds to step S230.

  In step S230, interpolation calculation is performed for the color difference components Cb and Cr. The input data control unit 30 supplies the interpolation calculation input data Dh to the interpolation processing unit 40. The coefficient unit 50 supplies a bilinear interpolation coefficient to the interpolation processing unit 40 based on the coefficient unit control signal S4 given from the control unit 10. The interpolation processing unit 40 receives the interpolation calculation input data Dh and the bilinear interpolation coefficient, and performs calculation processing based on the equations (16-1) to (16-4). The interpolation processing unit 40 supplies the result of the arithmetic processing to the output data control unit 60 as output data P1 to P4. Thereafter, the process proceeds to step S240.

  In step S240, the output data control unit 60 performs a calculation process based on Expression (12) based on the output data P1 to P4 supplied from the interpolation processing unit 40. Thereby, the interpolation value P is calculated. The output data control unit 60 supplies the interpolation value P to the image output unit 70 as output image data Do. When step S240 ends, the process proceeds to step S140 shown in FIG.

  As described above, when calculating the interpolation value P for the color difference components Cb and Cr, as the effective pixel value, the image data for four points around the interpolation point and the interpolation coefficient used for the interpolation coefficient by the bilinear method Is supplied to the interpolation processing unit 40. That is, for the color difference components Cb and Cr of the input image data, the interpolation value calculation formula of the bicubic method is used, but the interpolation value P is substantially calculated based on the interpolation value calculation method by the bilinear method. Yes.

  In step S <b> 140 shown in FIG. 12, output image data Do is transmitted from the image output unit 70 to the image memory 300. The image output unit 70 stores the output image data Do supplied from the output data control unit 60 in the output buffer 71 based on the image output unit control signal S6 supplied from the control unit 10. Thereafter, the image output unit 70 transmits the output image data Do stored in the output buffer 71 to the image memory 300 in predetermined block units. When the transmission of the output image data Do from the image output unit 70 to the image memory 300 is completed, the process proceeds to step S150.

  In step S150, it is determined whether or not output image data (interpolation value) processing (interpolation value calculation) corresponding to the output image size has been completed. As a result, if completed, the process proceeds to step S160. On the other hand, if not completed, the process returns to step S110.

  In step S160, it is determined whether or not processing for all color components has been completed. As a result, if it has been completed, the size conversion process is completed. On the other hand, if not completed, the process returns to step S100.

<5. Effect>
According to the present embodiment, only the interpolation processing unit 40 that performs arithmetic processing based on the bicubic interpolation value calculation formula is provided as a component for performing interpolation calculation of image data. The input data control unit 30 generates interpolation calculation input data Dh having different contents depending on the type of input image data, and the interpolation calculation input data Dh is supplied to the interpolation processing unit 40. The interpolation processing unit 40 is supplied with different values of interpolation coefficients depending on the type of input image data. Thus, the interpolation value P corresponding to the type of input image data is calculated by one interpolation processing unit 40. In addition, when the interpolation calculation is substantially performed by the bilinear method, power is not consumed in the portion where the zero value calculation is performed. Therefore, suitable interpolation processing is performed for each type of input image data without increasing the circuit scale and power consumption.

  In the present embodiment, the interpolation coefficient necessary for calculating the interpolation value is held in the coefficient unit 50 in advance. Although the calculation formula for calculating the interpolation coefficient in the bicubic method is relatively complicated, the processing time is shortened by using the interpolation coefficient held in the coefficient unit 50 in the interpolation calculation.

  Further, in the present embodiment, for the luminance component Y of the image data, the interpolation value P is calculated based on the interpolation value calculation method based on the bicubic method, and for the color difference components Cb and Cr of the image data, substantially. The interpolation value P is calculated based on the interpolation value calculation method by the bilinear method. For this reason, the interpolation processing of the image data is performed without reducing the image quality and without reducing the processing speed.

<6. Other>
In the above embodiment, the description has been made on the assumption that the YUV method is adopted as the format of the image data, but the present invention is not limited to this. For example, the present invention can be applied even if the RGB format or YIQ format is adopted as the format of the image data. In the above embodiment, for the luminance component Y of the image data, the interpolation value P is calculated from the image data for 16 points around the interpolation point and the bicubic interpolation coefficient, and the color difference component of the image data. For Cb and Cr, the interpolation value P is calculated from the image data for the four points around the interpolation point and the bilinear interpolation coefficient, but the present invention is not limited to this.

It is a conceptual diagram for demonstrating an interpolation point, a 1st pixel block, and a 2nd pixel block. It is a figure for demonstrating the interpolation value calculation method by a bicubic method. It is a figure for demonstrating the interpolation value calculation method by a bilinear method. It is a figure for demonstrating the interpolation value calculation method by the nearest neighbor method. It is a figure for demonstrating the interpolation method in this invention. It is a figure for demonstrating the interpolation method in this invention. It is a block diagram which shows the principal part of the image processing apparatus to which this invention was applied. 1 is a block diagram illustrating an overall configuration of an image processing apparatus according to an embodiment of the present invention. In the said embodiment, it is a block diagram which shows the structure of an interpolation process part. In the said embodiment, it is a figure for demonstrating the input-output data of the 1st-4th arithmetic unit. In the said embodiment, it is a circuit diagram which shows the structure of a 1st arithmetic unit. In the said embodiment, it is a flowchart which shows the procedure of a size conversion process. In the said embodiment, it is a flowchart which shows the procedure of an interpolation value calculation process. It is a block diagram which shows the structure of the principal part of the image processing apparatus which concerns on a prior art example. 1 is a block diagram illustrating a configuration of an image processing apparatus disclosed in Japanese Patent Laid-Open No. 9-298600. 1 is a block diagram illustrating a configuration of an image processing apparatus disclosed in Japanese Patent Laid-Open No. 2001-14454.

DESCRIPTION OF SYMBOLS 10 ... Control part 11 ... Control register 20 ... Image input part 30 ... Input data control part 40 ... Interpolation processing part 41-44 ... 1st-4th arithmetic unit 50 ... Coefficient part 60 ... Output data control part 70 ... Image output Unit 100 ... Size conversion processing unit 200 ... Input device 300 ... Image memory 400 ... Output device 500 ... Control processor Cb, Cr ... Color difference signal (color difference component)
Dh: Interpolation calculation input data Di: Input image data Do: Output image data Hk: Interpolation coefficient P: Interpolation value Y: Luminance signal (luminance component)

Claims (10)

An image processing apparatus that performs an interpolation calculation of image data based on a predetermined interpolation calculation expression,
According to the type of image data that is the subject of the interpolation calculation, an interpolation calculation input data generation unit that generates interpolation calculation input data to be substituted into the interpolation calculation formula;
An interpolation calculation unit that performs the interpolation calculation by substituting the interpolation coefficient calculated based on a predetermined interpolation coefficient calculation expression according to the type of the image data and the input data for interpolation calculation into the interpolation calculation expression; Prepared ,
The image data includes a first image component and a second image component,
The interpolation calculation input data generation unit includes:
When the image data to be subjected to the interpolation calculation is the first image component, the data is 16 points around the interpolation point whose value is to be obtained by the interpolation calculation, and is a third-order convolution interpolation. 16 points of data necessary for the interpolation calculation by the method are generated as the input data for the interpolation calculation,
When the image data to be subjected to the interpolation calculation is the second image component, four points around the interpolation point and four points necessary for the interpolation calculation by the bilinear interpolation method And data having data other than the four points surrounding the interpolation point out of the 16 points surrounding the interpolation point as zero value are generated as the input data for the interpolation calculation,
The interpolation calculation unit is configured to perform the cubic convolution as the predetermined interpolation calculation formula when the image data to be subjected to the interpolation calculation is either the first image component or the second image component. An image processing apparatus that performs the interpolation calculation based on an interpolation calculation expression based on an interpolation method .   An interpolation coefficient holding unit that holds an interpolation coefficient corresponding to the type of the image data in advance, and applies the interpolation coefficient held in advance to the interpolation calculation unit according to the type of the image data that is an object of the interpolation calculation; The image processing apparatus according to claim 1, further comprising:   The image processing apparatus according to claim 1, wherein the image data is YUV format data including a luminance component as the first image component and a color difference component as the second image component. The interpolation calculation unit includes four calculation units having the same configuration,
Each arithmetic unit obtains an intermediate value for calculating an interpolation value based on five data as the interpolation coefficient and four data as the input data for interpolation calculation by the following formula: The image processing apparatus according to claim 1.
P = (P1 · f (x1) + P2 · f (x2) + P3 · f (x3) + P4 · f (x4)) · f (y)
Here, P is the intermediate value, f (x1), f (x2), f (x3), f (x4), and f (y) are five data as the interpolation coefficients, P1, P2, P3, And P4 are four data as the input data for the interpolation calculation. 5. The arithmetic unit according to claim 4, wherein each arithmetic unit includes five multipliers that perform multiplication processing to obtain the intermediate value and three adders that perform addition processing to obtain the intermediate value. Image processing apparatus. An image data interpolation method for performing an interpolation calculation based on a predetermined interpolation calculation formula,
Interpolation calculation input data generation step for generating interpolation calculation input data to be substituted into the interpolation calculation formula according to the type of image data to be subjected to the interpolation calculation;
An interpolation coefficient calculation step for calculating an interpolation coefficient to be substituted into the interpolation calculation formula according to the type of image data to be subjected to the interpolation calculation;
An interpolation calculation step for performing the interpolation calculation by substituting the interpolation calculation input data and the interpolation coefficient into the interpolation calculation formula ,
The image data includes a first image component and a second image component,
In the interpolation calculation input data generation step,
When the image data to be subjected to the interpolation calculation is the first image component, the data is 16 points around the interpolation point whose value is to be obtained by the interpolation calculation, and is a third-order convolution interpolation. 16 points of data necessary for the interpolation calculation by the method are generated as the input data for the interpolation calculation,
When the input image data to be subjected to the interpolation calculation is the second image component, it is data of four points around the interpolation point and is necessary for the interpolation calculation by the bilinear interpolation method. Point data and data in which data other than the four points around the interpolation point out of the 16 points around the interpolation point are generated as the input data for the interpolation calculation,
In the interpolation calculation step, the cubic convolution is used as the predetermined interpolation calculation formula regardless of whether the image data to be subjected to the interpolation calculation is the first image component or the second image component. An interpolation method, wherein the interpolation calculation is performed based on an interpolation calculation formula by an interpolation method.   An interpolation coefficient acquisition step of acquiring, in place of the interpolation coefficient calculation step, an interpolation coefficient held in advance according to the type of image data that is a target of the interpolation calculation. 6. The interpolation method according to 6.   7. The interpolation method according to claim 6, wherein the image data is YUV format data including a luminance component as the first image component and a color difference component as the second image component. The interpolation calculation step includes four calculation steps for performing the same calculation,
In each calculation step, an intermediate value for calculating an interpolation value is obtained by the following equation based on the five data as the interpolation coefficient and the four data as the input data for interpolation calculation. The interpolation method according to claim 6.
P = (P1 · f (x1) + P2 · f (x2) + P3 · f (x3) + P4 · f (x4)) · f (y)
Here, P is the intermediate value, f (x1), f (x2), f (x3), f (x4), and f (y) are five data as the interpolation coefficients, P1, P2, P3, And P4 are four data as the input data for the interpolation calculation. 10. The calculation step according to claim 9, wherein each calculation step includes five multiplication steps for performing multiplication processing for obtaining the intermediate value and three addition steps for performing addition processing for obtaining the intermediate value. Interpolation method.

Images