JP2017215941A - Image processor and control method thereof - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processor capable of carrying out one-dimensional cubic interpolation, which is a primitive calculation of a two-dimensional bicubic interpolation, which is frequently used as a high quality image interpolation processing of an image, with a multiplier having a small circuit scale.SOLUTION: The image processor includes: a first linear interpolation computing unit that generates an interpolation pixel value at an interpolation position by means of linear interpolation from pixel values of a pair of pixels in plural pixels; plural second linear interpolation computing units that generate plural correction values by carrying out a linear interpolation from the pair of differential values in the plural differential values among the pixel values of the plural pixels; and a computing unit that generates a pixel value of the interpolation position based on the interpolation pixel value and the plural correction values.SELECTED DRAWING: Figure 2

Description

  The present invention relates to a technique for interpolating a pixel value of an intermediate coordinate from a pixel value of peripheral discrete coordinates.

  The imaging apparatus forms an image of light from a subject on an image sensor using an optical lens, and converts the light into an electrical signal using the image sensor. However, geometric distortion (hereinafter abbreviated as “geometric distortion”) tends to occur in the peripheral portion of the obtained image due to factors such as allowable accuracy at the time of designing the lens and variation accuracy at the time of manufacturing. As the number of imaging pixels increases to tens of millions, this distortion cannot be ignored.

  On the other hand, when processing and editing captured images for viewing on a display such as a television, there is no particular problem. However, when projecting onto a screen with a projector, the following factors can cause geometric problems. Distortion may occur.

  If the display and the screen are not strictly facing each other, the rectangle is transformed into a trapezoid or an unequal square and projected. In addition, the optical system of the display has a mechanical accuracy at the time of attachment and an accuracy related to the optical lens described above, and image distortion due to the accuracy occurs. As described above, various geometric distortions are superimposed on an image due to various factors during imaging and projection.

  Since an image is assumed to be viewed on both a display and a projector, the geometric distortion at the time of imaging needs to be corrected at the time of imaging, and the geometric distortion at the time of projector projection needs to be corrected at the time of projection. Each geometric distortion can be reduced by digital image processing.

  Some of the data and parameters necessary for geometric distortion correction can be obtained from simulation at the design stage, but generally obtained from a process called calibration. There are various methods for calibration, which is one technical area. For example, trapezoidal distortion correction during projector projection obtains the four vertex (four corners) coordinates of a distorted rectangle when the original rectangular image is projected without correction, and calculates a correction homography matrix from this information By doing so, parameters necessary for geometric correction are obtained.

  This specification is based on the premise that data and parameters necessary for geometric distortion correction have already been obtained by calibration or the like. That is, as described above, there are various methods for calibration, and it does not matter which one is used.

  In the block diagram of the geometric distortion correction process shown in FIG. 1, it is assumed that data and parameters necessary for correction have already been obtained by calibration and stored in the conversion parameter holding unit 101.

Whatever the calibration is, the resulting geometric distortion correction is composed of two main elements. One is a component related to coordinate transformation for geometrically deforming an image, and the other is a configuration related to coordinate interpolation for calculating a pixel value at a coordinate position below the decimal point generated by the configuration by interpolation. Is an element. Each is performed by the coordinate conversion unit 103 and the pixel interpolation unit 112 of FIG. The correction of geometric distortion is processed as follows in the block diagram of FIG.
(A1) The integer coordinate 102 when the image is regularly scanned is converted by the coordinate conversion unit 103, and the converted coordinate 104 is calculated.
(A2) The pixel interpolation unit 112 generates an interpolation pixel value for the converted coordinate.
(A3) The generated interpolation pixel value is stored in the image memory 113 as the pixel value of the integer coordinate 102.

  Since the converted coordinates 104 are generally coordinate data having a decimal part, it is necessary to read out and refer to pixel values of peripheral integer coordinates from the image memory 111 and generate them by interpolation processing.

  Typical interpolation methods include nearest neighbor, bilinear interpolation, and bicubic interpolation. From the viewpoint of the degree of polynomial interpolation, these are classified into 0th order interpolation, 1st order interpolation, and 3rd order interpolation, respectively. The higher the order, the better the interpolation accuracy and the better the image quality.

  As another interpolation method classified as cubic interpolation, there is a method called cubic spline interpolation, which is an interpolation method based on a spline function as the name indicates. The number of pixels to be referred to at the time of interpolation processing is 1 pixel, 4 pixels, and 16 pixels obtained by squaring the value obtained by adding 1 to the order, and the calculation amount and circuit scale are approximately proportional to the number of reference pixels.

  Most of the interpolation processing actually used is bilinear interpolation or bicubic interpolation, and there are many cases in which one is selected depending on whether image quality or circuit scale is prioritized. That is, in terms of image quality, it is desirable to use bicubic interpolation, but there are many cases in which bilinear interpolation is performed due to circuit scale limitations.

  Patent Document 1 attempts to reduce the circuit scale by realizing both functions of bilinear interpolation and bicubic interpolation with a single interpolation circuit, but the circuit scale itself of bicubic interpolation is reduced. is not.

  Patent Document 2 shows that cubic spline interpolation, which is one of cubic interpolation, can be realized with a configuration in which linear interpolation calculators are connected in three stages. However, the cubic spline interpolation does not have a function of adjusting the weighting coefficient of the reference 4 × 4 pixels, and the image quality cannot be adjusted according to the system in which the interpolation circuit is used.

  Bicubic interpolation has parameters that allow image quality adjustment, and image creation suitable for the system is possible. Prior art cubic spline interpolation schemes cannot be applied to bicubic interpolation.

JP 2007-219950 A JP 2007-87258 A

  To summarize the above, bicubic interpolation has a problem that the image quality is good but the circuit scale is large. In a three-plate projector, it is necessary to perform geometric correction different for R, G, and B, and it is also necessary to have an interpolation circuit for geometric correction for each channel.

  Therefore, several pixel interpolation processes are required in one video system, and if bicubic interpolation is employed, the arithmetic processing of the entire video system increases.

In order to solve this problem, for example, an image processing apparatus of the present invention has the following configuration. That is,
First linear interpolation means for generating an interpolation pixel value at an interpolation position from a pixel value of a pair of pixels among a plurality of pixels by linear interpolation;
A plurality of second linear interpolation means for generating a plurality of correction values by linearly interpolating each of the plurality of difference values between the pixel values of the plurality of pixels from a pair of difference values;
Generating means for generating a pixel value at the interpolation position based on the interpolation pixel value and the plurality of correction values.

  According to the present invention, it is possible to realize an interpolation operation such as one-dimensional cubic interpolation, which is a primitive operation of two-dimensional bicubic interpolation, which is often used as an image quality high-quality interpolation process, with a circuit configuration with few multipliers. Become.

The block diagram of a general geometric distortion correction process. The figure showing the structure of the cubic interpolation calculator of 1st Embodiment. The figure which shows the structure of a bicubic interpolation calculator using the cubic interpolation calculator of 1st Embodiment. The figure showing the structure of the 6 pixel reference cubic interpolation calculator in 2nd Embodiment. The figure showing the structure of the 8-pixel reference cubic interpolation calculator in 2nd Embodiment. The figure which compared the impulse response of 6 pixel reference cubic interpolation calculators and Lanczos-3. The figure which compared the impulse response of 8-pixel reference cubic interpolation calculator and Lanczos-4. The figure showing the structure of the cubic interpolation calculator of 3rd Embodiment. The figure showing the structure of the cubic interpolation calculator of 4th Embodiment. The figure showing the structure of the cubic interpolation calculator of 5th Embodiment. The figure showing the relationship between the interpolation position common to each embodiment, and the pixel referred by interpolation calculation. The figure showing the structure of the cubic interpolation calculator of 6th Embodiment. The figure showing the modification of the cubic interpolation calculator of 6th Embodiment. The figure showing the 2nd modification of the cubic interpolation calculator of 6th Embodiment.

  Hereinafter, embodiments according to the present invention will be described in detail with reference to the accompanying drawings. In the present embodiment, an example will be described in which a pixel value at a coordinate position that takes a value after the decimal point is applied to a pixel interpolation circuit that performs interpolation calculation from pixel values of 2N integer coordinates around it.

  Specifically, the pixel interpolation circuit is provided with a first linear interpolation calculator that generates an interpolation pixel value at an interpolation position by linear interpolation from the pixel values of a pair of pixels among a plurality of pixels. Then, a plurality of second linear interpolation calculators are provided that generate a plurality of correction values by performing linear interpolation from a pair of difference values among a plurality of difference values between pixel values of a plurality of pixels. Furthermore, an arithmetic unit that generates a pixel value at the interpolation position based on the interpolation pixel value and a plurality of correction values is provided.

[First Embodiment]
The first embodiment is applied to an interpolation calculation when generating an interpolation pixel with reference to pixel values of surrounding n × n pixels. Specifically, it applies to the first-stage cubic interpolation performed n times in units of n pixels on the same straight line in the vertical or horizontal axis direction, and the second-stage cubic interpolation performed by referring to the n interpolation results. It is what is done.

ここで、パラメータｓは、補間画素の座標から周辺参照画素までの符号付き距離を表している。座標が増加する方向は＋の符号で、減少する方向は−の符号が付く。また、パラメータａは、補間特性を調整するもので、本文中では「第１の画質調整パラメータ」と呼んでいる。 In general bicubic interpolation that refers to 4 × 4 pixels, based on Equation (1), for the coordinate position after the decimal point, four load coefficients to the pixel to be referred to in each axis direction, a total of eight coefficients. calculate.

Here, the parameter s represents a signed distance from the coordinates of the interpolation pixel to the peripheral reference pixel. The direction in which the coordinates increase is indicated by a plus sign, and the direction in which the coordinates decrease is indicated by a minus sign. The parameter a is used to adjust the interpolation characteristic and is referred to as “first image quality adjustment parameter” in the text.

When calculated in this way, a total of 16 multiplications are required for the calculation of | s | ² and | s | ³ , and there are 20 multiplications for the parameter a, and a total of 36 multiplications can be obtained simply by calculating 8 coefficients. is necessary. However, since the number of bits of the parameter a is generally small, the circuit scale of this multiplication is small.

  When the coefficient is multiplied to the reference pixel, if it is performed in two stages in each axis direction, 16 times are required for the first stage coefficient multiplication, and 4 times are required for the second stage coefficient multiplication, for a total of 20 times. Become.

  In total, 16 + 20 = 36 multiplications with a large circuit scale and 20 multiplications with a small circuit scale are required.

  In the present embodiment, the number of multiplications is reduced and the circuit scale is reduced by calculating interpolation pixels equivalent to cubic interpolation based on Expression (1) by a method different from the above description.

  In the cubic interpolation calculation of the present embodiment, the decimal part obtained by removing the integer part from the transformed coordinates (x, y) is set as a primary parameter (tx, ty), and tx · (1−tx) is given to the parameter. A quadratic polynomial parameter based on a quadratic polynomial of ty · (1-ty) is calculated.

  There are only two parameters to be calculated, and the other main operations are parallel operations of linear interpolation based on the primary parameter tx or ty.

  The calculation of the secondary parameter and the parallel operation of linear interpolation based on the primary parameter are operations common to all the embodiments described in this specification. In each embodiment described below, the secondary parameter is used as a calculated parameter, and the generation method thereof is omitted.

  In the first embodiment, which is the first, three component signals are generated by parallel calculation of linear interpolation based on the primary parameter. The configuration of the first embodiment is shown in FIG.

FIG. 11 shows the relationship between the arrangement order of 4 pixels referred to in the first embodiment and 6 and 8 pixels referred to in other embodiments described later, and the primary parameter t (interpolation position information) representing the interpolation position. Show. Here, the unit distance between the pixels is “1”. Therefore, it can be said that the primary parameter t has a relationship of 0 ≦ t ≦ 1. In all the embodiments, the interpolation position is determined between P ₃ and P ₄ , and a pixel value corresponding to the interpolation position is generated by interpolation calculation. In general, the pixel value at a specified position in the interpolation section between the Nth pixel and the (N + 1) th pixel in 2N pixels arranged in series is interpolated.

The calculation contents of the cubic interpolation calculator shown in FIG. When the calculation of the linear interpolation calculator having the primary parameter “t” and the two inputs {P _i , P _j } as inputs is expressed by Expression (2), the output P _out of the cubic interpolation calculator shown in FIG. It can be expressed as equation (3).

  Reference numerals 201, 202, and 203 are linear interpolation calculators that perform linear interpolation between two pixels (between input signals) separated by one pixel based on the primary parameter t, each having one component. Is generated. As the primary parameter t, tx or ty is input.

The linear interpolation calculator 201 calculates (1-t) · P ₃ + t · P ₄ and outputs it for the inputs P ₃ and P ₄ . This is linear interpolation connecting P ₃ and P ₄ . The linear interpolation calculator 201 performs an operation based on a calculation formula of P ₃ + t · (P ₄ −P ₃ ) so that an actual operation only needs to be performed once.

Similarly to the linear interpolation calculator 201, the linear interpolation calculators 202 and 203 also perform calculations on the illustrated input signal so that the number of multiplications is one. It should be noted that in the drawing, subtracters for generating input signals to the linear interpolation calculators 202 and 203 such as “P ₃ -P ₅ ” are omitted.

If the pixel value is defined only in integer coordinates before the interpolation process, the linear interpolation calculator 201 generates a component that linearly connects the discrete values P ₃ and P ₄ . That is, the linear interpolation calculator 201 can be interpreted as generating a component that makes discrete pixel values continuous. On the other hand, the linear interpolation calculators 202 and 203 can be interpreted as follows.

  When adjacent pixel values are connected linearly one after another, the waveform becomes uneven, and the first derivative of the waveform becomes a discontinuous waveform at integer coordinate positions. A component that makes the first derivative continuous is generated as a second component by the linear interpolation calculator 202.

Assuming that the distance between adjacent pixels is 1, the slope of the first component waveform when viewing from the direction P ₃ to P ₄ is “P ₄ −P ₃ ”, and the signal for canceling the slope is “− (P ₄ − P ₃ ) ”. Similarly, the slope of the waveform when viewing from P ₄ to P ₃ is “P ₃ −P ₄ ”, and the signal for canceling the slope is “− (P ₃ −P ₄ )”.

The linear interpolation calculator 202 receives the tilt cancellation signal at P _{3 and} the tilt cancellation signal at P ₄ as input and switches them linearly based on the primary parameter t.

If the switched signal is directly added to the output of the linear interpolation calculator 201, the values P ₃ and P ₄ in integer coordinates are changed, so that at t = 0, 1, the second component becomes the first component. It is necessary not to affect it. Hereinafter, this is expressed as a “boundary condition” in the embodiment. In order to satisfy the boundary condition, it is necessary and sufficient to multiply the second component by the secondary parameter. Reference numeral 231 in FIG. 1 is a multiplier for performing the multiplication.

  Now, when a linear interpolation signal based on the parameter t is multiplied by a secondary parameter related to t, a signal expressed by a cubic polynomial of t is obtained. This feature can be applied to the signal generated by the linear interpolation calculator 203. Therefore, the adder 221 adds the second component and the third component so that the signal generated by the linear interpolation calculator 203 is also multiplied by the secondary parameter, and then the multiplier 231 sets the secondary parameter. It was configured to multiply.

  By the way, even if the first derivative is made continuous, the second derivative is still discontinuous, and the second derivative is expected to be continued after the first derivative is made continuous. However, it is difficult to generate a component that makes the second derivative continuous by simply referring to four pixels. However, it is possible to reduce the discontinuity of the second derivative. The linear interpolation calculator 203 generates a third component that alleviates the discontinuity of the second derivative.

Linear characteristics of the input signal in the interpolation calculator 203, a difference signal P ₃ and P ₅ apart by one pixel to the right and left P ₄ as the base point, P ₄ apart by one pixel to the right and left of the P ₃ as a base point and P _{2 is} a difference signal.

The sum of the first component and the second component always has a property of being within the range of the pixel values P ₃ and P ₄ at both ends of the interpolation interval, but may exceed the range when the third component is added. . In other words, the third component can enhance the gradation difference by adjusting the amplitude of the component, and vice versa, and can adjust the image quality. In order to adjust the amplitude of the third component, the multiplier 211 multiplies the first image quality adjustment parameter Ka. The value of this parameter is sufficient in the range of 8/16 to 31/16, and a multiplier having a coefficient of about 5 bits may be used.

  It can be confirmed that the cubic interpolation calculation of the first embodiment is mathematically equivalent to the cubic interpolation based on the equation (1) by changing the equation with Ka = -a.

FIG. 3 shows a configuration in which bicubic interpolation processing is performed on 16 pixels (P _{00 to} P ₃₃ ) of 4 rows × 4 columns using five cubic interpolators with 4-pixel reference described above. Here, the coordinates of the pixel P _ij are (x _i , y _j ), and the coordinates of the interpolation pixel are (x, y). In the configuration shown in FIG. 3, four columns of pixel data are processed in parallel by four cubic interpolation calculators 301 to 304. Then, the four interpolation results obtained are cubically interpolated in the row direction by the cubic interpolation calculator 305 to obtain an interpolation result of coordinates (x, y). The same parameters (ty, ty · (1-ty), Ka in the drawing) are supplied to the four cubic interpolation calculators 301 to 304 that perform parallel processing. By performing pipeline processing with this configuration, it is possible to generate pixels that have undergone bicubic interpolation calculation for each cycle. Further, each row of 16 pixels of 4 rows × 4 columns is processed in parallel to obtain an interpolation result at four coordinate positions that are the same horizontal position as the interpolation pixel, and the obtained result is interpolated in the column direction. Also good.

[Second Embodiment]
There is an interpolation filter obtained by multiplying a basic sinc function (also referred to as an interpolation function) by a super-period sinc function as a window function. It is a filter called Lanczos interpolation. Cubic interpolation is said to be an approximation of this Lanczos interpolation.

  When performing the enlargement interpolation process using the Lanczos filter, if the number of reference pixels in one axis direction is 2n, a Lanczos-n filter is used. Generally used is Lanczos-3 which refers to 6 pixels. On the other hand, cubic interpolation is based on 4-pixel reference.

  An equation corresponding to the calculation of the weighting factor of the above equation (1) for extending the number of reference pixels for cubic interpolation to 6 pixels has been published in the paper, and in principle it can be expanded to 6 pixels. is there. However, there is no document showing the specific circuit configuration so far. Compared to a 4-pixel reference, the calculation of the six weighting factors is more complicated, which is considered to be less practical.

  However, in the present embodiment, the number of reference pixels for cubic interpolation in the uniaxial direction can be easily expanded to 6 pixels and 8 pixels. FIG. 4 shows a circuit configuration for cubic interpolation with reference to 6 pixels, and FIG. 5 shows a circuit configuration for cubic interpolation with reference to 8 pixels.

  The validity of the extension according to the present embodiment can be determined in terms of how close the cubic interpolation of the present embodiment is to the Lanczos interpolation based on the fact that the cubic interpolation is an approximation of the Lanczos interpolation.

First, the concept of expansion will be described, and then how close to the characteristics of Lanczos interpolation will be shown. Three basic ideas for expansion are shown in (1) to (3).
(1) The new signal component is multiplied by the secondary parameter so as to satisfy the boundary condition.
(2) The linear interpolation calculation is performed at the first stage so that the new signal component becomes a cubic polynomial.
(3) Following the form of the input signal of the third linear interpolation calculator, a difference signal between two pixels that are equidistant from the left and right from P ₃ and P ₄ is input to the linear interpolation calculation.

  The cubic interpolator with 6 pixels shown in FIG. 4 includes a fourth linear interpolation arithmetic unit 204 that operates in parallel, and calculates two inter-pixel difference signals to be input to the arithmetic unit as follows.

One is a difference “P ₂ −P ₆ ” between two pixels separated left and right from the P ₄ pixel as a base point, and the other is two pixels separated right and left from the P ₃ pixel as a base point. The difference between the two pixels is “P ₅ −P ₁ ”. These are input to the fourth linear interpolation calculator 204 to perform a linear interpolation calculation, and the output is multiplied by the second image quality adjustment parameter Kb. Multiply

  In addition, the 8-pixel reference cubic interpolation calculator shown in FIG. 5 is provided with a fifth linear interpolation calculator 205 operating in parallel to the 6-pixel reference cubic interpolation calculator shown in FIG. Two inter-pixel difference signals input to 205 are calculated as follows.

One is a difference “P ₁ -P ₇ ” between two pixels separated by 3 pixels left and right with the P ₄ pixel as the base point, and the other is ₃ pixels right and left with the P ₃ pixel as the base point. The difference between the two pixels is “P ₆ −P ₀ ”. These are input to the fifth linear interpolation calculator 205 to perform a linear interpolation calculation, and the output is multiplied by the third image quality adjustment parameter Kc. Multiply Similarly, the reference pixel is further expanded by using the difference between the pixel values of the pixels separated by the same number of pixels on the left and right from the base point and separated by 2, 4,..., 2 · (N−1) pixels. Can also be easily done.

When the output P _out of each of the cubic interpolation calculators shown in FIG. 4 and FIG. 5 described above is expressed by equations, equations (4) and (5) are obtained.

  In the cubic interpolation with 6-pixel reference shown in FIG. 4, the impulse response when the two image quality adjustment parameters are set to Ka = 14/16 and Kb = -3 / 16 is compared with Lanczos-3. The waveform shown in FIG.

  In the cubic interpolation with 8-pixel reference shown in FIG. 5, the impulse response when the three image quality adjustment parameters are set to Ka = 31/32, Kb = −12 / 32, and Kc = 3/32 The waveform shown in FIG.

  It can be seen that interpolation filter characteristics close to Lanczos-3 and Lanczos-4 can be realized by appropriately setting the image quality adjustment parameter group. Further, in actual use, it is possible to optimize the system by adjusting the parameters while confirming the image quality based on the parameter values.

  In the Lanczos interpolation filter, the filter coefficient is uniquely determined in accordance with the number of reference pixels and image quality adjustment cannot be performed. On the other hand, the cubic interpolation of the second embodiment allows easy image quality adjustment. This is a great advantage.

[Third Embodiment]
In the first embodiment, since the basic idea of pixel interpolation calculation is directly circuitized, the configuration is most easily understood.

  In the third embodiment, the calculation formula of the first embodiment is slightly modified to realize an operation equivalent to that of the first embodiment with fewer multipliers.

  Specifically, by using the internal calculation result of the first linear interpolation calculator 201, the second component is generated without using the second linear interpolation calculator 202. A specific configuration is shown in FIG.

The second linear interpolation calculator 202 in FIG. 2 linearly interpolates “− (P ₄ −P ₃ )” and “− (P ₃ −P ₄ )” based on the parameter t. , “(2t−1) · (P ₄ −P ₃ )”.

In the first embodiment, the actual calculation of the first linear interpolation calculator 201 is calculated by the calculation formula “t · (P ₄ −P ₃ ) + P ₃ ” so that only one multiplication is required. Stated.

It is obvious that the first linear interpolation calculator 201 calculates “(P ₄ −P ₃ )” and “t · (P ₄ −P ₃ )” in the course of the calculation. Can be used. If the latter “t · (P ₄ −P ₃ )” is doubled, that is, shifted to the left by 1 bit by the shift computing unit 301, and “(P ₄ −P ₃ )” is subtracted by the subtractor 302, “ (2t-1) · (P ₄ −P ₃ ) ”can be obtained. Therefore, it is not necessary to use the second linear interpolation calculator having a multiplier.

  The configuration shown in the third embodiment can be applied to the second embodiment and the fourth embodiment to be described next.

[Fourth Embodiment]
In the first and second embodiments, the input of the third linear interpolation calculator 203 is a difference value between pixels that are two pixels apart, and the input of the fourth linear interpolation calculator 204 is four pixels apart. The difference value between the pixels. In any case, it can be said that the difference between the pixels, that is, only the distance between the two pixels taking the first derivative is changed.

  Such a configuration is suitable for minimizing a difference in configuration when an n-pixel reference cubic interpolation calculator is expanded to an n + 2 pixel-reference cubic interpolation calculator. However, the input of each linear interpolation calculator is not limited to the form of the first and second embodiments.

  For example, the input of the second linear interpolation calculator is interpreted as the primary differential signal of the reference pixel value, the secondary differential signal of the reference pixel value is input to the third linear interpolation calculator, and the fourth linear interpolation calculator It is also possible to input the third derivative signal of the reference pixel value. Then, the meaning of the signal component generated by each linear interpolation calculator also changes.

Specifically, as shown in FIG. 9, the third linear interpolation calculator 203 receives (P ₄ −P ₅ ) − (P ₃ −P ₄ ) = 2 · P ₄ −P ₃ as a secondary differential signal. and _{-P 5, (P 3 -P 2} ) - enter the _{(P 4 -P 3) = 2} · P 3 -P 4 -P 2.

Then, the fourth linear interpolation operator 204 receives (2 · P ₅ −P ₆ −P ₄ ) − (2 · P ₃ −P ₄ −P ₂ ) = P ₂ −2 · (P ₃₋ P ₅ ) −P ₆ and (2 · P ₂ −P ₁ −P ₃ ) − (2 · P ₄ −P ₃ −P ₅ ) = P ₅ −2 · (P ₄ −P ₂ ) −P Enter ₁ .

In this case, the parameter for adjusting the amplitude of each component is multiplied by different parameters R ₁ , R ₂ and R ₃ different from those in the first and second embodiments. As specific values, when R ₁ = 0, R ₂ = 8/16, and R ₃ = −3 / 16, the impulse response characteristics shown in FIG. 6, that is, characteristics close to Lanczos-3 are obtained.

[Fifth Embodiment]
In the configuration of FIG. 4, there are three linear interpolation calculators before multiplication by the secondary parameter, but these can be combined into one.

  Since the linear interpolation calculation based on the parameter t, the multiplication of the image quality adjustment parameter, and the addition process are all linear calculations, the calculation order can be exchanged. Therefore, the linear interpolation calculation is moved after the addition processing, and the processing order is changed to multiplication of image quality adjustment parameters, addition processing, and linear interpolation calculation.

  Thereby, a plurality of linear interpolation operations can be combined into one, and the configuration shown in FIG. 10 can be obtained. In FIG. 10, calculators 501 and 502 are weighted adders that add weights to input signals, and are linear interpolation calculators combined by the calculator 503. The rest is the same as FIG. 4 of the second embodiment.

  The weighting for the input signal reflects the image quality adjustment parameters Ka and Kb in FIG. 4 of the second embodiment, and the configuration of FIG. 10 is functionally equivalent to the configuration of FIG.

  The method of the fifth embodiment can also be applied to the 8-pixel reference cubic interpolation shown in FIG. That is, the weighting adders 501 and 502 may be expanded to 8 pixel inputs and weighting reflecting the image quality adjustment parameter Kc may be performed.

[Sixth Embodiment]
The linear interpolation calculation of the difference signal between the pixels in the three cubic interpolation formulas (3) to (5) described so far can be performed by switching the order of the difference calculation and the linear interpolation calculation. It can be replaced with signal difference calculation. At this time, the linear interpolation is performed between the pixels in which two pixels whose sum of the subscripts of the pixel is 7 are paired. This is because linear interpolation calculation is performed between two pixels having a symmetrical positional relationship with an interpolation interval (between the pixels P3 and P4 in the embodiment) as a center.

When the above-described replacement of the calculation order is applied to the equations (3), (4), and (5), these can be transformed as the following equations (6), (7), and (8).

Expressions (6) and (7) are obtained by cutting out a part of Expression (8). Therefore, as a representative, the configuration of the arithmetic circuit of Expression (8) is shown in FIG. In the figure, reference numerals 601 to 607 are linear interpolation calculators, reference numerals 611 to 617 are subtractors, and the other components are the same as those of the same reference numerals in FIG. 5 of the second embodiment. It is.

  It can be seen from the configuration of FIG. 12 that if equation (8) is simply converted into an arithmetic circuit, the number of linear interpolation calculators increases from four to seven slightly less than twice. With this configuration, the number of multipliers used in the linear interpolation calculator increases, and the circuit scale also increases.

Incidentally, a linear interpolation of the input pixel P ₄ and P _3, the linear interpolation and the result of adding the of P ₃ and P ₄ replacing the input with each other, that constant P ₃ + P ₄ regardless of the interpolation position There is a relationship. A circuit configuration of cubic interpolation calculation using this relationship and having reduced multipliers is shown below.

  Here, two linear interpolation values before and after the two inputs of the linear interpolation calculator are exchanged are expressed as a linear interpolation pair, and an arithmetic unit that calculates the linear interpolation pair is expressed as a linear interpolation pair arithmetic unit. Express.

  In FIG. 12, three pairs of linear interpolation calculators 601 and 602, 603 and 604, and 605 and 606 are linear interpolation pair calculators.

Each linear interpolation pair calculation unit can reduce multiplication from two times to one time by calculating as shown in the following equation (9). It can also be seen that adding the right sides of the two equations gives P _i + P _j .

When the calculation is performed as described above, the calculation up to the term multiplied by t can be made common, and then the linear interpolation pair can be calculated only by performing addition and subtraction once. Since a linear interpolation pair can be calculated and generated with a circuit scale in which only one subtractor is added to a normal linear interpolation calculator, it is very efficient.
The configuration shown in FIG. 13 is a representation of the calculation unit as a single calculation unit called a linear interpolation pair calculation unit. Three reference numerals 621, 622, and 623 in the figure are linear interpolation pair calculators that perform the calculation of Expression (9).

この場合、２つ目の式の符号が式（９）と反転した関係になる。式（１０）の演算を行う線形補間対演算器を用いた場合は、減算器６１１〜６１４を加算器に置き換えれば、図１３の構成と等価になる。 The above equation (9) may be calculated as the following equation (10).

In this case, the sign of the second equation is reversed from that of Equation (9). When a linear interpolation pair computing unit that performs the calculation of Expression (10) is used, the configuration of FIG. 13 is equivalent if the subtracters 611 to 614 are replaced with adders.

  Furthermore, when the sign of the parameter to be multiplied by the image quality adjustment parameter multipliers 211 to 213 is inverted, it is necessary to replace the subtracters 615 to 617 with adders.

  As described above, the subtracters 611 to 617 are those that are replaced with adders depending on the previous stage of calculation and the sign of the parameters, and determine the type of the arithmetic unit so that the entire arithmetic circuit functions as a cubic interpolation calculation. There is a need.

  As an example, FIG. 14 illustrates a configuration of a 4-pixel reference cubic interpolation calculation circuit that performs the interpolation calculation of Expression (6) using the linear interpolation pair calculation unit 631 that performs the calculation of Expression (10). In the figure, adders 641 and 642 replace the subtracters 611 and 612 in FIG.

  In the configurations of the first and second embodiments, it is necessary to use the technique of the third embodiment together in order to further reduce the number of multipliers required for the linear interpolation calculation. On the other hand, the configuration of FIG. 13 of the sixth embodiment based on the equations (8) and (9) is the same as the number of multipliers when the third embodiment is used in combination with the second embodiment. .

  Although the cubic interpolation and the bicubic interpolation in which the cubic interpolation is applied to the two-dimensional pixel have been described in detail above in terms of hardware implementation, all operations can be replaced with software processing. In this case, when the processor that controls the apparatus such as the information processing apparatus executes the software, the processor in the block diagram shown in each embodiment is executed as a function or procedure. Therefore, the present invention can also be used as a cubic interpolation calculation method by software calculation.

(Other examples)
The present invention supplies a program that realizes one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in the computer of the system or apparatus read and execute the program This process can be realized. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

  201, 202, 203, 204, 205, 503 ... linear interpolation calculator, 301, 302, 303, 304, 305 ... cubic interpolation calculator, 211, 212, 213 ... image quality adjustment parameter multiplier, 231 ... secondary parameter multiplication , 601, 602, 603, 604, 605, 606, 607... Linear interpolation operator, 621, 622, 623, 631.

Claims (12)

First linear interpolation means for generating an interpolation pixel value at an interpolation position from a pixel value of a pair of pixels among a plurality of pixels by linear interpolation;
A plurality of second linear interpolation means for generating a plurality of correction values by linearly interpolating each of the plurality of difference values between the pixel values of the plurality of pixels from a pair of difference values;
An image processing apparatus comprising: generating means for generating a pixel value at the interpolation position based on the interpolation pixel value and the plurality of correction values.   The image processing apparatus according to claim 1, further comprising a first multiplication unit that multiplies the plurality of correction values by a parameter related to a boundary condition of the interpolation position.   The plurality of pixels are arranged on the same straight line as the interpolation position, where the distance between the pixels is 1, and the distance from one of the plurality of pixels to the interpolation position is t, the parameter is The image processing apparatus according to claim 2, represented by t · (1−t). The plurality of pixels are 2N pixels, and the plurality of second linear interpolation means include:
Linear interpolation for a difference value between pixel values of pixels separated by one pixel;
2. The plurality of correction values are generated using linear interpolation with respect to a difference value between pixel values of pixels that are separated from each other by 2, 4,..., 2 · (N−1) pixels. Image processing apparatus.   The image processing apparatus according to claim 1, further comprising a second multiplying unit that multiplies at least one correction value among the plurality of correction values by a parameter for adjusting an image quality. An image processing device that generates a pixel value at a position to be obtained from a plurality of pixels arranged in a plurality of rows and columns,
In each of the plurality of columns, the pixel value at the same vertical position as the position to be obtained is generated from the pixel values of the plurality of pixels in the column by the image processing device according to claim 1,
The image processing apparatus according to claim 1, wherein the pixel value at the position to be obtained is generated by the image processing apparatus according to claim 1 from the generated pixel value for each of the plurality of columns. An image processing device that generates a pixel value at a position to be obtained from a plurality of pixels arranged in a plurality of rows and columns,
In each of the plurality of rows, the pixel value at the same horizontal position as the obtained position is generated from the pixel values of the plurality of pixels in the row by the image processing device according to claim 1,
The image processing apparatus according to claim 1, wherein the pixel value at the obtained position is generated by the image processing apparatus according to claim 1 from the generated pixel value for each of the plurality of rows. An image processing apparatus that generates a pixel value at an interpolation position specified in an interpolation section between an Nth pixel and an N + 1th pixel from pixel values of 2N consecutive pixels,
First linear interpolation calculation means for generating a linear interpolation value for two pixels having a symmetrical positional relationship with respect to the interpolation section;
Second linear interpolation calculation means for generating two types of linear interpolation values for two pixels having a symmetrical positional relationship with respect to the interpolation section;
Addition / subtraction operation means for generating a value obtained by adding / subtracting between outputs of the first and second linear interpolation operation means;
An image processing apparatus comprising: means for multiplying the value generated by the addition / subtraction calculation means by a parameter relating to a boundary condition of the interpolation section. The second linear interpolation calculation means includes:
For each set of 2 pixels in the N-1 symmetric positional relationship excluding the pixels at both ends of the 2N pixels, a linear interpolation value of the pixel value of the set and a set obtained by replacing the two pixels of the set The image processing apparatus according to claim 8, wherein two types of linear interpolation values with a linear interpolation value of a pixel value are generated. A control method for an image processing apparatus, comprising:
A first linear interpolation means for generating an interpolation pixel value at an interpolation position by linear interpolation from a pixel value of a pair of pixels among a plurality of pixels;
A plurality of second linear interpolation means generating a plurality of correction values by linearly interpolating each of a plurality of difference values between pixel values of the plurality of pixels from a pair of difference values;
And a generation unit configured to generate a pixel value at the interpolation position based on the interpolation pixel value and the plurality of correction values. A control method of an image processing apparatus for generating a pixel value at an interpolation position designated in an interpolation section between an Nth pixel and an N + 1th pixel from pixel values of 2N consecutive pixels,
A first linear interpolation calculation step for generating a linear interpolation signal for two pixels having a symmetrical positional relationship with respect to the interpolation section;
A second linear interpolation calculation step in which a second linear interpolation calculation means generates two types of linear interpolation values for two pixels having a symmetrical positional relationship with respect to the interpolation section;
An addition / subtraction calculation step in which an addition / subtraction calculation means generates a value obtained by adding / subtracting between outputs of the first and second linear interpolation calculation means;
And a multiplication unit multiplying the value generated in the addition / subtraction operation step by a parameter related to a boundary condition of the interpolation section.   A program for causing a computer to function as each unit of the image processing apparatus according to claim 1 by being read and executed by a computer.

Images