JPH11252353A - Image interpolation method and its device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image interpolation method and its device capable of obtaining an interpolated image of a high quality by fast arithmetic processing at the obtaining of an entire or a part of a virtual image, which is obtained by transforming the original image through coordinate transformation involving rotation, as an outputted image. SOLUTION: A relation between an outputted image and the transformed image after coordinate transformation involving rotation comes to the conclusion of a relation between an XY-coordinate system and a UV-coordinate system. Based on the corresponding relation of both these coordinate systems, the pixel value of an output pixel (black circle) is obtained by interpolation. Thus, the pixel value of an intermediate pixel (white circle) is obtained by interpolating in a first direction, based on the pixel value of a reference pixel (white triangle) to be arranged in the first direction (U-axis direction) of the transformed image first. By repeating operations similar to this, an intermediate pixel value array is obtained in the second direction (Y-axis direction) of the output pixel. Then, the output pixel is obtained by interpolating in the second direction of the output pixel, based on the pixel value of the intermediate pixel value array in the neighborhood of the output pixel.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

The present invention relates to an original image
The present invention relates to a technique for obtaining a pixel value of an output pixel by interpolation in order to obtain, as an output image, all or a part of a virtual image subjected to coordinate transformation involving rotational movement.

[0002]

2. Description of the Related Art In order to obtain a scaled image of an original image as an output image, it is necessary to determine the pixel value of an output pixel at a position that does not exist in the original image. Therefore, the pixel value of the output pixel is obtained by interpolating the original image.

Conventionally, methods for determining a scaled image by interpolation include a method using orthogonal transformation, a bilinear method,
A method using a filter such as cubic convolution is known. In particular, the method using cubic convolution and the method using orthogonal transform can obtain a high-quality interpolated image.

[0004]

A method using cubic convolution is to perform a product-sum operation on each reference pixel in a filter area by a pixel value and a value of a weight function at the pixel position. , The interpolation value of the output pixel is obtained. However, since the method using a filter involves many sum-of-products operations, the throughput of processing operations decreases. In particular, this tendency is remarkable in the cubic convolution since the filter area is large.

On the other hand, if a method using orthogonal transformation is used, interpolation can be performed by high-speed calculation. However, a method of using an orthogonal transformation when outputting an image obtained by rotating an original image is not known.

In view of the above problems, the present invention obtains, as an output image, all or part of a virtual image obtained by transforming an original image by coordinate transformation involving rotation, and obtains the output image by high-speed arithmetic processing. It is an object of the present invention to provide an image interpolation method and an image interpolation device capable of performing the above.

[0007]

According to a first aspect of the present invention, there is provided an image interpolating method comprising the steps of: executing a coordinate transformation including a rotational movement on an original image; In obtaining the portion as an output image, an image interpolation method of obtaining a pixel value of an output pixel in the output image by interpolation, wherein (a) using a relational expression of the coordinate conversion, Determining a corresponding position of the output pixel; and (b) a one-dimensional first direction in the first direction.
(C) obtaining a pixel row of a virtual one-dimensional pixel array passing through the corresponding position by performing the interpolation on the converted image, and (c) obtaining a pixel row in a second direction intersecting the first direction.
Performing a second dimensional interpolation on the pixel values of the pixel array to obtain a pixel value of the output pixel, wherein the first direction is a row direction of a matrix array of pixels in the converted image. And the column direction, and the second direction is one of a row direction and a column direction of a matrix arrangement of pixels in the output image.

According to a second aspect of the present invention, in the image interpolation method according to the first aspect, the first and second image interpolation methods are used.
Is characterized in that it is interpolation using one-dimensional orthogonal transformation.

According to a third aspect of the present invention, in the image interpolation method according to the first aspect, the first interpolation includes:
The method is performed by referring to a predetermined number of reference pixels specified so that each pixel of the pixel array is located substantially at the center among the plurality of pixels arranged in the first direction.

According to a fourth aspect of the present invention, in the image interpolation method according to the first aspect, a combination of the first direction and the second direction is such that an interval between pixels forming the pixel value array is different. It is determined to be the minimum.

According to a fifth aspect of the present invention, in the image interpolating method according to the fourth aspect, the combination of the first direction and the second direction includes a rotation angle, a row direction, and a column direction in the coordinate transformation. It is characterized by being determined according to the magnification ratio to.

According to a sixth aspect of the present invention, in the image interpolation method of the second aspect, when performing the interpolation using the one-dimensional orthogonal transformation in the steps (b) and (c), A predetermined calculation result stored in advance in the means is used.

In order to achieve the above object, an image interpolating apparatus according to claim 7 obtains, as an output image, all or a part of a virtual converted image obtained by performing coordinate conversion including rotational movement on an original image. In this regard, an image interpolating apparatus that obtains a pixel value of an output pixel in the output image by interpolation,
(A) using a relational expression of the coordinate transformation, coordinate transformation means for determining the corresponding position of the output pixel on the transformed image,
(b) first direction interpolation means for performing a one-dimensional first interpolation in a first direction on the converted image to obtain a pixel row of a virtual one-dimensional pixel array passing through the corresponding position;
(c) second direction interpolating means for obtaining a pixel value of the output pixel by performing one-dimensional second interpolation on a pixel value of the pixel array in a second direction intersecting the first direction;
Wherein the first direction is one of a row direction and a column direction of a matrix array of pixels in the converted image, and the second direction is a row direction and a column direction of a matrix array of pixels in the output image. It is one of the features.

An image interpolation device according to an eighth aspect of the present invention is the image interpolation device according to the seventh aspect, wherein the first and second
Is characterized in that it is interpolation using one-dimensional orthogonal transformation.

According to a ninth aspect of the present invention, in the image interpolation apparatus according to the seventh aspect, the first interpolation is performed by:
The method is performed by referring to a predetermined number of reference pixels specified so that each pixel of the pixel array is located substantially at the center among the plurality of pixels arranged in the first direction.

According to a tenth aspect of the present invention, in the image interpolating apparatus according to the seventh aspect, (d) the first direction is set such that an interval between pixels forming the pixel value array is minimized. And a combination selecting means for determining a combination in the second direction.

According to an eleventh aspect of the present invention, in the image interpolating apparatus according to the tenth aspect, the combination of the first direction and the second direction includes a rotation angle, a row direction, and a column direction in the coordinate transformation. It is characterized by being determined according to the magnification ratio to.

According to a twelfth aspect of the present invention, in the image interpolating apparatus according to the eighth aspect, the first and second directional interpolating units perform interpolation using one-dimensional orthogonal transformation. In this case, a predetermined calculation result stored in the storage means in advance is used.

[0019]

[Definition of terms] In this specification, "row direction" and "column direction" represent two orthogonal directions of arrangement of pixels in an image plane. Optional.

"Interpolation using orthogonal transformation"
This means interpolation by orthogonal transformation and inverse orthogonal transformation, and interpolation equivalent thereto.

[0021]

Embodiments of the present invention will be described below with reference to the drawings.

<A. Embodiment> FIG.
FIG. 6 is a diagram showing a relationship between the output image 200 and a converted output image 200.
In the present embodiment, as shown in FIG. 1, by performing coordinate transformation such as scaling (enlargement, reduction), rotation, and translation on the original image 100, a part or the whole of the original image is output. An image interpolating apparatus and an image interpolating method to obtain 200 will be described.

To obtain the output image 200, first,
It is assumed that a virtual converted image 100B (see FIG. 3) is obtained by performing coordinate conversion including scaling and rotation on the original image 100. Then, an output image 200 is obtained by extracting all or a part of the virtual converted image 100B. At this time, the correspondence between the original image (converted image) and the output image can be obtained by a coordinate conversion formula. That is, it is possible to determine which position the output pixel in the output image corresponds to in the original image.

In the following, a predetermined number of reference pixels existing near the corresponding position are extracted from the original image (converted image).
By performing one-dimensional interpolation in the first direction, which is either the row direction or the column direction of the converted image, based on these reference pixels, the output image is in either the row direction or the column direction of the output image. It is possible to obtain a virtual pixel array in the second direction, and further obtain a pixel value of the output pixel by performing one-dimensional interpolation in the second direction of the output image based on the pixel values of the pixel array. Explain what you can do.

<A1. Outline of Apparatus> FIG. 2 is a functional block diagram of the image interpolation apparatus 1 according to the embodiment of the present invention. The image interpolation device 1 includes an original image storage unit 11, a coordinate conversion unit 13,
A combination selection unit 15, a reference pixel extraction unit 17, an interpolation unit 20,
An output image storage unit 31 and an output unit 33 are provided.

The original image storage section 11 has an original image 100. The original image 100 is subjected to coordinate transformation including rotation, as will be described in detail later, and a part or all of the image is output as an output image 200.

The coordinate conversion unit 13 associates the UV coordinate system where the virtual image after the coordinate conversion has been performed on the original image with the XY coordinate system where the output image is located. For example, the coordinate value of each output pixel in the output image in the UV coordinate system can be obtained (see FIG. 3).

The combination selecting section 15 selects a combination of the first direction and the second direction based on the values of the magnification and the rotation angle in the coordinate conversion. Here, the first direction is one of the row direction and the column direction of the pixel array in the virtual converted image, and the second direction is the one of the row direction and the column direction of the pixel array in the output image. Is. The combination of these directions has important significance when performing interpolation, as will be described in detail later.

The reference pixel extraction unit 17 extracts a predetermined number of reference pixels existing near the corresponding position of the output pixel from the original image based on the relationship between the two coordinate systems obtained by the coordinate conversion unit 13.

The interpolation unit 20 has a first direction interpolation unit 21 and a second direction interpolation unit 23. The interpolation unit 20 can obtain the pixel value of the output pixel by interpolation. The first direction interpolation unit 21 performs interpolation in a first direction, which is either the row direction or the column direction of the virtual converted image 100B, based on the pixel value of the reference pixel, and obtains the row direction and the column of the output image 200. A virtual intermediate pixel array arranged in the second direction which is one of the directions is obtained. The obtained pixel values of the intermediate pixel array are stored in the intermediate pixel storage unit 25. The second direction interpolation unit 23 obtains the pixel value of the output pixel by performing one-dimensional interpolation in the second direction of the output image based on the pixel value of the virtual intermediate pixel array. The obtained pixel values are stored in the output image storage unit 31.

By repeating the same interpolation operation as described above for all output pixels of the output image, the entire output image can be obtained by interpolation. The obtained output image is output by the output unit 33.

In the apparatus having the above configuration, the pixel value of the output pixel in the output image can be obtained by interpolation.

<A2. Operation Outline><Relationship Between Both Coordinates> FIG. 3 shows a UV coordinate system in which a virtual transformed image 100B obtained by performing coordinate transformation with rotation on the original image 100, and an output image 200 FIG. 6 is a diagram showing a correspondence relationship with an XY coordinate system where are arranged. The position of the output pixel G _pq represented by a coordinate value (q, p) in the XY coordinate system is represented by a coordinate value ( _Upq , _Vpq ) in the UV coordinate system.

Here, the original image is multiplied by _{Mu in the} U-axis direction,
After M _v times in the axial direction, and rotates theta, further assumed coordinate transformation to perform a translation of -v ₀ to -u _0, V-axis direction to the U-axis direction. At this time, the coordinate values (q, p) and the coordinate values ( _Upq , _Vpq ) have a relationship as shown in the following equation.

[0035]

(Equation 1)

The scaling factors M _u , M _v , rotation angle θ, and parallel movement amounts u ₀ , v ₀ are specified by the operator, and are known values here. Therefore, according to Equation 1,
The coordinate values (q, p) in the XY coordinate system can be converted to coordinate values ( _Upq , _Vpq ) in the UV coordinate system.

As described above, the corresponding position of the output pixel in the converted image can be obtained. A predetermined reference pixel existing in the vicinity of the obtained corresponding position is extracted from the original image and interpolation is performed. The reference pixel can be extracted from a parallelogram region R, for example, as shown in FIG. This region R includes a pair of sides parallel to the first direction and a second side.
It has a parallelogram shape composed of a set of sides parallel to the direction. The extraction of the reference pixel is performed by the reference pixel extraction unit 17.
Done by The operation of extracting the reference pixel is as follows.
It can be performed on all reference pixels prior to the interpolation operation (described later), or can be performed sequentially as needed when performing the interpolation operation.

In extracting reference pixels and interpolating output pixels, it is preferable to select an optimal combination of the first direction and the second direction. This selection operation is performed by the combination selection unit 1
5, the detailed operation of which will be described later.

<Interpolation Operation> In the following, a case will be described in which the combination selection unit 15 selects the virtual converted image row direction as the first direction and the output image column direction as the second direction. . In this case,
(1) By performing one-dimensional interpolation in the first direction on the basis of the pixel value of the reference pixel, a virtual intermediate pixel array arranged in the second direction is obtained. The pixel value of the output pixel is obtained by performing one-dimensional interpolation in the second direction based on the pixel value of the intermediate pixel array. Since an interpolated value is obtained by performing one-dimensional interpolation twice, high-speed processing can be performed by reducing the amount of calculation of the product-sum operation as compared with the case of performing two-dimensional interpolation.
FIG. 4 is a diagram illustrating a relationship between a reference pixel and an output pixel in the above case, and is an enlarged view of a portion corresponding to a parallelogram region in FIG. 3. Each output pixel of the output image exists on an integer lattice point of the XY coordinate system. On the other hand, the pixels of the original image to be referred to are
It exists on the integer lattice point of the UV coordinate system indicated by the dotted line.

In the following, the p-th row and q-th of the XY coordinate system
It is assumed that the pixel value of the output pixel G _pq indicated by a black circle existing at the position of the column is obtained.

Therefore, first, the q-th column of the XY coordinate system and the UV
The pixel value H _iu of a virtual pixel C _iu (shown by a white circle in the figure) existing on the intersection (u, i) with the i-th row in the coordinate system is obtained. Here, the U coordinate value u in the UV coordinate system of the intersection is
Is represented by the following equation.

[0042]

(Equation 2)

Note that this equation is based on the coordinate value (q, y) of the pixel C _{iu in} the XY coordinate system and the coordinate value (u, i) in the UV coordinate system.
And is derived by the following equation similar to Equation 1.

[0044]

(Equation 3)

In Equation 2, T ₀ , T ₁ , and T ₂ can be calculated in advance and stored in a memory table. By using the memory in which the values of T ₀ , T ₁ , and T ₂ are stored, the speed of the calculation processing can be improved.

In order to obtain each pixel value of the arrangement of virtual intermediate pixels (indicated by white circles) arranged on the q-th column of the output image, the values are determined in the row direction (first direction) of the converted image (original image). Interpolation is performed with reference to the arranged pixels. The reference pixel is a pixel that is present on an integer grid point of the UV coordinate system and is indicated by a white triangle in the drawing. In the figure, a plurality of pixels existing on both left and right sides of the intermediate pixel C _{iu in} each row of the converted image are shown as reference pixels. These pixels are represented by coordinate values (u, u) of the intermediate pixel C _{iu in} the UV coordinate system.
The selection can be made based on the value of i).

For example, as (u, i), (20.
When (3, 10) is selected, coordinate values (j, 10) (where j is 17 or more and 24
A total of eight pixels on the tenth row represented by the following integers) can be extracted from the converted image (original image) and referenced. By performing one-dimensional interpolation in the row direction (first direction) of the converted image based on the pixel values of these reference pixels (white triangles), the i-th row of the converted image and the q-th column of the output image are obtained. Virtual intermediate pixel C _iu arranged on the intersection of
A pixel value H _iu (shown by a white circle) is obtained.

<First Direction Interpolation Process> As one-dimensional interpolation in each row direction, interpolation using one-dimensional orthogonal transformation can be adopted. Hereinafter, an interpolation operation using a discrete cosine transform as the orthogonal transform will be described.

First, reference is made to pixels represented by coordinate values (m ', i) located on the left and right of the intermediate pixel in the row direction of the converted image. Here, m ′ is an integer not less than (E−3) and not more than (E + 4), and E represents an integer part of the U coordinate value u of the intermediate pixel C _{iu in} the UV coordinate system. Also, the decimal part is e. That is, a total of eight pixel values S (E−3 +
m, i). Here, m = 0, 1, 2,..., 7, and S (E−3 + m, i) is a reference pixel existing at a position represented by a coordinate value (E−3 + m, i) in the UV coordinate system. Represents a pixel value.

Next, these pixel values S (E−3 + m,
i) is subjected to a one-dimensional discrete cosine transform (DCT). The resulting DCT coefficient F _K is given by:

[0051]

(Equation 4)

Then, by using the obtained DCT coefficient F _K , an inverse discrete cosine transform (hereinafter, referred to as a phase angle) corresponding to a number e ′ (= e + 3) representing a pixel position in the block corresponding to the intermediate pixel position is performed. "IDCT"). As a result, the coordinate values (u,
The pixel value H _iu of the pixel existing at the position i) is obtained by interpolation. Pixel value H of intermediate pixel obtained by IDCT
_iu is given by the following equation.

[0053]

(Equation 5)

The above operations are obtained for a plurality of i. At this stage, the pixel values H _{iu of} eight virtual intermediate pixels present at the positions indicated by white circles in the figure are obtained.

Further, the one-dimensional DCT and ID
The CT may not actually be performed as a separate operation.

By substituting Equation 4 for Equation 5, the following equation is obtained.

[0057]

(Equation 6)

Further, in Equation 6,

[0059]

(Equation 7)

Assuming that the value for each m (0 to 7) shown in Expression 7 is V _m , Expression 6 is transformed into the following expression.

[0061]

(Equation 8)

Therefore, the amount of calculation can be further reduced by calculating the value of V _m in advance. For example, if the decimal part e is limited to 256 types of 8-bit values, then e ′ is also limited to 256 types of values. Therefore, it is limited to 256 kinds for values of up to V ₀ ~V _7. By calculating these values using Equation 7 in advance and storing them in the memory 29 provided in the interpolation unit 20, it is possible to improve the throughput of the interpolation calculation.

FIG. 5 is a diagram showing the values stored in the memory 29. A value V _m calculated in advance according to Equation 7 corresponding to each of the 256 kinds of values of e ′ is:
It is stored in the memory 29. This storage operation is performed in advance before performing the above-described interpolation operation. In the figure, "*" represents the corresponding values of each V _m. Memory 29
By calculating V _m for e ′ by using the value stored in Eq.
Therefore, the throughput of the interpolation operation can be improved. Incidentally, the interpolation value of the pixel, based on each obtained V _m, can be determined by the number 8.

In the above, a virtual intermediate pixel array arranged on the q-th column of the output image is obtained. These eight intermediate pixels are located on the q-th column of the output image.

It is preferable that the intermediate pixel value once obtained is stored in a memory. This is because the same intermediate pixel value may be used when obtaining the pixel values of different output pixels. In this case, there is no need to obtain the same intermediate pixel value redundantly, and the intermediate pixel value stored in the memory can be used. Therefore, the calculation amount can be reduced and the throughput can be improved.

<Interpolation in Second Direction> Next, based on the pixel values of the eight intermediate pixels located on the q-th column of the output image, a one-dimensional interpolation is performed in the column direction, which is the second direction of the output image. The pixel value of the output pixel is obtained by performing the interpolation.

The specific interpolation method is the same as in the first direction. The only difference is the base pixel value at the time of interpolation.

First, an intermediate pixel located at a position represented by the coordinate value (u, i) is referred to. Here, i is an integer not less than (F−3) and not more than (F + 4), F represents an integer part of the V coordinate value V _pq of the output pixel in the UV coordinate system, and its decimal part is f. In other words, in the column direction of the output image, a total of eight intermediate pixel values S (u, i), four pixels each from the upper side and the lower side of the output pixel, are referred to. Where S
(U, i) represents the pixel value of the intermediate pixel existing at the position represented by the coordinate value (u, i) in the UV coordinate system, and u can be a different value for each i. The intermediate pixel is
It is preferable that the reference pixels are located substantially at the center of these eight reference pixels, but it is acceptable that the number of upper reference intermediate pixels and the number of lower reference intermediate pixels are slightly different.

Next, these reference intermediate pixel values S (u,
One-dimensional DCT is applied to i). The resulting DCT coefficient _FL is given by:

[0070]

(Equation 9)

Then, by using the obtained DCT coefficient F _L , the ID at the phase angle corresponding to the number f ′ (= f + 3) representing the pixel position in the block corresponding to the intermediate pixel position
Perform CT. As a result, the pixel value H _{pq of} the output pixel existing at the position of the coordinate value ( _Upq , _Vpq ) in the UV coordinate system
Can be obtained by interpolation. The pixel value H _pq of the output pixel obtained by IDCT is given by the following equation.

[0072]

(Equation 10)

Further, the one-dimensional DCT and ID
The CT may not actually be performed as a separate operation. This is the same as in the case of the first direction interpolation.

As described above, in the image interpolation method and the image interpolation apparatus according to the present embodiment, first, (1) interpolation is performed in the first direction based on the pixel value of the reference pixel, so that the image is interpolated in the second direction. A virtual intermediate pixel array to be arranged is obtained. Then, (2) the pixel value of the output pixel can be obtained by performing one-dimensional interpolation in the second direction based on the pixel value of the virtual intermediate pixel array. Then, an output image can be obtained by performing this interpolation processing on all output pixels in the output image.

Here, it is preferable that the interval between the pixels of the intermediate pixel array arranged in the second direction is small. This is because, when the interval between the intermediate pixels is small, the output pixel is more likely to exist near the intermediate pixel, and thus the output pixel value can be obtained by interpolation with higher accuracy. In order to minimize the interval between the pixels of the intermediate pixel array, the above-described interpolation may be performed by selecting an optimum one from a combination of the first direction and the second direction. This selection may be determined according to the magnification and the rotation angle.
Hereinafter, selection of a combination will be described.

<Combination of First Direction and Second Direction> The combination selector 15 selects an optimum combination of the first direction and the second direction based on the values of the magnification and the rotation angle in the coordinate conversion. Select

As described above, the first direction is either the row direction or the column direction of the virtual converted image, and the second direction is either the row direction or the column direction of the output image. That is, there are a total of four types of combinations of the first direction and the second direction from the following (a) to (d).
That is, (a) (the row direction of the virtual converted image, the column direction of the output image), (b) (the column direction of the virtual converted image, the column direction of the output image), (c) (the virtual conversion (D) (the column direction of the virtual converted image, and the row direction of the output image). An interval D between pixels of the intermediate pixel array arranged in the second direction is determined according to these combinations. In addition,
In the following, the intervals D corresponding to each of the combinations (a) to (d) are represented as D1 to D4.

The values D1 to D4 are as follows.

[0079]

[Equation 11]

[0080]

(Equation 12)

[0081]

(Equation 13)

[0082]

[Equation 14]

FIGS. 7 to 10 show the case where each of the above four combinations (a) to (d) is selected when the XY coordinate system and the UV coordinate system have the relationship shown in FIG. , The distances D1 to D4 between the intermediate pixels are respectively shown. Straight line in the second direction (shown by a solid line in the figure)
Above, a virtual intermediate pixel (shown by a white circle in the figure) is arranged at a position intersecting with a straight line in the first direction (shown by a broken line in the figure). The intervals between the intermediate pixels are D1 to D4. Note that these intermediate pixels are
This is obtained by performing interpolation in the first direction.

For example, in FIG. 8, a virtual intermediate pixel is located at a position where a straight line (solid line) in the “column” direction of the output image intersects with a straight line (dashed line) in the “column” direction of the virtual converted image. (Open circles) are arranged. The interval between the intermediate pixels in this case is D2. Similarly, FIG. 7 shows the interval D1, FIG. 9 shows the interval D3, and FIG. 10 shows the interval D4.

The intervals D1 to D4 between these intermediate pixels
Among them, the interval Dmin which becomes the minimum value is determined. The minimum interval Dmin can be determined based on D1 to D4 represented by Expressions 11 to 14. Therefore, the above-described optimal combination that minimizes the interval D can be determined according to the scaling factors _Mu and _Mv and the rotation angle θ during coordinate conversion. Note that, for the rotation angle θ, 0 (rad) ≦ θ ≦ π
Convert to take a value within the range of / 2 (rad). That is,
Regarding a rotation angle exceeding this range, θ is regarded as an angle between a straight line in the X-axis direction and a straight line in the U-axis direction.
Convert to a value within this range.

FIG. 6 is a diagram schematically showing the operation of the combination selecting section 15. The above-described combination selection operation will be described with reference to FIG.

First, M _u > M _v , and 0 ≦ θ ≦ π / 4 (rad)
In this case, Dmin = D1. Therefore, in this case, among the above combinations, (a) (the row direction of the virtual converted image, the column direction of the output image) is the optimum combination. For example, this corresponds to the case where the XY coordinate system and the UV coordinate system have a relationship as shown in FIG. The intermediate pixels indicated by white circles in FIG. 4 are obtained in the same manner as the intermediate pixels shown in FIG.
D1.

Further, M _u > M _v , and π / 4 (rad) ≦ θ ≦
In the case of π / 2 (rad), Dmin = D3. In this case, the combination (c) is the optimum combination. FIG.
Is a diagram showing a relationship between the XY coordinate system and the UV coordinate system in this case.

Further, M _u <M _v and 0 ≦ θ ≦ π / 4 (ra
In the case of d), Dmin = D4. The optimal combination is the combination of the above (d). FIG. 12 is a diagram showing the relationship between the XY coordinate system and the UV coordinate system in this case.

Then, M _u <M _v , and π / 4 (rad) ≦ θ
When ≤π / 2 (rad), Dmin = D2. The optimal combination is the above (b). FIG. 13 is a diagram showing the relationship between the XY coordinate system and the UV coordinate system in this case.

As described above, the combination that minimizes the interval between virtual intermediate pixels can be determined in advance. Accordingly, as described above, one-dimensional interpolation is performed in the second direction of the output image on the basis of the pixel values of the virtual intermediate pixel array obtained for this optimum combination, thereby obtaining the pixel values of the output pixels. Can be requested. This makes it possible to obtain a more accurate interpolation value.

<B. Other Modifications> Although the DCT has been exemplified as the orthogonal transform in the above embodiment, the present invention is not limited to this, and can be applied to other orthogonal transforms such as Hadamard transform and Fourier transform. In this case, the corresponding inverse transform can be used as the inverse transform.

The one-dimensional interpolation in the predetermined direction can be performed by a method other than the method using the orthogonal transformation. That is, a method using a filter generally represented by the following equation can be adopted instead of Equations 4 and 5.

[0094]

(Equation 15)

Here, as described above, S (E−3 + m, i) is the coordinate value (E−3 + m, i) in the UV coordinate system.
It represents the pixel value of the reference pixel existing at the position represented by i). E represents the integer part of the U coordinate value u of the intermediate pixel C _{iu in} the UV coordinate system, and the decimal part thereof is e.
_Wm is a load value corresponding to the reference pixel. FIG. 14 is a diagram illustrating characteristics of the filter. If this is used, the sharpening of the image can be performed simultaneously with the interpolation. As shown in FIG. 14, a curve representing the characteristic of the filter, with respect to U coordinate value of the horizontal axis, the value W _m of the vertical axis is identified. The value W _m, can be identified using the value e instead of U coordinate values on the horizontal axis. That is, the value W _m
Can be determined depending on the value e. For example, if the value e is 8
When expressed in bits, the value W _m is limited to 256. By storing these 256 types of values in the table memory in advance, the speed of calculation processing can be improved.

According to the above equation (15), the pixel value of each intermediate pixel can be obtained by performing one-dimensional interpolation in each row direction. In order to obtain a plurality of intermediate pixels, similar processing may be performed on a plurality of rows in the converted image.

In the above embodiment, the same number of pixels are extracted as reference pixels for each row of the converted image. As a result, pixels on the lattice points existing inside the parallelogram region R are extracted, but the present invention is not limited to this. For example, after extracting a different number of reference pixels for each row of the converted image, it is possible to extract, for each row, pixels in a region having symmetry with respect to an intermediate pixel position in the row direction (first direction) of the converted image. it can. The number of pixels to be extracted for each row can be appropriately determined in consideration of the amount of calculation of the image quality of the output image.

Further, in the above embodiment, the intermediate pixels are obtained by extracting the same number of pixels in the horizontal direction of each row of the converted image with reference to the position of the output pixels in the column direction. It is desirable to extract the same number at a time, but it is not always necessary to extract the same number, and it is possible to allow some deviation. In addition, an output pixel value is also obtained in the vertical direction of a predetermined column of output pixels by referring to the same number of intermediate pixels based on the output pixel position. Similarly, the number is not necessarily the same.

In the present embodiment, the above-described pixel value has a gradation value. However, the pixel value may include not only monotone image data but also color components such as RGB. In that case, various image data such as an original image and an output image have color components, and DCT or IDC
T may be performed in parallel for each color component.

[0100]

As described above, claims 1 to 6 are as described above.
According to the image interpolation method described in the above, and the image interpolation apparatus described in the seventh to twelfth aspects, the original image is subjected to coordinate transformation including rotational movement, and all or part of the transformed image is obtained as an output image In doing so, a pixel value of a virtual one-dimensional pixel array passing through the corresponding position is obtained by performing a one-dimensional first interpolation in a first direction on the converted image. By performing one-dimensional second interpolation on the pixel values of the pixel array in the intersecting second direction, the pixel value of the output pixel after coordinate transformation with rotation is obtained. Therefore, an interpolated value can be obtained by performing one-dimensional interpolation twice instead of performing two-dimensional interpolation, so that the amount of calculation of the product-sum operation can be reduced and high-speed processing can be performed.

In particular, according to the image interpolation method according to the second aspect and the image interpolation apparatus according to the eighth aspect, since interpolation using one-dimensional orthogonal transformation is performed as one-dimensional interpolation, higher accuracy is achieved. Can be obtained at high speed.

According to the image interpolating method according to the fourth and fifth aspects and the image interpolating apparatus according to the tenth and eleventh aspects, the interval between pixels forming the pixel value array is minimized. Thus, a combination of the first direction and the second direction is determined. Therefore, the interpolation value is obtained by referring to the pixel value array arranged more densely, so that a higher precision interpolation value can be obtained, and a high quality output image can be obtained.

Further, according to the image interpolation method of the sixth aspect and the image interpolation apparatus of the twelfth aspect, when performing interpolation using one-dimensional orthogonal transform, a predetermined value stored in the storage means in advance is used. Since the calculation result of is used, the calculation amount can be further reduced and the processing can be performed at high speed.

[Brief description of the drawings]

FIG. 1 is a diagram showing a relationship between an original image 100 and an output image 200 after conversion.

FIG. 2 is a functional block diagram of the image interpolation device 1 according to the embodiment of the present invention.

FIG. 3 is a diagram illustrating a correspondence between a UV coordinate system and an XY coordinate system.

FIG. 4 is a diagram illustrating a relationship between an output pixel and a reference pixel.

FIG. 5 is a diagram showing values stored in a memory 29;

FIG. 6 is a schematic diagram showing an operation in a combination selection unit 15;

FIG. 7 is a diagram showing an interval D1 between intermediate pixels.

FIG. 8 is a diagram showing an interval D2 between intermediate pixels.

FIG. 9 is a diagram illustrating an interval D3 between intermediate pixels.

FIG. 10 is a diagram showing an interval D4 between intermediate pixels.

FIG. 11 is a diagram illustrating a relationship between an XY coordinate system and a UV coordinate system when Dmin = D3.

FIG. 12 is a diagram illustrating a relationship between an XY coordinate system and a UV coordinate system when Dmin = D4.

FIG. 13 is a diagram illustrating a relationship between an XY coordinate system and a UV coordinate system when Dmin = D2.

FIG. 14 is a diagram illustrating characteristics of a filter.

[Reference Numerals] 1 image interpolation device 20 interpolation section 100 original image 100B converted image 200 output image M _u, M _v magnification θ rotation angle C _iu intermediate pixel G _pq output pixel D, D1, D2, D3, D4 intermediate pixel Spacing R Parallelogram area

Claims (12)

[Claims] When obtaining, as an output image, all or a part of a virtual converted image obtained by performing coordinate conversion including rotational movement on an original image, pixel values of output pixels in the output image are obtained by interpolation. An image interpolation method, comprising: (a) obtaining a corresponding position of the output pixel on the converted image using a relational expression of the coordinate conversion; and (b) one-dimensionally in a first direction. Applying a first interpolation to the converted image to obtain a pixel row of a virtual one-dimensional pixel array passing through the corresponding position; and (c) one-dimensionally intersecting the first direction in a second direction. Obtaining a pixel value of the output pixel by performing a typical second interpolation on the pixel value of the pixel array, wherein the first direction is One of the column orientations , The second direction, the image interpolation method which is characterized in that either of the row direction and the column direction of the matrix arrangement of pixels in the output image. 2. The image interpolation method according to claim 1, wherein the first and second interpolations are interpolations using one-dimensional orthogonal transform. 3. The image interpolation method according to claim 1, wherein the first interpolation is performed such that each pixel of the pixel array among a plurality of pixels arranged in the first direction is located substantially at the center. An image interpolation method, which is performed with reference to a specified number of specified reference pixels. 4. The image interpolation method according to claim 1, wherein the combination of the first direction and the second direction is determined such that an interval between pixels forming the pixel value array is minimized. An image interpolation method characterized by the following. 5. The image interpolation method according to claim 4, wherein the combination of the first direction and the second direction is determined according to a rotation angle in the coordinate transformation and a magnification in a row direction and a column direction. An image interpolation method characterized in that: 6. The image interpolation method according to claim 2, wherein when performing the interpolation using the one-dimensional orthogonal transform in the steps (b) and (c), a predetermined calculation stored in a storage unit in advance. An image interpolation method characterized by using a result. 7. When obtaining, as an output image, all or a part of a virtual converted image obtained by performing coordinate conversion including rotational movement on an original image, obtain pixel values of output pixels in the output image by interpolation. An image interpolating apparatus, comprising: (a) coordinate conversion means for obtaining a corresponding position of the output pixel on the converted image using a relational expression of the coordinate conversion; and (b) one-dimensional in a first direction. First direction interpolation means for obtaining a pixel row of a virtual one-dimensional pixel array passing through the corresponding position by applying a first interpolation to the converted image, and (c) intersecting with the first direction A second direction interpolation means for performing a one-dimensional second interpolation on the pixel value of the pixel array in a second direction to obtain a pixel value of the output pixel;
Wherein the first direction is one of a row direction and a column direction of a matrix array of pixels in the converted image, and the second direction is a row direction and a column direction of a matrix array of pixels in the output image. An image interpolation device characterized by any one of the above. 8. The image interpolation apparatus according to claim 7, wherein the first and second interpolations are interpolations using one-dimensional orthogonal transformation. 9. The image interpolation device according to claim 7, wherein the first interpolation is performed such that each pixel of the pixel array is located substantially at the center among a plurality of pixels arrayed in the first direction. An image interpolating apparatus, which is performed by referring to a specified number of specified reference pixels. 10. The image interpolation apparatus according to claim 7, wherein: (d) determining a combination of the first direction and the second direction such that an interval between pixels forming the pixel value array is minimized. An image interpolation device further comprising a combination selection unit. 11. The image interpolation device according to claim 10, wherein the combination of the first direction and the second direction is determined according to a rotation angle in the coordinate transformation and a magnification in a row direction and a column direction. An image interpolating device, characterized in that: 12. The image interpolation apparatus according to claim 8, wherein the first direction interpolation unit and the second direction interpolation unit perform interpolation using one-dimensional orthogonal transformation, and are previously stored in a storage unit. An image interpolating device using a predetermined calculation result.