JP6326914B2 - Interpolation apparatus and interpolation method - Google Patents

Description

  The present invention provides an interpolating apparatus and an interpolation method for obtaining data between grid points by interpolating between grid points with respect to data to be assigned to a large number of grid points and to be processed in the order of the grid points. About.

  In recent years, a head-up display (hereinafter abbreviated as “HUD”) system has attracted attention as an in-vehicle information display device. An in-vehicle HUD system projects, for example, an image representing a vehicle speed, a remaining fuel amount, and the like on a windshield. However, if these images are projected as they are onto the non-planar windshield, they appear as distorted images to the driver. Therefore, many HUD systems have a function (warping function) for correcting this distortion.

  As a warping function, there is a method in which a correction value corresponding to the curved surface of the windshield is given to each pixel constituting an image. According to this method, when an image is projected onto the windshield, the distortion is eliminated from the image projected on the windshield by correcting the projection position of each pixel using the correction value given to each pixel. Can do.

  Furthermore, in the case of this method, for the purpose of reducing the data amount of the correction value given to each pixel, the correction values of some pixels are not given in advance to all the pixels constituting the video. It is determined by interpolation. By performing such interpolation, the amount of correction value data that must be stored in advance can be reduced.

  An example of such an interpolation method is a bilinear interpolation method (see Patent Document 1). In the bilinear interpolation method, the correction values of the four surrounding points are proportional to the distance between the pixel whose correction value is to be interpolated and the surrounding four points surrounding the pixel and each of the pixels having the correction value in advance. Is weighted. Then, an average of the weighted values is obtained, and the average value is set as a correction value to be given to the interpolation target pixel.

JP 2008-102519 A (published May 1, 2008)

  However, since the above-described bilinear interpolation method is linear interpolation using a linear expression, smooth interpolation cannot be performed when applied to a curved surface such as a windshield.

  Therefore, it is conceivable to perform interpolation using a quadratic expression instead of the primary expression. In this case, the horizontal scanning line direction and the vertical direction are each interpolated with a quadratic expression. Therefore, it is difficult to perform real-time processing such as drawing on a horizontal scanning line by sequentially using the interpolated correction values while interpolating the correction values.

  On the other hand, the present invention provides an interpolation apparatus and an interpolation method capable of realizing real-time processing for drawing on a horizontal scanning line by sequentially using the interpolated correction values while interpolating the correction values. Objective.

  In order to solve the above-described problem, the interpolation apparatus according to the present invention is assigned to each of a large number of grid points arranged in the first direction, and the data to be processed in the order of the grid points is set between the grid points. An interpolation apparatus for interpolating data, wherein interpolation means for performing interpolation between the first grid points using an interpolation formula set for the first grid points, and interpolation between the first grid points are performed. Interpolating equation setting means for setting an interpolating equation for the second lattice points that are arranged later than the interval between the first lattice points.

  According to the above configuration, when the data given to each grid point is processed in this order between the first grid point and the second grid point, the interpolation means performs interpolation with respect to the first grid point. During this period, the interpolation formula setting means determines an interpolation formula for interpolating between the second grid points and sets it between the second grid points.

  For this reason, when the interpolation means finishes processing the data of each grid point between the first grid points, and subsequently starts processing the data of each grid point between the second grid points, Is set between the second grid points.

  Therefore, immediately after the processing for each data between the first grid points is completed, it is possible to perform interpolation between the second grid points using the set interpolation formula.

  Therefore, it is possible to realize real-time processing that performs processing on each data by sequentially using the interpolated data while interpolating the data given between the respective grid points.

  Here, “multiple grid points” means that, in principle, it is sufficient if the number of grid points is 3 or more. However, when processing for pixels for displaying an image is assumed, more grid points are used. Will mean.

  The present invention has an effect that it is possible to realize a real-time process of drawing on a horizontal scanning line by sequentially using the interpolated correction values while interpolating the correction values.

It is a block diagram which shows the function structure of the interpolation apparatus which concerns on one Embodiment of this invention. It is a block diagram which shows schematic structure of the image processing apparatus to which the said interpolation apparatus is applied. It is a block diagram which shows the hardware constitutions of the said interpolation apparatus. It is a figure for demonstrating the said interpolation apparatus, (a) is a figure which shows the two-dimensional lattice on an original image, (b) is a figure which shows an example of the correction table in which the reference | standard correction value was described. It is a figure for demonstrating the interpolation method based on one Embodiment of this invention concretely, (a) And (d) is a figure explaining an original image, (b) And (c) explains the image for projection. It is a figure to do. It is a figure which shows the process sequence of the said interpolation method.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. The following embodiments are examples embodying the present invention, and do not limit the technical scope of the present invention.

(Image processing apparatus 1)
FIG. 2 is a block diagram showing a schematic configuration of the image processing device 1 to which the interpolation device 20 according to the embodiment of the present invention is applied. The image processing apparatus 1 is an image processing apparatus suitable for a display system that provides a user with an image projected on a projection plane from a display apparatus, such as an in-vehicle display system using a HUD system.

  As shown in FIG. 2, the image processing apparatus 1 includes a reference correction value storage unit 2, an address generation unit 3, a video capture buffer 4, a filter function unit 5, and a display control unit 6.

  The reference correction value storage unit 2 stores a correction value given in advance to each pixel constituting the original image for removing distortion generated when the original image assumed to be displayed on a plane is projected onto the curved surface as it is. Each correction value indicates a positional deviation between the position of each pixel of the original image and the position of each pixel corresponding to each pixel of the original image in the projection image projected onto the projection plane. .

  For example, each correction value indicates which position in the projection image each pixel of the original image should be displayed as (for example, the coordinates of the pixel in the projection image). In addition, each correction value may be a coefficient indicating a deviation of the coordinates of each pixel in the coordinate system in the projection image from the coordinates of each pixel in the coordinate system of the original image. In this case, the difference between the x coordinate of the original image in the XY coordinate system and the u coordinate of the projection image in the UV coordinate system, and the y coordinate of the original image in the XY coordinate system and the U− of the projection image. A difference from the v coordinate in the V coordinate system may be used as a correction coefficient.

  However, the reference correction value storage unit 2 does not store the correction values as described above for all the pixels constituting the original image. The reference correction value storage unit 2 assumes a two-dimensional grid on the original image and stores the above correction values in advance only for pixels located at the grid points of the two-dimensional grid (hereinafter referred to as “grid point pixels”). To do. On the other hand, correction values for pixels out of the lattice points of such a two-dimensional lattice, that is, pixels located between lattice points (hereinafter referred to as “inter-grid pixels”) are given to the lattice point pixels. It is calculated by interpolation using the correction value. This interpolation will be described later. Hereinafter, a correction value stored in advance in the reference correction value storage unit 2 is referred to as a “reference correction value”, and a correction value obtained by interpolation as described above is referred to as an “interpolation correction value”.

  The address generation unit 3 generates address data indicating the display position on the projection image described above for RGB data input from the outside of the image processing apparatus 1 and temporarily stored in the video capture buffer 4. The image processing apparatus 1 outputs RGB data to a display device (not shown), and displays an image on the display device based on the RGB data.

  Further, the address generation unit 3 has an interpolation device 20. As will be described later, the interpolation device 20 obtains an interpolation correction value for the inter-grid pixel by interpolation based on the reference correction value for the grid point pixel acquired from the reference correction value storage unit 2. When the interpolation correction value is given to the inter-lattice pixel, the address generation unit 3 uses the reference correction value acquired from the reference correction value storage unit 2 and the interpolation correction value obtained by the interpolation device 20 as described above. Is generated.

  The video capture buffer 4 temporarily stores RGB data of the original image input from the outside of the image processing apparatus 1. The RGB data is output from the video capture buffer 4 to the filter function unit 5 in accordance with the addresses described above.

  For example, the filter function unit 5 performs filter processing such as edge enhancement on the RGB data input from the video capture buffer 4. The filter function unit 5 outputs the RGB data subjected to such filter processing to the display control unit 6.

  The display control unit 6 controls the display device described above, and displays the projection image on the display device using the RGB data input from the filter function unit 5.

(Interpolation device 20)
As described above, the interpolation device 20 implements the process of obtaining the interpolation correction value based on the reference correction value stored in advance in the reference correction value storage unit 2. As shown in FIG. 1, the interpolation device 20 sets a first interpolation unit (interpolation for setting the interpolation equation for performing the first interpolation while setting the interpolation equation for performing the second interpolation). Expression setting means) 21, a second interpolation section (interpolation means) 22 for performing second interpolation, a horizontal direction interpolation coefficient storage section 23 for temporarily storing horizontal direction interpolation coefficients, and a first correction value temporarily. And a first correction value storage unit 24 for storing. The hardware configuration of the interpolation device 20 is shown in FIG. 3 and will be described in detail later.

(First interpolation unit 21)
The first interpolation unit 21 first calculates an interpolation correction value (hereinafter referred to as “first correction value”) to be given to each of two inter-lattice pixels on the horizontal scanning line by interpolation in the vertical direction. Ask. One interpolation section on the horizontal scanning line is a section between two grid point pixels to which the first correction value is given. Next, the first interpolation unit 21 interpolates an interpolation correction value (hereinafter referred to as “second correction value”) to be given to the pixels in the interpolation section in the horizontal scanning line direction, and calculates the scanning line direction. Determine the interpolation formula.

  As shown in FIG. 1, the first interpolation unit 21 that realizes such a function includes a reference pixel setting unit 30, a vertical direction interpolation coefficient calculation unit (sub-interpolation equation setting unit) 31, and a first correction value calculation unit. (Sub-interpolation means) 32, vertical coordinate calculation section (sub-interpolation means) 33, and horizontal direction interpolation coefficient calculation section 34.

  Hereinafter, as shown in FIG. 4A, a two-dimensional lattice is assumed on an original image composed of a plurality of pixels, and a reference correction value is given to a lattice point pixel located at a lattice point of the two-dimensional lattice. An embodiment is described as an example.

In this case, for example, the pixel in the projection image corresponding to the pixel located at the point p (0, 1) in the XY coordinate system of the original image is the q point ( u ₀₁ , v ₀₁ ). At this time, [u ₀₁ , v ₀₁ ] is given as the reference correction value d to the pixel located at the p point of the original image. That is, in this example, for each pixel located at the lattice point of the original image, the coordinates of the corresponding pixel located in the projection image are given as the reference correction value.

  As shown in FIG. 4B, such reference correction values are stored in advance in the reference correction value storage unit 2 in a table format. In this correction table, the reference correction value d given to the lattice point pixel located at the above-described p point is described as a matrix element corresponding to (x, y) = (0, 1) of the p point. Similarly, in the correction table, the reference correction value given to each grid point pixel of the original image is described.

  The reference pixel setting unit 30 sets a reference pixel region to be referred to when interpolating two inter-grid pixels on the horizontal scanning line. The reference pixel area includes a plurality of pixels (hereinafter referred to as “reference pixels”) positioned in the vertical direction (second direction) with respect to the interpolation target pixel among the lattice point pixels of the original image. The reference pixel setting unit 30 acquires the standard correction value of each reference pixel included in the reference pixel area from the standard correction value storage unit 2.

  The vertical direction interpolation coefficient calculation unit 31 uses the standard correction value of each reference pixel included in the reference pixel area set by the reference pixel setting unit 30 to obtain the first correction value to be given to the two inter-lattice pixels. A vertical interpolation formula for calculation is determined.

  Here, the above-mentioned “vertical direction” is the Y direction in the XY coordinate system of the original image. As described above, the reference correction value and the first correction value indicate the coordinates of each pixel in the projection image corresponding to each pixel of the original image. When the reference correction value and the first correction value indicate the coordinates of each pixel in such a projection image, the vertical direction interpolation formula is given to each lattice point pixel arranged in the Y direction in the XY coordinate system of the original image. It is represented by a curve that passes through the determined reference correction value.

In general, when n + 1 points (α ₀ , β ₀ ), (α ₁ , β ₁ ), (α ₂ , β ₂ ),..., (Α _n , β _n ) are given, The process of drawing a smooth curve passing through and obtaining the value β of the point on the curve at a value α between α _{0 and} α _n is called interpolation. Usually, a curve passing through n + 1 points is expressed by, for example, the following nth order expression (n is a natural number of 2 or more). In this embodiment, Newton's interpolation formula is used.

β = A ₀ + A ₁ (α−α ₀ ) + A ₂ (α−α ₀ ) (α−α ₁ ) +... + A _n (α−α ₀ ) (α−α ₁ ) (α− α _n-1 )
The above "to determine the vertical interpolation equation", in order to determine the n-th order equation means calculating the coefficients of the (interpolation coefficient) A ₀ to A _n.

  When such an n-order equation is used, the amount of calculation tends to increase as the number of n increases. For this reason, the vertical direction interpolation coefficient calculation unit 31 performs low-dimensional interpolation for the number of grid points in the X direction of the reference pixel region. Specifically, the vertical direction interpolation coefficient calculation unit 31 uses a quadratic expression using three grid point pixels as the vertical direction interpolation expression. Of course, the vertical interpolation formula is not limited to a quadratic formula, and may be a higher-order formula such as a cubic formula or a quaternary formula if an increase in the amount of calculation does not matter. The higher the expression, the wider the reference pixel area needs to be.

  The vertical direction interpolation coefficient calculation unit 31 calculates a coefficient of such a vertical direction interpolation formula and determines a plurality of vertical direction interpolation formulas. In the present embodiment, a quadratic equation is used as the vertical direction interpolation equation.

  The first correction value calculation unit 32 uses the vertical direction interpolation formula determined by the vertical direction interpolation coefficient calculation unit 31 to calculate a first correction value to be given to two interstitial pixels on the horizontal scanning line, which is an interpolation target. calculate. When calculating the first correction value, the first correction value calculation unit 32 calculates the vertical coordinates of the v coordinates in the U-V coordinate system of the projection image for the number of grid points in the X direction of the reference pixel area. Obtained from the unit 33. The u coordinate is acquired by substituting the acquired v coordinate into the corresponding vertical interpolation formula. These u and v coordinates are the first correction values.

  The vertical coordinate calculation unit 33 calculates the v coordinate on the horizontal scanning line to be interpolated by the following method from the reference correction values of the two grid points positioned above and below the horizontal scanning line to be interpolated.

  The difference between the v coordinate of the two lattice points in the UV coordinate system and the difference of the y coordinate in the XY coordinate system of the same two lattice points (distance between lattice points in the Y direction) are obtained.

  The v-coordinate is calculated from the relationship between the above two differences, the y-coordinate of the horizontal scanning line, and the y-coordinates of lattice points above and below the horizontal scanning line.

  The horizontal direction interpolation coefficient calculation unit 34 uses the first correction value calculated by the first correction value calculation unit 32 to calculate the second interpolation value to be given to the pixels in the pixels between the two grid points. First, a scanning line direction interpolation formula to be interpolated in the horizontal scanning line direction (first direction) is determined.

  The above-mentioned “horizontal scanning line direction” is the X direction in the XY coordinate system of the original image. Similar to the above-described vertical direction interpolation equation, the scanning line direction interpolation equation uses a quadratic equation (curve) using pixels between three grid points. The horizontal direction interpolation coefficient calculation unit 34 calculates the coefficient of the scanning line direction interpolation formula to determine the scanning line direction interpolation formula. In this embodiment, the scanning line direction interpolation formula is a quadratic formula.

(Second interpolation unit 22)
The second interpolation unit 22 uses the scanning line direction interpolation formula determined by the first interpolation unit 21 to apply the first correction value to the pixels located in the interpolation section that is between the inter-lattice pixels. 2 Determine the correction value. As illustrated in FIG. 1, the second interpolation unit 22 includes a second correction value calculation unit 40 and a horizontal coordinate calculation unit 41.

  The second correction value calculation unit 40 calculates a second correction value to be given to the pixels in the interpolation section using the scanning line direction interpolation formula determined by the horizontal direction interpolation coefficient calculation unit 34. When calculating the second correction value, the second correction value calculation unit 40 obtains the u coordinate of each pixel in the UV coordinate system of the projection image from the horizontal coordinate calculation unit 41.

  The horizontal coordinate calculation unit 41 uses the first correction value calculated by the first correction value calculation unit 32 to set the u coordinate of the pixel in the interpolation section in the horizontal scanning line direction in the UV coordinate system of the projection image. It calculates sequentially along.

(Horizontal interpolation coefficient storage unit 23)
The horizontal direction interpolation coefficient storage unit 23 temporarily stores the scanning line direction interpolation type coefficient calculated by the horizontal direction interpolation coefficient calculation unit 34 until the second correction value calculation unit 40 starts processing.

(First correction value storage unit 24)
The first correction value storage unit 24 temporarily stores the first correction value calculated by the first correction value calculation unit 32 until the horizontal coordinate calculation unit 41 starts processing.

(Operation of the interpolation device 20)
Hereinafter, the operation of the interpolation device 20, that is, the interpolation method according to the embodiment of the present invention will be described with reference to FIGS. FIG. 5 is a diagram for specifically explaining the interpolation method according to the embodiment of the present invention, and FIG. 6 is a diagram showing a processing procedure of the interpolation method.

6, the reference pixel setting section 30 sets a reference pixel region _{R 1} (S1). As shown in (a) of FIG. 5, the first pixel P ₁ , the second pixel P _2, and the third pixel P _{3, which} are pixels between lattice points, are interpolation targets. The reference pixel region R ₁ includes reference pixels that the first pixel P ₁ , the second pixel P _2, and the third pixel P ₃ should refer to in each interpolation. Specifically, the reference pixel region R ₁ includes a grid point pixel p ₁ ~p ₉ located at lattice points of the original image. The first pixel P ₁ is a pixel located in a section l ₁ between p ₁ and p ₄ , the second pixel P ₂ is a pixel located in a section l ₂ between p ₂ and p ₅ , a third pixel _{P 3} is a pixel located in a section _{l 3} between the _{p 3} and _{p 6.}

Reference pixels to be referred to when obtaining the first correction value of the first pixel P ₁ are p ₁ , p ₄ , and p ₇ . Reference pixels to be referred to when obtaining the first correction value of the second pixel P ₂ are p ₂ , p ₅ , and p ₈ . Reference pixels to be referred to when determining the first correction value of the third pixel _{P 3} is _{p 3, p 6, p 9} . The reason why three points are required for each reference pixel is that the vertical interpolation formula is a quadratic formula as described above.

The reference pixel setting unit 30 acquires the reference correction value given to each of p _{1 to} p ₉ from the correction table stored in the reference correction value storage unit 2.

Next, when the reference correction values given to each of p ₁ to p ₉ are input from the reference pixel setting unit 30 to the vertical direction interpolation coefficient calculation unit 31, the first pixel P ₁ and the second pixel P ₂ are input. and the coefficients of the vertical interpolation formula for calculating the first correction value each of the third pixel P ₃ is calculated to determine each of the vertical interpolation formula (S2).

For example, the first direction correction value for the first pixel P ₁ is interpolated in the vertical direction m ₀₁ , and the vertical direction interpolation formula m ₁ for calculation is determined as follows.

First, reference correction values p ₁ , p ₄ , and p ₇ that are reference pixels to be referred to when obtaining the first correction value of the first pixel P ₁ , that is, the coordinates in the UV coordinate system of the projection image are set. _{q 1, q 4, q 7} to the vertical interpolation formula _{m 1} is a quadratic equation below (1) through which position (see FIG. 5 (b)).

u = A ₀₀ (v−v ₀₀ ) (v−v ₀₁ ) + B ₀₀ (v−v ₀₀ ) + C ₀₀ (1)
The coordinates of q ₁ , q ₄ , and q ₇ in the UV coordinate system of the projection image are (u ₀₀ , v ₀₀ ), (u ₀₁ , v ₀₁ ), and (u ₀₂ , v ₀₂ ).

With v = v ₀₀ , u = u ₀₀ = C ₀₀ .

The v _{= v 01,} becomes _{u = u 01 = B 00 (} v 01 -v 00) + C 00. As a result, B ₀₀ = (u ₀₁ −u ₀₀ ) / (v ₀₁ −v ₀₀ ).

The v _{= v 02,} becomes _{u = u 02 = A 00 (} v 02 -v 00) (v 02 -v 01) + B 00 (v 02 -v 00) + C 00. The _{u 00 = C 00, B 00} = (u 01 -u 00) / (v 01 -v 00), A 00 = the _{(B 00 '-B 00) /} (v 02 -v 01). However, B ₀₀ ′ = (u ₀₂ −u ₀₀ ) / (v ₀₂ −v ₀₀ ).

Thus, the three coefficients A ₀₀ , B ₀₀ , and C ₀₀ in the above equation (1) are obtained, and the above equation (1) is determined.

Similarly, the first correction value of the second pixel P ₂ is interpolated in the vertical direction m ₀₂ and the vertical interpolation formula m ₂ for calculation is calculated, and the first correction value of the third pixel P ₃ is interpolated in the vertical direction m ₀₃ . Then, the vertical interpolation formula m ₃ for calculation is respectively determined.

Next, the first correction value calculation unit 32 uses the vertical direction interpolation formulas m ₁ , m _2, and m ₃ determined as described above to perform the first pixel P ₁ , the second pixel P _2, and the third pixel. calculating a first correction value for each of P ₃ (S3). Here, the first pixel Q ₁ , the second pixel Q _2, and the third pixel are points in the UB coordinate system of the projection image of the first pixel P ₁ , the second pixel P _2, and the third pixel P ₃ , respectively. and Q _3.

As shown in FIG. 5B, in the UV coordinate system of the projection image, the first pixel Q ₁ , the second pixel Q _2, and the third pixel Q ₃ are each composed of a horizontal scanning line and each vertical interpolation formula. Located at each intersection with m ₁ , m ₂ , m ₃ . The vertical coordinate calculation unit 33 calculates the v coordinate of each intersection point, that is, each v coordinate of the first pixel Q ₁ , the second pixel Q _2, and the third pixel Q ₃ . The v coordinates of the first pixel Q ₁ , the second pixel Q _2, and the third pixel Q ₃ are the difference in the v coordinate between q ₁ and q ₄ , the difference in the v coordinate between q ₂ and q ₅ , and q ₃ and it can be obtained from each of the difference v coordinates of the q _6. For example, in the case of the first pixel Q ₁ , a predetermined number of horizontal scanning lines are drawn in a section l ₁ between q ₁ and q ₄ . Dividing the difference in v-coordinate between q ₁ and q ₄ by the number of interpolations between the grid points in the Y direction of the XY coordinate system corresponding to section l ₁ , the unit distance in the V direction between the horizontal scanning lines is calculated. Is done. Vertical coordinate calculation unit 33, by the v coordinate of q ₁ by the number of horizontal scan lines when viewed from the q ₁ added unit distance in the V direction between horizontal scan lines, and calculates the first 1 v coordinates of the pixels Q ₁ be able to. The same applies to the second pixel Q ₂ and the third pixel Q ₃ .

The first correction value calculator 32 calculates the first pixel Q ₁ , the second pixel Q _2, and the second pixel Q calculated by the vertical coordinate calculator 33 for each of the above-described vertical interpolation equations m ₁ , m _2, and m ₃ . by substituting each v coordinates of three pixels _{Q 3,} calculating each u coordinates of the first pixel _{Q 1,} second pixel _{Q 2} and the third pixel _{Q 3.}

In this way, the first correction value calculation unit 32 uses the coordinates (first correction values) of the first pixel Q ₁ , the second pixel Q _2, and the third pixel Q ₃ in the UV coordinate system of the projection image. Is calculated. The first correction value calculation unit 32 outputs the coordinates of the first pixel Q ₁ , the second pixel Q _2, and the third pixel Q ₃ to the horizontal direction interpolation coefficient calculation unit 34 and also stores them in the first correction value storage unit 24. Store temporarily.

Next, when the coordinates given to each of the first pixel Q ₁ , the second pixel Q _2, and the third pixel Q ₃ are input from the first correction value calculation unit 32 to the horizontal interpolation coefficient calculation unit 34, the first pixel Q ₁ and interpolates between the second pixel Q _2, to calculate the coefficients of the scanning line direction interpolation formula for calculating the second correction value each in between the pixels, to determine the scanning direction interpolation formula (S4 (Interpolation formula setting process). Scanning line direction interpolation formula in which the first pixel Q ₁ and interpolating between the second pixel Q ₂ are determined similarly to the vertical interpolation equation described above.

For example, the scanning line direction interpolation formula m ₄ may be set to the following equation (2).

v = A (u−U ₀ ) (u−U ₁ ) + B (u−U ₀ ) + C (2)
Note that the first correction value of each of the first pixel Q ₁ , the second pixel Q ₂ , and the third pixel Q ₃ , that is, the coordinates in the UV coordinate system of the projection image is (U ₀ , V ₀ ), Let (U ₁ , V ₁ ), (U ₂ , V ₂ ).

  The three coefficients A, B and C in the above equation (2) are as follows.

A = (B′−B) / (U ₂ −U ₁ )
However, B ′ = (V ₂ −V ₀ ) / (U ₂ −U ₀ )
B = (V ₁ −V ₀ ) / (U ₁ −U ₀ )
C = V ₀
Thus, the three coefficients A, B, and C in the above equation (2) are obtained, and the above equation (2) is determined. As shown in (b) of FIG. 5, in a period during which scanning the interpolation interval between the 0th pixel Q ₀ and the first pixel Q _1, the first pixel Q ₁ and between the second pixel Q ₂ The above steps S1 to S4 for the interpolation section are performed.

Next, the second correction value calculation unit 40 uses the horizontal interpolation coefficient operation unit 34 the scanning line direction interpolation equation which is determined by, as shown in (c) of FIG. 5, a first pixel Q ₁ second pixel interpolation interval _{l 4} between the two pixels _{Q 2 (e.g., Q a,} Q _b) calculating a second correction value (S5) (interpolation step). As described above, the second correction value is a coordinate in the UV coordinate system of the projection image. The second correction value calculator 40, in accordance with the progress of the scanning point of the horizontal scanning lines of the drawing process, u of pixels in the interpolation interval l _4, successively calculates the v coordinate.

The second correction value calculation unit 40 substitutes the u coordinate of each pixel in the interpolation section l ₄ calculated by the horizontal coordinate calculation unit 41 into the above-described scanning line direction interpolation formula m ₄ , thereby obtaining each v coordinate. Is calculated.

Horizontal coordinate calculation unit 41, the U-V coordinate system of the projection image, and sequentially outputs the u coordinates of each pixel in the interpolation interval l ₄ in the horizontal scanning line direction. The interpolation section u coordinates of each pixel l ₄ can be determined from the difference between u coordinate of the first pixel Q ₁ and the second pixel Q _2. If divided by the number of interpolations between lattice points of the X-Y coordinate system X direction corresponding first pixel Q ₁ and the difference in u coordinates of the second pixel Q ₂ in the interpolation interval l _4, U direction between pixels The unit distance is calculated. The horizontal coordinate calculation unit 41, the thus calculated unit distance, by in accordance with the progress of the scanning point of the horizontal scanning lines, will in addition by one unit distance u coordinates of the first pixel Q _1, the interpolation interval The u coordinate of each pixel in l ₄ can be calculated.

Note that begins with the drawing of interpolation intervals l _4, the reference pixel setting section 30 sets a new reference pixel region R ₂ (S1). As shown in FIG. 5D, in the reference pixel region R ₂ , the first pixel P ₁ ′, the second pixel P ₂ ′, and the third pixel, which are pixels for which the first correction value is to be interpolated. Reference pixels to be referred to by P ₃ ′ are included. The reference pixel region R ₂ includes p ₁ ′ to p ₉ ′ located at lattice points of the original image. The first pixel P ₁ ′ is a pixel located in a section l ₁ ′ between p ₁ ′ and p ₄ ′, and the second pixel P ₂ ′ is a section l ₂ between p ₂ ′ and p ₅ ′. The pixel located at 'and the third pixel P ₃ ' are pixels located in the section l ₃ 'between p ₃ ' and p ₆ '. The first correction value of the first pixel P ₁ ′ is interpolated in the vertical direction m ₀₁ ′, the vertical direction interpolation formula for calculation, and the first correction value of the second pixel P ₂ ′ is interpolated in the vertical direction m ₀₂ ′. The vertical interpolation formula for calculation and the first correction value of the third pixel P ₃ ′ are interpolated in the vertical direction m ₀₃ ′, and the vertical interpolation formula for calculation is determined (S2).

  Here, as shown in FIG. 6, it is assumed that the interpolation section 1, the interpolation section 2, and the interpolation section 3 are drawn in this order. The interpolation device 20 performs the above-described steps S1 to S4 for the interpolation interval 2 within the period for performing the above-described step S5 for the interpolation interval 1. Similarly, the process of steps S1 to S4 described above is performed for the interpolation section 3 within the period during which the process of step S5 described above is performed for the interpolation section 2.

  In the interpolation device 20, before the drawing process of each interpolation section is started, the interpolation section necessary for scanning each interpolation section is interpolated, and the scanning line direction interpolation formula for obtaining the coordinates in the interpolation section is determined. . Thereby, the scanning line direction interpolation formula necessary for scanning the interpolation section which is the next drawing target is determined while performing the drawing process of the interpolation section.

  By doing so, it is possible to obtain the correction value of the pixel on the horizontal scanning line by interpolating on the horizontal scanning line while drawing on the horizontal scanning line.

  As described above, in the present embodiment, the processing based on the premise that the correction value of the inter-grid pixel is interpolated using the reference correction value assigned to the grid point pixel has been described. That is, it can be said that the processing of the present embodiment is intended for interpolation in a two-dimensional space. However, if attention is paid to the fact that the processing of steps S1 to S4 described above is performed for the subsequent interpolation section within the period of performing the processing of step S5 described above for the previous interpolation section, this point is one-dimensional. It can also be applied to interpolation in space. For example, it can be applied to a technique of reproducing audio data sampled with time while interpolating.

  The interpolation device 20 may be realized by a logic circuit (hardware) formed in an integrated circuit (IC chip) or the like, or may be realized by software using a CPU (Central Processing Unit).

  The interpolation device 20 can be realized by, for example, the hardware configuration shown in FIG. As illustrated in FIG. 3, the interpolation device 20 includes a first interpolation device 51 and a second interpolation device 52. The first interpolation device 51 includes the reference pixel setting unit 30, the vertical direction interpolation coefficient calculation unit 31, the first correction value calculation unit 32, the vertical coordinate calculation unit 33, and the horizontal direction interpolation coefficient calculation unit 34 illustrated in FIG. An adder 61, a multiplier 62, and a division table storage unit 63 realized by the configuration are included. Further, the first interpolation device 51 includes a temporary register 64 that implements the horizontal direction interpolation coefficient storage unit 23 and the first correction value storage unit 24 illustrated in FIG. 1 by a hardware configuration. The first interpolation device 51 calculates the reference correction value input from the reference correction value storage unit 2 using an adder 61, a multiplier 62, and a division table stored in the division table storage unit 63. Then, the horizontal interpolation coefficient and the first correction value described above are calculated and temporarily stored in the temporary register 64. As the division table, for example, a lookup table that can determine each digit of the quotient based on the dividend and the divisor can be used.

  The second interpolation device 52 calculates the horizontal interpolation coefficient and the first correction value input from the first interpolation device 51 using the multipliers 71 and 72 and the adder 73, and the second interpolation device described above. A correction value is calculated.

  When implemented by software, the first interpolation unit 21 and the second interpolation unit 22 of the interpolation device 20 are a CPU that executes instructions of a program that is software that implements each function, and the program and various data are computers (or CPUs). ROM (Read Only Memory) or storage device (referred to as “recording medium”) recorded in such a manner as to be readable, and a RAM (Random Access Memory) for expanding the program. And the objective of this invention is achieved when a computer (or CPU) reads the said program from the said recording medium and runs it. As the recording medium, a “non-temporary tangible medium” such as a tape, a disk, a card, a semiconductor memory, a programmable logic circuit, or the like can be used. The program may be supplied to the computer via an arbitrary transmission medium (such as a communication network or a broadcast wave) that can transmit the program. The present invention can also be realized in the form of a data signal embedded in a carrier wave in which the program is embodied by electronic transmission.

  The present invention is not limited to the above-described embodiments, and various modifications can be made within the scope shown in the claims. That is, embodiments obtained by combining technical means appropriately modified within the scope of the claims are also included in the technical scope of the present invention.

  The interpolation device according to each aspect of the present invention may be realized by a computer. In this case, the interpolation is realized by the computer by operating the computer as each unit included in the interpolation device. An apparatus control program and a computer-readable recording medium on which the apparatus control program is recorded also fall within the scope of the present invention. Furthermore, the interpolation device according to each aspect of the present invention may be realized as an integrated circuit, and in this case, a chip including the integrated circuit falls within the scope of the present invention.

  The present invention can be used for a display system that provides a user with an image projected on a projection plane from a display device, such as an in-vehicle display system using a HUD system.

DESCRIPTION OF SYMBOLS 1 Image processing apparatus, 2 reference correction value memory | storage part, 3 address production | generation part, 4 video capture buffer, 5 filter function part, 6 display control part, 20 interpolation apparatus, 21 1st interpolation part (interpolation formula setting means), 22 1st 2 interpolation units (interpolation means), 23 horizontal direction interpolation coefficient storage unit, 24 first correction value storage unit, 30 reference pixel setting unit, 31 vertical direction interpolation coefficient calculation unit (sub interpolation formula setting unit), 32 first correction value Calculation unit (sub interpolation unit), 33 vertical coordinate calculation unit (sub interpolation unit), 34 horizontal direction interpolation coefficient calculation unit, 40 second correction value calculation unit, 41 horizontal coordinate calculation unit.

Claims (2)

Interpolates the data between each grid point to the data to be processed in the order of the grid points given to each grid point arranged in the first direction which is the direction of the horizontal scanning line when drawing an image. An interpolation device,
Interpolation means for performing interpolation between the first grid points using the interpolation formula set for the first grid points;
An interpolation formula setting means for setting an interpolation formula between the second grid points whose arrangement order is later than between the first grid points during a period in which the interpolation between the first grid points is performed;
Equipped with a,
The interpolation formula setting means is arranged in the second direction, which is arranged in a second direction, which is a vertical direction when drawing the image, and is interpolated between the grid points of a plurality of grid points in which data is determined in advance. Sub-interpolation equation setting means for setting the equation;
By interpolating between the grid points in the second direction using the interpolation formula set by the sub-interpolation formula setting means, data is given to points located between the grid points in the second direction, Sub-interpolating means that uses the points to which the data is assigned as the grid points used by the interpolation means and the interpolation formula setting means,
The interpolation formula is an n-order formula (n is a natural number of 2 or more) passing through grid points at both ends between the grid points,
The interpolation apparatus according to claim 1, wherein the data is a correction value for correcting distortion generated when the image is projected onto a projection plane, and the lattice points are arranged on pixels constituting the image . Interpolates the data between each grid point to the data to be processed in the order of the grid points given to each grid point arranged in the first direction which is the direction of the horizontal scanning line when drawing an image. An interpolation method,
An interpolation step of performing interpolation between the first grid points using an interpolation formula set for the first grid points;
Interpolation formula setting for setting an interpolation formula between the second grid points whose arrangement order is later than that between the first grid points during the period in which the interpolation between the first grid points is performed in the interpolation step. Process,
Only including,
In the interpolation formula setting step, the interpolation in the second direction is performed between the grid points of a plurality of grid points that are arranged in a second direction, which is a vertical direction when the image is drawn, and in which data is determined in advance. Setting an expression; and
The points located between the grid points in the second direction by performing interpolation between the grid points in the second direction using the interpolation formula set in the step of setting the interpolation formula in the second direction. Including the step of setting the grid points used in the interpolation step and the interpolation formula setting step to the point to which the data is assigned,
The interpolation formula is an n-order formula (n is a natural number of 2 or more) passing through grid points at both ends between the grid points,
The interpolation method, wherein the data is a correction value for correcting distortion generated when the image is projected onto a projection plane, and the grid points are arranged on pixels constituting the image .

Images